Michel Vacher
email: michel.vacher@imag.fr

Benjamin Lecouteux
email: benjamin.lecouteux@imag.fr

Pedro Chahuara
email: pedro.chahuara@imag.fr

François Portet
email: francois.portet@imag.fr

Brigitte Meillon
email: brigitte.meillon@imag.fr

Nicolas Bonnefond
email: nicolas.bonnefond@imag.fr

The Sweet-Home speech and multimodal corpus for home automation interaction

Keywords: multimodal data acquisition, natural language and multimodal interactions, voice-activated home automation

come   L'archive ouverte pluridisciplinaire

Introduction

The goal of the Ambient Assisted Living (AAL) domain is to foster the emergence of ICT-based solutions to enhance the quality of life of older and disabled people at home, at work and in the community, thus increasing the quality of life, autonomy, participation in social life of elderly people, and reducing the costs of health and social care. Smart spaces, such as smart homes, are emerging as a way to fulfil this goal [START_REF] Chan | A review of smart homes-present state and future challenges[END_REF]. Smart homes can increase autonomy by enabling natural control of the people's environment through home automation, can support health maintenance by monitoring the person's health and behaviour through sensors, can enhance security by detecting distress situations such as fall. Typical processing undertaken as part of smart home systems includes, automatic location of the dwellers, activity recognition, Automatic Speech Recognition (ASR), dialogue, context-aware decision, sound analysis, behaviour analysis, etc. These tasks ask for a large amount of resources in order to acquire the most accurate models. However, one of the main problems that impede such research is lack of big amount of annotated data. The acquisition of datasets in smart homes is highly expensive both in terms of material and of human resources. Nowadays most of the available corpora recorded in smart spaces are outcomes of research projects (see for instance BoxLab1 or the MavHome website2 ). For instance, in the context of meeting room analysis [START_REF] Mccowan | The multichannel overlapping numbers corpus[END_REF] or of home automation order recognition from distant speech [START_REF] Cooke | An audio-visual corpus for speech perception and automatic speech recognition[END_REF][START_REF] Christensen | The CHiME corpus: a resource and a challenge for computational hearing in multisource environments[END_REF]. One drawback of these datasets is that they often do not include continuous recording of the audio modality. Either home automation sensor traces are recorded or speech data but rarely both at the same time (with the notable exception of [START_REF] Fleury | A French corpus of audio and multimodal interactions in a health smart home[END_REF]). Hence, it is still difficult to conduct study related to multimodal ASR, distant speech, grammar and emotion using these databases. This paper introduces the corpus composed of audio and home automation data acquired in a real smart home with French speakers. This campaign was conducted within the SWEET-HOME project aiming at designing a new smart home system based on audio technology [START_REF] Vacher | The SWEET-HOME project[END_REF]).

The developed system provides assistance via natural manmachine interaction (voice and tactile command) and security reassurance by detecting distress situations so that the person can manage, from anywhere in the house, her environment at any time in the most natural way possible. To assess the acceptance of vocal order in home automation, at the beginning of the project, a qualitative user evaluation was performed [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF] which has shown that voice based solutions are far better accepted than more intrusive solutions (e.g., video camera). Thus, in accordance with other user studies, audio technology appears to have a great potential to ease daily living for elderly and frail persons. To respect privacy, it must be emphasized that the adopted solution analyses the audio information on the fly and does not store the raw audio signals. Moreover, the speech recognizer is made to recognize only a limited set of predefined sentences which prevents recognition of intimate conversations. The paper introduces the experimental setting of the experiments conduced to acquired the SWEET-HOME corpus. The content of the corpus is then detailed and the results of some studies are then presented.

Smart Home Environment

The DOMUS smart home

To provide data to test and train the different processing stages of the SWEET-HOME system, experiments were con- 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 ducted in the DOMUS smart home of the LIG that was designed to observe users' activities interacting with the ambient intelligence of the environment (see Figure 1). It is a thirty square meters suite flat including a bathroom, a kitchen, a bedroom and a study, all equipped with sensors and actuators so that it is possible to act on the sensory ambiance, depending on the context and the user's habits. The flat is fully usable. The data acquisition was performed by recording all possible information sources from the house, including: home automation sensors and microphones. Actuators were driven by an operator (wizard of Oz) or by the intelligent system.

Home Automation Sensors

DOMUS smart apartment is part of the experimentation platform of the LIG laboratory and is dedicated for research projects. DOMUS is fully functional and equipped with sensors, such as energy and water consumption, level of hygrometry, temperature, and actuators able to control lighting, shutters, multimedia diffusion, distributed in the kitchen, the bedroom, the office and the bathroom. A framework has been designed in order to send orders to the different actuators, and to receive the changes of the sensor values [START_REF] Gallissot | Une plate-forme usage pour l'intégration de l'informatique ambiante dans l'habitat : Domus[END_REF]). An observation instrumentation, with cameras, microphones and activity tracking systems, allows to control and supervise experimentations from a control room connected to DOMUS. According to the different research projects, experimentations are conducted with users performing scenarios of daily housework and leisure. Multimodal corpus are produced, synchronized and analyzed in order to evaluate and validate the concerned concept or system.

Audio Recording

The flat also contains 7 radio microphones SENNHEISER ME2 set into the ceiling (2 per room except one for the bathroom) that can be recorded in real-time thanks to a dedicated software able to record simultaneously the audio channels coupled to a National Instrument PCI-6220E multichannel card, the sampling rate was 16 kHz. A 8 th channel can be used for a microphone set in front of a noise source (radio or vacuum cleaner). Two kinds of recording were operated with microphones (see Figure 2). Regarding the first one, a full soundtrack was obtained for each user experiment and may be used for other experiments thanks to StreamHIS [START_REF] Lecouteux | Distant speech recognition for home automation: Preliminary experimental results in a smart home[END_REF] or PATSH (Vacher et al., 2013c) softwares. These records were then annotated thanks to Transcriber3 , the speech sentences were extracted over the 7 channels and the Signal to Noise Ratio (SNR) was evaluated for each channel.

Regarding the second one, an automatic segmentation was operated by the PATSH software during interaction experiments with the SWEET-HOME system. For each speech event, some information including SNR is available (XML format file). Contrary to the precedent case, the audio data is related to an only channel, the PATSH analyser function has to select the best SNR channel.

Experiments

Before each experiment, the participant visited the home in order to become familiar with the place and the furniture.

The person was asked to read a short text aloud in the smart home in order to adapt the ASR acoustic model to her. To make the acquired data as natural as possible, no instruction was given about how to perform the activity and in which direction to utter sentences. All participants have given an informed and signed consent about the experiment. The acquisition protocol was submitted to the CNIL4 . CNIL is the French institution that protects personal data and preserves individual liberties.

Activity of Daily Living

The typical smart homes considered in the study are the one that permit voice based interaction. This kind of smart home can support daily life by taking context sensitive decisions based on the current situation of the user. More specifically, the smart home can be reactive to vocal or other commands to make the most adequate action based on context, and can act pro-actively by recognising a specific situation in which an action must be made (e.g., for security issue). Therefore, it must be able to recognize the activities of a dweller in order to operate properly in context aware interaction or to provide assistance. Activities can be seen as the atomic elements of a person's day and permit to recognize in which the person is currently, and to detect any deviance from his/her daily routine.

The traditional definition of basic activities, Activities of Daily Living (ADL), was given by [START_REF] Katz | A measure of primary sociobiological functions[END_REF]. While ADLs are related to daily human needs, instrumental ADLs (iADLs) are focused on the activities that involves higher level organisation (e.g., handling the phone) or handling of tools (e.g., preparing food). Table 1 presents some examples of these two kinds of activities.

ADL iADL Feeding

Using the phone Dressing Preparing and managing food Hygiene Doing shopping Locomotion Doing laundry

Table 1: ADL and iADL examples

In the context of our application, 7 activities were considered: (1) Sleeping; (2) Resting: managing emails, listening to the radio, reading a magazine; (3) Dressing/undressing; (4) Preparing the meal; (5) Feeding: having a meal; (6) Doing the laundry (dishes and vacuum); (7) Hygiene activity: washing teethes and having a shower, cleaning up the flat using the vacuum; and (7) Communicating: talking with the phone, going out to do the shopping. Twenty one participants (including 7 women) were asked to perform these activities without any condition on the time spent and or about the way for doing activities. Given the high number of necessary elementary activities and the duration of the experiment (mean: 1h 14mn), a detailed scenario simulating the sequences of a day, as well as a list of sentences to utter during the phone call were given to the participant and explained during the visit of the flat. Thanks to this experiment, a rich multimodal data set was collected associating speech data recorded in Distant Speech conditions with home automation data. It should be noted that these two types of data are very different from a temporal point of view: the first one is made of analog signals sampled at 16 kHz and the other of network frames transmitted over the home automation network when the person actuated a switch or moved in front of a infra-red detector.

Vocal Order for Home Automation

The project aims at designing a new smart home system based on audio technology to drive home automation thanks to vocal orders. Recent developments have produced significant results and enabled the Automatic Speech Recognizers (ASR) to be a component of many industrial products, but there are still many challenges to make this feature available for Smart Homes. For instance, ASR systems achieve good performance when the microphone is placed near the speaker (e.g., headset), but the performance degrades rapidly when the microphone is placed at several meters (e.g., in the ceiling). This is due to different phenomena such as the presence of noise background and reverberation [START_REF] Vacher | Preliminary evaluation of speech/sound recognition for telemedicine application in a real environment[END_REF] and these issues must be considered in the context of home automation. Moreover, the mouth does not emit sound in the same way for all frequencies, the emitting cone being narrower for high frequencies. These conditions are known as Distant Speech [START_REF] Wölfel | Distant Speech Recognition[END_REF], Given that, to the best of our knowledge, no dataset of French utterances of voice commands in a noisy multisource home exists, we conducted an experiment to acquire a representative speech corpus composed of utterances of not only home automation orders and distress calls, but also colloquial sentences. In order to get more realistic conditions, two types of background noise were considered while the user was speaking: the broadcast news radio and a music (classical) background. These were played in the study through two loud speakers, the 8 th channel was used to record the emitted sound thanks to a microphone set in front of one loud speaker. Note that this configuration poses much more challenges to classical blind source separation techniques than when speech and noise sources are artificially linearly mixed. Voice orders were defined using a very simple grammar as shown on Figure 3. Our previous user study showed that targeted users prefer precise short sentences over more natural long sentences [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF]. Each order belongs to one of three categories: initiate command, stop command and emergency call (see Figure 3). Except for the emergency call, all command starts with a unique keyword that permits to know whether the person is talking to the smart home or not. In the following, we will use 'Nestor' as key-word:

• set an actuator on: (e.g. Nestor ferme fenêtre) key initiateCommand object;

• stop an actuator: (e.g. Nestor arrête) key stopCommand [object];

• emergency call: (e.g. au secours).

The protocol was composed of two steps. During the first step, the participant was asked to go to the study, to close the door and to read a short text of 285 words. This data was used for the adaptation of the acoustic model of the ASR.

During the second step, the participant uttered 30 sentences in different rooms in different conditions. The first condition was without additional noise, the second one was with the radio turned on in the study and the third one was with classical music played in the study. Each sequence of 30 sentences was composed by a random selection of 21 home automation orders (9 without initiating keyword), 2 distress calls (e.g., "A l'aide" (help), "Appelez un docteur' (call a doctor)) and 7 casual sentences (e.g., "Bonjour" (Hello), "J'ai bien dormi" (I slept well)). No participant uttered the same sequences. The radio and the music were unique and pre-recorded and were started at a random time for each participant. 23 persons (including 9 women) participated to the experiment. The average age of the participants was 35 years (19-64 min-max). No instruction was given to any participant about how they should speak or in which direction. Consequently, no participant emitted sentences directing their voice to a particular microphone. The distance between the speaker and the closest microphone was about 2 meters. The total duration of the experiment was about 5 hours. During this experiment, only sound data were

Evaluation of the SWEET-HOME System

The system architecture is presented in Figure 4 and a complete description is available in (Vacher et al., 2013a). The input is composed of the information available from the home automation network and information from 7 microphones. The audio signals are analysed by the PATSH framework in real-time. The signals discriminated as speech are sent to the ASR if they satisfy the following conditions: SN R ≥ 0.1dB and 150ms ≤ duration ≤ 2.8s.

The Speeral tool-kit was used as ASR [START_REF] Linarès | The LIA speech recognition system: from 10xRT to 1xRT[END_REF]. Speeral relies on an A⋆ decoder with HMM-based contextdependent acoustic models and trigram language models. At the end, the most probable recognition hypothesis is then sent to the Intelligent Controller. Thus, information can be provided directly by the user (e.g., voice order) or via environmental sensors (e.g., temperature). The information coming from the home automation system is transmitted on-line to the Intelligent Controller (through several preprocessing steps). This controller captures all streams of data, interprets them and executes the required actions by sending orders to actuators through the home automation network. Moreover, the controller can send alert or information messages to a speech synthesiser in case of emergency or on request of the user ("Beware, the front door is not locked", "The temperature is 21 degrees") [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF]. This experiment aims at evaluating the context awareness performances of the complete SWEET-HOME system in realistic conditions. Two cases were considered: the first one implies users without disabilities and the second one aged or visually impaired persons.

Intelligent Controller

Population without disabilities

To validate the system in realistic conditions, we built scenarios in which every participant was asked to perform the following activities: (1) Sleeping; (2) Resting: listening to the radio; (3) Feeding: preparing and having a meal; and (4) Communicating: having a talk with a remote person thanks to a specialized communication system e-lio5 . Therefore, this experiment allowed us to process realistic and representative audio events in conditions which are directly linked to usual daily living activities. Moreover, to evaluate the decision making, some specific situations were planned in the scenarios. Each participant had to use vocal orders to make the light on or off, open or close blinds, ask about temperature and ask to call his or her relative. The instruction was given to the participant to repeat the order up to 3 times in case of failure. In that case, a wizard of Oz was used. Sixteen participants (including 7 women) without special needs were asked to perform the scenarios without condition on the duration after a visit of the flat allowing to explain the right way to utter vocal orders and to use the e-lio system. Before the experiment, the participant was asked to read a text of 25 short sentences in order to adapt the acoustic models of the ASR for future experiments. The average age of the participants was 38 years (19-62, min-max) and the experiment lasted between 23min and 48min. The scenario includes at least 15 vocal orders for each participant but more sentences were uttered because of repetitions.

Aged or visually impaired population

The method was also applied in the same context but the protocol was a little simplified and the experiment was cut into 4 independent scenarios: the light, close the blinds or ask for the temperature while doing these activities.

• The participant is coming back from shopping and is going to have a nap. He/she asks the same kind of commands but in this case, a warning situation alerts about the front door not being locked.

• The participant is going to the study to communicate with one relative through the dedicated e-lio system. After the communication, the participant simulates a sudden weakness and call for help.

• The participant is waiting in the study for friends going to visit her. She tests various voice orders with the radio, lights and blinds.

At the beginning of the experiment, the participant were asked to read a short text (288 words), this text was used to adapt the acoustical models to the speaker before performing the scenarios. Eleven participants were recruited to perform these scenarios either aged (6 women) or visually impaired (2 women, 3 men). During this experiment, about 4 hours of data were recorded.

The Acquired Corpus

The ADL annotation schema

The corpus was annotated by means of Advene 6 , an annotation tool that allows to organize annotations into different types and schemata. The main goal of this annotation was to set the ground truth for the location of the inhabitant 6 http://liris.cnrs.fr/advene/ and the activity she performs during the experiments. The state of the devices that gives information about the location and the activity are also part of this annotation. The speech was annotated using Transcriber and the Signal to Noise ratio (SNR) was evaluated for each channel. Table 3 shows the types of annotations that were set for the SWEET-HOME corpus. The apartment is composed of four rooms: a kitchen, a bedroom, a bathroom, and a study, therefore it was possible to consider the following activities for the experiment: eating and preparing a meal, cleaning and tiding, dressing/undressing, talking by the telephone, sleeping, reading and listening the radio, and hygiene actions. We consider that activities are composed of several actions.

For the activities we consider in our study, we have defined a set of actions. Thus, the actions involved in the activity eating and preparing a meal are: eating, drinking, washing dishes. While the actions included in hygiene are: washing hands, taking a shower, and tooth brushing. The others activities are composed of only one action.

In the case of Doors, the annotation do not contain the intervals in time with a certain state, but only the events indicating the door involved and its new state. The same has been applied for the position of the person.

General organisation of the corpus

The SWEET-HOME corpus, detailed in Table 2, is divided in four parts, each of which is presented in its own section.

The Multimodal subset: ADL experiment

This subset was recorded to train models for automatic human activity recognition and location. These two types of information are crucial for context aware decision making in smart home. For instance, a vocal command such as "allume la lumière (turn on the light)" cannot be handled properly without the knowledge of the user's location. The characteristics of the corpus are showed in the second column of Table 2. During the experiment, event tracks from the home automation network, audio (7 channels) and video sensors were captured. In total, more than 26 hours of data have been acquired (audio, home automation sensors and videos).

The French Home Automation Speech subset

As explained in section 3.2., this subset was recorded to develop robust automatic recognition of voice commands in a smart home in distant conditions (the microphones were not worn but set in the ceiling). Seven audio channels were recorded to acquire a representative multichannel speech corpus composed of utterances of not only home automation orders and distress calls, but also colloquial sentences.

The sentences were uttered in different rooms and conditions as described Table 4. The first condition was without noise, the second one was with the radio turned on in the study and the third one was with classical music played in the study. No instruction was given to the participants about how they should speak or in which direction. Each sentence was manually annotated on the best Signal-to-Noise Ratio (SNR) channel using Transcriber.

The third column of Table 2 shows the details of the record. It is composed, for each speaker, of a text of 285 words for acoustic adaptation (36 minutes for 351 sentences in total for the 23 and of 240 short sentences (2 hours and 30 minutes per channel in total for the 23 speakers) with a total of 5520 sentences overall. In clean condition, 1076 voice commands and 348 distress calls were uttered while they were respectively 489 and 192 in radio background noise and 412 and 205 with music.

The Interaction subset

The fourth column of Table 2 summarises the last subset of the SWEET-HOME corpus. In the matter of speech records, a text read aloud is available for each participant. Each of them had to use the grammar (see Figure 3) to utter vocal orders to open or close blinds, ask about temperature, ask to call his or her relative. . . The scenario included 15 vocal orders for each participant but more sentences were uttered because of repetitions due to errors of the recognition system or to a wrong use of the grammar by the participant. During the scenario, the participant coped with 4 situations: 1) Feeding: preparing and having a meal; 2) Sleeping; 3) Communicating: initiating and having a talk with a relative thanks to the specialised device e-lio; 4) Resting: listening to the radio. Globally, 443 vocal orders were recorded, and Table 5 shows for each participant the number of syntactically correct vocal orders or distress sentences and the corresponding SNR. The SNR varies between 11dB and 25dB and is below or equal to 17dB for 12 participants.

The User specific interaction subset

To assess the system with targeted users (aged or visually impaired), a user study was set up consisting of semidirected interviews and sessions in which the user, alone in the flat, had to interact with the SWEET-HOME system following predefined scenarios. The scenario was built from the last one of section 4.5. but simplified and cut into 4 short independent sequences. During the last sequence, the participant was able to use is proper syntax of command, and therefore, not all home automation orders are following the predefined grammar. Recorded data are presented Table 6, SI denotes a visually impaired participant and SA an aged participant. A part of the speech data are related to a talk conversation with the relative thanks to the e-lio system.

Some Studies using this Corpus

ADL Recognition

There is an increased interest in automatic human activity recognition from sensors just as well in Ambient Intelligence domain for context-aware interaction [START_REF] Coutaz | Context is key[END_REF] as in Ambient Assisted Living domain for behaviour monitoring in health applications [START_REF] Fleury | SVM-based multi-modal classification of activities of daily living in health smart homes: Sensors, algorithms and first experimental results[END_REF]. The Multimodal subset was used for a study related to ADL recognition from non-visual data [START_REF] Chahuara | Using Markov Logic Network for On-line Activity Recognition from Non-Visual Home Automation Sensors[END_REF]. We proposed a procedure that uses raw data from non visual and non wearable sensors in order to create a classification model leveraging logic formal representation and probabilistic inference. The Markov Logic Network method was evaluated and compared with SVM and Naive Bayes, as they have proved to be highly efficient in classification tasks. The overall accuracy achieved with MLN is 85.3%, it is significantly higher than the one obtained with SVM (59.6%) and Naive Bayes (66.1%).

Context Aware Interaction

The SWEET-HOME project addresses the issue of building home automation systems reactive to voice for improved comfort and autonomy at home. Context aware interaction is one of the challenges to take up. Our study focus on the context-aware decision process which uses a dedicated Markov Logic Network approach to benefit from the formal logical representation of domain knowledge as well as the ability to handle uncertain facts inferred from real sensor data [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF]. This study used the Interaction corpus. The decision process take into account some inferred variables, the location of the person and his activity. The correct decision rate are give Table 7 for the 4 situations described in section 4.5.

Distant Speech Recognition

The French Home Automation Speech subset was used for studies related to vocal order recognition in Distant Speech conditions [START_REF] Vacher | Recognition of Voice Commands by Multisource ASR and Noise Cancellation in a Smart Home Environment[END_REF]. A novel version of the Driven Decoding Algorithm (DDA-2) was evaluated. The a priori knowledge is obtained after decoding the best SNR channel: for decoding the second best SNR channel, the ASR system is driven by the 3 -nearest vocal orders of this hypothesis by using edit distance. This approach presents an important benefice, a strict usage of the grammar may bias toward vocal orders but the projection of vocal orders does not prevent the ASR recognising colloquial sentences. Moreover, an Acoustic Echo Cancellation (AEC) technique was used in the case of localized known sources (radio).

The corresponding recall results are given Table 8 for two kinds of noise, classical music(CM) and broadcast news (BN). Very good results were obtained using DDA in noisy and clean conditions, except for broadcast news. AEC improves greatly the performances in the case of broadcast news but leads to poor performances with music. This might be due to the fact that AEC introduces noise and non-linear distortion. Other methods like Blind Source Separation may be considered for performance improvement. The SWEET-HOME system was evaluated in the DOMUS flat with three user groups: seniors, visually impaired and people without special needs as related in section 3. (Vacher et al., 2013c). Regarding the technical aspect, DDA was not implemented in the system and consequently the vocal order error rate was high, 38%, and then the participant had often to repeat the orders. But an other objective was to evaluate the system regarding the user's interest, the accessibility the usefulness and the usability, particularly for the senior and visually impaired people. Hence the evaluation allowed to record their feedback about the system. None of the people said he had any difficulty in performing the experiment. It is worth emphasizing that aged people preferred a manual interaction because this was quicker, the voice warning was well appreciated. The visually impaired people found that the voice control would be more adequate if it could enable performing more complex tasks than controlling blinds or radio.

Distribution of the corpus

The multimodal and home automation parts are going to be released for research purpose only on a dedicated website http://sweet-home-data.imag.fr [START_REF] Vacher | The SWEET-HOME project[END_REF]. To respect privacy, no videos are released. Indeed, video data were only captured for manual marking up and then not available. The last part of the corpus will be released when the annotation process will be finished.

Conclusion

Smart Homes, despite to their recent developments, have led to few experimentations in daily living conditions. This paper presents a corpus made of 4 subsets acquired in daily living conditions in a fully equipped and complete smart home. Recorded data consist in home automation traces, sound and speech. The experiments that permitted these records are detailed, including the last one where aged and visually impaired users were implied. Finally, the results of experiments related to human location and activity recognition [START_REF] Chahuara | Using Markov Logic Network for On-line Activity Recognition from Non-Visual Home Automation Sensors[END_REF] and vocal command recognition and distress detection in real-time (Vacher et al., 2013b) are presented. Obtained results are not sufficient for a use in real conditions and future works include new studies related to activity recognition improvement and robust speech recognition in realistic noise conditions.

  (a) Screenshots of the recording video cameras.

  The DOMUS Smart Home with the position of some sensors.

Figure 1 :

 1 Figure 1: The DOMUS Smart Home.

Figure 2 :

 2 Figure 2: Sound recording.

Figure 3 :

 3 Figure 3: Excerpt of the grammar of the voice orders (terminal symbols are in French).

Figure 4 :

 4 Figure 4: Block diagram of the SWEET-HOME system.

•

  The participant is eating his/her breakfast and is preparing to go out. He/she asks the house to turn on

			The Sweet Home Corpus subsets
	Attributes	Multimodal Home Automation	Interaction	User Specific
	# Participants	21	23	16	11
	Age (min-max)	22-63	19-64	19-62	50-91
	Gender	7F, 14M	9F, 14M	7F, 9M	8F, 3M
	Ridden text for ASR adaptation	No	Yes (285 words)	Yes (288 words) Yes (288 words)
	Duration per channel	26h	3h 06mn	8h 52mn	4h 39mn
	Speech (#sentences), in French	1779	5520 (2760 with	993	907
			added noise effect)
	Sounds (#events)	-	-	3503	1866
	Home automation traces	yes	no	yes	yes
	Added noise effect	no	yes	no	no
			(vacuum, TV, radio)
	Home Automation orders	no	yes	yes	yes
	Distress call	yes	yes	yes	yes
	Colloquial sentence	yes	yes	yes	yes
	Interaction	no	no	yes	yes
	Segmented speech	manually	manually	manually and	automatically
				automatically
	Transcribed speech	yes	yes	yes	yes
	Transcribed SNR (at sentence level)	yes	yes	yes	yes
	Transcribed home automation traces	yes	no	no	no
	Transcribed sounds	no	no	automatically	automatically
				segmented	segmented

Table 2 :

 2 Description of the 4 subsets of the Sweet-Home Corpus

Table 3 :

 3 Annotation Types

	External noise	Study	Bedroom	Bathroom	Kitchen
	None	30	30	30	30
	Music (radio)	30	30	-	-
	Speech	30	30	-	-
	(broadcast news)

Table 4 :

 4 Home automation subset: Uttered sentence number as a function of the room

	Speaker	All	Sound	Speech	Order	SNR
	ID					(dB)
	S01	213	184	29	20	17
	S02	285	212	73	22	19
	S03	211	150	61	26	12
	S04	302	211	91	19	25
	S05	247	100	48	40	20
	S06	234	189	45	37	14
	S07	289	216	72	21	14
	S08	249	190	59	28	14
	S09	374	283	91	32	17
	S10	216	163	53	26	18
	S11	211	155	56	24	15
	S12	401	346	55	33	12
	S13	225	184	41	40	11
	S14	235	173	62	26	17
	S15	641	531	111	27	12
	S16	262	216	46	22	14
	All	4.595	3.503	993	443	-

Table 5 :

 5 Interaction subset: Recorded audio event characteristics

Table 6 :

 6 User specific interaction subset: Recorded audio event characteristics

Table 7 :

 7 Correct decision rate

Table 8 :

 8 Home automation order and distress detection 5.4. Evaluation of a context-aware voice interface for Ambient Assisted Living

	Signal	Method	Order recall	Distress
			(%)	recall (%)
	Without noise	ASR only	62.1	84.2
	Without noise	DDA	92.7	87.2
	BN	ASR only	29.3	74.3
	BN	DDA	57.2	75.2
	BN	AEC+DDA	83.5	81.2
	MC	ASR only	59.0	81.6
	MC	DDA	90.6	85.2
	MC	AEC+DDA	79.2	66.5

http://boxlab.wikispaces.com

http://ailab.wsu.edu/mavhome/research.html

http://trans.sourceforge.net/

http://www.cnil.fr/english/

http://www.technosens.fr/

Acknowledgements

This work is part of the SWEET-HOME project founded by the French National Research Agency (Agence Nationale de la Recherche / ANR-09-VERS-011). The authors would to thank the participants who accepted to perform the experiments. Thanks are extended to S. Humblot, S. Meignard, D. Guerin, C. Fontaine, D. Istrate, C. Roux and E. Elias for their support.




Abstract-A technological process to control the foam dielectric constant, an important issue for the design of microwave components and antennas, is described. For that purpose, the use of different commercial foam materials has been considered. This kind of foam substrate is made of original material (PVC, resin, …) into which gas is injected. So the dielectric constant of such foam is close to 1. It can be increased by expelling the gas out of the foam material. The authors are presenting the technological process used to expel the gas by pressing a foam slab at relatively low temperature (90°C). Thanks to this technological process, the dielectric constant variation can be controlled by the ratio between the initial and final slab thickness. It holds a great interest for the design of microwave antennas and circuits. Indeed, the dielectric constant inside gradient index lenses (Luneburg, Maxwell fish-eye and Fresnel lenses) must follow a particular law to obtain the desired radiation capabilities. Results of materials characterization are presented to validate the technological process. Foam based antennas and components are also shown to illustrate the interest of the process.

Index Terms-Foam material, controlled dielectric constant.

I. INTRODUCTION

ATERIALS with controllable permittivity are mainly used for electromagnetism and chemistry. Foam and composite materials exist with different dielectric constants [1,2]. They can be used as a support or directly to design microwave circuits or antennas [START_REF] Chainon | Etude et conception d"antennes composes de guides d"ondes en technologie mousse métallisée. Applications aux antennes à balayage électronique[END_REF]. The foam material used in [START_REF] Chainon | Etude et conception d"antennes composes de guides d"ondes en technologie mousse métallisée. Applications aux antennes à balayage électronique[END_REF] has the particularity to offer low dielectric constant (close to 1). This allows for the designing of antennas with high efficiency. This kind of foam is composed of basic initial PVC material into which air is injected or special gas. In microwave applications, substrate integrated non radiative waveguides can be manufactured with a particular dielectric constant material [START_REF] Xu | Substrate Intergrated Nonradiative Dielectric Waveguide Structures Direcly Fabricated on printed Circuit Boards and εetallized Dielectric δayers[END_REF]. In such applications, it could be interesting to adjust the dielectric constant for different waveguides and so to design phase shifters for example. In microwaves, antennas with inhomogeneous lenses can be designed to focus the radiation pattern [START_REF] Hristov | εillimeter-wave Fersnel-zone plate lens and antenna[END_REF] [6] [START_REF] Schulwitz | A new low loss Rotman lens design using a graded dielectric substrate[END_REF] but such lenses (Fresnel-zone plate lens, Luneburg lens, Maxwell fish-eye lens and Rotman lens) require the use of materials with different dielectric constant. For this kind of lens, the objective is to simplify the manufacturing and to reduce the cost by using only one material with controlled dielectric constant. The authors investigate a new and simple technological process to control the dielectric constant of a basic foam material. By expelling the gas contained inside a foam slab, the dielectric constant can vary, for example, between 1.3 and 3.2. In the first part, the technological process is related in details. In the second part, the measurement setup of dielectric constant is presented and results are shown to validate the principle of controlled dielectric constant based on one initial foam material. These experimental results are compared with theoretical data allowing to predict the dielectric constant of the manufactured material. In the third part, several applications at 60 GHz are presented.

II. TECHNOLOGICAL PROCESS

Four different foams are used and tested to validate the technological process [START_REF] Εerlet | εanufacturing method of a dielectric material and its application to millimeter-waves beam forming antenna systems[END_REF]. The first two kinds of foams (H200 and HCP50) are sold by Divinycell while the third and fourth ones (two different references of Airex PXc) are marketed by Corematerials.

A. Some instances of the foams used

These materials have dielectric constants close to one because they are filled with small gas bubbles as it is shown in Figure 1 for the H200 foam. There is four kinds of foams feature low dielectric constant and loss, as presented in Table 1. These characteristics are obtained from a free space measurements setup which is depicted in the second part of the paper. Foams are closed-cell foams and are used for their mechanical properties in the field of construction.

B. The main idea

In order to determine permittivity of a foam sample, several parameters are involved:

-Density: ρ x -Permittivity: x -Volume: V x -Thickness: h x Where x can be replaced by the appropriate material: g for the gas (air for us), p for the polymer, fi for the initial foam, ff for the final foam. Two different theories are reviewed: Knott and Plonus theories. The first one used the density and permittivity while the second one is based on the permittivity and volume. Both are computed and compared to measurements.

Firstly, the foam dielectric constant can be predicted by Knott"s formula [START_REF] Knott | Dielectric constant of plastic foams[END_REF]:

(1) Where f is the foam dielectric constant, p is the dielectric constant of the base polymer, g is the dielectric constant of the blowing agent gas (air in our case), and α is the volumetric fraction of polymer in the foam given by:

(2) The foam density depends on its volume:

(3) Then the main and innovative idea for microwave antennas is to reduce the height of the foam by pressing it (without altering the foam surface area S). When being pressed (Figure 2), the gas is expelled out of the foam, thus increasing the density and the dielectric constant of the foam. The density of the base polymer does not change during the process.

The density ratio between the initial foam height h i and the final height h f is given by: (4)

The new pressed foam density is given by: (5) Secondly, as mentioned in [START_REF] Plonus | Theorical investigations of scattering from plastic foams[END_REF], knowing the volume ratio, we can calculate the effective relative dielectric constant of pressed foam by ( 6): [START_REF] Fuchs | Comparative Design and Analysis of δuneburg and Half Maxwell Fish-Eye δens Antennas[END_REF] Where V g and V p are the volume of gas and polymer. In the reference [START_REF] Plonus | Theorical investigations of scattering from plastic foams[END_REF], Plonus finds the volume ratio of gas to base polymer in the foam as a function of foam density and uses this to determine the dielectric constant of the foam. The equation [START_REF] Schulwitz | A new low loss Rotman lens design using a graded dielectric substrate[END_REF] gives the volume of gas to volume of the base polymer ratio in considered foam knowing the density of base polymer (ρ p ) and gas (ρ g ). [START_REF] Schulwitz | A new low loss Rotman lens design using a graded dielectric substrate[END_REF] In order to verify the accuracy of the formulas ( 1) and ( 6), we used to predict the dielectric constant of H and HCP Divinycell foam [1]. In order to determine the dielectric constant and density of the base polymer ( p and ρ p ) of each foam material (unavailable data"s from manufacturers), the foam samples were powdered (Figure 3a) before being pressed (Figure 3b) to make sure there would be no gas left inside it during measurement. It will thus determine the utmost permittivity and density that can be reached (permittivity of the mixture composed of only core materials without gas). Every references of H or HCP foam (10, 20, 50, 100 …) are made of the same base polymer; the only difference is the final density of the foam.

-For the gas (air): ρ g =1.2 kg/m 3 and g =1.

-For H200 foam, the base polymer is PVC material, we measured: ρ p =1600 kg/m 3 and p =3.0.

-For HCP50 foam, the base polymer is PVC material, we measured: ρ p =1500 kg/m 3 and p =2.92.

-For Airex PXc 245, the base polymer is glass-reinforced urethane material, we measured: ρ p =1075 kg/m 3 and p =2.8.

-For Airex PXc 320, the base polymer is the same as PXc 245, but we measured: ρ p =1273 kg/m 3 and p =3.15. The formulas ( 1) and ( 6) compared to measurements of permittivity (10 GHz in the data sheet) show a good agreement. The measurements are made for different references of foams using the same base polymer but with different densities. Figure 4 and Figure 5 show this agreement

h i h f
Pressure at T° Fig. 2. Illustration of the process to increase foam density.

for both Divinycell foams (Airex does not present those values in their data sheet).

C. Pressing process

The first important parameter is the adjustment of the temperature during the pressing process. An optimized temperature compromise has to be determined in order to avoid melting and burning (with a too high temperature) or to avoid the sample to change size once pressure ceases (if temperature was not high enough). The optimized temperature depends on the type of foam material that is used. The second important parameter is the duration of the pressing. It can vary depending on the foam and will affect the stability of the foam. A sufficient pressing period is needed for the sample to keep its size after pressure. The temperature inside the whole sample of foam must be evenly spread.

For the four kinds of foams mentioned above, the samples are pressed during 20 minutes at 90°C.

A picture of a foam sample before and after being pressed is featured in Figure 6 and Figure 7. A stepped sample with different thicknesses has been pressed. In the figure 7, it is easy to see that the air bubbles have been expelled from the foam after pressing. The density of the foam material is higher after pressing because air has been expelled.

In the next section are presented results of dielectric constant characterization for different initial thicknesses of foam and with the same final thickness (3mm).

III. CHARACTERIZATION

A. Free space measurement setup

The measurement of dielectric constant is performed by using a free space measurement [START_REF] Friedsam | A Broadband Free-Space Dielectric Properties Measurement System at Millimeter Wavelengths[END_REF]12] bench (Figure 8), composed of an ABmm Vector Network Analyzer (working between 8 GHz and 75 GHz) and a computer with software to extract the dielectric constant from measurements. The free space setup consists of two horns associated with two polyethylene biconvex lenses in order to obtain a focusing plane where the sample under test is placed. The computation of dielectric constant can be obtained either from reflexion (S11) or from transmission measurement (S21), the latter being used in this paper. Permittivity and loss tangent can be determined with formulas given in [START_REF] Bourreau | A Quasi-optical Free-Space Measurement Setup Without Time-Domain Gating for Material Characterization in the W-Band[END_REF] and [START_REF] Ghodgaonkar | Free-Space Measurement of Complex Permittivity and Complex Permeability of εagnetic εaterials at εicrowave Frequencies[END_REF]. First, the bench is calibrated without any sample. Then, the same measurement is carried out with the sample under test and measurements results are computed in order to find out the dielectric constant of the sample. This measurement process is fast but limited in terms of bandwidth by the size of the horns and lenses (from 46 GHz, up to 74 GHz). It also requires samples at least a few millimeters thick and with a diameter at least greater than five lambda. To ensure stable results, the whole band has to be measured and results are given at the middle frequency of the band (60 GHz here).

B. Experimental results of electrical properties

Measurements are done in the V band and electrical properties are given at 60 GHz. The foam samples have been pressed at 90°C during 20 minutes. The final thickness of samples is consistently 3mm after pressing. The four different foams have been characterized using the same method and results of pressed foam dielectric constant versus densities The results show that it is possible to increase the dielectric constant from 1.33 to 2.8 for the Airex PXc245 foam and 1.43 to 3.15 for the Airex PXc320. For the H200 foam, the range of dielectric constant is reached from 1.22 up to 2.35 and from 1.3 to 2.2 for the HCP50 foam.

IV. MICROWAVES APPLICATIONS

In this section, applications on microwaves components and antennas are presented. Several applications require a control of the dielectric constant. Non-radiative waveguide is a simple one. It is composed of a dielectric waveguide with two parallel metal plates above and below it. A 3dB/90° dielectric coupler has been manufactured at 30 GHz (Figure 13 and14) with Airex PXc 245 and a permittivity of 2.1 inside the waveguide has been reached (after being pressed) while the dielectric constant is 1.33 outside of the waveguide. Although four types of foams have been investigated, the proposed process is general. This technology can be directly applied to the design of microwave components and antennas. It makes for instance the manufacturing of gradient index lenses much easier as detailed in a forthcoming publication.

Fig. 1 .

 1 Fig. 1. Cross sectional photograph of a 5mm thick H200 Divinycell Foam.

Fig. 3 .

 3 Fig. 3. Picture of the powdered foam (a) and pressed foam (b)

Fig. 4 .Fig. 5 .

 45 Fig. 4. Dielectric constant law versus density for H Divinycell foam

Fig. 6 .

 6 Fig. 6. Stepped sample before being pressed (5, 10, 15mm).

Fig. 7 .

 7 Fig. 7. Stepped sample after being pressed at 5mm.

Fig. 8 .

 8 Fig. 8. Photograph of the free space measurement setup composed of hornlens antennas and support of sample under test.

  ratio ξ are plotted in Figures 9, 10, 11 and 12. The values of density and permittivity of base polymer are available section II.b for all foam material.

Fig. 9 .

 9 Fig. 9. Dielectric constant law versus densities ratio for pressed H200 foam.

Fig. 10 .Fig. 12 .

 1012 Fig. 10. Dielectric constant law versus densities ratio for pressed HCP50 foam

Fig. 13 .

 13 Fig. 13. Dielectric coupler at 30 GHz before being pressed

Fig. 15 .

 15 Fig. 15. Foam lens before (a) and after (b) being pressed

TABLE 1 ELECTRICAL

 1 PROPERTIES OF THE FOUR FOAMS MEASURED AT 60 GHZ

	Technological Process to control the Foam
	Dielectric Constant
	Application to microwave components and
	antennas
	Foam type	r	tan
	H200	1.23	0.005
	HCP50	1.31	0.004
	Airex PXc 245	1.31	0.010
	Airex PXc 320	1.43	0.014

J. Bor, O. Lafond, Member, IEEE, H. Merlet, P. Le Bars, Member, IEEE, and M. Himdi M

ACKNOWLEDGMENT

The authors acknowledge L. Cronier, X. Morvan, X. Castel and C. Guitton from IETR de Rennes (France) for their support on mechanics.


Brice Videau

Vania Marangozova-Martin

Luigi Genovese

Optimizing 3D Convolutions for Wavelet Transforms on CPUs with SSE Units and GPUs

Keywords: 3D Convolutions, Optimization, GPU, Vectorization, OpenCL, Code Tuning, Portability

Nanosimulations present a big HPC challenge as they present increasing performance demands in heterogeneous execution environments. In this paper, we present our optimization methodology for BigDFT, a nanosimulation software using Density Functional Theory. We explore autotuning possibilities for BigDFT's 3D convolutions by studying optimization techniques for several architectures. Namely, we focus on processors with vector units and on GPU acceleration. We report on the portability and the performance gains of our approach (speedup x2 on CPU, x5 on GPU) and discuss the relation between algorithmic specifics, architecture and performance.

I. INTRODUCTION

Obtaining good application performance on a high performance computing (HPC) platform becomes a real challenge. Indeed, the HPC hardware landscape becomes very complex and each architecture brings its performance specifics. For instance, the behavior of the Nehalem processors is very different from that of NVIDIA GPUs. The first have a small number of general purpose high performance cores with vector units. The later have many specialized cores with no vector units. The first need coarse grain parallelism to be efficiently used while the later can manage much finer grain parallelism. CPUs have a cache hierarchy that can be transparently used while GPUs have a cache that needs to be explicitly managed and can prove inefficient if not used correctly. As a consequence, it is impossible to obtain satisfactory performances on different architectures with the same implementation and optimization of an algorithm.

In this paper we study the implementation, code restructuring and optimization of 3D convolutions. We have chosen convolutions as they are key components of BigDFT [START_REF] Genovese | Daubechies wavelets as a basis set for density functional pseudopotential calculations[END_REF], a software for simulating the electronic properties of future nano materials. The optimization of the BigDFT code on HPC platforms and especially the definition of autotuning strategies are two major goals of the joint work between the Nanosim team of the LIG laboratory and the L_Sim team of CEA-Grenoble.

Our methodology for evaluating the performance and define tuning strategies for 3D convolutions is the following.

• Choose a Target Architecture

We have worked on two target architectures, namely one containing CPUs with SSE and another based on GPUs.

Both architectures have been invested in by the CEA -Grenoble who demanded efficient code.

• Benchmark a Reference Implementation

For the CPU architecture, we have benchmarked the already existing algorithm in order to find performance issues. For the GPU architecture, we first implemented a basic version that we benchmarked before optimizing it according to the GPU specifics.

• Identify Optimization Issues

The major optimization issue concerns data locality. Indeed, playing with the placement of data in the registers and in the cache can yield very different performance results. In the CPU case, our contribution is the definition of locality patterns to be used for memory placement, as well as for computation. In the GPU case, we contribute on the efficent usage of the cache.

• Generate and Benchmark Different Optimizations

For each target architecture, we have explored different optimization strategies (in relation with the issues identifies in the previous point). For each strategy we have benchmarked and identified the best performing one.

• Autotuning

For each optimization strategy we have analyzed the possibility to generalize its use and apply it in a different context. The remainder of the paper is organized as follows. Setion II presents our use case, BigDFT, and details 3D convolutions. Section III explains how our optimization work is different from related projects. Section IV and Section V tackle optimizations on CPUs with vector units and on GPUs respectively. Finally, Section VI concludes and outlines our future works.

II. USE CASE: BIGDFT AND 3D CONVOLUTIONS

In this section we focus on the algorithm for 3D convolutions after presenting the general context of the BigDFT application.

A. The BigDFT Project

BigDFT [START_REF]The bigdft scientific application[END_REF] started as an EU FP6-STREP-NEST project with four partner laboratories: L_Sim -CEA Grenoble, Basel University -Switzerland, Louvain-la-Neuve University -Belgium and Kiel University -Germany. The goal of BigDFT is to develop a novel approach for electronic structure simulation In 2009, L. Genovese has received the French Bull-Fourier prize for having implemented and coordinated the realization of the hybrid version of BigDFT [START_REF] Genovese | Density functional theory calculation on many-cores hybrid central processing unit-graphic processing unit architectures[END_REF]. Thanks to its use of the wavelets formalism, as well as to the HPC structuring of the code, BigDFT is characterized by its high precision, efficiency and flexibility. Currently, it has been chosen as an official benchmark for the EU MontBlanc project [START_REF]The mont-blanc project[END_REF].

B. Algorithmic Overview

The usage of the wavelets formalism allows BigDFT to define nanosimulations in terms of 3D convolution operations. These operations consist in applying three-dimensional (3D) cubic filters to three-dimensional cubic zones (cf. Figure 1).

The wavelet transform consists in applying the cubic filter in all three dimensions. The transform takes a three-dimensional array in of dimension n 1 xn 2 xn 3 as an input and is transformed into the output array Ψ r given by [START_REF] Genovese | Daubechies wavelets as a basis set for density functional pseudopotential calculations[END_REF]. The values n 1 , n 2 and n 3 give the dimensions of the BigDFT simulation domain. To calculate the output values, the transform uses the ω k values composing the so called magic filter. The computation is done using an extension from -L to U . The filter dimension U + L + 1 is equal to the order l of the Daubechie wavelet family. In our use case, l has been chosen by the physicists to be 16. When the indexes are outside bounds, the data is used in a circular manner by using the index modulo the size of the data line 1 .

Ψ r (i 1 , i 2 , i 3 ) = U j1,j2,j3=-L ω j1 ω j2 ω j3 in(i 1 + j 1 , i 2 + j 2 , i 3 + j 3 ) . (1)
A direct implementation of this algorithm would contain six nested loops. The three outer loops scan through all elements of the input array. The filter application is shown for the computation of element Ψ r (i 1 , i 2 , i 3 ) (cf. Listing 1). It 1 This is called periodic boundary conditions. will incur n 1 n 2 n 3 l 3 reads, multiplications and additions, and

n 1 n 2 n 3 writes. c o n s t i n t L = 7 ; c o n s t i n t U = 8 ; d o u b l e W[ L+U+ 1 ] = {W0, W1, . . . , W15 } ; d o u b l e temp = 0 ; f o r ( j 1 = -L ; j 1 <= U ; j 1 ++) { f o r ( j 2 = -L ; j 2 <= U ; j 2 ++) { f o r ( j 3 = -L ; j 3 <= U ; j 3 ++) { temp += W[ j 1 +L] * W[ j 2 +L] * W[ j 3 +L] * i n [ i 1 + j 1 ] [ i 2 + j 2 ] [ i 3 + j 3 ] ; } } } p s i [ i 1 ] [ i 2 ] [ i 3 ] = temp ; Listing 1. Basic Implementation of the BigDFT Magic Filter Operation
What is interesting about this 3D convolution is that it can be separated in three independent 1D convolutions (2). 1)

f 1 (i 1 , i 2 , i 3 ) = U j=-L ω j s(i 1 + j, i 2 , i 3 ) 2) f 2 (i 1 , i 2 , i 3 ) = U j=-L ω j f 1 (i 1 , i 2 + j, i 3 ) 3) Ψ r (i 1 , i 2 , i 3 ) = U j=-L ω j f 2 (i 1 , i 2 , i 3 + j) (2)
This algorithm's implementation will generate only 3n 1 n 2 n 3 l reads, multiplications and additions, and 3n 1 n 2 n 3 writes. However, if we consider data to be allocated in columnmajor order, the first loop accesses elements with a stride of 1, whereas the second and third one accesses elements with a stride of n 1 and n 1 n 2 respectively. In other terms, the first loop accesses elements sequentially, the second loop needs to access elements at distance of n 1 elements and the third loop accesses n 1 n 2 -distant elements. On most architectures, such memory jumps prove to be harmful for the execution performance.

The most convenient way to compute a three dimensional convolution of this kind is by combining 1D convolutions and transpositions [START_REF] Goedecker | Rotating a three-dimensional array in an optimal position for vector processing: case study for a three-dimensional fast fourier transform[END_REF]. The computation is given in (3).

1)

F 1 (i 2 , i 3 , i 1 ) = U j=-L ω j s(i 1 + j, i 2 , i 3 ) 2) F 2 (i 3 , i 1 , i 2 ) = U j=-L ω j F 1 (i 2 + j, i 3 , i 1 ) 3) Ψ r (i 1 , i 2 , i 3 ) = U j=-L ω j F 2 (i 3 + j, i 1 , i 2 ) (3)
In this case, only writes have a stride different than 1 (respectively n 2 n 3 , n 3 n 1 and n 1 n 2 ). It can be noted that by processing the convolution along the third dimension first, the reads would have a stride while the writes would be sequential.

III. RELATED WORK

3D convolutions are optimized using the Discrete Fourier Transform or using 1D convolutions and transpositions, as presented in the previous section. et al. [START_REF] Nukada | High performance 3d convolution for protein docking on ibm blue gene[END_REF] for a software for the BlueGene machine. The problem with this solution is that the complexity of the algorithm is C ln(n) per element where C depends on the chosen radixes. For powers of 2, C is commonly accepted as being 5/ ln [START_REF]The bigdft scientific application[END_REF]. So considering a 128 line length, the cost of the 1D convolution is about 50 FLOP per element. In our case, with the size of the filter we need to consider, applying the DFT approach would be too expensive.

• 1D Convolutions and Transpositions

Another approach for optimizing 3D convolutions is to consider is to 1D convolutions and transpositions. The possibilities are to consider 1D+2D or 1D+1D+1D. For the considered size of the filter (i.e. 16x16x16), methods working on 2D are not applicable as the data size is too big to be interesting from the performance point of view.

If there are many references considering 2D convolution optimization using SSE units or GPUs [START_REF] Fialka | FFT and Convolution Performance in Image Filtering on GPU[END_REF][10], the projects focusing on 3D convolutions are rare. For works considering 3D convolutions with vectorization, we can cite Intel [START_REF] Danovich | 16bit 3D Convolution: SSE4+OpenMP implementation on Penryn CPU[END_REF] but the filter is very small (3x3x3) and the data limited to 16bit data. Gaussian 3D filtering has also been ported to SSE [START_REF] Vaško | Optimizing Gaussian Filtering of Volumetric Data Using SSE[END_REF], but in this case the filters are symmetric and the authors limit the use of buffers. As for 3D convolutions on GPU, [START_REF] Hopf | Accelerating 3d convolution using graphics hardware (case study)[END_REF] use a 2D+1D method but for very small filter size.

• Autotuning Auto-tuning is a difficult problem and as far as we know, nobody has tried to auto-tune the type of BigDFT convolutions . However, there are several libraries using auto-tuning for different problems. For linear algebra ATLAS [START_REF] Whaley | Automatically Tuned Linear Algebra Software[END_REF] is a well known reference. For fast Fourier transforms FFTW [START_REF] Frigo | A fast fourier transform compiler[END_REF] and SPIRAL [START_REF] Xiong | Spl: A language and compiler for dsp algorithms[END_REF] are the most well-known. However, these libraries are limited to CPUbased platforms.

IV. OPTIMIZING 3D CONVOLUTIONS BY VECTORIZATION

In this section we analyze the performances of several versions of the 3D convolution algorithm using various levels of optimization. Only the separated convolution in double precision is considered.

A. Preliminary Performance Evaluation

The test platform is a Lenovo D20 workstation using 1 Intel Xeon X5550 CPU (featuring 4 cores) and 8 GiB of RAM. Hyper-threading and turbo boost are deactivated and performance governor is set to performance, so the processor frequency is set to 2.67GHz. As the Nehalem architecture is capable of providing two multiplications and two additions per cycle in double precision, each core has a peak performance of 10.6 GFlop/s. The operating system is Ubuntu 10.10 using a backported 2.6.36 linux kernel. The compilers used are Intel Compiler Suite version 11.1 and Gnu Compiler Collection version 4.4.5. With the latter, the optimization option used is -O2, as -O3 proved harmful to performances. Performance counters are obtained using PAPI [START_REF] Mucci | Papi: A portable interface to hardware performance counters[END_REF].

We have studied several versions of the 3D convolution algorithm and the corresponding results are given in Table I. The numbers indicate the mean for 10 runs for convolving an array of size 128x126x130 double precision numbers (16 Mio) The versions include simple and simple t which correspond to the straightforward implementations of convolutions without and with transposition. unrolled and unrolled t are two versions in which the nested loops for the simple and the simple t algorithms are unrolled with a degree of 8. Loop optimization techniques, such as loop unrolling, are discussed in the work of Wolf and Lam [START_REF] Wolf | A loop transformation theory and an algorithm to maximize parallelism[END_REF].

The counters we have considered include the total number of computation cycles (TOT CYC) and the number of executed instructions (TOT INS). We also present several cache-related counters, namely the ones reflecting the number of data cache accesses (DCA) and of data cache misses (DCM) for cache levels 1 and 2.

Gflop/s are computed using the formula 3 * 32 * n elem /t. 3 comes from the number of dimensions (3D convolutions). 32 comes from the length of the filter ( 16) and the number of operations per element (one addition and one multiplication). t is the computation time derived from the TOT CYC counter.

The first thing to note is that unrolling greatly reduces the number of cycles needed to compute the 3D convolution, by a factor 6.25 for non transposed algorithm and 8.17 for the transposed one. This difference can be mainly accounted for by the number of instructions needed to compute the 3D convolution that is reduced by a factor of 6 thanks to the loop unrolling.

The second thing that can be noted is that transposed versions of the algorithm present a better instruction throughput than their counterparts. This is especially true for the unrolled versions, where the transposed one present 28% more throughput than its counterpart. This can be explained by comparing the data cache miss results: non transposed versions incur twice as many cache miss in L1 than transposed versions. For L2 this ratio is of 3.5 for simple convolutions and 1.6 for unrolled convolutions.

Finally, even if the unrolled algorithms are much more efficient than the simple ones, their performance are still unsatisfactory. Indeed they achieve respectively 21% and 28% of the peak performances of the core they use.

The total number number of floating point operations needed to compute the whole 3D convolution is: 128 * 126 * 130 * 32 * 3 = 201M . If we add to this result the number of data accesses (L1 DCA) in the case of the unrolled t version we obtain: 201M +137M = 338M instructions. If we consider the ratio between these values, it indicates that there has been poor register reuse as memory operations take about 2/3 of the arithmetic operations. If we compare this value to the one given in the table (TOT INS 386M ), this result leaves about 50M of operations that are not accounted for. These can be loop overheads, or copies in registers.

Vectorizing the code may allow a reduction by a factor of two of the number of arithmetic operations. It may also reduce the number of memory accesses by even bigger factor if registers are reused efficiently. We focus on vector instructions used in the x86 architecture, namely the SSE instruction set [START_REF] Thakkur | Internet Streaming SIMD Extensions[END_REF] and its extensions SSE2, SSE3 and SSE4.

B. 3D Convolutions Vectorization Approach

In order to vectorize efficiently 3D convolutions, we have taken into account memory alignment, register pressure, register reuse and memory access patterns. We have worked by separating data (filters and input array) in 2-element vectors. As the algorithm needs only 32 arithmetic operations (1 multiplication and 1 addition for 16 elements) per step and per resulting element, reordering data in memory will prove too costly. Indeed, in our case the complexity is of O(1) per element. Reordering methods are interesting on GEMM type operations with complexity of O(n) per element. n being the length of the rows of the first matrix, the cost is easily masked when n is big enough.

We have adopted a solution with two filters as illustrated in Figure 2.

Data

Filter U Filter A The first filter F A consists of values (F 0 , F 1 , ..., F 15 ). The second filter F U contains (F 15 , F 0 , ..., F 14 ). Both filters are aligned on 16 byte boundaries.

In our first approach, we assume that data is also aligned on 16 byte boundaries and thus allow the reuse of data registers as explained in the following.

Let us consider the computation of two successive data, R 0 and R 1 , defined by ( 4) and [START_REF]The mont-blanc project[END_REF]. In the equation, D 0 is the one also shown in Figure 2 and is aligned on a 16 byte boundary. ⊗ is the vectorized multiplication operation which obtains a two-element result by respectively multiplying the first and the second elements of the two-element vector operands.

It can be noted that seven of the eight two-element data vectors can be reused directly. The first one needs to be updated with D 16 . If we compute several values in parallel, filter vectors can also be reused.

R 0 = 15 i=0 D i F i = V 00 + V 01 where V 0 = 7 i=0 [D 2i , D 2i+1 ] ⊗ [F A 2i , F A 2i+1 ] (4) R 1 = 15 i=0 D i+1 F i = V 10 + V 11 where V 1 =[D 16 , D 1 ] ⊗ [F U 0 , F U 1 ] + 7 i=1 [D 2i , D 2i+1 ] ⊗ [F U 2i , F U 2i+1 ] (5)
The register reuse is limited by the number of available registers. Indeed, we need one register per result, half of the registers for data and two registers for the filters. Given this repartition, computing 8 values in parallel consumes 14 registers (8 results, 2 filters, 4 data). However, SSE provides only 16 registers! As a consequence, computing more than 8 values in parallel might create register spilling. On the contrary, using less registers should diminish register reuse and increase the ratio between memory operations and arithmetic operations.

C. Performance Study of Convolution Kernels

In this section we study the performance obtained through vectorization of different versions (we call kernels) of the convolution computation. First we consider variants without transposition and then variants with transposition (cf. Section II-B)

1) Kernels without Transposition: In order to experiment with the presented strategy, we have generated several patterns for computing data values (cf. Figure 3). The different patterns compute 2 to 12 values in one pass. Thus, if a transposition is to be added, several consecutive values can be written in memory. The vectorization is done using intrinsic operations, the compiler being responsible for register allocation and management.

The benchmarks use arrays of 32 columns of 5040 elements. The size is a multiple of the pattern dimension and fits in the L3 cache of the processor. The column size is much longer than what is used in BigDFT but our aim is to measure the asymptotic performances. The mean results of 20 runs are presented in table II. What we see is that even small patterns present improved performances compared to the non vectorized versions (cf. Table I). However, these patterns are not 3D and cannot be used "as is" for a 3D convolution. To be able to use the patterns, we should also use a transposition.

Performances are maximal and stable for patterns of 6 to 10 elements. Analyzing the generated assembly code reveals that the pattern of 12 elements suffers from register spilling. The other patterns, with element from 2 to 10 do not generate register spilling i.e. the compiler generates computations that fit in the 16 available registers.

2) Kernels with Transposition: For a transposed version of a convolution kernel to be efficient, it has to write several elements successively in memory. Having this in mind, we have chosen two candidate kernels: the 2*4 pattern and the 4*2 pattern.

To calculate the transposition, we have simply used the available SSE operation for manipulating vectors. The operation computes this sum for 2 vectors and stores the results in a third one. Using column-adjacent values creates a vector that can be directly stored.

The results are presented in table IV-C2. Computing more columns simultaneously is faster and the 4*2 pattern presents better performances (7.14 GFlops/s versus 6.32 GFlops/s for the 2*4 pattern).

Unfortunately, the version with transposition is less performant than the version without transposition. Attempts to use buffers to reduce the transposition cost proved unsuccessful. The 4*2 pattern was elected to create the final convolution.

D. 3D Vectorized Convolution

In order to build the magic filter in periodic boundary conditions, the pattern had to be adapted to take into account boundary conditions. In periodic conditions, column length is always a multiple of 2 and column number a multiple of 4, so alignment is not a concern. The first 4 and last 4 instantiations of the pattern have to be modified to load data from the end and the start of the column respectively. The 1D algorithm is then used 3 times to create the 3D version.

The performances of this implementation are compared to the previous ones in Table I. The SSE version with transposition reaches 6.52 GFlop/s which is more than twice the performances of the unrolled and transposed version. We can see that the generated code is more efficient as the number of instructions has been reduced by the same factor. In fact, the instruction throughput is the same, and while L1 data cache miss augmented slightly, behavior in other cache levels stay similar.

We believe that the generic case will benefit from the same optimization approach. Indeed, the technique already addresses one aligned and one unaligned element. It would just have to be adapted with an offset and modified filters.

V. OPTIMIZING 3D CONVOLUTIONS ON GPU

GPU computing [START_REF] Owens | GPU Computing[END_REF] in physics and chemistry is a strong trend. Indeed, GPUs offer impressive raw performances at the cost of refactoring part of the code. Indeed, some algorithms are not adapted to the GPU architecture and need a case by case treatment.

Most GPU programs are nowadays based on the NVIDIA's proprietary solution: CUDA [START_REF]NVIDIA Compute Unified Device Architecture[END_REF]. Nonetheless, there is a significant effort in defining standard: OpenCL [START_REF]OpenCL: Open Computing Language[END_REF]. OpenCL drivers are provided by manufacturers and almost every device nowadays support OpenCL. Portability stays a major issue.

In our convolution optimization work, we had to implement from scratch 3D convolutions for both NVIDIA and AMD architectures. In the following we start by presenting the principles of the OpenCL model. We then discuss the algorithmic details for implementing convolutions on a GPU architecture before pointing out AMD specifics. We finish with performance evaluation of our impementations.

A. OpenCL GPU Architecture

OpenCL is an open standard defined by the Khronos Group that aims at cross-platform parallel computing. Among the several types of devices defined by the standard, the GPU one corresponds to modern Graphical Processing Units. These accelerators present a high performance-over-cost, as well as performance-over-power consumption ratio. GPUs have become increasingly available in computing clusters and are targets of choice for numerical experimenters.

The GPU device in OpenCL is made of several address spaces and a set of multiprocessors. OpenCL is aimed at data parallel tasks and describes the computation in terms of workgroups composed of work-items. When executing an OpenCL function (also called kernel), work-items execute the same code.

The difference between work-items from different workgroups is the visibility of address spaces. The four address spaces are global, local, private and constant. The global address space is usually larger and with a higher latency than the local address space which is private to a workgroup. The first corresponds to the on-board RAM while the later corresponds to a user managed cache. The private memory corresponds to the registers of a work-item.

B. Convolution Implementation on GPU

The first thing to optimize when using a GPU is memory access. Reading and writing to a global memory should be coalesced. The easiest way to achieve this goal while transposing is to use a padded square buffer in local memory [START_REF] Nvidia | Opencl best practices guide[END_REF]. Transposition can be done while reading input data or writing output data. In BigDFT, data is read coalesced and,as the GPU processes the last dimension first, is stored transposed in the buffer. As the filter is 16 elements long, each column of a 16*16 buffer needs 15 more elements to be convoluted. In order for threads to execute the same code, each workitem loads 2 elements in a 32*16 buffer padded to 33*16. Work-items then compute a filtered value in private memory. The calculated value is finally stored in the result buffer in global memory. Figure 4 presents an example of work items assignation for a 4*4 block processing a filter of length 5. Buffer is in column major order. Thread 0,i 1,i ... k,i are processed simultaneously.

It has to be noted that this allocation allows memory accesses to be free from bank conflicts. This is true not only during the transposition, but also during the computation of the filtered values. Indeed, consecutive threads access consecutive elements in the buffer. As using constant memory for the filter values proved harmful to performances, the values were instead directly inserted into the code.

On more complex kernels found in BigDFT the order of the filtering operations also had an impact that could not be neglected. Indeed, once loaded into local memory, the order in which threads access the elements in the buffer has no real impact on performances (provided access do not produce bank conflicts). But trying to use identical filter coefficient in a grouped manner proved beneficial.

From the GPU parallelism point of view, there is a set of N independent convolutions to be computed. Each of the lines of n elements to be transformed is split in chunks of size N e . Each multiprocessor of the graphic card computes a group of N different chunks and parallelizes the calculation on its computing units. After the calculation of the convolution

0,0 1,0 2,0 3,0 0,1 1,1 2,1 3,1 1,2 0,2 2,2 3,2 0,3 1,3 2,3 3,3 0,0 1,0 2,0 3,0 0,1 1,1 2,1 3,1 1,2 0,2 2,2 3,2 0,3 1,3 2,3 3,3 1,1 3,3
0,0 0,1 0,2 0,3 values, these N e N elements are copied in the corresponding part of the output array, which is transposed with relation to the input. The size of the data fed to each block is identical and chosen to prevent block dependencies. When N and n are not multiples of N and N e , some data treated by different blocks may overlap. This fact has no double-counting effect since the overlap is reproduced also in the output array. Figure 5 shows the data distribution on the grid of blocks during the transposition.

C. Convolution Specifics on the AMD Architecture

AMD offers OpenCL with double precision support for it's Radeon HD line of GPU. There are 2 main differences between NVIDIA and AMD architecture regarding OpenCL programming. First AMD architecture is vector-based whereas NVIDIA architecture is scalar. Secondly AMD global memory access are split between channel and banks. Accessing data between workgroups with large power of 2 strides can prove really harmful on AMD architecture.

The most recent AMD architecture (Radeon HD 69xx) is based on vector operations of length 2 in double precision. To account for this, at runtime, when an AMD GPU is detected the filter multiply is vectorized. Large powers of 2 problem size are not the rule in BigDFT but rather the exception. So the memory channel/bank problems have not been addressed in the current version of the kernels. Nonetheless if such a stride was detected at runtime it would be possible to use a staggered offset version of the kernel.

D. Performance Evaluation of 3D Convolution Kernels

GPU performance of the kernels are evaluated using 2 test computers. Each runs an Ubuntu 11.04 linux based system using an unmodified 2.6.38-11 kernel. Compiler used is Intel 11.1 Compiler Suite. Processor is a X5550 Xeon with 4 cores and 8GiB of RAM. The setups only differ in their GPU and graphic drivers, the first one uses an NVIDIA TESLA C2070 using 270.40 drivers, the second one a RADEON HD6970 using 11.6 drivers. The 3D convolution results directly of 3 applications of the 1D convolution. Problem dimension are 124 x 17160 for 1D problems and 124 x 132 x 130 for 3D problems.

Two versions of the kernels have been tested one standard, and one where a work item computes 2 elements of the resulting array. Performance of the inverse operation is also considered, as well as the performance of the 3D convolutions. The 3D convolution results directly of 3 applications of the 1D convolution.

Results are presented in table IV. There is a slight performance advantage for AMD (10%) for these algorithm and those problem size. Using a 128 x 128 x 128 matrix halves performances on AMD while NVIDIA's performances are unaffected. Tis is due to the afore mentioned channel/bank problem on AMD hardware. This problem could be solved using staggered offsets. This approach does not need to embed the array in a larger one contrary to padding that would then need to be carried along the whole pipeline.

Similarly on NVIDIA hardware performance are decreasing for multiple of 7, but those dimension are forbidden in BigDFT for FFT performance reasons. Despite those specific problems performances are remarkably uniform regarding problem size, and variations in performances is mainly accountable on the overlapping part (and thus the waste of computing power) rather than on memory conflicts.

Most of the performance was gained on GPUs by optimizing memory accesses and by hiding computations between memory operations. Using constant memory for filter values proved less interesting than directly inserting the values inside the code using macro definitions. Computations are not completely hidden because vectorizing the computation loop on AMD hardware yields an increase of 10% in performance.

E. Performance Evaluation for BigDFT

In this section we present the impact of the OpenCL implementation and MPI configuration on the completion time of BigDFT. The system studied is a graphene sheet modeled The first important thing to note is that BigDFT is not scaling very well in MPI, due to bandwidth limitation. In our case GPUs bring not only their computing power, but also their impressive memory bandwidth. The second interesting trend is that NVIDIA GPU offer better (though comparable) performances than AMD on this problem. The situation is reversed compared to the unit tests presented in the previous section. The problem studied contains a dimension that is a power of 2: 32. So performances are seriously hindered on AMD's hardware in this case. As BigDFT is synchronous between MPI processes the hybrid case performances is limited by AMD node. Speedup is of 1.8 comparing 4 MPI + AMD and (4 MPI + NV) + (4 MPI + AMD). In this case, GPUs increase by a factor of about 10 the performance of the node considered.

For both architectures (GPU and CPU with small vector engines) the only way to increase the computation over memory access ratio and alleviate the memory bandwidth problem would be to merge different convolutions. This would yield longer filters but reduce the number of transposition needed. Strategies presented here can be adapted to fit those conditions, especially since later GPU OpenCL devices have more local memory. For the CPU our algorithm is oblivious to the filter length.

VI. TOWARDS AUTO-TUNING OF CONVOLUTIONS: CONCLUSIONS AND FUTURE WORK

In this paper we focused on optimizing the basic and most frequent operations, namely 3D convolutions, in the BigDFT nanosimulation application. This application simulates the properties of future electronic materials and because of its scalability properties has been elected official benchmark for the HPC EU project MontBlanc . [START_REF]The mont-blanc project[END_REF] We have considered optimizing 3D convolutions for two types of architectures: CPU architectures using SSE units, as well as GPU-based architectures. We have optimized BigDFT on three platforms, featuring respectively CPUs with SSE units, AMD GPUs and NVIDIA GPUs. For the first, we have explored vectorization techniques, while for the second we used intelligent cache utilization. For both types of work, the major principle has been data locality. In the case of CPUs, we have defined access patterns that define different locality groups used to optimize memory accesses, as well as computation. For GPUs, we have defined blocks of data for aggressive cache usage. All techniques are reusable in other contexts different from convolutions.

Concerning CPUs, we have used only 1 core and have not been limited by the CPU bandwidth. However, it would be interesting to consider using multiple cores, even if the performance gain would be limited. It could be beneficial to use cores for compute-intensive tasks or for network communication tasks [START_REF] Alam | Overlapping computations with communications and i/o explicitly using openmp based heterogeneous threading models[END_REF]. The transposition performance of our algorithm are 75% of those of a memory copy, so going back to a straight algorithm that would require each dimension to be treated differently is not interesting.

Concerning GPU architectures, both have presented similar behavior. A major problem has been the bandwidth usage so we should evaluate the trade-off between the block size and the consumed bandwidth. Anyway, if we consider performance on our target hardware, memory copies limit us to about 170 GFlops. Any further improvement is limited to a 70% increase in performance, and that would require efficient algorithms without overlaps and perfect recovering of computation time per memory accesses.

A pending challenge, raised by the experiments on vectorized convolutions, is the remaining parameter space to explore in order to find the best convolution version. The different configurations are meaningful not only from the computational point of view but also from a physicist point of view. Indeed, filters of different sizes have different properties in terms of convergence and accuracy. For now the filter size is fixed, but we could imagine changing this parameter in order to improve convergence speed and balance this gain with the computational performance impact.

Our future work will be to build a convolution generator that is able to produce a library of parametrized convolutions to be used for BigDFT's performance evaluation. This tool would allow a broader coverage of SSE versions as those are tedious to produce by hand, error-prone and difficult to maintain. The generator could also produce OpenCL kernels and allow better performance studies. These performance evaluations, studies and experimentations are a necessary step in understanding the behavior of HPC applications and in being able, one day, to produce a prediction performance model depending on the target architecture.

16 FFig. 1 .

 161 Fig. 1. 3D Convolution

Fig. 2 .

 2 Fig. 2. Two-filter computation where data and filters are loaded into registers.

Fig. 4 .

 4 Fig. 4. Thread allocation: example of a 4*4 block and filter of length 5.

Fig. 5 .

 5 Fig. 5. Data Distribution for 1D Convolution+Transposition on the GPU.Input data (left panel) is ordered along the N -axis, while output data (right panel) is ordered in n-axis direction (cf. Section 3). When executing GPU convolution kernel, each block of the execution grid (i,j) is associated to a set of N (N -axis) times Ne (n-axis) elements. The filled patterns in the figure indicate the overlap region, i.e. data which are associated to more than one block. Behind the (i,j) label, in light gray, is indicated the portion of data which should be copied to the shared memory to treat the data in the block, which contains also the buffers needed for computing the convolution.

• Discrete Fourier Transform (DFT)In this approach, the 3D convolution is calculated by applying the Fourier Transform to each dimension[START_REF] Goedecker | An efficient 3-dim fft for plane wave electronic structure calculations on massively parallel machines composed of multiprocessor nodes[END_REF]. It has been, for example, used in the work of Akira Nukada