M Pierrette
email: marjorie.pierrette@gmail.com

E Parizet

P Chevret

J Chatillon

Noise effect on comfort in open-space offices: development of an assessment questionnaire

This paper presents a short questionnaire aimed to evaluate the employees' comfort in an open-plan office and to propose optimal modifications of the office. Answers collected from 237 respondents showed that intelligible conversations represent the main source of noise annoyance; moreover, overall noise level is not related to this annoyance.

Abstract

It is currently accepted that noise is one of the most important annoyance factors in openspace offices. However, noise levels measured on open spaces of the tertiary sector rarely exceed 65 dB (A). It therefore appears necessary to develop a tool that can be used to assess the noise environment of these offices and identify the parameters to be taken into consideration when assessing the noise annoyance. This paper presents a questionnaire to be filled by people working in such environment, and a case study in different open plan offices.

The majority of the 237 respondents consider that the ambient noise level in their environment is high and that intelligible conversations between their colleagues represent the main source of noise annoyance. This annoyance was significantly correlated with their evaluation of sound intensity, which could not be represented by A-weighted level measurements.

Introduction

The modern open-space concept was developed by two German consultants, the brothers Eberhard and Wolfgang Schnelle, in the 1950s. This type of partitionless office layout found considerable success in the United States and has become much more widespread in Europe since 1980. Most companies have now adopted this type of work space (according to a survey conducted in 2008, 60 % French companies use open-space offices), in all business sectors.

There is no specific definition or size of the open-space office and the layout of these work spaces depends on the individual companies [START_REF] Bodin Danielsson | Office-type in relation to health, well-being and job satisfaction among employees[END_REF]. They may be simple collective offices or completely open platforms accommodating several dozen employees. The intention of these open-space offices is to improve communication between colleagues and therefore facilitate team or project work, save space and be closer to the managers. Employees nevertheless often complain that they permanently feel spied upon (e.g.

"L'open space m'a tuer" (Open-space killed me) by Alexandre des Isnards & Thomas Zuber, 2008) and that they suffer from a high level of ambient noise (noise annoyance related to the work of the other employees and the equipment).

According to one highly exhaustive survey conducted in 2010 by the Haute Ecole de Lucerne on behalf of the Swiss State Secretariat for the Economy [START_REF] Sbib | Enquête Suisse dans les bureaux[END_REF], noise is one of the main annoyance factors in open spaces. This survey agrees with several studies which demonstrated that the acoustic environment was considerably less satisfactory in open-space offices than in private offices (e.g. [START_REF] Nemecek | Noise in landscaped offices[END_REF][START_REF] Sundstrom | Office noise, satisfaction, and performance[END_REF][START_REF] De Croon | The effect of office concepts on worker health and performance: a systematic review of the literature[END_REF]Kaarlela-Tuomala et al., 2009). For example, Kaarlela-Tuomala et al. Numerous laboratory experiments have demonstrated that noise in offices has a disrupting effect on cognitive performance, such as mental arithmetic (e.g. [START_REF] Banbury | Disruption of office-related tasks by speech and office noise[END_REF], learning of associated words or a text (e.g. [START_REF] Lecompte | Extending the irrelevant speech effect beyond serial recall[END_REF][START_REF] Banbury | Disruption of office-related tasks by speech and office noise[END_REF], counting points displayed visually (e.g. [START_REF] Buchner | Irrelevant auditory material affects counting[END_REF], correction tasks (e.g. [START_REF] Jones | Disruption of proofreading by irrelevant speech: effects of attention, arousal or memory?[END_REF], understanding text and recall (e.g. [START_REF] Knez | Irrelevant speech and indoor lighting: effects on cognitive performance and self-reported affect[END_REF][START_REF] Oswald | Disruption of comprehension by the meaning of irrelevant sound[END_REF].

Noise in the work place would also appear to affect physical and mental health. Several researchers [START_REF] Pejtersen | Indoor climate, psychosocial work environment and symptoms in open-plan offices[END_REF][START_REF] Haapakangas | Perceived acoustic environment, work performance and well-beingsurvey results from Finish offices[END_REF] have stressed the importance of noise on health by comparing the declared health of people working in an open office and that of people working in a private office. They found that the percentage of occupants complaining about noise was ten times greater in large open spaces than in private offices.

The same study demonstrated an association between office size and several symptoms including headache, fatigue and difficulties in concentration. Open office occupants consider that they need to make significantly more cognitive efforts and have more symptoms related to stress than persons working in private offices. They also feel more tired and more exhausted, though contradictory results can be found in the literature. As an example, Meijer et al. (2009) noticed no long-term fatigue effects due to open plan office arrangement. But Bodin Danielsson et al.(2013) show, in a recent research, a higher 12-month prevalence of short sick leave spells among employees in open-plan offices. [START_REF] Marmot | Building health: an epidemiological study of "sick building syndrome" in the Whitehall II study[END_REF] nevertheless observed that persons able to adjust the environmental factors themselves (light, temperature) suffered less from SBS (sick building syndrome) than those who are unable to influence their environment.

Currently in France, standard NF EN ISO 3382-3, 2012 specifies the method used to measure the acoustic properties of open-space offices with furniture. This standard takes into account the factors influencing the acoustic performance of open-space offices such as furniture layout, acoustic absorption and background noise. It does not take into account, however, how the employees themselves perceive their workplace noise environment, while studies on the assessment of noise in general have demonstrated that the perceived intensity only accounted for 20 % [START_REF] Job | The influence of subjective reactions to noise and health effects of the noise[END_REF] to 25 % [START_REF] Landström | Exposure levels, tonal components and noise annoyance in working environments[END_REF] of the variance in noise annoyance felt by the individuals.

The various studies conducted on the perception of noise in open-space offices emphasise that other factors must be taken into account when assessing the noise annoyance: the noise source(s), the task to be performed, personal sensitivity to noise, working environment. The effect of these factors will be exposed in the following.

Noise source

Removal of partitions in the work space generates numerous noise sources: phones ringing, people speaking on the telephone, people speaking to each other, computer keyboards, office equipment, musical ambience or background noise, ventilation or air-conditioning system, noise outside the building, etc. [START_REF] Sbib | Enquête Suisse dans les bureaux[END_REF]. It would appear that the noise sources present in open-space offices are not all perceived in the same way and do not have the same impact on the annoyance felt. Several studies have confirmed in particular that noises considered as controllable and/or useful are less disturbing than noises considered to be uncontrollable and/or unnecessary [START_REF] Banbury | Office noise and employee concentration: Identifying causes of disruption and potential improvements[END_REF][START_REF] Haapakangas | Perceived acoustic environment, work performance and well-beingsurvey results from Finish offices[END_REF][START_REF] Kaarlela-Tuomaala | Effects of acoustic environment on work in private office rooms and open-plan officeslongitudinal study during relocation[END_REF][START_REF] Sailer | Assessing noise nuisance: an improvement-oriented approach[END_REF][START_REF] Sundstrom | Office noise, satisfaction, and performance[END_REF]. Similarly, a continuous noise such as that of the ventilation is generally considered as causing little annoyance. It is in fact easier to get used to a constant noise than to a variable noise [START_REF] Kjellberg | The effects of nonphysical noise characteristics, ongoing task and noise sensitivity on annoyance and distraction due to noise at work[END_REF]. According to these various studies, it seems that the noises considered most annoying and most disturbing for work are telephones ringing (more specifically those ringing in empty offices) and conversations (on the phone or between colleagues). Several studies indicate that the disturbance generated by conversation is largely due to the quality of speech transmission. [START_REF] Hongisto | A model predicting the effect of speech of varying intelligibility on work performance[END_REF] puts forward a model describing the disturbance in cognitive tasks according to a Speech Transmission Index (STI). To assess this model, [START_REF] Haka | Performance effects and subjective disturbance of speech in acoustically different office typesa laboratory experiment[END_REF] tested the impact of three STI levels on various cognitive tasks (2 verbal recall tasks, 1 visuospatial memorisation task, 2 verbal tasks based largely on semantics). This study demonstrated poorer performance between an STI of 0.65 and an STI of 0.10 or 0.35. However, they found no significant difference between 0.35 and 0.10. These results agree with the studies conducted by [START_REF] Jones | Auditory babble and cognitive efficiency: Role of number of voices and their location[END_REF] who demonstrated, through several laboratory experiments, that the number of errors on a short memorisation task decreases with the number of voices present during the task, i.e. when the STI decreases. The results are less good in the presence of one or two voices than in the presence of six voices. The speech level, the content and orientation of the source vary continuously, making it impossible to get used to the speech. Moreover, it has been found that reactions to noise largely depend on the nature of the task to be performed [START_REF] Beaman | Auditory distraction from low-intensity noise: A review of the consequences for learning and workplace environments[END_REF].

Task [START_REF] Kjellberg | Noise annoyance during the performance of different non-auditory tasks[END_REF] conducted a series of experiments with different more or less simple tasks (a simple and complex reaction time task, a proofreading task and a grammatical reasoning task (GRT)). They reported that the level of annoyance due to noise increases with the difficulty of the task. The disturbance is greater for the grammatical reasoning task than for a reaction time task. [START_REF] Haka | Performance effects and subjective disturbance of speech in acoustically different office typesa laboratory experiment[END_REF] indicate that a visuospatial memorisation task is not disturbed by the presence of speech. [START_REF] Baddeley | The phonological loop and the irrelevant speech effect: some comments on Neath[END_REF] explains this result by the fact that auditory information does not interfere with visual information (different coders are used).

Noise sensitivity

Individual factors may also explain the level of noise annoyance. Studies conducted on the annoyance level attributed to noise [START_REF] Moch | Les ambiances de la ville : du stress au confort[END_REF] indicate that the sensitivity level estimated by the respondents themselves is related to the perceived annoyance level. The most sensitive subjects claim that they are more exposed than the others and therefore more annoyed. According to a study conducted by [START_REF] Job | Community response to noise: a review of factors influencing the relationship between noise exposure and reaction[END_REF], noise sensitivity would be highly correlated with the subjective reactions to noise. It would explain approximately 9% of the variance in reaction. In a study conducted in 1998, Miedema and Vos reported that the difference in noise annoyance expressed between persons with low and high sound sensitivity was equal to the difference caused by a variation of 11 dB in the sound exposure.

Working environment

Some factors, not necessarily related to the sound aspect of the offices, may also be expressed in terms of perceived noise annoyance. It has been demonstrated that when employees consider that they are working in a satisfactory environment, they tend to attribute this satisfaction to their work, considering that a work situation is satisfactory when the work itself is satisfactory. On the contrary, when the work is considered unsatisfactory, the physical environment is in turn perceived negatively and, in this case, the individuals tend to see it as the source of their dissatisfaction [START_REF] Fischer | Psychologie des espaces de travail[END_REF]. Similarly, it seems important to understand how the physical comfort aspects are assessed by employees (visual comfort, thermal comfort, acoustic comfort) since each one may have an impact on the other. [START_REF] Sundstrom | Work places: The psychology of the physical environment in offices and factories[END_REF] demonstrated that assessment of comfort is subjective and that assessment of thermal comfort, for example, may be related to other factors such as noise. [START_REF] Haapakangas | Perceived acoustic environment, work performance and well-beingsurvey results from Finish offices[END_REF] also emphasised that persons working in open spaces consider the acoustic quality, as well as the thermal quality, lighting and air quality, of the offices to be significantly lower. [START_REF] Lee | Effects of control over office workspace on perceptions of the work environment and work outcomes[END_REF] studied how assessment of the working environment and job satisfaction depend on the work space layout. They measured that the more the respondents claim to be satisfied with their working environment, the less they perceive distractions. [START_REF] Lee | Can personal control over the physical environment ease distractions in office workplaces[END_REF] also showed that if employees can control their office work environment, this reduces the distraction. [START_REF] Huang | The role of environmental control on environmental satisfaction, communication, and psychological stress: effects of office ergonomic training[END_REF] demonstrate the importance of the ergonomic aspect in the workplace on improving the efficiency, perceived control and environmental satisfaction. The findings of their study indicate that environmental control is significantly and positively related to environmental satisfaction.

All these factors emphasise the importance of subjectivity of noise assessment by the employees and the need to develop a tool to record the various parameters to be taken into account when assessing noise annoyance in open-space offices. No tool capable of considering both the noise environment as perceived by employees and the consequences of this environment is currently available. The aim of this study is therefore to develop an efficient tool designed to assess how employees perceive their workplace noise environment, identify factors likely to influence this assessment and measure the impact of this environment on work and health. A questionnaire has therefore been drawn up and proposed to employees from several companies working in open-space offices. The results obtained are provided in the third section "application case".

Drawing up the questionnaire

The questionnaire was drawn up using all the information collected during the bibliographic search and during several semi-directive interviews. It consists of 67 questions grouped in four sections structured around an assessment of the employees' physical working environment, a more specific approach of the noise environment and an assessment of the consequences of this environment on the employees' health. The questionnaire is included in appendix.

In the first section of the questionnaire, "General information about yourself and your workstation", data such as sex, age, seniority in the company, seniority in the current job and number of persons working in the same work space are collected. This first section also allows us to assess the employees' satisfaction with their physical working environment.

Satisfaction regarding the physical working environment is assessed using a scale developed by [START_REF] Fleury-Bahi | Evaluer la satisfaction envers l'espace de travail: Développement d'une échelle et première validation[END_REF]. The scale, consisting of 14 items, measures satisfaction regarding the work space according to two dimensions: Control/Privacy (7 items) and Comfort/Functionality (7 items). For example, "Possibilities available to manage noise" for Control/Privacy and "Equipment available on your work space" for Comfort/Functionality. For each aspect of the physical working environment proposed, employees must indicate their satisfaction level on a 5-point scale ranging from 1 "Not at all satisfactory" to 5 "Quite satisfactory". This scale is used to obtain three scores: a global satisfaction average, a "Control/Privacy" satisfaction average and a "Comfort/Functionalities" satisfaction average. These two satisfaction domains as well as the general factor each exhibit good internal consistency. Cronbach's alpha (α) measured for this scale and each of its dimensions is 0.84 for the general factor, 0.78 for the Control/Privacy dimension and 0.77 for the Comfort/Functionality dimension) [START_REF] Fleury-Bahi | Evaluer la satisfaction envers l'espace de travail: Développement d'une échelle et première validation[END_REF]. Other scales are available to measure job satisfaction, such as the French version of the Karasek Job Content Questionnaire [START_REF] Brisson | Reliability and validity of the French version of the 18-item Karasek Job Content Questionnaire[END_REF], that of Veitch [START_REF] Veitch | A model of satisfaction with open-plan office conditions: COPE field findings[END_REF], and the French version of the Minesota-Satisfaction-Questionnaire [START_REF] Roussel | Rémunération, motivation et satisfaction au travail[END_REF]. However, a fairly short scale was preferable, in view of the number of subjects discussed in the questionnaire and a scale measuring exclusively satisfaction with the physical working space unrelated to the managerial aspect of the work.

The second section of the questionnaire, "Assessment of the noise environment of your work space", is used to assess the employees' noise environment. The general noise level perceived by the employees is measured first, then the perceived annoyance level. The noise environment is then detailed through 5 noise sources (operation of machines, ringing telephones, intelligible conversations, unintelligible conversations, people walking past), according to the perception frequency, the level of annoyance generated by each noise, the impact of these noises on work, and whether the noises are more annoying for some tasks.

These five noise sources are described in the literature as the main sources of noise annoyance in open-space offices (Nemeck et al., 1973;Sundström et al., 1994). They are also the noise sources mentioned by the employees interviewed. The perception frequency is assessed using a 5-point scale ranging from 1 "Never" to 5 "Permanently". The annoyance is also assessed using a 5-point scale ranging from 1 "Not at all (annoying)" to 5 "Quite (annoying)". The employees are then invited to indicate by "yes" or "no" whether the noise proposed is more annoying depending on some of their work activities and, if "yes", to list the various activities for which it seems more annoying.

The employees must then sort the sound sources from most annoying to least annoying. Two noise sources have been added to the five mentioned previously: noise generated by people (keyboard, opening and closing drawers) and noise related to one particular person. These two noise sources have been included with the previous ones since they were mentioned as annoying during the interviews.

A third section, "Your relation to noise in general", is dedicated to how people react to noise in general, i.e. their sensitivity level. There are 3 main noise sensitivity scales: the Weinstein Noise Sensitivity Scale their level of agreement with the statement proposed, using a 4-point scale ranging from 1 "Strongly disagree" to 4 "Strongly agree". The answers to all the questions are recoded from 0 to 3 and used to calculate a noise sensitivity score. A score of less than 1.11 indicates that the person is not sensitive to noise and a score of greater than 1.63 indicates that the person is sensitive to noise (Schütte et al., 2007a). The short version was tested [START_REF] Griefahn | Determination of noise sensitivity within an internet survey using a reduced version of the Noise Sensitivity Questionnaire[END_REF] and exhibits good internal consistency (α = 0.87).

Lastly, the fourth section of the questionnaire, "You and your health", is used to assess how the respondents perceive their own health. This last section, consisting of 15 questions taken from the SATIN questionnaire developed by Grosjean, Kop, Formet-Robert and Althaus (2012), allows the employees to self-assess their own physical and moral health. The 15 questions are presented in the affirmative and the employees indicate their level of agreement with the statement proposed, using a 5-point scale. The questions are used to calculate a general perceived health score and four specific scores: perceived physical health, perceived psychological health, perceived symptoms and perceived stress. The scores range from 1 to 5.

The scores are reversed for the last 10 questions. Scores close to 1 therefore indicate very poor health while scores close to 5 indicate a very satisfactory condition. More precisely, continuous scores greater than or equal to 3.5 indicate good health, continuous scores greater than or equal to 2.5 and strictly less than 3.5 indicate average health, and continuous scores strictly less than 2.5 indicate poor health (Grosjean, Kop, Formet-Robert & Althaus, 2012).

The scale exhibits good internal consistency (α=0.91 for global health; α=0.93 for perceived physical health; α=0.83 for perceived stress; α=0.81 for perceived pains; α=0.80 for perceived psychic health).

Application case

Completing the questionnaire

The questionnaire was proposed to employees of seven French companies working in openspace offices. Acoustic measurements, of ambient noise in particular, were taken on three of these companies (designated A, B and C). The measurements were taken at different points in space, the number varying depending on the area of the room. At each point, the A-weighted equivalent noise level was calculated over a period of 30 seconds, the measurement being 50 and 49 dBA for the three spaces. Note that the measurement variability is much higher in company C. Firstly, large differences are observed in the mean noise level between the measurement points, which is due to significant disparity in the layout (some work stations are much better insulated than others) and to different activities. In addition, for each measurement point, Figure 1 shows significant time variability. Work in this office is highly collaborative in fact, resulting in considerable movement of the employees and discussions at numerous different places during the day.

The respondents were requested to complete the questionnaire at work. They were asked to answer the questions spontaneously and anonymously. The answers are therefore subjective and specific to each person.

The answers collected were processed using statistical computation software (Statistica 10 -Statsoft).

Population

Out of all the employees interviewed in the various companies, we collected 237 questionnaires.

This figure is made up of 126 men and 111 women. The average age is 40 (SD = 11.8). The average seniority in the company is 19.3 years (SD = 12.9) and the average seniority in the current job is 3.5 years (SD = 3.3).

The scores obtained for assessment of global noise sensitivity show that, on average, the respondents are sensitive or even very sensitive to noise (mean = 2.2; SD = 0.5). The reliability analysis shows that the noise sensitivity scale exhibits good internal consistency. The Cronbach's alpha obtained is 0.84. It is consistent with the Cronbach's alpha of 0.87 obtained by Griefahn (1998).

The results of the perceived health indicate that, globally, the respondents consider themselves to be in good health (mean  3.5). However, the standard deviations are rather high, which would suggest that there is significant disparity between the answers, especially as regards the perceived stress. The reliability analysis shows that the perceived health scale exhibits good internal consistency. The Cronbach's alpha obtained for the scale measuring perceived general health is 0.89, 0.90 for perceived physical health, 0.83 for perceived psychic health, 0.77 for symptoms and 0.88 for stress. These results corroborate those of [START_REF] Grosjean | Un questionnaire d'évaluation de la santé au travail pour la prévention, le diagnostic et l'intervention -Manuel d'utilisation[END_REF].

Satisfaction regarding the work space

The scale of satisfaction regarding the work space (Fleury-Bahi & Marcouyeux, 2011) is used to obtain three scores: a global satisfaction average, a "Control/Privacy" satisfaction average and a "Comfort/Functionality" satisfaction average. The results indicate that, overall, the employees interviewed consider that their physical working environment is moderately satisfactory (mean = 2.9; SD = 0.7). However, when the "Control/Privacy" and "Comfort/Functionality" dimensions are assessed independently, we see that the employees are globally less satisfied by the aspects related to the control and private space of their physical working environment (mean = 2.5; SD = 0.8) than by the factors related to the comfort and functionality of their office (mean = 3.3; SD = 0.7). There is a significant difference between the two scores [z(236) = 11.87; p<.001].

The reliability analysis shows that the job satisfaction scale exhibits good internal consistency. We obtain a Cronbach's alpha of 0.88 for the overall scale, 0.85 for the Control/Privacy dimension and 0.81 for the Comfort/Functionality dimension. We obtain virtually the same results as Fleury-Bahi and Marcouyeux (2011).

Assessment of the workplace noise environment

The noise present on the open work space is perceived by the employees of the various companies as being high and annoying. Most employees (56 %) consider that the noise level of their working environment is high (27 %) or very high (29 %) (see Figure 2) and 58 % consider that it is annoying (32 %) or very annoying (26 %) (see Figure 3). Only 2 % of the employees consider the noise present on the work space as being "Not at all high" and 4 % as "Not at all annoying".

We calculated a Spearman's rank correlation between the global noise annoyance level and the global noise level felt on the work space (see Table 1). The result indicates that the noise level perceived and the annoyance felt are positively and strongly related (r s = 0.81; p<.01).

When the employees are requested to assess the frequency at which the various noise sources present on their workplace are perceived (operation of machines, ringing telephones, intelligible conversations, unintelligible conversations, people walking past), we see that all the noise sources proposed are in fact present on the workplace and more or less important in terms of perception frequency (see Figure 4). The noise source heard most frequently comes from intelligible conversations, with 41 % of the employees interviewed declaring that they hear it "Permanently", followed by ringing telephones, people walking past, operation of machines and unintelligible conversations. When they are requested to assess their annoyance level (see Figure 5), we observe that the noise source most present (intelligible conversations) is also the most annoying source. In contrast, the second most annoying noise source for employees, i.e. unintelligible conversations, is not the second noise source most often heard.

Ringing telephones, operation of machines and people walking are assessed in practically the same way. Concerning the noise of intelligible conversations, most employees (52 %) claim that the perceived annoyance is the same, whether they hear both people speaking or just one of them (telephone conversations).

For each noise aspect proposed (operation of machines, ringing telephones, intelligible conversations, unintelligible conversations and people walking past), we checked whether there was a link between the perception frequency and the perceived annoyance. The correlations presented in Table 1 indicate that, for all noise sources proposed, there is a positive and significant relation between the assessed perception frequency and the overall perceived annoyance level. We nevertheless observe a very low relation with the frequency at which operation of machines is perceived (r s = 0.18; p<.01).

When the employees claimed that they were "annoyed" by a noise source (answers 2 to 5 on the scale proposed), we asked them to indicate by "yes" or "no" whether they considered this annoyance to be more important depending on the task they were performing and if "yes, which". The results indicate that, for more than 50 % of the employees who answered that they were annoyed, the noise of machines (58%) and the noise of intelligible (67 %) and unintelligible (52 %) conversations seem to be even more annoying depending on the task being performed. "Telephone conversations" represent the main activity disturbed by noise, whatever the type. However, more than half of the respondents work in call centres. When we analyse these results according to the main activity of the employees (call centres or other), we observe in fact that the activity for which noise of intelligible conversations is more annoying is mainly "telephone conversations" for people working in call centres. For the others, however, the activities most often mentioned are "reading" and "writing", before "telephone conversations".

Noise annoyance and assessment of the physical working environment

The results presented in Table 1 also indicate that there is a significant relation between the level of noise annoyance perceived and the overall satisfaction regarding the work space (r s = -0.54; p<.01). This relation is negative: as the satisfaction level increases, the noise annoyance level decreases. We observe that the aspects related to the comfort and functionality of the offices are significantly correlated (r s = -0.32; p<.01) with the noise annoyance level, but less than the aspects related to control and privacy, which are both highly correlated (r s = -0.64; p<.01). The more the employees have the feeling that they are unable to control their environment and/or have no privacy, the more they claim to be annoyed by ambient noise.

Noise annoyance and individual factors

We calculated a Spearman's rank correlation between the global noise annoyance level and the level of sensitivity to the noise declared (see Table 1). The result indicates that there is a positive and significant relation between annoyance and noise sensitivity (r s = 0.34; p<.01).

We also measured the relation between the perceived noise annoyance and the declared health of the employees. We observe that there is no significant relation (p<0.01) between the global level of perceived noise annoyance and the physical health declared. In contrast, psychological health (r s = -0.26; p<.01), symptoms (r s = -0.24; p<.01), stress and overall health (r s = -0.29; p<.01) are significantly correlated with the noise annoyance level.

Considering the inversion of scores for the measurement of symptoms and stress (see Section 2. Drawing up the questionnaire), we observe that the greater the symptoms and the level of stress (score close to 1), the greater the noise annoyance.

Noise annoyance and sociodemographic factors

The following sociodemographic factors were recorded: sex, age, seniority in the company and seniority in the current job.

The Mann-Whitney variance test conducted between the "sex" and "annoyance" variables indicates that there is no difference between men and women as regards their assessment of annoyance.

We calculated a Spearman's rank correlation between the global noise annoyance level and the age of the respondents. We therefore categorised the "age" variable into five classes of ascending order. The result indicates that age and perceived annoyance are significantly related (r s = 0.18; p<.01), although the correlation is very low.

As with the "age" variable, we categorised the "seniority in the company" and "seniority in the job" variables in ascending order so as to correlate them with the noise annoyance. Noise annoyance is significantly and positively correlated with seniority in the company (r s = 0.22; p<.01) but not with seniority in the job.

Discussion and conclusion

The results of the application case demonstrate the relevance of our questionnaire to assess the physical working environment. The results corroborate the previous studies conducted insitu and can also be used to check other factors which may have an impact on assessment of the physical working environment.

In line with the previous studies of [START_REF] Landström | Exposure levels, tonal components and noise annoyance in working environments[END_REF] and [START_REF] Job | The influence of subjective reactions to noise and health effects of the noise[END_REF], the results indicate that, although the noise levels do not exceed the legal action thresholds (Directive 2003/10/EC), most of the employees interviewed consider that the noise in their work space is high or very high (56 %) and that it is annoying or very annoying (58 %). Consequently, the true noise intensity only partly accounts for the perceived noise intensity and the perceived annoyance.

The questionnaire is used to collect information on factors other than the acoustic environment in order to assess their influence on perception of the workplace noise environment or whether they are themselves affected by the presence of noise. The results of the practical case therefore demonstrate that the more the employees feel that they are unable to control their environment and have no privacy, the more they claim to be annoyed by ambient noise. These results confirm those of [START_REF] Lee | Effects of control over office workspace on perceptions of the work environment and work outcomes[END_REF].

Noise nuisances are clearly present in open-space offices. The annoyance felt by the employees and the consequences on their work and health must therefore be assessed. Lastly, note that the noise level measured is not representative of the annoyance expressed by the employees. Figure 6 compares objective noise level values and subjective annoyance values for the 3 offices in which these two measurements were taken. We subtracted 48 from the mean levels measured in each office for data comparison purposes. The subjective assessments represented are the answers to the questions "Generally, would you say that the noise level in your working environment is high" and "Generally, would you say that the noise level in your working environment is annoying". The maximum noise level is observed in office A, although the annoyance expressed by the occupants is much less than that expressed by the occupants of offices B and C. The opposite effect is observed in office C. Note that all the differences observed on Figure 5 are significant at the 0.05 level (Student's ttest), except as regards the mean noise level in offices B and C, which cannot be considered as different.

Factors other than the physical level must therefore be taken into account to describe the annoyance perceived by the employees.

In conclusion, the questionnaire would appear to be a complementary and necessary tool for physical measurements when assessing the noise environment of open-space offices. In particular, it may be used to best define the improvements required in an office and to measure the efficiency of these improvements.

(

  2009) studied employees who moved from a private office to an open space. The study highlights the negative effects of open space on 31 employees interviewed before and after the move. The sound level increased significantly and resulted in more disruptions during work, the feeling of privacy decreased, concentration difficulties increased. This study also demonstrates a lack of the beneficial effects generally associated with open-space offices: cooperation becomes less pleasant and the circulation of information is unchanged. The researchers conclude that work in open space is not recommended.

  (WNS) developed by Weinstein in 1978, the Fragebogen zur Erfassung der individuellen Lärmempfindlichkeit (LEF) developed by Zimmer and Ellermeier in 1999 and the Noise Sensitivity Questionnaire (NoiseQ) developed by Schütte in 2007. A shorter version has been created for each scale. We decided to use the reduced version of the Noise Sensitivity Questionnaire (NoiseQ) developed by Schütte et al. (2007a ; 2007b) since the constitution of the items on this scale is based on the WNS and LEF items which have been reformulated to obtain a better understanding of the content. The reduced version (NoiseQ-R) consists of 3 subscales (sleep, habitation, work) with 4 items each, making a total of 12 questions. The 12 questions are presented in the affirmative and the employees indicate

  repeated 40 times (making a total measurement duration of 20 minutes). The values collected are shown on Figure 1. Each point represents a measurement position. Since the areas of the rooms vary considerably, 6 measurements points are used for company A, 14 for B and 20 for C. For each measurement point, the figure represents the mean value of the 40 values calculated together with the associated standard deviation. The mean values obtained are 56,


Mercedes Garcia-Martinez

Loïc Barrault

Fethi Bougares

Mercedes García-Martínez

Factored Neural Machine Translation Architectures

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Neural Machine Translation (NMT) approach has been further developed in the last years [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF][START_REF] Sutskever | Sequence to sequence learning with neural networks[END_REF]. In contrast to the traditional phrased-based statistical machine translation [START_REF] Koehn | Moses: Open source toolkit for statistical machine translation[END_REF] that represents and translates the input sentence with a set of phrases, NMT uses the sequence to sequence learning architecture and the whole input sentence is considered as one unit for translation [START_REF] Cho | Learning phrase representations using RNN encoder-decoder for statistical machine translation[END_REF]. Recently, NMT is gaining more and more interest and showing better accuracy than phrase-based system translating several language pairs. In spite of these recent improvements, the NMT systems still have some restrictions and difficulties to translate. One of them is the high computational of the softmax function which requires to normalize all the output vocabulary size.

In order to solve this issue, a standard technique is to define a short-list containing only the most frequent words. This has the disadvantage of increasing the number of unknown words.

In [START_REF] Jean | On using very large target vocabulary for neural machine translation[END_REF], authors propose to carefully organise the batches so that only a subset K of the target vocabulary is possibly generated at training time. This allows the system to train a model with much larger target vocabulary without substantially increasing the computational complexity. Another alternative is proposed by [START_REF] Le | Large vocabulary SOUL neural network language models[END_REF] where a structured output layer (SOUL) is defined to handle the words not appearing in the shortlist. This allows the system to always apply the softmax normalization on a layer with reduced size.

Recently, some works have used subword units level instead of word-level for translation. In [START_REF] Sennrich | Neural machine translation of rare words with subword units[END_REF], the rare and unknown words are encoded as subword units with the Byte Pair Encoding (BPE) algorithm. The authors show that this method can generate words which are unseen at training time. Another lower level for translation is the character-level NMT, which has been presented in several works [START_REF] Chung | A character-level decoder without explicit segmentation for neural machine translation[END_REF][START_REF] Ling | Character-based neural machine translation[END_REF][START_REF] Costa-Jussà | Characterbased neural machine translation[END_REF] and showed promising results.

Factored Neural Machine Translation (FNMT) approach handles the output vocabulary size problem using factors as a translation unit. The main motivation behind this factored representation is related to the human way to learn how to construct correct sentences. In this work, the factors are referring to the linguistic annotation at word level like the Part of Speech (POS) tags. Some works have used factors as additional information for language modeling [START_REF] Bilmes | Factored language models and generalized parallel backoff[END_REF] and also applied for neural networks language models [START_REF] Niehues | Using factored word representation in neural network language models[END_REF][START_REF] Alexandrescu | Factored neural language models[END_REF][START_REF] Wu | Factored recurrent neural network language model in ted lecture transcription[END_REF]. Recently, factors have been used as additional linguistic input features to improve a word-level NMT system [START_REF] Sennrich | Linguistic input features improve neural machine translation[END_REF] as well.

This approach differs from previous works in the sense that it uses only the linguistic decomposition of the words (no surface form word level) and it is applied to the output language. Figure 1 presents the general architecture of our FNMT system where two different outputs are generated: [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF] the lemma of the word ; (2) its factors. Indeed, each word is represented by its lemma and its linguistic factors (POS tag, tense, gender, number and person). By these means, the target vocabulary size is reduced because all the derived forms of the verbs, nouns, adjectives, etc are not kept. Furthermore, new words that are not in the vocabulary using all the derived forms of the lemmas are produced.

Figure 1: Pipeline of NMT system with Factored output Multiple output neural networks were previously pro-posed [START_REF] Firat | Multi-way, multilingual neural machine translation with a shared attention mechanism[END_REF] using scheduled decoders with multiple source and target languages. In contrast to this, the FNMT system simultaneously produces several outputs. Given both outputs (lemma and factors) and linguistic resources, the final surface form is easily generated.

The rest of the paper is structured as follows: Section 2, about background work, describes the based NMT system and FNMT system. In Section 3, we describe the different experiments to go further with the FNMT architecture and to compare its results with the word-level NMT and other state of the art systems as BPE and multilingual NMT. Section 4 presents a deep analysis of the translation output of the FNMT systems. Finally, Section 5 includes the conclusions and future work.

Background systems 2.1. Attention-based Neural Machine Translation

The encoder-decoder architecture, used for NMT, consists of two recurrent neural networks (RNN), one for the encoder and the other for the decoder. The encoder maps a source sequence into a sequence of continuous space vectors and the decoder maps this representation back to a target sequence. Our trained neural translation models are based on an encoder-decoder deep neural network, equipped with an attention mechanism [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF], as described in Figure 2. This architecture consists of a bidirectional RNN encoder (as seen in stage 1 of Figure 2). An input sentence is encoded in a sequence of annotations (one for each input word), corresponding to the concatenation of the outputs of a forward and a backward RNN. Each annotation represents the full sentence with a strong focus on the current word. The decoder is composed of a conditional RNN as provided for the DL4MT winter school1 (see stage 3 of Figure 2), equipped with an attention mechanism (stage 2). The attention mechanism aims at providing weights for each annotation in order to generate a context vector (by performing a weighted sum over the annotations). The attention mechanism uses the hidden state at timestep j of the decoder RNN along with the annotation h i to generate a coefficient e ij . A softmax operation is performed over those coefficients to generate the annotation weights α ij . As described in [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF], the annotation weights can be used to align the input words to the output words. The RNN takes as input the context vector, the embedding of the previous output word (stage 4 of Figure 2), and of course, its hidden state. Finally, on stage 5 of the Figure 2, the output probabilities of the target vocabulary are computed. The word with the highest probability is selected to be the translation output at each timestep. The encoder and the decoder are trained jointly to maximize the conditional probability of the correct translation.

Factored Neural Machine Translation

The Factored neural machine translation is an extension of the standard NMT architecture which allows us generating several output symbols simultaneously as presented in For simplicity reasons, only two symbols are generated: the lemma and the concatenation of the different factors that we consider. For example, from the French word devient, we obtain the lemma devenir and the factors VP3#S, meaning that it is a Verb, in Present, 3rd person, irrelevant gender (#) and Singular. The morphological and grammatical analysis is performed with the MACAON toolkit [START_REF] Nasr | Macaon, an nlp tool suite for processing word lattices[END_REF]. MACAON POS-tagger outputs the lemma and factors for each word taking into account its context. For the very few cases when MACAON proposes multiple factors, the first proposition is taken. The decoder of the FNMT architecture presented in Figure 3 may lead to sequences with different length since lemmas and factors are generated in a synchronous stream, but in separate outputs. Indeed, each sequence of symbols ends when the end-of-sequence (<eos> ) symbol is generated with this architecture, and nothing prevents the lemma generator to output the <eos> symbol before or after the factors generator. To avoid this scenario, the length of the factors sequence is constricted to be equal to the length of the lemma sequence. This implies that to ignore the <eos> symbol for factors (to avoid shorter factors sequence) and stop the generation of factors when the lemma sequence has ended (to avoid longer factors sequence). This is motivated by the fact that the lemmas are closer to the final objective (a sequence of words) and that they are the symbols carrying most of the meaning.

In the NMT approach, the previous word is given as feedback for generating the next word. For the FNMT approach, multiple outputs are available which can be given as feedback. Consequently, several options are possible and will be explored in Section 3.3.

Once we obtain the factored outputs from the neural network, the post-process to fall back to the word representation is performed. This operation is also performed with the MACAON tool, which given a lemma and some factors, provides the word candidate. In the cases (e.g. name entities) that the word corresponding to the lemma and factors is not found, the system outputs the lemma itself.

Experiments

In order to study the behavior of the Factored model, we will explore different architectures and show their outcome. We study the different options of feedback since we have two outputs and, therefore, several combinations of these two values are possible for feedback. Moreover, we implemented a dependency in factors output in order to improve its performance. We compare them with the word-based NMT, subwords-based NMT and multiway, multilingual NMT systems.

Data processing and selection

We evaluate our experiments on the English to French Spoken Language Translation task from IWSLT 2015 evaluation campaign 3 . A data selection method [START_REF] Rousseau | XenC: An open-source tool for data selection in natural language processing[END_REF] consisting on scoring the sentences according to a in-domain language model has been applied. We have used as available parallel corpora (news-commentary, united-nations, europarl, wikipedia, and two crawled corpora) and Technology Entertainment Design (TED4 ) corpus as in-domain corpus. The data selection allows us to train the models in a faster way taking into account the sentences which contain relevant information of the domain and avoids noisy data. We also did a preprocessing to convert html entities and filter out the sentences with more than 50 words for both source and target languages. We finally end up with a selected corpus of 2M sentences (50.5 millions of words), leading to 147K unique tokens for English side and 266K unique tokens for French side.

Training

We chose the following hyperparameters to train the systems. The embedding and recurrent layers have a dimension of 620 and 1000, respectively. The model is trained with standard SGD and the minibatch size is set to 80 sentences. The learning rate is updated with the Adadelta method [START_REF] Zeiler | ADADELTA: an adaptive learning rate method[END_REF]. The norm of the gradient is clipped to be no more than 1 [START_REF] Pascanu | Understanding the exploding gradient problem[END_REF] and the weights are initialized with Xavier [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF]. The validations start at the second epoch and are performed every 5000 updates. Early stopping is based on BLEU [START_REF] Papineni | Bleu: a method for automatic evaluation of machine translation[END_REF] with a patience set to 10 (early stopping occurs after 10 evaluations without improvement in BLEU). The vocabulary size of the source and target languages is set to 30K (as the other state of the art models). For the Factored we have 30K vocabulary size for the lemmas and 142 for the factors. This allows the system to possibly generates 172K words with the MACAON tool. Once the model is trained, we set the beam size to 12 (as this is the standard value for NMT [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF]) when translating the development corpus. The models were trained during 6 days and translation takes 30 minutes, approximately.

Feedback

As explained in section 2.1, the RNN decoder is a conditional-GRU. The first GRU cell of the decoder is fed by its previous hidden state and the feedback (i.e. the previous generated symbol) with the following formulation. GRU 1 (y j-1 , s j-1 ) = (1 -z j ) s j + z j s j-1 , (1)

s j = tanh (Wfb(y j-1 ) + r j (Us j-1 )) , (2)
r j = σ (W r fb(y j-1 ) + U r s j-1 ) , (3)
z j = σ (W z fb(y j-1 ) + U z s j-1 ) , ( 4
)
where fb is the function which computes the feedback from the previous output y j-1 , s j is the internal representation, r j and z j being the reset and update gate activations. W, U, W r , U r , W z , U z are the trained parameters 5 . Tanh and σ refer to the hyperbolic tangent and the logistic sigmoid activation functions, respectively. Since we now have two outputs, we need to define what kind of feedback is more suitable for the Factored NMT system. Several solutions are possible to use either or both embeddings as feedback (see Figure 3).

The first assumption we made is highly dependent on the design of the considered factors, i.e. the lemmas are the most informative factors among all. Then, we tried using only the lemma embedding as feedback (see equation 5).

fb(y t-1 ) = y L t-1 [START_REF] Jean | On using very large target vocabulary for neural machine translation[END_REF] where y L t-1 is the embedding of the lemma generated at previous timestep.

In the same direction but with the other output information, we used only the factors embedding as feedback (see equation 6).

fb(y t-1 ) = y F t-1 [START_REF] Le | Large vocabulary SOUL neural network language models[END_REF] where y F t-1 is the embedding of the factors generated at the previous timestep.

Another straightforward operation is to sum the embeddings (technique used in [START_REF] Mikolov | Efficient Estimation of Word Representations in Vector Space[END_REF]) of the previous lemma with the embedding of the previous factors, as described in equation 7.

fb(y

t-1 ) = y L t-1 + y F t-1 (7)
While this could seem unnatural, by doing this, we hope to obtain a joint vector representation of both the lemma and the factors. We investigated whether the neural network can learn a better combination of the lemmas and factors embeddings using a linear (eq. 8) or non-linear (eq. 9) operation instead of a simple sum.

fb(y t-1 ) = (y L t-1 + y F t-1 ) • W f b (8) fb(y t-1 ) = tanh (y L t-1 + y F t-1 ) • W f b (9)
In this case, W f b are trained weights. In addition, we used the concatenation of both target embeddings as feedback using a linear (eq. 10) or non-linear (eq. 11) operation instead of the sum of them. The concatenation of the embeddings allows us to get full benefit of both outputs for the feedback of the model.

fb(y t-1 ) = [y L t-1 ; y F t-1 ] • W f b (10) fb(y t-1 ) = tanh [y L t-1 ; y F t-1 ] • W f b (11)
Table 1 presents the results obtained with systems integrating the different output embedding combinations as feedback when comparing with the state of the art systems standard NMT, NMT using BPE symbols and multilingual NMT systems.

For sake of comparison, we have computed BLEU at word level using BPE method [START_REF] Sennrich | Neural machine translation of rare words with subword units[END_REF]. We computed the subwords units in the output side of the neural network as done with Factored approach. We set the number of merge operations for the BPE algorithm, as explained in the paper [START_REF] Sennrich | Neural machine translation of rare words with subword units[END_REF], following equation 12.

#merge ops = vocab. size (30K) -#characters [START_REF] Niehues | Using factored word representation in neural network language models[END_REF] We can see that BPE performs worse (see Table 1) obtaining lower %BLEU than NMT and FNMT systems (lemma and tanh concatenation feedback). BPE system does not generate unknown words because all are encoded as BPE units. Besides this, BPE cannot improve in terms of %BLEU because the generation of BPE units sometimes provides incorrect words when producing the final word level translation.

Moreover, we also compared the FNMT system to the multiway, multilingual NMT system [START_REF] Firat | Multi-way, multilingual neural machine translation with a shared attention mechanism[END_REF]. This method can train several encoder and decoders sharing only the attention mechanism between them. In order to reproduce our experiments using the multilingual architecture, we used one input encoder (at word-level) for English and two separate decoders : French lemmas and French factors. The final word is obtained by the factors-to-word process as used with our FNMT system. As presented in Table 1, Multilingual approach performs better than all other systems at lemma and factors level. However, the performance at word level is the lowest due to the desyncronization of the two outputs which are trained independently.

On the other hand, FNMT can generate the words taking into account the relation between lemmas and factors. FNMT system performs better than BPE and multilingual systems but a bit worse than NMT system (-0.13). Nevertheless, the number of unknown words are reduced to more than halved which is not reflected by the automatic score. Section 3.6 shows examples of the FNMT performance managing the unknown words produced by NMT system. Furthermore, we see that the %BLEU score at lemma level performs almost (-0.28) like the lemma %BLEU of multilingual system. On the contrary, the %BLEU at factors level performs much lower (-3.37).

Then, we performed additional experiments to study how we can include the factors embedding for feedback to improve the performance. Firsly, we explored the possibility of having just the factors embedding as feedback. We observe in Table 1 that using factors embedding option, factors-level %BLEU score is the highest in the FNMT systems. By contrast, lemma %BLEU is very low (more than 3 point less) and has a great impact at word level evaluation performing with 3 point less of %BLEU.

Secondly, we performed the sum of the embeddings. The sum of the embeddings without the linear and tanh transformation gives better scores and less number of unknown words than using its linear and tanh operation. In addition, we have experimented the concatenation of the two outputs embeddings to give more information as feedback and learn better the combination of them. The sum and the linear transformation of the embeddings concatenation perform similar in %BLEU in all the levels. By contrast, the tanh transformation of the embeddings concatenation gets an improvement in terms of %BLEU and unknown words comparing to only lemma feedback and linear transformation of the concatenation of the embeddings. This can be due to the fact that we are using a more complete information as feedback and the model can learn how to represent the two embeddings.

Dependency models to improve factors prediction

We have evaluated the lemma and factors outputs, separately in the standard NMT system, having a different model for each one.

We saw that the lemma score performs similar for FNMT and NMT system. Nevertheless, the difference between the two systems is big (+2.5% BLEU) when evaluating the factors. Moreover, the prediction of factors should be an easy task considering that the output layer size is only 142. In order to compensate for this loss, we have explored different architectures aiming at improving the factors output.

The first experiment we performed to model the dependency consists on a chain of two models with standard NMT system. The first model has as input the source words as usual and as output the target lemmas. Then, the second model translates from the target lemmas from the first model to the target factors. This second model has the restriction of generating the same length sentences for source and target, due to the fact that each lemma has their corresponding factors. Finally, we construct the final words from the output lemmas from the first model and the output factors from the second model.

Model

%BLEU EN words -F R lemmas 37.38 F R lemmas -F R f actors 90.54 F actorstoword 33.82

Table 2: EN words -F R lemmas -F R lemmas -F R f actors chain model results in terms of %BLEU.

Table 2 shows the results of %BLEU of the output of each model step. The first model of the chain performs similar to the FNMT system at lemma-level (comparing with previous Table 1). Moreover, the score of the output of the second model is very high due to the easy task of translating from lemma to factors on the same language. Nevertheless, factors-to-word process to build the words from the outputs of the two models, obtains worse score than NMT and FNMT models (see Table 1 for comparison). This can be due to the difficulty of factors-to-word process handling the asynchronous outputs of both models trained separately, and having different alignments between the source and target words. This experiment gives us an idea of creating a dependency from lemmas to factors, due to the high performance, to help the factors output to produce higher %BLEU score.

Lemma dependency

One observation that can be made is that while generating factors could seem easier due to the small number of the possible outputs (only 142), the BLEU score is not as high as what we could expect. However, one could argue that generating a sequence of factors in French from a sequence of English words is not an easy task. In order to help the factors prediction, we contextualized the corresponding output with the lemma being generated. This creates a dependency between the lemma output and the factors output. We built models, where the factors output is directly dependent of the lemma, in order to receive more information of it as it has the main information of the word. The dependency has been implemented by including an extra input (see Figure 4 left side) which projects the lemma embeddings into the hidden layer used to generate factors. We use as main feedback only the previous lemma. We have implemented two possibilities for the lemma dependency model to give as additional input to the factors output:

1. The previous generated lemma to give it directly as recent context.

2. The current generated lemma corresponding to the factors to be generated.

Factors dependency

Another architecture has been implemented to improve the factors output performance. In order to take advantage of the information of the previous generated factors, we use them as feedback to the factors generation (see Figure 4 The results applying the dependency models techniques are presented in Table 3. We can observe that the dependency model using the previous lemma does not improve the results respect to the FNMT model without dependency. This can be due to the fact that we have already given the previous lemma as main feedback for the recurrent hidden state. On the other hand, the dependency model using the current lemma improves (+0.06) the performance of the FNMT without dependency in terms of %BLEU score on words. If we see the results of the lemma and factors level BLEU, we obtained 0.14 lower %BLEU at lemmas level when using this dependency. By contrast, the factors output obtains an improvement of almost one point of %BLEU. Moreover, the number of unknown words is the lowest value compared to the rest of the models. This model allows us to improve and specialize the produced factors with the generated lemma. However, it has an impact at lemmas output which is more correlated to the word evaluation.

The last model, using dependency with the previous factors to feed the factors output, increases the %BLEU value when comparing with the NMT and FNMT other systems. We can also observe that the %BLEU at lemma and factors level has improved. Furthermore, the number of unknown words has decreased by 4. This architecture has improvements due to the benefit of using all the previous information (lemma and factors) and input the previous factors in the specialized layer of this output.

Translating long sentences

In this section, we compare the results of the standard NMT system with the FNMT system when translating long sentences. For this, we observed the translation performance with respect to source sentences length. Figure 5 shows the %BLEU score for sentences between 10 and 100 words with intervals of 10. We can observe that FNMT system performs similar than NMT system in the intervals smaller than 80. By contrast, we can see that the FNMT system helps significantly with translating long sentences (between +1.5-3 more points of %BLEU). This improvement can be due to the expressiveness of the FNMT system generating new vocabulary.

Qualitative analysis

FNMT system generates much less unknown words than NMT system because it can cover more vocabulary. Then, we have observed some examples to confirm that the translation of the unknown words are correct and show the performance in a qualitative analysis

Translation examples of FNMT system performance

The translation examples in Table 4 show the FNMT system performance against the NMT system.

First example shows when Factored system can generate words when the NMT base system predicts unknown words. Firstly, the word 'lineage' in source sentence is translated as the reference 'lignée' by the FNMT system and mapped to 'UNK' by the NMT base system. Secondly, the word 'adaptive' is translated as 'adaptatifs' by the FNMT system, the reference translation is 'adaptés', but we can consider the FNMT choice a better translation. NMT system also mapped the word 'adaptive' to 'UNK'. Consequently, BLEU score penalizes this example in FNMT system being a correct translation.

In the second example, an NMT system translation has generated two unknown words. By contrast, FNMT system can generate correctly the two words producing 'réparent' and 'mettent ensemble'. This is due, on one hand, because the word 'réparent' appears 439 times in the word vocabulary of the NMT system so it is not sufficient frequency to include it in the shortlist. On the other hand, the lemma 'réparer' appears 8523 times in the lemmas shortlist so it is included and we are able to generate 'réparent' from the lemma and factors outputs (verb in present and third person in plural). Moreover, the verb in English 'put together' is translated to 'assamblent' in the reference which is a synonym of the FNMT system translation 'mettent ensemble'. Unfortunately, this is not well measured by %BLEU score.

Example 3 shows an FNMT system translation performing as the reference. We are able to generate the name entity 'marta' that it is not in the shortlist of the NMT system vocabulary. These examples show the potential of the FNMT system generating new words and reducing unknown words.

Dependency model comparison

We have compared the benefits of having the dependency in FNMT system to help the generation of better factors output. Table 5 shows one example where the FNMT without dependency generates a wrong factors output. Simple FNMT produces the lemma 'être' and its corresponding factors are cln-3-s (nominative clitique, third person of the singular), which are not correct. On the contrary, FNMT with dependency produces correctly, the lemmas 'nous faire' with their factors 'cln-1-p' (nominative clitique, first person of the plural) and 'v-P-1-p' (verb in present, first person of the plural), respectively. This example shows the effectiveness of the dependency method.

Conclusions

In this paper, we have proposed an NMT architecture which produces a factored representation of the target language words. Those factors are based on linguistics a priori knowledge. We have compared the Factored NMT system with other state of the art systems such as the subwords units and the multiway, multilingual NMT. We have explored different architectures of the Factored NMT system with the different options of feedback and adding a dependency in one of the outputs of the neural network. We showed that we are able to train Factored NMT systems with better performance than the state of the art systems. Hence, with the FNMT system we are able to substantially reduce the generation of unknown words. Also, the use of additional linguistic resources allows us to generate new word forms that would not be included in the standard NMT system with shortlist.

For future work, we would like to include linguistic features at the source language. It is known that this can be helpful for NMT [START_REF] Sennrich | Linguistic input features improve neural machine translation[END_REF]. Extending the approach with input factors could make the target language factors generation better. Furthermore, different attention mechanisms for each output will be explored because they could be aligned to different source words. The proposed FNMT architecture could even show better performance if applied when we translate to highly inflected languages like German, Arabic, Czech or Russian. Finally, FNMT approach will be explored in multimodal and multilingual tasks.

Figure 2 :

 2 Figure 2: Attention-based NMT system.

  Figure 3. The encoder and attention mechanism of the Figure 2 remain without modifications. However, the decoder has been modified to get multiple outputs 2 .

Figure 3 :

 3 Figure 3: Detailed view of the decoder of the FNMT system.

Figure 4 :

 4 Figure 4: Factors prediction dependency models decoder detail: lemma dependency (left) and factors dependency (right).

Figure 5 :

 5 Figure 5: Comparison in terms of %BLEU of the NMT and FNMT with factors dependency systems according to the maximum source sentence length.

Table 1 :

 1 Performance in terms of number of unknown words and %BLEU computed on word, lemma and factors of the state of the art NMT, BPE and multilingual systems and FNMT system when using different output embedding combinations as feedback.

	%BLEU

Table 3 :

 3 right side). Dependency different options of models results interms of %BLEU at word, lemma and factors level and the number of unknown words.

				%BLEU
	Model Depend.	word lemma factors #UNK
	NMT	-	34.69	-	-	1841
	FNMT -	34.56	37.44	42.44	798
	FNMT prev. lem. 34.34	37.39	42.33	831
	FNMT curr. lem. 34.62	37.30	43.36	690
	FNMT prev. fact. 34.72	37.56	43.09	794

Table 5 :

 5 Examples of translations with FNMT and FNMT with dependency.

		Words	Src			no one knows what the hell we do
		Words	Ref	personne ne	sait	ce	que	nous	faisons	.
		Words	FNMT	personne ne	sait	ce	qu'	être	l'	enfer	.
		Lemmas	personne ne	savoir	ce	qu'	être	l'	enfer	.
		Factors		pro-s	advneg v-P-3-s prep prorel cln-3-s det	nc-m-s poncts
		Words	FNMT dependency personne ne	sait	ce	que	nous	faisons	.
		Lemma	personne ne	savoir	ce	que	nous	faire	.
		Factors		nc-f-s	advneg v-P-3-s det	prorel cln-1-p v-P-1-p	poncts
		Src	set of adaptive choices that our lineage made
	1	Ref NMT	de choix adaptés établis par notre lignée de choix UNK que notre UNK a fait
		FNMT de choix adaptatifs que notre lignée a fait
		Src	enzymes that repair them and put them together
	2	Ref NMT	enzymes qui les réparent et les assemblent . enzymes qui les UNK et les UNK .
		FNMT enzymes qui les réparent et les mettent ensemble.
		Src	santa marta in north colombia
	3	Ref NMT	santa marta au nord de la colombie santa UNK dans le nord de la colombie
		FNMT santa marta dans le nord de la colombie

Table 4 :

 4 Examples of translations with NMT and Factored NMT.

https://github.com/nyu-dl/dl4mt-tutorial

The source code is available at https://github.com/merc85garcia/fnmt

https://sites.google.com/site/iwsltevaluation2015

https://www.ted.com

All the biases have been omitted for simplicity.

Acknowledgements

This work was partially funded by the French National Research Agency (ANR) through the CHIST-ERA M2CR project, under the contract number ANR-15-CHR2-0006-01.


Mercedes García-Martínez

Loïc Barrault

Anthony Rousseau

Paul Deléglise

Yannick Estève

The LIUM ASR and SLT Systems for IWSLT 2015

This paper describes the Automatic Speech Recognition and Spoken Language Translation systems developed by the LIUM for the IWSLT 2015 evaluation campaign. We participated in two of the proposed tasks, namely the Automatic Speech Recognition task (ASR) in German and the English to French Spoken Language Translation task (SLT). We present the approaches and specificities found in our systems, as well as our results from the evaluation campaign.

Introduction

This paper describes the ASR and SLT systems developed by the LIUM for the IWSLT 2015 evaluation campaign. We participated in the two tasks mentioned above, with German language for the ASR task; and English to French for the SLT task.

The remainder of this paper is structured as follows: in section 2.1, we describe the data used for both tasks and how the selection was performed. In section 2, we present the architecture of our ASR system and the results obtained on the various corpora used during the campaign. Then in section 3, we expose the architecture of our SLT system, along with its results during the campaign. Lastly, the section 4 concludes this system description paper.

Automatic Speech Recognition Task in English

In this section, we will describe the Automatic Speech Recognition system developed by the LIUM for the IWSLT 2015 campaign, as well as present the results (both in-house and official) obtained on various corpora.

Data selection for the ASR task

Performance of Natural Language Processing (NLP) systems like the ones we are going to present here can often be enhanced using various methods, which can occur before, during or after the actual system processing. Among these, one of the most efficient pre-processing method is data selection, i.e. the fact to determine which data will be injected into the system we are going to build. For this campaign, many data selection processing was done, both in ASR and SLT tasks.

Data selection for acoustic models training

For our acoustic modeling we used as a primary source the Euronews ASR 2013 dataset [START_REF] Gretter | Euronews: a multilingual speech corpus for ASR[END_REF] provided by the campaign organizers. In order to strengthen this base, we added data from various in-house sources. Then, we also collected a set of TEDx talks in German and carefully removed the off-limit talks. The Table 1 summarizes the characteristics of the data included in our ASR system acoustic models. 1: Characteristics of the acoustic data used in the LIUM ASR system acoustic models.

Corpus

Data selection for language models training

Since language models training data is constrained for the ASR task, we applied our data selection tool XenC [START_REF] Rousseau | XenC: An open-source tool for data selection in natural language processing[END_REF] on each allowed corpus at our disposal: basically all of publicly available WMT15 data and a collection of TEDx Talks closed-captions. Based on cross-entropy difference from a corpus considered as in-domain and out-of-domain data, our tool allows to perform relevant data selection by scoring out-of-domain sentences regarding their closeness to the indomain data. The table 2 summarizes the characteristics of the monolingual text data used to estimate our system language models.

Architecture of the LIUM ASR system

Our architecture is based on two separate systems, mainly based on the Kaldi open-source speech recognition toolkit [START_REF] Povey | The Kaldi speech recognition toolkit[END_REF] which uses finite state transducers (FSTs) for decoding.

A first pass is performed by using a bigram language model and deep neural network acoustic models.

5-gram Continuous Space

Language Model [START_REF] Schwenk | CSLM -a modular open-source continuous space language modeling toolkit[END_REF]. Last, an accelerated version of the consensus approach [START_REF] Mangu | Finding consensus in speech recognition: word error minimization and other applications of confusion networks[END_REF], which takes into account temporal information to speed up the processing, is applied on the confusion networks built from the 5gram rescored word-graphs.

Acoustic modeling

The GMM-HMM (Gaussian Mixture Model -Hidden Markov Model) models are trained on 13-dimensions PLP features with first and second derivatives by frame. By concatenating the four previous frames and the four next frames, this corresponds to 39 * 9 = 351 features projected to 40 dimensions with linear discriminant analysis (LDA) and maximum likelihood linear transform (MLLT). Speaker adaptive training (SAT) is performed using feature-space maximum likelihood linear regression (fMLLR) transforms. Using these features, the models are trained on the full 124.4 hours set, with 9 500 tied triphone states and 325 000 gaussians.

On top of these models, we train two separate deep neural networks (DNNs). The first one is based on TRAP features: For each frame, DNN inputs were composed of 368 TRAP coefficients computed on a sliding window of 31 frames. Each frame was constituted by the outputs of 23 Mel-scale filterbanks. Speaker adaptation was trivial: it only consists on mean subtraction applied on all frames associated to a speaker. It has been trained on the full 124.4 hours set. The DNN was built following the approach described in [START_REF] Vesely | Sequence-discriminative training of deep neural networks[END_REF] and it was composed of 6 hidden layers with 2048 units, while the output softmax layer had 4627 outputs. The second one is based on the same fMLLR transforms as the GMM-HMM models and on state-level minimum Bayes risk (sMBR) as discriminative criterion. Again we use the full 124.4 hours set as the training material. The resulting network is composed of 6 hidden layers with 2 048 units, while the output dimension is 7 827 units and the input dimension is 440, which corresponds to an 11 frames window with 40 LDA parameters each.

To speed up the learning process, each DNN is trained using general-purpose graphics processing units (GPGPU) and the CUDA toolkit for computations.

Language modeling

For language modeling, we rely on the SRILM language modeling toolkit [START_REF] Stolcke | SRILM -an extensible language modeling toolkit[END_REF] as well as the Continuous Space Language Model toolkit. The vocabulary used in the LIUM ASR systems is composed of 131 435 entries. The language models are trained on the data presented in section 2.1.2 and separate sets are trained for each system.

With the SRILM toolkit, a 2-gram LM is estimated for each corpus source, with no cut-offs and the modified Kneser-Ney discounting method. These 2-gram LM are then linearly interpolated to produce the final 2-gram LM which will be used in the final system, using the German IWSLT 2013 test corpora. To rescore the word-lattices produced by Kaldi, a trigram and a quadrigram language models (namely 3G and 4G) are estimated with the same toolkit, again by training one LM by corpus source and then linearly interpolating them. A 5G continuous-space language model (CSLM) is also estimated for the final lattice rescoring, with no cut-offs and the same discounting method as for the bigram language model. Table 3 and table 4 details the interpolation coefficients for the 2G, 3G and 4G language models as well as the final perplexity for each final model used in the two systems, respectively for the TRAP-based and the fMLLR-based system. , where n is the number of word-lattices to be merged (here, n = 2). In our experiments, we did not find significant improvements by using more tuned weights.

Corpus

For each speech segment, the use of weighted posteriors allows to merge starting (respectively ending) nodes from both lattices together into a single lattice in order to process directly with an optimized version of the consensus network confusion algorithm. This optimization reduces very significantly the computation time by managing temporal information during the clustering steps.

Results

The LIUM ASR system officialy achieved a Word Error Rate score of 17.8 on the 2015 test set, however, at this time of writing, ranks for each participant and full results have not been disclosed, thus we are not able to provide comparisons.

Spoken Language Translation Task

In this section, the architecture of our Statistical Machine Translation (SMT) system used in the SLT task is described.

Architecture of the LIUM SLT system

The SMT system is based on the Moses toolkit [START_REF] Koehn | Moses: open source toolkit for statistical machine translation[END_REF]. The standard 14 feature functions were used (i.e. phrase and lexical translation probabilities in both directions, seven features for the lexicalized distortion model, word and phrase penalty and target language model (LM) probability). In addition to these, a 5-gram Operation Sequence Model (OSM) [START_REF] Durrani | A joint sequence translation model with integrated reordering[END_REF] have been trained and included in the system.

Data processing and selection for the SLT task

All available corpora have been used to train the different components of the SMT system. The source side of the bitexts have been processed in order to make it more similar to speech transcriptions. After a standard tokenization, the processing mainly consisted in applying regular expressions to delete punctuations and unwanted characters, convert capital letters in lowercase and rewrite numbers in letters.

Once the processing performed, monolingual and bilingual data selection has been applied using XenC [START_REF] Rousseau | XenC: An open-source tool for data selection in natural language processing[END_REF]. For this purpose, the TED corpus has been used as in-domain corpus (to compute in-domain cross-entropy). The development corpus (named liumdev15 ) was used to determine the quantity of data by perplexity minimization. It is composed of the following corpora : dev2010, tst2010 tst2013.

Translation model

The translation models have been trained with the standard procedure. First, the bitexts are word aligned in both directions with GIZA++ [START_REF] Gao | Parallel implementations of word alignment tool[END_REF]. Then the phrase pairs are extracted and the lexical and phrase probabilities are computed. The weights have been optimized with MERT using two versions of the development data. For some systems, the provided transcriptions were used, and for others, the outputs of our ASR system was used. This was performed for the sake of comparing the impact of ASR systems improvement (observed during the last few years).

Language modeling

The language model is an interpolated 4-gram back-off LM trained with SRILM [START_REF] Stolcke | SRILM -an extensible language modeling toolkit[END_REF] on the selected part of the French corpora made available. The vocabulary contains all the words from the development sets, the target side of bitexts and only the more frequent words from the large monolingual corpora. The interpolation coefficient have been optimized using the standard EM procedure. The perplexity of this model on liumdev15 was 67.02.

Besides, two large context CSLM [START_REF] Schwenk | Continuous Space Language Models[END_REF] have been trained, each with a different architecture. Those models are used to rescore the n-best list of SMT hypotheses. the details of the architectures of the CSLMs as well as the perplexities obtained on the development corpus liumdev15.

Neural network machine translation system

In addition to the phrase-based SMT system, we trained a neural network machine translation (NNMT) system based on [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF] during 4 days. It is implemented in the Groundhog framework. It consists in a bidirectionnal encoder-decoder deep neural network equiped with an attention mechanism, as described in Figure 1.

We tried to translate using different values of the beam size. We can observe that in table 6 the more the beam size ture for neural machine translation. The new architecture coder (Sec. 3.2) and a decoder that emulates searching a translation (Sec. 3.1).

N x 1 x 2 x 3 x T + α t,1 α t,2 α t,3 α t,T y t-1 y t h 1 h 2 h 3 h T h 1 h 2 h 3 h T s t-1 s t Figure 1:
The graphical illustration of the proposed model trying to generate the t-th target word y t given a source sentence (x 1 , x 2 , . . . , x T ).

h conditional probability i 1 , s i , c i ), (4)
i, computed by i ). ng encoder-decoder apconditioned on a distinct equence of annotations the input sentence. Each he whole input sequence ing the i-th word of the the annotations are coma weighted sum of these [START_REF] Mangu | Finding consensus in speech recognition: word error minimization and other applications of confusion networks[END_REF] mputed by

exp (e ij ) P Tx k=1 exp (e ik ) , (6)
ij = a(s i 1 , h j )
ell the inputs around position j and the output at position idden state s i 1 (just before emitting y i , Eq. ( 4)) and the feedforward neural network which is jointly trained with ystem. Note that unlike in traditional machine translation, 3 Figure 1: Architecture of the encoder-decoder deep neural network machine translation system equiped with an attention-based mechanism. Taken from [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF]

Corpus

Beam is increased, the lower the results in BLEU.

An explanation to this is that the BLEU score differs from the internal score calculated by the model (at the output of the softmax layer). Consequently, a partial hypothesis with a low score which is pruned with a small beam size, is kept and extended when the beam size is greater. Moreover, the NN output probability distributions are known to be sharp, giving a high probability to a small number of outputs and a low probability to the rest. This can lead to worse hypotheses having higher results in final. This is an undesirable behavior, which a deeper analysis of the correlation between BLEU score and NN outputs probabilities could explain.

We used the trained NNMT model to rescore the 1000best list produced by the previous trained SMT model.

Submitted systems

A total of six systems were submitted for evaluation. Several rescoring process have been performed. For the sake of comparison, our best single SMT system has been submitted as contrastive2 as well as our best NNMT system based on Groundhog (contrastive5). This SMT system has been rescored with the two CSLM presented in previous section. contrastive3 and contrastive4 correspond to the rescoring with CSLM11 and CSLM19, respectively. Those two systems have also been rescored with the NNMT model obtained with Groundhog. The primary system corresponds to the contrastive3 rescored with Groundhog NNMT model and the contrastive1 corresponds to the contrastive4 rescored with the same NNMT model.

The results and discussion are presented in the next section.

Results and discussion

The results obtained on the development and test data are presented in Table 7.

The main observation that we can make is that all the results are coherent. Improvement obtained by rescoring with the CSLM and the NNMT model on the development corpus are reflected on the internal test (liumtst15) and the official evaluation test corpus (test2015). The gains observed by rescoring the 1000-best list of hypotheses with a high order CSLM are along previous results in the literature (around +1 BLEU point on development and test data). One can notice that the two different CSLM provide very similar results (in terms of perplexity during training and in terms of BLEU after rescoring).

During system development, we were surprised by the gains observed when rescoring with the NNMT system compared to the lower results obtained (on liumdev15 and li-umtst15). An interesting result is that the rescoring with the NNMT model provides similar results on the official test set.

A key point when applying a rescoring process is the optimization of the feature functions weights. The weights for the CSLM and the NNMT model have been optimized with CONDOR [START_REF] Berghen | CONDOR, a new parallel, constrained extension of Powell's UOBYQA algorithm: Experimental results and comparison with the DFO algorithm[END_REF], a numerical optimizer, with -BLEU as the objective function to minimize. The initial weights are set to those obtained with MERT during the SMT system tuning phase. The initial weights for the CSLM and NNMT features are set to the backoff LM weight (e.g. 0.0357). This is motivated by the fact that the LM and CSLM features have a similar distribution. After optimization, the LM had its weights decreased to 0.0314, the CSLM weight increased to 0.0391 while the NNTM feature function saw its weight highly increased (0.0486).

Conclusion

We presented the LIUM's ASR and SMT systems which participated in the ASR and SLT tracks of the IWSLT'15 evaluation campaign.

For ASR, we participated to the German transcription task, which is a new challenge to us since we built our first German systems for the campaign. We achieved an official WER of 17.8 of the 2015 test set which seems consistent with our experiments on previous development and test sets.

By rescoring with a continuous space language model, we obtained a gain of about 0.6% BLEU on the SLT test data. On top of that, an additional gain of almost %1 BLEU point is obtained by rescoring with a neural network translation model. The latter result is more surprising since the translation score of the NNMT system is significantly lower than the SMT systems.

Table 2 :

 2 Characteristics of the monolingual text data used in the LIUM ASR system language models.

	This pass gener-ates word-lattices: an in-house tool, derived from a rescoring tool included in the CMU Sphinx project, is used to rescore word-lattices with a 3-gram, then a 4-gram back-off LM and

  Before merging lattices, for each edge, these scores are replaced by its a posteriori probability. Posteriors are computed for each lattice independently, then weighted

	Coefficients 3G 4G manual transcriptions of speech 0.21 0.16 2G 0.16 Common Crawl 0.03 0.05 0.05 News Crawl 0.21 0.18 0.17 Europarl 0.04 0.06 0.07 News-Comm. 0.51 0.55 0.0.55 Perplexity 379 279 264 Table 3: Interpolation coefficients and perplexities for the bigram (2G), trigram (3G) and quadrigram (4G) language models used in the LIUM ASR TRAP-based system. 2.3. Word-lattice merging Both systems used the same audio segmentation, provided by the LIUMSpkDiarization[8] speaker diarization toolkit. Us-ing the same segmentation makes easier the merging between the two ASR outputs: final outputs were obtained by merging word-lattices provided by both ASR systems. Both systems provide classical word-lattices with usual information: words, temporal information, acoustic and lin-guistic scores. Corpus Coefficients 2G 3G 4G IWSLT14 0.016 0.014 0.012 Common Crawl 0.028 0.023 0.020 Europarl 0.075 0.090 0.097 News Crawl 0.872 0.866 0.865 News-Comm. 0.008 0.008 0.006 Perplexity 514 349 326 Table 4: Interpolation coefficients and perplexities for the bigram (2G), trigram (3G) and quadrigram (4G) language models used in the LIUM ASR fMLLR-based system. by 1 n

Table 5

 5

	shows

Table 5 :

 5 Architecture of the CSLM trained for rescoring the n-best list of SMT hypotheses. The third and fourth columns show the projection layer size and the number and size of the hidden layers, respectively.

Table 6 :

 6 Results obtained with the deep NNMT system with different values of beam size.

		size
	10	100	1000
	liumtst15 liumdev15 31.62 30.95 30.12 36.79 36.1 35.24

Table 7 :

 7 Results obtained with the submitted systems on internal dev and test corpora and the official evaluation test corpus.

	Name	CSLM rescoring rescoring NNMT	liumdev15 liumtst15 %BLEU %BLEU %BLEU %TER %BLEU %TER test2015 Case No-Case
	Primary Contrast1 CSLM19 yes CSLM11 yes Contrast2 -no Contrast3 CSLM11 no Contrast4 CSLM19 no Contrast5 --	33.81 33.82 31.81 32.81 32.70 31.62	39.61 39.65 37.35 38.36 38.28 36.79	18.51 18.53 16.95 17.54 17.56 14.88	79.06 78.96 80.61 80.04 80.07 84.69	20.02 20.10 18.36 19.02 19.03 16.98	76.41 76.29 78.01 77.31 77.45 80.38

Acknowledgements

This work was partially funded by the European Commission through the EUMSSI project, under the contract number 611057, in the framework of the FP7-ICT-2013-10 call. This work was also partially funded by the French National Research Agency (ANR) through the TRIAGE project, under the contract number ANR-12-SECU-0008-01.