Marie Garnier

Improving the Use of English in Requirements: Analysis, Results, and Recommendations

For most international industries, English is the main language of communication in technical documents. Among them, requirements are specifically designed to be easy to read and as efficient and unambiguous as possible for their users and readers. They must leave little space for personal interpretation. Non-native speakers of English often find themselves in the position of having to write requirements without extensive training in the use of English for this task, which may result in lexical, grammatical and style errors. Controlled languages are usually used as a way to go around this difficulty, but they fail to address the specific stumbling stones of nonnative speakers, and even have their own shortcomings. In this article, we present an analysis of the errors found in a corpus of requirements written in English by French speakers, and we attempt to highlight the most efficient ways to help requirements engineers limit the number of language errors in their work. Results are also relevant in the case of requirements written in English by speakers of other languages.

Introduction

As a text genre, requirements have to follow sets of rules in form and content. With the recent emphasis on the importance of requirement quality and proper training for requirements engineering, discussions have focused on higher order problems, such as gathering information for requirements, ensuring coherence in long documents or selecting criteria for requirement validation.

The role of linguistics and natural language processing (NLP) in requirement engineering and management has gained more importance over the last few years. In particular the International Requirements Engineering Conference regularly includes articles on that topic, as does the present magazine. The article "Readable requirements are not a matter of courseor are they?" (Rabeler, RE Magazine, issue 2014(4)) develops the complex notion of requirement readability, which shares connections with the present article. In addition, the two-part article "How requirements engineers can benefit from applying the NLP communication techniques" (Thomas and Georgieva, RE Magazine issues 2016(1, 2)) develops the notion of neuro NLP and discourse notions such as reframing or generalizations. Finally, the present article is a continuation of research on using NLP to improving quality in requirements, which is presented in the article "LELIE -An intelligent assistant for improving requirement authoring" (Saint-Dizier and Kang, RE Magazine issue 2015 (2)).

In this article, we propose to tackle requirement quality from another angle, that of the quality of their language and grammar. This aspect of requirements is just as crucial as questions of content in avoiding approximations and misunderstandings. Specifically, we focus on the use of English in requirements written by French native speakers.

The research presented here is the result of a project funded by the IREB Academy Program. The objective of the research was to gather data on the language errors produced by French native speakers writing requirements in English. Errors found in a corpus of requirements for two industrial domains were thoroughly analyzed and categorized in order to identify common error types in this specific type of writing, and find ways to improve language quality and readability in requirements.

Our project stems from the following initial observations:

• A preliminary analysis of non-native speakers' productions shows a large number of errors, making requirements sometimes obscure or prone to interpretation errors.

• Authoring norms exist, but they don't address the proper use of English grammar and lexicon in requirements for non-native speakers, nor do they issue warnings about the loss of intelligibility that may occur as a consequence of lexical and grammatical errors.

• As a result, requirements may include errors that decrease intelligibility and readability. This situation increases the risk of ambiguity and misinterpretation, leading to problems of misconception and lack of productivity and efficiency.

• There is a large pool of research on second language (L2) learners/users' errors, especially for English, as well as research on L2 corpora (e.g. [START_REF] Granger Sylviane | The International Corpus of Learner English. Version 2. Handbook and CD-Rom[END_REF]. However, since requirements form a specialized linguistic genre, we make the hypothesis that the task of writing requirements in English is associated with different types of writing behaviors and language errors, warranting a specific treatment.

• Editing and correcting errors manually in requirements is a very time-consuming task, especially when done collegially in meetings. Providing warnings and corrections during the initial writing process is a way to save time in the editing stage, and enable the editors to focus on other aspects of the quality of requirements [START_REF] Saint-Dizier | Features of an error correction memory to enhance technical texts authoring in LELIE[END_REF].

We have chosen to focus specifically on requirements written by French native speakers.

Research in second language acquisition has shown the influence of a speaker's first language on their use of a second language (e.g. [START_REF] Jarvis | Crosslinguistic Influence in Language and Cognition[END_REF], a phenomenon called transfer, or cross-linguistic influence. Language transfer plays a major role in error production, and gives precious indications as to the requirement engineer's intended meaning and possible remediation.

Research methodology

Research and analysis approach

Our research relies on the manual analysis of corpora of requirements written in English by French native speakers. We use the methodology of error analysis, a research method initially developed and used in the domain of Second Language Acquisition [START_REF] Corder | Error Analysis and Interlanguage[END_REF].

The main steps of error analysis include the identification, classification, and interpretation of errors.

The analysis is conducted by a single trained linguist specialized in English grammar with a background in research on linguistics-based automatic grammar checking for English. Due to the complexity of the task, the different steps are performed manually. Requirements are read and screened for errors, which are then tagged.

Overview of corpora

We analyzed 772 requirements extracted from 3 different sources in two technical domains. They are hereafter referred to as Corpus 1, Corpus 2 (both in aeronautics but coming from two different companies) and Corpus 3 (telecommunications).

Our corpora are composed only of requirements: technical documents usually contain textual parts that are of no interest for this research (e.g. introductions, summaries, definitions, contexts, diagrams, etc.), and which were therefore not included in the corpora. As a result, we present the numerical data in terms of number of requirements rather than number of words. The number of words in each requirement varies greatly, from under 10 words to over 100 words.

We controlled that these texts were indeed produced by French native speakers. It is extremely difficult to assess the actual level of English proficiency of the authors, as the corpora are compiled a posteriori. We thus don't have access to detailed information about the authors and have no opportunity to test them. However, we estimate that the authors have a B2 or at least B1 level (respectively higher and lower intermediate levels in the Common European Framework of Reference for Languages issued by the Council of Europe), partly because obtaining the jobs the authors have usually requires a B2 level. The names of the companies who provided these documents are kept anonymous at their request.

Requirement quality with respect to controlled language norms

Most companies ask that a given controlled natural language (or CNL) be used in their technical documentation. Resorting to a controlled language may be seen as a way to bypass the question of language quality in requirements, since they constrain and simplify the types of grammar and lexicon that can be used in such documents. As we explain below, it is unfortunately not a completely effective solution.

After our initial observation of the presence of errors in requirements that are supposed to follow the norms of a controlled language, we ran a preliminary investigation on the adequacy of the requirements in our corpus with respect to the standard guidelines and norms (i.e. INCOSE, IREB, general simplified natural language, [START_REF] Kuhn | A survey and classification of controlled natural languages[END_REF]). In parallel to our main investigation on errors in requirements, we have also tried to identify the main ways in which the requirements in our corpus fail to follow the recommendations of the controlled language used in the industry.

Here are some examples of the main deviations from controlled language norms found in our corpus (for a more detailed overview, see [START_REF] Saint-Dizier | LELIE -An Intelligent Assistant for Improving Requirement Authoring[END_REF]):

• Extensive use of the passive voice: the use of this form is usually not recommended.

• Extensive use of fuzzy terms (e.g. particular conditions, abnormal parameters, equivalent equipment, wherever possible, etc.): up to 4 fuzzy terms have been observed in single requirements.

• Long sentences (more than 30 words): whereas the norm usually states that sentences should not exceed 20 words, it is not unusual to find longer sentences in requirements. These long sentences often contain several subordinate clauses or complex coordination, which decrease intelligibility even further.

• Complex morphological forms: norms ban the use of the modal will to refer to the future (e.g. ventilation will be provided), as well as other complex constructions involving auxiliaries.

• Use of negation: while negation cannot be avoided in all circumstances, a number of cases should be rewritten (e.g. the solenoid shall be not activated as long as the input voltage is lower than 5mV -in this case, the sentence also includes a language error).

Definition and classification of errors

Identifying errors found in requirements

At its most basic, an error is defined as "an unsuccessful bit of language" [START_REF] James | Errors in Language Learning and Use: Exploring Error Analysis[END_REF]. The most common criterion for declaring a segment of language "unsuccessful" is grammaticality, that is to say whether or not the segment follows the rules of grammar. Acceptability is another criterion that focuses on whether or not the segment might be produced by a native speaker in an appropriate context [START_REF] Lyons | Introduction to Theoretical Linguistics[END_REF].

The concepts of "competence" and "performance" are also important in the definition of errors. Competence errors are attributable to a lack of knowledge in the language, while performance errors are due to external factors, such as lack of attention, stress or fatigue [START_REF] Corder | Error Analysis and Interlanguage[END_REF]. However, researchers have highlighted the fact that even though this distinction is theoretically relevant, it is practically impossible to distinguish competence errors from performance errors [START_REF] Thouësny | Modeling second language learners' interlanguage and its variablity: A computer-based dynamic assessment approach to distinguishing between errors and mistakes[END_REF].

We have adapted these commonly used criteria to the objectives of our project as well as to the nature of the documents in our corpus to define the types of segments we identify as errors in requirements:

• Grammaticality: segments that don't follow morpho-syntactic rules (e.g. plural agreement, modifier placement, stacks of Nouns), lexico-syntactic rules (e.g. choice of preposition after a verb), or other basic grammar rules are identified as errors.

• Acceptability: requirements need to be written as clearly and as intelligibly as possible so as to eliminate ambiguities and confusions. For this reason, we stretch the notion of acceptability to include clarity and intelligibility. As a result, segments that introduce ambiguities, lack clarity, or require an effort from the user to understand the intended meaning are identified as errors.

Classifying errors in our project

Error categories usually rely on a set of criteria, sometimes used in combination of two or three but very often used independently:

• The linguistic domain of the error (e.g. morphology, syntax, spelling, etc.);

• The part of speech bearing the error or that needs to be modified to correct the error; • The linguistic system linked with the error (e.g. agreement, verb complementation, etc.); • The description of surface phenomena (e.g. word omission, extra word, wrong word, word order, etc.).

Consider the following example:

The system shall includes a locking device.

In this sentence, the verb include should not have the ending -s since it follows a modal auxiliary (modals require the use of an uninflected verb form, or verb base, after them). Here is how this error would be described for each of the above criteria:

Linguistic domain Morphosyntax

The error is linked to the form the verb should have in its syntactic context

Part of speech Verb

Linguistic system

Complex verb forms

Surface phenomena

Wrong word

The word used should be include and not the form includes

We designed our classification system in order to obtain precise and comparable data about errors found in requirements. Previous research has posited that the type of classification needed to yield comparable data includes the rank of the linguistic unit to be taken into account for the error to become apparent (i.e. the error domain, [START_REF] Lennon | Error: some problems of definition, identification and distinction[END_REF], usually Noun Phrase, Verb Phrase, Clause, etc., as well as a number of sublevels of classification giving more detailed information, such as "Preposition selection", "Placement of modifiers", etc. [START_REF] Garnier | Utilisation de méthodes linguistiques pour la détection et la correction automatisées d'erreurs produites par des francophones écrivant en anglais[END_REF]. We also document spelling errors.

For the error given above (*The system shall includes a locking device), the form of the verb only appears as wrong if we take into account the presence of the modal auxiliary before it, as modals require an uninflected verb form. As a result the error domain is Verb Phrase. The second level of classification is Modality, since the error is linked to the use of a modal.

In addition, we introduce a distinction between what we call "central" and "marginal" categories. Central categories fit the description we gave above, while marginal categories allow for the researcher's own margin of error: since we are not specialists of the technical domains represented in the corpus, and don't have access to a list of expressions allowed in the companies the requirements come from, we use marginal categories to document segments that appear to us as errors, but may not actually be perceived as such by the requirements engineer and the readership.

Marginal categories include:

• The use of words that are either neologisms or don't normally have the meaning that seems to be intended in the requirement, but which could be specific to the field and thus acceptable in the context of requirements; • Expressions that are deemed ungrammatical or unacceptable by the researcher, but which might be standard practice in the field for which the requirements are written.

Results and discussion

Presentation of results

Overall, we have found 279 errors in 188 requirements, in a corpus of 772 requirements. This means that 1 out of 4 requirements contained at least one error. Nearly half of these contained more than one error, with a small proportion of them including up to 4 errors.

Surprisingly, Corpus 1 and Corpus 3 show a similar proportion of errors, with 18% of the requirements in these corpora containing at least one error, while Corpus 2 has nearly double the amount, with 33% of requirements containing at least one error. In all instances, the presence of errors is nowhere near negligible.

Diagram 2 shows how the proportion of errors coming from each corpora in our corpus of errors.

Errors are most often found in the domain of Noun Phrase, which accounts for about 46 % of errors, and the marginal categories, with about 26 % of errors. In each of them, one sublevel category holds the majority of errors: most errors in the Noun Phrase category are linked to modification (and mostly to modifier stacking, as we will see below), with 25 % of errors in total, and most errors in the marginal categories are linked to the use of non-standard expressions, with about 15 % of errors in total (which, as was stated above, may not be considered to be errors in the relevant technical domain).

Out the 29 final categories, marginal ones included, only 6 account for more than 5 % of errors, with 20 of them accounting for less than 3 %. However, when put together they gather 37 % of all errors. This is evidence of the wide diversity of errors found in requirements, highlighting the fact that more than a third of errors prove difficult to prevent, since each category stems from a different grammatical or lexical problem. The sublevel category we have identified as "modifier stacking" gathers segments in which a noun phrase is composed of a head noun and a string of modifiers to its left, sometimes with their own embedded modifiers. Here are a few examples of this error type:

EX1. the following probable average operational duty cycle of the X EX2. The system shall include a locking in full closed position device.

EX3. the reference computed ventilation flow (flight leg computed minimum reference flow i.e. X).

EX4. The maximum engine casing temperature shall be at 670°C and the inner fixed structure thermal blanket temperature shall be 350°C

The use of several noun modifiers in an NP is becoming increasingly common in English, especially in technical and journalistic English (Pastor-Gomez, 2011). They are favored in these two areas because they eliminate the need for prepositions and some determiners. Preposition selection and determiner selection are two of the main difficulties non-native English users face when writing in English, so we make the hypothesis that the use of noun modifiers appeals to this type of writers, as a way to try and control the number of errors they produce. However, by eliminating prepositions and sometimes plurals, such structures rely on implicit information that needs to be reconstructed by the reader. As a result, they may lead to longer reading times and difficulties in interpreting the requirement [START_REF] Biber | Longman Grammar of Spoken and Written English[END_REF]. Moreover, these errors overlap with controlled language recommendations on the use of noun complements and heavy noun phrases.

The category of "non-standard expressions", found in the marginal error categories, includes segments such as the following ones:

EX5. The tenderer shall indicate if Open source codes are embedded in this layer of the solution and case yes, the tenderer to confirm if Open source codes are compliant with Open source community licensing.

EX6. In case one analog acquisition is detected failed on Safety channel, corresponding X output label shall be sent with Failure Warning validity status.

The fact that we included such segments in the marginal categories means that even though their form may seem ungrammatical or unacceptable in general English, we recognize that they might be standard in the language of requirements. Having no way to ascertain the acceptability of their use, we chose to document them. The first requirement is an example of a structure found only in Corpus 2, and the second requirement represents a structure found only in Corpus 3. We didn't find any non-standard expressions in Corpus 1. This imbalance indicates that the use of non-standard expressions is not necessarily expected and/or accepted in all companies, and may be domain-or even company-dependent.

The second requirement is actually one example of a type of structure that takes several forms in Corpus 2. All instances of this structure contain a form of ellipsis; here are a few additional examples: EX7. On ground, when PACK is selected OFF, corresponding RARV shall stay in position From the point of view of surface syntax, these segments have different forms (e.g. a past participle followed by a participial adjective, a past participle followed by a preposition, a past participle followed by one adjective or by an adjective phrase composed of a head adjective and a modifying adverb or a PP complement), but from a semantic point of view they are built on the same model, which is close to that of verbal expressions such as to turn on, to switch off: The second term or phrase indicates the position or situation of a "mobile" element, such as a switch (e.g. OFF, ON, open, closed, failed), while the first one either specifies the action leading to that situation, or the observation of that situation (e.g. selected, detected). It is imaginable that this type of phrasing is accepted and even expected in the companies for which the requirements were written. This type of segment would therefore be an example of an ungrammatical but acceptable phrasing, and it can be seen as an efficient way to avoid the use of some prepositions.

This observation initially prompted us to exclude them from the error corpus. However, we found an alternative phrasing (selected to OFF), suggesting that the practice may not be as stable or as widely used as we initially thought. Furthermore, the ellipsis of prepositions and other words, which may be seen as increasing concision and simplicity, also creates a gap that must be filled by the reader, and may lead to ambiguities if the use of such structures is not used in the same way in all instances.

Specificity of the genre of requirements: comparison with other error corpora

In order to find out whether the errors found in requirements were similar to those found in other non-native productions, we compared our error corpus from requirements with the results of research on errors in English learner productions and research papers written by French native speakers writing in English [START_REF] Garnier | Utilisation de méthodes linguistiques pour la détection et la correction automatisées d'erreurs produites par des francophones écrivant en anglais[END_REF].

First, we looked at the distribution of errors according to the main categories, and more specifically those of Noun Phrase, Verb Phrase and Sentence and Clause. Since NPs and VPs are the minimal elements of sentences, they usually account for the highest number of errors. We did not count the proportions of NPs, VPs, clauses and sentences in each corpus, but we made the conservative assumption that they are similar. Diagram 4 shows the proportions of errors found in the comparable corpora for these three main categories. For this comparison, we only took central categories into account, since the marginal categories we identified are specific to requirements.

We notice that there is a great difference between the distribution of errors in our corpus of requirements and other corpora. Similar error rates are found for the three categories in learner productions and research papers, while requirements show significantly fewer errors linked to the VP or at the level of the clause and sentence, and significantly more on the NP.

The smaller proportion of errors in the VP in requirements could be attributed to the high level of proficiency of the writers and the fact that the requirements are proofread, which may eliminate most agreement errors. However, we make the hypothesis that the low frequency of errors in the VP is mostly due to controlled natural languages prohibiting the use of complex verb groups in requirements. This could also explain the low rate of errors at the level of the clause or sentence, as the constrained use of complex syntax helps limit errors. Conversely, requirements use more complex vocabulary and expressions, which may lead to a higher proportion of errors in the Noun Phrase.

In addition, we compared the proportion of the two most common error types in our requirements corpus with those found in our two comparison corpora. Diagram 5 shows the frequency for these error types. Diagram 4: Distribution of errors according to main category in comparable corpora

In the case of missing articles (ex. All components shall meet the requirements of [ ] table presented below), we notice that results in the two comparison corpora are similar, and are much lower than in the requirement corpus. However, determination errors in general account for 24 % (student essays) and 16 % (scientific papers) of all errors in the two comparison corpora, indicating that authors produce a more varied range of determination errors in these types of writing than in requirements, where determination errors other than missing articles are non-existent.

In the case of modifier stacking, there is a progression in the number of errors found in the three corpora, with them being marginal in the corpora of student essays. This is consistent with other studies on such structures (e.g. Pastor-Gomez, 2011), which identify them as a feature of technical, scientific or journalistic English. We should note that, since we are looking at "absolute" rather than "relative" error numbers (i.e. only the total number of errors, not the number of errors over the number of total uses of the structure), it is not surprising to see lower error rates on these structure in types of writing that typically don't make use of them. Errors linked to modifier stacking are also more varied and complex in the corpus of requirements, with up to 5 modifiers on the left of a head noun (see examples above).

Overall, the most frequent error categories correspond to "simplification" strategies, with the omission of function words or punctuation that may be perceived by the author as superfluous or expendable. Three out of the six categories have to do with "missing" words or parts of words, while the use of noun modifiers eliminates the need for prepositions. The two types of non-standard expressions we reviewed also seem to be indicative of simplification strategies. In the example from Corpus 2 (and case yes, the tenderer to confirm if…), a more grammatically acceptable version of the requirement would include more words and a more complex syntax (e.g. and if it is the case, the tenderer must confirm that…). It is also the case for the example from Corpus 3 (the segment in case one analog acquisition is detected failed can be corrected as e.g. in case one analog acquisition is detected as having failed / as being in a state of failure

Diagram 5: Frequency of two error categories in comparable corpora

We conclude from this comparison that the distribution of errors reflects the specificities of the genre of technical documents, warranting the collection and use of data in this genre, and specific treatment as an L2 production.

Summary and recommendations for training

In this article, we presented the results of a research project devoted to the analysis of language errors in requirements written in English by native speakers of French. We analyzed 772 requirements from 3 corpora in 2 different technical domains, and detected 279 language errors. These errors were categorized using a tailored classification system which includes marginal (e.g. non-standard expressions) and central (e.g. agreement errors) categories.

We found a large variety of errors, with only 6 of the total 29 categories accounting for more than 5% of the errors each. However, some general tendencies were identified. A majority of errors (62%) occur in the Noun Phrase. This result is at odds with results from error analysis in non-native speaker productions from other genres, indicating that requirements form a specialized text genre that reflects specific writing behaviors. In addition, a significant proportion of errors is linked to missing punctuation and misspelled words, and can be remedied through the use of spellcheckers and grammar checkers.

A high number of segments were found to be unacceptable, or even ungrammatical from the point of view of standard English, but might be deemed acceptable in the context of requirements. Finally, we found numerous errors linked to the use of multiple adjectives or nouns in front of the head noun of the phrase. This type of structure can lead to interpretation errors and decrease the readability of the requirement.

The 6 most frequent error types, which account for 62% of errors in total, are not equal in terms of impact on readability, and most importantly in terms of ease of correction and prevention. For example, errors on the use of articles are notoriously frequent in the productions of non-native speakers writing in English, but they are also very difficult to address in automatic grammar checking or even in in-person teaching. Training providers should therefore focus on the errors for which the remediation is relatively simple (ex. spelling errors), or errors that are really detrimental to the readability of the requirement. (ex. use of several nouns and adjectives in an NP, see our discussion or nocuous ambiguity below). The error types that we recommend trainers to address are not directly linked to the native language of the requirement authors, therefore our recommendations can be used with authors having native languages other than French, with the adjustments which the training providers will think necessary.

The importance of using a spellchecker

Despite the ubiquitous presence of spellcheckers, 7% of all errors found in requirements are linked to spelling errors.

In addition to the fact that the errors themselves may decrease the clarity of the requirements, the main problem is that they affect the credibility and image of the company or requirement engineer directly. In the eyes of a client or contractor, the presence of easily avoidable errors in technical documentation may be the telltale sign of more significant errors in other areas.

Fortunately, the distribution of these errors shows that they are not the result of a lack of spelling skills, but rather of momentary lapses (e.g. one common word spelled correctly most times and wrong a few times), and subsequent lack of editing that can be fixed relatively easily. Writing requirements is a difficult task, therefore requirements writers should be heavily encouraged to rely on spellcheckers for part of the editing process. This can be achieved by making sure authors are familiar with the use of spellcheckers and notice the corrections proposed.

Policing the use of non-standard expressions

Roughly 15% of the errors found in requirements came from the use of non-standard expressions. However, it is perfectly acceptable for requirements authors to use nonstandard expressions in their writing, since requirements writing is a technical task using specialized English. Moreover the constraints of the form of controlled English used might call for dedicated expressions.

Nevertheless, resorting to a non-standard expression should be a conscious choice. In order for their use not to decrease the readability of requirements, alternative syntax and expressions should only be used if the following criteria are met:

• The alternative expression fills a need that cannot be filled using standard syntax. For example, the alternative expression uses fewer words, or avoids the use of a structure that is not allowed in the controlled English in use.

• The non-standard expression is used by other requirements writers in the same field or the same company, and with the same exact meaning. Writers are aware that they are not using standard syntax, and know why they are using an alternative expression. Ideally, the non-standard expression is included in a writing guide or manual for the requirements writers of this field or company.

• The use of the non-standard expression is completely stable. It is used in the same way every time, with no hesitation between the non-standard expression and standard syntax, or between two similar non-standard expressions.

Improving the readability of Noun Phrases

NPs that include several modifiers, and especially in the form of other nouns, are very common in technical writing because they reduce the number of words by eliminating the need for prepositions and some determiners, and give the impression of a compact delivery of information. However, when several modifiers are stacked in front of an NP, the readability of the NP decreases, and interpretation errors may occur as readers of requirements are left to reconstruct the intended meaning.

In particular, ambiguity arises from the fact that the modifiers used in these NPs often contain their own modifiers, making it difficult to identify the exact scope of each element, and determine whether the NP demonstrates stacked (e.g. [thermal [system breakdown]]) or embedded (e.g. [[thermal system] breakdown]) modification. The use of noun modifiers further complicates the issue, since they can function as heads as well as modifiers in NPs and embedded nominal modifiers.

When discussing ambiguous structures, we must address the question of nocuous ambiguity. Ambiguity is said to be innocuous when a theoretically ambiguous text is interpreted in the same way by different readers regardless of its ambiguity; it is said to be nocuous when the ambiguity in the structure actually yields different interpretations [START_REF] Willis | Automatic identification of nocuous ambiguity[END_REF]. According to this study, nearly half of the cases of syntactic ambiguity were attributable to the use of nominal modifiers.

Research on the nocuous status of modifier stacking including nominal modifiers would be very useful in helping to identify the structures that should be corrected. However, as is visible from the examples from our corpus given in section 5, most NPs with modifier stacking include embedded and stacked modification, and usually more than 2 modifiers. We posit that these two factors are enough to create nocuous ambiguity. In addition, the lack of prepositions and determiners clarifying the relationships between the elements of the NP may lengthen reading times and mobilize cognitive resources to the detriment of other elements of the requirement.

As a consequence, in order to minimize the risk of nocuous ambiguity in NPs and to maintain fluidity in reading, we recommend that the number of modifiers placed before the head noun of an NP be limited to two adjectives or two nouns, or one of each, therefore ensuring that no more than 3 elements will be found in succession in an NP without a preposition or conjunction. There is an overlap between our recommendations and other constraints on requirements writing, since the use of heavy NPs is often discouraged in controlled English.

Training providers should make sure that requirements authors are familiar with the structure and pay attention to the readability issues that may arise when using them. In addition, trainers should help authors choose adequate ways to rewrite heavy NPs.

EX8.

  Both FCVs are selected open EX9. During PACK starting sequence the FCV is not commanded fully open EX10. In the event the RARV is failed stuck in position

  ).

	Learner productions	7% 3%
	Research papers	8%	14%			Missing article
						Modifier stacking
	Requirements	12%		24%
	0%	10%	20%	30%	40%

Further work: Implementation in the LELIE Research Platform

This article mainly deals with the analysis of non-native authors writing in English. The main errors that were identified can be expressed on the basis of patterns and implemented in the LELIE authoring platform (Garnier, 2014), (Saint-Dizier, 2015), (Saint-Dizier and Kang, 2015). In this platform, errors can be signaled in the text either by means of dedicated tags, or via specific comments when texts are in Word or Excel. When a correction is automatically induced by the system, it is then suggested in the comment.

Such an implementation would allow us to test our diagnosis and the possibility to provide authors with automatic corrections, which is often welcome but needs some control from the authors. Such corrections can also be automatically learned when they turn out to be recurrent.

The LELIE technical text authoring platform is a university prototype. It has been plugged into Word and Excel to allow authors to call LELIE from their document and to make corrections directly on their document. The LELIE platform is freely available from the authors under a creative commons license.


Minh-Quang Nguyen
email: minhquang.nguyen@telecom-sudparis.eu

Hang Nguyen
email: hang.nguyen@telecom-sudparis.eu

Eric Renault
email: eric.renault@telecom-sudparis.eu

Phan-Thuan Do

Joint source-channel coding with adaptation

Keywords: joint source channel coding, Shannon thoery, information theory, speech, unequal error protection

According to Shannon's source-channel separation theorem, the source coding and the channel coding operations should be performed independently. However, such a method does not exploit the nature of unequiprobable source output in a finite block regime so that it can not achieve good qualities in several scenarios in practical systems. In this study, we propose a JSCC strategy which takes into account the unequiprobable source output to attain better data quality than traditional methods. An adaptation is performed in order to map source code outputs to appropriate inputs of channel code. We also propose a greedy algorithm to construct parameters for the adaptation in polynomial time. Our simulation is performed on AMR compression to estimate the effect of our approach as compared to the traditional strategy. Using this approach we find significantly improved speech quality after transmission in both cases: transfer of only one AMR codec mode and transfer of all AMR codec modes.

I. INTRODUCTION

Joint source-channel coding (JSCC) is considered as one of the enabling technologies for reliable communications in which a source code and a channel code are performed sequentially on data before transmission over noisy channels. During source coding, we choose the most effective representation of data to remove all the redundancy and form the most compressed version possible. Conversely, during channel coding, we add redundant information to the compressed data prior to data transmission, and a receiver can then recover original data which contains no apparent errors. In practical systems, a perennial question is how to design a source code and a channel code that can transfer data effectively.

The relationship between the source code and the channel code was first mentioned in Shannon theory [?] which provided a remarkable insight for optimal transmission systems. It includes two parts. The direct part stated that a source can be transmitted over channels in a reliable way if the source coding rate is strictly below the channel capacity. The converse part stated that if the source coding rate is smaller than the channel capacity, reliable transmission is impossible. Shannon's theory shows the limit of what is achievable in the case of infinite time to transmit information. According to the proof of Shannon theorems about source code and channel code, traditional communication systems perform source coding and channel coding separately and independently to transfer data.

Shannon theory shows the limits of what is achievable in the case where time is infinite, i.e. the blocklength of channel code output goes to infinite. In real world systems, especially in modern real time applications, because of their finite blocklength, one cannot wait forever. Kostina et al. [?] show that data in practical systems can be transferred more efficiently than traditional methods by choosing an appropriate source and channel code: a source code is chosen with knowledge of the channel, and a channel code is chosen with knowledge of the source's distribution and distortion measurement. In their work, a bounds to show that k, the maximum number of source symbols transmissible using a given channel blocklength n, must satisfy

nC -kR(d) = nV + kV (d)Q -1 ( ) + O(logn)
under the fidelity constraint of exceeding a given distortion level d with probability . Where Q is the standard Gaussian complementary cumulative distribution function, C, V, R(d), V (d) are the channel capacity and dispersion, ratedistortion and the rate-dispersion function respectively. This is a remarkable insight, but to achieve the fundamental limit, a lot of work must be done to design codes reaching that limit.

There are many approaches to improve the traditional method in different scenarios. One of the main trends of JSCC is using Unequal Error Protection (UEP). It is based on an assumption: not all information bits are equally important, due to the different sensitivity of the source decoder to errors. UEP consists of allocating coding redundancy depending on the importance of the information bits. In published literature, mainly two different channel packet formats have been considered: variable-length channel packets with fixed-length information block (fixed-k approach) [?], fixed-length channel packets with variable length information block (fixed-n approach) [?] and variable k, n for different packets (variable-(n, k) approach) [?], [?], [?]. However, little attention has been paid to exploit the unequiprobable distribution and the unequal importances of source code output values.

In this paper, we extend the concept of UEP: instead of differing redundancy allocated to information bits, we map important source output values to appropriate channel input values which are less distorted by the noisy channel than others. Hence, by adding the same redundancy to information bits, important parts of source output are protected more efficiently than others. Simulation results are compared in terms of quality of speech data with the traditional method.

II. JOINT SOURCE CHANNEL CODING IN FINITE BLOCKLENGTH

A. Asymptotic optimality

Given a channel blocklength n, a source blocklength k, a rate distortion measurement R(d) and a channel capacity C. The output of the most advantageous source encoder is approximately equiprobable over a set of roughly exp(kR(d)) distinct messages (for large k). The use of this source encoder enables to represent most of the source outcomes within distortion d. On the other hand, the channel coding theorem shows that there exists a channel code that can distinguish M = exp(kR(d)) < exp(nC) messages with high probability if a maximum likelihood decoder is equipped in the system. Therefore, it is shown in the JSCC theorem that one can achieve high-quality transmission (in which there is small probability of distortion exceeding d) by a simple combination of both source and channel codes. However, in practical systems where n is finite, the output of the source encoder is not always equiprobable. Hence schemes that perform source decoding and channel decoding separately without exploiting unequal message probabilities may not achieve near-optimal non-asymptotic performance. In the nonasymptotic regime, there is a need to design a intelligent code that take into account the residual encoded source redundancy at the channel decoder.

B. Definitions

For practical systems where blocklengths are limited, with a specific source code and channel code equipped, we propose a method to take into account the nature of the channel and the unequiprobable source outputs to improve traditional JSCCs. In source coding, there are several values in the compressed data that are more important than others: values which appear in headers regularly or values with high probability distribution. On the other hand, there are several values of the channel code output which are less distorted than others after transfer over a noisy channel. We introduce here a new concept called adaptation, which is considered as a bijection between source code outputs and channel code inputs. The adaptation is performed at the transmitter and the receiver in order to map important source output values to channel code inputs which are less distorted by the channel.

Unlike traditional UEP strategies where protection is brought about based on the unequal importance of data positions in compressed data, our proposed adaptation capitalizes on the unequal importance of source output values. Therefore, the indispensable values are masked by those which are less distorted while being transferred. Moreover, instead of different redundancy amount allocated to compressed data, we use the same channel code rate for every source output values. As a consequence, we only need to use one channel code to protect data.

Assume that we have a source that produces sequences X ∈ X . The output messages of source code are indexed by the set {1, .., M }. Before entering the channel decoder, each source

f = f (M ) c × ad × f (M ) s g = g (M ) c × ad -1 × g (M ) s where f (M ) s : M → {1, .., M } ad : {1, .., M } → {1, .., M } f (M ) c : {1, .., M } → X g (M ) c : Y → {1, .., M } ad -1 : {1, .., M } → {1, .., M } g (M ) s : {1, .., M } → M
For a channel characterized by P Y |X and a system equipped with a specific channel code, there is an adaptation ad which maps source code outputs to channel code inputs to attain the least distortion. Work of our proposed JSCC with adaptor is described in Figure 1.

In lossy compression, if compressed data is decompressed without transferring over channel (i.e. it does not suffer any error) the rate distortion is greater than 0. If it is transferred over a noisy channel before being decompressed, errors caused by the channel make the rate distortion greater than that of decompressed data without transmission error. We define the quality degradation of data after transferring the noisy channels as follow: Definition 2 Let w 1 ∈ {1, ..., M } be the outputs of the source code from an input m; w 2 ∈ {1, ..., M } be a received value after channel coding and transfering w 2 over channel. With a specific source decoder, the rate distortion augmentation of w 1 to w 2 is given by:

s(w 1 , w 2 ) = d(g s (w 1 ), m) -d(g s (w 2 ), m)
where d( m, m) is the rate distortion between decompressed message m and original message m. Rate distortion augmentation can be interpreted as the difference of the rate distortion when source code output w 1 of m is transformed to w 2 by the noisy channel. Definition 3 With a JSCC in a finite blocklength mode, we define the probability of receiving a message ŵ after transfering over channel and channel decoding when the input of the channel encoder is w :

p( ŵ |w ) = P Y |X (y|x = f c (w ))
where ŵ = g (M ) c (y). There are many ways to map the set M to itself to form an adaptation. Adaptations have different effects. To estimate the effect of an Adaptations, we calculate the expectation of the rate distortion augmentation by the formula below:

E = w ∈{1,...,M } ŵ ∈{1,...,M } p(w ).p( ŵ | w ).s(w , ad -1 ( ŵ ))
where w is the output of the source coder and ŵ is the output of the channel decoder. Based on this formula, we proposed 2 different algorithms to find appropriate adaptations in the next section.

III. ALGORITHMS FOR THE ADAPTATION

A. Adaptation for one-value-protection

The purpose of an adaptation in this scenario is to minimize the rate distortion augmentation which is caused by errors appearing on a particular value w ∈ {1, ..., M }. If w is assigned to w, i.e. ad(w) = w and ad -1 (w) = w, the expectation of rate distortion augmentation caused by errors on w becomes:

E er(w) = ŵ∈{1,...,M }

p(w).p(w| ŵ).s(w, ad -1 ( ŵ))

To find the optimal adaptation in this situation, we solve the optimization problem:

min ad(w)=ŵ E er(w)
The minimization is over all possible assignments for ad(w), w ∈ {1, .., M }, w = w. Lemma 1. Fix ad(w) = ŵ, the optimal adaptation to minimize quality degradation caused by errors on w can be determined by the following formula:

min w2 p(w 1 , ŵ2 )s( ŵ1 , w 1 ) = N i=0 p(w 1 , ŵi 2 )s( ŵi 1 , w 1 ) in which, for 0 ≤ i < j ≤ N p(w 1 , ŵ2 i ) < p(w 1 , ŵ2 j )
and s( ŵ1 , w 1 ) < s( ŵ1 , w 1 )(i < j)

Proof. According to the rearrangement inequality:

N i=1 x i y i ≤ N i=1
x σ(i) y i for every choices of real number x 1 ≤ x 2 ... ≤ x N and y 1 ≤ y 2 ... ≤ y N , and every permutation {x σ(i) } of {x i (i = 1, ..., N )}.

Applying the rearrangement inequality for x i = p(w 1 , ŵ2 i ) and y i = s( ŵ1 , w 1 )(i = 0, ..., N ) we have the minimum of w2 p(w 1 , ŵ2 )s( ŵ1 , w 1 ). To find the optimal adaptation, we calculate over all possible assignments for ad(w) and find the minimum E er(w) by applying this lemma. In the case of one-value-protection, the steps to find the optimal adaptation in the case of one-valueprotection is described in the Algorithm 1. Solving this problem by means of an complete search is impossible since M ! cases have to be considered. Instead, we propose this greedy algorithm with fast running time to find a feasible solution. The idea is as follow: from the list of all w ∈ M in increased order of importance, at each step, we find the most appropriate value of w ad and map it to the most important value from the list. The pseudo code of this greedy algorithm is described in Algorithm 2. The effect of our JSSC with adaptation is estimated by comparing with the traditional UEP strategy

IV. SIMULATION AND RESULTS

We have conducted some experiments with and without an adaptation in two scenarios. In the first scenario, we use an adaptation to protect a particular value of compressed speech data in a fixed compression mode. Parameters for the adaptation are calculated by Algorithm 1. In the second scenario, our algorithm minimizes the rate distortion of speech data for all codec modes with the adaptation parameters calculated by Algorithm 2.

A. Simulation description

Our simulation is based on speech data with the Adaptive Multirate (AMR) audio compression [?] as the source code and the traditional convolution code [?] as the channel code. Raw speech data is compressed by the AMR compression to obtain 8 different compression rates. The convolution code with rate 1/3 is used to add redundant data to the compressed For the sakes of clarity and brevity, we use binary asymetric channel. Other channels will be considered in future works. We use Mean Opinion Score to calculate the quality of speech data after transfer over the channel. Our simulation is described in Fig. 1.

B. Simulation parameter

The blocklength of the channel code input is 8 bits. To estimate the importance of each 8-bit value in an AMR frame, we calculated the probability distribution of channel code input values in a set of 556.300 frames. We remind that an AMR frame includes 4 types of bits with different sensitivities to error: header, class A, class B and class C. We assume that the importance of each channel code input is proportional to its probability distribution. Interestingly, the highest probability distribution falls into the value of the 8-bit header which is the most important part of each AMR frame. In addition, we further assume that in each type, the rate distortion augmentation of 2 source output values is proportional to the Hamming distance between them. Consequently, a coefficient C type is a multiple of the Hamming distance between 2 values of each type to calculate the rate distortion augmentation as follows:

s(w C. Performance evaluation 1) Adaptation for one-value protection: This scenario is to protect a specific output value from the source code, when the speech data is compressed with a particular compression rate. Precisely, in AMR compression format, data are divided into frames, each frame has a header and we want to protect the 8-bit-header value of every frame in an AMR codec. The performance of the adaptation for the first scenario is shows in Fig. 2.

As seen in the figure, while the traditional method is ineffective for transfer of data over a noisy channel, by using different adaptations on different scenarios, protection for header values of every AMR frames attains considerably high quality of speech each mode, with 6 of 7 received codec modes having MOS greater than 2.5. However, we can apply these adaptations only in the case of knowing what particular AMR codec mode is transferred. The performance of adaptation given by the Algorithm 2 is estimated for the second scenario when the codec mode is not specified.

2) Adaptation for general case: For models in which the source and channel code are performed separately and independently, Fig. 3 represents a comparison of our JSCC with the adaptation model and the traditional UEP model. It was apparent beforehand that the quality of files encoded by lower bit-rates is smaller than that of files with greater bit-rates. In Fig. 3, one can observe this standard trend of the speech quality attained by different AMR compression rates. In both experiments with the traditional strategy and the JSCC with adaptation, speech quality quantified by MOS is increased over all the AMR codec mode from 4.75 to 10.02. In traditional method, although AMR data are protected by channel code, the presence of channel errors leads to disastrous quality: MOS less than 2.0 in 6 codec modes, which is a level of speech quality that is hard for human ears to distinguish. In contrast, as can be seen in the figure, the JSCC system equipped with an appropriate adaptation attained considerably higher quality speech data than the traditional model, with 6 out of 7 received codec modes having MOS greater than 2.0.

V. CONCLUSION

To improve the quality of data transmission in JSCC strategy, we have shown a new method to exploit the unequal importance of source outputs and the nature of noisy channels. As evidenced by the numerical results, the adaptation, which applies to map the source outputs to appropriate channel code inputs, can outperform the traditional UEP method to transmit speech data over noisy channel. Unless the messages has the same effect that caused by the channels while they are transferred, our JSCC with adaptation offers significant advantage in the finite blocklength mode.

Fig. 1 .

 1 Fig. 1. JSCC with adaptor

Algorithm 1 1 :Assign i to w 8 : 9 :

 189 Input: p(w 1 |w 2 ), s(w 1 , w 2 ) for all w 1 , w 2 ∈ {1, .., M } and ŵ ∈ {1, .., M } 2: Output: Set S of M pairs (w 1 , w 2 ) assigned to adaptor. Sort p() in increased order to get array p() Sort p() in increased order to get array p() algorithms for unequal error protection This section describes an algorithm to find an adaptation to decrease the rate distortion in JSCC. Let i(w, w ad ) be the indicator function of the adaptation, i.e.:i(w, w ad ) = 1 if w is mapped to w ad 0 otherwiseThe optimal adaptation can be determined by solving the following problemp( ŵad |w ad )s(w, ŵad )i(w, w ad ) subjects to: M w=1 i(w, w ad ) = 1 M w=1 i(w, w ad ) = 1

2

 2 [?]. It is clear that the complexity of Algorithm Greedy algorithm 1: Input: p(w), p( ŵad |w ad ), s(w, w ad ) for all w, w ad , ŵad ∈ {1, .., M } 2: Output: i(w, w ad ) for all w, w ad ∈ {1, .., M } 3: procedure 4:Sort {w} in decreased order of importance: S = {m i ∈ {1, .., M }|m i < m j with i < j} 5:for i ← 1 to M do 6: for j ← 1 to M do 7: i(m i , m j ) = 0 8: for doi ← 1 to M = p(w|m i ).s(w, m i ) is O(M 2 )where M is the number of source outputs.

Fig. 2 .

 2 Fig. 2. Comparison of UEP and JSCC with adaptation in one particular AMR codec mode

  Parameters ValuesCoefficients of bit typesC h = 4, C a = 3, C b = 2, C c = 1Channel Binary asymetric channel p 00 = 0.9, p 01 = 0.1 p 10 = 0.2, p 11 = 0.8,

Fig. 3 .

 3 Fig. 3. Comparison of UEP and JSCC with adaptation in one particular AMR codec mode

  1 , w 2 ) = (C h p h (w 1 )+C A p A (w 1 )+C B p B (w 1 )+C C p C (w 1 ))H(w 1 , w 2 )where p h , p a , p b , p c and C h , C a , C b , C c are the probabilities of each value and the coefficients of header, class A, class B and class C values, respectively. The parameters of the simulation are presented in TableI.


Duy-Huy Nguyen
email: duy_huy.nguyen@telecom-sudparis.eu

Hang Nguyen
email: hang.nguyen@telecom-sudparis.eu

Éric Renault
email: eric.renault@telecom-sudparis.eu

A new non-intrusive assessment method for VoLTE quality based on extended E-model

Keywords: VoLTE, voice quality, LTE, Extended E-model, MOS

Voice over Long Term Evolution (called VoLTE) is always a main service that brings a big benet for mobile operators. However, the deployment of VoLTE is very complex, specially for guaranteeing of Quality of Service (QoS) to meet quality of experience of mobile users.

The key purpose of this paper is to present an object, non-intrusive prediction model for VoLTE quality based on LTE-Sim framework and the extended E-model. This combination allows to overcome the lack of the E-model is that how to determine exactly its inputs. Besides, we propose to add the jitter buer as an essential input factor to the E-model. The simulation results show that our proposed model can predict voice quality in LTE network via both the E-model and the Extended E-model. The proposed model does not refer to the original signal, thus, it is very suitable for predicting voice quality in LTE network for many dierent scenarios which are congured in the LTE-Sim framework. The simulation results also show that the eect of jitter buer on user perception is very signicant.

Introduction

LTE network is developed by the Third Generation Partnership Project (3GPP) [START_REF]3rd Generation Partnership Project (3GPP)[END_REF]. It is a mobile network which has high data rates, low latency and fully packet-based. It means to improve the capability of legacy system by increasing data rates and extending superior QoS for various multimedia applications. Voice over LTE network (called VoLTE) is a main service of LTE network. Since LTE network is a full packet-switched, thus, the deployment of VoLTE service is very complicated. All voice tracs over LTE network are VoIP (Voice over Internet Protocol) including VoLTE. According to [START_REF] Hyun | A volte trac classication method in lte network[END_REF], there are two types of voice trac over LTE network, those are VoLTE and VoIP. VoLTE is really the VoIP service with QoS guaranteed [START_REF] Hyun | A volte trac classication method in lte network[END_REF]. Table 1 represents the service classes in LTE network where voice service is a Guaranteed Bit Rate (GRB) service which has the second priority just after IP Multimedia Subsystem (IMS) signaling. However, in order to guarantee VoLTE quality is an extreme challenge. In communications systems, the perceived voice quality is usually represented as the MOS. MOS can be attained by many methods. These methods are divided into two groups called subjective methods and objective ones. Subjective methods humans listening to a live stream or a recorded le and rating it on a ratio of 1 (poor) to 5 (excellent) [START_REF]Methods for subjective determination of transmission quality[END_REF]. These methods have some disadvantages such as too expensive, time consuming and are not suitable for a large network infrastructure. Otherwise, objective methods have more advantages, they eliminate the limitations of subjective methods. Objective methods are classied into two approaches: intrusive and non-intrusive ones. The intrusive methods (e.g.

Perceptual evaluation of speech quality (PESQ) [START_REF]Perceptual evaluation of speech quality (pesq): An objective method for end-to-end speech quality assessment of narrowband telephone networks and speech codecs[END_REF]) are more exact and are widely utilized to predict aware voice quality. However, they are not suitable for real-time services such as VoIP because they require original signals to refer. The non-intrusive methods (e.g. ITU-T E-model [START_REF]The e-model, a computational model for use in transmission planning[END_REF]) are computational models that are used for transmission planning purposes. They are not as accurate as the intrusive approaches and they do not have complex mathematical operations.

The obtained results from objective methods do not always well relate to human perception. The main advantage of the non-intrusive methods are they predict voice quality without any reference to the original signals and they require less parameters than the intrusive methods. For several typical non-intrusive methods, authors in [START_REF] Radhakrishnan | A non-intrusive method to assess voice quality over internet[END_REF] proposed to use Random Neural Network (RNN) to assess voice quality over internet. Voice quality assessment was predicted in [START_REF] Larijani | Voice quality in voip networks based on random neural networks[END_REF] using RNN. Another non-intrusive method was proposed in [START_REF] Ghalut | Non-intrusive method for video quality prediction over lte using random neural networks (rnn)[END_REF] based on RNN for evaluating video quality in LTE network. Authors in [START_REF] Nguyen | A dynamic rate adaptation algorithm using wb e-model for voice trac over lte network[END_REF] proposed to use Wideband (WB) E-model to predict VoLTE quality for minimizing redundant bits generated by channel coding. In [START_REF] Ding | Speech quality prediction in voip using the extended emodel[END_REF], authors investigated the eects of PLR and delay jitter on VoIP quality to assess prediction errors of MOS for the E-model.

A new model called Packet-E-Model was proposed in [START_REF] Meddahi | packet-e-model": e-model for wireless voip quality evaluation[END_REF] to measure speech quality perception for VoIP in Wimax network. In [START_REF] Carvalho | An e-model implementation for speech quality evaluation in voip systems[END_REF], authors presented a voice quality measurement tool based on the E-model. A framework of objective assessment method for estimating conversational quality in VoIP was proposed in [START_REF] Takahashi | Objective assessment methodology for estimating conversational quality in voip. Audio, Speech, and Language Processing[END_REF]. In [START_REF] Assem | Monitoring voip call quality using improved simplied e-model[END_REF], a simplied versions of the E-model were proposed to simplify the calculations and focus on the most important factors required for monitoring the call quality. According to our knowledge, at present, there are not any proposals which allow to predict VoLTE quality using LTE-Sim framework and extended E-model.

We see that, the lack of the E-model is how to measure exactly the its input parameters. Therefore, we need a method to measure the input factors of the Emodel. In this paper, we propose to use the LTE-Sim [START_REF] Piro | Simulating lte cellular systems: an open-source framework[END_REF] to calculate delay and PLR for voice users. LTE-Sim is a famous framework which allows to simulate entire LTE network, thus, it is quite similar to a real system. Outputs of LTE-Sim are Delay and PLR which are essential input parameters of the E-model.

In addition, we also take eects of network jitter into account by considering jitter parameter (I j ) as an input parameter of the E-model. In order to obtain more real results, we simulate voice service with mobility in LTE heterogeneous network.

The rest of this paper is organized as follows: Overview of the system model is described in section 2. In section 3, we present the proposed model. The simulation results and performance evaluation of the proposed model are analysed in section 4. The conclusion and future work is represented in section 5.

2

The system model The IP header is then compressed by RoHC down to only 1-4 bytes, normally 3 bytes.

LTE-Sim allows to simulate a heterogeneous trac over LTE network. Therefore, it is quite similar to a real system. The outputs of this software including Fig. 1. The implemented protocol stack in LTE-Sim [START_REF] Piro | Simulating lte cellular systems: an open-source framework[END_REF] many parameters where there are delay and PLR. LTE-Sim supports multicell/multiuser with mobility in a heterogeneous network. The details of the simulation scenario is represented in Section 3.

2.2

The extended E-model E-model is a computational model developed and standardized by ITU-T [START_REF]The e-model, a computational model for use in transmission planning[END_REF]. It is used to estimate the MOS for narrow band audio quality. The output of the model is R-factor. The values of this R-factor in range of 0-100 where 100 is the best and 0 is the worst quality. And then, it is mapped to the corresponding MOS value. The standard R-factor in the E-model is dened as follows:

R = R 0 -I s -I d -I ef + A (1)
In which: R 0 : The basic signal-to-noise ratio which consists of noise sources such as circuit and room noise. In this model, its value is set to 94.2. I s : The simultaneous impairment factor, it is the sum of all impairments which may occur more or less simultaneously with the voice transmission. In this model, the default value is set to 0. I d : The delay impairment factor, representing all impairments due to delay of voice signals. I ef : The equipment impairment factor, capturing the eect of signal distortion due to low bit rates of the codec and packet losses of random distribution. A: The advantage factor, capturing the fact that some users can accept a reduction of quality due to the mobility of cellular networks. In this model, this factor is set to 0.

In above factors, I d and I ef are aected by end-to-end delay and packet loss, respectively, while R 0 and I s do not depend on network performance. The R-factor is then translated into the MOS as follows [START_REF]The e-model, a computational model for use in transmission planning[END_REF]:

M OS =          1, if R < 0 1 + 0.035 × R + 7 × 10 -6 × R × (R -60) × (100 -R), if 0 ≤ R ≤ 100 4.5, otherwise (2)
The relation between R-factor, user perception, and MOS is described in the

R = 94.2 -I d -I ef (3)
We see that, when voice packet transmitted over an IP network, it is aected by many network impairments such as PLR, delay, jitter, etc. In the E-model, there is no presence of network jitter. In order to improve use satisfaction, we propose to add the I j factor to the E-model. It's obvious, when add I j to the E-model, user perception of the extended E-model will be lower than E-model,

and the E-model can be described as the following formula:

R = 94.2 -I d -I ef -I j (4)
Equation ( 4) shows that the R-factor depends on end-to-end delay (I d ), total loss probability (I ef ), and network jitter (I j ). Hence, in order to compute the R-factor, we must to count these factors. The I d is a factor which is aected by end-to-end delay and is calculated as follows [START_REF] Olariu | Voip quality monitoring in lte femtocells[END_REF]:

I d = 0.024 × d + 0.11 × (d -177.3) × H(d -177.3) (5)
In which: H(x) is the Heavyside function:

H(x) = 0, if x < 0 1, otherwise (6)
In equation ( 5), d represents the total end-to-end delay (or mouth-to-ear delay) of speech packet. It is one of output results of LTE-Sim software. The I ef is determined according to packet loss. In order to compute this factor, we use the equation in [START_REF] Mushtaq | Qoe-based lte downlink scheduler for voip[END_REF] as follows:

I ef = λ 1 + λ 2 × ln(1 + λ 3 × e l ) (7)
Where: The λ 1 represents the voice quality impairment factor caused by the encoder, λ 2 and λ 3 represent the eect of loss on voice quality for a given codec.

Such that, these factors depend on the voice codec used. In this study, we use LTE-Sim [START_REF] Piro | Simulating lte cellular systems: an open-source framework[END_REF] to simulate. This simulation tool supports only G.729 codec, thus, for this codec, the factors above has values as follows: λ 1 = 11, λ 1 = 40, λ 3 = 10. While e l is the total loss probability (consisting of network and buer layout) which has the value in range of 0..1. This factor is also obtained from the output results of LTE-Sim software.

The I j represents the impacts of network jitter to voice quality. It also depends on the voice codec. In this paper, we use the method proposed in [START_REF] Ding | Speech quality prediction in voip using the extended emodel[END_REF] as follows:

I j = C 1 × H 2 + C 2 × H + C 3 + C4 × exp( -T K ) (8)
In which:

C 1 , C 2 , C 3 , C 4
The R-factor is then mapped to the MOS via equations [START_REF] Hyun | A volte trac classication method in lte network[END_REF]. MOS reects the perception of user. It is obvious, the higher MOS, the higher user satisfaction and inverse. How to determine exactly the perception of voice user is a big challenge of mobile operators.

The proposed model

Voice over LTE network (VoLTE) is a real-time service, and is fully deployed over an IP network, thus, in order to ensure VoLTE quality is very complex. There are very little methods that allow to monitor and to predict VoLTE quality and most of them are subjective ones which have to refer to the original signal, thus, they are not suitable for real-time services such as VoIP, Video, Gaming, etc.

In this paper, we propose a new non-intrusive voice quality assessment method which based on LTE-Sim software [START_REF] Piro | Simulating lte cellular systems: an open-source framework[END_REF] and the extension of the E-model [START_REF]The e-model, a computational model for use in transmission planning[END_REF].

LTE-Sim is a software which allow to simulate VoIP ow that is quite similar to real ow. Therefore, we see that, the simulation results are quite exact. The E-model is a computational model which allows to predict voice quality when it is transmitted from source to destination. However, in this model, there is not presence of network jitter. For VoIP ow, network jitter aects signicantly on voice quality, thus, we propose to add the eects of network jitter to E-model via I j factor. When the I j is added to the E-model, the value of R factor will be lower, it leads to the lower user perception when compared to the standard E-model. The proposed model is represented on Figure 2. factor, and then it is mapped to MOS score. This MOS score performs the user satisfaction. The steps of the proposed model can be described as follows:

Step 1: Setting input parameters of the simulation scenario.

Step 2: Simulating the scenario in the LTE-Sim software

Step 3: Collecting the delay, PLR results from Step 2, and selecting values of JB.

Step 4: Feeding the input parameters taken from Step 3 to the Extended E-model which is programmed in Matlab software [START_REF][END_REF] as follows:

• Calculating R factor of the extended E-model according to Equation ( 9).

• R is then mapped to MOS score based on Formula (2).

After Step 4, we will receive the user satisfactions according to the number of user.

LTE is a packet-switched network, thus, in order to simulate it as a real system, we select input data ows including a VoIP, a Video and a non real-time ow. In addition, the mobility is also included in our scenario, specically, we select the speed of user is 30 and 120 km/h. We select the M-LWDF scheduler which is very suitable for real-time services. Besides predicting the user perception, we assess the eects of the Delay, PLR on user satisfaction according to the number of user.

4 Simulation environment and Performance evaluation is a bursty application that is modelled with an ON/OFF Markov chain [START_REF] Chuah | Characterizing packet audio streams from internet multimedia applications[END_REF].

For the video ow, a trace-based application that generates packets based on realistic video trace les with a bit-rate of 242 kbps was used in [START_REF] Reisslein | H. 264/AVC and SVC Video Trace Library: A Quick Reference Guide[END_REF] and it is also available in [START_REF] Piro | Simulating lte cellular systems: an open-source framework[END_REF]. In order to obtain a realistic simulation of an H. Simulation parameters: In this paper, we use M-LWDF scheduler for simulating. This scheduler is very suitable for voice trac ow. The simulation process is performed in a single cell with interference with the number of users in the interval [START_REF] Ding | Speech quality prediction in voip using the extended emodel[END_REF]50] which move randomly at a speed of 30 and 120 km/h. The other basic parameters used in the simulation are represented in the Table 3.

Performance evaluation

In order to simulate the trac model in LTE-Sim, we use Modied Largest

Weighted Delay First (M-LWDF) [START_REF] Ameigeiras | Performance of the m-lwdf scheduling algorithm for streaming services in hsdpa[END_REF] scheduler with mobility in LTE heterogeneous network. The M-LWDF scheduler is very suitable for real-time services such as VoIP, Video, Gaming, etc. The analyses of the simulation results are represented in the following subsections.

Eects of Jitter buer on voice quality: In order to evaluate the eects of JB on voice quality, we set values of this factor of 40, 60, 80, 100, and 120.

Figure 3 shows the relationship of the number of user (NU) vs user perception This gure also shows that the higher JB, the higher MOS. However, in fact, JB should not be too high. As shown on this gure, when JB is more than 80, the MOS increase not signicantly. This is quite clear when JB corresponds to 100 and 120. It can be said that, when JB is more than 100, its eect on user perception is not signicant.

Figure 4 shows the eects of JB when the speed is 120 km/h. It's clear that, when the speed is high, the user satisfaction is decreased clearly. All cases have the MOS score heavily decreasing when NU increasing. It's obvious, but the reduction is quite dierent from the case of the speed of 30 km/h. It's similar to the previous case, the higher JB, the higher user perception. The JB is directly proportional to user satisfaction while the speed is inverse.

Eects of Delay on voice quality: Figure 5 shows the relationship between delay vs user perception according to NU. It's clear that, the delay increases when NU increase. This leads to lower MOS score. The E-model has the highest MOS while the Extended E-model has the lower one. It can be said that, the higher NU and the higher delay, the lower MOS score (or lower user perception).

This principle is suitable for both the E-model and the Extended E-model (for all cases of JB). The detailed results of this case are represented in Table 4.

It's similar to the case of delay for the user speed of 30 km/h, the eects of delay on user satisfactions when the user speed is 120 km/h is shown on Figure 6. The principle is similar to the user speed of 30 km/h, however, the MOS score is lower and heavily decreases when the NU or delay increases. This is right in both E-model and Extended E-model. The detailed results are described in Table 4. Normally, PLR increases when NU increases, but there is special case for the NU equals 10. This may be due to the system is usually not stable in fact. For both E-model and Extended E-model, the higher NU and the higher PLR, the lower MOS score. The E-model has the higher MOS when compared to the Extended E-model. For the Extended E-model, rule for the higher JB, the higher MOS is still met. For the detailed results, refer to Table 4.

For the case of the user speed equals 120 km/h, the rules for the case of user speed is 30 km/h is still met. The dierence here is that in this case, the user perception decreases in comparison with the user speed of 30 km/h and it heavily decreases when the NU or PLR increases. This is fully suitable. The detailed results of this case are represented in Table 4. This is because in our model, we take eects of network jitter into account. This also means the eect of JB on user perception is very signicant. The simulation results also show that user perception decreases when the user speed increases.

For the case of user speed of 30 km/h, the user satisfaction slightly reduces when NU raises while it heavily decreases for the user speed of 120 km/h. For both cases of the user speed, the user perception increases when the JB increases.

However, when JB is more than 100, the user satisfaction increases not signicantly. For the delay and PLR, both increase when NU increases and meet the principle: the higher delay and PLR, the lower user satisfaction. It can be concluded that, the proposed model is very suitable for predicting VoLTE quality be cause it does not refer to the original signal. The user perception can be not as exact as the intrusive methods such as PESQ but it is quite simple, calculate very quickly. It is also very suitable for purpose of transmission planning and for voice assessment in laboratory for academic community and researchers.

The proposed model can be used to predict VoLTE quality for many dierent scenarios which can be congured in the LTE-Sim. In the near future, we will evaluate our proposed model for more NU, other voice codecs, and complement more parameters into the E-model.

  are coecients, K is time instant. These factors depend on the voice codec, for the G.729 codec, these factors have the values as follows: C 1 = -15.5, C 2 = 33.5, C 3 = 4.4, C 4 = 13.6, K = 30. The factor of T is the xed buer size of the voice codec. For the G.729 codec, the packet size is 20ms, thus, normally T =∝ ×20 where ∝= 2, 3, 4, 5, 6, etc. The H is a factor of Pareto distribution and in range of 0.55 to 0.9. According to [10], the MOS slightly drops when H increases and it does not aect signicantly on MOS score, thus, in this study, we select H = 0.6 for the simulation. The nal expression of the R-factor when utilizing the G.729 codec is described in Equation (9). R = 64.28 -[0.024 × d + 0.11 × (d -177.3) × H(d -177.3)]-40 × ln(1 + 10 × e l ) -13.6 × exp( -43 )

Fig. 2 .

 2 Fig.2. The proposed implement model

4. 1

 1 Simulation environment Trac model: In our scenario, the eNB is located at the center of the macrocell using an ommi-directional antenna in a 10 MHz bandwidth. Each UE uses a VoIP ow, a Video ow, and a INF-BUF ow at the same time. For the VoIP ow, a G.729 voice stream with a bit-rate of 8 kbps was considered. The voice ow

  264 SVC video streaming, we used an encoded video sequence foreman.yuv , which is publicly available. The LTE propagation loss model is formed by four dierent models including: Path loss, Multipath, Penetration and Shadowing [21]. Path loss: P L = 128.1 + 37.6 × log(d), with d is the distance between the UE and the eNB in km. Multipath: Jakes model Penetration loss: 10 dB Shadowing: Log-normal distribution with mean 0 dB and standard deviation of 8 dB.

Fig. 3 .

 3 Fig.3. MOS vs number of VoIP user at the speed of 30 km/h

Fig. 4 .Fig. 5 .

 45 Fig.4. MOS vs number of VoIP user at the speed of 120 km/h

Fig. 6 .Fig. 7 .

 67 Fig.6. Eects of Delay on voice quality at the speed of 120 km/h

Fig. 8 .

 8 Fig.8. Eects of PLR on voice quality at the speed of 120 km/h

Table 1 .

 1 LTE service classes with QoS requirements

	Resource Type Priority 2 Guaranteed Bit Rate 4 (GBR) 3	Packet Delay Budget (ms) 100 150 50	Packet Error Loss Rate 10 -2 Conversational voice 10 -3 Conversational video (live streaming) 10 -3 Real-time gaming	Example services
		5	300	10 -6 Non-conversational video (buered stream)
		1	100	10 -3 IMS signaling
	Non-GBR	6	300	10 -6 Video (buered streaming) TCP-based (e.g. www, e-mail, chat, FTP, P2P sharing, progressive video, etc.)
		7	100	10 -6 Voice, Video (live streaming, Interactive Gaming)
		8 9	300	10 -3 Video (buered streaming) TCP-based (e.g. www, e-mail, chat, FTP, P2P sharing, progressive video, etc.) 10 -6

Table 2 .Table 2 .

 22 R-factor and MOS with corresponding user satisfaction

	R	User satisfaction MOS
	90 ≤ R < 100 Very satised	4.3-5.0
	80 ≤ R < 90 Satised	4.0-4.3
	70 ≤ R < 80 Some users dissatised	3.6-4.0
	60 ≤ R < 70 Many users dissatised	3.1-3.6
	50 ≤ R < 60 Nearly all users dissatised 2.6-3.1
	R < 50	Not recommended	< 2.6

After setting the default values for the E-model, Equation (

1

) can be rewritten as follows:

Table 3 .

 3 Simulation parameters Simulation Parameters Values MOS score) when the speed is 30 km/h. It's clear that, in this case, the Emodel has the highest MOS score while the proposed model has the lower MOS. This is obvious because our proposed model considers eect of network jitter on the E-model. All cases have the slightly degraded MOS when NU increases.

	Simulation duration	100 s
	Frame structure	FDD
	Cell radius	1 km
	Bandwidth	10 MHz
	Video bit-rate	242 kbps
	VoIP bit-rate	8 kbps
	User speed	30, 120 km/h
	Number of user	10, 20, 30, 40, 50 UEs
	Maximum delay	0.1 s
	Packet Scheduler	M-LWDF
	Trac model	VoIP, Video, and INF-BUF

(