Abir Masmoudi
email: masmoudiabir@gmail.com

Fethi Bougares
email: fethi.bougares@univ-lemans.fr

Mariem Ellouze
email: mariem.ellouze@planet.tn

Yannick Esteve
email: yannick.esteve@univ-lemans.fr

Lamia Belguith
email: l.belguith@fsegs.rnu.tn

Y Estève

Automatic Speech Recognition System for Tunisian Dialect

Keywords: Under-resourced Language, Rule-based Grapheme-to-phoneme conversion, Automatic speech recognition, Tunisian dialect

Although Modern Standard Arabic is taught in schools and used in written communication and TV/radio broadcasts, all informal communication is typically carried out in dialectal Arabic. In this work, we focus on the design of speech tools and resources required for the development of an Automatic Speech Recognition system for the Tunisian dialect. The development of such a system faces the challenges of the lack of annotated resources and tools, apart from the lack of standardization at all linguistic levels (phonological, morphological, syntactic and lexical) together with the mispronunciation dictionary needed for ASR development. In this paper, we present a historical overview of the Tunisian dialect and its linguistic characteristics. We also describe and evaluate our rule-based phonetic tool. Next, we go deeper into the details of Tunisian dialect corpus creation. This corpus is finally approved and used to build the first ASR system for Tunisian dialect with a Word Error Rate of 22.6%.

Introduction

Arabic is a Semitic language among the oldest in the world. The evolution of this language from antiquity to the present day has given birth to several linguistic registers; some forms have disappeared and others are still spoken. In accordance with the great periods of the Arabic language history, four linguistic registers can be cited: (1) Old Arabic, which is not used currently; it is found in ancient literary works (mainly poems); (2) Classical Arabic or literary Arabic, which is the language of Islams Holy Book; spreaded through Islamic conquests; (3) Modern Standard Arabic (MSA) is the o cial language of all the Arab countries and it has extensive presence in various settings: education, business, arts and literature, and o cial and legal written documents;

(4) dialectal Arabic, which is the mother tongue of every Arabic speaker and the main variety used in daily life for spoken communication. There are many Arabic dialects across the Arab World.

The dialects are not standardized, they are not taught, and they do not have an o cial status. Indeed, most native speakers of Arabic are unable to produce a sustained spontaneous discourse in MSA; in unscripted situations where spoken MSA would normally be required (such as talk shows on TV); speakers usually resort to repeated code-switching between their dialect and MSA [START_REF] Abdel-Rahman | Code-switching and Linguistic Accommodation in Arabic[END_REF].

In the Natural Language Processing (NLP) context, MSA has received the lions share of attention. There are lots of parallel and monolingual data collections, richly annotated collections (e.g., treebanks1 ), sophisticated tools for morphological analysis and disambiguation, syntactic parsing, etc. [START_REF] Habash | Introduction to Arabic Natural Language Processing, Synthesis Lectures on Human Language Technologies[END_REF]. Unfortunately, a large number of these MSA tools cannot e↵ectively model Arabic dialects because the latter diverge considerably from MSA and di↵er widely among themselves. Indeed, these di↵erences make the direct use of MSA NLP tools and applications for handling dialects impractical. Therefore, it is necessary to construct dedicated tools and resources to process dialectal Arabic.

Tunisian dialect is a subset of the Arabic dialects usually associated with the Arabic of the Maghreb. In this work, we aim to develop speech processing tools and resources for the purpose of creating an ASR system in the framework of human/human conversation between customers and agents of the Tunisian Railway Transport Network. Despite the significant progress and the considerable e↵ort recently made to study this dialect, much work remains to be done. On the other hand, the development of such tools faces many challenges: [START_REF] Abdel-Rahman | Code-switching and Linguistic Accommodation in Arabic[END_REF] The limited number and the non-availability of dedicated resources.

(2) The lack of standard orthographies for this dialect. Therefore, the task of resources creation is more complicated because there is no consensus about the transcription conventions. (3) The impossibility to apply phonetic transcriptions techniques dedicated to MSA, since the Tunisian dialect is characterized by the addition of other supplementary phonemes. [START_REF] Andersen | Comparison of two tree-structured approaches for grapheme-to-phoneme conversion[END_REF] The irregularities relative to the lack of correspondence between the spelling of the lexical items and their sound contents.

The main contributions of this paper are as follows:

• Presenting the characteristics of the Tunisian dialect at di↵erent levels i.e., lexical, morphological, phonological and syntactic. • Designing an accurate approach for G2P conversion of the Tunisian dialect in order to develop a phonetic dictionary. The latter is an important ASR component that serves as an intermediary between acoustic models and language models in ASR systems [START_REF] Masmoudi | A Corpus and Phonetic Dictionary for Tunisian Arabic Speech Recognition[END_REF]. • Describing the creation process of our Tunisian dialect corpus and giving the most important characteristics of this corpus. • Developing and analyzing a dedicated ASR system for the Tunisian Railway Transport Network.

The remainder of this paper is organized as follows. Section 2, briefly overviews earlier research in automatic speech recognition for under-resourced languages. Section 3 sets a historical review of the Tunisian dialect and presents its linguistic characteristics. Section 4 describes the variety of published G2P conversion approaches and presents our methodology to develop G2P rulebased approach for Tunisian dialect. We present, discuss and analyse the experimental results of the proposed method as well. In section 5, we explain how we proceeded to create our manually transcribed Tunisian dialect corpus. In this section we describe the steps taken in order to implement and analyse the results of our ASR system in the Tunisian Railway Transport Network domain. Finally, we conclude and present future work in section 6.

Background and Related Works

In order to train automatic speech recognition (ASR) systems very large amounts of speech and textual data are generally required. Such spoken language resources are available for a large variety of languages such as English and some European languages. However, these resources are still unavailable or very limited for other languages. The latter are generally defined as underresourced languages. It should be noted that under-resourced languages include languages spoken by a minority of the population such as Breton, Castilian, Catalan... or spoken languages in regions of underdeveloped countries. They include also Arabic dialects that are considered as the first language of several Arab countries and used for everyday communication.

Recently we have seen an increasing interest in Automatic Speech Recognition (ASR) for under-resourced languages. Several systems were presented and discussed over the last couple of years. These systems are di↵erent in terms of their origins and in the techniques used to solve the problem of lack of resources.

For instance, the earliest works on ASR for Vietnamese and Khmer lowresourced languages were proposed by [START_REF] Besacier | Reconnaissance automatique de la parole pour des langues peu dotees: Application au vietnamien et au khmer[END_REF]. Afterward [START_REF] Vu | Rapid building of an asr system for underresourced languages based on multilingual unsupervised training[END_REF] used a method for a rapid adaptation of an automatic speech recognition system for Vietnamese. Another interesting work was carried out for the Khmer language [START_REF] Seng | Letter-to-Phoneme Conversion Based on Two-Stage Neural Network Focusing on Letter and Phoneme Contexts[END_REF]. In this work, the problem of the lack of data was addressed. Authors crawled the web to extract a large number of khmer documents. These documents were pre-processed and used for language model training and a part of them was uttered by several speakers in order to constitute training data for acoustic models.

Other researchers, proposed di↵erent works for African languages: Somali [START_REF] Nimaan | Boites a outils tal pour les langues peu informatisees: Le cas du somali[END_REF], Amharic [START_REF] Pellegrini | Transcription automatique de langues peu dotees[END_REF], South African languages [START_REF] Barnard | Speech technology for information access: a south african case study[END_REF] and West African languages [START_REF] Schlippe | hause large vocabulary continuous speech recognition[END_REF]. For more detailed information, readers may refer to the survey on ASR for under-resourced languages presented in [START_REF] Besacier | Automatic speech recognition for under-resourced languages: A survey[END_REF].

With respect to the Arabic dialects, a large gap is still present between them and the MSA. Indeed, MSA is characterized by a large amount of resources to develop an ASR system in contrast to the Arabic dialects, which su↵er from lack of resources. In the Arabic dialect context, we point to the earlier work on the Qatari dialect [START_REF] Elmahdy | Development of a tv broadcasts speech recognition system for qatari arabic[END_REF]. Although the resources for the Qatari dialect are derisory, it is possible to benefit from the MSA resources: spoken and textual corpora. In this respect, authors proposed to add MSA corpora to the small specific dialectal data. Adding MSA data enhance the acoustic modeling of the Qatari dialect [START_REF] Elmahdy | Development of a tv broadcasts speech recognition system for qatari arabic[END_REF].

3 The Tunisian dialect

Historical Overview of the Tunisian dialect

The Tunisian dialect is considered as the current mother tongue shared by all Tunisians independently of their origin and their social belonging. Therefore, this dialect occupies an important linguistic situation in Tunisia. Historically, Berber was the original mother tongue of the North Africa inhabitants. Later on, the Arab invasions and the Islamic conquests introduced the Arabic language which is the language of Islams Holy Book. The prolonged Ottoman Turkish political domination of North Africa, roughly from the mid-fifteenth to the late nineteenth century, and the French colonization, from 1881 to 1956, had an impact on the absorption of foreign vocabulary into the lexicon of the local Arabic dialect. In addition to Turkish and French, we find plentiful examples of lexical elements from the European languages in the Tunisian dialect. We can identify a significant number of expressions and words of Spanish and Italian origin, and even of Maltese origins.

In this regard, the interactions between the Tunisian dialect and these foreign languages a↵ect the di↵erent levels of the spoken language, i.e phonology, morphology, lexical and syntax, which have rendered the linguistic situation in Tunisia rather complex. The linguistic situation in Tunisia is described in [START_REF] Lawson | Accommodation communicative en Tunisie: une tude empirique, Plurilinguisme et identits au Maghreb[END_REF] as poly-glossic where multiple languages and language varieties coexist. In the following sub-sections, we present some characteristics of the Tunisian Dialect at di↵erent levels.

Linguistic characteristics

Phonological level

There are many substantial phonological di↵erences between Tunisian dialect and MSA [START_REF] Mejri | Pluringuisme et diglossie en Tunisie[END_REF][START_REF] Zribi | A Conventional Orthography for Tunisian Arabic[END_REF]. This includes the introduction of some foreign phonemes like /P/, /V/ and /G/ and the neglection of short vowels when they are located at the end of a syllable. Furthermore, we can also quote the pronunciation variation of some consonants. For example, the consonant Ä /s/ [s] can be pronounced as Ä /s/ [s] or ê /S/ [S], as in the example of the word »ÒÉ ↵ P /raswl/ [Prophet] is pronounced »ÒÉ ↵ P /raswl/ or »Òì ↵ P /raSwl/.

Morphologic level

Compared to MSA, the Tunisian Dialect shows many divergences at the morphological level. The overarching themes are those of simplifying inflections and introducing new clitics [START_REF] Zribi | A Conventional Orthography for Tunisian Arabic[END_REF]. The Tunisian Dialect has lost the feminine singular and the feminine plural in the conjugation of verbs. Similarly, there is no di↵erence between the feminine and the masculine plural in the Tunisian Dialect. In addition, there is no longer any di↵erence between the conjugation of the first and second person singular. In order to indicate duality, the Tunisian uses a numeral word P P /zwz/ [two] before or after the plural noun. New clitics are also introduced such as negation clitic ˙⌦Ê ⌘ Ö ↵ /Siy/ + verb + A ↵ " /mA/ which are manifested in MSA with di↵erent particles ie A ↵ " /mA/, B ↵ /lA/, • ↵ À /lan/ and ' ↵ À /lam/.

Syntactic level

Like other Arabic dialects, the Tunisian dialect respects in most cases the regular grammar of MSA or that of Classical Arabic [START_REF] Graja | Statistical Framework with Knowledge Base Integration for Robust Speech Understanding of the Tunisian Dialect[END_REF]. Yet it has some syntactic particularities. Indeed, the order of syntactic constituents in the sentence seems to relatively be more important. The canonical order of words in Tunisian dialect verbal sentence is SVO [Subject Verb Object] [START_REF] Baccouche | Larabe, dune koin dialectale une langue de culture, Mmoires de la socit linguistique de Paris, TomeXI[END_REF]. In MSA, the word order can have the following three forms (SVO, VSO and VOS) [START_REF] Baccouche | Larabe, dune koin dialectale une langue de culture, Mmoires de la socit linguistique de Paris, TomeXI[END_REF]. Among the Tunisian dialect syntactic particularities, we can also cite the reduction of the number and the form of MSA pronouns [START_REF] Mejri | Pluringuisme et diglossie en Tunisie[END_REF] from twelve to seven personal pronouns. Moreover, we have reported the disappearance of the duel of the second person, the feminine plural of the second person and the feminine plural of the third person.

Lexical level

As mentioned above, the Tunisian dialect is an outcome of the interactions between Berber, Arabic and many other languages.

A ↵ ÓD ⌦ " Q ↵ ✓ J ⌦ É /syprpyhA/ [delete it].

CODA writing convention

CODA is a conventionalized orthography for dialectal Arabic [START_REF] Habash | Conventional Orthography for dialectal Arabic[END_REF]. In CODA, each word has a single orthographic representation. CODA has five key properties. First, CODA is an internally consistent and coherent convention for writing dialects. Second, it is primarily created for computational purposes, but it is easy to be learnt and recognized by educated Arabic speakers. Third, it uses the Arabic script as used for MSA, with no extra symbols from (i.e Persian or Urdu). Fourth, it is intended as a unified framework for writing all dialects. CODA has been defined for Egyptian [START_REF] Habash | Conventional Orthography for dialectal Arabic[END_REF] and was extended to the Tunisian dialect [START_REF] Zribi | A Conventional Orthography for Tunisian Arabic[END_REF] (some authors of our paper were involved in this work). Finally, it aims to maintain a level of dialectal uniqueness while using conventions based on similarities between MSA and the dialects. For a full presentation of CODA and a justification and explanation of its choices, see [START_REF] Habash | Conventional Orthography for dialectal Arabic[END_REF], [START_REF] Saadane | A Conventional Orthography for Algerian Arabic[END_REF] and [START_REF] Zribi | A Conventional Orthography for Tunisian Arabic[END_REF].

G2P conversion of the Tunisian dialect

Grapheme-to-Phoneme (G2P) conversion is the task of converting a sequence of graphemes (written letters) to a sequence of phonetic symbols (representation of speech sounds). This conversion is usually generated using an automatic G2P conversion tool [START_REF] Bisani | Joint-Sequence Models for Grapheme-to-Phoneme Conversion[END_REF]. It has an application in many areas such as speech recognition and speech synthesis systems. A fairly large number of researches have been conducted on phonetic conversion. Proposed techniques could be classified into three categories briefly described in the following sub-sections.

G2P approaches

Look-up approach

This approach involves a manually created phonetic dictionary. A good example of this approach is the CMU dictionary 2 for North American English, which has been built by human experts (linguists) and contains over 125,000 words and their transcriptions. Such a resource is particularly useful for speech recognition and synthesis, as it has mappings from words to their pronunciations into a given phoneme set. Despite its good quality and usefulness for di↵erent NLP applications, creating phonetic dictionaries manually is a very expensive and tedious task. To overcome the limitations of this approach, the rule-based and data-driven approaches were proposed as alternatives.

Rule-based approach

This type of approach uses a comprehensive set of G2P rules and a lexicon of exceptions. The development of the G2P rules requires a highly qualified phonetician. For morphologically rich languages, such as MSA, the manual creation of pronunciation dictionaries is a painful task because of the large number of word forms, each of which has a large number of possible pronunciations.

Fortunately for MSA-language, the relationship between orthography and pronunciation is relatively regular and well understood [START_REF] Biadsy | Improving the Arabic Pronunciation Dictionary for Phone and Word Recognition with Linguistically-Based Pronunciation Rules[END_REF]. More recently, most MSA speech recognition systems are trained using pronunciation dictionary constructed by mapping every undiacritized 3 word to all its diacritized versions. Morphological disambiguation obtained using tools such as MADA 4 are used to determine the most likely pronunciation of the word given its context. A typical example of this approach is presented in [START_REF] Vergyri | Development of the SRI/Nightingale Arabic ASR system[END_REF] where morphological information is used to predict word pronunciation.

In the context of Arabic dialect G2P conversion, [START_REF] Harrat | Grapheme To Phoneme Conversion-An Arabic Dialect Case[END_REF] used the rule-based approach in order to generate the pronunciation of Algerian dialect 5 . To avoid ambiguity caused by the absence of diacritics, they first added diacritic marks to the texts using ADAD (Automatic Diacritizer of Algerian dialect), a system for automatic diacritization. Then, they applied several manually defined 2 http://www.speech.cs.cmu.edu/cgi-bin/cmudict 3 Undiacritized or unvowelized word refers to a word without short vowels. 4 MADA is a POS tagger for Arabic languages 5 The Algerian dialect is the language used in the daily spoken communication of Algerian.

phonetic rules to perform the G2P transformation. The techniques described above show that the rule-based approach provides decent quality pronunciation dictionaries without requiring a great amount of data. However, this approach requires specific linguistic skills to design the rules.

The data-driven approach

In contrast to the rule-based approach outlined above, the G2P data-driven approach is example based. Hence, this approach make uses of a manually annotated corpus rather than a linguistic one. These examples are used to train a machine learning classifier to predict the pronunciation of unseen words [START_REF] Bisani | Joint-Sequence Models for Grapheme-to-Phoneme Conversion[END_REF]. In the literature, several machine learning algorithms have been used to train data-driven G2P conversion including pronunciation by analogy, local classification: decision trees [START_REF] Häkkinen | Assessing text-to-phoneme mapping strategies in speaker independent isolated word recognition[END_REF], Hidden Markov Models (HMMs) [START_REF] Taylor | Hidden Markov models for grapheme to phoneme conversion, INTER-SPEECH[END_REF], Conditional Random Fields (CRFs) [START_REF] Illina | Grapheme-to-Phoneme Conversion using Conditional Random Fields[END_REF] and Joint Sequence Model (JMM) [START_REF] Bisani | Joint-Sequence Models for Grapheme-to-Phoneme Conversion[END_REF].

Methodology of G2P conversion of the Tunisian dialect

Pronunciation dictionaries map words to one or more pronunciation variants, taking account of pronunciation variability. In our study, we have chosen to use the rule-based approach. The reasons for this choice were threefold: first, the lack of hand-crafted corpus for Tunisian Dialect needed to train a G2P classifier; second the results from previous studies show the e↵ectiveness of rule based approach and give a pronunciation dictionary of good quality; and third the similarity between Tunisian dialect and MSA, gives us the possibility to benefit from the well defined phonetic rules of MSA [START_REF] El-Imam | Phonetization of Arabic: rules and algorithms[END_REF] and build on it.

Format of phonetic rules

We have identified a set of pronunciation rules. The development and the conception of these rules were carried out collaboratively with linguistic and phonetic experts. Rules are provided for each letter in the Tunisian dialect. Each rule tries to match certain conditions relative to the right and left context of the letter and provide subsequently a replacement. Each rule is read from right to left by following this format:

P honetisation {Lef t Condition} + {Grapheme} + {Right Condition}
• Right-Condition context before the current position "Grapheme".

• Grapheme: is the current letter in the word.

• Left-Condition context after the current position "Grapheme".

• Phonetisation: is either a phoneme or more of a phoneme or a vacuum if the "Grapheme" is omitted in pronunciation.

Lexicon of exceptions

The Tunisian dialect spelling is fairly regular, except in certain situations where some words do not comply with the regular pronunciation rules. It is therefore necessary to define a lexicon of exceptions (93 exceptions of unvowelized words and 90 exceptions of vowelized words). The phonetic form of each exceptional word is entered with its graphemic form. The exceptions dictionary is scanned first. When the word does not exist in that dictionary, the pronunciation rules are applied in order to generate the corresponding phonetic form. By doing this, we could solve some problems related to the phonetic and phonological variations in the Tunisian dialect [START_REF] Masmoudi | A Corpus and Phonetic Dictionary for Tunisian Arabic Speech Recognition[END_REF].

Application of phonetic rules

This section highlights the phonetic rules application strategy in order to generate the phonetic form of vowelized and no-vowelized words. It is worth noting that no-vowelized category also includes the partially vowelized words.

Vowelized words

In this case the application of the phonetic rules straightforward. Rules are applied in the word reading direction (from right to left). This begins with the first letter of each word whilst taking into account the order and the context of each letter.

No-vowelized words

As it is the case for MSA, vowels are generally omitted in Tunisian dialect written texts. Therefore, the determination of the phonetic form of a given word is ambiguous. It is thus quite natural that G2P conversion becomes more di cult compared to vowelized words. This ambiguity was targeted for MSA and several solutions have been proposed in the NLP community to over-come the lack of vowels. For example, [START_REF] Alghamdi | Automatic restoration of Arabic diacritics : a simple, purely statical approach[END_REF] proposed an automatic restoration of arabic diacritics. However, the proposed method is specific for undiacritized MSA text and cannot handle the dialects given the morphological, syntactic, lexical and phonetic di↵erences.

In order to get the exact pronunciation of words unvowelized many challenges must be addressed. Firstly, the primary challenge consists in the determination of the phonemes corresponding to the short vowels location. In this regard, we have agreed to put short vowels during the manual transcription. Then, according to the phonetic study that we have performed in this corpus, we have identified these three phonetisation principles:

1. Tunisian dialect words end with either a silent consonant (with Sukun) or a long vowel. 2. Each long vowel is always followed and preceded by a silent consonant (without short vowels). 3. Tunisian dialect words cannot have two successive consonants that carry a vowel (long or short) with the exception of words with "Shadda" (Doubling of consonants).

Based on these previous principles and the pronunciation rules that we presented above, we were able to distinguish four classes of G2P conversion of no-vowelized words:

1. First class contains words with long vowels. 2. Second class contains words with Shadda. 3. Third one includes words with long vowels and Shadda together. 4. The last class integrates simple words without Shadda and long vowels.

Unlike the case of vowelized words, the application of phonetic rules to no-vowelized words does not follow the reading direction of the word. Indeed, the phonetic rules for no-vowelized words are divided into two groups: support rules and secondary rules.

As a first step, the support rules are applied. This produces the graphemes and the long vowels phonemes and facilitates the application of secondary rules. Secondary rules generate the production of the short vowels, Tanween and "Sukun" phonemes [START_REF] Masmoudi | A Corpus and Phonetic Dictionary for Tunisian Arabic Speech Recognition[END_REF]. Support rules are applied in the following order:

1. The grapheme rules and foreign-letter rules; 2. Long-vowel rules; 3. Rules of the Ya-Maqsoura; 4. Gemination rule; 5. Ligature rules; 6. Elision rules; 7. Shamsi and Ghamari rules.

Once the support rules is applied and based on the three phonetisation principles identified above, the secondary phonetic rules are applied in the following sequence: first short-vowel and sukun rules, then tanween rules. Table 1 shows an example of the phonetisation process of no-vowelized word. First, the support rules are applied (phase 1 in table 1) and produce the phonemes: "KH R JH UW". Afterwards, the secondary rules are applied (phase 2 in table 1) in order to produce the phonetisation results of the last column of the table.

Table 1 The G2P conversion process of the word @ Òk . Q k. /xrjwA/ [they go out]

Word

Phonetisation Rules & comments Phonetisation @ Òk . Q k

Phase 1: Application of support rules -By applying graphemes rules we obtain the phoneme "KH" which corresponds to the grapheme p, the phoneme "R" which corresponds to the grapheme P, and the phoneme "JH" which corresponds to the grapheme h . . -The grapheme  /w/ is a part of ⇣ È´A ↵ "m . Ã '@ @ ↵ . Thus, the long vowel rule must be applied to obtain the "UW" phoneme directly.

KH AE R JH UW KH IH R J UW KH UH R J UW

Phase 2: Application of secondary rules -Based on the principle that each long vowel is always followed and preceded by a mute consonant, in this case, the insertion of phoneme of the short vowels "AE" or "IH" or "UH" must be only between the first phoneme "KH" and the second phoneme "R".

Evaluation and discussion

In order to evaluate our G2P conversion tool, an expert-annotated Tunisian dialect corpus is needed. This kind of manually annotated corpus does not exist for the Tunisian dialect. To cope with this situation, we create our evaluation set from the three corpus described in table 2. 1), ( 2) and ( 3) 4083

As a first step, these corpora are standardized using the CODA convention [START_REF] Zribi | A Conventional Orthography for Tunisian Arabic[END_REF], then about 4,000 di↵erent words are randomly selected in order to build the G2P evaluation set. This evaluation subset is then automatically phonetized using our rule based G2P system. The error rate is calculated as a function of phonemes, by estimating the following errors in the hypothesis compared to the reference:

• Substitutions (Sub): misrecognized phonemes.

• Insertions (Del): represent extra phonemes.

• Deletions (Ins): represent unrecognized phonemes.

The evaluation was performed based on two distinct levels using Sclite6 , according to word and phoneme levels. At the word level the generated phonetic form of a given word is considered correct when all its phonemes are correctly generated. The Final score is calculated according to the following formula :

WER/PER = (Sub + Del + Ins) / N (where N is the reference size) Table 3 reports the obtained scores using our G2P conversion system on the evaluation set. As reported in the table, the G2P conversion system always predicts the correct phonemes when the test words are vowelized. Otherwise, when words are non-vowelized the G2P conversion system predicts the phonetic form with an accuracy of 99.6% and 99.2% at the phoneme and word levels respectively.

A list of erroneous words was compiled, in order to determine the source of errors made with the no-vowelized G2P conversion. It turned out that the errors are caused by the presence of foreign words, irregular words and some proper names. The rule-based approach adopted in this work is unable to generate the correct G2P conversion of the foreign words used in the Tunisian dialect and pronounced the same way as in the original language. One possible solution to deal with foreign words would be statistical approaches. Indeed, with statistical approaches we can train the G2P including foreign phonemes.

5 Tunisian dialect speech recognition system

Tunisian Arabic Railway Interaction Corpus (TARIC)

The quality and number of NLP tools available for a given language are usually correlated with the amount of annotated resources (transcribed speech corpora, phonetic dictionaries, vocabularies, text corpora) for that language. Although these resources are limited for the under-resourced languages, they are still needed to develop tools such as ASR systems. Tunisian dialect is an example of this so called under-resourced languages with limited amount of available annotated resources.

In this context, we took the initiative to create our own spoken dialogues corpus. This corpus is gathered in collaboration with the Tunisian National Railway Company (SNCFT 7 ) and corresponds to recordings of information request in railway stations. This corpus, called TARIC (Tunisian Arabic Railway Interaction Corpus), contains information requests in the Tunisian dialect about the railway services such as train type, train schedule, train destination, train path, ticket price and ticket booking. TARIC corpus is created in three stages : first, audio recording; second the orthographic transcription, and finally the transcription normalization. The following sub-sections give more details about these stages.

Speech data collection

This first stage consists in speech audio recordings. This task was performed in the ticket o ces at the railway station in Tunis. We recorded conversations in which there was a request for information about train schedules, fares, bookings, etc. The equipment we used included two portable PCs using the Audacity software 8 and two microphones, one for the ticket o ce clerk and the other for the client. The micro of the client is installed invisibly in order to collect spontaneous speech using the daily form of the Tunisian dialect. The recording was carried out in full respect of the laws and regulations in force and the internal procedures of the SNCFT.

In order to increase the vocabulary size, we recorded at various periods, particularly holidays, weekends, festival days, and sometimes during the week. This strategy leads us to create a heterogeneous corpus with a rich vocabulary, given that fares and schedules vary in holiday periods compared to normal days requests. Finally, we recorded also in the promotion days and on festival days. At the end of this stage, we came up with 20 hours of audio recording. We note, however, that we opt for the re-recording solution when the original session was very noisy (i.e. some sentences were re-recording by another person for a better acoustic conditions).

Private life policies

Since the speech corpus collection is based on real conversations between the ticket o ce clerk and the client, personal data of the client may be present in these recordings. These data can be references to places or human characteristics such as the national identity cards numbers, the work card numbers or the fidelity card numbers. Consequently, the presence of such personal data in a corpus engenders a lack of respect for people's privacy. To prevent this, we have manually identified all passages of the corpus including personal information about the speaker "client". Thereafter, these passages are deleted. Doing so ensures that we fulfill our commitment with the SNCFT in addition to compliance with local applicable laws.

Orthographic transcriptions

The transcription is the first level of a symbolic representation of an oral communication. It is an essential step in the constitution of the TARIC corpus. The goodness of the transcription is crucial and it has repercussions for all posterior treatments. Yet, the transcription in itself is not a trivial task. This task has been even more di cult since no transcription conventions are available for the Tunisian dialect. For this reason, we have developed first our own orthographic guidelines called CODA [START_REF] Zribi | A Conventional Orthography for Tunisian Arabic[END_REF] (see section 3.3 for more details), which was later adopted during the annotation of our speech corpus.

In practical terms, we used Transcriber 9 to perform the alignment of transcription and audio data. Using the 20 hours described in previous sections, we ended up creating a corpus of roughly 10 hours of transcribed speech. This corpus is then divided into three parts: training, development and testing sets. The main characteristics of the three sets are shown in Table 3. The TARIC corpus contains more than 21k statements. The corpus contains 82k running words with a vocabulary of 3207 words. As presented in Table 4, about 9.5 hours of the transcribed data are devoted to both training and development while the 43 remaining minutes composed the test set.

Table 5 reports the speakers distribution over train, dev and test set. Overall, we recorded 108 di↵erent speakers 60 males and 37 females. Among these speakers, 11 are on dev and 6 on test. We wish to point out that development and test sets include 7 and 4 unseen speakers with roughly 15 minutes and 24 minutes respectively.

Corpus distribution

TARIC will be the first freely distributed transcribed speech corpora for Tunisian dialect. This will be distributed as a package available for research purposes 10 . This package will be constituted of the audio recordings along with their corresponding aligned transcripts and the pronunciation dictionary. We will provide also the language model and the monolingual data used to train it. Finally we plan to continuously enhance the corpus with data acquired from other domains (mainly from broadcast news) with more speakers having di↵erent accents and di↵erent speaking styles.

Tunisian dialect ASR system

In this section we describe our ASR system for the Tunisian dialect built using the TARIC corpus. This ASR system is developed using the KALDI speech recognition toolkit [START_REF] Povey | The Kaldi Speech Recognition Toolkit[END_REF]. KALDI toolkit is very actively developed over the past few years and is widely used by the research community. Moreover, several recipes for training ASR systems with various speech corpora have been made available and frequently updated to include the newest techniques developed in the community. In the following subsections, we describe the main design characteristics of our Tunisian dialect ASR system.

Acoustic Modeling

Acoustic model is trained using PLP (Perceptual linear predictive) features.

For each frame, we also include its neighboring ±4 frames and apply Linear Discriminative Analysis (LDA) transformation to project the concatenated frames to 40 dimensions, followed by Maximum Likelihood Linear Transform (MLLT). We also applied a speaker adaptation through feature-space Maximum Likelihood Linear Regression (fMLLR) technique. These models are all standard 3-states context-dependent triphone models. The GMM-HMM model has about 15K Gaussians for 2.5K tied states.

Language Model and lexicon

The language model (LM) was built using TARIC training data transcripts and corpora described in table 2 (section 4.2.4). A total corpus of 100K words is obtained and normalized following the CODA recommendations. We trained different 3-gram language models with modified Kneser-Ney discounting. These models were interpolated to create the final LM by minimizing the perplexity on the development corpus. Given the limited size of our training data, we did not apply neither cut-o↵s nor pruning to the final language model. Moreover, we would point out we deliberately chose to avoid out-of-vocabulary (OOV) words. This was done by including all dev and test words in the lexicon.

Results

The KALDI system produces lattices as recognition results. The language model scaling factor is optimized on the development set and used to compute the best path through the lattice. The results of the first large vocabulary ASR system for the Tunisian dialect are presented in table 6. A WER of 22.6% is obtained on the test set. In order to have a clear analysis about the type of errors made by our ASR system, we looked at the frequently confused pairs on the development set. Given the ASR results presented in previous table, we especially focused on the substitute and we found that the top 6 frequent substitutions are words with Shadda, which represent the doubling of consonants. These errors will be deeply studied in future work in order to improve this baseline ASR system. -" @ ë ⌦  Q ⌦ J . À P P ÈK ⌦ @ >yh zwz lbrwys >mm

We showed examples of the system output in Table 7. For each example we presented the ASR output and the reference. We presented also the Arabic script and the Buckwalter transliteration. The two last examples showed the case in which our system recognize correctly all the words except these with Shadda ( ⇣ È ✏ ' Ø and ⌘ Ä@ ↵ ✏ Y ⌘ AEK . ).

Conclusion and future work

In this paper, we presented some historical features of the Tunisian dialect. We also reported the linguistic characteristics of this dialect at multiple levels including phonological, morphological, syntactic as well as the lexical level.

We then exposed an in-depth analysis of the problems of converting G2P in the Tunisian dialect and the challenges related to spelling irregularities. An outcome of this study was the development of the G2P tools based on a comprehensive set of G2P rules and a dictionary of exceptions. This G2P tools was evaluated and validated using a manually annotated evaluation set.

On a further step, we created a spoken dialogues corpus in the Tunisian dialect in the Tunisian Railway Transport Network domain called TARIC. This corpus was manually transcribed and used together with our G2P tool to run our experience in order to develop an automatic speech recognition system of the Tunisian dialect. This ASR reaches a word error rate of 22.6% on a held-out test set. G2P tool and corpus are freely distributed for the research community.

For the future research, we are planning to improve the G2P conversion for unvowelized words. We also plan to expand our ASR system of the Tunisian dialect in the Tunisian Railway Transport Network to more generic field like broadcast news. Last, starting from the baseline ASR system presented in this paper, we will train deep neural network acoustic models in order to apply some transfert learning techniques that have been showed as to be very e↵ective [START_REF] Samson | Using Resources from a Closely-related Language to Develop ASR for a Very Under-resourced Language: A Case Study for Iban[END_REF] to take benefit from resources available in closely-related language. In the case of Tunisian dialect, resources related to MSA should be really helpful to improve the acoustic models.

  The outcome of this interaction is manifested in the introduction of borrowed words in the Tunisian daily communication. Examples include, but are not limited to: †A ↵

	[shoes] from Spanish and ⇣ È ↵ ¢ ⌘ Ç⌧ ⌦ Ø	J . ↵ É /sabAT/

/fySTap/ [holiday] from Italian. Some foreign words undergo an addition of the Arabic enclitic or proclitic to express either an action or an order or possession of a thing etc. An example of this, the French verb /supprimer/ [delete] which undergoes generally with the addition of the Arabic enclitic : the Arabic enclitic A ↵ Î /hA/ [her] is attached at the end of the verb to indicate an order

  The exceptions lexicon is subdivided into four predominant categories. The first category includes 10 dialecatel demonstrative pronouns of the Tunisian dialect such as @ ↵ YÎ /h*A/ [This male] or ⌦ YÎ /h*y/ [this female]. The second one contains some exceptional names like the names of ALLAH È< ✏ À@ /Allah/ [GOD]. we recorded 6 exceptional names. The third category contains 9 elements representing the personal pronouns of the Tunisian dialect like A ↵ K @ /<nA/ [I]. The last category gathers about 65 various exceptions which do not belong to any previous category.

Table 2

 2 Corpus details used in G2P experiments. 4k words are randomly selected to create the evaluation set

	Corpus	Description	Size (#words)
	(1) Blogs corpus	texts collected from Tunisian blogs of various	21917
		fields (politics, sports, culture, science ...)
	(2) Arabizi conv corpus	automatic converted texts originally written	8057
		in Latin script (called Arabizi)
	(3)TARIC	Tunisian Arabic Railway Interaction Corpus	4648
	(4) Evaluation set	random selection from (

Table 3

 3 Word/Phone Error Rate of the rule based G2P on the evaluation set (row (4) table 2)

		Words (WER)	Phonemes (PER)
		Vowelized	Unvowelized	Vowelized	Unvowelized
	Substitutions	0%	0.3%	0%	0.1%
	Insertions	0%	0.3%	0%	0.2%
	Deletions	0%	0.2%	0%	0.1%
	Overall Error Rate	0%	0.8%	0%	0.4%

Table 4

 4 Main characteristics of the TARIC training, development and test sets.

		# of hours	# of statements Vocabulary size
				(di↵erent words)
	Train	8 Hours and 57 Minutes	18027	3027
	Dev	33 Minutes and 40 Seconds	1052	612
	Test	43 Minutes and 14 Seconds	2023	1009

Table 5

 5 Speakers distribution over train, dev and test set

		Number of speaker #unseen speaker (duration) Male Female
	Train	60	37	-
	Dev	3	8	7 ( 1 5 m i n 1 9 S e c )
	Test	1	5	4 ( 2 4 m i n 1 5 S e c )

Table 6

 6 Results of the first large vocabulary ASR system for the Tunisian dialect.

		LM PPL	WER (%)	Substitutions	Deletion	Insertions
	Dev	41.69	21.5	15.2	4.1	2.3
	Test	53.71	22.6	16.0	4.0	2.5

Table 7

 7 Examples of output our Tunisian dialect ASR system.

	Reference ASR output Reference ASR output	Arabic script ⇣ Ȫ @ ↵ YÎ ⇣ Èí AE ⌘ Ø ®A º@ ↵ YÎ ⇣ Èí AE ⌘ Ø ®A ↵ ⇣ J" ©K . P Q ⌦ ´⇣ 骂 ⇣ ⌧ ↵ À ↵ ⇣ J" ©K . P  ⇣ 骂 ⇣ ⌧ ↵ À ⌦ A ↵ ae ⇣ JÀ@ ˙⌦OE™ ¢ ⇣ ⌘ Ä@ ↵ ✏ Y ⌘ AEK . bqd⇠A$ ttlely tkAy Buckwalter transliteration ↵ @ Altsep gyr rbe mtAe qfsp h*Akp ↵ @ tsep w rbe mtAe qfsp h*Akp ⌦ A ↵ ae ⇣ JÀ@ ˙⌦OE™ ¢ ⇣ Y ⇣ Ø qd ttlely tkAy
	Reference	⇣ È ✏ ' Ø ë ⌦  Q ⌦ J . À P P ⌦ @ >y zwz lbrwys fm⇠p
	ASR output

Treebanks are language resources that provide annotations of natural languages at various levels of structure: at the word level and the sentence level.

http://www1.icsi.berkeley.edu/Speech/docs/sctk-1.2/sclite.htm

http://www.sncft.com.tn/

https://sourceforge.net/projects/audacity/

Transcriber is distributed as free software and is available at http://trans.sourceforge.net

http://www-lium.univ-lemans.fr/ ~bougares/ressources.php