Professeur Benoît Crabbé

Patrice Bellot

Directeur Karlheinz Mörth

Mohamed Khemakhem Viii

Prof. Dr Rute Costa

Richard Eckart

Standard-based Lexical Models for Automatically Structured Dictionaries

Keywords:

Dictionaries could be considered as the most comprehensive reservoir of human knowledge, which carry not only the lexical description of words in one or more languages, but also the commun awareness of a certain community about every known piece of knowledge in a time frame. Print dictionaries are the principle resources which enable the documentation and transfer of such knowledge. They already exist in abundant numbers, while new ones are continuously compiled, even with the recent strong move to digital resources.

However, a majority of these dictionaries, even when available digitally, is still not fully structured due to the absence of scalable methods and techniques that can cover the variety of corresponding material. Moreover, the relatively few existing structured resources present limited exchange and query alternatives, given the discrepancy of their data models and formats.

In this thesis we address the task of parsing lexical information in print dictionaries through the design of computer models that enable their automatic structuring. Solving this task goes hand in hand with finding a standardised output for these models to guarantee a maximum interoperability among resources and usability for downstream tasks.

First, we present different classifications of the dictionaric resources to delimit the category of print dictionaries we aim to process. Second, we introduce the parsing task by providing an overview of the processing challenges and a study of the state of the art. Then, we present a novel approach based on a top-down parsing of the lexical information. We also outline the architecture of the resulting system, called GROBID-Dictionaries, and the methodology we followed to close the gap between the conception of the system and its applicability to real-world scenarios.

After that, we draw the landscape of the leading standards for structured lexical resources. In addition, we provide an analysis of two ongoing initiatives, TEI-Lex-0 and LMF, that aim at the unification of modelling the lexical information in print and electronic dictionaries. Based on that, we present a serialisation format that is inline with the schemes of the two standardisation initiatives and fits the approach implemented in our parsing system.

After presenting the parsing and standardised serialisation facets of our lexical models, we provide an empirical study of their performance and behaviour. The investigation is based on a specific machine learning setup and series of experiments carried out with a selected pool of varied dictionaries. We try in this study to present different ways for feature engineering and exhibit the strength and the limits of the best resulting models. We also dedicate two series of experiments for exploring the scalability of our models with regard to the processed documents and the employed machine learning technique.

Finally, we sum up this thesis by presenting the major conclusions and opening new perspectives for extending our investigations in a number of research directions for parsing entry-based documents.

Résumé

Les dictionnaires peuvent être considérés comme le réservoir le plus compréhensible de connaissances humaines, qui contiennent non seulement la description lexicale des mots dans une ou plusieurs langues, mais aussi la conscience commune d'une certaine communauté sur chaque élément de connaissance connu dans une période de temps donnée. Les dictionnaires imprimés sont les principales ressources qui permettent la documentation et le transfert de ces connaissances. Ils existent déjà en grand nombre, et de nouveaux dictionnaires sont continuellement compilés.

Cependant, la majorité de ces dictionnaires dans leur version numérique n'est toujours pas structurée en raison de l'absence de méthodes et de techniques évolutives pouvant couvrir le nombre du matériel croissant et sa variété. En outre, les ressources structurées existantes, relativement peu nombreuses, présentent des alternatives d'échange et de recherche limitées, en raison d'un sérieux manque de synchronisation entre leurs schémas de structure.

Dans cette thèse, nous abordons la tâche d'analyse des informations lexicales dans les dictionnaires imprimés en construisant des modèles qui permettent leur structuration automatique. La résolution de cette tâche va de pair avec la recherche d'une sortie standardisée de ces modèles afin de garantir une interopérabilité maximale entre les ressources et une facilité d'utilisation pour les tâches en aval.

Nous commençons par présenter différentes classifications des ressources dictionnaires pour délimiter les catégories des dictionnaires imprimés sur lesquelles ce travail se focalise. Ensuite, nous définissions la tâche d'analyse en fournissant un aperçu des défis de traitement et une étude de l'état de l'art. Nous présentons par la suite une nouvelle approche basée sur une analyse en cascade de l'information lexicale. Nous décrivons également l'architecture du système résultant, appelé GROBID-Dictionaries, et la méthodologie que nous avons suivie pour rapprocher la conception du système de son applicabilité aux scénarios du monde réel.

Ensuite, nous prestons des normes clés pour les ressources lexicales structurées. En outre, nous fournissons une analyse de deux initiatives en cours, TEI-Lex-0 et LMF, qui visent à unifier la modélisation de l'information lexicale dans les dictionnaires imprimés et électroniques. Sur cette base, nous présentons un format de sérialisation conforme aux schémas des deux initiatives de normalisation et qui est assorti à l'approche développée dans notre système d'analyse lexicale.

Après avoir présenté les facettes d'analyse et de sérialisation normalisées de nos modèles lexicaux, nous fournissons une étude empirique de leurs performances et de leurs comportements. L'étude est basée sur une configuration spécifique d'apprentissage automatique et sur une série d'expériences menées avec un ensemble sélectionné de dictionnaires variés. Dans cette étude, nous essayons de présenter différentes manières d'ingénierie des caractéristiques et de montrer les points forts et les limites des meilleurs modèles x résultants. Nous consacrons également deux séries d'expériences pour explorer l'extensibilité de nos modèles en ce qui concerne les documents traités et la technique d'apprentissage automatique employée.

Enfin, nous clôturons cette thèse en présentant les principales conclusions et en ouvrant de nouvelles perspectives pour l'extension de nos investigations dans un certain nombre de directions de recherche pour l'analyse des documents structurés en un ensemble d'entrées.

xi [START_REF] Ma | Bootstrapping structured page segmentation[END_REF] and [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF] . . . . . . . . 3.7 Illustration of the Hyper-Level zones in a scientific paper recognised by Header Segmentation Model in GROBID . . . . . . .

List of Figures

4.1

First and second segmentation levels of a dictionary page (Larousse, 1972) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Example of the segmentation performed by the Lexical Entry model (Larousse, 1972) . . . . . . . . . . . . . . . . . . . . . . . 4.3 Left: Excerpt from Bibliographic References Section in [START_REF] Khemakhem | Enhancing Usability for Automatically Structuring Digitised Dictionaries[END_REF]. Right: Excerpt of Lexical Entries in (Publishing, 2009) . . . . . . . . . . . . . . . . . . . . 4.4 Excerpt from GROBID's architecture 1 . . . . . . . . . . . . . . 4.5 Sequence labelling a dictionary article (Larousse, 1972) TIONARY" in [START_REF] Ernest | A Comprehensive etymological dictionary of the English language[END_REF] (Mueller, 1878) . . . . . . . . . . . 6. 13 Minimal TEI Encoding as an Output of the Etym/Quote Model 6.14 Minimal TEI Encoding as an Output of the Etym Model . . . . 6.15 Internal Serialisation of GROBID-Dictionaries's Models . . . . 6.16 Final Serialisation of the <entry> Model in GROBID-Dictionaries102 7.1 Excerpt from a Mixtec-Spanish Dictionary [START_REF] Alvarado | Vocabulario en Lengua Mixteca[END_REF] . . 7.2 Excerpt from the Dictionnaire de la Langue Française Dictionary [START_REF] Littré | Dictionnaire de la langue française[END_REF]

Introduction 1.1 Overview

Computational lexicography is a field which has emerged from the combination of lexicography -the field dealing with the design and compilation of lexica for human use -with computational methods. This "adventurous" combination, as called by [START_REF] Hanks | Lexicography from Earliest Times to the Present[END_REF], that started in the late 60's freed western lexicographers from the content dictated alphabetical order compilation to the logical order proceeding. The impact of further computerising lexicography has been significant and led to the establishment of a large field which, according to [START_REF] Gibbon | Computational lexicography[END_REF], covers tasks such as text mining for corpus-based lexicon construction, the construction of lexica for Natural Language Processing (NLP) applications, automatic acquisition of syntactic or semantic information from texts, re-use of Machine-Readable Dictionaries (MRDs) for new lexica, and computer production of lexica for human use. The creation of lexica for NLP use has been focused in the last two decades on corpus based approaches at the expense of endeavours dealing with the reuse of MRDs which represent in general the digitised version of print dictionaries. This imbalanced situation has actually been the consequence of increased accessibility of large corpora coupled with a rapidly growing number of advanced dedicated processing tools. To favorise lexical acquisition from corpora over exploiting MRDs, [START_REF] Lemnitzer | Lexical Acquisition[END_REF] further argued that such resources are too old, internally and mutually inconsistent, narrow in scope, missing important information like frequency and distribution information and finally biased towards infrequent phenomena such as obsolete senses and usages.

In the previous decade, such a claim used to be relatively valid for wellresourced languages (e.g., English, French, German, Dutch..) and was strongly questionable for under-resourced languages where print dictionaries always represented the backbone for creating NLP lexica. In fact, print dictionaries encapsulate semi-structured lexical information vital for either rapidly harvesting material for building lexica in less resourced languages or for enriching well established resources with information about the diachrony of words. But after the retro-digitisation movement following the breakthroughs in Optical Character Recognition (OCR) techniques, legacy dictionaries have returned in the recent years to the spotlight. Currently an abundant number of digitised dictionaries are constantly uploaded to publicly accessible repositories 1 2 . In addition, the copyright clearance of such documents, as result of ageing or freely born intellectual properties, has opened up the possibility of their re-use. In parallel, established concepts in corpus linguistics, such as comparable corpora [START_REF] Kenning | What are parallel and comparable corpora and how can we use them?[END_REF], have raised the question of applying the same approaches for lexical acquisition from the large newly available corpora of dictionaries.

These facts, along with relatively primitive approaches to automatically structuring such resources, have unwittingly created a huge gap where most of the dedicated methods are ad-hoc and, consequently, unable to cover the important stream of dictionary material for NLP downstream applications.

Task Definition & Research Questions

In this thesis, we aim to reduce the aforementioned gap by studying the nature of digital print dictionaries and finding a suitable approach for structuring such resources on a large scale.

The task we are trying to solve comprises two major steps: the first deals with automatically parsing lexical information encapsulated in the text of a digital dictionary. The identification of such constructs greatly depends on a second milestone which studies the design of a scheme that supports different organisations of lexical information in the target classes of dictionaries. Therefore, we are considering the exploration of dedicated standards for modelling lexica as they provide a suitable framework for scaling up the creation of interoperable lexica [START_REF] Calzolari | Approaches towards a'Lexical Web': the Role of Interoperability[END_REF]. Switching back and forth between the two sub-tasks is necessary as the complexity of the lexical information needs to be supported by the parsing technique and the parser(s)'s grammar.

For the first sub-task, most existing approaches have been focused on rule-based methods [START_REF] Mykowiecka | Building an electronic dictionary of old polish on the base of the paper resource[END_REF][START_REF] Fahmy | Towards Structuring an Arabic-English Machine-Readable Dictionary Using Parsing Expression Grammars[END_REF][START_REF] Maxwell | Endangered data for endangered languages: Digitizing print dictionaries[END_REF]Ranaivo-Malançon et al., 2017;[START_REF] Steingrímsson | Digitizing the Icelandic-Danish Blöndal Dictionary[END_REF] with few attempts to make use of machine learning techniques [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF][START_REF] Crist | Processing the Text of Bilingual Print Dictionaries[END_REF][START_REF] Bago | Using machine learning for language and structure annotation in an 18th century dictionary[END_REF]. The dictionaries tested in these approaches have a relatively flat structure and the main task addressed has been tagging the tokens of dictionary articles. None of the state of the art methods has proposed an endto-end architecture for structuring print dictionaries that have different and deep structures such as nested entries or senses. The only exception to this is the work of [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF] who have built a framework for digitising, parsing and generating lexica focused on salient structures in bilingual dictionaries.

In the above mentioned related works, few addressed [START_REF] Maxwell | Endangered data for endangered languages: Digitizing print dictionaries[END_REF] or mentioned [START_REF] Mykowiecka | Building an electronic dictionary of old polish on the base of the paper resource[END_REF] the need to generate standardised lexica using Text Encoding Initiative (TEI) [START_REF] Budin | Creating Lexical Resources in TEI P5. A Schema for Multi-purpose Digital Dictionaries[END_REF] or Lexical Markup Framework (LMF) [START_REF] Francopoulo | Lexical Markup Framework (LMF)[END_REF]. Other lexicographic projects (Eckle-Kohler and [START_REF] Gurevych | Uby: A large-scale unified lexical-semantic resource based on LMF[END_REF][START_REF] Czaykowska-Higgins | Using TEI for an endangered language lexical resource: The Nxa?amxcín Database-Dictionary Project[END_REF] have been more focused on the standardisation of existing NLP resources and shared the use of TEI and LMF, which are considered to be key frameworks within the lexicographic community. The need to find a unified scheme and guidelines that combine the best of the two standards has been clearly formalised [START_REF] Czaykowska-Higgins | Using TEI for an endangered language lexical resource: The Nxa?amxcín Database-Dictionary Project[END_REF][START_REF] Romary | Using the TEI framework as a possible serialization for LMF. Rendering endangered languages lexicons interoperable through standards harmonization[END_REF] with some practical suggestions, as the division in their use within the lexicographic community represents an obstacle to the large scale interoperability dream. It is important to point out that the task of categorising lexical structures and their modelling using possible alternatives offered by a standard already represents a highly debatable subject among lexicographers.

These challenges raise the following research questions, which we tackle in this thesis:

• RQ1 -Sample-agnostic Models for Print Dictionary Parsing: The first question concerns the study of different existing print dictionaries by analysing their physical and logical facets and the consequent processing complexity. Then we want to explore building generic lexical models to parse different categories of dictionaries. We call a model a parser that enables the analysis and the structuring of text blocks in a dictionary according to a defined scheme.

• RQ2 -Unified Scheme for Structured Dictionaries: The second investigation will be carried out on the leading standards for modelling lexica, namely TEI and LMF. The goal is to find a compromising scheme that meets both the lexicographic requirements and parsing constraints identified in the first question.

• RQ3 -Scaling-up Lexical Models: In the last question we will empirically address the leverage of the constructed lexical models by studying the improvement of their performance on the same dictionary and other dictionaries. The possibility of finding more generic models covering documents with similar content is by no means excluded from our investigation.

Approaching all these research questions at once represents in itself a core research question in Digital Humanities (DH): to what extent could a collaboration be possible between the computational and the humanist?

Thesis Organisation

This research work is structured in eight chapters. In the current section, we present an overview of the organisation of the thesis:

Chapter 2 -Dictionaric Resources

Having established the context, in this chapter we will present a classification of existing digital print dictionaries. The goal of such a step is to gradually Chapter 1. Introduction delimit, based on different aspects of such material, the category of dictionaric resources that fall within the scope of our research.

Chapter 3 -Parsing Lexical Structures in Print Dictionaries

In the third chapter we present the challenges related to the parsing of lexical information in the already defined target dictionaric resources. Then we will draw the state of the art by reporting related studies and explain how they approached the aforementioned difficulties. We will also discuss the most advanced techniques to position our work within the computational landscape.

Chapter 4 -Lexical Models for Automatically Structuring Print Dictionaries

The fourth chapter is dedicated to the presentation of our approach and the architecture of the lexical models. We show how we were inspired by the analogy between the parsing of bibliographic data and our task and we will demonstrate how we succeeded in adapting an existing machine learning infrastructure to the requirements of our context. Details about the process of building these models for the new infrastructure called GROBID-Dictionaries, the challenges, and the consequent conceptual and technical solutions will be provided.

Chapter 5 -Standards for Structured Lexical Resources

In this chapter we give an overview of the most widely used standards within the lexicographic community. We set out to explain the shortcomings of these frameworks with respect to the requirements of automatic processing tasks, mainly the parsing application presented in Chapter 4.

Chapter 6 -Novel Standardised Schemes for Encoding Dictionaries

The pitfalls identified in Chapter 5 will be tackled in this chapter, where we present standardisation initiatives in line with our unification goals. We will present our involvement in shaping TEI-Lex-0 and the new LMF frameworks and how such interaction has been translated into the definition of the scheme of our lexical models.

Chapter 7 -GROBID-Dictionaries in Action

In this chapter more experiments, carried out with GROBID-Dictionaries and different dictionaries in different setups, will be described and evaluated. The goal here is to provide an extensive overview of the performance of our machine learning models when they are exposed to different dictionaric material. The limits of the architecture and the possibilities to scale up parsing dictionaries based on the collected data and more advanced techniques will be discussed. Experiments performed on non-dictionaric resources will also be presented to show the genericity of our approach and resulting models.

Chapter 8 -Summary and Perspectives

The last chapter will be dedicated to the conclusions drawn from the results of the experiments presented and community feedback. We will also present our vision for building on our work with respect to the requirements of the task undertaken and the advances in machine learning techniques.

Chapter 2

Dictionaric Resources

Introduction

Dictionaries or lexica (the plural of lexicon) have different representations and content, and therefore can be categorised according to different aspects. This thesis is focused on a parsing task applied to a specific category of dictionaric resources. To define this task, we firstly aim in this chapter to present the most common classifications of dictionaries and establish the nature of the material we are dealing with. The following categories are based on several dimensions that touch on the macro-logical structure, format, age and content of a digital dictionary.

Macro-Logical Structure

By macro-logical structure in this context, we denote the representation of relationships among words, meanings and concepts in a dictionary. Semasiology and Onomasiology are directly opposite approaches for studying and presenting such relationships.

Onomasiological Dictionaries

An onomasiological -from the Greek ónoma (name) and logos (study) -approach focuses on categorising words expressing a certain concept based on a possible synonymy of the words. It tries to answer the question "How is a concept expressed?". Figure 2.1 illustrates the word synonymy mechanism addressed in an onomasiological system.

Such an approach is commonly applied for building terminologies, but is also followed by lexicographers for compiling thesauri [START_REF] Roget | Roget's Thesaurus of English Words and Phrases[END_REF][START_REF] Lamy | The Cambridge French-English Thesaurus[END_REF] and synonymy dictionaries [START_REF] Urdang | Longman synonym dictionary[END_REF] .

Onomasiological resources are beyond the scope of the dictionaries studied in this thesis.

Semasiological Dictionaries

Semasiology -from the Greek semasia (meaning) -deals with polysemy, the coexistence of possible meanings of a particular word. The question to be Classic dictionaries [START_REF] Hornby | Oxford advanced learner's dictionary of current English[END_REF][START_REF] Hindley | Old French-English Dictionary[END_REF], used by language learners, are the most common form of dictionaries compiled following the semasiological approach.

In the context of this thesis, it is semasiological dictionaries that we are interested in.

Format

The format of a dictionary is determinant for the techniques to be used for the NLP task that we are addressing. We are interested in dictionaries in digital formats having specific features. In this section, we present a classification of possible formats of dictionaries and we specify the ones that fall within the present context.

Format

Born-digital Dictionaries

This category denotes dictionaries that have been originally compiled using computers and generated as a file that could be displayed using a web browser, a text editor or any other dedicated software (e.g., Microsoft Word, PDF viewer, etc). Their text content is usually represented as a sequence of text searchable by a machine and its quality greatly depends on the tool used to generate the file.

Digitised Dictionaries

This class gathers dictionaries in digital format resulting from the application of an Optical Character Recognition (OCR) system on scans or images of a print dictionary. Such resources are called also "retro-digitised" or "OCR'd" dictionaries.

The automatically recognised text, also known as OCRs, can be exported by the OCR tool as a separate text document or on the original scan as text layer, visible upon selection. The quality of the text and its layout information in such dictionaries varies widely, depending on the characteristics of the original scans, the OCR tool and the conditions in which the digitisation process has been carried out.

Both digital born and digitised dictionaries represent a typical format for resources to be structured using methods developed in the context of this work.

Machine Readable Dictionaries

Certain born digital dictionaries are generated from MRDs [START_REF] Atkins | Building a Lexicon The Contribution of Lexicography[END_REF][START_REF] Muller | Moving into XML Functionality: The Combined Digital Dictionaries of Buddhism and East Asian Literary Terms[END_REF][START_REF] Dendien | Le trésor de la langue française informatisé. Un exemple d'informatisation d'un dictionnaire de langue de référence[END_REF] which are resources that consist of text files controlled by pieces of code guiding the typesetting process. Such a format is commonly used by publishers to produce print and digital versions of dictionaries.

For the structuring task we are addressing, we do not consider processing this kind of resources.

Computerised Dictionaries, Lexical Databases and NLP Lexica

Computerised dictionaries are structured resources reflecting the logical structure of lexical entries such as part of speech, definitions, examples, etc. The explicit structure can be easily transformed into resources, suitable for computational tasks, called "Lexical Databases" or "NLP Lexica".

In the context of the present thesis, such a class of resources represents the target format.

Time

Time is reflected in dictionaries through certain lexicographic designs and practices which could lead to different dictionaries for very similar content. We distinguish two broad categories of resources: modern and legacy. It is hard to draw a clear line between the two of them as the classification is relatively subjective. But we will try to describe both categories based on our experience with different materials we have been dealing with.

Modern Dictionaries

These are the dictionaries that we use nowadays where, from the layout perspective, there are clear boundaries among the lexical entries and their main lexical information. Typographic information is omnipresent and in most cases consistently used as they have been generated and controlled automatically. The lexical information in such resources is usually expressed in modern language with fewer prolix descriptions.

Legacy Dictionaries

The age of existing legacy dictionaries ranges from a few decades [START_REF] Larousse | Petit Larousse illustré[END_REF][START_REF] Hornby | Oxford advanced learner's dictionary of current English[END_REF] to few centuries [START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF][START_REF] Littré | Dictionnaire de la langue française[END_REF]. Prose lexical descriptions, poetic examples and old language constructs and formulations are very common in the content of such resources. Besides the complex semasiological system in representing relatively similar lexical structures such as entries, sub-entries, related entries and senses, inconsistency in the use of the very system can be noticed in different parts of the same dictionary. This fact is totally natural as such resources took years, in some cases decades, to be compiled and often by more than one lexicographer. Typographic information is usually poor, which is not helpful for a human reader to infer the lexicographic system followed by the creator of the dictionary. Figure 2.3 shows a case of a less informative typography in a legacy dictionary along with a complex logical structure of entries, senses and related entries that could be interpreted differently.

Most digital legacy dictionaries were originally retro-digitised from print dictionaries that were produced before the use of computers was introduced. In addition to the logical complexity briefly explained above, issues related to the OCR process, which will be explained in Section 3.2, make the recognition of the text in such documents very challenging for any advanced OCR system and consequently result in a noisy text in the digitised version.

Modern and legacy dictionaries are both in the center of focus of this thesis.

Content

We see the content of a dictionary as having two dimensions: multilinguality and of lexicographic information. [START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF] of the entry ABORDER

Content

Multilinguality Dimension

The number of languages to be used in a semasiological system has a direct impact on the constructs in the content of a dictionary.

a. Monolingual

Monolingual dictionaries represent the basic form of dictionaries where words in a certain language are explained and described in the same language. Referencing words in other languages remains however possible, especially for expressing relationships between words and their origins, a study called etymology or diachrony.

b. Bilingual

Translation equivalents, along with explanations of varying length, constitute the core of bilingual dictionaries. Such resources are often used by learners of new languages to find translations of words in a newly learnt language or the other way around. In such dictionaries, it is very common to find two parts: one part for language A to language B and the second part for language B to language A. Compared to monolingual dictionaries, bilingual ones have shorter entries as the goal in this case is not to give an exhaustive lexical description.

c. Multilingual

Multilingual dictionaries are the least common form of dictionaries as multilingualism is relatively less frequent among people. The lexical description in such a category is even more compact than in bilingual dictionaries as an alignment between translation equivalents in three or more languages should be preserved for readability and pedagogical purposes.

Lexicographic Dimension

The last dimension in our classification of dictionaric resources represents the category of the lexicographic content.

a. Lexical

Lexical dictionaries are known as the default dictionaries for most dictionary users. This kind of document has the goal of representing the meaning and other lexical aspects related to lexemes, or words of a language. Named entities, such as person or city names, are not in the scope of such dictionaries, but are included in the following category.

b. Encyclopaedic

As mentioned earlier, named entities fall in the scope of encyclopaedic content in dictionaries. Such a content can be found in separate documents called encyclopaedic dictionaries or encyclopaedias [START_REF] Hindley | Larousse encyclopedia of music[END_REF][START_REF] Larousse | Grand dictionnaire encyclopédique Larousse[END_REF] as it can usually constitute a separate part in a dictionary. However, in many cases, the encyclopaedic content is mixed with the lexical content in one dictionary and the alphabetical order is the only sorting system that applies.

c. Etymological & Diachronic

As signalled earlier, etymology and diachrony are disciplines for studying the origin of words, which implies investigating relationships between words not only in the same language but also in different ones. Such a description could be condensed within the description of an entry in a lexical dictionary but may be separated and exhaustively described in dedicated documents called etymological dictionaries [START_REF] Ernout | Dictionnaire étymologique de la langue latine: histoire des mots[END_REF][START_REF] Ernest | A Comprehensive etymological dictionary of the English language[END_REF].

Chapter Summary

In this chapter we presented a classification of dictionaries that goes from macro structures and format to the class of content that a dictionary can encompass. In Figure 2.4 we summarise the consequent classification of digital dictionaries we are dealing with in the following chapters.

The rectangles in green represent the class of dictionary material we might have as input to structure, whereas the one in orange is the target category we want to reach.

Introduction

According to the Oxford English Dictionary1 , parse as a verb has two meanings. The first is defined as "resolve (a sentence) into its component parts and describe their syntactic roles" while the second is a sub-sense of the former, related to the computing domain, and has the meaning of "analyse (a string or text) into logical syntactic components". Both senses apply in the context of the parsing task we are aiming at. The second has a broader computational meaning and in the case of dictionaries could be parsing the text of a page to recognise structures such as headers, footers, lexical entries or their components.

Dictionaries falling in the target categories defined in the previous chapter represent several challenges for their parsing. Previous studies have tried to overcome these obstacles partially or entirely to reach different levels of text analysis. In the current chapter we showcase the challenges for parsing print dictionaries and present the state of the art.

Challenges

Parsing print dictionaries comes up against several obstacles that need to be studied in order to have a better understanding of the computational challenges involved. We try to decode these aspects by focusing on two axes.

Born-digital vs Digitised Documents

Print dictionaries in digital format have been produced using computers by means of either Dictionary Writing Systems (DWSs) for the case of born-digital dictionaries or digitisation softwares obviously for digitised ones.

Abel, 2012 gives a general overview of the different DWS uses and new trends in compiling dictionaries, following the era of pen and paper. In-house systems, implemented in academic projects such as DEB [START_REF] Horák | Dictionary management system for the DEB development platform[END_REF] and Jibiki [START_REF] Mangeot-Nagata | Dictionary building with the Jibiki platform[END_REF], and off-the-shelf tools, used Chapter 3. Parsing Lexical Information in Print Dictionaries extensively by publishers as well as in academia, for instance TshwaneLex [START_REF] Joffe | TshwaneLex: a state-ofthe-art dictionary compilation program[END_REF], DPS (McNamara, 2003), or ABBYY Lingvo [START_REF] Kuzmina | ABBYY Lingvo electronic dictionary platform and Lingvo ?ontent dictionary writing system[END_REF], along with other commercial tools and ad-hoc editors have helped lexicographer to compile, retype or convert dictionaries.

The diversity of the technologies used remained, however, uncontrolled which resulted in pre-print versions that can represent serious challenges for automatic processing. For instance, but not limited to, a large number of dictionaries are exported and available as PDF files. PDF stands for Portable Document Format (PDF) which is the de facto and de jure exchange format that still causes computational headaches and remains an active research topic especially for extracting text and its typography from the original documents [START_REF] Tiedemann | Improved text extraction from PDF documents for large-scale natural language processing[END_REF][START_REF] Thaiprayoon | PDF Extraction Based on Lexical Analysis for Thai Texts[END_REF]. Figure 3.1 shows an example of anomalies that can be introduced by a PDF engine. In this case, metadata information has been found in almost all the pages of the dictionary but it is only seen in the raw text when extracted from the PDF file.

Processing digitised dictionaries is, in its turn, facing more complexity, not only because of the different compilation workflows and OCR systems, but also given the state of the raw documents. Unclear scans resulting from damaged pages or low resolutions, obsolete fonts, old scripts and old orthography among many other issues, challenge the most advanced digitisation techniques. [START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF] An OCR software sometimes puts spaces in the middle of words, and it can incorrectly recognise an individual letter in a word, such as misrepresenting the letter "I" for the letter "i." or the sequence "rn" for the letter "m". Correctly and consistently identifying typography used to markup microstructure in dictionaries also remains questionable for state of the art OCR systems, such as OCRpus [START_REF] Breuel | The OCRopus open source OCR system[END_REF], Transkribus [START_REF] Kahle | Transkribus-a service platform for transcription, recognition and retrieval of historical documents[END_REF], Calamari [START_REF] Wick | Calamari-A High-Performance Tensorflow-based Deep Learning Package for Optical Character Recognition[END_REF], or ABBYY Finereader2 . In Figure 3.2 we can clearly notice many text recognition errors in the OCR output for a legacy dictionary [START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF]. We see here an illustration of the impact of the scan quality and the old orthography on the digitisation outcome.

Several studies in the literature have investigated the impact of noisy data on basic downstream NLP tasks such as sentence boundary detection, tokenisation, and POS tagging [START_REF] Lopresti | Optical character recognition errors and their effects on natural language processing[END_REF] as well as more advanced computational applications like topic modelling [START_REF] Mutuvi | Evaluating the Impact of OCR Errors on Topic Modeling[END_REF].

This digitisation issue represents a serious obstacle for any parser aiming at decoding the logical system behind the typography conventions implemented in a dictionary, as we explain in the following section.

Logical Structure vs Physical Structure

Dictionaries as a whole and their lexical entries are highly structured objects embedding recurrent components. The organisation of a print dictionary can be analysed from two related perspectives: logical and physical structures

Logical Structure

The logical structure of a dictionary represents the functional constructs and their mutual connections, enabling the representation of the lexical information designed by the creator of the lexicon.

Semasiological dictionaries for alphabet-based languages share a general logical organisation. Such a dictionary is usually segmented into chapters where each chapter contains a list of dictionary articles, also called lexical entries, starting with the same letter and ordered alphabetically. In bilingual dictionaries, it is common to have chapters organised by pairs of languages.

Lexical entries are considered the elementary constructs of semasiological dictionaries and therefore it is important to define their key components to understand their logical representation and the links between them. Each entry contains a range of predetermined possible constructs that may differ, depending on the entry type and the purpose of the dictionary. It is beyond the scope of this thesis to give an exhaustive list of the possible lexical elements in lexical entries in all kinds of dictionaries. But to give some indication of the complexity of logical structures in lexical entries it is useful to sum up these constructs in the following items:

• headword: called also lemma. This is considered to be the key element in a dictionary article which is the subject of the lexical or encyclopaedic description and represents a written or spoken form. It usually occupies the first position in the lexical entry and marked with bold characters.

• variant form: a variant form of a headword. For instance, "color" is the American variant form of a headword "colour" in a British dictionary

• inflected form: a form resulting from the application of a regular or irregular inflection paradigm (i.e. pattern) to an original word. For example, "colored" is the inflected form of "color" representing its past participle.

• part of speech (POS): grammatical category of a word that may be a verb, noun, adjective, preposition, etc

• morphological information: any kind of information related to the morphology of a word form such as orthography, variant and inflected forms, etc. It describes the headword as well as its related forms.

• grammatical information: carries the grammatical description of a form such as the POS, gender, number, etc.

• sense: represents one possible meaning of the headword and is expressed through definitions, examples, usage domain, etc. A sense could carry a multi level nesting of sub-senses and may replace the function a lexical entry in some dictionaric designs (see Figure 3.3)

• etymology: provides information about the origins of the headword and its senses. Etymology may figure as a brief description in several possible spots within the dictionary article, or it can occupy the whole focus of the lexical entry's description in the case of etymological dictionaries.

• translation equivalent: maps a headword to its equivalent in another language and can be accompanied by definitions or examples. Such a description is the core representation in bilingual dictionaries.

• related entry: also called compound or Multi Word Expression (MWE). It represents the use of the headword or one of its senses in more complex constructs such as idioms, collocations, etc.

• cross references: are links to other lexical entries having a semantic relation such as synonymy or antonymy and are usually triggered by the use of "see X" or "cf. X" or their equivalents in other languages. In Figure 3.3 the red, green, purple and pink fields represent respectively the headword, pronunciation, POS and related entry. The article act is a homograph which has been represented in this dictionary as a lexical entry having one sense as noun and another one as a verb (the blue fields). Each sense has sub-senses and the second has an embedded related entry. This representation may also be interpreted as a noun entry having a sub-entry as a verb. In both cases, the sense representation differs from the logical pattern followed to model another homograph in the same dictionary, abuse, which Chapter 3. Parsing Lexical Information in Print Dictionaries could also be a noun and a verb. The logical structure of morphological and grammatical information remains the same, however.

The highlighted inconsistency in modelling entries is one of a number of phenomena that can be observed in modern dictionaries and is more frequent in legacy materials. The logical structure, or the syntax, of lexical entries translated in the presence of certain information or their order may totally change in a different dictionary. The difference depends on the multilingualty, lexicographic and time dimensions introduced in Chapter 2.

Figure 3.4 illustrates a high degree of diversity in the structures of entries from different dictionaries. Such internal and external variations represent a true challenge for having a unified parser that covers all these varieties.

Physical Structure

The logical structure of entries is translated in print dictionaries through a series of navigational components and markers such as font, font size and symbols (e.g. squares, bullets, diamonds). A fixed and predictable typographical system is essential to allow dictionary users to quickly and easily find the information they are looking for. The same applies to a parser which will decode the syntax of lexical entries based on their markup system.

Such a structure remains relatively preserved in the case of the same borndigital dictionary. But given the highly probable deformation of both the text and its typography in digitised documents, explained in Section 3.2.1, the physical structure would be greatly impacted. Consequently, features reflecting the typography of the lexical components would be weak, and in many cases, biasing for a parser aiming at predicting the labels to be assigned to the tokens of lexical constructs.

For both, digitised and born-digital dictionaries, the physical structure depends on the purpose of the lexicographic document as well as the choices of the lexicographer who will have a certain background, practices and typography preferences. Therefore, the combination of all these factors yields a wide range of physical representations of the lexical information (see Figure 3.4) which challenges their uniform parsing.

State of the art

The problem of parsing the structure of print documents, including dictionaries, has been addressed in the literature by different approaches. In this section, we provide on overview of the commonly used techniques. We put the focus on the most advanced methods which have made use of machine learning to solve the parsing task as well as similar tasks.

Rule-based

Rule-based techniques are very common in computational fields. They have been popular since the early use of computers as, for instance, they had been used for building early expert systems [START_REF] Buchanan | Principles of rule-based expert systems[END_REF]. Such a The set of defined rules constitutes a grammar that gives a formal description of certain patterns within the observed sample. A parser relies on such Chapter 3. Parsing Lexical Information in Print Dictionaries a grammar to perform the analysis of text structures.

Many ad-hoc rule-based approaches are still being used today to parse dictionaries using tailored grammars. In fact, one way [START_REF] Khemakhem | Towards an automatic conversion approach of editorial Arabic dictionaries into LMF-ISO 24613 standardized model[END_REF][START_REF] Fayed | Towards Structuring an Arabic-English Machine-Readable Dictionary Using Parsing Expression Grammars[END_REF] is to rely mainly on textual markers in the lexical description to infer the function of the parsed structures. Such an approach is commonly used when typography information do not exist and the text of the digitised or retyped dictionary is the only available material to analyse, as is the case for several MRDs.

Other studies [START_REF] Mykowiecka | Building an electronic dictionary of old polish on the base of the paper resource[END_REF][START_REF] Maxwell | Endangered data for endangered languages: Digitizing print dictionaries[END_REF][START_REF] Steingrímsson | Digitizing the Icelandic-Danish Blöndal Dictionary[END_REF] make use of the typography information, collected from the OCR output, combined with the textual markers to build the parsing grammar. Old but efficient platforms for a flexible writing grammar for new parsers are still in use, such as for example the LexParse tool [START_REF] Hauser | Dictionary entry parsing using the LexParse system[END_REF][START_REF] Lemnitzer | Lexical Acquisition[END_REF].

On the one hand, such an approach has several attractive sides, especially for small projects. First, the rules could be quickly implemented, or just written in the case of existing platforms like LexParse, and results could be obtained within a few days or weeks. Second, the system can be considered as a simulator of the decision process of an expert, but on a larger scale. This fact makes the interpretation of the results straightforward by humans and consequently the rules easy to fine-tune. It also represents a major upside for independent lexicographers who have limited little IT skills, as it frees them from the intervention of IT staff in their own workflow. Finally, since no advanced IT knowledge is needed to develop such a technique, such a choice can drastically reduce the costs for limited budget projects.

On the other hand, the above-mentioned studies have shown that such an approach can be useful to parse dictionaries with a flat structure, bilingual dictionaries for instance, or the shallow constructs in relatively more complex dictionaries. It remains, however, limited for parsing deep and extensive lexical descriptions where the lexicographic information becomes more complex along with possible inconsistencies in the representation of the logical and physical structures. For instance, the dictionary article FRANCE in La grande encyclopédie [START_REF] Berthelot | Annuaire-almanach du commerce, de l'industrie, de la magistrature et de l'administration : ou almanach des 500.000 adresses de Paris[END_REF] has 91 pages, AIR has 52 pages and FRANCO-ALLEMANDE (GUERRE) 31 pages. Any rules defined by humans are too subjective to cover all the patterns hidden in the body of such large lexical entries. The scalability question is not limited to large or legacy samples. The adaptation of the extracted rules to new samples may be costly and in many cases impossible as the dimensions of the new dictionary can vary widely.

Probabilistic Models

The need for scalable methods to parse print dictionaries has become clearly apparent. The use of machines can go beyond simply looking for patterns defined by a human expert. Probabilistic models have shown in the literature a great potential to leverage the capacities of machine learning techniques when human observation is limited or impossible.

In this section we introduce a family of probabilistic models gathering certain techniques suitable for our target parsing task.

Text Sequence Labelling

The parsing task we are addressing may be seen as the task of assigning a label from a set of possible tags to each token in a sequence of text. Let us assume we want to label this excerpt of an entry "act /akt/ noun 1. something which is done". A probabilistic model has the aim of predicting the probability of assigning a label to each token based on a learnt distribution from previously seen data. A label starting with "I-" marks the beginning of a new field in the sequence, whereas other labels (i.e "<def>") mean we are in the middle or end of a field. In this example: <lemma> marks a lemma, <pron> pronunciation, <pos> part of speech of the headword, <num> numbering tokens and <def> definition of a sense.

<I-lemma> act <I-pron> /akt/ <I-pos> noun <I-num> 1 <num> . <I-de f > something <de f > which <de f > is <de f > done FIGURE 3.5: Text Sequence Labelling
Such a text sequence labelling task is common in the NLP field, where POS Tagging and Named Entity Recognition (NER) are among the most popular tasks that have been addressed in the literature by applying a family of probabilistic models called graphical models.

Graphical Models for Sequence Labelling

The key assumption in graphical modeling is that a distribution over many variables can often be represented as a product of local functions that each depends on a smaller subset of variables. This factorisation shows a close connection to certain conditional independence relationships among these variables. The graphical appellation comes from the fact that both types of information can be represented by a graph.

These models describe the probability distribution P based on X and Y, where X and Y are random variables respectively ranging over observation sequences and their corresponding label sequences. The probability distribution P is calculated differently from one graphical model to another and the difference lies in how the independence among variables is modelled.

In this section, we present the conceptual difference between two types of graphical models which have been the most widely used in the literature for sequence labelling: Hidden Markov Models (HMMs) and Conditional Random Fields (CRF)s.

Hidden Markov Models (HMMs) An HMM is a generative model for describing a probability p(y 1:N , x 1:N ) over observation x 1:N and label y 1:N sequences. It is defined as the product:

Chapter 3. Parsing Lexical Information in Print Dictionaries p(y 1:N , x 1:N ) = N ∏ n=1 p(y n |y n-1 )p(x n |y n ) (3.1)
where p(y n |y n-1 ) denotes the transition between two successive states and p(x n |y n ) represents the distribution of an observation given its state.

To estimate such a joint probability distribution, a HMM needs to enumerate all possible observation sequences for tasks where an observation typically represents an atomic entity, a word in a document or a token in a sentence. Theoretically, such a model can not take into account features of each observation or those of neighbouring entities.

To calculate, for instance, the probability p(y 3 , x 3 ) of the sequence depicted in Figure 3.5, a typical HMM considers the token "noun" as an observation and the label "I-<pos>" as its state. The label and its transition are taken into account but no typographic features (e.g. bold, italic, etc) of "noun", "1.", "/akt/" or any token in the sequence can be considered.

Conditional Random Fields (CRFs) Conditional Random Fields are also a probabilistic framework for labeling a sequence of observations. Whereas an HMM is generative assuming independence among input sequence x 1:N , a CRF tries to relax this independence relationship by describing a conditional probability distribution p(y n , x n ) defined as:

p(y 1:N |x 1:N ) = 1 Z exp N ∑ n=1 F ∑ i=1 λ i f i (y n-1 , y n , x 1:N , n) (3.2)
where x n is an observation and y n is a state which respectively belongs to the x 1:N and y 1:N sequences. f i denotes an arbitrary set of feature functions and λ i: F are their associated parameters to be learned. The scalar Z is a normalisation factor to make p(y 1:N |x 1:N ) a valid probability over label sequences. Z is defined as

Z = ∑ y1:N exp N ∑ n=1 F ∑ i=1 λ i f i (y n-1 , y n , x 1:N , n) (3.
3) which has an exponential number of terms, difficult to compute in general. Note that Z implicitly depends on x 1:N and the parameters λ.

A CRF model removes the constraint that observations only depend on the hidden states in the same event. In fact, CRF provides the ability to model how observations affect each other. Such an ability is enabled through the f i:N functions, also called feature functions, which look at pairs of adjacent states y n-1 and y n , the set of input x 1:N , and where we are in the sequence.

For example, we can define a simple feature function for the text sequence in Figure 3.5 which produces binary values: it is 1 if the current token is "noun", and 0 if the current state y n is "I-<pos>":

f 1 (y n-1 , y n , x 1:N , n) = 1 if y n = I-< pos > and x n = noun 0 otherwise (3.4)
A feature function is used depending on its corresponding weight λ 1 . If λ 1 > 0, whenever f i is active (i.e. we see the token "noun" in the sentence and its corresponding tag is "I-<pos>"), it increases the probability of the tag sequence y 1:N . This is another way of saying "the CRF model should prefer the tag I-<pos> for the token "noun". If on the other hand λ 1 < 0, the CRF model will try to avoid the tag "I-<pos>" for "noun". And when λ 1 =0, this feature has no effect.

To define λ 1 one may set λ 1 by domain knowledge (most probably being positive), or learn λ 1 from corpus (let the data tell us), or both.

As another function, let us consider:

f 2 (y n-1 , y n , x 1:N , n) = 1 if y n-1 = I-< pron > and x n+1 = 1 0 otherwise (3.5)
This feature function is active if the previous tag is "I-<pron>" and the following token is "1". One would therefore expect a positive λ 2 to go with the feature. Consequently, f 1 and f 2 can both be active for a sequence as in Figure 3.5. This is an example of overlapping features describing the observations and states of a sequence.

The input sequence x 1:N is not limited to tokens. x 1:N can be the font of token or an attribute marking if a token is bold or not, italic or not etc. In practice, such a representation would be possible by generating for each token (i.e. observation) a vector containing the identity of the token along with its typographic description. Defining a feature function would then consist of selecting attributes from the vector representing the current observation. In such a setup, a feature can, for instance, be:

f 3 (y n-1 , y n , x 1:N , n) = 1 if y n = I-< pos > and x n = ITALIC 0 otherwise (3.6)
Such a feature is active when the current tag is I-<pos> and the current font is italic.

At this point, it becomes clear how much more flexible CRFs are than HMMs for defining candidate features extracted from the observation and state sequences. Such a flexibility gives the model the ability to use the different hints about the physical structure of dictionary articles to structure their text sequences.

Evaluation metrics

The performance of a model (i.e. tagging algorithm) is usually evaluated using a variety of different metrics, measured against gold standard data which have been tagged by human annotators and are known to be correct. To calculate them, it is necessary to count the different possible decisions:

• True Positives (TP), i.e., positive instances correctly predicted • True Negatives (TN), i.e., negative instances correctly predicted (the target label has not been used to tag wrong tokens)

• False Positives (FP), i.e., instances which are tagged with a target label where it should not be • False Negatives (FN), i.e., instances which are not tagged with the target label but should be Precision reports how many of the model's decisions to tag a token are correct, i.e. the higher the precision of a tagging algorithm is, the more confident we can be about the labelling of tokens with the current tag. Precision is formally defined as: Precision = TP TP + FP Recall reports how many of the positive examples in the gold standard are found by our algorithm, i.e. the higher the recall of our algorithm the more confident we can be that all tokens that should have the target label are correctly tagged. Recall is formally defined as: Recall = TP TP + FN F1-score (or F-measure ) represents a weighted average of precision and recall. It is usually considered as the most significant metric, as precision and recall are not useful in isolation. These two measures can be considered antagonistic: good precision might be achieved by tagging few but correctly (no incorrect prediction has been made), while perfect recall can be achieved by tagging everything with the target label. The F-measure is defined as:

F1 = 2 • Precision • Recall Precision + Recall
Accuracy reports how many of the decisions made by the algorithm are correct in total, i.e. considering both positive and negative examples. While this is also an indicator of tagging quality, it should be carefully judged depending on the dataset. It is considered as a good measure only when we have balanced datasets where the values of false positives and false negatives are almost same. Otherwise, a good accuracy can easily be achieved by always assigning the majority class. Thus, the F1-score is usually considered to be the more meaningful measure. Accuracy is formally defined as: Accurracy = TP + TN TP + TN + FP + FN Token/Field level Token, or word, level reports the labelling quality for each different token, where field, also called phrase, level evaluates the quality over a whole sequence of tokens. Field level quality is usually harder to achieve as one wrongly labelled token in a sequence is enough to consider the labelling has failed for the whole sequence.

Previous Work

The are not too many studies in the literature that have made use of machine learning for the purpose of parsing dictionaries. The few relevant endeavours have carried out experiments using mainly one of the two categories of the previously introduced graphical models: HMMs and CRFs.

HMM-based

Approach: [START_REF] Ma | Bootstrapping structured page segmentation[END_REF] presented a system for the acquisition and parsing of lexical information in print bilingual dictionaries. Processing of the content of an OCR'd dictionary page is carried out in two stages before the generation of a structured lexicon. The first step called "Dictionary Parsing" has the task of extracting each entry and marking its main fields (i.e. functional properties of a group of tokens). It is broken into 3 phases:

• Dictionary acquisition: by adjusting the setup for scanning and storing dictionary page scans

• Entry Segmentation: the step addresses the automatic identification of the starting and ending lines of each entry based on textual and typographic features (e.g. special symbols, word font and size, indentation, spacing..) that are manually extracted and then used to train a Bayesian framework. The process is iterative and relies on a human in the loop to provide corrections of the results to generate better samples for retraining the framework. The workflow yielded accurate segmentation results (between 96% and 99% accuracy)

• Functional Labelling: Each identified entry in the previous phase is processed at this level to differentiate fields containing Latin scripts from those that do not. Such fields, called "functional", can be pronunciation or translations, etc. A Nearest Neighbor Matching [START_REF] Ma | Bootstrapping structured page segmentation[END_REF] and Support Vector Machine [START_REF] Burges | A tutorial on support vector machines for pattern recognition[END_REF] classifiers have been experimented to identify these functional fields. The Support Vector Machine classification has been reported to perform better (above 90% average accuracy).

The second step comes after applying an OCR system and is dedicated to tagging entry tokens, which represents the previously explained sequence labelling task. [START_REF] Ma | Bootstrapping structured page segmentation[END_REF] experimented two approaches:

Chapter 3. Parsing Lexical Information in Print Dictionaries

• Rule-based: the approach is very similar to those presented in Section 3.3.1 and based on observing the lexical description typography to tune a tagger's grammar incrementally. Experimented with dictionaries with highly regular structures, this approach has been reported to work "efficiently".

• HMMs-based: for dictionaries with relatively more complex structures and noisy OCRs, HMM models have been used to leverage the joint distribution of features in the dictionaric data. [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF] give an exhaustive description of the same approach which has been used to parse similar bilingual dictionaries (see Figure 3.6). Manual annotation has been carried out for 400 randomly selected tokens to be tagged with their functional categories. [START_REF] Ma | Bootstrapping structured page segmentation[END_REF] and [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF] On the token level, both studies [START_REF] Ma | Bootstrapping structured page segmentation[END_REF][START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF] have reported comparable F1-scores (around 0.75 for French-English) and (around 0.87 for English-Turkish) using rule-based and HMM-based parsers. On the field level, the former performed clearly better (around 0.75 for French-English and around 0.88 for English-Turkish) than the stochastic approach (around 0.70 for French-English and around 0.77 for English-Turkish). [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF] reported lower stochastic performance with a Hindi-English sample on the field level (0.51 F1-score). They tried to boost the results by applying a rule-based postprocessing to the stochastic output. The hybrid method gave relatively better results but remained limited for certain samples.

Discussion: [START_REF] Ma | Bootstrapping structured page segmentation[END_REF] succeeded in the design of a pipeline comprising several modules implementing different techniques to parse bilingual dictionaries using ad-hoc and adaptive methods. The attempt to implement a scalable technique with a human in the loop to adjust the entry extraction results of a Bayesian system was successful. However, the random selection of tokens to build a HMM model, is questionable for the sake of learning labelling lexical sequences, even after tweaking the HMM modelling setup.

The HMM-based approach presented for the identification of salient logical structures in bilingual lexical entries can not be reliable for a scalable processing, given the limitation by design of such models.

CRF-based

Approach 1: The exhaustive study carried out by Crist, 2011 can be considered the most relevant work to this part of the thesis. He first explained the conceptual complementarity between logistic regression and HMMs for the purpose of modelling observations and their hidden states in a sequence labelling setup. This analogical study, also shared by several introductory studies to graphical modelling [START_REF] Zhu | Cs838-1 advanced nlp: Conditional random fields[END_REF][START_REF] Sutton | An introduction to conditional random fields[END_REF], has been used to justify his choice for CRFs, which is considered a combination of the best of both techniques, to parse two digitised dictionaries.

The first sample was a bilingual Lau-English dictionary and has relatively few and simple structure entries. All of the 1365 entries of the dictionary, containing over 15 000 tokens, were tagged with 10 different tags marking (i) headwords, (ii) integers distinguishing homographs, (iii) morphological category, (iv) indentation, (v) single letter headers, (vi) use of the headword in a context (vii) English definitions, (viii) abbreviations indicating POS or other morpho-syntactic categories, (ix) cross references and (x) English prose discussions. For each token, a set of 24 features is generated to describe the typography (e.g. bold, italic) and the category of the current token, part of it or preceding token (e.g, final character is semi colon or not, its preceding token ends with a comma, etc).

The second sample of the experiment was an old English dictionary with over 1300 pages and 60000 articles. 13 randomly selected pages containing 306 entries and over 20000 tokens were manually tagged with 17 labels. The Chapter 3. Parsing Lexical Information in Print Dictionaries new tags give more precision regarding the categories of the logical lexical structure of each token, such as Latin translations, Greek translations, morpho-syntactic information, etymlogy, etc. Each token was extensively described using 35 features, with more focus on the language, the morphology and belonging of each token to existing vocabularies (e.g. abbreviations used in the dictionary).

The training of the two CRF models was performed through the use of the MALLET framework [START_REF] Mccallum | Mallet: A machine learning for language toolkit[END_REF]. Accuracy was the measure chosen to evaluate the results and behaviour of each model. In the first experiment, a plateau of 95% was reached by using 3100 tokens, where the mean number of tokens per entry is around 10. The second experiment reported 94% accuracy reached by using over 7400 tokens, where an entry contains on average 65 tokens.

The remainder of the work studies in depth the structure of 100 digitised dictionaries and focuses on the crucial steps for digitising and parsing dictionary highlighting the role played by the OCR quality in such a process. Discussion 1: Through his exhaustive survey, Crist, 2011 managed to expose the diversity and the complexity of the logical and physical structures in digitised dictionaries, and consequently, the need for a flexible parsing framework as CRFs. Empirically, he succeeded in leveraging the modelling capacity of CRFs to machine learn the parsing of two different dictionaries. Moreover, his experiments showcased the difference in complexity between dictionaries and the consequent need for more descriptive features and larger training datasets.

However, the implemented technique remains experimental and focused on parsing already extracted lexical entries, with no practical solutions for identifying entries' boundaries which is a non-negligible issue. The general accuracy measures provided do not give a precise evaluation of the models for parsing the different structures exhaustively analysed. In fact, the large number of labels to infer at once seems to be overwhelming for one model. We also have reservations about the representativity of the randomly selected entries in the training dataset to cover all the structures to be predicted in the evaluation dataset. Regarding the features used, their selection was tailored to the sample, which limits the ability of the resulting models to scale up. Less language dependent along with long-range features (i.e. features describing several tokens preceding and following the current token) could have resulted in a CRFs model that supports both samples with only more training.

Approach 2: Another work presented by Bago and Ljubešić, 2015 has studied the use of CRFs for the purpose of speeding up language and structural annotation in a multilingual legacy dictionary.

The first use case has the goal of learning the inference of a token's language based on its context in the dictionary article. Over 100 entries, comprising around 2% of all the tokens of the dictionary, were randomly selected.

These tokens were annotated with 3 labels indicating the language: (i) Croatian, (ii) Italian, (iii) and Latin, with a domination of Croatian tokens (i.e. over half). A variable set of around 20 features, mostly focused on the text of the token and its neighbouring, was generated for each token. A set of features includes: Boolean variable whether the token is lower-case or not, the surrounding tokens and their lower-case form, and the frequency of the token's sequential trigraphs. An incremental experiment was carried out to find the final subset of best features giving the highest accuracy and F1-score: (i) 0.98 for Croatian, (ii) 0.97 for Italian, (iii) and 0.99 for Latin,.

The second use case deals with parsing the lexical structures carried by the tokens by applying the same approach with a slight adjustment of the features and new task-specific labels. Language and suffix of 4 character length were added to the feature set of the incremental experiment and kept in the final subset of informative features. 19 different tags were used to label the data with flat structural category of a token, such as POS, citation, cross reference, bibliography, punctuation, etc. Given the fact that the entries were randomly selected, the performance of the trained CRFs model significantly varied from zero, for non frequent labels in the training dataset, to 0.99 and 1.0 for punctuation and line beginning tags.

The CRFsuite [START_REF] Okazaki | CRFsuite: a fast implementation of conditional random fields (CRFs)[END_REF] has been used to train the CRFs models with the annotated data. The learning curve shows that 40% of the data brings the model close to the plateau, which confirms the findings of [START_REF] Crist | Processing the Text of Bilingual Print Dictionaries[END_REF] showing the efficiency of CRFs models to quickly learn the hidden states in dictionary text sequences.

Discussion 2: Through the language labelling use case, Bago and Ljubešić, 2015 have demonstrated the positive impact of long range features on the performance of CRFs models. They have also shown the power of text based features for some specific tasks when there are no structure categories to be identified. However, the second use case has exposed the need to have observations describing the physical structure to infer the logical structures in a dictionary.

Statistics about each label in the training and test batches are missing but it is clear that the dataset is imbalanced for the use case of structural aspects learning. Thus, this is another experiment showing the specificity of lexical data and the need to have lexicographic expertise for the selection of a training sample. Another part of the issue of having low performance over all the labels in the second experiment could be explained by the fact that several heterogeneous labels have to be learned by one model.

The approach is reported to speed up the annotation process but still not reliable for full automatic labelling of lexical constructs, even with a sample that has no lexical depth.

Probabilistic Models for Parsing Bibliographic Data

All the related studies presented so far have followed a bottom-up approach for decoding lexical information in print dictionaries. The reviewed machine Chapter 3. Parsing Lexical Information in Print Dictionaries learning approaches have manifested difficulties in capturing heterogeneous lexical structures using one model.

Our study of the literature led us to find a top-down a approach for parsing bibliographic information in a category of print documents, representing several similarities with the parsing task we are addressing. In what follows we present this approach implemented in a framework called GROBID.

GROBID (GeneRation Of Bibliographic Data)

GROBID (GROBID 2008(GROBID -2020;;[START_REF] Romary | GROBID-Information Extraction from Scientific Publications[END_REF] is a machine learning framework that implements a modular approach for the analysis and extraction of bibliographic constructs in scientific papers and patent documents.

GROBID has used over 10 CRFs models to orchestrate the parsing of such documents in a cascade fashion. Each model is responsible for the identification of a homogeneous set of structures. At its first extraction level, GROBID detects the main blocks of a paper such as the header, the body, the references, annexes, etc. These main parts are further structured at the following level, like the header which is recognised and parsed in a second stage to extract the title, authors, their affiliations, abstract and keywords. The references are also extracted in separate items and then parsed one by one to detect the titles, the authors and the other publication details. GROBID (up to version 0.5.3) relies on an advanced version of CRFs [START_REF] Lavergne | Practical very large scale CRFs[END_REF] which allows the modelling of a large number (millions) of features per model. Each model is designed with dedicated features and labels supporting the parsing of a certain level of the bibliographic information. For the first segmentation model, for instance, each line in a document is described with over 30 seed observations3 that are expanded in more local and long range features based on the text, layout and typography.

Such an extensive use of features results in more complex feature engineering and a longer learning process but the outcome of the models shows it pays off. The framework has been ranked for years among the best system for such a task [START_REF] Saleem | Information extraction from research papers by data integration and data validation from multiple header extraction sources[END_REF][START_REF] Lipinski | Evaluation of header metadata extraction approaches and tools for scientific PDF documents[END_REF][START_REF] Torre | MATESC: Metadata-Analytic Text Extractor and Section Classifier for Scientific Publications[END_REF][START_REF] Tkaczyk | Machine Learning vs. Rules and Out-ofthe-Box vs. Retrained: An Evaluation of Open-Source Bibliographic Reference and Citation Parsers[END_REF] with high accuracy information extraction, thanks to its cascading sequence labelling and the leverage of CRFs modelling capacities.

Chapter Summary

In this chapter, we have shown the bottlenecks for parsing lexical structures in print dictionaries. The difficulties lie mainly in the noisy digital material to process and the complexity and possible inconsistency of the logical structure to dismantle, through physical clues highly impacted by the varying noise in the material.

The study of the literature has shown that it is still of actuality to use ad-hoc approaches [START_REF] Maxwell | Endangered data for endangered languages: Digitizing print dictionaries[END_REF][START_REF] Steingrímsson | Digitizing the Icelandic-Danish Blöndal Dictionary[END_REF], based on parsing grammars defined on the basis of human expert observations. HMMs-based methods [START_REF] Karagol-Ayan | Acquisition of bilingual MT lexicons from OCRed dictionaries[END_REF][START_REF] Ma | Bootstrapping structured page segmentation[END_REF] have tried to relax the dependence on human expertise by relying on probabilistic models to learn the distribution of labelled sequences. The attempt was partly successful as comparable results have been achieved by an adaptive HMMs system for several dictionaric samples. However, the approach remained focused on capturing salient structures in a category of resources (bilingual dictionaries) and did not leverage the context of tokens, given the conceptual limits by design of HMMs.

CRFs based approaches [START_REF] Crist | Processing the Text of Bilingual Print Dictionaries[END_REF][START_REF] Bago | Using machine learning for language and structure annotation in an 18th century dictionary[END_REF] have been adopted to overcome the limits of HMMs and provide a scalable framework that integrates noise and uncertainty along with the rich contextual clues that are often present in print dictionaries. The visited CRFs setups have illustrated ways to benefit from the textual and typographic information of tokens in a context. However, the solution for the different sequence labelling tasks suffered from the "one model does it all" strategy. It risks sanctioning certain labels which are less represented in the training dataset. Moreover, the evaluation of such an issue is usually hard given the large number of Chapter 3. Parsing Lexical Information in Print Dictionaries heterogeneous labels per model. The attempt to reduce labels per model has shown positive results with dictionaries [START_REF] Bago | Using machine learning for language and structure annotation in an 18th century dictionary[END_REF]. A larger modular approach [START_REF] Romary | GROBID-Information Extraction from Scientific Publications[END_REF] has also given highly accurate results with a similar complex parsing task, which supports the "divide and conquer" strategy.

We hypothesise that the use of CRFs can be improved and tuned to build models that are generic enough to cover several categories of lexica. Following a waterfall approach for the parsing case of print dictionaries seems to have the potential to solve the issue of overwhelming lexical labels. These assumptions are our starting point to find answers to the generecity and modularity questions.

Chapter 4

Lexical Models for Parsing Print Dictionaries

Introduction

Previous work introduced in Chapter 3 has left open questions regarding the ability of probabilistic models to parse complex lexical information along with the variations and inconsistencies in different dictionaric print material. Parsing print dictionaries using probabilistic models, CRFs more specifically, seems to have the potential to overcome the scalability issues. However, more investigations need to be carried out to find additional assets for using such probabilistic models as well as their limits.

Given the unrealistic short-term goal of structuring all kinds of dictionaries by the click of one button, our aim in this thesis is not limited to a proof of the scalable structuring concept but we also aim at building an end-to-end infrastructure for structuring dictionaries which can be easily adapted and extended. Getting domain experts, who may have limited IT skills, involved in shaping the lexical models is on our agenda for two reasons: first because we do not have the knowledge required to annotate data in several categories of dictionaries in a multitude of languages. Second, we do not have enough resources to perform such large-scale and complex annotations, which are the cornerstone for conducting any machine learning experiment.

In this chapter, we present our approach which has been inspired from another task dealing with a different category of text documents. We also give details about building GROBID-Dictionaries1 , a new architecture of machine learning models dedicated to the analysis and extraction of lexical structures from digital and digitised dictionaries, following the novel approach. An overview on the challenges encountered and the solutions to implement our approach, are also discussed in the following sections.

Cascading Parsing

Lexical information in print dictionaries has several granularity levels which represent the logical structure of the lexicon. Recognising all these elements 36 Chapter 4. Lexical Models for Parsing Print Dictionaries at once is not an obvious task as the complexity we want to dismantle involves hierarchies of lexical components and different constructs that do not have the same physical structure. Therefore, we addressed the parsing task by adopting a top-down approach where several parsers are built in cascade to go from the less to the more granular. To be easily adaptable to new dictionary samples, we chose machine learning over rule-based techniques to build the cascading models. And since CRF models have been the more successful in the literature for sequence labelling tasks, we decided to explore them to implement our approach.

Approach

We followed a divide-and-conquer strategy to dismantle text constructs in a print dictionary, based initially on observations of their layout. Main pages (see Figure 4.1) in almost any dictionary share three blocks: a header (green), a footer (blue) and a body (orange). The body is, in its turn, made up of several entries (red).

Each lexical entry can be further broken down (see Figure 4.2) into: form (green), etymology (blue), sense (red) or/and related entry. The same logic could be applied further for each extracted block, as long as the finest lexical structures have not yet been reached.

Such a cascading approach ensures a better understanding of the learning process's output and consequently simplifies the selection process of the machine learning features. Limited exclusive text blocks per level help to diagnose the cause of prediction errors significantly. Moreover, it would be possible to detect and replace at an early stage any irrelevant selected features that can bias a trained model. In such a segmentation, it becomes more straightforward to notice that, for instance, the token position in the page is very relevant to detect headers and footers but has almost no relevance for capturing a sense in a lexical entry, which is very often split over two pages.

Such a generic approach enables the modular creation and the flexible tuning of any required number of models to parse certain constructs. But at the same time, the flexibility should be controlled to avoid ending up with an architecture tailored for one sample or category of lexica. The balance we are aiming to strike should be translated in an architecture of models that is generic enough to be adapted to any dictionary by simply annotating a small sample, a key known asset for CRFs. Therefore, this milestone is highly connected to the modelling decisions and best practices that will be discussed in Chapter 6. Switching between the two processes to adapt either the architecture or the modelling to each other, is necessary to make sure that the outputs of the two stages are compatible.

Bibliographic Information and Lexical Information: Parsing Similarities and Differences

Our study of the literature led us to notice a remarkable analogy between the structures that can be extracted by GROBID [START_REF] Romary | GROBID-Information Extraction from Scientific Publications[END_REF], Figure 4.3 illustrates visually such an analogy between the structure of bibliographic entries in a scientific paper and lexical entries in a print dictionary. The underlined fields highlight the analogue structures we noticed respectively between: author names versus headword (red), publication year versus pronunciation (green), publication title versus POS (blue) and book title versus sense (orange). To these backbone structures in both categories of documents, additional fields can be joined to the description of an entry such as location and month of a conference versus a general note about a dictionary article (purple). Note that the transitions between the fields of bibliographic and lexical entries are marked by a consistent change in either the typographic features (e.g. bold, italic, font change, etc.), or/and textual markers (e.g. parenthesis, slash, number, dot, etc.).

This correspondence is reinforced by the fact that GROBID actually relies on these features to perform a cascading parsing of the text of a scientific paper and, in particular bibliographic references, in the same way shown in Figure 4.3. These facts incite us to investigate the adaptation of such models to the parsing of lexical entries.

Scientific papers processed by GROBID and the print dictionaries we are targeting represent, however, logical and physical differences. First, the logical structure of many categories of dictionaries is more granular than bibliographic information and can support different interpretations of structure classifications, depending on the background of the expert (see Figure 2.3). Consequently, annotating lexical information is more costly and less consistent than bibliographic data in scholarly articles. Second, the vast majority of available scholarly papers are the product of work dating back to few decades ago which consequently means that most of them are born-digital as opposed to the available dictionaries we are targeting, which are mostly digitised. This implies more obstacles for processing and classifying the text they contain.

GROBID

GROBID was initially implemented as a machine learning system for parsing and extracting bibliographic data from scholarly articles, mainly text documents in PDF format. It relies on CRF models [START_REF] Lavergne | Practical very large scale CRFs[END_REF] to perform a multi-level sequence labelling of text blocks in a cascade fashion which are then extracted and encoded in TEI elements.

Such an approach has been very accurate for that use case and the system's Java API2 has been one of the most widely used by bibliography research platforms and research bodies worldwide, including ResearchGate3 , HAL4 , Mendeley5 , CERN6 among many others. GROBID as a platform for manipulating PDF documents has powerful functionalities to extract and manipulate text in digital and digitised material. In addition, its API provides a benchmark for measuring the performance of newly developed CRF models and represents a suitable framework for conducting machine learning experiments. The API benefits from an active developer community guaranteeing the sustainability and the constant evolution of the embedded NLP libraries.

On the other hand, the cascading behaviour of the system comes from a modular structure of the API which makes CRF models, implemented following the same logic, portable and easily pluggable to the core GROBID models [START_REF] Lindemann | Retrodigitizing and Automatically Structuring a Large Bibliography Collection[END_REF].

Given all these conceptual analogies and technical conveniences, we chose GROBID as a core platform to implement our approach. In the following section, we present the novel architecture for parsing lexical information called GROBID-Dictionaries.

GROBID-Dictionaries

GROBID-Dictionaries [START_REF] Khemakhem | Automatic extraction of TEI structures in digitized lexical resources using conditional random fields[END_REF][START_REF] Khemakhem | Enhancing Usability for Automatically Structuring Digitised Dictionaries[END_REF]) is a machine learning infrastructure which relies on cascading CRF models for parsing text content of dictionary pages. It takes as an input a PDF text document and generates a TEI P5 [START_REF] Budin | Creating Lexical Resources in TEI P5. A Schema for Multi-purpose Digital Dictionaries[END_REF] compliant encoding where the various segmentation levels are associated with an appropriate XML tessellation.

Cascading Lexical Models

Our cascading models are designed in a way to support the encoding of the detected structures in multiple TEI constructs. The TEI schemes and the decisions behind modellings granular information will be discussed in detail in Chapter 6. But for the purpose of explaining the scope of each model, we are focusing more on the adequacy of a certain TEI modelling to the implemented cascading mechanism.

After having fully encoded a lexical entry, the task becomes more specific and more challenging when it comes to defining the TEI structures to be extracted by each model. It is a question of finding the appropriate mapping between the TEI elements and the labels to be set for the models that share the task of structuring the text in cascade. In addition, the process is at the same time constrained by the need to avoid having structures from different hierarchy levels being extracted at once. In fact, the CRF models, as they could be used from the GROBID core, do not allow the labelling of nested text sequences. A modification of such an aspect is still possible but it could be costly and is not necessary for implementing our approach, as we want to keep the models the least complex possible to ease the feature selection process.

The matrix in Figure 4.5 represents a set of feature vectors (see Appendix A for the description of each feature) describing a lexical entry condenser, which will be labelled by the "Lexical Entry" model. The latter has the task of detecting the main blocks in a lexical entry, if they exist. For the sense information, the model has been trained to extract each parsed text sequence representing a sense. Each vertical column is a specific feature for all the tokens of the lexical entry and each horizontal line corresponds to all the features of each token. The feature vectors and columns serve as the basis for the feature selection process that will be explained in Section 4.3.2.

In the second phase, comes the role of the trained model to give a prediction of a suitable label for each token, based on all its feature values. A structure corresponds then to the sequence of tokens having the same label, Having this logic in mind and after observing different TEI encoding schemes for a wide range of dictionaric samples, we defined the lexical models of our parsing architecture iteratively. The hierarchy of the CRF models implemented in GROBID-Dictionaries is illustrated in Figure 4.6

We give below a description of each model and its labels :

• Dictionary Segmentation: This is the first model and has the goal of segmenting each dictionary page into three main blocks: headnote for header and head-note information, body for all the text area containing the lexical entries of a page labelled as body, and footnote for footer and footnote information. • Lexical Entry: The third model parses each "Lexical Entry", recognised by the second model, to segment it into: lemma, variant, or inflected for morphological and grammatical information related to a "Form", etymology for etymological and diachronic description, senseGramGroup for grammatical information related to a "Sense", sense for semantic information or a "Sense", sub-entry for an embedded entry, reference for a reference to another entry, related entry for a related entry, note for general remarks about the whole entry, and formGramGroup for general grammatical description.

Another instance of this model can be used recursively to parse generated labelled sub-Entry and Related entry text blocks, as the logical structure of these sub-structures follows the same scheme as a Lexical Entry construct.

• Form: This model analyses any "Form" block such as lemma, variant, or inflected, generated by the Lexical Entry model, and segments the information it contains. The list of possible labels for this model contains for the moment: orthography to contain the orthography of a "Lemma" or a "Variant", part to contain the extent of an othography of an "Inflected From", pronunciation for pronunciation, gramGroup for grammatical information, such as POS, gender, number, etc, language for language information about a "Form", name for headwords in encyclopedic dictionaries, description and note for prose descriptions and notes related to morphology or grammar, and usage for usage information of the analysed form.

• Grammatical Group: This model is plugged at several levels of the architecture and has the task of parsing a group of grammatical information generated by Lexical Entry, Form, Sense or Sub-Sense models.

The actual list of labels contains: pos for POS, gramElement for a piece of grammatical information that is going to be typed, tense for tense of a verb, gender for gender information, number for number, subcategorisation for information about transitivity, count-ability, etc., and note for prose notes about grammar.

• Cross Ref: Such a model makes it possible to parse cross reference constructs generated by the Lexical Entry model and potentially at other levels where the logical and physical structures are represented in a similar way. The main labels predicted by this model are: label for textual triggers of a reference (e.g. "See", "voir" or "V.", etc), reference for a internal or external reference, and relation to type the reference instance (e.g. synonym, antonym, false friend, etc). Other labels are also enabled for this segmentation level such as bibliography for bibliographic information or note for further notes.

• Sense & Sub-Sense: These two models orchestrate the decomposition of the hierarchy of senses, if sense nesting occurs. For the Sense model, the supported labels are subSense for an embedded sense, gramGroup for existing grammatical information, num for sense numbering, and note for any prose description related to the upper sense. The mission of this segmentation model is to extract senses that will be parsed by the Sub-Sense model to recognise possible gramGroup for embedded grammatical information, definition containing a definition of a sub-sense, example for sense illustration, translation for translation equivalents, usage for usage information, related entry for possible embedded related entries, etymology for diachronic information related to the sense, and finally reference for recognised cross references. Note that, if no sense nesting occurs, these two steps still need to be followed one after the other. For an entry that has only one sense, this sense needs to be annotated first as a sense for the Lexical Entry model and then as a subSense for the Sense model. A control for displaying the final output will make sure that redundant sense tags are removed. Finally, an additional sense nesting level can be triggered when the senseGramGroup label is used and all the sense elements recognised by the Lexical Entry model along with senseGramGroup will be wrapped in a sense construct inside a Lexical Entry.

• Etym/Quote & Etymology: parsing etymological and diachronic information is carried out through two successive steps. The first makes use of the Etym/Quote model to differentiate quote text blocks from the rest of the etymological description, which is labelled as segment.

Then both recognised blocks are parsed in the second step with an Etymology model to extract definition, mentioned for etymologically related words, language for information about the related word's language, bibliography for bibliographic details, and possible nested segment(s). Such cascading processing is necessary as quote could have a complex structure that needs to be processed in the same way as segment constructs. These cascaded models can be used to parse diachronic information wherever it appears (e.g. under subSense, related entry, etc).

For all the models presented above, two more labels are required: punctuation, which represents an abstract label for any punctuation or symbol marking the separation among fields recognised by a CRF model, such as the full stop marking the end of a lexical entry's description, bullets marking the beginning of a definition or an example, etc. Such an abstraction helps the models to converge quickly, especially for some feature setups where the label of the previous token in a sequence is used as a feature to predict the label of a current token. This aspect will be explained further in Section 4.3.2.

The second label we added to the set of elements to be recognised by a model is dictionary scrap, which can be considered as a trash collector used to encode any text sequence that can not be labelled with any of the tags permitted for a model. Such a label is very useful when it comes to labelling a noisy text resulting from an OCR output or meta-data information that could figure in the text of the dictionary at any random position.

Finally, all these tags can be used more than once per level, as for multiple lexical entry to segment a body or definition to structure a subSense.

Feature Engineering

Given the impact of selected features on the learning of CRF models explained in Section 3.3.2, the feature engineering or selection step is crucial.

The process begins with preparing vectors describing the text sequence we want to structure (i.e. the text of a whole page, the text of a lexical entry, the text of a sense, etc). To do so, we designed the vectors based on two variations:

• Token vs Line based clues: The descriptive vectors are based on text and layout features of single tokens (see Figure 4.5). The vector generation mechanisms we implemented can adjust the level on which the descriptive matrix is focused. For the first model, Dictionary Segmentation, which has the task of parsing all the text of the document, is more efficient in terms of learning the distributions and guaranteeing a fast processing to use the first two tokens (punctuation marks with their preceding token are considered as one token) of each line rather than using all its tokens. Our decision was initially based on the observation of similar behaviour of analogue models in GROBID. Moreover, the experiments on several samples confirmed the rapid and efficient prediction results. This choice was dropped starting from the second model, Dictionary Body Segmentation, where lexical markers at different positions of the lines are often crucial for the model to learn the triggers for starting labelling a new field. For that, each vector describes each token of the text sequence.

• Generic vs Lexical clues: For the Dictionary Segmentation model, we used features based on a descriptive matrix analogous to the one defined for GROBID's first segmentation model (a detailed description of the vectors is provided in A. 1.1). This choice was motivated by the power of such a model to detect macro areas in a document, a task which is similar to the scope of the first model in our architecture. For the rest of the models in GROBID-Dictionaries, we chose to rely on restricted descriptive vectors where we drop the information that is unlikely to be useful. The token position in the page, for instance, is very informative for the Dictionary Segmentation model to differentiate a page body from headnotes and footnotes but has almost no relevance, and is probably misleading for capturing a sense in a lexical entry that is very often split over two pages. Excluding such input for the CRF models results in reducing the number of feature functions of a model and consequently making its training process and its size more compact. Therefore, we extracted only 16 descriptive features (see columns in Figure 4.5): 8 based on the text and the rest carrying layout and typography clues about each token, such as a change of font or line breaks (for more details, see Appendix A.1.2). However, we tried to maximise the abstraction over key lexicographic markers, namely field separators.

In fact, we dedicated one flag feature to mark the tokens that are a punctuation, opening or closing brackets (parenthesis, square brackets, and braces are considered brackets in this case).

From the resulting descriptive matrix we used Wapiti8 , a CRF library implemented by Lavergne, Cappé, and Yvon, 2010 and adapted within GRO-BID to perform the feature engineering process by experimenting combinations of selected features from each descriptive vector and its neighbouring tokens. Wapiti allows feature tuning to be performed through dedicated files specifying a set of feature templates (i.e. selected features). We studied the tuning of feature templates empirically to find the best combination for each model, based on the various samples we collected and annotated. Given the fact that the templates select the features from the descriptive matrix, the feature engineering process follows the same separation between the first model and the rest of GROBID-Dictionaries' architecture. In other terms, we prepared three classes of template combinations and for each class we define one combination for the first model of the architecture and another one for the rest of the models. We detail the combinations in the following: .2.4). We defined these templates based on those used for the Dictionary Segmentation model and followed the same logic to define the descriptive matrices, by dropping information like the position of a token within a block or a page, or the description of the current line. For the remaining templates, we defined a combination that has a restricted bidirectional window of information about neighbouring tokens.

• Engineered Feature Templates: As a third class, we used Engineered feature templates that we define based on rounds of feature selection. Enlarging the bidirectional window mentioned above is the main engineering action for this class. For the Dictionary Segmentation model we were not able to find a better combination that could significantly improve the existing Bigram combination. For the rest of the models, the idea was to find an advanced set of feature templates that could operate on data which have different levels of complexity and for a mid-level complexity sequence labelling model. For that, we relied on the corresponding set of Bigram feature templates and we tried to tweak it (see Appendix A.2.5) using two different dictionaries, a modern English born-digital dictionary (see Section 7.2.3) and a legacy French digitised dictionary (see Section 7.2.3), and the Lexical Entry model, which we consider to be the component of our architecture that has the average complexity level of parsing (i.e. in terms of granularity). For all features focusing on information about neighbouring tokens, we enlarged the window by 3 in both directions to provide longer range clues for predicting the class of a token. For instance, the window of templates focusing on the token and its neighbours goes from 4 to 7 and templates pointing to the typography of tokens in the sequence go from 1 to 4 neighbouring words.

For Unigram templates, the total T of generated distinct features is L × N, where L is the number of labels and N is the number of unique features generated by the templates. For Bigram and Engineered templates, T equals L × L × N. As a result of feature engineering, a larger L in the case of Bigram templates and a larger N in the case of Engineered templates produce a larger number of distinct features T. It is worth pointing out that the cost of a growing T is a slower training process and heavier resulting models which could slow similarly heavy models when they are called in cascade.

The impact of the presented set of template combinations on the labeling performance of the models will be exhaustively presented in Chapter 7.

Model Activation & Call

To use the models of the architecture, two stages need to be followed through the functionalities available in the two facets that the system represents.

Models Activation: MATTER Workflow

The activation of the architecture presented is enabled by following the MAT-TER methodology (Model-Annotate-Train-Test-Evaluate-Revise, see Figure 4 Model: define a CRF model for predicting different text structures at one stage and determine the corresponding feature set. This phase requires a programmer to create the defined models and integrate them into the cascading architecture.

Annotate: assign a TEI tag to each text block representing a lexical entity defined within a model's scope. This task must be performed on an XML representation of the data and must be strictly synchronised with the corresponding feature matrix file. The annotation guidelines 9 need to be respected.

Train: use each annotated batch of data to train a corresponding model. The cascading architecture of the models should be respected here.

Test: this step gives just a rough idea about how the trained model behaves on unseen data. There are many ways to accomplish this goal. The easiest one is to run the corresponding web service from the web application on a held-out sample.

Evaluate: a precise evaluation with different measures is possible at the end of the training process as long as annotated data are provided under the dedicated location in the dataset.

Revise: the last stage concerns reviewing the modelling and annotation steps that have been described in the guidelines. Four possible measures are the outcome of this step:

• annotate more data when an improvement in the results was achieved,

• refine the annotation guidelines for new variations observed in the last training batch

• proof-read the performed annotations when minor anomalies are noticed

• think about redefining the modelling when the results represent unexplainable anomalies. This could be translated either into a simple feature engineering process or into a change of the logic behind and the scope of the models or their architecture.

Models Call: Cascading REST Services

After activation/training of the selected parsing models, these are called through REST services of the system's web application facet 10 . After calling the macro levels parsing service, the micro level parsing is organised according the required depth of the analysis of the lexical information.

After giving an input dictionary file, calling the Parse dictionary service triggers the Dictionary Segmentation model and the parsing result is then displayed in the web navigator. If the result is good enough, calling the second service on the list is then meaningful and the second model in the architecture is launched to parse the extracted body. Upon having good segmentation results, the Lexical Entry model can be called through the third service on the list. The final service on the list, Parse full dictionary, triggers the customisable orchestration process of the rest of models. At this stage, the users have full control over calling the models they need for the parsing depth they want. Figure 4.8 shows a case of the cascading model activation process, where the users have the choice of calling one or several models for parsing the constructs of detected by the Lexical Entry model. They can then combine the output of the models they choose to get the best possible parsing result. More screenshots of the web application are provided in Appendix B.

It is worth pointing out that after each step at any parsing level, the users have the option to export the output they find the best as a TEI valid document. They can then continue the encoding of the structures in the exported document.

Lexicographic Knowledge Acquisition

Throughout the development of our approach and the implementation of the models in GROBID-Dictionaries, we faced several challenges that impacted on our priorities and pushed us to carry out more experiments in order to identify the nature of the issues and think about possible ways of overcoming them.

Extensive lexicographic expertise was required to define generic cascading models that satisfy the maximum of the lexicographic needs at each segmentation level. Making sure that the lexical models are applicable to different categories of dictionaries and languages that we do not master, was also a priority for us after encountering several modelling challenges. Therefore, we thought of collecting early feedback from domain experts through three measures. First, easing the setup of the system to attract more users, in particular those who have no IT knowledge or support to install and operate relatively complex tools in different running environments. Second, relaxing the complexity of the annotation workflow with more user-assistance and steps controls. Finally, offer collective training sessions to test the enhanced setup and collect feedback directly from the end users.

Easing Setup

To explain the improvement targeting the setup of the tool by a user with little IT skills, one needs to have an idea about the initial configuration of the tool.

Initial Configuration: IT Expert Use

GROBID-Dictionaries depends on core utilities and libraries provided by GROBID. The installation of the system must be preceded by the installation and setup of the parent project. Therefore GROBID-Dictionaries needs to be cloned as an extension module within GROBID's project structure and must be built after its parent project.

Due to differences in technical preferences of the project leaders, two different automation build technologies need to be used to build each project:

Gradle11 for GROBID and Maven12 for GROBID-Dictionaries. Successful builds of the system are packaged as Java libraries in two formats:

• a JAR (Java ARchive): this file is required for all processing stages which precede the training of each model, and

• a WAR (Web Application Resource or Web application ARchive): in the case of GROBID-Dictionaries this is not only a standalone web application but also a self-contained one that can be run after the training of the CRF models. It provides a graphical user interface to the existing web services, each corresponding to one or more of the cascading classification models.

GROBID-Dictionaries has been developed, tested and documented for the Linux and Mac operating systems. The behaviour of the resulting libraries is expected to be the same when run on other operating systems. However, there is no explicit guarantee for such uniform behaviour.

Enhanced Usability & Unified Execution Environment: DH Use

As a first measure, we investigated different ways to streamline the setup process and to guarantee a unique behaviour of the system across different execution environments.

One possible solution would have been to use a system image runnable on a virtual machine. Such an image should have a Linux-based operating system, a Java development kit (JDK) and the different automated build systems installed. GROBID and GROBID-Dictionaries should also already be cloned and built correctly. This type of solution suffers from two main issues. Firstly, the size of the image would be huge as it would include several unnecessary tools and system files that are still part of the operating system. Secondly, the static nature of such an image would make it complicated to update after a new version of GROBID-Dictionaries has been released. Updates to GROBID-Dictionaries are published frequently since the tool is under continuous development. However, a system image containing the above-mentioned components can be built in a more efficient way using a different technique. Docker13 is a state of the art software technology which is also based on the virtualisation of the execution environment. In contrast to the static image approach sketched out initially, Docker allows for the flexible composition of an image. An image is shaped by instructions written in a Docker file 14 . These instructions ensure that only the required components are included in the image. Moreover, several alternatives are available to efficiently update a build within an image starting from pushing a newly created image to the online Docker Hub repository 15 To run a Docker image of GROBID-Dictionaries (see Figure 4.9), a user needs to install the version of the Docker software corresponding to the user's operating system and pull the latest image of the tool from Docker Hub. The pulled image (orange box) will not be run directly on top of the operating system of the host machine but rather inside a Docker controlled container (yellow box). Thus testing the tool on Docker is enough to guarantee a unified behaviour, regardless of the particular system configuration of a user's computer environment.

It is also possible to synchronise files on the host machine with a running image in the Docker container. This feature allows the tool hosted inside a Docker container to directly interact with files stored on the host machine. We took advantage of this alternative to make the dataset directory shared between the two environments. With this mechanism, the user can exploit the full functionality of the tool living in the Docker image to train the machine learning models on the data residing locally on the user's machine.

In addition, thanks to the self-contained nature of the tool's web application coupled with its fluid setup and manipulation through the Docker image, using the GROBID-Dictionaries image enables both the desktop and web based functionality to be run on the user's local machine. Such a feature represents an asset for researchers who are concerned about the security of their data and experiments.

Lightening Annotation

The second major category of improvements specifically targets the annotation workflow. Annotating data for the training process involves challenging manual work and requires precautionary measures to ensure data integrity and validity.

Creating Training Data

To train a model in GROBID-Dictionaries based on a PDF file containing the raw text and the typographical features of a lexical resource, two additional files are necessary: a TEI document containing the corresponding reference encoding and a feature file describing textual and typographical information of each printed line or token.

To generate the training files, embedded functionalities of the tool should be used following one of the two following options:

• pre-annotated training data: this used to be the default mode for automatically creating training data, inherited directly from GROBID's core functionality. This mode is useful when a model was trained on a substantial amount of data. The task of the annotator is then to correct the automatically placed TEI tags by moving, adding or removing them.

• raw training data: this constitutes new functionality we have implemented to shortcut the checkout and cleaning of the tags automatically generated by using the default mode. The idea is simply to create training data without pre-annotations. Despite being obvious, starting to annotate a document from scratch was not possible before integrating this new feature. Such a mode breaks with the old practice of correcting the predictions made by a model trained on different samples, to make it possible to start annotating totally fresh data. Besides giving more choices to the annotator, such a mode saves time and effort, especially if an old model was trained with multiple TEI elements.

A legitimate question remains as yet unanswered: how can a user generate training data based on a selection of specific pages from the possibly hundreds of pages a dictionary may comprise?

After annotating different lexical samples in PDF format, we could qualify splitting an existing document into separate pages, or sequences of pages, as a very critical step. With some supposedly dedicated PDF manipulation tools producing damaged pages, we found only one tool reliably useful for the purpose of separating PDF pages 16 which seems to produce a quality split as good as the original document. Using workaround solutions for this purpose, such as the print-to-file functionality in web browsers, is also not recommended.

Training Data Annotation

As previously stated, GROBID-Dictionaries generates a preprocessed XML representation from PDF files containing the raw text of a lexical resource. To create training data for the tool, the user is then required to introduce semantic mark-up for the different models. Typically, an XML aware editor should be used to perform this task. Some advanced editors such as oXygen 17 We aimed to take advantage of the visual feature to avoid performing inline annotation directly on the text of the XML elements. This is catered for by a new feature in GROBID-Dictionaries that for each model now provides both a schema description (in Relax NG) 18 and a presentational stylesheet (in CSS). The schema description enables the editing software to check or even enforce schema compliance of the training data. The stylesheet can be exploited by the editing software to allow users to mark up the training data semantically by highlighting portions of the text and then enclosing the highlighted portion with a suitable XML tag. The colours attributed to each element can be customised by a simple modification in the stylesheet.

Training Domain Experts and Collecting Feedback

To test and adjust the two previous categories of enhanced usability measures, we chose to collect feedback from domain experts while and after manipulating and getting familiar with the GROBID-Dictionaries ecosystem.

Hands-on Sessions with End-Users

Besides the motivation, we had the opportunity to expose the under-development infrastructure, during different periods of its implementation process, to real users. These user experiences were organised as workshop series in the context of Masterclasses for lexicographic data 19,20 , inter-disciplinary consortium gathering21 , and dedicated events for promoting the new system 22,23,24 .

The goal of the workshop was to familiarise the participants with the MATTER workflow as implemented in GROBID-Dictionaries (see Section 4.3.3), while excluding the first modelling step which requires programming skills. Note that none of the participants was familiar with the tool prior to the tutorial.

The average duration of a workshop is 5 hours. After a short introduction to the architecture of the system, the users were guided through the process of installing and running the docker image 25 . Once the docker image was running, the participants were then able to reproduce the results reported in [START_REF] Khemakhem | Automatic extraction of TEI structures in digitized lexical resources using conditional random fields[END_REF] which are based on a modern English monolingual dictionary (Publishing, 2009). As the next step, several users used the possibility to experiment with their own lexical samples by repeating the workflow they had learnt and crafting new models for their individual datasets 26 .

Participants with Diverse Backgrounds

The tutorial groups consisted of users with various profiles, such as lexicographers, linguists, computational linguists, computer scientists, philologists and translators. Most of the participants had not previously trained machine learning tools.

After the very first workshop, we asked the participants of our tutorial to respond to a questionnaire created as a Google Form27 . The questionnaire gives an overview of the typical profiles and user experience, which we tried to improve after collected feedback. Although it was our first tutoring experience and the usability improvements were still freshly implemented, all the participants reported being confident that they were able to re-apply what they had learnt on other lexical resources.

The workshop series was held 8 times with pending ones to be organised after writing this thesis. The workshop series allowed us to train and collect feedback from over 100 users from different academic and industrial institutions, who also succeeded in tutoring new users in their institutions. Over 10 users are actively using the system and providing us with feedback and samples.

Various tested Material

The lexical resources brought to the tutorials were considerably varied. They included different types of dictionaries (some digitised, some born-digital with no explicit semantic markup) such as general monolingual, bilingual and etymological dictionaries as well as a dictionary from a language documentation field project (see Table 4.1). The tested samples confirmed the generic nature of our models and the fact they are language agnostic, as long as the samples are from alphabet-based languages.

Some separate experiments with arabic samples were not successful given a problem originating from the library responsible for extracting the text from the original document. We assume that the problem would be the same for other languages sharing the same writing system (from the right to the left).

Outcome & Gathered insights

Having motivated inter-disciplinary experts participating in the tutorial as well as testing the tool on new lexical samples provided us with the opportunity to spot some issues and several possible improvements.

We were able to fix some of the minor triggered implementation issues in the course of the tutorials. Other issues have been filed as new tickets on GitHub, e. g. issues concerning the treatment of lexical entries that stretch over more than two pages in print. Some technical issues related to the GRO-BID core still need to be resolved such as support for some classes of special characters which are wrongly encoded in the pre-processing of the raw input text. The annotation guidelines were also further refined to provide clearer definitions of constructs to be annotated. But we made sure that the changes are not great to avoid confusing the trained users after the tutorials.

Lexicographers and linguists among the participants who gave us useful domain expertise comments, were many of which were taken up and resulted in implementing new labels for our models or even modifying the models behaviour. The limits of our modelling were exposed when the system had the challenge of parsing new samples according to different lexicographic practices.

Such a user experience helped us not only to verify our assumptions and improve our implemented approach but also to provide us with data we can use for our advanced experiments (see Chapter 7) and initiate cooperation that allowed us to widen the scope of our lexical models (see Section 7.6.1).

Chapter Summary

This chapter provided an in-depth presentation of our approach and the different solutions we found for the theoretical and practical challenges that we encountered.

We have presented GROBID-Dictionaries, a double-faceted ecosystem for parsing print dictionaries, which allows users with no advanced IT skills to train and customise the use of an architecture of machine learning models. We have also sketched out possible ways of leveraging the training of CRF models from an engineering perspective. This aspect will be showcased in Chapter 7 where we will present in detail the performances of the different models, given different combinations of features and varied tested samples.

Solutions to overcome the lack of lexicographic knowledge were also explained through practical and empirical measures. Our investigations in this direction may not be qualified as a user study but we managed to bridge the gap between engineers and domain experts by foreseeing such an obstacle. The positive results of such an early measure and the common issues discovered in related fields in humanities encouraged us to dig in new directions and open up new perspectives for our endeavours. We will further develop this aspect of our work in Section 7.6.1.

We have also highlighted some decisions regarding the encoding of lexical structures with respect to the implemented cascading mechanism. In Chapter 6 we will provide more details about the challenges of using a standard to encode dictionaric resources and coming up with generic schemes that ease exchange and automatic processing. But before we dive into new standardisation schemes for lexical resources, the following chapter gives an overview of the state of the art of the standards for such resources and the obstacles facing the community regarding scalability and exchange.

Chapter 5

Standards for Structured Lexical Resources

Introduction

The dictionary parsing architecture we are presenting in this thesis aims not only at generating structured lexica for NLP downstream tasks, but also at allowing high exchange and interoperability with existing resources and querying tools. Thus, awareness and compliance with the existing practices and standards for structured lexical resources is required for the purpose of scalability.

Our study of the literature has shown a dominance of two standards, namely Text Encoding Initiative (TEI), Lexical Markup Framework (LMF) and one standardisation initiative, OntoLex-Lemon, for modelling lexical resources. Each of these modelling frameworks has been initiated to satisfy specific needs that represent several overlappings.

In this chapter we present the history and the motivations behind the foundation of these standards. We then provide a comparative study 1 of all of them with a focus on the strengths of each framework, and how it could fit the requirements of te structured output we wish to deliver.

TEI

TEI is a well-established standard that has proved popular within the lexicographic community. In this section, we give an overview of the standard from an engineering and lexicography newbie perspective.

Background

TEI [START_REF] Sperberg-Mcqueen | Guidelines for electronic text encoding and interchange[END_REF]) is a standardisation initiative that was launched in the late 80's with a view to finding a common framework for encoding text documents. The initiative has become a de facto standard and has been widely adopted in major humanities and documentation fields for academic, governmental and industrial projects 2 . 1 Our survey is based on the state and the guidelines of these standards at the time when the work on this thesis started, specifically, September 2016 2 https://tei-c.org/activities/projects/

In fact, the TEI guidelines give flexible and effective alternatives to structure texts for various purposes. They offer a strong document representation framework, with over 600 XML elements covering the needs to model almost any text. Its flexibility also lies in its specification language, ODD -One Document Does it all, which makes it straightforward to adapt the existing guidelines for close structuring requirements and supports the creation of new extensions for totally new needs.

The TEI guidelines are actively maintained and the P5 revision [START_REF] Budin | Creating Lexical Resources in TEI P5. A Schema for Multi-purpose Digital Dictionaries[END_REF] devotes an extended chapter to encoding lexical resources of different kinds. The impact of this initiative is clear in the lexicography domain, where it has been adopted as a main encoding format for large scale projects and resources in the field. Such success comes from the fact that a varied community of contributors is involved in shaping the standard and the reviewing process is pretty flexible and quicker than those for de jure standards.

Modelling Perspectives

The TEI guidelines provide a formal modelling of text in documents through a set of categories gathering related XML elements, called modules. The P5 version of the standard comprises 21 modules3 for marking up almost any piece of text, where the ninth is dedicated to encoding dictionaries.

Encoding Level

The TEI guidelines allow two levels of text encoding in print documents, where often one or the other is targeted by a user of the standard. The first level aims at reflecting the physical structure of a document by using elements from the "core module". Such TEI elements can be used to encode paragraph and line beginnings, highlighted words, etc. Some elements can also be typed in to provide more precision on how they are typographically presented in the original print document. Figure 5.1 (C) illustrates the use of elements from the core module (i.e. <p>4 , <lb>5 , and <hi>6 ) to encode respectively a paragraph beginning, a line beginning and highlighted text segments in the lexical entry. Note the typed <hi> elements to markup the italic -i -and bold -b -in the text. Such a use is very common in the documentation and archive projects where the goal is to preserve every detail of the physical aspects of the text material.

The second level of encoding enabled by the TEI guidelines deals with the semantic and logical function of text structures. For lexica encoding, the "dictionaries module" provides a lexicon designer with an exhaustive set of TEI elements modelling different linguistic levels of the lexical information.

TEI

61 Such an encoding is also influenced by the linear description of print dictionaries, where common practices involve respecting the order of fields as they appear in the original document. Figure 5.1 (B) depicts the use of the elements of the specific module and shows the difference between this and the previous encoding level. Such a use is more popular within the lexicographic community and projects.

Encoding Workflow & Choices

Each module is described in the guidelines in a chapter that contains a formal description of the use of its elements. The use of TEI elements of a certain module does not exclude those from other modules.

A newbie willing to follow these guidelines needs to verify the adequacy of an element he/she chooses for the piece of information he/she wants to model by:

• reading the prose description provided in the guidelines for the chosen element (e.g. <entry>7 )

• exploring the different instances of the use of the element in different contexts of the text to encode, which are often provided in more than one language 62

Chapter 5. Standards for Structured Lexical Resources

• verifying the validity to use a certain element as a child of one of its possible containers, indicated in the section "contained by"

• anticipating the modelling of child components by checking out the allowed elements for lower level, specified in the section "may contain"

In addition, a lexicon designer can stop at any depth of encoding when he/she considers that the necessary needs have been met. Figure 5.2 illustrates two valid modellings with different encoding elements and depths for the same dictionary article. Note the use of <entry> and <entryFree> 8 to encode the same macrostructure. <entryFree> is more favoured when the encoded entry has no clearly structured description. Moreover, using <entryFree> can substitute, to a large extent, the use of <entry>, and even gives more options to encode child components not allowed within <entry> (e.g <orth> 9 for encoding orthography or <superEntry> 10 for certain cases of nesting entries). The modelling of orthography is also different in the two TEI samples, where the encoding depth varies. In fact, the use of <entry> forces the use of <form> to enable the encoding of orthography with <orth> element, where the same information can be encoded by directly using the <orth> element within <entryFree>. The granularity of the <etym> 11 block is also different in the two encoding examples. The designer of the lexicon on the left seems to be interested only in differentiating the lemma and the etymological information, while in the example on the right, more focus is given to identifying the micro-structure of the <etym> block.

TEI-based Lexical Resources

It is hard to enumerate all the existing lexical resources encoded in TEI, given its previously mentioned popularity. Nevertheless, we can differentiate between TEI resources that describe the physical structure and those which focus on the logical constructs.

For the first category, we can cite the work12 of the Berlin-Brandenburg Academy of Sciences for digitising and transcribing legacy dictionaries such as Mueller, 1878; [START_REF] Uhlenbeck | Kurzgefaßtes etymologisches Wörterbuch der gotischen Sprache[END_REF][START_REF] Goedel | Etymologisches Wörterbuch der deutschen Seemannssprache[END_REF][START_REF] Kluge | English Etymology. A Select Glossary Serving as an Introduction to the History of the English Language[END_REF] For the second, we can cite EDBL [START_REF] Aduriz | EDBL: a Multi-Purposed Lexical Support for the Treatment of Basque[END_REF], MULTEXT-East (Erjavec, 2004) or the work by [START_REF] Declerck | Accessing and standardizing Wiktionary lexical entries for the translation of labels in Cultural Heritage taxonomies[END_REF] to convert Wiktionary data 13 into a TEI compliant resource.

It is also worth mentioning that some OCR systems, such as Transkribus [START_REF] Kahle | Transkribus-a service platform for transcription, recognition and retrieval of historical documents[END_REF], offer the option to export the OCR'd documents in TEI format. Such an output carries only markups for the physical description of an input print dictionary and can be referred to as "TEI resource" or "TEI version" of the original dictionary.

Discussion

Given the flexibility to choose from different encoding options for the same piece of information, lexica encoded in TEI can have different schemes. This freedom coupled with the wide adoption among lexicographers, having their own backgrounds and visions of the logical structure of a dictionary, has yielded a big bang of lexicon schemes and practices. Ironically, a standard that was supposed to unify the encoding formats under the umbrella of a common framework has turned into an uncontrolled modelling space.

TEI schemes are influenced by the documentation aspect of the standard given the fact that the linear aspect of the encoded constructs highly impacts the possible modelling options. Such a factor constrains the choices for a lexicographer and results in a non-optimal scheme dictated by all the possible orders of the lexical information in a sample, even the inconsistent ones. Such a fact adds a layer of complexity to the diversity of practices and schemes. Consequently, interoperability and exchange between resources compliant to the same standard have been significantly reduced.

LMF (2008)

LMF is a de jure standard published by the International Organization for Standardisation (ISO) for modelling lexical databases and MRDs. It was launched as ISO 24613 and is maintained by the standardisation sub-committee ISO-TC 37/SC 414 , and more specifically Working Group (WG) 4. In this section, we present a brief history of this standard as well as a theoretical and technical survey concerning its usage.

Background

LMF was initiated a bit later than TEI, as a followup initiative to multiple international projects such as ACQUILEX [START_REF] Copestake | The ACQUILEX LKB: representation issues in semiautomatic acquisition of large lexicons[END_REF], EAGLES/ISLE [START_REF] Calzolari | Towards a standard for a multilingual lexical entry: The EAGLES/ISLE initiative[END_REF] and MILE [START_REF] Bertagna | Content Interoperability of Lexical Resources: Open Issues and" MILE" Perspectives[END_REF]. The most stable and well-known version of LMF, published by ISO in 2008 [START_REF] Francopoulo | Lexical Markup Framework (LMF)[END_REF] 15 , has offered a framework for modelling, publishing and sharing lexical resources with a special focus on requirements arising from the NLP domain.

The LMF workflow follows the standardisation practices within ISO where proposals and decisions are discussed and made within a high-level committee (i.e. ISO-TC 37/SC 4/WG 4) of experts and then validated/modified through an international balloting process. Compared to TEI, there is less transparency of the "in-between decisions" and less interaction with the future users. The use of the LMF standard is also charged, unlike TEI which is free. Nevertheless, it has gained a considerable number of users from academia and governmental institutions who have adopted the standard to take advantage of the combination of lexicographic and engineering expertise for the purpose of building universal and sustainable lexical databases.

Meta-model

LMF has proposed a meta-model for designing lexical resources formalised mainly in Unified Modelling Language (UML) [START_REF] Fowler | UML distilled: a brief guide to the standard object modeling language[END_REF]. The meta-model is composed of a core component and pluggable interlinked extensions defining an abstract model for different linguistic levels of the lexical information (see Figure 5.3). Examples of such extensions are Machine Readable Dictionaries, Morphology, Semantic and Syntax extensions. The meta-model is linked to data category registry (ISOcat -ISO 12620) 16 that represents the elementary linguistic properties of the model components.

The meta-model introduces a number of interlinked classes and relationships, which are categorised into one of the aforementioned extensions. An instance of the diagram can be enabled by the instantiation of these classes and relationships and respecting their specified multiplicities. Figure 5.4 depicts an instance of the LMF meta-model.

Lexical Entry is the key class of any LMF meta-model and it holds the backbone of the lexical description. Morphological information and semantic information are respectively presented by means of Form and Sense classes and their sub-classes. List Of Components and Components classes, belonging to several overlapping extensions, represent the core modelling mechanism 15 the academic publication about the standard was a mature draft of the main aspects of LMF that were accepted within the ISO-TC 37/SC 4/WG 4 and later went through the ISO validation and publication processes to be published later on in 2008 16 ISOcat is no longer an ISO project after the decision to make such a project open to the user community. DatCatInfo took up the mission of representing such categories. It is maintained by LTAC Global / TerminOrgs. Further details can be found in: http://www. datcatinfo.net/#/history

Serialisation

LMF recommends an ad-hoc serialisation that represents a direct mapping between the class names in the meta-model and the XML element names or attributes. This serialisation figures in the standard as an Annex to the meta-model and through some scattered possible serialisation examples. But there is no comprehensive guide for serialising each element, defined in the standard's manual. Note that the serialisation can be carried out in any other format, since the XML option remains the recommended alternative but is not exclusive.

The standard is more NLP-oriented, which means that it gives almost no importance to the linear structures of lexical information in print dictionaries. The physical structure is also not considered as an aspect to be represented with the LMF meta-model. These facts give an idea about the scope of the resulting serialisation, in particular in comparison to a standard like TEI.

LMF-based Lexical Resources

A number of projects have adopted LMF as the main modelling framework to build lexical databases in different formats. The most known publicly available project resources is probably UBY [START_REF] Gurevych | Uby: A large-scale unified lexical-semantic resource based on LMF[END_REF] which UBY has been natively implemented as an SQL Database but the UBY API enables the export of lexica as an XML serialised resource. Several downstream applications have been using UBY to carry out different tasks such as semantic annotation [START_REF] Miller | Sense-annotating a Lexical Substitution Data Set with Ubyline[END_REF], machine translation [START_REF] Beinborn | Cognate production using character-based machine translation[END_REF], personality profiling [START_REF] Flekova | Personality profiling of fictional characters using sense-level links between lexical resources[END_REF], and many others. The high coverage and the usability alternatives of a resource like UBY are to a great extent enabled thanks to the LMF metamodel behind.

El Madar [START_REF] Khemakhem | ISO standard modeling of a large Arabic dictionary[END_REF], a lexicon for the Arabic language modelling of MRDs, is another LMF-based resource that shows the metamodel's lexical coverage and extendibility. In fact, [START_REF] Khemakhem | ISO standard modeling of a large Arabic dictionary[END_REF] managed to model over 37,000 lexical entries using the existing extensions and building on them new components, extending the existing meta-model. Using LMF to model dictionary entries in a highly inflectional and syntactically rich language like Arabic proves the genericity and the flexibility of the meta-model.

The list of resources instantiating the standard goes beyond these two resources and includes lexica in several languages or pairs of languages [START_REF] Maks | Standardising Bilingual Lexical Resources According to the Lexicon Markup Framework[END_REF][START_REF] Mykowiecka | Building an electronic dictionary of old polish on the base of the paper resource[END_REF].

Discussion

LMF managed to overcome several obstacles towards a uniform representation of lexical resources for the NLP usages. The standard left, however, different gaps that restrained its wide adoption and becoming the de facto framework for modelling lexica. On the form level, the standard has an imbalanced structure with just 20 pages for the main content and over 60 pages for normative and informative annexes. In addition, only a few modelling examples come with a recommended serialisation. Consequently, the balance and the analogy of the content, required for an intuitive understanding are highly questionable for the case of this standard.

On the modelling level, the potential richness and the multi-layered nature of linguistic descriptions in lexical resources has resulted in the LMF meta-model taking on a great deal of complexity in its attempt to reflect these various different linguistic facets. Complex relationships between classes (e.g. useless Component class in Figure 5.4) and redundant mapping mechanisms (different mechanisms for the same abstract phenomenon e.g. synonymy, MWE, etc) were the side effects of the modelled lexical complexity. The latter has been propagated to the ad-hoc serialisation, which made the querying and the enrichment of the resulting resources challenging. Finally, key areas of linguistics such as etymology (and diachronic lexical information in general) were not covered by the meta-model which sanctions a whole Chapter 5. Standards for Structured Lexical Resources category of valuable resources that are supposedly within the scope of such a standard.

OntoLex-Lemon

OntoLex-Lemon is an evolving initiative for the representation of lexical data on the Semantic Web. In this section we present the history of its development, the theoretical and technical foundations, and the limitations of the resulting modelling.

Background

The rise of the Linked Open Data movement, mirrored on the linguistics field as Linguistic Linked Open Data 17 , has created the need to represent lexical data as an ontology for the semantic web usage. LExicon Model for ONtologies (Lemon) [START_REF] Mccrae | Linking lexical resources and ontologies on the semantic web with lemon[END_REF] was the first attempt to find a dedicated model. After the foundation of the OntoLex Community18 at the end of 2011, the group took on the improvement of Lemon and its upgrade to create the so called OntoLex-Lemon [START_REF] Mccrae | The OntoLex-Lemon Model: Development and Applications[END_REF]. W3C, however, frames its role into a host of the discussions of the group and remains distant from the views and decisions of the latter 19Since its creation, the OntoLex group has focused on more structuring of the model and the collection of several use cases for the purpose of widening the coverage of the Lemon model. In parallel, the proliferation of the tools that facilitate querying, linking and visualising of such resources has been attractive for many projects. The transparency of the workflow has to some extent helped the standard to strengthen interest in its use within the lexicographic community, aiming at representing their resources on the Semantic Web.

Modelling

The most recent version of OntoLex-Lemon, published in the official report of the OntoLex Community Group, defines a model structured into four main modules, besides a core model. The latter represents the description of a lexical entry as a constellation of concepts (see Figure 5.5) such as Lexical Concept and Concept Set, and main components, like Form, Lexical Sense and Affix. The other modules carry the representation of Syntactic and Semantics, the Decomposition of multi-word lexical entries, Variation and Translation aspects, and Linguistic Metadata. OntoLex-Lemon's modelling remains faithful to the semantic web formalism which is based on representing concepts and their relationships using Resource Description Framework (RDF) triplets: subject, predicate and object. The resulting model follows the principle of semantics by reference [START_REF] Mccrae | Interchanging lexical resources on the Semantic Web[END_REF], where the logical structure of a lexical entry is expressed by reference to an individual, class or property defined in the ontology. In some cases, the lexicon itself can reference object properties that belong to other ontological models such as LexInfo20 . In fact, the defined triple-based vocabulary specifies lexical concepts and possible internal links, forming the definition of a lexical entry, and external relationships that represent links among lexical entries. The modelling principles are heavily inspired by those implemented in other models, like LMF's.

In Figure 5.6, we show an instance of the OntoLex-Lemon model for representing the same compound (i.e MWE) "dead centre", already modelled using LMF and depicted in Figure 5.4. We can notice the LMF inspiration in modelling the components of the MWE and in the representation of Sense and Form information. The modelling diagram is however ad-hoc, despite a great resemblance to UML schematisation.

The triple-based vocabulary is manifested in each concept, as for instance: centre_n is a Lexical Entry, and between two concepts, such as: centre_n is a noun.

Note the complex modelling of the geographical variant forms of the canonical form (i.e. lemma) in centre_n_form concept, where the written and phonetic representation have composite values. In addition to being less intuitive, in comparison to other modelling frameworks like LMF, such a representation makes it more complicated for automatic systems to enrich or query atomic values for each attribute of a property. We noticed that the official report of OntoLex-Lemon does not provide guidelines for modelling other categories of lexical information like deeper sense relations or Etymology. But defining customised extensions for supporting information, having such granularity and category, is also possible.

And as long as the design in the OntoLex-Lemon report and the triplet modelling principles are respected, a new extension can be qualified as "OntoLex-Lemon compliant".

Related work has succeeded in modelling such advanced extensions like polyLemon [START_REF] Khan | The Challenges of Converting Legacy Lexical Resources to Linked Open Data using Ontolex-Lemon: The Case of the Intermediate Liddell-Scott Lexicon[END_REF] for deeper sense information, and lemonEtym [START_REF] Khan | Towards the Representation of Etymological Data on the Semantic Web[END_REF] for etymological description. The ongoing revisions of the model are also focused on its enrichment by improving existing extensions as for morphology21 , or defining new ones as for frequency 22 .

Serialisation

The serialisation of OntoLex-Lemon relies mainly on the syntax and vocabulary of RDF [START_REF] Klyne | Resource description framework (RDF): Concepts and abstract syntax[END_REF]) and RDFS [START_REF] Mcbride | The resource description framework (RDF) and its vocabulary description language RDFS[END_REF]. Another Semantic Web framework, OWL [START_REF] Mcguinness | OWL web ontology language overview[END_REF], is also used within OntoLex-Lemon's serialisation to express more complex relationships along with constraints on data values, such as cardinality and data range.

The defined serialisation can be created in different formats, such as RDF/XML and JSON, while the Ontolex documents uses Turtle23 as the main language to serialise the examples of modelling diagrams. The choice of a certain serialisation format does not exclude a straightforward migration to the others. In fact, through the existing translation tools 24,25 for semantic web technologies, it is possible to transform any RDF serialisation into other formats, like JSON and Turtle, upon the click of a button. Such a feature ensures more interoperability and exchange for the Ontlex-Lemon resources on the Semantic Web.

OntoLex-Lemon-based Resources

Theoretically, OntoLex-Lemon allows the modelling of unstructured lexical data in print dictionaries. In practice, the ontological model has been mostly and its Components used as a scheme to transform already structured lexical data into lexical databases. To our knowledge, only a very small number of small sized lexica have used it as a native model/serialisation. From our newbie experience, our explanation for such a phenomenon is the need to break down everything in a lexical description into at least one triplet and often into more than one. Such a modelling principle is not very intuitive and not practical, especially when it comes to encoding from scratch complex lexical information that involves composition, rich atomic values (e.g. the phonetic and written representations in Figure 5.8) or structure nesting. A number of transformation experiments have been successfully carried out with TEI and LMF-based databases among other native formats. The best known examples of such a conversion are probably the resources resulting from the transformation of NLP lexica, widely used in field, like UBY [START_REF] Gurevych | Uby: A large-scale unified lexical-semantic resource based on LMF[END_REF] transformed into UBY-Lemon (Eckle-Kohler, Mc-Crae, and Chiarcos, 2015), and BabelNet (Navigli and Ponzetto, 2012) converted into lemon-babelnet [START_REF] Ehrmann | A multilingual semantic network as linked data: lemon-babelnet[END_REF], or the TEI version of Liddell-Scott [START_REF] Liddell | An intermediate greek-english lexicon[END_REF] compiled by the Perseus project [START_REF] Magazine | The Perseus Project and beyond[END_REF] which has been transformed into an OntoLex-Lemon lexicon .

Discussion

OntoLex-Lemon managed to open new perspectives for lexical data by offering a framework for its representation on the Semantic Web and consequently overcoming ad-hoc serialisation and format issues.

Nevertheless, this initiative represents several limitations for the use of its scheme, as a native format for modelling lexical structures and their relationships. Firstly, this is due to the relative complexity of its triplet-based vocabulary, compared to standards like TEI and LMF. The use cases, based mostly on conversion scenarios, support this claim. Secondly, the different possibilities to design the same lexical concept, such as the case of MWEs (see Figures 5.6 and 5.7), represent an obstacle towards reaching a unified representation of the lexical information which results in reducing exchange and comparability alternatives. Moreover, the standard is under active revision and great progress has been already achieved to support the modelling of more granular and new classes of information. However, it remains insufficiently mature to cover the modelling requirements in print dictionaries.

Chapter Summary

This chapter gave an overview of the state of the art of the leading standards and initiatives for modelling structured lexical resources and we highlighted their strengths and shortcomings.

The TEI guidelines, being the most adopted within the community, have provided lexicon designers with multiple mechanisms to model similar lexical structures according to different perspectives and practices. The flexibility offered, however, needs to be restrained by finding more compromising encoding schemes/practices to ensure a maximum interoperability of resulting resources. Our aim is to find a modelling that converges towards a unified TEI scheme for our parsers.

LMF has more control on the encoding practices and provides a stronger formalism for modelling the lexical information, making the approach of the standard a candidate framework for defining the target output we want to generate from the parsed dictionaric material. However its complexity, lack of coverage and NLP orientation need to be relaxed in order to support the specificity of our target dictionaries.

Ontolex-Lemon has great potential for enabling high interoperability and exchange allowed by the Semantic Web technologies. But the lack of coverage and the differences in representing the same information are not in favour of directly adopting it for modelling the lexical output of our parsers. Nevertheless, the active revisions and the previous conversion experiments leave the door open for the use OntoLex-Lemon to represent the output of our models on the Semantic Web, by using TEI or LMF serialisation as a pivot format. Such a choice has been already adopted in ELEXIS26 , a major European project for electronic lexicography that shares with us the same parsing and modelling goals, which confirms the limitations of the ontological model.

In Chapter 6, we present the endeavours targeting the improvement of our two candidate standards TEI and LMF. We show how these initiatives overlap with our work and how our models can generate a comparable output.

Chapter 6

Novel Standardised Schemes for Encoding Dictionaries

Introduction

The issues highlighted in Chapter 5 regarding the modelling aspects of two leading standards for lexical resources, TEI and LMF, have been already reported by the lexicography community and some concrete proposals have consequently been made to find better modelling frameworks with mutual improvements [START_REF] Romary | Using the TEI framework as a possible serialization for LMF. Rendering endangered languages lexicons interoperable through standards harmonization[END_REF][START_REF] Czaykowska-Higgins | Using TEI for an endangered language lexical resource: The Nxa?amxcín Database-Dictionary Project[END_REF][START_REF] Romary | Using the TEI framework as a possible serialization for LMF. Rendering endangered languages lexicons interoperable through standards harmonization[END_REF]. But these observations and suggestions lacked the context that moves towards gathering individual interests in such a direction under one umbrella.

The two standards have different approaches and revision workflows. Thus, combining the best of the two standards represents a considerable modelling challenge and requires an active involvement in both standardisation ecosystems. Fortunately, this thesis began when there was already consensus within European and International projects on the need to act and find better TEI and LMF. In addition, we had the opportunity to be closely involved in such revision endeavours, which helped us to be aware of standardisation issues and important modelling challenges of lexica that we took into consideration to shape the output of GROBID-Dictionaries.

In this chapter, we present two novel standardised schemes for encoding lexical resources: TEI-Lex-0 [START_REF] Romary | TEI Lex-0: A Target Format for TEI-Encoded Dictionaries and Lexical Resources[END_REF] and the recently revised version of LMF [START_REF] Romary | LMF Reloaded[END_REF]. We report on the modelling aspects brought by these new frameworks and the main challenges which impacted on some key encoding choices. We then show the overlap with the output of GROBID-Dictionaries and the main differences.

TEI-Lex-0

The definition of a baseline encoding that enables easier exchange and exploitation of TEI lexical resources was the key driving force behind the initiation of TEI-Lex-0. Such an encoding scheme in no way replaces the TEI dictionaries module. In fact, it has the goal of defining guidelines towards a more systematic use of TEI elements, based on observations of common practices followed to encode existing TEI-based lexical resources. The guidelines should support newly created TEI resources, as well as, the unequivocal transformation of existing TEI dictionaries into a unified baseline encoding.

In this section, we refer by minimal encoding to the representation of macro structures which does not present the exhaustive encoding of fine grained lexical information, such as <orth> within <form> or <def> within <sense> constructs.

Context

The preparations for the TEI-Lex-0 initiative were initially discussed within the "Retrodigitised Dictionaries" Working Group1 under the umbrella of the COST Action European Network of e-Lexicography (ENeL). The DARIAH Working Group "Lexical Resources"2 took up the work of establishing the TEI-Lex-0 guidelines and support from the H2020 funded project PARTHENOS3 , which has been accorded within its standardisation Work Package 4 [START_REF] Romary | Report on Standardization[END_REF]. The TEI-lex-0 has been chosen to be a pivot format for comparing, querying and visualising structured lexical resources. Upon the kick-off of ELEXIS4 and given high overlapping interests, partners have also been involved in the discussion around the expected interoperability requirements expected from TEI-Lex-0.

Technically, the main discussions are carried out through several face-toface meetings, organised every few months. Experts, mastering several languages and having different lexicographic backgrounds, bring and discuss different raw or already encoded lexical samples. Critical and open questions are publicly posted as github tickets5 to push the discussion forward and collect feedback from the community. Finalised decisions are then implemented in dedicated public guidelines 6 .

A number of simplification principles were obvious and easily got the green light from the majority of the experts around the table (e.g. main elements of a lexical entry, lemma encoding, sense made mandatory for each dictionary article). However, several recommendation decisions were not straightforward to reach, as compromising comes at a cost and each expert has his/her own legitimate vision and preferences for modelling TEI structures.

In the following section, we sketch out some challenging modelling choices to illustrate the obstacles encountered and how they were overcome.

Modelling challenges

The complexity of modelling logical structures in TEI dictionaries was dismantled based on the main macro structures of a lexical entry in a print dictionary. These structures are then studied individually based on their components as well as their belonging to higher level constructs (i.e., contained by and may contain sections in the description of each TEI element).

We present encoding use cases that resulted in major recommendations reached within TEI-Lex-0 and which were propagated to the TEI Dictionaries module itself. For the remainder of this thesis, we denote by element@attribute the attribute of an XML element.

Recursive entries

<entry> is a TEI element containing the description of a lexical entry in a dictionary. The TEI guidelines provide a lexicon designer with additional elements to encode specific categories of lexical entries and entry-like constructs like <entryFree> for less structured entries, <superEntry> for higher level entries (e.g. roots in Arabic dictionaries) and <re> 7 for related entries (e.g. collocations, idioms, etc).

To ease search in multiple resources and extract comparable structures, the TEI-Lex-0 guidelines recommend the use of only <entry> to encode all entry and entry-like constructs. The differentiation among these different constructs is specified by means of type values on the <entry> elements, where these values denote properties of the entry as a whole and not expressed by means of other elements within the encoding of the entry. Such a classification can be seen from different perspectives. Therefore, TEI-Lex-0 guidelines recommend several domain lists for such values.

The unified entry-like structure is vowed to be an autonomous and referenceable construct in a number of scenarios, such as internal mapping, indexing, inter-resources exchange or/and language disambiguation of lexica. Therefore, a recommendation, of making @id and @language attributes mandatory for each entry construct, reached the absolute agreement among the TEI-Lex-0 experts. Figure 6.1 shows an instance of such a scheme in a typical case of nested entry-like constructs in Arabic dictionaries, where dictionary articles are roots that gather all their morphologically derived forms.

In such an example, nesting typed <entry> is required to reflect the hierarchy within the lexical entries and avoid using different representations like <superEntry> or <re>. Such a decision triggered the need to allow such modelling within the TEI Dictionaries modules. Upon exhaustive argumentation in favour, the proposal was adopted and the <entry> model in the TEI guidelines was extended to allow recursive entries and <entry> has been made possible within <sense> construct as related entries might be related to a sense and not the whole entry8 . As can be noticed, the values of the <entry> containers provide another classification of the constructs encoded in the nested <entry>s (i.e entry@type="wordFamily" vs form@type="root"). It is also worth mentioning that each entry that is not typed has by default "main" value, usually meaning that it carries at least a lemma and other sense information.

Revising the entry model

The existing <entry> model, prior to the TEI-Lex-0 discussion, required further revision to unify the representation of its macro-structures. Besides the required nesting of <entry> and the deletion of <re>, the use of <hom> 9 for the representation of homographs was deprecated. Two recommendations for a more unified representation of such a phenomenon, by means of either nested <entry> or <sense> elements, were proposed as alternatives. Figure 6.2 shows an example of the first encoding proposal 10 of homographs that have different POS (print version of the entry is provided in Figure 3.3).

Such a representation replaces the use of <hom> elements to encode the entry as two constructs with two different POS (such as a verb and as a noun). The use of <entry> again makes it possible to have a unique representation that helps an automatic system to look for entry constructs that might be inconsistently, or just differently, modelled. Some reluctance was, however, manifested within the TEI-Lex-0 group to the adoption of such a modelling for some dictionaries and the use of <sense> 11 construct was suggested instead 12 . Besides respecting the original modelling in the print version, such a position can also be understood by the fact that some information 9 https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-hom.html 10 https://github.com/DARIAH-ERIC/lexicalresources/issues/48 11 https://tei-c.org/release/doc/tei-p5-doc/en/html/ref-sense.html 12 https://github.com/DARIAH-ERIC/lexicalresources/issues/14 related to the highest <entry> structure (e.g. etymology, usage, etc) would be falsely included inside the entry "actNoun". Excluding the <gramGrp> 13from that entry would leave an imbalanced representation with regard to entry "actVerb". The second encoding proposal, using <sense> element, is depicted in Figure 6.3. This encoding remains faithful to the modelling choices of the lexicographer of the original print dictionary, despite the inconsistency illustrated in Figure 6.3 and its analysis. But the visibility of the entry-like constructs encoded as sense will be sanctioned, as we have explained the advantage for the alternative encoding in Figure 6.2. To maintain analogy and allow automatic comparability between semantic structures, such an encoding requires wrapping all the first four senses of the entry "actNoun" in an additional <sense> block. We can notice in these cases, the influence of the linear aspect of the lexical data in print dictionaries and the dilemma of choosing one option between different highly supported proposals.

Written and Spoken Forms

The differences of opinion to represent written and spoken forms following the TEI-Lex-0 guidelines are relatively less important and a consensus has been reached for a unified encoding of morphological and a large part of grammatical information.

The representation of the lemma is recommended to be within a form@type"lemma"/orth construct, which should also hold further information specific to the lemmalike pronunciation. Grammatical information, such as POS, number, or tense, related to the entire lexical entry, should be encoded directly under <entry> by means of a <gramGrp> construct, as illustrated in Figure 6.4. information by means of either a simplified gramGrp/gram@type or <gram-Grp> and an explicit TEI element. The proposal 14 to choose between one of these two alternatives requires further discussion. These two recommendations are also valid for encoding grammatical information within <sense> structures.

For encoding information related to variants or inflected forms, it is recommended to use the <form> construct as shown in Figure 6.5. In the Mixtec-Spanish example 6.5, we can see how the tense has been encoded within the <gramGrp> construct for an inflected form, whose orthography has been specified as partial in the attribute of <orth> 15 element.

Further details on more recommendations about modelling morphological and grammatical in formation have been described by [START_REF] Banski | TEI-lex0 Guidelines for the Encoding of Dictionary Information on Written adn Spoken forms[END_REF].

Miscellaneous

The TEI-Lex-0 experts could make great progress on the representation of other aspects of the lexical description. The scope of the <usg> 16 element, which allows the encoding of usage information, has undergone a drastic restriction of its scope and more control over the classification of its possible types. usage@type="language" has been deprecated and such use should be encoded by means of the <lang> 17 element. Furthermore, a unified list of possible values of usage@type has been compiled by taking into account the existing suggested values in TEI P5 and the classifications defined by Atkins and[START_REF] Atkins | The Oxford guide to practical lexicography[END_REF][START_REF] Svensén | A handbook of lexicography[END_REF] To encode prose information that lacks structure or does not fall within the scope of the possible TEI elements allowed at the encoding level of the parent element, a lexicon designer is allowed, following the TEI-Lex-0 guidelines, to make use of the <dictScrap> 18 element. Moreover, and as it has been shown in the encoding examples of this section, the <pc> 19 element is recommended to be used to tag the separators between different physical structures, such as dots, commas, parentheses, etc. Besides the need to exclude such separators, as shown for pronunciation in Figure 6.4, such an encoding helps a parser to learn the features marking the transition between two fields. We will develop this idea more in Chapter 7.

The work on the TEI-Lex-0 guidelines is still ongoing and ideas should be further refined, as for the mentioned pending decisions, as well as for other important constructs like etymology and collocations.

Serialisation Model

To summarise the major decisions that have so far been made for the recommendations of TEI-Lex-0, we present a UML diagram that sketches the main TEI components of a lexical entry and their different relationships as they are defined in the actual guidelines.

Entry Class Diagram

Each class in the diagram presented in Figure 6.6 represents a TEI element that holds part of the definition of a lexical entry. Elements from modules other than the TEI dictionaries chapter are not represented in this diagram.

The serialisation model of TEI-Lex-0 represents a rich constellation of TEI elements. <entry>, being the central component and allowing recursivity, can be composed of:

• <gramGrp>: groups grammatical information, such the grammatical function (i.e. pos), number, gender, etc.. • <xr>: enables the encoding of cross-references within all the components of a lexical entry, even itself (recursive). The mapping mechanism is enabled by means of type (i.e, xr@type) to express the relationship between the source and a target encoded with <ref> 20 .

• <usg>: contains usage information of a higher level component that might be an <entry> and all its sub-elements. Typing <usg> is recommended to specify the category of the modelled usage.

• <form>: holds morphological information of an entry or its components. It can be recursive and might comprise <gramGrp>, <usg> and <xr> constructs.

• <sense>: contains the semantics of an entry and can accommodate all its components and even an entire entry structure (e.g. the case of related entries).

• <etym> 21 : allows the encoding of etymological and diachronic description. It can be recursive and permits the expression of usage and crossreferences by means of respectively <usg> and <xr> constructs.

• <dictScrap>: as has been previously explained, this element could be employed to tag less structured text segments within any element of the model.

• <lbl> 22 : this element is often used within an <xr> structure to encode labels triggering a cross-reference, like "See." in English, "Voir" or "V." in French, etc. It can also be used everywhere to designate any text segment that has a label function.

Discussion

The resulting model has managed to reduce alternatives for encoding several categories of lexical information and the scope of TEI key elements of the dictionaries chapter. But, despite the revised scope and new recommendations for a unified use of elements, the model remains complex with many bidirectional and recursive relationships. Such a fact represents an obstacle to finding a unified scheme for heterogeneous lexica, as the joint encoding possibilities can grow exponentially. In other terms, a query system, developed for querying different resources following the TEI-Lex-0 scheme, might miss the extraction of some embedded structures (e.g. <entry>s, <sense>s, etc) as there is no nesting limit. Such an issue can be resolved by defining multiplicities of the relationships which are missing in the guidelines. We do not know, for example, what represents the minimal structure of an entry (e.g. a lemma, a form, a lemma and a sense,..). Such constraints can ensure sanity check and prohibit falsely encoded structures that might be the result of a human manipulation of a wrongly defined transformation scheme from an existing TEI resource.

From our experience in the TEI-Lex-0 group, we have also noticed reservations about making important changes to the model of some problematic existing TEI structures and practices. In addition, the order of the occurrence of lexical information in a print dictionary dictates the constraints for the revised encoding which reduced the scope for changing old practices and left too much choice than if the approach had been based on the logical structure of a lexicon. In Section 6.3, we show how such constraints are relaxed in a different approach, where the NLP requirements have more control on the modelling choices of the lexical structure over the documentation and human readability needs.

LMF Reloaded

As an ISO standard, LMF is the subject of revision every few years. The process is triggered by a dedicated ballot and the availability of volunteering experts who will lead the revision project. In this section we report the features of the new version of the standard as a result of the recent revision work within the ISO/TC 37/SC 4/WG 4 committee.

Context

In Section 5.3, we showed how the version of LMF, published 2008, suffered from several shortcomings at the structure and the content levels. These limitations were the motive for the revision work that started in 2015 and which we joined in 2017. It was decided to structure the standard into several parts with more focused scope and total abstraction of the meta-model from its serialisation. A number of simplifications and improvements have been carried out to reduce the complexity of the modelling. In addition, the connection with leading standards, in particular TEI, has allowed LMF to gain more simplicity and acquire a flexible serialisation model that enables interoperability and exchange with the abundant number of TEI-based resources.

We present below the new features of the standard, with a special focus on the modelling principles and serialisation, as well as the progress made and our criticism of the resulting work.

Modelling Challenges

The LMF version [START_REF] Francopoulo | Lexical Markup Framework (LMF)[END_REF], the subject of the ongoing revision, tried to bring structure into the meta-model and its description by organising them into different extensions that complement a core model. The definition of such packages and the conformity of the classes they contain lacked precision and the separation among extensions was blurry in some cases.

The experts within the ISO/TC 37/SC 4/WG 4 committee took up, firstly, the restructuring of the standard. Secondly, they worked on simplifying the existing classes and their relationships. Finally, the meta-model has been enriched with new classes for existing and newly introduced extensions, resulting from the restructuring step.

Our main contribution was focused, at a first stage, on the consistency management of the UML abstract models during the Restructuring and Simplification processes. In addition, we were involved in finding a suitable TEI serialisation of the resulting models.

Restructuring

The restructuring project was initially based on the already existing extensions that came as a whole in the standard. A user who might be interested in only parts of the standard had to get the entire package and pick up what he/she needs. The organisation of the standard has been thoroughly discussed and revised to create a new structure based on complementary modules. The multi-part standard now has the following structure:

• ISO 24613-1 -LMF Part 1 -Core Model: this represents the minimal structure of a lexicon where the defined classes provide a lexicon designer with the necessary elements to model basic lexical information, like lemma, form and sense.

• ISO 24613-2 -LMF Part 2 -Machine Readable Dictionary (MRD) Model: this part complements the core model by means of classes that bring more precision to the semantic information, like examples and their textual representation, and allows a differentiation between written and spoken forms.

• ISO 24613-3 -LMF Part 3 -Etymological Extension: this is a new extension that carries a whole new range of information describing etymology and diachrony of lexical information.

• ISO 24613-4 -LMF Part 4 -TEI Serialisation: this part defines mappings between the defined classes in the first three parts of the standard and constructs in the different TEI P5 modules, with a focus on the dictionaries chapter.

• ISO 24613-5 -LMF Part 5 -LBX Serialisation: a second serialisation is proposed in this module using the Language Base Exchange (LBX), which had been introduced earlier by George, 2013 as an external application of the standard.

• ISO 24613-6 -LMF Part 6 -Syntax and Semantics Extension: this part has the role of providing deep modelling of semantics and syntax along with their mutual relationships. The work on this extension is mainly focused on remodelling the existing extension by following the new principles defined in Parts 1 and 2.

• ISO 24613-7 -LMF Part 7 -Morphological Extension: this extension has also a great input from what had been already defined for the old version of LMF, touching on the deep modeling of morphology. Enriching the restructured morphological classes and patterns with more precision and new classes is on the agenda of the LMF experts.

Simplification

The emphasis on abstraction and modularisation led to a series of major simplifications affecting several classes of the meta-model. One key feature that has been recently introduced is the CrossREF class which is a pointing/mapping mechanism that can be used to model a wide array of lexical features and relationships such as semantic relations, cross references, related entries and others witin the meta-model. As a result, some classes (e.g. List of Components and Component) whose features have been taken on, in part, by CrossREF have been removed altogether. Figure 6.7 illustrates the simplicity of the new mechanism used to model an MWE previously represented by classes which are now obsolete (see Figure 5.4).

Enrichment

New information has been introduced to describe essential aspects of lexical information such as Bibliography. Such information is required to specify references for some usages, definitions, examples, etc. Therefore the new class is kept multi-functional to be used in case of need as determined by the editor of a lexicon. Additionally, the differentiation of Orthographic Representation into Form Representation and Text Representation has been designed to enable more precision in the encoding of written forms touching on respectively Sense and Form sub-classes. As a second enrichment aspect, Etymology and Diachrony come out in the new version of LMF with a whole range of information useful for the study of words and their origins, designed for the NLP usage. It is beyond the scope of this thesis to give an exhaustive description of the decisions made to shape this new extension, but we can give a flavour of the new dimension that such a package has given to the meta-model. Figure 6.8 shows an instance of the object diagram of the Etymology extension in connection with the Core and MRD packages. We can observe the complementarity of the three extensions for the description of etymological links among different forms. Etymon and Etylink represent the core mechanisms for expressing an etymological relationship, in this case borrowing.

TEI Serialisation Model: ISO 24613-4

The resulting meta-model can be serialised by using any language that suits the needs of the user of the standard, like XML or SQL. Given the fact that TEI benefits from widespread adoption within the lexicographic community, the LMF expert committee has decided to connect with the de facto standard by proposing an XML serialisation based on mapping between the LMF classes and TEI elements. In Figure 6.9, we introduce a novel model representing key components of the new TEI serialisation extension. The UML class diagram represents a serialisation of the meta-model of the lexical entry structure by means of TEI elements. For the proposed serialisation, unequivocal mappings between the LMF meta-model classes and TEI P5 elements are kept to a maximum. Where one-to-one alignments are not possible, we opt for the translation of the LMF classes into TEI constructs that can be composed of more than one element or/and element types.

The <entry> element, representing the serialisation of the Lexical Entry class in the meta-model can contain:

• <xr>: to allow a cross-reference to other lexical entries having a relationship with the source entry, such as synonym, antonym or translation equivalent that do not yet have senses to map to. This element also comes to replace old mapping mechanisms within the entire metamodel, as it can be activated to enable the cross-references among nearly all the elements of the serialisation. The mapping mechanism is enabled by means of (i) typing xr@type, to specify the relationship, (ii) <ref> element to contain the text of the lemma or phrase representing the target, (iii) and specifying the ID (i.e. ref@corresp) to link to an internal or external target, if it exists, that can be typed (i.e, ref@type) with the category of the target (e.g. entry, sense, etc).

• <usg>: contains subject field information of an <entry> and all its components. The typing of <usg> is recommended to specify the category of the modelled usage and the idea is to converge to the same list of categories that has been compiled for TEI-Lex-0.

• <form>: holds morphological information of an entry. Typing is the key mechanism to differentiate between the possible forms that might be, in the first place, the lemma, along with other morphological structures like inflected forms, stem, root, etc. It might also contain grammatical information, encoded within <gramGrp>, as well as the <usg> and <xr> constructs.

• <sense>: serialises a sense of an entry and allows recursivity to contain sub-senses. It represents the provision of sense information such as <def> 23 for sense definition, <cit>24 typed with example or translation equivalents or usage information encoded within <usg>. It might contain grammatical information encoded within a <gramGrp> element and can reference other senses (i.e synonyms, antonyms, etc) by means of the <xr> mechanism. In some cases, when an entry has multiple forms and senses (e.g. the case of homographs) and the grammatical information is specified only for the forms, a link between the <sense> and <form> constructs needs to be established to differentiate the belonging of the senses (see Figure 6.10). Such a detail is crucial for NLP application where the missing link can be propagated to the syntax of the lexical entry.

• <etym>: is the entry point to the etymological description modelled in the third part of the standard. etym@type carries the type of etymology description, such as borrowing, inheritance, etc. It may contain the <usg> and <xr> constructs.

• <dictScrap>: contains elements of the lexical descriptions that do not fall into any of the previous categories but is allowed only for the lexical entry. It can be used by a lexical designer to markup information that he/she wants to exclude from indexing.

• <re>: has the same logical structure as the <entry> element in the actual specifications of the standard. This element is used to encode derived and related forms and can also be used as an alternative mechanism to encode MWEs (see Figure 6.11). There are, however, strong proposals to replace it by a recursivity of the <entry> element.

Figure 6.11 shows an example of a serialisation of the MWE "dead center" and its components. We can see how the <form> element is enriched by means of fine-grained elements such as <orth>25 for lemma, <pron>26 for pronunciation and <seg>27 for components of the MWE. The latter represents a use of the second mapping mechanism of the proposed standard, which allows a pointing to other forms by means of @corresp attribute.

Discussion

The new version of LMF brought new changes to the existing extensions, that helped to improve them, while they persisted to be mutually complementary with the other modules of the standard. The new serialisation answers more to the encoding requirements of print dictionaries, as it has adopted TEI guidelines and practices. In parallel, the new meta-model translated into TEI constructs, remained focused on how the lexical description could be modelled to carry the richest possible different information in the clearest and more simplified way.

The differences in modelling the same lexical structures are less important than those in TEI-Lex-0, as recursivity has been drastically reduced and mutual inclusion between elements has been disabled. However, different alternatives are still possible to model the same information like the case for homographs (see Figure 6.7 and Figure 6.10). In fact, the discussions within the expert committee were mostly focused on stabilising the core and the MRD models. The etymology model and serialisation modules managed to achieve relatively important progress, given the fact they are totally new. The work on the TEI serialisation extension was mostly focused on finding the mapping between the two standards. Consequently, the ongoing and future revisions should build upon the resulting serialisation model and define more restrictions on the scope of each employed TEI element to be fully compliant with the new LMF specifications. The serialisation examples made publically available would also help to obtain feedback from the community and more use cases for future discussions of the ISO committee.

GROBID-Dictionaries Output Scheme

GROBID-Dictionaries takes into account the recommendations in the TEI-Lex-0 and LMF to organise the recognition of the different lexical structures and their serialisation. But the modelling and the serialisation in the novel system also come up with explorative suggestions that were influenced by the use cases we encountered during the annotation of different samples in the experiments carried out.

In this section, we present main modelling challenges, the resulting serialisation of the CRF models introduced in Section 4.3.1 and their serialisation.

Modelling Challenges

The modelling challenge at this stage lies in the translation of all the recommendations dictated by lexicographic practices -TEI-Lex-0 -and NLP standardisation requirements -LMF -into CRF models able to extract lexical information and serialise it into a TEI scheme that is compliant with the novel standardisation schemes. The workflow first begins by guaranteeing a mapping to TEI compliant output. Then we study the possibility of converging to a scheme that maximally satisfies the requirements of TEI-Lex-0 and LMF.

Cascading Modelling

In section 4.2, we have already explained that the architecture of the cascading CRF models is defined by the ability to match TEI elements in a target scheme with the labels of each model. Avoiding the extraction of constructs that have different granularities is a requirement for the definition of the labels of each model.

We explain this further by illustrating the process of defining the model chain for the extraction of different structures from entries in an etymological dictionary. Figure 5.1 shows an entry from the print version of the dictionary. For this example, we rely on a TEI encoding defined by a lexicographer and neither the TEI-Lex-0, nor the LMF guidelines are followed. An etymology extension is the models chain responsible for parsing diachronic information encapsulated in an <etym> block detected by the Lexical Entry model and allows the clustering of specific constructs into TEI sub-blocks. Each subblock corresponds to a TEI compliant element whose significance in this use case is described in the following table :   TEI Element Description <seg> unclassified parts of an entry <quote> a quotation, text quoted from another source (typically from another dictionary) <def> a definition, i. e. a semantic description (gloss or paraphrase) associated with a given wordform <lang> a language identifier (often abbreviated) <mentioned> a mentioned wordform in any language, i. e. a wordform that is considered in the etymological description <bibl> a bibliographic citation (typically a scribal abbreviation) Projected on the print sample, such constructs have a sparse distribution over the etymological description of a lexical entry (see Figure 6.12).

Where <def>, <lang>, <mentioned>, <seg> and <bibl> structures are represented respectively in red, orange, green, black and blue, a <quote> construct has more complex content illustrated by the whole underlined segment in the above entry. The issue of having scattered information to be recognised in the presented use case has been solved by the effectiveness of the chosen machine learning technique. An experiment on a first etymological CRF model trained with carefully annotated data, gives very accurate results for the extraction of <def>, <lang>, <mentioned> and <bibl> components. However, the recognition of complex structures such as <quote> within the whole etymological description represents a second bottleneck. In the case of Cabbage, extracting a <quote> segment and its encapsulated etymological components requires a cascading processing. Our solution is to start by extracting complex segments by a first model (see Figure 6.13). The resulting <seg> and <quote> blocks will be parsed by a second model to extract the elementary structures (see Figure 6 .14).

This use case showcases the need to have 2 models to parse the sparse etymological constructs. We name the first Etym/Quote, as it has the goal of isolating the <quote> blocks from the rest of the structures. The second model we name it Etym, since it allows the full parsing of the pre-extracted structure by the first etymological model. The Etym model can be recursive to parse the blocks generated by a higher level Etym model. These two models form the etymology extension we define in Figure 6.15.

Satisfying different encodings for the same structures

Our goal, to find a unequivocal mapping between the structures recognised by the CRF models and the TEI-Lex-0 and LMF guidelines, has been challenged by the fact that, in some cases, more than one alternative has been defined in each of these two formats for encoding same structures (e.g., the case of homographs, explained in Section 6.2.2). Such a need pushed us to study different heuristics to reach a target encoding by allowing required labels for the suitable models in different modelling scenarios. From a user perspective, the activation of such labels and the generation of one of the wanted target encoding is possible by following two steps:

• After choosing the chain of models permitting to reach a structure and the depth of the target TEI output, the user proceeds by annotating the training data for the corresponding models.

• After training each model properly, the user can compose the output he/she wants by selecting in the web application (see Figure 4.8) the models he/she trained according the scenario he/she chose.

Following this setup, we managed to enable two encodings for modelling structures that could be interpreted differently. For instance, we managed to allow the modelling of homographs following two different scenarios. The first scenario, having the output presented in Figure 6.2 as the target, should allow a "Lexical Entry" model to recognise sub-entries within <entry> and then parse the sub-entry, serialised as <entry> (i.e. the entry having as ID "actVerb"), by using a second entry model. The second scenario, followed to generate the output of Figure 6.3, relies instead on a "Sense" model to parse a <sense> block and isolate <gramGrp> from senses that will be parsed by a "Sub-Sense" model to extract related entries and other components like definitions and examples (not represented in the figure). The two figures, representing TEI-Lex-0 serialisation instances, recommend the use of typed <entry> to represent a related entry. The models of GROBID-Dictionaries were not all implemented in parallel with the discussions of the TEI-Lex-0. As the decision to use <entry> for all entry-like constructs is relatively recent, related entries are still serialised as <re> the models which remain an LMF compliant choice.

The guidelines for encoding the two formats, TEI-Lex-0 and LMF, are also not fully compatible. For instance, there is a difference between modelling a recognised <gramGrp> block related to the entire entry, such as the POS. We can see in Figure 6.3, that TEI-Lex-0 recommends the encoding such a <gramGrp> as a direct child of <entry>, whereas in LMF, such an encoding is not allowed by the serialisation model (see Figure 6.11) and it is only allowed inside the <form type="lemma"> element. We chose to allow the first alternative, as it reflects the physical structure of most of the dictionaries we tested and our users give more importance such a representation. Encoding such a logical structure according to the LMF serialisation model remains, Chapter 6. Novel Standardised Schemes for Encoding Dictionaries however, easy to implement thanks to a dedicated separation between the recognition and the serialisation stages that we made sure to have.

Further details about the encoding supported by GROBID-Dictionaries are summed up in the serialisation model presented in Figures 6.15 and 6.16.

Serialisation Models

The parsing models within GROBID-Dictionaries generate two XML serialisations: the first is for internal use and the second represents the final output that results from the call of the REST services through the web application facet (see Figure 4.8).

Internal Serialisation

The first serialisation, illustrated in Figure 6.15, uses XML elements that in many cases share the same labels as homologous TEI elements. This serialisation is used to annotate data, train and evaluate the CRF models. The reason behind the use of labels different from those existing in the TEI guidelines is to give more semantics the tagged components. For instance, we make use of <fromGramGrp> to denote <gramGrp> related to the entire entry and to differentiate such a component from <senseGramGrp> which represents a <gramGrp> related to a sense. There are also some cases where we use labels for elements, that might be confused with TEI elements, but we use them to denote different structures. For example, we use <body> element to mark the body part of page whereas <body>28 in the TEI guidelines is usually used to mark the entire body matter of a monograph or a dictionary.

We present below both serialisations of the elements of each model in the architecture. In the remainder of this section, by serialised in we mean the element serialised internally and transformed into the second (i.e. final) serialisation as:

• Dictionary Segmentation: The first model of the architecture has three main labels:

-<headnote> for header and head-note information, serialised in <fw type="header">29

-<body> for all the text area containing the lexical entries of a page, serialised in <ab>30

-<footnote> for footer and footnote information serialised in <fw type="footer">

• Dictionary Body Segmentation: The second model parses a <body> block, recognised by the first model, and processes it to recognise <en-try> constructs that are the same in both serialisations.

Chapter 6. Novel Standardised Schemes for Encoding Dictionaries

• Lexical Entry: The third model parses each <entry>, to segment it into:

-<lemma> serialised in, <form type="lemma"> -<variant> serialised in, <form type="variant"> -<inflected> for morphological and grammatical information related to a "Form", serialised in <form type="inflected"> -<senseGramGrp> for grammatical information related to a "Sense", and <formGramGroup> for a general grammatical description, are both serialised in <gramGrp> -<subEntry> for an embedded entry, serialised in <entry> -<reference> for a reference to another entry, serialised in <xr> -<sense> for semantic information, <re> for a related entry, <etym> for etymological description, all have the same labels in both serialisations.

• Form: This model analyses blocks representing a "Form" such as <lemma>, <variant>, or <inflected>, generated by the Lexical Entry model, and segments the information it contains. The current list of possible labels for this model contains:

-<orth> to contain the orthography of a "Lemma" or a "Variant", <pron> for pronunciation, <gramGrp> for grammatical information, such as part of speech, gender, number, etc, <lang> for language information about a "Form", <name> for headwords in encyclopedic dictionaries, <desc> to encode the brief description coming after the headword of an article and <usg> for usage information of the analysed form. All these labels remain the same for both serialisations -<part> to contain the extent of an orthography of an "Inflected From", serialised in <orth type="part">

• Grammatical Group: This model has the task of parsing a <gramGrp> wherever it appears in the architecture. The actual list of labels contains:

-<pos> for POS, <gram> for a piece of grammatical information that is going to be typed, <tns> for tense of a verb, <gen> for gender information, <number> for number, <subc> for information about transitivity, countability, etc., There is no difference in labels between the two serialisations for these elements.

• Cross Ref: Such a model makes it possible to parse cross references <xr> wherever they appear in the architecture. The main labels predicted by this model are:

-<ref> for an internal or external reference, and serialised in <xr> -<relation> to type the reference instance (e.g. synonym, antonym, false friend, etc) serialised in the type for the parent <xr> element -<bibl> for bibliographic information and <lbl> for textual triggers of a reference (e.g. "See", "voir" or "V.", etc), both remain the same in the two serialisations.

• Sense & Sub-Sense: These two models orchestrate the decomposition of the hierarchy of senses, if sense nesting occurs. For the Sense model, the supported labels are:

-<subSense> for an embedded sense, serialised in <sense> -<gramGroup> for existing grammatical information, serialised in <gramGrp> -<num> for sense numbering, and note for any prose description related to the upper sense, <def> containing a definition of a subsense, <usg> for usage information, <re> for possible embedded related entries, <etym> for diachronic information related to the sense, and <xr> for recognised cross references. These elements have the same labels in the internal and final serialisations.

-<example> for sense illustration, serialised in a <cit type="example"><quote> construct.

-<translation> for translation equivalents, serialised in <cit type="translation"><quote> construct.

• Etym/Quote & Etymology: the labels of these models remain identical to what has been already described in Section 6.4.1 for the internal and final serialisation.

Additionally, we use the elements <dictScrap> and <pc> for all models to mark, respectively, junk text resulting from digital conversion (e.g. metadata, conversion errors, etc) or digitisation (i.e. OCR noise), and punctuation. We make use of another label <note>31 for related prose descriptions and notes, replacing the use of <dictScrap> in TEI-Lex-0 and LMF, given the fact that their guidelines do not recommend any element to use to encore noisy text in digitised dictionaries. All these elements are serialised using the same labels in the internal and the final serialisations.

Final Serialisation & Discussion

The model of the final TEI serialisation can be partially represented as a UML class diagram, to have an overview of the main serialisation elements that can be compared to the TEI-Lex-0 (Figure 6.6) and LMF (Figure 6.9) schemes.

Figure 6.16 represents the final serialisation model of the <entry> structure, its components and their mutual relationships.

On the one hand, we can notice that GROBID-Dictionaries's entry model contains all the TEI elements used for the serialisations of TEI-Lex-0 and the .16 -are also included in the GROBID-Dictionaries's serialisation. Thus, using the parsing models of our system enables, in a great number of modelling use cases, the generation of macro TEI serialisation that is compliant to the two new standardised schemes.

On the other hand, GROBID-Dictionaries's final serialisation model represents more constrained relationships than TEI-Lex-0 and gives more encoding alternatives than LMF. The affected relationships, like the mutual inclusions or the recursivity of certain elements in TEI-Lex-0 (i,e. <from> and <etym>) or the cross-reference modelling in LMF (i.e. <xr> being n-ary relationship), are not driven from print dictionaries modelling but rather the needs of NLP databases. These relationships can be added to the output of our system by applying a post-processing, such as enabling the mapping between recognised entries and their cross-references triggered in the source or the target by an <xr> element.

Our modelling anticipates some serialisation choices and provides dedicated serialisation, such as the different use of <note> and <dictScrap> which is driven from practical use cases encountered during the annotation of diverse print dictionaries. The need to encode the entire text of a parsing level forced us to find suitable encoding elements in the TEI guidelines, such as <dictScrap> for OCR noise or <num> for the numbering of entries or senses in some dictionaries.

Finally, the composition associations in this diagram, as in Figure 6.6 and as opposed in Figure 6.9, mean that none of the child elements can exist in the final output without its parent. In fact, this precision reflects the origin of the serialisation, which is not the result of the conversion of a relational database, but rather an output generated from a cascading approach where a parent triggers and accommodates its child components.

Chapter Summary

This chapter presented two novel standardised schemes for lexical resources, TEI-Lex-0 and LMF, and their inspiration for the serialisation of the parsing models in GROBID-Dictionaries.

TEI-Lex-0, having the goal of defining a baseline encoding for print dictionaries, has managed to reduce TEI modelling alternatives towards more unified encoding guidelines. LMF has undergone an important revision work that has brought more structuring and richer information to the meta-model and its serialisation. The TEI serialisation for LMF managed to propose initial mappings between the two standards. More work is needed to investigate the applicability of the primary serialisation model on complex modelling scenarios and granular lexical information. We have shown the different modelling challenges, as well as the resulting serialisation for both new schemes.

We have also presented the two serialisation facets of GROBID-Dictionaires's models and the overlapping of its final TEI serialisation with TEI-Lex-0 and the new LMF. The compatibility with these two new standardised schemes has been kept maximal, despite the differences between the standardisation approaches. The degree of compliance of our models depends on the evolution of these two novel formats, as they are not yet finalised.

In Chapter 7 we present parsing experiments on different print resources using the described models of GROBID-Dictionaries. We give an exhaustive analysis of the strengths and weaknesses of the models employed and, more importantly, we focus on the possibility of scaling up the parsing while using the presented standardised scheme.

Chapter 7

GROBID-Dictionaries in Action

Introduction

After having presented the parsing architecture and the corresponding serialisation scheme, we now aim at assessing the performance of each lexical model against different print dictionaries. To this end, we need appropriate annotated data in order to conduct the various machine learning experiments.

In fact, the parsing scenarios we want to experiment require real world data that are able to exhibit different categories of digitisation anomalies (see Section 3.2.1). Consequently, the annotation of the lexical structures should take into consideration the specificities of our new serialisation model, along with these anomalies in raw documents. Besides, given the fact that the existing TEI lexical resources do not cover both specificities, we have carried out the work from scratch and annotated a pool of dictionaries that covered the diversity we needed for our experiments.

This final round of large-scale experiments has allowed us to use, in a uniform manner, the most stable version of the annotation scheme to a representative collection of dictionaries and thus annotate uniformly substantial amounts of data for several cascading models. Through the annotation and the training of the lexical models, we encountered additional challenges related to the quality and the balance of the data in terms of observable features and complexity. Besides the design and the feature selection for the parsing models, such data-related aspects appeared to significantly impact the machine learning experiments.

In this chapter, we present our machine learning setup by explaining our experimental goals and major factors interfering with the introduced sequence labelling task. Then we give an overview of the dictionaries pool and main challenges encountered while applying the annotation scheme to each dictionary. Next, we detail the series of experiments we carried out with various print dictionaries, using a selected set of cascading models of GROBID-Dictionaries' architecture, and show the extent to which our technique and the performed feature engineering can support their parsing. Finally we explore, through another series of experiments, the possibility of scaling up our system either by applying the system on similar structures in print documents or/and by integrating new generation models.

Machine Learning Experiment Setup

Before presenting the details and the evaluation of our experiments, we define the goals of our investigation and outline the factors interfering with the experimental setup to prepare a solid ground for the analysis of the results.

Experimental goals

Our experiments focus on studying the variation in performance observed for each model in the cascading chain, along with their learning behaviour when exposed to diverse dictionaric content. The performance of each lexical model is studied with regard to:

• Feature engineering: our aim is to investigate the impact of basic and advanced tweaking of certain features that can improve the performance of essential sequence labelling models in our architecture. For that, we used the combinations of feature templates introduced in Section 4.3.2. A first set of experiments presents the difference in performances resulting from the use of the Unigram and Bigram templates.

Another set of experiments is carried out with the Engineered templates to observe the impact of advanced feature engineering.

• Generalising capacities: given the best overall performance of a feature combination, we want to explore the ability of each model to generalise over the data that were used for its training. Two main variations are to be investigated: testing the models with data from already seen dictionaries and from unseen samples.

• Learning curve: for experiments that have the best performances, we want to study the learning behaviour of the models involved. More precisely, we want to gain insight into the evolution of each model's learning given a number of annotated pages used for its training. Such an aspect is studied with regard to the category and the quality of each dictionary.

To this end, our experiments focus on a selection of models from different levels of lexical parsing. We have chosen seven models of the cascading architecture. Besides the first three models necessary to activate the rest of the architecture explained in Figure 4.6, we selected Form, GramGrp, Sense and Sub-sense models. The choice of these models was based on the stability of their TEI encoding, as well as the fact that they are the most commonly used models by the early users of the system.

Interfering Factors

In the early stages of this work, we assumed that the task we are addressing starts with a dictionary file to parse and ends with the corresponding standardised structured output. Although such an assumption sound plausible and even relatively straightforward after gaining hands-on experience, we discovered that the task is more complex than it might at first appear and starts much earlier. Drawing up an exhaustive list of the obstacles and issues involved is beyond the scope of this thesis. Nevertheless, we shed light on different aspects of sample representativity and the importance of digitisation details explored in different categories of print material. This will help later on in the evaluation of the various experiments that we carried out with different dictionaries.

Sample Selection and Annotation

To train a supervised machine learning model, the annotation of a representative sample is required. In the case of GROBID-Dictionaries, a representative sample can vary from one model to another. For instance, a representative sample for the Dictionary Body Segmentation model, which has the task of recognising the boundaries of each dictionary article in a page body, should contain dictionary pages having enough:

• entries of different sizes,

• and different spots of the entries in a page: beginning of page, beginning of a column, end of a page, split over several pages or columns, etc.

On the other hand, a representative sample for the Lexical Entry model, which is responsible for parsing the main components of a lexical entry, should contain, for instance:

• entries having a simple structure • entries having a complex structure. Such entries should comprise:

items representing different physical structures (typographic, layout, etc.) items representing different logical structures, especially less frequent components: for instance entries with and without related entries or cross-references that could be less present in dictionary articles than forms or senses

• entries with more textual markup variation, especially when typographic information is poor

• in some cases, entries with and without hyphenation where the latter is frequent and occurring at the beginning or the end of the text sequences, or when the dictionary articles are very short (i.e. few tokens per entry)

The representativity of samples varies from one dictionary to another. And given the differences in the representativity needed to train each model of our machine learning architecture, selecting pages that contain all the variations required for the convergence of all models is extremely challenging. Selecting different pages for training different models can lead to the propagation of the recognition errors by higher level models in the training data of subsequent models. This would result in an imprecise evaluation as the model being tested may not itself be responsible for any potential recognition errors. In addition, the conventional dataset split ratios1 , for training, development and testing, can not always be respected. [START_REF] Bowers | TEI Encoding of a Classical Mixtec Dictionary Using GROBID-Dictionaries[END_REF] Table 7.1 gives an overview of the statistics of a page sampling process for an experiment [START_REF] Bowers | TEI Encoding of a Classical Mixtec Dictionary Using GROBID-Dictionaries[END_REF] with a Mixtec-Spanish dictionary represented by the sample in Figure 7.1.

Model

For training 5 models of the architecture, we selected and annotated 14 pages from different spots in the dictionary: 10 for training and validation, and 4 for testing. We detail the annotated instances for each model, except for the first one dealing with the prediction of main regions of a page. The page sampling shows different ratios for each label of each model. For example, for the Sub-Sense model, the ratios for the instances of <def> labels in the training and evaluation datasets are very close to the 2/3, 1/3 ratios, whereas the <gramGrp> testing proportion exceeds the training one. The number of <gramGrp> instances is also small compared to <def>. Annotating more separate instances of <gramGrp> and adding them to the training and testing datasets would be misleading for the model as the context of the sequence must be preserved for the model to learn the right features. The sample in Figure 7.1 shows several hyphenated tokens in small size dictionary articles. The hyphenation occurs in different spots in the entries. Thus, including such a variation in the training dataset is crucial for more than one parsing model.

OCR Impact

The second major factor that impacts on the machine learning experiments that we want to conduct is related to the OCR quality, particulary in the case of digitised material.

To investigate such an impact, we carried out an experiment [START_REF] Khemakhem | How OCR Performance can Impact on the Automatic Extraction of Dictionary Content Structures[END_REF] with two versions of a retro-digitised legacy dictionary [START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF]. We use the term OCRisation to designate the process of using an OCR system, or any character recognition system, to recognise the layout and the text of a document. For this experiment, we carried out the OCRisation by using the Transkribus platform [START_REF] Kahle | Transkribus-a service platform for transcription, recognition and retrieval of historical documents[END_REF] and following the workflow described by [START_REF] Lindemann | Retrodigitizing and Automatically Structuring a Large Bibliography Collection[END_REF]. The process consists of using a default OCR model to produce a first layer of OCRs that will be manually corrected and then used to train a Handwritten Text Recognition (HTR) model to produce higher quality digitised text.

The first sample was compiled from a low image quality document that was OCRised with an HTR model trained with 28 pages. The second sample was created using an HTR model trained with 108 pages applied on a high image quality document. We annotated the same 45 pages from each sample of the dictionary. We tried to select the pages that covered the maximum number of variations required for training the first three models of GROBID-Dictionaries. For each parsing level, two instances of the CRF models with the same features were trained and evaluated. The results2 of the experiment, represented in the evaluation of the CRF models in Table 7.2,7.3 and 7.4, show the different measures on the field level. We can notice the differences in the performance of each model to recognise the different labels.

Three main conclusions can be drawn from this experiment: firstly the OCR quality plays an important role in boosting or deteriorating the performance of our machine learning models. Secondly, despite the noisy OCRs, the CRF models can still operate and produce for the recognition of some labels comparable results to high quality OCRised material. Finally, the impact of the OCRs is more visible for the extraction of fine grained information which can be explained by the importance of the textual and markup at such levels.

Dictionary Samples & Annotation

Selecting dictionaries for our experiments was a challenge in itself as several anomalies were often only discovered after many rounds of cascading annotation, especially for OCRised samples. Our final dictionaries pool is composed of 5 dictionaries, each reflecting several aspects of the classification established at the beginning of this thesis (see Figure 2.4): monolingual or bilingual, born-digital or digitised, modern or legacy, and with lexical or/and encyclopaedic content.

The multi-stage annotation of the different dictionaries was carried out manually and semi-automatically by using the two modes introduced in Section 4.4.2 (raw and pre-annotated training data). One person was in charge of the whole annotation/correction task for all the dictionaries. The raw and annotated data, along with the results of the experiments have been released on a freely available repository 3 .

In this section we present each dictionary with regard to these criteria as well as the information about its structural complexity. We also highlight main consequential specificities encountered during the annotation of a number of lexical structures.

Chapter 7. GROBID-Dictionaries in Action

Dictionnaire de la Langue Française (DLF) Description: "Dictionnaire de la Langue Française" [START_REF] Littré | Dictionnaire de la langue française[END_REF]) is a legacy monolingual French dictionary that contains both lexical and encyclopaedic articles. Besides the classic explanation of entries, exceeding 78 000 articles, the lexicographer provides an exhaustive historical and etymological description of lexemes with nearly 300 000 citations. The retro-digitised version4 that we used for the purpose of our experiments comprises 4 volumes with over 5 000 article pages each organised in 3 columns (see Figure 7.2). We do not have enough information about the digitisation process but we assess the OCRs quality in the PDF we used to be relatively good.

Dictionary & Annotation Specificities: We consider this dictionary to be the most complex sample we used for our study, as it represents challenges at almost all parsing levels. Lexical markups, namely parenthesis, bold and italic, are omnipresent, especially for distinguishing the structures at the Lexical Entry level. Such markers are much less present at the Sense and Sub-Sense levels where pipes and numbering are visually the most helpful clues. We can group the main challenges we faced during the annotation of DLF by model:

• Lexical Entry: The dictionary articles have a high variation in length, from few tokens up to several columns. Inflected forms such as "Etudiant, E" were not marked as inflected forms at the Lexical Entry level, as it will break the annotation of grammatical information related to the main form. We also made the same decision for variants that occur after lemmas with no geographic or other more elaborate information. Consequently, we postponed the annotation of these structures to the Form level, where we annotate them as <part>s.

• Sub-Sense: DLF has a very rich semantic representation and it is often a challenge to human experts to differentiate the boundaries of definitions and examples. In fact, most of the examples are citations from classic French literature although some brief usage examples may occur just after a definition. To overcome this issue, we based our approach on the choices made to compile a structured version5 of the dictionary and we considered examples to be only citations.

Easier English Basic Dictionary (EEBD)

Description: "Easier English Basic Dictionary" (Publishing, 2009) is a monolingual dictionary for English which contains over 5,000 entries, published in 2009. For our experiments, we used the 370 pages containing the body of the dictionary. The PDF version which we used, is a digitally born one that has two columns per page. Dictionary & Annotation Specificities: As Figure 7.3 illustrates, the dictionary has a very modern and clear typography and its markup system. Pronunciation, sense definition, examples and related entries are clearly marked up. We consider such a dictionary to have medium complexity level as it represents a few parsing challenges:

• On the structural level, the major challenge is the representation of related entries and notes which occur at the Lexical Entry and Sub-Sense levels. We annotate the notes only at the Sub-Sense level. We are aware that a post-processing is necessary to reassign some notes which are related to the whole entry. Related entries marked with diamonds are annotated within the lexical entry, the rest are annotated within Sub-Sense

• Other issues present in the versions of the dictionary that we had to deal with are related to the character encoding, especially for pronunciation constructs, which is different from the one supported by GROBID core. Lemmas are represented physically by redundant strings which we annotated as <dictScrap>s. Finally, the PDF we used originated from different steps, starting from the first download till the sampling process. We suspect that one of the PDF engines we used resulted in the introduction of some meta-data at the end of almost all the pages. We used that version for the tutorial6 series we organised for early system users. This anomaly surprisingly disappeared when we restarted the workflow from scratch to generate data for our experiments.

Mixtec-Spanish Dictionary (MxSp)

Description: We refer by Mixtec-Spanish7 dictionary to a PDF document that was compiled and published in 2009 by Jansen and Perez Jiménez from "Voces del Dzaha Dzahui'", a historical lexical resource published by the Dominican fray Francisco Alvarado in the year 1593. This bilingual dictionary, documenting a legacy lexicon, has over 370 pages with modern layout and short entries arranged in two columns (see Figure 7.1).

Dictionary & Annotation Specificities: We consider this dictionary to be the easiest to parse, given the length of the entries and the clear and clean typography. The parsing of this dictionary was the fruit of a collaboration with a linguist working on endogenous languages. The most challenging part was the sampling process, since the annotator did not speak both languages of the dictionary and had to collaborate with the linguist for the sampling process to cover the required logical and physical representativity explained in Section 7.2.2

Chapter 7. GROBID-Dictionaries in Action

Fang-French (FangFr) & French-Fang (FrFang) Dictionary Description: The Fang-French8 & French-Fang dictionary (Galley, 1964) is a bilingual dictionary gathered in one volume that has over 500 pages of lexical entries split into two parts. The dictionary was published in 1964 but later digitised and the PDF version we used has a medium OCR quality.

For our experiments, we consider the first part FangFr, containing over 390 pages, to be a separate dictionary from the second part FrFang, with over 140 pages.

Dictionary & Annotation Specificities: As Figures 7.4 and 7.5 show, the markup system is based on typographic change along with textual clues to markup field transition. For the page sampling milestone, we detected different anomalies in the OCRs such as messy text order of footnotes and some important text blocks (e.g. section titles, page numbers, parts of entries, etc.) disappearing in the extracted text. We suspect the origin of such anomalies to be the recognition of text regions as images by the OCR system. We encountered several challenges during the annotation process and the decisions we made could be gathered by parsing level for each part of the dictionary:

• FangFr: Two models represented difficulties for their annotation -Sense: Finding the boundaries of non-numbered senses was not obvious -Sub-sense: We used <note> for any prose description. However, it was hard to distinguish the different fields, especially <note> from <def>. We chose to annotate the first sentence or any short description (gloss) as <def>. Any that comes after, which is neither an example nor a cross reference, is considered a <note>

• FrFang:

-Lexical Entry: We used <sense> to annotate * Definitions followed by a set of translation equivalents. For instance:

• ANCÊTRE aïeul, grand-père, grand-mère, mvam (h)

* A definition and one translation or more. For example:

• AMULETTE fétiche, grigri, ñgîr (b) * Simple translation equivalents and their pronunciation. For instance:

• ANANAS aloès, ñkuba (h), ñkôkh ô sekh (hm)

We used <re> to annotate related entries, which often represents the rest of the lexical description of a dictionary article * For the MxSp dictionary, the differentiation between translations and definitions in the target language is too fuzzy. The recommendation made by the linguist was to annotate information that does not represent grammatical or usage information, as one or many definitions.

* For the FrFang dictionary, it is straightforward to recognise the French definitions followed by translation equivalents in Fang

Experiment Series 1: Training with One Dictionary

Having introduced the general setup of the experiments we want to conduct, in this section we present a first series of experiments. Each of the experiments uses one dictionary to train all the selected models of the architecture. First, we report and discuss the evaluation of each model using the three feature templates introduced in Section 4.3.2. Then, we present the results of our experiments revealing further the behaviour of the best models for each dictionary, in particular the learning curve.

Feature Engineering Experiments

The goal of these experiments is to understand the impact of feature engineering on the outcome of each model based on common characteristics of the dictionary samples at each parsing level. For training 9 and testing each model, we selected annotated data taken from different parts of a dictionary. We tried to respect as much as possible 3 4 1 4 ratios for splitting the training/testing data, but it was not always possible given the reasons explained in Section 7.2.2. For MxSp, we wanted to challenge the models by trying 2 3 1 3 ratios.

Experiments

In the following, we present, model by model, the evaluation10 of seven parsing levels for each sample of the five selected dictionaries. For each model, except Dictionary Segmentation, we experimented with the three categories of templates: Unigram, Bigram and Engineered. The number of pages used for training/testing is different from one dictionary to another, depending on the complexity of the sample. More information about the number of training pages is provided in Section 7.3.2.

We report the macro-average F1-score11 of each label recognised in every sample, as well as of the macro-average F1-score of field level evaluation for all the labels of a model. We chose macro-average measurement because of the imbalanced number of instances of the classes of each model. The field level choice made the feature engineering more challenging, as we have often noticed an improvement at the token level but one single wrongly recognised token, quite often a full stop or an OCR garbage, sanctions the performance on a whole text block. For labels not encountered in a dictionary sample, their scores are rendered as -1. One dictionary, FrFang, did not not have any grammatical information to be parsed and consequently the GramGrp model was not activated, which explains why its scores are rendered as -1.

Dictionary Segmentation Figure 7.6 shows an identical performance of the two categories of templates with perfect recognition for 3 dictionaries and lower recognition error by the bigram variation (less than 4%) for DLF, despite the sample's relatively complex layout. The lowest score of both templates observed for FrFang reflects the struggle of the model to differentiate Chapter 7. GROBID-Dictionaries in Action headnotes from the bodies of pages. This is mainly due to the fact that some headnotes were not recognised as text regions by the OCR system and it consequently affects the learning of the model when they disappear from the text sequences. The fluctuant score for the <dictScrap> label, which we used to tag dictionary sections, is related to its limited occurrences in the training data (a ratio of less than 2 per 400 instances). In general, engineered templates seem to combine the best of the bigram and unigram features. They have slightly better results, especially for recognising less frequent instances, with Macroaverage F-1 scores exceeding 95% for 4 dictionaries.

Lexical Entry For this model, the comparison of performances represented in Figure 7.8 is reduced between the bigram and the engineered templates with a mutual competition on certain samples.The boosted general scores for EEBD and DLF illustrate the feature engineering impact that we aimed at by tweaking the bigram templates. The difficulty for both templates concerns the recognition of less frequent constructs: notes, related entries, usage, cross references and OCR garbage, tagged as <dictScrap>. Nevertheless, longer range features again prove to be useful for recognizing less frequent labels in relatively long sequences. The parsing performance for inflected forms and etymology is insensitive to the addition of templates to the bigram combination. More information about neighbouring tokens seem to slightly harm the identification of lemma blocks coming always at the beginning of a main sequence, where the recognition of semantic blocks presents a small variation between the two best templates that depends on the sample.

Form As shown in Figure 7.9, the three classes of templates have very similar performances for all the dictionaries except for MxSp, where the bigram templates win. For the latter, the overall recognition is good with individual F1-score for labels exceeding 95%, except for <part>, <lbl>, <pc>, and <usg>. The difficulties in recognising these constructs mainly stems from their low number in the training data. For <part> the model with bigram templates manages to reach a score of 85%, despite the occurrence limitation. This shows the positive impact of the previous label and the negative impact of long range features on the predictions of the model.

GramGrp

Unigram templates beat the rest of the templates with almost perfect recognition, where the worst score exceeds 97% for the FangFr sample (see Figure 7.10). For this fine grained lexical information, the sequence labelling model gives its best when the clues used are more focused on the current token.

Sense

The sense model has the smallest number of labels to be predicted on relatively long text sequences. Figure 7.11 shows a stable performance of the bigram templates over all samples. But all the templates still have comparably good, often perfect, results.

Sub-Sense This is the model with the biggest number of classes to be differentiated with over 10 labels. From the results reported in Figure 7.12, we notice that the longer the text of the sense is, the worse the Unigram templates perform. Except for the less frequent labels, the other two templates

Discussion

The previous experiments can be summarised by Table 7.5, where U stands for Unigram templates, BB for Basic Bigram templates, EB for Engineered Bigram templates and A for any category of templates.

From these experiments, we can draw the following conclusions:

• For parsing short text sequences, as in the case of the GramGrp model, simple Unigram features are more efficient.

• For the rest of the models, where text sequences are relatively longer, the label of the previous token is a very important clue.

• Engineered templates, with long-range features, are generally the best choice for labelling lexical constructs in longer text sequences

• The Bigram templates beat in many cases the Engineered templates, despite the previous conclusions made by [START_REF] Lavergne | Practical very large scale CRFs[END_REF], favouring larger numbers of features per model.

Model MxSp EEBD FangFr FrFang DLF Best Dictionary Segmentation

Learning Curve Experiments

A second behaviour of the models we want to know more about, is actually the learning curve of the best models selected in the previous experiments.

Experiments

To generate such a learning curve, we designed our experiments by having for each model to be trained on one dictionary sample, 4 batches that do not necessarily contain the same number of annotated pages. The pages in each batch are chosen from different parts of the dictionary. For each dictionary we report the number of annotated pages for each model and the performance of each model after adding a new batch of annotated pages. As pointed out earlier, the number of pages annotated for each dictionary varies according to the complexity of the sample. In what follows, we present the learning curve of all models observed for each sample in our dictionary pool. EEBD The curve reported in Figure 7.14 shows that fewer pages are required for this less complicated sample to reach the best performance and stability of the different models. Moreover, 10 pages were enough to achieve a high labelling accuracy. The additional pages were useful to stabilize the recognition of certain labels and further strengthen the quickly achieved accuracy.

DLF

MxSp Fewer than 10 pages were enough for the second digital-born bilingual dictionary to reach perfect performance for 4 models. Except for the Form model, the other the models seem to be reaching a plateau by the 8th page. It is worth remembering, that the low F1 score in such a curve (see Figure 7.15) is actually due to difficulties recognising less frequent labels.

FangFr For 5 models, 10 pages enabled the labellers to reach a plateau of their best performance (see Figure 7.16). The Sub-Sense model gives better FrFang The threshold of 10 pages also remains enough for 4 models, applied to the FrFang sample, to reach the plateau (see Figure 7.17). Sub-Sense and Dictionary Body Segmentation needed respectively 10 and 15 pages more before their learning curve stabilised.

Discussion

Through these experiments we can notice a rapid convergence of the different models, where a plateau could often be reached after annotating 10 pages. For models operating on longer text sequences, such as the case for the Dictionary Segmentation model, more pages are required. In general, the difference in the rapidity and the evolution of the learning are related to a number of factors. Digitisation origin, the quality of the OCRs, the representativity of the sample and the presence of less frequent labels are among these determinant factors.

Experiment Series 2: Training with Multiple Dictionaries

The models, upon selecting their best feature templates, often yield a very accurate recognition of the mono-sample lexical structures they are exposed to. We aim at further investigating the behaviour of these models, tuned with the same feature templates, by experimenting their training with multisamples at once. The goal is to study the generalisation capacities of each model and the impact of the combination of samples on the learning curves.

To do so, we selected 5 sample combinations:

• 2 BL: stands for 2 Bilingual dictionaries that have the same pair of languages and are most likely designed and compiled by the same lexicographer(s). From our pool, this represents the case of the two parts of the Fang-French and French-Fang dictionary

• 3 BL: stands for 3 Bilingual dictionaries where the language pairs are different and the design and the compilation of the lexicons are carried out by different lexicographers. For that, we added the Mixtec-Spanish sample to the previous combination.

• DB: stands for Digital-Born samples, which are represented in our dictionary pool by the English and Mixtec-Spanish dictionaries. Such samples have a very little amount of noisy text as the main source, the OCRs, are present in this combination.

• ML: stands for Monolingual samples where the lexical description in the dictionary articles is in the same language and have, on average, longer text sequences than those in the bilingual samples. The modern English and the legacy French dictionaries are the constituents of this combination.

• ALL: The final combination gathers all the samples we collected and annotated, where the multi-linguality is mixed with the different digitisation and aging aspects. Form On the partial combinations of samples, the Bigram and Engineered templates enable the model to reach its best performances with the best results on the ML sample achieved by the Bigram variation. Nevertheless, the evaluation in Figure 7.21 shows identical performances of all templates when training with a combination of all samples. For the best templates, mixing samples boosts the macro average performance reported for single sample training. Such a behaviour is observed for DB and 3 BL. However, that is not always the case as we see a little deterioration in the overall performance for All.

GramGrp As for the mono-sample experiments, the Unigram templates achieve the best scores with almost perfect recognition for all combinations.

And compared to the performance on individual samples, the multi-sample training shows only a slight regression. The suitability of simple features for this model is also further confirmed.

Sense Another model, with the limited number of labels but operating on longer text sequences, that shows its preference for simple features, as illustrated in Figure 7.23. The overall performance is also very comparable to the ones achieved by models trained on one sample.

Sub-Sense

Macro-average scores reported in Figure 7.24 show a better generalisation capacity for the Engineered templates when the model is trained with all the samples of the pool. Good results are also achieved on other combinations but the Bigram variation has better results. We also notice no deterioration in the performances of the models as a result of combining the training data. • a relatively small number of mixed samples comes from the same compilation process, such as the case of 2 BL compared to 3 BL or DB

• a bigger number of samples are mixed, as observed for All compared to 3 BL and ML. Such a combination seems to help the models to have a more abstract prediction over the variety of the training samples.

We can also observe, as for models trained on mono-samples (see Section 7.3.2), that a model has a more stable learning curve when it is operating on long text sequences.

Finally, the convergence of models remains quick as a proportion of pages between 1/4 and 1/3 of the total number of annotated pages allows most of the models to reach performances close to the plateau.

Experiment Series 3: Testing with Unseen Dictionaries

After studying the behaviour of our models on seen samples, we want to get more insight into the performance of the best models trained with one or more dictionaries and tested with a different sample. In fact, this section sets out to answer a question that we not only asked ourselves, but one that we have been asked several times by the community: "Could the models of GROBID-Dictionaries parse dictionaries that the tool was not initially trained on?" To answer this question, we carried out the following experiments.

Experiments

We train each model using for each sample the templates that give the best performance, as observed in Tables 7.6 and 7.5, and we change the test dataset to have a sample that the model did not see during the training. We designed 5 experiments where we used samples from previous experiments. Therefore, the previous abbreviations for naming the sample remain valid and we add a dash to differentiate respectively the training from the testing datasets.

The training and testing pairs are the following:

• EEBD-DLF & DLF-EEBD: the training and the testing is performed with one monolingual dictionary in each dataset then we permute the sample, train and test again.

• 2BL-MxSp & MxSp-2BL: the same approach applies for this experiment with bilingual dictionaries.

• ML-BL: for this experiment, we want to further push the models to their limits by training on monolingual samples and testing on bilingual ones.

Discussion

Figures 7.30, 7.31, 7.32, 7.33, 7.34, 7.35 and 7.36 show very deteriorated performance of the models at different levels of the lexical parsing. The best average performances, around 50% F1-score, are achieved by the pairs ML-BL for the Dictionary Segmentation model and DLF-EEBD for the GramGrp model. Some individual good predictions for some labels are also visible such as for <orth> or <subSense> but not for all pairs of samples. This behaviour was somehow expected as theoretically, and often, either the models are trained to predict certain labels which do not exist in the testing sample or the testing sample has labels that the model was not trained to recognise. That is the case of <usg> and <xr> labels of lexical entry models where these classes of structure exist in the 2 BL sample but the model trained with the MxSp sample has no clue about their recognition as they were not among the labels to predict during the training. For the labels which are in common between the training and test sets, the difference in the logical and physical structures seems to be too hard to be contained by the features activated in the training.

The question asked at the beginning of this section seems to be straightforward to answer, given the results of these experiments: in order to achieve accurate predictions, the models need to be trained on the testing sample.

Scaling up

The scalability aspect of our models was a constant concern during the preparation of this thesis. In this section, we address the issues involved, while putting our work in context with regard to other research projects. We carried out this investigation through two series of experiments: the first addresses scalability by using different categories of data and the second explores the possibility to shift to a new generation of machine-learning techniques.

Experiment Series 4: Beyond dictionaries

Exposing our system at the early stages of its development to scholarly users led us to explore the applicability of the models we designed for similarly structured documents, mainly entry-based ones. Such "side" investigations helped us to gain a better understanding of the behaviour of our parsing models, to find ways to widen their scope and to scale up their performance. Consequently, we were driven to become familiar with common issues in the digital humanities that could be solved by a tool like GROBID-Dictionaries.

Legacy Manuscript Auction Catalogs

Introduction Manuscript Sales Catalogues (MSC)s, called also Manuscript Auction catalogues, are essential documents for Catalogue business as well as for the traceability of the movements of historical documents. An increasing number of such documents is made available to the public and researchers in related humanities fields are keen to acquire methods and tools for extracting structured information from such material. Most of these available catalogues are legacy documents that may date back to the beginning of the 19th century.

The documents in Figure 7.38 represent excerpts from an Encyclopaedic Dictionary and an MSC. One could see how both samples appear similar and that it could be difficult to differentiate between them at first sight. Such an observation pushed us to investigate, as a first step, the applicability of our lexical encoding schema to this new class of documents. To reach this target, we use the following from our modelling schema:

• <name> is used to encode headwords at the Form level to be then parsed by a future subsequent model to differentiate <persName> <addName> and <surName> constructs.

• <desc> element to encode the brief description coming after the headword of an article in both types of documents

• <def>, <bibl> and <note> to model semantic information

We customised the schema of <entry> to allow <num> to markup the ordering on entries in dictionaries and MSCs. And given the fact that the whole semantic description is related to one sense, the humanities experts in charge of the annotation chose to annotate <note> at the Sense level and not Sub-Sense.

As a second step, we wanted to experiment the ability of our lexical models to learn the recognition of constructs in such documents. In the following section, we describe the experiments we conducted to this end, based on a study focused on parsing MSCs [START_REF] Rondeau | Scaling up Automatic Structuring of Manuscript Sales Catalogues[END_REF]. encapsulate lexical and encyclopedic-like content, and are organised in an entry-based layout, is a second aspect that is confirmed by these experiments.

Address Directories

Introduction The lack of adequate information extraction tools and the enhanced usability of the GROBID-Dictionaries and its workflow attracted more researchers in the DH fields to try our models on newer categories of documents. In what follows we present preliminary work that we carried out in this direction, in collaboration with other researchers . Time Machine12 is a major large-scale project, that was launched after the interest raised by the potential of a seed project [START_REF] Kaplan | The venice time machine[END_REF], aiming at analysing and valorising the content of legacy documents for the ultimate purpose of redrawing the historical, social and economical heritage of Europe. Browsing a legacy map representing a geographical snapshot of historical cities is far from being accomplished. The difficulty is firstly due to the lack of structured data allowing a system to map a given address to a throwback location. Such information is abundantly available in dedicated paper resources, such as legacy address directories. But even digitised, mining the content of these resources remains limited due to the ad-hoc information extraction techniques that are currently employed. Experiments In the context of joint studies within a local consortium, Paris Time Machine 13 , we explored the possibility of applying GROBID-Dictionaries' models on legacy address directories "Annuaires-almanach" of Paris, made available by the French National Library14 . We were struck by the similarities between the structures of dictionaries and address directories, where both resources share a semasiological representation. In fact, such directories could be perceived as encyclopedic resources where locations are described as unique concepts.

As a first step, we tried to find a TEI encoding that converged to the modelling of our lexical parsers. In Table 7.7, we distinguish between two categories of entries. The first is reserved for each entry describing a single occupant in a unique or a shared address. In other terms, to each number in a street, one or many occupants could be assigned and an entry for each one of them. The second category of entries gathers the description blocks of a common street. An entry in this case encapsulates information like the name of the street, its length, neighbouring streets, etc.

In [START_REF] Khemakhem | Fueling Time Machine: Information Extraction from Retro-Digitised Address Directories[END_REF], we present promising preliminary results of first experiments carried out to extract macro-structures of the entries of these directories. The OCR quality was a serious obstacle for us to pursue more advanced experiments. Nevertheless, such attempts confirmed the potential of our approach and the resulting models to be applied on a larger scale for entry-based documents.

Introduction

Deep Learning (DL) is a relatively new machine-learning paradigm, based on artificial neural networks, that is constantly gaining more territory in research and industry. In fact, the growing computing capacities, the abundance of data generated daily by the Internet users, and the democratisation of libraries and best practices for data science has resulted in a large movement to deep learn almost any computational task. In the NLP field, Machine Translation [START_REF] Wu | Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation[END_REF][START_REF] Artetxe | Unsupervised Neural Machine Translation[END_REF][START_REF] Cheng | Semi-supervised learning for neural machine translation[END_REF], Part of Speech Tagging [START_REF] Plank | Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss[END_REF][START_REF] Gui | Part-of-speech tagging for twitter with adversarial neural networks[END_REF][START_REF] Meftah | A neural network model for part-of-speech tagging of social media texts[END_REF][START_REF] Martin | Dijkstra-wsa: A graph-based approach to word sense alignment[END_REF] and Named Entity Recognition [START_REF] Lample | Neural Architectures for Named Entity Recognition[END_REF][START_REF] Peters | Semi-supervised sequence tagging with bidirectional language models[END_REF][START_REF] Martin | Dijkstra-wsa: A graph-based approach to word sense alignment[END_REF][START_REF] Le | FlauBERT: Unsupervised Language Model Pre-training for French[END_REF] are typical tasks that have benefited from the deep learning race, where the last two tasks represent a certain analogy with the lexical parsing of print dictionaries.

Many readers of this thesis, like many fellow researchers, might well ask the obvious question: why did we not investigate the use of DL and gain inspiration from these analogous tasks to solve the present parsing task? In fact, several reasons lie behind our choice not to favour the investigation of such advanced machine-learning techniques. First, at the outset of our work, we did not have enough understanding of the obstacles that left state-ofthe-art methods for parsing dictionaries so limited and did not profit from simpler machine learning techniques. Second, at the time we started our investigations, the understanding of DL architectures was not as developed as it has subsequently become. In addition, deep architectures still required a heavy implementation effort and resources that we were not sure to have. So we considered that it was a bit risky to rely on such a technology for a very little explored territory. Therefore, we made our choice to build our work upon an evolving implementation, GROBID, which was based on established machine-learning techniques and offered us a minimal framework to start with and adapt.

By the time we had gathered enough knowledge about the requirements of the task we wanted to solve, we discovered several practical limitations of DL models in this sense. One could notice that most of the tasks solved by such models deal with text sequences that do not exceed a few words or sentences. With regard to the specificity of our parsing task, it is still technically impossible to use deep learning models to process batches of text (exceeding 1024 tokens) like the whole text of a dictionary to find macro structures such as the <body> in a dictionary page or <entry>s within a <body> construct. Even parsing separate dictionary articles can encounter this issue for the case of many legacy dictionaries, where the lexical description can take up several columns or pages. Parsing such large structures is vital for the first three models of our architecture, which are key models in the cascading parsing chain. For the rest of the models, we did not find enough annotated datasets that contained real-world data, as we explained at the beginning of this chapter (see Section 7.1).

The simpler technique that we used helped us to quickly explore the parsing complexity for the different lexical models. Moreover, the quick convergence and high accuracy of the models built enabled the annotation of raw data to be speeded up, by using automatic annotation and fewer manual post-corrections. Luckily, our bet on using GROBID did pay off in the end, as in parallel with our work, core functionality in this scholarly parsing system evolved and it became possible to integrate DL models.

In the following section, we present preliminary experiments with deep learning models, when it was technically possible.

Experiments

Through these experiments we aim at showing the possibility to integrate a new generation of DL models within our system. We also want to get insight into preliminary results of an example of a more advanced machine learning technique compared to the classic CRF we used to implement our sequence labelling models.

We used DeLFT 2018-2020, an advanced library implementing Deep Learning Architectures, which benefits from a native integration within GROBID. From this framework, we used an implementation of the architecture proposed by [START_REF] Lample | Neural Architectures for Named Entity Recognition[END_REF] which relies on a BidLSTM-CRF model with glove-840B word embeddings. For these experiments, we skip the evaluation of the Dictionary Segmentation and Dictionary Body Segmentation models, given the technical limitation of such deep architectures to handle the corresponding long text segments, and we present the results of the other models. To train and test the new labelers, we chose the EEBD sample given its midcomplexity and its fitness of the used word embeddings to the language and its modernity. In Figures 7.44,7.45,7.46,7.47 and 7.48, we report the evaluation of the DL and Wapiti models trained and tested with the same English dictionary. The results of these experiments show comparable performances with identical scores for one model, GramGrp, and better results achieved by Wapiti for the rest of the models. More data and different tuning of the deep models need to be further experimented in order to improve the labelling capacities of the models at a larger scale and to overcome the limitations of classic machine learning models. Such a study goes beyond the goals of this

Chapter Summary

This chapter breaks down the complexity of parsing print dictionaries from an empirical perspective. We also presented the different factors and features that impact on the performance of our lexical models and showed different scaling up alternatives.

The introduction of the machine-learning setup, with regard to the specificity of the lexical parsing task, gave an overview of aspects and details that should be taken into consideration to properly conduct machine-learning experiments. Through multiple extensive experiments, we studied the impact of feature engineering and observed how it can vary according to the lexical parsing level and the sample. The CRF models implemented in GROBID-Dictionaries showed the ability to adapt and parse all the dictionaries we presented, to varying degrees. The positive combination of samples was also an aspect that we explored along with the limitation of trained models to differentiate structures in unseen samples. Through drawing the learning curves, we also demonstrated that the technique we have used does not require a large amount of annotated data to achieve accurate parsing results. More side exploratory investigations on the application of our architecture were successful and we managed by simply applying our annotation scheme to enable the parsing of new categories of entry-based documents. Scaling up the technique to more sophisticated models was also addressed and preliminary experiments showed the possibility of integrating DL models at different levels of our architecture.

the engineering part, we managed to adapt core functionalities of GROBID, the implementation of our above-mentioned inspiration, and build on top of them a platform for training multi-level CRF models for parsing dictionaries. For the lexicographic part, we succeeded in enhancing the usability of the system for its use in the humanities. In fact, we have lightened the setup and the annotation processes and exposed the enhanced environment to real users who, consequently, provided us with valuable feedback to shape the design of our models and to spot real expert issues.

Building the parsing infrastructure went hand in hand with the definition of the final standardised output. Actually, finding the match between the final output and the designed cascading models was the guiding line of this thesis and the most challenging part, given the implementation constraints and the fact that the final output relies on the outcome of ongoing standardisation discussions.

To define the final output, we studied the most widely-used standards and formats for modelling lexical resources, namely TEI, LMF and OntoLex-Lemon, and we discussed their strengths as well as their limitations. Based on that, we presented our contribution in the shaping of two emerging schemes, TEI-Lex-0 and the new LMF, that try to unify the lexical modelling practices by following two different approaches. Our standardised output was massively inspired from these evolving schemes and this can be observed in the serialisation model we have presented in this thesis. We managed to make our output converge to a model for serialising the lexical entry macrostructure that satisfies most of the modelling of TEI-Lex-0 and LMF models. For more granular lexical parsing, we have anticipated based on the samples we encountered.

After building the architecture of lexical models, we aimed at studying their performance and behaviour at each parsing level. To this end, we needed annotated data that suited the design and the encoding of our models. Such a requirement was not easy to satisfy given the challenging aspects of lexical data annotation and real-world data. Thanks to the efficiency of the technique implemented for our models and the enhanced usability of the system, we managed to semi-automatically create sliver-standard data carrying suitable annotations.

Through an extensive experimentation, we analysed the behaviour of our CRF models as well as ways to improve their performance, chiefly through feature engineering. We also showed the impact of combining samples for the training on the generalisation capacities of such sequence labellers, that could be maintained and often improved. In addition, we have shown empirically the limits of these models in the parsing of samples that were not used in the training.

Our cascading approach showed a great ability to adapt for parsing new categories of documents. The flexibility of the models has been tested with non dictionaric resources that share certain physical and logical aspects with 8.2. Perspectives 163 print dictionaries. Simply through a proper annotation of the new samples, the implemented models demonstrated a significant genericity for parsing and encoding dictionaries and entry-based documents, mainly with encyclopaedic content. Finally, the implementation of our approach showed enough flexibility to easily integrate a more advanced category of machinelearning models, in particular DL technology. Such a scalability can unlock and speed up experiments aiming at advancing research in such a direction.

From a standardisation perspective, this thesis gives ideas and practical solutions for structuring an increasing number of raw lexical and entry-based documents into exchangeable resources. Such research work represents a concrete use case where related initiatives are exposed to real extraction scenarios to confirm choices and to open up discussion about the applicability of some lexical modelling decisions.

Perspectives

The work presented in this thesis has the potential to pave the way for massive information extraction from a wide range of digitised documents, mainly dictionaries and other entry-based documents. What we have so far achieved can be extended in several ways.

Besides carrying on the work on implementing the missing models in GROBID-Dictionaries, in parallel with the refinement of the final encoding, scaling up the performance of the built architecture can be investigated by either performing more feature engineering or integrating more DL models on more data. Regarding data, after major obstacles were clarified and overcome, this thesis can be used to develop guidelines and a protocol for largescale annotation of more diverse samples. Given the rapid convergence of our models, the generation of training data can be exponentially accelerated to find larger sets of data to enable meaningful investigations of advanced machine-learning techniques. For that, user interaction for the training functionalities could be further improved by making them more accessible to a greater number of users with more enhanced usability (e.g., implementing a user interface). Given the very promising results on Catalogs, we have already started to concretise ideas in such research directions by launching GROBID-Cat1 where we plan to make an in-depth customization of the architecture and encoding for more fine grained extraction. Such collaboration can unlock research in related fields by making more structured legacy documents available.

Being compatible with the GROBID-family tools, our models can be exploited to build new tools based on models coming from different systems. A first attempt in this direction to parse large bibliographic collections (see Figure 8.1) was successful by combining models of GROBID-Dictionaries and GROBID [START_REF] Lindemann | Retrodigitizing and Automatically Structuring a Large Bibliography Collection[END_REF]. We also plan to further enrich the output of GROBID-Dictionaries by applying models from 164 Chapter 8. Summary & Perspectives Named Entity Recognition and Disambiguation system [START_REF] Foppiano | entity-fishing: a DARIAH entity recognition and disambiguation service[END_REF]. Another avenue that is worth exploring in order to enrich the output of our system is the application of tools for downstream tasks.

  using the "Lexical Entry" segmentation model . . . . . . . . . . . . . 4.6 GROBID-Dictionaries's Cascading Architecture . . . . . . . . . 4.7 Implemented MATTER Workflow . . . . . . . . . . . . . . . . 4.8 Cascading Model Selection in GROBID-Dictionaries . . . . . . 4.9 A GROBID-Dictionaries image in a Docker container . . . . . 4.10 Training data annotation in oXygen author mode for the first model: page headers vs. page body . . . . . . . . . . . . . . . . 5.1 Entry CABBAGE in (Mueller, 1878) and examples of its two Levels of TEI encoding . . . . . . . . . . . . . . . . . . . . . . . 5.2 Different Depths of TEI encoding for Logical Structure . . . . xx 5.3 Dependencies between the LMF (2008) core and extension packages (Soria, Monachini, and Vossen, 2009) . . . . . . . . . . . . 5.4 LMF (2008) Object Diagram for Modelling the MWE "DEAD CENTER" . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 OntoLex-Lemon Core Model (McCrae et al., 2017) . . . . . . . 5.6 First Instance of OntoLex-Lemon Model for the MWE "DEAD CENTRE" . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Another Example of Instantiating OntoLex-Lemon Model for the MWE "DEAD CENTRE" . . . . . . . . . . . . . . . . . . . . 5.8 Turtle Serialisation of the MWE "DEAD CENTRE" and its Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Left: Dictionary Article in an Arabic Dictionary (Almonjid, 2014). Right: Corresponding Minimal TEI-Lex-0 Encoding . . 6.2 Minimal Encoding of Homographs as Entries in TEI-Lex-0 (Dictionary Articles from (Publishing, 2009)) . . . . . . . . . . . . . 6.3 Minimal Encoding of Homographs as Senses in TEI-Lex-0 (Dictionary Articles from (Publishing, 2009)) . . . . . . . . . . . . . 6.4 Left: POS Encoding Using <gram> Element. Right: POS Encoding Using <pos> Element . . . . . . . . . . . . . . . . . . . 6.5 Left: Extract from Mixtec-Spanish Dictionary (Alvarado, 1593) Containing Inflected Forms. Right: Corresponding Encoding in TEI-Lex-0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 TEI-Lex-0 Serialisation for <entry> Model . . . . . . . . . . . . 6.7 Example of Modelling the MWE "DEAD CENTRE" and its Components using the New LMF Core and MRD Models . . . . . . 6.8 Instance of the New Etymology Extension for the Entry "DIC-

  . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Excerpt from the Easier English Basic Dictionary (Publishing, 2009) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Excerpt from the Fang-French & French-Fang Dictionary (Galley, 1964) (FangFr) . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Excerpt from the Fang-French & French-Fang Dictionary (Galley, 1964) (FrFang) . . . . . . . . . . . . . . . . . . . . . . . . . . xxi 7.6 Mono-sample Evaluation of the Dictionary Segmentation Model Using two Classes of Templates . . . . . . . . . . . . . . . . . . 7.7 Mono-sample Evaluation of the Dictionary Body Segmentation Model Using three Classes of Templates . . . . . . . . . . 7.8 Mono-sample Evaluation of the Lexical Entry Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . 7.9 Mono-sample Evaluation of the Form Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.10 Mono-sample Evaluation of the GramGrp Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . 7.11 Mono-sample Evaluation of the Sense Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.12 Mono-sample Evaluation of the Sub-Sense Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . 7.13 Learning Curve of the Different Models Given the Number of Training Pages from DLF . . . . . . . . . . . . . . . . . . . . . . 7.14 Learning Curve of the Different Models Given the Number of Training Pages from EEBD . . . . . . . . . . . . . . . . . . . . . 7.15 Learning Curve of the Different Models Given the Number of Training Pages from MxSp . . . . . . . . . . . . . . . . . . . . . 7.16 Learning Curve of the Different Models Given the Number of Training Pages from FangFr . . . . . . . . . . . . . . . . . . . . 7.17 Learning Curve of the Different Models Given the Number of Training Pages from FrFang . . . . . . . . . . . . . . . . . . . . 7.18 Multi-sample Evaluation of the Dictionary Segmentation Model Using two Classes of Templates . . . . . . . . . . . . . . . . . . 7.19 Multi-sample Evaluation of the Dictionary Body Segmentation Model Using three Classes of Templates . . . . . . . . . . . . . 7.20 Multi-sample Evaluation of the Lexical Entry Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . 7.21 Multi-sample Evaluation of the Form Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.22 Multi-sample Evaluation of the GramGrp Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . 7.23 Multi-sample Evaluation of the Sense Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.24 Multi-sample Evaluation of the Sub-Sense Model Using three Classes of Templates . . . . . . . . . . . . . . . . . . . . . . . . 7.25 Learning Curve of the Different Models Given the Number of Training Pages from 2 BL . . . . . . . . . . . . . . . . . . . . . 7.26 Learning Curve of the Different Models Given the Number of Training Pages from 3 BL . . . . . . . . . . . . . . . . . . . . . 7.27 Learning Curve of the Different Models Given the Number of Training Pages from DB . . . . . . . . . . . . . . . . . . . . . . 7.28 Learning Curve of the Different Models Given the Number of Training Pages from ML . . . . . . . . . . . . . . . . . . . . . . xxii 7.29 Learning Curve of the Different Models Given the Number of Training Pages from All . . . .

FIGURE 2 . 1 :

 21 FIGURE 2.1: Onomasiological Approach

FIGURE 2

 2 FIGURE 2.2: Semasilogical Approach

FIGURE 2 . 3 :

 23 FIGURE 2.3: Excerpt from Basnage Dictionary[START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF] of the entry ABORDER

FIGURE 2

 2 FIGURE 2.4: Classification of Dictionaric Resources

  FIGURE 3.1: Metadata text introduced by a PDF engine in the text of dictionary pages (Publishing, 2009)

  FIGURE 3.2: Digitised entry in Basnage dictionary[START_REF] Furetière | Dictionnaire Universel, contenant généralement tous les Mots François tant vieux que modernes[END_REF]

FIGURE 3

 3 FIGURE 3.3: Left: Entry ACT in (Publishing, 2009). Right: Entries ABUSE (Publishing, 2009)

  FIGURE 3.4: Excerpts from different dictionaries with different logical structure

FIGURE 3 . 7 :

 37 FIGURE 3.7: Illustration of the Hyper-Level zones in a scientific paper recognised by Header Segmentation Model in GROBID

FIGURE 4 . 1 :

 41 FIGURE 4.1: First and second segmentation levels of a dictionary page(Larousse, 1972)

FIGURE 4

 4 FIGURE 4.3: Left: Excerpt from Bibliographic References Section in[START_REF] Khemakhem | Enhancing Usability for Automatically Structuring Digitised Dictionaries[END_REF]. Right: Excerpt of Lexical Entries in(Publishing, 2009)

FIGURE 4 . 4 :

 44 FIGURE 4.4: Excerpt from GROBID's architecture 7

  FIGURE 4.5: Sequence labelling a dictionary article(Larousse, 1972) using the "Lexical Entry" segmentation model

Chapter 4 .

 4 Lexical Models for Parsing Print Dictionaries

FIGURE 4

 4 FIGURE 4.6: GROBID-Dictionaries's Cascading Architecture

  .7) introduced by Pustejovsky and Stubbs, 2012. Projected onto GROBID-Dictionaries and the processing of lexical resources, the individual steps are as follows:

FIGURE 4 . 7 :

 47 FIGURE 4.7: Implemented MATTER Workflow

FIGURE

  FIGURE 4.8: Cascading Model Selection in GROBID-Dictionaries

  , to linking the corresponding GitHub and 52 Chapter 4. Lexical Models for Parsing Print Dictionaries Docker Hub repositories coupled with activating the automatic build to synchronise the image after each update of the code.

  FIGURE 4.9: A GROBID-Dictionaries image in a Docker container

  allow for the visual annotating of XML files (see Figure 4.10 for an example). 54 Chapter 4. Lexical Models for Parsing Print Dictionaries

FIGURE 4 . 10 :

 410 FIGURE 4.10: Training data annotation in oXygen author mode for the first model: page headers vs. page body

  FIGURE 5.1: Entry CABBAGE in (Mueller, 1878) and examples of its two Levels of TEI encoding

FIGURE 5 . 2 :

 52 FIGURE 5.2: Different Depths of TEI encoding for Logical Structure

8

  https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-entryFree.html 9 https://www.tei-c.org/release/doc/tei-p5-doc/en/html/examples-orth.html 10 https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-superEntry.html 11 https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-etym.html5.3. LMF (2008) 63

FIGURE 5

 5 FIGURE 5.3: Dependencies between the LMF (2008) core and extension packages[START_REF] Soria | Wordnet-LMF: fleshing out a standardized format for wordnet interoperability[END_REF]

FIGURE

  FIGURE 5.4: LMF (2008) Object Diagram for Modelling the MWE "DEAD CENTER"

  FIGURE 5.5: OntoLex-Lemon Core Model (McCrae et al., 2017)

FIGURE 5

 5 FIGURE 5.6: First Instance of OntoLex-Lemon Model for the MWE "DEAD CENTRE"

  Figure 5.7 depicts such a different design.

FIGURE 5 . 7 :

 57 FIGURE 5.7: Another Example of Instantiating OntoLex-Lemon Model for the MWE "DEAD CENTRE"

Figure 5 .

 5 8 illustrates a Turtle serialisation of the entry, modelled in Figure 5.6, by following the OntoLex-Lemon vocabulary. Note that every line represents a triplet expressed in the diagram. The referencing to the internal modules is manifested in, for example, the definition of components of the MWE: centre_component a decomp:Component There are also instances of external links to object properties, such as the one to LexInfo in: lexinfo:partOfSpeech a lexinfo:adjective

FIGURE 5

 5 FIGURE 5.8: Turtle Serialisation of the MWE "DEAD CENTRE" and its Components

FIGURE 6 . 1 :

 61 FIGURE 6.1: Left: Dictionary Article in an Arabic Dictionary (Almonjid, 2014). Right: Corresponding Minimal TEI-Lex-0 Encoding

FIGURE 6 . 2 :

 62 FIGURE 6.2: Minimal Encoding of Homographs as Entries in TEI-Lex-0 (Dictionary Articles from (Publishing, 2009))

FIGURE 6

 6 FIGURE 6.3: Minimal Encoding of Homographs as Senses in TEI-Lex-0 (Dictionary Articles from (Publishing, 2009))

FIGURE 6

 6 FIGURE 6.4: Left: POS Encoding Using <gram> Element. Right: POS Encoding Using <pos> Element

FIGURE 6

 6 FIGURE 6.5: Left: Extract from Mixtec-Spanish Dictionary (Alvarado, 1593) Containing Inflected Forms. Right: Corresponding Encoding in TEI-Lex-0

FIGURE 6

 6 FIGURE 6.6: TEI-Lex-0 Serialisation for <entry> Model

FIGURE 6 . 7 :

 67 FIGURE 6.7: Example of Modelling the MWE "DEAD CENTRE" and its Components using the New LMF Core and MRD Models

FIGURE 6 . 9 :

 69 FIGURE 6.9: LMF Serialisation for <entry> Model

FIGURE 6 . 10 :

 610 FIGURE 6.10: Example of LMF Modelling of Homographs with two Form Objects

FIGURE 6 .

 6 FIGURE 6.12: Etymological Constructs in (Mueller, 1878)

FIGURE 6 .

 6 FIGURE 6.13: Minimal TEI Encoding as an Output of the Etym/Quote Model

FIGURE 6 .

 6 FIGURE 6.16: Final Serialisation of the <entry> Model in GROBID-Dictionaries

FIGURE 7 . 2 :

 72 FIGURE 7.2: Excerpt from the Dictionnaire de la LangueFrançaise Dictionary[START_REF] Littré | Dictionnaire de la langue française[END_REF]

FIGURE 7 . 4 :

 74 FIGURE 7.4: Excerpt from the Fang-French & French-Fang Dictionary (Galley, 1964) (FangFr)

FIGURE 7 . 5 :

 75 FIGURE 7.5: Excerpt from the Fang-French & French-Fang Dictionary (Galley, 1964) (FrFang)

FIGURE 7 .

 7 FIGURE 7.6: Mono-sample Evaluation of the Dictionary Segmentation Model Using two Classes of Templates

FIGURE 7 .

 7 FIGURE 7.7: Mono-sample Evaluation of the Dictionary Body Segmentation Model Using three Classes of Templates

  The evolution of the learning of the different models illustrated in Figure7.13 shows a convergence and a stability for a number of pages between 35 and 40 for 6 models. The seventh model, Dictionary Segmentation, reaches such convergence after approximately twice the number of pages. This is due to the fact that the scope of such a model requires more pages to cover most of the variations in the structure and layout of the pages that have a dense text and slightly changing clues.

FIGURE 7 . 13 :

 713 FIGURE 7.13: Learning Curve of the Different Models Given the Number of Training Pages from DLF

FIGURE 7 .

 7 FIGURE 7.14: Learning Curve of the Different Models Given the Number of Training Pages from EEBD

FIGURE 7 .

 7 FIGURE 7.16: Learning Curve of the Different Models Given the Number of Training Pages from FangFr

FIGURE 7 . 17 :

 717 FIGURE 7.17: Learning Curve of the Different Models Given the Number of Training Pages from FrFang

FIGURE 7 . 23 :

 723 FIGURE 7.19: Multi-sample Evaluation of the Dictionary Body Segmentation Model Using three Classes of Templates

FIGURE 7 .

 7 FIGURE 7.25: Learning Curve of the Different Models Given the Number of Training Pages from 2 BL

FIGURE 7 .

 7 FIGURE 7.30: Evaluation of the Dictionary Segmentation Model Using the Best Templates and Testing with Unseen Sample

FIGURE 7 .

 7 FIGURE 7.31: Evaluation of the Dictionary Body Segmentation Model Using the Best Templates and Testing with Unseen Sample

FIGURE 7 .

 7 FIGURE 7.33: Evaluation of the Form Model Using the Best Templates and Testing with Unseen Sample

FIGURE 7 .

 7 FIGURE 7.34: Evaluation of the GramGrp Model Using the Best Templates and Testing with Unseen Sample

FIGURE 7 .

 7 FIGURE 7.35: Evaluation of the Sense Model Using the Best Templates and Testing with Unseen Sample

FIGURE 7 .

 7 FIGURE 7.36: Evaluation of the Sub-Sense Model Using the Best Templates and Testing with Unseen Sample

  FIGURE 7.37: Resemblance between MSCs and Encyclopedic Dictionaries

FIGURE 7 .

 7 FIGURE 7.38: Encoding of an MSC

FIGURE 7 .

 7 FIGURE 7.39: Evaluation of the Dictionary Segmentation Model Trained and Tested with MSCs and Dictionaries

FIGURE 7 .

 7 FIGURE 7.40: Evaluation of the Dictionary Body Segmentation Model Trained and Tested with MSCs and Dictionaries

FIGURE 7 .

 7 FIGURE 7.41: Evaluation of the Lexical Entry Model Trained and Tested with MSCs and Dictionaries

FIGURE 7 .

 7 FIGURE 7.42: Evaluation of the Form Model Trained and Tested with MSCs and Dictionaries

FIGURE 7 .

 7 FIGURE 7.43: Evaluation of the Sense Model Trained and Tested with MSCs and Dictionaries

FIGURE 7 .

 7 FIGURE 7.44: Evaluation of the Lexical Entry Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers

FIGURE 7 .

 7 FIGURE 7.46: Evaluation of the GramGrp Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers

FIGURE 7 .

 7 FIGURE 7.47: Evaluation of the Sense Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers

FIGURE 7 .

 7 FIGURE 7.48: Evaluation of the Sub-Sense Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers

FIGURE 8 . 1 :

 81 FIGURE 8.1: Excerpt from the Bibliography Collection (Wiegand, 2014) used for Experimenting the Combination of GRO-BID and GROBID-Dictionaries Models

  2.1 Onomasiological Approach . . . . . . . . . . . . . . . . . . . . 2.2 Semasilogical Approach . . . . . . . . . . . . . . . . . . . . . . 2.3 Excerpt from Basnage Dictionary (Furetière, 1701) of the entry ABORDER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Classification of Dictionaric Resources . . . . . . . . . . . . . . 3.1 Metadata text introduced by a PDF engine in the text of dictionary pages (Publishing, 2009) . . . . . . . . . . . . . . . . . . . 3.2 Digitised entry in Basnage dictionary (Furetière, 1701) . . . . . 3.3 Left: Entry ACT in (Publishing, 2009). Right: Entries ABUSE (Publishing, 2009) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Excerpts from different dictionaries with different logical structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Text Sequence Labelling . . . . . . . . . . . . . . . . . . . . . . 3.6 Example of bilingual dictionaries processed by

  . . . . . . . . . . . . . . . . . . . . 6.9 LMF Serialisation for <entry> Model . . . . . . . . . . . . . . . 6.10 Example of LMF Modelling of Homographs with two Form Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 Example of the new LMF Serialisation for the MWE "DEAD CENTER" and its Components . . . . . . . . . . . . . . . . . . 6.12 Etymological Constructs in

  . . . . . . . . . . . . . . . . . . 7.30 Evaluation of the Dictionary Segmentation Model Using the Best Templates and Testing with Unseen Sample . . . . . . . . 7.31 Evaluation of the Dictionary Body Segmentation Model Using the Best Templates and Testing with Unseen Sample . . . . . . 7.32 Evaluation of the Lexical Entry Model Using the Best Templates and Testing with Unseen Sample . . . . . . . . . . . . . 7.33 Evaluation of the Form Model Using the Best Templates and Testing with Unseen Sample . . . . . . . . . . . . . . . . . . . . 7.34 Evaluation of the GramGrp Model Using the Best Templates and Testing with Unseen Sample . . . . . . . . . . . . . . . . . 7.35 Evaluation of the Sense Model Using the Best Templates and Testing with Unseen Sample . . . . . . . . . . . . . . . . . . . . 7.36 Evaluation of the Sub-Sense Model Using the Best Templates and Testing with Unseen Sample . . . . . . . . . . . . . . . . . 7.37 Resemblance between MSCs and Encyclopedic Dictionaries . 7.38 Encoding of an MSC . . . . . . . . . . . . . . . . . . . . . . . . 7.39 Evaluation of the Dictionary Segmentation Model Trained and Tested with MSCs and Dictionaries . . . . . . . . . . . . . . . 7.40 Evaluation of the Dictionary Body Segmentation Model Trained and Tested with MSCs and Dictionaries . . . . . . . . . . . . . 7.41 Evaluation of the Lexical Entry Model Trained and Tested with MSCs and Dictionaries . . . . . . . . . . . . . . . . . . . . . . . 7.42 Evaluation of the Form Model Trained and Tested with MSCs and Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . 7.43 Evaluation of the Sense Model Trained and Tested with MSCs and Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . 7.44 Evaluation of the Lexical Entry Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers . . . . . . . . 7.45 Evaluation of the Form Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers . . . . . . . . . . . . 7.46 Evaluation of the GramGrp Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers . . . . . . . . 7.47 Evaluation of the Sense Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers . . . . . . . . . . . . 7.48 Evaluation of the Sub-Sense Model Trained and Tested with EEBD Using Deep Learning and Wapiti Labelers . . . . . . . . 8.1 Excerpt from the Bibliography Collection (Wiegand, 2014) used for Experimenting the Combination of GROBID and GROBID-Dictionaries Models . . . . . . . . . . . . . . . . . . . . . . . . . Selecting Lexical Entry Model for Parsing Sub-Entries . . . . . 185 B.7 State after of Full Parsing Selection . . . . . . . . . . . . . . . . 186 Some of the Dictionaries experimented with in some sessions of the workshop series . . . . . . . . . . . . . . . . . . . . . . . 6.1 Main labels of the etymology extension . . . . . . . . . . . . . 7.1 Page Sampling Statistics (Bowers, Khemakhem, and Romary, 2019) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Field Level Evaluation of the Dictionary Segmentation Model 7.3 Field Level Evaluation of the Dictionary Body Segmentation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Field Level Evaluation of the Lexical Entry Model . . . . . . . 7.5 Summary of the First Series of Experiments . . . . . . . . . . . 7.6 Summary of the Second Series of Experiments . . . . . . . . . 7.7 Proposed TEI Encoding of Entries in Address Directories (Didot-

B.1 Selecting Morphological and Grammatical Models . . . . . . . B.2 Selecting Semantic Models . . . . . . . . . . . . . . . . . . . . . B.3 Selecting Etymological Models . . . . . . . . . . . . . . . . . . B.4 Selecting Lexical Entry Model for Parsing Related Entries . . . B.5 Selecting Cross-Reference Models . . . . . . . . . . . . . . . . . xxiii B.6 xxv List of Tables 4.1 Bottin, 1901) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

TABLE 6 .

 6

1: Main labels of the etymology extension

TABLE 7 . 1 :

 71 Page Sampling Statistics

	Training	Evaluation

TABLE 7 .

 7

5: Summary of the First Series of Experiments

  Chapter 7. GROBID-Dictionaries in Action Dictionary Body SegmentationFigure 7.19 shows the Bigram templates taking the lead from the Engineered ones experimented with the mono-sample training. Average score is reached when the two samples containing the <dictScrap> label (i.e. 2 BL) are mixed. The model a little confused when only one sample is added to the training (i.e. 3 BL). However, trained with more samples (i.e. ALL) enables good performances to be achieved again.Lexical EntryAs for the mono-sample experiments, this model, with text sequences from different sizes and multiple classes to predict, gives its best performances with the Bigram and Engineered templates. The latter outperform the former on the 2 BL, 3 BL and DB combinations. The generalization of the model over all samples is identical for both categories of templates. The most interesting aspect at this lexical parsing level, observed with the Engineered Templates, is that the F1-score obtained with all samples is identical to, and often better than individual scores obtained with the best model from mono-sample training.

https://archive.org/search.php?query=dictionaries

https://galica.bnf.fr/conseils/content/dictionnaire

https://en.oxforddictionaries.com/definition/parse

https://www.abbyy.com/en-eu/finereader/

https://github.com/kermitt2/grobid/blob/master/grobid-core/src/main/ java/org/grobid/core/features/FeaturesVectorSegmentation.java

https://traces1.inria.fr/grobid-dictionaries/

https://github.com/kermitt2/grobid

researchgate.net

https://hal.archives-ouvertes.fr/

https://www.mendeley.com

https://home.cern/

the full architecture can be found in GROBID documentation https://grobid. readthedocs.io/en/latest/grobid-04-2015.pdf

for the sake of conciseness, Figure4.6 represents only some of the possible labels supported by certain models. The full list of labels is given in the descriptive text of the figure.

https://wapiti.limsi.fr/manual.html

https://github.com/MedKhem/grobid-dictionaries/wiki/How-to-Annotate%3F

https://traces1.inria.fr/grobid-dictionaries/

https://gradle.org

https://maven.apache.org

https://www.docker.com

https://github.com/MedKhem/grobid-dictionaries/blob/master/Dockerfile

https://hub.docker.com/r/medkhem/grobid-dictionaries/

http://community.coherentpdf.com

https://www.oxygenxml.com/

http://www.relaxng.org

https://digilex.hypotheses.org/250

https://lexmc18.sciencesconf.org/resource/page/id/3

https://cahier.hypotheses.org/3640

https://www.eventbrite.com/e/sadilar-grobid-dictionaries-workshoppretoria-tickets-49730494247

[START_REF]Punctuation information[END_REF] https://www.eventbrite.com/e/sadilar-grobid-dictionaries-workshoppotchefstroom-tickets-

49732031846 24 https://www.eventbrite.com/e/sadilar-grobid-dictionaries-workshopstellenbosch-

tickets-49731001765 25 see instructions at https://github.com/MedKhem/grobid-dictionaries/wiki

/ Docker_Instructions26 A more detailed description of the conditions of the experiment can be found in a blogpost at https://digilex.hypotheses.org/250 and https://digilex.hypotheses.org/ category/posts as shared by several participants.

https://goo.gl/Zt2gDy

https://tei-c.org/release/doc/tei-p5-doc/en/html/ST.html#STMA

https://tei-c.org/release/doc/tei-p5-doc/en/html/ref-p.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-lb.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-hi.html

https://tei-c.org/release/doc/tei-p5-doc/en/html/ref-entry.html

https://gitlab.com/xlhrld/retro-dict

https://www.wiktionary.org/

https://www.iso.org/committee/297592.html

http://linguistic-lod.org/

https://www.w3.org/community/ontolex/

An explicit note from the consortium in the community page clarifies its position regarding the work of the Ontolex Community group : "Community Groups are proposed and run by the community. Although W3C hosts these conversations, the groups do not necessarily represent the views of the W3C Membership or staff."

https://www.lexinfo.net/

https://www.w3.org/community/ontolex/wiki/Morphology

https://acoli-repo.github.io/ontolex-frac/

https://www.w3.org/TR/turtle/

https://rdf-translator.appspot.com/

http://www.easyrdf.org/converter

https://elex.is

http://www.elexicography.eu/working-groups/working-group-2/objectives/

https://www.dariah.eu/activities/working-groups/lexical-resources/

http://www.parthenos-project.eu

https://elex.is

https://github.com/DARIAH-ERIC/lexicalresources/issues

https://dariah-eric.github.io/lexicalresources/pages/TEILex0/TEILex0. html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-re.html

https://github.com/DARIAH-ERIC/lexicalresources/issues/43

https://tei-c.org/release/doc/tei-p5-doc/en/html/ref-gramGrp.html

https://github.com/DARIAH-ERIC/lexicalresources/issues/31

https://tei-c.org/release/doc/tei-p5-doc/en/html/ref-orth.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-def.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-cit.html

https://tei-c.org/release/doc/tei-p5-doc/en/html/ref-orth.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-pron.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-seg.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-body.html

https://www.tei-c.org/release/doc/tei-p5-doc/de/html/ref-fw.html

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-ab.html

FIGURE 6.15: Internal Serialisation of GROBID-Dictionaries's Models

https://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-note.html

There are two main conventions for dataset splitting ratios. First,

2/3 for training and validation and 1/3 for testing. Second, 80% for training and validation and 20% for testing. The choice of these conventional ratios depends on the amount of data available and the preferences of designer of the machine learning experiment, as the first convention is more challenging for a trained model.

In this chapter, the labels presented in the evaluation of the models are of the internal serialisation of the architecture and not the final one

https://github.com/MedKhem/grobid-dictionaries_data

https://gallica.bnf.fr/ark:/12148/bpt6k5406710m

https://www.littre.org

https://github.com/MedKhem/grobid-dictionaries/wiki/Docker_Instructions

Mixtepec-Mixtec is an Otomonguean language spoken by roughly 9,000 -10,000 people, and in addition to the native communities in Mexico, it is also spoken by small communities of people spread over several cities in the United States. The Spanish variation of this dictionary is Castilian.

Fang is a language spoken by around 1 million people in several central African countries

Training data in this context also includes the development set. The GROBID training module automatically manages the split between the two datasets

all evaluation figures in this chapter are generated using https://github.com/ MedKhem/grobid-benchmark

We have noticed that some experiments do not have always the exact same results, when they are run on different occasions. The difference is estimated to be around 1.5% in the F1-Score. As we discovered such a minor anomaly at the very end of this thesis, we did not have enough time to find the exact origin of such a behaviour in the WAPITI implementation we used. But we suspect the dynamic allocation of threads, used to perform the training, to be one main source of this inaccuracy. Therefore, our comparison for the outcome of the different templates takes this into consideration by tolerating such differences in the selection of the best template category

http://timemachineproject.eu

https://paris-timemachine.huma-num.fr/groupe-adresses-et-annuaires/

https://gallica.bnf.fr/ark:/12148/bpt6k9763088f/f1198.image

https://github.com/MedKhem/grobid-cat

Acknowledgements

Acknowledgements xi

As it can be noticed, two alternatives are provided to encode grammatical 6.2. TEI-Lex-0

This thesis has been supported by the European PARTHENOS project, as a part of the Work Package 4 under grant No. 654119, and by the BasNum project under the ANR grant No. ANR-18-CE38-0003.

This work would not have seen the light without the support of people who believed in my personal and scientific capacities. It goes without saying that I had a great chance to have Prof. Dr. Laurent Romary as a PhD advisor.

Chapter 6. Novel Standardised Schemes for Encoding Dictionaries Entry "DICTIONARY" in [START_REF] Ernest | A Comprehensive etymological dictionary of the English language[END_REF] Chapter 7. GROBID-Dictionaries in Action

Feature engineering Experiments

The first set of experiments focuses on investigating the stability of the performance of the templates experimented with separate samples, by training the labelling models on different dictionaries at once.

Experiments

Dictionary Segmentation As illustrated in Figure 7.18, the Bigram templates remain the best variation of this labelling level, despite the relatively good performances of the Unigram templates. None of the combinations of samples seems to harm the performance of the models.

Discussion

We sum up the results of this series of experiments in The main conclusion to be drawn from these experiments is that, in most cases, mixing samples does not harm the learning of a model and can often boost it. Such a combination seems to be healthy for the lexical models to overcome potential over-fitting to data coming from one source. Compared with the analogous table for mono-sample experiments, in Table 7.6 we notice changes of the best templates for each model. The Unigram templates appear to be the most suitable for parsing lexical structures in relatively short text sequences with models having few classes to predict. Engineered templates are the best for the Lexical Entry model over most of the combined samples and remain in tight competition with the Bigram variation. The latter dominates the competition with other templates for the rest of the parsing levels.

Learning Curve

After investigating the impact of mixing samples on the feature engineering choices, we aim at studying the impact on the learning curve of each model for the same combinations used in the previous experiments.

Experiments

The same design of the experiments with mono-sample training was followed to generate the learning curve for the different combinations of these experiments. We simply summed the page numbers of analogous batches and selected the best models observed in the previous series of experiments. The learning curves per combination of samples are presented in Figures 7.25,7.26,7.27,7.28 and 7.29.

Discussion

Given the limited number of samples we used for this experiment, we can not be sure about the origin of some behaviours when certain categories of

Chapter 7. GROBID-Dictionaries in Action

Experiments To study the performance of the models on MSCs, we carried out two experiments. The first, named Cat, is focused on training and testing the models with different catalogues, whereas the second, named Cat+Dict, makes use of the same catalogues as well as all the dictionaries in our pool to train and test the models.

We used 5 models only after we excluded the GramGrp and Sub-Sense models, since the information in the MSCs did not require an activation of these two models. Figures 7.39,7.40,7.41,7.42 and 7.43 present the evaluation of the different models with the Bigram templates, which gave the best results. In these Figures, we also recall the results of the models trained with only the combination of all dictionaries, already reported in previous experiments as All. Since the focus of these experiments is on the MSCs, we report the F1-score of individual labels that occur in these documents and exclude the tags which occur only in dictionaric samples.

Chapter 8

Summary & Perspectives

Summary

Parsing print dictionaries using lexical models that are able to dismantle the multi-fold complexity of such resources and the generation of standardised output enabling a maximum exchange, are two tasks that we addressed in this thesis. We presented a novel approach relying on standard-based lexical models that build on the leading encoding standards and extends the hitherto modest state-of-the-art of parsing techniques.

A standard-based lexical model, as presented in this thesis, can be assimilated as an automatic labeller:

• that uses a supervised machine-learning technique • based on typographic and textual features • to learn the recognition of structures in text sequences

• and serialises the recognised constructs into a standardised resource Our initial goals were to:

• Understand why parsing dictionaries did not benefit from the great advances in machine-learning techniques

• Find an encoding that can generalise over most of dictionaries

• Build models that can fit this encoding and accelerate the structuring of print dictionaries

We tackled the analysis of the challenges for such a parsing task by presenting the specificity of digital-born and digitised documents, and the varieties in the logical and physical structures of print dictionaries. We also presented related work, which, in general, is limited and experimental, and our inspiration from an existing successful implementation in an infrastructure for parsing bibliographic information.

We have accordingly implemented an end-to-end system, called GROBID-Dictionaries, which relies on cascading machine-learning models to orchestrate the analysis and the extraction of lexical structures from an input dictionary. We presented the different challenges that we encountered to build our models, which required both engineering and lexicographic skills. For