Keywords: Système de recommandation, ontologie, alignement d'ontologies, sourcing. 5 Recommender system, ontology, ontology alignment, sourcing domain. 7

Cette thèse de doctorat CIFRE s'inscrit dans le cadre d'un projet de recherche collaboratif entre le laboratoire I3S de l'Université Côte d'Azur et la société Silex et aborde le domaine des systèmes de recommandation. De nos jours, les systèmes de recommandation sont devenus populaires et se sont répandus dans de nombreux domaines d'application (recommandation de films, actualités, restaurants, produits, services financiers, etc). Ils fournissent des suggestions aux utilisateurs pour les aider dans leur recherche d'information, en essayant de répondre au mieux à leurs besoins. Silex est une start-up qui développe un outil de sourcing Software-as-a-Service permettant aux entreprises de fournir une description de leurs activités professionnelles, de leurs offres et/ou des services qu'elles recherchent en langue naturelle (actuellement le français). Dans ce contexte, l'objectif de cette thèse est de proposer un système d'aide à la décision en exploitant les connaissances sémantiques extraites à partir des descriptions textuelles des demandes de prestation et des prestataires, afin de recommander des prestataires pertinents pour une demande de prestation.

Les contributions de cette thèse sont les suivantes. Premièrement, nous avons proposé un vocabulaire pour le domaine du sourcing afin d'annoter sémantiquement les descriptions textuelles des prestataires et des demandes de prestation. Ce vocabulaire a été construit en réutilisant et en intégrant des vocabulaires existants. Deuxièmement, nous avons proposé une méthode d'alignement automatique afin d'établir la correspondance entre différents concepts des vocabulaires considérés. Cette approche se base sur des règles exploitant l'espace des plongements lexicaux et des mesures sur des groupes d'étiquettes pour découvrir les relations entre concepts. Troisièmement, nous avons proposé un algorithme d'extraction des entités nommées à partir des descriptions textuelles des demandes de prestation et des prestataires et un algorithme d'annotation sémantique de ces descriptions, basé sur le liage des entités extraites avec les concepts du vocabulaire défini. Quatrièmement, nous avons proposé un algorithme de recommandation de prestataires qui exploite ces annotations sémantiques. Finalement, nous avons étudié l'apport de l'utilisation de connaissances ontologiques afin d'améliorer notre système d'aide à décision pour le domaine du sourcing.

To your soul my dear daddy, You never left my mind, I miss you ...

To Anas, Ahmed and Tasnime

Je remercie aussi toute l'équipe produit pour les échanges que nous avons eus, et surtout pour leur travail d'annotation manuelle qui était précieux pour la validation de mes travaux. Merci à ma chère famille et mes proches de me soutenir, de m'encourager et m'entourer d'amour depuis longtemps. J'ai une pensée particulière à ma mère pour son soutien, son amour, et son appui moral malgré la distance qui me sépare d'elle. Merci à mon frère, ma soeur et leurs petites familles pour leur amour, et ma belle-famille qui m'a beaucoup encouragée, et particulièrement ma belle-mère qui n'a pas hésité à faire des sacrifices pour me soutenir pendant ces années.

Enfin, je ne pourrai jamais exprimer assez de gratitude à ceux qui m'apportent l'amour, la tendresse et le réconfort dont j'ai besoin pour avancer. Je remercie mon bien-aimé Anas d'avoir toujours cru en moi, de toujours faire de son mieux pour me soutenir, m'encourager et me donner la force de surmonter les difficultés rencontrées pendant ma thèse. Merci d'être si patient, si arrangeant et d'avoir toujours su être là. Je remercie mes petits trésors, Ahmed et Tasnime, d'être toujours ma source de bonheur, d'amour et de motivation. Tout seul, on va plus vite, ensemble on va plus loin. [START_REF] Euzenat | Ontology matching[END_REF]

Context

Nowadays, companies are collecting masses of data through multiple channels with multiple formats, and facing big challenges to analyze them and create value. To deal with that, companies are accelerating the integration of digital technologies, especially Artificial Intelligence (AI) technology that is revolutionizing their processes.

The sourcing domain is one of the most promising domains for AI. Used in the procurement, sourcing is the action of searching, identifying and evaluating an ad hoc supplier, in order to meet an identified need (for goods or services) formulated by a company or a service or a department of that company. The evaluation of providers is based on multiple criteria, such as cost, deadline, innovation, quality, proximity, response capacity on needs, and production capacity. Traditional sourcing methods are based on using online paid databases or specialized databases in a specific domain, and on trade shows, federations, and unions. Unfortunately, with today's fastpaced and ever-changing environment, manual sourcing procedures often lead to a lack of transparency, incorrect supplier lists, and inconsistent purchasing management. These drawbacks involve big losses for both buyers and suppliers.

The Silex company1 develops an e-sourcing tool with the ultimate goal of providing a framework to steer and optimize an lookup and collection of supplier offers following a model facilitating their analysis. Using this framework, the purchasers will collect information on providers, their products or services and prices. Based on this information and the purchasers' requirements, reliable comparisons will be made to suggest a list of potential providers.

Automatizing the e-sourcing process has multiple benefits: on the one hand, the purchasing department has a better follow-up, real time savings on order and delivery times, and more efficient and transparent purchasing; on the other hand, the provider could boost their activities, improve collaboration, increase their revenue, and increase transparent exchange.

This CIFRE thesis takes place in the context of a collaborative research project between the Silex company and the WIMMICS research team2 from Inria and I3S at Université Côte d'Azur, aiming to develop IA methods and tools to be integrated within the Silex sourcing platform. This platform supports two main user communities: (i) service requestors and (ii) providers. Users provide a textual description of their professional activities, their offers and/or services in the French language. The platform automatically analyzes these textual descriptions in order to better and faster evaluate opportunities, with more targeted sourcing.

The goal of this thesis is to design a recommender system (RS) relying on the prediction of relevant providers that are likely to be of interest for a service request based on contextual analysis of these data. Two main challenges have been identified:

1. How can we analyze expressions in natural language (national localized context specific to each country)?

2. How can we model these expressions in a formal representation allowing to reason about them?

Our research focus

In this thesis, we focus on the introduction of semantics into the Silex platform by conceptualizing the knowledge involved in sourcing, in order to be able to automatically reason on service requests and providers descriptions, and improve the recommender process.

Domain knowledge modeling for sourcing data representation

A first ontological engineering work consists of the design of a semantic repository to describe the sourcing data. This involves exploiting heterogeneous sources such as classification or metadata repositories to build it. There are two main challenges in this first part:

1. Define a precise and complete domain knowledge that describes the sourcing domain, in phase with the Silex data.

2. Integrate the multilingual dimension of the data sources used into the domain knowledge to position the service request or provider in both French and European markets.

Design and adaptation of algorithms for knowledge extraction

This involves the implementation of Natural Language Processing (NLP) algorithms to identify the relevant key phrases to be extracted from the textual descriptions of providers and service requests. Our challenges in this part are:

1. Deal with short descriptions and distinguish between the real need and the general context of the service request.

Recommend providers based on the formal representation of descriptions

The aim is to define a recommender approach based on the matching of formal representations of providers and service requests, by exploiting the semantic proximity captured by the designed domain knowledge (for instance, a company with a hiring need will be interested in service providers presenting themselves as recruiters, but perhaps also in others presenting themselves as headhunters, specialized in the recruitment of highly qualified people). The provider recommender then involves developing evaluation metrics between an expressed need and the matching providers.

Our methodology

In our recommender scenario, when a new service request is published on the Silex platform, a set of the most relevant providers for it should be automatically suggested, either directly to the user or to the Silex sale's staff for validation. Figure 1.1 shows the proposed overall workflow. It comprises five steps: (i) construction of a vocabulary for the sourcing domain, (ii) entity recognition from the textual descriptions of service requests and providers, (iii) entity management, (iv) vector representation of service requests and providers, and (v) recommender algorithm.

In the following, we summarize the methodology we developed to achieve this goal.

Building a vocabulary for the sourcing domain

The preliminary step of our approach is the construction of a vocabulary to capture the sourcing knowledge. Our proposal relies on the use of domain knowledge that can be captured into a thesaurus or an ontology. In the following, we will refer to vocabulary indistinctly as a thesaurus or an ontology. The aim is to semantically annotate the textual descriptions of companies and service requests with five types of knowledge: (i) skills, (ii) occupations, (iii) products (i.e. goods and services), and (v) business activities. We built a modular sourcing vocabulary by identifying and combining several relevant standard metadata repositories.

Proposal of an ontology alignment approach

The second step is the proposal of an alignment method and its development to automatically align the above-mentioned vocabularies, which where partially overlapping, in order to get a complete vocabulary.

Proposal of an approach for sourcing named entity recognition and linking

The third step is the definition of an NLP algorithm to extract and link the relevant knowledge pieces, i.e., occupations, skills, products, and business activities, from the textual descriptions, i.e., service requests and company descriptions.

Proposal of a recommender algorithm

The final step is to propose a formal representation of each service request or provider that summarizes the semantics of the entities extracted from their descriptions and define metrics to measure the similarity between these representations used to make the matching. This similarity measure is the backbone of our recommendation algorithm.

Our contributions

In this thesis, our main contributions are as follows:

1. The construction of a sourcing domain vocabulary by using external and internal metadata repository. The results of our work have been published in the proceedings of French Knowledge Engineering Days (IC) [START_REF] Dhouib | Construction d'ontologie pour le domaine du sourcing[END_REF].

2. The proposal of a new ontology alignment approach using embedding and radius measure. The results of our work have been published in the proceedings of the International Conference on Semantic Systems [START_REF] Tounsi Dhouib | An ontology alignment approach combining word embedding and the radius measure[END_REF].

3. The proposal of a new named entity recognition algorithm combining several types of extracted features describing the textual content such as: (i) semantics, (ii) syntax, (iii) word characters, and (iv) position of words. The results of our work have been published in the proceedings of the National Conference on Practical Applications of Artificial Intelligence (APIA) [START_REF] Daoud | A hybrid bi-lstm-crf model for sequence labeling applied tothe sourcing domain. 6ème conférence sur les Applications Pratiques de l'Intelligence Artificielle APIA[END_REF].

4. The proposal of a recommender system approach based on domain knowledge and similarity between the vector representations. The results of our work have been published in the proceedings of the Web Intelligence conference [Dhouib et al., 2020].

Thesis outline

This thesis is organized into the following chapters:

• Chapter 2 defines the relevant notions used in this thesis.

• Chapter 3 presents the modular vocabulary we developed for the sourcing domain.

• Chapter 4 describes the ontology alignment approach we developed to match the different ontologies we reused. It also presents the evaluation of this approach conducted several other ontologies.

• Chapter 5 presents our knowledge extraction approach by defining a named entity recognition approach and an entity linking method.

Introduction

In this chapter, we introduce some background knowledge that will be used through out the thesis.

Semantic Web

In 1999, Tim Berners-Lee presented for the first time his vision of the Semantic Web. "The Semantic Web is not a separate Web, but an extension of the current one, in which information is given well-defined meaning, better enabling computers and people to work in co-operation" [Berners- [START_REF] Berners-Lee | The semantic web[END_REF]. Tim Berners-Lee's goal was to extend the current Web with metadata by allowing both machines and humans to better manipulate information and make meaningful interpretations. Therefore, we are no longer talking only about a Web of documents but a Web of data [START_REF] Bizer | Linked data: The story so far[END_REF].

Knowledge organization systems

With the emergence of the semantic Web, the notion of ontology has experienced a new rise. Formerly reserved for the field of philosophy, and later on that of knowledge representation, ontologies now represent the backbone of the semantic Web technology. In the literature, several definitions of ontology have been presented and evolved over the time. The most quoted one is given by Gruber [START_REF] Thomas R Gruber | Toward principles for the design of ontologies used for knowledge sharing[END_REF] "an ontology is a formal, explicit specification of a shared conceptualization". Typically, an ontology is a set of concepts or classes of objects that share common characteristics and relationships or properties [START_REF] Antoniou | A semantic web primer[END_REF]. In addition, an ontology can include a set of axioms which ascertain the coherence of the model, such as disjointness, equivalence, cardinality axioms for concepts, transivity, and functional or inverse axioms for properties.

There are several related conceptual structures to capture knowledge [Zacklad, 2007]:

• A thesaurus is set of terms of a specific domain. These terms are enriched by semantic relations such as hierarchical relation, equivalence relation and association relation.

• A taxonomy is a list of terms of a domain organized in a hierarchical way.

• The term of knowledge graph has been presented by Google as a new trend in 2012. A knowledge graph is usually interpreted as a collection of interlinked entities and relations between those entities. So it may used as a synonym for ontology. In some contexts, it is used to refer to any knowledge base that is represented as a graph [START_REF] Krötzsch | Ontologies for knowledge graphs? In Description Logics[END_REF][START_REF] Ehrlinger | Towards a definition of knowledge graphs[END_REF].

Based on the definitions of the semantic Web and ontologies, we can highlight two main benefits of these technologies: (i) Interoperability, which reposes on sharing and exchanging data across Web applications and agents; (ii) Inferencing, which means the ability of the system to derive new knowledge and new facts.

Semantic Web languages

Resource description framework

Resource Description Framework (RDF)1 is a graph model for representing and structuring data and metadata. RDF is based on the notion of triple (subject, predicate, object), where the subject represents the resource to be described, the predicate represents the property, and the object represents a literal value or another resource. Triples can also be seen as the arcs of a labeled oriented graph that would be distributed on the web. We take for example the following sentence "Silex provides a sourcing platform". This sentence describes the company Silex by the fact that it provides a sourcing platform. This sentence can be represented in RDF by the triple (Silex, provides, Sourcing_platform). RDF statements can be represented in a variety of syntaxes, among which RDF/XML,2 Terse RDF Triple Language (Turtle),3 and JSON-based serialization (JSON-LD).4

Resource description framework schema

Resource Description Framework Schema (RDFS)5 is a language to represent the vocabulary used to describe properties and classes used in RDF knowledge graphs. In our example, the provides property and the class Company are part of the Silex RDFS vocabulary.

Ontology web language

Ontology Web Language (OWL)6 is a W3C recommendation to add more vocabulary for the description of properties and classes, which allow supplementary inference capabilities. We can express for example restrictions on the value of properties or their cardinality, or algebraic properties (symmetrical, transitive, functional, inverse property, disjointness).

Simple knowledge organization system

Simple Knowledge Organization System (SKOS)7 is a W3C recommendation based on RDF and OWL dedicated to represent terminological resources, thematic classifications, glossaries, thesauri, or any other type of controlled and structured vocabulary. SKOS provides several primitives to declare preferred labels or synonymous labels for each concept in each language and synonymy or hyponymy relations [START_REF] Gandon | Graphes RDF et leur Manipulation pour la Gestion de Connaissances[END_REF].

SPARQL protocol and RDF query language

SPARQL Protocol and RDF Query Language (SPARQL)8 provides a query language to manipulate RDF graphs.

Ontology alignment

Ontology alignment is the process of discovering correspondences between concepts and relations from different ontologies. It represents the key ingredient for the semantic interoperability and to solve the semantic heterogeneity problem.

We adopt the ontology alignment definition introduced by [START_REF] Euzenat | Ontology matching[END_REF][START_REF] Shvaiko | Ontology matching: state of the art and future challenges[END_REF]. A correspondence between a source ontology O 1 and a target ontology O 2 is defined as a tuple {(e 1 , e 2 , r, con)}, where:

• e 1 is an entity in O 1 , • e 2 is an entity in O 2 ,
• r is the semantic relationship between e 1 and e 2 such as equivalence(≡), more general ( ), and

• con is the confidence score (typically in the [0, 1] range) holding for the correspondence between e 1 and e 2 .

Generally, the most commonly used semantic relations are equivalence and subsumption relations. For OWL ontologies we can use owl:equivalentClass for equivalent alignment of classes, owl:sameAs for equivalence alignment of individuals and rdfs:subClassOf for subsumption alignment of classes. For SKOS vocabularies, we can use skos:narrowMatch and skos:broadMatch for hyponymy relations between concepts, and skos:exactMatch or skos:closeMatch for synonymy relations. Alignments can be of various cardinalities: (i) one-to-one (1:1), (ii) one-to-many (1:m), (iii) many-to-one (n:1), or (iv) manyto-many (n:m). There are two kinds of matches: (i) a simple match is about linking two atomic entities represented by their identifiers; and (ii) a complex match allows to express logical formulas between entities [START_REF] Thiéblin | Un jeu de données d'évaluation de correspondances complexes entre ontologies[END_REF].

Information and knwledge extraction

Information extraction (IE) [START_REF] Daya | Ontology-based information extraction: An introduction and a survey of current approaches[END_REF][START_REF] Singh | Natural language processing for information extraction[END_REF] is the task of automatically extracting relevant knowledge from unstructured and/or semi-structured data, and converting that into a representation suitable for machine processing.

Similar to IE, knowledge extraction (KE) is the task of extracting knowledge from structured and unstructured data. The main difference between KE and IE is in the case of IE the extraction result is converted in to a relational schema, whereas for KE the result of extraction requires the reuse of existing formal knowledge or the generation of a schema based on source data.

Natural language processing

Natural Language Processing (NLP) [START_REF] Assal | Semantically-enhanced information extraction[END_REF] is a scientific and engineering field concerned with studying the structure and rules of languages to allow computers to analyse and process in order to derive meaning from text and speech.

Sequence labeling

Sequence labeling [Jagannatha and [START_REF] Abhyuday | Structured prediction models for rnn based sequence labeling in clinical text[END_REF] is a NLP task which involves of assigning a class or label to each token in a given input sequence. It includes named entity recognition (NER), syntactic chunking and part of speech (POS) tagging.

Named entity recognition

Named entity recognition (NER) [START_REF] Grishman | Message understanding conference-6: A brief history[END_REF][START_REF] Nadeau | A survey of named entity recognition and classification[END_REF]] also called entity identification or entity extraction, is probably the first step towards knowledge extraction from text. NER seeks to locate entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, time expressions, quantities, etc. It is generally composed of two main phases: (i) identify named entities, and (i) classify entities into predefined categories.

Named entity linking

Named Entity linking (NEL) [START_REF] Rao | Entity linking: Finding extracted entities in a knowledge base[END_REF] is the task of linking an entity mention that has been identified in a text with its corresponding entities in a knowledge base such as Wikidata,9 DBpedia,10 YAGO11 or any other knowledge graph.

Information filtering

Information filtering (IF) [START_REF] Hanani | Information filtering: Overview of issues, research and systems[END_REF] is about removing redundant or unwanted information from large information flows, and managing the information overload in order to expose to users only information that is relevant to them. Recommender systems (RS) is an active information filtering systems. The first recommender system definition was given by [START_REF] Resnick | Recommender systems[END_REF]: "people provide recommendations as inputs, which the system then aggregates and directs to appropriate recipients". Later on, [START_REF] Burke | Hybrid recommender systems: Survey and experiments[END_REF] refines this definition and introduces RS as "any system that produces individualized recommendations as output or has the effect of guiding the user in a personalized way to interesting or useful objects in a large space of possible options". So, we can define RS as a system that helps users to answer their need and provide them with a personalized suggestion in order to pick the most relevant elements. RS are composed of two main entities: (i) the user who plays two roles at the same time, providing opinion about items and receiving the recommendation, and (ii) items such as products, services, articles, movies, music, or social connections, etc. Some examples of e-commercial RS are Amazon12 and Netflix. 13 In 1998, Amazon introduced its item-to-item collaborative filtering algorithm [START_REF] Linden | Amazon. com recommendations: Item-to-item collaborative filtering[END_REF]. [START_REF] Mackenzie | How retailers can keep up with consumers[END_REF] estimated that 35% of consumer purchases on Amazon come from product recommendations. In 2006, Netflix created the Netflix award competition to improve the accuracy of its film recommendation system by 10%. Netflix mentioned that 70% of what users watch is the result of personalized recommendation [START_REF] Mackenzie | How retailers can keep up with consumers[END_REF].

Many challenges are facing the development of RS [START_REF] Singh | Natural language processing for information extraction[END_REF][START_REF] Shah | Survey on recommendation system[END_REF]:

• Cold start: means that the information about items or users are not yet sufficient to provide the best results. This problem is usually associated with the lack of valuable user interactions when a new item or new user is added to the system.

• Sparsity: is also related to the lack of information because the user rated just some items, but no interaction is available on other items. So if only few items are evaluated, it will be difficult to determine his/her taste.

• Popularity bias: means that the system fails to recommend items that are not popular.

• Over-specialization: is the fact that the system recommends only the items rated by users and ignores all the items that are different from anything that user has seen before. This can be a problem if the user wants to try something new.

• Scalability: relies on the complexity of the RS and the ability of a system to react with a large dataset.

Vector representations of textual data

Bag of words

A Bag of words (BOW) represents a text as a list of tokens based on either the occurrence or frequency of each word in the textual document. The dimension of the feature space is the number of all different words in all documents [START_REF] Feldman | The text mining handbook: advanced approaches in analyzing unstructured data[END_REF]. The limitation of this model is that it ignores the semantic relations between words.

Bag of concepts

A Bag of concepts (BOC) represents the text as a list of concepts and not a list of tokens. [START_REF] Sahlgren | Using bag-of-concepts to improve the performance of support vector machines in text categorization[END_REF] introduce this representation by assuming that the meaning of a text can be approached by the union of keyword concepts in the text. Thus, a text is represented by the weighted vector sum of concept vectors corresponding to the terms present in the document [START_REF] Täckström | An evaluation of bag-of-concepts representations in automatic text classification[END_REF].

Word embedding

Word embedding is a distributed word representation that leverages the semantics of words by mapping them to vectors of real numbers, where each dimension of the embedding represents a latent feature of the word [START_REF] Turian | Word representations: a simple and general method for semi-supervised learning[END_REF][START_REF] Gromann | Comparing pretrained multilingual word embeddings on an ontology alignment task[END_REF]. Word embedding models are trained through a shallow neural network architecture to embed words in a dense continuous vector space, based on their linguistic contexts in a corpus, to preserve the semantic and syntactic similarities between words. As a result, words appearing in similar contexts in a text are represented by similar vectors.

Many models produce word embeddings: (i) Word2vec [START_REF] Mikolov | Computing numeric representations of words in a high-dimensional space[END_REF] provides two neural architectures, namely Skip-Gram and CBOW. Skip-Gram takes as input a word and tries to predict its context, whereas CBOW receives as input the context of a word (i.e. the words around it in a sentence) and tries to predict the word in question, (ii) Glove [START_REF] Pennington | Glove: Global vectors for word representation[END_REF] is a count-based model using the global matrix factorization to calculate the co-occurrence of words in the corpus, (iii) fastText [START_REF] Bojanowski | Enriching word vectors with subword information[END_REF] is a library for learning word representations that published a pre-trained word vectors for several languages trained on Wikipedia. Unlike the other models, fastText provides a vector representation for each character n-gram [START_REF] Gromann | Comparing pretrained multilingual word embeddings on an ontology alignment task[END_REF].

The state of the art of word embedding models has recently evolved to what is known as contextual embedding. Embedding from Language Models (ELMo) [Peters et al., 2018a] is a deep contextualized word representation using a Bi-directional LSTM architectures able to create contextual word embeddings. The ELMo idea is to not use a fixed embedding for each word, but to look at the entire sentence before generating the embedding. As a result, the same word can have multiple representations based on its context. Bidirectional Encoder Representation from Transformers (BERT) [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] relies on an architecture that uses attention layers to better capture the semantic relations inside the embedding. BERT is pre-trained on a large corpus of unlabelled texts composed from the entire Wikipedia and book corpus. Bert has two main pre-training objectives, the prediction of "hidden words" and the next sentence of a sequence. CamemBERT [START_REF] Martin | Camembert: a tasty french language model[END_REF] is based on the RoBERTa [START_REF] Liu | Roberta: A robustly optimized bert pretraining approach[END_REF] architecture which is a robust and optimized approach from BERT. The difference between CamemBERT and BERT can be summarized in these three points: (i) CamemBERT has only one pre-training objective which is the prediction of "hidden words"; (ii) it is pre-trained on 138 GB of French text; and (ii) it uses new hyper-parameters. In fact, it chooses the words to be predicted dynamically by randomly masking certain words in a sequence.

Graph embedding

Inspired by word embedding, graph embedding methods consist in learning a continuous vector space for each entity (node/or edge) of a graph. As a result, similar entities have similar vector representations. Several graph embedding methods are presented in the literature. Node2vec [START_REF] Grover | node2vec: Scalable feature learning for networks[END_REF] is an algorithmic framework that aims to create embedding for nodes in a graph. Node2vec is based on random walks which means that the algorithm starts a walk at a random node and performs a series of steps where each step goes to a random neighbor. Translating Embedding for Modeling Multi-relational data (TransE) [START_REF] Bordes | Translating embeddings for modeling multi-relational data[END_REF] provides a very simple and efficient way to capture the structure of a knowledge graph. It generates embedding entities based on the sum of the source entity vector and the translation vector of relation in such a way that they are as close as possible to the target vector. However, this model is not able to represent all other relations that do not link only two entities (i.e. 1-N, N-1 and N-N relations). To overcome this limitation, many models have been developed to extend TransE. TransH [START_REF] Wang | Knowledge graph embedding by translating on hyperplanes[END_REF] presents a more flexible model by interpreting a relation as a translating operation on a hyperplane. TransR [Lin et al.] is a generalization of TransH. The basic idea of TransR is to use two distinct spaces, one for entities and the other for multiple relations. TransG [START_REF] Ling | Design challenges for entity linking[END_REF], produces multiple vectors for a relation to take into account the multiple meanings of this relation. pTransE [START_REF] Yao | Biomedical named entity recognition based on deep neutral network[END_REF] takes into account the relation paths between entities by the summing of all relations in a path.

Machine learning and deep learning algorithms

Hidden markov model

The hidden Markov model (HMM) is a statistical model used in sequence labeling problem. HMM analyzes the sequence pattern of observed symbols in order to interpret the non observable process. An HMM consists of a double stochastic process, in which the hidden stochastic process can be indirectly inferred by analyzing the sequence of observed symbols of another set of stochastic processes [START_REF] Awad | Efficient learning machines: theories, concepts, and applications for engineers and system designers[END_REF][START_REF] Nguyen | Comparisons of sequence labeling algorithms and extensions[END_REF].

Maximum entropy markov model

Maximum entropy Markov model (MEMM) [START_REF] Mccallum | Maximum entropy markov models for information extraction and segmentation[END_REF]] is a discriminative model that combines HMM and maximum entropy. The basic idea of MEMM relies on the fact that the unknown values to be learnt are connected in a Markov chain rather than being conditionally independent of each other.

Support vector machine

Support Vector Machine (SVM) [START_REF] Cortes | Support-vector networks[END_REF] is a supervised learning algorithm which transforms training data into higher dimensions, and searches for a linear optimal separating hyperplane.

Bayesian network

Bayesian Network (BN) is a probabilistic graphical model that represents knowledge about an uncertain domain. Each node in the directed acyclic graph corresponds to a random variable, and each edge represents the conditional probability for the corresponding random variables [START_REF] Yang | Introduction to Algorithms for Data Mining and Machine Learning[END_REF].

Neural network

A Neural Network (NN) is a mathematical model inspired by the human brain to process information and it is composed by many layers. The perceptron in Figure 2.1 is the simplest kind of NN, which has only two layers (the input nodes receive the feature values and the output nodes produce the NER result), and the link weights represent dependence relations [START_REF] Naji | Arabic named entity recognition using artificial neural network[END_REF]].

Convolutional network

A Convolutional Network (CNN) is a regularized version of a multilayer perceptron or fully connected network. This regularization is based on two types of layers: (i) a convolutional layer based on a convolutional filtering, and (ii) a pooling layer that reduces the spatial size in order to limit the amount of neurons and the complexity of the minimization problem [START_REF] Cagli | Convolutional neural networks with data augmentation against jitter-based countermeasures[END_REF].

Recurrent neural networks

Recurrent Neural Networks (RNN) [START_REF] Felix A Gers | Learning to forget: Continual prediction with lstm[END_REF] are a family of neural networks operating on sequential data. In contrast to traditional neural networks which assume that all inputs and outputs are independent from each other, RNN performs the same task for every element of a sequence, and the output depends on the previous computations.

Long short-term memory networks

The main problem of RNN is that they are looking back only a few steps. Long Short-term Memory Networks (LSTM), a particular type of RNN, are designed to avoid this problem through the use of a memory-cell to capture long-term dependencies [START_REF] Felix A Gers | Learning to forget: Continual prediction with lstm[END_REF]. The goal of LSTM is to learn information from context. The main characteristics of LSTM are: (i) the ability to operate on sequential data, and (ii) the ability to capture long term dependencies thanks to a memory-cell.

2.6.9 Bidirectional long a short-term memory-conditionalrandom fields networks

Bidirectional Long Short-Term Memory-Conditional Random Fields network (Bi-LSTM-CRF) as shown in figure 2.2 is obtained from the combination of a Bi-LSTM model and a CRF classifier. Bi-LSTM model [START_REF] Graves | Framewise phoneme classification with bidirectional lstm and other neural network architectures[END_REF] is composed of two LSTMs, one that processes the input sequence from left to right, and the other that processes the input in the reverse direction (i.e., from right to left). CRF [START_REF] Lafferty | Conditional random fields: Probabilistic models for segmenting and labeling sequence data[END_REF] is an undirected graphical model and is partially similar to HMMs. The difference between CRF and HMM is that HMM simply works on the word type of tokens, while CRF works on a set of features defined automatically from input tokens during the training process [START_REF] Sutton | An introduction to conditional random fields[END_REF][START_REF] Poostchi | Bilstm-crf for persian named-entity recognition armanpersonercorpus: The first entityannotated persian dataset[END_REF].

Evaluation measures

General evaluation metrics are used to analyse the performance of a model for different tasks (e.g. ontology alignment, NER, RS). These metrics are based on the followings: (i) True Positive (TP) is the number of positive instances correctly assigned; (ii) False Positive (FP) is the number of positive instances incorrectly assigned; (iii) False Negative is the number of negative instances incorrectly assigned; and (iv) True Negative (TN) is the number of negative instances correctly assigned.

Precision

Precision is used to check the degree of correctness of the model.

Precision = TP TP + FP . (2.1)

Recall

Recall is used to check the degree of completeness of the model.

Recall = TP TP+FN . (2.2)

F1-measure

F1-measure is the harmonic average of recall and precision.

F-measure = 2 • recall • precision recall + precision . (2.
3)

The precision, recall and F1-measure are commonly used to evaluate NER [START_REF] Konkol | Crf-based czech named entity recognizer and consolidation of czech ner research[END_REF]Jiang et al., 2016a;[START_REF] Tzong-Han | Various criteria in the evaluation of biomedical named entity recognition[END_REF], ontology alignment [START_REF] Euzenat | Semantic precision and recall for ontology alignment evaluation[END_REF][START_REF] Ochieng | Large-scale ontology matching: State-ofthe-art analysis[END_REF], and RS [START_REF] Lerato | A survey of recommender system feedback techniques, comparison and evaluation metrics[END_REF][START_REF] Gunawardana | A survey of accuracy evaluation metrics of recommendation tasks[END_REF][START_REF] Hernández | Evaluation of recommender systems: A new approach[END_REF]].

Precision at N

This measure is used especially to evaluate RS [START_REF] Lerato | A survey of recommender system feedback techniques, comparison and evaluation metrics[END_REF]. Given that, from the perspective of the user of a RS, it is highly desirable that at least the first recommendations be highly relevant. So, it is interesting to evaluate the proposed settings based on the precision score considering the N top ranking providers (up to the tenth) according to the usual formula for "precision at N ". "Precision at N " represents the portion of the top-n documents that are relevant to the user. P @N = relevant items in the top N recommended items N .

(2.4)

Cross-validation

Cross-validation [START_REF] Refaeilzadeh | Cross-Validation[END_REF] is a technique that is used to evaluate and compare the performance of models. This technique is used in the case of insufficient data available for partitioning them into training and test sets. K-fold cross-validation consists in randomly partitioning the data into K equal folds. Subsequently, one of the folders is used as the test set while the remaining k-1 folds are used for training set. This operation is repeated until each unique folder is used as the test set.

Conclusion

In this chapter, we provided some background notions on the semantic Web, ontology alignment, knowledge extraction, and recommender systems. All these notions are the backbone of this thesis since we intend to propose an approach to inject domain knowledge in a recommender system for the sourcing domain.

Chapter 3

Ontology engineering for the sourcing domain

Introduction

In recent years, various companies have been moving towards the integration of ontologies within their processes to better structure their knowledge and improve the performance of their automatic processing. In this chapter, we present our domain knowledge modeling specific to the sourcing domain with the goal of reasoning on knowledge to improve the providers' recommender.

This chapter is organized as follows: we first present relevant works on ontology engineering in Section 3.2. Subsequently, we propose in Section 3.3 the main lines of our approach to build an ontology for the sourcing domain. Sections 3.4, 3.5 and 3.6 detail the three steps of our approach. Finally, Section 3.7 concludes this chapter.

Related works

Since 1996, several surveys on ontology engineering methodology have been written [START_REF] Uschold | Ontologies: Principles, methods and applications[END_REF][START_REF] Fernández | Overview and analysis of methodologies for building ontologies[END_REF][START_REF] Corcho | Methodologies, tools and languages for building ontologies. where is their meeting point?[END_REF][START_REF] Iqbal | An analysis of ontology engineering methodologies: A literature review[END_REF][START_REF] Stadlhofer | An overview of ontology engineering methodologies in the context of public administration[END_REF][START_REF] Simperl | Collaborative ontology engineering: a survey[END_REF][START_REF] Yadav | Ontology engineering and development aspects: a survey[END_REF][START_REF] Konstantinos I Kotis | Ontology engineering methodologies for the evolution of living and reused ontologies: status, trends, findings and recommendations[END_REF].

In this section, we describe well-known and relevant methodologies:

Uschold and King's methodology [START_REF] Uschold | Towards a methodology for building ontologies[END_REF]: was designed at the Artificial Intelligence Applications Institute (AIAI) of Edinburgh to build an ontology for enterprise modeling processes. It defines four steps to build an ontology: (i) identify the purpose; (ii) build the ontology; (iii) evaluate it; and (iv) document it. Three strategies for identifying the main concepts in the ontology are also presented: (i) in a top-down approach the most abstract concepts are identified first; (ii) a bottom-up approach starts by the identification of the most specific concepts, and then their generalization into more abstract concepts; and (iii) a middle-out approach starts by identifying the important concepts, and then either generalizes or specializes into other concepts.

Grüninger and Fox methodology [START_REF] Grüninger | Methodology for the design and evaluation of ontologies[END_REF]: was designed in the context of the development of knowledge-based systems using first order logic, and is based on two steps: (i) identify the main applications and scenarios for which the ontology will be used; (ii) determine the scope of the ontology using a set of competency questions.

KACTUS approach [START_REF] Bernaras | Building and reusing ontologies for electrical network applications[END_REF]: is designed in the context of the Esprit KACTUS project with the aim of studying the feasibility of knowledge reuse in complex technical systems, and the role of ontologies to support it. Ontologies are built following a bottom-up strategy. The idea is to refine the ontology each time that the application is built. The KACTUS approach proposes three steps: (i) specification of the application; (ii) preliminary design based on relevant top-level ontological categories; (iii) ontology refinement and structuring.

METHONTOLOGY [START_REF] Fernández-López | Methontology: from ontological art towards ontological engineering[END_REF]: created in the Artificial Intelligence Lab from the Technical University of Madrid, this methodology is used to build ontologies either from scratch, reusing other ontologies, or for re-engineering ontology [START_REF] Corcho | Methodologies, tools and languages for building ontologies. where is their meeting point?[END_REF]. It enables the construction of ontologies at the knowledge level. This methodology divides the ontology development life cycle into: (i) specification; (ii) conceptualization; (iii) formalization; (iv) implementation; and (v) maintenance.

SENSUS [START_REF] Swartout | Toward distributed use of large-scale ontologies[END_REF]: was developed by the Information Sciences Institute (ISI) natural language group to provide a conceptual structure for developing machine translators. The process of this methodology starts by extracting information from various electronic sources of knowledge. The basic ontology is manually aligned first with PENMAN Upper Model [START_REF] Bateman | Kpml: the komet-penman (multilingual) development environment[END_REF] and ONTOS [START_REF] Nirenburg | Application-oriented computational semantics[END_REF] and then with WordNET.1 SENSUS is composed by the following steps: (i) seed identification using series of terms; (ii) manually link these seed terms to SENSUS; (iii) include all concepts from the new terms to the root of SENSUS; (iv) add all new terms that could be relevant within the domain; (v) add the entire subtree under the nodes that have a large number of paths through them; (vi) add new domain terms.

NeOn [START_REF] Carmen | NeOn Methodology for building ontology networks: specification, scheduling and reuse[END_REF][START_REF] Carmen Gómez-Pérez | Neon methodology for building ontology networks: a scenario-based methodology[END_REF]: is a scenariobased methodology that supports the collaborative aspects of ontology construction. It emphasizes the development of ontology networks as well as the reuse of existing ontological and non-ontological resources to the development of an ontology [START_REF] Konstantinos I Kotis | Ontology engineering methodologies for the evolution of living and reused ontologies: status, trends, findings and recommendations[END_REF]. This methodology presents a set of nine scenarios: (i) from specification to implementation; (ii) reusing and re-engineering non-ontological resources; (iii) reusing ontological resources; (iv) reusing and re-engineering ontological resources; (v) reusing and merging ontological resources: ontology matching tools enable ontology aligning or merging; (vi) reusing, merging and re-engineering ontological resources; (vii) reusing ontology design patterns (ODPs); (viii) restructuring ontological resources; and (ix) localizing ontological resources to translate all the terms of the ontology into another natural language.

On-To-Knowledge [START_REF] Staab | Knowledge processes and ontologies[END_REF]: is based on analyzing use cases and includes the identification of goals of the knowledge management tools. This methodology is composed of four steps: (i) Kick-off to capture the ontology requirements, identification of the competency questions, and study of the potentially reusable ontologies, (ii) refinement; (ii) evaluation; and (iv) ontology maintenance.

These methodologies focus on different aspects of ontology engineering [START_REF] Corcho | Methodologies, tools and languages for building ontologies. where is their meeting point?[END_REF][START_REF] Konstantinos I Kotis | Ontology engineering methodologies for the evolution of living and reused ontologies: status, trends, findings and recommendations[END_REF]:

• Some methodologies are designed to build ontologies from scratch, or promote the reuse of existing ontologies (i.e. NeOn, METHONTOL-OGY). KACTUS proposes to build an ontology based on an abstraction process from an initial knowledge base, while SENSUS proposes to automatically generate the ontology's skeleton from a huge ontology.

• Degree of application dependency: (i) application dependent (i.e. KAC-TUS and ON-To-Knowledge methodology) since the ontology is built based on a given application; (ii) semi application-dependent for example Gruninger and Fox's method and SENSUS; (iii) application independent (i.e. other methodologies) since the ontology development process is totally independent of the uses of the ontology.

• Cooperative construction: only the NeOn methodology takes this aspect into account.

• Life cycle proposition: only METHONTOLOGY and On-To-Knowledge methodologies propose a life cycle to identify the set of phases through which the ontology moves during its life.

• Strategies for identifying concepts: (i) top-down approach (i.e. KAC-TUS); (ii) bottom-up approach (i.e. SENSUS); and (iii) middle-out approach (i.e. METHONTOLOGY, On-To-Knowledge )

Our ontology design approach

We adapted the NeON methodology to build the Silex vocabulary based on the scenario of reuse, fusion, and re-engineering of resources. We identified three main questions in our ontology engineering approach:

1. What types of knowledge do we need to represent in order to reason on their representation and to improve the provider recommendation quality?

2. Which existing ontologies can we reuse?

3. Do we need to develop a new complementary ontology?

With the collaboration of Silex managers, and after an analysis of textual descriptions of providers and service requests, we decided to represent four types of knowledge:

1. Skills defined as a set of required knowledge and capacity to realize a daily task in a specific field [START_REF] Amourache | Construction d'une ontologie basée compétence pour l'annotation des cvs/offres d'emploi[END_REF].

2. Occupations2 performed by a person in a specific field.

3. Business activities3 refer to any activity a business engages in for the primary purpose of making profit. This is a general term that encompasses all the economic activities carried out by a company during the course of business.

4. Products defined as goods, services or ideas that can be offered for sale.

We imagined scenarios relevant for the Silex company and based on them we identified competency questions, i.e. the questions that our ontology must enable to answer.

Scenarios

1. The user is looking for a special product or service. The ontology must link the descriptions of the service request and providers based on this need.

2. The user is looking for a special product or service. But no provider description contains directly this need. The ontology must allow to navigate through this knowledge to answer a more generic or more specific need.

3. The user wants to have an idea about a product or a service but also about related services.

Competency questions

In order to specify correctly our ontology, and with the help of the Silex managers, we have defined a set of competency questions [START_REF] Uschold | Ontologies: Principles, methods and applications[END_REF][START_REF] Natalya | Développement d'une ontologie 101: Guide pour la création de votre première ontologie[END_REF]:

1. Which skills / occupations / business activities/products appear in the description of a provider or a service request?

2. What are the links between two skills / business activities / products? 3. Which service providers have a given occupation / skill / business activity/ product?

4. What are the skills related to a given occupation / business activity?

5. What are the products related to a given occupation / business activity?

Our approach for building an ontology involves:

• Identifying current metadata repositories that represent skills, occupations, products and business activities.

• Building an internal Silex metadata repository to represent the company's own knowledge.

• Aligning the identified metadata repositories and ontologies with the internal ones.

This is further detailed in the following sections.

Identification and reuse of existing metadata

In the context of Silex, our need is based on the use of domain knowledge that can be captured in a thesaurus of concepts for the procurement domain, as we exploit different types of relationships between concepts and their labels (not intentional definitions). In the rest of this thesis, we will refer to vocabulary indistinctly as a thesaurus or an ontology.

We reviewed the state of the art of existing metadata repositories and identified the most interesting ones for our context. We defined four selection criteria: (i) the source of the metadata repository; (ii) its freshness; (iii) the supported languages; and (iv) the supported formats.

We studied three types of vocabularies: (i) the first one represents skills and occupations; (ii) the second one represents the business activities; and (iii) the third one represents products. The ROME code, consisting of a letter and four digits, is structured into four levels:

Skills and occupations vocabulary

• The letter (from A to N) represents a family of occupations;

• The letter and the first two digits identify the professional field;

• The letter and the first three digits further specifies the professional field;

• The letter and the four digits, representing the ROME code, refer to the occupations as concepts.

We present in figure 3.2 an example of the ROME hierarchy knowledge. ROME is available in Excel format, but we were planning on building a vocabulary in SKOS-RDF format, therefore, as a preprocessing step, we had to transform rome into SKOS-RDF format. Our vocabualry containing 12255 concepts.

The nomenclature of Cigref 's IS professions

The professions 6 that provides a description of existing professions in the IT departments of the large companies that are members of CIGREF. This nomenclature contains seven sub domains of computing and 36 occupation names and descriptions for each business. We built a vocabulary in SKOS-RDF format for this repository in the same way as for ROME, which contains 43 concepts. We present in figure 3.3 an example of the CIGREF hierarchy knowledge. 1735 concepts with both French and English labels. NAF rev.2 has a tree structure with five levels:

Business activity vocabulary

• 21 sections (1 letter), common to NAF, NACE and ISIC.

• 88 divisions (2 digits), common to NAF, NACE and ISIC.

• 272 groups (3 figures), common to both NAF and NACE.

• 615 classes (4 digits), common to both NAF and NACE.

• 732 sub-classes (4 digits and 1 letter).

We present in figure 3.4 an example of the NAF hierarchy knowledge.

United Nations standard product and services code (UN-SPSC)

UNSPSC8 is an open and international classification of business activities, goods and services, owned by the United Nations Development Programme (UNDP). The UNSPSC is an eight-digit coding system with a four-level hierarchical structure:

• Segment is the logical aggregation of families for analytical purposes.

• Family is a commonly recognized group of interrelated commodity categories.

• Class is a group of commodities sharing common characteristics.

• Commodity is a group of substitutable products or services. Services 81100000 Professional engineering services 81110000 Computer services 81111600 Computer programmers 81111602 Programming for Java 81111800 System administrators 81111900 Information retrieval systems 81111901 Database information retrieval 81111902 On line database information retrieval This classification is available in 11 languages. A commercial version is available in Microsoft Excel format. In the same way as we did for ROME, we transformed the Excel file of UNSPC into a SKOS-RDF vocabulary. For that, we started by analyzing the format of this file, which contains 18 columns. For each level of the UNSPC tree, there are four columns. For example, the columns associated to the segment level are: (i) "segment" which represents the key of segment; (ii) "segment title" which represents the title of the segment; (iii) "segment definition" which presents a description of the segment; and (iv) "segment synonyms" which provides all title synonyms in all languages. We used an automatic language detector to extract the French language appellation from the synonyms columns, and we built our vocabulary in SKOS-RDF format. Our UNSPSC version contains 87470 concepts.

We present in figure 3.5 an example of the UNSPSC hierarchy knowledge.

Kompass

Kompass9 is the author of the most extensive international classification of activities. • 15 families,

• 67 sectors (2 digits),

• 3014 branches (5 digits),

• 55450 products and services (7 digits).

The disadvantage of this classification is that it is exposed via a website only. We extracted this classification using a crawler to transform it into the SKOS-RDF format. Focusing on the IT field, we identified the family "IT, Internet and R&D" which contains 3 sectors: (i) IT and the Internet, (ii) Architects, technical offices and engineering consulting firms, and (iii) Research and testing. The Kompass vocabulary contains 1370 concepts.

We present in figure 3.6 an example of the KOMPASS hierarchy knowledge.

Products vocabulary

French classification of products (CPF)

The nomenclature of activities and products10 have been developed mainly to facilitate the organization of economic and social information. It aims to classify goods and services resulting from economic activities. Each NAF code is associated with a link to the CPF to view the codes and titles of the products associated with each activity and to access the entire CPF. We transformed the Excel file of CPF into a SKOS-RDF vocabulary. This vocabulary contains 5522 concepts. We present in figure 3.7 an example of the CPF hierarchy knowledge.

Construction of internal vocabularies at Silex

By comparing Silex 's internal repositories with the above mentioned resources, we identified new concepts specific to Silex that were used for sourcing purposes. In order to capture the richness of these internal repositories, we decided to build two internal vocabularies.

Construction of a Silex internal skills/ occupations vocabulary

The Silex platform contains a skills repository stored into its database and enriched by users. We used this text file containing 8470 terms to build a vocabulary in SKOS-RDF format. We faced two difficulties: (i) the repository mixes terms from different semantic fields such as skills, occupations, business activities, cities, countries, languages (Java, Marketing, Security agent, France, English, Paris); and (ii) the repository contains compound terms, in French and English, with spelling errors and abbreviations. So we started with a normalization step to remove all duplicates and group synonyms. We obtained 6479 terms. Then, we used the hierarchical clustering [START_REF] Berkhin | A survey of clustering data mining techniques[END_REF][START_REF] Kuchaki Rafsanjani | A survey of hierarchical clustering algorithms[END_REF][START_REF] Dipak | Extensive survey on hierarchical clustering methods in data mining[END_REF]] method using word embedding [Mikolov et al., 2013a] and the cosine similarity metric [START_REF] Singhal | Modern information retrieval: A brief overview[END_REF] to identify groups of terms which are relatively homogeneous. We obtained 101 categories.

When analyzing the results of this clustering, we could clearly associate a category name to a cluster of terms such as:

• "Test sub category" groups the following terms: "Tester", "Test and validation engineer" ,"Unit test", "User test", "Functional test", "Recruitment test".

• "Development sub category" contains for example "C++", "Java", "cackephp", "MangoDB", "Joomla", "SQL", "full stack developer".

• "Occupation sub-category" contains for example "IT pro ject manager".

Computing

Cloud computing and security consulting and computing expertise Web and mobile application development • "General services" groups the following terms: "gardening", "paint and flooring" or "floor covering.

It is true that we made good progress in the construction of this vocabulary, but we had to face a real lock: How to automatically assign labels to the nodes of the formed clusters? A potential solution is to compare the distribution of words in the hierarchy (i.e the label of the node, the label of the parent node) to assign labels to each cluster [START_REF] Treeratpituk | Automatically labeling hierarchical clusters[END_REF]. Even if we have not dealt with this topic and preferred to focus on our recommendation approach, building an internal repository should be prioritized for the next steps to benefit of the rich vocabulary of Silex.

Construction of the internal business activities vocabulary for Silex

The Silex business activity repository is stored in a database and maintained by sales representatives. This repository contains six main business activities: (i) "Computing", (ii) "Marketing and commercial", (iii) "Human resources management", (iv) "General services", (v) "Finance and Administrative", and (vi) "Industrial services".

We extracted this repository in CSV format and built a vocabulary in SKOS-RDF format containing 90 concepts.

Ontology alignment

After building separate vocabularies as described above, we worked on the construction of a single modular vocabulary integrating them all, by aligning them with each other. Capturing these alignments helps to identify the semantic links between the different concepts (skills, occupations, business activities and products).

We started by doing manual alignment work for the specific field of computing. The manual alignment work was carried out to discover correspondences between these vocabularies: (i) ESCO to Cigref, (ii) ESCO to ROME, (ii) NAF to UNSPSC, and (iv) NAF to Silex business activities. This work allowed us: (i) to develop a proof of concept of our ontology-based recommendation of providers approach for Silex related to this domain, and (ii) to use manual alignments as a test-bed for our automatic alignment approach presented in Chapter 4.

To define the semantic links between the concepts, we used the following properties:

• skos:broadMatch to define a generalization relationship between two concepts;

• skos:exactMatch to define a high level of similarity between two concepts (same labels);

• skos:closeMatch to express that two concepts are quite similar (with different labels);

• dcterms:references11 to express a reference to a related resource.

Figure 3.9 explains the defined alignment approach.

Figure 3.9: Overall knowledge engineering process to built a modular vocabulary for the sourcing domain.

Alignment of the skills and occupations vocabulary

To align the metadata repositories of the selected skills and occupations, we considered ESCO as the reference because of its multilingual characteristic and its completeness compared to the others. We conducted three alignment phases: (i) between ROME and ESCO (ii) between Cigref and ESCO, and As stated before, there are different levels of structuring for these ontologies. The highest levels can be seen as domains, while the lowest levels represent the occupations or skills. We started by aligning the occupations (lowest level) by looking for the correspondences between the concept names. To align the vocabulary domains (highest level), we defined the following rule: if there is a correspondence between the source vocabulary occupation and the target vocabulary occupation, we establish a correspondence between the target vocabulary domain and the source vocabulary domain.

For instance, as Figure 3.10 shows, we matched the ESCO concept "IT consultant" with the concept "IT consultant" of ROME and the concept "Information Systems Consultant" of Cigref using the property skos:closeMatch.

The second step of our alignment process was to match Silex _skills with ESCO. We can easily establish a direct match between some occupations or programming languages. For example, we match the "IT project manager" Silex concept with the "ICT project manager" ESCO concept, and we match the "Java" Silex concept with the ESCO concept having the same name.

Regarding the Cigref/ESCO alignment, Cigref vocabulary is limited to popular occupations from the Information Systems Departments. In spite of that, we failed to match "Support and Assistance" Cigref domain which contains "Functional Assistant" and "User Support Technician" occupations. Also, we could not match "information Systems Planner", "Coach agile" or "Product Owner" Cigref occupations.

In the case of ROME/ESCO matching, we aligned all the occupation domains of ROME with ESCO concepts. But we failed to link all occupations of ROME. For example, for the "Information Systems Department" occupation domain, we could not identify a match to "Director of the IT department" or "Head of IT division" occupations. For "Production and operation of information systems" domain, we could not link "IT Production Team Leader" or "IT Operating Assistant" occupations.

For Silex _Skills/ESCO matching, we encountered several problems related to the heterogeneity of the Silex vocabulary: (i) it contains not only skills and occupations but also terms which represent sub-categories such as "design digital"; (ii) Silex details more than ESCO the different software packages and programming languages, e.g. with concepts like "mongodb", "windev", "openerp"; (iii) finally, the Silex vocabulary is sometimes using a commercial oriented language style, for example "front_end developer" or "full stack developer".

We obtained 229 links between ESCO and ROME and 56 links betwwen ESCO and CIGREF.

Alignment of the business activity vocabulary

We considered NAF as the business activity vocabulary reference, and the alignment consists of searching a correspondence between NAF concepts, Kompass concepts, and UNSPSC concepts based on their labels. We chose to proceed from the top level of hierarchies down in order to achieve the best precision.

For example, the NAF division 62 and the group 62.0 have the same name "Programming, consulting and other IT activities". We aligned the NAF group 62.0 with the Kompass branch 57830 "IT audit and consulting". We also used the skos:narrowMatch property to establish a generic relation between the "Computer programming" NAF concept and "Language and programming software" and "Computer programming services" Kompass concepts.

To align UNSPSC vocabulary to NAF vocabulary, we used not only the labels of concepts but also their definitions. We browsed all the commodities of a given class in order to define the matching between that class and NAF concepts. Therefore, the granularity of our matching is fixed at the class level. For example, we linked the UNSPSC concept "Computer programmers" to the NAF concept "Computer programming services" with the skos:exactMatch property. We also linked the UNSPC concept "Software maintenance and support" to the NAF concept "Computer facilities management services" with the skos:closeMatch property. We could not identify a matching for these classes: "Graphic display services", "Art de- sign services", "Electronic mail & messaging service", "Data voice or multimedia network" and "Access management service".

We matched Silex _activity to NAF. We aligned for example "Cloud computing and security", "IT and IS consulting", and "Web and mobile application development" concepts, but we did not match"Design and artistic direction", "3 D Impression" and "Conversational Digital Assistant" concepts.

We obtained 77 links between NAF and UNSPSC, 19 links between NAF and Silex_activity, and 11 links between NAF and KOMPASS.

Alignment between the skill/occupation vocabulary and the business activity vocabulary

The last step of the alignment process was the elaboration of links between the skill and occupation ontology and the business activity vocabulary. The best way to do that is to align the two reference ontologies ESCO and NAF. We used the Pôle Emploi documentation, which gives the correspondences between ROME and NAF [pôle emploi, 2017]. Having already established the links between ESCO and ROME, we thus deduced by transitivity the links between ESCO and NAF. The total number of links between these two ontologies is 34 links. For example, we had already aligned the ESCO concept "Application Programmers" with the ROME concept "Computer studies and development". The Pôle Emploi documentation establishes a correspondence between the latter and NAF divisions "Computer programming, consultancy and related activities" and "Information service activities". As a result, we have defined a correspondence between the ESCO concept and NAF divisions "Computer programming, consultancy and related activities" and "Information service activities". We obtained 34 links between ESCO and NAF.

Conclusion

In this chapter, we described our approach to design the Silex modular vocabulary intended to support an ontology-based approach of automatic sourcing. The resulting vocabulary makes it possible to semantically annotate text descriptions of providers and service requests based on four knowledge types: (i) skills, (ii) occupations, (iii) business activities and (iv) products, in order to automatically generate high-quality recommendations of service providers. We adopted a top-down approach, reusing metadata repositories such as ESCO, ROME, NAF, UNSCPC and CPF. Table 3.1 summarizes the existing repositories used for the sourcing domain knowledge. We adopted a bottom-up approach to build the company's internal vocabulary. The final step of aligning the various vocabularies chosen to build up the targeted modular vocabulary was first carried out manually for the computing sector only, as a proof of concept and a way to build up a test bed. Chapter 4 presents an approach that we designed to automatically align candidate vocabularies for any other sector.

Introduction

Ontology alignment plays a key role in the management of heterogeneous data sources and metadata. Many reasons can explain this: (i) there are different actors with different interests, (ii) the use of different tools, knowledge with some different levels of details [START_REF] Euzenat | Ontology matching[END_REF], (iii) and also different methodologies followed to construct the ontologies, etc. As a result, several ontologies exist in the same or different domains with some level of heterogeneity among them. Ontology alignment is thus a crucial yet difficult task to deal with this heterogeneity and achieve interoperability on the semantic web. In this context, we address in this chapter the following research questions:

• How can we align two ontologies?

• How can we define a similarity measure between the entities of ontologies?

• How can we refine the nature of the relationship between two entities?

We propose a novel approach to ontology alignment based on a set of rules exploiting mainly semantic information using a similarity measure defined in the embedding space of a word embedding. The underlying assumptions behind our approach are:

• All the labels of the entities which share the same parents are close to each other in the embedding space;

• Each entity in an ontology can be represented as a cluster of its instances in the embedding space and such a cluster can be described by its centroid and its radius [START_REF] Ristoski | Large-scale taxonomy induction using entity and word embeddings[END_REF]Alshargi et al., 2018b,a];

• A cluster whose radius is smaller than the radius of another cluster whose centroid coincides or is very close to its centroid is likely to represent a specialization of the entity associated with the broader cluster.

Our major contributions include:

• Our capability to handle not only the equivalence relationship, but also the hierarchical relationship between entities;

• The introduction of the radius notion as a dispersion measurement of a label cluster that enables to refine the nature of the relationship (equivalence or hierarchical) between two matching entities;

• Our capability to discover rich n-m relationships between entities;

• The evaluation of our system on several open datasets from the Ontology Alignment Evaluation Initiative (OAEI)1 benchmark and a real-world case study provided by the Silex company2 and another provided by ONISEP3 .

This chapter is organized as follows: Section 4.2 discusses previous works on ontology alignment. Section 4.3 describes the different steps of our ontology alignment approach. Section 4.4 reports and discusses the results of our experiments on several datasets. Section 4.5 concludes with an outline of future work.

Related work

A variety of ontology alignment techniques has been presented in the literature, and probably over a hundred different alignment systems exist to date. Due to this wide scope, we choose not to capture all research directions in this domain. Instead, we focus in this section on giving an overview of alignment techniques with some references of systems. Several surveys on ontology alignment techniques have been written [START_REF] Ardjani | Ontology-alignment techniques: Survey and analysis[END_REF][START_REF] Euzenat | Ontology matching[END_REF][START_REF] Kalfoglou | Ontology mapping: the state of the art[END_REF][START_REF] Otero-Cerdeira | Ontology matching: A literature review[END_REF][START_REF] Shvaiko | A survey of schema-based matching approaches[END_REF][START_REF] Rahm | A survey of approaches to automatic schema matching[END_REF][START_REF] Doan | Semantic integration research in the database community: A brief survey[END_REF]. Most of these surveys focus on input and process dimensions to classify the ontology alignment techniques. [START_REF] Doan | Semantic integration research in the database community: A brief survey[END_REF] consider both input and process dimensions and differentiate in their classification between: (i) rule-based techniques that exploit schema-level information in specific rules; and (ii) learning-based techniques that exploit data instance information with machine-learning or statistical analysis. However, [START_REF] Rahm | A survey of approaches to automatic schema matching[END_REF] analyze the two dimensions in a different way. For the input dimension they distinguish between instance classification matchers (i.e. exploiting information from the TBox) and schema classification matchers (i.e. exploiting information from the ABox). For the process dimension they introduce classification axes such as element vs structure or linguistic vs constraint-based. But the most complete and extensive classification of ontology alignment techniques available to date is probably the one proposed by [START_REF] Euzenat | Ontology matching[END_REF] depicted in Figure 4.1.

This classification can be read in a top-down way focusing on the granularity of the matcher and the interpretation that the different techniques offer to the input information, or in a bottom up way focusing on the origin and the kind of input information used by the matching techniques. In the top-down interpretation, the matching techniques can be classified in the first level as: • Element-level techniques, which find out correspondences by considering the entities in isolation.

• Structural-level techniques, which rely on the analysis of the neighbourhood of two entities to determine their similarity.

In the second level of the top-down interpretation, the matching techniques include:

• Syntactic techniques, which limit their input interpretation to the instructions stated in their corresponding algorithms.

• Semantic techniques, which interpret their input using formal semantics.

In the bottom-up interpretation, the matching techniques can be classified in the first level as:

• Content-based techniques, which focus on the internal information of the two ontologies.

• Context-based techniques, which take into account external information that may come from relations between ontologies or external resources [START_REF] Otero-Cerdeira | Ontology matching: A literature review[END_REF].

Most researches on ontology alignment fall into the content-based category, which is divided into four groups or feature approaches, ranging from lexical information to semantics information, passing through structural and extensional information [START_REF] Kolyvakis | Deepalignment: Unsupervised ontology matching with refined word vectors[END_REF].

Lexical information

Lexical information presents an important source of information to ontology alignment systems. This type of information is deduced on the Element-level and is based on computing the similarities between the lexical information of the entities (i.e. names, labels, descriptions, comments). There are three main categories of similarity measures to compare two strings [START_REF] Cheatham | String similarity metrics for ontology alignment[END_REF][START_REF] Stoilos | A string metric for ontology alignment[END_REF][START_REF] Van Tien Nguyen | Mesure de la similarit\'e entre termes et labels de concepts ontologiques[END_REF][START_REF] Wael H Gomaa | A survey of text similarity approaches[END_REF]]:

• String-based similarity measures: consider a string as a sequence of characters: (i) The Hamming distance is a simple way to compare two strings based on counting the number of positions in which two strings differ [START_REF] Euzenat | Ontology matching[END_REF]. (ii) The edit distance (also called the Levenshtein distance) is a basic character-based metrics that represents the minimal cost of editing operations (i.e insertion, deletion, and substitution) to be applied to one string in order to obtain the other one. This metric is very popular in ontology alignment systems such as RIMOM [START_REF] Li | Rimom: A dynamic multistrategy ontology alignment framework[END_REF], ASMOV [START_REF] Yves R Jean-Mary | Ontology matching with semantic verification[END_REF] and AgreementMaker [START_REF] Isabel F Cruz | Agreementmaker: efficient matching for large real-world schemas and ontologies[END_REF]. (iii) The character-based metrics, also called the Jaro distance, measures the similarity between two strings based on the number of common characters that are present in them. The main limitation of this technique is that it does not allow to discover the equivalent entities described by different terms (synonyms), while different concepts described by equal terms (homonyms) will mistakenly be detected as a perfect match [START_REF] Granitzer | Ontology alignment-a survey with focus on visually supported semi-automatic techniques[END_REF].

COMA [START_REF] Do | Coma-a system for flexible combination of schema matching approaches[END_REF] and COMA++ [START_REF] Aumueller | Schema and ontology matching with coma++[END_REF],

OLA [START_REF] Euzenat | Similarity-based ontology alignment in owl-lite[END_REF], Anchor-Prompt [Noy and [START_REF] Noy | Anchor-prompt: Using non-local context for semantic matching[END_REF], S-Match [START_REF] Giunchiglia | S-match: an algorithm and an implementation of semantic matching[END_REF] are examples of systems that use string-based metrics for alignment.

• Token-based similarity measures: consider a string as a vector, which makes it possible to apply metric space distances: (i) The Term Frequency-Inverse Document Frequency (TF-IDF) [START_REF] William W Cohen | A comparison of string distance metrics for name-matching tasks[END_REF] measures the relevant of a word in the document; it is defined as the product of the frequency of the word in the document (TF) and its importance according to its distribution and use in the document set (IDF). (ii) The Jaccard similarity measure [START_REF] Hadjieleftheriou | Weighted set-based string similarity[END_REF] is defined as the ratio between the number of common characters between two strings and the total number of characters. (iii)

The Euclidean distance represents the geometric distance between two data points in an n-dimensional space, (iv) The Manhattan distance calculates the distance between two real-valued vectors. (v) The cosine similarity is a measure of similarity between two vectors of an inner product space that measures the cosine of the angle between them [START_REF] Wael H Gomaa | A survey of text similarity approaches[END_REF] • Hybrid similarity measures: combine string-based and token-based approaches: (i) SoftTF-IDF [START_REF] William W Cohen | A comparison of string distance metrics for name-matching tasks[END_REF] determines similar token pairs by improving the TF-IDF method by a character-based metric (Jaro), (ii) TagLink [START_REF] Camacho | A string metric based on a one-to-one greedy matching algorithm[END_REF] first calculates a similarity score for each pair of tokens by comparing the characters of one token to those of the other token, and then computes a global similarity score which is calculated for both strings by comparing the tokens in one string to those in the other string, and (iii) [START_REF] Alvaro E Monge | The field matching problem: Algorithms and applications[END_REF]] use a character-based metric (Jaro). For each token of the first string, they look for the closest token in the second string and the corresponding score. The global similarity score between the two strings corresponds to the average value of these scores.

Structural information

The lexical information is not enough to discover the matching when the vocabulary of ontologies differs. Hence, many ontology alignment systems consider the structure level for this. Ontology is represented as a kind of graph. The position of the entities in the graph and their relations with others entities can be a very good source of information to be analyzed and used to discover the similarity structure. Among the systems which use this kind of information, we can mention: Yam++ [START_REF] Ngo | Yam++: a multi-strategy based approach for ontology matching task[END_REF], MEDLEY [START_REF] Hassen | Medley results for oaei[END_REF], Cupid [START_REF] Madhavan | Generic schema matching with cupid[END_REF], Anchor-Prompt [Noy and [START_REF] Noy | Anchor-prompt: Using non-local context for semantic matching[END_REF], COMA [START_REF] Do | Coma-a system for flexible combination of schema matching approaches[END_REF], OLA [START_REF] Euzenat | Similarity-based ontology alignment in owl-lite[END_REF], QOM [START_REF] Ehrig | Qom-quick ontology mapping[END_REF], RiMOM [START_REF] Li | Rimom: A dynamic multistrategy ontology alignment framework[END_REF], and there are many others.

External information

Despite the fact that lexical and structural information are widely used in ontology alignment, these techniques suffer from their weakness in capturing the semantics of lexical information of entities. To overcome this problem, many systems consider linguistic-based similarities: it is the case of AROMA [David, 2007], Falcon [START_REF] Jian | Falcon-ao: Aligning ontologies with falcon[END_REF], OLA [START_REF] Euzenat | Similarity-based ontology alignment in owl-lite[END_REF], Cupid [START_REF] Madhavan | Generic schema matching with cupid[END_REF], COMA [START_REF] Do | Coma-a system for flexible combination of schema matching approaches[END_REF]. This technique involves exploiting an auxiliary resource, such as WordNet, to add lexical relationships (e.g. synonym, antonyms, hypernyms or hyponyms) to the system [START_REF] Mohammadi | Comparison of ontology alignment systems across single matching task via the mcnemar's test[END_REF]. This information not only improves the alignment quality, but also allows to define the type of relationship, such as equivalence or generalization [START_REF] Granitzer | Ontology alignment-a survey with focus on visually supported semi-automatic techniques[END_REF]. There are three methods to calculate similarities using WordNet [START_REF] Lin | A survey of exploiting wordnet in ontology matching[END_REF]: (i) an edge-based method, which estimates the semantic relatedness between two concepts in WordNet by accumulating the lengths of all edges on the shortest path to quantify the semantic similarity, (ii) an information-based method, which estimates the semantic similarity between two concepts using information contained in related nodes (i.e., related concepts) in WordNet, and (iii) a hybrid method, which combines the information-based method and the structural information (i.e edge-based method) from WordNet to estimate the semantic similarity between words. The main drawbacks of this approach are: (i) that thesauri for languages other than English are generally of poor quality or simply not available; (ii) specialized application areas require a domain-specific thesaurus.

Semantic information

If it is true that the use of an auxiliary resource can resolve part of the synonymy problem when searching for similarities between entities, auxiliary resources still suffer from the incompleteness and non-exhaustiveness of their entries. For that, word embedding techniques are now used more and more in the ontology alignment task [START_REF] Zhang | Ontology matching with word embeddings[END_REF][START_REF] Vieira | Using word semantics on entity names for correspondence set generation[END_REF][START_REF] Kolyvakis | Deepalignment: Unsupervised ontology matching with refined word vectors[END_REF][START_REF] Juan | A reproducible survey on word embeddings and ontology-based methods for word similarity: linear combinations outperform the state of the art[END_REF]. The main drawback of semantic word embedding is that it tends to coalesce the notions of semantic similarity and conceptual association [START_REF] Hill | Simlex-999: Evaluating semantic models with (genuine) similarity estimation[END_REF], especially because they depends on the corpus from which this embedding is derived. Still, word embedding has the potential to bring significant value to ontology matching given the fact that a great deal of ontological information comes in textual form [START_REF] Kolyvakis | Deepalignment: Unsupervised ontology matching with refined word vectors[END_REF]. The first approach that explored word embedding in the ontology alignment task is described by [START_REF] Zhang | Ontology matching with word embeddings[END_REF]. The authors proposed a hybrid method to combine word embedding and the edit distance together. The matching strategy is to consider the maximum similarity, i.e to return for every entity in the source ontology the most similar entity in the target ontology. Nkisi-Orji et al. [2018] introduce a classifier-based approach for ontology alignment which combines string-based similarity, semantic similarity, and semantic context. Word embedding was used to generate semantic features for a random forest classifier. [START_REF] Kolyvakis | Deepalignment: Unsupervised ontology matching with refined word vectors[END_REF] use information from ontologies and additional knowledge sources to extract synonymy and antonymy relations. These information are then used to refine and adapt pre-trained word vectors to compute the similarity distance between entities. [START_REF] Schmidt | Matching domain and top-level ontologies exploring word sense disambiguation and word embedding[END_REF] compare two similarity measures for synset disambiguation: (i) the Lesk measure [START_REF] Lesk | Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone[END_REF] and (ii) the distance between word embedding to match domain and top-level ontologies. Based on their experiments, the authors show that the results obtained using word embed-ding are better than the results obtained with Word Sense Disambiguation. [START_REF] Gromann | Comparing pretrained multilingual word embeddings on an ontology alignment task[END_REF] use a multilingual word embedding for multilingual ontology alignment. Alshargi et al. [2018b,a] extend the state of the art by providing a framework containing three distinct tasks related to the individual aspects of ontological concepts: (i) the categorization aspect, (ii) the hierarchical aspect, and (iii) the relational aspect. Several intrinsic metrics are proposed for evaluating the quality of the embedding. Furthermore, multiple experimental studies were run to compare the quality of the available embedding models. This work highlights that (i) there is no single embedding model which shows superior performance for all tasks, and (ii) that the embedding learned from the knowledge graph (i.e. DBpedia) does not have a higher quality in comparison to the embedding learned from unstructured data (i.e. Wikipedia).

The semantic information category also contains other techniques than word embedding for the alignment task, by reducing the graph matching problem to pairwise node matching problems solved through the validation of a logical formula using a SAT solver. Among the systems which use this approach, let us cite CtxMatch [START_REF] Bouquet | Bootstrapping semantics on the web: meaning elicitation from schemas[END_REF]] and S-Match [START_REF] Giunchiglia | S-match: an algorithm and an implementation of semantic matching[END_REF].

When compared to the state of the art, we propose a hybrid approach combining three types of information: (i) lexical, (ii) structural, and (iii) semantic information to align ontologies. Our first challenge was to fit with the real-world use cases of the Silex company. The analysis of the Silex data showed that the labels of the entities of ontologies to be aligned are not very close at the lexical level. Therefore, string-based metrics are not very useful in this case. Then we moved towards word embedding. We experimented training our own embedding model, but we got poor results as the available corpus is not rich enough. Finally we decided to use the fastText model as it is the only model that provides word embedding for French. Based on the results obtained by [START_REF] Schmidt | Matching domain and top-level ontologies exploring word sense disambiguation and word embedding[END_REF], which prove that word embedding preformed this task better than Word Sense Disambiguation, we decided to ignore the use of Word Sense Disambiguation in our approach.

Additionally, we considered extracting the semantics of the concepts based on the structure of the ontology. According to Aristotle's fundamental predictive theory, the semantics of a concept is mainly defined by the difference between this concept and its genus, or more generally its ascendants in the ontology [START_REF] Parrochia | Taxinomie et réalité: vers une métaclassification[END_REF]. Therefore, in the ontology alignment literature, several works use information associated to more general concepts when searching matchings between two concepts, as this generalization of concepts is bringing more context. In our approach, we also consider taking into account the specialization of concepts when computing matchings, considering that more specific concepts will also bring additional context and semantics. To the best of our knowledge, there is no previous work considering this information in the ontology alignment process.

Overview of our ontology alignment approach

Problem statement

The goal of ontology alignment is to discover the relationships between entities of ontologies.

Our alignment process, illustrated in Figure 4.2, is a hybrid approach combing lexical information, structural information and semantic information expressed in the embedding space to refine the nature of the relationship between entities. In the rest of this section, we detail the four successive steps of our approach.

We consider indifferently RDFS, OWL or SKOS vocabularies, and two languages, namely French and English. The language must be chosen at the beginning of the alignment process to ensure that the right word embedding model is selected.

Extracting lexical and structural information from ontologies

The first step of our approach is to extract lexical information and structural information from the ontologies to be aligned. To achieve this, the two ontologies are parsed with rdflib and queried with the SPARQL query shown in Listing 4.1. Lexical information is extracted from the values of the properties rdfs:label for RDFS or OWL ontologies or skos:prefLabel for SKOS vocabularies.

Structural information is captured by associating the labels of all child entities to their parent entities, considering rdfs:subClassOf or rdfs:subProper-tyOf properties instead of skos:broader. As a result, we consider clusters of entities specializing the root entity in each cluster. • lexical_information(#J) = {Information and communication}.

• structural_information(#J) = {Information and communication, Publishing activities, Computer programming, consultancy and related activities, Telecommunications, Wired telecommunications activities, Wireless telecommunications activities, Satellite telecommunication activities, Other telecommunications activities}.

Computing word embedding representations

Based on the extracted information, we compute the word embedding representation of entities. We define two types of vector representations: (i) the vector representation of an entity (lexical information) and (ii) the vector representation of a cluster of entities (structural information).

We use the pre-trained word vectors for French and English, learned using fastText4 on a Wikipedia dump. The French model contains 1,152,449 tokens, and the English model contains one million tokens. Both are mapped to 300-dimensional vectors [Mikolov et al., 2013b].

A pre-processing step is required to convert words to lower case and remove all stop words.

The process of computing the vector representation of the entities is similar to creating the vector representation of sentences since in several cases the label of an entity is composed of multiple words. So the vector representation of the entity is computed by averaging the word embedding vectors along each dimension of all the words contained in its label and occurring in the dictionary:

entityW ordEmbedding(c) = 1 n n i=1 w i , (4.1)
where n is the number of words in the dictionary occurring in the label of an entity c and w i ∈ R 300 denotes the word embedding vector of the ith such word (if a word in a label does not appear in the dictionary, it is just ignored).

The vector representation of a cluster of entities is constructed by averaging the word embedding vector representations of the entities belonging to it:

end i f end f o r end f o r clusterW ordEmbedding(cl) = 1 k k i=1 entityW ordEmbedding(c i ), (4.2)
where c i is an entity in the cluster cl and k = |cl|.

Searching for matching entities

The semantic similarity between an entity of the source ontology and an entity of the target ontology is calculated by considering their vector representations.

The common similarity metric for embedding is the cosine similarity measure. We consider that a correspondence exists between two entities when the cosine similarity between them is bigger than a given threshold. Our algorithm aims at collecting all the possible correspondences between entities to propose many-to-many mappings (i.e. one entity from one ontology can correspond to more than one entity in the other ontology). Listing 4.2 shows the pseudo-code of our algorithm to discover the correspondences.

Refining the nature of the relationship between two matching entities

At this stage, for each entity in the source ontology we have a list of matching entities in the target ontology. We must now decide of the nature of the relationships holding between entities of the source and target ontologies: an equivalence relationship or a hierarchical relationship depending on the degree of similarity between two matching entities, considering the clusters of which they are the root. More precisely, the relationship between two matching entities e 1 and e 2 is refined by comparing the radii of their respective embedding vector clusters, computed by taking into account the hierarchical structure of the two ontologies: The radius of a cluster is the maximum distance between the centroid of the cluster and all the other entities in the cluster. We define the radius of a cluster of entities as the standard deviation of their cosine dissimilarity with respect to the centroid:

radius = 1 N N i=1 1 - w i • w |w i | • |w| 2 , (4.3)
where w i ∈ R 300 is the vector representation of the i th entity in the cluster, N is the size of the cluster, and w ∈ R 300 is the centroid of the cluster, defined as 4.4 shows two example clusters associated to entity Telecommunication (the bigger circle) and the entity Telecommunication service (the smaller circle). To define the type of the relationship we compare the radii of two matching clusters. These two clusters are formed mainly using structural information. We suppose that the cluster whose result has the smallest average distance between a label and the centroid is in broader relation with the cluster which has the largest radius. As shown in Figure 4.4, the blue circle (which represents the cluster of telecommunication service, voice mail administration service, and mobile phone administration services) is in broader relation with the big circle (which represent the cluster including telecommunications, wired telecommunications activities, and satellite telecommunications activities).

w = 1 N N i=1 w i .
We define the two following rules to identify the relationship holding between two similar entities:

|radius(e 1 ) -radius(e 2 )| < 0.1 ⇒ e 1 closeM atch e 2 (4.4) radius(e 1 ) -radius(e 2 ) > 0.1 ⇒ e 1 narrowM atch e 2 ∧e 2 broadM atch e 1 (4.5)

In particular, the first condition above is trivially satisfied when both e 1 and e 2 are leaf nodes of their respective ontologies and their radii are both zero.

We represent equivalence relationships by using owl:sameAs properties when aligning RDFS or OWL vocabularies and skos:closeMatch properties when aligning SKOS vocabularies. We represent hierarchical relationships by using rdfs:subClassOf or rdfs:subPropertyOf properties for RDFS or OWL vocabularies and skos:broader and skos:narrower properties for SKOS vocabularies.

Experiments

In this section we describe the experiments conducted to evaluate the above described proposed approach to ontology alignment: the datasets we considered, the experimental protocol we adopted and the results of these experiments.

Datasets

We experimented our proposed approach to ontology alignment on ontologies coming from a public benchmark and two specific use cases described in the following.

Experiments on task-oriented complex Alignment on conference Organization

We experimented our approach on the conference complex alignment benchmark [START_REF] Thieblin | Task-oriented complex alignments on conference organisation[END_REF] for ontology merging. This benchmark has been 4.1 summarizes the number of entities by type contained in these ontologies [START_REF] Thiéblin | Task-oriented complex ontology alignment: Two alignment evaluation sets[END_REF].

Silex use case

As described in Chapter 3, an ontology engineering work was carried out in the Silex company, including a manual ontology alignment task to establish correspondences between targeted referentials in the computing sector: (i) ESCO to Cigref, (ii) ESCO to ROME, (ii) NAF to UNSPSC, (iv) NAF to Silex_activity. Table 4.2 presents the number of concepts in each of the modules building up the Silex vocabulary for the computing sector, and Table 4.3 presents the number alignment per relation. We consider the set of the manually stated alignments as a test-bed for the automatic alignment approach we propose.

Evaluation protocol

The performances of our approach are measured by calculating precision, recall and F-measure [START_REF] Ochieng | Large-scale ontology matching: State-ofthe-art analysis[END_REF]. In addition to this stateof-the-art evaluation method and taking into account the fact that our system was not designed to achieve a fully automatic sourcing process but rather to support end-users responsible for the sourcing task, by presenting a list of possible matches, we defined another evaluation method assuming that if a system is able to propose a list of k best possible matches which includes the correct match, we consider that the matching is correct. This way of evaluation does not only concern the precision metric but also the recall and F1 metrics since the correspondence is no longer considered as False Positive but as True Positive. We conducted the parameter learning (i.e threshold) through 5-fold cross validation.

Results and discussion

Experiments on task-Oriented complex alignment on conference organisation

We compared our matching results with the results of three state-of-the-art complex ontology matchers that were evaluated in [START_REF] Thiéblin | Task-oriented complex ontology alignment: Two alignment evaluation sets[END_REF], namely: [START_REF] Ritze | A pattern-based ontology matching approach for detecting complex correspondences[END_REF]and implementing a rulebased approach mostly relying on string similarity; 2. the system presented in [START_REF] Ritze | Linguistic analysis for complex ontology matching[END_REF] and implementing another rule-based approach using linguistic evidence; and 3. the KAOM system presented in [Jiang et al., 2016b] and using a probabilistic framework based on Markov Logic networks.

We searched the literature for other, more recent ontology alignment systems evaluated against the same benchmark, but we could not find any, probably due to the novelty of the benchmark. Table 4.4 shows that our system clearly outperforms the others on this benchmark, with an F1 of 0.27 and we can reach an F1 of 0.51 using our evaluation method, confirming the interest of looking at clusters of entities in an embedding space both to establish correspondences between them and to resolve the nature of their relations.

Silex and ONISEP use cases

Table 4.5 presents the results of our system in the real world data from Silex and ONISEP use cases. For the Silex data, the F1 value is around 0.5 and we can reach an F1 ranges between 0.8 and 0.97 using our evaluation method.

For the ONISEP data, the F1 value is 0.52 and we can reach an F1 of 0.93 using our evaluation methods. We conducted some additional experiments in which we add the parent label to the label of a concept. We decided to conduct these experiments because we noted that several state-of-the-art proposals [START_REF] Gracia | Semantic heterogeneity issues on the web[END_REF] have been made on the basis of such a bottom-up approach instead of a top-down approach. The experiment shows that the use of this information severely decreases the performance of our alignment system. For example, the F1 value when matching NAF and UNSPSC decreases from 0.50 to 0.11, and when matching ESCO and cigref it decreases from 0.60 to 0.1.

Although it looks like a dramatic step ahead with respect to the state of the art, our system still has much room for further improvement. There are four main issues that could be addressed:

1. The cosine similarity between some entities that should be matched is much lower than the matching threshold and as a consequence these matches are ignored. For example the cosine similarity between 'chairman' and 'demo chair' is 0.37. 2. Our system is not designed to test hierarchical relations between two leaf nodes. This type of relationship must pass through the structural information to calculate the radius and, thus, infer the relationship. For example, in the benchmark, 'country' and 'location' are two leaf nodes that have been matched by rdfs:subClassOf.

3. Based on Equations 4.4, our system can assign an equivalence relation instead of a hierarchical relation because the threshold of the difference of radius between two classes is smaller than 0.1.

4. The quality of the embedding space depends on the context of the data and the similarity between the training data and the ontology data. Therefore, the quality of our system is tightly dependent on the embedding model.

Conclusion

In this chapter we presented a novel approach of ontology alignment, based on measuring the clusters of labels in an embedding space to refine relations in ontology alignment. We reported the results of our experiments on multiple datasets: (i) the OAEI conference complex alignment benchmark; (ii) the realworld use case encountered by the Silex company, namely matching skills and competences from several ontologies in the computing field; and (iii) the realworld use case encountered by the ONISEP, namely matching occupations between the ONISEP and the ROME vocabularies. These experiments show that our approach outperforms state-of-the-art approaches and is well suited to real world use cases, where the goal would be to propose possible alignments to experts that should be validated, as it is the case for Silex or ONISEP. In the next chapter, we present our approach for semantically annotating descriptions of providers and service requests and recommend relevant providers for a given service request, based on the aligned Silex vocabulary.

Chapter 5

Named Entity Recognition and Linking

Introduction

In a number of areas, companies are often faced with the task of dealing with large amounts of textual data. Automating knowledge extraction can help to accelerate the processing of data and this by giving the machines the possibility to execute certain tasks. In this chapter, we report our knowledge extraction approach which is composed of: (i) a named entity recognition (NER) approach based on Bi-LSTM-CRF architecture able to analyze textual descriptions (service providers and service requests) and extract the relevant parts of the text that summarize a provider's offer/ a request need (such as services, products, occupations, skills); (ii) a named entity linking (NEL) algorithm based on semantic similarity to link the extracted entities from the descriptions of service requests and providers with the concepts in the sourcing vocabulary.

In this chapter, we address the following research questions:

• Which is the best approach to extract knowledge from short texts?

• Which types of embedding must we use to extract relevant knowledge in our case?

• How can we link the extracted entities with our sourcing vocabulary?

This chapter is organized as follows: Section 5.2 presents the related works for NER and NEL. Section 5.3 describes our knowledge extraction approach. Section 5.4 describes our data and our implementation. Section 5.4.2 reports and discusses the results of our experiments. Section 5.5 concludes with an outline of future work.

Related Work

Named entity recognition

In this section, we focus on the related works for named entity recognition. In the literature, there are three common techniques for the NER task [START_REF] Yadav | A survey on recent advances in named entity recognition from deep learning models[END_REF][START_REF] Li | A survey on deep learning for named entity recognition[END_REF][START_REF] Huu Nguyen | Toward mention detection robustness with recurrent neural networks[END_REF][START_REF] Li | A survey on deep learning for named entity recognition[END_REF]: (i) knowledge-based systems; (ii) feature-engineered supervised systems; and (iii) feature-inferring neural network systems.

Knowledge-based systems

Knowledge-based systems rely basically on lexical resources and specific domain knowledge. They are called unsupervised systems as they do not need annotated training data. These systems have the advantage of having a high precision, but they present two main drawbacks: (i) most of the time, the recall is low due to incomplete dictionaries; and (ii) domain experts are needed to construct and maintain the knowledge vocabulary.

Feature-engineered supervised systems

In these systems, the NER task can be seen as a multi-class classification or sequence labelling task. Given inputs and their expected outputs, supervised systems learn how to make predictions.

Feature-engineered supervised systems are based on two main approaches: (i) an approach based on the representation of each training example. Each word in the text is represented using one or more features like word-level features (e.g. case, morphology and POS), list lookup features (e.g. Wikipedia gazetteer and DBpedia gazetteer), and corpus feature (e.g. local syntax and multiple occurrences); (ii) an approach based on a machine learning to learn a model to recognize similar patterns from unseen data. Among the common machine learning systems used for NER, we can cite Hidden Markov Models (HMM) [START_REF] Eddy | Hidden markov models[END_REF], Support Vector Machines (SVM) [START_REF] Hearst | Support vector machines[END_REF], Conditional Random Fields (CRF) [START_REF] Lafferty | Conditional random fields: Probabilistic models for segmenting and labeling sequence data[END_REF], and decision trees [START_REF] Quinlan | Simplifying decision trees[END_REF].

IdentiFinder [START_REF] Daniel | Nymble: a high-performance learning name-finder[END_REF]], is the first NER system based on HMM to identify names, dates, time expressions and numerical quantities. [START_REF] Malouf | Markov models for language-independent named entity recognition[END_REF] uses Maximum Entropy (ME) with multiple features like capitalization and a list of first names collected from various dictionaries. [START_REF] Mcnamee | Entity extraction without languagespecific resources[END_REF] train SVM classifiers using 1000 language-related features and 258 orthography and punctuation features. [START_REF] Mccallum | Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons[END_REF] propose a NER system using a feature induction method for CRF.

Feature-inferring neural network systems

There are many existing taxonomies for feature-inferring neural network systems in the literature [START_REF] Yadav | A survey on recent advances in named entity recognition from deep learning models[END_REF][START_REF] Huu Nguyen | Toward mention detection robustness with recurrent neural networks[END_REF]].

Here we adopt the presentation of [START_REF] Li | A survey on deep learning for named entity recognition[END_REF] because it seems to us the most structured and understandable, especially with the distinction of three steps. According to [START_REF] Li | A survey on deep learning for named entity recognition[END_REF], a NER feature-inferring neural network system or a NER Deep Learning system can be broken down into three steps: (i) distributed representations for input; (ii) context encoder; and (iii) tag decoder to predict tags.

Distributed representations for input

Word-level representation [START_REF] Collobert | A unified architecture for natural language processing: Deep neural networks with multitask learning[END_REF] is one of the first neural network architectures for NER. This system uses feature vectors constructed from orthographic features (e.g., capitalization of the first character), dictionaries and lexicons. With the advent of word embedding, [START_REF] Collobert | Natural language processing (almost) from scratch[END_REF] propose a semi-supervised method where these manually craffed feature vectors are replaced with word embedding using a convolutional neural network (CNN). Word embedding is then intensively used as input of many NER systems [START_REF] Yao | Biomedical named entity recognition based on deep neutral network[END_REF][START_REF] Zhou | Joint extraction of multiple relations and entities by using a hybrid neural network[END_REF][START_REF] Huu Nguyen | Toward mention detection robustness with recurrent neural networks[END_REF] in various domains. Word embedding can be either fixed or further fine-tuned during NER model training.

Character-level representation Several studies introduce a characterlevel representation learned by a neural model as an input of their systems. The advantages of this representation are basically: (i) to exploit sub-wordlevel information such as prefix and suffix; (ii) to handle out-of-vocabulary words. CNN-based model or RNN-based model (e.g. LSTM and Gated Recurrent Unit (GRU)) can be used to extract character-level representations. CharNER [START_REF] Kuru | Charner: Character-level named entity recognition[END_REF] is a character-level tagger for language independent NER. CharNER uses LSTM to extract character-level representations and produces a tag distribution for each character instead of each word. Furthermore, [START_REF] Lample | Neural architectures for named entity recognition[END_REF]; [START_REF] Ma | End-to-end sequence labeling via bi-directional lstm-cnns-crf[END_REF] use a bidirectional LSTM to extract character-level representations of words. Each input vector is a concatenation of pre-trained word-level embedding and character-level representations. [START_REF] Chiu | Named entity recognition with bidirectional lstm-cnns[END_REF] present a hybrid bidirectional LSTM and a bidirectional CNN neural network architecture that help to exploit explicit character-level features such as prefixes and suffixes, which could be useful especially with rare words for which word embedding are poorly (or not) trained. Santos and Guimaraes [2015] introduce the neural character embedding in the NER task for English, and achieve the state-of-the-art. [START_REF] Jie | Dependency-guided lstm-crf for named entity recognition[END_REF] propose a simple LSTM-CRF model for NER that takes the complete dependency trees. [START_REF] Anastasyev | Improving part-ofspeech tagging via multi-task learning and character-level word representations[END_REF] explore ways to improve point-of-sale labeling using different types of auxiliary losses and different representations of words. They built their model based on Bi-LSTM layers, and showed that introducing word representations through their characters gives better results.

Hybrid representation

In additional to word-level and character-level representations, some systems introduce additional information (e.g. lexical similarity, gazetteers, linguistic dependency and visual features) into the final representation of words. Enriching the input sequences of the neural network with accessible additional data can also help to have better results. Many systems prove that adding additional information improves the NER performance. [START_REF] Huang | Bidirectional lstm-crf models for sequence tagging[END_REF] use a BiLSTM-CRF with four types of features, namely spelling features, context features, word embedding, and gazetteer features. Their results show that combining multiple features improve the tagging accuracy. [START_REF] Wei | Disease named entity recognition by combining conditional random fields and bidirectional recurrent neural networks[END_REF] present a CRF system for NER of disease names, by employing multiple features, such as word embedding, POS tags, chunking, and word shape features (e.g., dictionary and morphological features). [START_REF] Bill Yuchen Lin | Multi-channel bilstm-crf model for emerging named entity recognition in social media[END_REF] build their word representation by concatenating character-level representations, word-level representations, and syntactical word representations (i.e., POS tags, dependency, word positions, head positions).

Context encoder

The second step of NER deep learning is capturing the context dependencies using CNN, RNN or other network architectures. [START_REF] Collobert | Natural language processing (almost) from scratch[END_REF] propose a NER system using a CNN. The main problem with this system is that it does not take into account long-term dependencies between words, because it is based on a simple feedforward neural network, and limits the use of context to a fixed-size window. To overcome this limitation, [START_REF] Collobert | Natural language processing (almost) from scratch[END_REF] propose a RNN deep learning algorithm for the sequence labeling task. [START_REF] Zhai | Comparing cnn and lstm character-level embeddings in bilstm-crf models for chemical and disease named entity recognition[END_REF] provide a comparison between NER systems using a CNN or RNN models. LSTM networks are a particular type of RNN that are designed to avoid this problem through the use of LSTM cells, which make it easy to learn about long-term dependencies [START_REF] Felix A Gers | Learning to forget: Continual prediction with lstm[END_REF]. [START_REF] Lample | Neural architectures for named entity recognition[END_REF]; [START_REF] Huang | Bidirectional lstm-crf models for sequence tagging[END_REF] propose a more powerful neural network model that incorporates Bi-LSTM and CRF. The Bi-LSTM model takes into account the whole context, which enables it to effectively train a model with the flexible use of long-range context [START_REF] Graves | Speech recognition with deep recurrent neural networks[END_REF].

Tag decoder

The final stage in a NER model is tag decoder. It takes context-dependent representations as input, and produces a sequence of tags corresponding to the input sequence. Multi-layer perceptron and a Softmax layer are used early as a tag decoder as numbers of NER models [START_REF] Strubell | Fast and accurate entity recognition with iterated dilated convolutions[END_REF]Li et al., 2017;[START_REF] Xu | A local detection approach for named entity recognition and mention detection[END_REF][START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF][START_REF] Cui | Hierarchically-refined label attention network for sequence labeling[END_REF]. CRF represents the most common choice for tag decoder in many NER deep learning systems. Some are using CRF on top of the bidirectional LSTM layer [START_REF] Huang | Bidirectional lstm-crf models for sequence tagging[END_REF][START_REF] Zheng | Joint extraction of entities and relations based on a novel tagging scheme[END_REF]Peters et al., 2018b;[START_REF] Lin | Reliability-aware dynamic feature composition for name tagging[END_REF]; others use CRF on top of the CNN layer [START_REF] Collobert | Natural language processing (almost) from scratch[END_REF][START_REF] Strubell | Fast and accurate entity recognition with iterated dilated convolutions[END_REF][START_REF] Yao | Biomedical named entity recognition based on deep neutral network[END_REF].

When compared to the sate-of-the-art, our NER approach for the service request is mostly related to feature-inferring neural network systems. Character-based representations are very important in our use case. Since our data are user-generated, it is important to capture morphological and orthographic patterns.

Named entity linking

In this section, we focus on the related works for named entity linking. In the literature, many surveys are devoted to the named entity linking task [START_REF] Al-Moslmi | Named entity extraction for knowledge graphs: A literature overview[END_REF][START_REF] Ling | Design challenges for entity linking[END_REF][START_REF] Rao | Entity linking: Finding extracted entities in a knowledge base[END_REF][START_REF] Sevgili | Neural entity linking: A survey of models based on deep learning[END_REF][START_REF] Wu | Entity linking: an issue to extract corresponding entity with knowledge base[END_REF][START_REF] Italo | Towards holistic entity linking: Survey and directions[END_REF][START_REF] Shen | Entity linking with a knowledge base: Issues, techniques, and solutions[END_REF]. These surveys present the different methods to perform the NEL task by following the general architecture of the NEL process which is composed of three steps:

(i) candidate entity generation which aims to retrieve a list of all possible entities in the knowledge base that may refer to each entity mention and filter out the irrelevant ones; (ii) candidate entity ranking aims to rank the list of candidate entities selected from the last step (i.e. candidate entity generation) and return the closest one for each entity mention; and (iii) NIL clustering or unlinkable mention prediction tries to deal with entities that do not have their corresponding entities in the knowledge base.

Candidate entity generation

There are three main methods to generate the entity candidate:

Dictionary based methods: These methods are based on the construction of a dictionary by using some features from Wikipedia such as entity pages, redirect pages, disambiguation pages, bold phrases from the first paragraphs, and hyperlinks. This type of dictionary provides information about name variations and represents a good way to generate candidate entities [START_REF] Shen | Entity linking with a knowledge base: Issues, techniques, and solutions[END_REF].

Surface form matching methods: These methods use the surface forms of mentions in the text to compose the candidates entities list.

Probability based methods: These methods compute the matching probability between the mentioned entity and the corresponding entity in the knowledge base to select the candidates entities. Usually a high value of probability implies that the entity in the knowledge base can be a selected candidate.

Candidate entity ranking

Three methods are presented in the literature to rank the candidate entity:

Similarity computation methods: These methods are based on the comparison of the similarity between the vector representation of each entity mention and the vector representation of the candidate entity in the knowledge graph using similarity measures (i.e cosine similarity, Jaccard similarity, etc.) To rank candidates entities, [START_REF] Bunescu | Using encyclopedic knowledge for named entity disambiguation[END_REF] use the cosine similarity between the bag of word vector of the context of an entity mention and the bag of word vector of the Wikipedia page candidate entity. Then the target entity is represented by the entity with the maximum cosine similarity score.

Machine learning and deep learning methods: These methods use either a binary classification model such as SVM or Naive Bayes, or the deep learning methods to decide if a candidate entity is the target entity.

Graph methods:

These methods are based on graph construction, in which the nodes represent the entities mentions and candidates entities, and the edges link each mention node with its candidates entities. After that, several techniques can be used to select the best candidate such as random walk for example.

NIL clustering

In this part, we give a brief overview of the three main approaches used in the literature to tackle with entities that have not been linked with mentions in the KB.

String matching methods:

These methods compute the string similarity between entities to group them into clusters.

Hierarchical agglomerative clustering methods: These methods try to merge entities of each cluster until the distance between clusters is smaller than the threshold.

Graph methods:

These methods are based on the construction of a semantic graph of entities before applying the hierarchical agglomerative clustering method.

Our NEL approach focuses on the first two steps of the NEL architecture and is based on the similarity score to choose the best concept for each extracted entity.

Our approach

Named entity recognition

Our aim in this step is to automatically analyze the textual descriptions of service requests and providers in order to extract the relevant entities. Those entities can be of different types. In general-purpose NER they are names of persons, organizations, places [START_REF] Cardellino | A low-cost, high-coverage legal named entity recognizer, classifier and linker[END_REF]. For the sourcing domain, we will consider names of skills, occupations, products (goods and services), and business activities.

The processing of these descriptions must address many challenges: (i) texts are generally short (50 words on average in our case); (ii) these texts are user-generated, and thus subject to typing errors; (iii) in some requests, users may describe their own products to contextualize their request, which would create confusion. This raises the issue of distinguishing between the user's real need and the general context of the sourcing request in its description.

To handle this type of textual data, our approach [START_REF] Daoud | A hybrid bi-lstm-crf model for sequence labeling applied tothe sourcing domain. 6ème conférence sur les Applications Pratiques de l'Intelligence Artificielle APIA[END_REF]] is inspired by [START_REF] Bill Yuchen Lin | Multi-channel bilstm-crf model for emerging named entity recognition in social media[END_REF] and is based on a Bi-LSTM-CRF architecture, able to analyze textual descriptions of service providers and extract the relevant parts of the text that summarize a customer need or company offer. In addition to word embedding, we extract three other kinds of embedding for each word in the textual description: (i) a syntax based embedding; (ii) a character-level based embedding; and (iii) a position-based embedding. These three types of embedding are extracted with Bi-LSTM, and concatenated with a word embedding. As there is no large enough corpus available that specifically fits our use case to train a word embedding model, we resort to pre-trained word vectors for French, learned using fastText. The vector representation resulting from the concatenation of the four embeddings is given as input to our main Bi-LSTM-CRF model.

Syntactic word representations (SWR)

The manual analysis of our data shows that syntax plays an important role to locate the user's need in a description. For example, many service requests are using specific verbs like "rechercher"/"chercher" (to look for), or "souhaiter" (to wish) before explaining their needs. Therefore, recognizing the sentence object would help the model to recognize the customer's need. We then trained part-of-speech (POS) embedding, using a Bi-LSTM Model.

Character-level word representations (CLWR)

This representation is used to represent rare words or words with spelling errors, which cannot be captured with word embedding, since fastText embedding is limited to the vocabulary on the training corpus. For every word, we use a Bi-LSTM that takes as input the sequence of the characters of the words, and returns the vector of the last hidden states. We consider this vector as a character-level based representation.

Position representation (PR)

Based on manual analysis of our data, it turns out that usually the main subject is mentioned at the beginning of the text. Hence, we used this embedding type to push the model to understand that it is highly likely that the words at the beginning of the text are relevant information. We use a Bi-LSTM model to extract this type of embedding.

All these types of representations are concatenated and used as input of the main Bi-LSTM-CRF bloc as shown in Figure 5.1.

Named entity linking with the sourcing vocabulary

In order to link the named entities extracted from the descriptions of service requests and providers with the concepts in the sourcing vocabulary, we defined a similarity measure between an entity and a concept, and we link an entity to the closest concept in the vocabulary. We mention that in this step we do not use all the repositories building up the Silex vocabulary, we used ESCO, ROME, NAF, UNSPSC and CPF. So, we ignore the silex internal repository, cigref as well as kompass. This decision is based on the fact that the selected ontologies are more structured and cover all domains. We first represent each extracted entity and each concept of the sourcing vocabulary by an embedding vector which is computed as the average of the word embedding vectors of all the words participating in the entity or label of concept and occurring in the dictionary. Here again, in the absence of a large corpus available to train a word embedding model for our use case, we use pre-trained word vectors for French learned using fastText.

The embedding vector for an entity or a (label of) concept x is thus computed as

V (x) = 1 n n i=1 w i , (5.1)
where n is the number of words of the dictionary occurring in x and w i ∈ R 300 denotes the word embedding vector of the ith word of x occurring in the dictionary. If a word of x does not belong to the dictionary, it is just ignored.

Then we define the similarity between an entity e extracted from a request or provider description and a concept c in the sourcing vocabulary as the cosine similarity between their embedding vectors V (e) and V (c):

sim(e, c) = V (e) • V (c) ||V (e)|| • ||V (c)|| . (5.2)
Finally, we link each entity with the most similar concept in the vocabulary O: linked 0 (e, c) ⇐⇒ sim(e, c) = max

c i ∈O
sim(e, c i ).

(5.3)

Experiments and results

Dataset and protocol

Our dataset for NER is composed of 883 descriptions of service requests distributed as follows: (i) 594 service request descriptions in the train set, (ii) 198 service request descriptions in the development set, and (iii) 90 service request descriptions in the test set. Data are annotated by sourcing experts at Silex according to the BIOES format which stands for Beginning (B-) to mark the beginning of an entity, Inside (I-) to mark the inside of an entity, Outside (O) to mark a token outside all of entities, End(E-) to mark the end of an entity, and Single (S-) to mark a single entity. Table 5.1 shows an example annotation using the BIOES format. We conducted four experiments to compare the performance of four kinds of models:

Result and discussion

In order to evaluate the performance of the NER approach to extract knowledge from the textual description of service request, we used the precision, recall, F1 score to measure the match between the entities automatically extracted by the system and the entities manually produced by the experts.

Let us note that even expert annotators find it sometimes hard to decide on the segment to annotate. In our evaluation, we do not consider the complete annotated entity, but rather words of the entity separately. For example, in the sentence "I am looking for a plumber able to repair a faucet Sprinkle", we do not fully penalize the algorithm if it does not detect the complete repair a faucet Sprinkle segment. But we count words that it could detect in that segment. Indeed, if we suppose that the model detects only repair a faucet, this may be enough to understand the need's subject. We also ignore conjunctions, determinants and punctuation in the evaluation. and the evolution of the loss function across epochs.

One can see that model IV, which uses all types of features, is the best model across all epochs. Model I, which takes as word representations only word embedding with logistic regression for tagging, has the lowest score. Using syntactic information with POS tagging significantly improves Dev and Test recall, and balances well precision and recall (see Table 5.2). We also note that with this model, we were able to get a similar F1 scores in dev and test data (difference of 0.39%).

Conclusion

In this chapter, we detailed our knowledge extraction step by firstly proposing a method that relies on Bi-LSTM-CRF for sequence labeling to summarize service requests. We combined several types of features to represent every word in a sequence: (iii) character-level based embedding, (ii) syntax based embedding, and (iii) position embedding. These additional embeddings are extracted using Bi-LSTM and concatenated with word embedding. We showed that the syntax and position of words help to improve the quality of the knowledge extraction in our use case. Moreover, we showed that Bi-LSTM-CRF architectures for information extraction can provide value even in a small-data context. Secondly, we presented our NEL algorithm, which is based on textual context to measure the textual similarity between the context around the extracted entity and the concept of our sourcing vocabulary. We mentioned that we adopted a knowledge-based system approach to extract named entities from the textual description of providers. We didn't detail this work in this chapter as it is a simple lexical search of the different words of concepts in the textual descriptions of the providers.

Introduction

Recommender Systems (RS) are a subclass of information filtering systems that seek to predict user preferences among a large selection of items [START_REF] Ricci | Introduction to recommender systems handbook[END_REF][START_REF] Pn | A survey on recommender systems (rss) and its applications[END_REF]. Nowadays, RS are primarily used in commercial applications to provide a personalized experience and suggest relevant items to users such as music, movies, books, trips, products, etc.

A strict word matching method between textual descriptions for the recommendation task would perform badly, as it does not take synonyms or polysemous words into account. To address that, word embedding models have been widely used to represent textual descriptions for the recommendation task in order to preserve the semantic and syntactic similarities between words. However, their main drawback is that they may lead to too generic a representation of texts, in the case where the available corpus is too small to train a specific word embedding model, and a pre-trained model is used instead, as it is the case in the context of the Silex platform.

In this chapter, we propose to combine a conceptual representation of texts to their representation based on word embedding to enhance the recommendation in the sourcing domain. Our main research question in this chapter is: Can the integration of domain knowledge enhance the performance of a RS in our use case in the sourcing domain? We focus on the following sub-questions:

• What is the best way to integrate domain knowledge into the representation of service requests and providers in order to enhance the quality of recommendations?

• To what extent does the injection of domain knowledge improve the performance of the system?

The remainder of this chapter is organized as follows. Section 6.2 gives an overview of state of the art on RS. Section 6.3 presents our approach. Section 6.4 reports and discusses the results of our experiments in the sourcing domain. Section 6.5 concludes and provides an outline of future work.

Related Work

RS generate meaningful recommendations to users for items that might be interesting to them. Several surveys on RS have been written [START_REF] Guo | A survey on knowledge graph-based recommender systems[END_REF][START_REF] Singh | Natural language processing for information extraction[END_REF][START_REF] Dong | Trust in recommender systems: A deep learning perspective[END_REF]. We can distinguish four types of RS:

Content-based recommender

A content-based recommender [START_REF] Lang | Newsweeder: Learning to filter netnews[END_REF] analyzes the content (i.e. set of attributes or metadata) of items liked by users in the past, and suggests items with similar content [START_REF] Philip | Data mining and personalization technologies[END_REF]. The similarity score between the user profile and the item profile is calculated using various methods such as Bayesian networks [START_REF] Park | A context-aware music recommendation system using fuzzy bayesian networks with utility theory[END_REF], neural networks [START_REF] Hidasi | Session-based recommendations with recurrent neural networks[END_REF], decision trees [START_REF] Golbandi | Adaptive bootstrapping of recommender systems using decision trees[END_REF], and TF-IDF [START_REF] Huang | A text similarity measurement combining word semantic information with tf-idf method[END_REF]. The top-scoring items are then recommended to the user [START_REF] Cheung | Mining customer product ratings for personalized marketing[END_REF]. The quality of the content-based recommendation can be improved over the time, and it can produce recommendations even with special tastes. However, it suffers from many drawbacks: (i) a poor recommendation result when the system does not have sufficient information about the particular preference of a new user (lack of information problem). (ii) The content-based system is not able to recommend new items that are different from the user preference, which limits the recommendation scope, and hides other items that are potentially interesting to the user (overspecialization).

Many content-based recommender systems use word embedding to represent the features of the processed texts. [START_REF] Musto | Word embedding techniques for content-based recommender systems: An empirical evaluation[END_REF] present a comparison of three kinds of vector representations based on word embedding to recommend movies and books: Latent Semantic Indexing, Random Indexing and word2vec. [START_REF] Shin | Recommending tumblr blogs to follow with inductive matrix completion[END_REF] use word2vec to compute a vector representation of tags and recommend Tumblr blogs to users. Ozsoy [2016] uses word2vec to recommend the next check-in location based on social networks. Finally, [START_REF] Elsafty | Document-based recommender system for job postings using dense representations[END_REF] use word embedding to build a recommender system for job postings.

Collaborative filtering

Collaborative filtering [START_REF] Shardanand | Social information filtering: algorithms for automating "word of mouth[END_REF] is the most successful approach. It is based on the feedback (rating) on items provided by similar users. There are two main techniques of collaborative filtering: (i) Memorybased techniques try to find neighbors for a user or an item by computing the similarity between the active user and the other users or by focusing on the similarities between items; and (ii) Model-based techniques build an inference model using different machine learning algorithms, such as Bayesian networks, clustering, Markov decision processes, Sparse factor analysis, and rule-based approaches.

The advantage of collaborative filtering is that it does not require massive amounts of data to yield good results, and the overspecialization problem can be resolved by finding similar users in the system. However, the recommendation of a new item or new user can be poor when the system does not have any interaction information (users, previous recommendations), a problem known as cold start. There are also other problems such as the sparsity problem and synonym problem.

Knowledge-based recommendation

Knowledge-based recommendation [START_REF] Ameen | Knowledge based recommendation system in semantic web -a survey[END_REF] is about using ontologies to model knowledge about the user context, the item context and the domain, and to compute the similarity between items. Using knowledge-based recommendation, many problems of common RS are eliminated such as the cold start problem, having a very good accuracy and explainable recommendation. In contrast, the major limitation for this RS approach is the cost of knowledge modelling. Many recommender systems use domain knowledge to enhance the performance of their systems. In the e-learning domain, [START_REF] Zhuhadar | Multimodel ontology-based hybrid recommender system in e-learning domain[END_REF] propose a hybrid recommender system based on a multi-model domain ontology to represent the learning materials and a rule-based recommendation; [START_REF] Yu | Ontology-based semantic recommendation for context-aware e-learning[END_REF] propose an ontology-based approach for content recommendation by representing knowledge about the content, domain and the user context into an ontology; [START_REF] John K Tarus | A hybrid knowledgebased recommender system for e-learning based on ontology and sequential pattern mining[END_REF] propose an ontology-based method to model and represent the domain knowledge about the learner and learning resources. In the field of economics, [START_REF] Cantador | A multilayer ontologybased hybrid recommendation model[END_REF] propose a multilayered approach for a hybrid recommendation model, where the user interests are represented as concepts of domain ontologies, and a collaborative recommendation mechanism is applied based on the similarities between such content-based user profiles. [START_REF] Cui | Ontology-based top-n recommendations on new items with matrix factorization[END_REF] propose an approach for the recommendations of new items based on an ontology-based similarity and Matrix Factorization to predict the missing value in the user-item matrix. The similarity between items is dependent on their properties, that is, common features tend to increase similarity and non-common ones tend to diminish it. [START_REF] Werner | Ontology-based recommender system of economic articles[END_REF] present a system to recommend a set of economic articles based on a set of ontologies used to describe both articles and user profiles. Other works use embedding-based methods [START_REF] Zhang | Collaborative knowledge base embedding for recommender systems[END_REF][START_REF] Zhang | Learning over knowledge-base embeddings for recommendation[END_REF][START_REF] Ernst | Knowlife: a knowledge graph for health and life sciences[END_REF] to either build knowledge graphs to enrich the representation of items or build user-items graphs by introducing users into the graph which can directly model the user preference. The traditional path-based methods [START_REF] Zhao | Meta-graph based recommendation fusion over heterogeneous information networks[END_REF]; [START_REF] Shi | Semantic path based personalized recommendation on weighted heterogeneous information networks[END_REF][START_REF] Shi | Heterogeneous information network embedding for recommendation[END_REF]; [START_REF] Wang | Explainable reasoning over knowledge graphs for recommendation[END_REF] build a user-item graph and enrich the user and/or item representation using the path connectivity. Finally, unified methods [START_REF] Li | Unifying task-oriented knowledge graph learning and recommendation[END_REF][START_REF] Sha | Attentive knowledge graph embedding for personalized recommendation[END_REF] try to benefit from both the semantic embedding of knowledge graphs and semantic path patterns to fully exploit the information in the knowledge graph. This approach is based on the notion of embedding propagation [START_REF] Wang | Ripplenet: Propagating user preferences on the knowledge graph for recommender systems[END_REF]].

Hybrid recommendation

Hybrid recommendation [START_REF] Bibliography | Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions[END_REF] combines the three above, to take advantage of the benefits of each of them, and to overcome the limitations of using only one type of method.

In our use case, since we do not have a large enough historical corpus of service requests associated to the relevant recommended providers, and since we do not have any feedback on recommended providers, we adopt a knowledge-based approach combined with embedding methods to represent the description of service requests and providers. Our choice is motivated by the fact that knowledge-based recommendations are neither dependent on ratings, nor on information about a particular user to give recommendations. Our method is at the intersection of the content-based and knowledge-based approaches. The rationale behind our approach is to take advantage of the power of word embedding, which approximates a general semantic relation, and the power of domain knowledge, which models a more specific semantic relation, to provide high-quality recommendations.

Proposed Approach

We suppose that (i) a provider is relevant for a service request if the descriptions of the service request and the provider are semantically similar and (ii) the semantic similarity of the descriptions of service requests and providers is a function of the similarity of the domain named entities or concepts they are using.

Vector representation of service requests and providers

We aim to represent each service request or provider by a vector that summarizes the semantics of the entities extracted from its description. For each description of a service request or provider, we consider three alternative vector representations: (i) the average of the embedding vectors of the entities in the textual description; (ii) a bag of concepts representation; and (iii) a vector representation combining the two former ones.

Word Embedding of entities

The base vector representation V Emb (x) of a service request or provider x is the average of the embedding vectors of all the entities e i , i = 1, ..., n extracted from its description:

V Emb (x) = 1 n n i=1 V (e i ), (6.1)
where V (e i ) is the vector representation of entity e i as defined in Equation 5.1.

Bag of concepts

Using the result of the above described entity linking process, we consider an alternative representation V BoC (x) of a service request or provider x based on the sourcing vocabulary S: the bag of the concepts (BoC) in S which the entities e i extracted from x are linked to according to Equation 5.3:

V BoC (x) = BoC S (x) = (b 1 , ..., b m ) (6.2)
where m is the size of the sourcing vocabulary S and b i = 1 if ∃e ∈ x, linked S (e, c i ), and b i = 0 otherwise. Additionally, we considered enriching the BoC representation of a service request or provider, by considering not only the concepts linked to the entities it contains but also some neighbors in the vocabulary to the linked concepts. More precisely, we considered the parents of the concepts linked to the entities (skos:narrower relation) or those semantically close (skos:closeMatch relation). Formally, we define three alternative BoC representations:

V BoC (x) = BoC S (x) = (b 1 , ..., b m ) (6.3)
where m is the size of the sourcing vocabulary S and b i = 1 if ∃e ∈ x, linked S (e, c i ) or linked S (e, c j ) with c j skos:narrower c i ; and b i = 0 otherwise.

V

BoC (x) = BoC S (x) = (b 1 , ..., b m ) (6.4)
where m is the size of the sourcing vocabulary S and b i = 1 if ∃e ∈ x, linked S (e, c i ) or linked S (e, c j ) with c j skos:closeMatch c i ; and b i = 0 otherwise.

V BoC (x) = BoC S (x) = (b 1 , ..., b m ) (6.5)
where m is the size of the sourcing vocabulary S and b i = 1 if ∃e ∈ x, linked S (e, c i ) or linked S (e, c j ) with c j skos:narrower c i or c j skos:closeMatch c i ; and b i = 0 otherwise.

Combination of vector representations

We define a third type of vector representation of a service request or provider as the concatenation of the vector representations defined in Equation 6.1 and one of the BoC representations defined in Equations 6.2, 6.3, 6.4, and 6.5, respectively:

V Conc (x) = V Emb (x) V BoC (x), (6.6) V Conc (x) = V Emb (x) V BoC (x), (6.7) V Conc (x) = V Emb (x) V BoC (x), (6.8) V Conc (x) = V Emb (x) V BoC (x). (6.9)
Here, the symbol denotes the vector concatenation operator.

Recommendation algorithm

We define two metrics to measure the similarity between a service provider p and a service request r. The first one is the cosine similarity between the vector representations of p and r: 6.10) where V stands for one of the representations defined in Equations 6.1 to 6.9.

sim 1 (p, r) = V (p) • V (r) ||V (p)|| • ||V (r|| , (
The second metrics is (6.11) where sim 1 is then computed with the base vector representation V 1 . A service provider p is recommended for a service request r if sim 1 (p, r) or sim 2 (p, r) is greater than a given threshold, depending on the chosen similarity measure. An empirical study was conducted to choose the value of the threshold, making its value vary and computing the recall and precision measures.

sim 2 (p, r) = 1, if V (p) ∩ V (r) = 0, sim 1 (p, r), otherwise,

Experiments and results

Dataset and protocol

We evaluate the performance of our recommendation approach on a dataset provided by sales experts at Silex. This dataset comprises 109 descriptions of service requests and the 649 providers which were manually selected and recommended for these requests. They are in various areas: Computing, Marketing and commercial, Human resources management, General services, Finance, Industrial services, etc.

To evaluate and compare the different recommendation approaches defined in Section 6.3, we consider two sets of annotations of this dataset: A is the set of annotations automatically performed by our NER approach; A is the result of a manual cleaning of A that we performed, after realizing that, for some descriptions, automatic extraction is still far from perfect and can introduce noise in the process; this noise can consist of entities that are useless for the description of exceedingly long entities. For example, for the following service request description : "Cabinets en étude et évaluations environnementales, etudes d'impacts, environnementales, d'actualisation de plan massif incendie, natura 2000. Evaluations environnementales", the named entity extracted is "Cabinet en étude" which is too generic and can lead to an erroneous recommendation, because it would match any research firm in any domain, not just environmental evaluation.

V Emb sim 1 BoC V BoC sim 1 BoC V BoC sim 1 BoC V BoC sim 1 Boc V BoC sim 1 Conc V Conc sim 1 Conc V Conc sim 1 Conc V Conc sim 1 Conc V Conc sim 1 BoC→Emb V BoC and V Emb sim 2 Conc→Emb V Conc and V Emb sim 2
To decide on the optimal vector representation and algorithm to recommend service providers, we conducted eleven experiments whose settings are depicted in Table 6.1, the baseline being experiment Emb.

In order to evaluate the performance of the proposed settings, and therefore the interest of injecting domain knowledge into vector representations, we used the precision, recall, F1 score and "precision at N " metrics to measure the match between the recommendations automatically produced by the system and the recommendations manually produced by the experts. We conducted the parameter learning (i.e threshold) through 5-fold cross-validation.

Results and discussion

Table 6.2 and 6.3 presents the performance of our system for each tested setting in terms of precision, recall and F1 score with datasets A and A respectively.

In order to evaluate to what extent the performance of our proposed approach depends on the method adopted to automatically annotate the descriptions of the service requests and providers, we conducted some additional experiments in which we used DBpedia spotlight1 for NER. Table 6.4 presents the results obtained in terms of precision, recall and F1 score on dataset A when named entities are extracted with DBpedia spotlight [START_REF] Mendes | Dbpedia spotlight: Shedding light on the web of documents[END_REF][START_REF] Daiber | Improving efficiency and accuracy in multilingual entity extraction[END_REF]. Figures 6.1a and 6.1b present the performance of our system for each tested setting in terms of P@N on dataset A and dataset A respectively.

Let us first discuss the results on dataset A. The best results were achieved with the Conc method, where the recommendation is based on a vector representation concatenating word embedding and BoC, and the even when one cannot rely on a high-quality NER method.

Conclusion

In this chapter, we proposed a sourcing recommender system approach that exploits knowledge extracted from textual descriptions of providers and service requests to automatically suggest the best providers. In this work, we study the benefits of using ontological knowledge to improve our recommender process. We focus especially on the vector representation of the descriptions by evaluating the performance of the system using word embedding or injecting domain knowledge into the representation. We reported the results of our experiments on the Silex dataset. These experiments show that recommendation accuracy can be greatly improved through the injection of domain knowledge and especially the bag of concept representation in the recommendation process.

Chapter 7

Conclusion

103

This thesis falls within the framework of the RS domain. RS have become a crucial necessity for the companies and do not stop evolving these last years. In the context of Silex, RS is meant to be a specific type of decision support system that guides either directly the user or the Silex sale's staff to find the best providers for a service request. Silex collects data both by allowing users to provide a textual description of their professional activities, and by collecting the open data on the Web. As a result, the Silex dataset gathers data from all domains (i.e. computing, general service, marketing, Humans resources ...).

Motivated by the concerns of Silex, we addressed in this thesis the issue of defining a RS based on domain knowledge. Our approach relies on the following four bricks: (i) building a sourcing domain vocabulary, (ii) defining an ontology alignment approach to integrate various domain metadata repositories, (iii) defining a NER approach and a NEL approach to extract and link entities from the textual description of service requests and providers, and finally (iv) proposing a RS using conceptual representations to suggest the best suited service providers to answer a service request.

In this chapter, we summarize the work conducted in this thesis and we highlight our contributions. We report the limitations of our work and we discuss several future directions.

Summary of the thesis

The first contribution of this thesis is the construction of a vocabulary for the sourcing domain. This vocabulary has been used to semantically annotate text descriptions of service requests and providers based on four knowledge types: (i) skills, (ii) occupations, (iii) products, and (iv) business activities, in order to automatically generate high-quality recommendations of service providers. This vocabulary has been built using the NeON methodology by imagining usage scenarios and defining some competency questions. A top-down approach has been applied to reuse metadata repositories such as ESCO, ROME, NAF, UNSPSC and FCP. A bottom-up approach has been also applied to build an ontology from the company's internal data. Our sourcing domain vocabulary contains 125,488 concepts.

The second contribution of this thesis is the proposal of a novel approach to ontology alignment. The motivation of this work is to automatically align all metadata repositories used in the sourcing vocabulary. Our approach relies on the use of word embedding and measuring the spread clusters of labels to discover the relationship between entities. We tested our approach on multiple datasets: (i) the OAEI conference complex alignment benchmark, (ii) the real-world use case encountered by the Silex company, and (iii) the real-world use case encountered by the ONISEP. The experimental results show that the combination of word embedding and a measure of dispersion of the clusters of labels, which we call the radius measure, makes it possible to determine, with good accuracy, not only equivalence relations, but also hierarchical relations between entities.

The third contribution of this thesis is the proposal of a knowledge extraction approach by presenting a NER approach based on a BiLSTM-CRF model, which combines four features types (i.e. word embedding, characterlevel based embedding, syntax based embedding, and position embedding), and a NEL algorithm based on computing the semantic similarity between the extracted entities and the concept of the sourcing vocabulary using word embedding. The experimental results relevant to the NER approach show that the combination of these four type of features helps to improve the quality of the knowledge extraction in the Silex use case.

The final contribution of this thesis addresses the proposal of a sourcing RS based on domain knowledge. We focused on the vector representation of the descriptions. Our proposal is to study the benefits of using knowledge, and more specifically, the 'bag of concepts' representation to enhance a RS in the sourcing domain. We tested our approach in a real-world case study provided by the Silex company. The experimental results show that injecting knowledge in the recommendation process outperforms word embedding approaches.

Technological transfer

As a result of the work carried out for this thesis, Silex now owns an aligned sourcing vocabulary. The sourcing vocabulary was integrated into the platform and was used to extract knowledge from the textual description of providers. This point is only briefly introduced in Chapter 5, because we have not evaluated the performance of this method yet. The NER approach applied to the service requests was integrated into the Silex platform.

Currently, usability tests for our RS approach are in progress and are based on a specific keyword set provided by the Silex product team to automatically recommend providers covering the whole provider database of Silex. We can also imagine that these tests can be useful not only for the integration of our approach in the platform, but also to know if our approach is sensitive to different business activities.

Limitations and perspectives of the proposed approach

The work presented in this thesis explores a range of techniques to automatically suggest the providers that best suit a service request. For some of them, there are several directions of improvements. In this section, we present some possible improvements and perspectives we aim at undertaking to overcome some of the limits identified in our work and to open follow-up research directions. Table 7.1 summarizes the perspectives of this section.

Construction of the sourcing vocabulary

The first improvement concerns the construction of the sourcing vocabulary.

The actual vocabulary comprises a big number of concepts, but we regret that we do not use the richness of the internal vocabulary of Silex in an efficient way. We have started a vocabulary reconstruction work using a clustering approach, but what is currently lacking is a principled way to assign labels to the formed clusters to be able to align them with the the current vocabulary, and to give Silex the chance to have its own specific vocabulary. As a perspective, we aim at automatically labeling hierarchical clusters. A starting point will be the work of [START_REF] Treeratpituk | Automatically labeling hierarchical clusters[END_REF], who suppose that by comparing the word distribution of the hierarchy, it should be possible to assign appropriate labels to each cluster in the hierarchy.

They propose an algorithm exploiting information on the cluster and the parent cluster and corpus statistics to assign a label to each cluster. Another potential direction is to support Silex in its internationalisation phase, and that by improving the sourcing vocabulary with other multilingual metadata repositories. This is a key feature for Silex to open up to the inter-national market. In the current version of our vocabulary, the most complete metadata that supports several European and non-European languages is ESCO, followed by UNSPSC. Adding the USA Occupational Information Network (O*NET),1 the China Industry Classification System2 and other industrial taxonomies can be a good track. We also encountered a small problem with UNSPSC: in the Excel file of the UNSPSC repository, the synonyms column provides the title synonyms of the concept in all languages separated with comma. We used the langdetect Python package to detect and automatically extract only the French language appellation from the synonyms columns. However, the extraction is not reliable enough, so we will have to think about another solution to have a good multilingual vocabulary. It will be good to find and test another language detector such as textblob3 or langrid library4 to extract more multilingual labels from UNSPSC repositories.

Another ambitious direction is to propose a continuous ontology updating solution that takes into account the permanent evolution of sourcing knowledge in the context of each country. We know that an ontology must to be modified over a period of time in order to reflect changes in the real word: a new concept might emerge, or a new concept might be introduced as the combination of two or more existing concepts (or conversely might be split into several concepts) [START_REF] Wardhana | Review of ontology evolution process[END_REF], etc. Therefore, capturing or even predicting these different changes is a real need in the field of sourcing.

Ontology alignment

The second improvement path concerns ontology alignment. The current algorithm presents the disadvantage of not being able to define the type of the relationship between two leaf nodes of ontologies. This is due to the fact that we use the notion of cluster, which is constructed in a top-down way in the ontology to deduce the type of relationship. To overcome the limitation of our ontology alignment algorithm on the leaf nodes, a potential solution would be to simply manipulate leaves differently from other nodes. Another improvement path would be to better capture the context of words using a contextual embedding. A now obvious perspective is to use BERT [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] or CamemBERT [START_REF] Martin | Camembert: a tasty french language model[END_REF] to represent each concept as a contextual embedding. On another note, it would also be useful to extend the evaluation of our system to the entire dataset or other datasets, and to empirically determine the optimal threshold value for the radius difference to be used in our alignment algorithm.

Named entity recognition and linking

As far as the Named entity recognition and linking is concerned, several improvement paths could be explored. To begin with, it might be interesting to improve the NER method. We discussed before the fact that, for some descriptions, the extraction is not yet perfect (extraction too generic or too long). A possible path to improve our NER algorithm is to use again a contextual embedding to extract entities or ideally to train our embedding model on our data in order to better extract the relevant entities. Concerning the NEL algorithm, we used the same principle for entity linking as for ontology alignment, i.e. to identify links between extracted entities and concepts, or links between two concepts. As a consequence, at first we considered that the evaluation of our approach for ontology alignment was enough. It is time now to evaluate it. The improvement of our EL algorithm first goes through evaluating the results of our approach (and not only on ontology alignment). And just like for ontology alignment, a possible improvement path is to use a contextual embedding.

Recommender system

The most prominent limitation of our recommender system approach is the absence of link between entities and concepts. This may be due to a false extraction produced by the NER or to the inability of the EL algorithm to produce the link with ontology concepts or simply to the absence of this concept in our ontology. The bulk of this limitation can be resolved by improving the previous steps of our approach. A promising direction is to exploit additional metadata on service requestors and providers to improve their matching, like location, number of employees, sales turnover. In addition, the inclusion of historical user data constitutes an interesting extension of our work. Another ambitious direction will be to study the manner to present the results of the recommendation to users. This line of research is not yet addressed for at Silex. We think that from the user's point of view, it will be more interesting to present the results of recommendations in the form of a graph whose nodes are the providers, the service requestors as well as the ontology concepts, and whose edges are the different links between these three types of nodes. We can present these different types of nodes with different colors and vary the size of the nodes based on the number of concepts in common between a provider node and a service request node. This will naturally highlight the best recommendation results to the users. Finally, our RS should be integrated within the Silex platform and usability tests should be conducted.

In conclusion, it is true that the path of ontology engineering seems a long and difficult one, especially when applied in industry, but it always proves its performance and effectiveness.

  . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Workflow of the proposed ontology alignment approach. . . . 4.3 An example of a hierarchy of concepts. . . . . . . . . . . . . . 4.4 Two example clusters of entities, one included into the other. 5.1 Model architecture (Embedding extraction + Main Bi-LSTM-CRF). 5.2 Evolution of score F 1 Dev . . . . . . . . . . . . . . . . . . . . . . 5.3 Evolution of the function. (loss) . . . . . . . . . . . . . . . . . . 6.1 Precision@N. . . . . . . . . . . . . . . . . . . . . . . . . . . . List of Tables 3.1 Metadata repositories. . . . . . . . . . . . . . . . . . . . . . . 4.1 Number of entities by type for each ontology. . . . . . . . . . 4.2 Number of concepts for the Silex ontology for the computing domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Number of relation types between concepts for the Silex ontology for the computing sector. . . . . . . . . . . . . . . . . . . 4.4 Evaluation of our approach on the OAEI benchmark. . . . . . 4.5 Evaluation of our approach on real world data from the Silex and ONISEP use cases. . . . . . . . . . . . . . . . . . . . . . 5.1 Example of input training data. . . . . . . . . . . . . . . . . . . 5.2 Precision, Recall and F1-score. . . . . . . . . . . . . . . . . . . . 6.1 Experimental settings ON RS. . . . . . . . . . . . . . . . . . . 6.2 Evaluation of the proposed experimental settings with dataset A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Evaluation of the proposed experimental settings with dataset A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Evaluation of the proposed experimental settings with dataset A, using DBpedia spotlight for NER. . . . . . . . . . . . . . . 7.1 Summary of our work perspectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Our research focus . . . . . . . . . . . . . . . . . . . 1.2.1 Domain knowledge modeling for sourcing data representation . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Design and adaptation of algorithms for knowledge extraction . . . . . . . . . . . . . . . . . . . . . . . 1.2.3 Recommend providers based on the formal representation of descriptions . . . . . . . . . . . . . . . 1.3 Our methodology . . . . . . . . . . . . . . . . . . 1.3.1 Building a vocabulary for the sourcing domain . . 1.3.2 Proposal of an ontology alignment approach . . . . 1.3.3 Proposal of an approach for sourcing named entity recognition and linking . . . . . . . . . . . . . . . . 1.3.4 Proposal of a recommender algorithm . . . . . . . 1.4 Our contributions . . . . . . . . . . . . . . . . . . 1.5 Thesis outline . . . . . . . . . . . . . . . . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: Overview of our recommendation approach.

Figure 2 .

 2 Figure 2.1: A simple NN architecture. Source: [Mohammed and Omar, 2012]

Figure 2 .

 2 Figure 2.2: A Bi-LSTM-CRF-architecture. Source:[START_REF] Huang | Bidirectional lstm-crf models for sequence tagging[END_REF]

Figure 3 . 2 :

 32 Figure 3.2: ROME example.

Figure 3 . 4 :

 34 Figure 3.4: NAF example.

Figure 3

 3 Figure 3.5: UNSPSC example.

Figure 3 . 6 :

 36 Figure 3.6: KOMPASS example.

Figure 3

 3 Figure 3.7: CPF example.

Figure 3 . 8 :

 38 Figure 3.8: Silex business activities vocabulary.

Figure 3 .

 3 Figure 3.10: Alignment example between ESCO, ROME, and Cigref.

Figure 3 .

 3 Figure 3.11: Example of ESCO-NAF alignment based on ROME.

  Figure 3.11 presents an alignment example between ESCO and NAF based on ROME.

Figure 4 .

 4 Figure 4.1: Matching techniques classification. Source [Euzenat and Shvaiko, 2013].

Figure 4 . 2 :

 42 Figure 4.2: Workflow of the proposed ontology alignment approach.

  Listing 4.1: SPARQL query to extract lexical and structural information from a SKOS vocabulary SELECT ? u r i ? l a b e l ( group_concat (DISTINCT ? mid_label ; s e p a r a t o r=":" ) AS ? l i n e a g e ) WHERE { ? u r i s k o s : p r e f L a b e l ? l a b e l FILTER ( l a n g ( ? l a b e l )='fr ' ) ? u r i ^s k o s : b r o a d e r * ?mid . ?mid s k o s : p r e f L a b e l ? mid_label . FILTER ( l a n g ( ? mid_label )='fr ' ) } GROUP BY ?mid ORDER BY count ( ? l a b e l ) Let us illustrate it using the hierarchy of Figure 4.3 as an example: • lexical_information(#61) = {Telecommunications} • structural_information(#61) = {Telecommunications, Wired telecommunications activities, Wireless telecommunications activities, Satellite telecommunication activities, Other telecommunications activities}.

Figure 4 .

 4 Figure 4.3: An example of a hierarchy of concepts.

Listing 4 . 2 :

 42 Pseudo-code to search for matching entities input : s o u r c e o n t o l o g y O 1 , t a r g e t o n t o l o g y O 2 , t h r e s h o l d _ s i m output : l i s t o f c o r r e s p o n d e n c e s l i s t=n u l l f o r each e 1 in O 1 do f o r each e 2 in O 2 do sim=c os ine _s im {O 1 , O 1 } i f sim> t h r e s h o l d _ s i m then l i s t . append ( e 1 , e 2 , sim )

Figure 4 . 4 :

 44 Figure 4.4: Two example clusters of entities, one included into the other.

Figure

  Figure 4.4 shows two example clusters associated to entity Telecommunication (the bigger circle) and the entity Telecommunication service (the smaller circle). To define the type of the relationship we compare the radii of

Contents 5 . 1

 51 Introduction . . . . . . . . . . . . . . . . . . . . . 78 5.2 Related Work . . . . . . . . . . . . . . . . . . . . . 78 5.2.1 Named entity recognition . . . . . . . . . . . . . . 78 5.2.2 Named entity linking . . . . . . . . . . . . . . . . . 81 5.3 Our approach . . . . . . . . . . . . . . . . . . . . . 83 5.3.1 Named entity recognition . . . . . . . . . . . . . . 83 5.3.2 Named entity linking with the sourcing vocabulary 84 5.4 Experiments and results . . . . . . . . . . . . . . 86 5.4.1 Dataset and protocol . . . . . . . . . . . . . . . . . 86 5.4.2 Result and discussion . . . . . . . . . . . . . . . . 87 5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . 89

Figure 5 . 1 :

 51 Figure 5.1: Model architecture (Embedding extraction + Main Bi-LSTM-CRF).

•

  I: Bi-LSTM model with only word embedding and logistic regression classification model • II: Bi-LSTM model with only word embedding and CRF classification model • III: Bi-LSTM-CRF model with word embedding, character-based representations, and Bi-LSTM position based representation. • IV: Bi-LSTM-CRF model with word embedding, character-based representations, Bi-LSTM position-based representation and syntactic word representations.

Figure 5 . 2 and

 52 Figure 5.2 and Figure 5.3 respectively show the evolution of the F1 score

Figure 5 . 3 :

 53 Figure 5.3: Evolution of the function. (loss)

  Figure 5.3 shows that from epoch 30, the loss function continues to decrease without improving the F1 score. From this epoch, the model starts to overfit the training data.

Contents 7 . 1

 71 Summary of the thesis . . . . . . . . . . . . . . . 7.2 Technological transfer . . . . . . . . . . . . . . . . 7.3 Limitations and perspectives of the proposed approach . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Construction of the sourcing vocabulary . . . . . . 7.3.2 Ontology alignment . . . . . . . . . . . . . . . . . 7.3.3 Named entity recognition and linking . . . . . . . 7.3.4 Recommender system . . . . . . . . . . . . . . . .

  Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Our research focus . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Domain knowledge modeling for sourcing data representation . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Design and adaptation of algorithms for knowledge extraction . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.3 Recommend providers based on the formal representation of descriptions . . . . . . . . . . . . . . . . . . . . 1.3 Our methodology . . . . . . . . . . . . . . . . . . . . . . . . . Proposal of a recommender algorithm . . . . . . . . . 1.4 Our contributions . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . Semantic Web languages . . . . . . . . . . . . . . . . . 2.2.3 Ontology alignment . . . . . . . . . . . . . . . . . . . 2.3 Information and knwledge extraction . . . . . . . . . . . . . . 2.3.1 Natural language processing . . . . . . . . . . . . . . . 2.3.2 Sequence labeling . . . . . . . . . . . . . . . . . . . . . 2.3.3 Named entity recognition . . . . . . . . . . . . . . . . 2.3.4 Named entity linking . . . . . . . . . . . . . . . . . . . 2.4 Information filtering . . . . . . . . . . . . . . . . . . . . . . . 2.5 Vector representations of textual data . . . . . . . . . . . . . 2.5.1 Bag of words . . . . . . . . . . . . . . . . . . . . . . . Summary of the thesis . . . . . . . . . . . . . . . . . . . . . . 7.2 Technological transfer . . . . . . . . . . . . . . . . . . . . . . 7.3 Limitations and perspectives of the proposed approach . . . . 7.3.1 Construction of the sourcing vocabulary . . . . . . . . 7.3.2 Ontology alignment . . . . . . . . . . . . . . . . . . . 7.3.3 Named entity recognition and linking . . . . . . . . . . 7.3.4 Recommender system . . . . . . . . . . . . . . . . . . Overview of our recommendation approach. . . . . . . . . . . .10 Alignment example between ESCO, ROME, and Cigref. . . . 3.11 Example of ESCO-NAF alignment based on ROME. . . . . .

	CONTENTS
	7 Conclusion
	7.1 Bibliography List of Figures
	1 Introduction 1.1
	1.1 11

1.3.1 Building a vocabulary for the sourcing domain . . . . 1.3.2 Proposal of an ontology alignment approach . . . . . . 1.3.3 Proposal of an approach for sourcing named entity recognition and linking . . . . . . . . . . . . . . . . . . 1.3.4 2 Background knowledge 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Semantic Web . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Knowledge organization systems . . . . . . . . . . . . 2.2.2 2.5.2 Bag of concepts . . . . . . . . . . . . . . . . . . . . . . 2.1 A simple NN architecture. Source: [Mohammed and Omar, 2012] 2.2 A Bi-LSTM-CRF-architecture. Source: [Huang et al., 2015] . 3.1 ESCO example. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 ROME example. . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 CIGREF example. . . . . . . . . . . . . . . . . . . . . . . . . 3.4 NAF example. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 UNSPSC example. . . . . . . . . . . . . . . . . . . . . . . . . 3.6 KOMPASS example. . . . . . . . . . . . . . . . . . . . . . . . 3.7 CPF example. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Silex business activities vocabulary. . . . . . . . . . . . . . . . 3.9 Overall knowledge engineering process to built a modular vocabulary for the sourcing domain. . . . . . . . . . . . . . . 34.1 Matching techniques classification. Source

  Section 2.2 presents the different notions and languages related to the semantic Web domain. Sections 2.3 and 2.4 introduce respectively the knowledge extraction and the recommender systems domains. Section 2.5 reports a brief overview of the state of the art of vector representations relevant to textual data. Section 2.6 gives a brief introduction on the machine and deep learning algorithms used in the named entity recognition task. Section 2.7 defines the different evaluation metrics used throughout our work. Finally, Section 2.8 concludes this chapter.

  Alignment between the skill/occupation vocabulary and the business activity vocabulary . . . . . 3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . .

	Contents
	3.1 Introduction . . . . . . . . . . . . . . . . . . . . .
	3.2 Related works . . . . . . . . . . . . . . . . . . . .
	3.3 Our ontology design approach . . . . . . . . . . .
	3.4 Identification and reuse of existing metadata . .
	3.4.1 Skills and occupations vocabulary . . . . . . . . .
	3.4.2 Business activity vocabulary . . . . . . . . . . . .
	3.4.3 Products vocabulary . . . . . . . . . . . . . . . . .
	3.5 Construction of internal vocabularies at Silex .
	3.5.1 Construction of a Silex internal skills/ occupations
	vocabulary . . . . . . . . . . . . . . . . . . . . . .
	3.5.2 Construction of the internal business activities vo-
	cabulary for Silex . . . . . . . . . . . . . . . . . .
	3.6 Ontology alignment . . . . . . . . . . . . . . . . . .
	3.6.1 Alignment of the skills and occupations vocabulary
	3.6.2 Alignment of the business activity vocabulary . . .
	3.6.3

  3.4.1.1 European skills, competences, qualifications and occupations European Skills, Competences, Qualifications and Occupations (ESCO) 4 is a multilingual classification of occupations and skills, containing 17091 concepts and available in SKOS-RDF format. The main goal of ESCO is to bridge the gap between the world of education and training and the European Union (EU) labor market by identifying and categorizing skills, qualifications and occupations in the EU. ESCO provides descriptions of 2942 occupations and 13485 skills linked to these occupations, translated into 27 languages (all official EU languages plus Icelandic, Norwegian and Arabic). The hierarchical structure of ESCO is composed of five levels. We present in figure 3.1 an example of ESCO hierarchy knowledge. Operational directory of trades and jobs Operational Directory of Trades and Jobs or Répertoire Opérationnel des Métiers et des Emplois (ROME) 5 is a directory created in 1989 by the Na-

	Professionals
	Information and communications technology professionals
	Software and applications developers and analysts
	Systems analysts
	ICT system architect
	Figure 3.1: ESCO example.
	3.4.1.2

tional Employment Agency (ANPE), today Pôle emploi, in France. The last version of ROME contains more than 10,000 different names of job titles grouped by families; for example, administration of information systems (administration de systèmes d'information), database administrator (administrateur / administratrice de bases de données) or production and operation of information systems (production et exploitation de systèmes d'information).

  This classification, whose original version was created in 1947, makes it possible to systematically classify companies in the 66 countries of the Kompass network, according to the products and services they provide.

	Computing, Internet, R&D
	57 Computing and Internet
	57910 Computing security
	5791002 Network Security Services
	57750 Computing maintenance
	84 Architects, technical offices and engineering
	consulting companies
	84780 Consulting in robotic engineering
	85 Research and testing
	Its
	58,000 entries make up a unique directory, continuously updated, translated
	into 26 languages, offering very important development perspectives. In 2014,
	the WF13 new version of Kompass International was built: this classification
	proposes a new hierarchical structure that takes into account the latest
	developments in the various sectors of the world economy. WF13 offers a
	ranking of 55,000 products and services presented in a tree of 4 levels:

Table 3 .

 3 1: Metadata repositories.

	Repository Knowledge Coverage	Format	Languages	Number
			domains
	ESCO	Skill&	All	RDF	Multilingual 17091
		Occupation domains
	ROME	Occupation All	SKOS-RDF	French	12255
			domains	transformation
	Cigref	Occupation Computing SKOS-RDF	French	45
			domain	transformation
	NAF	Business	All	RDF	French	1735
		sector	domains		English
	UNSPSC	Business	All	SKOS-RDF	French	87470
		sector	domains	transformation English
	kompass	Business	Computing SKOS-RDF	French	1370
		activity	domain	transformation
	CPF	Product	All	SKOS-RDF	French	5522
			domains	transformation

Table 4 .

 4 1: Number of entities by type for each ontology.

	Ontology Classes	Object	Data
			properties properties
	cmt	30	49	10
	conference	60	46	18
	confOf	39	13	23
	edas	104	30	20
	ekaw	74	33	0
	constructed within the framework of the OAEI and it contains 57 correspon-
	dences and five ontologies (cmt, conference, confOf, edas, ekaw ) available in
	OWL format. Table

Table 4 . 2 :

 42 Number of concepts for the Silex ontology for the computing domain. State operator that reports to the Ministry of National Education and Youth and the Ministry of Higher Education, Research and Innovation. As a public publisher, ONISEP produces and distributes all information on training and trades. It also offers services to students, parents and educational teams. In this context, ONISEP provided us with an occupation directory Table4.3: Number of relation types between concepts for the Silex ontology for the computing sector. , and the goal was to align it with ROME. The ONISEP vocabulary contains 5325 concepts and the ROME vocabulary contains 12255 concepts. We started by transforming the ONISEP vocabulary into a SKOS vocabulary then we applied our approach to align ONISEP and ROME. A gold standard, composed of 290 links and produced by an expert, is used for the evaluation of our automatic alignment approach. It contains 259 close relations and 31 hierarchical relations.

	Skills and Occupations	Business activity
	Ontology	Number	Ontology Number
	ESCO	160	NAF	53
	ROME	117	UNSPSC	153
	Cigref	42	Silex	14
	4.4.1.3 ONISEP use case
	ONISEP (Office national d'information sur les enseignements et les profes-
	sions) is a

Table 4 .

 4 4: Evaluation of our approach on the OAEI benchmark.

	Systems		Precision Recall f-measure
	Our system with standard 0.32	0.31	0.27
	evaluation methods
	Our system with custom	0.70	0.43	0.51
	evaluation methods
	Ritze et al. 2009	0.30	0.13	0.19
	Ritze et al. 2010	0.83	0.09	0.18
	Jiang et al. 2016	0.09	0.11	0.10
	Table 4.5: Evaluation of our approach on real world data from the Silex and
	ONISEP use cases.
	Dataset	Evaluation Threshold Precision Recall F1
	ESCO-ROME	Standard	0.85	0.49	0.74	0.58
		Custom	0.7	0.99	0.94	0.96
	ESCO-Cigref	Standard	0.8	0.51	0.72	0.59
		Custom	0.8	0.92	0.72	0.80
	NAF-UNSPSC	Standard	0.8	0.40	0.71	0.50
		Custom	0.7	1	0.95	0.97
	ONISEP-ROME Standard	0.87	0.42	0.73	0.52
		Custom	0.7	1	0.88	0.93
	1. the system presented in

Table 5 .

 5 1: Example of input training data.

	Word	POS	Label
	Je	PRON O
	recherche	VERB O
	un	DET	O
	plombier	NOUN S-
	La	DET	O
	société	NOUN O
	BNB	NOUN O
	souhaite	VERB O
	créer	VERB O
	des	DET	O
	supports	NOUN B-
	de	ADP	I-
	communication NOUN E-
	.	.	.
	.	.	.

Table 5 .

 5 2 presents the results obtained in terms of precision, recall and F 1 score.

		Recall	Precision	F1
	Model Dev	Test	Dev	Test	Dev	Test
	I	58.88 61.31	77.02 76.61	66.74	68.11
	II	64.03 66.61	75.21 76.66	69.17	71.29
	III	63.06 66.61	75.62 80.11 68.77	72.74
	IV	67.57 70.20 76.03 73.77	71.55 71.94
		Table 5.2: Precision, Recall and F1-score.

Table 6 .

 6 1: Experimental settings ON RS.

	Experiment Vector representation Similarity measure
	Emb

Table 6 . 2 :

 62 Evaluation of the proposed experimental settings with dataset A.

	Experiment Threshold Precision Recall	F1
	Emb	0.76	0.848	0.397	0.530
	BoC	0.239	0.887	0.137	0.235
	BoC	0.16	0.852	0.247	0.378
	BoC	0.15	0.728	0.187	0.296
	BoC	0.069	0.697	0.287	0.402
	Conc	0.58	0.875	0.612 0.717
	Conc	0.55	0.793	0.457	0.575
	Conc	0.58	0.791	0.231	0.365
	Conc	0.52	0.797	0.258	0.389
	BoC→Emb	0.70	0.600	0.580	0.579
	Conc→Emb	0.24;0.76	0.83	0.429	0.558

Table 6 .

 6 3: Evaluation of the proposed experimental settings with dataset A .

	Experiment Threshold Precision Recall	F1
	Emb	0.73	0.877	0.562	0.678
	BoC	0.01	0.909	0.338	0.487
	BoC	0.16	0.890	0.456	0.596
	BoC	0.13	0.819	0.414	0.541
	BoC	0.10	0.841	0.497	0.618
	Conc	0.58	0.875	0.612	0.717
	Conc	0.55	0.912	0.540	0.674
	Conc	0.55	0.883	0.420	0.568
	Conc	0.52	0.920	0.430	0.584
	BoC→Emb	0.7	0.72	0.694 0.768
	Conc→Emb	0.05;0.73	0.860	0.627	0.719

Table 6 . 4 :

 64 Evaluation of the proposed experimental settings with dataset A, using DBpedia spotlight for NER.

	Experiment Threshold Precision Recall	F1
	Emb	0.79	0.346	0.169	0.219
	BoC	0.11	0.418	0.313 0.357
	Conc	0.39	0.401	0.193	0.260

Table 7 .

 7 1: Summary of our work perspectives.

			Roadmap
	Steps	Terms	Perspectives
	Construction of the	Short	-Test other language detectors.
	sourcing vocabulary	Medium -Automatically label hierarchical
			clusters.
		Long	-Add other multilingual metadata.
			-Evolution of sourcing knowledge
			in the context of each country.
	Ontology alignment	Short	-Use a contextual embedding.
		Medium -Evaluation on the entire
			Silex dataset or other datasets.
		Long	-Deal with leaf nodes.
	NER	Short	-Evaluate our NEL algorithm.
	and linking		-Use a contextual embedding.
		Medium -Deal with unlinkable entities.
		Long	-Train our embedding model.
	Recommender system Short	-Integrate our work within
			the Silex platform.
		Medium -Exploit additional metadata.
		Long	-Data visualization.

https://www.silex-france.com/silex/

https://team.inria.fr/wimmics/

NLP algorithms have to be able to process texts in different languages.

https://www.w3.org/RDF/

https://www.w3.org/TR/rdf-syntax-grammar/

https://www.w3.org/TR/turtle/

https://www.w3.org/2018/jsonld-cg-reports/json-ld/

https://www.w3.org/TR/2000/CR-rdf-schema-20000327/

https://www.w3.org/OWL/

https://www.w3.org/2004/02/skos/

https://www.w3.org/TR/rdf-sparql-query/

https://www.wikidata.org/wiki/Wikidata:Main_Page

https://wiki.dbpedia.org/

https://yago-knowledge.org/

https://www.amazon.fr/

https://www.netflix.com/fr/

https://wordnet.princeton.edu/

https://fr.wikipedia.org/wiki/Profession

https://www.investopedia.com/terms/b/business-activities.asp

https://ec.europa.eu/esco/portal/home

http://www.pole-emploi.org/accueil/mot-cle.html?tagId=94b2eaf6-d7bd-4244-bddc-01415605563b

https://www.cigref.fr/publication-mise-a-jour-2018-de-la-nomenclature-des-metierssi-du-cigref

7 https://www.insee.fr/fr/information/2406147

https://www.unspsc.org/

http://www.kompass-international.com/Corporate/home/kompass-knowhow/processing-the-data/classification.html

https://www.insee.fr/fr/information/2493496

https://terms.tdwg.org/wiki/dcterms:references

http://oaei.ontologymatching.org/

https://www.Silex-france.com/Silex/

http://www.onisep.fr/

https://fasttext.cc/docs/en/pretrained-vectors.html

https://www.dbpedia-spotlight.org/

https://www.onetonline.org/

https://www.china-data-online.com/info/hyfl.asp

https://textblob.readthedocs.io/en/dev/

https://langrid.org/

Remerciements

cosine similarity measure. The second best results were achieved with the BoC→Emb method which combines a BoC representation and word embedding. Unsurprisingly, considering the BoC representation increases the precision and F1 measures, even though at the expense of the recall score. With dataset A , the best results based on the precision measure were achieved with the Conc"' method, where the recommendation is based on a vector representation concatenating word embedding and BoC with subsumption and skos:closeMatch relations. The best results in terms of F1 score were obtained using the BoC→Emb method which combines a BoC representation and word embedding. Using this cleaned dataset, the performance of all methods increases. This emphasizes the fact that all methods are very sensitive to the quality of entity linking.

All other methods, where a word embedding representation is enriched with domain knowledge, obtain a better precision measure than the baseline method using word embedding alone. Focusing on the precision@N results, with both datasets, we can conclude that injecting domain knowledge is highly beneficial to our RS. Although a BoC representation performs very well from the 2 top-ranking items on dataset A, we can observe that its performance keeps steady on both datasets up to the tenth item.

All in all, it appears that injecting domain knowledge into the vector representations is all the more beneficial, the greater the quality of the annotations. Also, enriching the conceptual representations by considering the subsumption relation and skos:closeMatch relation clearly gives better results. Finally, the comparison with the experiments using DBpedia spotlight for NER confirms that the introduction of domain knowledge in the recommendation process is beneficial and helps enhance the performance of the system