Lynda Said Lhadj

Mohand Boughanem

Karima Amrouche
email: k_amrouche@esi.dz

Enhancing Information Retrieval Through Concept-Based Language Modeling and Semantic Smoothing

Traditionally, many information retrieval models assume that terms occur in documents independently. Although these models have already shown good performance, the word independency assumption seems to be unrealistic from a natural language point of view, which considers that terms are related to each other. Therefore, such an assumption leads to two well-known problems in information retrieval (IR), namely, polysemy, or term mismatch, and synonymy. In language models, these issues have been addressed by considering dependencies such as bigrams, phrasal-concepts, or word relationships, but such models are estimated using simple n-grams or concept counting. In this paper, we address polysemy and synonymy mismatch with a conceptbased language modeling approach that combines ontological concepts from external resources with frequently found collocations from the document collection. In addition, the concept-based model is enriched with subconcepts and semantic relationships through a semantic smoothing technique so as to perform semantic matching. Experiments carried out on TREC collections show that our model achieves significant improvements over a single word-based model and the Markov Random Field model (using a Markov classifier).

Introduction

In most traditional information retrieval (IR) approaches, queries and documents are represented by single words or word stems (typically referred to as a bag of words representation). The relevance score of a document with respect to a query is estimated using the frequency distribution of query terms over the documents. The latter is carried out under the assumption that query words occur independently in documents [START_REF] Salton | An evaluation of term dependence models in information retrieval[END_REF]. However, given the common knowledge about natural language, such an assumption might seem unrealistic (or even wrong), leading to the long-standing IR problems of synonymy and polysemy:

• The synonymy issue (also known as term mismatch issue) occurs when users and authors use different terms for the same concept [START_REF] Gonzalo | Semantic matching in information retrieval[END_REF][START_REF] Li | Semantic matching in search[END_REF][START_REF] Wei | Managing word mismatch problems in information retrieval: A topic-based query expansion approach[END_REF]. More precisely, authors use a large vocabulary to express the same concepts while user queries are often a short or incomplete description. For example, given the query "Auto Race," a document containing related concepts such as "Automobile Race," "Grand Prix," or "Rally" would be not retrieved if both "Auto" and "Race" do not occur within this document. • The polysemy issue concerns the ambiguity of single words, viz., that a word could have several meanings [START_REF] Krovetz | Lexical ambiguity and information retrieval[END_REF]. For example, given a user query "Dog Bark" and a single word-based model, irrelevant documents dealing with "The Bark of a Dog" or "The Bark of a Tree" might be returned if the words "Bark" and "Dog" occur frequently therein.

These issues have been largely discussed in IR and have given rise to semantic IR approaches [START_REF] Alvarez | Word pairs in language modeling for information retrieval[END_REF][START_REF] Baziz | A conceptual indexing based on document content representation[END_REF][START_REF] Boughanem | A new factor for computing therelevance of a document to a query[END_REF]Cao, Nie, & Bai, 2005;[START_REF] Li | Semantic matching in search[END_REF]. Generally, these approaches rely on additional sources of evidence such as semantic resources (e.g., dictionaries or ontologies) or corpus features (e.g., term co-occurrence) to represent queries and documents with specific meaning of terms rather than single words. However, research developed so far has led to mixed results. We present the following questions about how to improve the mismatches that result from polysemy and synonymy between query term and corpus: (a) how to identify query and document terms denoting concepts and (b) how to use a concept-based model to enhance retrieval. Aiming to go beyond bag of word issues, we focus in this paper on language-based models (LMs) [START_REF] Ponte | A language modeling approach to information retrieval[END_REF]. Indeed, during the two last decades LMs have attracted increased interest in the IR community mainly due to their reliance on probabilities. In addition, LMs have shown successful results over the other models such as probabilistic and vector space models [START_REF] Bennett | A comparative study of probabilistic and language models for information retrieval[END_REF][START_REF] Zhai | Statistical language models for information retrieval a critical review[END_REF]. More particularly, in the context of semantic IR, a wide range of LM approaches have been proposed to relax the word independency assumption. Most approaches have focused on considering term dependencies such as n-grams, phrases, co-occurrence, or ontological concepts [START_REF] Cao | Integrating word relationships into language models[END_REF][START_REF] Gao | Dependence language model for information retrieval[END_REF][START_REF] Hammache | Combining compound and single terms under language model framework[END_REF]Srikanth & Srihari, 2002;[START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] in an attempt to capture user query and document semantics. Two main types of approaches can be distinguished, namely, corpus-based approaches and external resource-based approaches. The former attempts to capture term dependencies within a corpus using statistical measures or learning techniques [START_REF] Berger | Information retrieval as statistical translation[END_REF][START_REF] Lavrenko | Relevance based language models[END_REF] while the second category relies on external semantic resources (ontologies, encyclopedia) to recognize query and document concepts [START_REF] Meij | Parsimonious concept modeling[END_REF][START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] or single-word relationships [START_REF] Cao | Integrating word relationships into language models[END_REF].

In this paper, we propose a novel concept-based language model to address the problem of word dependence. The proposed model is inspired by previous approaches [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF][START_REF] Bendersky | Modeling higher-order term dependencies in information retrieval using query hypergraphs[END_REF][START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF][START_REF] Zakos | A novel concept and context-based approach for Web information retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] which have proven the usefulness of concept-based representation in IR. More specifically, we assume that documents and queries are represented as a bag of concepts, instead of a bag of words, to meet the polysemy problem. Document and query concepts are identified using two sources of evidence: a semantic resource (an ontology) and the document collection. We view both sources as complementary since important concepts, such as neologisms or proper names, generally, are not found in semantic resources. We consider that such concepts might be frequent collocations in the document collection. Thus, a concept can be viewed as a single word, a frequent collocation, or an ontology entry. The idea of combining multiword phrases and ontological concepts has also been proposed [START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF]. However, the concept model they propose is based on counting only ontological concepts or frequently found multiword phrases. From our point of view, the estimation of concept-based models in IR can go beyond concept counting. Indeed, semantic information a priori defined in semantic resources such as subconcepts (component concepts corresponding to ontology entries) and semantic relatedness (e.g., hypernymy, hyponymy, etc.) could be integrated into the model through a semantic smoothing technique [START_REF] Berger | Information retrieval as statistical translation[END_REF]Lafferty & Zhai, 2001;[START_REF] Zhai | Statistical language models for information retrieval a critical review[END_REF] to perform semantic matching. The intuition is that the authors tend to use concept relationships or subconcepts to avoid repetition or to refer to a concept they have previously used. In the same spirit, [START_REF] Cao | Integrating word relationships into language models[END_REF] proposed an approach to incorporate individual word relationships into a language model framework without considering word relationships. In this work, we combine single words, frequent collocations, ontological concepts, subconcepts, and semantic relatedness into a unified language model so as to perform query and document matching at a semantic (concept) level. Accordingly, for a given query the retrieval model should be able to retrieve documents that contain both the same terms as those of the query and those that contain related concepts such as synonyms, hypernyms, and hyponyms. We evaluated various scenarios of our model on two TREC collections to demonstrate the effect of these elements on retrieval. Our results show significant improvements over stateof-the-art models: the unigram language model (Dirichlet) and the Markov Random Field model [START_REF] Metzler | A Markov random field model for term dependencies[END_REF].

The remainder of the paper is organized as follows. In the next section we describe the general language model principle and review previous works dealing with word independence. Concept-Based Retrieval Model follows and then Experimental Evaluation and Results performed on two TREC collections are presented. The final section summarizes the contributions and suggests some future research directions.

Related Work

Language models have steadily grown in popularity since their introduction in IR [START_REF] Ponte | A language modeling approach to information retrieval[END_REF]. They have been successfully applied in various applications such as ad hoc retrieval [START_REF] Ponte | A language modeling approach to information retrieval[END_REF], expert finding [START_REF] Macdonald | Voting techniques for expert search[END_REF], and social retrieval [START_REF] Zhou | Exploring social annotations for information retrieval[END_REF]. Specifically, in the context of ad hoc retrieval, LMs have significant performance compared to some traditional IR models, such as probabilistic and vector space models [START_REF] Bennett | A comparative study of probabilistic and language models for information retrieval[END_REF][START_REF] Zhai | Statistical language models for information retrieval a critical review[END_REF]. The main idea behind LM in IR is to view a document as a language sample generated according to some probability distribution of word sequences [START_REF] Zhai | Statistical language models for information retrieval a critical review[END_REF]. The relevance of a document D with respect to a query Q is seen as a probability of producing the query from the document language model. Several variations have been proposed to estimate this probability [START_REF] Berger | Information retrieval as statistical translation[END_REF][START_REF] Ponte | A language modeling approach to information retrieval[END_REF][START_REF] Song | A general language model for information retrieval[END_REF]. The most common one is the query likelihood approach [START_REF] Ponte | A language modeling approach to information retrieval[END_REF] where the relevance score is given by the conditional probability P(Q|D) of Q to be generated by the underlying language model D expressed as follows:

Score Q D P Q D ( , ) ( | ) = (1)
To more easily estimate the above probability, query terms are assumed to occur independently. Thus, D is commonly estimated using the unigram model, so Equation 1 becomes the product of individual query term probabilities given the document model:

Score Q D P t D i t Q i ( , ) ( | ) = ∈ ∏ (2)
Probability P(ti|D) is calculated using the maximum likelihood PML(ti|D) of individual terms estimated as follows:

P t D f t D D ML i i ( | ) ( ) =
where f(ti, D) is the frequency of term ti within D, and |D| is the total number of terms in that document.

According to this model, when a query term ti does not occur in D, PML(ti|D) is zero, making the overall Score(Q, D) zero, even if the remainder of query terms {tj}j≠i are seen in D. To cope with this problem, a number of smoothing techniques have been proposed [START_REF] Chen | An empirical study of smoothing techniques for language modeling[END_REF]Zhai & Lafferty, 2001). The general principle is to assign a nonzero probability to unseen query terms in documents and adjust low probabilities.

Using bag of word representations that assume word independence is prevalent within language model approaches. It is commonly accepted in IR that such an assumption is a matter of mathematical convenience because terms in natural language are often dependent. Therefore, much research goes beyond the bag of word representation. Earlier works captured dependencies between words by using units longer than single words such as bigrams and concepts [START_REF] Song | A general language model for information retrieval[END_REF]Srikanth & Srihari, 2002;[START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF]. The intuition is the following: the longer an indexing unit is than a single word, the less is its ambiguity [START_REF] Shi | Integrating phrase inseparability in phrasebased model[END_REF]. More advanced works have attempted to deal with the synonymy and polysemy issues using smoothing techniques, for instance, the translation model [START_REF] Berger | Information retrieval as statistical translation[END_REF][START_REF] Cao | Integrating word relationships into language models[END_REF]. These works rely on a specific resource, which may be either the document collection viewed as an internal resource in the IR process or a semantic resource viewed as an external resource. Accordingly, we classify these works into two categories of approaches depending on the source of word dependencies: (a) corpus-based approaches integrating dependencies extracted from a corpus such as bigrams or concepts defined on the basis of co-occurrence information and (b) external resource-based approaches enabling the capture of semantic dependencies from external resources (such as dictionaries, thesauri, or ontologies).

Corpus-Based Approaches

Most of these approaches have examined different ways to exploit text features, such as bigrams or longer collocations to relax the word independency assumption [START_REF] Bai | Query expansion using term relationships in language models for information retrieval[END_REF][START_REF] Berger | Information retrieval as statistical translation[END_REF][START_REF] Gao | Dependence language model for information retrieval[END_REF][START_REF] Petrovic | Comparison of collocation extraction measures for document indexing[END_REF]. Generally, these dependencies are learned from the document collection. [START_REF] Berger | Information retrieval as statistical translation[END_REF] proposed one of the earliest works in the LM framework that exploits dependencies between query and document words. Word dependencies are explicitly expressed through a translation probability estimating the degree of link between query and document words using word co-occurrence in a training corpus. [START_REF] Song | A general language model for information retrieval[END_REF] integrated ordered and adjacent dependencies between pairwise words in the so-called bigram model. Empirical results were not successful for two reasons: (a) bigrams cannot cover all useful word dependencies, given that there are more distant dependencies than bigrams. Indeed, terms occurring in a specific context can be related even though they are not adjacent. In addition, the bigram-based model assigns higher probabilities to documents containing query bigrams (e.g., "Information Retrieval") than those where component terms "Information" and "Retrieval" occur separately; and (b) bigrams introduce noise in the retrieval process. For example, the query "Computer-aided Crime" (TREC topic 94) contains the following bigrams: "Computer-aided" and "aided Crime." For this query, documents containing "Computeraided Design" with higher frequencies are likely to be topranked by the bigram model as relevant. To overcome these limitations, Srikanth and Srihari (2002) extended the bigram model to a biterm one. They define biterms as bigrams in which the constraint of word ordering is relaxed. Given a document containing the phrase "Information Retrieval" and another containing "Retrieval of Information," the biterm language model would assign the same probability of generating the query "Information Retrieval" for both documents, unlike the bigram model. The results of this approach were slightly better than the unigram and bigram models. In Srikanth and Srihari (2003), the authors extended their model and integrated higher-level dependencies in queries seen as a sequence of concepts that are themselves viewed as sequences of single words. These concepts are identified using a syntactic parser, and their probabilities are estimated using a smoothed bigram model. Experiments have shown that the concept-based unigram model provides better results than both bigram and biterm models [START_REF] Song | A general language model for information retrieval[END_REF]Srikanth & Srihari, 2002). With a similar intuition, [START_REF] Metzler | A Markov random field model for term dependencies[END_REF] proposed a dependency-based framework using Markov Random Field (MRF). They exploited text features to detect longer dependencies between query terms. Thus, they examined three types of dependencies: the Full Independence (FI), where query terms are assumed to be independent of each other, the Sequential Dependence (SD), where query terms are neighboring, and the Full Dependence (FD), where query terms are dependent on each other. The results have shown that modeling such dependencies significantly improves retrieval effectiveness. In particular, the authors noticed that the SD model is more effective on smaller collections with longer queries, while the FI model is best for larger collections with shorter queries. Similarly, [START_REF] Shi | Integrating phrase inseparability in phrasebased model[END_REF] proposed a phrase-based model in which phrases and single words are used to estimate a document model. Only phrases having a higher inseparability factor than their component words have been used [START_REF] Shi | Integrating phrase inseparability in phrasebased model[END_REF]. The inseparability of a phrase is defined as a discriminant factor based mainly on IDF (Inverse Document Frequency). The experimental results stressed the usefulness of integrating phrase's inseparability in the language model. Along these lines, [START_REF] Hammache | Combining compound and single terms under language model framework[END_REF] combined single words and filtered bigrams into a language model. They proposed an approach for filtering bigrams and weighting them by considering both their frequencies and their component terms in the document collection. [START_REF] Gao | Dependence language model for information retrieval[END_REF] considered only dependencies among pairs of terms. Thus, a query is modeled by an undirected acyclic graph that expresses the most distant and robust dependencies among query words.

The results of the above approaches show that exploiting dependencies extracted from the corpus may have a positive influence on retrieval effectiveness. However, the bigram language model has not shown consistent performance beyond the unigram model because many dependencies are incorrect and introduce noisy dependencies into the retrieval process.

Although the bigram model has been extended by relaxing order and adjacency constraints or by considering more distant dependencies [START_REF] Bai | Query expansion using term relationships in language models for information retrieval[END_REF][START_REF] Gao | Dependence language model for information retrieval[END_REF]Srikanth & Srihari, 2003), the retrieval effectiveness can be further enhanced by considering implicit dependencies such as synonymy or any semantic relationship (e.g., hypernymy). More recently, a number of LM-based approaches have attempted to use semantic information a priori defined in external resources such as WordNet [START_REF] Bao | LSM: Language sense model for information retrieval[END_REF][START_REF] Cao | Integrating word relationships into language models[END_REF], Unified Medical Language System (UMLS) [START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF], and Wikipedia [START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF]. We give an overview of this category of models in the next section.

External Resource-Based Approaches

Most of the underlying approaches are built upon the translation model [START_REF] Berger | Information retrieval as statistical translation[END_REF], which provides a straightforward way to incorporate a kind of semantic smoothing by mapping query and document terms [START_REF] Berger | Information retrieval as statistical translation[END_REF]Lafferty & Zhai, 2001;[START_REF] Zhai | Statistical language models for information retrieval a critical review[END_REF]. Ontological relationships have also been exploited to estimate dependencies between query and document terms. In particular, [START_REF] Cao | Integrating word relationships into language models[END_REF] extended the translation model [START_REF] Berger | Information retrieval as statistical translation[END_REF] by considering word relationships in order to match query and document terms at a semantic level. For this purpose, they assumed that dependencies between query and document words are generated through two sources: (a) direct connection (matching) using a unigram model and/or (b) indirect connection using a link model expressed throughout co-occurrences and WordNet relationships. These two sources are used to smooth the document model. This approach has shown consistent improvement over the unigram model on TREC collections, although word ambiguity had not been discussed. In contrast, [START_REF] Bao | LSM: Language sense model for information retrieval[END_REF] proposed a Language Sense Model (LSM) where the single-word unigram model is smoothed with WordNet synsets [START_REF] Miller | WordNet: A lexical database for english[END_REF]. More precisely, the appropriate sense of each single word in both document and query is selected a priori using the Word Sense Disambiguation System they developed. Afterward, the document model is smoothed with hyponyms and hypernyms found in WordNet. However the experiments did not show strong conclusions except for long queries (having more than 20 words). Other works [START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] have followed another direction to exploit word dependencies. Typically, [START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] proposed a semantic smoothing approach to enhance the document model and address the problem of word ambiguity using the so-called topic signatures. The latter is a set of synonyms, senses, and collocations (word pairs) corresponding to named entities. These topic signatures are recognized in documents using both MaxMatcher1 and XTRACT.2 Experiments on a domain collection (TREC Genomic3 ) have shown the effectiveness of such an approach, with significant improvement over the unigram model. With this aim, [START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF] proposed a semantic smoothing approach based on Wikipedia. More precisely, they consider the titles of Wikipedia articles instead of topic signatures to smooth the document model proposed in [START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF]. They have shown that a Wikipedia article title has the same syntactic structure (generally a nominal phrase) as an ontological concept. [START_REF] Meij | Conceptual language models for domain-specific retrieval[END_REF] proposed generative concept models to improve the query model. More precisely, the query is first translated into a conceptual representation obtained using ontology and feedback documents (issued from an initial retrieval run). Then the conceptual query model is translated into a textual query model. The intuition is that the textual representation is more detailed than the conceptual representation. Thus, retrieving with a textual query representation translated from a conceptual form yields a better performance than a strict concept-based matching.

The approach we propose in this paper is at the confluence of both described categories. We propose a conceptbased model by considering two sources of word dependencies: frequent collocations and ontology entries. A similar proposition has been described previously [START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] where frequent collocations and domain concepts are combined in a language model. In [START_REF] Bendersky | Modeling higher-order term dependencies in information retrieval using query hypergraphs[END_REF] and [START_REF] Zhang | Recognition and classification of noun phrases in queries for effective retrieval[END_REF], several types of terms (bigrams, noun phrases, and named entities) denoting concepts and modeling high-order dependencies have been considered in a retrieval framework. However, none of those works exploited concepts and their relationships in the same model. We think that both are important sources of relevance for estimating the concept-based language model. Indeed, [START_REF] Cao | Integrating word relationships into language models[END_REF] have shown the effectiveness of integrating different types of word relationships into LMs at the word level without considering word meanings. With the same spirit, we exploit the translation model but with a different formulation to incorporate concepts, their relationships, and subconcepts to enhance the concept-based language model.

Concept-Based Language Model

In this section, we describe the concept-based language model we propose. Our goal is to address term independency with two contributions:

• We consider documents and queries as a bag of concepts instead of words. We assume that a concept might be a single word or a multiple words and in both cases it might be an ontology entry or a frequent word collocation in the document but having no entry in the ontology. We assume that a frequent collocation can refer to a neologism or a proper name that has not been recorded in the ontology. A collocation as defined in Petrovic et al. (2006, p. 321) refers to ". . . a set of words occurring together more often than by chance. . .." Thus, our definition of a concept is roughly equivalent to the ones given in Bendersky and Croft (2012, p. 941) where ". . . concepts may model a variety of linguistic phenomena, including n-grams, term proximities, noun phrases, and named entities."

In our approach both types of concepts (i.e., frequent collocations and ontology entries) are combined in a unique language model framework. The example shown in Figure 1 is a TREC 4 document taken from the AP (Associated Press) data set. We see in this document that "Iran-Contra" is a frequent collocation in the document and is indeed an important concept because the document deals with 4 trec.nist.gov. FIG. 1. An example of a TREC document containing frequent collocations and ontology entries.

"Iran-Contra scandal." The terms "Iran" and "Contra" are entries in the WordNet ontology.

• We use a semantic smoothing method based on the translation model [START_REF] Berger | Information retrieval as statistical translation[END_REF] to map query concepts to document concepts through semantic relationships defined in the ontology. We consider such relationships as dependencies and additional sources of importance in estimating concept models. The intuition is to exploit concept relationships during query evaluation to retrieve documents dealing with the same or related query concepts. Thus, the concept-based language model is estimated according to occurrences of both concepts and their related ones, unlike previous approaches [START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] where concept models are only estimated by counting concepts whether they are ontology entries or not. For example, for the TREC query "Iran-Contra Affair," the retrieval model should take into account of the presence of the concept "Nicaraguan" occurring twice in the document illustrated in Figure 1. It is a related concept to query concept "Contra" (in WordNet, "Nicaraguan" is a hypernym of "Contra"). Indeed, the document in Figure 1 deals with the "Nicaraguan scandal," a related concept to "Iran-Contra Affair." Concept relationships are measured according to their semantic similarity [START_REF] Resnik | Using information content to evaluate semantic similarity in a taxonomy[END_REF]. In this context, most concept-based IR approaches merge generic and specific concept relationships even though they have a different effect on retrieval performance [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF][START_REF] Cao | Integrating word relationships into language models[END_REF][START_REF] Zakos | A novel concept and context-based approach for Web information retrieval[END_REF]. We will show the positive effect of considering concepts and semantic relationships into a unified language model using smoothing techniques.

Overview

Given query Q, document D and ontology O. In our approach, we assume that D and Q are represented by concepts, Q = {c1, c2, . . . cm} and D = {c1, c2, . . . cn} respectively, where ci is a concept that can be either an ontology entry or a frequent collocation in document collection, and in both cases it can be a single word or multiple words. Therefore, the relevance score RSV(Q, D) of D with respect to query Q is given by the probability of query concepts to be generated by the document model described below.

RSV Q D P Q D P c D i i n ( , ) ( | )) ( | ) = = ∏ (3)
The estimation of P(Q|D) attempts to abstract the unigram model described in Equation 1 (Related Work, above) at the concept level to which we refer as a concept-based document model, where P(ci|D) is the probability of concept ci in D estimated as follows:

P c D P c O D if c O P c O D otherwise i i i i ( | ) ( , | ) ( , | ) = ∉    (4)
For better clarity, we can rewrite probability P(ci|D) as:

P c D P c O D P c O D i i i ( | ) ( , | ) ( , | ) = + (5)
where | ) corresponds to the probability of ci in D given the information that ci is a frequent collocation having no entry in the ontology O.

• P c O D i ( ,
• P(ci, O|D) is the probability that ci has an ontology entry in the document model.

Assuming that concept ci is an ontological entry, its probability P(ci, O|D) is estimated using the translation model [START_REF] Berger | Information retrieval as statistical translation[END_REF], which is the most appropriate one to take into account semantic dependencies.

P c O D P c O c P c D i i j sem j c D j ( , | ) ( , | ) ( | )) = ∈ ∑ (6)
Equation 6 can also be seen as semantic smoothing and highlights the intuition that it allows incorporating semantic relationships between query and document concepts. Thus, when a query concept ci is effectively seen in the document, its probability is adjusted, or more precisely, enhanced with probabilities of its related concepts within the document. Moreover, the estimation of P(ci, O|D) is carried out proportionally to the relationship degree estimated through the probability P(ci, O|cj). Thus, the probability of query concept ci is estimated by highlighting concept centrality. This concept centrality is a factor which ". . . measures how much a query concept is related to a document concept" (Boughanem et al., 2010, p. 2).

Substituting probability P(ci, O|D) (Equation 6) in 5, the latter becomes:

P Q D P c O D P c O c P c D i i j sem j c D c Q j i ( | ) ( , | ) ( , | ) ( | ) = +       ∈ ∈ ∑ ∏ (7)
The above equation shows the general principle of our approach, which combines frequent collocations and ontology entries into a unified language model framework. In our contribution, we exploit relationships such as synonymy, hypernymy, and hyponymy because they have been shown to be useful for IR [START_REF] Cao | Integrating word relationships into language models[END_REF]. We notice that the synonymy is taken into account when representing documents and queries by ontological concepts viewed as sets of synonyms. Besides, [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF] and [START_REF] Cao | Integrating word relationships into language models[END_REF] showed empirically that each relationship has a specific impact on retrieval effectiveness [START_REF] Cao | Integrating word relationships into language models[END_REF]. Accordingly, probability P(ci, O|cj) is estimated by combining and weighting concept relationships differently. Thus, P(ci, O|cj) is expressed by:

P c O c P c O c P c O c i j hyper i j hypo i j ( , | ) ( , | ) ( ) ( , | ) = + - α α 1 (8)
where α and (1α) are the mixture weights of each relationship, Phyper(ci, O|cj) is the hypernymy model, and Phypo(ci, O|cj) is the hyponymy one. According to Equation 8, we end up with the following ranking function:

P Q D P c O D P c O c P i hyper i j c D c Q hypo j i ( | ) ( , | ) [ ( , | ) ( ) ( = +   + - ∈ ∈ ∑ ∏ α α 1 c c O c P c D i j sem j , | )] ( | )   (9)
In what follows, we define the three conditional probabilities P c O D i ( , | ) , Psem(ci|D), and P(ci, O|cj).

Probability Estimation | ) of query concept ci in document D given the information that ci is not an ontology entry is smoothed using Dirichlet smoothing, which has shown good results in previous studies (Zhai & Lafferty, 2001). Indeed, ci may not be effectively seen in the document. Thus, its probability P c O D i ( , | ) is given by:

Probability P c O D i ( , | ). Probability P c O D i ( ,
P c O D count c D P c C count c D i i ML i k c D k ( , | ) ( , ) ( | ) ( , ) = + + ∈ ∑ µ µ ( 10
)
where Count(ci, D) is ci frequency in the document D, μ is the Dirichlet smoothing parameter, ck is a document concept, and PML(ci|C) corresponds to the background collection language model estimated by the maximum likelihood estimator.

P c C count c C count c C ML i i k c k ( | ) ( , ) ( , ) = ∑ (11)
Semantic probability Psem(cj|D). This probability can be estimated in different ways. For instance, using the maximum likelihood estimator based on a simple concept counting such as in [START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF]. In our work, probability Psem(cj|D) is estimated by smoothing the maximum likelihood (ML) model of concept cj with the ML of its component concepts called subconcepts and also corresponding to ontology entries. [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF] and [START_REF] Hammache | Combining compound and single terms under language model framework[END_REF], in the same way, estimate concept weights. The intuition is that the authors tend to use subconcepts to refer to a given concept they have previously used in the document. More precisely, when a multiterm concept, for instance, "Military Coup," occurs in a document, the concept "Coup" which appears later is very likely used to refer to a "Military Coup" than to another sense, for example, "a brilliant and notable success" (see the example in Figure 2). Thus, Psem(cj|D) is given by:

P c D P c D length sc length c P sc D sem j ML j j ML sc ( | ) ( | ) ( ) ( ) ( ) ( | ) = + - θ θ 1 ∈ ∈ - ∑ sub Concepts c j ( ) (12)
where length(sc) is the number of words of subconcept sc that corresponds to an ontology entry. The ratio length sc length c j ( ) ( ) is a factor that adjusts the subconcept model according to the relative length of sc and cj [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF]. The intuition is to strengthen long subconcepts. Smoothing parameter θ is ∈ [0, 1]. The probabilities PML(cj|D) and PML(sc|D) are respectively estimated using the Dirichlet smoothing as in Equation 10.

Probability P(ci, O|cj). P(ci, O|cj) is the probability translation into the strength of the association between concepts ci and cj. Most of methods proposed in the literature for estimating word relationships are based on variants of co-occurrence in a training corpus [START_REF] Bai | Query expansion using term relationships in language models for information retrieval[END_REF][START_REF] Berger | Information retrieval as statistical translation[END_REF][START_REF] Cao | Integrating word relationships into language models[END_REF]. For instance, [START_REF] Cao | Integrating word relationships into language models[END_REF] estimated such a probability at a word level (i.e., P(wi|wj)) by counting word co-occurrences in the collection and checking whether both words (wi and wj) are linked in WordNet. [START_REF] Bai | Query expansion using term relationships in language models for information retrieval[END_REF] exploited other information such as Information Flow degree between single words within a certain context (a text passage, the whole of the document, or a window of fixed length). In this work, we estimate P(ci|cj, O) using the relationship degree between ci and cj in the ontology relative to whole relationships between ci and all document concepts. Formally, P(ci, O|cj) is estimated according to two possible cases summarized as follows:

P c O c Rel c c Rel c c if c c otherwise i j i j i k c i j k ( , | ) ( , ) ( , ) = ≠      ∑ 1 (13)
where ck, k = {1..n}, is a document concept and n is the number of concepts within the document.

Rel(ci, cj) is a function estimating the ci and cj relationship. We use here a variant of Resnik Semantic Similarity [START_REF] Resnik | Using information content to evaluate semantic similarity in a taxonomy[END_REF] based on the Information Content (IC) metric revisited by [START_REF] Zakos | A novel concept and context-based approach for Web information retrieval[END_REF]. This metric highlights the specificity of concepts [START_REF] Resnik | Using information content to evaluate semantic similarity in a taxonomy[END_REF][START_REF] Zakos | A novel concept and context-based approach for Web information retrieval[END_REF]. Indeed, it has been shown in previous work [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF][START_REF] Boughanem | A new factor for computing therelevance of a document to a query[END_REF][START_REF] Cao | Integrating word relationships into language models[END_REF][START_REF] Zakos | A novel concept and context-based approach for Web information retrieval[END_REF] that specific concepts are more likely to be useful in IR than generic ones.

Formally, Resnik Semantic Similarity is based on an "is-a" relation and the Information Content (IC) metric proposed in [START_REF] Seco | An intrinsic information content metric for semantic similarity in WordNet[END_REF] given as:

Rel c c sim c c max IC c i j res i j c S c c o i j ( , ) ( , ) ( ) *
( , ) = = ∈ ( 14
)
where S(ci, cj) is the set of concepts subsuming ci and cj. ICO has the particularity to be estimated according to the hierarchical structure of the ontology O, unlike the basic IC metric relying on word occurrences in a given corpus [START_REF] Seco | An intrinsic information content metric for semantic similarity in WordNet[END_REF]. Its principle is the following: the more descendants a concept has, the less information it expresses. Moreover, concepts that are leaves, 5 namely, specific ones, have more Information Content than the ones located higher in the hierarchy. Accordingly, the ICO metric highlights concept specificity and is defined in [START_REF] Seco | An intrinsic information content metric for semantic similarity in WordNet[END_REF] as:

IC hypo c max O O = - + 1 1 log( ( )) log( ) ( 15
)
where hypo(c) is a function that returns the number of hyponyms of concept c. maxO is a constant. It generally takes the value of the maximum number of concepts in the ontology hierarchy.

Experimental Evaluation

To evaluate the retrieval effectiveness of our model, we used standard IR collections issued from TREC 6 evaluation campaigns. Our objectives were:

Experimental Setting

We used two data sets issued from disk 1&2 of the TREC ad-hoc collection: the Associated Press 1989 (AP 89) and the Wall Street Journal 1986-1987 (WSJ 86-87) subcollections. For each data set, queries and relevance judgments are provided.

6 trec.nist.gov. FIG. 2. An example of a TREC document illustrating the intuition of considering subconcepts: "Coup" is a subconcept of the concept "Military Coup." Document indexing and query processing. For both data sets we indexed all documents with single words, frequent collocations, and ontological concepts. For this purpose we used the WordNet 7 v. 2.1 as ontology, its depth is maxwn = 117659. Thus, each document is processed using the following approach: a) Terms (single words) and multiterms are identified using a collocation extractor, the Text-NSP tool [START_REF] Banerjee | The design, implementation and use of the n-gram statistics package[END_REF]. The latter is a software tool for extracting n-grams (sequences of n-tokens in text corpora) and provides statistics to detect frequent collocations such as frequencies and Mutual Information. The multiterm size is limited to three or less. b) Detected terms are then processed in order to remove "not valid terms," mainly those beginning or ending with a stopword defined in the Terrier stopword list [START_REF] Ounis | Terrier information retrieval platform[END_REF]. We avoid pretreatment of the text before detecting multiterms in order to retain all potential concepts. Indeed, terms such as "Chief of State" or "Obstruction of Justice" could be important concepts. 8 They are generally called "complex phrases" [START_REF] Zhang | Recognition and classification of noun phrases in queries for effective retrieval[END_REF] and are frequently monosemic. c) For validating that a given multiterm is a concept, we keep only those occurring at least twice. d) We check whether a concept (a single word or a frequent collocation) occurs in WordNet. Those having an entry are selected and represented by their Synset Number. The latter is a set of synonyms having a unique identifier. For instance, the concepts "Coup," "Coup d'etat," "Takeover," and "Putsch" are grouped in the Synset, whose identifier is 01147528. Therefore, the synonymy relationship is automatically incorporated in the model. e) When a given concept has several entries (polysemy), the first sense in WordNet is selected as the default. f) The remainder of concepts (including single words) are retained as non-WordNet entries (single words and frequent collocations) and weighted with simple count of occurrences.

A set of search topics (numbers 51-100) were used as queries. Each topic is composed of three parts: Title part, which is the short query, Description part, which is a long query, and Narrative part, describing previous parts and precisely what relevant documents should contain.

In our experiments, for all data sets, we used only the Title part of topics as queries for two main reasons: (a) Title parts are as short as user queries and have the same syntactic form, usually nominal phrases such as the query "Information Retrieval System" (TREC topic 65), and (b) Terms of Title part are more important than the remainder of the topic parts [START_REF] Metzler | A Markov random field model for term dependencies[END_REF]. During querying, the extraction of topic concepts is the same as the process of document indexing. Notice that some topics have no relevant documents: topic 63 on the WSJ 86-87 data set and topics 65, 66, and 69 on the AP 89.

Evaluation Metrics

Our model is compared with baseline models using the standard Text Retrieval Conference method. It is reported in [START_REF] Buckley | Evaluating evaluation measure stability[END_REF] that the mean average precision (MAP) and the precision at the rank x noted P@x, x ∈ {10, 20} (the ratio evaluating the number of relevant documents in the top x retrieved documents) are the most used metrics to evaluate the overall effectiveness of an IR system. As MAP is estimated over all the queries, some details about the performance of our model can be hidden. For this reason we conducted a per query analysis (by comparing average precision per query). All performance measures are obtained by evaluating our retrieval runs using the trec_eval 9 standard tool. In addition, to show the consistency of our results, we perform statistical significance testing. We use the Student's t-test shown in [START_REF] Hull | Using statistical testing in the evaluation of retrieval experiments[END_REF] to be suitable for information retrieval systems.

Baseline and Evaluation Scenarios

The baseline we used to compare our model is the unigram model-based Dirichlet prior smoothing [START_REF] Smucker | An Investigation of Dirichlet Prior Smoothing's Performance Advantage[END_REF][START_REF] Zhai | A study of smoothing methods for language models applied to information retrieval[END_REF]) available on the Terrier system [START_REF] Ounis | Terrier information retrieval platform[END_REF]. The relevance score is given by:

RSV Q D P q D tf q D P q C tf w D i q Q i i w D q Q i i ( , ) ( | ) ( , ) ( | ) ( , ) = = + + ∈ ∈ ∈ ∏ ∑ ∏ µ µ (16)
where tf(qi, D) is the query term frequency in document D, μ is a smoothing parameter, and probability P(qi|C) corresponds to the background collection language model estimated by the maximum likelihood estimator as follows.

P q C count q C count w C ML i i w V ( | ) ( , ) ( , ) = ∈ ∑ (17)
where V is the vocabulary.

For comparison purposes, we performed experiments using three variants of our model considering the concept unigram model and the smoothed one with concept relationships:

• CLM_I is the individual concept model which corresponds to Equation 7 without considering concept relationships, thus the ranking function becomes:

P Q D P c O D P c O D i i c Q i ( | ) [ ( , | ) ( , | )] = + ∈ ∏ (18)
where P(ci, O|D) is simply estimated using Dirichlet smoothing without considering relationships (see Equation 10).

• CLM_R is the concept-based model integrating concept relationships corresponding to Equation 9. 7 wordnet.princeton.edu.

8

These examples are taken from TREC documents. 9

See http://trec.nist.gov/trec_eval/.

• CLM_R* is the concept-based model integrating concept relationships in which query concepts are disambiguated according to their centrality in the document. This disambiguation method has shown powerful results [START_REF] Boughanem | A new factor for computing therelevance of a document to a query[END_REF]. The centrality of a concept is equivalent to the number of concepts related to it in document D. We notice that relationships are taken from WordNet.

Centrality c R c c c D

i i k k ( ) # ( , ) = ∀ ∈ (19)
where #R(ci, ck) is the number of ci relationships ck in D.

In this experiment, the value of μ is set to 2,500 for all Dirichlet smoothing models and data sets. This value corresponds to the optimal value recommended in the literature (Zhai & Lafferty, 2001).

Results and Evaluation

In the following sections, we discuss the results obtained in the experimental evaluation of the three variants of our model. We show results in MAP and precision at the 10-document level (P@10), which is comparable to MAP and easier to score. At the end, we compare the variant achieving the best MAP to the MRF model [START_REF] Metzler | A Markov random field model for term dependencies[END_REF].

Impact of combining frequent collocations and WordNet concepts.

We aim here at evaluating the impact of combining frequent collocations and WordNet entry concepts. We performed a four-stage evaluation of CLM_I. a) CLM_IF using only frequent collocations. b) CLM_IC using only WordNet concepts (we recall that a WordNet concept is a set of synonyms) without considering their subconcepts as detailed in Equation 12 where θ value is set at 1. c) CLM_IS using WordNet concepts with considering their subconcepts. This variant returns the best performances when θ = 0.6. This value has been set by tuning θ ∈ [0, 1] with increments of 0.1. Therefore, we kept the optimal value of θ for the remainder of the experiments. d) CLM_I considering both frequent collocations and WordNet concepts with their subconcepts.

Figure 3 shows clearly that the combined model (CLM_I) outperforms all of the individual models based on frequent collocations (CLM_IF) and the one based on WordNet concepts (CLM_IC and CLM_IS ) in MAP and P@10. Indeed, for the WSJ 86-87 data set, CLM_I achieves a MAP value of 0.2376 and P@10 value of 0.3640. As for the AP 89 data set, the MAP and P@10 are 0.1908 and 0.3280. This result could be explained by the fact that CLM_IC, CLM_IF, and CLM_IS used individually do not cover all document content, whereas CLM_I captures more semantic content by considering both frequent collocations and concepts. Moreover, the concept model includes synonyms and subconcepts. Accordingly, CLM_I was used in the remaining experiments.

We compare CLM_I and ULM performance in Table 1, where the line gain (%) denotes the percentage of noticed improvements. The reported precisions show that CLM_I gives significant improvement over ULM (Unigram Model) for both data sets. More precisely, for the WSJ 86-87 data set we notice a significant improvement at P@10 than for MAP, with the respective values of +10.98% and +3.21%. For the AP 89 data set, we notice the most important improvement on MAP with a value of +6.41%. However, the P@10 and P@20, improvements are less important than the ones noticed for the WSJ 86-87.

In what follows, we perform a deep analysis per topic to highlight the factors that contribute to this improvement and to illustrate through examples of queries how relevant documents are promoted with our model (CLM_I).

Per-topic analysis. In this analysis, queries are separated depending on whether or not they contain concepts. Accordingly, we have 34 concept queries and 16 nonconcept queries for the WSJ 86-87 data set, while we have 27 concept queries and 20 nonconcept queries for the AP 89 data set. Concept queries correspond wholly to a WordNet entry (there are eight) or a frequent collocation, or contain a FIG. 3. Overview of performances (MAP and P@10) of CLM_IF, CLM_IC, CLM_IS, and CLM_I. long concept (more than one word). Examples of concept and nonconcept queries are given in Table 2.

Table 3 compares performance of our model CLM_I with respect to ULM for the two underlying categories. Columns noted (-, =, +) indicate the number of queries for which CLM_I achieved a worse, as equal as, or better average precision (AP) than the ULM. The column change (%) presents the rate of improvement in MAP over each category of query.

For both data sets, our model outperforms ULM mainly on queries containing concepts. Particularly for the WSJ 86-87 data set, we observed the best performance by yielding an improvement rate of (+38.61%) in MAP. For instance, for the query "Iran-contra Affair" (topic 99), CLM_I and ULM achieved respectively an average precision of 0.2706 and 0.069. This result could be explained by the fact that the query itself and "Iran-contra" are concepts (frequent collocations which occur more than twice in the data set). We also notice for nonconcept queries an important improvement of (+21.63%). For example, the query "Computer-aided crime" does not contain any collocation or long concepts, but the single words "Computer" and "Crime" are WordNet entries. For this query, our model achieves an AP value of 0.0526, whereas ULM achieves 0.0035. This enhancement is mainly due to the representation of query words "Computer" and "Crime" by their Synsets in WordNet (a synset is essentially a set of synonyms) because it does not contain any long concept.

We focused our analysis on the ranking of relevant documents 10 with ULM and CLM_I. The query "Military Coup d' etat" (Topic-62) is an example of a concept query for which our model achieves an AP of 0.2911, whereas ULM achieves 0.1407. This query contains the following concepts: two WordNet entries "Military" and "Coup d'etat" and two terms that are not WordNet entries "Military Coup" (a frequent collocation) and "Etat." The WSJ 86-87 data set contains 13 relevant documents according to the relevance judgment file of the TREC Adhoc Collection. These documents are ranked with ULM and CLM_I as showed in Table 4.

We notice that most documents have their ranks promoted with CLM_I. In particular, the respective ranks of 10 Taken from the relevance judgments file. does not occur in that document, but synonyms (which belong to the same synset) such as "Coups" and "takeover" occur respectively seven and six times. However, for queries corresponding to a WordNet entry and composed of a unique term, our model performs equally or nearly as well as ULM. For instance, on topic 78 "Greenpeace," our model and ULM achieve respectively an AP of 0.2951 and 0.2929.

For the AP 89 data set, the results are slightly lower than those achieved for the WSJ 86-87 data set. For nonconcept queries, we notice a degradation of performance over ULM with the value of -18.00%. However, we notice improvement on some queries. For instance, for the query "Hostagetaking" (topic 64) our model returns a better AP than ULM, contrary to what we observed for the WSJ 86-87, where ULM is better. The reason is that the query is wholly a frequent collocation in the AP data set, in contrast to WSJ 86-87.

The same point has been observed for the query "Israeli Role in Iran-Contra Affair" (topic 61). CLM_I and ULM achieved respectively an AP of 0.1041 and 0.0733. The AP 89 data set contains eight relevant documents. Table 5 reports the ranking of these documents to this query (topic 61) with ULM and our model. This table shows that the ranks of all relevant documents have been promoted with CLM_I. For example, the rank of document AP890106-0010 has been promoted from 41 to 1. This is mainly due to the fact that this document contains almost all of the query concepts. On the one hand, "IRAN," "Contra," "Affair," and "Israeli" are WordNet entries that occur respectively 25, 14, 3, and 2 times. In addition, the related concept "matter," which is a synonym of query concept "Affair," occurs once. On the other hand, "Iran-contra" and "Irancontra Affair" are frequent collocations in the document and occur respectively two and six times.

In the light of these results, we can conclude that incorporating diversified sources of concepts, such as frequent collocations, ontology concepts, and subconcepts may be effective.

Impact of incorporating concept relations. We evaluated integrating concept relationships into the retrieval model. This evaluation concerns CLM_R, which integrates hypernymy and hyponymy ("IS-A") relationships.

It has been shown that retrieval performance is sensitive to smoothing parameter values (Zhai & Lafferty, 2001). Thus, in our experiment we tuned parameter α in Equation 9(corresponding to CLM_R), where α is varied in [0, 1] in increments of 0.1. Figure 4 illustrates the variation of retrieval effectiveness mainly in P@10 and MAP according to α. It shows that the best performance is achieved at α value 0.3. We have also observed that the same α value corresponds to the best retrieval performance for the AP 89 data set. This shows that combining the concept relationships hypernymy and hyponymy is more effective than exploiting each one individually (when α = 0 or α = 1). Therefore, α = 0.3 is used throughout the remainder of the experiments. We recall that α and (1α) are respectively the mixture weights of the hypernym and hyponym models.

Table 6 summarizes the retrieval performance of CLM_R over CLM_I and ULM. We notice that for both data sets CLM_R significantly outperforms ULM, mainly on P@10 and MAP. It also achieves better performance than CLM_I.

In particular, we observe significant improvement over ULM on P@10 for both data sets. However, the improvement achieved by CLM_R over CLM_I is less important for the AP 89 data set. Nevertheless, these observations lead us to conclude that integrating concepts and semantic relationships are effective for IR. In what follows, we will further show this on a per-topic analysis.

Per-topic analysis. Here we refine the analysis by topic with a focus on AP. The results of comparison of the three models ULM, CLM_I, and CLM_R overall long and short queries are summarized in Table 7. This comparison shows that CLM_R is enhancing retrieval over ULM and CLM_I. For both data sets, CLM_R shows the best performance on concept queries. This enhancement in MAP is mainly due to long concepts in queries that are often monosemic (a concept having a unique sense). Therefore, their probabilities are enhanced with those of their hyponyms and hypernyms in returned documents. To illustrate our statement, we give in Table 8 a focus on ranks of relevant documents for the query "Military Coup d'etat" (topic 62). The achieved AP with ULM, CLM_I, and CLM_R are respectively 0.1446, 0.2911, and 0.3985.

It can be seen that most relevant documents were promoted. The analysis of some document content confirms that these contain, as expected, hyponyms and (or) hypernyms of query concepts. For example, document WSJ870128-0030 is promoted from 6 to 2 because it contains the concept "Forces," which is a direct hypernym of the query concept "Military" (it occurs four times just within that document).

The same observation has been noticed for AP 89 results. Table 9 presents in more detail the results of the reranking achieved per query "Israeli Role in Iran-Contra Affair" (TREC topic 61).

Table 9 highlights that most documents have been promoted with CLM_R. For example, document AP890504-0157, after being ranked at 144 with ULM, has been promoted to 43 with our model. This occurs because query concepts appear in that document as follows: concepts that are not WordNet entries, such as "Iran-Contra" occurs three times, "Iran-contra Affair" occurs twice, and WordNet entries "Israeli," "Affair," and "Nicaraguan" (hypernym of Contra) occur respectively 1, 5, and 3 times in that document. The probability of "contra" is boosted the presence of its hypernym "Nicaraguan" as shown in Equation 9. Another example illustrating concept relationships is the query "Crude Oil Price Trends" (topic 88). For this query, CLM_R reached an AP value of 0.0654, whereas ULM achieved 0.0332. This enhancement in AP is explained by the presence of the query concept "Crude Oil" (also a WordNet entry) occurring in relevant documents. In addition, the weight of that concept is boosted with its hyponyms ("Residual Oil") and hypernyms ("Fossil Fuel"). However, for nonconcept queries, CLM_R has not shown as strong an improvement as ULM and CLM_I. For most of these queries, ULM outperforms our model. One can explain this by the fact that terms of these queries are sometimes wrongly disambiguated with their first sense in WordNet, making the relationships wrong also. Another possible explanation is that there are no hypernyms and hyponyms of query concepts in relevant documents. We can conclude in general that incorporating some semantic relationships between concepts and weighting them proportionally to their importance (see Equation 13) yields significant improvements in retrieval effectiveness.

Impact of incorporating disambiguated concept relations. We evaluate our retrieval model by disambiguating query and document concepts by concept centrality described above [START_REF] Boughanem | A new factor for computing therelevance of a document to a query[END_REF]. We recall that for previous experiments we selected the first sense of the concepts.

Table 10 recapitulates the performance effectiveness of CLM_R*,CLM_R, and ULM. The reported precisions show that the improvement achieved with CLM_R* over ULM is significant. However, the comparison over CLM_R indicates mixed results. For WSJ 86-87, we notice that CLM_R* slightly outperforms CLM_R on P@10 with an improvement of +1.94%, while the improvement +9.36% is higher for the AP 89 data set. In contrast, the noticed improvement in MAP is clearly significant for WSJ 86-87. Although improvements achieved by CLM_R* are variable, we can conclude that integrating correct relationships into the retrieval model increases to some extent the retrieval effectiveness.

Per-topic analysis. To show how the disambiguation of concepts enhances retrieval effectiveness, we perform in what follows a per-topic analysis.

Table 11 shows that CLM_R* clearly outperforms ULM, while it slightly outperforms CLM_R, particularly for concept queries. Nevertheless, there are some nonconcept queries for which CLM_R* has achieved better results than did CLM_R. For instance, for the query "Information Retrieval System" (topic 65), CLM_R* and CLM_R achieved high APs of 0.3652 and 0.3519, respectively, while the AP achieved by ULM is 0.0361. The WSJ 86-87 data set contains five documents relevant for this query and they are ranked with the three compared models as follows.

The example shown in Table 12 confirms the enhancement observed on average precision, notably with the document WSJ870304-0091, whose rank jumps from 250 with CLM_R to 36 with CLM_R*. WSJ870429-0078 has also seen its rank promoted from 4 to 1. This enhancement is mainly due to the right disambiguation of query terms as follows.

a) The term "Information," which corresponds in this query to the fourth sense in WordNet (i.e., "Data: a collection of facts from which conclusions may be drawn"). For the remaining words, the first sense is indeed the right one. Relevant documents contain the term "Data" as a synonym for "Information." b) The right sense of the term "Retrieval" is still the first one in WordNet (Computer Science). This sense is also related to the concept "Storage," which is omnipresent in relevant documents such as WSJ870304-0091, where it occurs five times.

These statements confirm that incorporation of correct concept relationships into the concept-based language model improves document retrieval.

Comparison with the MRF model. We also compare our model CLM_R* to a language model named MRF [START_REF] Metzler | A Markov random field model for term dependencies[END_REF]. We used the Sequential Dependency (MRF-SD) variant of MRF (see above). The value of free parameters of the MRF-SD are taken from [START_REF] Metzler | A Markov random field model for term dependencies[END_REF]. Table 13 compares MAP achieved by these models for both data sets.

The change (%) column in Table 13 highlights that our model achieves the best MAP over MRF-SD for both data sets. The enhancement for the WSJ 86-87 is significant (+13.71%) while the improvement is less noticeable for the AP 89 data set (+3.81%) but statistically significant. However, on P@10, MRF-SD is marginally more effective than our model. This can be explained by the fact that the collocations of MRF-SD are better filtered than ours, which are somewhat noisy (adjacent and order constraint).

To further clarify results achieved by CLM_R* and MRF-SD, we give below an analysis per topic.

Per-topic analysis. We also perform an analysis per topic to show how our model outperforms the MRF-SD. The results of the comparison are given in Table 14, which shows that CLM_R* is better than MFR-SD especially for concept queries in both data sets.

Take the example of the concept query "Attempts to Revive the SALT II Treaty" (topic 69) which contains an ontological concept "SALT II." For this query, our model achieved a better AP (with a value of 0.6003) than MRF-SD and ULM, which achieved respectively APs of 0.4725 and 0.4709. We show in Table 15 the ranking of relevant documents under the three models. We can observe that ranks of some documents have decreased markedly, for instance, WSJ861201-0004, WSJ861209-0002, WSJ870507-0018, and WSJ870120-0047.

When examining document WSJ870120-0047 ranked at 175 with ULM, it appears that it does not contain some query words such as "Attempt" and "Treaty." However, it is matched to the query and is promoted from 175 to 32 and 24 with MRF-SD and CLM_R*, respectively. The explanation is that this document contains concepts related to "Treaty" such as "Accord" (synonym) and "Agreement" (hypernyms) and "SALT II" (hyponym). These concepts occur once, twice, and once, respectively. Thus, the frequency of "Treaty" is not null, and is enhanced with related concept frequencies. The same has been observed in the document WSJ870130-0003 which has been promoted from 44 and 40 with ULM and MRF-SD to 19 with our model. Although the concept "Treaty" only occurs once, its probability is enhanced with its related concepts occurring in the document, namely, "Pact"(synonym), "Accord" (synonym), "Agreement" (hypernym), and "SALT II" (hyponym).

For nonconcept queries, we can see in Table 14 that for both data sets, MRF-SD outperforms our model in MAP with the value of (+2.54%) for the WSJ 86-87 data set and (+3.15%). Let us examine for example query "1988 Presidential Candidate Platforms" (topic 80). Our model performs nearly as well as ULM, with an AP of 0.0625 (AP achieved by ULM is 0.0655), whereas the MRF-SD registered an AP with value of 0.1047. This can be explained by two main reasons: the first, the numerical date "1988," which is an important information in this query. The second is related to noisy collocations such as "Candidate Platforms" kept as a frequent collocation in our approach since (its frequency >2) but appears also in documents irrelevant to this query (topic 80).

Overall, this study allows us to confirm that our model outperforms the MRF model particularly for queries containing concepts (WordNet entries as well as frequent collocations). For nonconcept queries, the improvement for both data sets is marginal or even worse. Thus, we can expect further improvements by carrying out deeper analysis to capture semantic information proximity features, for example.

Discussion

Our research is mainly related to harnessing concepts extracted from both a semantic resource and documents (frequent collocations) to enhance document retrieval via concept queries. We have shown through the experiments reported in this paper that estimating an accurate conceptbased document model by considering concepts, subconcepts, and their relationships (synonyms, hypernyms, hyponyms) can be effective for IR.

Previous works [START_REF] Baziz | A conceptual indexing based on document content representation[END_REF][START_REF] Tu | Wikipedia-based semantic smoothing for the language modeling approach to information retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] have shown the advantage of using units (phrases or concepts) larger than single words. They exploit either an external resource (ontology) or a resource built upon a corpus to capture document and query concepts [START_REF] Bendersky | Modeling higher-order term dependencies in information retrieval using query hypergraphs[END_REF]. In the context of LMs, most research has relied exclusively on concept frequencies to estimate the concept-based document models, whereas LM variants such as the translation model provide a way to exploit semantic information such as synonymy. In addition, it is known that terminology is continuously enriched with new concepts including neologisms, named entities, or acronyms that are not yet integrated into semantic resources. That is why researchers have focused on looking for more effective approaches to capture the semantics of a document.

As mentioned previously (see the Introduction), this paper deals with two main research questions that aim to investigate the effectiveness of our proposed concept-based language model. First, we have been answering the first question and showed that a concept can be an ontological entry or a frequently found collocation (occurring more than twice in the text collection). Indeed, the latter may denote important concepts such as neologisms, proper names, or specific concepts that have no entry in the ontology. The results of the experiment showed that combining both types of concepts is effective. We believe that concept queries are predominant in domain-specific retrieval such as medical or biological queries where users use specific concepts to express their queries [START_REF] Meij | Conceptual language models for domain-specific retrieval[END_REF][START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF]. However, on the web we rely on a study of [START_REF] Bendersky | Analysis of long queries in a large scale search log[END_REF] where they showed that long queries (containing more than four words) also called verbose queries are more likely to contain concepts.

Second, to estimate the concept-based model, we have shown the possibility of integrating concepts from the ontology as well as from the document collection into a unified language model framework. Our intuition is similar to the one used in [START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF], which also considered frequently found collocations and ontological concepts (a concept is a set of synonyms). However, in the [START_REF] Zhou | Topic signature language models for ad hoc retrieval[END_REF] approach, both frequently found collocations and ontological entries are used equally; that is, the document model is smoothed with concept frequencies, which are either ontology entries or not. In our approach, the conceptbased model is estimated by considering subconcepts and synonyms (since a concept is a set of synonyms). The results we obtained by comparing the model combining these elements to single word and concept unigram models have shown improvements in terms of MAP. Furthermore, we exploit semantic relationships between concepts to semantically match query and document concepts in such a way that generic concepts (hypernyms) are separated from specific concepts. Therefore, more distant dependencies than word proximity [START_REF] Bai | Query expansion using term relationships in language models for information retrieval[END_REF][START_REF] Gao | Dependence language model for information retrieval[END_REF][START_REF] Zhao | A proximity language model for information retrieval[END_REF]) are integrated into the retrieval language model. [START_REF] Cao | Integrating word relationships into language models[END_REF] also integrated more distant dependencies between query and document words through word relationships (cooccurrence, synonymy, hypernymy, and hyponymy). However, they have not considered the meaning of single words, which are often ambiguous, so the problem of polysemy is still present. We demonstrated here that concept relationships are effective to improve IR performance in comparison to state-of-the-art models in cases in which the queries are concepts.

Our work has some limitations:

• The proposed concept-based model, in particular the collocation-based one, is mainly estimated using frequencies.

We think that it can be further enhanced by considering frequent subconcepts as the approach proposed in [START_REF] Hammache | Combining compound and single terms under language model framework[END_REF]. • Our approach of recognizing concepts is somehow limited by order because we extract first the collocations that vary from one to three words. We can overcome this limitation by relaxing concept word adjacency and recognizing concepts in a larger context such as a text passage. • The number of trials for this experiment are the total number of queries in the WSJ data set (49) plus the total number from the AP data set (47). The number of trials is acceptable, but we need to perform further experiments using a larger set of queries (more than 100) to better assess the robustness of our model. • Word collocation would need to be performed upon document ingest, and whenever new items are added to the collection (rather than at query time) in order for the system to retrieve at decent speed. Web-wide, this method would require massive item indexing.

Conclusion and Future Work

In this paper we introduce a concept-based language modeling approach to enhance information retrieval. In our approach, document and queries are represented through concepts. We consider concepts (that vary from one to two or three words) that can be an ontology entry or a frequent collocation having no entry in the ontology. This leads to a rich representation of document content and closes the semantic gap between query and document (brought about assuming the word independence). In addition, we integrate concept relationships a priori defined in the ontology WordNet to better model the document. Indeed, we consider that a document can be represented by concepts (synonyms, hypernyms, hyponyms, subconcepts, frequent collocations, and single words).

Our experimental results on TREC data sets showed that our model yields significant improvements over the unigram model and the MRF-SD by [START_REF] Metzler | A Markov random field model for term dependencies[END_REF]. This has been noticed for queries containing concepts that are long queries or queries that are wholly ontology entries. We showed also that differentiating concept relationships such as the hypernymy and the hyponymy is promising for retrieval.

We plan in the short term to test our model on medical collections, where the notion of concept is important. In the long term, our model could be further improved by integrating additional Natural Language Processing rules for recognizing useful phrases. For nonconcept queries, it would be interesting to use an additional resource such as Wikipedia, Dbpedia, or Yago to identify concepts. This might also enhance the query model with term relationships through the graphical structures of these resources.

  a) Evaluating the combination of ontological and the nonontological concepts in a language model. b) Highlighting the impact of incorporating semantic information such as subconcepts and concept relationships (hyponyms, hypernyms). c) Assessing the importance of disambiguating concepts in retrieval. d) Comparing our model to two language models, namely, the unigram model smoothed with Dirichlet prior and the MRF language model.

TABLE 1 .

 1 Comparison between performances of the unigram model (ULM) and the combined concept-based model CLM_I. Signs + and ++ indicate that the difference is statistically significant according to t-test at p-value < .05 and p-value < .01, respectively.

				Performance evaluation
	Collection	Model	P@10	P@20	MAP
	WSJ 86-87	ULM	0.3280	0.2790	0.2302
		CLM_I	0.3640	0.3030	0.2376
		Gain over ULM(%)	+10.98 ++	+8,60 ++	+3,21 +
	AP 89	ULM	0.3160	0.2810	0.1809
		CLM_I	0.3280	0.2910	0.1925
		Gain over ULM(%)	+3,80 +	+3,56 +	+6,41 ++

TABLE 2 .

 2 TREC-query examples of concept and nonconcept queries (OE and FC indicate ontology entry and frequent collocation, respectively).

	Collection	Concept queries	Nonconcept queries
	WSJ 86-87 Query 51: Airbus Subsidies	Query 85: Official corruption
		(FC)
		Query 65: Military Coups	Query 94: Computer-aided
		D'etat (OE)	crime
	AP 89	Query 64: Hostage-Taking	Query 59: Weather-related
		(FC)	Fatalities
		Query 79: FRG Political	Query 95: Computer-aided
		Party Positions (OE)	crime detection

TABLE 3 .

 3 Per-topic analysis of ranking models (CLM_I vs. ULM).

						TABLE 4. Ranking of relevant document TREC topic 62 with ULM and
						CLM_I.
							Document ranking
						Document number	ULM	CLM_I	Change in ranking
						WSJ871016-0026	88	34	-54
						WSJ870928-0164	29	8	-21
						WSJ870831-0105	2	1	-1
						WSJ870828-0026	13	9	-4
						WSJ870828-0019	80	45	-35
						WSJ870518-0113	98	19	-53
						WSJ870514-0016	25	67	+42
						WSJ870128-0030	8	6	-2
						WSJ870127-0005	103	132	+29
						WSJ870526-0068	114	22	-92
				CLM_I vs. ULM
	Collection	Query category	-	=	+	Change (%)
	WSJ 86-87	Concept queries (35)	13	3	19	+38,61
		Nonconcept queries (14)	6	2	6	+21,63
	AP 89	Concept queries(27)	8	3	16	+4,79
		Nonconcept queries (20)	12	2	6	-18,00

TABLE 5 .

 5 Ranking of AP 89 relevant document to TREC topic 61 with ULM and CLM_I.

			Document ranking
	Document number	ULM	CLM_I	Change in ranking
	AP890105-0053	26	22	-4
	AP890106-0010	41	1	-40
	AP890125-0124	98	71	-27
	AP890328-0075	54	16	-48
	AP890329-0168	11	31	+20
	AP890408-0099	65	53	-12
	AP890413-0132	6	18	+12
	AP890504-0157	144	60	-84
				FIG. 4. Variation of P@10 and MAP with α using CLM_R for the WSJ
				86-87 data set.

TABLE 6 .

 6 Comparison between ULM and concept-based models CLM_I and CLM_R. Signs + and ++ indicate that the difference is statistically significant according to t-test at p-value < .05 and p-value < .01, respectively.

				Performance evaluation
	Collection	Model	P@10	P@20	MAP
	WSJ 86-87	ULM	0.3280	0.2790	0.2302
		CLM_I	0.3640	0.3092	0.2376
		CLM_R	0.376	0.317	0.2477
		Gain over ULM (%)	+14,63 ++	+13,62 ++	+7,60 ++
		Gain over CLM_I (%)	+3,23 +	+2,52 +	+4,25 +
	AP 89	ULM	0.3160	0.2810	0.1809
		CLM_I	0.3280	0.2910	0.1925
		CLM_R	0.3420	0.2940	0.1932
		Gain over ULM (%)	+8,23 ++	+4,63 +	+6,79 +
		Gain over CLM_I (%)	+4,27 ++	+1,03	+0.36

TABLE 7 .

 7 Per-topic analysis of ranking models (CLM_R vs. ULM, and CLM_I).

				CLM_R vs. ULM			CLM_R vs. CLM_I
	Collection	Query category	-	=	+	Change	-	=	+	Change
	WSJ 86-87	Concept queries (35)	8	4	23	+34,06	15	2	18	+7,15
		Nonconcept queries (14)	6	1	8	+21,00	6	2	6	-4,5
	AP 89	Concept queries (27)	7	2	18	+15,40	11	3	13	+10.12
		Nonconcept queries (20)	11	2	9	-2,72	10	3	7	-12,04

TABLE 8 .

 8 Ranks of relevant documents to TREC topic 62 with ULM, CLM_I, and CLM_R.

			Document ranking
	Document number	ULM	CLM_I	Change	CLM_R	Change
	WSJ871016-0026	88	34	-84	28	-6
	WSJ870928-0164	29	8	-11	3	-5
	WSJ870831-0105	2	1	-1	1	0
	WSJ870828-0026	13	9	-5	4	-1
	WSJ870828-0019	80	45	-35	125	+122
	WSJ870518-0113	98	19	-79	14	-5
	WSJ870514-0016	25	67	+42	121	+54
	WSJ870128-0030	8	6	-2	2	-4
	WSJ870127-0005	103	132	+31	95	-37
	WSJ870526-0068	114	22	-92	18	-4

TABLE 9 .

 9 Ranking of relevant documents to TREC topic 61 with ULM, CLM_I, and CLM_R.

			Document ranking
	Document number	ULM	CLM_I	Change	CLM_R	Change
	AP890105-0053	26	22	-4	20	-2
	AP890106-0010	41	1	-40	1	0
	AP890125-0124	98	71	-27	59	-12
	AP890328-0075	54	16	-38	13	-3
	AP890329-0168	11	31	+20	11	-20
	AP890408-0099	65	53	-12	37	-16
	AP890413-0162	53	18	-35	6	-12
	AP890504-0157	144	60	-84	43	-17

TABLE 10 .

 10 Comparison of CLM_R*, CLM_R, and ULM performances. Signs + and ++ indicate that the difference is statistically significant according to t-test at the level of p-value < .05 and p-value < .01.

				Performance evaluation
	Collection	Model	P@10	P@20	MAP
	WSJ 86-87	ULM	0.3280	0.2790	0.2302
		CLM_R	0.376	0.317	0.2477
		CLM_R*	0.3833	0.3102	0.2827
		Gain over ULM (%)	+16,85 ++	+15,05 ++	+22,81 ++
		Gain over CLM_R (%)	+1,94	-2,14	+6,12 ++
	AP 89	ULM	0.3160	0.2810	0.1809
		CLM_R	0.3420	0.2940	0.1932
		CLM_R*	0.3740	0.3090	0.1935
		Gain over ULM (%)	+14,63 ++	+13,62 ++	6.97 ++
		Gain over CLM_R (%)	+9,36 ++	+5,10 ++	+0.16

TABLE 11 .

 11 Per-topic analysis of ranking models (ULM, CLM_R, and CLM_R*).

				CLM_R* vs. ULM			CLM_R* vs. CLM_R
	Collection	Query category	-	=	+	Change	-	=	+	Change
	WSJ 86-87	Concept queries (35)	12	1	26	+41,27	11	10	15	+6,15
		Nonconcept queries (14)	6	2	7	+21,00	7	2	5	+1,08
	AP 89	Concept queries (27)	8	4	15	+14,50	4	3	16	+30.35
		Nonconcept queries (20)	7	2	11	+1,09	6	4	10	+2,35

TABLE 12 .

 12 Ranking of relevant documents to TREC topic 65 with ULM, CLM_R, and CLM_R*.

				Document ranking
	Document number	ULM CLM_R Change CLM_R* Change
	WSJ870304-0091	137	250	-113	36	-137
	WSJ870331-0042	98	71	-27	59	-12
	WSJ870429-0078	48	4	-44	1	-43
	WSJ871103-0021	17	16	-1	11	-5
	WSJ871202-0145	311	19	-292	8	-11

TABLE 13 .

 13 Comparison between CLM_R* and MRF-SD performances. Signs + and ++ indicate that the difference is statistically significant according to t-test at p-value < .05 and p-value < .01, respectively.

		Performance	MRF -SD	CLM_R*	Change (%)
	WSJ 86-87	MAP	0.2486	0.2827	+13,71 ++
		P@10	0.3857	0.3833	-0.62
	AP 89	MAP	0.1864	0.1935	+3,81 +
		P@10	0.3851	0.3740	-2,88 +

TABLE 14 .

 14 Per-topic analysis of ranking models (CLM_R* vs. MRF-SD).

				CLM_R* vs. MRF-SD
	Collection	Query category	-	=	+	Change (%)
	WSJ 86-87	Concept queries(35)	12	3	20	+4,93
		Nonconcept queries (14)	8	1	6	-2,54
	AP 89	Concept queries (27)	12	2	13	+2,08
		Nonconcept queries (14)	10	2	10	-3,15

TABLE 15 .

 15 Ranking of relevant documents to TREC topic 69 with ULM, MRF-SD and CLM_R*.

				Document ranking
	Document number	ULM MRF-SD Change CLM_R* Change
	WSJ861201-0004	53	40	-13	17	-27
	WSJ861202-0040	3	2	-1	2	0
	WSJ861204-0019	52	44	-12	36	-8
	WSJ861205-0001	40	58	+18	35	-23
	WSJ861209-0002	105	104	-1	39	-65
	WSJ861216-0141	7	15	+8	13	-2
	WSJ861218-0172	2	1	-1	1	0
	WSJ861222-0149	4	6	+2	4	-2
	WSJ861229-0047	27	31	+4	28	-5
	WSJ870106-0081	25	30	+5	21	-9
	WSJ870120-0047	175	32	-143	24	-8
	WSJ870130-0003	44	40	-4	19	-21
	WSJ870203-0101	8	4	-4	7	+3
	WSJ870305-0116	1	9	-8	9	0
	WSJ870506-0144	15	25	-10	16	-9
	WSJ870507-0018	82	67	-15	34	-33

Extraction tool of UMLS concepts.

Collocation extraction tool.

trec.nist.gov.

Acknowledgments

We thank the reviewers for their helpful comments and suggestions.