Nicolas Marie
email: nicolas.marie@inria.fr

Fabien Gandon

Alain Giboin

Émilie Palagi

Exploratory search on topics through different perspectives with DBpedia

Keywords: [Graph Theory]: Graph algorithms; [HCI design and evaluation methods]: User studies Algorithms, experimentation Exploratory search, Multi-perspectives exploration, DBpedia, Discovery engine, Discovery Hub

A promising scenario for combining linked data and search is exploratory search. During exploratory search, the search objective is ill-defined favorable to discovery. A common limit of the existing linked data based exploratory search systems is that they constrain the exploration through single results selection and ranking schemes. The users can not influence the results to reveal specific aspects of knowledge that interest them. The models and algorithms we propose unveil such knowledge nuances by allowing the exploration of topics through several perspectives. The users adjust important computation parameters through three operations that help retrieving desired exploration perspectives: specification of interest criteria about the topic explored, controlled randomness injection to reveal unexpected knowledge and choice of the processed knowledge source(s). This paper describes the corresponding models, algorithms and the Discovery Hub implementation. It focuses on the three mentioned perspective-operations and their evaluations.

INTRODUCTION

As stated by White in [START_REF] White | Supporting exploratory search, introduction, special issue, communications of the acm[END_REF]: "search is only a partially solved problem". Exploratory search refers to cognitively consuming search tasks like learning or investigation [START_REF] Marchionini | Exploratory search: from finding to understanding[END_REF]. The term learning is employed in the broad sense and can concern educational, professional and personal contexts. During exploratory search the search objective is ill-defined, favorable to discovery. Actual popular search engines are evolving in the right direction but are still not supporting exploratory search efficiently today. This is notably due to their keyword-search paradigm and lack of assistance during the results consultation. First it is impossible to capture complex information needs in few keywords, especially for vague ones. Second the users have to synthesize an important amount of information without support from the system. They have to rely on their own search strategies leading to a considerable cognitive load. It increases the information integration work needed to understand and to use the information collected: "the human user's brain is the fundamental platform for information integration" [START_REF] Brambilla | Designing exploratory search applications upon web data sources[END_REF]. There is a need to complete the actual widely-used solutions by designing and popularizing systems optimized for exploratory search tasks. This challenge requires contributions from several research fields including in particular information retrieval and interaction design1 .

Linked data [START_REF] Bizer | Linked data-the story so far[END_REF] and DBpedia [START_REF] Lehmann | Dbpedia-a large-scale, multilingual knowledge base extracted from wikipedia[END_REF] have been extensively described in the scientific literature and respectively correspond to an approach for publishing and linking data on the web and its application to data extracted from Wikipedia2 . The improvement of search through incorporation of semantics is referred to as semantic search. Semantic search has been the subject of numerous researches targeting a wide range of search objectives [START_REF] Tran | A survey of semantic search approaches[END_REF]. Today important initiatives emerge from major players including the knowledge graphsbased search engines functionalities (based on Bing Satori3 , Google 4 and Yahoo Knowledge Graphs [START_REF] Blanco | Entity recommendations in web search[END_REF]), Apple Siri5 and the Facebook Graph Search 6 . Such technologies and functionalities open the door to a better support of exploratory search tasks ("explore your search", "help you research a topic more in depth than before" 4 ). Even if some of these graphs are partially built from open data sources they are not publicly accessible and are consequently not part of the linked open data cloud.

Linked data are promising for supporting exploratory search. Their richness and structured aspect allow proposing new al-gorithms and interaction models optimized for exploration purposes. The research in this field still faces several challenges. One limit is that existing linked data based systems often offer only one exploration perspective i.e. the users can not or hardly influence the queries' results in a direction of interest. This paper supports [START_REF] Bizer | Linked data-the story so far[END_REF] the idea that a plurality of relevant exploration perspectives can be offered to the users starting from a topic of interest, (2) that some linked data datasets constitute a valuable source of knowledge for such multi-perspectives exploratory search. Indeed, the objects described in linked data datasets can be rich, complex and approached in many manners. For example, a user can be interested in a painter (e.g. Claude Monet or Mary Cassat) in many ways: works, epoch, movement, entourage, social or political contexts and more. The user may also be interested by basic information or by unexpected and unusual ones depending on his actual knowledge about the painter. He may also want to explore the topic through a specific culture or area e.g. impressionism in American or French culture. A single interest can be explored through many perspectives corresponding to different knowledge nuances. In the graph context of linked data these perspectives correspond to different non exclusive sets of objects and relations that are informative on a topic regarding specific aspects.

In our proposition such exploration perspectives are obtained by increasing the results relevance thanks to users' query refinement/personalization and provoking discoveries by injecting randomness during the results' computation. Such topics are not new in the general field of information retrieval but, to the best of our knowledge, no approaches were formalized, implemented and evaluated in the context of linked data based exploratory search. This flexibility in the query processing is reached thanks to a framework that computes the results at query time and that is loosely coupled to the data source queried.

STATE-OF-THE-ART

In this state-of-the-art we consider exploratory search systems (ESS) and recommenders. As mentioned in [START_REF] Mirizzi | Exploratory search and recommender systems in the semantic web[END_REF] despite their differences recommenders and exploratory search systems show interesting intersections. They share the objective of assisting the users in information or resource discovery in a collection. However they achieve it in very different ways. The recommenders provide direct suggestions that require no or minimal users interactions whereas the exploratory search systems lead to discoveries through users' engagement and high interactivity. Moreover some linked data-based exploratory search systems integrate recommendation functionalities e.g. query terms recommendation for assisting and inspiring the users in their query formulation as in LED 7 . Thus it is interesting to review linked-data based recommenders today as few exploratory search systems based on such data exist.

A list of recommenders and exploratory search systems based on linked data was presented in [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF]. In the present paper we focus on applications having a user interface allowing the users to define the perspective they want to explore about topics of interest. By performing this state-of-theart, we wanted in particular to review how existing recommenders' and exploratory search systems implemented such a perspective-setting, and to determine how this perspective-setting could be improved. The systems presented hereafter are summarized in Table 1. We observed that the following limits are recurrent:

• The results selection process is generally fixed.

• The ranking scheme is generally fixed.

• The data source(s) used to process the query is fixed. LED (Lookup Explore Discover) [START_REF] Mirizzi | Exploratory search and recommender systems in the semantic web[END_REF] is an exploratory search system that suggests related query-terms starting from a user initial query. It implements the DBpedia Ranker algorithm for such recommendations. Seevl8 is a music discovery platform that offers DBpedia-based artists' recommendations thanks to the DBrec algorithm [START_REF] Passant | Dbrec-music recommendations using dbpedia[END_REF]. Yovisto9 is an academic video platform offering related-queries suggestions computed with a set of heuristics on the German and English DBpedia chapters [START_REF] Waitelonis | Augmenting video search with linked open data[END_REF]. These 3 applications do not allow the users to influence the results retrieved.

Starting from a resource of interest Aemoo10 visually presents its direct neighborhood filtered with semantic-based patterns called Encyclopedic Knowledge Patterns (EKPs) [START_REF] Musetti | Aemoo: Exploratory search based on knowledge patterns over the semantic web[END_REF]. EKPs are selections of the most informative classes regarding a specific class: "the most relevant types of things that people use for describing other things". For instance the DBpedia Actor class EKP includes the classes Actor, City, Film, TelevisionShow, etc. Aemoo proposes a "curiosity" function which displays the queried resource neighborhood through an inverted EKP filtering. This function offers an exploration perspective that aims to unveil unexpected knowledge. The MORE movie recommender11 [START_REF] Mirizzi | Exploratory search and recommender systems in the semantic web[END_REF] is based on a semantic adaptation of the Vector Space Model and allows the users to tune the importance of each vector/property for the recommendation e.g. director, genre, starring.

There is a certain lack of flexibility in the existing approaches based on linked data. In other words considering a topic, captured in the form of a resource, there is only one or few processing and consequently one or few results space(s) available. This lack of flexibility is notably due to the fact that the results are pre-computed and stored for later retrieving. The users retrieve the pre-computed results and are consequently not able to influence the computation parameters. We propose a flexible framework offering several exploration perspectives on the users' topics of interest. Contrary to other approaches the results are computed at query-time allowing the users to have the hand on several parameters through the interface of an application implementing the framework.

RELEVANCE AND SURPRISE AS USER-CENTRIC CRITERIA

Framework basis

The framework we propose for linked data based exploratory search was described in [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF], its general architecture is presented on Figure 1. It is based on a semantic-sensitive spreading activation algorithm that is coupled to an incremental and live graph sampling technique. Spreading activation is a class of algorithms that iteratively diffuse a value [START_REF] Cohen | Information retrieval by constrained spreading activation in semantic networks[END_REF]. The value distribution and the stop conditions depend on the implementation objective. In our case the activation spreads only to nodes belonging to a subset of classes identified as relevant (CP D(o) below) and favors the nodes similar to the initial node of interest thanks to a triple-based similarity measure (ctriple(i, o) below). Concerning the architecture the algorithm is applied on a small sub-graph incrementally imported at query-time. The imported triples are stored in a local KGRAM 12 triple store using IN SERT queries 13 sent to a targeted SPARQL endpoint. The full algorithm formalization is published in [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF].

Definition 1: semantic spreading activation

a(i, n + 1, o) = s(i, n, o) + w(i, o) * j∈N eighbors(i) a(j,n,o) degree j
where:

• a(i, n+1, o) is the activation value of node i at iteration n + 1 for an initial stimulation at o (the starting point of the spreading activation). o being the user's query e.g. the DBpedia resource Claude Monet 14 ;

• s(i, n, o) is the external stimulation value of the node i at iteration n for an initial stimulation at o; • N eighbors(i) is the set of neighbors of the node i in the linked data graph:

N eighbors(i) = x; (i, p, x) ∈ KB ∨ (x, p, i) ∈ KB ∧ p = rdf : type ∧ x ∈ ∪B
• degreej is the number of edges involving the node j: degreej = |(j, p, x) ∈ KB ∪ (x, p, j) ∈ KB | • where w(i, o) is a semantic-based pattern that constrains the propagation to a subset of resources' types:

w(i, o) = 0 if ∄ t ∈ T ypes(i) ; t ∈ CP D(o) 1 + |ctriple(i, o)| otherwise where:
• t is a type of i;

• T ypes(i) is the set of types of i;

• CP D(o) is the Class Propagation Domain i.e. a subset of classes that are relevant to the query and that

12 http://wimmics.inria.fr/corese 13 http://www.w3.org/TR/sparql11-update/#deleteInsert 14 Claude Monet is a recurrent example for Discovery Hub. It was initially chosen for a screencast presented during the Semanticpedia presentation which was strongly related to culture. Claude Monet was the first art-related query entered by a user in Discovery Hub constrains the propagation. These classes are selected among the types of neighbors of the initially stimulated node o. The idea behind it is that informative types regarding a topic of interest must be present in its direct neighborhood. It aims to augment the quality and lower the cost of the spreading activation by applying it only to nodes belonging to this subset of types, see [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF] for more information.

• ctriple(i, o) (common triples) is a semantic similarity function that calculates the amount of triples having the activated node i or the origin o as objects and similar property (p) and value (v). ctriple(i, o) = (i, p, v) ∈ KB; ∃ (o, p, v) ∈ KB; • KB is the set of all the triples (subject,predicate,object) in the triple store. The objective of the spreading activation step is to identify a set of relevant results related to the initial topic of interest that will be explored by the user. They constitute an exploration perspective. The live graph importation and computation allow to adjust several important computation parameters before the processing. The changes in the algorithm configuration lead to substantial variations in the results list retrieved (composition and ranking).

DBpedia was chosen for our implementation, called Discovery Hub, because it captures a rich knowledge about many domains (arts, politics, history, etc). We also wanted to leverage the advantage that DBpedia international chapters now exist in 15 languages 15 e.g. the French, German, Italian and Spanish ones. The Discovery Hub exploratory search engine is accessible online 16 and was showcased in several screencasts 17 , see also Figure 2. A video of an exploration performed with Discovery Hub and related commentaries are available online 18 . The details of the implementation including its main parameters and the graph sampling functioning are published in [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF]. Discovery Hub offers functionalities that helps exploration such as faceted browsing and results explanations features. It presents the results sorted by types (corresponding to CP D(o), used as facets, to ease the results understanding and browsing e.g. artist, museums for a painter. The users have the possibility to import their Facebook likes in order to query them. In the context of long-lasting interests they can also gather the results of their searches in collections. An important part of the DBpedia knowledge is captured by the hierarchy of categories. In Wikipedia the articles are classified into hierarchically organized categories that appear at the bottom of the pages. For instance Claude Monet 19 belongs to, among others: artists from Paris, French impressionists painters, alumni of the école des Beaux-Arts. In DBpedia the resources are related to their categories with the dcterms:subject property 20 . The value of using the DBpedia categories to compute recommendations was shown in [START_REF] Di Noia | Linked open data to support content-based recommender systems[END_REF]. To demonstrate the impact of categories in our case we reused the results of the previous experimentation detailed in [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF]. During this experimentation we asked 15 participants to rate the results retrieved by our framework. The results lists were constituted of 20 films that originated from 5 queries having a film as input. These films were randomly selected in the 50 films to see before you die list 21 : 2001 a space odyssey, Erin Brokovitch, Terminator 2, Princess Mononoke and Fight Club (mentioned later as query-films).

The participants rated the results on their relevance and discovery aspects. 16 http://discoveryhub.co 17 https://www.youtube.com/user/wearediscoveryhub 18 https://www.youtube.com/watch?v=MUK01T-n1Ks 19 http://en.wikipedia.org/wiki/Claude Monet 20 http://purl.org/dc/terms/subject 21 http://www.listal.com/list/film-4-50-films-see

We observed the categories consideration influence on the results lists rankings. We constituted 2 golden truth results lists for each 5 query-films by ordering them by their relevancy and unexpectedness in decreasing order. Then we compared our framework results with (ctriple(i, o) with p = dcterms : subject) and without (ctriple(i, o) always equal to 0) categories consideration to these golden truth lists. The lists were compared to the golden truth ones thanks to a Kendall-Tau rank correlation coefficient [START_REF] Kendall | Rank correlation methods[END_REF] where 1 corresponds to a perfect rank correlation, see the results on Figure 3. It is observable that on average the ranking correlation with the golden truth is better when the categories are considered for both the relevance and the discovery for this set of movies. This goes in the sense of [START_REF] Di Noia | Linked open data to support content-based recommender systems[END_REF].

Specification of Interest Criteria

Using Discovery Hub the users can specify criteria of interest and disinterest that are used by the framework during the sample importation and its computation, see Figure 4. The objective is to guide the spread and modify the activation values in order to retrieve results that are more specifically related to aspects that interest the users. The criteria specification function modifies the definition 1 as follows:

Definition 2: semantic spreading activation, criteria specification variant a(i, n + 1, o, V ) = s(i, n, o) + wcrits(i, o, V ) * j a(j,n,o) degree j where:

w(i, o, V ) = 0 if ∄ t ∈ T ypes(i); t ∈ CP D(o) 1 + |ctriplecrits(i, o, V )| else
where:

• ctriplecrits(i, o, V ) = p ∈ P, v ∈ V, (i, p, v) ∈ KB, ∃ (o, p, v) ∈ KB;
• P is the set of properties used for the triple-based similarity measure,

• V is the set of criteria of interest specified by the user.

In our implementation the DBpedia categories are used as criteria of interest/disinterest i.e. p = http://purl.org/dc/ terms/subject. We made this choice because as shown in the For instance by declaring such criteria the user can perform a query specifying that he is interested in Claude Monet because he is an impressionist but not because he is French. Examples of such queries and corresponding top results are presented in Table 2. The results presented are the Artist facet results. The DBpedia categories often reflect pieces of information associated to a class e.g. movement, origin for artists. The influence of the criterions selection is consequently easily observable on this facet (but have also influence on the others). The top 10 results lists presented in Table 2 are all related to Claude Monet but constitute different perspectives:

• The first query where no category of interest are specified retrieves artists that are strongly related to France and impressionism: 9 on 10 are French, 8 on 10 are impressionists.

• The second query where all the categories related to France were declared as uninteresting and the category impressionist painters declared of interest retrieves 9 non-French (and 1 French) impressionists painters. American artists are well represented.

• The third query where all the categories related to France were declared interesting and the category impressionist painters declared uninteresting retrieve 10 French painters where 5 are not impressionists (realist, fauvist, romantic), 4 are impressionist but not only (fauvist, cubist, modern artist) and only 1 who is only declared as impressionist and post-impressionist.

Randomness injection

It is possible to inject randomness into the propagation process in order to modify the ranking scheme and expose more unexpected results. It changes the algorithm behavior that was originally designed to retrieve the results that are the most related regarding the activation's origin i.e. the most obvious ones. This randomizing operation is particularly interesting for experts who want to retrieve unusual information in order to deepen their peripheral knowledge on a topic. To avoid to confront quickly the user with too surprising results the algorithm is different if the chosen level of randomness is inferior or superior to 0.5 (minimum 0, maximum 1). If the value is inferior or equal to 0.5 the results are randomized only at the last iteration. In other words the spreading activation occurs normally till the last iteration. If the desired serendipity level is superior to 0.5 the randomization occurs at each iteration influencing strongly the spreading activation algorithm and consequently the results list. where:

• r is the level of randomness specified by the user, comprised between 0 and 1;

• random() retrieves a random value between 0 and 1;

• maxP ulse is the maximum number of spreading activation iterations.

Data source selection

The data source used to process the query can be easily changed with the proposed framework. It is especially interesting in the distributed context of the LOD where many datasets capturing different knowledge are accessible through their public SPARQL endpoints22 . In the case of DBpedia it enables the use of the local DBpedia version SPARQL endpoints. Today 15 chapters are accessible online 23 . The DBpedia chapters vary significantly in what they describe and how they describe it 24 . In this publication we focus on the 5 DBpedia chapters that propose more than 100 millions triples: the English, French, German, Italian and Spanish ones.

The differences between the results when using different SPARQL endpoints are substantial. They act as a cultural prism, quantitatively studied in the next section of this paper. Continuing with the example of Claude Monet we executed the query on the 5 previously mentioned SPARQL endpoints. Regarding the Artist and Museum facets, both interesting because strongly associated to a country and culture, we observed that:

• Using the English DBpedia chapter, 4 artists and 5 museums in the tops 10 are from English-speaking countries (as officially recognized) i.e. United Kingdom and the United States. Contrary to other languages the Art Institute of Chicago is ranked as the first museum (instead of the Orsay Museum).

• Using the French DBpedia chapter 9 artists are French in the top 10 artists and 9 museums are situated in French-speaking countries (as officially recognized): 1 in Switzerland and 8 in France. • The Botero museum, situated in Columbia, appears only in the Spanish chapter results.

EVALUATIONS

One of the toughest difficulty encountered by the exploratory search community is the evaluation of the systems [START_REF] Kules | What do exploratory searchers look at in a faceted search interface[END_REF]. In fact there is no standard metric for this purpose. The high human engagement in the search process necessitates adapted evaluation protocols that both evaluate the IR and the HCI aspects of the systems. The traditional information retrieval evaluations metrics such as recall and precision are not sufficient to evaluate this human engagement. More user-centric metrics are necessary. In this article we decided to focus on the quality of the retrieved results, on their relevance and discovery potential, rather than on the interactions.

Specification of Interest Criteria

The positive impact of criteria of interest specification was briefly studied in [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF] where the top 10 films results of Fight Club were evaluated a second time with 3 criteria of interest specified by each participant. The average relevance rating rose from 1.42 to 1.94 (with 0 corresponding to not interesting at all and 3 corresponding to very interesting). In order to confirm this positive influence on the results we built another protocol. This protocol also covers the evaluation of the randomness injection influence, discussed in the following sub-section. We wanted to confirm that the model we propose for capturing topic-related criteria of interest is relevant and modifies efficiently the results retrieved by the algorithm. For this we formulated the following hypotheses:

• Hypothesis 1: Users who specify their criteria (categories) of interest about a topic before launching the search, find the results of the search more relevant than users who did not specify their criteria.

• Hypothesis 2: Users who specify their criteria (categories) of interest about a topic do not find the results of the search less novel than users who do not specify their criteria. In other words, there is no loss of discovery power due to the specification of interest criteria.

First we randomly choose a set of 20 queries -i.e. DBpedia resources -from the query log of Discovery Hub. These queries or resources are hereafter referred to as exploration topics. Second we asked 16 participants to select in this list the 4 exploration topics that were the most interesting to them. We retained the 2 topics selected by the largest number of participants: information visualization and the singer Serge Gainsbourg. This selection is interesting regarding exploratory search as it is composed of a topic mainly related to a professional interest (information visualization) and an exploration topic related to a personal interest (Serge Gainsbourg). Then we asked each participant to specify the categories associated to each topic they considered either as interesting or uninteresting. 5 categories were available for information visualization and 19 for Serge Gainsbourg. Finally a list of results was generated with 4 algorithm configurations: with the basis formula (definition 1), with categories consideration (personalized per participant, definition 2), with a random level of 0.5 and 1 (definition 3). All these results were randomized in a single list. The participants evaluated them with the Discovery Hub application notably using the explanation features. An evaluator was present during the test to help them and to collect their impressions for further research. The participants were composed of 6 females and 10 males of 31 years old on average.

Our experimentation aimed to evaluate the interest and the surprise of the users regarding the perspectives they can explore. The framework notably proposes operations that constrain (criteria specification) or free (randomness injection) the spread over the data graph in order to increase the users' interest or surprise. We wanted to measure the influence of such variants on both the interest and surprise. For a precise evaluation we proposed the following definitions to the participants. A result is surprising if:

• You discovered an unknown resource or relation.

• You discovered something unexpected.

A result is interesting if:

• You think it is similar to the topic explored.

• You think you will remind or reuse it.

The users were invited to evaluate the interestingness and surprisingness of each result by indicating their degree of agreement or disagreement about the four following statements, presented in the form of a 4-point Likert scale:

• S1: This result in itself is surprising: Not agree at all 1-2-3-4 Totally agree;

• S2: This relation between the topic searched and the result is surprising: Not agree at all 1-2-3-4 Totally agree;

• S3: This result is interesting: Not agree at all 1-2-3-4 Totally agree;

• S4: This result is too distant from the topic searched: Very close 1-2-3-4 Too distant. The first interesting observation is that the selection of categories was very diverse. Only 2 criteria selections on 16 were doubloons for information visualization and 1 for Serge Gainsbourg. It confirms that regarding a topic the users are interested in different aspects and that allowing the exploration of topics through different perspectives might be relevant. Looking at Figure 5 we observe that the results generated by the algorithm considering the users' categories are judged more interesting than the results generated by the other algorithms thus the hypothesis 1 is validated. Conversely, we observe that these results are judged a bit less surprising thus the hypothesis 2 is not validated. Otherwise the loss in term of surprise is minor and does not require in our sense a modification of the algorithm. It is certainly due to the prior knowledge of the users' about the criteria of interests they specified. Concerning the agreement the standard deviation was of 0.54 on average for all the different metrics and algorithm variants. The maximum average standard deviation was 0.68 (surprisingness of the relation, 0.5 randomized variant) and the minimum was 0.37 (perceived distance, basis formula).

Randomness injection

We formulated 2 hypotheses related to the controlled randomness injection functionality.

• Hypothesis 3: The stronger is the level of randomness the more surprising the results are for the users.

• Hypothesis 4: Even if the level of surprise is high, the majority of the top results are still relevant to the users.

Looking again at Figure 5 we observe that the results with a randomness set at 1 are judged more surprising than the ones with a randomness set at 0.5. Thus the hypothesis 3 is validated. We also observe that a majority are judged irrelevant (>2.5). Thus the hypothesis 4 is not validated. The intersection of the results evaluated as very interesting and very surprising is in favor of the lower randomness value (0.5). Indeed, their percentage reaches only 3.3% for the randomness value of 1 versus 7.5% for the 0.5 value (4.5% for the other algorithm variants). Thus we might use lower levels of randomness in the future to obtain a better trade-off between relevance and surprise.

Data source selection

The following hypothesis was formulated about the selection of the DBpedia chapter used to process the query.

• Hypothesis 5: Significant results variations exist when using different DBpedia chapters for the same query.

In others words, the results reflect the knowledge variation present in the DBpedia chapters.

It is especially hard to evaluate the result relevance according to cultural criteria as it is profoundly subjective. Thus we decided to evaluate quantitatively the difference between the results lists obtained from different DBpedia local chapters. We filtered the whole list of distinct queries entered in Discovery Hub (2302) 25 to keep only the entities that were described in the 5 biggest DBpedia chapters: the English, French, German, Italian and Spanish ones. The amount of query-entities that was described in all this 5 chapters was 739 (32%) 26 . Then we processed these queries with the SPARQL endpoints, which expose the localized DBpedia versions. We compared the French, German, Italian and Spanish results with the English chapter ones. We chose to compare them with the English chapter results because the vast majority of existing applications uses it and only it today. The results are shown on Figure 6. It is interesting to notice that the top 100 shared results are relatively low and that a consequent proportion of results does not exist in the English DBpedia chapter. Thus the hypothesis 5 is verified. The average execution time on each chapter was few seconds (maximum 5 seconds for the English chapter and minimum 3 seconds on average for the Spanish one). It proves that the framework is adapted in the distributed context of the LOD where many knowledge sources emerge. Our configuration for tests was:

• Application server: 8 processors Intel Xeon CPU E5540 @2.53GHz 48 Go RAM

• SPARQL endpoint: 2 cores Intel Xeon CPU X7550 @2.00GHz 16Go RAM 25 http://discoveryhub.co/querylog-DH.txt 26 http://discoveryhub.co/querylog-DH-multilingual.txt

CONCLUSION AND PERSPECTIVES

This paper was motivated by the idea that the linked data richness allows to explore topics of interest through several perspectives. The contributions of this paper are the formalization, implementation and evaluation of such search approach over a linked data source: DBpedia. Through the interactions with such perspectives the users unveil knowledge nuances about the topic explored. This multi-perspectives exploratory search functionality over DBpedia was achieved thanks to a framework that computes the results at querytime. It allows the users to have the hand on important computation parameters that change the processing and consequently the results list according to specific aspects of interest. At the time of writing 3 main operations are available within the framework: the specification of criteria of interest, the randomness injection and the choice of the data source.

During our analysis we observed that the interest criteria specification with DBpedia categories leads to more interesting results according to the users. Concerning the randomness injection we observed that the level of randomness is correlated with the level of surprise. We also observed that the gain of surprise is at the cost of a consequent loss of relevance. The execution of a set of queries on the English, French, German, Italian and Spanish SPARQL endpoints and their comparison showed considerable differences between the results lists. It is also noticeable that an important part of the non-English results does not exist in the English DBpedia chapter. It is not a surprise as it is known that the knowledge captured by the DBpedia chapters is different but it is the first time that such variations in search results were quantified. Thus it is important to pursue the efforts of DBpedia's internationalization both in quantity and quality to avoid an over-use or misuse of the DBpedia English chapter.

The randomness injection and the choice of the knowledge source will be integrated in Discovery Hub as components of the advanced search functionality. The current implementation already integrates the criteria of interest specification. We will propose a randomness value between 0 and 0.5 in order to get a better tradeoff between relevance and surprise than the one we obtained during our experimentation.

Several linked data based functionalities are relevant to enhance the concept of multi-perspectives exploratory search. A promising one for Discovery Hub is to allow the users to tune the criteria of interest when they examine the results in order to re-rank them without launching another query. The classification of the results, leveraged by faceted browsing, could also be improved by integrating the users' specified criteria of interest. The introduction of such functionalities will also require new evaluations of the users' interface.

To conclude an evaluation of the whole application and the interactions it proposes is necessary and planned.

Figure 1 :

 1 Figure 1: Discovery Hub general architecture

Figure 2 :

 2 Figure 2: The Discovery Hub exploratory search engine interface

Figure 3 :

 3 Figure 3: Kendall-Tau rank correlation coefficient between results with and without categories consideration and golden truth results list

Figure 4 :

 4 Figure 4: Criteria of interest specification through the Discovery Hub interface

Definition 3 :( 1

 31 semantic spreading activation, controlled randomness variant a rand (i, n, o, r)r) * a(i, n, o) + r * random() if r > 0.5 | a(i, n, o) if n < maxP ulse (1r) * a(i, n, o) + r * random() else | else

Figure 5 :

 5 Figure 5: Interest, surprise and perceived distance of results according to 4 algorithm configurations

Figure 6 :

 6 Figure 6: Percentage of shared results with top 100 English chapter results and percentage of top 100 results that are specific to the chapter

Table 1 :

 1

			Summarization of closest systems
	Application	Aemoo	LED	MORE	Seevl	Yovisto
	Purpose	ESS	ESS	Recommender	Recommender	Video exploration
	Data	DBpedia subset	DBpedia subset	DBpedia subset	DBpedia subset DBpedias subset
	Data choice	No	No	No	No	No
	Algorithm	EKP-based view DBpedia Ranker sVSM	DBrec	Heuristics
	Computation	Offline	Offline	Offline	Offline	Offline
	Perspect.-setting	Yes	No	Yes	No	No
	Perspect. number 2	1	n	1	1
	Perspectives types Core, curiosity	/	Property weighting /	/

initially associated to one or several nodes representing the user's interest

Table 2 :

 2 Results of three queries about Claude Monet using the criteria specification Query

		Claude Monet (1)	Claude Monet (2)	Claude Monet (3)
	Criteria	None	Impressionist painters +	Impressionist painters -
			Artists from Paris -	Artists from Paris +
			People from Le Havre -	People from Le Havre +
			Alumni of the École des Beaux-Arts -Alumni of the École des Beaux-Arts +
			French painters -	French painters +
	Results
	1	Pierre-Auguste Renoir	Theodore Robinson	Pierre-Auguste Renoir
	2	Alfred Sisley	Édouard Manet	Gustave Courbet
	3	Édouard Manet	Alfred Sisley	Edgar Degas
	4	Mary Cassatt	W ladys law Podkowiński	Jacques-Louis David
	5	Camille Pissarro	Leslie Hunter	Jean-Baptiste-Camille Corot
	6	Edgar Degas	Theodore Earl Butler	Jean-François Millet
	7	Charles Angrand	Lilla Cabot Perry	Paul Cézanne
	8	Gustave Courbet	Frank Weston Benson	Marc Chagall
	9	Berthe Morisot	Childe Hassam	Camille Pissarro
	10	J.-Baptiste-Camille Corot	Edward Willis Redfield	Édouard Manet
	• The Kunsthalle Bremen, Alte Nationalgalerie, Museum
	Folkwang, Wallraf-Richartz-Museum and the Fonda-
	tion Corboud, situated in Germany as well as the Ger-
	man artist Max Liebermann appear only in the Ger-
	man chapter results.

• The Galleria nazionale d'arte moderna e contemporanea, situated in Italy, appears only in the Italian chapter results.

https://sites.google.com/site/hcirworkshop/

http://www.wikipedia.org/

http://www.bing.com/blogs/site blogs/b/search/archive/ 2013/03/21/satorii.aspx

google.com/insidesearch/features/search/knowledge.html

http://www.apple.com/ios/siri/

https://www.facebook.com/about/graphsearch

http://sisinflab.poliba.it/led/

http://seevl.fm

http://www.yovisto.com/

http://wit.istc.cnr.it/aemoo/

https://apps.facebook.com/new-more

http://dbpedia.org/Internationalization

http://www.w3.org/wiki/SparqlEndpoints

wiki.dbpedia.org/Internationalization/Chapters?v=190k

dbpedia.org/Datasets39/CrossLanguageOverlapStatistics


Nicolas Marie

Fabien Gandon

Survey of linked data based exploration systems

Linked datasets now constitute a valuable background knowledge for supporting exploration and discovery objectives through browsers, recommenders and exploratory search systems in particular. Today there is a need to look at the achievements and tendencies in this rapidly developing field in order to better orient the future research works. In this paper we propose a survey of such systems from the earliest semantic browsers to more recent and innovative ones.

1 Introduction [START_REF] Marchionini | Exploratory search: from finding to understanding[END_REF] stressed the need for efficient exploratory search systems (ESS) i.e. systems optimized for investigation and learning tasks. Linked data [START_REF] Bizer | Linked data-the story so far[END_REF] and DBpedia [START_REF] Lehmann | Dbpedia-a large-scale, multilingual knowledge base extracted from wikipedia[END_REF] respectively correspond to an approach for publishing and linking data on the web and its application to data extracted from Wikipedia 3 . The semantic search community [START_REF] Tran | Towards exploratory video search using linked data[END_REF] enlightened the possibilities brought by semantics incorporation in search systems, their capability to support advanced search experiences and to solve complex information needs. Today major initiatives in the field of semantic search emerge from important web players 4 .

The amount and heterogeneity of the contributions in the field of linked data based exploration is growing. There is a need to look at the achievements and tendencies in order to orient the future research works. We propose the following diagram in order to structure the reflection, see Figure 1. The exploratory search tasks characteristics on the left are summarized from [START_REF] White | Exploratory search: Beyond the query-response paradigm[END_REF]. We derived from these characteristics a list of desired effects of the systems and linked them to the widespread features implemented in the existing applications.

To the best of our knowledge no previous survey specifically dedicated to linked data based exploration and discovery systems exist. We review [START_REF] Becker | Dbpedia mobile: A location-enabled linked data browser[END_REF] the linked data browsers, (2) the linked-data based recommenders, (3) the linked data based exploratory search systems.

Linked data browsers

The first generation of tools designed to explore linked data was semantic browsers. At their beginning semantic browsers gave a very simple view of the data in graph form or were strongly inspired by the web pages browsing. More recently innovative browsing paradigm were proposed. In this section we review textual, visual, faceted and more singular browsers.

Text-based browsers

A plethora of semantic text-based browsers exists. We propose a selection of some of the most cited in the literature to give an idea of the functionalities this class of browsers proposes. A more complete survey is available in [START_REF] Dadzie | Hide the stack: toward usable linked data[END_REF]. Noadster [START_REF] Rutledge | Making rdf presentable: integrated global and local semantic web browsing[END_REF] is an early (2005) and generic RDF browser that performs a property-based clustering on the data. The properties-based clusters are used to structure the results: frequent properties appear higher in the results-tree. Disco5 (2007) is a simple server-side browser that render the RDF data in columns of propertyvalue pairs associated to the currently browsed resource (object) in a table. Marbles6 (2007) formats the RDF triples using the Fresnel vocabulary7 . Marbles retrieves additional data about the browsed resources from semantic index, search and review systems. Colored bubbles (the "marbles") help the users to identify the sources of provenance of the data retrieved.

Visualization-based browsers

Visualization-based browsers lower the users' cognitive load and help them overcoming the data complexity by relying on their natural visual ability. For a complete survey the reader may refer to [START_REF] Dadzie | Approaches to visualising linked data: A survey[END_REF]. RDF Gravity 8 (2004) is a RDF and OWL visualization tool [START_REF] Goyal | Rdf gravity (rdf graph visualization tool[END_REF]. It offers filtering, search functionalities and allows the users to reposition the nodes for better visualization. The resources are displayed in different colors according to their RDF types for a more comprehensive layout. Tabulator 9 (2005) [START_REF] Berners-Lee | Tabulator: Exploring and analyzing linked data on the semantic web[END_REF] is a generic RDF browser that displays the data through a tree-view. By clicking the hierarchically displayed trees the users browse through increasing levels of details. DBpedia Mobile 10 (2008) is a location-aware mobile web application [START_REF] Becker | Dbpedia mobile: A location-enabled linked data browser[END_REF]. It displays the geolocated DBpedia resources surrounding the users (or elsewhere) on a map by using geo:lat and geo:long properties. Filtering and context-aware strategies are used to minimize the amount of displayed triples. LENA 11 (2008) offers a mechanism to build Fresnel-based visualization templates using SPARQL queries [START_REF] Koch | Lena-browsing rdf data more complex than foaf[END_REF]. The objective is to propose specific views that correspond to users in respect to their interests and expertise.

Faceted browsers

A faceted search system "presents users with key-value metadata that is used for query refinement" [START_REF] Koren | Personalized interactive faceted search[END_REF]. Each time a constraint (a facet value) is applied the results are filtered, the available facets and facets' values are updated. Longwell 12 (2005) is a web application for browsing and searching large RDF datasets. Longwell offers a faceted filtering mechanism. The facets and their values are heuristically derived from the dataset. These heuristics can be pre-configured by the users. MuseumFinland 13 (2005) is a faceted semantic browser that was implemented on the top of a Finnish art-collections RDF knowledge base [START_REF] Hyvönen | Museumfinlandfinnish museums on the semantic web[END_REF]. Preview counts showing the amount of results per facets help the users in their manipulation. mSpace (2005) uses the facets dependencies as a support for browsing [START_REF] Wilson | Mspace mobile: A ui gestalt to support on-the-go info-interaction[END_REF]. The facets are organized horizontally e.g. era, composer, place in the domain of classical music. The order is not neutral and constitutes a hierarchy where the left-most column is the top level. The value(s) applied in each facet constrain(s) the displayed values of the facets on the right. The users can change the orders of facets, swap, delete and add new ones. /facet (2006) automatically set up faceted browsing over a heterogeneous linked dataset where manual configurations are too fastidious or unfeasible [START_REF] Hildebrand | /facet: A browser for heterogeneous semantic web repositories[END_REF]. The facets and their hierarchical dependencies are generated with the help of the ontological knowledge e.g. the RDFS classes and their associated properties. Such facets generation has the advantage to be applicable on every linked database. Humboldt (2008) is a faceted browser that helps the users to explore a local RDF graph [START_REF] Kobilarov | Humboldt: Exploring linked data[END_REF]. Contrary to the majority of semantic web faceted browsers Humboldt proposes facets corresponding to classes, for example Person appears instead of directed, stars in, was awarded for, etc.. Proposing class-based facets simplifies the interface by decreasing their amount. More details about the relations are given on demand.

Others browsing paradigms

The following browsers propose innovative and singular interaction paradigms implemented over linked data sources. Parallax (2009) aims to overtake the classic one resource at a time interaction paradigm [START_REF] Huynh | Parallax and companion: Set-based browsing for the data web[END_REF]. The authors of Parallax introduce the "set-based browsing" paradigm that allows to browse several links at the same time: from a set of entities to another set of entities e.g. all the US presidents to their children. Along the exploration the users can apply filters on the currently displayed data. A browsing history helps the users to edit their previous interactions. RelFinder 14 (2009) is a web application that displays the paths existing between two resources of interest in a graph form [START_REF] Heim | Relfinder: Revealing relationships in rdf knowledge bases[END_REF]. The path identification remains at the instances level and do not traverse rdf:type properties. Identifying such paths can be very cognitive and time consuming by manual browsing. gFacet (2010) is a prototype combining graph-based visualization and faceted filtering techniques [START_REF] Heim | Facet graphs: Complex semantic querying made easy[END_REF]. The facets are represented in the form of nodes in a graph, the arcs represent their dependencies (e.g. birthPlace links the facets Person and Place). When the users select a filtering value in a facet it recomputes the values in the others facets as well as in the results set. Visor (2011) is a semantic web browser that implements the "multi-pivoting browsing" paradigm [START_REF] Popov | Connecting the dots: a multi-pivot approach to data exploration[END_REF]. Visor allows multi-directional exploration by following several properties at a time. The users first select one or several classes thanks to a keyword-search functionality. The selected classes and the properties which connect them are displayed in a graph form. The majority of interactions occurs at the schema level (e.g. adding, removing classes). The instances are only shown on demand.

Recommenders

Several research works have shown the great potential of linked data and especially DBpedia to compute semantic similarities. Such similarity measures are mainly used by recommenders. At the beginning domain-constrained approaches were researched (first subsection). More recently more complex recommendations were investigated (second subsection). The domain-specific approaches target a single sub-ensemble of a dataset for which they are optimized e.g. DBpedia actors, directors and movies and related properties. They can be constrained to a single type e.g. movies recommendations starting from a movie. Recently Bing, Google and Yahoo deployed their own entities' recommendation approach. If an entity is recognized in the users' query these search engines display structured data about it in a knowledge panel. The Google (2012) and Bing (2013) methods are unpublished. Yahoo partially revealed the functioning of its SPARK recommender (2013) which is based on heterogeneous features extraction (e.g. co-ocurrence, type) and a stochastic gradient boosted decision tree, [START_REF] Blanco | Entity recommendations in web search[END_REF].

Domain-specific recommenders

The Linked Data Semantic Distance (LDSD, 2010) is based on direct and indirect paths counting between a pair of resources [START_REF] Passant | Dbrecmusic recommendations using dbpedia[END_REF]. It was applied on the music domain for computing similarities between all the Bands and Musical Artists instances contained in DBpedia. The LDSD results were successfully evaluated against the LastFM15 recommendations. The DBpedia Ranker (2010) proposes a similarity measure which is a combination of several weights including external services (search engine correlations measure with Google, Yahoo, Bing and Delicous 16 ), one link analysis measure based on the existence of a relation between the resources (no links, a link in one direction, two links in both directions) as well as a literal (label) based analysis. It was implemented over DBpedia resources belonging to the ICT domain. MORE17 (2010) is a DBpedia-based movie recommender [START_REF] Mirizzi | Exploratory search and recommender systems in the semantic web[END_REF]. It is based on a semantic adaptation of the vector space model. The more features two resources share the more similar they are. They can be linked through direct properties (e.g. "subsequentWork"), be the subject of triples having the same property and object ("starring" "Robert de Niro") or be the objects of two RDF triples having the same property and subject. The users can ask explanations about the recommendations by requesting the shared properties between the queried movie and the results.

Cross-domains recommenders

The recommenders previously described are focused on a specific domain and/or limited to a specific type of items. This constraint is motivated by performance and recommendations quality concerns. This can be considered as a severe limitation in the linked data context. Indeed the richness of the linked data datasets offers an unprecedented ground for computing complex similarities metrics. Fernandez and al. (2011) proposed a framework to compute recommendations in a defined domain starting from instances that belong to another domain e.g. music starting from places [START_REF] Kaminskas | Ontology-based identification of music for places[END_REF]. The instances belonging to the 2 domains are linked in an acyclic graph. This acyclic graph is composed of a set of classes and properties that are selected by an expert. Finally a weighted traversal algorithm is performed over this graph to generate the recommendations. The evaluations showed very encouraging results and confirmed the potential of DBpedia for cross-domains/types recommendations. [START_REF] Stankovic | Linked-data based suggestion of relevant topics[END_REF] proposed the hyProximity algorithm (2011), designed to support industrial problem solving. A valuable approach in open innovation is to identify topics that are lateral to a given problem in order to find innovative solution coming from another domain. The goal of the hyProximity algorithm is to identify hidden and unexpected associations between industrial topics. A set of DBpedia entities is first extracted from the targeted industrial problem description. Then the hyProximity algorithm traverses the DBpedia graph and finds laterals topics (and consequently associated experts) that might propose an innovative solution.

4 Linked data based exploratory search systems Exploratory search systems are the most advanced systems in the field of linked data based exploration. They all propose a variety of functionalities dedicated to exploration purposes. There are two main currents in the literature in terms of processing and interaction model. The first one is to allow the users to produce views on the graph through rich operators (first subsection). The second approach consists in applying algorithms that leverage the semantics in order to select and rank a small amount of data that are presented to the users (second subsection). In the second case the graph is not displayed as it is, computed meaningful relations (similarity, relatedness) are shown instead.

View-based exploratory search systems

Aemoo 18 (2012) is an exploratory search system based on Encyclopedic Knowledge Pattern19 (EKP) [START_REF] Musetti | Aemoo: Exploratory search based on knowledge patterns over the semantic web[END_REF]. EKP are knowledge patterns that define the typical classes used to describe entities of a certain class. For instance "airport", "aircraft", "military conflict" or "weapon" are part of the "aircraft" EKP. They were built by using a DBpedia graph analysis. Aemoo presents the resource explored direct neighborhood filtered with its class' EKP in the form of a graph. It also proposes a "curiosity" function which displays the resource of interest neighborhood through an inverted EKP filtering. This function aims to reveal surprising knowledge. Aemoo also provides explanations by showing the cross-references in Wikipedia pages between the topic of interest and the results. Linked Jazz20 (2013) aims to reveal the network of the social and professional relations within the American jazz community [START_REF] Pattuelli | Crafting linked open data for cultural heritage: Mapping and curation tools for the linked jazz project[END_REF]. The objective is to capture the relations semantics in RDF from jazz people interviews' transcripts. The method used to reach this goal is a combination of automated processing and crowd-sourced curation. The dataset obtained can be explored through a graph visualization web application. It gives an overview of the jazz community network and supports more focused explorations e.g. individual connections, shared connections, dynamic network creation by manual selections.

Algorithm-based exploratory search systems

Yovisto21 (2009) is an academic videos search engine that provides a linked data based exploratory search feature [29]. The exploratory search feature retrieves topics suggestions that are semantically related to the users' query. When the user enters a query Yovisto retrieves a set of semantically interrelated suggestions. These suggestions are computed over DBpedia by using a property ranking process based on a set of heuristics. Semantic Wonder Cloud (SWOC, 2010) 22is based on the DBpedia Ranker [START_REF] Mirizzi | Exploratory search and recommender systems in the semantic web[END_REF]. First the users select a starting topic by using the DBpedia lookup 23 (rapid resource selection by typing). Then SWOC surrounds it by its 10 most similar concepts in a graph form. SWOC exposes associations that does not exist in the original dataset. The second system based on the DBpedia Ranker is Lookup Explore Discover (LED, 2010)24 [START_REF] Mirizzi | Exploratory search and recommender systems in the semantic web[END_REF]. The users first enter a topic of interest using a DBpedia lookup. Then LED suggests semantically similar query-resources that are presented in the form of a tags cloud. Thanks to them the users can refine the query and easily explore an unknown domain. Once the query is composed, LED retrieves an aggregation of results coming from several major search engines (Google, Yahoo, Bing) as well as a news feed and a microblogging service. The Seevl25 musical discovery platform (2012) uses the LDSD measure to retrieve artist recommendations. The users can also browse through the properties-values associated to an artist in order to explore and discover other artists e.g. its musical genres, collaborations. Shared-properties based explanations are available. Discovery Hub26 (2013) implements a linked data exploration framework based on a semantic-sensitive traversal algorithm coupled to a live graph sampling technique [START_REF] Marie | Discovery hub: on-the-fly linked data exploratory search[END_REF]. Discovery Hub offers faceted browsing and multiple results explanations features. Several advanced query modes are supported by the framework including the specification of facets of interest and disinterest about the topic explored and the injection of randomness during the data processing to retrieve more surprising results. in-Walk27 (2014) is a web application for high level linked data exploration [START_REF] Castano | inwalk: Interactive and thematic walks inside the web of data[END_REF]. It allows the users to interact with a graph of clusters named inCloud. An inCloud is a set of linked data based clusters, computed according to a manually declared list of properties. The clusters are related each others by a semantic proximity link. The current inWalk web application is implemented on a Freebase subset related to athletes and celebrities. InWalk aims to give an high-level overview on a domain and to ease its appropriation by the users.

Discussion

Evolution over time

The Figure 2 shows the evolution of the research in terms of linked data based exploration systems. During the first development phase of the semantic web (2001 -2007) several types of browsing paradigms were investigated. Text-based browsers inspired by the classic web browsing experience appeared. In their simplest form they allowed to navigate one resource-at-atime using resources outgoing relations. During this period visual (graph) and faceted browsers were also investigated. The small size and the relative homogeneity of the available datasets were favorable to such approaches. In 2007 the Linked Open Data initiative renewed the research. The quality, size and coverage of generic datasets like DBpedia and Freebase opened the door to innovative processing and interactions models. New browsing paradigms like set-based, multi-pivoting and hierarchical faceting ones were investigated. An important shift in the research started with new forms of linked data based algorithms. The objective was to compute and expose computed links instead of showing the graph as it is. Such algorithms leverage the semantics to select and rank a small amount of data that are then explored by the users. In a first time the computation of linked data based recommendations brought very encouraging results. The similarity computation was domain-constrained at the beginning (MORE, LDSD, DBpedia Ranker). Then cross-domain (Fernandez and al.) and lateral (hyProximity) approaches were researched. Some of these recommenders constituted the basis of linked data based exploratory search search systems. The DBpedia Ranker was notably integrated in SWOC and LED. Later the LDSD measure was implemented for computing the Seevl recommendations. Several from-scratch exploratory search system, associated data processing and interactions models were developed. Recently (2012 -2013) the 3 major search engines deployed their entity-recommendation solutions, confirming the potential of such approaches for mainstream services.

Achievements and opportunities

The table 3 gives an overview of the most advanced systems for exploration and discovery on linked data i.e. innovative semantic browsers, recommender systems and exploratory search systems. As the human factor is crucial for exploration tasks we only considered the systems having a user interface in this table. We comment the table below, reusing the desired effects of the systems listing proposed in the introduction.

The system provides efficient overviews: The majority of systems do not implement specific over-viewing features. Some layout like the Discovery Hub (pictures board) or Linked jazz ones (whole graph visualization) are helpful but not sufficient. Several applications are specifically built to favor the understanding of the data space by adopting a schema first, instances second design (Visor, InWalk). However these approaches might be confusing for the lay users. In order to mitigating this problem we see an opportunity in exposing both schema and instances information at the same time in order to obtain a good tradeoff between information richness and intuitiveness.

The system helps the user to understand the information space and to shape his mental model: faceted interfaces, used by 5 systems out of 14, constitute a powerful approach for understanding the domain explored as they expose explicitly its important dimensions. At the same time 6 out of 14 systems propose results explanations features which are crucial components to rise the information space understanding. There is an opportunity in investigating new forms of faceted interfaces and explanations e.g. personalized ones which put the information space in perspectives with resources already known by the users.

The user explores multiple, heterogeneous results and browsing paths: concerning the results heterogeneity several systems have shown that it is possible to propose linked data exploratory search systems without targeting a specific type or domain (e.g. Aemoo, Discovery Hub). It is noticeable that 7 out of 14 systems propose in-session memory feature (also known as breadcrumbs). Such features allow the users to easily come back to a previous state. They considerably lower the perceived cost of browsing and consequently encourage the exploration. We see an opportunity in setting up rich interfaces to reduce even more the perceived cost of browsing.

The system eases the memorization of relevant results: in-session memory features avoid the users to memorize their sequences of browsing, freeing their cognitive resources that can be used for more valuable exploratory sub-tasks e.g. results analysis and comparison. There is a need to develop account-related memory features that can support knowledge acquisition for long-lasting interest. Collaborative approaches (gathering several points of view) and advanced knowledge organization (e.g. mind maps) are relevant axes of development for such functionalities.

The system inspires the user and shapes his information need: there are only two systems which propose query suggestions: LED and Yovisto. To overtake the actual limitation there is a need to propose a query model that is more flexible, expressive and that can open the door to finer assistance in the composition e.g. declaring that some characteristics of an input-resource are important. Today the query-unit remains at resource level. There is a need to propose alternative to this monolithic approach for more precise query expressiveness and corresponding assistance.

The system provokes discoveries: by nature all the tools listed in this review provoke discoveries at some point. In this ensemble, two systems go one step further by implementing dedicated strategies to expose unexpected knowledge. Aemoo proposes a curiosity functionality inverting its habitual view on data. The framework implemented by Discovery Hub can inject randomness in its processing to retrieve unusual results. We see an opportunity in personalized discovery approaches using profiles. It seems relevant to adapt the level of evidence/surprise of the results according to the users' prior state of knowledge.

Conclusion

In this paper we reviewed the systems designed for linked data exploration. The earliest approaches were inspired by the classic web browsing experience. Visual and faceted browsers over datasets having a limited size were also common. The LOD initiative and the appearance of DBpedia in particular offered new possibilities and challenges in terms of data processing and interactions for linked data based exploration and discovery. Innovative approaches were researched from recommenders to exploratory search systems. Last but not least the users are increasingly familiar with structured data in search through the major search engines. In our opinion the massive use of linked data based exploratory search functionalities and systems will constitute a decisive improvement for the future of web search experience and its outcomes.

Fig. 1 .

 1 Fig. 1. Relation between exploratory search tasks characteristics and widespread systems features

Fig. 2 .

 2 Fig. 2. Systems timeline

Fig. 3 .

 3 Fig. 3. Summary of the systems according to their main information retrieval and human-computer interaction aspects.

http://www.wikipedia.org/

e.g. http://www.bing.com/blogs/site blogs/b/search/archive/2013/03/21/satorii.aspx

http://wifo5-03.informatik.uni-mannheim.de/bizer/ng4j/disco/

http://wiki.dbpedia.org/Marbles

http://www.w3.org/2005/04/fresnel-info/

http://www.visualdataweb.org/relfinder.php

www.lastfm.com

https://delicious.com/

http://apps.facebook.com/new-more/

http://wit.istc.cnr.it/aemoo

http://www.ontologydesignpatterns.org/ekp/

http://linkedjazz.org/

http://www.yovisto.com/

http://sisinflab.poliba.it/semantic-wonder-cloud/index/

http://wiki.dbpedia.org/Lookup

http://sisinflab.poliba.it/led/

http://play.seevl.fm/

http://discoveryhub.co

http://islab.di.unimi.it/inwalk