Keywords: separation of concerns, multidisciplinary development, modeling, domain specific languages

Introduction

A thought provoking article published by Nature [START_REF] Merali | Computational science[END_REF] makes a critical analysis of the way software is developed and used in the context of scientific research and gives some examples of situations where this affected the quality of the scientific research results by leading to wrong conclusions. As computers and programming environments have grown more complex, the development of software by scientists with practically no training in computer programming, according to ad-hoc or self-taught methods, incurs at best a waste of time and energy. The previously cited analysis does not hesitate to use unflattering adjectives such as "awful programming", "mangled coding", that leads to "monster code".

As the research teams and topics are covering larger research areas, the interactions between various scientific disciplines increase and are often materialized by the computational science, as highlighted by this year's ICCS theme. While this is a perfectly legitimate aim and it corresponds to a real trend encountered in practice, we are concerned about the potential to worsen the quality of software currently used in scientific research.

The aim of the current paper is firstly to warn against the fact that increased multidisciplinarity poses a new risk on the quality of the produced software. Moreover, we see the current Abstract This year's ICCS conference theme promotes the use of computational science as a means to foster multidisciplinarity and synergies with other fields. Our thesis is that this trend towards multidisciplinarity should be accompanied by the use of best practices issued from the software engineering community in order to avoid obtaining overly complex and tangled code, difficult to validate, to maintain and to port. In this paper we argue for the need of applying separation of concerns principles when the development involves scientists from various application fields. We overview several strategies that may be used to achieve this separation, focusing mainly on two approaches drawn from our previous experiences with multidisciplinary projects, addressing two distinct patterns of multi-domain interaction that may occur in scientific software development.

On Patterns of Multi-domain Interaction for Scientific Software Development focused on Separation of Concerns

Ileana Ober and Iulian Ober

IRIT -University of Toulouse, France {Ileana.Ober, Iulian.Ober}@irit.fr trend towards multidisciplinarity as a huge opportunity to correct some of the deviations seen in the scientific code crafting and introduce rigorous software engineering best practices. This shift is both needed and feasible. Scientific research is not the only field where several domain experts need to work together and software plays the role of intermediary, as we will show through examples in the paper.

The issue on which we focus in this paper is separation of concerns, which is of paramount importance in a multidisciplinary computational science project. After stressing out its importance, we overview several strategies that may be used to achieve this separation. We focus mainly on two approaches drawn from our previous experiences with multidisciplinary projects, addressing two distinct patterns of multi-domain interaction.

The rest of this paper is structured as follows: in Section 2 gives a brief historical perspective on the notion of separation of concerns in software development and its importance. In Section 3 we describe two strategies for introducing and managing separation of concerns, issued from our previous experience, before concluding the paper.

Separation of concerns in software engineering

It is widely recognized that the growing complexity of scientific software is a major engineering challenge (see for example [6,[START_REF] Joppa | Troubling trends in scientific software use[END_REF]), which can only be magnified in a multidisciplinary context. The path towards a solution for this problem necessarily involves engineering techniques that have proved efficient in taming the complexity in other application fields. Software complexity has been an subject of study in search for solutions for a long time, and separation of concerns was advocated very early on, as the following excerpt from E.W. Dijkstra's 1976 A Discipline of Programming [START_REF] Edsger | A discipline of programming[END_REF] reminds us: "The purpose [...] is to reduce the detailed reasoning needed to a doable amount, and a separation of concerns is the way we hope to achieve this reduction. The crucial choice is, of course, what aspects to study in isolation".

Yet the finding of the right separation lines that allow to cut through the complexity of a large software system remains a difficult task. The years since Dijkstra's remark have brought a profusion of possible solutions in the form of concepts, language constructs and methodologies for structuring software and its development. Modularity percepts [START_REF] Parnas | On the criteria to be used in decomposing systems into modules[END_REF] and standard paradigms such as object-oriented design [START_REF] Meyer | Object-Oriented Software Construction[END_REF] go a long way towards this goal. However, the design of complex systems having multiple stakeholders and integrating multiple heterogeneous components often requires an intent focus on separation of concerns and more specific weapons geared towards it. This has given rise to approaches such as subject-oriented design and multidimensional separation of concerns [START_REF] Tarr | N degrees of separation: multi-dimensional separation of concerns[END_REF], aspect-oriented programming and design [START_REF] Kiczales | Aspect-oriented programming[END_REF][START_REF] Jacobson | Aspect-Oriented Software Development with Use Cases[END_REF], role-based design [START_REF] Bruun | Object-oriented modelling with roles[END_REF], to name only a few.

The existence of these approaches proves that simply relying on modularity and component based software design is not always enough to tackle the complexity of nowadays systems. If long-term sustainability of a software development project is aimed, the separation and tracking of concerns is of the essence, as it has been advocated in [START_REF] Robillard | Representing concerns in source code[END_REF].

Another perspective on the problem of multidisciplinary development and on separation of concerns is provided by systems engineering, and in particular the recently developing field of Model-Based Systems Engineering (MBSE) [START_REF]INCOSE. Systems Engineering Handbook -A Guide for System Life Cycle Processes and Activities, version 3.2[END_REF]. This approach has emerged in the context of the development of complex cyber-physical systems (CPS) in automotive, aerospace or industrial supervisory control applications. As noted in [START_REF] Derler | Modeling cyber-physical systems[END_REF], the development of these systems is by nature multidisciplinary and the effective means to manage its complexity nowadays is through the use of models. However, since CPS integrate computation, networking and physical dynamics, modeling techniques have to deal with this heterogeneity by allowing the different aspects to be modeled separately, while providing a semantic basis for these models to inter-operate. We arrive thus at multi-modeling approaches [START_REF] Brooks | Model engineering using multimodeling[END_REF] and at modeling languages such as SysML [START_REF] Omg | Systems Modeling Language (SysML)[END_REF] that support multiple integrated views of the different aspects of a system. Model-driven engineering, a paradigm that has matured over the last twenty years [START_REF] Brambilla | Model-Driven Software Engineering in Practice[END_REF], provides the technical support allowing to create, analyze, transform and integrate these multi-models.

3 Strategies for introducing separation of concerns.

In this section we discuss two strategies for managing and fostering separation of concerns, inspired from our previous work on large multidisciplinary projects. Both approaches, which may be qualified as vertical and horizontal, leverage the idea of domain-specific models addressing the concerns of experts in a particular discipline. The vertical approach is organized into abstraction layers, each layer corresponding to a particular expertise and adding detail to the layers above. The horizontal approach, useful when several expert teams work on different models at the same level of abstraction in related yet domain-specific languages, aims at the mechanized integration of the resulting models.

A layered approach to model-based HPC application development In previous work [START_REF] Ober | On the use of models for highperformance scientific computing applications: an experience report[END_REF], we were involved in a four-year project that aimed to address the complexity resulting from the frequent evolutions of supercomputers and the tight coupling between software and hardware aspects, in the context of the development of scientific applications relying on highperformance computing. The technique used in this approach relied on the use of models.

Our approach, called MDE4HPC, follows a classical model based strategy: model all artifacts -including those that are non-software dependent -produced by the range of experts who typically take part in the development (physicists, applied mathematicians, computer scientists, hardware and software architects) and combine them to generate the application code. To that end, MDE4HPC offers a multi-layered architecture where each abstraction level corresponds to specific profiles of experts. Each expert will handle his dedicated model in a hardwareindependent way, at the right level of abstraction, in a user-friendly modeling environment. The different concerns are separated thanks to the use of several models, as well as several modeling viewpoints on these models.

MDE4HPC uses: models (one type per domain) and HP CM L (High Performal Computing Modeling Language) which is defined on three layers: one dedicated to physics, one dedicated to the numerical model and another one dedicated to execution related matters (software and hardware). The layered definition of the HP CM L enforces the separation of concerns, by allowing each domain expert to focus on artifacts relevant to its work. Obviously, the resulting layers are not disjoint. The parts that are common to various domains help with ensuring the consistency of the transformations required when going back-and-forth between two domains. At the surface syntax level, the domain-specific language constraints proposed to various domain experts may even vary (in practice this was seldom the case, yet it occurred), however as the underlying metamodel (for the concerned parts) is the same, the resulted transformation is accurate.

Our approach for the development of scientific applications permits a selective view of the system and the separation of concerns, through the layered architecture of MDE4HPC. It is obviously not the only technique permitting a clean separation of concerns, but it proved quite effective.

A constructive approach for interoperability In the context of computational science applications that involve scientists from several domains, the software development may use different programming languages, either due to existence of programming languages tailored for the needs of a specific field (e.g. Tonto [START_REF] Jayatilaka | Tonto: A fortran based object-oriented system for quantum chemistry and crystallography[END_REF] -the Fortran based language for computational chemistry), or due to programming language preferences issued from background constraints (in terms of legacy code or team training). In order to avoid imposing unnecessary constraints on each of the different teams involved in the development, it is sometimes interesting to allow each of them to do the software development according to its experience and devise an automated approach for the interoperability between the artifacts produced by each of them.

In previous work [START_REF] Ober | Dealing with variability within a family of domain-specific languages: comparative analysis of different techniques[END_REF] we have addressed the problem of allowing domain experts to cooperate using programming languages that are closely related, yet not identical. Here, we discuss these results as an example of constructive approach for interoperability of related languages. The context of our work was similar to what is encountered in certain computational science software development projects in a multidisciplinary environment. For a more detailed presentation of the approach presented in this section, the reader is referred to [START_REF] Ober | Dealing with variability within a family of domain-specific languages: comparative analysis of different techniques[END_REF]. The problem addressed by this approach was issued from a case study that we developed with colleagues from the French National Space Agency (CNES) in the context of the DOMINO French National project 1 . The case study revealed the need to deal with a set of software components specified using related -yet different -domain specific languages (DSLs).

One critical issue in this case study is how to handle the differences existing between languages dedicated to the definition of operational procedures for space system testing and operations. Indeed, different space agencies or equipment builders use different languages to remotely control the satellites: Pluto [START_REF]A. space engineering standard -ground systems and operations -monitoring and control data definition standard[END_REF] for the European Space Agency, Stol 2 by NASA, and Elisa by EADS/Astrium. These languages contain primitives to deal with algorithmic constructs, task scheduling, remote commands and measures in an imperative style. In spite of the need to occasionally use them jointly, several attempts to impose a unique language have failed, for both economical and political reasons. The various languages of the family differ at the syntactic and semantic level; nevertheless their compatibility is needed in order to achieve interoperability between agencies and user organizations and it is necessary for the development of a space mission infrastructure.

Although issued from a particular case study, this is a more general problem, as the need to use several closely related languages in the same development project occurs quite frequently in practice, in particular when the project brings together experts from various domains.

Conclusion and discussions

In this paper we argue for the need to apply proven state of the art software engineering techniques in the context of computational science. The evolution towards multidisciplinarity in scientific software development is bound to create a complexity problem that will provide the incentive to move away from artisanal software crafting towards a more professional approach to code development. Thus the problem could be turned into an opportunity and the use of best practices could significantly increase the quality and the sustainability of software development.

After overviewing the notion of separation of concerns in engineering, the challenges it raise and the issues it may solve, as well as some classical approaches to address it, we overview two strategies to achieve it, corresponding to two patterns of multidisciplinary interaction. The first one, that we qualify as vertical, occurs when scientific software involves challenging computational problems that cannot be addressed by the field scientists alone and need the collaboration of numerical analysts and computer platform specialists (including specialists in hardware, compilers, particular libraries, etc.). The second approach, qualified as horizontal, may occur when multiple scientific fields contribute contribute components written in different yet related languages to a multidisciplinary software project. The problem in this case is the integration of these components and our approach aims at automating the interoperability of these components.

Although the focus of this paper is separation of concerns, other software engineering techniques, such as rigorous requirement engineering, requirement traceability and rigorous validation processes, provide great potential for improving the quality of scientific software.

http://www.domino-rntl.org/


Hong Diep
email: hong-diep.tran@irit.fr

Guillaume Cabanac
email: guillaume.cabanac@irit.fr

Gilles Hubert
email: gilles.hubert@irit.fr

Expert suggestion for conference program committees

Keywords: Expertise, academia, program committee

With the rapid development of science, the academic community requires higher and higher quality of the published articles. This great responsibility is placed on editorial boards of journals, on program committees of conferences and their members. In addition, with a large number of scientific conferences held each year, searching for experts that would be invited to join the program committees is an increasingly hard task. In this paper, we propose an expert modelling method on different indicators of expertise, in particular on citation networks in order to build suggestions of members for program committees of conferences. Contrary to most of the works about expert search in the literature which are based on textual sources of evidence to define the expertise of an expert candidate, our model leverages non-textual indicators only. We evaluate the proximity of an expert candidate according to the weights of his/her various relations with the conference for which the candidate is proposed to be member of the next program committee. We report various experiments in the context of the SIGIR conference editions. These experiments give hints on the composition of program committees and on the effectiveness of our approach.

I. INTRODUCTION

The peer review process in science is the cornerstone supporting the communication of scientific progress [START_REF] Zuckerman | Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system[END_REF]. Manuscripts submitted to a scientific journal are handled by the journal's editorial board composed of gatekeepers who serve as referees [START_REF] Crane | The gatekeepers of science: Some factors affecting the selection of articles for scientific journals[END_REF]. Journal editorial boards are rather stable in time: only a fraction of the board is usually renewed each year. For some scientific fields, however, journals are not the only way to communicate research results. In computer science, for instance, international conferences play a key role in this respect. Paper submissions -usually full papers, not just abstracts as in other fields -are refereed by three or more members of the conference Program Committee (PC). Unlike for journals, PCs are renewed each year upon suggestion of the PC chairs. Constituting an efficient PC is a big challenge, as one needs to identify and invite scholars both active in all the topics addressed by the conference and diverse in many aspects (location, age, gender, and so on). It is an even more daunting task for large conferences such as CIKM, SIGIR, or WWW that attract hundreds of submissions [START_REF] Cabanac | Capitalizing on order effects in the bids of peer-reviewed conferences to secure reviews by expert referees[END_REF]. Various works showed the interest of automating different tasks related to conference organisation [START_REF] Rodriguez | An algorithm to determine peer-reviewers[END_REF], [START_REF] Han | Recommending program committee candidates for academic conferences[END_REF], [START_REF] Sfyris | Profile-based selection of expert groups[END_REF].

The issue tackled in this paper relates to the expertise retrieval question raised some fifteen years ago [START_REF] Balog | Expertise retrieval[END_REF]. Diverse contexts find an interest in expert search, such as determining peers to review submitted manuscripts [START_REF] Rodriguez | An algorithm to determine peer-reviewers[END_REF], suggesting researchers matching scientific interests [START_REF] Cabanac | Accuracy of inter-researcher similarity measures based on topical and social clues[END_REF], finding supervisors for PhD students [START_REF] Liu | Using semantic web technologies to improve expertise matching within academia[END_REF], or recommending helping experts for technical support of medical software [START_REF] Mcdonald | Expertise recommender: A flexible recommendation system and architecture[END_REF]. However, we failed to find much work about suggesting expert candidates to invite as members of a renewed conference PC.

The literature features work about finding experts that could respond to an expertise need [START_REF] Rodriguez | An algorithm to determine peer-reviewers[END_REF], [START_REF] Fang | Probabilistic Models for Expert Finding[END_REF], [START_REF] Afzal | Expertise Recommender System for Scientific Community[END_REF]. Such works take advantage of information retrieval (IR) models by processing documents related to experts to represent experts through textual profiles. Those expert profiles are then matched with the expertise need to find the most similar ones.

Most works build expert profiles from documents related to expert activities in enterprise or academic domains. However, some studies showed an interest in exploiting citations [START_REF] Strohman | Recommending citations for academic papers[END_REF], [START_REF] Liang | Finding relevant papers based on citation relations[END_REF] and their combination with other scientific activities for researchers such as participations to conferences [START_REF] Cabanac | Accuracy of inter-researcher similarity measures based on topical and social clues[END_REF]. Citations are one of the main subject of Sociology of Science [START_REF] Kessler | Bibliographic coupling between scientific papers[END_REF], [START_REF] Small | Co-citation in the scientific literature: A new measure of the relationship between two documents[END_REF], [START_REF] Cronin | The Citation Process: The Role and Significance of Citations in Scientific Communication[END_REF]. However, identifying automatically the exact meaning of a citation remains an open issue. To this day, few approaches proposed to use citation networks to find experts, most of them focusing on searching articles or scientific documents.

This paper introduces a model for suggesting PC candidates for a given conference. It exploits scholars' bibliographies and citation networks to suggest candidates active in the topics of the conference. These suggestions are deemed to inform PC chairs about active albeit perhaps overlooked scholars who would serve the scientific community by joining the conference program committees.

This paper aims at suggesting expert candidates to help renewing the program committee of a given conference. This context differs from those aforementioned by the limited textual content describing a conference. Indeed, the description of a conference is usually limited to a set of topics outlined by some keywords. Searching in a bibliographic database with these keywords would inevitably return a highly "noisy" result. Thus, the approach proposed in this paper is based on non textual elements only.

The contribution of this paper is twofold, by proposing:

1) a modelling through a graph of the elements involved in the context of conferences (such as the conference editions, the articles, and the expert-candidates) and their different types of relations (such as being author of an article, citing an article, and participating to a previous PC), 2) a method for evaluating the proximity of an expert candidate with a conference. A set of experiments in the context of the SIGIR conference editions was carried out to evaluate the effectiveness of our approach.

This paper is organized as follows. The next section introduces the representative prior literature related to PC recommendation and expertise retrieval. Our approach is described in section III and section IV. Section V presents the experiments carried out and the results obtained applying our approach on various collected data about the SIGIR conference editions. Finally, conclusions and future work are given in last section.

II. RELATED WORK

This paper can be related to what is called "expertise retrieval" in the literature [START_REF] Balog | Expertise retrieval[END_REF], [START_REF] Mangaravite | The LExR Collection for Expertise Retrieval in Academia[END_REF]. Expertise retrieval gathers two main tasks: expert profiling and expert finding. Expert profiling aims at answering the question: What topics a person is expert on?, whereas expert finding aims at answering the question: Who are the experts on this topic?

Expert profiling usually intends to construct automatically expert profiles, and in particular topical profiles [START_REF] Balog | Determining expert profiles (with an application to expert finding)[END_REF], [START_REF] Rybak | Temporal expertise profiling[END_REF]. Such topical profiles are keyword-based, and built from sources of expertise evidence, which are mainly documents such as enterprise documents [START_REF] Serdyukov | Automatic people tagging for expertise profiling in the enterprise[END_REF], curricula vitae (CVs) [START_REF] Ribeiro | On tag recommendation for expertise profiling: A case study in the scientific domain[END_REF], or research publications [START_REF] Cabanac | Accuracy of inter-researcher similarity measures based on topical and social clues[END_REF].

Expert finding intends to identify the persons with a high expertise for a given topic. The majority of the published work focuses on this task thanks to the datasets proposed by the TREC Enterprise track organised during four years [START_REF] Balog | Overview of the TREC 2008 enterprise track[END_REF]. Expert finding can be seen as a ranking task and many approaches took inspiration from information retrieval models [START_REF] Fang | Probabilistic Models for Expert Finding[END_REF]. Expert profiles are usually represented as vectors of terms as well as the topics and classical measures, such as the cosine measure, are used to evaluate the similarity between the user profiles and the topics [START_REF] Demartini | A vector space model for ranking entities and its application to expert search[END_REF].

A key step to address these tasks, is to find suitable sources of expertise evidence. Most approaches consider documents directly linked to expert candidates, such as the documents they authored or the documents mentioning their professional activities. Some works proposed to weight the importance of the documents used benefiting from bibliographic networks [START_REF] Hashemi | Expertise retrieval in bibliographic network: a topic dominance learning approach[END_REF].

In addition, some approaches that could be related to expertise retrieval showed that other sources of evidence such as citations [START_REF] Strohman | Recommending citations for academic papers[END_REF], [START_REF] Liang | Finding relevant papers based on citation relations[END_REF] or co-authorship and participations to conferences [START_REF] Cabanac | Accuracy of inter-researcher similarity measures based on topical and social clues[END_REF] are of interest to link researchers to scientific domains. Such approaches take advantage of the works on the building of bibliographic databases, providing resources to build co-authorship networks [START_REF] Ley | The DBLP Computer Science Bibliography: Evolution, Research Issues, Perspectives[END_REF] and citation graphs [START_REF] Garfield | Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas[END_REF], [START_REF] Tang | Arnetminer: extraction and mining of academic social networks[END_REF]. However, one problem is that citations convey different meanings, and each citation meaning corresponds to an implicit level of link between the citing article and the cited article [START_REF] Cronin | The Citation Process: The Role and Significance of Citations in Scientific Communication[END_REF]. [START_REF] Tang | Arnetminer: extraction and mining of academic social networks[END_REF] proposed to classify the citations according to their extracted contexts, while [START_REF] Huang | A multiple-perspective approach to constructing and aggregating citation semantic link network[END_REF] focused on mining the semantics of citation relations applying various NLP (Natural Language Processing) methods.

A citation graph (or citation network) is a directed graph representing documents and their citation relations, i.e., the links between the citing articles and the cited articles. On one hand, such a graph enables one to determine direct and indirect relations between documents. On the other hand, some properties related to the citations, such as the number of citations within a document and the time between the date of the citing article and the date of the cited article, can be used to differentiate the weights of the links between the documents and estimate some proximities between two documents.

Two types of approaches are distinguished in order to estimate the proximities between the nodes of a citation graph. The first type focuses on the direct citations to define the proximity. [START_REF] Kessler | Bibliographic coupling between scientific papers[END_REF] introduced the concept of bibliographic coupling as two documents citing the same third one and as representing a proximity between the two citing documents. Later, [START_REF] Small | Co-citation in the scientific literature: A new measure of the relationship between two documents[END_REF] considered co-citation patterns as supporting relationships between documents. Two documents frequently cited together by other ones convey a topical similarity. More recently, [START_REF] Gipp | Citation Proximity Analysis (CPA) -A New Approach for Identifying Related Work Based on Co-Citation Analysis[END_REF] introduced a new approach for measuring the similarity between documents based on the proximity of cocitations within an article's full-text. A comparison of various citation-based similarity measures is presented in [START_REF] Gipp | CITREC: An Evaluation Framework for Citation-Based Similarity Measures based on TREC Genomics and PubMed Central[END_REF]. However, these measures are interesting for document retrieval tasks, which do not correspond to the issue tackled in this paper.

The second type of approaches intends to leverage the structure of the graph. In this case, the proximity between two nodes of the graph is determined by the paths connecting them. In expertise retrieval, most of the approaches use the graph measure which was proposed in [START_REF] Katz | A new status index derived from sociometric analysis[END_REF]. The Katz measure is based on the sum of the path weights linking two nodes, giving more weight to short paths. [START_REF] Liben-Nowell | The link-prediction problem for social networks[END_REF] studied several node proximity measures for inferring new interactions among the members of a social networks. The experiments showed good performance of the Katz measure for this purpose. For paper recommendation, [START_REF] Liang | Finding relevant papers based on citation relations[END_REF] considered three types of citation relations and proposed a method using a graph distance based on the Katz distance and weighted links to measure the relevance between two papers. This work is close to our approach in the way the relations between papers are considered. However, the issue we tackle involves other elements and relations (e.g., coauthorship).

Various approaches of expertise retrieval were proposed with different goals, for instance, for finding reviewers for journal submissions [START_REF] Rodriguez | An algorithm to determine peer-reviewers[END_REF], [START_REF] Afzal | Expertise Recommender System for Scientific Community[END_REF], for recommending researchers matching some scientific interests [START_REF] Cabanac | Accuracy of inter-researcher similarity measures based on topical and social clues[END_REF], finding PhD supervisors for students [START_REF] Liu | Using semantic web technologies to improve expertise matching within academia[END_REF], or finding helping experts for technical support of medical software [START_REF] Mcdonald | Expertise recommender: A flexible recommendation system and architecture[END_REF]. Helping to elaborate or renew the program committees of conferences has recently aroused interest. [START_REF] Sfyris | Profile-based selection of expert groups[END_REF] tackles the issue of selecting experts meeting some criteria, such as covering a given set of competences for staffing an organization's board or for covering the topics of a conference. This approach builds a collection of textual candidate profiles and propose an IR-based approach to perform the expert group selection. Despite, experimenting the approach in the context of conference PCs, the suggested PCs were not compared with official PCs but evaluated with questionable measures of researcher performance, such as the H-index. In addition, this approach is based on textual descriptions only. [START_REF] Han | Recommending program committee candidates for academic conferences[END_REF] studied the possibility of recommending PC members by combining a content-based approach for expert finding and additional indicators related to the publication history of the candidates, their social closeness and their authority. The experiment results showed that the publication history and the social closeness are good indicators while the authority is not. This work gives hints on some indicators suitable for suggestions of PC members, which our approach is also based on.

Since the descriptions of the conference topics are limited, usually quite general, and sometimes in other languages than English, searching for expert candidates in a bibliographic database would inevitably return a highly "noisy" result. Thus, we propose an approach based on non textual elements only, modelled through networks.

Our approach gathers two aspects described in the following sections:

• The modelling of data related to the conferences, the publications, and the researchers; • The evaluation of the proximity between the expert candidates (i.e., researchers) and the given conference edition concerned by the suggestion of PC members.

III. BIBLIOGRAPHIC GRAPH-BASED DATA MODEL

Considering a given conference for which to suggest candidates to join the PC of the next edition, we distinguish two subsets of data related to the given conference : the data related to the expert candidates (who are directly involved in an edition of the given conference) and the data related to the expert candidates who are involved in the conferences close to the given conference. These two subsets of data are represented by two sub-models that form a final bibliographic model, which are detailed in the next sections.

A. Bibliographic model of the given conference

To suggest committee members, a first issue is to get and to exploit appropriate pieces of evidence about the expert candidates w.r.t. the conference. Firstly, we base our approach on the publications related to the given conference and their authors. Secondly, our approach relies on the past program committees of the given conference, and more particularly their members.

Our model differs from the existing work with regard to different aspects :

• The pieces of evidence are modelled through a 3-mode network; • Various relationships are modelled between the conference and the expert candidates and are considered of different weights;

• The graph is considered as non-oriented to identify the paths linking the conference to the expert candidates, including the co-citation as well as the co-coupling relations.

We model the bibliographic data as a 3-mode network, in which the nodes represent the concerned conference, the articles related to the conference editions (i.e., the articles published in the editions of the conference, the articles cited by or citing some articles published in the conference), and the expert candidates (i.e., authors and/or past committee members). Four types of edges are distinguished, connecting the different types of nodes: 1) the citation links between two articles (citing-cited), 2) the publication links between the conference editions and the articles, 3) the authoring links between the articles and the researchers, 4) the membership links between the conference editions and the researchers who participated to a program committee of a conference edition. A network example is shown in Figure 1. The relations are represented as oriented to facilitate their comprehension but the orientation is not considered in our approach. We consider four types of relations between the expert candidates and the given conference (as illustrated in Figure 2):

• The expert was PC member of an edition of the given conference; • The expert was author of an article published in an edition of the given conference; • The expert was author of an article that cites an article published in an edition of the given conference; These types of nodes and edges constitute the base of our model.

B. Bibliographic model of a close conference

Each conference relates to a particular scientific domain. A scientific domain gathers several conferences, which take interest of some common topics. The researchers interested in a particular scientific domain submit papers to the various conferences of this domain, and thus tend to be authors of papers published in these conferences. We call "close conferences" the conferences of a scientific domain that share common topics. Consequently, for a given conference the researchers involved in the close conferences represent expert candidates that could be invited to join the PC.

We follow a similar approach for modelling data about the close conferences, considering the papers published in these conferences, and the researchers concerned by authoring relations as well as PC membership relations. Two types of relations are thus considered: 1) the expert candidate authored a paper of the close conferences, 2) the expert candidate was member of one PC of the close conference.

The pieces of evidence about the given conference (see section III-A) and those about the close conferences are consequently connected through three types of relations as illustrated in Figure 3: 1) an article of the given conference cites or is cited by an article of the close conference, 2) an author of one conference (the given one or the close one) was member of a program committee of the other, and 3) an expert was both member of one edition of the given and of the close conference.

To reduce the complexity introduced by the various types of relations between the given conference and the close conferences, we combine them into a single one as illustrated in Figure 4. We first calculate the weight of the relations between the given conference and the close conferences, then a link between the given conference and each close conference is added to the graph.

(2') 1) Final bibliographic data model: Eventually, our model corresponds to a 3-mode network with three types of nodes and five types of edges. Figure 5 shows a network example w.r.t. our model. We do not consider the articles that cite or are cited by the articles of the close conferences, and the authors of such articles. This could be an extension of our model.

IV. EXPERT CANDIDATE PROXIMITY

To evaluate the proximity between an expert candidate and an edition of the given conference, we define different weights w.r.t the different relations represented in our model, and then deduce the weights of the paths linking the expert candidate to the conference. Finally, the proximity is based on all the paths between an expert candidate and the given conference.

A. Weights of the direct links in the network of the given conference

The weights of the direct links existing in a given conference network defined from the considered data (see section III) as follows:

• l C,d corresponds to the weight of a link between the edition of the conference C at the year t x and an article d published in the previous conference edition of the year t d :

l C,d = Q d Q max .e -tx -t d ∆t1max
Where Q d is the number of articles that cite the article d, Q max is the maximum number of articles that cite another article, and ∆t1 max is the maximum difference of time between t x and the publication year of an article. This definition promotes the articles that are very cited and more particularly the most recent ones. • v citing,cited corresponds to the weight of a link between a citing article and a cited article:

v citing,cited = Q cited Q max .e t citing -t cited ∆t2max
Where Q cited is the number of articles that cite the cited article, Q max is the maximum number of articles that cite another article, t citing is the publication year of the citing article, t cited is the publication year of the cited article, and ∆t2 max is the maximum difference between the publication years of a citing article and a cited article. This definition promotes the citation links with the most cited articles and more particularly the oldest ones. • w d,c corresponds to the weight of a link between the article d and the author c:

w d,c = 1 A d
Where A d is the set of authors of the article d. This definition considers equal contributions of the coauthors of the article. A variant could also consider the position of each co-author in the list of authors. • u C,c corresponds to the weight of a link between the edition of the conference C at the year t x and a member c of the PC of the year t p :

u C,c = e -tx -tp ∆t3max
Where ∆t3 max is the maximum difference between t x and the PC membership year of an expert candidate. This definition promotes the expert candidates who participated to the recent editions of the conference.

B. Weights of the links with the close conferences

As aforementioned in section III-B, various relations between the given conference and a close conference are considered. For simplification, these relations are combined into a single link between the given conference and a close conference, and then the links between the expert candidates and this close conference are considered. The weight of the link between the two conferences C 1 and C 2 is based on all the paths between them. The weights of the three path types (see section III-B) are defined as follows:

Sth 1 = l C1,citing l max + v citing,cited v max + l C2,cited l max Sth 2 = l C1,d l max + w d,c w max + u C2,c u max Sth 3 = u C1,c + u C2,c u max
The sum of the weights according to each path type is then computed. For the relation between a close conference and the given conference, we calculate the sum of the weights on the same type. For instance, Sth * 1 corresponds to the sum of path weights in type (1) between the given conference and the close conference.

Sth * i = θ∈P athC 1 ,C 2 Sth i
Where θ is a path of type i, P ath C1,C2 is the set of all the paths between the two conferences.

Finally, the weight of the relation between two conferences is defined as:

m C1,C2 = Sth * 1 Sth * 1max + Sth * 2 Sth * 2max + Sth * 3 Sth * 3max

C. Paths between the given conference and the expert candidates

To suggest committee members, we compute a proximity score between the new conference edition and each expert candidate. We assume that this proximity depends on:

• the number of paths between the conference node and the expert node. The proximity increases according to this number; • the weight of the paths between the conference node and the expert node. The weight of a path depends on the weight of each segment composing the path.

The weight of a path between the given conference node and an expert node depends on the six factors corresponding to the possible segments that can be involved in a path. A path comprises from one to three segments depending on the roles played by the expert candidate. comprises one segment corresponding to the membership link. Therefore, the weight of the path is defined as: The path contains three segments corresponding to the publication link, the citation link, and the authoring link.

Weight 1 = u C,
Weight 3 = l C,d l max + v d,d ′ v max + w d ′ ,c w max
• Role 4: the expert candidate c authored the article d ′ that cites the article d published in the given conference C.

The path contains three segments corresponding to the publication link, the citation link, and the authoring link.

Weight 4 = l C,d l max + v d ′ ,d v max + w d ′ ,c w max
• Role 5: the expert candidate c was member of a program committee of the close conference C ′ . The path contains two segments corresponding to the inter-conference link and the membership link.

Weight 5 = m C,C ′ m max + u C ′ ,c u max
• Role 6: the expert candidate c authored the article d ′ published in the close conference C ′ . The path contains three segments corresponding to the inter-conference link, the publication link, and the authoring link.

Weight 6 = m C,C ′ m max + l C ′ ,d ′ l max + w d ′ ,c w max

D. Proximity between the given conference and an expert candidate

For an expert candidate, the sum of the path weights for each path type is computed:

Weight * i = θ∈P athC,c Weight i
Where θ is a path of type i and P ath C,c is the set of all the paths between the given conference and the expert candidate c.

Finally, the proximity between the given conference C and an expert candidate c is defined as :

Proximity C,c = 6 i=1 η i Weight * i Weight * i max
Where η i are parameters that modulate the importance of the expert roles in the evaluation of the proximity between the given conference and the expert candidate.

V. EXPERIMENTS

A. Dataset

We experimented our approach using data about all the editions of the international conference on research and development in information retrieval (SIGIR). This conference is of interest because it is one of the main conferences in the field of information retrieval. It exists since 1971 and comprises a lot of editions, and thus a lot of program committees. Furthermore, it gathers a lot of publications, citing other important publications, as well as being cited by other important publications. Finally, a lot of researchers participated to the different editions as authors of papers and/or as members of the program committees. An additional interest of this conference is the availability of the data related to the conference through the bibliographic databases ACM and DBLP.

The preliminary work consisted in collecting all the papers published in the conference editions, i.e., 3,554 papers of the 40 editions of the SIGIR conference from 1971 to 2015. This is the initial set of papers, it is denoted B1 in the remainder of the paper.

The second work consisted in collecting the publications citing the SIGIR papers as well as the publications cited by the SIGIR papers. This set of 29,907 additional papers is denoted B2.

With regard to the close conferences, we considered eight prominent conferences that share some topics of interest with SIGIR: CIKM, WWW, ECIR, RecSys, IJCAI, KDD, ACL, and WSDM. The set of 35181 additional articles that were published in these conferences is denoted B3.

Finally, all the participations of experts to the various program committees were collected in the available editions, from 2004 for SIGIR, CIKM, WWW, ECIR, KDD, from 2007 for RecSys, from 2003 for IJCAI, from 2005 for ACL, and from 2010 for WSDM. The number of collected editions for each conference depends on the available editions.

B. Results

The objectives of the experiments were to answer two questions:

Q1. To what extent our approach can suggest the members of the official PCs?

Q2. To what extent our approach can suggest relevant new members for the next official PCs?

Q1 is related to the capacity of our approach to identify known PC members. To answer Q1 we compared the PCs suggested by our approach with the official PCs. Q2 is related to the capacity of our approach to identify some potential new members. Since there are no sets of experts identified as potential new SIGIR PC members available to serve as ground truth, we compared the new members of the PCs suggested by our approach with the new members of the official PCs to answer Q2. Since our approach aims at helping to renew PCs Q2, answering Q2 is decisive. We evaluated the suggestions according to the usual measures: precision (number of correct suggestions/number of suggestions) and recall (number of correct suggestions/number of members to suggest).

To present the results of our experiments, we use the notations listed in Table I.

Ci

Intersection between Sy and a combination of some official committees (or sub-committees) M .

We tested the seven configurations presented in Table II of the parameters involved in the proximity definition defined in section IV.

1) Suggested PCs compared with official PCs:

The first series of experiments consisted in suggesting committees for the last 5 editions, from 2011 to 2015. For each of these considered editions, the sets of papers B2 and B3 were built using the data about the previous SIGIR editions (e.g., bibliographic data from 2004 to 2013 were used to construct the suggestions for the 2014 PC) and the related close conferences.

We performed two types of comparisons for the suggested lists (Sy) of PC members for the year y:

• the comparison with the official program committees (M y) of the same year y to determine the number of correct suggestions, i.e., C1 = Sy ∩ M y; • the comparison with the set of experts who participated at least to one past program committee (M y -) before the year y, i.e., C2 = Sy ∩ M y -. For each experiment, we suggested three different lists: a suggested list comprising the same number of expert candidates as the corresponding official PC (Nb of experts = |M y |); a list with 50 % of expert candidates more than the corresponding official PC (Nb of experts = 150 % |M y |) and a list with 100 % of expert candidates more than the corresponding official PC (Nb of experts = 200 % |M y |).

Table III reports the comparison results (according to precision and recall) when applying the seven configurations of our approach, compared with the official PC of the corresponding year y (C1) .

The results show that our approach suggests a PC that shares from about 40 % to 53 % of official PC members. Up to more than 67 % can be suggested when doubling the suggestions, but to the detriment of precision. The best results were reached by the configuration 2 of our approach, which promotes the participations of expert candidates to the past PCs of the conference. The lowest results are obtained with configurations 5 and 6, which promote the participations to PC of close conferences and the publications in close conferences, respectively. The results shown in in Table IV, which compare the suggestions with the whole set of past PCs (M y -). A recall up to 96 % can be reached by the configuration 2 when doubling the size of the suggested list (Table IV, |Sy| = 200 %|M y -|). Since the configurations 5 and 6 are those identifying less existing PC members, they are likely to suggest more candidates that could be relevant new PC members. This is the concern of the experiments reported in the next section.

2) New members suggested compared with the new members of the official PCs: There are annually some new members who join the PC of a conference for the first time. Table V shows that there is a not negligible proportion of new members each year in the PC of the conferences related to our experiments. This observation supports the interest of our approach, helping to find expert candidates to invite for joining a new PC. To evaluate the capacity of suggesting relevant expert candidates applying our approach, we should compare our suggested new candidates with a set of identified relevant candidates (representing a ground truth) or submit our suggestions to expert assessors. Since there are no sets of experts identified as potential new SIGIR PC members for a given year y available to serve as ground truth, we compared the new members of the PCs suggested by our approach with the new members of the official PCs of the years after y to answer Q2. The precision and recall measures were still used. These experiments have thus some limitations because the new SIGIR PC members appearing in the next conference editions represent an incomplete ground truth. There is a need for a more complete ground truth, which should not depend on the current practice of constituting PCs. However, finding experts to suggest expert candidates or to assess suggestions is a difficult task. This constitutes an important future work.

For these experiments, as the first series of experiments run, we suggested PCs for the editions from 2011 to 2015. For each suggested PC for a given year y we considered the new suggested members N Sy , i.e., the members who were not present in one of the official PCs before the year y. We considered three different cases:

1) The set of new suggested experts who are actually new members in the corresponding official PC, i.e.,

C3 = N Sy ∩ (M y \ M y -)
For instance, an expert who is suggested for 2011 is actually participating to the official PC in 2011 as a new member.

2) The set of new suggested experts who are actually new members in the corresponding official PC or the next one, i.e.,

C4 = N Sy ∩ ((M y +1 ∪ M y) \ M y -)
For instance, an expert who is suggested for 2011, is actually joining the official PC as new member in 2011 or in 2012.

3) The set of new suggested experts who are actually new members of the corresponding official PC or the next two ones, i.e.,

C5 = N Sy ∩ ((M y +1 ∪ M y +2 ∪ M y) \ M y -)
For instance, an expert who is suggested for 2011 is actually joining the official PC as new member in 2011, in 2012, or in 2013.

The results for the first case, i.e., the comparison with the new members of the corresponding PC (C3), reported in Table VI show that a low proportion of suggested members corresponds to official new members of the corresponding year. However, the results show that the configuration of our approach that promotes the participations to PCs of close conferences (Configuration 6) yields the best results on average w.r.t. the other configurations. Doubling the suggestions can reach a higher recall but lowers precision.

The cases 2) and 3) were intended to verify if a delay may exist for an expert candidate to be invited to join the PC of a conference. The results reported in Table VII for the case 2) and in Table VIII for the case 3) show that more new suggested expert candidates become relevant since they join the officials PCs in the next years. These results confirm the

VI. CONCLUSION AND FUTURE WORK

We introduced an approach for helping program chairs of conferences to find expert candidates who could be invited to join the next conference PC. The proposed approach exploits bibliographic data to built a 3-mode network modelling the pieces of evidence of expert candidates w.r.t a given conference. Such pieces of evidence are related to the various relations between the experts and the given conference, such as authoring, PC membership, and citation, as well as the close conferences. A measure to evaluate the proximity between a conference edition and an expert candidate is proposed. It is based on the definition of weights for the various types of relations that connect the expert candidates to the conference editions.

Different configurations of our approach can be obtained by modulating the importance assigned to the different types of relations. These configurations yield PC suggestions composed quasi-entirely of members from past PCs, or PC suggestions mainly composed of "new" candidates, passing by more balanced suggestions.

Future work will be devoted to:

• enhance the evaluation protocol to get more accurate evaluations, by involving some assessments of the suggestions, for example asking some experts involved in the studied conferences, such as the previous PC chairs; • test different configurations of our approach to evaluate the impact of each considered relation; • experiment our approach on other conferences, to detect the differences which could exist in the PC compositions between some close conferences, and between the different PC chairs; • refine our approach by considering different types of citations or incorporating information about the conference topics; • propose a multi-criteria approach to build a program committee and then apply the appropriate configuration of our approach to fit the expected criteria.

Fig. 1 .

 1 Fig. 1. Example of a 3-mode network modelling bibliographic pieces of evidence about a set of expert candidates and their relations with the given conference.

Fig. 2 .

 2 Fig. 2. The relation types between the expert candidates and the given conference.

Fig. 3 .Fig. 4 .

 34 Fig. 3. The types of relations between the given conference and a close conference.

Fig. 5 .

 5 Fig. 5. Example of a 3-mode network modelling the bibliographic pieces of evidence about the expert candidates and their relations with the conference.

c u max ••

 max Role 2: the expert candidate c authored the article d published in the given conference C. The path comprises two segments corresponding to the publication link and the authoring link. Role 3: the expert candidate c authored the article d ′ cited by the article d published in the given conference C.

  Set of new members of the official program committee of the suggested year y. N y = M y \ M y - M y +i Set of members of the official program committee of the i-th year after the suggested year y Sy Set of experts who are suggested for the year y N Sy Set of new experts who are suggested for the year y. N Sy = Sy \ M y -

		TABLE I
	THE NOTATIONS OF THE SETS USED IN THE EXPERIMENTS
	Notation	Description
	M y	Set of members of the official program committee of the
		suggested year y
	M y -	Set of members of all official program committees of the
		years before the suggested year y
	N y

TABLE II THE

 II SEVEN CONFIGURATIONS TESTED WITH THE VALUES OF THE PARAMETERS

TABLE VI NEW

 VI MEMBERS OF PROGRAM COMMITTEE IN THE CORRESPONDING YEAR WHO ARE SUGGESTED = η 2 = η 3 = η 4 = η 5 = η 6 = 1 Conf iguration4 : η 1 = 0.5; η 2 = 0.5; η 3 = 1; η 4 = 0.5; η 5 = 0.5; η 6 = 0.5 Conf iguration5 : η 1 = 0.5; η 2 = 0.5; η 3 = 0.5; η 4 = 1; η 5 = 0.5; η 6 = 0.5 Conf iguration6 : η 1 = 0.5; η 2 = 0.5; η 3 = 0.5; η 4 = 0.5; η 5 = 1; η 6 = 0.5 Conf iguration7 : η 1 = 0.5; η 2 = 0.5; η 3 = 0.5; η 4 = 0.5; η 5 = 0.5; η 6 = 1

	Year	|N y|	C3	|Sy| = |M y| Recall N Sy	Precision C3	|Sy| = 150 %|M y| Recall N Sy Precision C3	|Sy| = 200 %|M y| Recall N Sy Precision
	Conf iguration1 : η 1 2011 92 0	0.00 %	102	0.00 %	3	3.26 %	199	1.51 %	6	6.52 %	337	1.78 %
	2012	96	5	5.21 %	114	4.39 %	7	7.29 %	239	2.93 %	10	10.42 %	391	2.56 %
	2013	55	2	3.64 %	82	2.44 %	2	3.64 %	182	1.10 %	3	5.45 %	292	1.03 %
	2014	87	0	0.00 %	88	0.00 %	2	2.30 %	198	1.01 %	4	4.60 %	312	1.28 %
	2015	82	0	0.00 %	75	0.00 %	1	1.22 %	165	0.61 %	3	3.66 %	274	1.09 %
	Conf iguration2 : η 1 = 1; η 2 = 0.5; η 3 = 0.5; η 4 = 0.5; η 5 = 0.5; η 6 = 0.5
	2011	92	0	0.00 %	49	0.00 %	1	1.09 %	122	0.82 %	3	3.26 %	233	1.29 %
	2012	96	3	3.13 %	64	4.69 %	5	5.21 %	156	3.21 %	8	8.33 %	286	2.80 %
	2013	55	0	0.00 %	33	0.00 %	2	3.64 %	107	1.87 %	2	3.64 %	202	0.99 %
	2014	87	0	0.00 %	37	0.00 %	0	0.00 %	113	0.00 %	2	2.30 %	218	0.92 %
	2015	82	0	0.00 %	28	0.00 %	0	0.00 %	90	0.00 %	2	2.44 %	183	1.09 %
	Conf iguration3 : η 1 = 0.5; η 2 = 1; η 3 = 0.5; η 4 = 0.5; η 5 = 0.5; η 6 = 0.5
	2011	92	1	1.09 %	98	1.02 %	4	4.35 %	194	2.06 %	6	6.52 %	321	1.87 %
	2012	96	6	6.25 %	113	5.31 %	8	8.33 %	234	3.42 %	10	10.42 %	378	2.65 %
	2013	55	2	3.64 %	78	2.56 %	3	5.45 %	171	1.75 %	3	5.45 %	280	1.07 %
	2014	87	0	0.00 %	82	0.00 %	2	2.30 %	184	1.09 %	5	5.75 %	305	1.64 %
	2015	82	0	0.00 %	70	0.00 %	1	1.22 %	155	0.65 %	5	6.10 %	264	1.89 %
	2011	92	0	0.00 %	103	0.00 %	3	3.26 %	200	1.50 %	6	6.52 %	333	1.80 %
	2012	96	5	5.21 %	117	4.27 %	7	7.29 %	239	2.93 %	10	10.42 %	385	2.60 %
	2013	55	2	3.64 %	82	2.44 %	2	3.64 %	181	1.10 %	3	5.45 %	290	1.03 %
	2014	87	0	0.00 %	87	0.00 %	2	2.30 %	195	1.03 %	4	4.60 %	308	1.30 %
	2015	82	0	0.00 %	74	0.00 %	1	1.22 %	164	0.61 %	3	3.66 %	271	1.11 %
	2011	92	1	1.09 %	103	0.97 %	3	3.26 %	199	1.51 %	6	6.52 %	331	1.81 %
	2012	96	5	5.21 %	116	4.31 %	8	8.33 %	236	3.39 %	10	10.42 %	387	2.58 %
	2013	55	2	3.64 %	79	2.53 %	2	3.64 %	179	1.12 %	3	5.45 %	291	1.03 %
	2014	87	0	0.00 %	86	0.00 %	2	2.30 %	194	1.03 %	4	4.60 %	310	1.29 %
	2015	82	0	0.00 %	72	0.00 %	1	1.22 %	164	0.61 %	3	3.66 %	272	1.10 %
	2011	92	3	3.26 %	133	2.26 %	4	4.35 %	252	1.59 %	5	5.43 %	399	1.25 %
	2012	96	6	6.25 %	147	4.08 %	9	9.38 %	291	3.09 %	10	10.42 %	467	2.14 %
	2013	55	2	3.64 %	111	1.80 %	3	5.45 %	223	1.35 %	4	7.27 %	355	1.13 %
	2014	87	1	1.15 %	109	0.92 %	3	3.45 %	230	1.30 %	3	3.45 %	367	0.82 %
	2015	82	1	1.22 %	96	1.04 %	2	2.44 %	198	1.01 %	5	6.10 %	322	1.55 %
	2011	92	0	0.00 %	146	0.00 %	2	2.17 %	255	0.78 %	5	5.43 %	393	1.27 %
	2012	96	5	5.21 %	158	3.16 %	7	7.29 %	299	2.34 %	8	8.33 %	454	1.76 %
	2013	55	2	3.64 %	115	1.74 %	2	3.64 %	234	0.85 %	2	3.64 %	360	0.56 %
	2014	87	1	1.15 %	124	0.81 %	2	2.30 %	247	0.81 %	4	4.60 %	380	1.05 %
	2015	82	0	0.00 %	105	0.00 %	2	2.44 %	217	0.92 %	3	3.66 %	331	0.91 %

TABLE VIII THE

 VIII SUGGESTED EXPERTS WHO ARE NEW MEMBERS OF PROGRAM COMMITTEES FOR THE NEXT TWO YEARS|M y ∪ M y +1 ∪ M y +2 | |Sy| = 150% |M y ∪ M y +1 ∪ M y +2 | |Sy| = 200% |M y ∪ M y +1 ∪ M y +2 |

	Year	N y +1 ∪ |N y∪			|Sy| =
		N y +2 |	C5	Recall	N Sy	Precision C5	Recall	N Sy	Precision C5	Recall	N Sy	Precision

• Role 1: the expert candidate c was member of a past program committee of the given conference C. The path


The Search Engine  The analysis of political tweets during the election campaigns, or specific events, is increasing and can be seen as a specific type of political discourse (Longhi, 2013)  Extensive literature on the analysis of political tweets: but these works are difficult to gather because they come either from the computer sciences, either from the humanities and social sciences.

 Despite the unquestionable interest in outlining political facts, citizens do not have access to these results.

Challenges of #Idéo2017

The #Idéo2017 platform  For the period from September 1st 2016 to May 7th 2017, 42290 tweets were extracted for the 11 candidates. These tweets were gathered in a collection published in a TEI corpus in the standards of the Ortolang platform, thanks to the Corli Consortium.



  Development of the Interaction Space (IS) model to model CMC interaction (Chanier & Jin, 2013).  Includes descriptions of time, set of participants, online location(s) defined by the properties of the set of environments used by participants.  Description of the IS within the TEI header and messages and turns encoded in the TEI body using a common <post> element Tool Development Analysis The #Ideo2017 platform proposes: The Analysis "I Analyze the Tweets that Contain the Word [Word]" The Analysis "I Analyze the Tweets of [Candidate]"



  Julien Longhi Université de Cergy-Pontoise -Centre de Recherche AGORA (julien.longhi@u-cergy.fr) Members of the project: Hassine N., Marinica C., Borzic B., Alkhouli A Openess  Corpora released as open-datapaves way for scientific examination, replication and cumulative analyses  Released on ORTOLANG (French equivalent of DARIAH the European infrastructure for Humanities) Bibliographic reference created for each corpus and given in <titleSmt> of TEI header. e.g. Longhi J., Marinica C., Borzic B., Alkhouli A., 2014: Polititweets, corpus de tweets provenant de comptes politiques influents. In Chanier T. (ed) Banque de corpus CoMeRe. Ortolang.fr : Nancy. [cmr-polititweets-tei-v1]. https://repository.ortolang.fr/api/content/comere/v3.3/cmrpolititweets.html Interests and limits  Aims to build a kernal corpus of computer-mediated communication (CMC) genres in French  Mono and multimodal interactions stemming from networks including the Internet and telecommunications that may be synchronous or asynchronous  Members had previously collected and structured different types of CMC corpora within their local teams (in a variety of formats with disparities in corpus compilation choices)  Corpora are structured and referred to in a uniform way in order that they may form part of the forthcoming French National Reference Corpus  #Idéo2017 combines different technologies and inputs, which give citizens the opportunity to grasp a part of the discursive issues of the election.  This development, which can be enriched, allows citizen to easily use a set of features usually accessible by software requiring different transformations of the data.

Tools and analysis

 Citizens can make their own queries (based on linguistic and textometric criteria, more precisely, the most commonly used words by political personalities, analyses of similarities, ALCESTE algorithm, etc.) and obtain comprehensible results.