Robin Brochier
email: robin.brochier@univ-lyon2.fr

Adrien Guille
email: adrien.guille@univ-lyon2.fr

Benjamin Rothan
email: benjamin@peer.us

Julien Velcin
email: julien.velcin@univ-lyon2.fr

Impact of the Query Set on the Evaluation of Expert Finding Systems

Keywords: expert finding, recommender system, evaluation

Expertise is a loosely defined concept that is hard to formalize. Much research has focused on designing efficient algorithms for expert finding in large databases in various application domains. The evaluation of such recommender systems lies most of the time on humanannotated sets of experts associated with topics. The protocol of evaluation consists in using the namings or short descriptions of these topics as raw queries in order to rank the available set of candidates. Several measures taken from the field of information retrieval are then applied to rate the rankings of candidates against the ground truth set of experts. In this paper, we apply this topic-query evaluation methodology with the AMiner data and explore a new document-query methodology to evaluate experts retrieval from a set of queries sampled directly from the experts documents. Specifically, we describe two datasets extracted from AMiner, three baseline algorithms from the literature based on several document representations and provide experiment results to show that using a wide range of more realistic queries provides different evaluation results to the usual topic-queries.

Introduction

It is common to consider expertise as an implicit knowledge about a domain that someone carries and shares in different manners. Expertise retrieval aims at identifying this knowledge through explicit artifacts such as communications, actions or interactions between people. When someone call for an expert, she expects to find a candidate able to understand a specific query. Whereas most evaluations for expertise retrieval consist in directly querying the namings or descriptions of the ground truth topics of a given dataset, we claim that these queries do not show much interest for a real case scenario since:

the textual content of the topics namings are very limited in terms of language. Using richer (hence noisier) descriptions might better test the robustness of the evaluated algorithms. For example, it is better to query multiple times a retrieval algorithm with several texts relevant to the field of "data mining" than only once with the naming of the field itself. In real case scenarios, users have a wide range of behaviors and seldom use the same queries when looking for the same thing no one really seeks for experts in so broad subjects. Most of the time, someone looks for an expert with a very specific application in mind. Indeed, if a recruiter from a company is looking for a researcher to work on a specific subject, it is more likely that she will use the detailed description of the project instead of a generic naming of the job to find the right person.

In this paper, we first provide in Section 3 a formal definition of the expert finding task applied to the data extracted from AMiner 3 . In particular, we describe two protocols: the topic-query evaluation and the document-query evaluation. We then describe in Section 4 three baseline algorithms from the literature that we reimplemented and tested using several document representations. Finally we show and analyze in Section 5 the results of our experiments, demonstrating the impact of the type of query on the behaviors of the algorithms and document representations.

Precisely, our contribution is fourfold:

1. we propose two different procedures for generating queries and study their impact on the evaluation results 2. we describe two ways of using AMiner's data for expert finding and detail the preprocessing needed 3. we reimplement and evaluate 3 algorithms from the literature based on several document representations 4. the corresponding Python code is made publicly available 4 which makes it easy to reproduce the experiments or even expand the proposed pipeline.

Related Works

The automation of expert finding appeared as a research field along with the creation of large databases when started the digitalization of libraries and of the communication tools in big companies. P@noptic Expert [START_REF] Craswell | P@ noptic expert: Searching for experts not just for documents[END_REF] is one of the first published works on expertise retrieval. The proposed model transforms the expert finding task in a text similarity task by building a meta-documents for each candidate, aggregating all documents where the name of this candidate appears.

In 2005, the research around expert finding received a boost with the TREC-2005 Enterprise Track, Expert search task. They provided a dataset extracted from the World Wide Web Consortium (W3C). Moreover, they shared an evaluation toolkit to allow researchers to confront their algorithms. As a result, a formal definition of the problem emerged [START_REF] Craswell | Overview of the trec 2005 enterprise track[END_REF]. As presented in [START_REF] Balog | Formal models for expert finding in enterprise corpora[END_REF], the generative document-model of Balog et al., we denote q a query, d a document and e a candidate. The expert finding task consists in estimating the probability of a candidate to be an expert given a query P (e|q) = P (q|e)P (e) P (q)

. Voting models as in [START_REF] Macdonald | Voting for candidates: adapting data fusion techniques for an expert search task[END_REF] relax the probabilistic view of the latter equation. As an example, the score of a candidate can be computed by ranking all documents against the query with a document representation such as the bag-of-words based model term frequency. Then each candidate is provided a score given the ranks of the documents she is associated to. In [START_REF] Zhang | Expert finding in a social network[END_REF] and [START_REF] Serdyukov | Modeling multi-step relevance propagation for expert finding[END_REF], the authors propose to propagate the affinity between the query and the documents across the collaboration graph in a similar manner as PageRank [START_REF] Page | The pagerank citation ranking: Bringing order to the web[END_REF]. More recently, [START_REF] Van Gysel | Unsupervised, efficient and semantic expertise retrieval[END_REF] adapted a word embedding technique to embed words and candidates in the same vector space. Many algortihms presented recently in the field of representation learning such as TADW [START_REF] Yang | Network representation learning with rich text information[END_REF] and metapath2vec [START_REF] Dong | metapath2vec: Scalable representation learning for heterogeneous networks[END_REF] can be adapted to the task of expert finding but their authors did not experiment them on this specific task. Much work has been done for expert finding in community-based question answering as shown in [START_REF] Zhao | Expert finding for communitybased question answering via ranking metric network learning[END_REF] and their ranking metric network learning framework and in [START_REF] Zhao | Cold-start expert finding in community question answering via graph regularization[END_REF] which adresses the cold-start expert finding problem.

Framework for Expert Finding Evaluation

In this Section, after formally describing the expert finding task, we present two methodologies to generate queries. The first, usually used in the literature, directly sets topics labels as queries whereas the second, which we introduce in this paper, samples documents from the experts of each topic. Finally we detail how we used the data from AMiner to generate two datasets for the expert finding task.

Formal Description

Let G = (V, E) be a bipartite graph with nodes V = V C ∪ V D corresponding to a set of candidates C and a set of documents D, where the links are undirected associations candidate-documents. Let X ∈ R |D|×N be the textual features of the D documents. The expert finding task, given such (G, X) dataset (see Figure 1), consists in scoring the set of candidates given a textual query q ∈ R N , in order to answer the question "who are the candidates more likely to be experts in the topics present in the query ?". Given a set of queries Q = (q 1 , ..., q i , ..., q M ) each associated with an identified set of experts E i ⊂ E ⊂ C, E being the global set of known experts among the candidates, we want to optimize the ranking of the ground truth experts E i among the global set of experts E.

Evaluation

To evaluate the ranking of experts produced by an algorithm given a query, we use several common metrics from information retrieval such as Precision at rank K (P@K), Average Precision (AP) and Reciprocal Rank (RR). Moreover, to better understand the behavior of the algorithms tested, we construct the Fig. 1: Hypothetical example of a dataset for expert finding.

Receiver Operating Characteristic (ROC) curve and compute its Area Under the Curve (AUC). For each of these metrics, we also compute their standard deviations along the queries which shows the robustness of the tested algorithms against the variations in the data. Moreover, when we have multiple queries per topic, we compute the standard deviation along the topics. We now present two ways of generating queries and their corresponding ground truth experts.

Topic-query evaluation This approach is straightforward and is commonly adopted in the expert finding community. For a specific topic, its naming or description is directly used as a query and its associated experts are the ground truth list of candidates to be retrieved. Algorithm 1 shows the complete evaluation procedure. As a result, if the dataset is composed of 10 topics, the protocol of evaluation consists of 10 queries. We call this approach the topic-query evaluation. For each measure described above, we are interested in its mean (Mean) and standard deviation (STD ) along the queries.

Algorithm 1 Topic-query evaluation procedure. The function Evaluate generates metrics such as P@10 and the ROC AUC based on the produced ranking and the ground truth expert set of a given topic.

Require: Ranking Algorithm scores ← [ ] for all topics do candidates ranking = Ranking Algorithm(current topic textual expression) current score ← Evaluate(candidates ranking, ground truth experts set) scores.append(current score) end for return Mean(scores), STD(scores) Document-query evaluation We propose to sample the documents linked with the experts of a given topic in order to use them as queries. Instead of using the topic description, we use the set of documents associated to the ground truth experts of a given topic. Precisely, we create a set of queries and their associated experts by selecting each document of the dataset linked with the ground truth experts. As such, the evaluated algorithm produces a ranked list of candidates for each document-query and its performance is measured by comparing the ranking with the experts of the same topic as the expert who produced the document-query. Since several document-queries are sampled for each topics, we also compute the means and standard deviations along the topics, by computing these values along the averaged measures intra-topics. To avoid any bias in the metrics, when evaluating an algorithm on a sampled document, we leave it out of the data. We call this approach document-query evaluation. Algorithm 2 shows the complete evaluation procedure.

Algorithm 2 Document-query evaluation procedure. Note that the computed metrics are also averaged for each topic in order to compute the inter-topic standard deviation.

AMiner Data

The AMiner project aims to provide tools for mining researcher's social network. They provided several datasets5 [START_REF] Tang | Arnetminer: extraction and mining of academic social networks[END_REF] collecting papers, authors, co-authorship and citations links extracted from DBLP [START_REF] Ley | The dblp computer science bibliography: Evolution, research issues, perspectives[END_REF], ACM (Association for Computing Machinery) and other sources in the field of computer science. For the task of expert finding, they provided two lists of experts 6 . The first, the machineannotated list, is composed of 13 topics and has been built from topical web search. The second, the human-annotated list, is composed of 7 topics built with the method of pooled relevance judgments together with human judgments as described in [START_REF] Zhang | A mixture model for expert finding[END_REF]. We used the machine-annotated list with the citation dataset V2 and the human-annotated list with the citation dataset V1 available on the AMiner website 7 .

We preprocessed the two datasets based on the distribution of links between candidates and documents. We also took into account the document string length (number of letters). First we kept only authors with less than 100 documents links and with at least one link. This reduces author name ambiguity by discarding authors who were originally connected to tens of thousands documents. Then we composed the textual content of the documents by concatenating their titles and abstracts and by keeping only those with string length greater than 50. As a result, we ended up with two datasets:

-AMiner expert dataset 1: using the machine-annotated list of experts, is composed of 996,110 candidates, 1,125,082 documents, 1,269 experts in 13 topics. The distribution of the experts across topics is given in Table 1a (one expert can be linked to several topics) with the total number of documents linked to those experts -AMiner expert dataset 2: using the human-annotated list of experts, is composed of 532,968 candidates, 480,630 documents, 210 experts in 7 topics.

The distribution is given in Table 1b.

Baseline Algorithms

After a short description of document representation, we describe three baseline algorithms taken from the literature. We reimplemented them since their original codes were not available or hardly reusable. Moreover, we could easily extend them to work with any kind of document representation.

Document Representation

Our three baseline algorithms rely on a measure of semantic similarity between the queries and the corpus of documents. We chose to try several document representations: term frequency (TF), term frequency -inverse document frequency (TF-IDF) and latent semantic indexing (LSI) [START_REF] Papadimitriou | Latent semantic indexing: A probabilistic analysis[END_REF]. We tokenized the text of the documents by lowercasing the characters, removing stop words and concatenating tokens based on their co-occurrence counts to compound 2-grams and 3-grams. Then, words appearing less than 3 times in the corpus or in more than 50% of the documents were discarded to reduce the computational cost (a) AMiner dataset 1. without affecting the retrieval performance. The number of dimensions of the singular value decomposition for the LSI is 300. This number was chosen to ensure components above noise level are retained as proposed in [START_REF] Kaiser | The application of electronic computers to factor analysis[END_REF].

Topic

Text-based Approach 1: P@noptic Model

P@noptic Expert [START_REF] Craswell | P@ noptic expert: Searching for experts not just for documents[END_REF] is a simple algorithm which creates meta-documents for each author. Our implementation first concatenates the contents (title+abstract) of all documents linked with each candidate, then vectorizes this meta-documents using the pretrained documents representation models. Finally, it computes the cosine similarities between a query and the meta-documents and ranks the candidates by descending order of their scores.

Text-based Approach 2: Voting Model

Our voting model based on [START_REF] Macdonald | Voting for candidates: adapting data fusion techniques for an expert search task[END_REF] first computes the cosine similarities between the query and the documents of the dataset and then ranks all documents by descending order of their score. The algorithm then sums the inverse value of the rank (Reciprocal Rank -RR) of each document a candidate is linked with. If a candidate is linked with the 2nd, 3rd and 7th closest documents to the query, its score will be 1 2 + 1 3 + 1 7 = 0.976. This algorithm gives a huge boost to candidates who have at least one document well ranked and tends to promote candidates with more documents than others. We also tried other fusion techniques than the RR such as CombSUM and CombMNZ, described in [START_REF] Macdonald | Voting for candidates: adapting data fusion techniques for an expert search task[END_REF], but they provided weaker results.

Graph and Text-based Approach: Propagation Model

The propagation model we made is a simpler version of those described in [START_REF] Serdyukov | Modeling multi-step relevance propagation for expert finding[END_REF]. The algorithm first computes the cosine similarities between the query and the documents and it initializes a score vector S 0 of length |C| + |D| with zeros for candidates and the documents-query scores for documents. It then operates several two-steps random walks with restart until the score vector converges (until the L 2 norm of the difference of its previous value and current value is below 10 -6 ). These random walks are done iteratively: S i+1 = (1-η)×A(AS i )+ ηR where η is the jumping factor, a scalar between 0 and 1, which controls the restart, R = S 0 is the restart vector that represents the global probability of a random walk to restart from its original node, A is the column-wise L 1 normalized adjacency matrix of the bipartite graph, also known as the PageRank transition matrix [START_REF] Page | The pagerank citation ranking: Bringing order to the web[END_REF]. At each step, scores jump from documents to candidates then from candidates to documents. A last step is finally done to propagate scores back to the candidates. These scores are then ranked by descending order.

Experiments

In this section, we present the experiments we did with both topic-query and document-query evaluations. We first show some general results before analyzing the effect of the type of query and finally focusing on the variations of ranking along the queries and the topics.

Settings

We evaluated our baseline models on the topic-query and the document-query methodologies. We made two evaluations for the propagation model using η = 0.1 where the restart is weak, hence the propagation is wide, and η = 0.5 where the scores stay close to their initial values. Moreover, for each model, the semantic similarity was computed with TF, TF-IDF and LSI document representations. Table 2 shows the results on the AMiner expert dataset 1 and Table 3 shows the results on the AMiner expert dataset 2.

General Results

For both datasets, the document representation TF-IDF performs generally better except for the AUC score, where LSI performs best, especially on topic queries. Actually, taking a closer look at the ROC curve, we could see that LSI is better in ranking for the worst ranked experts. It smoothes the curve in the top right corner and hence improves the area under the curve. Most metrics (P@10 and RR for example) are intended to focus on the quality of the very first ranked experts but the ROC AUC allows us to analyze the behavior of a ranking algorithm over the entire ranking. It is also important to note that the results are more stable across the choice of document representation for the second dataset. This behavior is expected since the ground truth experts have been human curated.

Effect of the Type of Query

We observe different rankings of the baseline algorithms depending on the type of evaluation performed. For the topic-query procedure, the propagation model performs best (with η = 0.5 for the first dataset and η = 0.1 for the second) whereas the voting model is the best for the document-query evaluation. Our explanation is that voting models are good when queries and documents are of the same type since we only need one candidate's document to be similar to the query to push her to the top of the ranking. When the query is as short as "data mining", the chance to find such a similar document is low since few documents about data mining have the words "data mining" in their content. Indeed, scientific articles rarely deal with data mining in general but rather focus on particular aspect of this field.

In contrast, the propagation model can give a good score to a candidate if in her neighborhood, the query is similar to some documents. Even if this candidate is an expert of "data mining" without never actually using the expression, there are quite some chances that in its close social network, some other candidates used these two words.

Then, the voting model might perform best than propagation for document queries because the latter tends to mistaken an information retrieval expert who worked closely with data mining experts. This situation is less likely to happen when the query is a short and very specific description than with paper contents that share a lot of similar terms between topics.

Moreover, this difference of results between query types are weaker when using LSI, which is due to the ability of this document representation to capture a similarity between two texts that do not share any word in common. The effect of short query is thus highly reduced compared to TF and TF-IDF.

Standard Deviation along Queries and Topics

One important aspect is the amount of dispersion the sets of scores have around their means. We computed the standard deviations for each evaluation to have an insight of the robustness of the algorithms to queries and to topics. Interestingly, for the document-query, the standard deviations along topics evaluation are lower than the deviations along queries. This shows that the robustness of the algorithms are not that much impacted by the variation of topics, as could have suggested the standard deviations for the topic-query evaluation, but merely by the variety of queries intra-topics. As a result, using only a few topic queries is statistically biased since some topic namings might have lesser chance to appear in their related documents. Finally, in the second dataset, the deviations along topics for the voting model are significantly lower than other models which is a precious information that cannot be revealed by a topic-query evaluation if one wants to favorite stability over the searched topics of expertise.

Pros and Cons of the Document-Query Evaluation

Beside the fact that the document-query evaluation seems to better represent a real case application of expert finding, we showed that it provides a deeper insight on the robustness of an algorithm. The different rankings of algorithms for both evaluations and their corresponding inter-topics standard deviations prove that using only the namings of the topics is not a satisfactory protocol to compare expert finding systems. However, in a general manner, measures are much better with the topic-query evaluation. This is due to two aspects:

document-queries are semantically fine grained and it is more difficult to separate two queries of different topics. This makes the expert finding task harder to solve but it is not a bad thing for the evaluation. -in our current configuration, document-queries do not rely on an annotated dataset. As a consequence, some sampled documents might not actually belong to the topic their authors are associated with. This motivates the construction of a ground truth set of documents associated to at least one of the human-annotated expert topics.

Summary and Future Work

We compared two evaluation protocols for scientific expert finding that rely on two types of query generation. Evaluating our baseline models with this framework, we showed that using the documents written by the ground truth experts brings different results than with the usual topic queries. Specifically, short queries can profit of a propagation model whereas longer queries are better handled by a simpler voting model. Moreover, the lower standard deviations along topics for the document-query evaluation shows that there is a bias in using only one topic naming as query since the document representations do not handle well such short query similarity to the documents.

To improve the document-query evaluation with the AMiner data, we would like to filter the set of sampled documents by human annotation in order to keep only those that match the expertise of their authors. This would then justify a deeper analysis of the significance of the measurements to consider the variations of ranking of the evaluated algorithms along the queries. Another interesting work would be to perform an online evaluation of the same expert finding algorithms in the case of a reviewer assignment application in order to compare the results with our framework.

(a) Baseline mean scores and their query (same as topic) standard deviations for the topic-query evaluation.

  G adjacency matrix of the bipartite graph and the X features matrix of the documents.

Table 1 :

 1 Distribution of experts and their documents counts across topics in both AMiner expert datasets.

				(b) AMiner dataset 2.
		Exps Docs	Topic	Exps Docs
	neural networks	105 1941	intelligent agents	29	685
	ontology alignment	49	908	planning	33	510
	boosting	49	1062	machine learning	38	628
	support vector machine	91	1145	natural language processing 38	453
	intelligent agents	28	729	information extraction	18	290
	machine learning	33	1229	semantic web	41	521
	computer vision	174 3636	support vector machine	29	323
	data mining	285 8034	TOTAL	210 3410
	natural language processing 40	1480
	semantic web	332 6435
	planning	21	531
	cryptography	139 3033
	TOTAL	1269 30802

Table 2 :

 2 Results of the evaluations with the AMiner dataset 1, composed with the machine-annotated experts and the biggest candidates-documents set. Bold values are the best scores across the algorithms for each document representation.(a) Baseline mean scores and their query (same as topic) standard deviations for the topic-query evaluation. AUC 0.860±0.097 0.866±0.100 0.834±0.052 P@10 0.829±0.148 0.843±0.118 0.686±0.181 AP 0.647±0.142 0.676±0.139 0.564±0.140 RR 1.000±0.000 1.000±0.000 1.143±0.350 Prop (η = 0.5) AUC 0.860±0.098 0.864±0.101 0.843±0.057 P@10 0.786±0.146 0.814±0.125 0.686±0.181 AP 0.646±0.139 0.671±0.140 0.579±0.138 RR 1.000±0.000 1.000±0.000 1.286±0.452 (b) Baseline mean scores and their query standard deviations for the document-query evaluation.

				TF	TF-IDF	LSI
			AUC 0.777±0.099 0.778±0.102 0.832±0.083
		P@noptic	P@10 0.662±0.262 0.685±0.260 0.615±0.301 AP 0.398±0.193 0.415±0.204 0.395±0.233
			RR 1.615±0.923 1.538±0.746 1.769±0.890
			AUC 0.714±0.137 0.714±0.138 0.800±0.107
		Vote	P@10 0.608±0.312 0.608±0.287 0.538±0.325 AP 0.373±0.212 0.381±0.209 0.390±0.247
			RR 2.308±2.398 1.538±0.929 2.769±3.765
			AUC 0.834±0.093 0.834±0.096 0.824±0.085
		Prop	P@10 0.669±0.270 0.677±0.264 0.615±0.298
		(η = 0.1)	AP 0.458±0.239 0.473±0.243 0.389±0.230
			RR 1.462±0.929 1.538±1.082 1.692±1.136
			AUC 0.842±0.086 0.842±0.088 0.833±0.083
		Propagation	P@10 0.677±0.255 0.685±0.274 0.615±0.296
		(η = 0.5)	AP 0.457±0.232 0.472±0.242 0.395±0.231
			RR 1.308±0.606 1.385±0.738 1.692±1.136
	(b) Baseline mean scores and their query standard devia-	(c) Topic standard de-
	tions for the document-query evaluation.	viations.
		TF	TF-IDF	LSI	TF TF-IDF LSI
		AUC 0.593±0.131 0.618±0.134 0.620±0.139	0.114 0.112 0.119
	P@noptic	P@10 0.255±0.266 0.335±0.294 0.302±0.298 AP 0.150±0.117 0.181±0.133 0.174±0.133	0.174 0.191 0.195 0.097 0.102 0.104
		RR 9.153±16.028 6.210±13.323 8.663±16.262	6.063 3.676 6.088
		AUC 0.606±0.131 0.630±0.137 0.637±0.142	0.109 0.109 0.111
	Vote	P@10 0.284±0.263 0.322±0.280 0.275±0.276 AP 0.169±0.131 0.193±0.145 0.187±0.152	0.166 0.181 0.174 0.102 0.110 0.115
		RR 6.819±15.421 5.783±13.983 8.438±16.987	3.400 2.789 4.168
		AUC 0.591±0.140 0.617±0.143 0.612±0.145	0.128 0.126 0.131
	Propagation	P@10 0.256±0.253 0.336±0.292 0.300±0.288	0.156 0.187 0.184
	(η = 0.1)	AP 0.152±0.114 0.183±0.132 0.173±0.127	0.093 0.102 0.101
		RR 9.035±16.718 6.773±14.641 8.948±17.422	9.175 6.139 9.245
		AUC 0.598±0.141 0.623±0.142 0.621±0.147	0.125 0.122 0.128
	Propagation	P@10 0.255±0.245 0.333±0.283 0.298±0.282	0.147 0.177 0.175
	(η = 0.5)	AP 0.155±0.115 0.185±0.133 0.177±0.130	0.093 0.101 0.102
		RR 8.994±17.224 6.419±14.088 9.089±18.358	8.488 5.059 8.944

Table 3 :

 3 Results of the evaluations with the AMiner dataset 2, composed with the human-annotated experts and the smallest candidates-documents set. Bold values are the best scores across the algorithms for each document representation.

https://AMiner.org/data

https://AMiner.org/lab-datasets/expertfinding/#expert-list

https://AMiner.org/citation


Julián Andrés Galindo
email: julian.galindo@epn.edu.ec

Sophie Dupuy-Chessa

Nadine Mandran

Eric Céret

Using user emotions to trigger UI adaptation

Keywords: UI adaptation, emotion, usability, aesthetics I

User experience is an important user interface quality factor that needs to be maintained during interaction. With this goal, we propose to adapt user interfaces to users' emotions. This requires being able to detect at runtime negative emotions to trigger adaptations. However, emotions depend on users' gender and age. This makes it difficult to identify negative emotions and their cause. This paper studies the categorization of emotions depending on users' age and gender and two user experience dimensions (aesthetics and usability). Thanks to an experiment, it proposes some emotions thresholds that can be used for detecting usability and aesthetics problems from users' age and gender. These results are used in a user interface adaptation system to infer negative emotions and user interface problems.

INTRODUCTION

Beyond usability, user experience (UX) is nowadays recognized as an important quality factor to make systems or software successful in terms of user take-up and frequency of usage [START_REF] Winckler | Identifying user experience dimensions for mobile incident reporting in urban contexts[END_REF]. For instance, according to [START_REF] Bernhaupt | User experience evaluation in entertainment[END_REF], UX depends on dimensions like emotion, aesthetics or visual appearance, identification, stimulation, meaning/value or even fun, enjoyment, pleasure, or flow. Among these dimensions, the importance of usability and aesthetics is recognized [START_REF] Tractinsky | What is beautiful is usable[END_REF]- [START_REF] Angeli | Interaction, usability and aesthetics: what influences users' preferences?[END_REF]. So, both of them need to be considered while designing user interfaces (UI) [START_REF] Stone | User interface design evaluation[END_REF], [START_REF] Galitz | The essential guide to user interface design: an introduction to GUI design principles and techniques[END_REF].

Beyond design, it would be interesting to check UX at runtime and improve it if necessary. To achieve a good UI quality in any context of use (i.e. user, platform and environment), plasticity [START_REF] Calvary | Plasticity of user interfaces: A revisited reference framework[END_REF] proposes to adapt UI to the context while preserving usercentred properties. In a similar way, our goal is to preserve or improve UX at runtime, by proposing UI adaptations. Adaptations can concern aesthetics or usability. They can be triggered by the detection of negative emotions [START_REF] Paterno | User Interface design adaptation[END_REF], as these emotions can express a problem with the UI.

Complexity is then related to the categorization of emotions as negative because they depend on various individual 1 www.noldus.com/human-behavior-research/products/facereader characteristics such as age [START_REF] Cyr | Gender and website design in e-business[END_REF], [START_REF] Steenkamp | How country characteristics affect the perceived value of web sites[END_REF] and gender [START_REF] Steenkamp | How country characteristics affect the perceived value of web sites[END_REF]- [START_REF] Djamasbi | Generation Y, web design, and eye tracking[END_REF]. It is even higher while considering categorization at runtime.

We present an experiment that allows us to categorize users' emotions according to the levels of UI usability and/or aesthetics and to the users' age and gender. Indeed, even if there are other emotion's sources (e.g. personality, day time-line, weather or social activities) [START_REF] Galindo | Toward a UI Adaptation Approach Driven by User Emotions[END_REF], we focus on these two static sources for stating the interest of the approach.

From these categorized users' emotions, it is then possible to identify if a problem exists in UI usability and/or aesthetics and to choose if some adaptation must be triggered. Lastly, these results are used to trigger adaptations in the prototype proposed in [START_REF] Galindo | Toward a UI Adaptation Approach Driven by User Emotions[END_REF].

Our long-term motivation is to propose a system able to adapt UI to emotions. The contribution of this work is a step toward this goal by making it possible to identify usability or aesthetics problems thanks to emotions.

The next section of the paper presents work related to affective computing and adaptation to emotions. Section 3 describes the experiment and its results. In section 3, we use these results to propose some triggers for UI adaptation. Section 4 presents a prototype to implement them. The paper ends with some conclusions and future work.

II. RELEATED WORK

In human-computer interaction, users' emotions are recognized as being important in the design and evaluation processes [START_REF] Hook | Affective Computing[END_REF]. They cannot be ignored while designing for UX [START_REF] Hook | Affective Computing[END_REF]. In this context, the field of Affective computing [START_REF] Picard | Affective computing[END_REF] aims at creating « machines that relate to, arise from, or deliberately influence emotion or other affective phenomena » according to [START_REF] Hook | Affective Computing[END_REF]. It provides models [START_REF] Ekman | An argument for basic emotions[END_REF], [START_REF] Russell | Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant[END_REF] and tools such as FaceReader 1 or Affdex2 to measure affects in terms of basic emotions from Ekman's model (surprise, happiness, sadness, disgust, anger, fear) [START_REF] Ekman | An argument for basic emotions[END_REF] plus neutral. It also studies emotions related to users' features such as learnability, performance, communication in some specific domains such as tutoring systems [START_REF] Forbes-Riley | Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system[END_REF]- [START_REF] Graesser | The relationship between affective states and dialog patterns during interactions with AutoTutor[END_REF].

Beyond models, emotions can be used at runtime in closed-loop applications [START_REF] Esposito ; Egon | Affective Signal Processing (ASP): Unraveling the mystery of emotions[END_REF]: emotions are measured and interpreted in terms of affects; then, based on the affects, a decision is made concerning the applicable actions; these actions are executed; and the loop starts again with emotions measures. In this way, adaptations can be triggered either on the functional part of the application [START_REF] Janssen | Tune in to your emotions: a robust personalized affective music player[END_REF] -e.g. for changing the content -or on the UI part -e.g. for adapting the design or the interaction modality. In our work, we focus on the UI adaptation.

Only few works [START_REF] Meudt | Going Further in Affective Computing: How Emotion Recognition Can Improve Adaptive User Interaction[END_REF]- [START_REF] Märtin | Designing Responsive Interactive Applications by Emotion-Tracking and Pattern-Based Dynamic User Interface Adaptation[END_REF] studies UI adaptation based on emotions even if the user's emotional state is cited as one of the elements of the context of use for UI adaptation [START_REF] Paterno | User Interface design adaptation[END_REF] . [START_REF] Meudt | Going Further in Affective Computing: How Emotion Recognition Can Improve Adaptive User Interaction[END_REF] proposes an approach which aims at identifying the cause of an emotion from the history of interactions and then at proposing an appropriate adaptation. In this work, interaction concerns humans and robots, which is a very specific kind of human-computer interaction domain, particularly in terms of usability and aesthetics. Moreover, the approach does not seem to have been implemented in systems. The ABAIS approach (Affect and Belief Adaptive Interface System) [START_REF] Hudlicka | Assessment of user affective and belief states for interface adaptation: Application to an Air Force pilot task[END_REF] follows user's anxiety while interacting with a complex air force system. In this work, adaptation is triggered by only one emotion (stress) and pilot's beliefs (e.g. hostile aircraft approach) in a specific context. The results cannot be reused in other contexts for more widespread UIs. Interestingly, the proposed adaptations can affect icons, displays, notifications and custom configuration which reflects both usability and aesthetics changes. But the distinction between usability and aesthetics problems is not realized as the goal is not to improve UX. Nasoz [START_REF] Nasoz | Adaptive intelligent user interfaces with emotion recognition[END_REF] proposed a more general approach for UI adaptation to emotions. It consists in implementing an adaptive virtual reality system relying on the recognition of affective states from physiological signals. It includes a user model with the following features: age, gender, recognized emotion (sadness, anger, surprise, fear, frustration, and amusement) and frequency of experiencing a specific emotion. From these data, an avatar mirroring the user can be shown and a Bayesian belief network can propose some actions adapted to user's emotions, like "make a joke" or "suggest to stop the car". Here a personalized avatar (skin colours, voices, hair, make-up, accessories) and a background are proposed based on user's preferences. Then adaptation is the mirroring of the user's current emotional state with the avatar design (face shape and orientation changes). The detected emotions are not related to the UI but to the driving situation. So, if the work proves that it is possible to trigger adaptations from several emotions, it does not explicit how detecting UI usability or aesthetics problems.

Finally, [START_REF] Märtin | Designing Responsive Interactive Applications by Emotion-Tracking and Pattern-Based Dynamic User Interface Adaptation[END_REF] proposes an architecture where automatically detected emotions can trigger UI adaptation based on patterns. Adaptation can be related to colours or structure and is performed at runtime. The approach is very interesting as it proposes an adaptation of UI aesthetics and usability based on emotions at runtime. It has very similar goals to ours. However, adaptations are based on one dominant emotion over a period of time. This emotion is compared with a dominant emotion determined at design time from a panel of users for each application. The way this comparison is realized is not made precise: the period of time and the threshold that triggers adaptation are not specified. There is also no evidence about how to identify the cause of non-expected emotions and then proposing appropriate modifications. To sum up, only few works studied UI adaptations based on emotions at runtime. None of them seems able to detect at runtime the cause of a UX problem in terms of usability or aesthetics and then to appropriately trigger UI adaptations. The next section presents an experiment, which allows us to identify usability and aesthetics problems based on age and gender at runtime. Then these results can be integrated in a prototype for UI adaptation proposed in section 4.

III. EXPERIMENTAL STUDY

One of the main challenges of UI adaptation to emotions is to identify, at runtime, emotions that must trigger adaptations. To address it, we conduct an experimental study, which allows us to better understand users' emotions based on their age and gender and to analyze it in regard with the level of usability and aesthetics. We take advantages of the seven emotions (anger, contempt, disgust, sadness, fear, neutral, happiness) detected from captured images by commercial tools like FaceReader in order to obtain a fine grain analysis and to detect negative emotions at runtime.

A. Goal and hypothesis

The experiment aims at identifying usability and/or aesthetics problems based on users' age and gender at runtime. To achieve this goal, it is necessary to detect fine grain emotions at runtime. From them, some categories of emotions, depending on users' profile as well as UI aesthetics and usability quality levels, can be identified. They represent some typical emotional behaviours, defined by emotions intervals. These intervals determine emotions thresholds from which it is possible to detect a problem. Our hypotheses are thus: H1: it is possible to categorize emotions depending on users' age and gender and on UI aesthetics and usability levels; H2: there exist emotions thresholds for detecting usability and aesthetics problems from users' age and gender.

B. Experimental protocol

Four UI versions of a travel website were designed and developed [START_REF] Dupuy-Chessa | Considering aesthetics and usability temporalities in a model based development process[END_REF]. The websites vary by two factors or UI qualities: usability and aesthetics. Consequently, the combination of two values (low vs. high) for these two factors (aesthetics and usability) released four versions of the website as evidenced in Fig. 1. [START_REF] Dupuy-Chessa | Considering aesthetics and usability temporalities in a model based development process[END_REF] shows the relevance of using such websites versions to identify the impact of usability and aesthetics on UX. These four used websites followed (1) usability and (2) aesthetics design modifications previously implemented by [START_REF] Dupuy-Chessa | Considering aesthetics and usability temporalities in a model based development process[END_REF]. First, usability was considered at the task level (e.g. interaction distance and workload) as well as at concrete user interface (CUI) level. The CUI was redesigned for the low usable version for increasing the workload by changing the UI components or the navigation easiness (e.g. increasing interaction distance or adding needless articulatory subtasks). Second, the aesthetics quality level (high vs. low) was achieved by modifications only in the graphic factors. As illustrated in Fig. 1 (e.g. website version number 1 vs number 2), this included the background-colour (bright vs. light), the framework shape (square vs. rounded), and the format style (non-aligned vs. aligned).

Each participant was assigned to one version of the travel website. While he/she was interacting with the website, his/her emotions were detected automatically based on captured user's photos in a time period of 10 seconds. Photos are sent to an emotion detection tool, Microsoft Emotions API 3 . This tool detects five negative (anger, contempt, disgust, sadness, fear) and one positive (happiness) emotions which were stated as universal by Ekman [START_REF] Ekman | An argument for basic emotions[END_REF]. It also detects neutral even if it is not part of the set of universal emotions, it is important to differentiate a calm (neutral) against an arousal user response (six Ekman emotions). This set of 7 emotions is recognized as a basic set proposed by many emotion detection tools based on images. Secondly, to ensure that participants are involved enough in the web interaction, they had to answer an online questionnaire after each task.

Three tasks are required for each participant: (1) look for a 5stars hotel which offers a visit in the desert, (2) look for a specific hotel with must include a breakfast plan and (3) look for a hotel circuit in a specific given location. For each participant, we registered the UI version used and his/her seven emotions detected every 10 seconds as an experimental time period. This period of time was chosen because a face macro-expression takes almost 4 seconds [START_REF] Mann | Emotion: All That Matters[END_REF] and it is thus expected that a 10-second-period is good enough to capture a proper emotional response. As some images are not good enough for being useable by the emotion detection tool, we finally obtain 1730 usable images with an average of 38 images by user.

Moreover, each image is marked with a timestamp, the UI in use (website version and current page) and the user's characteristics. We name an observation the tuple of data with the timestamp, the seven detected emotions (disgust, anger, fear, sadness, contempt, neutral and happiness), the current UI and the user's age and gender. An observation corresponds to an image, so we got 1730 observations with 11 data inside each tuple, giving a total of 19,030 data.

D. Data analysis

This section discusses the intermediary results while the next section presents results related to the two experimental hypotheses.

The data analysis process, illustrated in figure 2, started with a multivariable analysis ❶ called Principal Component Analysis (PCA) in order to find discrimination values in sociodemographic variables. Principal Components Analysis is a statistical analysis which extracts some significant factors that resume data information.

It uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called factors or principal components. Only significant factors are kept (eigenvalues greater than 1). Here PCA allowed us to observe a difference between men and women and between ages in emotions expression in accordance with previous studies [START_REF] Cyr | Gender and website design in e-business[END_REF]- [START_REF] Djamasbi | Generation Y, web design, and eye tracking[END_REF].

Then we divided the sample into 4 categories depending on gender and age ❷. For age, we aimed at keeping the same number of persons in each age category in order to have enough individuals in each category to conduce a statistical analysis. The threshold was 27 years old. So, the 4 categories are: women less than 27, women more than 27, men less than 27 and men more than 27.

From these categories, we identified similar emotion behaviours thanks to another PCA ❸ and a Hierarchical Cluster Analysis (HCA) ❹. A PCA analysis was realized to study the links between emotions by individual. It limits the variance quantity between individuals and removes data noise. PCA creates new coordinates (scores) of emotions for each observation.

Table 1 shows how two retrieved principal components (PC1 & PC2) in each case (women < 27, women >= 27, men < 27, men >= 27) contributes more than others based on the variance(σ 2 ). For instance, PC1 and PC2 reached in total 65% of variance while all other ones accumulated 35% in women less than 27. Similarly, the two components (PC1 & PC2) accumulated more variance in the three remain cases with 59, 71 and 70 per cent respectively. Furthermore, except from women over 27 years old, the two mentioned principal components (PC1 & PC2) were the most important ones by releasing standard deviation values (σX) higher than 1 for each case.

These new coordinates were computed with a Hierarchical Cluster Analysis ❹. The goal of this technique is to create some clusters sufficiently distinct from each other but with an intrinsic homogeneity. Here the Hierarchical Cluster Analysis produced significant partitions with 3 clusters for each case (women<27, women>27, men>27, men<27, see Table 2).

Then for each cluster, we studied the relationship between emotions detected and the website version used (low/high usability/aesthetics). We computed its mean and its confidence interval ❺ for each emotion detected and each website version. We obtained 7 box-plot for each cluster (Fig. 3).

By analysing these clusters, 4 of them appeared to have not enough discriminating behaviours (see Table 2, Women > 27 in clusters 1 and 3 and Men > 27 in clusters 2 and 3, which are the strikethrough numbers). More than 83% of women's scores who are more than 27 years old are in the second cluster, making the two other ones not enough representative. In the group of men over 27 years, one website version is not represented in the data of cluster 2 (website 4) and cluster 3 (website 1). So, we did not use these 4 clusters from our results analysis in order to keep relevant results.

E. Results

1) Hypothesis 1

Our first research question is related to the possibility to categorize emotions depending on users' age and gender and on UI aesthetics and usability levels. This first hypothesis is partly validated by our experimental results. Indeed, the identification of clusters by gender and age tends to show that it is possible to categorize emotions to obtain emotions behaviour depending on usability and aesthetics levels. However, 4 of our clusters are not relevant currently. More data would be required to show the relevance of all clusters. Anyway, the current relevant clusters are significant and can be used for hypothesis 2.

2) Hypothesis 2

The second hypothesis concerns the emotions thresholds for detecting usability and aesthetics problems from users' age and gender.

First of all, we need to identify emotions behaviours (in terms of one emotion or a combination of emotions) that can permit to identify a website version and so a usability and/or an aesthetics problem. For instance, we can see on Fig. 3 that the means for fear, disgust and contempt for website 3 (low usability and low aesthetics version, women under 27 years) are very different from the values obtained on the other websites for these same emotions. So, a low usability and a low aesthetics problem can be identified thanks to the detection of a certain level of fear, disgust or contempt for young women.

Secondly, we can see from Fig. 3 that each mean is given with a confidence interval. Here confidence intervals are of 95%, results can be reproduced in 95 percent of cases. This 5% of uncertainty can be used to determine the interval of values expected to determine a behaviour. For instance, the emotion behaviour studied previously is based on fear, disgust or contempt. The confidence interval for fear is between 8.79*10 -4 and 1.93*10 -3 (even if these numbers are close to 0, the difference is significant at 5% thresholds). These intervals provide us the emotions thresholds for identifying an UI problem. In our example, for women under 27 years, if fear is between 8.79*10 -4 and 1.93*10 -3 , or disgust between 4.11*10 -4 and 5.42*10 -4 or contempt between 8.10*10 -4 and 1.5*10 -3 , the corresponding website version can only be website 3. So, there are problems of aesthetics and usability.

To know if there is only a problem of usability (resp. aesthetics), we must study the behaviour of website 4, i.e., high aesthetics/low usability version, (resp. website 1, i.e., low aesthetics/high usability version). By analysing Fig. 3, we can provide conclusions for women under 27 years of age of cluster 1: • Website 1 (low aesthetics version) is characterized by the level of anger (between 2.84*10 -4 and 3.84*10 -4 ) AND the level of disgust (between 1.04*10 -4 and 2.16*10 -4 ). A problem with aesthetics can thus be identified if both emotions are detected between these levels.

• Website 4 (low usability version) can be identified by the level of disgust (between 2.5*10 -4 and 3.28*10 -4 ). So, a usability problem can be detected for young women, if emotions are inside these thresholds.

• Even if this does not help in identifying UI problem, we can notice that the version with high aesthetics and high usability (website 2) can be identified by the low level of anger (between 3.23*10 -5 and 6.88*10 -5 ).

We studied the 8 clusters in a similar manner. Table 3 for women and Table 4 for men summarize the results of this analysis. If it is the combination of several emotions values that discriminates, all emotions listed are required. This information is given by an "&" between emotions in the tables. Otherwise any of the listed emotions can be used to determine a problem.

We can notice that for men, there are some cases where no emotion data can discriminate an emotional behaviour. For men under 27 years old, for only one cluster, there is one behaviour that we cannot identify (cluster c2 and website3). For men older than 27 years, only one emotional behaviour is significant: this is the one related to problem in aesthetics and usability.

F. Discussion

The experiment allows us to partly validate our hypotheses: it shows that H1) it seems possible to categorize emotions depending on users' age and gender and on UI aesthetics and usability levels and H2) some emotions thresholds can be used for detecting usability and aesthetics problems from users' age and gender. As we choose confidence intervals of 95%, results can be reproduced in 95 percent of cases. Of course, an uncertainty of 5 percent remains. This is thus a first answer the lack identified in the related work: identifying potential issues to keep or improve the UI (H2) requires to establish links between UI quality factor (aesthetics and usability), emotions and user features (age and gender).

Although these results are encouraging, they must be mitigated as we designed this experiment considering very obvious differences between perceived aesthetics and perceived usability for distinguishing clearly these two factors. For instance, we carefully designed the low aesthetics version in order to avoid impacting also readability (guidance criteria) with inappropriate colours for labels. We also underline that: the tasks are non-critical and are without time and security constraints and that the experiment is limited to a single handcrafted web-site. Finally, we notice that users can be inconsistent in their rational and emotional thinking [START_REF]Cognitive dissonance: Progress on a pivotal theory in social psychology[END_REF], [START_REF] Festinger | A Theory of Cognitive Dissonance[END_REF], can feel misleading emotions [START_REF] Goldie | Explaining expressions of emotion[END_REF] and can vary in emotion arousal when facing the same stimuli [START_REF] Hume | Emotions and moods[END_REF]. It advocates for additional experiments to strengthen these results.

Lastly, the main limit of this work is that it does not allow us to fully cluster emotional behavior. Some clusters are not relevant and others do not discriminate enough emotions. Even if this is Table 3 -Thresholds and UI problems detected for women a limit, it does not challenge our approach. It simply implies that more data would be necessary to fully validate our hypotheses. However, results are currently good enough to be used in Perso2U, the prototype presented in the following section.

IV. INTREGRATION INTO PERSO2U

Perso2U is a prototype for adapting dynamically UIs to user's emotions.

A. Perso2U architecture UI adaptation is based on the choice of the most appropriate variant for the current context of use (user, environment, platform). A variant is a variation of an UI created for a specific context of use. For instance, there can be two different structures for a laptop and a smartphone, leading to two variants of the same UI.

Once a variant is chosen at runtime, some parameters can be applied to it. UI Parameters define a set of variables which personalize UI components (e.g., font-size, widgets, audio use or level, display and dialogues) regarding with identified context values (e.g., user's emotions). For instance, if the user's emotions are in the thresholds that correspond to an aesthetics problem then the UI parameter named background-color could be set to light-yellow. Then, these variables are sent to the displayed UI for applying the corresponding changes. Fig. 4 exemplifies this by showing different UI parameters applied to the same UI: here parameters are color and font size.

Based on the principles of variants and parameters, we proposed an architecture (Fig. 5) to adapt, at run-time, UI to the context of use including users' emotions [START_REF] Galindo | Toward a UI Adaptation Approach Driven by User Emotions[END_REF]. This architecture is based on three components: the Inferring Engine , the Adaptation Engine and the Interactive System . An automatic adaptation process starts from either (a) a need for a new UI to be displayed or (b) a change in the context of use. (a) can be exemplified by the user entering a web site: the home page has to be displayed. An example of (b) is the ambient light: when it increases, the contrast on the UI might be increased as well. In (b), the overall process is the following: the Inferring Engine monitors sensors to detect changes in the context of use. From these values, it deduces the new context of use dynamically. It embeds an Emotion Wrapper which provides emotions values, coming from emotion detection tools ❹ and included in the user model. The Inferring Engine sends the computed context of use to the Adaptation Engine , which elicits accordingly a suitable UI variant and the UI parameters for personalising it. Finally, the Interactive System displays the variant and executes the changes related to the UI parameters. The whole process runs cyclically by following a time period defined by the designer.

B. Extension of the Perso2U inferring engine

The experiment described in section 3 shows that it can be possible to detect a negative emotion behaviour depending on user's gender and age. The Inferring Engine is in charge on computing this behaviour. The calculation relies on the user's emotions -detected at runtime by emotion detection tool and used as dynamic variables -and his/her age and gender -which can be declared by the user or deduced from pictures by the emotion detection tool and are used as static variables. Then the Inferring Engine sends the context of use with the user's emotions to the Adaptation Engine for adapting UI appropriately.

But to propose a relevant personalisation to emotions, the Adaptation Engine must also know if the problem is related to UI aesthetics or usability. So, the Inferring Engine must also send the identified problem source to the Adaptation Engine. This means that our prototype must be modified to add a parameter, the problem source that we name UI-problem, from the Inferring Engine to the Adaptation Engine. From the experimental results, we can propose some inference rules about UI quality, that provide emotions (+, -, neutral) and the eventual problem.

Let's take as example: the case of women under 27 years of age for which the experimental results show that if fear is between 8.79*10 -4 and 1.93*10 -3 , or disgust between 4.11*10 -4 and 5.42*10 -4 or contempt between 8.10*10 -4 and 1.5*10 -3 , there are problems of aesthetics and usability. As any of the emotions detected in these intervals can discriminate the aesthetics and usability problem, the conditions about emotions thresholds are thus linked by an "or". When discrimination requires the combination of several emotions, conditions are linked by an "and". This is the case for the level of anger (between 2.84*10 - 4 and 3.84*10 -4 ) and the level of disgust (between 1.04*10 -4 and 2.16*10 -4 ) that are both needed for discriminating a problem of aesthetics for women younger than 27 years old.

The implementation of this case in Perso2u is realized in two steps. First, Fig. 6 shows the thresholds definitions inside the inferring engine for women under 27 years. This section serves to include all needed thresholds variables previously defined in Tab. 2 and 3 without making distinction between clusters.

Secondly, a new function checkUIquality was coded to identify UI quality problem from users' current detected emotions as illustrated in Fig. 7. Here, the Inferring Engine reads the static (gender, age) and dynamic (emotions) variables to make a comparison between the predefined thresholds and the current emotions. The conditions follow the criteria defined in Table 2 cluster 1. The implementation of the rules for inferring negative emotions contains a lot of conditions which correspond to the verification of the thresholds defined by the statistical analysis. However, the implementation removes the clusters of the analysis, as the Inferring Engine cannot know in advance which cluster a person belongs to. So, we need to test carefully if the results of the Inferring Engine are correct.

The test aims at verifying if a correct emotion and UI issue is provided to the Adaptation Engine by using the inference rules. The inference rule for women less than 27 years old was tested with 105 images of three users. The test was done by comparing the expected and observed values (inferred and sent to the Adaptation Engine). These values include the inferred emotion (negative, positive) and the UI issue: aesthetics (A), usability (U) or aesthetics & usability (AU) or not available (n.a.) when a negative emotion is not found. To illustrate the results of the test of the inferring rule algorithm, Table 5 shows ten results for one user. The "Test results" column shows if the observed value matches the expected one (OK) or not (not OK). For instance, for image #1 in Table 4, has expected values {negative, A} (emotion is negative, there is an aesthetics problem), while its observed values are {negative, A}, that correspond exactly to the expectations.

The tests allowed us to identify 4 errors corresponding to thresholds badly reported in tables 3 and 4. The code of the inferring rules was corrected accordingly. Now the code is robust and the Inferring Engine is able to deduce a negative emotion and UI issue according to Tables 3 and4.

V. CONCLUSIONS AND PERSPECTIVES

This paper presents the results of an experiment and states that 1) it can be possible to categorize emotions depending on users' age and gender and on UI aesthetics and usability levels and that 2) some emotions thresholds can be defined for detecting usability and aesthetics problems from users' age and gender. These results are used in a UI adaptation system to infer negative emotions and UI problems based on users' emotions, age and gender. Adaptations triggered by emotions are expected to improve UX.

Other experiments could strengthen the current results. An experiment, similar to the one described in this paper with more participants, could be realized to characterize other users' clusters. This would permit to obtain a more precise adaptation system. Another experiment could validate the proposed inference rules.

It would also be interesting to strengthen the statistical analysis to obtain a statistical model, which would allow us to obtain a more precise determination of UI problems. Indeed, a logistic regression can estimate the probability being in front of a type of website (e.g. low usability or low aesthetics) depending on the gender, the age and the detected emotions. This model is interesting because it makes it possible to obtain the level of confidence of the prediction, while in our rules the thresholds are at 5% level. The statistical model is more precise than the rules model. Currently, our prototype proposes simple adaptations (colour and text font) to emotions without referring to relevant work in psychology concerning preferences by gender or age. Coupling this existing knowledge with our system would allow us to provide a more relevant adaptation system. Moreover, our experimental results can also be used in UI evaluation as they allow a system to detect some usability or aesthetics problems. This opens a large spectrum of possibilities for UI evaluation based on emotions. However even if lots of questions remain opened, this work is a first step toward providing better UX thanks to emotions.

  3 www.microsoft.com/cognitive-services/en-us/emotion-api C. Participants and interaction data 45 persons participated to the study, with an average of approximately 11 participants per website and a distribution of: 13 (website1-low aesthetics/high usability), 10 (website2 -high aesthetics/high usability), 12 (website3 -low aesthetics/low usability), and 10 (website4 -high aesthetics/low usability). The experiment sample included 53% (24) male from 19 to 67 years old and 47% (21) female from 23 to 63 years old participants (mean male age 34, σ=11.66 and mean female age 27 with σ=12.25).

Figure 1 .

 1 Figure 1. Four versions of a web site user interface.

Figure 2 .

 2 Figure 2. Research process for data analysis.

Figure 3 -

 3 Figure 3-Emotion means and standard deviations for cluster 1 for women under 27 years

Figure 5 .

 5 Figure 5. Global schema of the Perso2U architecture.

Figure 4 .

 4 Figure 4. Parameter changes

Table 2 -

 2 Number of scores in the HCA process

		Cluster 1	Cluster 2	Cluster 3
	Women < 27	216	259	304
	Women > 27	74	508	28
	Men < 27	337	558	310
	Men > 27	422	259	97

Table 1 :

 1 Standard deviation and variance values for each principal component across women and men clusters.

		women < 27		women >= 27
		σX	σ 2	σX	σ 2
	PC1	1,66	0,40	1,58	0,36
	PC2	1,33	0,25	1,28	0,23
	PC3	0,92	0,12	1,05	0,16
	PC4	0,86	0,11	0,84	0,10
	PC5	0,63	0,06	0,76	0,08
	PC6	0,49	0,03	0,56	0,05
	PC7	0,47	0,03	0,41	0,02
		men < 27		men >= 27
		σX	σ 2	σX	σ 2
	PC1	1,96	0,55	1,93	0,53
	PC2	1,07	0,16	1,09	0,17
	PC3	0,85	0,10	0,96	0,13
	PC4	0,79	0,09	0,79	0,09
	PC5	0,67	0,06	0,58	0,05
	PC6	0,39	0,02	0,33	0,02
	PC7	0,24	0,01	0,31	0,01

σX: standard deviation and σ 2 : variance

Table 4 -

 4 Thresholds and UI problems detected for men

	User category	Clus ter	Website version	emotions	Thresholds (min and max)	Problem
				anger	3.97*10 -2 and 6.46*10 -2
			website1	disgust	1.46*10 -3 and 2.03*10 -3	aesthetics
				happiness	6.32*10 -4 and 2.41*10 -3
		c1	website3	happiness & neutral	2.68*10 -4 and 4.97*10 -3 9.05*10 -1 and 9.24*10 -1	aesthetics and usability
			website4	anger sadness	4.45*10 -4 and 6.17*10 -3 1.20*10 -1 and 1.55*10 -1	usability
			website1	anger	1.01*10 -3 and 3.57*10 -3	aesthetics
	Men	c2	website3		no discriminant values
	younger than 27		website4 contempt	1.31*10 -4 and 4.18*10 -3	usability
				fear	1.46*10 -1 and 2.66*10 -1
				anger	1.83*10 -2 and 3.27*10 -2
			website1	disgust	6.45*10 -3 and 1.09*10 -2	aesthetics
				neutral	5.82*10 -1 and 6.38*10 -1
		c3		contempt	3.30*10 -3 and 6.54*10 -3
				fear	1.7*10 -2 and 3.09*10 -2
			website3	happiness	1.01*10 -2 and 4.07*10 -2	aesthetics and usability
				neutral	6.98*10 -1 and 7.48*10 -1
			website4	sadness	4.06*10 -1 and 4.06*10 -1	usability
			website1		no discriminant values
	Men older than 27	c2	website3	anger	1.85*10 -1 and 3.55*10 -1	aesthetics and usability
			website4		no discriminant values

www.affectiva.com/solutions/affdex/

ACKNOWLEDGMENT

This work has been supported by the Labex PERSYVAL-Lab in the University Grenoble-Alpes.

Of course, other conditions are required for the other behaviours of Tables 2 and3. Every condition results from the deduction of two variables: the user emotion (negative) and the UI quality problem (aesthetics, usability or aesthetics & usability). Finally, user emotion and UI-problem are stored in the context of use (user model).

Currently, detected emotions are used to identify some UI problems and for sending negative emotions to the Adaptation Engine. All emotions data that are not in the identified thresholds are considered as positive. A more precise algorithm could use the discriminant values for the website with high aesthetics and high usability to detect positive emotions and keep the neutral values for all data that do not permit discrimination. For instance, in section 3, we have identified that the website with high aesthetics and high usability can be discriminated by a low level of anger for a cluster of women under 27 years of age. So, the low level of anger could be interpreted as a positive emotion. Currently positive emotions are not used by the adaptation engine and no distinction is made between positive and neutral emotions.

The execution of the proposed implementation in the global architecture (Fig. 5) produces the following process. First, user's gender and age are stored in the context of use in the user model. Then, while the user interacts with the UI, the emotion wrapper retrieves the detected emotions from the emotion recognition tool (Microsoft Emotion API). From detected emotions, the Inferring Engine calls the function checkUIquality to look for predefined negative behaviours. Then, if a negative behaviour is detected (i.e. if emotions values followed the conditions exposed in Table 3), user emotions and UI-problem parameters are modified and sent with other contextual elements to the Adaptation Engine. The Adaptation Engine reads the UI-problem and the user-emotions and can trigger an appropriate UI adaptation.


Fabrice Zaoui

A GPU-enabled finite volume solver for large shallow water simulations

Keywords: GPU computing, 2D shallow water problems, Unstructured grids

HAL is

Introduction

Models based on Shallow Water Equations (SWE) are useful in Computational Fluid Dynamics. They can simulate a wide range of geophysical flows like rivers, channels, estuarine and coastal circulations, dam breaks, etc. These phenomena occur in large domains and may require long simulation time for a real study. Consequently the computation times can be extremely high if an efficient solver is not implemented. Fortunately, the numerical solution of the 2D hyperbolic systems of conservation laws SWE can be easily implemented on parallel machines to keep the execution times acceptable. The parallelism is mainly concerned with the domain decomposition method on cluster, multithreaded programs and vectorization techniques.

Most of the codes solving 2D SWE in parallel are historically based on CPUs. Even if the GPU architecture is made for massive parallelism, its use to obtain the solution of non-graphics applications like mathematical problems in science and engineering is more recent, and requires a very different programming model from that of CPU [START_REF] Lastra | Simulation of shallow-water systems using graphics processing units[END_REF][START_REF] Brodtkorb | Efficient shallow water simulations on GPUs: Implementation, visualization, verification, and validation[END_REF][START_REF] Vacondio | GPU-enhanced finite volume shallow water solver for fast flood simulations[END_REF]. When a code is developed and optimized to benefit from the large memory bandwidth and Floating Point Operations Per Seconds (FLOPS) of the GPU architecture, simulations can run significantly faster than a CPU model [START_REF] Kalyanapu | Assessment of GPU computational enhancement to a 2D flood model[END_REF].

In this paper, NVIDIA Quadro cards are used to implement the solution of 2D SWE with the finite volume method on unstructured grids [START_REF] Castro | GPU computing for shallow water flow simulation based on finite volume schemes[END_REF]. Performances are compared with a mono-core CPU version of the same algorithm.

Method

Mathematical model

The conservative form of 2D SWE is:

𝜕𝐔 𝜕𝑡 + 𝜕𝐆(𝐔) 𝜕𝑥 + 𝜕𝐇(𝐔) 𝜕𝑦 = 𝐒(𝐔) 𝑖𝑛 Ω 𝑡 × [0, 𝑇 𝑠 ] (1

]

where ℎ = 𝜂 -𝑧 is the water depth (𝑚), 𝜂 is the free surface (𝑚), 𝑧 is the bathymetry (𝑚), 𝑢 and 𝑣 are the velocity components in the 𝑥 and 𝑦 directions (𝑚 𝑠) ⁄ , 𝑔 is the acceleration due to gravity (𝑚 𝑠 2 ) ⁄ , 𝑛 is the Manning roughness coefficient (𝑠 𝑚 1 3 ⁄ ) ⁄ and 𝑇 𝑠 is the simulation time (𝑠).

Discretization

The computational domain is divided into a set of triangles and the partial differential equation ( 1) is solved with a cell-centered finite volume method. This implies the discretization of the SWE in integral form:

𝜕 𝜕𝑡 ∫ 𝐔 Ω dΩ + ∫ 𝛁 • 𝐅 Ω dΩ = ∫ 𝐒dΩ Ω (2)
where 𝐅(𝐔) = (𝐆(𝐔), 𝐇(𝐔)) 𝑇 and Ω denotes the volume of the cell 𝑖. The application of the divergence theorem for a triangular cell gives: Assuming the state vector 𝐔 to be piecewise constant over cells and a simple explicit Euler scheme for the time discretization, the solution for a new time step 𝑛 + 1 is computed as:

∫ 𝛁 • 𝐅 Ω dΩ = ∮ 𝐅(𝐔) Γ • 𝐧dΓ = ∑ 𝐅 𝑘 (𝐔) • 𝐧 𝑘 𝑙 𝑘 3 𝑘=1 (3)
𝐔 𝑖 𝑛+1 -𝐔 𝑖 𝑛 ∆𝑡 = - 1 Ω 𝑖 ∑ 𝐅 𝑘 (𝐔) • 𝐧 𝑘 𝑙 𝑘 3 𝑘=1 + 𝐒 (4)
The time step ∆𝑡 must be selected carefully to maintain the stability of the scheme. Therefore the Courant-Friedrichs-Lewy (CFL) stability condition is applied with a Courant number equal to 0.7.

The HLLC approximate Riemann solver [START_REF] Toro | Restoration of the contact surface to the Harten-Lax-van der Leer solver[END_REF] is used to compute the numerical fluxes 𝐅 𝑘 (𝐔) of mass and momentum at interfaces. Source terms are treated with a hydrostatic reconstruction for the topography 𝐒 𝟎 and a semi-implicit method for the friction 𝐒 𝐟 .

Implementation

The implementation of algorithm ( 4) is written in the C programming language for CPU and duplicated in CUDA C for GPU. CPU has to perform in a wide range of applications. It is traditionally optimized for singlethread performance with a big cache memory, sophisticated control logic and high clock frequency. GPU is specialized for highly parallel tasks with its high number of processing cores optimized for a multithreaded execution. The best performance for CUDA code is expected if the number of data transfer is minimized between HOST and DEVICE through the PCI express bus, see Figure 2. Consequently, the loop time is on the HOST with calls for calculation on DEVICE. Transfer copy is limited to the initial data structure from the input data (copy from HOST to DEVICE) and to output calculation results at 𝑡 = 𝑇 𝑠 (copy from DEVICE to HOST).

Functions to compute (4) are executed by many threads on GPU. The threads are grouped in grids of blocks that run in parallel to compute the numerical flux 𝐅 𝑘 (𝐔) and to update the new state 𝐔 𝑛+1 . These two steps are mainly concerned by a parallel loop over the number of interfaces and cells respectively. The algorithm of the finite volume solver is verified on a set of well-known benchmark cases. For example, Figure 4 shows the water depth evolution of a dam-break flooding over three mounds. This case considers a non-trivial wetting and drying process.

Results of the comparison tests

Four cards of the NVIDIA Quadro line are tested, see Table 1. These GPUs are professional graphics solutions. They are compared with the INTEL Xeon CPUs working in a sequential mode. These CPUs are Xeon E3 3.2 GHz for the Quadro K600, Xeon E3 3.4 GHz for Quadro K2000, and model E5 2.6 GHz for K5200/K6000. The test case is a water drop in a large basin. The fluid is initially at rest with a Gaussian free surface at the center of a large square domain with reflective conditions on all the boundaries. The longest studied simulation time 𝑇 𝑠 is equal to 2,400 s. Comparison tests are carried out on unstructured grids of different sizes ranging from 1,000 to 10,000,000 cells. Table 2 gives information about sizes of the grids. The tests are run with a double precision representation. Final results for CPU and GPU at 𝑡 = 𝑇 𝑠 are verified as being very close.

The CPU/GPU ratios of computation times are shown in Figure 6 for the first four tests. As the necessary amount of memory required by the test number #5 is large, it can only be run on the GPUs K5200 and K6000 and is not shown in Figure 6. Considering the mean speed-up offered by each GPU for the first four tests, a ranking can be established as shown in Figure 7. This can be interpreted as a work distribution according to individual capacity. K5200 and K6000 are the only ones GPUs able to load the problem test #5 in memory as it requires more than 2 GB of RAM. The performance of these two graphics cards are important for large grids as shown in Figure 9. The speed-up can also depends on the simulation time 𝑇 𝑠 . Figure 10 shows for the model K6000, how the speed-up varies with the simulation time.

Discussion

The exploitation of the parallel processing power of GPU is interesting in comparison to a single-core GPU if the problem size of the finite volume solver for shallow water systems is significant. This minimum size of interest lies in the range starting at 10,000 cells for high performance graphics card like the Quadro K5200/K6000 to 100,000 cells for the entry-level K600.

The maximum speed-up of the execution time is also bounded by the model of GPU. Its value is approximately equal to × 3 for the Quadro K600 to more than × 40 for K6000. This confirms that the performance highly depends on the model of the graphics card. Nevertheless, for all the simulations times and grids, the speed-up factor remains more dispersed for high-end GPUs if one takes a look at the statistical variability as shown in Figure 11. All the studied GPU in this paper come from the NVIDIA Quadro line and mainly differ by the following technical specifications: processing cores, RAM and memory bandwidth. Whereas the amount of RAM is important for dealing with large size problems, the number of processing cores and the rate at which data can be transferred to or from memory are important for the overall performance. The joint analysis of Table 1 and Figure 7 exhibits the direct relationship between performance and technical specifications, as shown in Figure 12. In Figure 10, the speed-up depends on the value of the simulation time with largest values for longest times except for one point. This behavior still needs to be investigated on a larger variety of problems in order to conclude.

Conclusions

This paper presents a parallelization of SWE for GPU under the NVIDIA CUDA framework. GPU computations are compared with a sequential approach on CPU. Several comparison tests are done involving different grid sizes, simulation times and GPU models. GPU can significantly speed-up the simulations according to the choice of computational parameters.

More tests still have to be done to investigate the influence of the simulation time on the speed-up. And finally, a logical continuation of this work would also compare the multi-GPU approach vs multi-CPU.

Fig. 1 .

 1 Fig. 1. Control volume Ω 𝑖 of cell-centered scheme.

Fig. 2 .

 2 Fig. 2. CUDA source code can address HOST (CPU) and DEVICE (GPU).

Fig. 3 .

 3 Fig. 3. Flowchart of parallel computing.

Fig. 4 .

 4 Fig. 4. Topography, mesh and water depth evolution of the three-mound problem.

Fig. 5 .

 5 Fig. 5. Initial state of the water drop.

Fig. 6 .

 6 Fig. 6. Relative performance assessment on processors.

Fig. 7 .

 7 Fig. 7. Relative computing capacity.

Fig. 8 .

 8 Fig. 8. GPU Execution times for test number #4.

Fig. 9 .

 9 Fig. 9. Execution times for test number #2, #3, #4 and #5.

Fig. 10 .

 10 Fig. 10. Speed-up as a function of the simulation time.

Fig. 11 .

 11 Fig. 11.Dispersion of results on the speed-up factor.

  Fig. 12. GPU performance as a function of the technical specifications.

Table 1 .

 1 GPU solutions for the tests.

	Model name	K600	K2000	K5200	K6000
	Number of processing cores	192	384	2,304	2,880
	TeraFLOPS (Single precision, Peak)	0.34	0.73	3.5	5.1
	Memory (GB)	1	2	8	12
	Memory bandwidth (GBps)	29	64	192	288

Table 2 .

 2 Unstructured grids.

	Test number	1	2	3	4	5
	Number of cells (triangles)	1,036	10,132	104,788	1,056,518 10,261,932
	Number of edges	1,588	15,310	157,542	1,585923	15,396,468
	Number of nodes	553	5,179	52,755	529,406	5,134,537