Imen Bizid
email: imen.bizid@univ-lr.fr

Nibal Nayef
email: nibal.nayef@univ-lr.fr

Patrice Boursier
email: patrice.boursier@univ-lr.fr

Sami Faiz
email: sami.faiz@insat.rnu.tn

Jacques Morcos
email: jacques.morcos@univ-lr.fr

Prominent Users Detection during Specific Events by Learning On-and Off-topic Features of User Activities

Keywords: user identification, On-and off-topic features, Events in Twitter, SVM Classification and Ranking, Real-time feature computation

Microblogs such as Twitter are characterized by the richness and recency of information shared by their users during major events. However, it is very challenging to automatically mine for information or for users sharing certain information due to the huge variety of unstructured stream of data shared in such microblogs. This work proposes a ranking and classification model for identifying users sharing useful information during a specified event. The model is based on a novel set of features that can be computed in real time. These features are designed such that they take into account both the on and off-topic activities of a user. Once users are characterized by a feature vector, supervised machine learning tool is trained to classify users as either prominent or not. Our model has been tested on data shared during a flooding disaster event and performed very well. The achieved results show the effectiveness of the proposed model for both the classification and ranking of prominent users in such events, and also the importance of the adjustment of the on-topic features by the off-topic ones when describing users' activities.

I. INTRODUCTION

Microblogging platforms offer the services of convenient access to and sharing of fresh data on any topic. Shared data is usually limited to a specified number of characters, for example, tweets are limited to 140 characters. This results in a huge stream of unstructured data, which makes information retrieval within such data very challenging, specially when having to perform such tasks in real time.

Having the aforementioned particularities of microblogs in mind, many research works have associated the relevance and the quality of the shared messages in microblogs with the user's prominence in terms of the network and the targeted topic but not in terms of messages' content [START_REF] Wagner | It's not in their tweets: Modeling topical expertise of twitter users[END_REF], [START_REF] Liao | Understanding experts' and novices' expertise judgment of twitter users[END_REF].

Those work have focused on the identification of influencers [START_REF] Romero | Influence and passivity in social media[END_REF], [START_REF] Chen | Efficient influence maximization in social networks[END_REF] and domain experts [START_REF] Xianlei | Finding domain experts in microblogs[END_REF], [START_REF] Bozzon | Choosing the right crowd: Expert finding in social networks[END_REF] in microblogs. The identification of the latter type of users is usually based either on their centrality and popularity in the network or on their frequent networking activities domains. From another aspect, domain experts are generally identified by analyzing their historic information regarding a topic of interest. However, the prominence of microblogs users during a specific event cannot be evaluated according to the user's centrality or prior activities in the network.

The features characterizing prominent users should reflect the nature of a user's behavior during such events. Consider the case of a disaster where alerts and emergency states are being rapidly updated over time. The prominent users -whom we are interested in -would focus their attention and communication mostly on the topical information related to the disaster. These users are not necessarily experts in disasters, they may be ordinary microblogs users geolocated in the disaster area and who are sharing what they are seeing and experiencing. Hence, they would share a lot of exclusive information. While users such as media channels toggling between several topics would share many event-related information which are already diffused in the network. Thus, features based on traditional metrics such as the number of shared on-topic tweets cannot be directly used to identify user's prominence. Therefore, identification models which are based on traditional metrics would be sensitive to users interested in several topics and sharing outdated information.

In this paper, we propose a model that alleviates these shortcomings first by designing a set of metrics which represent Twitter users interested in a specific event. These metrics characterize each active user by considering both the on-and off-topic activities of the user during the event. Based on those metrics, our list of features promote users who focus only on the event under consideration, and penalize those who are toggling among several topics. Using these representative features, we use supervised learning to train an SVM model to identify the most prominent users in real time during an event.

The rest of this paper is organized as follows: Section II reviews related work. Section III presents the set of our proposed features used to model microblogs' users. Section IV summarizes the classification and ranking approach employed to identify and detect the best prominent users. Section V presents the experiments and results obtained by our model. Section VI concludes the paper and discusses future work.

II. RELATED WORK

To the best of our knowledge, the issue of prominent users' identification has never been explored in the context of specific events. However, there have been several attempts proposing new measures identifying influential users and domain experts in microblogs and specially Twitter [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF], [START_REF] Freeman | Centrality in social networks: ii. experimental results[END_REF].

As the identification of social networks influencers problem has been potentially associated with the global issue of central users finding in any kind of network, most of the proposed approaches identifying social media influencers are based on standard centrality measures such as PageRank [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF] and HITS [START_REF] Agichtein | Finding high-quality content in social media[END_REF]. These measures have proved their potential to identify influencers according to their social position in microblogs. However, they are still sensitive to celebrities and news channels that have a large number of followers. Moreover, microblogs are richer than simple links relating users.

Therefore, many approaches have adapted PageRank and Hits algorithms to the specificities of microblogs (e.g. the number of shared tweets, number of mentions and number of retweets etc.) [START_REF] Cappelletti | Iarank: Ranking users on twitter in near real-time, based on their information amplification potential[END_REF], [START_REF] Silva | Profilerank: Finding relevant content and influential users based on information diffusion[END_REF]. TwitterRank [START_REF] Weng | Twitterrank: Finding topicsensitive influential twitterers[END_REF] measures the user's influence based on both the topical similarity between users and their link structure. IP-influence [START_REF] Romero | Influence and passivity in social media[END_REF] weights the edges by taking into account microblogs features related to user's passivity and influence rates and measures the user's influence using HITS algorithm. This approach is similar to KHYRank [START_REF] Zhang | Identifying influential users of microblogging services: A dynamic action-based network approach[END_REF] which uses the number of retweets and mentions to weight the graph. While these adapted PageRank and Hits algorithms have yielded better results for influencers identification, their iterative process is still time consuming and unsuitable for the identification of prominent users during an event.

Few prior studies have explored the detection of topical authorities or domain experts in real-time scenarios [START_REF] Pal | Identifying topical authorities in microblogs[END_REF], [START_REF] Xianlei | Finding domain experts in microblogs[END_REF], [START_REF] Ghosh | Cognos: Crowdsourcing search for topic experts in microblogs[END_REF]. Cognos system [START_REF] Ghosh | Cognos: Crowdsourcing search for topic experts in microblogs[END_REF] is built for finding domain experts in Twitter by mining Twitter lists created by individual users to include experts on topics. Xianlei et al. [START_REF] Xianlei | Finding domain experts in microblogs[END_REF] proposed a Gradient Boosted Decision Tree to identify domain experts in Sina Microblog over state-of-the-art [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] and new linguistic features. Pal et al. [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] proposed a clustering and ranking procedure based on a set of features including nodal and topical metrics characterizing microblog user's activities.

All these studies have focused on measuring the attachment of microblogs users only for specific analyzed topics while neglecting their activities on other topics. These approaches are sensitive to active users interested in several topics and who are not necessarily sharing updated or detailed information about the topic of interest. Whereas in this paper, we propose a new model built based on features that describe the user's activities on both the analyzed topic and the other ones. This model is capable of identifying prominent users in specific and unexpected events such as natural or human disasters.

III. DESCRIBING USERS BY THEIR ON-AND OFF-TOPIC ACTIVITY FEATURES

In order to build a classification model for identifying prominent users interacting about an event, we propose a set of new features to describe and characterize users and their behavior during that event. These features are composed of a set of metrics computed according to both the on-and off-topic activities of microblogs users during the event.

On-topic: An activity is considered on-topic when it contains a subset of a list of keywords and hashtags which are defined to describe the event under consideration. Off-topic: an off-topic activity refers to any activity that was not recorded as an on-topic one.

Nevertheless, if a captured tweet referring to the event includes some keywords reflecting non-serious or non-valuable contents (e.g. advertising or joke words and symbols such as sale, rent, pub, lol and so on), it will be directly recorded as an off-topic one. Our rationale behind the extraction of on-topic and off-topic activities is based on penalizing users who are toggling among several topics, and who may share outdated information.

A. Characterizing of Users' Activities

Users' activities in Twitter falls into three types: original tweets are tweets originally expressed by a user, retweets are tweets already shared by someone else in the network and forwarded later by a user and mentions are tweets directed to particular users mentioned using the @ symbol.

Table I shows the complete list of set of our proposed and the state-of-the art metrics used in this work to describe the different user's activities during a specific event. [START_REF] Boyd | Tweet, tweet, retweet: Conversational aspects of retweeting on twitter[END_REF] + + Mentions M1 : Number of mentions of other users by the author [START_REF] Honey | Beyond microblogging: Conversation and collaboration via twitter[END_REF], [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] + + M2 : Number of unique users mentioned by the author [START_REF] Honey | Beyond microblogging: Conversation and collaboration via twitter[END_REF], [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] + + M3 : Number of mentions by others of the author [START_REF] Honey | Beyond microblogging: Conversation and collaboration via twitter[END_REF], [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] + + M4 : Number of unique users mentioning the author [START_REF] Honey | Beyond microblogging: Conversation and collaboration via twitter[END_REF], [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] + + Graph G1: Number of active followers [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF] + + G2: Number of active followees [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF] + +

B. Building a User-level Feature Vector

Inspired by the features presented in Pal et al. [START_REF] Pal | Identifying topical authorities in microblogs[END_REF], we propose a new list of features to represent users by adjusting their on-topic activities by their off-topic ones. The features are based on the list of metrics presented in Table I. In the following, we explain those features in detail: Topical Strength: estimates the value (or worthiness) of the author's topical tweets with respect to the off-topic ones. F 1 promotes users that have collected more favorite points regarding their on-topical tweets than off-topical ones.

F 1 = T 4 on T 4 of f + 1 (1)
Topical Attachment: indicates the involvement rate of the user regarding the analyzed topic by referring to the number of his original on-topic tweets adjusted by the off-topic ones.

The more a user produces on-topical tweets compared to offtopical ones, the higher his Topical Attachment score is.

F 2 = T 1 on + T 2 on T 1 of f + T 2 of f + 1 (2)
Retweeting Rate: measures the impact of the original tweets shared by other users on the user's topical activities. This measure is adjusted by the retweeting activity of the user regarding off-topic original tweets.

F 3 = R1 on * log(R2 on + 1) -R1 of f * log(R2 of f + 1) (3)
Retweeted Rate: calculates the impact of the topical original tweets produced by the author on other network users. This feature is adjusted by the user's influence rate on other topics.

F 4 = R3 on * log(R4 on + 1) -R3 of f * log(R4 of f + 1) (4)
Incoming Mention Rate: measures the diversity of mentions that the user has received regarding the specific topic. This measure is adjusted by the flow rate of off-topic mentions intended to the user.

F 5 = M 3 on * log(M 4 on + 1) -M 3 of f * log(M 4 of f + 1) (5)
Outcoming Mention Rate: promotes users producing many on-topic mentions intended to several users on the one hand and penalizes users having more off-topic mentions addressed to different users than on-topic ones on the second hand.

F 6 = M 1 on * log(M 2 on + 1) -M 1 of f * log(M 2 of f + 1) (6)
Centrality Degree: adjusts the number of on-topic followers and followees of each user with the number of his off-topic relations. This feature promotes users having more on-topic followers than off-topic ones.

F 7 = log( G1 on + 1 G1 of f + 2 ) -log( G2 on + 1 G2 of f + 2 ) (7)
These hand-crafted features combine different metrics which can be extracted in real time from users' time-line using Twitter APIs. Using these features, we offer a better representation of users by combining different metrics which have significant relations among themselves.

Thus by computing the above described features, we model each user by the following feature vector composed of seven features describing his on-and off-topic activities.

x i = (F 1 i , F 2 i , F 3 i , F 4 i , F 5 i , F 6 i , F 7 i , T 5 i ) (8)

IV. CLASSIFICATION AND RANKING OF PROMINENT USERS

To identify the prominent users from within the huge number of users that may be interacting during a specific event, we model this problem into a binary classification problem (i.e. 1 for prominent users or -1 for non-prominent ones). We use a supervised learning method in order to build our classification model. The role of the classification is to detect the prominent users and reject the others. Then, in the ranking stage, we mainly focus on identifying the top prominent users regarding the specific event or topic under consideration.

V. PERFORMANCE EVALUATION

A. Dataset

To conduct experimental performance evaluation on real data, we collected most of the tweets shared during the floods that have occurred from 29 th to 30 th September 2014 in the Herault area, situated in the south of France. Data collection was processed using our multi-agent System called MASIR [START_REF] Bizid | Masir : A multiagent system for real-time information retrieval from microblogs during unexpected events[END_REF]. At the lowest level, the system detects the different users who have shared at least one on-topic tweet (i.e. talking about the floods) during the analyzed period. On-topic tweets are detected using the hashtags and keywords which were employed by Twitter users to share information about the disaster. The system then crawls all the on-topic and off-topic tweets shared by the detected users during the event. We collected 60195 tweets composed of on-and off-topic tweets shared by 3332 users during the two days of the disaster.

Ground Truth: For the purposes of training and evaluation, we conducted a subjective study for labeling each user in the dataset as one of the two classes: C1 for prominent users, or C2 for non-prominent ones according to the relevance and freshness of his tweets' content. In this study, multiple persons have manually labeled each user in the dataset. Then, they assigned a score indicating the prominence degree of each prominent user. The first label is to be used for training and evaluating our classification model, while the second label (prominence degree score) is to be used to evaluate our user ranking model.

B. Experimental Setup

The dataset described in the previous Subsection has been divided into training and test sets using two different partitions as described in Table II. Furthermore, in order to avoid any bias in experiments, we have applied the principle of 3-fold cross validation for both partitions 1 and 2.

C. Experiments and Results

1) Performance of our Classification Model: We compared our proposed classification model with several baselines and state-of-the-art methods as described in the following: Our model: using our proposed features (Subsection III-B) computed using both the on and off-topic metrics. B1: this model also uses the features described Subsection III-B, but without any adjustment of on-topic metrics with off-topic ones. B2: this model uses all the on-topic metrics presented in Table I. B3: this model uses the features proposed by Pal et al. [START_REF] Pal | Identifying topical authorities in microblogs[END_REF].

Figure 1 shows the precision and recall results of our classification model to identify the prominent users compared to the other baselines. We note that the results of our classification model are significantly higher than the other baseline models. According to the recall results of the two partitions, we observe that our model detects most of the true prominent users, and achieves between 8% to 20% higher recall than the baseline methods. Additionally, we note that the precision of the different models for class C1 is well under 50%. However, this result remains important, as the different classification models have rejected most of the non-prominent users and performed worse than our model. Overall, through these experiments, we establish that our model outperforms other baseline methods which use only on-topic features to represent the user importance. Hence, we demonstrate that adjusting on-topical metrics with off-topical ones improves the classification results.

2) Performance of our Ranking Model: According to the classification results, our model has identified most of the true prominent users in the different partitions. However, it misclassified a small number of non-prominent users. Therefore, we need to evaluate the efficiency of our ranking model in the detection of the top prominent users. We compare our model with the following two baseline methods: Our model: we use our proposed features (Subsection III-B) computed using both the on and off-topic metrics. Baseline 1: this model uses the features proposed by Pal et al. [START_REF] Pal | Identifying topical authorities in microblogs[END_REF]. Baseline 2: this model uses the PageRank algorithm in order to measure the score of each user according to its centrality in the network. Thus, we have constructed a network relying on the different users who have shared at least one on-topic tweet about the event. We have ranked the set of users identified in class C1 using the different ranking baseline models. Then, we picked the top 15 users detected by each baseline. The precision of the ratings accorded by each baseline are computed by counting the number of correctly detected users in the top 15 with respect to our ground truth. The results of these experiments are illustrated in Figure 2. According to these results, we observe that our model achieves the highest precision compared to other baseline methods, with a precision of 86% in Partition 2. Therefore, our designed high level features outperform the state-of-the-art features for both the identification of prominent users and the detection of the top ones. Moreover, we note that PageRank algorithm achieves the worst results compared to the models constructed using machine learning techniques.

VI. CONCLUSION AND FUTURE WORK

This paper has presented a classification and ranking model to identify prominent users in a specific topic or event. Through the conducted experiments, we have shown that models learned according to high or low level features computed from both on-and off-topic metrics outperform other models that are based only on on-topic features. Despite the challenges posed by the nature of our real data, we have shown how the used supervised learning algorithm (SVM) can still be effective with appropriate features and using different weights for imbalanced data classes.

For future work, we aim to explore the different features proposed in the literature to model microblogs' users in order to select the most representative features for prominent users. In addition, we plan to investigate the use of other machine learning models to predict the top important users.

Fig. 1 .

 1 Fig. 1. Classification results of our proposed method for class C1 according to precision and recall metrics.

Fig. 2 .

 2 Fig. 2. Performance evaluation of our model according to Ranking precision of users classified in class C1 (prominent users).

TABLE I .

 I LIST OF METRICS THAT DESCRIBE USERS AND THEIR ACTIVITIES IN TWITTER. (NEW) DENOTES THE NEW PROPOSED METRICS, ON(+) AND OFF(+) REFER TO THE RECORDED METRICS FOR BOTH ON-

	AND OFF-TOPIC ACTIVITIES OF USERS
		On	Off
	Original tweets
	T1: Number of original tweets [14], [5]	+	+
	T2: Number of links shared [16]	+	+
	T3: Number of keyword and hashtags [14]	+	-
	T4: Number of favorites of original tweets (new)	+	+
	T5: Number of tweets geo-located in the event area (new)	+	-
	Retweets
	R1: Number of retweets of other's tweets [17], [5]	+	+
	R2: Number of unique users retweeted by the user (new)	+	+
	R3: Number of retweets of author's tweets (new)	+	+
	R4: Number of unique users who retweeted author's tweets

TABLE II

 II

	.	THE DIFFERENT PARTITIONS OF DATA USED IN THE
		TRAINING AND TEST PHASES.
		Partition 1	Partition 2
	Training1 (60%)	Test1 (40%)	Training1 (80%)	Test1 (20%)
	C1	54	36	72	18
	C2	1945	1297	2593	649


Imen Bizid
email: imen.bizid@univ-lr.fr

Patrice Boursier
email: patrice.boursier@univ-lr.fr

Jacques Morcos
email: jacques.morcos@univ-lr.fr

Sami Faiz
email: sami.faiz@insat.rnu.tn

A Classification Model for the Identification of Prominent Microblogs Users during a Disaster

Keywords: Information retrieval, Microblogs, Disaster management

Content shared in microblogs during disasters is expressed in various formats and languages. This diversity makes the information retrieval process more complex and computationally infeasible in real time. To address this, we propose a classification model for the identification of prominent users who are sharing relevant and exclusive information during the disaster. Users who have shared at least one tweet about the disaster are modeled using three kinds of time-sensitive features, including topical, social and geographical features. Then, these users are classified into two classes using a linear Support Vector Machine (SVM) to evaluate them over the extracted features and identify the most prominent ones. The first results using the actual dataset, show that our model has a high accuracy by detecting most of the prominent users. Moreover, we demonstrate that all the proposed features used by our model are indispensable to achieve this high accuracy.

INTRODUCTION

The rapid development of communication technologies has offered access to rich information which was not previously accessible using traditional communication techniques. These technologies, such as microblogs, have become indispensable within everyday life and have been significantly implicated in several domains, particularly within the field of disaster management.

With the emergence of microblogs such as Twitter, geographic information became easier to reach by harnessing Ambient Geographic Information (AGI) posted on the field by microblogs' users. Moreover, the message behind this feed of information is not only related to geography, but is indeed very rich, opening unprecedented insights into various research domains, particularly as they relate to human and social dynamics. Controlling and analyzing this rich information eases many disaster management challenges [START_REF] Bizid | Integration of Heterogeneous Spatial Databases for Disaster Management[END_REF].

The need for emergency teams to go on the disaster affected zones, risking their lives, in order to collect information about what is taking place diminishes greatly. People from surrounding areas can provide nearly real-time observations about disaster scenes by interacting in microblogs. Citizens in the affected zones can share on the field information about what they are experiencing; watching and hearing during a disaster. This information is fundamental to have an accurate insight into what is happening on the ground, in order to manage emergency situations in a responsive manner.

However, a huge amount of information is shared in these microblogs during disasters referring mostly to irrelevant, noisy or redundant information which is not useful for emergency responders. In fact, in order to manage critical situations in real-time, emergency responders need to access and identify the relevant and useful information shared in microblogs. Nevertheless, data shared in these microblogs could be expressed in several languages and in various forms (i.e. image, text, link and videos) that have to be analyzed using various time consuming techniques (e.g. text mining, image analysis…). These techniques make the retrieval task more complex and computationally infeasible in real time.

To address this problem, we propose a classification model which detects the prominent users who are sharing valuable and fresh information in a given disaster. The contribution proposed through this model is based on a set of new time-sensitive features characterizing the topical, social and geographical position of each user regarding the disaster. These features are computed using various metrics extracted from the user's profile and timeline as well as some other metrics describing the disaster. We use a linear SVM to classify each user who has shared at least one tweet related to the disaster as a prominent or nonprominent user.

The rest of this paper is organized as follows, Section 2 reviews related works that addressed the identification of prominent users in online social networks. Section 3 details the proposed features. Section 4 describes the learning step of our classification model. Section 5 outlines the dataset used to learn and test our classification model. Section 6 surveys the experiments and results. Section 7 concludes and outlines future works.

RELATED WORK

According to recent internet live stats, 90% of information produced daily is provided essentially from social networks. The microblog Twitter is ranked in the second position with around 700 million tweets generated daily. This data has to be explored daily in order to retrieve relevant and exclusive information which was not yet shared in traditional communication channels. To achieve this, many organizations have opted to employ certified persons to continuously follow the information shared about a given topic in real time. However, this practice is tedious and infeasible in real-time, especially when there's a surge of new information in a short period of time, a phenomenon that commonly observed with each new disaster.

To address this problem, some works have proposed to coordinate between human and machine intelligence in order to classify the information shared in microblogs [START_REF] Imran | Coordinating human and machine intelligence to classify microblog communications in crises[END_REF]. Others have opted to analyze tweets contents according to their types by applying text mining techniques if it includes a text, image processing if it contains an image and so on [START_REF] Imran | Extracting Information Nuggets from Disaster-Related Messages in Social Media[END_REF]. However, these techniques cannot be applied for real-time disaster relief as they are computationally unfeasible in real time. Therefore, many researchers have opted to identify key users who tend to share relevant information during the event, in order to follow their activities instead of focusing on the entire host of information shared on microblogs [START_REF] Ghosh | Cognos: crowdsourcing search for topic experts in microblogs[END_REF][START_REF] Hajian | Modelling Influence in a Social Network: Metrics and Evaluation, Privacy, Security, Risk and Trust (PASSAT)[END_REF][START_REF] Romero | Influence and passivity in social media[END_REF][START_REF] Wagner | It's Not in Their Tweets: Modeling Topical Expertise of Twitter Users, Privacy, Security, Risk and Trust (PASSAT)[END_REF]. This approach is agnostic of the content type of the shared tweets by the user, it depends on simple measures that can be computed according to the features characterizing users who are classified into three categories

The first category defines key users as the central ones in the network who can be identified using standard centrality measures such as PageRank [START_REF] Heidemann | Identifying key users in online social networks: a PageRank based approach[END_REF] and HITS [START_REF] Cappelletti | IARank: Ranking Users on Twitter in Near Real-Time, Based on Their Information Amplification Potential[END_REF]. As these measures depend only on the structure of the network, many works have proposed to adapt these measures and more precisely PageRank and HITS algorithms to the specificities characterizing microblogs (Shoubin and Ling, 2011; Silva et al., 2013; [START_REF] Tang | Social influence analysis in largescale networks[END_REF][START_REF] Weng | TwitterRank: finding topicsensitive influential twitterers[END_REF]. Measures proposed in these two categories are more suitable for the identification of influential users who are more prominent than others in a specific topic. Moreover, they are still time consuming and sensitive to well-connected users (e.g. CNN, BBC…) as they are using mainly the PageRank or Hits algorithms. Other works have identified topical key users according to the features describing their activities shared in microblogs [START_REF] Pal | Identifying topical authorities in microblogs[END_REF]. These approaches are computationally feasible in real time, nevertheless, they are sensitive to old fashioned information as they do not take into account neither the topical user's behavior features over time nor the spatiotemporal features.

USERS' MODELING

In order to learn and apply our classification model, we have to represent each user who has shared at least one tweet related to the disaster according to a set of time-sensitive features reflecting his social, geographical and topical positions during the event. The three categories of features are described in depth in the next subsections.

Topical user's position features

The features proposed in this category estimate the extent to which a user is connected to the disaster according his tweeting behavior. These features rely on the frequency of on-topic and off-topic tweets shared by each user during the disaster. A tweet is recorded as on-topic if it meets two criteria. It must contain at least one keyword or hashtag identifying the disaster and it should not include any words reflecting non-serious or non-valuable contents (i.e. advertising or joke words and symbols such as sale, rent, pub, lol and so on).We use the following notations in the sequel: OnT, OfT represent respectively the number of on-topic and off-topic tweets shared by the user from t0, time of the occurrence of the disaster, to tOnT+1, extraction time of the user's tweets. t is the time of share of the on-topic tweet i. OfTi is the number of off-topic tweets shared between ti and ti+1.

On-topic attachment: the more the user is sharing consecutive on-topic tweets over time the more he is interested in the disaster.

Son(u) =

Off-topic attachment: the more the on-topic shared tweets are interrupted by offtopic ones the more the user is potentially detached from the disaster. Many news channels like CNN and BBC would fall into this category, as they are interested in many topics at once and do not particularly exclusive information.

Soff(u)=

Social user's position features

This category of features evaluates users according to their position in the microblogs' social graph. They promote users having a high number of on-topic followers OnF and on-topic followees OnFe who have shared their first on-topic tweet ti after a while from the occurrence of the disaster. These features are adjusted by the number of the user's followers F and followees Fe.

User's Followers Implication: indicates how much the user's followers are implicated in the disaster.

User's Followees Implication Feature: estimates how much the user's followees are interested in the disaster.

Geographical user's position features

These features evaluate each user according to his geographic position regarding the disaster. Users who are geolocated in the disaster area may share relevant information not only about what they are hearing but also about what they are seeing and experiencing.

Spatial Co-location Feature (SCF)

SCF indicates the matching rate between the set of unique locations Lu specified by each user and the set of unique locations included in the disaster zone Ld. The extracted locations are drawn from the user's profile location.

SCF(u)=

Spatial Global Co-geolocation Feature (SGCF)

SGCF indicates the inclusion rate of the geolocation specified by the user in the territory threatened or affected by the disaster. The disaster area is represented by a polygon or a set of polygons Pg that may include many distant zones. This feature takes into account only specific geographic coordinates Cu.

SGCF(u)=

Spatiotemporal Local Co-geolocation Feature (STLCF)

STLCF is motivated by the fact that in one disaster many sub-disasters may occur in a shorter period and a more specific location. Considering only the geographic coordinates shared by each user, we extract the geolocation list Cu for each user according to the period of time Ti corresponding to the duration of each defined sub-disaster i. Each user u is represented by a sub-disasters vector SGCF(U) = [SGCF1(u,T1),SGCF2(u,T2)… SGCFn(u,Tn)] and each sub-disaster can be weighted according to its importance during the disaster.

USERS' CLASSIFICATION

After computing the different features scores, we have to assign the vector of features V representing each user.

Then, these features are used to classify all the users interacting about the disaster into two classes: Class 1 (C1) includes prominent users who are sharing valuable and exclusive information indispensable for disaster management Class 2 (C2) represents users who are sharing non-valuable or/and not exclusive information Since we generate only 7 features at this stage of work, we decided to keep things as simple as possible by using a linear Support Vector Machine (SVM) to learn the classification model. The two types of SVMs; linear and non-linear have been tested in order to decide which SVM classifier is more suited to identify prominent users (C1). As proved in the next sections, the linear SVM is the most suited for our case.

DATA AND METHODOLOGY

In order to learn the classification model, we collected most of the tweets shared during the floods that have occurred from 29 th to 30 th September 2014 in the Herault area, situated in the south of France. The flooded area witnessed recordshattering 252 mm of rainfall in just three hours, causing important damages estimated between 500 and 600 million Euros.

Data collection was processed using our multi-agent System called MASIR. At the lowest level, the system detects the different users who have shared at least one on-topic tweet (i.e. talking about the floods) during the analyzed period. On topic-tweets are detected using the hashtags and keywords "#Herault, Hérault, #Hérault, #interpériés, #crues, #flooding, #Montpelier, #Alert, #Inondations, #RedAlert" which were employed by Twitter users to share information about the disaster. The system then crawls all the on-topic and off-topic tweets shared by the detected users from 29 th September at 00 am (t0) to 1 st October at 00 am (tOnt+1). We collected 63 195 tweets composed of on and off-topic tweets shared by 3 163 users during the 2 days of the disaster. 18% of these tweets are geolocated with exact GPS coordinates and 95% of the crawled profiles contain the location name of the user. The set of polygons representing "Montpellier", "Pinet" and "Sète" cities placed in red alert were assigned in order to compute the STLCF score of each user by fixing the same weight for each city (0.33).

We asked 45 volunteers from our laboratory to evaluate these users anonymously according to the relevance and recency of their tweets regarding the event (66 users/volunteer). These volunteers have to study the tweets of each user (from 1 to 82 tweets/ user) and attribute them to one of the 2 classes. According to this study, 88 and 3075 were classified respectively in C1 and C2.

To train our model, we divided the set of users into two sets: a training set Tr; test set Ts. Tr contains 45 and 1602 attributed respectively to C1 and C2. Ts includes 43 and 1473 assigned respectively to C1 and C2. Then, we used the linear Support Vector Machine (SVM) to model our classification problem using Tr and we tested the efficiency of the resulted model using Ts.

EVALUATION AND RESULTS

We compared our model with 2 baseline models learned by non-linear SVM classifiers as described below: Baseline1: The model has been learned using the kernel radial basis function in SVM.

Baseline2: The model has been learned using a quadratic SVM.

Our model: We used a linear SVM to learn our classification model.

The classification results obtained by these baselines are detailed in Table 1. According to these results, the linear SVM outperforms the other baselines as it detects 39 prominent users among 43 users confirmed by the ground truth. Overall, the classification results obtained by our model achieved a high accuracy (96%).

Our model

Baseline1 Baseline2

True

CONCLUSION AND FUTURE WORK

In this paper, we proposed a new classification model based on new topical, social and spatiotemporal features. This model could be used in order to detect the prominent users who are sharing valuable and exclusive information during a disaster. According to the first experimental results, our learned classification model gives a very high accuracy by using the three categories of features to represent users interacting during the disaster. Moreover, we showed that topical and spatiotemporal features are necessary to detect most of the prominent users.

For future work, we aim to enrich our classification model with further features by taking into account the nature of tweets. Furthermore, as the results obtained by our model are not precise enough, we wish to propose a ranking model in order to be able to identify the top N prominent users. Moreover, we aim to test our final model with other natural disasters in order to evaluate it in real time and in other contexts.

Table 1 :

 1 Classification results using different modelsIn order to test the performance of the three categories of features proposed in this paper, we train the linear SVM classifier using firstly only Topical Features (TF), then, using TF and Spatiotemporal Features (SpF) and finally using TF and Social Features (ScF). The results returned by the learned model using each set of features are reported in Table2. These results show that TF are necessary to detect the most prominent users. The resulted model using only TF and SpF has yielded promising results as 41 prominent users have been retrieved among 43. However, using only these two features makes the model sensitive to false positives. Hence, according to the reported results, all the categories of features are valuable to achieve the best result in terms of the classification model accuracy.

					True Retrieved Prominent	False Retrieved Prominent
					Users	Users
				TF	39	51
				TF+SpF	41	440
				TF+ScF	39	52
				All features	39	49
	Positives	39	34	38
	False Positives	49	40	53
	True Negatives	1424	1433	1423
	False Negatives	4	9	5

Table 2 :

 2 Classification results using different categories of features


Imen Bizid
email: imen.bizid@univ-lr.fr

Patrice Boursier
email: patrice.boursier@univ-lr.fr

Jacques Morcos
email: jacques.morcos@univ-lr.fr

Sami Faiz
email: sami.faiz@insat.rnu.tn

MASIR : A Multi-agent System for Real-time Information Retrieval from Microblogs during Unexpected Events

Keywords: multi-agent systems, microblogs, real-time, information retrieval, unexpected events

Microblogs have proved their potential to attract people from all over the world to express voluntarily what is happening around them during unexpected events. However, retrieving relevant information from the huge amount of data shared in real time in these microblogs remain complex. This paper proposes a new system named MASIR for real-time information retrieval from microblogs during unexpected events. MASIR is based on a decentralized and collaborative multi-agent approach analyzing the profiles of users interested in a given event in order to detect the most prominent ones that have to be tracked in real time. Real time monitoring of these users enables a direct access to valuable fresh information. Our experiments shows that MASIR simplifies the real-time detection and tracking of the most prominent users by exploring both the old and fresh information shared during the event and outperforms the standard centrality measures by using a time-sensitive ranking model.

Introduction

Since the emergence of Twitter, microblogs have experienced a new social aspect allowing users to express in real time their opinions, perceptions and expectations. The usefulness of such networks has been demonstrated during the Arab Spring revolutions and the last memorable natural disasters. Microblogs were the main source of information used by official organizations and news channels in the different event phases (i.e. event detection, situation awareness, alert dissemination) [START_REF] Bizid | Integration of heterogeneous spatial databases for disaster management[END_REF]. Since, these online social platforms have become indispensable to acquire and monitor information in real time during unforeseen events.

Unexpected events, in the context of this paper, refer to the events which occur without any warning or preparation such as natural and human disasters. Information retrieval during this kind of events is more complex than during simple ones as it is not possible to gain a priori information before its occurrence. Most of the proposed IR approaches in this context are based on both machine and human intelligence [START_REF] Imran | Coordinating human and machine intelligence to classify microblog communications in crises[END_REF]. Such techniques are time consuming and require the presence of an important number of volunteers for the collection and the annotation of relevant information tasks.

Indeed, to enhance situation awareness of such events, it is important to find new ways coping with the limited data access using microblogs APIs on the one hand and to be able to retrieve valuable information shared in real time on the other hand [START_REF] Nakamura | Effects of social participation and the emergence of voluntary social interactions on household power-saving practices in post-disaster kanagawa, japan[END_REF]. Identifying and tracking prominent users who are behind the relevant and exclusive information shared during the targeted event can offer a rapid access to the required information. The prominence of users in these events relies on the relevance and freshness of their information independently of their social influence. Hence, using traditional techniques would fail to track and identify such users in real time due to the strict microblogs conditions for tracking users and the complexity of this type of events. The original contributions of this paper are twofold. First, we propose a multi-agent system to insure a real-time identification and an extensive tracking of prominent users in the context of managing unexpected events. Second, we propose a novel approach to detect prominent users based on both their geographic and social positions over time during an unexpected event.

This paper is organized as follows: Section 2 reviews existing approaches for information retrieval from social networks. Section 3 describes our proposed system and its functional model. Section 4 presents our experiments and evaluates our results. Section 5 summarizes our conclusions and future work.

Related Work

Tracking of social media users for information retrieval has been studied using mainly three ways: social networks' interface crawling through public and false profiles and phishing techniques [START_REF] Canali | Dataacquisition in social networks: Issues and proposals[END_REF], social networks applications such as Netvizz [START_REF] Rieder | Studying facebook via data extraction: The netvizz application[END_REF], as well as social networks Application Programming Interfaces (APIs) [START_REF] Abdulrahman | Data extraction from online social networks using application programming interface in a multi agent system approach[END_REF]. These techniques have provided a direct access to an important number of users. However, they have no selection strategy to evaluate the prominence of tracked users.

To the best of our knowledge, the issue of prominent users identification has never been explored in the context of unexpected events. However, there have been several attempts proposing new measures identifying influential users and domain experts in microblogs and specially Twitter [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF][START_REF] Smailovic | Calculating user's social influence through the smartsocial platform[END_REF][START_REF] Smailovic | Advanced user profiles for the smartsocial platform: Reasoning upon multi-source user data[END_REF]. Most of the proposed approaches identifying social media influencers are based on standard centrality measures such as eigenvector centrality and its variants HITS [START_REF] Agichtein | Finding high-quality content in social media[END_REF] and PageRank [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF]. These adapted measures to microblogs specificities (e.g. number of tweets, mentions, retweets ...) have yielded promising results for the identification of influential users [START_REF] Silva | Profilerank: Finding relevant content and influential users based on information diffusion[END_REF][START_REF] Weng | Twitterrank: Finding topic-sensitive influential twitterers[END_REF]. However, they are computationally expensive and sensitive to well-connected users (e.g. celebrities, communication channels...) [START_REF] Cappelletti | Iarank: Ranking users on twitter in near real-time, based on their information amplification potential[END_REF]. On the other hand, low research works have proposed domain experts iden-tification based on efficient ranking models using a set of features describing the activities of users in particular topics [START_REF] Pal | Identifying topical authorities in microblogs[END_REF][START_REF] Xianlei | Finding domain experts in microblogs[END_REF]. These approaches are efficient to identify in real time topical authorities sharing relevant information based on their previously shared information. While somewhat similar to [START_REF] Pal | Identifying topical authorities in microblogs[END_REF][START_REF] Xianlei | Finding domain experts in microblogs[END_REF], prominent users identification in the context of unexpected events differs in different points. Firstly, there is no a priori information describing the user's prominence that may be extracted before the occurrence of the event. In our context, information related to interested users in an event has to be explored in real time. Secondly, many features describing the prominence of a user in specific events have never been explored in the prior approaches such as the geolocation position, the time of the first user's activity regarding the event and the social position of the user in both the network and the event. Thirdly, our approach is tested in a real case from the identification of all users interested in the event to the detection and the tracking of the most prominent ones. The problem of prominent users identification has been studied only theoretically using test databases. Microblogs APIs constraints related to extensive crawling have never been taken into account. Moreover, existing approaches for microblogs users tracking have focused only on the quantity of users while neglecting their qualities. In this paper, we propose a multi-agent system for relevant and exclusive information retrieval in Twitter by identifying and tracking the most prominent users in real time cases.

Decentralized Collaborative System for Information Retrieval

This section describes the operation details of MASIR for information retrieval from microblogs during unexpected events. MASIR is based on a multi-agent approach compliant with Twitter APIs specifications. The idea behind modeling a multi-agent system for information retrieval relies on its ability to retrieve information using parallel processing which makes the analysis process computationally feasible in real time. Moreover, the multiple used agents boost the number of monitored users in parallel and ease the detection of the most prominent users during the event.

Fig. 1 describes the decentralized structure of our system. MASIR is composed of 6 different kinds of agents designed to execute a well-defined related tasks. The process starts when the keywords and/or hashtags representing the targeted event were specified to the Stream Retrieval Agent (SRA). Using these parameters, SRA searches for the list of new users sharing real-time information about the event and sends it continuously to the Historic Listener Agents Manager (HLAM). HLAM assigns a Historic Listener Agent (HLA) to each identified user by SRA in order to extract and store his historic in the Historic and Social Information Base (HSIB). The collected data is analyzed by the Prominent Users Detector (PUD) in order to detect the most prominent users that have to be tracked in real time by the Stream Listener Agents (SLAs). These agents are described in detail in the following sub-sections. Fig. 1. A decentralized multi-agent system for real-time information retrieval from Twitter

The Stream Retrieval Agent (SRA)

SRA retrieves the tweets published in real time about an event and extracts the identities of users who are sharing it by following these monitoring operations:

1. Streaming search: SRA remains connected to Twitter during the event in order to search in real time for new tweets using the assigned hashtags or keywords identifying the targeted event. 2. Users' identification: SRA extracts the identity of users sharing on-topic tweets. 3. Users' filter: SRA applies dynamically new filters with reference to the Identified User Base (IUB) in order to force the streaming search to retrieve only tweets shared by new users. 4. Users' storage: SRA stores in IUB the identifier of any new detected user posting information related to the event. 5. List of users sending: SRA has to send the list of new detected users interested in the event every 30 seconds to HLAM.

The Historic Listener Agents Manager (HLAM)

HLAM manages the extraction process of the social and historic information from the identified users profiles. It controls multiple HLA agents which are in charge of the historic and social information collection from each identified user profile. HLAM can undergo different transitions according to the processed operations:

1. Users assignment: When HLAM receives the list of users sent by SRA, it adds the new identified users in a waiting list. Then, HLAM assigns each user to one of the available HLAs by respecting the FIFO (First In First Out) principle. 2. Information reception: This operation is processed after the reception of a message from HLA precising that the historic extraction process was accomplished. Then, at this stage HLAM saves the returned information collected by HLA in HSIB. 3. HLA status change: Once HLAM has received all the extracted information from a HLA, it sets this HLA status to "free" in order to be able to assign it to a new user.

The Historic Listener Agents (HLAs)

HLAs have to extract historic information shared by each assigned user. Once a HLA has finished the extraction of the needed information related to a user, it sends a message to HLAM to store the collected information in HSIB. Then, the HLAM will change this HLA status to "free". Each HLA have to be able to process the three following operations:

1. Receiving a user's identity: When HLA status is set to "free", HLA could be assigned to a unique user recognized by his unique identifier.

Historic information extraction:

HLA extracts all the historic information shared by the assigned user. 3. Social information extraction: HLA extracts the list of the followers and followees of the assigned users. 4. Extracted information Sending: HLA sends all the information collected to HLAM in order to store it in HSIB and to change its status.

The Prominent Users Detector (PUD)

PUD acts as the intermediary between the historic extraction process and the streaming process. This agent detects the most prominent users using the data collected during the historic extraction process. The identification of these prominent users insures the attribution of the limited number of parallel SLAs to the most central users during the event. PUD detects the most prominent users by calculating and updating periodically the Prominence Score (PS) of the already watched users. This final score is estimated according to the geo-location and social positions of the user and the recency of his first interaction regarding the event. PS is computed using the following ranking model:

P S(u) = w 1 * RS(u) + w 2 * GP S(u) + SP S(u) (1)
Where 0.38 and 0.02 are the weights reflecting the importance of RS and GPS. All the weights' values (from w 1 to w 6 ) used during the calculation of PS and SPS were estimated a priori through a user study evaluating the active Twitter users in the South Korea ferry disaster. This study was conducted by a group of volunteers who have evaluated the Twitter users according to the relevance and recency of their information about the disaster. These volunteers have noted these users from 1 to 10 according to their prominence. These notes were used for fitting a linear regression model composed of the different predictor scores proposed in this paper to evaluate the prominence of each user. The weights evaluating each predictor were normalized to form the sum 1 for all the weights. The Recency Score (RS) indicates the recency of the first on-topic information shared by the user regarding the time of occurrence of the event (t event ).

The difference in time between the first shared information (t) and (t event ) is measured in minutes.

RS(u) = 1 t -t event + 1 (2)
The Geo-location Position Score (GPS) indicates the inclusion rate of the geo-location(i.e. longitude, latitude) specified by the user in the territory concerned by the event. The event area is represented by a polygon or a set of polygons (P e) that may include many distant zones. For each user u, we extract from his different historic tweets collected by HLAs the set of his geo-locations (Cu). For example, if all the geolocations specified by the user are included in the event area, his GPS will be set to 1.

GP (u) = Cu ∩ P e Cu ∪ P e (3)
The Social Position Score (SPS) indicates how much the user's followers (F ) and followees (F e) are interested in the analyzed event. The more a user has many on-topic followers (OnF ) and followees (OnF e) having a high RS, the more his final SPS is high. As well-connected users such as CNN and BBC may have a large number of OnF and OnF e even they are not sufficiently prominent regarding the analyzed event, these numbers are adjusted by F and F e which makes our SPS insensitive to well connected users. SPS is computed as follows using the social information already extracted by HLA and stored in HSIB:

SP (u) = w 3 * OnF i=1 RS(i) log(OnF +1) + w 4 * OnF log(F ) + w 5 * OnF e i=1 RS(i) log(OnF e+1) + w 6 * OnF e log(F e) (4)
Where w 3 = 0.21, w 4 = 0.1, w 5 = 0.23 and w 6 = 0.04 are the weights reflecting the importance of the different predictors contributing in the final SPS score of each user.

The Stream Listeners' Agents Generator (SLAG)

SLAG manages the tracking process of the most prominent users during the event. It starts the generation and management process when it receives the list of prominent users by PUD. SLAG generates a SLA for each user in the list. These SLAs are generated in different hosts in order to avoid the risk of IP banning by Twitter. Hence, SLAG processes the following operations :

1. Receiving detected users: SLAG receives periodically an updated list of the most prominent users that have to be tracked in real time. 2. Killing existing SLAs: After receiving the updated list, LAG kills SLAs which are tracking users who do not exist in the new list. By killing these SLAs, SLAG will release the place in some hosts in order to be able to track the new prominent users. 3. Generating a new SLA: After liberating the place for the new prominent users, SLAG generates SLAs for these users.

Streaming Listener Agents (SLAs)

While HLA extracts the historic and social information of one assigned user and once it has finished this task, it listens to another user, SLAs differ in various points. First, SLAs keep listening to a user profile in order to detect any new update. Second, SLAs are dynamically generated by the SLAG. Each SLA is in charge of tracking the assigned user profile in real time. SLAs store in real time any new detected information shared by its assigned user in the Retrieved Information Base (RIB). RIB contains the tweets extracted in real time from the most prominent users' profiles.

Experiments and Results

The architecture of MASIR was implemented using Java Agent DEvelopment framework (JADE). Using this framework, each agent was created in a running instance named container. MASIR agents were executed in various containers distributed in 5 hosts connected via a Virtual Private Network. Our system was implemented using two Twitter APIs; the Search API for the historic information extraction process and the Streaming API for the real-time tracking of prominent users.

As the Streaming and Search APIs limit the number of the crawled profiles simultaneously to around 5, MASIR used to encounter this limit by distributing SLAs and HLAs in various hosts and by using different Twitter accounts. This distribution aims not only to avoid IP banning when the authorized crawling limit rate is reached but also to boost the number of listened profiles. The 5 used hosts incorporate all a main container in order to enable SLAG to monitor all the created agents and automate the agents generation process according to the number of available hosts. Using this strategy, HLAM managed up to 75 HLAs (15/host) and SLAG generated up to 175 SLAs (35/host).

MASIR was run during the Herault floods which have occurred in the Herault country in France. These floods have lasted two days from 29 to 30 September 2014. They have caused important damages estimated between 500 and 600 million Euros. MASIR was launched after a while from the official announcement of the event using the hashtag "Herault" which was used by Twitter users to refer to the event. Our system has collected 41.064 historic tweets and 22.136 fresh tweets shared respectively by 3.143 users listened by HLAs and 604 users tracked by SLAs. Hence, MASIR has coped with the limits imposed by the Twitter APIs by tracking an important number of users in real time. In order to evaluate the quality of the users tracked in real time by MASIR. We have collected all the tweets shared by the identified users by SRA from the announcement of the event to its end using the Search API after two days of the disaster. Then, these users were evaluated by a group of volunteers to define our ground-truth. This group were asked to note these users from 1 to 10 according to the relevance and freshness of their tweets. The top 175 users were selected in order to evaluate the users tracked by MASEA periodically. Table 1 presents the total number of prominent and non prominent users identified by MASIR during the two days of the disaster and the number of the true prominent users tracked at each period of time with reference to the ground truth results. According to these results, an important number of the ground-truth prominent users were identified by SRA and tracked by SLAs from the first day of the disaster. We also note that the precision of our detection process was improved during the end of the second day by tracking 46% of the ground-truth prominent users continuously. We compared the used model by MASIR to detect prominent users with three baseline algorithms: the eigenvector centrality, PageRank and HITS algorithms. These measures were chosen as they were used in the literature for the detection of prominent users in various contexts.

To measure the quality of results returned by each baseline in each period of time, we calculated the precision of the returned prominent users by each algorithm. The obtained results are shown in Fig. 2.

Compared with the time consuming centrality measures, our model gains a significant increase in performance at the different stages of the event. We also note that the performance of the baseline measures decreases over time as they are sensitive to well-connected users. Moreover, we note that the centrality measures are not suitable for the detection of prominent users in the context of unexpected events.

According to the results of our experiments, MASIR outperforms the centrality measures in terms of performance and time as the most prominent users were detected at an early stage of the event. Moreover, the used distributed architecture and our time sensitive ranking model have made the detection and the tracking of these users feasible in real time. Even MASIR has not detected all the top 175 users, these results are promising. Similar to recommendation and information retrieval engines, we can argue that if from the 9 recommended users, 8 are bad and 1 is extremely good and is sharing the required information, there is a high chance that the user will be satisfied by the retrieved information in real time.

Conclusion and Future Work

This paper highlights the power of multi-agent systems based architecture for real-time information retrieval from microblogs during unexpected events. MASIR uses various collaborative agents enabling a real-time detection of the most prominent users who tend to share valuable information. This first research effort to deal with the detection and tracking of prominent users in real unexpected events cases has provided promising results. The straightforward time sensitive measures used by our ranking model have outperformed the standard centrality measures. Moreover, the employed multi-agent architecture has coped with the Twitter APIs limits to be able to track in real time 175 users using only 5 hosts.

For future work, we aim to propose new features reflecting the user behavior during the event in order to improve the detection process of prominent users. In addition, we would like to optimize the distribution of our agents and minimize the number of exchanged messages between the different agents.

Fig. 2 .

 2 Fig. 2. The precision of MASIR vs Eigenvector, Hits and PageRank during the disaster

Table 1 .

 1 The simulation results of the identified and the true detected users by MASIR

		Identified	Ground-truth	True prominent
		users by SRA	prominent users	users listened by
				SLAs
	1st day 12 am-00 pm	1254	157	67
	00 pm-12 am	1264	157	67
	2nd day 12 am-00 pm	2433	173	57
	00 pm-12 am	3143	175	81


Imen Bizid
email: imen.bizid@univ-lr.fr

Nibal Nayef
email: nibal.nayef@univ-lr.fr

Oumayma Naoui

Patrice Boursier
email: patrice.boursier@univ-lr.fr

Sami Faiz
email: sami.faiz@insat.rnu.tn

A Comparative Study of Microblogs Features Effectiveness for the Identification of Prominent Microblog Users during Unexpected Disaster

Keywords: Effectiveness of feature categories, prominent microblog users, disaster management

This paper presents a learning-based approach for the selection of relevant feature categories in the context of information retrieval from microblogs during unexpected disasters. Our information retrieval strategy consists of identifying prominent microblog users who are susceptible to share relevant and exclusive information in a disaster case. To identify these users, we evaluate the effectiveness of the state-of-the-art features characterizing microblog users for the identification of prominent users in a specific context. We experimented with a different sets of feature categories to determine those that discriminate prominent users sets from non-prominent ones interacting in Twitter during the 2014 Herault floods that occurred in France. The achieved results show that onand off-topical user activities features are the most representative features for identifying prominent users in a disaster context. We also note that SVM outperforms the ANN learning algorithm for this classification context especially when it is trained with additional spatial features.

Introduction

The climate change unleashes a multitude of unexpected disaster characteristics and effects that have never been perceived in our planet. Heat waves in summer, winter without snow, climate disruption, floods in some regions of Europe while other neighboring regions suffer from terrible droughts. Climate change manifests itself in diverse unexpected forms. Such phenomena still turn into disasters, causing irreversible damages in many places of our planet.

The challenges of managing such disasters are related especially to situation awareness and real-time information collection. The need for emergency teams to go on the disaster affected zones, risking their lives, in order to collect information about what is taking place diminishes greatly. People from surrounding areas can provide nearly real-time observations about disaster scenes by interacting in microblogs. Citizens in the affected zones can share information about what they are experiencing; watching or hearing during a disaster. These microblogging platforms represent a rich source of information fundamental to have an accurate insight into what is happening on the ground in order to efficiently manage these unexpected disasters.

Although these microblogs such as Twitter provide many specificities (e.g. number of favorites, number of retweets of a tweet, etc.) reflecting other microblog users feedback regarding the shared information, it is still challenging to retrieve relevant and exclusive information from the huge amount of shared data. These microblogs specificities remain inaccurate as they refer mostly to the information shared by popular users independently of the relevance and freshness of their content. Therefore, it is more rational to associate the relevance and the quality of the shared information with user's prominence during the disaster.

By tracking prominent microblog users who are sharing relevant and exclusive information during an unexpected event, emergency first responders can have a real-time global view of what is happening in the threatened or affected areas. The identification of these key users have been widely explored in the context of influencers and domain experts identification. However, it has never been explored in the context of prominent users identification during unexpected disasters.

Prominent users, in the context of this paper, refer to microblog users who are susceptible to share exclusive and relevant information during a given unexpected event. Finding such users depends generally on the effectiveness of the selected categories of features describing these microblog users according to the specific context.

In this paper, we aim at evaluating the effectiveness of both the state-of-theart and our prior proposed features describing microblog users for the identification of prominent microblog users in the context of unexpected disasters. This study focuses on the selection of the most descriptive categories of features that may lead us to the identification of these microblogs key users during disasters.

The rest of this paper is organized as follows. Section 2 reviews related works for identifying prominent microblog users. Section 3 describes the different categories of features evaluated in this paper. Experimental results are presented in Section 4. The experiments are discussed in Section 5. Finally, we conclude this paper with direction to future work in Section 6.

Related Work

Current Information retrieval systems in microblogs for disaster management are mostly based on the content analysis of microbogs posts. Tweak the tweet system [START_REF] Starbird | Tweak the tweet: Leveraging microblogging proliferation with a prescriptive syntax to support citizen reporting[END_REF] provided a hashtag based syntax to make text mining of the huge amount of information shared in microblogs during disasters easily processed. Imran et al. [START_REF] Imran | In: Coordinating human and machine intelligence to classify microblog communications in crises[END_REF] proposed a classification model for disaster-related information extraction by analyzing tweets text content. MicroFilters system [START_REF] Ilyas | Microfilters: Harnessing twitter for disaster management[END_REF] extracted the valuable disaster-related-images shared in microblogs based on image analysis techniques. These systems have yielded promising results for the identification and classification of disaster-related-content. However, they are computationally expensive, on the one hand, and they are still sensitive to redundant and outdated information on the other hand. Moreover, it is more logical to identify prominent users that may share relevant and exclusive information during the disaster and track them in order to access in real time to their shared disaster-related-information.

To the best of our knowledge, the issue of prominent users' identification has never been studied in the context of disasters. However, it has been widely explored in the different contexts defining key users as influential users in the network or as domain experts who are active and popular in a specific topic or domain [START_REF] Weng | Twitterrank: Finding topic-sensitive influential twitterers[END_REF][START_REF] Zhang | Identifying influential users of micro-blogging services: A dynamic action-based network approach[END_REF][START_REF] Silva | Profilerank: Finding relevant content and influential users based on information diffusion[END_REF].

Existing approaches for the identification of social media influencers are based on standard centrality measures such as eigenvector centrality and its variants HITS [START_REF] Agichtein | Finding high-quality content in social media[END_REF] and PageRank [START_REF] Kwak | What is twitter, a social network or a news media?[END_REF]. These adapted measures to microblogs specificities (e.g. number of tweets, mentions, retweets ...) are computationally expensive and sensitive to well-connected users (e.g. celebrities, communication channels...) [START_REF] Silva | Profilerank: Finding relevant content and influential users based on information diffusion[END_REF]. Therefore, these approaches could not be used in real time scenarios, on the one hand, and they could not lead us to identify users sharing fresh information during unexpected disasters on the other hand.

Apart from the above research studies, domain experts identification has been explored using supervised and unsupervised learning techniques based on a set of features describing the activities of users regarding only the particular analyzed topics [START_REF] Pal | Identifying topical authorities in microblogs[END_REF][START_REF] Xianlei | Finding domain experts in microblogs[END_REF][START_REF] Ghosh | Cognos: Crowdsourcing search for topic experts in microblogs[END_REF]. IA-Rank [START_REF] Cappelletti | Iarank: Ranking users on twitter in near real-time, based on their information amplification potential[END_REF] ranked users based on the features characterizing how the user name is amplified via mentions, replies or retweets by other users. Pal et al. [START_REF] Pal | Identifying topical authorities in microblogs[END_REF] proposed a new identification model using a set of features characterizing microblog users according to the different nature of their activities and their social position in the network. Xianlei et al. [START_REF] Xianlei | Finding domain experts in microblogs[END_REF] proposed a Gradient Boosted Decision Tree to identify domain experts in Sina Microblog based on profile and tweeting behavior features.

These features have yielded promising results in the identification of domain experts. However, they have never been explored in the context of the identification of prominent users during disasters. Hence, in this paper, we evaluate the effectiveness of the different categories of both state-of-the-art features and our prior proposed features [START_REF] Bizid | A classification model for the identification of prominent microblogs users during a disaster[END_REF][START_REF] Bizid | Prominent users detection during specific events by learning on-and off-topic features of user activities[END_REF] in a disaster context.

User Modeling Using Microblogs Features

The identification of prominent users problem in the context of disasters can be casted into a binary classification problem. Many supervised learning algorithms can be used to learn the classification model for this purpose. The performance of the used algorithms is potentially associated to the strength of the selected features to model the user behavior in a disaster context. The more the features are representative for the prominent and non-prominent users behavior during a disaster the more the learned classifiers are efficient.

To learn the classification model, we study a large set of the state-of-the-art features and some new features proposed in our prior work [START_REF] Bizid | Prominent users detection during specific events by learning on-and off-topic features of user activities[END_REF][START_REF] Bizid | A classification model for the identification of prominent microblogs users during a disaster[END_REF]. The listed features may reflect the behavior and the importance of each user interacting about the disaster. We have split these features into five broad categories: profile features (PrF), on-topical features (OnAF), off-topical features (OfAF), spatial features (SpF) and social network structure features (SnF). The rest of this section describes in depth these main categories of user features.

Profile Features

Profile Features (PrF) characterize the user profile description. This description is registered by the user himself (e.g. location, domains of interest...) or generated automatically by the microblogging service in order to report the user activeness rate in the network (e.g. Number of collected favorites, Number of followers...). Table 1 presents the set of user profile features. These features are extractable from any user profile using Twitter APIs. PrF give a general representation of each user independently of his activeness rate during the disaster. At the first sight, we can note that P2 and P1 may be descriptive for prominent users during disasters. P2 refers to the user information precision during the disaster where the geographic information is important. P1 can be used as a strong proof or indicator to evaluate the veracity of the information shared by each user. P7 and P8 are generally used in order to detect celebrities and domain experts. These features are evaluated in order to study if there is a correlation between the user's popularity in the network and their prominence during unexpected disasters. Moreover, P4 and P5 which refer to the user activeness in the network are studied in order to evaluate if users who are generally active in the network may be prominent during unexpected disasters or not and vice versa.

User Activity Features

Microblog users can express what they are seeing, hearing and experiencing during a disaster using different nature of tweets: User's own produced tweets are original tweets shared by the profile owner. These tweets are expressed by a simple content which do not include any retweet or mention symbols. Mention tweets are tweets destined specially to particular users to make them aware about a particular information. These tweets include the @ symbol followed by the name of users to whom the tweet content is destined. Repeated tweets are original tweets posted by someone else and rebroadcasted by the user in order to share it with his followers. These tweets are informally called retweets and can be identified by the RT@username label that is automatically inserted at the beginning of the tweet.

All these three TYPES of tweets can refer to valuable contents that are indispensable to manage unexpected disasters. Hence, we need to analyze any nature of tweets shared by users interacting during a disaster in order to identify the prominent ones.

Moreover, in order to differentiate prominent users activities from nonprominent ones, we analyze both the user's on-topic tweets related to the disaster and the off-topic ones. The categorization of on and off-topic user activities was proposed in our prior work [START_REF] Bizid | Prominent users detection during specific events by learning on-and off-topic features of user activities[END_REF] under the assumption that users affected by the disaster would be interested only by the disaster news and would neglect any other off-topical information diffused in the network.

Thus, we divide the different user activities features during the disaster into two categories On-topic Activities Features (OnAF) and Off-topic ones (OfAF). These feature categories are measured respectively according to the user on-topic and off-topic activities:

On-topic an activity is considered on-topic when it contains a subset of a list of keywords and hashtags which are defined to describe the unexpected disaster under consideration Off-topic an off-topic activity refers to any activity that was not recorded as an on-topic one Additionally, in this paper we assume that tweets referring to the disaster and including at least one keyword reflecting non-serious or non-valuable contents (e.g. advertising or joke words and symbols such as sale, rent, pub, lol and so on), it will be directly recorded as an off-topic one. Thus, users who share non valuable contents could be penalized.

Our rationale behind the extraction of on-topic and off-topic activities is based on penalizing users who are toggling among several topics, and who may share outdated information. Using this strategy, users are evaluated based on their impact on the analyzed disaster, and on the strength of their attachment to that disaster. For example, Top news outlets sharing news about several topics are penalized as they do not focus mainly on the analyzed disaster.

Table 2 presents both the state-of-the-art features characterizing user activities during the disaster and our prior proposed features. On-and Off-topical user activity features will be studied separately in this paper in order to estimate the effect of each category of features in the identification of prominent users in the context of unexpected disasters.

Spatial Features

Spatial Features (SpF) characterize microblog users according to their assigned location and geolocation regarding the threatened disaster zone. Such features may be essential to determine which are the users geolocated in the disaster zone and who may play the role of sensors to diffuse information about what is really happening on the ground. The following spatial features described in Table 3 are studied in the context of disasters : Table 3. Spatial Features (SPF) characterizing the geographic position of microblog users regarding the disaster.

Name Features

S1 User Location [13] S2

Shared geo-coordinates [START_REF] Bizid | A classification model for the identification of prominent microblogs users during a disaster[END_REF] S1 indicates if the location indicated in the user profile has been stricken by the disaster or not.

S2 measures the inclusion rate of the geo-coordinates related to the user shared tweets are included in the territory threatened by the disaster or not.

Network Structure Features

Many works have explored the microblogs structure features in order to identify mostly influential and popular users. However, all of these works have used mainly time consuming algorithms that are not feasible in real time and unsuitable for the disaster context. Moreover, the used profile features referring to the number of user's followers and followees may promote popular users who are toggling between several topic and who are sharing outdated information and neglect real prominent users having a small number of connections in the network. Thus, in this category of features, we focus only on the user followers and followees who are interacting about the disaster. Table 4 presents the network structure features studied in this paper. 4 Experiments and Results

Dataset

To conduct experimental performance evaluation on real data, we collected most of the tweets shared during the floods that have occurred from 29th to 30th September 2014 in the Herault area, situated in the south of France. The flooded area witnessed record-shattering 252mm of rainfall in just three hours, causing important damages estimated between 500 and 600 million Euros. Data collection was processed using our multi-agent System called MASIR [START_REF] Bizid | Masir : A multi-agent system for real-time information retrieval from microblogs during unexpected events[END_REF]. At the lowest level, the system detects the different users who have shared at least one ontopic tweet (i.e. talking about the floods) during the analyzed period. On-topic tweets are detected using the hashtags and keywords: #Hérault, #Herault, #intempéries, #crues, #flooding, #Montpellier, #Alert, #Inondations, #RedAlert, which were employed by Twitter users to share information about the disaster.

The system then crawls all the on-topic and off-topic tweets shared by the detected users from 29th September at 00:00AM to 1st October at 00:00AM. We collected 60195 tweets composed of on and off-topic tweets shared by 3332 users during the two days of the disaster.

Ground Truth: For the purposes of training and evaluation, users tracked in our dataset have to be assigned labels indicating whether they are prominent users or not. We have asked some volunteers from our laboratory to classify the tracked users according to the relevance and exclusivity of their tweets during the event. The number of users to be labeled varied from 66 to 200 users per volunteer.

These volunteers labeled each user as one of the two classes: C1 for prominent users, or C2 for non-prominent ones. The labeling is based on the subjective study of the tweets' content of each user (varying from 1 to 82 tweets per user). According to this study, 90 users have been classified as C1, and 3242 users have been classified as C2.

Evaluation Tools and Metrics

We describe in this section the methodologies used to evaluate the effectiveness of the different categories of features for identifying prominent microblog users in a disaster context.

Features categories selection approaches

We studied the effectiveness of the different feature categories in a disaster context with two learning algorithms. Support Vector Machine (SVM) [START_REF] Osuna | Support vector machines: Training and applications[END_REF] and Artificial Neural Networks (ANN) [START_REF] Zhang | Neural networks for classification: a survey[END_REF] are used for this study. Using these algorithms, we tested all the combination of feature categories that may represent prominent microblog users interacting during a disaster.

SVM is used in order to learn a linear SVM model for the identification of microblog prominent users in a disaster case based on different feature categories.

ANN is used in order to train and test a multi-layer ANN classification model by experimenting several combination of feature categories representative to microblog users.

In order to deal with the unbalanced data classification problem, we gave a more important weight to the class C1 of prominent users (W 1 = 10) than the class C2 of non prominent users (W 2 = 1). These parameters were set experimentally in the training phase of SVM. As there is no parameters to tune the class weights using ANN, we have duplicated the dataset of prominent users by 30 in order to balance the two datasets of prominent and non prominent users in the training phase of ANN.

Experimental setup and Evaluation metrics

For experimental set-up, we randomly sampled 60% of both prominent and nonprominent labeled users datasets as training data for learning the classification and ranking models based on different feature categories, and the remaining 40% as test data to test the efficiency of the learned model. Through the different experiments conducted to evaluate the efficiency of the different feature categories for the classification of microblog users, we have used standard precision, recall and F1-score (i.e. F-measure) evaluation metrics.

Feature Categories Effectiveness

In order to select the most representative feature categories for microblog prominent users in a disaster, we evaluate the effectiveness of each category of the state-of-the-art features separately. Table 6 reports the experimental results evaluating the effectiveness of each category of features using two different learning algorithms. According to the test results returned by both the learned SVM and ANN models, the category of features characterizing the on-topical user activity in microblogs during the disaster (OnAF) is the most representative category for prominent users in a disaster context. The remaining categories of features have yielded poor results and failed to identify microblog prominent users. However, these categories may yield improvement in terms of precision and recall if they are combined with other categories. Therefore, we study the effectiveness of these categories when they are combined with the OnAF feature category. Table 7 reports the results of the different pairs of categories used to learn the ANN and SVM models for the prominent user identification. We note that combining the two categories of features OnAF and OfAF improves the identification results using both ANN and SVM. Thus, these two categories are more efficient for the identification of microblog prominent users in the context of disasters when they are combined. Regarding the other combinations, we observe that they have a negative effect when they are combined with the OnAF category. Thus, we select the most efficient combination (OnAF+OfAF) in order to enrich it with the remaining categories of features. The identification of prominent users results using the new categories of features combination are reported in Table 8. According to these results, there is no significant enhancements when adding a third category of features to OnAF and OfAF. Only the spatial category of features slightly improves the identification results obtained by learning the SVM model. However, training an ANN classification model based on these same categories, decreases the identification performance compared to the previous resulted ANN learned based on OnAF and OfAF categories.

We experiment in Table 9 OnAF OfAF and SpF categories with additional categories in order to evaluate the effectiveness improvement rate using our two learning approaches.

We notice that all the trained combinations reported in Table 8 have failed to improve prominent users identification effectiveness.

Discussion

The obtained results in this study have led us to conclude on the importance of using the on-topical and off-topical activities features categories and the spatial features category to learn an efficient classification SVM model identifying prominent microblog users during real-time disasters. On-and off-topical features are extremely useful in disaster management scenarios where prominent users focus mainly in sharing disaster-related information. Thus, using off-topical activity features, users toggling between different topics will be penalized. In addition, referring to the on-topical activities features, users focusing potentially on the unexpected disaster will be promoted. Such a property has shown that users faced by a disaster would share mainly on-topical information and neglect the other topics-related information. Moreover, we showed that users geolocated in the disaster area are more susceptible to share fresh and relevant information about what is happening around them than the others. These results have been validated using the Herault Floods database. An open access to Twitter would be necessary to evaluate these features using more microblogs disasters databases which could not be afforded.

Conclusion and Future Work

In this paper, we analyzed the effectiveness of different state-of-the-art categories of features and the additional categories of features proposed in our prior work in the context of microblog prominent users identification during unexpected disasters. We tested different combinations that may lead to an efficient classification model using two different learning algorithms ANN and SVM. We found that on-topic activities category of features and the off-topic one are almost the only categories that can differentiate microblog prominent users from non prominent one in a disaster context. Moreover, we noted that the SVM algorithm is more suited to identify prominent users especially when it is learned based on the spatial, on-and off-topic categories of features.

For future work, we aim to analyze the effectiveness of each feature characterizing prominent users independently of their category using different feature selection algorithms. Moreover, we wish to propose additional engineered features which are more representative for active microblog users in the context of disaster management.

  Precision = #Correctly classified prominent users #Users classified as prominent users Recall = #Correctly classified prominent users #Ground truth prominent users F1-score = 2×Precision×Recall Precision+Recall

Table 1 .

 1 Profile Features (PrF) from the microblogging platform Twitter.

	Name Features
	P1	Certified user [10]
	P2	Enabled geolocation [14]
	P3	Protected [10]
	P4	Number of produced tweets [10]
	P5	Number of collected favorites [14]
	P6	Creation date of the Twitter account [14]
	P7	Number of followers [10]
	P8	Number of followees [10]

Table 2 .

 2 On-topic User Activities Features (OnAF) and Off-topic User Activities Features (OfAF) in Twitter.

	Name Features

Table 4 .

 4 Network Structure Features (SnF) characterizing the social position of microblog users.

	Name Features
	NS1	Number of user's topical followers [13, 9]
	NS2	Number of user's topical followees[13, 9]
	NS3	Number of user's topical followers adjusted by the total number of followers [13, 14]
	NS4	Number of user's topical followees adjusted by the total number of his followees [13, 14]

Table 5 .

 5 Training and test datasets description

		Training Dataset (60%) Test Dataset (40%)
	Number of Prominent users	54	36
	Number of Non-prominent users	1945	1297

Table 6 .

 6 Effectiveness of each feature categories for prominent users identification in terms of Precision, Recall and F1-score evaluation metrics.

	Feature Categories	N o of features	SVM Precision Recall F1 Precision Recall F1 ANN
	OnAF*	12	0.43	0.86 0.57	0.29	0.80 0.42
	OfAF	11	0	0	0	0.04	0.33 0.07
	PrF	8	0	0	0	0.01	0.33 0.03
	SnF	4	0.05	0.02 0.03	0.09	0.61 0.15
	SpF	2	0	0	0	0	0	0

Table 7 .

 7 Effectiveness of each pair of feature categories (OnAf, An additional Feature Category) for prominent users identification in terms of Precision, Recall and F1-score evaluation metrics.

	Feature Categories	N o of features	SVM Precision Recall F1 Precision Recall F1 ANN
	OnAF+OfAF*	23	0.47	0.75 0.58	0.43	0.80 0.56
	OnAF+PrAF	20	0.42	0.86 0.56	0.36	0.86 0.51
	OnAF+SnF	16	0.40	0.86 0.55	0.24	0.66 0.35
	OnAF+SpF	14	0.43	0.86 0.57	0.39	0.88 0.54

Table 8 .

 8 Effectiveness of 3 combined feature categories (OnAf, OfAF, An additional Feature Category) for prominent users identification in terms of Precision, Recall and F1-score evaluation metrics.

	Feature Categories	N o of features	SVM Precision Recall F1 Precision Recall F1 ANN
	OnAF+OfAF+SpF	25	0.48	0.75 0.60	0.41	0.80 0.54
	OnAF+OfAF+PrAf	31	0.43	0.72 0.54	0.32	0.75 0.45
	OnAF+OfAF+SnF	27	0.45	0.75 0.56	0.36	0.77 0.50

Table 9 .

 9 Effectiveness of 4 combined feature categories (OnAf, OfAF, SpF, An additional Feature Category) for prominent users identification in terms of Precision, Recall and F1-score evaluation metrics.

	Feature Categories	N o of features	SVM Precision Recall F1 Precision Recall F1 ANN
	OnAF+OfAF+SpF+SnF	29	0.45	0.75 0.56	0.38	0.80 0.52
	OnAF+OfAF+SpF+PrAf	33	0.42	0.72 0.53	0.30	0.77 0.43