Maria Gomez

Romain Rouvoy

Martin Monperrus

Lionel Seinturier

María Gómez

A Recommender System of Buggy App Checkers for App Store Moderators

Keywords: Android Permissions, Android App Bugs, App Checkers, Online reviews

The popularity of smartphones is leading to an ever growing number of mobile apps that are published in official app stores. However, users might experience bugs and crashes for some of these apps. In this paper, we perform an empirical study of the official Google Play Store to automatically mine for such error-suspicious apps. We use the knowledge inferred from this analysis to build a recommender system of buggy app checkers. More specifically, we analyze the permissions and the user reviews of 46, 644 apps to identify potential correlations between error-sensitive permissions and error-related reviews along time. This study reveals error-sensitive permissions and patterns that potentially induce the errors reported online by users. As a result, our systems give app store moderators efficient static checkers to identify buggy apps before they harm the reputation of the app store as a whole.

The popularity of smartphones is leading to a rapidly growing number of mobile apps, which are distributed through dedicated stores. According to a recent study, 70% of mobile application developers targeted the Android platform in Q3 2014 [START_REF] Visionmobile | Developer economics q3 2014: State of the developer nation[END_REF], and the Google Play Store 1 currently offers over 1, 300, 000+ apps [3] for download.

However, there are many different app stores, depending on the platform, the country, and the business model. The role of an app store moderator is a complex one as s/he stands between app developers and end-users. On the one hand, endusers want high-quality apps, on the other hand, app developers want open platforms to express their creativity. Thus, app store moderators constantly make business and technical trade-offs to keep both users and developers happy and stay competitive in the ever-changing mobile computing landscape.

Any smartphone user knows that the quality of mobile apps greatly varies. For instance, we discovered a large number of exceptions raised in the wild on Android apps [START_REF] Haderer | Dynamic deployment of sensing experiments in the wild using smartphones[END_REF]. As Google Play Store lacks of source inspection before publishing apps, low-quality and malicious apps (e.g., malware) easily reach the store. Recently, CHABADA [START_REF] Gorla | Checking app behavior against app descriptions[END_REF] and WHYPER [START_REF] Pandita | WHYPER: Towards automating risk assessment of mobile applications[END_REF] introduced novel approaches for identifying malware Android apps, which misbehave with regard to their description. However, these approaches do not cover another challenging threat for app stores: buggy apps, which disrupt user experience and inevitably store quality.

An app checker is a mechanism to determine whether an app is potentially error-prone when the developer uploads it. If an app store use app efficient checkers, the overall quality of apps would raise. An app checker can be either dynamic or static, and in this paper, we focus on the latter. As all checkers 1 https://play.google.com/store (whether on mobile, desktop or server applications), app store checkers may suffer from false positives. In our case, it means that an app would be flagged as being potentially buggy while it is actually working fine. If an app store moderator enables checkers with too many false positives, it would be a dealbreaker for app developers. On the other end, if s/he disables all checkers, s/he risks hosting buggy apps, which would degrade the app store reputation.

To solve this problem, this paper proposes a recommender system of mobile app checkers for application moderators. This system enables app store moderators to make informed decisions on which checkers to enable based on quantitative measurements of their capabilities. Beyond existing researches on the use of permissions by Android apps [START_REF] Barrera | A methodology for empirical analysis of permission-based security models and its application to android[END_REF], [START_REF] Felt | Android permissions demystified[END_REF], [START_REF] Frank | Mining permission request patterns from android and facebook applications[END_REF], [START_REF] Stevens | Asking for (and about) permissions used by android apps[END_REF], we investigate the correlation between permission requests and errors reported by end-users. In particular, we perform an empirical study of the official Google Play Store to automatically mine permissions that correlate with bugs. From this study, we devise a taxonomy of permissions and a family of app checkers that can predict error-proneness upon submission of new apps.

To validate our approach, we identify a set of buggy apps using a data-mining technique and we compute the accuracy of each checker. From an analysis of the negative user reviews of 46, 644 apps, we build an oracle of error-proneness to measure the quality of our app checkers. These checkers exhibit an accuracy between 61.42% to 61.96%. Note that we identify symptoms of bugginess and we do not intend to point out the underlying causes of bugs. Our evaluation shows that app store moderators can be provided with large scale quantitative information regarding the relevance of enabling app checkers.

To sum up, the contributions of this paper are:

• We set up a family of permission-based checkers for mobile apps and integrate them into a recommender system for app store moderators; • We propose a novel evaluation scenario based on automatically mining 1, 400, 000+ online user reviews to identify a set of error-suspicious apps; • We report on a taxonomy of permissions that Android apps can request based on the study of 46, 644 apps available on Google Play Store. Using a grounded theory method the taxonomy classifies permissions under 4 categories and 5 specific classes; • We conduct an extensive analysis of the relevance of the recommender system of app checkers and the performance of the considered permission based checkers.

The remainder of this paper is organized as follows. Section II provides an overview of the Android permission model and a summary of our proposal. Section III describes the dataset we built for performing the empirical study. Section IV describes an approach for mining online user reviews and discover apps with reviews discussing about errors. Section V proposes a taxonomy of permission types for Android apps. Section VI presents the recommender system of buggy app checkers proposed. Section VII evaluates the approach. Section VIII summarizes the related work. Finally, Section IX concludes the paper and outlines future work.

II. OVERVIEW

In this section, we first provide a brief background on the Android permission model, and we then present an overview of our proposal.

A. Background on Android Permissions

Android apps run in a sandbox-i.e., an isolated area of the system that does not have access to system resources by default. Android provides a set of APIs to enable the application to access to data, resources, and privileged operations. The APIs are protected by a permission model: Apps must explicitly request the required permissions to access protected resources. Every app has a manifest file (AndroidManifest.xml) that summarizes essential information about the app. In particular, the manifest declares the permissions that the app requires for its execution, and a package name that serves as unique identifier for the app. When an app attempts to access a resource protected by a permission, the system checks the content of the manifest at runtime.

App developers are solely responsible for identifying the set of permissions required by their apps. Sometimes, developers fail assigning the adequate permissions, often because the available documentation of the Android permission system is incomplete [START_REF] Au | Pscout: Analyzing the android permission specification[END_REF], [START_REF] Felt | Android permissions demystified[END_REF], [START_REF] Vidas | Curbing android permission creep[END_REF]. However, a permission error often results in an exception (e.g., SecurityException) being thrown back to the app during execution [START_REF]Android-System Permissions[END_REF], often leading to the app crash.

As a matter of clarification, we distinguish between two terms that are used along the paper: Permission requests refer to permissions declared by apps in their manifest file ; permission types group available permissions that any app can request.

B. Overview of the Proposal

Our proposed approach consists of five steps, illustrated in Figure 1 together with the techniques and tools used: (1) We start by collecting Android app metadata from Google Play Store (Section III). ( 2) Then, for each app, we retrieve online reviews made by users in Google Play Store (Section IV-A).

(3) Using Latent Dirichlet Allocation (LDA) on the reviews, we isolate the reviews discussing about errors, and we identify a set of error-suspicious apps (Section IV-C). (4) The fourth step consists in analyzing the existing permissions declared by apps and proposing a taxonomy of these permissions (Section V). ( 5) Taking as input the permission classification performed in step 4 and the error-suspicious apps in step 3, we automatically mine buggy-permission patterns using a data mining algorithm and propose a family of buggy app checkers that constitutes the basis of a recommender system for assisting app store moderators (Section VI).

III. THE APP ANALYTICS FRAMEWORK

In this section, we describe the dataset we build to perform our empirical study and some key statistics about this dataset.

A. Dataset Collection

We build a dataset that consists of a random sample of all the mobile apps available on the Google Play Store. The apps belong to the 27 different categories 2 defined by Google, and the 4 predefined subcategories (free, paid, new free, and new paid). For each category-subcategory pair (tools-free, tools-paid, sports-new free, etc.), we collect a maximum of 500 samples 3 . In addition, for each app, we retrieve the following metadata: name, package, creator, version code, version name, number of downloads, size, upload date, star rating, star counting, and the set of permission requests.

Since the Google Play Store is continuously evolving (adding, removing and/or updating apps), we updated the dataset twice. We started by collecting available apps in November 2013 (D 0 ). Then, we updated the dataset in January 2014 (D 1 ) and March 2014 (D 2 ). In the rest of this paper, we use D 1 for performing the empirical study and D 2 for assessing the approach. confirms that the trend of adding more and more permissions pointed out by previous studies [START_REF] Wei | Permission evolution in the android ecosystem[END_REF] still holds. We have also observed that some apps are adding unnecessary permission requests when updating to a new version. For example, there are 74 apps adding the deprecated permission [appPackage].permission.MAPS_RECEIVE previously required to use the Google Maps API v1. The update of Google Play Services 3.1.59 (in July 2013) made this permission useless 4 . Furthermore, apps request 6 permissions by median, and at maximum, request 83 different permissions. Figure 3 reports on the number of apps requesting different number of permissions. For example, there are 180 apps requesting more than 35 different permissions. In contrast, there are 2, 592 apps that do not request any permission. Figure 4 compares our dataset with the datasets used in the related work in order to evaluate its representativity. The left Y axes reports in logarithmic scale the number of different apps considered. Our dataset is the second largest among the datasets reported in the literature on this topic. The right Y axes focuses on the number of different permissions. Our dataset contains the largest number of permissions: 1, 874 different permissions. While related work only focus on Android permissions (≈140 permissions), we consider other types of permissions available for Android apps, e.g. Google-specific permissions (cf. Section V for details). In conclusion, we believe the dataset is representative for performing the study.

B. Statistics on the Collected Apps

C. Representing Apps Ecosystem as a Graph

To store the dataset, we created a graph database with Neo4J5 . This dataset therefore consists of a graph describing the apps as nodes and edges. Graph databases provide a powerful and scalable data modelling and querying technique capable of representing any kind of data in a highly accessible way [START_REF] Robinson | Graph Databases[END_REF]. We chose a graph database because the graph visualization helps to identify connections among data (e.g., clusters of apps sharing similar sets of permission requests). Our dataset graph contains five types of nodes: APP nodes grouping the details of each app, PERMISSION nodes describing permission types, CATEGORY nodes describing app categories, SUBCATEGORY nodes describing app subcategories, and REVIEW nodes storing user reviews. Furthermore, there are four types of relationships between APP nodes and each of the remaining nodes. In total, our graph contains 1, 449, 361 nodes and 1, 901, 703 relationships among them.

To detail statistics about the dataset, we can query the graph database using one of the available graph query languages (e.g., Cypher, SPARQL, Gremlin). We chose Cypher [START_REF]Cypher[END_REF], which is a widely used pattern matching language. This graph database is available for download 6 .

IV. MINING ERROR-SUSPICIOUS APPS

We are interested in revealing potential correlations between the usage of error-sensitive permissions and the emergence of bugs, thus we first have to mine buggy apps. To automatically distinguish buggy apps that tend to crash during execution, we analyze the online reviews written by users in Google Play Store. This gives us an oracle of error-proneness. However, it is inconceivable to manually analyze millions of reviews available in the Google Play Store. Consequently, we use unsupervised machine learning (specifically topic mining) to automatically identify clusters of error-related topics.

A. User Review Mining

For each app, we collected the reviews posted by users in the Google Play Store. For each review, we extracted the title, description, associated device, and version of the app. None of these fields are mandatory, thus several reviews lack some of these details. As app updates might fix bugs from previous releases, some errors reported in the version 1.1 of a given app may be fixed in the new release 1.2. Therefore, from all the reviews attached to an app, we only consider the reviews associated with the latest version of the app. We collected up to a maximum of the latest 500 reviews per app, resulting in a corpus of 1, 402, 717 reviews. Each app has 36 reviews on average associated with the latest version of the app.

B. Topic Mining

To identify online reviews that treat topics related to bugs and errors, we extract topics discussed in these reviews using a topic modelling technique. Specifically, we apply the Latent Dirichlet Allocation (LDA) [START_REF] Blei | Latent dirichlet allocation[END_REF] algorithm. LDA identifies topics discussed in an entire corpus of documents and is able to analyze a large volume of unlabelled text. For each document, LDA estimates a probability distribution over the mined topics. For example, one document may have a probability of 0.7 to relate to topic #2 and of 0.3 to belong to topic #1 (the sum is always 1). In this case, the document mostly belongs to topic #2. The LDA model takes as an input parameter the number of topics to extract. The rational of using topic models is to automatically (1) cluster reviews discussing about errors, and (2) extract keywords that characterize error themes without sketching them beforehand. We are particularly interested in topic models that reveal bug-related keywords, which were unforeseen initially.

To implement this approach, we use MALLET (MAchine Learning for LanguagE Toolkit) [START_REF] Mccallum | MALLET: a machine learning for language toolkit[END_REF], a Java library that provides an implementation of LDA. We consider each single review of apps as a document, thus leading to 1, 402, 717 documents. To reduce noise in the modeled topics, we filter out English stop words 7 from the entire corpus. In order to achieve a better precision, in a preliminary exploratory phase, we ran the LDA algorithm using different number of topics. The best results are obtained for 100 topics, this is the value used in the experiments presented below.

C. Identifying Error-suspicious Apps

Our approach to identify error-suspicious apps comprises the following steps: 3) Select error-related reviews. To select error-related reviews, we analyze the probability distribution given by 7 We used the list of 524 common English stop words included in Mallet.

LDA for each review. Since a review can discuss several topics, we consider as error-related reviews those with at least 5% of its probability related to a topic discussing buggy behaviour. Examples of error-related reviews obtained8 : Astoundingly buggy. Get ready for the app to crash when you want it to work, and for it to linger in your notification bar even after you force close. I have no idea how such a phenomenal show has such a disappointing app.

Crashes all the time. I love TAL but this app is horrible. Actually it is a good app but it literally crashes every time I put it to use. Sometimes it will open up and I can start a show and listen to the whole thing and when it is done it crashes. Sometimes it crashes as it is opening.

4) Select error-suspicious apps. We consider as errorsuspicious apps, the set of apps containing at least n error-related review. The minimum number of errorrelated reviews is a parameter to be decided by app store moderators depending on the quality they want to ensure in their stores, and the confidence they have on the users reporting reviews. In this work, we have considered the minimum possible evidence of problems (n > 1 review).

To sum up, by applying unsupervised machine learning on the reviews of apps, we automatically learn a class label for each app: "Buggy" or "NonBuggy".

D. Empirical Results

Our process enables the identification of 10, 658 errorsuspicious apps (27.48%) in our dataset. Figure 5 shows the number of error-suspicious apps identified in each category of the Google Play Store. For each category, we illustrate the distribution of error-suspicious apps according to the number of error-related reviews published by users. With 68.90% of apps having error-related reviews, the category GAME contains the largest number of error-suspicious apps. These data are in line with the results of the study performed by Crittercism based on observations in real devices, showing that gaming apps have the highest crash rate [START_REF]Mobile experience benchmark[END_REF]. V. ANALYSING APP PERMISSIONS Now that we have identified error-suspicious apps, the next step is to analyze app permissions, in order to find correlations between permission and error-proneness. We start by proposing a taxonomy to classify the types of permissions that Android apps can request. This classification will later support the identification of error-sensitive permissions to propose a family of app permission checkers. To the best of our knowledge, we are the first to provide such a kind of taxonomy.

G A M E P E R S O N A L IZ A T IO N T O O L S A P P _ W A L L P A P E R P H O T O G R A P H Y E N T E R T A IN M E N T P R O D U C T IV IT Y M U S IC _ A N D _ A U D IO A P P _ W ID G E T S C O M M U N IC A T IO N M E D IA _ A N D _ V ID E O E D U C A T IO N B O O K S _ A N D _ R E F E R E N C E L IF E S T Y L E H E A L T H _ A N D _ F IT N E S S S O C IA L T R A V E L _ A N D _ L O C A L B U S IN E S S W E A T H E R S P O R T S F IN A N C E C O M IC S M E D IC A L L IB R A R IE S _ A N D _ D E M O N E W S _ A N D _ M A G A Z IN E S T R A N S P O R T A T IO N S H O P P IN G #Error-suspicious Apps

A. Taxonomy Creation

To create the taxonomy, we follow a grounded theory approach [START_REF] Strauss | Grounded theory methodology: An overview[END_REF]. We first review the literature and define a draft taxonomy based on observations made by prior works [START_REF] Barrera | A methodology for empirical analysis of permission-based security models and its application to android[END_REF], [START_REF] Felt | Android permissions demystified[END_REF]. As a matter of example, Felt et al. [START_REF] Felt | Android permissions demystified[END_REF] state that they consider neither developer-defined permissions nor Googledefined ones in their analysis. In addition, they reveal the existence of a set of non-official Android APIs. We include those types of permissions as categories in the taxonomy and we investigate them in depth. Another example is Barrera et al. [START_REF] Barrera | A methodology for empirical analysis of permission-based security models and its application to android[END_REF], who manually observed that some apps in their dataset request non-existent permissions and deprecated permissions. We also consider information provided in the official Android documentation for developers 9 and in Android community forums (e.g., Android Developers Blog10 ).

B. Permission Types Taxonomy

The taxonomy classifies permissions into four main categories:

1) Android-specific permissions refers to permission types to use the official APIs provided by the Android SDK 11 . For example, android.permission.READ_CONTACTS allows an app to read the user's contacts data. 2) Google-specific permissions groups the permission types to use the APIs provided by the Google Play Services SDK 12 , which include services such as: Google Play Licensing, Google Play In-app Billing, Google Maps, Google Cloud Messaging (GCM), etc. For example, the permission com.android.vending.BILLING is required to use the In-app Billing service that enables to sell products from inside an app. 3) Vendor-specific permissions encloses permission types to use the APIs included in SDKs provided by specific mobile device vendors (e.g., Samsung, HTC) to create apps specialized for their devices. As an illustration, Samsung provides the AllShare Framework SDK that includes APIs 13 to implement convergence services (e.g., media sharing, screen sharing). Apps using AllShare APIs should request the following permission: com.sec.android.permission.PERSONAL_ME-DIA 14 . Another example is the permission com.sonymobile.permission.CAMERA_ADDON that is part of the Sony Add-on SDK 15 (Camera Add-on API) required to develop apps that can be launched from the native Xperia camera. 4) Developer-specific permissions finally refers to permission types defined by third-party developers. Android enables apps to define their own permissions to protect the functionality they expose to other apps. For example, android.webkit.permission.PLUGIN from Adobe Flash plugin.

We group permissions within each category around 5 classes of permissions:

(a) Official permissions groups permissions that are available in the public SDK's documentations for developers. (b) Removed permissions refers to permissions that were available in previous versions of APIs and do not exist anymore. (c) Internal permissions encloses permissions that are intended for internal use by the system and system apps. (d) Incorrect permissions groups permissions defined erroneously by developers (e.g., misspelled permissions). (e) Unclassified permissions groups permissions that do not fit into any of the previous classes.

As we will show later, all the classes of permissions have been identified in the four categories of permissions.

C. Taxonomy Exploitation

We extracted all the permissions collected in our graph database and classify them according to the proposed taxonomy. We use heuristics, based on regular expression matching and string analysis, to catalog permissions under categories: Heuristic 1 (Android-specific permissions): First, to identify permissions belonging to the Androidspecific category, we filter permissions which follow the patterns: android.permission. Heuristic 4 (Classes refinement): We build lists of permissions to refine the permissions contained in each category under specific classes: official, removed, and internal. These lists are manually created by collecting documentation available in different sources. As a matter of example, the Android SDK contains a set of internal APIs that are intended for exclusive use by the system and system apps. However, they are available from the Android source code and third-party apps can access internal APIs using Java reflection [START_REF] Felt | Android permissions demystified[END_REF]. The internal APIs reside in the Android source code in the package com.android.internal, and in the other packages with the annotation @hide.

Heuristic 5 (Identifying incorrect permissions):

We select all the permissions within a category that are considered as unclassifiable in previous classes. To automatically identify incorrect permissions, we compute the Damerau-Levenshtein distance [START_REF] Damerau | A technique for computer detection and correction of spelling errors[END_REF] to measure the similarity between two input strings. We normalize the distance to the range [0, 1] by dividing the distance by the length of the longest string, thus defined as:

dist N DL (s1, s2) = dist DL (s1, s2) max(|s1|, |s2|) (1)
We compute the normalized Damerau-Levenshtein distance between each unclassified permission P and each official permission O i . Finally, we set the similarity score of an unclassified permission as the minimum distance of all the obtained normalized Damerau-Levenshtein distances:

score(P ) = min{dist N DL (P, O i )}, i = {0...n} (2)
Considering different similarity ranges, we observed 5 types of permission request mistakes in the Google Play Store:

Misspell.

For example, requesting android.permission.READ_CONCACTS instead of android.permission.READ_CONTACTS.

Wrong prefix. Each permission is identified by a unique label. Typically, the label is defined by a string starting with a prefix (e.g. android.permission.) followed by a constant in capital letters. We have identified a set of permissions defining incorrect prefixes, specifically two common mistakes: Prefix absence. For example, INTERNET is missing the prefix android.permission. The rational for this mistake seems to come from the official Android documentation on permissions, which first provides a table with the permissions showing only the constant part. Prefix confusion. For example, requests to android.permission.SET_ALARM, instead of com.android.alarm.permission.SET_ALARM.

Misuse. It is a special case of prefix misuse. Apps request as permissions other Android elements (e.g., libraries, features). For example: android.hardware.CAMERA.

Lacking. Finally, we also observed some permissions that look like being incomplete: android.permission.

In addition to these mistakes, we build a cluster of unclassified permissions: all the permissions within a category that do not fit in any of the defined classes. Finally, the developerspecific permissions category covers permissions that do not fit into any of the previous categories.

We manually validated the resulting permission classification. We took a random sample of 300 permissions. Our manual inspection found 5.33% of false positives. Some false positives derive from the interpretation of permissions. For example, the permission android.permission.SEND_MMS 17 was tagged as Misspelled Android permission while it is obviously not. The nearest match is the permission android.permission.SEND_SMS. However, we consider the developer is trying to access to a functionality related with MMS instead of SMS, and it is not a misspelling.

Table I synthesizes the results of measuring the abundance of each class of the the permission taxonomy to our dataset. For each category and class of the taxonomy we show the number of permissions and permission requests. We observed that official Android-specific permissions are the most commonly requested ones among apps. Removed permissions is the second most popular class among the dataset.

We consider as error-sensitive permissions the set of permissions belonging to the classes removed, internal, incorrect and unclassified, since the use of non official and valid permissions can be the source of problems. Now, we aim to identify potential permission patterns that correlate with bugs.

D. Influence of Permissions on Error-related Reviews

First, we ask the following research question: To what extent apps which have error-related reviews request errorsensitive permissions? In our dataset, there are 10, 658 apps having error-related reviews. Out of these 10, 658 apps, 1, 073 request error-sensitive permissions (10.07%). Thus, we raise the second research question: Do apps that request error-sensitive permissions have error-related reviews? In our dataset, there are 4, 288 apps requesting error-sensitive permissions, and from these, there are 2, 598 apps with at least one review (not necessarily bad). Out of these 2, 598 apps, 1, 304 have error-related reviews. This means that 50.19% of the apps requesting error-sensitive permissions can be considered as error-suspicious. This can be explained because some permissions that we have identified as error-sensitive do not really exhibit errors at runtime. For example, including some deprecated permissions in the manifest does not necessarily produce any side-effects. Furthermore, as pointed by prior studies, many apps declare more permissions than they actually use [START_REF] Felt | Android permissions demystified[END_REF].

This confirms that: 1) app bugs may be related to permissions and 2) there are many other reasons behind app bugs. In the next section, we further explore the statistical relationship. To clarify, not all the bugs are related to the suspicious permissions themselves (e.g., missing permission, wrong permission due to a typo), but rather to the fact that some APIs for which the permission is requested are buggy or obsolete [START_REF] Linares-Vásquez | API change and fault proneness: A threat to the success of android apps[END_REF]. We study the correlation between permission requests and buginess without claiming the underlying causes of bugs.

VI. DESIGNING A RECOMMENDER SYSTEM OF BUGGY APP CHECKERS

Based on our identification of error-suspicious apps (cf. Section IV) and error-sensitive permissions (cf. Section V), we now study the relationship between them in order to identify potential correlations between apps that use errorsensitive permissions and those reported as error-suspicious by end-users. We use the knowledge inferred from this study to propose buggy app checkers that could be embedded in app stores, such as the Google Play, to anticipate the emergence of app crashes in the wild.

A. Generating App Permission Checkers

Now, we ask the following research question: Are there some common permission patterns in the presence of errorsuspicious apps? With the objective of mining common permissions in error-suspicious apps, we use supervised machine learning, and specifically a classifier where independent variables are permissions and the dependent variable is errorproneness.

1) Dataset Preprocessing: Let D be a dataset of apps, we represent each app (A) as a binary vector: A = [p 1 , ..., p n , c 1 , ..., c n , L], where p i ∈ {0, 1}, c i ∈ {0, 1}, and L ∈ {Buggy, N onBuggy}. Each p i depicts a permission available in our database, c i represents a class of permission (according to our taxonomy), and L represents the class label learnt from its reviews (cf. Section IV-C). The value p i =1 indicates that the app requests the permission p i , and p i =0 indicates the absence of the permission. Similarly, c i =1 indicates that the app requests a permission that belongs to the class c i (e.g., INCORRECT class) in our taxonomy.

For the purpose of this analysis, we group all the classes of infrequent single permissions (e.g., the misspelled ones) within a common abstract class. Indeed, a specific misspelled permission usually only appears in one app, but we are rather interested in knowing if the group of incorrect permissions is frequent in error-suspicious apps. There are also some permission names that are customized in each app. For example, apps requesting the GCM Google service must include a specific permission for receiving messages:

[appPackage].permission.C2D_MESSAGE. The permission name must exactly match the pattern, but each app substitutes [appPackage] by its own package name in the manifest. This permission prevents other apps from registering and receiving their messages. Therefore, each specific GCM permission is only requested once at maximum. Moreover, we grouped all the official Android permissions in a single dimension 18 . We notice that the 10 most requested official Android permissions in our dataset are the same top requested permissions observed by other studies [START_REF]Android Observatory[END_REF], [START_REF] Frank | Mining permission request patterns from android and facebook applications[END_REF] 19 .

Finally, the 4 classes of permissions considered are: ANDROID, INCORRECT, GCM, and REMOVED-GOOGLE permissions. Each app is represented by a 1,563-dimensions vector, where the first 1, 558 dimensions represent single permissions (p i ), the following 4 dimensions refer to classes of permissions (c i ), and the last dimension is the assigned class label (L).

2) Mining Error-suspicious Permission Patterns:

To identify permission patterns that correlate with bugs, we use J48 Decision Tree algorithm (a Weka20 implementation of C4.5 [START_REF] Quinlan | C4. 5: programs for machine learning[END_REF]), which is a predictive machine-learning model. J48 predicts the value of a dependent variable based on the value of various attributes (independent variables) of the data. In our setup, the independent variables are permissions, and the dependent variable is bugginess. From the resulting decision tree model, we extract permission patterns that lead to the label 'Buggy'. We only consider presence of permissions in the patterns. However the absence of a permission also causes the app to crash. This kind of bug is partially handled by our approach, when the permission is missing because of an incorrect or incomplete declared permission. Furthermore, there already exists some developer support [START_REF] Bartel | Automatically securing permission-based software by reducing the attack surface: An application to android[END_REF] to automatically check if apps declare all the permissions required to run.

The algorithm takes as input parameters a confidence factor and a minimum number of objects (minN umObj). The conf idencef actor limits the prediction error. For example, a confidence factor of 0.25 indicates that a permission pattern fails as maximum in the 25% of predictions. Thus, the lower the confidence factor, the more accurate the classifier. The minN umObj parameter sets the minimum number of instances that reach a leaf of the tree model. In our case it represents the minimum number of apps that must exhibit a permission pattern to be considered as a predictor of bugginess.

We train a J48 decision tree model using the dataset D 1 that contains 10, 658 apps labelled as Buggy and 11, 539 labelled as NonBuggy. Varying the conf idence and minN umObj thresholds impacts the number of permission checkers obtained and their respective performance. We run the algorithm several times for different input values in order to identify the best calibration. We set cross-validation 10 folds. Then, we train the model with different confidence factors (ranging from 0.05 to 0.50 by increments of 0.05) with three different minN umObj limits-i.e., 100, 50 and 20 apps. Figure 6 reports on the results of the sensitivity analysis performed. The resulting family of checkers ranges from 4 to 12 different permission checkers. The family of permission checkers obtained form the knowledge of the recommender system. Using buggy app checkers, app store moderators can make informed decisions on which checkers to enable in order to identify potential buggy-apps before they are published in the store. Table II details 4 families of permission checkers obtained varying the conf idence factor and minN umObj values. For example, the family F 2 includes the 4 permissions pointed by F 1 and one additional permission. In the family F 1, we observe two official Google permissions: CHECK_LICENSE and BILLING. Contrary to our expectations, these official Google permissions are involved in some bugs. After searching in online forums for Android developers, we realize that there are many Android developers complaining because they have experienced crashes (due to security exceptions) in their apps after the update of Google Play services 4.3 (March 2014).

Note that the buggy permission checkers presented in this paper are not meant to be permanent. The app ecosystem is continuously evolving, and app store moderators can use our approach regularly (say monthly) for updating existing checkers, discovering new ones, and discarding outdated ones.

The proposed system reveals interesting insights for isolating bugs in real devices. Nevertheless, we do not claim causality, but rather we suggest permissions that correlate with bugs.

VII. EVALUATION

To assess the effectiveness of our approach, we investigate three main research questions: RQ1: What is the accuracy of the inferred checkers? RQ2: To what extent checkers are able to flag new apps as buggy? RQ3: What is the effect of removing error-sensitive permissions on error reviews?

A. Evaluating the Accuracy of App Checkers

Checkers often have false positives, which in our case means that the app is flagged as buggy while still works fine. To sum up, if the moderator enables checkers with too many false positives, it can often flag apps as having bugs while they are working fine, then disturbing app developers. On the other end, if s/he disables all checkers, s/he risks hosting buggy applications, which disrupt app user experience and degrade the store reputation. For answering RQ1, we therefore evaluate the accuracy of each checker. To evaluate the accuracy of the proposed app checkers, we use the Laplace expected error estimate [START_REF] Clark | Rule induction with cn2: Some recent improvements[END_REF] which is computed as follows:

LaplaceAccuracy = (n c + 1)/(n tot + k) (3)
where k is the number of classes in the domain, e.g. Buggy/NonBuggy. n tot is the total number of examples covered by the checker. In our case, is the total number of apps requesting the permissions captured by the checker. n c is the number of examples in the predicted class by the checker.

In our case, is the number of Buggy apps that request the permissions captured by the checker.

We set cross-validation 10-folds and we perform a sensitivity analysis of the accuracy of the checkers for different values of input parameters. Figure 7 shows the accuracy of the recommender system. The family of checkers exhibits an accuracy that ranges from 61.42% to 61.96%. We observe that the reported accuracy does not change significantly for different values of the parameters. Table II shows the accuracy (acc.) values for some of the permission checkers identified. We also report the total number of Buggy (#B) and NonBuggy (#NB) apps which request the error-sensitive permissions captured by the checkers.

B. Evaluating Checkers with New Apps

For answering RQ2, we use the dataset D 2 updated in March 2014 (cf. Section III-B). We select the subset of new apps that appeared in D 2 and did not exist in D 1 . D 2 contains 6, 783 new apps (with reviews). We use the user reviews as our ground-truth for determining if an app is buggy or not 21 . Thus, we check if the D 2 -version apps flagged as suspicious by our checkers have been reported as buggy by end users (after the publication of the version of D 2 ). Our system flagged 1, 896 apps as error-suspicious. Out of 1, 896 suspicious apps, 56% of apps were also reported as buggy by end users.

The 56% of new apps flagged as suspicious by our checkers were also reported as buggy by end users a posteriori.

C. Impact of Removing Error-sensitive Permissions

To answer RQ3, we investigate if the apps that remove error-sensitive permissions (captured by the checkers) in the update get less error-related reviews. We noticed 30 apps (with error-related reviews) that remove error-sensitive permissions after updating. For these apps, we compute the percentage rate of error-related reviews before and after the update. Figure 8 illustrates the evolution of the reviews in these apps. We observe that after removing error-sensitive permissions, 22 apps (out of 30) remove error-related reviews. If the checkers had been enabled in the app store, all those apps would have been flagged as suspicious, thus avoiding buggy apps reaching end-users.

D. Discussion and Threats

We focus on permission requests and user reviews to build a family of buggy app checkers that can help app store moderators to score the quality of a submitted app. One threat to the internal validity of our study is that we have not analyzed the source or binary code of apps to ensure that the declared permissions are actually used. 21 We are working on deploying experiments in real devices in the wild to complement the quantitative study performed in this paper We consider user reviews as a ground truth of bugginess, but we are aware that this measure is only an approximation. Users do not always report crashes when faced, and some bugs can be reported later in time. In addition, apps can crash for many reasons, not only due to permission-related issues. Currently, we are working on experiments in real devices to address this threat.

0% 10% 20% 30% 40% 50% 60% A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 A 1 0 A 1 1 A 1 2 A 1 3 A 1 4 A 1 5 A 1 6 A 1 7 A 1 8 A 1 9 A 2 0 A 2 1 A 2 2 A 2 3 A 2 4 A 2
The conceptual foundations of our approach are independent of Android. We only need an oracle of bugginess and some observation features (in our case, the user reviews and the requested permissions, respectively). To gain in confidence in the external validity of our evaluation, more evaluations are needed on other platforms, using other oracles of bugginess and features.

Let us now discuss the sensitivity of our approach. The proposed approach is based on predefined thresholds specified by various parameters. Table III summarizes these parameters together with the reasonable values used in the experiments reported in this paper. The calibration of these parameters can impact the results and constitute a potential threat to validity. We have performed a sensitivity analysis for different values of these parameters. The reported accuracy does not change significantly for different values.

VIII. RELATED WORK

We divide related work into two major groups, and survey the literature in each group. Android Permission analysis. Frank et al. [START_REF] Frank | Mining permission request patterns from android and facebook applications[END_REF] propose a probabilistic model to identify permission patterns in Android All previous studies only focus on official Android-specific permissions, we also consider Googledefined, Vendor-defined and Developer-defined permissions in our analysis.

Xu in his thesis [START_REF] Xu | Techniques and Tools for Analyzing and Understanding Android Apps[END_REF] presents a systematic study of apps from Google Play Store. With reference to permissions, the study observed developer-related errors. However, Xu does not propose a technique for setting up buggy app checkers.

Our approach is also related to approaches that focus on identifying malicious behaviour in Android apps. CHABADA [START_REF] Gorla | Checking app behavior against app descriptions[END_REF] proposes an API-based approach to detect apps which misbehave regarding their descriptions. CHABADA clusters apps with similar descriptions and identifies API usage outliers in each cluster. These outliers point out potential malware apps. Similarly to our approach, CHABADA uses topic modelling techniques. They apply LDA in the app descriptions for grouping apps with similar themes. In contrast, we use topic models on online user reviews (and not on app descriptions) to automatically identify buggy apps (as opposed to identify malware). Our aim is to identify permission patterns that correlate with bugs.

On the static analysis side, there are several works focusing on analyzing the source code and API calls in Android apps to check if the declared permissions are actually used: COPES [START_REF] Bartel | Automatically securing permission-based software by reducing the attack surface: An application to android[END_REF], PScout [START_REF] Au | Pscout: Analyzing the android permission specification[END_REF], and Permlizer [START_REF] Xu | Permlyzer: Analyzing permission usage in android applications[END_REF]. This is only indirectly related to checkers for identifying buggy apps.

Online user review analysis. Several approaches have analysed online user reviews posted on the Google Play Store with different purposes.

Ha et al. [START_REF] Ha | Do android users write about electric sheep? examining consumer reviews in google play[END_REF] manually analysed user reviews available on Google Play Store to understand what users write about apps. Performing this task manually becomes infeasible due to the large amount of available reviews.

Chen et al. [START_REF] Chen | AR-Miner: mining informative reviews for developers from mobile app marketplace[END_REF] present AR-Miner, a tool for mining reviews from Google Play Store and extract user feedback. They filter reviews that contain useful information for developers to improve their apps. As us, they use LDA to group the reviews discussing about the same thematics. Fu et al. [START_REF] Fu | Why people hate your app: Making sense of user feedback in a mobile app store[END_REF] propose WisCom, a system to analyze user reviews and ratings in order to identify the reasons why users like or dislike apps. Iacob and Harrison [19] propose MARA, a prototype for extracting feature requests from online reviews of mobile applications. First, they identify the sentences of the online reviews referencing to feature requests using a set of predefined rules. Finally, they use LDA for identifying the most common topics among the feature requests. As our approach, all these approaches use LDA for identifying topics discussed in the reviews. However, none of them use reviews as an oracle of error-proneness. We focus on reviews to identify a set of error-suspicious apps and to automatically learn a class label for each app.

Linares-Vásquez et al. [START_REF] Linares-Vásquez | API change and fault proneness: A threat to the success of android apps[END_REF] demonstrate a correlation between the stability of APIs and the success of Android apps. They also use user reviews for finding evidence of problems. They manually analyze the reviews of apps which contain low ratings. They look for reviews that contain some predefined error-related keywords. Their aim is to assess that the identified unsuccessful apps express the cause of the errors in their reviews. On the contrary, we automatically analyze the reviews of all apps, and identify error-related reviews using topic models.

IX. CONCLUSION AND FUTURE WORK

In this paper, we propose a recommender system that detects potentially buggy apps by using the correlation between permission patterns and error-proneness. We collected 46, 644 Android apps from Google Play Store with the aim of studying the correlation between permission requests and the emergence of errors and crashes. We started by mining 1, 400, 000+ online user reviews to identify error-related reviews that point out error-suspicious apps. We used unsupervised machine learning techniques to automatically identify error-suspicious apps available in the Google Play Store. We then propose a taxonomy of types of permissions requested by Android apps that supports the analysis of permissions. With the knowledge inferred from this analysis, we built permission-based checkers that are then ranked by our the recommender system. The permission-based checkers recommended by this system have an accuracy between 61.42% to 61.96%.

Currently, we are working on developing experiments in real devices in the wild with the set of error-suspicious apps identified. We will use the insights obtained in this study as starting point for isolating errors.

Figure 2

 2 Figure2depicts the evolution of our dataset along time in terms of additions, removals and updates of apps and permission types. D 1 contains 38, 781 apps requesting 7, 826 different permissions, while D 2 contains 46, 644 apps and 9, 319 different permission requests. The observed evolution tends to add new apps, more than updating or removing existing ones. In January, 15, 023 new apps appeared, 9, 001 apps (33, 13%) were updated, and 3, 411 apps (12, 55%) were removed. Similarly in March, 12, 543 new apps appeared, 8, 970 apps (23, 13%) were updated and 4, 680 apps (12, 07%) disappeared.

Fig. 2 .

 2 Fig. 2. Evolution of apps and permissions in Google Play Store.

Fig. 3 .

 3 Fig. 3. Distribution of apps per number of permission requests.

FFig. 4 .

 4 Fig. 4. Comparison of number of apps and permission types used in our dataset and related work's datasets.

ErrorFig. 5 .

 5 Fig. 5. Distribution of error-suspicious apps by categories in Google Play.

Fig. 6 .

 6 Fig. 6. Calibration of confidence factor and minNumObj parameters.

Fig. 7 .

 7 Fig. 7. Evaluation of the accuracy of the recommender system.

Fig. 8 .

 8 Fig. 8. Reviews evolution after removing error-sensitive permissions.

  Un Système de Recommandation de Vérificateurs d'Applications Boguées pour les Modérateurs de Magasins d'Applications Résumé : La popularité des smartphones engendre un nombre croissant d'applications mobiles qui sont publiés dans des magasins officiels d'applications. Cependant, les utilisateurs peuvent rencontrer des bogues et incidents avec certaines de ces applications. Dans cet article, nous effectuons une étude empirique du magasin officiel Google Play Store pour extraire automatiquement ces applications suspectes. Nous utilisons ensuite la connaissance construite au cours de cette analyse pour proposer un système de recommandation de vérificateurs d'applications boguées. Plus précisément, nous analysons les permissions et les commentaires des utilisateurs de 46, 644 applications pour identifier des corrélations potentielles entre des permissions identifiées comme sensibles et les commentaires liés à des erreurs au cours du temps. Cette étude révèle des permissions particulièrement sensibles aux erreurs ainsi que des associations qui sont potentiellement la source des erreurs signalées par les utilisateurs en ligne. En conséquence, notre système de recommandation offre aux modérateurs de magasins d'applications une gamme de vérificateurs statiques pour identifier efficacement les applications boguées avant qu'elles ne nuisent à la réputation du magasin dans son ensemble.

Mots

-clés : Permissions Android, Bogues d'Applications Android, Vérificateurs d'Applications, Commentaires des Utilisateurs I. INTRODUCTION

time in terms of additions, removals and updates of apps and permission types. D 1 contains 38, 781 apps requesting 7, 826 different permissions, while D 2 contains 46, 644 apps and 9, 319 different permission requests. The observed evolution tends to add new apps, more than updating or

	removing existing ones. In January, 15, 023 new apps appeared, 9, 001 apps (33, 13%) were updated, and 3, 411 apps (12, 55%) were removed. Similarly in March, 12, 543 new apps appeared, 8, 970 apps (23, 13%) were updated and 4, 680 apps (12, 07%) disappeared.
	We observe that the app updates tend to rather add permissions than remove existing ones. Out of the 3, 411 apps that updated to a new version in January 2014, there are 2, 677 apps (78.48%) that update their set of permis-sion requests. Specifically, there are 3, 007 permission re-quest removals and 5, 225 permission request additions. This
	2 Books&Reference, Business, Comics, Communication, Education, En-tertainment, Finance, Games, Health&Fitness, Libraries&Demo, Lifestyle, Live Wallpaper, Media&Video, Medical, Music&Audio, News&Magazines, Personalization, Photography, Productivity, Shopping, Social, Sports, Tools, Transportation, Travel&Local, Weather, Widgets.

3

This number is a constraint enforced by the Google Play Store.

1 )

 1 Extract topics. Using LDA, we mine the topics discussed in the app reviews. With our default parameter, we obtain

	100 topics.
	2) Select relevant topics. From the mined topics, we manually select the topics that are related to bugs and crashes. From our dataset, we automatically identify the following 3 relevant topics: #24: fix update problem fixed bug issue crashes phone stars bugs plz pls time crashing problems working issues crash hope #48: work doesn doesnt won didn working open kit show load kat properly anymore sucks bad note worked android wont #81: app crashes force time open fix close closes won crashing work crash start freezes working times constantly closing doesn

  [ * ] or com.android.[ * ].permission.[ * ]. From these permissions, we tag as official the permissions that match the list provided in the public Android documentation for developers according to Android 4.4 (API level 19)16 .

	gle.android.voicesearch.SHORTCUTS_ACCESS in the Voice Search app.
	Heuristic identify permissions 3 (Vendor-specific belonging specific category, we filter permissions following the permissions): To to the Vendor-patterns com.[vendor].[ * ].permission.[ * ] or android.permission.[vendor].[ * ]. We compile a predefined list of available vendor names: sonymobile, htc, huawei, sec (Samsung Electronics Co), dell, and motorola.
	Heuristic 2 (Google-specific permissions): To identify permissions belonging to the Google-specific category, we filter permissions which follow the patterns: com.google.android.[ * ].permission.[ * ], or com.android.[ * ]. The last pattern excludes the word ''permission'' as it matches the Android-specific pattern. Examples of Google-specific permissions that follow this pattern are com.android.vending.BILLING and com.android.vending.CHECK_LICENSE to use the billing and licensing services, respectively. From these permissions, we define the class of internal Google permissions with the permissions defined by Google apps (e.g., Voice Search, GMail). These permissions follow the patterns com.google.android.apps.[Go-ogleAppName].[ * ] or com.google.android.google-apps.permission.[ * ]. For example, com.goo-

TABLE I .

 I SUMMARY OF APPLYING THE PERMISSION TAXONOMY TO OUR GOOGLE PLAY STORE DATASET

	Android Google Vendor Samsung Sony HTC Dell Motorola Huawei Developer	Official #Perm #Req 137 245, 274 5 18, 246 184 1, 514 88 302 38 291 28 251 14 14 12 649 4 7 1, 085 2, 875	Removed #Perm #Req 19 1, 699 837 1, 107 ----------------	Internal #Perm #Req 50 1, 040 51 200 ----------------	Incorrect #Perm #Req #Perm Unclassified #Req 144 561 152 1, 240 39 61 3 22 --------------------------------
	TOTAL	1,411	267,909	856	2,806	101	1,240	183	622	155	1,262

TABLE II .

 II EVALUATION: APP PERMISSION CHECKERS AND THEIR ACCURACY.

	C	M		Error-sensitive Permission Checker	#B	#NB	%Acc
				android.permission.WRITE_INTERNAL_STORAGE
	F1 0.1 100		android.permission.ACCESS_SUPERUSER com.android.vending.CHECK_LICENSE	2, 888 1, 789	61.74
	com.android.vending.BILLING F2 0.2 100 F1 com.android.launcher.permission.READ_SETTINGS	2, 932 1, 824	61.64
	F3 0.2	50	F1	android.permission.WRITE_OWNER_DATA com.sonyericsson.extras.liveware.aef.EXTENSION_PERMISSION	2, 928 1, 834	61.48
				com.google.android.googleapps.permission.GOOGLE_AUTH
	F4 0.2	20	F3	android.permission.STORAGE com.android.email.permission.READ_ATTACHMENT com.google.android.gm.permission.READ_CONTENT_PROVIDER	2,986	1,841	61.86

TABLE III .

 III SUMMARY OF THE APPROACH'S PARAMETERS ] studied the permission requests made by apps in the different categories of the Google Play store by mapping apps to categories based on their set of requested permissions. They show that a small number of Android permissions are used very frequently while the rest are only used occasionally. In[START_REF] Jeon | android and mr. hide: Fine-grained permissions in android applications[END_REF], Jeon et al. proposed a taxonomy that divides official Android permissions into four groups based on the permission behaviours (e.g., access to sensors, access to structured user information). For each category, they propose new fine-grained variants of the permissions. Chia et al.[START_REF] Chia | Is this app safe?: A large scale study on application permissions and risk signals[END_REF] performed a study on Android permissions to identify privacy risks on apps. They analyze the correlation between the number of permissions requested by apps and several signals such as app popularity and community rating. Our taxonomy has a different goal, the aim of our classification is helping to identify errorsensitive permissions.

	Parameter LDA model number of topics threshold (IV-B) Error-related reviews threshold in suspicious apps (IV-C) Composition threshold in error-related reviews (IV-C) J48 model conf idence factor (VI-A2) J48 model minN umObjects (VI-A2)	Value 100 > 1 0.05 0.2 20
	and Facebook apps. They found that permission patterns differ between high-reputation and low-reputation apps. Barrera et al. [5

http://www.neo4j.org

https://sites.google.com/site/androidbuggyappcheckers

The reviews belong to the app: "This American Life" (version 2.1.8), https://play.google.com/store/apps/details?id=org.prx.talbot. The dates of the reviews are13-12-2013 and 18-12-2013, respectively.

http://developer.android.com

http://android-developers.blogspot.com

http://developer.android.com/sdk

http://developer.android.com/google

Samsung documentation for developers: http://developer.samsung.com/ develop

"sec" comes from: Samsung Electronics Co

Sony documentation for developers: http://developer.sonymobile.com

http://developer.android.com/reference/android/Manifest.permission.html

The request of Android permissions should not make the app crash, and we are rather interested in revealing the existence of patterns in error-sensitive permissions. First, we run the experiment considering all the official Android single permissions, but the most requested permissions appeared in many checkers, leading to high amount of irrelevant patterns.

The most requested permissions in our dataset are: INTERNET, ACCESS_NETWORK_STATE, READ_EXTERNAL_STORAGE, WRITE_EXTERNAL_STORAGE, READ_PHONE_STATE, ACCESS_WIFI_STATE, WAKE_LOCK, ACCESS_FINE_LOCATION, ACCESS_COARSE_LOCATION, and VIBRATE

Weka is an open-source Java library that provides implementations of several data mining algorithms: http://www.cs.waikato.ac.nz/ml/weka

https://developers.google. com/maps/documentation/android/releases?hl=en#july 2013

RESEARCH CENTRE LILLE -NORD EUROPE


Keywords: Concept extraction from text, information integration, ontology-based data access, semantic search

The problem that we address in this paper is how to improve the accuracy of retrieving specialized information within a textual scientific corpus. We present a new approach in which the keywords expressing the bibliographical needs of a researcher are related to a domain ontology. We illustrate how such a declarative ontolology-based approach can be used both for computing varied statistics, and also for helping experts to find useful fine-grained information within a textual corpus.

I. INTRODUCTION

OR researchers, finding the bibliographical articles likely to improve their knowledge on their domain of expertise is a crucial but time-consuming task. Subscription keywords-based search tools exist to help researchers monitoring the web or bibliographic databases of their domain. However, due to the limitation of keywords to capture fine-grained knowledge, the bibliographical articles returned by keywords filtering need a further analysis to retain those that are really relevant to the needs of the researcher. Up to now, this analysis is done manually by the researcher herself of by a human assistant. In this paper, we present a new approach in which the keywords expressing the bibliographical needs of a researcher are related to a fine-grained description of her domain of expertise in the form of an ontology. An ontology is a formal description providing human users a shared understanding of a given domain, and can also be interpreted and processed by machines thanks to a logical semantics that enables inference and reasoning. In this paper, we first summarize the methodology underlying our approach, that takes as input a domain ontology and a corpus of articles, and constructs automatically a knowledge base enriching the input domain ontology with the relevant content and structure of the corpus. Then we illustrate through examples, the interest and the power of such a declarative ontology-based approach both for computing varied statistics on the corpus Manuscript submitted July 2, 2014. This work has been partially supported by the LabEx PERSYVAL-Lab (ANR-11-LABX-0025-01) and by IUF (Institut Universitaire de France).

(1) University Grenoble Alpes, LIG, FR-38000 Grenoble, France (contact e-mail: marie-christine.rousset@imag.fr, others email: firstname.lastname@imag.fr).

(2) University of Crete, Greece.

(3) CHU, INSERM UMR1043 and University of Toulouse III, FR-31062 Toulouse, France (e-mail: tauber.mt@chu-toulouse.fr). and its correlation with formal terms of a domain of interest, and also for helping experts to find useful fine-grained information within a textual corpus.

II. METHODOLOGY

Our approach consists in using a domain ontology for extracting the relevant content of a given bibliographical corpus and for representing it as a knowledge base equipped with reasoning and querying capabilities.

Step 1 is the design of ontology that can be either created manually by a domain expert or extracted from a preexisting one. The ontology is provided using Semantic Web standards [START_REF] Hayes | Recommendation, World Wide WebConsortium[END_REF] (namely, RDFS enriched with rules), and stored as a set of RDF triples and a set of rules. Each formal term introduced by the expert is associated with a set of textual expressions that correspond to it in the scientific literature. This results in a table of correspondence used as input in the second step.

Step 2 consists in the automatic extraction from each paper in the corpus of the sentences that contain textual expressions corresponding to formal terms of the ontology. This is done using a suite of simple Perl scripts that take as input the corpus and the table of correspondence.

Step 3 consists in automatically assigning each paper and each extracted sentence to unique identifiers that are typed by the classes Papers and Sentences respectively. This information is encoded into RDF triples that are automatically inserted into the domain ontology. RDF triples expressing that each paper's identifier is related to the paper's title are also inserted in the ontology by using the RDF property rdfs:label. Similarly, each sentence identifier is associated to its textual content by using the RDF property rdf:comment and the corresponding triples are injected in the ontology. We have also defined the following properties for which the corresponding triples are also automatically injected in the ontology:

-publishedIn: to relate a paper's identifier to its year of publication, -isaSentenceOf: to relate a sentence's identifier to the paper it belongs to -mentions: to relate a sentence to a formal term (instance, class or property) of the domain ontology, -isAbout: to relate a paper to a formal term (instance, class or property) of the domain ontology. A paper (respectively a sentence) may be about (respectively may mention) several formal terms. In fact, this last property isAbout is defined from the properties mentions and isaSentenceOf by a rule saying that: if a sentence ?s belonging to a paper ?p mentions a formal term We have chosen to store and process the resulting knowledge base as a deductive RDF triple-store using, but not limited to, TopBraid Composer [START_REF]TopBraid Composer[END_REF]. TopBraid Composer is a commercial tool specifically designed for RDF, which is also available as free version. It fully supports the SPARQL query language for expressive querying and also has built-in support for custom inference rules. These rules are applied to infer automatically all the RDF triples that can be entailed from the original triples and the rules.

Step 4 consists in exploiting the resulting knowledge base with reasoning and querying capabilities supported by TopBraid Composer. This knowledge base, made of a set of RDF triples and a set of rules, is a deductive database that can be saturated [START_REF] Abiteboul | Web Data Management[END_REF] and then queried using the query language SPARQL [START_REF] Prud'hommeaux | SPARQL query language for RDF[END_REF]. SPARQL is the SQL of RDF: it is a W3C standard allowing to ask possibly complex queries on demand. SPARQL is a powerful structured query language that provides a full set of query operations such as SELECT, JOIN, SORT and aggregation operators based on GROUP-BY. It can be used to compute simple statistics and also to compute answers to queries entered by human experts.

III. NOVELTY

Compared to existing works on semantic search issued from the Information Retrieval community (such as [START_REF] Malo | Semantic content filtering with Wikipedia and ontologies[END_REF], [START_REF] Perez-Iratxeta | Update on XplorMed: a web server for exploring scientific literature[END_REF], [START_REF] Webber | Evaluating the Effectiveness of Keyword Search[END_REF], [START_REF] Qiu | Concept based query expansion[END_REF], [START_REF] Santos | Improving search engine Query Expansion techniques with ILP[END_REF], [START_REF] Perez-Iratxeta | Update on XplorMed: a web server for exploring scientific literature[END_REF], [START_REF] Tsuruoka | FACTA: a text search engine for finding associated biomedical concepts[END_REF], [START_REF] Egozi | Concept-Based Feature Generation and Selection for Information Retrieval[END_REF]), the distinguishing point of our approach is to offer a powerful SQL-like query language that enables a unified and powerful approach to express fine-grained queries combining structure, content and semantics. The results returned to the users are not a list of ranked documents but precise fine-grained answers and correlations between formal terms of a specialized domain knowledge and relevant sentences within documents.

The only code that has to be used is generic and is restricted to implement the steps 2 and 3 described above in the methodology. The queries and statistics of interest can be specified declaratively and on demand using SPARQL, and computed by any SPARQL query engine. Exploiting the SPARQL power allowed us to develop a uniform and declarative query-based approach for a fine-grained analysis of a specialized bibliographical corpus. This approach enables querying capabilities that goes much beyond the concept-based search capabilities offered by semantic search engines such as TextPresso [START_REF] Müller | Textpresso: an ontology-based information retrieval and extraction system for biological literature[END_REF] for example, and more generally by information retrieval systems. In addition, in contrast with our approach, the computation of statistics of interest is not supported by (semantic) search engines and must be done by writing Perl or Python scripts or by using dedicated tools.

IV. ONTOLOGY-BASED DESCRIPTION OF A TEXTUAL CORPUS

Our approach is generic but in order to explain it and to show its feasibility, we have focused on a specialized medical domain related to a rare disease (Prader-Willi syndrome [START_REF] Cassidy | Prader-Willi syndrome[END_REF]) of which one of us is an expert and for which we had access to a corpus of almost 6000 articles representative of the broad spectrum of the literature relevant to this rare syndrome. We first give an overview of the domain ontology and of its automatic enrichment. We then show through examples the power of SPARQL queries both for formulating expert demands of information and for computing some useful statistics on the corpus and its content. The triple <obesity, rdf:type, Symptom_PW> is the declaration of the fact that obesity is a symptom encountered in Prader Willi syndrome, while the following triple <Behaviour_Troubles_PW,rdfs:subClassOf, Troubles_PW encodes the knowledge that behaviour troubles are a special kind of troubles occurring for patients suffering from Prader Willi syndrome. Figure 1 shows an extract of the resulting class hierarchy.

A. Overview of the Prader-Willi ontology

Example 2:

If <?i , rdf:type, ?c1> and <?c1, rdfs:subClassOf, ?c2> Then <?i, rdf:type, ?c2>. The variables (indicated by ?) are placeholders that can be replaced by constants denoting specific instances and classes accordingly. Such a rule will infer for instance that boulimia (declared as an instance of hyperphagia, itself asserted as a subclass of Eating_Disorder_PW)) is a special case of eating disorder related to Prader Willi syndrome.

Querying the resulting saturated RDF dataset using SPARQL guarantees to be sound and complete, i.e., to return all the answers satisfying any input query given the input RDF facts and the rules. The current domain ontology is small (25 classes, 61 instances and 3 relations) and far from being complete. It would require more interaction with the expert. Nevertheless, the important point is that it can be easily updated by adding new classes, instances or properties using TopBraid Composer, or by enriching it using medical ontologies available in Linked Data cloud.

B. Incorporating relevant information extracted from the corpus into the domain ontololy

This step relies on a table of correspondence provided by the expert where each formal term of the Prader-Willi ontology is associated with a list of textual expressions that correspond to it in the scientific literature.

Example 3:

The current table of correspondence lists feeding problems, eating disorders, food intake disorders, appetite regulation problems as textual expressions to the formal term Eating_Disorder_PW (declared as a class in the domain ontology). First, for each paper, the sentences that contain one of the textual expressions corresponding to a formal term are automatically extracted.

Example 4: From the following inserted triples (where p1608s14 is the identifier of the 14 th sentence extracted from the paper number 1608, and HH denotes the class of Hypothalamic Hormones):

<p1608s14, mentions, Oxytocin>, <Oxytocin, rdf:type, HH> , <HH, rdfs:subClassOf, Hormone> It will be inferred and added to the saturated ontology: <pw:p1608s14 pw:mentions pw:HH> <pw:p1608s14 pw:mentions pw:Hormone> Fig. 4 shows a screenshot of the combined ontology mapping textual content and formal terms of the domain ontology. The second category illustrates how to use queries for a simple statistical analysis of corpus, while third category enables fine-grained semantic analysis of its content.

A. Query-based information retrieval Fig. 5 shows a query (and its results) asking for papers that contain sentences mentioning both hormones and symptoms, and if such papers exist, to display their titles and the corresponding sentences. It is worthwhile to note that, thanks to the triples inferred by rules, this general query (in which no specific hormone and no specific symptom is made specific) allows to return very relevant specific information. For instance, the sentence whose full content is displayed in Fig. 5 contains obesity (that is a specific symptom encountered in Prader-Willi) and insulin (that is a specific hormone) that are not explicit in the query. Using search engines based on keyword-based queries, it would have been impossible to express such a query and to obtain such precise answers.

Proceedings of the World Congress on Engineering and Computer Science 2014 Vol II WCECS 2014, 22-24 October, 2014, San Francisco, USA Fig. 5 How to find papers with sentences mentioning some hormones and some symptoms related to Prader-Willi Fig. 6 shows a query that is more specific than the previous one in which the expert is interested in finding papers in which a same sentence mentions explicitly an association between a symptom and a specific neurotransmitter (namely the dopamine). These two queries clearly illustrate the added-value of our ontology-based approach equipped with querying and inference capabilities compared to keyword-based or even concept-based search.

B. Query-based simple statistical analysis of the corpus

First, we have measured to what extent the corpus is related to the Prader-Willi ontology described in Section IV. Fig. 7 and Fig. 8 show that, despite the small size of the current Prader-Willi domain ontology, our approach allowed to automatically extract from the corpus a significant amount of information relevant to the Prader-Willi domain. Fig. 7 shows the result of the following SPARQL query that asks for displaying the 30 papers that contain the most sentences related to the current Prader-Willi ontology. ! Fig. 7 Titles of papers found with the most sentences related to Prader-Willi Fig. 8 shows the distribution of the number of the relevant sentences among the papers of the corpus. Around 98% of the papers corpus contain from 1 to 100 sentences mentioning at least one of the formal terms of our ontology, and 1.5% of papers contain between 101 and 200 such sentences. The Prader-Willi entities with the corresponding expressions can be matched to more than 100 sentences in average. This is a very encouraging results considering the limitation of our domain ontology so far. It can be assumed that with a more complete ontology the number of sentences found relevant to the ontology will increase significantly. Not surprisingly, the most frequent formal terms that are mentioned in the papers are the general entities (Symptom, Gene, Hormone, Troubles). This is because our approach is able to infer that a paper implicitly mentions a given formal term (e.g., a Symptom) if it mentions explicitly a subclass of an instance of it. Interestingly, among the specific formal terms (i.e., instances in the ontology), Obesity is the most frequent in the papers of the Prader-Willi corpus. This is a strong hint (validated by the expert) that obesity is one of the main symptoms of Prader-Willi syndrome.

C. Query-based fine-grained analysis of the corpus

We have written several queries to investigate cooccurrences within a same sentence of different types of formal terms related to the Prader-Willi domain. ! Fig. 10 The query-based computation of the number of co-occurrences of each of the 3 relations with a formal concept Fig. 11 shows the query for computing the 30 most frequent co-occurrences of symptoms and hormones, and the results ordered by their frequency. As we can see, the number of (explicit or implicit) co-occurrences of symptoms and hormones is big (approximately 21.000 sentences), which suggests a strong relation between them. The most frequent co-occurrence between two specific entities respectively instances of the classes Hormones and Symptoms is Insulin with Obesity. This suggests a possible link between insulin and obesity in Prader-Willi domain. 12 clearly shows the increasing importance taken by oxytocyn within the scientific literature about Prader-Willi syndrome since 2006. This reflects the current advances on this rare disease for which the role of oxytocyn has been investigated quite recently. Note that the decreasing of the curve for 2013 may be due to the fact that our corpus contains much less papers published in 2013 than in the previous years.

VI. CONCLUSION

We have designed a novel approach to filter and query a corpus of scientific articles related to a domain of specialization for which an ontology exists or can be built. The novelty of this approach is to represent the textual content relevant to the domain of interest as knowledge added to the ontology, thus making explicit the link between sentences and papers with formal terms they refer to. This enables to use a powerful SQL-like query language, namely SPARQL to express possibly complex queries on demand. Our approach is declarative and generic. The domain ontology can be simply updated by an expert who just has to add or delete triples using the editor of TopBraid Composer, and additional articles can be added to the corpus but the whole algorithmic machinery remains unchanged. In this paper, this approach is illustrated on a specialized medical domain, namely the Prader-Willi syndrome. However, it can be easily applied to another domain simply by providing as input an ontology of this domain and a corpus of articles. We have implemented this approach using W3C standards that make it conform to Linked Data [START_REF]Linked Data[END_REF]. This opens the possibility to easily extend it to ontologies available in the Linked Data cloud.

As future work, in line with existing works combining ontologies and text mining (e.g., [START_REF] Müller | Textpresso: an ontology-based information retrieval and extraction system for biological literature[END_REF], [START_REF] Spasic | Text mining and ontologies in biomedicine: making sense of raw text[END_REF], [START_REF] Maier | LitMiner and WikiGene: identifying problemrelated key players of gene regulation using publication abstracts[END_REF], [START_REF] Cheng | PolySearch: a web-based text mining system for extracting relationships between human diseases, genes, mutations, drugs and metabolites[END_REF]), we plan to exploit text mining techniques in order to (semi)automatically enrich the domain ontology. The fact to have a preliminary ontology is important because it can be exploited to facilitate concept extraction. For instance, by focusing on sentences recognized by our method as mentioning one relation (e.g, Associated_With), extracting expressions corresponding to new concepts based on their position in the sentence compared to the expressions known to refer to the recognized relation is a simple but promising approach. In turn, the enriched ontology will enable to extract more sentences relevant to the domain of interest. This iterative process, combining simple text mining techniques and ontology-based corpus filtering is likely to be robust while limiting the manual help of human experts.

  Gene (where the suffix _PW denotes that the anomalies, symptoms, etc. that we consider are those specific to Prader-Willi setting). Instances and Subclasses can be declared in RDF by triples built with the RDF property rdf:type and the RDFS property subClassOf, as shown in Example 1. Example 1:

Fig. 1

 1 Fig. 1 Hierarchy of classes in the Prader-Willi ontology (extract) We give below the different domain properties that we have defined: -Associated_With: to relate a symptom to a trouble. -Caused_By: to relate a trouble to an anomaly. -Stimulates: to relate an hormone to another hormone. Rules enable to express transitivity of the subclass relation and also inheritance of instances between classes like the rule shown in Example 2.

  Proceedings of the World Congress on Engineering and Computer Science 2014 Vol II WCECS 2014, 22-24 October, 2014, San Francisco, USA As an illustration, Figures2 and 3are screenshots of SPARQL queries and their answers using TopBraid Composer.

Fig. 2

 2 Fig. 2 SPARQL query asking for symptoms instances in Prader Willi

Fig. 4

 4 Fig. 4 Screenshot showing the combined ontologyIn terms of size, the domain ontology alone is 160 KB while the combined ontology (i.e., the domain ontology enriched with knowledge about the textual content of the corpus) is much bigger: 264 MB corresponding to 1.627.068 RDF triples built on 27 classes, 7 properties and 365.798 instances. The number of scientific papers in the corpus is 5950 from which 359.787 sentences have been extracted (because relevant to at least one formal term of the Prader-Willi ontology). Finally, the size of the saturated combined ontololy is 393 MB corresponding to 2.941.688 RDF triples.V. QUERY-BASED ANALYSIS AND SEMANTIC SEARCHIn this section, we show through examples the power of SPARQL queries both for formulating expert demands and for computing some useful statistics on the corpus and its content. The queries are classified into three categories. The first category illustrates typical queries of experts to search in a corpus information related to their domain of expertise. The second category illustrates how to use queries for a simple statistical analysis of corpus, while third category enables fine-grained semantic analysis of its content.

!

Fig. 6

 6 Fig. 6 How to find papers with sentences mentioning an association between a Prader-Willi symptom and dopamine.

Fig. 8 Fig. 9

 89 Fig. 8 Histogram showing the distribution of the number of sentences within papers

Fig. 10

 10 reports on the co-occurrences that have been queried and found between one of the 3 domain specific relations and the one hand, and a formal concept of the domain on the other hand. The most frequent relation co-occurring with other concepts related to Prader-Willi is clearly the relation Associated_With. Exploring the corresponding sentences could help to find new concepts to enrich the current ontology.

  Fig.11The query-based computation of the number of co-occurrences of hormones and symptomes Among these correlation results, the expert wanted to zoom on the correlation between a specific hormone (namely Oxytocin) and the general formal term Symptom_PW denoting the set of symptoms of the Prader-Willi declared in our ontology. Her demand was then to visualize the evolution over time of the correlation of oxytocin with Prader-Willi syndrome. Fig.12shows the query corresponding to this demand, and the resulting curve.