José A Galindo
email: jagalindo@us.es

David Benavides
email: benavides@us.es

Pablo Trinidad
email: ptrinidad@us.es

Antonio-Manuel Gutiérrez-Fernández

Antonio Ruiz-Cortés

Automated analysis of feature models: Quo vadis?

Keywords: Software product lines, Automated analysis, Feature models, Variability-intensive systems Calls, Screen, Color 3 Mobile Phone, Calls, Screen, High Resolution 4 Mobile Phone, Calls, Screen, High Resolution, Media, Camera 5 Mobile Phone, Calls, Screen, Basic, Media, MP3 6 Mobile Phone, Calls, Screen, Color, Media, MP3 7 Mobile Phone, Calls, Screen, High Resolution, Media, MP3 8 Mobile Phone, Calls, Screen, High Resolution, Media, Camera, MP3 9 Mobile Phone, Calls, Screen, Color, GPS 10 Mobile Phone, Calls, Screen, High Resolution, GPS 11 Mobile Phone, Calls, Screen, Color, GPS, Media, MP3 12 Mobile Phone, Calls, Screen, High Resolution, GPS, Media, MP3 13 Mobile Phone, Calls, Screen, High Resolution, GPS, Media, Camera 14 Mobile Phone, Calls, Screen, High Resolution, GPS, Media, Camera, MP3

Feature models have been used since the 90's to describe software product lines as a way of reusing common parts in a family of software systems. In 2010, a systematic literature review was published summarizing the advances and settling the basis of the area of Automated Analysis of Feature Models (AAFM). From then on, different studies have applied the AAFM in different domains. In this paper, we provide an overview of the evolution of this field since 2010 by performing a systematic mapping study considering 423 primary sources. We found six different variability facets where the AAFM is being applied that define the tendencies: product configuration and derivation; testing and evolution; reverse engineering; multi-model variability-analysis; variability modelling and variability-intensive systems. We also confirmed that there is a lack of industrial evidence in most of the cases. Finally, we present where and when the papers have been published and who are the authors and institutions that are contributing to the field. We observed that the maturity is proven by the increment in the number of journals published along the years as well as the diversity of conferences and workshops where papers are published. We also suggest some synergies with other areas such as cloud or mobile computing among others that can motivate further research in the future.

Introduction

Software Product Lines (SPLs) are about developing a set of different software products that share some common functionality [START_REF] Clements | Software product lines[END_REF]. Documented benefits of SPLs are the increment of reuse, quality and reduction of time to market to achieve mass customization of software products in order to satisfy the customer needs as well as decreasing the effort of personalization.

Feature Models (FMs) [START_REF] Kang | Feature-oriented domain analysis (foda) feasibility study[END_REF] are tree-like structures that define the set of products that belongs to a SPL and have become the de facto standard to represent common and variable characteristics in a SPL. It is easy to find different graphical and textual notations in the literature to represent FMs [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]. The information that can be obtained from FMs is extensive, and the mechanisms for obtaining it are likewise varied. In fact, this area known as Automated Analysis of Feature Models (AAFM) [START_REF] Batory | Automated analysis of feature models: Challenges ahead[END_REF] has recently been identified as one of the most important areas in the SPL [START_REF] Heradio | A science mapping analysis of the literature on software product lines[END_REF] community.

The AAFM is the computer-aided extraction of information from feature models [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF] and can be summarized in three steps. First, FMs are translated to a logical representation. Second, an off-the-shelf solver or specific algorithm is used to perform a given analysis operation (such as counting the number of products or checking the consistency of a FM). Finally, the result is obtained and used in a determined context to perform other tasks such as product configuration or derivation.

A comprehensive list of proposals and operations for the AAFM was presented by Benavides et al. [4] in 2010 that settled the conceptual underpinnings of the discipline. In particular, thirty analysis operations were presented, and further formalized in [START_REF] Durán | FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing[END_REF], different automated mechanisms identified such as SAT, CSP or BDD solvers, a conceptual framework described and directions for future work established. Operations range from determining if a product is valid with respect to a FM to the calculation of the number of different products in a product line.

AAFM has been applied in different activities along the SPL process such as product configuration and derivation, reverse engineering or SPL testing. In this paper, we present a Systematic Mapping Study (SMS) to identify the evolution and trends in the application of the AAFM since 2010. Concretely, we have performed a search on different databases of AAFM-related papers. We selected 423 primary sources (papers) that followed the defined inclusion and exclusion criteria. The primary sources were classified according to different variability facets that were found during the reading and key-wording phase. It is important to remark that before 2010, AAFM was not well defined and it was referenced using an amalgam of names and concepts. Therefore, we consider that in 2010 the concept of AAFM was coined and then used in different domains and scenarios. This paper studies how AAFM has been used since its definition.

We discovered six different variability facets where the AAFM is being applied: i) product configuration and derivation. Automated support is used to guide the configuration process and the derivation of specific products. This is the most traditional usage of automated analysis mechanisms; ii) testing and evolution. Specific configurations are selected for testing purposes using automated mechanisms. These can support the automated-guided evolution of feature models; iii) reverse engineering. Extracting feature models from product descriptions and, in some situations, from logical formulas; iv) multi-model variability-analysis. Traditionally, automated analysis operations have been proposed over a single model. However, there are situations where analyses are performed over more than one model; v) variability modelling. The basic modelling constructs are not enough in some specific situations and other information such as attributes are used for modelling different situations and analysis are performed with these new modelling elements, and; vi) variability-intensive systems. AAFM is used in other application domains not directly related with SPLs.

We also observed that there are only a few industrial and real evidences of the application of AAFM techniques in most of the cases. We detect in detail where and when the papers have been published and who are the authors and institutions that are contributing to the field. We saw that the maturity is proven by the increment in the number of journals published along the years as well as the diversity of conferences and workshops where papers are presented. Finally, we devise some research opportunities and applications in the future as well as synergies with other research areas.

The remainder of this paper is organized as follows: Section 2 presents the background required to understand the scope of this paper; Section 3 discusses the previous related works that are related to this research area; Section 4 covers the methodology used in this mapping study; Section 5 presents results from analyzing the primary studies; and Section 6 presents concluding remarks.

Background

Feature models

Feature Models (FMs) are one of the most used artefacts to describe the set of products in a SPL in terms of features and relationship among them. In FMs, features are hierarchically arranged in a tree-like structure. In addition, cross-tree constraints can be used to connect features. Figure 1 shows an example of a FM describing a mobile-phone SPL, using the most common notation (Figure 1a) and its represented products (Figure 1b). There are different proposals for FM notations (see [START_REF] Schobbens | Generic semantics of feature diagrams[END_REF] for a detailed survey) but most of the proposals have the following common elements:

-Mandatory: a child feature has a mandatory relationship with its parent feature when it appears in a product whenever its parent does. In Figure 1, Screen has a mandatory relationship with Mobile Phone, i.e. any mobile phone must have a screen. -Optional : a child feature has an optional relationship with its parent feature when the child can appear or not in a product whenever its parent does. In the example in Figure 1, Media has an optional relationship with Mobile Phone, i.e. A mobile phone can have support for media features or not depending on the configuration chosen. (a) A sample feature model of a SPL for mobile phones using a FODA-like notation. (Source: [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]) Fig. 1: A sample feature model among it set of represented products -Or-relationship (also known as OneOrMore): a set of child features has an or-relationship with its parent when one or more child features can be selected when the parent is. Figure 1 contains an or-relationship between Camera and MP3 with Media. Whenever Media is present in a product, Camera, MP3, or both have to be present. -Alternative (also known as OnlyOne): a set of child features have an alternative relationship with their parent when one and only one of them can be selected in a given product whenever their parent is selected. Figure 1 shows an alternative relationships among Basic, Colour and High Resolution, so a given mobile phone can only have a specific type of Screen in a product. -Requires, Excludes: Cross-tree relationships like A requires B means that whenever feature A appears in a product, feature B must also appear. Also, a relationship like A excludes B means that both features cannot appear in the same product at the same time. Figure 1 shows two examples of these kinds of relationship: the Camera requires a High Resolution screen and the GPS excludes the Basic screen.

Automated analysis of feature models

The AAFM deals with extracting information from FMs by using computer-aided mechanisms. SPL engineers use the information to improve their business strategies as well as to take technical decisions.

The process to extract such information is shown in Figure 2. It starts by translating the features and relationships encoded in the FM and any other additional information (e.g. market share [START_REF] Galindo | Testing variability-intensive systems using automated analysis: an application to android[END_REF]) to a knowledge base described in a logic paradigm. Later, queries to the knowledge base can be performed using existing solvers or tools thus, obtaining the analysis results. In [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF], different analysis operations on FMs were reported. According to that study, we present some of them:

-Finding out if a product is valid. This operation checks if a product (i.e. set of features) belongs to the set of products represented by a FM or not. It is helpful for SPL engineers and managers to determine whether a given product is available in a SPL. -Obtaining all products. This operation lists the products represented by a FM. It allows practitioners to identify the final products that they can manage in their SPL. For example, the model shown in Figure 1a represents the products in Figure 1b. -Calculating the number of products. This operation counts the number of products of a FM. This provides information about the size and complexity of the SPL represented by a FM. It is commonly used to perform more complex operations such as calculating the amount of reuse metrics of a SPL. For example, in Figure 1a there are fourteen products. -Detecting errors. The large number of different features used in a FM increases its complexity as well as the probability of introducing errors. There are several types of errors that can be detected by using the AAFM. For example, it is important to determine if a FM is void, i.e. whether it represents no product at all because of contradicting relationships. Another common error is the detection of dead features, i.e. features that cannot appear in any of the products derived from the model. Dead features are clearly undesired since they are the result of a wrong domain modelling. -Explaining errors. As shown before, there are circumstances where FMs contain errors. In such situations, it is important to assist on resolving them. An explanation operation takes a FM as input and a set of previously identified errors, trying to provide insights to correct them. These operations are performed automatically using different approaches. Most of them translate FMs into specific logic paradigms such as propositional logic, constraint programming or description logic. Others propose ad-hoc algorithms and solutions to perform these analyses [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]. Finally, these analysis capabilities can also be found in several commercial and open source tools such as pure::variants 1 , SPLOT [START_REF] Mendonca | Software product lines online tools[END_REF], FaMa [START_REF] Benavides | FaMa[END_REF], FeatureIDE [245] or FAMILIAR [START_REF] Acher | FAMILIAR: A domain-specific language for large scale management of feature models[END_REF]

Logic Paradigm

Solver or Tool

Literature review methods

To crawl the existing knowledge in the literature there are different kinds of reviews [START_REF] Grant | A typology of reviews: an analysis of 14 review types and associated methodologies[END_REF], but we focus on two of them. Figure 3 -Systematic Literature Reviews (SLRs) take primary sources, i.e. papers under study, that are homogeneous in terms of comparability and compare them to get conclusions [START_REF] Kitchenham | Systematic literature reviews in software engineering -a systematic literature review[END_REF]. All papers of the review are comparable among different aspects. SLRs provide a synthesis of the knowledge existing in the core content of primary sources of a specific field and, to do so, researchers have to read, understand and classify the whole content of the studies. SLRs provide as output as many details and comparisons as possible among the content of primary sources. In other words, they are focused on "how" the things have been done in a concrete field. This is, SLRs target a reduced set of papers and extract all the insights possible from them. -Systematic Mapping Studies (SMSs) take primary sources that are heterogeneous in terms of comparability but are related to a broader area and provide a mapping and categorization of the different facets detected in the studies [START_REF] Petersen | Systematic mapping studies in software engineering[END_REF]. Researchers read the title, abstract, and optionally other parts of the paper stepwise. The idea is to get the whole picture of a broad research area. SMSs provide visual outputs that allow an easy identification of research gaps. For example, most of SMSs provide bubble plots or heatmaps that show how well covered is an intersection between the categories analysed. Note that SMSs do not analyse specific research but allow the characterization and classification of more heterogeneous papers. In other words, the main focus of an SMS is to detect "what have been done".

Petersen et al. [START_REF] Petersen | Systematic mapping studies in software engineering[END_REF] detailed the process of building systematic mappings and compared them with SLRs. SMSs take as input as many papers as possible reducing the bias of only reading portions of the papers in the case of a mistake in the classification.

There are potential relations between SMSs and SLRs as shown in Figure 3. The results of an SMS can be the basis for a more in depth SLR of one or several of the research facets that are part of the mapping. For instance, an SLR of testing in mobile phone applications can then serve as the basis for performing an SLR of techniques for automated unit tests in that context, which is a more concrete field. SLRs and SMSs usually provide common outputs such as the fora where authors have published, temporary evolution of primary sources as well as authors and institutions contributing to the area of study.

Related work

As explained in Section 4 there are different kinds of review methods. In this Section we go through other systematic reviews identifying the context, year and period. In the area of SPLs, several SMSs or SLRs can be found [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF][START_REF] Engström | Software product line testing -a systematic mapping study[END_REF][START_REF] Laguna | A systematic mapping study on software product line evolution: From legacy system reengineering to product line refactoring[END_REF][START_REF] Lopez-Herrejon | A systematic mapping study of search-based software engineering for software product lines[END_REF][START_REF] Montalvillo | Requirement-driven evolution in software product lines: A systematic mapping study[END_REF][START_REF] Da Mota | A systematic mapping study of software product lines testing[END_REF] as shown in Table 1, most of them being published in recent years. The period of time used for the reviews varies depending on the topic, ranging from 1990 (where SPLs started to be popular) to the date where the study was performed2 . Also, the number of analysed contributions of our work reviewed more studies. As explained in Section 4.2, we used 423 papers for extracting some general data and 242 for more concrete insights. Heradio et al. [START_REF] Heradio | A bibliometric analysis of 20 years of research on software product lines[END_REF] presented a bibliometric analysis of 20 years of SPLs, from 1995 to 2014. One of the conclusions was that "feature modelling has been the most important topic for the last fifteen years, having the best evolution behaviour in terms of number of published papers and received citations" [START_REF] Heradio | A bibliometric analysis of 20 years of research on software product lines[END_REF].

Also, in the 2009-2014 period, the AAFM was the most influential topic according to the number of papers produced and the centrality and density of papers. Note that [START_REF] Heradio | A bibliometric analysis of 20 years of research on software product lines[END_REF] is not an SMS nor an SLR and only performs data analysis based on automated mechanisms.

3.1 Need for a systematic mapping study Benavides et al. [4] analysed in detail the relevant literature that builds up the body of knowledge of the AAFM. That work analysed and categorized concrete methods for performing automated analysis of feature models. Also, authors identified different analysis operations and several automated mechanisms for performing the operations (see Section 2.2). We conjecture that in [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF] the grounds of the field of AAFM were settled and from then, most of the papers are using analysis operations to solve different problems in SPL engineering instead of defining new operations or proposing new analysis mechanisms or algorithms.

In this paper, we want to detect the main trends where the AAFM is being used, this is, we want to include in this study any paper that uses the AAFM in any area of the software product line engineering process (e.g. testing, requirements, derivation,...). In this context, we performed an SMS which is the most suitable review method to cover a wider set of publications and understand the current and future state of this area. Section 4.1.1 shows the set of questions that will guide our study and will help us to achieve our goal.

The authors of this work belong to one of the most active research groups in the area of feature modelling, and automated analysis according to [START_REF] Heradio | A bibliometric analysis of 20 years of research on software product lines[END_REF], being these topics on the ones that attracted more attention in recent years. As the field evolves, there is a need to evaluate the trends in the area and assess their maturity. With this work, we aim to serve as evidence for research opportunities both in the kind of research (i.e. more empirical vs more theoretical) and in the topics to be covered.

SMS process definition

This SMS follows a process inspired by the one proposed by Petersen et al. [25] summarized in Figure 4. The main steps to perform a systematic mapping study are:

1. Planning the review, which includes the definition of the search protocol, the survey of the literature and the definition of the research questions; 2. Study identification; where the databases are crawled and the primary studies selected; 3. Data extraction and classification, where the mapping is developed and conclusions are obtained. : Systematic mapping study process [START_REF] Montalvillo | Requirement-driven evolution in software product lines: A systematic mapping study[END_REF][START_REF] Petersen | Systematic mapping studies in software engineering[END_REF] Next, we detail the process that guides this SMS.

Phase 1: Planning the Review

This phase comprises the execution of three process steps as shown in Figure 5: i) protocol definition, where we decide how to do the review and how to minimize the threats to validity; ii) literature survey, that consists of getting the base of the body of knowledge of the area, and; iii) the definition of research questions, where the questions to be answered are stated. Next, we detail how we performed each of these steps. Protocol definition. The protocol that we defined follows the guidelines of Petersen et al. [START_REF] Petersen | Systematic mapping studies in software engineering[END_REF] to perform systematic mapping studies. Then, we compared the research questions with Jia et al. [START_REF] Jia | 5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing[END_REF] 5W+1H model to check their accuracy (see Section 4.1.1). Also, we modified the protocol to obtain not only the results from databases but also considering the papers quoting Benavides et al. [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF] (see Section 4.2).

Literature survey. In order to survey the current state of the art in the AAFM, we read the papers published in the main conferences by means of authors expertise and journals as well as taking the following papers as the main input defining the body of knowledge [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF][START_REF] Durán | FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing[END_REF]. Also, we relied on the expertise of some of the authors of this paper that have been working in the AAFM and collaborating with the community for the last ten years or so.

Definition of research questions

Defining a set of research questions that guide an SMS is a difficult task that could lead to a biased study, which is aggravated if the authors have experience in the area. To guide and reduce the arbitrariness of the questions definition, we were inspired by the 5W+1H pattern proposed by Jia et al. [START_REF] Jia | 5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing[END_REF] using it to verify the completeness of our proposed questions. This pattern relies on the 5W+1H model, a pillar of journalism to report stories originally proposed by Kipling [START_REF] Kipling | Just so stories[END_REF]. 5W+1H is an abbreviation of Who, Why, What, Where, When and How, the six questions to be answered in order to know the most important aspects about a story. Thus, we defined the following research questions for our study:

RQ1 Where are the papers published? Which fora are being targeted by practitioners in the last six years? We think that this question can help researchers to know which communities are interested in which topics. Therefore, to target the most suitable forum for a new contribution. RQ2 Who are the authors and institutions that make research into AAFM? Which authors have been more active regarding the AAFM area and which institutions are currently hosting them? This question aims at fostering researchers collaboration and to determine which institutions hold the knowledge of a topic. RQ3 What are the areas for which AAFM has been applied? What topics have attracted more researchers attention in the last years? This research question aims at discovering "Why" the researchers have applied AAFM techniques in the sense of the applications where AAFM has been attracting researchers attention.

RQ4 Which kind of publications are used to address the challenges? This question gets insights about the maturity of the area by relying on Wieringa et al. [START_REF] Wieringa | Requirements engineering paper classification and evaluation criteria: a proposal and a discussion[END_REF] taxonomy research which includes validation research, evaluation research, solution proposal, philosophical papers, opinion papers and experience papers. RQ5 When have the papers been published? This question tries to determine the temporary evolution of the publication types and fora. This question aims at helping researchers to determine the current state of a topic, thus, to see if there are chances of further collaboration in that area or not. RQ6 How are the interrelationships among the papers? This question tries to find research gaps whether to invest more research efforts in the future by analysing the already covered areas.

Note that most of the questions could be rewritten differently and that the pattern [START_REF] Jia | 5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing[END_REF] is used for the sake of coverage w.r.t. the research questions.

Phase 2: Study identification

In this section we explain how we identified the studies to include in the mapping study as shown in Figure 6. Please note that in Figure 7 we provide details about the evolution of the selection of papers used in this review: i) conducting search, where we perform a raw search in different data sources for primary studies; ii) filtering studies, that consists in applying inclusion and exclusion criteria. iii) deep search, where new studies are added manually to improve the quality of the primary sources. iv) evaluate search, where we get the final set of primary sources after an evaluation. Next, we detail how we performed each of these steps.

Conduct search for primary sources

The process followed to conduct the search for primary studies is shown in Figure 7 (step 1). We decided to crawl Google Scholar3 and Scopus4 databases because both of them provide easy mechanisms to export the resulting data ordered by number of citations. In each database, we performed two different queries5 : one with the following search string "feature model" AND ("reasoning" OR "analysis" OR "automated" OR "analyses") and the other with the papers citing [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]. We specified that the publication date should be between 2010 and 2017. We obtained 15,300 (using the search string) and 945 (citing [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]) for Google scholar queries and 1167 (using the search string) and 572 (citing [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]) for Scopus queries. Then, we selected only the first 200 most cited papers from each data source and removed duplicates getting a total of 445 initial studies. Also note that in this paper, we want to determine in which scenarios the AAFM has has been used. It is important to remark that [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF] was published in 2010 and that is the reason why we start the search for papers from that date.

Step Step 2: Filtering studies

Step 4: Evaluating the search

Primary studies First Batch [RQ1] -[RQ2] -[RQ5] (423) Primary studies Second Batch [RQ3] -[RQ4] -[RQ6] (242)
Step 3: Snowball reading

Filtered by title and abstract

Selected conferences (VaMoS, SPLC, ASE, ICSE) and the top 10% percentiles according to SciVal [START_REF] Mendonca | Software product lines online tools[END_REF].

To minimize the impact of the ordering by citation reading the first 50 of each result.

Filtering studies

From the 445 papers of previous step, we discarded non-peer reviewed material such as technical reports, secondary studies on AAFM and thesis documents and performed a detailed screening of the papers removing papers not related to AAFM. This resulted in a set of 402 primary studies.

Deep search

To prevent missing some relevant works, we conducted a special search for the papers published from 2015 to 2017 that had no time to gain citations and tried to detect some relevant papers missing from the initial search as proposed by [START_REF] Montalvillo | Requirement-driven evolution in software product lines: A systematic mapping study[END_REF]. This search was relying on snowball reading and the experience of the authors. We removed duplicates and added 19 additional papers, obtaining a total number of 423 papers for the next step. Also, we checked if there was a missing highly cited paper considering it for inclusion. This is done to reduce the chances that a non included paper biases the observations.

Evaluate search

The resulting 423 papers, which are taken as input for this step, were used for RQ1, RQ2 and RQ5, this is, to determine where and when the papers were published and who are the authors and institutions that publish results. We were able to use this large amount of papers since we used automated mechanisms to gather this data from our local database.

For the remaining research questions, we only considered the papers published in journals, and conferences with a high acceptance rate of papers related to variability-management and AAFM, according to our own classification of the previous step, i.e. the answer of RQ1 (see Table 4). This resulted in a total of 242 papers for questions RQ3, RQ4 and RQ6. Also, for this second set of papers, we considered all contributions in the top 10% percentiles according to SciVal 6 to avoid missing relevant papers.In the study, we used two different set of papers to answer the RQs as illustrated in Figure 7. For RQ1, RQ2 and RQ5, we used 423 papers and for RQ3, RQ4 and RQ6 we used 242. Note that 242 is a subset of the 423 papers. The reason to use two different sets is due to the fact that, on the one side, for RQ1, RQ2 and RQ5 we used semi automated mechanisms that made it feasible to handle that big amount of papers. While, on the other side, for RQ3, RQ4 and RQ6 we had to analyse the content of the papers what would have been unfeasible with the bigger set.

Phase 3: Data extraction and classification

This phase comprises the execution of two process steps as shown in Figure 8: i) topics keywording, where we detect the topics to answer RQ3, RQ4 and RQ6; ii) data extraction and mapping, where the mapping itself is done. Next, we detail how we performed each of these steps. 8: Systematic mapping study process [START_REF] Petersen | Systematic mapping studies in software engineering[END_REF] 6 www.scival.com

Topics keywording

Publications are classified in two dimensions: i) the variability context facet; and ii) the research type. We defined these dimensions by following up the process described by Petersen et al. [START_REF] Petersen | Systematic mapping studies in software engineering[END_REF]. Petersen et al., propose to use a keywording method to define the research focus to group the papers. This process is divided in two steps. First, researchers read the abstracts in the collection to review and identify the keywords and concepts that reflects the paper contribution. If the abstracts are not enough, then researchers take a look at the introduction and optionally more parts of the papers until the concepts can be identified. Finally, researchers define the set of categories to do the mapping by identifying the main paper contributions. Also, if after this reading is unclear where a paper belongs, we take a look at the conclusions and then, the category where the paper is more focused on. As a result of the keywording process, the following variability context facets were defined. Note that we understand the term "variability context" as the different AAFM application domains. It is worth noting that a paper can belong to more than one category:

-Product configuration and derivation, papers coping with the derivation or the configuration of products. For example, [237] enables the configuration of FMs but taking into account the preferences of multiple stakeholders. -Testing and evolution, these papers focus on the use of automated analysis techniques to optimize the testing of the products derived from a FM such as those presented in [START_REF] Apel | Detection of feature interactions using feature-aware verification[END_REF] to detect feature interactions which are error sources. -Reverse engineering, these papers describe different techniques to build up FMs from a variety of product descriptions such as product description matrices [START_REF] Acher | Extraction and evolution of architectural variability models in plugin-based systems[END_REF] or lists of products. -Multi-model variability analysis, these papers focus on the analysis when variability is not described in a single model but in several models (e.g. merging and slicing operations between models). For example, Dhungana et al. [START_REF] Dhungana | Integrating heterogeneous variability modeling approaches with invar[END_REF] presented a solution to enable the configuration of diverse and inter-operable variability models. -Variability modelling, papers focussing on the description and modelling of the variability to perform further analysis and extraction of relevant information. For example, Berger et al. [START_REF] Berger | A study of variability models and languages in the systems software domain[END_REF] presented a study on the variability models and languages in the systems software domain. -Variability intensive systems analysis, these papers focus on applying automated analysis techniques into variability intensive systems (usually not categorized as SPLs) that have to cope with variability requirements such as the Linux kernel [232].

Different kinds of publications were also taken into account for this mapping study. Concretely, we followed up the proposal of Wieringa et al. [START_REF] Wieringa | Requirements engineering paper classification and evaluation criteria: a proposal and a discussion[END_REF] and encouraged in [START_REF] Petersen | Systematic mapping studies in software engineering[END_REF], which propose the following types:

-Opinion papers, papers showing the author opinions over a concrete technique but not relying on methodologies or related work. -Philosophical papers, these papers help structuring the area such as taxonomies, literature reviews and mapping studies. -Solution proposal, in these papers authors propose solutions to problems but relying in existing techniques -even improving them. There is no need for evaluation or validation. -Evaluation research, implemented techniques with evaluation and conclusions. Here the technique has been evaluated in front of examples or small datasets but there is no validation by final users yet. -Validation research, this research type focuses on techniques already evaluated with end users.

-Experience papers, papers explaining industrial or personal experiences in the field.

Data extraction and mapping

Once the facets and topics are gathered in the previous steps, the mapping is ready to be performed. When classifying the data obtained from Phase 3, we proceeded with the following steps to increase the confidence about its correctness.

1. We created a database with all the papers to be classified. In our case, we decided to use a bibtex database and exploited it by using JabRef 7 . The database contained the following fields for each paper: i) title ; ii) abstract in plain text ; iii) link to reach the full paper; iv) proposed variability facet; and v) type of research facet. 2. We formed two groups of researchers to carry the classification task independently. 3. Each group performed the keywording process in order to identify the minimal set of categories that provides a good separation of concerns within the papers. Then, we had a meeting and decided the final classification schema to use. 4. Then, the two teams performed the classification individually on the two proposed dimensions (variability context and research type). 5. A single member identify the papers that were classified differently. 6. We held a meeting in which all the papers classification identified in the previous step were discussed until a consensus was reached.

Threats to validity

Even though the research we present in this paper aims to be as systematic as possible, there are some assumptions that we made that may affect its validity.

External validity. As mentioned before, we included works directly citing Benavides et al. [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF] previous work and the result of different queries in scientific databases. This may have biased the process by increasing the papers that cited a single paper (e.g. authors not knowing that study). However, we minimised the impact of this threat by including as many papers as possible.

The major threats to the external validity are:

-Population validity, we included a large set of papers to reduce the possibility of missing relevant works. However, using the number of citations as threshold might prevent the inclusion of some interesting works. However, we consider that with a set with 423 papers for the RQ1, RQ2 and RQ5 and 242 papers for RQ3, RQ4 and RQ6 we provide a good coverage of the existing research and reduced the probability of missing interesting works and the impact of a bad classification in the keywording process. Moreover, we cross-checked other SMS verifying that this amount is above the average number of papers (See Section 3). -Ecological validity, it is focused on possible errors in the experiment materials and tools used. We relied on automated mechanism when possible instead of relying on manual methods to prevent this error. Also, there is a threat inherent to the use of the number of papers published in a venue to map dimensions and discard papers in SMSs. To prevent this thread, we also included those papers ranked in the top 10% percentile according to SciVal.

Internal validity is a measure which ensures that a researcher's experiment design closely follows the principle of cause and effect. In this mapping study we have tried to be as methodologically exhaustive as possible. However, a manual classification process such as keywording might introduce some errors. Again, we considered a large set of papers to minimise the impact of an erroneous classification.

Results

In this section we revisit the different research questions defined in Section 4.1.1.

RQ1: Where are the papers published?

This question is used twofold. First, to identify which conferences and journals are accepting AAFM results. Second, to reduce the number of papers of the first batch as detailed in Section 4.3 by means of a clear criteria.

Table 2, shows the top-ten journals depending on the number of papers published on them for this mapping study. A total of fifty-eight different journals were detected in this study (see Appendix B). We noted that the top-ten journals were indexed in journal quality rankings such as JCR

1 JSS 1 1 4 8 1 1 1 1 1 1 9 0 2 SOSYM 2 1 4 4 3 2 0 2 0 1 7 1 3 SQJ 0 2 2 5 2 0 0 1 0 1 7 0 4 TSE 0 0 3 2 1 3 0 0 0 0 5 2 5 IST 1 1 3 2 3 0 0 0 0 1 4 1 6 SCP 1 1 4 0 3 0 0 0 0 2 4 0 7 STTT 0 1 3 2 3 0 0 1 0 1 2 1 8 ESA 0 0 1 2 2 0 0 0 0 0 4 0 9 Computer 1 1 2 1 0 0 1 0 2 0 0 0 10 ESE 0 0 0 2 1 2 0 0 0 0 2 1
Table 3: Top-ten journals with variability facets and research type Table 3 presents the journal names among the number of papers classified in each category. For the category of each paper we used the second set of papers only containing all journals and conferences with more than ten contributions. This table aims at helping deciding where to submit a new contribution. We observe that testing and evolution is the most common topic on the first members in the classification. Also, the same happens with the type of research, where the most common type for the first members of the rank is evaluation research. 4, shows the top-ten conferences based on the number of papers related to automated analysis in their proceedings. We see that the MODELS conference is accepting several AAFM related contributions. This might be indicative of the increasing importance of variability modelling in modelling specific conferences. Also, it is remarkable that even though there are conferences traditionally accepting AAFM papers, there are other conferences such as ICSE, MODELS, ECSA among others that are also accepting contributions.

#papers

To reduce the number of papers to review in RQ3, RQ4 and RQ6, we only considered papers published in journals and in the conferences having more than ten contributions (i.e. SPLC, VAMOS, ASE, ICSE). Note that we also considered for this second batch the papers within the top 10% percentile according to SciVal. The full list of journals and conferences is provided in Appendix B.

Acronym

Variability Table 5: Top-ten conferences with variability facets and research type Table 5 presents the conferences among the number of observations done in for each topic. This table was built using the second batch of papers only containing papers from journals and conferences with more than ten contributions as well as the papers present in the top 10% percentile according to SciVal. This table shows that more traditional subareas such as product configuration and derivation are more present in more generalist conferences such as ICSE and ASE while the niche conferences of the area such as VAMOS or SPLC hold a more varied set of contributions. Also it worth mentioning that regarding the research type it was similarly distributed among all conferences.

RQ2: Who are the authors and institutions that research on AAFM?

In this question, we want to discover the most active researchers and institutions using AAFM. For that, we took the first batch of papers without excluding any conferences. Then, we counted how many papers were published by each author. Table 6, presents those first authors with more than three AAFM related publications since 2010. Also, we added the columns were the author contributions were categorized. Note that while for the ranking of papers we relied on the first batch of papers, we used the second batch for the categories. This table aims at fostering future collaboration between researchers working on similar areas. 10 .

After identifying the most prolific first authors in the area, we wanted to highlight which institutions are currently hosting them. This is, we care about the current institution of an author, not the one at the time of writing the paper. This is done to identify which institutions currently have the know-how in AAFM. To perform this analysis, we relied on Google Scholar author search. Again, we used the first batch of papers without excluding any conferences. We searched by each first author in the paper and retrieved its current verified institutional email in Google Scholar and then, we manually looked for the institutions associated with the email domain. Henard, Christopher To discover why researchers are using AAFM methods, we explored the different identified AAFM topics showing our major findings and identifying potential research gaps. These research gaps are subjective interpretations according to the screening of papers and the experience of the authors. Also, we show quantitative data such as the number of identified papers per topic. This is the output of the process defined in Section 4.3.1, where keywords were assigned to papers after reading their titles and abstracts and required portions. For this purpose, only the filtered batch of papers was used (see Figure 7).

0 0 1 3 0 1 0 0 0 0 5 0 5 Pereira, J.A. 0 0 1 0 1 0 0 0 0 0 0 1 5 Thüm, Thomas 1 0 2 1 2 1 0 1 0 1 1 0 5 Galindo, José A 0 1 2 1 1 0 0 0 0 0 3 0 5 Pleuss, Andreas 0 0 3 1 2 0 0 0 0 4 0 0 4 Apel, Sven 0 0 1 1 0 0 0 0 0 0 2 0 4 Berger, Thorsten 0 0 1 0 3 0 0 0 0 1 0 3 4 Arcaini, P. 0 0 0 2 0 0 0 0 0 1 1 0 4 Mazo, Raúl 0 0 1 1 0 0 0 1 0 0 1 0 4 Bagheri, Ebrahim 0 1 2 1 2 0 0 1 0 0 3 0
Product configuration and derivation. Papers that deal with the configuration and/or further product derivation of feature models are in this category. The configuration of FMs can be defined as the process of selecting and deselecting features in a FM to obtain a concrete product instance. After and complementary to the previous task, the product derivation process is activated. Product derivation uses concrete composition mechanism and variability management techniques to obtain a working product [START_REF] Capilla | Variability realization techniques and product derivation[END_REF]. A total of sixty-nine papers were also classified in this category, making this topic the one that attracted more attention. For example, the use of feature-oriented techniques to implement SPLs has been extensively documented (e.g. [245]). Also, researchers proposed the improvement of the scalability of product configuration [216]. Finally, there are works that focus on configuration, such as Asadi et al. [START_REF] Asadi | Toward automated feature model configuration with optimizing non-functional requirements[END_REF], who provide mechanisms to configure feature models while optimizing non-functional properties, and [245], where a tool to guide practitioners while configuring SPLs is presented.

Research opportunities

Traditionally, configuration technologies are applied in a closed SPL context, this is, all or most of the features are available in a controlled and closed environment, where all the decisions for configuring a product are taken by a very small group of people. An emerging challenge in this area is the configuration of a diversity of distributed SPL descriptions, sometimes known as software ecosystems. As SPLs grow, configuring and maintaining them become an unfeasible task for a small group. Also, different stakeholders and privacy policies encourage the use of visibility restrictions for the configurable parts. We think that there is still work to be done in this area like, for example, to enable the parallel configuration of those open and distributed SPLs.

Testing and evolution. When a large number of products is encoded in a FM the testing and evolution processes become expensive and tedious. Two main approaches have been followed to reduce the testing costs. First, combinatorial testing and more concretely T-wise methods to narrow the number of products [130]. Second, test prioritisation to order the execution of critical tests in a time or resource constrained environment [214]. These techniques become also relevant when coping with software evolution. SPL evolution happens when it is required to add, remove or modify features or relationships to an existing SPL. In this scenario, it is required to test if the SPL is error-free before, during and after the evolution actually happens. In this case, the rationale for grouping evolution and testing is that, after the keywording process, some papers were in both sub-areas, probably, because testing is a key aspect to consider when evolving variability-intensive systems.

We have identified sixty-two papers referring to SPL testing and evolution. We also noticed that the works coping with testing costs reduction are focusing in scenarios where there is more than one objective -maybe contradictory-to be satisfied at the same time. This is, to optimize different aspects of the same test-suite [130]. For example, to find the test-suite that minimises the testing cost while maximising the market-share. Recently, researchers have been looking for the most convenient evolutionary algorithm to test and select best SPL products, finding that IBEA was returning better results than NSGA-II [216] when coping with multi-objective testing objectives. In terms of evolution management different automated analysis have been proposed to guarantee the safe transition between the different evolution phases of the SPL. For example, White et al. [258], propose the use of CSP solvers to grant the validity of so-called FM drifts.

Research opportunities

A challenging task is to evolve a SPL while maintaining support for existing products. This is, to verify that the products already being used are still valid in the next evolution of the SPL. SPL researchers use information encoded in variability models when selecting and prioritising testcases. However, other information related to SPL activities such source code management, bug tracking system can be considered to select and prioritise test-cases. The exploration of different testing techniques such as metamorphic, mutation, graph-based among others can still explored in the SPL context.

Reverse engineering. There are two main strategies to adopt an SPL approach. First, a proactive approach when a company already starts by planning the construction of the SPL. Second, a reactive approach when individual products are first developed one after another and at a certain point, when the number of similar products is big enough, the company transitions to an SPL engineering approach. To help in this transition, researchers have proposed several reverse-engineering methods.

The last years have been fruitful in this area with a total of twenty-two papers classified. Researchers have extracted variability encoded in product comparison matrices [215] and CNF formulas [START_REF] Andersen | Efficient synthesis of feature models[END_REF]. Moreover, Becan et al. [START_REF] Bécan | Breathing ontological knowledge into feature model synthesis: an empirical study[END_REF] worked on benefiting from ontological knowledge to help in the task of reverse engineering.

Research opportunities

Nowadays, the variability description of variability-intensive systems is getting more complex. This is done by introducing non-boolean information [145], and by using several variability models [118] in a multi-layer fashion. However, we found no proposals to reverse engineer the existing variability information of variability-intensive systems with more than one variability model or non-boolean information. Also, researchers had not relied on low level assets such as source code or package management systems descriptions when reverse engineering feature models. This kind of assets can provide valuable information because they can contain implicit variability information to be exploited in the reverse engineering task.

Multi-model variability analysis. The AAFM started by only considering one FM description at a time. However, the more the SPL becomes larger, the more complex the variability description becomes. Nowadays, it is common to find the variability description of a variability-intensive system in several variability models that can be depicted using different formats.

A total of twenty-five papers cope with multi-model SPLs. For example, Dhungana et al. [START_REF] Dhungana | Integrating heterogeneous variability modeling approaches with invar[END_REF], proposed the distributed modelling of SPLs providing examples from the industry. Also, new tools have been provided to determine the set of implementation artifacts that supports a concrete feature specification [START_REF] Abele | The cvm framework-a prototype tool for compositional variability management[END_REF]. But also, because of the need of describing a variability-intensive system in smaller artifacts so it can be maintained by different practitioners, new operations to merge models appeared, thus introducing new FM operations such as model merging or slicing [START_REF] Acher | Slicing feature models[END_REF].

Research opportunities

There is a lack of support for quality attributes when coping with multi-model SPL descriptions. For example, we did not find techniques showing how quality attributes domains are impacted by the selection of certain features at specification level. Also, it is possible to implement distributed analyses of FMs when having multiple models. For example, to first slice the model, and then distribute the analysis in different computation nodes and finally merge the results. This can be helpful in scenarios with large models such as the Linux kernel containing more than 8,000 features.

Variability modelling. Encoding the variabilities and commonalities of a SPL requires to find a tradeoff between the expressivity of the language and its usability [START_REF] Méndez-Acuña | Leveraging software product lines engineering in the development of external dsls: A systematic literature review[END_REF]. There are multiple ways of encoding variability such as the one described in [START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF]. Some variability modelling languages in real context have been analysed [START_REF] Czarnecki | Cool features and tough decisions: a comparison of variability modeling approaches[END_REF] to learn how variability is managed and modelled in realistic systems [START_REF] Berger | Variability modeling in the real: A perspective from the operating systems domain[END_REF].

This area has gained momentum in the last years (see Figure 9), grouping a total of sixty papers. Several proposals appeared to cover different domain specific requirements. For example, new cardinalities regarding features have been considered [175] to enable the proper description of systems when a feature is present multiple times in a configuration. Existing languages for feature modelling have been documented [101] and new language constructs have been introduced to improve the analysis of FMs. For example, FAMILIAR [START_REF] Acher | Familiar: A domain-specific language for large scale management of feature models[END_REF] introduced different constructs and translations to solvers that make the AAFM scaling over previous approaches.

Research opportunities

Nowadays, AAFM is applied beyond product lines in scenarios like cloud computing or the operating systems domain [START_REF] Dougherty | Model-driven auto-scaling of green cloud computing infrastructure[END_REF] (see the next challenge). We think that the use of new available information in those systems can be used to leverage the AAFM in other domains by means of the definition of new analysis operations or the composition of existing ones. This motivates the need of new constructs encoding this information and thus, improving the AAFM.

Variability intensive systems analysis. The AAFM was initially developed thinking in SPLs. However, nowadays it is used for a widespread number of applications. Currently, the scenarios where AAFM is used range from variability analysis in other domains such as mobile apps testing [117], cloud systems [119] or bioinformatics [START_REF] Costa | A scientific software product line for the bioinformatics domain[END_REF]. Note that there is a clear trend of using AAFM for systems beyond SPL however, authors tend to use it for a concrete scenario such as testing [117] or configuration [119].

A total of twenty-three papers referring to this trend were identified within the corpus of papers. Different variability models benefit from automated analysis techniques to extract information in a similar way. While diverse models such as OVM [209] and DOPLER [170] models had automated analysis support in 2010, now, there are other models such as BPMNs [START_REF] Del-Río-Ortega | On the definition and design-time analysis of process performance indicators[END_REF], that use similar techniques to extract information and metrics. In the open source domain, the AAFM was applied to existing systems such as the Linux [232] kernel. Also, the AAFM has profited from the open-source community which helped to understand the nature of models and the constraints flavours existing in the wild [180].

Research opportunities

The number of different scenarios where AAFM can be applied have been proved to be very large and we envision that there will be more and more scenarios where it will be applied in the future. Ecosystems, cyber-physical systems, robotics, big data or the internet of the things are only some examples where variability is a first-class citizen and can use AAFM techniques in their development.

Quantitative analysis

The variability context dimension of this mapping study aims to identify the current trends in the area of AAFM. In the previous Section, we already presented them while highlighting some of the most interesting findings. Now we present quantitative data that provides the weight of each trend in the area.

Table 8 presents the distribution of the different trends of the full set of papers. We see that the areas that more attracted researchers attention were testing and evolution, product configuration and variability modelling. When we take a look to the list of papers we see that the product configuration was more present within the time lapse considered for this research while the use of AAFM in a diversity of systems is more recent. Also, we see that, because of this new AAFM usage the variability, modelling papers started to introduce new forms of variability in the reasoning process such as features cardinalities.

Variability context

Citations #papers Product configuration and derivation [START_REF] Acher | Familiar: A domain-specific language for large scale management of feature models[END_REF][START_REF] Ajoudanian | Automatic promotional specialization, generalization and analysis of extended feature models with cardinalities in alloy[END_REF][START_REF] Apel | Feature-interaction detection based on featurebased specifications[END_REF][START_REF] Asadi | Development and validation of customized process models[END_REF][START_REF] Asadi | Toward automated feature model configuration with optimizing non-functional requirements[END_REF][START_REF] Bagheri | Stratified analytic hierarchy process: Prioritization and selection of software features[END_REF][START_REF] Bagheri | Configuring software product line feature models based on stakeholders' soft and hard requirements[END_REF][START_REF] Baresi | Service-oriented dynamic software product lines[END_REF][START_REF] Ter Beek | Statistical model checking for product lines[END_REF][START_REF] Ter Beek | Supervisory controller synthesis for product lines using cif 3[END_REF][START_REF] Ter Beek | Statistical analysis of probabilistic models of software product lines with quantitative constraints[END_REF][START_REF] Mht | Using mcrl2 for the analysis of software product lines[END_REF][START_REF] Benavides | Automated analysis in feature modelling and product configuration[END_REF][START_REF] Berger | Feature-to-code mapping in two large product lines[END_REF][START_REF] Boškovi | Automated staged configuration with semantic web technologies[END_REF][START_REF] Boucher | Deriving configuration interfaces from feature models: A vision paper[END_REF][START_REF] Buchmann | Mod2-scm: A model-driven product line for software configuration management systems[END_REF][START_REF] Camacho | Cost-related interface for software product lines[END_REF][START_REF] Capilla | An overview of dynamic software product line architectures and techniques: Observations from research and industry[END_REF][START_REF] Capilla | Context variability for context-aware systems[END_REF][START_REF] Chavarriaga | Using multiple feature models to specify configuration options for electrical transformers: An experience report[END_REF][START_REF] Chen | Optimizing the product derivation process[END_REF][START_REF] Chrszon | Profeat: feature-oriented engineering for family-based probabilistic model checking[END_REF][START_REF] Classen | Symbolic model checking of software product lines[END_REF][START_REF] Cordy | Beyond boolean product-line model checking: dealing with feature attributes and multi-features[END_REF][START_REF] Dhungana | Configuration of multi product lines by bridging heterogeneous variability modeling approaches[END_REF][START_REF] Diskin | Faithful modeling of product lines with kripke structures and modal logic[END_REF][START_REF] Dumitrescu | Bridging the gap between product lines and systems engineering: an experience in variability management for automotive model based systems engineering[END_REF][START_REF] Duran-Limon | An ontology-based product architecture derivation approach[END_REF][START_REF] Durán | Flame: a formal framework for the automated analysis of software product lines validated by automated specification testing[END_REF]100,103,105,106,107,108,109,110,112,114,116,118,119,120,123,125,126,128,133,134,135,136,138,140,144,145,146,149,153,154,155,156,161,169,171,172,173,174,176,177,178,182,183,185,189,190,191,194,195,197,198,199,201,204,205,206,208,209,210,216,219,221,222,224,234,236,237,238,242,244,245,246,247,249,250,251,252,253,259,260,263,264,265,266,267] 115

Testing and evolution [START_REF] Abal | 42 variability bugs in the linux kernel: a qualitative analysis[END_REF][START_REF] Acher | Separation of concerns in feature modeling: Support and applications[END_REF][START_REF] Ajoudanian | Automatic promotional specialization, generalization and analysis of extended feature models with cardinalities in alloy[END_REF][START_REF] Al-Hajjaji | Similarity-based prioritization in software product-line testing[END_REF][START_REF] Apel | Detection of feature interactions using feature-aware verification[END_REF][START_REF] Arcaini | Generating tests for detecting faults in feature models[END_REF][START_REF] Arcaini | Automated repairing of variability models[END_REF][START_REF] Arrieta | Automatic generation of test system instances for configurable cyber-physical systems[END_REF][START_REF] Assunção | Multi-objective reverse engineering of variability-safe feature models based on code dependencies of system variants[END_REF][START_REF] Eb | Assessing the maintainability of software product line feature models using structural metrics[END_REF][START_REF] Bezerra | Exploring quality measures for the evaluation of feature models: a case study[END_REF][START_REF] Brabrand | Intraprocedural dataflow analysis for software product lines[END_REF][START_REF] Bürdek | Reasoning about product-line evolution using complex feature model differences[END_REF][START_REF] Camacho | Cost-related interface for software product lines[END_REF][START_REF] Capilla | The promise and challenge of runtime variability[END_REF][START_REF] Cordy | Beyond boolean product-line model checking: dealing with feature attributes and multi-features[END_REF][START_REF] Dintzner | Extracting feature model changes from the linux kernel using fmdiff[END_REF][START_REF] Dintzner | Analysing the linux kernel feature model changes using fmdiff[END_REF][START_REF] Duran | Investigation of feature run-time conflicts on goal model-based reuse[END_REF]104,105,107,111,113,114,115,117,121,122,124,128,129,130,132,134,135,139,141,143,144,150,152,159,160,161,162,164,165,166,168,170,176,183,186,187,188,192,193,195,196,200,207,212,213,214,222,225,226,227,228,229,230,235,239,241,244,248,250,253,254,255,256,257,258,261] 85

Reverse engineering [START_REF] Acher | On extracting feature models from product descriptions[END_REF][START_REF] Acher | Support for reverse engineering and maintaining feature models[END_REF][START_REF] Acher | Extraction and evolution of architectural variability models in plugin-based systems[END_REF][START_REF] Andersen | Efficient synthesis of feature models[END_REF][START_REF] Assunção | Multi-objective reverse engineering of variability-safe feature models based on code dependencies of system variants[END_REF][START_REF] Bécan | Breathing ontological knowledge into feature model synthesis: an empirical study[END_REF][START_REF] Ter Beek | Supervisory controller synthesis for product lines using cif 3[END_REF][START_REF] Bécan | Synthesis of attributed feature models from product descriptions[END_REF][START_REF] Davril | Feature model extraction from large collections of informal product descriptions[END_REF]126,127,131,134,144,148,149,151,157,163,164,180,181,191,215,233,244,268] 27

Multi-model variability analysis [START_REF] Abele | The cvm framework-a prototype tool for compositional variability management[END_REF][START_REF] Acher | Slicing feature models[END_REF][START_REF] Acher | Composing multiple variability artifacts to assemble coherent workflows[END_REF][START_REF] Acher | Separation of concerns in feature modeling: Support and applications[END_REF][START_REF] Acher | Support for reverse engineering and maintaining feature models[END_REF][START_REF] Bagheri | Configuring software product line feature models based on stakeholders' soft and hard requirements[END_REF][START_REF] Boškovi | Automated staged configuration with semantic web technologies[END_REF][START_REF] Bąk | Clafer: unifying class and feature modeling[END_REF][START_REF] Capilla | The promise and challenge of runtime variability[END_REF][START_REF] Classen | A text-based approach to feature modelling: Syntax and semantics of tvl[END_REF][START_REF] Dhungana | Configuration of multi product lines by bridging heterogeneous variability modeling approaches[END_REF][START_REF] Dhungana | Integrating heterogeneous variability modeling approaches with invar[END_REF]100,113,118,121,134,144,158,179,182,206,210,220,222,237,241,246,262] 29

Variability modelling [START_REF] Antkiewicz | Clafer tools for product line engineering[END_REF][START_REF] Asadi | Toward automated feature model configuration with optimizing non-functional requirements[END_REF][START_REF] Bagheri | Stratified analytic hierarchy process: Prioritization and selection of software features[END_REF][START_REF] Bagheri | Formalizing interactive staged feature model configuration[END_REF][START_REF] Bécan | Breathing ontological knowledge into feature model synthesis: an empirical study[END_REF][START_REF] Ter Beek | Statistical model checking for product lines[END_REF][START_REF] Ter Beek | Supervisory controller synthesis for product lines using cif 3[END_REF][START_REF] Mht | Using mcrl2 for the analysis of software product lines[END_REF][START_REF] Berger | Variability modeling in the real: A perspective from the operating systems domain[END_REF][START_REF] Berger | A study of variability models and languages in the systems software domain[END_REF][START_REF] Berger | What is a feature? a qualitative study of features in industrial software product lines[END_REF][START_REF] Bezerra | Exploring quality measures for the evaluation of feature models: a case study[END_REF][START_REF] Bécan | Synthesis of attributed feature models from product descriptions[END_REF][START_REF] Bąk | Clafer: unifying class and feature modeling[END_REF][START_REF] Chimiak-Opoka | Ocl tools report based on the ide4ocl feature model[END_REF][START_REF] Chrszon | Family-based modeling and analysis for probabilistic systems -featuring profeat[END_REF][START_REF] Chrszon | Profeat: feature-oriented engineering for family-based probabilistic model checking[END_REF][START_REF] Classen | A text-based approach to feature modelling: Syntax and semantics of tvl[END_REF][START_REF] Costa | A scientific software product line for the bioinformatics domain[END_REF][START_REF] Czarnecki | Cool features and tough decisions: a comparison of variability modeling approaches[END_REF][START_REF] Db | Ontology-based feature modeling: An empirical study in changing scenarios[END_REF][START_REF] Dintzner | Extracting feature model changes from the linux kernel using fmdiff[END_REF][START_REF] Dougherty | Model-driven auto-scaling of green cloud computing infrastructure[END_REF]101,102,108,109,110,118,134,136,137,139,141,142,144,145,146,147,149,157,158,167,169,175,179,183,184,194,197,199,202,203,210,211,217,218,219,221,223,224,225,231,232,240,242,243,244,252,261,262,265,266,269] 74

Variability intensive systems analysis [START_REF] Abal | 42 variability bugs in the linux kernel: a qualitative analysis[END_REF][START_REF] Asadi | Development and validation of customized process models[END_REF][START_REF] Ter Beek | Statistical model checking for product lines[END_REF][START_REF] Ter Beek | Supervisory controller synthesis for product lines using cif 3[END_REF][START_REF] Ter Beek | Statistical analysis of probabilistic models of software product lines with quantitative constraints[END_REF][START_REF] Boškovi | Automated staged configuration with semantic web technologies[END_REF][START_REF] Buchmann | Mod2-scm: A model-driven product line for software configuration management systems[END_REF][START_REF] Capilla | The promise and challenge of runtime variability[END_REF][START_REF] Costa | A scientific software product line for the bioinformatics domain[END_REF][START_REF] Davril | Feature model extraction from large collections of informal product descriptions[END_REF][START_REF] Del-Río-Ortega | On the definition and design-time analysis of process performance indicators[END_REF][START_REF] Dintzner | Extracting feature model changes from the linux kernel using fmdiff[END_REF][START_REF] Dougherty | Model-driven auto-scaling of green cloud computing infrastructure[END_REF]119,120,134,153,156,167,178,190,204,213,214,232,244,259,267,269] 29

Table 8: Classification of papers based on the variability context facet q q q q q q q q 0% 20% 40% 60% 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 Figure 9 shows the evolution of the different trends depending on the year of publication. In the y axis, the percentage of papers per year of each variability context facet is presented which can show the interest of the facet in the corresponding year. We see that there are some areas that stayed with minor variations, such as multi-model variability analysis and variability modelling, while there are others that clearly have hyped during the last years, such as reverse engineering.

We want to highlight that "Product configuration and derivation" has a good percentage of papers over the time span of the review. This is probably due to the fact that it is one of the initial and more established usages of AAFM. Also, the contributions targeting "Multi-model variability analysis" had an stable percentage of papers every year, and has not hyped yet. We conjecture that this is due to the fact of the complexity of the underling analysis on multiple models and the lack of well recognized case studies and examples on this field. Finally, another observation is that before 2014 there were two groups of trends that evolved similarly. First, "Multi-model variability analysis", "Reverse engineering" and "Variability intensive systems analysis"; and second, "Product configuration and derivation", "Testing and evolution" and "Variability modelling". In the last years, we see the hype of "Variability modelling". We conjecture that the hype of this last trend is due to industrial needs of specifying specific variability properties within their domains [START_REF] Alférez | Modeling variability in the video domain: language and experience report[END_REF]. 5.4 RQ4: Which kind of publications are used to address the challenges?

The research facet classifies the papers depending on its research type. In Table 9 we observe the number of papers within our set of papers that fits each research type.

We observe that there is a clear higher amount of papers on the evaluation, philosophical and solution proposal papers categories. This indicates that even this area is maturing and we can foresee some papers focusing on real-world problems, most papers are not related to the industry. We notice that most of the research lacks validation and focuses on evaluating concrete theoretical approaches. This pinpoints the need for this community to work closer to the industry and to validate methods and techniques in a more practical way.

Research facet Citations #papers

Opinion Paper [START_REF] Boucher | Deriving configuration interfaces from feature models: A vision paper[END_REF][START_REF] Capilla | The promise and challenge of runtime variability[END_REF][START_REF] Capilla | Context variability for context-aware systems[END_REF]144,164,193, 248] 7 Philosophical Paper [START_REF] Bagheri | Formalizing interactive staged feature model configuration[END_REF][START_REF] Capilla | An overview of dynamic software product line architectures and techniques: Observations from research and industry[END_REF][START_REF] Durán | Flame: a formal framework for the automated analysis of software product lines validated by automated specification testing[END_REF]101,134,137,171,175,196,201,244] 11 Solution Proposal [START_REF] Acher | Slicing feature models[END_REF][START_REF] Acher | Familiar: A domain-specific language for large scale management of feature models[END_REF][START_REF] Arcaini | Automated repairing of variability models[END_REF][START_REF] Mht | Using mcrl2 for the analysis of software product lines[END_REF][START_REF] Berger | Feature-to-code mapping in two large product lines[END_REF][START_REF] Boškovi | Automated staged configuration with semantic web technologies[END_REF][START_REF] Camacho | Cost-related interface for software product lines[END_REF][START_REF] Chimiak-Opoka | Ocl tools report based on the ide4ocl feature model[END_REF][START_REF] Chrszon | Family-based modeling and analysis for probabilistic systems -featuring profeat[END_REF][START_REF] Chrszon | Profeat: feature-oriented engineering for family-based probabilistic model checking[END_REF][START_REF] Classen | A text-based approach to feature modelling: Syntax and semantics of tvl[END_REF][START_REF] Diskin | Faithful modeling of product lines with kripke structures and modal logic[END_REF]102,106,140,141,142,145,151,158,162,165,172,176,184,185,191,195,197,198,199,200,203,205,207,210,217,219,220,225,236,239,243,246,247,262,264,265,268] 49

Evaluation Research [START_REF] Abal | 42 variability bugs in the linux kernel: a qualitative analysis[END_REF][START_REF] Abele | The cvm framework-a prototype tool for compositional variability management[END_REF][START_REF] Acher | Composing multiple variability artifacts to assemble coherent workflows[END_REF][START_REF] Acher | Separation of concerns in feature modeling: Support and applications[END_REF][START_REF] Acher | Extraction and evolution of architectural variability models in plugin-based systems[END_REF][START_REF] Ajoudanian | Automatic promotional specialization, generalization and analysis of extended feature models with cardinalities in alloy[END_REF][START_REF] Al-Hajjaji | Similarity-based prioritization in software product-line testing[END_REF][START_REF] Andersen | Efficient synthesis of feature models[END_REF][START_REF] Antkiewicz | Clafer tools for product line engineering[END_REF][START_REF] Apel | Detection of feature interactions using feature-aware verification[END_REF][START_REF] Apel | Feature-interaction detection based on featurebased specifications[END_REF][START_REF] Arcaini | Generating tests for detecting faults in feature models[END_REF][START_REF] Arrieta | Automatic generation of test system instances for configurable cyber-physical systems[END_REF][START_REF] Asadi | Development and validation of customized process models[END_REF][START_REF] Asadi | Toward automated feature model configuration with optimizing non-functional requirements[END_REF][START_REF] Assunção | Multi-objective reverse engineering of variability-safe feature models based on code dependencies of system variants[END_REF][START_REF] Bagheri | Stratified analytic hierarchy process: Prioritization and selection of software features[END_REF][START_REF] Bagheri | Configuring software product line feature models based on stakeholders' soft and hard requirements[END_REF][START_REF] Eb | Assessing the maintainability of software product line feature models using structural metrics[END_REF][START_REF] Bécan | Breathing ontological knowledge into feature model synthesis: an empirical study[END_REF][START_REF] Ter Beek | Statistical model checking for product lines[END_REF][START_REF] Ter Beek | Supervisory controller synthesis for product lines using cif 3[END_REF][START_REF] Benavides | Automated analysis in feature modelling and product configuration[END_REF][START_REF] Brabrand | Intraprocedural dataflow analysis for software product lines[END_REF][START_REF] Buchmann | Mod2-scm: A model-driven product line for software configuration management systems[END_REF][START_REF] Bürdek | Reasoning about product-line evolution using complex feature model differences[END_REF][START_REF] Bécan | Synthesis of attributed feature models from product descriptions[END_REF][START_REF] Bąk | Clafer: unifying class and feature modeling[END_REF][START_REF] Chen | Optimizing the product derivation process[END_REF][START_REF] Classen | Symbolic model checking of software product lines[END_REF][START_REF] Cordy | Beyond boolean product-line model checking: dealing with feature attributes and multi-features[END_REF][START_REF] Czarnecki | Cool features and tough decisions: a comparison of variability modeling approaches[END_REF][START_REF] Davril | Feature model extraction from large collections of informal product descriptions[END_REF][START_REF] Del-Río-Ortega | On the definition and design-time analysis of process performance indicators[END_REF][START_REF] Db | Ontology-based feature modeling: An empirical study in changing scenarios[END_REF][START_REF] Dhungana | Configuration of multi product lines by bridging heterogeneous variability modeling approaches[END_REF][START_REF] Dhungana | Integrating heterogeneous variability modeling approaches with invar[END_REF][START_REF] Dintzner | Analysing the linux kernel feature model changes using fmdiff[END_REF][START_REF] Dougherty | Model-driven auto-scaling of green cloud computing infrastructure[END_REF][START_REF] Dumitru | On-demand feature recommendations derived from mining public product descriptions[END_REF][START_REF] Duran | Investigation of feature run-time conflicts on goal model-based reuse[END_REF][START_REF] Duran-Limon | An ontology-based product architecture derivation approach[END_REF]100,103,104,107,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,135,138,139,143,146,147,148,149,150,152,153,154,155,157,159,160,161,163,166,167,168,169,170,173,174,177,178,179,180,181,182,183,186,188,189,190,192,202,204,206,208,209,212,214,215,216,218,222,223,224,226,227,228,229,230,231,233,234,235,237,238,240,241,245,249,250,252,255,256,257,258,260,261,263,266,267,269] 148

Validation Research [START_REF] Acher | On extracting feature models from product descriptions[END_REF][START_REF] Acher | Support for reverse engineering and maintaining feature models[END_REF][START_REF] Ter Beek | Statistical analysis of probabilistic models of software product lines with quantitative constraints[END_REF][START_REF] Berger | Variability modeling in the real: A perspective from the operating systems domain[END_REF][START_REF] Berger | A study of variability models and languages in the systems software domain[END_REF][START_REF] Berger | What is a feature? a qualitative study of features in industrial software product lines[END_REF][START_REF] Costa | A scientific software product line for the bioinformatics domain[END_REF][START_REF] Dintzner | Extracting feature model changes from the linux kernel using fmdiff[END_REF]105,136,156,187,194,211,213,221,232,251,253,254,259] 21

Experience Report [START_REF] Baresi | Service-oriented dynamic software product lines[END_REF][START_REF] Bezerra | Exploring quality measures for the evaluation of feature models: a case study[END_REF][START_REF] Chavarriaga | Using multiple feature models to specify configuration options for electrical transformers: An experience report[END_REF][START_REF] Dumitrescu | Bridging the gap between product lines and systems engineering: an experience in variability management for automotive model based systems engineering[END_REF]108,242] 6

Table 9: Classification of papers based on the research facet 5.5 RQ5: When have the papers been published?

In this question, we want to determine the dates where the papers were published. Also, we want to see how the number of papers published in the different fora types used across the paper evolved across the different years. To answer this question we relied on the first batch of papers containing 423 contributions. Note that the papers are normalized by the number primary studies published that year. We observe in Figure 10 that there is an increment of papers published in journals. This is specially remarkable in the years 2017 were the papers published in journals were more than in conferences and workshops. This points out that AAFM research works have matured when published. Also, the number of papers published in SPLC, VAMOS, ICSE, and ASE is lower than the number of papers published in the rest of conferences when taking into account the publication year.

The difference between variability related conferences and the non related ones stays mostly unaltered in the last three years covered by this study. However, we can see that the percentage of variability related conferences paper is decreasing since 2013. We conjecture that this change in the trend is due to the fact that the AAFM is now more widespread in other communities and therefore, non variability related conferences accept more papers on this topic. 5.6 RQ6: How are the papers related among them?

This question aims at finding research gaps to drive future work in the area of AAFM. By identifying the variability-context and type of papers where there are fewer publications we can infer the lacks of current research.

To answer this question we relied on the reduced dataset only considering variability-related conferences and journals. It is interesting to see the global distribution (see Figure 11) of the number of papers based on the two main dimensions considered in this mapping study. This heatmap shows the research gaps where the SPL community should invest more efforts. Firstly, we observe that there is still room for improvement regarding experience reports because only eleven contributions fall in this category. Moreover, we almost found papers about product configuration and derivation and variability modelling in such research facet. This is a clear gap that evidences the distance to industry of the other trends. Secondly, we observe that opinion and philosophical papers have low weight which may show that the discipline is getting mature because there are taxonomies, literature reviews or mapping studies on the different trends. Finally, we see that most of the papers are on the evaluation research category and testing and evolution context facet, i.e. the papers evaluate the solution in front of small examples or datasets. We conjecture that this is due to the fact of the availability of well known examples and datasets on these trends.

Generally speaking, we observe that the area is getting mature but, still misses some more collaboration with industry. In fact, those variability contexts that lately attracted the most researchers attention did not have time to report on experience reports. Also, we observe a tendency of publishing conceptual ideas that are later evaluated in terms of efficiency (evaluation) instead of customer satisfaction (validation).

Conclusions

In this paper we went through different research questions to understand the current state of the area of AAFM by applying a systematic mapping method. We identified where relevant papers are being published; who are the authors and institutions that currently holds the know-how of AAFM; which trends in the usage of AAFM attracted more researchers attention in the last years; what kind of publications are being proposed; when and where the papers were published.

The surveyed primary sources point out that the AAFM is a subject that is getting mature and that has driven some other subjects such as product configuration, derivation, testing, evolution and reverse engineering. However, regarding the distribution depending on the nature of primary sources, the main fact detected is that FM practitioners are not validating the research as much as they evaluate it. This actually should encourage the community to work closer to industry and provide better ready-to-themarket solutions instead of toy techniques and prototypes.

The aim of this study is to guide future research on the application of the AAFM in new domains such as images creation, cloud management, mobile computing, operating systems dependencies or internet of the things among others. We believe that it is time to stop gleaming luster to the analysis techniques that are in general mature enough and find application domains and real evidence where the AAFM can be successfully applied.

Fig. 2 :

 2 Fig. 2: AAFM framework.

Fig. 3 :

 3 Fig. 3: Systematic mapping studies and literature reviews

Fig. 4

 4 Fig.4: Systematic mapping study process[START_REF] Montalvillo | Requirement-driven evolution in software product lines: A systematic mapping study[END_REF][START_REF] Petersen | Systematic mapping studies in software engineering[END_REF]

Fig. 5 :

 5 Fig. 5: Planning the review phase

Fig. 6 :

 6 Fig. 6: Study identification phase

Fig. 7 :

 7 Fig. 7: Search citing publications process

  Fig.8: Systematic mapping study process[START_REF] Petersen | Systematic mapping studies in software engineering[END_REF]

Fig. 9 :

 9 Fig. 9: Temporary distribution of the variability context facet

Fig. 10 :

 10 Fig. 10: Temporary distribution of papers.

Fig. 11 :

 11 Fig. 11: Visualization of the systematic map

Table 1 :

 1 Comparison of related reviews

	Review work	Kind	Year	Context	Period	# papers studied
	Benavides et al. [4]	SLR	2010	Automated analysis of feature	1990-2010	53
				models
	Da Mota et al. [23]	SMS	2011	SPL testing	1993-2009	120
	Engström and Runeson [9]	SMS	2011	SPL testing	1990-2008	64
	Laguna and Crespo [18]	SMS	2013	SPL evolution	1990-2011	74
	López-Herrejón et al. [19]	SMS	2015	Search-based SPLs	2011-2014	77
	Montalvillo and Díaz [22]	SMS	2016	Requirement-driven evolution in	1990-2015	107
				SPLs
	This paper	SMS	2018	Automated analysis of feature	2010-2017	423/242
				models

  1: Conducting search

	Google scholar	Google scholar	Scopus	Scopus
	Search (15300)	Citing (945)	Search (1167)	Citing(572)
	01/2010 -12/2017	01/2010 -12/2017	01/2010 -12/2017	01/2010 -12/2017
	by citations	by citation	by citation	by citation
		Merged Studies
		(800)
		Initial set of studies
		(445)

Table 2 :

 2 8 or SCImago9 . Top-ten journalsThe Software Quality Journal appeared in third place, this might be indicative of the interest in testing variability-intensive systems and SPL

		#papers Journal										Acronym SCImago JCR
	1	13	Journal of Systems and Software					JSS
	2	11	International Journal on Software and Systems Mod-	SOSYM
			eling,Software and Systems Modeling
	3	9	Software Quality Journal							SQJ
	4	7	IEEE Transactions on Software Engineering		TSE
	5	6	Information and Software Technology				IST
	6	6	Science of Computer Programming				SCP
	7	5	International Journal on Software Tools for Technol-	STTT
			ogy Transfer
	8	4	Expert Systems with Applications				ESA
	9	3	Computer										Computer
	10	3	Empirical Software Engineering					ESE
		Rank Journal	Variability intensive	systems analysis	Multi-model	variability analysis	Product configuration	and derivation	Testing and evolution	Variability modelling	Reverse engineering	Experience Report	Philosophical Paper	Opinion Paper	Solution proposal	Evaluation Research	Validation Research

Table 4 :

 4 Top-ten conferencesTable

	Conference

  Table 7 presents the institutions publishing more than four AAFM related papers. It is remarkable that the AAFM is mostly attracting European institutions.

	#Papers Author	Variability intensive	systems analysis	Multi-model	variability analysis	Product configuration	and derivation	Testing and evolution	Variability modelling	Reverse engineering	Experience Report	Philosophical Paper	Opinion Paper	Solution proposal	Evaluation Research	Validation Research
	14	Acher, Mathieu	0	4	1		1	0	3	0	0	0	2	3	2
	7	Lopez-Herrejon, Roberto E	0	0	0		4	0	2	0	0	1	2	2	0
	7	ter Beek, Maurice H	3	0	3		0	2	1	0	0	0	0	2	1
	6	Segura, Sergio	0	0	0		5	0	0	0	0	0	0	5	0
	6	Wang, Shuai	0	0	2		4	1	0	0	0	0	0	3	2
	6

Table 6 :

 6 First authors having more than four papers in the survey

	# papers	Domain	Country	Institution
	15	irisa.fr	France	Institut National de Recherche en Informatique et Automatique de Rennes
	12	us.es	Spain	University of Seville
	7	uni.lu	Luxembourg	Université du Luxembourg
	7	berkeley.edu	USA	University of Berkeley
	7	etsmtl.ca	Canada	Université du Québec
	7	isti.cnr.it	Italy	Institute of the National Research Council of Italy
	5	osumc.edu	USA	Ohio State university
	5	lero.ie	Ireland	Lero -The Irish Software Research Centre
	4	uni-passau.de	Germany	Universität Passau
	4	univ-paris1.fr	France	Unversité Paris 1
	4	tu-dresden.de	Germany	Technische Universität Dresden
	4	unamur.be	Belgium	Université de Namur
	4	chalmers.se	Sweden	Chalmers University of technology
	4	ryerson.ca	Canada	Ryerson University
	4	mcmaster.ca	Canada	McMaster University
	4	ovgu.de	Denmark	Otto-von-Guericke-Universitaet Magdeburg

Table 7 :

 7 Institutions working on AAFM 5.3 RQ3: What are the areas for which AAFM has been applied?

 reduce the period of time due to the topic handled.

http://scholar.google.com

http://www.scopus.com

Note that, as well as the standard process proposed defined by[START_REF] Petersen | Systematic mapping studies in software engineering[END_REF] to query bibliographic databases, we added a second group of papers citing the paper that settle the body of knowledge of AAFM[START_REF] Benavides | Automated analysis of feature models 20 years later[END_REF] as justified in Section 3.1.

www.jabref.com

https://jcr.incites.thomsonreuters.com

https://www.scimagojr.com

Acknowledgements

This work was supported, in part, by the European Commission (FEDER), by the Spanish government under BELi (TIN2015-70560-R) project and by the Andalusian government under the COPAS (TIC-1867) project. You can find all the material used in this paper in the website https://isa-group.github.io/ aafm-quo-vadis/.

B Conferences and Journals

This section contains all the material used in this mapping study.

Rank