1) Comment fournir une aide dès la conception de nouvelles MSPLs avec de nouveaux langages dédiés ?

2) Comment personnaliser le support de dérivation de produit de CVL ?

3) Comment amener la séparation des préoccupations en matière de modélisation de la variabilité en ingénierie système ? 4) Comment intégrer la gestion de la variabilité en ingénierie système d'une manière transparente et non intrusive ? 5) Comment exploiter la génération valide des produits ?

Contributions

Nos contributions sont basées sur le fait qu'une solution générique, pour tous les domaines, et qui dérive des modèles corrects n'est pas réaliste, surtout si on prend en considération le contexte des systèmes complexes décrits précédemment. Par conséquent, au lieu d'essayer de trouver la solution miracle (par exemple, un vérificateur générique de modèles de ligne de produits ou une notation unifiée pour exprimer les MSPLs en ingénierie système), nous prenons le parti inverse et proposons une approche indépendante du domaine pour générer des contre-exemples de MSPLs, révélant des erreurs de conceptions de modèles et supportant les parties prenantes à construire de meilleures MSPLs et des mécanismes de dérivation plus efficaces.

Plus précisément, la première et principale contribution de la thèse est un processus systématique et automatisé, basé sur CVL, pour la recherche aléatoire de contreexemples de MSPL dans un langage donné. Les contre-exemples sont des exemples de MSPLs qui autorisent la dérivation de modèles invalides, syntaxiquement ou sémantiquement, malgré une configuration valide dans le modèle de variabilité.

Ces contre-exemples visent à révéler des erreurs ou des risques -soit dans le moteur de dérivation ou dans le modèle de réalisation -aux parties prenantes de MSPLs. D'une part, ces contre-exemples font office d'oracles de tests pour augmenter la robustesse des mécanismes de vérification de la MSPL. Les développeurs peuvent alors utiliser des contre-exemples pour prévoir des valeurs limites et les types de MSPLs qui sont susceptibles de permettre des dérivations erronées. D'autre part, les parties prenantes peuvent répéter le même type d'erreurs lors de la spécification des correspondances entre un modèle de variabilité et d'un modèle de base. Les contre-exemples agissent alors comme des anti-patrons qui devraient éviter de mauvaises pratiques ou diminuer le nombre d'erreurs pour un langage de modélisation dédié.

Nous validons l'efficacité de ce processus autour de trois formalismes (UML, Ecore et une simple machine à états finis) à différentes échelles (jusqu'à 247 méta-classes et 684 règles) et différentes façons d'exprimer les règles de validation. De plus, nous l'appliquons dans un scénario industriel réel (en partenariat avec Thales Research & Technology) démontrant comment notre approche s'exécute dans la pratique.

Nous explorons aussi l'hypothèse exposée ci-dessus : qu'un moteur de dérivation générique ou un support de base pour la gestion de la couche de réalisation est susceptible d'autoriser MSPLs incorrectes. Nous discutons dans quelle façon les contre-exemples pourraient guider les praticiens lors de: la personnalisation des moteurs de dérivation, la mise en oeuvre des règles de vérification qui empêchent la création d'un modèle CVL incorrect, ou tout simplement lors de la spécification d'un MSPL. Les techniques génératives et l'étude exploratoire suggèrent l'utilisation de solutions au courant de la sémantique des langages de modélisation ciblés lors de l'élaboration des MSPLs.

La seconde contribution de la thèse est un étude sur les mécanismes pour étendre la sémantique des moteurs de dérivation, offrant une approche basée sur des modèles à fin de personnaliser leurs sémantique opérationnelle. Cette approche facilite la définition des opérateurs de réalisation corrects par l'ingénieur du domaine. Il s'agit d'une étape naturelle après les preuves empiriques obtenues par le procédé automatisé mentionné comme première contribution.

Dans la troisième contribution de la thèse , nous extrapolons les limites de langages de modélisation. Nous présentons une étude empirique à large échelle sur le langage Java qui comprend : une évaluation automatisée de tous ses éléments de langage et comment les opérateurs de réalisation de CVL sont pertinents ou non à modifier et faire varier les programmes Java. Nous proposons une classification complète des transformations de variabilité. Nos données statistiques aident à caractériser quelles constructions du langage sont susceptibles de varier ou nécessitent des transformations spécifiques. D'un point de vue qualitatif, nous examinons et analysons les variantes de programmes Java générées à l'aide d'outils dédiés. Cette expérience sert aussi à démontrer les premières étapes après avoir généré les contre-exemples: l'analyse, l'ordonnancement et la classification.

La quatrième et dernière contribution de la thèse est une méthodologie pour intégrer notre travail dans une organisation qui cherche à mettre en oeuvre les lignes de produit logiciels basées sur des modèles pour l'ingénierie des systèmes. Nous nous concentrons sur la réponse à l'événement de devoir concevoir une MSPL pour un nouveau langage dédié, montrant les différentes activités, comment elles se succèdent et les rôles impliqués dans chacune.

Résumé en Français

Les applications et les systèmes logiciels sont devenus omniprésents dans notre quotidien, qu'il s'agisse des véhicules de transport, sur nos routes, à la maison dans nos appareils multimédias mais aussi notre électroménager voire les bâtiments eux-mêmes, etc. Ces systèmes cyber-physiques [START_REF]Cyber-physical systems[END_REF] (i.e., les systèmes avec des éléments computationnels collaboratifs qui contrôlent des éléments physiques) imposent aux ingénieurs de travailler avec des logiciels de taille très importante --une nouvelle voiture a environ 100 millions de lignes de code. Pour faciliter le développement de tels systèmes, les ingénieurs adoptent l'approche diviser pour régner et de séparation des préoccupations. Le système est donc développé en utilisant de nombreux et différents langages dédiés et impliquant des différentes parties prenantes.

Cette observation est valable pour la plupart des acteurs industriels, quelque soit leur secteur d'activité, et est particulièrement vérifiée dans le cas de Thales: une entreprise multinationale qui opère dans les domaines complexes, tels que l'aéronautique, l'espace, les systèmes de défense ou encore le transport. Leurs ingénieurs utilisent différents langages dédiés à fin de développer des ensembles intégrés de systèmes. Ces langages sont construits dans un ensemble de représentations dédiées à l'analyse de problèmes spécifiques à un domaine et s'appuient sur l'ingénierie dirigée par les modèles (MDE)! [START_REF] Douglas | Model-Driven Engineering[END_REF].

Ces entreprises ont également besoin de construire des versions\variantes légèrement différentes d'un même système. Ces versions partagent des points communs et des différences, le tout pouvant être géré à l'aide d'une approche ligne de produits (SPL -Software Product Line, même si on pourrait dorénavant remplacer la spécificité du logiciel par une encapsulation plus large, à savoir le système lui même) [START_REF] Clements | Software Product Lines : Practices and Patterns[END_REF][START_REF] Pohl | Software Product Line Engineering: Foundations, Principles and Techniques[END_REF]. L'objectif principal d'une SPL est d'exploiter la personnalisation de masse, dans laquelle les produits sont réalisés pour répondre aux besoins spécifiques de chaque client! [START_REF] Benavides | Automated Analysis of Feature Models 20 Years Later : A Literature Review[END_REF]. Pour répondre à ce besoin de personnalisation, les systèmes doivent être étendus de manière efficace, ou modifiés, configurés pour être utilisé dans un contexte particulier! [START_REF] Svahnberg | A taxonomy of variability realization techniques: Research articles[END_REF][START_REF] Chen | Variability management in software product lines: a systematic review[END_REF].

Une approche encourageante consiste à connecter l'approche MDE (séparation des préoccupations, modélisation et langages dédiés) à l'approche SPL (gestion de la variabilité) -les SPL basées sur les modèles (MSPL). Les produits de la ligne de produits sont ainsi exprimés sous la forme de modèles conformes à un méta-modèle et ses règles de bonne formation. De nombreuses techniques de MSPL ont été proposées ([PBvdL05, PKGJ08, HSS+10, CA05a, CHS+10b, CP06, ZJ06, VG07]). Ces approches sont composées généralement par i) un modèle de variabilité (e.g., un modèle de caractéristiques ou un modèle de décision), ii) un modèle de base (e.g., une machine à états, un diagramme de classes) exprimé dans un langage de modélisation spécifique (par exemple, le langage de modélisation UML de l'OMG (Unified Modeling Language! [Gro07]), et iii) une couche de réalisation qui met en correspondance les points de variations et les éléments d'un modèle de base.

Basée sur une sélection de caractéristiques souhaitées dans le modèle de variabilité, un moteur de dérivation peut synthétiser automatiquement des modèles personnaliséschaque modèle correspondant à un produit individuel de la ligne de produits. Dans ce contexte de MSPLs, le langage \emph {CVL (Common Variability Language}! [START_REF] Fleurey | Standardizing Variability -Challenges and Solutions[END_REF] a récemment émergé comme un effort de standardisation et la promotion des MSPLs. Dans cette thèse, nous adoptons CVL comme langage de construction de MSPLs.

Challenges

L'espace de conception, l'environnement du système logiciel que l'on construit (i.e., l'ingénierie du domaine) d'une MSPL est extrêmement complexe à gérer pour un ingénieur. Tout d'abord, le nombre possible des produits d'une MSPL est exponentielle au nombre d'éléments ou de décisions exprimé dans le modèle de variabilité. Ensuite, les modèles de produits dérivés doivent être conformes à de nombreuses règles liées au domaine métier mais aussi aux langages de modélisation utilisés. Par exemple, UML présente 684 règles de validation dans l'implémentation EMF. Par conséquent, un développeur doit comprendre les propriétés intrinsèques du langage de modélisation pour concevoir une MSPL. Troisièmement, le modèle de réalisation qui relie un modèle de variabilité et un modèle de base peut être très expressif, spécialement dans le cas de CVL. La gestion des modèles de variabilité et de modèles de conception est une activité non triviale, relier les deux parties et par conséquent l'ensemble des modèles est une tâche non négligeable et susceptible à erreurs.

En plus de ces défis intrinsèques aux MSPLs, il faut ajouter que les ingénieurs système utilisent différents langages de modélisation dédiés dans le cadre de projets pour la réalisation de systèmes critiques. Comme les modèles sont conformes à leurs propres règles de bonne formation et des règles spécifiques du domaine, chaque utilisation d'un nouveau langage de modélisation pour l'élaboration d'une MSPL implique la révision de la couche de réalisation.

Nous pouvons résumer ces défis autour de cinq questions de recherche que nous abordons dans cette thèse. Ces questions sont la conséquence de l'effort de gestion de la variabilité, en ingénierie système chez Thales, autour d'un scénario utilisant plusieurs langages et avec une nécessité d'avoir des modèles corrects.

Introduction Context

Software and Systems are becoming increasingly essential for daily life; they are omnipresent in the different transportation, home-appliances, civil infrastructures, entertainment or healthcare devices. These Cyber-Physical Systems [START_REF]Cyber-physical systems[END_REF] (i.e., systems of collaborating computational elements controlling physical elements) impose engineers to deal with massive pieces of software -a typical new car has about 100 million lines of code 1 . To ease the development of such systems, engineers adopt a divide and conquer approach: each concern of the system is engineered separately, with several domain specific languages and stakeholders. This is the case in many companies in different industry sectors, being also true in Thales 2 : a large company actuating in complex domains, such as aerospace, space, defence and transportation. Their stakeholders use numerous domain specific modeling languages to develop integrated sets of systems. These languages are built within a set of dedicated representations to analyze domain-specific problems and they rely on the Model-driven Engineering (MDE) [START_REF] Douglas | Model-Driven Engineering[END_REF] paradigm.

On the other hand, these companies also need to construct slightly different versions/variants of a same system; these variants share commonalities and variabilities that can be managed using a Software Product Line (SPL) [START_REF] Clements | Software Product Lines : Practices and Patterns[END_REF][START_REF] Pohl | Software Product Line Engineering: Foundations, Principles and Techniques[END_REF] approach. The main goal of an SPL is to leverage mass customization, in which products are made systematically to meet individual customer's needs [START_REF] Benavides | Automated Analysis of Feature Models 20 Years Later : A Literature Review[END_REF]. To meet this need for customization, systems have to be efficiently extended, changed or configured for use in a particular context [START_REF] Svahnberg | A taxonomy of variability realization techniques: Research articles[END_REF][START_REF] Chen | Variability management in software product lines: a systematic review[END_REF].

A promising approach is to ally MDE with SPL -Model-based SPLs (MSPL)-in a way that the products of the SPL are expressed as models conforming to a metamodel and well-formedness rules. Numerous MSPL techniques have been proposed (e.g., see [PBvdL05, PKGJ08, HSS + 10, CA05a, CHS + 10b, CP06, ZJ06, VG07]). They usually consist in i) a variability model (e.g., a feature model or a decision model), ii) a model (e.g., a state machine, a class diagram) expressed in a specific modeling language (e.g., Unified Modeling Language (UML) [Gro07]), and iii) a realization layer that maps and transforms variation points into model elements. Based on a selection of desired Introduction features in the variability model, a derivation engine can automatically synthesise customized models -each model corresponding to an individual product of the SPL. The Common Variability Language (CVL) [FHMP + 11a] has recently emerged as an effort to standardize and promote MSPLs; it is our adopted language for constructing MSPL.

MSPLs in systems engineering), we provide a domain-independent approach to generate counterexamples of MSPLs, revealing dangerous designs and assisting stakeholders to construct better MSPLs and derivation mechanisms.

Specifically, the first and core contribution is a systematic and automated process, based on CVL, to randomly search for counterexamples of MSPL of a given language.

Counterexamples are examples of MSPLs that authorize the derivation of syntactically or semantically invalid product models despite of a valid configuration in the variability model. These counterexamples aim at revealing errors or risks -either in the derivation engine or in the realization model -to stakeholders of MSPLs. On the one hand, counterexamples serve as testing "oracles" for increasing the robustness of checking mechanisms for the MSPL. Developers can use counterexamples to foresee boundary values and types of MSPLs that are likely to allow incorrect derivations. On the other hand, stakeholders may repeat the same kind of errors when specifying the mappings between a variability model and a base model. Counterexamples act as "antipatterns" that should avoid bad practices or decrease the amount of errors for a given modeling language.

We validate the effectiveness of this process for three formalisms (UML, Ecore and a simple finite state machine) with different scales (up to 247 metaclasses and 684 rules) and different ways of expressing validation rules. In addition, we apply it in a real industry scenario (a partnership with Thales Research & Technology), demonstrating how our approach performs in practice. We also explore the hypothesis exposed above, i.e., that a generic derivation engine or a basic support for managing the realization layer is likely to authorize incorrect MSPLs. practitioners when customizing derivation engines, when implementing checking rules that prevent early incorrect CVL models, or simply when specifying an MSPL. Overall, the generative techniques and exploratory study call for solutions aware of the semantics of the targeted modeling languages when developing MSPLs.

Our second contribution is a study on the mechanisms to extend the semantics of derivation engines, providing a model-based approach to customize their operational semantics. This approach eases the definition of safe realization operators by the domain engineer. It is a natural step after the empirical evidences obtained by the automated process mentioned as first contribution.

As a third contribution, we extrapolate the limits of modelling languages. We perform a substantial empirical study on Java: an automated assessment of all its language constructs and how CVL-based realization operators adequate or not to vary Java programs. We provide a comprehensive classification of variability transformations. Statistical data help to characterize which language constructs are likely to vary or require specific transformations. From a qualitative perspective, we review and analyze the resulting Java variants with the help of dedicated tools. This experiment also serves to demonstrate the first steps after having generated counterexamples: analysing, ordering and categorizing them.

Our fourth and final contribution is a methodology to integrate our work into any organization that seeks to implement model-based software product Introduction lines for systems engineering. We focus on the response to the event of having to engineer an MSPL for a new DSL, showing the different activities, how they follow each other and the roles involved in each of them.

Part I

Context and State of the Art

Chapter 1

Context

In this chapter, we contextualize the thesis by presenting the scenario of Systems Engineering: a complex development activity that involves various stakeholders and technologies. In Section 1.1, we show that the diversity of languages is an important factor in this scenario, exemplifying with the case of the Thales group (see Section 1.1.2). We motivate that systems engineering could benefit from a product line approach once the variability inside these languages is externally handled (Section 1.2), by using the Common Variability Language (CVL).

Continuing, we present the challenges of managing variability in such a complex context, justifying and illustrating the current practice of specializing the derivation semantics of CVL (Section 1.3), which makes the challenge of deriving correct products even harder (Section 1.4). We then conclude that engineering software product lines for systems engineering needs special assistance (Section 1.5) and, finally, we synthesize the challenges in Section 1.6.

Systems Engineering: a tale of many languages and variants

The system of a typical new car has about 100 million lines of code1 ; it is inconceivable to make, manage or evolve such a big and complex piece of software by considering it as a monolithic system -a divide and conquer approach is imperative to succeed. The need for this division outperforms the problem of modularizing and separating concerns of the system and its software, it also involves the issue of dealing with several different stakeholders in charge of conceiving and engineering each part [START_REF] Andrew | Handbook of systems engineering and management[END_REF].

In the Systems Engineering field, stakeholders are often domain specialists (e.g, in the automotive domain there are engineers responsible for: safety, vehicle dynamics, performance, drivability, etc.); each one of them needs to deal with particular information of their particular domain, and they are not necessarily concerned by the other domains. Having one general language to design all parts and reason about all domains 10 Context is inefficient and hard to manage. Therefore, they use many specialized languages to build systems for their field. The use of Domain-Specific Languages (DSL) allows to abstract from unneeded complexity and to have efficient dedicated tooling.

Model-driven Engineering in the state of practice of DSLs

One way to implement DSLs is using Model-driven Engineering (MDE) [START_REF] Douglas | Model-Driven Engineering[END_REF]. Instead of having a grammar specifying the abstract syntax of a language, a Domain-Specific Modeling Language (DSML) uses a metamodel to express the concepts of a domain. Metamodels are the central artifact of a DSML; they can be complemented by constraints expressed in a declarative language like OCL (Object Constraint Language), by a concrete syntax, by an execution semantics, etc.

Recent investigations of the state of practice in industry reveal that the use of MDE is widespread and that the majority of MDE usage examples follows domain-specific modeling paradigms. Whittle and others [START_REF] Whittle | The state of practice in model-driven engineering[END_REF] report that most of the companies that succeeded to adopt MDE in their development process created or used languages specialized for their own domains; also, they confirmed that the number of DSMLs in a company tends to be large and that, sometimes, several "mini"-DSMLs are used within a single project [START_REF] Hutchinson | Empirical assessment of mde in industry[END_REF]. Therefore, DSLs make part of the current practice in Systems Engineering, being used extensively and having metamodels as primary artifacts; we attest this in the next section by presenting this scenario in a large company.

DSLs in the wild: the Thales scenario

Thales2 is a large company involved in different industry sectors (aerospace, space, defence and transportation areas, etc.); they produce software intensive systems, extensively using and evolving the ARCADIA method [START_REF] Jl Voirin | Method and Tools for Constrained System Architecting[END_REF]. This method is a viewpointbased architectural description, defining five different abstraction levels (further explained as phases) of a system, following the ISO/IEC 42010, Systems and Software Engineering -Architecture Description [START_REF]International organization for standardization: Iso/iec fcd 42010: Systems and software engineering -architecture description[END_REF].

Thales' engineers use numerous domain specific modeling languages to develop integrated sets of systems according to ARCADIA. These languages are built within a set of dedicated representations to analyze domain-specific problems. The language workbench provides a set of highly dynamic (changes are often) notations working seamlessly together on top of models; in addition, they can be combined and customized according to the concept of Viewpoints. Views, dedicated to a specific Viewpoint, can adapt both their display and behavior depending on the model state and on the current concern. The same information can also be simultaneously represented through diagram, table or tree editors. We enumerate following three examples of Viewpoints.

• Performance. It is the view in charge of analysing properties such as CPU usage, bus overload and latency.

• Fault Propagation. It is the view in charge of analysing and act over system failures, handling the system exceptions.

• Spatial Arrangement. It is the view in charge of analysing the spatial constraints impacting the physical architecture.

ARCADIA also defines five main phases of Thales' systems engineering, which we briefly explain following.

• Customer Operational Need Analysis. This step focuses on analyzing the customer needs, goals, expected missions and activities; it ensures that the system definition is adequate to its real operational use. This phase also defines the IVVQ (Integration, Verification, Validation, Qualification) conditions.

• System Need Analysis. This step focuses on defining how the system can satisfy the operational need, along with its expected behavior and qualities.

• System Logical Architecture Definition. This step focuses on identifying the system parts, their contents, relationships and properties, excluding implementation or technical/technological concerns. In order to assure that these parts are stable and safe to the further steps, all major non-functional constraints, such as safety, security and performance, are taken into account to find the best trade-off among them.

• System Physical Architecture Definition. This step also focuses on defining the system architecture, but in the physical components level, making them ready to be developed in a low-level engineering.

• Definition of Components Development & IVVQ technical ContractEPBS. This step focuses on supporting the construction of the EPBS (End-Product Breakdown Structure), benefiting from the previous architectural definition, defining components requirements, and preparing a safe IVVQ (Integration, Verification, Validation, Qualification).

Each of these Viewpoints and phases contributes to the number of DSLs used in Thales. These languages are defined as a set of 20 metamodels with about 400 metaclasses and about 200 validation rules; they model the ARCADIA method in an eclipsebased environment. Besides, this workbench is extensible and new languages can be defined to design novel systems. Therefore, those numbers are constantly growing; the diversity of DSLs in Thales is a first and critical aspect of its development process.

Managing variability in DSLs

Given a DSL definition, whether it is expressed with metamodels, grammars or even code, we can know the different combinations of models/programs that can be made for that given domain. Therefore, a DSML expresses all possible models in the domain, and Context in this set, there are different valid products that fit the needs of clients. Orthogonally to this, the main goal of Model-driven/based Software Product Lines (MSPL) [CAK + 05] is to manage the commonalities and variations among the possible products, in this case, the possible models.

Managing this variability from a product line perspective allows to raise the level of abstraction (even more), gaining productivity by exploring reuse and all the other benefits that comes along with using Software Product Line Engineering (SPLE) [START_REF] Clements | Software Product Lines : Practices and Patterns[END_REF]. However, there is a price to pay: the variability of the DSL must be captured and expressed in a model -the variability model -and it must work seamlessly with all the machinery of the given domain (e.g., language workbenches, frameworks, editors, etc).

Externalizing variability and standardizing it for any DSL

From a practical point of view, such a variability model could be amalgamated to the DSL itself, having the metamodel of the variability language together with the metamodel of the DSL. On the one hand, this amalgamated approach can save effort on the integration of the variability and the domain-specific language machinery. On the other hand, it can add complexity to something that should be simple -"DSLs are often made simple and should stay simple" [HMlPO + 08]. Another drawback of an amalgamated approach is that it does not scale in real scenarios. For example, in a company that uses several DSLs and can invent and implement a new one in around two weeks 3 , the cost and redundancy of such approach undermine its adoption.

In these amalgamated approaches, variability can be modelled in existing models, such as requirement or design models. Some approaches, as those proposed by Gomaa [START_REF] Gomaa | Designing software product lines with uml 2.0: From use cases to pattern-based software architectures[END_REF], Gomaa and Shin [START_REF] Gomaa | Multiple-view meta-modeling of software product lines[END_REF], Ziadi and others [START_REF] Ziadi | Towards a uml profile for software product lines[END_REF], propose including commonality and variability in the UML models, opposed to orthogonal modeling. However, this strategy has some limitations: the mixing of variability and other modelling concepts [START_REF] Halmans | Documenting application-specific adaptations in software product line engineering[END_REF][START_REF] Pohl | Variability management in software product line engineering[END_REF] and the fact that variability is implicitly spread across the product line software artifacts.

In the opposite direction to the amalgamated approach, the separated-languages approach [HMlPO + 08] aims at having the variability modeling language as an independent part of the whole. In this way, the constructs of a variability model simply refer to the constructs of the DSL, which makes it orthogonal to the concerns of the domain. This idea gave birth to the initiative of the Common Variability Language4 (CVL): a domain-independent language to specify and resolve variability over any instance of any language defined based on MOF (Meta-Object Facility5 ). CVL results from a large consortium involving both industry and academia, being an emerging standard and continuously increasing its adoption.

A software product line defined using CVL has four main parts: a variability model, a base model (i.e., an instance of a DSL), a realization model (contains relationships between the variability and the base models) and a derivation engine (i.e., the semantics of the relationships in the realization model and the algorithm that executes them). Having separate models for each concern favors modularization and reusability; this is a step towards externalizing variability from the domain language and standardizing it for any DSL. More details on CVL can be found in Section 2.5.

Fragile -please do not break

As stated in Section 1.1.2, Thales already has an established and efficient model-based method in their systems development processes, their goal is to leverage this process from single systems to family of systems, maintaining their safety and quality standards. A well-defined and -implemented model-driven development process has good properties, such as reliability, traceability and automation. These properties are explicit and can be verified in the case of Thales approach. In their process, we can state that a composition of model elements that will be part of a given system, is effectively consistent, due to their model verification techniques.

Integrating SPL into this well-defined process may lead to some complex challenges. If we integrate the variability model with their current models, we can imagine to occur that decisions made in the variability model may be inconsistent with selected elements in the models and artifacts level. The consistency of variability models and their configurations can be checked with current variability management tools (satisfiability algorithms can be used to check whether a selection of features is valid or not [FAM]). In the same way, consistency in well-defined models, such as the ones used by Thales, can also be checked [START_REF] Voirin | Method & tools to secure and support collaborative architecting of constrained systems[END_REF]. One issue is: how to assure the consistency of a feature selection with respect to the semantics of the artifacts related to these features?

In other words, it is difficult to assure that constructing their systems in an SPL fashion (configuring a set of features, pressing a button and having an end-product) is at least as safe as constructing them in the traditional way. Reaching such a level of confidence is an open problem even in academia, so in industry, there is still a long way road. Therefore, leveraging product line engineering for each of these languages and domains is very expensive and error-prone; it has to be supported by automated tools.

Several stakeholders have to work during the design and development process of the tool chain for supporting SPLE in the Thales context:

• Product-line engineers who have to identify the commonalities and the variants and in charge of designing the VAM and the VRM.

• Product engineers who have to create specific products, focusing on creating valid products regarding a set of requirements.

• DSL designers who are in charge of creating or extending existing DSLs (base metamodel).

The ultimate goal of such a tool chain is to enable the generation of systems by configuring options in a variability model; CVL has been prototyped to leverage this, Context however there are some open issues around its generality and safety, which we discuss in the next two sections, respectively.

One language to rule them all?

Product Derivation is a key activity in Software Product Line Engineering. During this process, the core assets (i.e., the base model) are customized and selected according to a given configuration of the variability model. Derivation engines have been designed and implemented to automate Product Derivation. These engines mainly work by removing, replacing or adding elements from the set of core assets in order to derive a concrete product; in the case of an MSPL defined with CVL, we would obtain as output of the derivation an instance model of the used DSL.

Ideally, CVL derivation should be adequate in any working domain, however, in the practice of Systems Engineering, we observe some challenges on using CVL. The derivation operators of CVL can be too generic and thus ignorant from the domain knowledge; their semantics can be specialized for better fitting a domain. To illustrate this, we present following the different scenarios in which the semantics of a simple derivation operator (ObjectExistence) can vary according to different factors.

Semantics variation scenarios

The primary semantics of the ObjectExistence variation point is to determine whether a model element exists or not in the materialized model. Considering a negative derivation, this is done by checking the resolution of a feature, if it is set to yes, nothing is done, if it is set to no, the referred model element is excluded. However, we have to consider that excluding a model element may lead to secondary operations to complement the primary semantics of the variation point. This semantics complements, or secondary operations, may vary according to different scenarios.

The first scenario to be considered is that this materialization semantics can vary within the same metamodel. In Figure 1.1 (a), we have a base model that conforms to the UML Class Diagram metamodel. This model has three classes: Garage, Car and Sedan. The class Sedan is a subclass of the class Car, which represents a car that can be parked in a garage. Therefore, the class Garage represents a place that can accommodate cars. To exemplify the semantics variation of excluding a model element, we remove each class of this model and observe the possible outcome for each class.

Although we exclude the same type of element (Class) in this example, we can see that the semantics of the exclusion operation leads to different possible secondary operations in the model. A possible result from excluding the class Garage is shown in Figure 1.1(b), which is to exclude the class Garage and all its relationships. This outcome is reasonable, considering that the other classes are not depending on the class Garage to exist. This same scenario can be observed in Figure 1.1(c). After removing the class Sedan , the inheritance relationship is removed and the other classes remain in the model. On the other hand, as illustrated in Figure 1 e l e m e n t i t i s p o i n t i n g , w h e t h e r t h e y a r e i n t h e s a m e D S L o r n o t . T h i s F i g u r e g i v e s u s t h e b i g p i c t u r e t h a t o n e s i n g l e s e m a n t i c s f o r a n o p e r a t o r d o e s n o t s u i t t h e n e e d s a n d t h a t i s w h y practitioners tend to customize the CVL semantics for their own DSLs. Besides the OVP, practitioners can also introduce new derivation semantics statically inside the code of the CVL derivation engine and the second one is using the strategy pattern, but also inside the derivation semantics. We have analysed the different mechanisms used to customize the CVL semantics in 4, as well as their advantages and disadvantages.

And after all, is it safe?

The design space (also called domain engineering) of an MSPL defined with CVL is extremely complex to manage for a developer. First, the number of possible products of an MSPL is exponential to the number of features or decisions expressed in the variability model. Second, the derived product models 6 have to conform to numerous well-formedness and business rules expressed in the modeling language (e.g., UML exhibits 684 validation rules in its EMF implementation). The number of derived models can be infinite while only part of the models are safe and conforming to numerous well-formedness and business rules. Consequently, the engineer has to understand the intrinsic properties of the modeling language when designing an MSPL.

The two modeling spaces (variability and working domain) should be properly connected so that all valid combinations of features (configurations) lead to the derivation of a safe model. In CVL, as in many MSPL approaches, the realization layer is crucial and should be properly managed. Specifically, managing the design space of an MSPL raises two key issues.

First, the realization model specifies how to remove, add, substitute, modify (or a combination of these operations) model elements. Elaborating such a model is errorprone because, for example, it is easy for an SPL designer to specify instructions that delete model elements that are dependent on others (e.g., deleting a super class of a class without deleting also the class) for a given combination of features [START_REF] Czarnecki | Verifying feature-based model templates against well-formedness ocl constraints[END_REF], or perhaps to forget a constraint between features in a variability model and allow a "valid" configuration despite the derivation of an unsafe product (this is illustrated in detail in Section 2.6 of Chapter 2).

Second, the derivation engine executes the realization model and produces a product model that has to conform to the syntax and the semantics of the modeling language. Assuring the correctness of the derivation engine for a given modeling language is still a theoretical and practical problem.

Engineering MSPL in industry needs special assistance

Because of the combinatorial explosion of possible derived variants, the great variety and complexity of its models, correctly designing a Model-based Software Product Line Context has proved to be challenging. It is easy for a developer to specify an incorrect set of mappings between the features/decisions and the modeling assets, thus authorizing the derivation of unsafe product models in the MSPL. At the tooling level, CVL leads engineers to customize its semantics to better tackle their specific domains; it is hard to know whether these modifications will only produce safe products or not.

In this thesis, we want to provide assistance for both levels: the modeling level, helping designers of CVL to make safer models; and at the tooling level, assisting engineers of derivation engines to better implement custom operators. Our approach must differentiate from others in the literature and consider the challenges previously presented in this chapter.

The majority of standard Verification & Validation (V&V) techniques for SPL relies on assumptions that do not hold in our context. Our approach cannot assume:

• a single formalism in which all base models are expressed, like in model checking techniques (e.g., [CHS + 10b, CHSL11, AtBGF11]), as we must consider a multi-DSL scenario.

• existing variability, realization or even base models; the ideal would be to provide assistance even before starting to create the MSPL's artifacts, assuming only the DSL definition in a metamodel as input.

• that the derivation engine is correct. CVL derivation engine is not yet safe, assuring the derivation of a correct model is still a theoretical and practical problem; besides, engineers tend to customize CVL's derivation semantics to adequate it to a domain, as already discussed, such customizations may introduce many errors.

Synthesis

In Figure 1.4, we synthesize the specific challenges of leveraging variability management in a systems engineering context. They will guide the remainder of this thesis; our goal is to address the five challenges. The challenges are extracted from the Thales scenario, where multiple languages and dimensions are used to engineer a system. These languages are supported by model-driven workbenches, which are also important to leverage automation in the development process.

The five challenges can also be seen as research questions:

1. How can we provide early assistance for designing SPLs for new languages?

2. How can we customize the derivation support of CVL?

3. How can we provide separation of concerns in variability modeling for systems engineering?

4. How can we integrate variability management in systems engineering in a nonintrusive and seamless way?

5. How can we leverage consistent generation of products?

Chapter 2

State of the Art

In this chapter, we present the State of the Art of Variability Management and Analysis. First, we present how variability has been evolving as a core concept on software and systems development, wrapped in a trending paradigm called Software Product Lines 2.1. After, we introduce Model-driven Engineering, a strong ally to catalyse the benefits promoted by SPL (Section 2.2), also explaining the conjunct concept of Model-based Software Product Lines in Section 2.3.

Once the foundations are presented, we review the literature of variability modeling approaches (Section 2.4), assessing them according to the issues raised in the previous chapter. The literature review of variability modeling approaches supports the use of the Common Variability Language (CVL), which we then explain in Section 2.5.

We dedicate the rest of the State of the Art to the issues on realizing variability in SPL (Section 2.6) and the existing analysis techniques for this purpose (Section 2.7). Finally, we make a synthesis of the issues addressed by the approaches of the state of the art and the open challenges that we seek to address in this thesis (Section 2.8).

Software Product Lines

As in any engineering branch, constructing software is subject to constant evolution. The way we build systems has been changing over the last decades; innovative methods, tools, languages and paradigms shape the future to increasingly more efficient software design and development. A notorious evolution is concerning automation. Just like in the industrial revolution, software engineering has been progressively increasing automation in its processes, quiting handcrafting and entering industrialization.

We can think about this automation in a broader sense than just the execution of tasks by the machine. In the case of software development, automation begins with the need to reuse knowledge and code, to simply avoid unneeded repetition, which obviously promotes time and economic advantage. To achieve this reuse, once again software engineering reproduces methods from other engineering disciplines, adapting them to the software reality. It is the case of assembly lines; perfected by Henry Ford in the beginning of the twentieth century [START_REF] Ford | My Life and Work[END_REF], they were responsible for leveraging 22 State of the Art the mass production of cars.

What if we could make software and systems like we make cars? This vision leads us to think that software production can be systematized and automated -we could assemble different pre-built parts of a software product in a systematic way. If only software artifacts were as easy to handle as car parts or Lego bricks. One key and challenging difference is that software is far more customizable than standard goods. Instead of producing one single standardized product for every customer (mass production), industrialized software production is closer to mass customization, in which products are made to meet individual customer's needs [START_REF] Benavides | Automated Analysis of Feature Models 20 Years Later : A Literature Review[END_REF].

To meet this need for customization, systems have to be efficiently extended, changed or configured for use in a particular context [START_REF] Svahnberg | A taxonomy of variability realization techniques: Research articles[END_REF][START_REF] Chen | Variability management in software product lines: a systematic review[END_REF]. The challenge for practitioners is to develop and maintain multiple similar products (variants), exploiting what they have in common and managing what varies among them [START_REF] Apel | An overview of feature-oriented software development[END_REF]. Software Product Line (SPL) engineering has emerged to address the problem [START_REF] Clements | Software Product Lines : Practices and Patterns[END_REF][START_REF] Pohl | Software Product Line Engineering: Foundations, Principles and Techniques[END_REF], involving both the research community and the industry.

Domain Engineering and Application Engineering

Constructing software and systems following a product line approach requires to think about these artifacts in a broader sense: instead of dealing only with the tasks to build a single product, engineers must also take into account the family of similar products in a domain. This last concept is the fundamental idea of Domain Engineering; its main motivation is to leverage the development of parts of software for reuse in a family of applications. These parts can be seen as features of the system. Many definitions of feature have been proposed in the SPL literature, it can be roughly defined as an end-user visible functionality of the system [START_REF] Czarnecki | Generative Programming: Methods, Tools, and Applications[END_REF]. These features can be common to many products of a domain or vary among them; they are often implemented as reusable artifacts.

Complementary to the Domain Engineering, the Application Engineering of an SPL is the activity of developing products with reuse, by exploiting the reusable artifacts, composing and adapting them to the specific needs of a single product. Figure 2.1 shows an overview of the Application Engineering Activity; when this activity is automated, a derivation engine is responsible to transform the core assets into a product corresponding to a configuration.

Ideally, the functionalities in the final product can be mapped to the features of the family of applications anticipated during the Domain Engineering phase. As Domain and Application Engineering are continuous processes in an SPL, it is common to have feedback from one to the other. For example, features that appear in an individual product during its development and that were not thought before in terms of an entire domain can be promoted as reusable assets among other products. Clearly, managing all the commonalities and variabilities of a family of products is challenging; however, it can be facilitated with variability modeling (see next section 2.1.2). A f e a t u r e m o d e l r e p r e s e n t s a h i e r a r c h y o f c h a r a c t e r i s t i c s o f a s o f t w a r e / s y s t e m t h a t a r e v i s i b l e t o e n d -u s e r s ( i . e . , f e a t u r e s ) . F i g u r e 2 . 2 s h o w s a n e x a m p l e o f f e a t u r e m o d e l : e a c h b o x r e p r e s e n t s a f e a t u r e , w h i c h c a n m a n d a t o r y , o p t i o n a l , m u t u a l l y e x c l u s i v e ( X o rg r o u p ) o r a l t e r n a t i v e ( O r -g r o u p ) c h i l d r e n -t h i s i s h o w v a r i a b i l i t y i s m o d e l e d i n f e a t u r e m o d e l s . T h e y c a n a l s o h a v e c r o s s -t r e e c o n s t r a i n t s , e x p r e s s i n g t h a t o n e f e a t u r e r e q u i r e s o r e x c l u d e s a n o t h e r .

SPL

Three

Feature Models

Feature modeling is a variability modeling technique, which has generated a lot of interest in SPL engineering since their introduction by [START_REF] Kang | Feature-oriented domain analysis (foda) feasibility study[END_REF]] in the FODA method. Feature models are currently the de-facto standard for representing variability. We first describe the essential principles and semantic foundation of feature models (see Section 3.1). We then surveyed in which contexts and for which purposes feature models are used in SPL engineering (see Section 3.2).

3.1 SEMANTICS OF FEATURE MODELS Figure 3.1 gives a first visual representation of a feature model. Throughout the thesis, we will rely on the same graphical notation used in this figure, largely inspired by the one proposed in [START_REF] Czarnecki | Generative Programming: Methods, Tools, and Applications[END_REF]. Features are graphically represented as a rectangles while some graphical elements (e.g., unfilled circle) are used to describe the variability (e.g., a feature may be optional). Intuitively, the feature model depicted in Figure 3.1 compactly describes a family of medical images, where each member of the family is a medical image corresponding to an unique combination of features.

The Essence of Feature Modeling

In essence, a feature model is a hierarchy of features with variability. From a general conceptual perspective, a feature model of a concept describes a set of valid feature combinations, each representing an instance of that concept. For example, in Figure 3 FM has also been used like DM-as a central variability model and a basis for derivation (e.g., [38,48,47]). Interestingly, Gears and Pure::Variants, which are industrial SPL tools supporting FM, have been used predominantly for variability modeling and derivation support. 1 However, FM is also used as a general concept modeling technique, e.g., in comparative surveys [24]. Thus, in contrast example, the engineer needs to decide whether a particular phone will support the GSM 1900 protocol (cf. Fig. 1(b) and(c)).

The term "feature" is highly overloaded among different FM approaches [19] and also in the wider context of software engineering. FODA defines a feature as "a prominent or distinctive user-visible aspect, quality or characteristic of a software system or systems." [42] Some authors define features as requirementslevel entities, e.g., "a cohesive set of individual requirements" [16] or "a set of related requirements, domain properties, and specifications" [19]. FOSD considers a feature as "an increment in product functionality" [11], and focuses on representing features explicitly throughout the development life cycle, including requirements, design, implementation, and tests. Thus, FM has represented a wide range of system and environment properties as featuresfunctional or non-functional, and pertaining to different life cycle Mapping to Artifacts. This is an important characteristic of variability modeling techniques that can handle product derivation. In order to reflect decisions made in the variability model into the actual artifacts, we need to map both levels. Section 2.1.3 discuss the different ways of realizing variability after a configuration and a mapping to domain artifacts.

Modularity. This is a key characteristic to enable reuse and separation of concerns in variability modeling. As variability models may express an entire product line of a complex organization, it must provide means to encapsulate and abstract parts of its information [START_REF] Kästner | The road to feature modularity[END_REF].

Realization/Derivation techniques in SPL

There are different approaches to manage variation at the assets level of the SPL; they differ in the way they transform or compose the core assets to generate the product.

Annotative approaches derive concrete variants by activating or removing parts of a model or a program. Variant annotations define these parts with the help of, for example, UML stereotypes [START_REF] Ziadi | Software product line engineering with the uml: Deriving products[END_REF] or presence conditions [CHS + 10b, CP06, CA05a]. The directives of the C preprocessor (#if, #else, #elif, etc.) conditionally include parts of files and can be used to activate or deactivate a portion of code [KAK08, LAL + 10].

Compositional approaches associate reusable fragments (e.g., feature modules or model fragments) with features that are then composed for a particular configuration. For instance, Perrouin et al. offer means to automatically compose modeling assets based on a selection of desired features [START_REF] Perrouin | Reconciling automation and flexibility in product derivation[END_REF]. Superimposition is a generic composition mechanism to produce new variants, being programs (written in C, C++, C#, Haskell, Java, etc.), HTML pages, Makefiles, or UML models [START_REF] Apel | Model superimposition in software product lines[END_REF][START_REF] Thaker | Safe composition of product lines[END_REF]. Software artefacts are composed through the merging of their corresponding substructures [START_REF] Apel | Languageindependent and automated software composition: The featurehouse experience[END_REF].

Model-driven Engineering

MDE is a software development paradigm originated at the beginning of the 2000s, with its first concepts designed and presented in the Model-driven Architecture approach, an OMG standard [S + 00]. MDE advocates the use of models as the main artifacts in the software and systems development. Models help engineers to abstract from unneeded details by allowing to represent structural or behavioural problems in a simpler way [START_REF] Douglas | Model-Driven Engineering[END_REF].

In MDE, each kind of model can be specific to represent problems of a distinct domain of knowledge. Instead of having one single kind of model to serve as unified language to represent any problem, Domain Specific Modeling Languages (DSML) are dedicated to areas of expertise, such as medical or avionics systems. Their specificity can also be based on the kind of information that will be modeled, for example, to represent data flows, security, requirements or architectural components.

As with any language, DSMLs have two main components: syntax and semantics. The syntax of a DSML can be divided into the abstract and the concrete syntax. The abstract syntax of a DSML defines its concepts and the relationships between them, while the concrete syntax maps these concepts to visual elements used in models. The semantic of a DSML is the actual meaning of the syntax representations. From the two types of syntax and the semantics, the most important component of a DSML is its abstract syntax; it is common to find DSMLs without formal semantics definition or without a concrete representation, but the abstract syntax is imperative.

A metamodel is the model that defines the abstract syntax of a DSML and, therefore, it is the central artifact of a DSML definition. As the metamodel is also a model, it is also expressed in a third-level language like MOF, E-MOF, Ecore, etc; these third-level languages are bootstrapped, defining themselves their abstract syntax. Figure 2.3 shows a metamodel of a finite-state machine DSML, expressed using Ecore. This metamodel has three concepts (named EClass in Ecore): F SM, representing the finite-state machine itself; State; and T ransition. It also has the relationships between the concepts, defining compositions or associations.

The essential and most important function of the metamodel of Figure 2.3 is that it can describe what is and what is not a finite-state machine. The concepts and relationships serve to define well-formedness rules. For example, the arrow from FSM to State, labeled initialState, with the number 1 in its end, determines that a finite-state machine must have exactly one initial state; in the same way, a Transition must have exactly one source State and one target State.

Therefore, if a model M does not violate any of the well-formedness rules of its corresponding metamodel, we say that M conforms to its metamodel. In Figure 2.4, we illustrate the design space of finite-state machines, i.e., the possible ways of designing an FSM model, being conforming or not to its metamodel. The models that do not violate any well-formedness rule, expressed in the FSM metamodel, are conforming models, belonging to a correctness envelop (the subset of all valid instaces of a metamodel and that also conform to all the well-formedness rules).

Model-based Software Product Lines

I n i t i a l l y , i t i s h a r d t o i m a g i n e c o m p l e t e l y a u t o m a t e d p r o d u c t l i n e s o f c o m p u t e r p r og r a m s ; t h e fi n e -g r a i n e d c o m p l e x i t y o f t h e i r l a n g u a g e e l e m e n t s , t h e i r i n s t r u c t i o n s a n d t h e i r c o n t r o l fl o w m a k e t h e d e s i g n o f a s a f e S P L c l o s e t o i m p o s s i b l e -b u t a t l e a s t w e h a v e m o d e l s t o a b s t r a c t t h e p r o g r a m c o m p l e x i t y . T h e r e f o r e , o n e p o s s i b i l i t y t o e a s e t h i s c o m p l e x i t y i s t o a l l y S P L w i t h M o d e l -d r i v e n E n g i n e e r i n g ( M D E ) , a n d t h e n h a v i n g M o d e l -b a s e d S P L s ( M S P L s ) . M S P L s h a v e t h e s a m e c h a r a c t e r i s t i c s a n d o b j e c t i v e s o f a n S P L , e x c e p t t h a t t h e y e x t e n s i v e l y r e l y o n m o d e l s . M o d e l s , a s h i g h -l e v e l s p e c i fi c at i o n s o f s y s t e m s , a r e t r a d i t i o n a l l y e m p l o y e d t o a u t o m a t e t h e g e n e r a t i o n o f p r o d u c t s a s w e l l a s t h e i r v e r i fi c a t i o n s [ S c h 0 6 ] . T h e y a r e s i m p l e r a r t i f a c t s t h a n s t a n d a r d p r o g r a m s ; t h e i r c o n c e p t s a n d r e l a t i o n s h i p s a r e l e s s d i v e r s e a n d e a s i e r t o h a n d l e t h a n t h e o n e s i n a g e n e r a l p r o g r a m m i n g l a n g u a g e , f o r e x a m p l e . A v a r i e t y o f m o d e l s m a y b e u s e d f o r d i ffe r e n t d e v e l o p m e n t a c t i v i t i e s a n d a r t i f a c t s o f a n S P L -r a n g i n g f r o m r e q u i r e m e n t s , a r c h i t e c t u r a l m o d e l s , s o u r c e c o d e s , c e r t i fi c a t i o n s a n d t e s t s t o u s e r i n t e r f a c e s . L i k e w i s e , d i ffe r e n t s t a k e h o l d e r s c a n e x p r e s s t h e i r e x p e r t i s e t h r o u g h s p e c i fi c m o d e l i n g l a n g u a g e s a n d e n v i r o n m e n t s , a n i m p o r t a n t r e q u i r e m e n t i n l a r g e c o m p a n i e s l i k e T h a l e s [ V o i

c o r r e c t o r i n c o r r e c t . T h e i d e a l w o u l d b e t h a t a n M S P L d o e s n o t g e n e r a t e w r o n g m o d e l , h o w e v e r i t i s v e r y h a r d t o a s s u r e t h a t a n M S P L w i l l o n l y g e n e r a t e c o r r e c t p r o d u c t s ( w e d i s c u s s t h i s i s s u e i n S e c t i o n 2 . 6 ) .

Reviewing the literature of variability modeling languages

V a r i a b i l i t y m o d e l i n g i s a t r e n d i n g t o p i c o n S o f t w a r e E n g i n e e r i n g , n u m e r o u s l a n g u a g e s a

n d a p p r o a c h e s h a v e b e e n p r o p o s e d d u r i n g t h e l a s t d e c a d e s . A t t h e b e g i n n i n g o f t h e P h D , w e c o n d u c t e d a s y s t e m a t i c r e v i e w o f t h e l i t e r a t u r e i n a q u e s t f o r a s u i t a b l e l a ng u a g e / a p p r o a c h t o a d d r e s s t h e v a r i a b i l i t y m o d e l i n g c h a l l e n g e s o f T h a l e s , e x p o s e d i n t h e C o n t e x t c h a p t e r . T h e m a i n g o a l o f t h e r e v i e w i s t o s u p p o r t t h e d e c i s i o n o f w h e t h e r c o ns t r u c t i n g a n e w v a r i a b i l i t y l a n g u a g e / t o o l o r s i m p l y u s i n g a n e x i s t i n g o n e a n d s u i t i n g i t t o t h e T h a l e s ' n e e d s . A l t h o u g h T h a l e s w a s e n d o r s i n g t h e

C o m m o n V a r i a b i l i t y L a n g u a g e c o n s o r t i u m a s a s u b m i t t e r , t h e y w e r e c o n s t a n t l y i n v e s t i g a t i n g a l t e r n a t i v e a p p r o a c h e s f o r v a r i a b i l i t y m o d e l i n g .

Literature Review Process

T h e fi r s t d e fi n i t i o n t h a t s h o u l d b e t a k e n i n t o a c c o u n t w h e n c a r r y i n g o u t a l i t e r a t u r e r e v i e w i s i t s g o a l [ K C 0 7 ] . B a s i c a l l y , w e w a n t e d t o b e a b l e t o a n s w e r fi v e q u e s t i o n s . T h e fi r s t q u e s t i o n i s t h e m o s t g e n e r a l :

• Q 1 . W h a t a r e t h e e x i s t i n g V a r i a b i l i t y M o d e l i n g a p p r o a c h e s ?

Reviewing the literature of variability modeling languages

29

After answering this most general question, we will have a set of variability modeling approaches and then we can address four more specific questions:

• Q2. What kind of derivation mechanisms are supported by these approaches?

• Q3. How these approaches address the multi-level issue?

• Q4. Which and how approaches are related to the MDE paradigm?

• Q5. Which approaches provide tool support?

We motivate Q2 and Q4 with the fact that Thales already works with code and model generation, so they are always seeking to automate their development process; therefore, using a language that could support at maximum the product derivation and the MDE paradigm is essential. The motivation of Q3 is based on the complexity of the Thales development process: many stakeholders, at different levels and using different languages, need to work independently developing the product line -the approach must be able to be used in many levels with many DSLs and stakeholders. Q5 is important to identify the approaches that have already implementations, academic or commercial.

Once we set up the goals, we divide the Literature Review Process into three main steps. The first step is the definition of a search strategy. This definition is important to systematically acquire the papers related to the research question Q1.

The second step is the clustering and the selection of studies. This selection is made in order to discard irrelevant papers according to all the research questions.

The last step consists on performing a quality assessment over the selected studies, retrieving useful data and presenting it in a synthesised way. With this quality assessment we are able to answer the specific questions Q2, Q3, Q4 and Q5. These steps are presented in the next three subsections.

Search Strategy

We considered three different strategies to collect the papers in order to answer the research question Q1.

Existing Literature Reviews

The first strategy is to identify the existing literature reviews that focused on gathering variability modeling approaches; they gather important results from the SPL community and, as peer-reviewed papers, they are reliable sources of relevant articles.

Chen et al. [START_REF] Lianping Chen | Variability management in software product lines: a systematic review[END_REF] present a systematic review of variability management approaches in the software product line context. The work is a collection of general approaches selected based on two main criteria: if it introduces an approach to dealing with some aspect of Variability Modeling (we will abbreviate as VM) in SPLE or if it reports an evaluation of a VM approach. The paper also analyses the kind of issue addressed by each approach. This review was performed in the year of 2007.

After the first review, in 2011, Chen and Ali Babar [START_REF] Chen | A systematic review of evaluation of variability management approaches in software product lines[END_REF] conducted another systematic review, but now addressing how the variability management approaches in State of the Art SPLE have been evaluated, and what is the quality of the reported evaluations. Still in 2011, Chen and Ali Babar [CA11b] also investigated the contemporary industrial challenges using the approaches already captured in their previous work.

On the other hand, Benavides et al. [START_REF] Benavides | Automated Analysis of Feature Models 20 Years Later : A Literature Review[END_REF] conducted a literature review in order to identify papers regarding automated analysis of feature models 20 years after their creation. They selected and studied 53 papers in order to identify any automated operation done over feature diagrams. As a result of the study, they come up with a framework to understand the different proposals as well as categorise future contributions in the automated analysis field.

Rabiser et al. [START_REF] Rabiser | Requirements for product derivation support: Results from a systematic literature review and an expert survey[END_REF] also presented a systematic review in the SPLE area, however they focused on the product derivation aspect. The review was motivated by the lack of requirements definition for product derivation support. Therefore, they researched which approaches exist in SPLE that support product derivation, identifying the features of each approach that enables it to support product derivation.

Hubaux et al. [START_REF] Hubaux | A preliminary review on the application of feature diagrams in practice[END_REF] have initiated a systematic review in order to identify the usage of feature diagrams in practice. Their study is still a pilot for a possible full systematic review and is still biased for few conferences. Otherwise, they found 29 papers that fit into their search string and they could extract preliminary evidences that few reports about feature usage have been performed

Djebbi et al. [START_REF] Djebbi | Industry Survey of Product Lines Management Tools: Requirements, Qualities and Open Issues[END_REF] present an industry survey of product line management tool. The tools identification in their work was not done systematically. However, they found 12 tools for product line management, but only evaluated 4 tools. They used 13 requirements which they categorized in three different kinds: Product Line Engineering criteria, Management criteria and Technical criteria. They analyse if the requirements were matched, assigning a mark to each tool with respect to the given requirement.

Eichelberger et al. [START_REF] Eichelberger | A systematic analysis of textual variability modeling languages[END_REF] performed a systematic analysis of textual variability modeling languages. They chose ten different languages and evaluate them according to their configurable elements, constraint support, configuration support, scalability support and other additional characteristics.

Berger et al. [BRN + 13] investigated the actual use of variability modeling techniques in the industrial practice. They concluded that industry uses different notations and tools, which emphasizes the heterogeneity of the SPL industrial practice. Their survey is important in order to identify the approaches that are sufficiently mature to be prototyped or even consolidated in real scenarios.

All the papers before mentioned contributed to our knowledge of variability modeling and analysis approaches. They cover a great amount of the state of the art and can serve as basis for more sophisticated reviews, as the one we are performing in this chapter by adding new research questions adjusted to our needs.

Selection Criteria and Methods

Articles of interest for this literature review have been published in software engineering conferences. Available and properly referenced technical reports about variability modelling techniques were also valid candidates. We extracted the papers from electronic data sources including the ACM Digital Library, the IEEE Xplore Digital Library, the RefDoc Service, the HAL open access archive, the IEEE Computer Society Digital Library, the CiteSeerX Digital Library, the IBM Technical Journals, and the book series of Lectures Notes in Computer Science available on SpringerLink.

These data sources were crawled using web-based services to retrieve the references of the articles to include in the selection. We use mainly the Google Scholar service but also to some extent the Researcher, the SciVerse ScienceDirect, the Oxford Journals, the ArnetMiner and CiteULike websites, and several authors or research team projects homepages. Because we reused results from the existing literature reviews exposed in section 2.4.2.1, we avoided making the same search and rather focus on papers published after 2008 (Chen et al. [START_REF] Chen | A systematic review of evaluation of variability management approaches in software product lines[END_REF] provide a large set of approaches in their systematic review, but all of them were published before 2008). We perform our selection of articles using the following string:

(variability (model* OR management) OR feature model* OR software product line OR product derivation) AND (year >2008)

The articles that are selected for this review are only those which discuss variability modeling or management. Relevant articles are identified first by their title and second by reading the abstract. If the articles do not discuss variability modelling techniques, the article is not included in the selection. Once a paper has been selected for review, we proceed the read to capture valuable information. The adequacy of articles, regarding the selection criteria, is then checked.

Study Selection

In this subsection, we present the application of the process to select the studies for being part of the Data Extraction step.

Among the articles that match the research criteria, some of them do not propose any variability modelling technique or approach, but deal instead with industrial requirements for variability modelling. The articles that were not proposing any variability modelling technique or approach explicitly were removed from the set of selected articles for this study. We also excluded papers that do not propose new approaches for modeling variability and that focus only on the analysis of SPLE -for this specific purpose, Section 2.7 presents the different techniques and approaches to analyse SPLs.

Considering the captured papers, we list bellow the tools studied during this work and their respective URLs:

• CaptainFeature, http://sourceforge.net/projects/captainfeature/

Data Extraction

We have conducted a study over the selected papers in order to extract data to answer the questions Q2, Q3,Q4 and Q5. Initially, we classified if the given approach answers or not the corresponding research question, considering also when the approach just partially handles the given issue. Table 2.4.4 shows the result for each question. For Q2, we can refer to [START_REF] Djebbi | Industry Survey of Product Lines Management Tools: Requirements, Qualities and Open Issues[END_REF][START_REF] Troyer | Feature Modeling Tools: Evaluation and Lessons Learned[END_REF] for a more detailed investigation of tools for SPLs. Similar to [START_REF] Troyer | Feature Modeling Tools: Evaluation and Lessons Learned[END_REF], we can point as possible answers if the given approach has tooling support used in academia (A), industry (I) or in both (B).

For Q3, we are interested in how the MDE assets are related to the VM approach. Each model asset conforms or not to a metamodel with its own semantics and its own set of constraints. The support of a consistent derivation semantics depends on the model-driven assets semantics. This is an important challenge in order to adequate the SPL in a model-based development process. We mark Y if the approach has means to relate to modeling assets and N if it does not.

For Q4, possible answers can be regarding to what are the product derivation activities supported by the VM approach. Product derivation activities are defined in [START_REF] Rabiser | Key activities for product derivation in software product lines[END_REF], and can be divided into a preparation activity (A), a configuration activity (B) and an additional development and testing activity (C). Besides following these derivation activities, managing variability modelling in a model-based context also leads to integrate an extension mechanism to refine the derivation semantics, depending on the asset's metamodel.

For Q5, we consider as premise to deal with multi-dimensions the fact that the approach is able (M) or not (S) to manage multiple variability models, or even, how the modularization of these models is tackled. This question shows that it is important to

Remarks on the Data Extraction

We can conclude after planning and applying the review process that it is not a trivial task. Most of its complexity is due to the fact that variability modeling is still a recent research area. Therefore, by the time the review was applied, standardization was still missing to define variability techniques contributions and what they are able to express or compute. It is a hard task to identify if an approach can manage, for example, the product derivation, since it is a complex process and encompasses several steps. Another issue is how to categorize if an approach follows or not the model-driven paradigm. The multi-leveling characteristic is also not trivial to be recognized, mainly due to the fact that one could claim that several dimensions could be expressed in only one feature diagram. Thus, from this preliminary discussion towards the suitability of existing approaches, the main challenge is probably not to define a new VM language, but to extend or facilitate the adoption of one. Existing VM approaches target each challenge: (i) provide a language to capture commonalities and variation points (Q1), (ii) provide tool support for these languages and include a support to automatically derive a concrete product (Q2,Q4), (iii) use these languages at different phases of the software development process (Q5), and support model-driven assets (Q3). We highlight that CVL matches all the criteria of our research questions, which strongly encouraged its adoption. We present CVL and its main concepts in Section 2.5.

The Common Variability Language

In this section, we present the main concepts of CVL and introduce some formal definitions that are useful for the remainder of this thesis. CVL is a domain-independent language for specifying and resolving variability over any instance of any MOF1 -compliant metamodel. The current revised submission document can be accessed at CVL's webpage: http://www.omgwiki.org/variability

The overall principle of CVL is close to many MSPL approaches: (i) A variability model formally represents features/decisions and their constraints, and provides a highlevel description of the SPL (domain space);

(ii) a mapping with a set of models is established and describes how to change or combine the models to realize specific features (solution space);

(iii) resolutions of the chosen features triggers modifications in the base models to derive the final product model.

Figure 2.5 presents the overall modeling structure of an MSPL defined using CVL: a variability abstraction model (VAM), expressing the variability units (VSpecs) and their relationships; a variability realization model (VRM), containing the mapping relations between the VAM and the artifacts; the resolutions (i.e, configurations) for the VAM; and the base models conforming to a DSL.

• Variability Abstraction Model (V AM ) expresses the variability in terms of a tree-based structure. Inspired by feature and decision modeling approaches [CGR + 12], the main concepts of the V AM are the variability specifications, called VSpecs.

The VSpecs are nodes of the V AM and can be divided into three kinds (Choices, Variables, or Classifiers). In the remainder of the thesis, we will focus on the Choices VSpecs, making the V AM structure as close as possible to a Boolean feature model-the variant of feature models among the simplest and most popular in use [START_REF] Benavides | Automated Analysis of Feature Models 20 Years Later : A Literature Review[END_REF] -as current implementations of attributed feature models are still being investigated to assist on the design of CVL models. Another reason is because our work focus on the derivation process of the SPL, rather than its configuration process. These Choices can be decided to yes or no (through ChoiceResolution) during the configuration process.

• Base Models (BM s) a set of models, each conforming to a domain-specific modeling language (e.g., UML). The conformance of a model to a modeling language

The Common Variability Language Using CVL, the decision of a Choice will typically specify whether a condition of a model element, or a set of model elements, will change after the derivation process or not. In this way, these choices must be linked to the model elements, and the links must explicitly express what changes are going to be performed. The aforementioned links compose the V RM, determining what will be executed by the derivation engine. Therefore, these links contain their own meaning. We consider that these links can express three different types of semantics:

• Existence. It is the kind of VP in charge of expressing whether an object (Ob-jectExistence variation point) or a link (LinkExistence variation point) exists or not in the derived model.

• Substitution. This kind of VP expresses a substitution of a model object by another (ObjectSubstitution variation point) or of a fragment of the model by another (FragmentSubstitution)

• Value Assignment. This type of VP expresses that a given value is assigned to a given slot in a base model element (SlotAssignment V P ) or a given link is assigned to an object (LinkAssignment V P ).

Using the models provided by CVL, one can completely express the variability over any MOF-compliant BM . In addition, it is possible to derive a family of models that will compose an MSPL. Therefore, it is possible to properly define an MSPL in terms of CVL (see Definition 1).

Definition 1 (Model-based SPL) An MSPL = hCVL, BMS, i is defined as follows:

• A CV L = hV AM, V RM i model is a couple such that:
-V AM is a tree-based structure of VSpecs. We denote C V AM the set of possible valid configurations for V AM ;

-V RM is a model containing the set of mapping relationships between the V AM and the BM S2 ;

• BM S = {BM 1 , BM 2 , . . . , BM n } is a set of models, each conforming to a modeling language;

• : VAM ; w e p r e s e n t s o m e a n a l y s i s t e c h n i q u e s a n d t h e i r c h a r a c t e r i s t i c s i n S e c t i o n 2 . 7 .

CV L ⇥ c ⇥ BM S !
!"# !$# !%# &'()*!#+,-.!)/*)"# 0-/1#+,-.!)/*)"# 0-/1#+,-.!)/*)%# 0-/1#+,-.!)/*)$#

g o i n g t r a n s i t i o n m u s t h a v e a t l e a s t o n e t a r g e t s t a t e , w h i c h d o e s n o t h o l d f o r t r a n s i t i o n t 1 . I n t h e c a s e o f Conf iguration 3, t h e d e r i v e d p r o d u c t m o d e l h a s t h e i n c o m i n g t r a n s i t i o n t 3 w i t h o u t a s o u r c e s t a t e , w h i c h a l s o i s i n c o r r e c t w i t h r e s p e c t t o t h e

State of the Art

!"# !$# % &# '# "# "# '# '((" # "# "# "# "# '# '# '((" # '# "# !)# % &# "# "# '# "# '((" # "# "# !"# !)# % *# &# Derivation Derivation Derivation Configuration 1# Configuration 2 Configuration 3

Analysis of Software Product Lines

A u t o m a t e d p r o d u c t d e r i v a t i o n i s t h e H o l y G r a i l o f S P L , h a v i n g t h e m e a n s t o a u t om a t i c a l l y g e n e r a t e a p r o d u c t a f t e r a s e l e c t i o n o f f e a t u r e s c a n s a v e t i m e a n d m i n i m i z e c o s t s f o r t h e c o m p a n y . H o w e v e r , a c h i e v i n g s u c h a l e v e l o f a u t o m a t i o n a n d m o s t i m p o rt a n t l y , i n a r e l i a b l e w a y , i s s t i l l a b i g c h a l l e n g e . W e d i s c u s s i n t h i s s e c t i o n t h e d i ffe r e n t a p p r o a c h e s t o a n a l y s e S P L s , s p e c i fi c a l l y t h o s e r e l a t e d t o t h e a n a l y s i s o f i s s u e s i n t h e v a r i a b i l i t y r e a l i z a t i o n ( a n a l y s i s o n l y a t t h e l e v e l o f v a r i a b i l i t y m o d e l s a r e n o t c o n s i de r e d ) . W e r e l y o n r e c e n t e x t e n s i v e c l a s s i fi c a t i o n a n d s u r v e y e ffo r t s a b o u t p r o d u c t l i n e a n a l y s i s s t r a t e g i e s [ T A K

+ 1 4 a , T A K + 1 2 , M T S + 1 4 ] .

Classification of SPL analysis approaches

S P L E i s e v o l v i n g a n d , a s t h e n u m b e r o f a n a l y s i s a p p r o a c h e s i n c r e a s e s , m e t a s t u d i e s c o m e u p t o s t r e n g t h e n t h e fi e l d . I n [ T A K + 1 4 a ] , T h ü m a n d o t h e r s p r o p o s e a c l a s s i fic a t i o n t o t h e s e a n a l y s i s ; c a t e g o r i z i n g t h e m a l l o w s t o b e t t e r k n o w t h e i r c a p a b i l i t i e s a n d w e a k n e s s e s , t h u s f a c i l i t a t i n g t h e r e t r i e v a l a n d u s e o f a p p r o a c h e s f o r a s p e c i fi c n e e d ( e . g . , c h o o s i n g a m o d e l c h e c k i n g a p p r o a c h t o a n a l y s e a s m a l l S P L b u t t h a t n e e d s a h i g h d e g r e e o f c o n fi d e n c e ) .

T h e a p p r o a c h e s a r e d r i v e n b y e x i s t i n g t e c h n i q u e s t o a n a l y s e s o f t w a r e , which we enumerate in the following; they are incorporated for the reality of multiple products and variability management.

1. Type Checking. This method of analysis consists on verifying whether a program is well-or ill-typed with respect to a type system [START_REF] Benjamin | Types and programming languages[END_REF], i.e., a formal specification defining rules that contained systems must follow. Type checking is the foundation of the concept of conformance in model-driven engineering; a model is well-formed if its syntactic structure does not violate any rule expressed in its metamodel [START_REF] Atkinson | Model-driven development: a metamodeling foundation[END_REF] (i.e., its type system).

Advantage: Can be fully automated and scalable.

Limitation: Rules in type systems are constrained by their decidability/checkability.

2. Static Analysis. This type of analyses works at compile-time by extracting semantic information and approximating the behaviours of a program [START_REF] Landi | Undecidability of static analysis[END_REF].

An example of application of static analysis is to check live-variables (i.e., whether a variable is accessed during the execution of its statement or not).

Advantage: Can be fully automated and often does not need user input.

Limitation: Some analysis techniques can be undecidable or uncomputable and have to work with approximation.

3. Model Checking. It is an analysis technique that verifies if a formal model, which represents a system, satisfies its specification [START_REF] Edmund M Clarke | Model checking[END_REF]. Model checkers rely on specific languages to make their computation; these languages are usually based on finite state machines and offer a precise abstraction of the systems behaviour.

Advantage: Can be exhaustive and find errors that other analysis techniques cannot, because of its precise representation of the system.

Limitation: Due to the state space explosion, model checking does not easily scale for large systems with many interacting parts. Another limitation is that the system must be encoded in a formal language, as well as its specification.

4. Theorem Proving. An automated theorem prover is a program that, given a logic formula, evaluates whether it is universally valid or not, according to an automatic deduction with the application of inference rules [START_REF] Schumann | Automated theorem proving in software engineering[END_REF].

Advantage: Has a high precision and often generalizes over technologies.

Limitation: It also requires the system to be encoded in a formal specification and does not scale for large programs.

In SPLE, besides the before mentioned techniques, an important aspect of the analysis approaches is in which part of the product line they focus. The most common types of analysis are at the following levels:

State of the Art 1. Product-based analysis. In this kind of analysis, the products of the SPL are generated and then analysed one by one.

Advantage: An advantage of this technique is that it is easier to apply any software analysis technique to the products, as they are individual programs.

Limitation: The main issue with this kind of analysis is the fact that the number of possible products in an SPL is exponential. Thus, product-based analysis can either rely on optimizations to reach a reasonable coverage of the problem space or just be exhaustive.

2.

Family-based analysis. It operates only over domain artifacts and it considers the knowledge about valid combinations of features, which can be expressed in the variability model.

Advantage: It is not necessary to generate and analyse all the products of the SPL; its complexity is generally reduced to a satisfiability problem.

Limitation: The standard techniques for software analyses must be adapted to take into account the knowledge about variability.

3.

Feature-based analysis. This type of analysis takes into account the features in an isolated way, without considering their valid combinations, as done in the family-based; it also operates only on domain artifacts.

Advantage: As in the family-based, it is not necessary to generate and analyse all the products of the SPL Limitation: The main limitation of feature-based analysis is their primary assumption that features can be analysed in a modular way; it is still hard to imagine a legacy and complex system being divided in composable and completely independent parts.

These approaches can be combined among them; Thüm and others [TAK + 14a] explain in details the possible combinations and the advantages and disadvantages of each combined approach. The combinations are: Feature-Product-based Type Checking, Feature-Product-based Model Checking, Feature-Product-based Theorem Proving; Feature-Family-based Type Checking, Feature-Family-based Theorem Proving; Family-Product-based and Feature-Family-Product-based.

We can conclude that there are many ways of increasing or just assess the safety of an SPL design; all of them have advantages and disadvantages, their suitability depends on the domain of the SPL and the desired properties for the checking mechanisms.

Synthesis

In Figure 2.8, we present the issues raised in the context chapter (see Section 1.6) and we relate them to techniques/features of existing variability modeling and analysis approaches that try to address the issue. The top boxes represent characteristics from

State of the Art

Seamless non-intrusive integration

As in separation of concerns, orthogonality of variability models is an efficient nonintrusive technique, opposed to amalgamated approaches [START_REF] Roos | Automated analysis of orthogonal variability models. a first step[END_REF]. From the analysis perspective, product-based approaches can be easily integrated to existing product lines, as they can work directly in the notation/language of the product itself. Methodological and organizational challenges are important and current subjects of investigation in order to facilitate variability management in a seamless and non-intrusive way. Unfortunately, reengineering is still the most common practice to move from single systems to SPL [START_REF] Miguel | A systematic mapping study on software product line evolution: From legacy system reengineering to product line refactoring[END_REF].

Consistent generation of artifacts

This is one of the most challenging and also studied issue in the SPLE community; many approaches explore automated product derivation and program analysis techniques to ensure consistent generation of models or programs. In the case of modelbased approaches, type checking techniques are used to constrain the set of possible valid products and try to ensure well-formedness of derived models [START_REF] Buchmann | Ensuring well-formedness of configured domain models in model-driven product lines based on negative variability[END_REF][START_REF] Zhang | Towards correct product derivation in model-driven product lines[END_REF]. Some techniques specifically address the problem of verifying SPL or MSPL[ARW + 13]. The objective is usually to guarantee the safe composition of an SPL, that is, all products of an SPL should be "safe" (syntactically or semantically). In [START_REF] Thaker | Safe composition of product lines[END_REF], Batory et al. proposed reasoning techniques to guarantee that all programs in an SPL are type safe: i.e., absent of references to undefined elements (such as classes, methods, and variables). At the modeling level, Czarnecki et al. presented an automated verification procedure for ensuring that no ill-structured template instance (i.e., a derived model) will be generated from a correct configuration [START_REF] Czarnecki | Verifying feature-based model templates against well-formedness ocl constraints[END_REF]. In [CHS + 10b, CHSL11], the authors developed efficient model checking techniques to exhaustively verify a family of transition systems against temporal properties. Asirelli et al. proposed a framework for formally reasoning about modal transition systems with variability [START_REF] Asirelli | Formal description of variability in product families[END_REF].

In

Early support for new DSLs

We consider early support for new DSLs as the capacity of providing ways to analyse the domain even before the domain engineering phase -before domain engineering, there is no variability model, configurations or even base models. The motivation is when one wants to build an SPL using only a language syntax definition as input. To the best of our knowledge, all approaches cited in 2.8 and in [TAK + 14b] consider existing variability models, configurations or base models, therefore we could not find approaches providing early support for new DSLs.

Specialized derivation support

In a context with several languages and stakeholders, we cannot assume that derivation operators will have the same operational semantics in every situation [START_REF] Bosco | Customizing the common variability language semantics for your domain models[END_REF]. Some variability modeling approaches propose extension points to add customized semantics or extend the current one of the product derivation. For example, CVL has opaque variation points, which work as black boxes components that can be plugged to the language. In delta-oriented approaches, one can create custom derivation deltas to specific languages [START_REF] Schaefer | Delta-oriented programming of software product lines[END_REF][START_REF] Clarke | Abstract delta modeling[END_REF]. Yet, the majority of product line approaches assume a unique/unchanged semantics in their derivation engines.

Conclusions

We presented the foundations of this thesis and the state of the art of variability modeling and analysis. From the numerous approach, we chose to present CVL as our working variability language, justified by a comparison to the other approaches of the literature review and its standardization effort. We have shown the critical issue of realizing variability in a model-based context and the numerous approaches that try to analyse SPL and address this challenge.

Observing the state of the art, we could assess if and how our context challenges were addressed. Our conclusions are:

1. There are no approaches providing support for MSPL engineering that do not consider domain engineering artifacts or existing products as input. Therefore, early support for leveraging SPL in new DSLs is absent.

2. Few approaches provide means to customize their derivation support mechanisms, the existing ones use extension points.

3. Many approaches address separation of concerns with modularization and orthogonality mechanisms.

4. Product-based analysis and orthogonal modeling are non-intrusive techniques to integrate variability modeling in existing systems development life cycle. Several approaches use both techniques and we can consider that one can analyse products without interfering in their development process; however they lack efficiency.

5. Many approaches progressed on the consistent generation of artifacts by considering valid combination of features, composition algebras or conformance to wellformedness rules. These advances are important, but assuring that all generated products will be valid is still a theoretical and practical problem.

Part II

Contributions

Overview of the Contributions

In this part of the thesis, we present our contributions to the state of the art of engineering model-based software product lines for the systems engineering domain. Our contributions are driven by the challenges raised at the end of the context chapter (see Section 1.6) and by the gaps left by the current approaches in the state of the art in relation to these challenges (see Section 2.8).

To illustrate this, Figure 2.9 shows the five challenges in the middle surrounded by our contributions, using the arrows to point to the challenge that is being tackled. We placed CVL at the top because it plays a major role in our work. CVL is not our contribution, however, as INRIA is a submitter of CVL to OMG's RFP, we participated on the consortium meetings, helping on the definition of some concepts of the language. Besides, we implemented our version of CVL tooling in INRIA, the kCVL 4 .

In Chapter 3, we present -the core contribution of this thesis -our generative and automated approach to produce counterexamples of MSPLs (see 1 in Figure 2.9). The work in this chapter has as main motivation to provide a novel approach to identify erroneous designs of MSPLs, that, despite of having correct variability models, can still derive wrong products with valid configurations. This approach needs only a metamodel as mandatory input, contributing to the early support for new DSLs. Because it uses the existing V&V mechanisms of systems engineering (e.g., conformance checking of models), the approach is less intrusive than the family-based or feature-based ones.

In Chapter 4, we show the different mechanisms for customizing the semantics of CVL's realization model and derivation engine, promoting specialized derivation support (see 2 in Figure 2.9). Adjusting CVL's semantics is already a practice in Thales; engineers want to get rid of tedious operations, like always having to delete dangling references to a no longer existing model element; and they also want to enhance the reliability of their derivation engines -by incorporating secondary operations that will be executed automatically, complementary to the original semantics of a CVL variation point. We present this practice in a more structured way, showing three possibilities for performing the customizations; they can be used according to some non-functional requirement, like reusability or checkability.

In Chapter 5, we present an empirical study on the application of the derivation operators of CVL in real Java programs, assessing their capacity of generating correct programs (see 3 in Figure 2.9). This chapter helps to know the adequacy of the CVL's

Generating Counterxamples of MSPL

A one-size-fits-all support for designing MSPLs is unlikely, since models have to conform to their own well-formedness rules and business rules. Each time a new modeling language is used for developing an MSPL, the realization layer should be revised accordingly. We observed this kind of situation in the context of prototyping the use of CVL with Thales.

Without adequate support, a developer of an MSPL is likely to introduce errors. The tooling support can provide different facilities: antipatterns (counterexamples) to document what should be avoided during the design of an MSPL; domain-specific rules to avoid earlier the specification of incorrect mappings; examples to show possible correct MSPL, etc. Moreover, the support offered to domain experts should be ideally specific to a domain metamodel. Methodological support and guidelines are also needed to identify what constructs of a metamodel are likely to vary; to define an accurate realization model; or to develop specific derivation engines for a given modeling language.

In this Chapter, we provide a way to generate counterexamples of MSPLs, which are examples of MSPLs that authorize the derivation of syntactically or semantically invalid product models despite of a valid configuration in the variability model. These counterexamples aim at revealing errors or risks -either in the derivation engine or in the realization model -to stakeholders of MSPLs. On the one hand, counterexamples serve as testing "oracles" for increasing the robustness of checking mechanisms for the MSPL. Developers can use counterexamples to foresee boundary values and types of MSPLs that are likely to allow incorrect derivations. On the other hand, stakeholders may repeat the same kind of errors when specifying the mappings between a variability model and a base model. Counterexamples act as "antipatterns" that should avoid bad practices or decrease the amount of errors for a given modeling language.

We provide a systematic and automated process, based on CVL, to randomly search the space of MSPLs for a specific formalism (see Section 3.1); this process is implemented in a tool named LineGen (see Section 3.2). In Section 3.3, we validate the effectiveness of this process for three formalisms (UML, Ecore and a simple finite state machine)

Generating Counterxamples of MSPL

with different scales (up to 247 metaclasses and 684 rules) and different ways of expressing validation rules. We also explore the hypothesis exposed above, i.e., that a generic derivation engine or a basic support for managing the realization layer is likely to authorize incorrect MSPLs. In Section 3.5, we extend the evaluation to our industrial case in Thales.

We discuss how counterexamples can guide practitioners when customizing derivation engines, when implementing checking rules that prevent early incorrect CVL models, or simply when specifying an MSPL (see Section 3.4). Overall, we conclude in Section 3.6 that the generative techniques and exploratory study help to construct solutions aware of the semantics of the targeted modeling languages when developing MSPLs.

The contributions of this chapter are published in [FBA + 13] and in [FBA + 14].

Our Approach

Our approach seeks to reveal MSPL designs that can generate invalid products even after a satisfiable set of Boolean choices (i.e., counterexamples, see Definition 2), as explained in Section 2.6. Our approach can help at least two kinds of users:

• designers of MSPLs in charge of specifying the VAM, the BMs, as well as the relationships between the VAM and the BMs (V RM)(see CV L of Definition 1);

• developers of derivation engines in charge of automating the synthesis of model products based on a selection of features (Choices) (function of Definition 1);

Incorrect derivation engines or realization models may authorize the building of unsafe products. The majority of the existing work target scenarios in which an existing MSPL has been designed and seeks to first check its consistency, then to generate unsafe product models -pointing out errors in the MSPL. These techniques are extremely useful but assume that a generic derivation engine exists and is correct for the targeted modeling language -which is hardly conceivable in our case. Moreover, designers of MSPLs are likely to perform typical errors for a given modeling language (e.g., FSM).

Counterexamples to the Rescue

Specifically, we are interested on finding MSPLs that apparently would derive models that respect the domain modeling language, as they have a correct variability model and a conforming base model, but however, either their VRM or their derivation engine were incorrectly designed. We precisely want to support the two kinds of users before mentioned in their activities, by exploring the design space of their DSLs.

The expected benefits are as follows:

• SPL designers in charge of writing CVL models, can better understand the kinds of errors that should be avoided (Figure 3.1 gives an "antipattern").

Our Approach

51

• developers of derivation engines can exploit counterexamples as testing oracles, anticipating the kinds of inputs that should be properly handled by their implementation. Furthermore, they can enrich the derivation engine with domain specific validation rules (customizing their operational semantics with one of the mechanisms described in Chapter 4). In addition, specific error reports can be generated when an MSPL is incorrect, inspired by the catalogue of counterexamples.

In our approach, we will randomly explore MSPL designs that are possible to be a counterexample. For doing this, we first define the concept; definition 2 formalizes this kind of MSPL as counterexamples, while in Figure 3.1, we show an example of a counterexample resulting in a wrong finite state machine when derivation is executed. Having this definition set, we show in the next sections how we proceeded to explore the design space of an MSPL of a given DSL.

Definition 2 (Counterexample of MSPL) A counterexample CE is an MSP L in which:

• CVL is well-formed;

• There exists at least one valid configuration in VAM: C V AM 6 = ;;

• The set of BM conforms to its modeling language.

• 9 c 2 C V AM , (CV L, c , BM ) = DM 0 such that DM 0 does not conform to its modeling language.

Overview of the Generation

In order to systematically generate counterexamples of MSPLs, we have defined a set of activities that can be performed for this purpose. Figure 3.2 presents an overview of the process that generates a single counterexample, as well as the input and output for the different phases. We have divided the process into four phases, which are explained in details in the following subsections; the second and the third phases are part of the greater activity of generating a CVL model.

1. The first phase is the set up of the input that will be taken into account; different activities can be performed, depending on the input.

2. The second phase is the generation of a random variability model and of a valid random configuration.

3. The third phase is the generation of the relationships between the VAM and the base model elements, i.e., the variability model (VRM).

4. The fourth and last phase is to identify whether the generated model is a counterexample or not. In case it is not, we go back to the second step.

Generating Counterxamples of MSPL

Set up input

G e n e r a l l y , c o m p a n i e s t h a t u s e o r d e c i d e t o s e t u p a p r o d u c t l i n e a l r e a d y h a v e a n i n i t i a l s e t o f c o r e a s s e t s . I n t h e c a s e o f M S P L s , i f t h e m o d e l s a r e n o t a v a i l a b l e , i t i s c o m m o n t o h a v e t h e m e t a m o d e l a n d t h e w e l l -f o r m e d n e s s r u l e s o f t h e m o d e l i n g l a n g u a g e . C o ns i d e r i n g t h i s , t h e m e t a m o d e l a n d t h e r u l e s o f t h e d o m a i n -s p e c i fi c m o d e l i n g l a n g u a g e a r e a s t a r t i n g p o i n t t o g e n e r a t e a C V L m o d e l . O u r a p p r o a c h i s a d a p t a b l e t o w o r k w i t h b o t h c a s e s , w h e t h e r t h e m o d e l s a r e a v a i l a b l e o r o n l y t h e i r m e t a m o d e l . I n t h e c a s e t h e y a r e n o t a v a i l a b l e , w e a p p l y r a n d o m i z a t i o n s o v e r t h e m e t a m o d e l t o c r e a t e r a n d o m m o d e l s . T h e s e r a n d o m i n s t a n c e s p o p u l a t e t h e B a s e M o d e l , a n d t h e i r c o r r e c t n e s s i s c h e c k e d a g a i n s t t h e m e t a m o d e l a n d t h e w e l l -f o r m e d n e s s r u l e s . I f a c r e a t e d m o d e l i s n o t c o r r e c t , t h i s i n s t a n c e i s d i s c a r d e d . I n t h e c a s e o f t h e F S M m o d e l i n g l a n g u a g e , t h e c h e c k e d w e l l -f o r m e d n e s s r u l e s a r e : i f t h e i n i t i a l s t a t e i s d i ffe r e n t o f t h e fi n a l , i f t h e F S M i s d e t e r m i n i s t i c a n d i f a l l t h e s t a t e s a r e r e a c h a b l e . O n t h e o t h e

Generate VAM and Resolution

F o r g e n e r a t i n g t h e V AM a n d t h e V RM , t h e f o l l o w i n g p a r a m e t e r s a r e r e q u i r e d :

• Generating Counterxamples of MSPL also rely on existing frameworks like FaMa [START_REF] Benavides | Automated Analysis of Feature Models 20 Years Later : A Literature Review[END_REF]). The kinds of VAM we consider in this thesis are amenable to boolean feature models supported by FAMILIAR. Using FAMILIAR, we check whether the variability model is valid or invalid. If it is an invalid model, we discard it and return to the V AM generation step. A resolution model is necessary in order to resolve the variability expressed in the V AM . To generate the configuration, we create the corresponding resolution CV L element for each V Spec. Meanwhile, random values (true or false) are set for each ChoiceResolution that has been created. We use standard satisfiability techniques to randomly generate a resolution, which is, by construction, a valid configuration of the V AM .

O n c e t h e B M i s e s t a b l i s h e d a n d t h e p a r a m e t e r s h a v e b e e n s e t , w e t a k e t h e m a s i n p u t t o s t a r t t h e g e n e r a t i o n o f t h e C V L m o d e l . F i r s t , i f t h e V AM i s n o t p r o v i d e d b y t h e u s e r , w e g e n e r a t e i t , c r e a t i n g a r o o t V Spec a n d i t s c h i l d r e n . T h e n u m b e r o f c h i l d r e n i s d e c i d e d r a n d o m l y , r a n g i n g f r o m 0 t o M A X _ C H I L D R E N . T h e V Spec c r e a t i o n i s r e p e a t e d f o r e a c h g e n e r a t e d c h i l d u n t i l t h e ( M A X _ D E P T H ) i s r e a c h e d o r t h e r e a r e n o m o r e V Specs w i t h c h i l d r e n . T h e o n l y i m p o s e d g e n e r a t i o n i s o f t h e r o o t n o d e o f t h e t r e e , a f t e r , i t i s a r a n d o m d e c i s i o n b e t w e e n c r e a t i n g ( o r n o t ) e a c h c h i l d . A f t e r g e n e r a t i n g t h e V AM , i t i s n e c e s s a r y t o c h e c k i t s c o r r e c t n e s s , a s w e a r e n o t i n t e r e s t e d i n w r o n g V AM s . F o r t h i s r e a s o n , w e t r a n s l a t e t h e V A M t o a l a n g u a g e t h a t c a n p r o v i d e u s a b a c k g r o u n d f o r a n a l y s i n g i t . T h e F A M I L I

Generate VRM

Once we have a correct V AM and a correct BM , we can generate the V RM to link each other. To do this, we iterate over the set of choices in the V AM , deciding if the given choice is pointed or not by a Variation Point. This decision is done based on the (LINK_PERCENT) parameter. If the decision is true, we create the V P in the V RM. The type of the V P is also random. To finish the creation of the VP, we also randomize its target over the set of model elements of the BM . Naturally, we restrict the set of the randomization with respect to the kind of V P , e.g., a LinkExistence has a random target randomized over the subset of BM references. The VRM generation can also be independent, from existing VAMs and BMs, one could then explore the possibilities of relationships between them.

Detect Counterexample

Although Figure 3.2 describes the process of generating one single counterexample, we iterate the process to produce a set of counterexamples. For this reason, the first parameter to be taken into account is the stopping criteria. The stopping criteria can be specified in two different ways. The first one defines a target number of counterexamples, making the process repeat until this number is reached. The second one is to set an amount of time, stopping the process after it has elapsed.

After the aforementioned steps have been performed, we have a correct CVL model, composed by a correct V AM and a V RM created in conformance to the CVL metamodel. We also have a valid configuration c and a correct set of models composing the BM . The next step is to derive a product model using the CVL, c and the BM . If the derived model is incorrect, in other words, having (CV L, c , BM ) incorrect, we have found a counterexample as states the Definition 2, and consequently, we add it to the oracle. If the model is correct then we discard it and we come back to the generate VAM phase, synthesising a new entire CVL model.

The derivation engine is an algorithm that visits each of the variation points in the CVL model, executing them according to the resolution of the variability model. Our implementation of the CVL derivation engine follows the operational semantics of each variation point defined in the CVL specification (for further details, see the Annex A of the CVL revised submission provided in http://www.omgwiki.org/variability).

Tool Support

55

To check if the derived model is correct, we relied on the EMF Diagnostician, using it as a black box to validate the conformance of the generated instance of the given metamodel.

As we will discuss in Section 4, these counterexamples can be helpful to the domain experts in charge of designing the CVL model or developing their derivation engines for their domain.

Tool Support

To support the process of generating counterexamples of MSPLs (exposed in the previous section), we developed a dedicated tool, called LineGen. Figure 3.3 gives an overview of the main features of LineGen. Depending on the inputs, the tool addresses different scenarios of counterexamples' generation -from the whole exploration of a modeling space (in the case only a metamodel is given) to the design of a specific MSPL (the variability model and the base model can be given by the user).

Specifically, the only mandatory input for LineGen is the metamodel of the base language. Additionally, the user can choose to provide existing base model and variability model; if this is the case, LineGen will not modify these models, setting them as immutable during the generation. To generate an MSPL example or counterexample, LineGen synthesizes a variability model, a configuration, a base model, and a set of realization relationships. LineGen calls the EMF's Diagnostician and checks the conformance of the base model with its input metamodel. After, LineGen checks the correctness of the variability model and the satisfiability of the configuration; to do so, it uses the reasoning engine within the FAMILIAR language. If they pass, LineGen carries on generating the realization relationships, finishing the CVL model.

After everything is generated, LineGen calls the CVL derivation engine, giving as input the generated CVL model (the triplet: variability model, realization model and base model) and a configuration. The goal of the call to the derivation engine is to determine whether the derived model is conforming to its modeling language. If it is, the CVL model given as input to the derivation engine is considered as an example of MSPL; otherwise it is considered as a counterexample.

We used different technologies as part of the LineGen implementation. As the user interface is an Eclipse 4 RCP application, it is written in Java. The core algorithms of the model generation parts are written in Scala. We used the EMF API to manipulate and check the Ecore metamodels and model instances. To benefit from automatic analysis of the variability model, we translated the VAM to the FAMILIAR language [FAM].

Figure 3.3 shows the graphical user interface of LineGen. The user must load the Ecore metamodel of the modeling language to be able to perform the generation steps (see 1 ). Once the metamodel has been successfully loaded-the Console (see 6 ) shows whether LineGen successfully completed an operation or not-it is possible to generate a base model by pressing the Generate BM button (see 2 ); a file named BaseModel is created with the chosen extension. The Max Many field should be set to limit the number of instances of a given model element. search. The Console tab also provides exception messages, in case an unexpected error occurs. More details about LineGen can be found online: https://code.google.com/ p/linegen/wiki/LineGen.

Generating Counterxamples of MSPL

F i g u r e 3 . 3 : L i n e G e n u s e r i n t e r f a c e T h e s a m e p r o c e s s a p p l i e s t o t h e V A M g e n e r a t i o n ( s e e 3 i n F i g u r e 3 . 3 ) . T h e u s e r s p e c i fi e s t h e m a x i m u m d e p t h o f t h e V A M , a s w e l l a s t h e m a x i m u m n u m b e r o f c h i l d r e n p e r f e a t u r e . A f t e r p r e s s i n g t h e G e n e r a t e V A M b u t t o n , L i n e G e n c r e a t e s a C V L m o d e l w i t h j u s t t h e V A M p a r t d e fi n e d . I n t h e V R M t a b , t h e u s e r c a n d e fi n e t h e p e r c e n t a g e o f f e a t u r e s l i n k e d t o a v a r i a t i o n p o i n t i n t h e V R M ( s e e 4 ) . I n t h e C o u n t e r e x a m p l e s a n d E x a m p l e s t a b s ( s e e 5 ) , t h e u s e r c a n s t a r t t h e g e ne r a t i o n p r o c e s s t h a t w i l l r a n d o m l y s e a r c h f o r e x a m p l e s o r c o u n t e r e x a m p l e s o f M S P L s . I f t h e u s e r c h o o s e s t o u s e t h e L o a d E x i s t i n g B a s e M o d e l o r L o a d E x i s t i n g V A M t a b s , L i n e G e n u s e s t h e l o a d e d m o d e l s w i t h o u t m o d i f y i n g t h e m a n d j u s t g e n e r a t e s

Evaluation

The goal of this evaluation is to verify the applicability and effectiveness of the proposed approach, as well as to assess important properties of the generated counterexamples. Regarding the effectiveness, we formulated the following question:

• RQ1. Can the approach generate counterexamples in a reasonable amount of time?

Then we seek to answer questions about the properties of the generated counterexamples, such as:

• RQ2. Does the number of counterexamples increase in a more complex domain?

• RQ3. With respect to the metamodel or the OCL rules, what errors are the most common in the counterexamples?

• RQ4. Is it possible to prevent the generation of counterexamples by the designer?

RQ1. Applicability and Effectiveness

Answering this question will allow us to know if the approach can actually generate counterexamples and how long it takes to generate a range of counterexamples. Objects of Study. To answer RQ1, we need to apply the proposed approach to specific scenarios and verify if it effectively produces counterexamples. As a first scenario, we use the FSM modeling language that was presented in previous sections. As second and more complex scenario, we use the Ecore modeling language. We provide the corresponding metamodel and validation rules as input for both scenarios. As previously mentioned, the FSM metamodel has 3 classes and 4 rules, while the Ecore metamodel has 20 metaclasses, 33 datatypes and 91 validation rules. We set up the parameters equally for both scenarios: the stopping criteria is set to the number of 100 counterexamples, the MAX_DEPTH is set to 5, the MAX_CHILDREN is set to 10 and the LINK_PERCENT is set to 30%.

Experimental Setup. Once the parameters and the input are ready, we start the automatic generation of the counterexamples. The generation was performed in a machine with a 2nd Generation Intel Core I7 processor -Extreme Edition and 16GB of 1333MHz RAM memory, running under a linux 64bit with a 3.8.0 kernel, Scala 2.9.3 and an oracle Java Runtime Environment 7.

Experimental Results. The times are shown in Figure 3.4, ranging from 0 to 12625 seconds. For both FSM and Ecore, we could successfully find and generate counterexamples in a reasonable time. The time for generating 10 counterexamples for the Ecore-based MSPL was approximately 15 minutes, which is acceptable, considering For generating valid UML model, we do not create UML models from scratch, but we mutate existing UML models. We chose the footnote referred set of UML models to create the BM 1 .

Generating Counterxamples of MSPL

RQ2. Counterexamples vs Domain Complexity

T h i s r e s e a r c h q u e s t i o n a i m s a t a n a l y s i n g t h e c o n s e q u e n c e s o f a p p l y i n g t h e a p p r o a c h i n a m o r e c o m p l e x d o m a i n . A n s w e r i n g t h i s q u e s t i o n h e l p s w h e t h e r a n d t o w h i c h e x t e n t i t i

Experimental Results. The experiment resulted in the generation of 469 correct DM s for 100 counterexamples for FSM, 292 correct DM s for 100 counterexamples for Ecore and 52 correct DM s for 100 counter examples for UML. We can therefore verify the ratio of incorrect per correct derived models. In the case of FSM, the ratio is 1 incorrect DM to 5 correct DMs, while in the case of Ecore, this ratio is 1 to 3, and for UML the ratio is 1 to 0,5. These results provide evidence that, as the domain modeling language becomes more complex, the chance to get a correct DM becomes lower. In a sense, it confirms the relevance of our procedure for generating counterexamples. More importantly, the practical consequence is that the designer is likely to produce much more unsafe MSPLs when the targeted modeling language is complex.

RQ3. Nature of the errors

The purpose here is to evaluate whether the errors are a violation to the structural properties of the metamodel or to the validation rules (i.e., OCL rules). Answering this question can help to understand which part of the modeling language is more likely to reveal more errors. Hence, we conducted the following experiment to investigate the research question.

Objects of Study. To identify the nature of the errors in the counterexamples, we used the generation of the 100 counterexamples for the three modeling languages that were previously used to answer RQ2. Our object of study is the quantity of counterexamples with errors violating the metamodel or the OCL rules.

Experimental Setup. For each modeling language, we applied our approach to obtain 100 counterexamples under the same parameters, and then we identify in which part of the modeling language definition is the error of the DM. The evaluation was performed using the same computer of the previous experiment.

Experimental Results. For the FSM language, among the 100 counterexamples, we generate 10 models that do not conform to the metamodel and 90 models that violate one of the validation rules. For the Ecore modeling language, among the 100 counter examples, we generate 64 models that do not conform to the metamodel and we generate 36 models that violate one of the validation rules. For the UML modeling language, among the 100 counter examples, we generate 22 models that do not conform to the metamodel and we generate 78 models that violate one of the validation rules.

We now correlate these numbers with the properties of the modeling language. FSM contains only three structural rules (i.e., a state-machine must contain at least one state, one initial state and at least one final state). Most of the errors are the validation rules that are violated. Ecore contains much more structural rules (mainly lower case constraints for cardinality). Therefore lots of errors come from structural inconsistencies. Finally UML contains so many validation rules that it is unfeasible to create a valid

1 http://goo.gl/kC0sx
Generating Counterxamples of MSPL UML model randomly. (That is why we used mutation from a set of valid UML models.) For this case we obtained much more DM s that violate validation rules expressed in OCL.

Yet, it is hard to draw definitive conclusions on whether structural or validation rules expressed in OCL participate the most in generating incorrect MSPLs. The results indicate that the kind of errors that are the most common in the counterexamples depend mainly on the domain modeling language (Ecore vs UML). It is well known, for instance, that some OCL rules can be refactored as structural constraints in the metamodel. In a sense, it partly confirms -in the context of CVL -some of the results exposed in [START_REF] José | Searching the boundaries of a modeling space to test metamodels[END_REF] showing there exists different "styles" of expressing business or domain-specific rules within a metamodel.

RQ4. Antipattern detection

The purpose of RQ4 is to evaluate the feasibility of expressing validation rules on the triplet V AM , BM , V RM to decrease the risk of creating invalid DM s from a valid CV L model and a correct BM , being C the set of possible valid configurations for a valid V AM . This question helps to know if it is possible for a domain designer to detect early "bad" CVL models (acting as "antipatterns") for a given domain.

Objects of Study. To evaluate this research question, we created two validation rules to detect antipattern for the FSM modeling language. Rule 1 prevents a substitution between a final state and an initial state, and vice versa. Rule 2 constrains the fact of having an object existence that targets the initial state of an FSM. These rules have been implemented in Scala and can be written in few lines using an OCL writing style, as shown in Listing 3.1. Experimental Setup. For the FSM modeling language, we applied our approach to obtain 100 counterexamples and we compare the number of valid DM s we obtain either checking the antipatterns rules or not. The evaluation was performed on the same computer that the previous experiment, as well as with the same parameters.

Experimental Results. The experimental results show that we generate 1860 correct DM s for 100 counterexample for FSM when the antipattern rules for CVL are activated, against 469 correct DM s for 100 counter examples for FSM when the CVL validation rules for CVL are not activated. For this domain, writing only 2 rules on the triplet of V AM , V RM, BM allowed us to decrease 4 times the risk of generating an invalid DM . Therefore, it is feasible to detect identified antipatterns using our approach, writing validation rules that detect a priori and therefore earlier these errors.

Discussion

Besides the checking operations, the time results presented in Figure 3.4 are mainly dependent on the following factors:

1. The time to generate a correct set of models to compose the BM; 2. The time to generate a correct VAM;

The time to generate a VRM;

These three factors are resulting from the generality and the full automation of our approach that does not require any input models. The approach gives the ability of finding possible design errors without having yet designed the MSPL. This allows users to explore the design space of an MSPL, given a modeling language -this is the main scenario we initially target. However, it is possible to predefine some inputs. It could enhance the scalability of our generative process, since there is no need to spend time in generating these inputs. It may be the case when a designer of an MSPL already has an established BM. Another possible situation is when the VAM has been previously designed, as it is often one of the starting points of an MSPL. Therefore, we can claim that the conducted experiment address the worst case input for our approach. Consequently, our approach is sufficiently generic, as it does not assume that it is always the case of having a VAM or the BM as input. In addition, because it is fully automated, the approach does not demand a great effort to be used. Another benefit of predefining some inputs is that we could address other scenarios, like the debugging of an existing MSPL or the definition of various realization models given predefined BMs and VAMs.

By definition, an MSPL is a complex structure, composed by different connected models. This characteristic makes hard to design a correct MSPL, as errors can occur in any design phase. Given this great proneness to error, it is relevant to discuss the causes and to reason where is the lack of safety. For this purpose, we can analyse and give a rationale about two questions:

1. How a VAM and its analysis tools check and prevent configurations that result in incorrect DMs?

2. Is the fact of a derivation operator generate an incorrect DM fault of the own derivation operator (derivation engine) or is it fault of how it was invoked (realization model)?

Generating Counterxamples of MSPL

Regarding the first question, it seems unfeasible to have a generic checker that, for any domain, could detect whether a configuration derives or not an incorrect model. It is rather needed to customize a derivation engine and/or a consistency checker (e.g., a simulator [START_REF] Zhang | Towards correct product derivation in model-driven product lines[END_REF]) that takes into account the syntactic and semantic rules of the domain. Likewise, faulty configurations, currently not supported by the MSPL, could be better identified and located. From this aspect, counterexamples can help to devise such specific simulators and oracles. For the second question, we can argue that there is a trade-off between the expressiveness of the realization model and the safeness of the derivation. On the one hand, if more restrictions are applied to the derivation engine, we limit what could be generated. Also, a realization design can be wrong in one domain, but correct in another. On the other hand, if the derivation engine is not customized to address the specific meanings of a modeling language, then it is necessary to have checking mechanisms for the VRM that takes into account the syntax and semantics of the domain. More practical investigations are needed to determine when to customize the derivation engine or when to develop specific checking rules for the VRM. Counterexamples can be used for implementing both solutions.

Approach in an Industrial Case

In the last session, we evaluated our approach against well-known modelling languages; we could verify that it produces counterexamples in a reasonable time and we could also assess properties of the counterexamples. In this section, we present how the approach performs facing an industrial case. First, we describe the company's scenario; second, we report on how we could successfully apply the approach on it; and finally, we reproduce the applicability and effectiveness experiments done in the RQ1 of the evaluation.

Thales Scenario

Thales is a large company involved with different industry sectors (aerospace, space, defence and transportation areas, etc.); they produce software intensive systems, using model-based technologies, and they seek to evolve towards a product line approach. Thales already has a well-established and functional model-based method for developing their systems and software, the ARCADIA, however they seek to leverage this development from single software to families of software, maintaining their safety and quality standard [START_REF] Bosco Ferreira Filho | Leveraging variability modeling for multi-dimensional model-driven software product lines[END_REF].

The ARCADIA method is a viewpoint-based architectural description, defining 5 different abstraction levels of a system, following the ISO/IEC 42010, Systems and Software Engineering -Architecture Description [START_REF]International organization for standardization: Iso/iec fcd 42010: Systems and software engineering -architecture description[END_REF]. Thales' engineers use numerous domain specific modeling languages to develop integrated sets of systems according to ARCADIA. These languages are built within a set of dedicated representations to analyze specific problems. The language workbench provides a set of customizable and highly dynamic representations working seamlessly together on top of models. These representations can be combined and customized according to the concept of Viewpoints.

Approach in an Industrial Case

63

Views, dedicated to a specific Viewpoint, can adapt both their display and behavior depending on the model state and on the current concern. The same information can also be simultaneously represented through diagram, table or tree editors.

These languages are defined as a set of 20 metamodels with about 400 metaclasses and about 200 validation rules; they model the ARCADIA method in an eclipse-based environment. Besides, this workbench is extensible and new languages can be defined to design specific viewpoints of a system. Therefore, leveraging product line engineering for each of these languages and domains is very expensive and error-prone; it has to be supported by automated tools.

Several stakeholders have to work during the design process on the tool chain:

• Product-line engineers who have to identify the commonalities and the variants and in charge of designing the VAM and the VRM.

• Product engineers who have to create specific products, focusing on creating valid products regarding a set of requirements.

• DSL designers who are in charge of creating or extending existing DSLs (base metamodel). They define where and how we can put variability within (at the M2 level) the architecture and the derivation semantics [START_REF] Bosco | Customizing the common variability language semantics for your domain models[END_REF] The use of the proposed counter example framework aims at easing the correct cooperation between these stakeholders. It is used to provide a pragmatic approach to guide these stakeholders to design CVL model that provides only valid products.

Approach Application and Results

We applied the approach to the Thales' representative sample model of weather balloons; this base model has 2079 model elements and 563Kb and, despite of being one single subdomain, it can serve as a pilot application for other similar areas of the organization. The set of metamodels and validation rules of ARCADIA are considered as input to the approach. In contrast of what we did to evaluate the approach in a generic way (generating everything else besides the metamodel), we could simplify the generation because Thales provided a variability model and the aforementioned preliminary base model, narrowing down the problem space. Therefore, we fixed the VAM and the BM, randomizing only over the configurations of their variability model and generating the set of variation points to compose the realization model. However, it was necessary to adapt the implementation to meet some technical requirements from Thales for loading and saving the models.

Reproducing the same experimental setup of RQ1, we performed 10 rounds and measured the average time for generating 100 counterexamples. The results in seconds are shown in Figure 3.5. We could verify that in a situation where the VAM and the BM are provided, it is around 27 times faster to generate the same amount of counterexamples, and the curve still behaves linearly. plained in details each step of our generative approach and illustrated it with a running example. The tool LineGen, built on top of CVL and modeling technologies, supports the generative process. It enables practitioners to explore the whole design space of a given modeling language but also to focus on a specific MSPL with a pre-defined variability and base models. We performed experiments to assess the applicability and effectiveness of the tool-supported approach. The conducted experiments allowed us to evaluate the approach when applied to different modeling languages, at different scales of complexity. We could successfully generate counterexamples for each modeling language in a reasonable amount of time, which could be drastically reduced when the approach received additional input. In addition, we explored the natures of errors found in the counterexamples and our ability to detect antipatterns. We also reported on our experience when instantiating the approach and LineGen in an industrial context.

!"# $$%# $&'# $!(# $($# )$$# ))%# )&)# )!*# )%$# +# "+# $++# $"+# )++# )"+# *++# $+# )+# *+# &+# "+# !+# '+# %+# (+# $

Conclusions

Chapter 4

Customization of Derivation Semantics

Recalling the motivations of Section 1.3.1 for varying the operational semantics of CVL, we have that: the semantics can vary within different metamodels (e.g., it is different to exclude a UML Class and a BPMN activity, however both are model elements that can be pointed by an Object Existence); within the same metamodel and with the same model elements (e.g., excluding a Singleton Class is different of excluding a Parent Class, in terms of secondary operations it may lead).

In this chapter, we first introduce the two possibilities to exploit the generated counterexamples (see Section 4.1). We then expose the mechanisms to implement and extend the semantics of CVL's variation points (VP) (see Section 4.2), showing how this semantics can be customized in practice according to a domain or to different model elements. In Section 4.3, we summarize the mechanisms presented and give a brief comparison. Some of the contributions presented in this chapter are published in [START_REF] Bosco | Customizing the common variability language semantics for your domain models[END_REF].

What to do after generating counterexamples?

After generating counterexamples for a given DSL, the MSPL infrastructure engineer (see Roles in Section 6.1) can decide to use the knowledge acquired to: design checking rules to detect errors and develop repair actions in an MSPL design; or to change the derivation semantics and avoid these errors when executing the derivation.

In Figure 4.1, we illustrate the following situation. After having successfully generated counterexamples for finite state machines, we observe a type of counterexample that is frequently produced: the case of excluding a state and letting dangling references that used to point to it. This counterexample is due to an object existence, referring to a state with incoming and outgoing transitions, that had its operational semantics executed during the derivation algorithm.

As shown in Figure 4.1, there are two options to handle this situation. The first one would be at design time, including a checking rule that would detect if an object 6 8

Customization of Derivation Semantics

O n e a d v a n t a g e o f t h i s o p t i o n i s t h a t t h e d e s i g n e r o f t h e M S P L c a n k n o w t h a t h i s d e s i g n i s f a u l t y a n d t h e n c o r r e c t i t . A d i s a d v a n t a g e i s i f t h e e r r o r i s v e r y f r e q u e n t , h e / s h e w i l l h a v e t o s p e n d a l o t o f t i m e c o r r e c t i n g t h e d e s i g n , i n c l u d i n g a d d i t i o n a l v a r i a t i o n p o i n t s . T h e s e c o n d o p t i o n i s t o c u s t o m i z e t h e s e m a n t i c s o f t h e o b j e c t e x i s t e n c e v a r i a t i o n p o i n t .

T h e n e w s e m a n t i c s w o u l d a u t o m a t i c a l l y m a k e t h e s e c o n d a r y o p e r a t i o n s a f t e r r e m o v i n g a s t a t e , c l e a n i n g a l l t h e d a n g l i n g r e f e r e n c e s . T h e d i s a d v a n t a g e i s t h a t t h i s c a n o n l y b e d o n e i f t h e e r r o r i s a p a t t e r n i n t h e d o m a i n -i n t h i s c a s e i t i s t r u e , a t r a n s i t i o n o f a fi n i t e s t a t e m a c h i n e m u s t a l w a y s h a v e a t l e a s t o n e s o u r c e a n d o n e t a r g e t s t a t e -otherwise, the semantics would be adequate to one case and broken for others.

The main advantage of customizing the derivation semantics is that engineers could get rid of tedious operations, such as with dangling transitions; it also increases the safety of the MSPL, in this case, assuring that it would never generate finite state machines with dangling transitions. In this chapter, we choose to further explore this idea of customizing the semantics of the CVL's variation point

Approaches to customize CVL's derivation semantics

An engineer that wants to specialize the CVL's derivation semantics can do it in different ways; we present three in this section, as illustrated in Figure 4.2. The first approach is the static introduction of semantics. It consists on directly redefining new semantics into the derivation engine. The meaning of the variation point is changed to cope with other specific need. It is the simplest way to customize the semantics, as the engineer re-implement the code of the CVL derivation engine, developing a new behaviour.

The second approach is the opaque customization. CVL proposes a set of VPs with a well-defined semantics and keeps one type as an extension point to implement its own semantics: the Opaque Variation Point (OVP). The OVP is a black box that can define an arbitrary behaviour to execute during derivation; they can be unknown algorithms from third parties and just be invoked by the derivation mechanism. The use of OVPs can be seen as a mechanism to propose a particular semantics for the derivation engine.

The third approach is the extensible customization. As the name says, this approach consists on extending the original semantics of the derivation engine, but without changing the original one; the new semantics is added as an explicit and focused extension. An additional logic is included in the derivation engine, in order to either switch between the extensions or just choosing which should be executed according to the nature of the elements to which the VP points.

Semantics in CVL

Before presenting how the customization approaches can be implemented in practice, we show the basic concepts of how the original semantics of CVL was defined and developed. We consider CVL to be a modeling language as any other. Therefore, next subsection shows what is the practice of implementing the semantics of a modeling language.

Weaving semantics into a modeling language

Metamodeling is the current practice when defining a modeling language; it consists on defining the structure of a language, using concepts and relationships. This activity is supported by metalanguages like MOF, EMOF [START_REF] Omg Mof | Omg meta object facility (mof) specification v1[END_REF] and Ecore [START_REF] Steinberg | EMF: eclipse modeling framework[END_REF]. However, in many cases, it is not sufficient to define only the language structures; the

Customization of Derivation Semantics

!" !" !" !"##"! $" ##! %"& " !'(')*"*+!',-).('),/"
!" Semantics of one VP ,!" New opaque semantics Original derivation engine 01'0/!)230" *+!',-).('),/" !"

!"# , 4 ( 5 + 0 "* + ! ' , -) . ( ' ) , / " !" !"

,!" ,!" F i g u r e 4 . 2 : T h r e e a p p r o a c h e s t o c u s t o m i z e t h e s e m a n t i c s o f t h e d e r i v a t i o n e n g i n e . b e h a v i o u r i s a l s o a m a j o r c o n c e r n i n m a n y l a n g u a g e s . I t c a n b e s e e n a s t h e a c t u a l m e a n i n g a n d t h e a c t i o n s o f t h e l a n g u a g e ; i t i s w h a t m a k e s a l a n g u a g e e x e c u t a b l e .

O n e c o u l d i m p l e m e n t t h i s b e h a v i o u r i n a g e n e r a l p u r p o s e i m p e r a t i v e l a n g u a g e , l i k e J a v a , o r i n a d e c l a r a t

i v e o n e l i k e O C L . H o w e v e r , t h e y a r e n o t t h e b e s t fi t f o r t h i s p r a c t i c e , f o r e x a m p l e , J a v a d o e s n o t c o n t e m p l a t e s o m e o f t h e c o n c e p t s e x i s t i n g i n M O F , s u c h a s a s s o c i a t i o n s , e n u m e r a t i o n s , o p p o s i t e p r o p e r t i e s , m u l t i p l i c i t i e s , d e r i v e d p r o p e r t i e s , e t c . ) . I n o u r t e a m

, w e u s e K e r m e t a [ M F J 0 5 ] t o h a n d l e t h i s c h a l l e n g e o f i m p l e m e n ti n g a b e h a v i o u r f o r a m o d e l i n g l a n g u a g e . K e r m e t a i s a l a n g u a g e w o r k b e n c h m a d e f o r s p e c i f y i n g a n d d e s i g n i n g d o m a i n s p e c i fi c l a n g u a g e s ( D S L ) . F o r t h i s , i t i n v o l v e s d i ffe r e n t l a n g u a g e s , d e p e n d i n g o n t h e c o n c e r n : a b s t r a c t s y n t a x ( i . e . , m e t a m o d e l1 ) , s t a t i c s e m a nt i c s a n d b e h a v i o u r a l / o p e r a t i o n a l s e m a n t i c s .

T h e w o r k b e n c h i n t e g r a t e s t h e O M G d e f a c t o s t a n d a r d s E M O F a n d O C L , r e s p e c t i v e l y f o r s p e c i f y i n g t h e a b s t r a c t s y n t a x a n d t h e s t a t i c s e m a n t i c s ; i t a l s o p r o v i d e s t h e K e r m e t a L a n g u a g e t o a d d r e s s t h e s p e c i fi c a t i o n o f t h e o p e r a t i o n a l s e m a n t i c s a n d t o i n t e g r a t e a n a c t i o n l a n g u a g e i n E M O F . T h e w o r kb e n c h c o m p o s e s t h e s e d i ffe r e n t c o n c e r n s i n t o a s t a n d a l o n e e x e c u t i o n e n g i n e ( i n t e r p r e t e r

or compiler) of the DSL.

In Kermeta, all pieces of static and behavioral semantics are encapsulated in metamodel classes. The aspect keyword enables DSL engineers to relate the language concerns (abstract syntax, static semantics, behavioral semantics) together. It allows DSL engineers to reopen a previously created class to add some new pieces of information such as new methods, new properties or new constraints. It is inspired from openclasses [CL00] -the keyword require enables the composition. A DSL implementation requires an abstract syntax, a static semantics and a behavioral semantics. The require mechanism also provides some flexibility with respect to static and behavioral semantics. For example, several behavioral semantics could be defined in different modules (all on top of the same metamodel) and then chosen depending on particular needs (e.g., simulation, compilation).

Weaving Semantics into CVL

We translated CVL's metamodel to the .ecore notation; it is the base metamodel in which we will introduce operational semantics. Consequently, the CVL metamodel is required as an input and can be easily invoked in Kermeta using the require keyword, (e.g., require CVLMetamodel.ecore).

Once the CVL metamodel is loaded, it is possible to weave the operational semantics into any model element in the metamodel. The Listing 4.1 shows how we can simply attach operational semantics into the CVL abstract syntax (metamodel). The code that implements the operational semantics of a model element is placed in an aspect class block, named according to the model element name (line 1). In the case of the variation points, we define an abstract operation to evaluate the operational semantics of the variation point (line 2). This operation is abstract because the actual implementation is in the concrete class that inherits from the Variation Point class (e.g., Object Existence). Each concrete variation point has an eval method, which overrides the abstract operation and must contain the operational semantics to be executed, following an implementation of the Interpreter Design pattern [START_REF] Gamma | Design Patterns: Elements of Reusable Object-Oriented Software[END_REF]. An execution context (CVLEx-ecutionContext ) is provided as a parameter of the eval method. This context stores relevant information to the execution of the materialization engine, such as the set of selected/unselected V Specs and the set of model elements in the resolved model. The operational semantics in Kermeta of all concrete variation point is inside the CVL submission document2 , in the Annex A: More on semantics. To be concise, we will adopt the Object Existence variation point as working example.

The Listing 4.2 presents the operational semantics of the ObjectExistence variation point. First, in line 3, it is verified whether the binding V Spec (i.e., similar to feature, see Section 2.5) of the current variation point is selected or not. If the V Spec is not selected for the current materialization, we need to remove the corresponding element in the base model. In line 4, we navigate to the binding object of the current variation point (self) to provide the optional element in the base model that is inside the collection ctx.domainResource to the method remove. This operational semantics is executed whenever the derivation engine runs to produce a product after a configuration. In a negative derivation algorithm, the Object Existence is linked to existing model elements, and if the binding choice is set to false, the operational semantics is executed, in this case, the object is removed from the base model.

Static customization

By using the built-in require composition mechanism of Kermeta, it is possible to statically customize the semantics of a CVL variation point. Indeed, require provides a mechanism to weave aspect in an existing metamodel. Therefore, the DSL engineer can reopen a previously created metaclass to add new pieces of information such as new methods, new properties or new constraints. It also allows engineers to easily replace the behaviour of an existing method. The method (eval, see Listing 4.1), which is introduced in all the variation points (ObjectSubstitution, ObjectExistence, . . . ) can be changed by requiring a new Kermeta file. This modification is static, modifying the types and requiring to recompile all the CVL's Kermeta implementation.

This extension mechanism has two main drawbacks. First, Kermeta does not allow the new implementation to call the previous aspect implementation, contrarily to the situation in which we can call the code of an operation contained in the super-class with the keyword super. Secondly, using this mechanism, the DSL engineer can change (and potentially break) completely the CVL implementation. Kermeta does not provide any checker to ensure that a new implementation is a refinement of the previous implementation. The main advantages is the fact that the extension is modular and can be statically plugged or unplugged. As an example, we present in Listing 4.3, an excerpt of a customization of the CVL Object Existence, which, besides removing the model element (line 4), also fixes dangling references (line 5).

Extensible customization

The basic semantics used in the default CVL implementation is the following. Each variation point can modify the model to change relationships between model elements and can introduce new model elements. To remove a model element, each variation point acts on a context that contains a list toRemove of the model elements that must be removed. Removing elements of a base model is performed at the end of the derivation to avoid side-effects among variation points.

With this behaviour, CVL combines positive variability and negative variability. The default semantics for the remove implementation is the following. Each element contained (containment relationship) by an element that must be removed is also removed. All the references of an element that must be removed are set to null. All the elements, that reference an object that must be removed through a reference with a violation of the lower cardinality are also removed, e.g. if a : A references b : B and A is associated exactly with one B, if b is removed, a is also removed. This can already be seen as a semantic customization as it has an additional load of operations that must be performed for a given variation point.

We can introduce in the default semantics a strategy pattern [START_REF] Gamma | Design Patterns: Elements of Reusable Object-Oriented Software[END_REF] to provide the ability of dynamically specializing the default semantics. The idea is that a domain expert can define a new CVL semantic extension and can register it. During the derivation, when a model element has to be removed, all the registered extensions are called to determine the list of model elements to be removed (as depicted in Figure 4.3). To implement a new metamodel extension, the DSL expert has to create an object that respects the following interface (see Listing 4.4). This extension mechanism provides several benefits. First, it ensures that the default semantics of the CVL variation point is respected. Indeed, domain engineers can only refine the semantics in removing elements, and not directly in the variation point. It can be compared to the idea of post directives in Kompose [START_REF] Fleurey | A generic approach for automatic model composition[END_REF]. Second, new strategies can be registered or unregistered dynamically. Finally, each specialization can be modularized in a distinct building block.

Opaque customization

The last way to customize the CVL derivation semantics is the use of Opaque Variation Points (OVP). An OVP is the Variation Point in which the behavior is defined by using an expression defined in an action language. We currently propose an implementation that supports OVP definition in Groovy3 , in Javascript or in Kermeta. With these action languages, designers can modify the base model directly. Each of this Variation point can access to a context that contains the list of Objects to remove (toRemove), the list of objectHandles associated to this Variation Point (ctx), the list of variable and their associated value defined in the the resolution model (args), and a map of key value that can be used to propagate value between the execution of variation points map (see Figure 4.4). An example of OVP is defined in Listing 4.6 as an expression attribute of the OVP, it adds all the UML properties that references a base model element that must be removed.

an existing OVP. The first mechanism is built-in within Kermeta but seems to be dangerous for the case of CVL because experts should be perfectly aware of the previous implementation to change it without introducing side-effects.

Conclusion

As CVL is a generic language for handling variability in any domain, there is often a need for specializing its semantics. We have shown in this chapter three different ways of making this specialization, comparing them and showing examples. We believe that these customizations are the key for building derivation engines better fitted to the specific domains. The customizations can be assisted by the counterexamples approach presented in Chapter 3, using its random explorations of a particular domain to find antipatterns of MSPLs. The customizations can help engineers to ease the task of defining a realization model, by encapsulating tedious operations or reassuring the correction of the transformations.

Chapter 5

Experimenting CVL variation points with Java program constructs

Each time a domain specific modeling language or a programming language is used for developing an SPL, practitioners (e.g., domain experts or software developers) need to understand the language constructs subject to variation and the means to realize a variant. In the case of Java, numerous constructs are subject to transformation: we can add a parameter into a constructor, remove a statement, substitute a field, etc. Some transformations lead to errors; some others not.

In particular, not all constructs of a language are subject to variation because of the numerous well-formed and domain-specific rules. For instance, removing a return statement in a non-void Java method is not possible; adding a try block without a catch block either; replacing the type of a parameter by another unrelated type is unlikely to produce a correct variant, etc.

Our industrial experience with Thales confirmed the relevance of the problem and its practical difficulty, as explained in previous chapters. Each time a new modeling language is used for developing a variability-intensive system, domain experts need to understand the language constructs subject to variation and the means to realize a variant.

Though generic foundations and tools (independently of any particular technology) for describing variations are emerging [START_REF] Erwig | The choice calculus: A representation for software variation[END_REF][START_REF] Apel | Languageindependent and automated software composition: The featurehouse experience[END_REF], the specificity of the language's syntactic structure and semantics has to be considered at some points. In practice, each time software artefacts are concerned with variation, the what and how (i.e., the removal, adding, or replacement of a program or model element) of the conforming languages should be considered carefully. The problem impacts both users of languages (e.g., Java) and developers of tools (e.g., integrated development environment). A traditional approach is to hand-craft a solution by relying on domain knowledge and empirical observations.

In this chapter, we empirically answer the question: which transformations (i.e.,

Experimenting CVL variation points with Java program constructs derivation/realization operators, or variation points) can synthesize variants of Java programs that are incorrect, correct and perhaps even conforming to test suites? We adopt an approach with no assumptions about the targeted language that relies on full extensive automations for exploring a variation space. We implement source code transformations, based on the Common Variability Language, that add, remove, substitute any kind of element of a Java program. We automatically synthesize 376,185 program variants based on source code elements in a set of 8 real large Java projects (up to 85000 lines of code). We obtain a comprehensive panorama of the sanity of the transformations based on statistical data collected and qualitative reviews of synthesized Java variants. This chapter is organized as follows. Section 5.1, introduces our model-based approach for combining variability modelling and automatic program transformation to understanding what can be vary in a Java program with CVL derivation operators. Section 5.2 presents the experiment, its methodology and the hypotheses to be tested. Section 5.3 and Section 5.4 analyse the results, discuss them and present the threats to validity of this experiment. Section 5.5 concludes the paper and presents future work.

Automatic synthesis of Java Programs with CVL

This section presents an overview of the approach to automatically synthesize variants of Java programs using CVL. The goal of this approach is to empirically analyse the suitability of CVL derivation operators when they transform real Java programs.

Definition

In CVL, the operators are always linked to a target element. Consequently, we will further refer to the definition of a transformation -T as a pair hO, Ei, in which O is a kind of operator (from the aforementioned list) and E is a type of targeted program element (e.g., code statement, class, package).

Given a program transformation T , a program P that successfully compiles and a test suite T S that passes on P , the possible results for a transformed program P 0 = T (P ) are:

1. P 0 is syntactically incorrect and contains compilation errors-P 0 is a counterexample;

2. P 0 is syntactically correct and successfully compiles but at least one test case in T S fails-P 0 is a variant;

3. P 0 compiles and all test cases in T S passes-P 0 is a sosie 1 . } Differently, we have cases in which a substitution of a program element does not imply any error. Listing 5.2 shows one of these cases and, like in the previous transformation, a statement is replaced by another. In this case, the replaced statement is an independent method call, as well as the inserted one, which is a static method call. However, the transformed program does not have the same behaviour of the original one, therefore it does not pass on the test suite of the original one. The reason is because the replaced statement plays a role on the functionality of the time method. In some situations, a transformation can generate compilable variants and, at the same time, preserve the behaviour of the original program-sosie. Following, Listing 5.3 presents a sosie generated from a replacement of a literal value by another of the same type (the string"csv-reporter" is replaced by "m5_rate"), therefore not leading to compilation errors; besides, this literal did not play an important role on the program execution and its behaviour remained unchanged. The success of a transformation depends on the kind of the targeted element. It is expected that it is not possible to modify or remove some program elements without leading to compilation errors (e.g., remove a return keyword from a method with nonvoid value type). On the other hand, we can easily expect that some statements that do not have any impact on the program execution, like log statements, can be removed without any further problem.

Process overview

In the reminder of this chapter, we empirically study the results of applying product line variation points into real Java programs, exploring the possibilities of transforming a Java program in an SPL fashion.

Experiment

In this Section, we present in details the empirical study we conducted in order to assess the product line derivation operators in the context of Java programs.

Goal

The main objective of the experiment is to answer the following questions. How suited are existing product line derivation operators when used as source code transformations? Can we assess the safety of transformations with respect to the program elements?

Measurement Methodology

Our empirical evaluation is focused on analysing the applicability of the aforementioned operators in the elements of a Java program. We measure the percentage of noncompilable, compilable program variants and of sosies generated by a given operator applied to different program elements. We want to observe these percentages both with respect to the operators, the program elements and the pairs operator & program element (transformation). For each analysed program, our experimentation algorithm performs one transformation per time and tries to compile the transformed program, if it compiles, we proceed to run the test suite, checking whether it passes or not.

Experiment Variables

We define our variables according to the theory of scales of measurements; additionally, they are also classified as independent, dependent or controlled variables. Independent and controlled variables influence dependent variables, but the controlled ones remain unchanged during the entire experimentation. Table 5.2.3 presents the experiment variables with their classification and the range of values they can assume during the ObjectSubstitution is greater than with O = Object Existence).

An Object Substitution can be seen as a combination of two Object Existences (i.e., A ceases to exist; while B starts to exist in A's place). Therefore, we expect that randomly succeeding to create and apply a transformation depends on its complexity (in terms of number of instructions).

• H 3 b : Removing program elements that are blocks of code, instead of single instructions, is likely to generate correct programs (we will consider a compile percentage greater than 70%).

Testing H 3 b : can give us initial insights on how the granularity affects the chances of succeeding on varying a Java program.

Subject Programs

The dataset of our experiment is composed by 8 widely-used open source projects. One important selection criterion is that they need to have a good test suite (a statement coverage greater than 70%); they are all expressed in JUnit. Table 5.2 shows the included projects and some relevant properties for the experiment. The size of each program ranges from 1 to 80 KLOC and the number of classes from 23 to 803; they are in the category of APIs and frameworks-programs that are used by other programs. All of them have a good test coverage percentage, ranging from 79% to 94%; we consider that test suites with many assertions and high coverage indicate an important effort and care put into their design.

Table 5.2 also provides the number of statements for each program. Since the transformations manipulate statement for object substitutions, this number is an indicator of the size of the search space for it. None of the programs have a compilation time greater than 10 seconds, which helps on the total time for running the experiments. However, their testing time ranges from 7 to 144 seconds4 .

Protocol

The experiment is designed to randomly explore the possible transformations that can be done in a given program, having its AST nodes and the four operators as the universe Experimenting CVL variation points with Java program constructs to be sampled. Algorithm 1 defines the protocol to run the experiments. It takes as input the program to be transformed and returns the data we use further to analyse the transformations and the AST elements (we get either a counterexample, a variant or a sosie for each random transformation applied). Our stopping criteria is not strict and it is defined by the amount of computational resources available in the Grid5000 The experimental protocol for creating and applying the transformations.

Analysis

Results

Table 5.3 presents the results after running the experiments for the 8 subject programs, having 196816 lines of code in total. We calculate the number of possibilities of applying a specific operator in the universe of the 8 programs (the candidate column). The candidates for Object Existence is simply the number of nodes in the AST; for Object Substitution is each node type of the AST squared, than the sum of them; for Link Existence is the number of fields plus the number of inheritance links; and for Link End Substitution is the number of fields squared plus the number o inheritance links squared.

The Trial column describes how many times we applied a transformation containing the given operator. Given the number of candidates, the number of trials, and a confidence of 99%, we calculate the margin of error for each operator. This margin holds meaning if one wants to consider the results as probabilities inside our universe. An example of interpretation is: the probability of having a program that compiles after removing a random program element is between 10.97% and 11.97% (compile% = 11.47 and margin of error = 0.50).

There are 86 possible combinations between operators and program elements. In Table 5.4, we show the 15 first and the 15 last transformations ordered by their compilation percentage. The first column refers to the type of operator: OE (Object Existence), OS (Object Substitution), LE (Link Existence) and LS (Link End Substitution). The second column is the affected program element. We also show the number of possibilities for each transformation and how many times we actually apply them in our experiments. Figure 5.2 shows the results for 7 groups of program elements, independent on the kind of operator. Each of the vertical bars represent the compilable and sosie percentages for a given project (they are arranged in the same order of Table 5.2). We measured the dependent variables for all 42 program elements. However, to fit the results to be seen here, we have selected 20 elements and grouped them according to their common function. The first group is the if, containing the if and the conditional (i.e., A?B:C) program elements. The loops group contains the do, while, foreach and for elements. The invocations group is composed by the invocation (i.e, a method call such as .a() , where a is a method), unary operators and binary operators; we see the two last as invocations of methods (e.g., a + b is equivalent to add(a,b)). The read group contains the program elements in charge of access: variable, field and array access. The write group is composed by the assignment and operator assignment program elements. The new group contains the elements responsible to create objects or primitive types (the case for literal): new array, new class and literal. In the exception group, we gathered the catch, try and throw elements. In Table 5.3.1, we show the values for the variance, standard deviation, mean and margin of error for each of the aforementioned group of AST elements.

We excluded from Figure 5.2 program elements that have never compiled after being affected, which is, for example, the case for methods, classes, interfaces and packages.

Visualizing the Results

Due to the large amount of data produced as result of the experiment, we had to provide means to ease the visualization of the transformations. Figure 5.3 shows the web-based . Once zoomed, it is possible to see and access each of the colored lines that make part of a class; they represent code locations (a line number) that received transformations. The red portions of the lines represent the amount of transformations in that place of the code that did not succeed to compile; while blue portions represent the ones that compiled and the green portion the ones that resulted on sosies. Furthermore, we made possible to click on each line to visualize the list of the transformations done in a given place (see 3 ). This third view provides details on the actual number of transformations performed in that code location, the name of the applied transformation and their status ( 0 means it compiled and passed the tests, -1 it compiled and -2 it did not compile). In the transformed code, we comment everything that was supposed to be removed/substituted in a transformation in order to let the user compare the before and after the transformation. In the specific case of Figure 5.3, the first transformation erased the first parameter of a method call and the second one replaced the "null" keyword by the "unchecked" string.

Hypotheses Testing and Discussion

Overall Safety of CVL to Java

With respect to H 1 , we can use the data from table 5.3 to see that the chances of generating incorrect programs is much higher than of generating valid ones (see Total line). According to the experiments, around 10% of the transformed programs compiled and 6% were sosies, therefore 90% were counterexamples, which validates H 1 .

Discussion. H 1 confirms that the CVL operators are not safe and do not take into account the syntax or semantics of the target language. As CVL is designed to be generic to any target language, this is acceptable and even expected; it is unfeasible to anticipate every possible domain-specific syntax or semantics rules. On the other hand, observing Figure 5.2 and Table 5.3.1, we can see that there are groups of elements that In order to test H 3 b, we pick the transformations that have Object Existence as their operation and blocks of code as elements: Do, For, ForEach, While, If, Throw. We can observe in Table 5.4 that their variant percentage range from 70%, in the case of the Throw, to 98%, in the case of the ForEach. We can also observe from Table 5.3.1 and Figure 5.2 that the mean for loops, which are blocks of code, is the greatest comparing to the others program elements. We only partially confirm H 3 b and we discuss the reason following.

Discussion. Although the promising results for the aforementioned program elements, we can just partially confirm the idea that blocks of code are easier to vary. The reason is because not every block of code is self-contained, and therefore they refer to other blocks or are referenced in other blocks. For example, the probability to have compilable programs after removing a Class or a Method is less than 0.1%.

Discussion

Modeling Languages: Comparison with Chapter 3

Our previous effort provides evidence that the usage and tooling support of CVL should be specialized for a given domain-specific modeling language [FBLNJ12, FBA + 13] if one wants to achieve safe product derivation.

The experiment presented in this chapter provides further evidence that the direct use of CVL leads to improper support -this time we consider a programming language (not a modeling language). Our empirical study confirms that the syntactic structure and semantics of the targeted language (here Java) should be taken into account. Otherwise, developers will spend their time specifying variability over language elements that lead to unsafe variants. In Chapter 3, we have identified a tendency to generate incorrect product variants for three modeling languages. The evidences pointed that the more complex is a language (quantity of syntactic and semantic rules), the easier is to synthesise wrong product variants. The results in Chapter 3 shows that: in a small modeling language such as the Finite State Machine, the percentage of incorrect variants is around 16%; in the Ecore language it jumps to 25%; at last, in a very big modeling language like UML, the incorrect variants represent 66% of the total. We confirmed this tenet for Java, which raised the percentage of incorrect variants to 90%.

For applying the approach and fully exploring the variation space of a language (Java), we had to develop novel automated techniques. The techniques exposed in Chapter 3 do not seek to categorize which model elements are likely to produce unsafe variants. We also develop a large-scale infrastructure to launch 86 kinds of transformations while checking some properties of the variants. Such an infrastructure was required due to the computational resources involved in the experiment.

Experimenting CVL variation points with Java program constructs

Diversity: Comparison with [BAM14]

Comparing to the experiment in [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF], our approach to synthesize sosies is clearly different. In [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF], program transformations are built based on domain knowledge and empirical observations to decide which transformation can be used. In this chapter, we use an automatic, agnostic method to explore the variability space. We use four CVL realization operators on any kind of Java AST node types. The approach has the merit to discover new kinds of transformation (86 against 9 in [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF]) that have not been used in [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF]. We empirically show that a substantial number can also produce sosies. In particular, the Link Existence realization operator has not been used in [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF]. Another example is the Object Existence realization operator used on specific Java AST node type, -e.g. the call to the super operation -that allows to discover numerous sosies. As a result, we discover that (i) some CVL transformations have a good percentage of sosies and can compete with program transformations introduced in [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF] (ii) as recognized by the authors of [START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF], CVL transformations are sometimes unintuitive for a human and surprisingly work. The removal of a parameter in methods is an example. This result demonstrates the benefits of the use of full automated techniques that do not make any assumption and that visit the whole problem space; (iii) finally, some CVL transformations could be adapted (leading to the creation of new ones) in the context of diversification.

Towards a Methodology and Systematic Approach

We chose Java as it is a widely used programming language, however our experiments can be reproduced to analyse other languages. Following we enumerate the essential steps to reproduce the approach, learnt from the application of the experiment in Java. After acquiring all the data and performing analysis, an immediate usage is to serve as basis to enhance the transformations and create more robust ones. In Chapter ??, this enhancement was done having as basis only the domain expert knowledge. We believe that our methodology could better guide customizations by showing real examples of Java program variants. Second, we can envision practical applications, such as heuristics to enhance design of the mapping relationships between features and program elements.

-ranging from syntax highlighters to recommender systems.

Threats to Validity

Our experiment has the necessary conditions for causality. No other changes are done in the programs by each iteration, we only perform one transformation at a time (see 1), therefore the changes in our dependent variables are only related to the execution of the given transformation.

Internal Validity: One potential threat for internal validity is the number of trials with respect to the possible number of transformations in our universe. We have addressed this threat by controlling the margin of error for the transformations, having it always less than 1% and using a confidence level of 99%. However, some program elements were not numerous enough for being representative, such as annotation types; they are rarely used and therefore our experiments may not be conclusive for these program elements.

External Validity: Regarding the threats to external validity, we relied on the fact that our subject programs are widely used by programmers and that they make part of lots of projects that use them as APIs and frameworks. Besides, they have a considerable amount of lines of code (200K in total). Regarding the representativeness of the transformations, in total, we generated and executed 376,185 transformations. We tried to compute the maximum number of transformations as possible, given the available resources; if we multiply the total number of transformations we have performed by, the compilation time added to the testing time (when compilation is ok) of the 8 projects, we have a total of around 97 days of computation in a personal computer.

By calculating the variance and standard deviation of transforming specific program elements over our 8 different subject programs, we could notice some discrepancy among them. This fact can be an evidence that factors such as choices of design and programming style, can have an influence on the compilation percentages; it needs to be further explored.

Conclusions

We executed 376,185 variability transformations over 8 real large Java programs with up to 85,000 lines of code. The transformations consist in adding, removing, or replacing all kinds of Java elements (classes, loops, fields, assignments, etc.) -they respresent the main variation points in CVL realization layer. We obtained 376,185 variants of Java programs, out of which 41,042 compile and 23,232 pass the test suite. We could obtain a panorama of the suitability of CVL derivation operators applied to such a complex language like Java. The results were as expected: CVL operators are dangerous to be applied in fine-grained programing language constructs; calling for specialized semantics or more sophisticated (de)composition languages. However, we could also verify that there are categories of constructs that can better behave with variational transformations, like blocks of codes.

Chapter 6

Towards a Methodology

In this chapter, we provide a methodology to integrate our work into any organization that seeks to implement model-based software product lines. We focus on the response to the event of having to engineer an MSPL for a given DSL, showing the different activities, how they follow each other and the roles involved in each of them. We model the set of processes using the Business Process Modeling Notation (BPMN) [START_REF]Business Process Modeling Notation OMG. Version 1.0. OMG Final Adopted Specification[END_REF], a widely used and standardized language for this purpose. Instead of explaining the technical details of the approach like in previous chapters, we rather provide high-level tasks for the purpose of situating our approach into the development process.

Roles

Although the construction of an MSPL can involve several stakeholders, there are two roles particularly interested in our methodology, and we will further concentrate on the activities related to them. The two fundamental roles in our methodology are: the MSPL Designer and the MSPL Infrastructure Engineer. Both are directly related to our counterexamples generation approach described in Chapter 3 and are essential to leverage variability management in a model-based context.

The MSPL Designer is the role responsible for constructing the models of an MSPL. This role requires a solid knowledge about the domain, as designing the MSPL models is essentially deciding which and how products can vary. Therefore, the MSPL Designer can be thought as a domain engineer with considerable knowledge of the product engineering phase (required to foresee the outcome of possible products when designing the realization mappings).

The MSPL Infrastructure Engineer is the role responsible for building, extending and maintaining the mechanisms and tools that support the activities of the MSPL designer role. This includes the construction of model editors, interpreters, derivation engines and verification & validation techniques to increase the safety of the designed MSPLs (we concentrate on these last in this chapter). Besides the knowledge of the domain, infrastructure engineers are also required to master model-driven technologies and their workbenches.

Part III

Conclusion and Perspectives

Chapter 7

Conclusion and Perspectives

In this chapter, we first synthesize and conclude all the contributions of this thesis, re-enumerating the challenges and how we addressed each of them. Next and finally, we discuss some perspectives for future research.

Conclusion

This thesis is an effort to support the tasks of managing variability in systems engineering. This management needs special assistance due to the complexity of the development process and of all the machinery involved. Indeed, we showed that, developing a system in Thales (our case organization that uses systems engineering) is a complex task due to two main characteristics: their diversity of domain specific languages and their existing model-based software development life cycle.

We concluded that leveraging variability management in this context raises five big challenges that need to be addressed:

1. provide early support for constructing product lines for new domain specific languages;

2. provide specialized support for product derivation based on each particular domain;

3. provide separation of concerns at both modeling and V&V levels;

4. integrate the modeling and verification approaches in a seamless and non-intrusive way into the existing systems development process;

5. facilitate the consistent generation of artifacts.

From the five challenges identified, we considered those which the scientific community had less advanced in the state of the art and focused on them. The ones more studied could be contemplated in our choice to use the Common Variability Language. Nevertheless, we found limitations on CVL and we concentrated on overcoming them, 104

Conclusion and Perspectives

by addressing the issues on realizing variability and the need for customized assistance in the realization layer.

We introduced and explored the concept of counterexamples of model-based software product lines, having it as basis for engineering better derivation engines and verification mechanisms. We automated the generation of counterexamples in a systematic and domain-independent approach, so that we could synthesize MSPLs starting only from a metamodel of the domain -the fundamental characteristic of our approach in order to provide early support.

We then validated our approach with four different modeling languages, being one of them acquired from a real industry scenario. The approach could generate counterexamples in a reasonable time, both when only the metamodel of the language was available, as well as when we could use existing variability or base models. After, we gave first insights on how to customize the derivation engine based on the counterexamples knowledge and how one could weave new semantics into the derivation process.

Using the same idea of the counterexamples generation, we extrapolated our experiments to the vast scenario of Java programs. The experiments aimed to analyse how adequate was the use of CVL in a complex programming language like Java. We concentrated on the adequacy of the derivation operators of CVL in fine-grained program elements. Following the empirical method, we could systematically asses the safety of product-line-based code transformations. This experiment also served to demonstrate the first steps after having generated counterexamples: analysing, ordering and categorizing them.

Finally, we synthesized our contributions in the form of a methodology, defining the roles and the high-level activities for an organization that wants to benefit from our approach.

We conclude that our approach contemplates the five challenges raised in the beginning:

1. we provided early support by enabling the approach to work in the case of nonexisting models (e.g., initial MSPLs can be generated even before domain engineering phases, before the variability model definition);

2. we provided mechanisms to customize the CVL derivation engines, adapting the operational semantics of its variation points;

3. we used CVL and implemented tooling support following its concepts, providing an orthogonal and modular way to express variability on top of multiple modeling languages;

4. our methodology showed that the approach can be used in parallel to the other activities of the development process -it does not require the engineer to learn and use any new notation -and there is no need to change it, avoiding intrusive mechanisms (e.g., other approaches require augmenting the current notations with variability information);

Conclusion and Perspectives

concrete syntax of the variability model would already prohibit feature combinations that would lead to incorrect products, or maybe mutate the links so they could exclude mutually exclusive parts or include absent dependent parts (e.g., two classes that could not be at the same product, because of a mutually exclusive pair of methods, could select a method and erase or comment the other one).

The IDE would capture the intention of the engineer. If he/she defines a feature called "Log System", the realization layer would suggest automatically to link to, for example, the code statements using methods/classes from the java.util.logging, but still considering the impacts of removing, replacing or changing the different statements, based on the gained knowledge from the counterexamples. This could be true for any language incorporated in the IDE, as the counterexamples approach would work independently and agnostic to the domain.

Self-tuning of product line checkers

Many verification mechanisms of SPL's models are based on type checking. With a Type System, they have a set of rules that constrain the possible correct model designs, detecting defects that makes a model ill-typed. However, designing such a type system for the triplet (Variability Model, Realization Model, Base Models) is a very hard task. Therefore, a promising vision is to automate the construction of these rules. From the generated counterexamples, we can extract valuable knowledge about common errors and synthesize new rules to be incorporated into checkers -a self-tuning of checking mechanisms. An initial process to do such a self-tuning system is to take the triplet elements that always resulted on counterexamples and synthesize preventive rules (e.g., if in (Variability Model, Realization Model, Base Models) contains (feature A, object existence, mandatory model element E) then return error).

Narrowing down the search space

A short-term perspective is to narrow down the search space, decreasing the time to find counterexamples and, perhaps, finding more meaningful ones. Search-based algorithms can be of great value if one can define good fitness functions for finding and generating counterexamples; they already proved value on software test data generation [START_REF] Mcminn | Search-based software test data generation: a survey. Software testing[END_REF]. Another possibility is to limit the search on the possible configurations of the feature model; t-wise algorithms have been developed and used to bypass the combinatorial explosion imposed by variability models [PSK + 10, JHF12]. Therefore, our idea is to incorporate a t-wise coverage criteria into our algorithm.

Integration with model slicers

Another short-term perspective is to see and implement the realization model as a model slicer, which is a mechanism to extract a subset of a model [START_REF] Blouin | Modeling model slicers[END_REF]; this could be particularly helpful in a negative derivation scenario. The advantage would be to reuse the slicing techniques to better depict model elements from the base model; for

  .1(d), removing the class Car

  l e a d s t o e x c l u d e n o t o n l y i t s e l f a n d i t s r e l a t i o n s h i p s , b u t a l s o r e m o v i n g i t s s u b c l a s s e s , i n t h i s c a s e , t h e c l a s s Sedan. (a) Original Model! (b) Remove Garage! (c) Remove Sedan! (d) Remove Car! F i g u r e 1 . 1 : D i ffe r e n t s e m a n t i c s f o r r e m o v i n g a c l a s s . T h e s e m a n t i c s o f e x c l u d i n g a n e l e m e n t c a n v a r y f o r t h e s a m e t y p e o f m o d e l e l e m e n t s , b u t c a n a l s o v a r y f o r d i ffe r e n t t y p e s . E x c l u d i n g a p a c k a g e o f a c l a s s d i a g r a m c a n i m p l y o n r e m o v i n g a l l i t s c l a s s e s . H o w e v e r , e x c l u d i n g a c l a s s a t t r i b u t e m a y n o t l e a d t o a n y f u r t h e r o p e r a t i o n s .An o t h e r s c e n a r i o t o c o n s i d e r i s r e l a t e d t o t h e o t h e r k i n d s o f b a s e m o d e l s , s u c h a s t h e a c t i v i t y d i a g r a m o f t h e U M L , i n w h i c h w e c a n o b s e r v e t h a t t h e m a t e r i a l i z a t i o n s e m a n t i cs c a n v a r y e v e n m o r e . I n F i g u r e 1 . 2 , e x c l u d i n g t h e F a s t e n S e a t B e l t a c t i v i t y c a n i m p l y o n f o u r o p e r a t i o n s : r e m o v e F asten Seat Belt m o d e l e l e m e n t , r e m o v e t h e i n c o m e l i n k , r e m o v e t h e o u t c o m e l i n k a n d c r e a t e a n e w l i n k f r o m t h e Get in a c t i v i t y t o t h e Start Engine a c t i v i t y . T h i s i s a l s o d i s c u s s e d i n [ C A 0 5 b ] .

6

 6 (a) Original Model! (b) Remove Fasten Seatbelt! F i g u r e 1 . 2 : R e m o v i n g a n a c t i v i t y . 1 Context T h e n e e d f o r c u s t o m i z i n g t h e o p e r a t i o n a l s e m a n t i c s o f C V L f o r a s p e c i fi c d o m a i n d o e s n o t m e a n t h a t C V L i s i n a d e q u a t e ; i t r a t h e r c o n fi r m s t h a t C V L i s e x t e n s i b l e a n d c a n b e u s e d a s b a s i s . I n f a c t , o n e c a n a l s o a r g u e t h a t t h i s s e m a n t i c s v a r i a t i o n c a n b e e x p r e s s e d w i t h t h e C V L s t a n d a r d s e m a n t i c s , b y c o m p o s i n g v a r i a t i o n p o i n t s . H o w e v e r , w e n o t i c e d f r o m p r a c t i c a l e x p e r i e n c e t h a t s u c h c u s t o m i z a t i o n i s i m p o r t a n t t o l e v e r a g e a b s t r a c t i o n o v e r t h e b a s e m o d e l s e m a n t i c s , e n c a p s u l a t i n g s e c o n d a r y o p e r a t i o n s . T h e r ef o r e , t h e e n g i n e e r i n c h a r g e o f d e s i g n i n g a C V L r e a l i z a t i o n m o d e l c a n , f o r e x a m p l e , a b s t r a c t o v e r t e d i o u s o p e r a t i o n s , s u c h a s r e m o v i n g d a n g l i n g r e f e r e n c e s o r e x c l u d i n g c o n t a i n e d e l e m e n t s . F i g u r e 1 . 3 i l l u s t r a t e s t h e s e m a n t i c s v a r i a t i o n s o f d e r i v a t i o n o p e r a t o r s . F o r e x a m p l e , D 1 i s a n o p e r a t o r t h a t c a n a s s u m e t h r e e d i ffe r e n t m e a n i n g s , a c c o r d i n g t o t h e b a s e m o d e l

F i g u r e 1 . 3 :

 13 S e m a n t i c s v a r i a t i o n o f d e r i v a t i o n o p e r a t o r s .1.3.2 Semantics specialization mechanismsC V L p r o v i d e s m e a n s t o b e s p e c i a l i z e d : i t h a s a s e m a n t i c a l l y c u s t o m i z a b l e d e r i v a t i o n o p e r a t o r c a l l e d O p a q u e V a r i a t i o n P o i n t ( O V P ) . A n O V P w o r k s l i k e a n e x t e n s i o n p o i n t , a l l o w i n g t h e i m p l e m e n t a t i o n o f " h o m e -m a d e " s e m a n t i c s a s a b l a c k b o x t h a t c a n d e fi n e an arbitrary behaviour to be executed during derivation.

2. 1 . 2 h i c h c a n b e t a k e n b y a p p l i c a t i o n e n g i n e e r s t o r e s o l v e t h e v a r i a t i o n s f o r a p r o d u c t o f a s y s t e m i n t h e d o m

 12 Variability modelingV a r i a b i l i t y m o d e l i n g i s a k e y a c t i v i t y i n S P L , i t c r o s s c u t s b o t h d o m a i n a n d a p p l i c a t i o n e n g i n e e r i n g . O v e r t h e p a s t t w e n t y y e a r s , a n u m b e r o f v a r i a b i l i t y m o d e l l i n g a p p r o a c h e s h a v e b e e n p r o p o s e d . R o u g h l y , t h e y c a n b e g r o u p e d i n t w o m a i n c a t e g o r i e s : F e a t u r e m o d e l i n g a n d d e c i s i o n m o d e l i n g . Decision modelling f o c u s e s o n d e c i s i o n s r a t h e r t h a n d o m a i n d e s c r i p t i o n . D e c i s i o n s w e r e fi r s t i n t r o d u c e d b y C a m p b e l l a n d o t h e r s [ C F W 9 0 ] a s " a c t i o n s " w a i n . T h e S y n t h e s i s m e t h o d [ B u r 9 3 ] d r i v e s t h e m a j o r i t y o f e x i s t i n g d e c i s i o n m o d e l i n g a p p r o a c h e s . A d e c i s i o n m o d e l i s d e fi n e d a s " a s e t o f r e q u i r e m e n t s a n d e n g i n e e r i n g d e c i s i o n s t h a t d e t e r m i n e t h e v a r i e t y o f w o r k p r o d u c t s i n t h e d o m a i n , a n d m u s t b e r e s o l v e d b y a n a p p l i c a t i o n e n g i n e e r t o d e fi n e a n d c o n s t r u c t w o r k p r o d u c t s " . S c h m i d a n d J o h n [ S J 0 4 ] , F o r s t e r a n d o t h e r s [ F M P 0 8 ] , D h u n g a n a a n d o t h e r s [ D R G N 0 7 ] , a m o n g s t o t h e r s , u s e d e c i s i o n m o d e l s a s v a r i a b i l i t y m o d e l l i n g l a n g u a g e . Feature modeling i s t h e m o s t p o p u l a r n o t a t i o n a n d h a s g a i n e d a t t e n t i o n o f b o t h r e s e a r c h a n d i n d u s t r y . T h e fi r s t f e a t u r e m o d e l w a s p r o p o s e d b y K a n g a n d o t h e r s [ K C H + 9 0 ] , i n 1 9 9 0 , a s p a r t o f t h e m e t h o d F e a t u r e -O r i e n t e d D o m a i n A n a l y s i s ( F O D A ) . S i n c e t h e n , s e v e r a l o t h e r f e a t u r e m o d e l i n g a p p r o a c h e s w e r e p r o p o s e d b a s e d o n F O D A [ C B U E 0 2 , C H E 0 4 , F F B 0 2 , G F A 9 8 , K T S + 0 9 , K K L + 9 8 , K L D 0 2 , R F S 0 8 , S H T B 0 7 ] . T h e y u s u a l l y State of the Art f o c u s o n d e s c r i b i n g t h e p r o d u c t l i n e d o m a i n a n d t h e i r k e y i d e a i s t o c a p t u r e i n a f e a t u r e m o d e l t h e s e t o f p o s s i b l e p r o d u c t s o f a p r o d u c t l i n e .

Figure 3 . 1 :

 31 Figure 3.1: A family of medical images described with a feature model

F i g u r e 2 . 2 :

 22 E x a m p l e o f a f e a t u r e m o d e l a d a p t e d f r o m [ C G R + 1 2 ] . V a r i a b i l i t y m o d e l i n g h a s s o m e i m p o r t a n t c h a r a c t e r i s t i c s ; s o m e w o r k s t r y t o c a p t u r e t h e m , f o r e x a m p l e , C z a r n e c k i a n d o t h e r s [ C G R + 1 2 ] h a v e i d e n t i fi e d t e n d i m e n s i o n s o f v a r i a b i l i t y m o d e l i n g a p p r o a c h e s : a p p l i c a t i o n s , u n i t o f v a r i a b i l i t y , o r t h o g o n a l i t y , d a t a t y p e s , h i e r a r c h y , d e p e n d e n c i e s a n d c o n s t r a i n t s , m a p p i n g t o a r t i f a c t s , b i n d i n g t i m e a n d m o d e , m o d u l a r i t y a n d t o o l a s p e c t s . I s t o a n a n d o t h e r s [ I K P 1 1 ] p r o p o s e f o u r d i ffe r e n t c a t e g o r i e s o f v a r i a b i l i t y m o d e l i n g a p p r o a c h e s . F o l l o w i n g , w e b r i e fl y d i s c u s s s om e c h a ra c t e r i s t i c s o f v a r i a b i l i t y m o d e l s t h a t w e b e l i e v e t o b e i m p o r t a n t t o t h e r e m a i n d e r o f t h i s t h e s i s .Unit of variability. T h e s e a r e t h e k e y c o n c e p t s o f a v a r i a b i l i t y m o d e l . T h e s e u n i t s a r e f e a t u r e s i n t h e c a s e o f F e a t u r e M o d e l i n g a n d d e c i s i o n s i n t h e c a s e o f D e c i s i o n M o d e l i n g . T h e u n i t o f v a r i a b i l i t y d e fi n e s h o w g r a i n e d a r e t h e v a r i a b l e a n d c o m m o n p a r t s e x p r e s s e d i n t h e v a r i a b i l i t y m o d e l . Orthogonality. T h e g o a l o f t h i s d i m e n s i o n i s t o i n t r o d u c e e x p l i c i t l y d o c u m e n t a t i o n o f t h e v a r i a b i l i t y i n s o f t w a r e p r o d u c t l i n e s i n t o a s e p a r a t e m o d e l . T h e y p r o v i d e a c r o s ss e c t i o n a l v i e w o f t h e v a r i a b i l i t y o f t h e p r o d u c t l i n e a c r o s s a l l s o f t w a r e d e v e l o p m e n t a r t i f a c t s . O n e -w a y r e f e r e n c e s t o t h e b a s e m o d e l d e s c r i b e h o w t h e b a s e m o d e l e l e m e n t s c a n v a r y . B a c h m a n n a n d o t h e r s [ B G d P L + 0 3 ] p r o p o s e d t h e u s e o f o r t h o g o n a l v a r i a b i l i t y m o d e l s t o p r o v i d e t h i s s e p a r a t e v i e w o f t h e v a r i a b i l i t y b y d o c u m e n t i n g e x p l i c i t l y t h e v a r i a t i o n p o i n t s . T h e O r t h o g o n a l V a r i a b i l i t y M o d e l ( O V M ) w a s i n i t i a l l y p r o p o s e d i n [ P B L 0 5 ] . T h i s a p p r o a c h p r o p o s e s t o d o c u m e n t v a r i a b i l i t y i n a s e p a r a t e m o d e l , a n d i n t e r r e l a t e s v a r i a b i l i t y o n t h e b a s e p r o d u c t l i n e m o d e l s . O V M i s m a i n l y c h a r a c t e r i s e d b y c o n s i d e r i n g v a r i a t i o n p o i n t s a s fi r s t -c l a s s c i t i z e n s . A n o t h e r a p p r o a c h p r o p o s e d t o m a k e variability models orthogonal to the product line models is the Common Variability Language (CVL) [FHMP + 11b].

F

  i g u r e 2 . 3 : F S M m e t a m o d e l .

  0 8 a ] . N u m e r o u s M S P L t e c h n i q u e s h a v e b e e n p r o p o s e d ( e . g . , s e e [ P B v d L 0 5 , P K G J 0 8 , H S S + 1 0 , C A 0 5 a , C H S + 1 0 b , C P 0 6 , Z J 0 6 , V G 0 7 ] ) . T h e y u s u a l l y c o n s i s t i n i ) a v a r i a b i li t y m o d e l ( e . g . , a f e a t u r e m o d e l o r a d e c i s i o n m o d e l ) , i i ) a m o d e l ( e . g . , a s t a t e m a c h i n e , a c l a s s d i a g r a m ) e x p r e s s e d i n a s p e c i fi c m o d e l i n g l a n g u a g e ( e . g . , U n i fi e d M o d e l i n g L a ng u a g e ( U M L ) [ G r o 0 7 ] ) , a n d i i i ) a r e a l i z a t i o n l a y e r t h a t m a p s a n d t r a n s f o r m s v a r i a t i o n p o i n t s i n t o m o d e l e l e m e n t s . B a s e d o n a s e l e c t i o n o f d e s i r e d f e a t u r e s i n t h e v a r i a b i li t y m o d e l , a d e r i v a t i o n e n g i n e c a n a u t o m a t i c a l l y s y n t h e s i s e c u s t o m i z e d m o d e l s -e a c h m o d e l c o r r e s p o n d i n g t o a n i n d i v i d u a l p r o d u c t o f t h e S P L . T h e k e y p o i n t o f a n M S P L i s t h e a b i l i t y t o c h e c k w h e t h e r t h e d e r i v e d p r o d u c t i s u r e 2 . 4 : D e s i g n s p a c e a n d c o n f o r m i n g m o d e l s .

  u r e 2 . 5 : O v e r v i e w o f C V L m o d e l s a n d a b a s e m o d e l . d e p e n d s b o t h o n w e l l -f o r m e d n e s s r u l e s ( s y n t a c t i c r u l e s ) a n d b u s i n e s s , d o m a i ns p e c i fi c r u l e s ( s e m a n t i c r u l e s ) . T h e O b j e c t C o n s t r a i n t L a n g u a g e ( O C L ) i s t y p ic a l l y u s e d f o r s p e c i f y i n g t h e s t a t i c s e m a n t i c s . I n C V L , a b a s e m o d e l p l a y s t h e r o l e o f a n a s s e t i n t h e c l a s s i c a l s e n s e o f S P L e n g i n e e r i n g . T h e s e m o d e l s a r e t h e n c u s t o m i z e d t o d e r i v e a c o m p l e t e p r o d u c t . • Variability Realization Model (V RM ) c o n t a i n s a s e t o f V a r i a t i o n P o i n t s ( V P ) . T h e y s p e c i f y h o w V S p e c s ( i . e . , C h o i c e s ) a r e r e a l i z e d i n t h e b a s e m o d e l ( s ) . A n S P L d e s i g n e r d e fi n e s i n t h e V R M w h a t e l e m e n t s o f t h e b a s e m o d e l s a r e r em o v e d , a d d e d , s u b s t i t u t e d , m o d i fi e d ( o r a c o m b i n a t i o n o f t h e s e o p e r a t i o n s , s e e b e l o w ) g i v e n a s e l e c t i o n o r a d e s e l e c t i o n o f a C h o i c e i n t h e V A M . B u t i n t h e l a s t i t e r a t i o n w e c o u l d i d e n t i f y d i s c r e p a n c i e s . W i t h r e s p e c t t o t h e v a r i a b i l i t y m o d e l , w e h a v e f o u n d e v i d e n c e s t h a t i t i s a t o u g h t a s k t o d e s i g n i t w i t h o u t l e a d i n g t o a n y w r o n g p r o d u c t m o d e l s . I t i s a l s o u n f e a s i b l e t o p r e d i c t e v e r y p o s s i b l e c o n fi g u r a t i o n , State of the Art once this number can reach exponential.

F

  i g u r e 2 . 6 : C V L m o d e l o v e r a n F S M b a s e m o d e l . C o n s i d e r i n g t h e M S P L o f F i g u r e 2 . 6 , i t i s a c t u a l l y p o s s i b l e t o d e r i v e i n c o r r e c t F S M m o d e l s e v e n s t a r t i n g f r o m a v a l i d B M a n d v a l i d c o n fi g u r a t i o n s o f V A M . T h i s i s i l l u st r a t e d i n F i g u r e 2 . 7 . C o n fi g u r a t i o n 1 g e n e r a t e s a c o r r e c t F S M m o d e l , i . e . , c o n f o r m i n g t o i t s m e t a m o d e l . Conf iguration 2 a n d Conf iguration 3, d e s p i t e b e i n g v a l i d c o n fi g u r at i o n s o f t h e V A M , l e a d t o t w o u n s a f e p r o d u c t s . I n d e e d , t h e F S M m o d e l g e n e r a t e d f r o m Conf iguration 2 i s n o t c o r r e c t : a c c o r d i n g t o t h e m e t a m o d e l , a n o u t

  m e t a m o d e l . E v e n f o r a v e r y s i m p l e M S P L , s e v e r a l n o n -c o n f o r m i n g p r o d u c t m o d e l s c a n b e d er i v e d i n c o n t r a d i c t i o n t o t h e i n t e n t i o n o f a n M S P L d e s i g n e r . I n p r a c t i c e , s p e c i f y i n g a c o r r e c t M S P L i s a d a u n t i n g a n d e r r o r -p r o n e a c t i v i t y d u e t o t h e f a c t t h a t t h e n u m b e r o f c h o i c e s i n t h e VA M , t h e n u m b e r o f c l a s s e s a n d r u l e s i n t h e m e t a m o d e l a n d t h e s i z e o f t h e V R M c a n b e b i g g e r .

F i g u r e 2

 2 . 7 : C o n fi g u r a t i o n a n d d e r i v a t i o n o f F S M s . T h e p r o b l e m o f s a f e l y c o n fi g u r i n g a f e a t u r e o r a d e c i s i o n m o d e l i s n o w w e l l u n d e rs t o o d [ B S R c 1 0 ] . M o r e o v e r , s e v e r a l t e c h n i q u e s e x i s t f o r c h e c k i n g t h e c o n f o r m a n c e o f a m o d e l f o r a g i v e n m o d e l i n g l a n g u a g e . H o w e v e r , t h e c o n n e c t i o n o f b o t h p a r t s ( t h e V A M a n d t h e s e t o f b a s e m o d e l s ) a n d t h e m a n a g e m e n t o f t h e r e a l i z a t i o n l a y e r a r e s t i l l c r u c i a l i s s u e s [ T B K C 0 7 , A t B G F 1 1 , C H S + 1 0 b , S H M P 1 1 , T A K + 1 4 a ]

  r e n t v a r i a b i l i t y m o d e l i n g a p p r o a c h e s , w h i l e t h e b o t t o m o n e s r e p r e s e n t t h e t y p e s o f p r o d u c t -l i n e a n a l y s i s t e c h n i q u e s . T h e a r r o w f r o m o n e b o x t o a n o t h e r m e a n s t h a t t h e f e a t u r e / c h a r a c t e r i s t i c i s s u p p o s e d t o s u p p o r t t h e p o i n t e d i s s u e . T h e c o l o r o f t h e i s s u e i s h o w w e s e e t h e l e v e l o f e x i s t i n g s u p p o r t : v e r y l o w ( d a r k r e d ) , l o w ( s o f t r e d ) , a d v a n c e d ( s o f t b l u e ) , v e r y a d v a n c e d ( d a r k b l u e ) . F o l l o w i n g , w e e x p la i n h o w e a c h c h a l l e n g e h a s b e e n a d d r e s s e d i n t h e s t a t e o f t h e a r t ( i n c a s e i t h a s ) a n d w h i c h f e a t u r e s o f e x i s t i n g a p p r o a c h e s a r e r e l a t e d t o t h e m a n d h o w .Modeling !"#$%&'())*#+&,*#& -./&012'& 1).34"$45.6& 6.#47"8*-& '())*#+& 1.)"#"8*-&*,& 3*-3.#-'& 1."9$.''&-*-: 4-+#('47.& 4-+.;#"8*-&& <*-'4'+.-+& ;.-.#"8*-&*,& "#8,"3+'& Verification & Analysis =#+>*;*-"$& ?*6.$'& !@+.-'4*-&A*4-+'& B(+*9"83& A#*6(3+& 0.#47"8*-& A#*6(3+:C"'.6& D"94$%:C"'.6& D."+(#.:C"'.6& ?*6($"#4+%& 1).34"$45.6& 1."9$.''&-*-: <*-'4'+.-+& '())*#+& 4-+.;#"8*-&& "#8,"3+'& 4-+.;#"8*-&& 4-+.;#"8*-&& 4-+.;#"8*-&& 4-+.;#"8*-&& Support F i g u r e 2 .8 : S y n t h e s i s o f i s s u e s a n d e ffo r t s i n t h e s t a t e o f t h e a r t .Separation of concernsA t t h e m o d e l i n g a n d a n a l y s i s l e v e l s , m o d u l a r i z a t i o n m e c h a n i s m s a r e t h e m a i n f e a t u r e s t o l e v e r a g e s e p a r a t i o n o f c o n c e r n s i n v a r i a b i l i t y m a n a g e m e n t a p p r o a c h e s . S o m e m e c h an i s m s r e l y o n a s p e c t -o r i e n t a t i o n [ N K 0 8 , M F B + 0 8 ] , o t h e r s i n f e a t u r e -o r i e n t a t i o n [ K T S + 0 9 , M O 0 4 ] ; a n d C V L h a s c o n fi g u r a b l e u n i t s a n d v a r i a b i l i t y i n t e r f a c e s , w h i c h f a c i l i t a t e t h e s p e c i fi c a t i o n o f c o n fi g u r a b l e c o m p o n e n t s . B e s i d e s m o d u l a r i z a t i o n , t h e o r t h o g o n a l p r o pe r t y o f a p p r o a c h e s l i k e C V L a n d O V M p l a y s a n i m p o r t a n t r o l e t o s e p a r a t e v a r i a b i l i t y c o n c e r n s f r o m t h e w o r k i n g d o m a i n . A l t h o u g h w e b e l i e v e t h a t s u p p o r t f o r a d d r e s s i n g t h i s c h a l l e n g e i s v e r y a d v a n c e d , c r u c i a l i s s u e s , l i k e g r a n u l a r i t y a n d f e a t u r e i n t e r a ct i o n s [ K A O 1 1 ] , s t i l l n e e d s p e c i a l a t t e n t i o n .

4 8

 4 Overview of the Contributionsv a r i a t i o n p o i n t s w h e n a p p l i e d t o a c o m p l e x p r o g r a m m i n g l a n g u a g e s u c h a s J a v a . I t a l l o w s u s t o e v a l u a t e n o t o n l y C V L a n d o u r c o u n t e r e x a m p l e -b a s e d a p p r o a c h , b u t a l s o J a v a , a s w e a r e a b l e t o a s s e s s h o w h a r d i s t o r a n d o m l y v a r y J a v a p r o g r a m s i n a w a y t h a t t h e y c o n t i n u e t o b e c o m p i l a b l e o r t e s t a b l e , a n d t h e r e f o r e , h o w f r a g i l e t h e y a r e . O u r c o n t r i b u t i o n t o t h e s e p a r a t i o n o f c o n c e r n s c h a l l e n g e r e l i e s o n t h e u s e o f C V L , c o n s i d e r i n g t h a t i t c a n g u a r a n t e e t h e d e g r e e s o f m o d u l a r i t y a n d o r t h o g o n a l i t y w e n e e d w h e n d e s i g n i n g M S P L s f o r s y s t e m s e n g i n e e r i n g . O u r c o n t r i b u t i o n s a r e i n t r i n s i c a l l y l i n k e d t o r e a l i n d u s t r i a l c h a l l e n g e s e x t r a c t e d f r o m T h a l e s G r o u p ; t h e r e f o r e w e a l s o p r o v i d e i n C h a p t e r 6 a m e t h o d o l o g y t o a p p l y t h e m i n t h e i n d u s t r y , s h o w i n g t h e e n g i n e e r s r o l e s a n d a c t i v i t i e s i n a d e fi n e d p r o c e s s .

!

  c " C VAM | !(CVL, c, BM) = DM "F i g u r e 3 . 1 : A n e x a m p l e o f c o u n t e r e x a m p l e .

  r h a n d , i f w e a l r e a d y h a v e a s e t o f m o d e l s , w e c a n u s e m u t a t i o n o p e r a t o r s t o i n c r e a s e t h e n u m b e r o f s a m p l e s , o r j u s t n o t m o d i f y t h e b a s e m o d e l s . M u t a t i o n s o p e r a t o r s a r e b a s i c C R U D ( C r e a t e , R e a d , U p d a t e , D e l e t e ) o p e r a t i o n s o n t h e b a s e m o d e l t h a t a r e a p p l i e d r a n d o m l y .

  T h e m a x i m u m d e p t h o f t h e V AM ( M A X _ D E P T H ) a n d t h e m a x i m u m n u m b e r u r e 3 . 2 : O v e r v i e w . o f c h i l d r e n f o r e a c h V Spec ( M A X _ C H I L D R E N ) .• T h e p e r c e n t a g e o f V Specs t h a t w i l l b e l i n k e d t o v a r i a t i o n p o i n t s ( L I N K _ P E R C E N T ) .F o r e x a m p l e , i n F i g u r e 3 . 2 , t h e V AM w a s g e n e r a t e d w i t h a p e r c e n t a g e o f 6 6 % , a s f o u r o u t o f s i x V Specs a r e l i n k e d t o V P s.

  A R l a n g u a g e i s e x e c u t a b l e a n d g i v e s s u p p o r t t o m a n i p u l a t e a n d r e a s o n a b o u t f e a t u r e m o d e l s [ A C L F 1 3 ] ( w e c o u l d

  V R M m o de l s . T h e fi e l d N u m b e r o f C o u n t e r e x a m p l e s d e t e r m i n e s w h e n L i n e G e n h a s t o s t o p t h e Evaluation 57

F

  t h e c o m p l e x i t y o f t h e E c o r e m e t a m o d e l . T h u s , a s t h e t a r g e t n u m b e r o f c o u n t e r e x a m p l e i n c r e a s e s , w e c a n c o n fi r m a l i n e a r g r o w t h o f t h e t i m e . T h e l i n e a r t r e n d l i n e s a r e a g o o d fi t t o t h e o b t a i n e d t i m e v a l u e s , w i t h R 2 v a l u e s c l o s e t o 1 . E a c h t i m e v a l u e i s a n a v e r a g e o f 1 0 e x e c u t i o n s , t h i s w a s d o n e t o m i n i m i z e t h e r a n d o m e ffe c t . i g u r e 3 . 4 : C o u n t e r e x a m p l e s f o r F S M a n d E c o r e .

s m o r e l i k e l y t o d e s i g n c o u n t e r e x a m p l e

  s ( i . e . , u n s a f e M S P L s ) w h e n t h e d o m a i n b e c o m e s m o r e c o m p l e x o r n o t . Objects of Study. T o a d d r e s s R Q 2 , w e c o m p a r e d t h e r a t i o b e t w e e n t h e n u m b e r o f i n v a l i d DM s a n d v a l i d DM s. W e m a d e t h i s c o m p a r i s o n w i t h t h r e e d i ffe r e n t m o d e l i n g l a n g u a g e s : F S M , E c o r e ( w i t h t h e E c l i p s e M o d e l i n g F r a m e w o r k i m p l e m e n t a t i o n ) a n d U M L ( w i t h t h e E c l i p s e U M L 2 p r o j e c t i m p l e m e n t a t i o n . W e c l a s s i fi e d t h e s e m o d e l i n g l a n g u a g e s i n t h e f o l l o w i n g i n c r e a s i n g s e q u e n c e o f c o m p l e x i t y : F S M < E c o r e < U M L . I n d e e d , t h e F S M m e t a m o d e l c o n t a i n s o n l y 3 m e t a c l a s s e s 1 d a t a t y p e a n d 4 v a l i d a t i o n r u l e s . T h e E c o r e m e t a m o d e l c o n t a i n s 2 0 m e t a c l a s s e s , 3 3 d a t a t y p e s a n d 9 1 v a l i d a t i o n r u l e s . F i n a l l y , t h e U M L c o n t a i n s 2 4 7 m e t a c l a s s e s , 1 7 d a t a t y p e s a n d 6 8 4 v a l i d a t i o n r u l e s . Experimental Setup. F o r e a c h m o d e l i n g l a n g u a g e , w e a p p l i e d o u r a p p r o a c h t o o b t a i n 1 0 0 c o u n t e r e x a m p l e s , u s i n g t h e s a m e p a r a m e t e r s o f t h e fi r s t e x p e r i m e n t , a n d w e c o l l e c t t h e n u m b e r o f c o r r e c t DM s w e o b t a i n . T h e e v a l u a t i o n w a s p e r f o r m e d o n t h e same computer of the previous experiment.

Listing 3 . 1 : 3 / * Rule 1 :

 3131 Antipattern rules for FSM 1 def checkVRM(f:FSM,vrm: VPackage):Boolean = { 2 vrm.asInstanceOf[VPackage].getPackageElement().foreach(e=> { Replacing a final state by an initial one, and vice versa , is 4 forbidden. * / 5 if (e.isInstanceOf[ObjectSubstitution]){ 6 var p = e.asInstanceOf[ObjectSubstitution].getPlacementObject().getReference() 7 var p1 = e.asInstanceOf[ObjectSubstitution].getReplacementObject(

F i g u r e 3 . 5 :

 35 C o u n t e r e x a m p l e s f o r A R C A D I A s a m p l e m o d e l . A b o u t 3 0 % o f t h e m o d e l s g e n e r a t e d f o r t h i s d o m a i n w e r e w r o n g , m e a n i n g t h a t , i n a v e r a g e , i f w e r a n d o m l y d e fi n e r e a l i z a t i o n r e l a t i o n s h i p s a m o n g t h e f e a t u r e s a n d t h e m o d e l e l e m e n t s o f t h i s d o m a i n , a l m o s t o n e t h i r d c a n r e s u l t o n c o u n t e r e x a m p l e s . A n o t h e r i n t e r e s t i n g r e s u l t i s t h e f a c t t h a t o n l y o n e O C L r u l e a d d e d t o t h e V R M c a n r e m o v e 5 0 % o f t h e c o u n t e r e x a m p l e s ( F o r b i d d e n a n o b j e c t e x i s t e n c e o n a s p e c i fi c k i n d o f m o d e l e l e m e n t " E v e n t S e n d C a l l A c t i o n " ) a n d 8 0 % o f t h e c o u n t e r e x a m p l e s c a n b e r e m o v e d i n w r i t i n g 8 b a s i c O C L r u l e s . W i t h t h i s e x a m p l e , w e c a n s h o w t h a t t h e g e n e r a t i o n o f c o u n t e r e x a m p l e s f r o m a r e f e r e n c e m o d e l c a n h e l p t o d e t e c t s o m e a n t i -p a t t e r n s t h a t c a n b e e a s i l y c o n s t r a i n e d a n d d e t e c t e d f o r a p a r t i c u l a r d o m a i n . T h e r e s u l t i s t h e i m p r o v e m e n t o f t h e u s e o f C V L i n t h i s i n d u s t r i a l c o n t e x t a n e a r l y d e t e c t i o n o f C V L m o d e l t h a t c a p t u r e i n v a l i d p r o d u c t s .

B e c a u s e o f t h e c o m b i n a t o r i a l e x p l o s i o n o f p o s s i b l e d e r i v e d v a r i a n t s , t h e g r e a t v a r i e t y a n d

  c o m p l e x i t y o f i t s m o d e l s , c o r r e c t l y d e s i g n i n g a M o d e l -b a s e d S o f t w a r e P r o d u c t L i n e ( M S P L ) h a s p r o v e d t o b e c h a l l e n g i n g . I t i s e a s y f o r a d e v e l o p e r t o s p e c i f y a n i n c o r r e c t s e t o f m a p p i n g s b e t w e e n t h e f e a t u r e s / d ec i s i o n s a n d t h e m o d e l i n g a s s e t s , t h u s a u t h o r i z i n g t h e d e r i v a t i o n o f u n s a f e p r o d u c t m o d e l s i n t h e M S P L . I n t h i s c h a p t e r , w e h a v e p r e s e n t e d a s y s t e m a t i c a n d f u l l y a u t o m a t e d a p p r o a c h t o e x p l o r e t h e d e s i g n s p a c e o f a n M S P L . T h e m a i n o b j e c t i v e o f t h e a p p r o a c h w a s t o g e n e r a t e c o u n t e r e x a m p l e s o f M S P L s , i . e . , M S P L s t h a t c a n p r o d u c e i n v a l i d p r o d u c t m o d e l s . T h i s k i n d o f M S P L c a n b e u s e d t o t e s t d e r i v a t i o n e n g i ne s o r p r o v i d e e x a m p l e s o f i n v a l i d V R M s , w h i c h c o u l d s e r v e a s a b a s i s t o e s t a b l i s h a n t i p a t t e r n s f o r d e v e l o p e r s . F o r t h i s p u r p o s e , w e h a v e f o r m a l i z e d t h e c o n c e p t s o f a n M S P L , b a s e d o n t h e C o mm o n V a r i a b i l i t y L a n g u a g e ( C V L ) , a s w e l l a s t h e c o n c e p t o f a c o u n t e r e x a m p l e . W e e x -

F i g u r e 4 . 1 : C u s t o m i z i n g t h e d e r i v a t i o n s e m a n t i c s o r i n c l u d i n g c h e c k i n g r u l e s e x i s t e n c e t h a t p o i n t s t o a s t a t e w i l l b e e x e c u t e

 41 if (s.isPointedByObjectExistence() && s.bindingVSpecResolution().getResolution() == false) then foreach s.getAllTransitions() t { if (t.isPointedByLinkExistence() && (t.bindingVSpecResolution.getResolution == false)) then return error } end !"#$$!"#$%&'()*+'%,&%!-!%&'()*!%./01!&')!2!345()%&678,38,'%)'22345()%&678,38,'%)'9!2!+),*-,$-*)--,.-1/)'!&'):;%&*+*8,9!'(&/--!&')!:!'8<%=8.%:/;;1+%0>:8?78,/0"#$%&':8#$%&'9! !@)<%>%A%,&%+1+%0>:8?78,/0"#$%&':8#$%&'B!&')d , i n c a s e i t w i l l ( i . e . , i t s b i n d i n g V S p e c i s n e g a t i v e l y d e c i d e d ) , t h e r e m u s t b e l i n k e x i s t e n c e s p o i n t e d t o a l l i t s i n c o m i n g a n d o u tg o i n g t r a n s i t i o n s , a n d t h e y s h a l l a l s o p o i n t t o n e g a t i v e l y d e c i d e d V S p e c s ( i . e . , f e a t u r e s c o n fi g u r e d t o f a l s e ) .

Listing 4 . 1 :

 41 Eval method header in all the VPs 1 operation eval(ctx:CVLExecutionContext::CVLExecutionContext):Void is abstract 2 }

Listing 4 . 2 :

 42 Excerpt of the ObjectExistence semantics 1 aspect class ObjectExistence { 2 method eval( ctx : CVLExecutionContext::CVLExecutionContext) : Void is do 3 if (not ctx.decision) then 4 ctx.toRemove.add(self.optionalObject.object)

Listing 4 . 3 :

 43 Excerpt of the ObjectExistence semantics with fix references procedure 3 class ObjectExistence { method eval( ctx : CVLExecutionContext::CVLExecutionContext) : Void is do 4 if (not ctx.decision) then Approaches to customize CVL's derivation semantics 73 5 ctx.toRemove.add(self.optionalObject.object) 6 fixReferences ( self .optionalObject.object, ctx)

Listing 4 . 4 : 2 } 4 . 5 :Figure 4 . 3 :

 4424543 Figure 4.3: Strategies Sequence Diagram

Figure 5 .

 5 Figure 5.1 shows an overview of the process of transforming an input program P . First, we use Spoon[START_REF] Pawlak | Spoon: Program analysis and transformation in java[END_REF] to extract P 's Abstract Syntax Tree (AST), which provides the 1 Sosie is a French noun that means "look alike" and it has been previously defined in[START_REF] Baudry | Tailored source code transformations to synthesize computationally diverse program variants[END_REF].

Listing 5 . 2 :

 52 Object Substitution generating a variant. //class com.codahale.metrics.Timer, line 101 public <T> T time(Callable<T> event) throws Exception { final long startTime = clock.getTick((clock.getTick() -startTime); com.codahale.metrics.ThreadLocalRandom. current().nextLong(); } }

Figure 5 . 2 :

 52 Figure 5.2: Results by categories of program elements for the 8 projects.

1 2 3 F

 3 i g u r e 5 . 3 : V i s u a l i z i n g t h e t r a n s f o r m a t i o n s . a r e r e a s o n a b l y a b o v e t h e a v e r a g e c o m p i l e % . F o r e x a m p l e , c h a n g i n g i f b l o c k s o r l o o p s h a s i n a v e r a g e 2 6 % a n d 3 4 % c h a n c e s t o g e n e r a t e s a f e p r o g r a m s , r e s p e c t i v e l y . 5.3.3.2 Safe and Unsafe Transformations T o t e s t H 2 a, w e r e f e r t o t a b l e 5 . 4 , s p e c i fi c a l l y t o i t s 1 2 l a s t l i n e s , i n w h i c h w e c a n fi n d 1 2 t r a n s f o r m a t i o n s t h a t h a v e n e v e r w o r k e d , t h e r e f o r e v a l i d a t i n g H 2 a. I n t h e s a m e w a y , w e v a l i d a t e H 2 b b y o b s e r v i n g t h e fi r s t 2 t r a n s f o r m a t i o n s i n t a b l e 5 . 4 t h a t h a v e a l w a y s g e n e r a t e d v a r i a n t s o r s o s i e s . Discussion. S o m e e l e m e n t s o f a J a v a p r o g r a m h a v e a n o p t i o n a l n a t u r e w i t h r e s p e c t t o c o r r e c t n e s s . F o r e x a m p l e , i f w e r e m o v e a " c o n t i n u e " f r o m a l o o p i t w i l l c o n t i n u e s y n t a c t i c a l l y c o r r e c t , a n d m a y b e e v e n s e m a n t i c a l l y , s i n c e a " c o n t i n u e " c a n b e u s e d f o r o p t i m i z a t i o n p u r p o s e s , n o t i m p l i c a t i n g c h a n g e s i n t h e l o o p s e m a n t i c s . O n t h e o t h e r h a n d , t h e r e a r e s o m e J a v a c o n s t r u c t s t h a t c a n b e c o n s i d e r e d a s m a n d a t o r y w i t h r e s p e c t t o o t h e r s a n d t h e r e f o r e m u s t b e h a n d l e d c a r e f u l l y . F o r e x a m p l e , w e k n o w t h a t a " t r y " b l o c k i s o f t e n f o l l o w e d b y a " c a t c h " o n e , t h e r e f o r e r e m o v i n g a ca t c h h a v e g r e a t c h a n c e s o f g i v i n g c o m p i l a t i o n e r r o r s ( s t i l l , t h e r e a r e c a s e s i n w h i c h i t c a n w o r k , s u c h a s w h e n a t r y h a s m o r e t h a n o n e c a t c h , t h e r e f o r e r e m o v i n g a c a t c h d o e s n o t l e a d t o c o m p i l a t i o n e r r o r ; h o w e v e r t h i s w a s n o t t h e c a s e i n a n y o f t h e s u b j e c t p r o g r a m s ) . I n t h e s a m e w a y , r e p l a c i n g a " fi e l d " b y a n o t h e r , w i l l r a i s e u p e r r o r s i n t h e r e s t o f t h e c o d e t h a t r e m a i n s u s i n g t h e o l d n a m e a n d t y p e o f t h e" fi e l d " . D e s p i t e o f t h e v a l i d a t i o n o f H 2 a, w e o b s e r v e t h a t t h e t r a n s f o r m a t i o n s t h a t n e v e r g e n e r a t e c o r r e c t p r o g r a m s a r e m i n o r i t y : 1 4 % ( 1 2 o u t o f 8 6 ) . T h i s i n d i c a t e s t h a t t h e r e a r e s e v e r a l p o s s i b i l i t i e s f o r v a r y i n g a J a v a p r o g r a m w i t h o u t c r a s h i n g i t . B e s i d e s , w e c a n t h i n k o f c o m b i n i n g t w o o r m o r e t r a n s f o r m a t i o n s s o t h e y c a n w o r k t o g e t h e r , l i k e i n t h e a f o r e m e n t i o n e d e x a m p l e ( r e m o v e a c a t c h t o g e t h e r w i t h i t s t r y ) .5.3.3.3 Safety of Transformations vs. Types of Operators and ProgramElements W e r e f e r t o t a b l e 5 . 3 t o v a l i d a t e H 3 a. W e c a n s e e t h a t o n l y a b o u t 9 % o f t h e t r a n s f o rm a t i o n s b a s e d o n O b j e c t S u b s t i t u t i o n h a v e s u c c e e d t o g e n e r a t e c o r r e c t p r o g r a m s , w h i l e the ones based on Object Existence had about 21% of success.

  h i s s e c t i o n , w e p r e s e n t t h e m a i n a c t i v i t i e s o f M S P L e n g i n e e r i n g t h a t r e l a t e t o o u r g e n e r a t i v e a p p r o a c h o f c o u n t e r e x a m p l e s . T h e g o a l o f t h e s e a c t i v i t i e s i s t o l e v e r a g e p r o d u c t l i n e e n g i n e e r i n g f o r a g i v e n D S L , a n d t h e y c o n c e r n b o t h r o l e s b e f o r e m e n t i o n e d . A s i l l u s t r a t e d i n F i g u r e 6 . 1 , i t s t a r t s w i t h a r e q u e s t i n s i d e t h e o r g a n i z a t i o n t o b u i l d a n M S P L f o r a g i v e n D S L . A f t e r , t h e e n g i n e e r s c a n c a r r y o n w i t h t h r e e m a i n p a r a l l e l t a s k s : t h e M S P L m o d e l i n g , t h e E n g i n e e r i n g o f v e r i fi c a t i o n m e c h a n i s m s a n d t h e g e n e r a t i o n a n d o r g a n i z a t i o n o f c o u n t e r e x a m p l e s ; t h e y a r e e x p l a i n e d i n t h e n e x t s u b s e c t i o n s t o g e t h e r w i t h t h e s h a r e d a c t i v i t y C o n s u l t c o u n t e r e x a m p l e s .

F

  i g u r e 6 . 1 : L e v e r a g e M S P L f o r a D S L6.2.1 Generate and organize counterexamples T h i s a c t i v i t y i s t h e b a s i s o f o u r m e t h o d o l o g y , i t p r o v i d e s a s o u t c o m e t h e m a t e r i a l t o h e l p d e s i g n e r s a n d i n f r a s t r u c t u r e e n g i n e e r s o f M S P L w i t h t h e i r a c t i v i t i e s o f b u i l d i n g s a f e r m o d e l s a n d t o o l s . T h e c o u n t e r e x a m p l e s g e n e r a t i o n p a r t i s e x p l a i n e d i n d e t a i l s i n C h a p t e r 3 ; i t f e e d s t h e c o u n t e r e x a m p l e s d a t a b a s e , a s s h o w n i n F i g u r e 6 . 2 . A f t e r g e n e r a t i n g t h e c o u n t e r e x a m p l e s , w e c a n o r g a n i z e t h e m t o b e b e t t e r e x p l o i t e d . T h e y c a n b e c l u s t e r e d a n d c a t e g o r i z e d t o r e p r e s e n t g r o u p s o f e r r o r s a n d t h e n s o r t e d a c c o r d i n g t o t h e i r s a f e t y o r o t he r c r i t e r i a . F o r e x a m p l e , i n T a b l e 5 . 4 o f C h a p t e r 5 , w e o r d e r e d t h e t y p e s o f c o u n t e r e x a m p l e s ( i n t h a t c a s e , t h e p a i r s o f v a r i a t i o n p o i n t + p r o g r a m e l e m e n t ) b y t h e i r p e r c e n t a g e o f c o m p i l a t i o n ; w h i l e i n F i g u r e 5 . 2 o f t h e s a m e c h a p t e r , w e c a t e g o r i z e t h e c o u n t e r e x a m p l e s b y t y p e o f p r o g r a m e l e m e n t s .

6. 2 . 2

 22 Consult counterexamples B e f o r e e x p l a i n i n g t h e t w o o t h e r m a i n a c t i v i t i e s , w e p r e s e n t t h e C o n s u l t c o u n t e r e x a m p l e s , w h i c h i s a n i m p o r t a n t t a s k s h a r e d b y b o t h . E v i d e n t l y , i t n e e d s t o b e p e r f o r m e d a f t e r

F

  i g u r e 6 . 2 : G e n e r a t e a n d o r g a n i z e c o u n t e r e x a m p l e s t h e p r e v i o u s a c t i v i t y o f g e n e r a t i n g c o u n t e r e x a m p l e s . T h e r e a r e s e v e r a l p o s s i b i l i t i e s t o e x p l o i t t h e g e n e r a t e d c o u n t e r e x a m p l e s ; w e c o n c e n t r a t e o n f o u r b a s i c s u b a c t i v i t i e s t h a t c a n s e r v e b o t h t o t h e M S P L D e s i g n e r a n d t h e I n f r a s t r u c t u r e E n g i n e e r , a s s h o w n i n F i g u r e 6 . 3 . T h e c h e c k u n s a f e a s s e t s a c t i v i t y c o n s i s t s o n e v a l u a t i n g w h e t h e r a b a s e m o d e l e l e m e n t i s i n t r i n s i c a l l y d a n g e r o u s t o b e m o d i fi e d o r n o t ; i n C h a p t e r 5 , w e h a v e i d e n t i fi e d J a v a c o n s t r u c t s t h a t w e r e v e r y l i k e l y t o l e a d t o e r r o r s w h e n m o d i fi e d ( e . g . , r e a d s t a t e m e n t s ) . I n c r e m e n t a l l y , o n e c a n a l s o a cc e s s t h e s a f e t y o f t h e a s s e t s a s s o c i a t e d w i t h a v a r i a t i o n p o i n t ( c h e c k u n s a f e V P + a s s e t s a c t i v i t y ) a n d t o g e t h e r w i t h a c o n fi g u r a t i o n o f t h e v a r i a b i l i t y m o d e l ( c h e c k u n s a f e c o n fi g u r a t i o n + V P + a s s e t s ) . T h e b e f o r e m e n t i o n e d a c t i v i t i e s r e l y m a i n l y o n c o m p a r i n g t h e c o u n t e r e x a m p l e s w i t h a c a n d i d a t e M SP L d e s i g n , i n a n a p p r o x i m a t e d w a y , l i k e c o m p a r i n g t y p e s / c a t e g o r i e s o f e r r o r s . W h e r e a s i n t h e M a t c h e x i s t i n g c o u n t e r e x a m p l e s a c t i v i t y , t h e g o a l i s t o a c t u a l l y u s e c o u n t e r e x a m p l e s a s t e s t i n g m o d e l s , t h i s i s p a r t i c u l a r l y u s e f u l i n s m a l l e r p r o b l e m s p a c e s . T h e s e a c t i v i t i e s c a n s e r v e a s b a s i s f o r m o r e s o p h i s t i c a t e d o n e s , l i k e c o d e r e c o mm e n d a t i o n a n d s t a t i c a n a l y s i s o f a r t i f a c t s .

6. 2 . 3

 23 MSPL modelingM o d e l i n g a n M S P L u s i n g C V L i s e s s e n t i a l l y c o n s t r u c t i n g i t s t h r e e m a i n m o d e l s : B a s e , V a r i a b i l i t y a n d R e a l i z a t i o n m o d e l s . I n F i g u r e 6 . 4 , w e i l l u s t r at e t h e s e t h r e e m a i n a c t i v it i e s , p r e c e d e d b y t h e u s e r t a s k a c q u i r e a v a i l a b l e m o d e l s a n d , d e p e n d i n g o n t h e e x i s t i n g m o d e l s , t h e o t h e r c o n s t r u c t i o n a c t i v i t i e s a r e p e r f o r m e d o r j u s t p a s s e d . T h e M S P L D es i g n e r i s t h e r o l e r e s p o n s i b l e f o r h a n d l i n g t h e s e a c t i v i t i e s . T h e t h r e e m o d e l c o n s t r u c t i o n a c t i v i t i e s o fF i g u r e 6 . 4 c a n b e e x p a n d e d i n s u b t a s k s ; i n t h e n e x t s e c t i o n , w e p r e s e n t t h e C o n s t r u c t r e a l i z a t i o n m o d e l e x p a n d e d , a s i t i s t h e m a i n a c t i v i t y t h a t c a n b e n e fi t f r o m o u r a p p r o a c h o f c o u n t e r e x a m p l e s g e n e r a t i o n .

F

  i g u r e 6 . 3 : C o n s u l t i n g c o u n t e r e x a m p l e s F i g u r e 6 . 4 : A c t i v i t i e s t o d e s i g n a n M S P L Construct realization model T h e r e a l i z a t i o n m o d e l i s t h e s e t o f m a p p i n g s b e t w e e n t h e f e a t u r e s i n t h e v a r i a b i l i t y m o d e l a n d t h e e l e m e n t s o f t h e b a s e m o d e l s ; t h e r e f o r e , i t a s s u m e s t h a t t h e s e m o d e l s e x i s t a n d c a n b e r e f e r e n c e d . F i g u r e 6 . 5 i l l u s t r a t e s t h e m o d e l i n g p r o c e s s o f a r e a l i z a t i o n m o d e l . B e f o r e c h o o s i n g t h e m a p p i n g r e l a t i o n s h i p i t s e l f , t h e M S P L D e s i g n e r h a s t o c h o o s e t h e c o n c e r n e d f e a t u r e ( o r s e t o f f e a t u r e s , o r a f e a t u r e e x p r e s s i on ) . A f t e r , t h e d e s i g n e r h a s t o i d e n t i f y t h e a s s e t s ( b a s e m o d e l e l e m e n t s ) t h a t c o r r e s p o n d t o t h e c h o s e n f e a t u r e a n d t h e n t o p i c k t h e d e s i r e d v a r i a t i o n p o i n t t o l i n k t h e m . I n p a r a l l e l , o r r i g h t a f t e r t h e s e t h r e e s t e p s :C h o o s e f e a t u r e ( s ) , C h o o s e c o r r e s p o n d i n g a s s e t ( s ) a n d C h o o s e v a r i a t i o n p o i n t ( s ) , t h e M S P L D e s i g n e r c a n c o n s u l t c o u n t e r e x a m p l e s t o a s s e s s t h e s a f e t y o f t h e c a n d i d a t e M S P L d e s i g n . I f h e / s h e c o n s i d e r s t h e d e s i g n t o b e s a f e -a u t o m a t i c a l l y o r m a n u a l l y c h e c k i n g i f t h e c a n d i d a t e d e s i g n i s s i m i l a r o r e q u a l t o t h o s e o f s o m e c o u n t e r e x a m p l e st h e p r o c e s s i s r e p e a t e d f o r o t h e r m a p p i n g s o r fi n i s h e d ; e l s e , t h e m o d e l m u s t b e c o r r e c t e d .

F i g u r e 6 . 5 :

 65 A c t i v i t i e s t o c o n s t r u c t t h e r e a l i z a t i o n m o d e l6.2.4 Engineer verification mechanismsA t a d i ffe r e n t l e v e l o f t h e M S P L D e s i g n e r , t h e M S P L I n f r a s t r u c t u r e E n g i n e e r h a s t o d e a l w i t h t a s k s t h a t w i l l s u p p o r t t h e m o d e l i n g a c t i v i t i e s p r e v i o u s l y p r e s e n t e d . W e c h o o s e t o p r e s e n t t h e a c t i v i t y o f e n g i n e e r i n g v e r i fi c a t i o n m e c h a n i s m s ( s e e F i g u r e 6 . 6 ) , i n w h i c h w e c o n s i d e r t h a t o u r c o u n t e r e x a m p l e s a p p r o a c h c a n b e u s e f u l . T h e i n f r a s t r u c t u r e e n g i n e e r c a n d e c i d e e i t h e r t o c o n s t r u c t a d o m a i n s p e c i fic c h e c k e r o r t o c u s t o m i z e t h e d e r i v a t i o n e n g i n e , o r b o t h , a s t h e y a r e n o t m u t u a l l y e x c l u s i v e ; w e p r e s e n t n e x t t h e s e t w o p o s s i b l e a c t i v i t i e s e x p a n d e d .

F

  i g u r e 6 . 6 : I n f r a s t r u c t u r e m e c h a n i s m s f o r M S P L v e r i fi c a t i o n 6.2.4.1 Construct domain specific checker O f t e n , p r o d u c t l i n e e n g i n e e r s s e e k t o h a v e c h e c k i n g m e c h a n i s m s t o v e r i f y t h e i r d e s i g n c h o i c e s a t d e s i g n t i m e , a v o i d i n g i n fi r s t -h a n d b a d d e s i g n s . W e c a n i m a g i n e m e c h a n i s m s s i m i l a rt o s y n t a x h i g h l i g h t i n g i n p r o g r a m i n g l a n g u a g e s e n v i r o n m e n t s t o r e v e a l t h e s e e r r o r s , b e f o r e d e r i v a t i o n ( c o m p i l a t i o n i s t h e a n a l o g y ) . H o w e v e r , t h e y a r e o f t e n l i m i t e d t o s y n t a x c h e c k i n g , a s i t i s e a s i e r t o b e s t a t i c a l l y c h e c k e d u s i n g t y p e c h e c k i n g m e c h a n i s m s . T h i s a c t i v i t y i s i l l u s t r a t e d i n F i g u r e 6 . 7 a n d s t a r t s w i t h c o n s u l t i n g t h e c o u n t e r e xa m p l e s i n o r d e r t o a n a l y s e t h e u n s a f e d e s i g n s t h a t c a n s e r v e a s a n t i p a t t e r n . A f t e r , t h e e n g i n e e r c a n e n c o d e v e r i fi c a t i o n r u l e s i n a c h o s e n f o r m a l i s m t o d e t e c t a u t o m a t i c a l l y t h eu n s a f e d e s i g n s . F o r e x a m p l e , i n S e c t i o n 3 . 3 . 4 o f C h a p t e r 3 , w e e n c o d e d t w o r u l e s f o r t h e F i n i t e -S t a t e M a c h i n e d o m a i n . F i n a l l y , t h e e n g i n e e r c a n c h o o s e t o i n c o r p o r a t e t h e s e r u l e s i n a c h e c k i n g m e c h a n i s m , d e c i d i n g w h e t h e r t o u s e t h e m a t d e s i g n t i m e o r a f t e r , i n c r e m e n t a l l y o r a t o n c e .

F 9 6. 2 . 4 . 2

 9242 i g u r e 6 . 7 : A c t i v i t i e s t o c o n s t r u c t a c h e c k i n g m e c h a n i s m f o r a M S P L Conclusion 9 Customize derivation engine T h e i n f r a s t r u c t u r e e n g i n e e r c a n d e c i d e , b a s e d o n t h e c o u n t e r e x a m p l e s , t h a t t h e d e r i v at i o n e n g i n e m u s t b e s p e c i a l i z e d f o r a g i v e n d o m a i n . T h i s c a s e i s i l l u s t r a t e d i n F i g u r e 6 . 8a n d i t i s d e t a i l e d i n C h a p t e r 4 . L i k e i n t h e c o n s t r u c t i o n o f a c h e c k e r , t h e fi r s t t a s k i s t o c o n s u l t t h e c o u n t e r e x a m p l e s a n d t h e n e x t r a c t t h e d o m a i n s p e c i fi c r u l e s , h o w e v e r , t h e e n g i n e e r d o e s n o t n e c e s s a r i l y n e e d t o e n c o d e t h e r u l e s i n a f o r m a l i s m . T h e r u l e s w i l l s e r v e a s b a s i s t o s p e c i a l i z e t h e e x i s t i n g s e m a n t i c s o f t h e v a r i a t i o n p o i n t s . I n t h e s i t u a t i o n s t h a t v a r i a t i o n p o i n t s r e s u l t e d i n c o u n t e r e x a m p l e s , t h e e n g i n e e r h a s t o t h i n k h o w t h es e m a n t i c s c o u l d b e e x t e n d e d t o a v o i d t h e e r r o r s . F o r e x a m p l e , w e s h o w e d i n S e c t i o n 4 . 2 . 1 . 2 o f C h a p t e r 4 c u s t o m i z a t i o n s f o r t h e O b j e c t E x i s t e n c e v a r i a t i o n p o i n t t h a t c o u l d r e d u c e e r r o r s d u e t o d a n g l i n g r e f e r e n c e s .

F

  i g u r e 6 . 8 : A c t i v i t i e s t o c u s t o m i z e t h e d e r i v a t i o n e n g i n e6.3 Conclusion I n t h i s c h a p t e r , w e h a v e p r e s e n t e d a m e t h o d o l o g y t o l e v e r a g e M S P L e n g i n e e r i n g f o r a g i v e n D S L , d e fi n i n g t h e r o l e s a n d a c t i v i t i e s i n v o l v e d u s i n g B P M N . T h e m e t h o d o lo g y s e r v e s t o g u i d e t h e o r g a n i z a t i o n s t h a t w a n t t o u s e o u r a p p r o a c h t o s u p p o r t t h e i r a c t i v i t i e s . O u r c o u n t e r e x a m p l e s g e n e r a t i o n a p p r o a c h c a n b e i n t e g r a t e d a s a p a r a l l e l a c t i v i t y t o b o t h t h e M S P L m o d e l i n g a n d i t s i n f r a s t r u c t u r e e n g i n e e r i n g , e m p h a s i z i n g i t sn o n -i n t r u s i v e n a t u r e .

decision name description type Range cardinality/constraint visible/relevant if

		(a) Feature model in a tree notation-slightly adapted from FODA [42]
	GSM_Proto-	Support GSM 1900 protocol?	Boolean true | false
	col_1900
	Audio_Formats	Which audio formats shall be	Enum	WAV | MP3	1:2
		supported?
	Camera	Support for taking photos?	Boolean true | false
	Camera_Resolu-	Required camera resolution?	Enum	2.1MP | 3.1MP | 5MP	1:1	Camera == true
	tion
	MP3_Recording	Support for recording MP3 audio?	Boolean true | false	ifSelected Audio_For-
					mats.MP3 = true
		.1, a feature model (b) Decision model in a tabular notation [59, 28]
	GSM_Protocol_1900: one of (GSM_1900, NO_GSM_1900)	{indicates whether support for making and receiving calls using
				GSM 1900 is available}
	Audio: list of (WAV, MP3)		{indicates the types of supported audio formats}
	Camera: composed of
	Presence: one of (Camera, NO_Camera)		{indicates whether camera support is available}
	Resolution: one of (2.1MP, 3.1MP, 5MP)		{resolution of the camera}
	MP3_Recording: one of (MP3, NO_MP3)		{indicates whether MP3 recording is available}
	Constraints
	Resolution is available only if Presence has the value Camera
	MP3_Recording requires that also MP3 Audio is supported
		(c) Decision model in the textual notation of Synthesis [64]
	Figure 1: A feature model and two decision models for a fictitious mobile phone product line; same variability, while commonalities are only
	shown in the feature model.
	modeling in service-oriented systems [34], to use DM with model-
	driven architectures [31], to use DM for code generation [70], and
	to support personalization of ERP software by end-users [53].
	FM has targeted a broader set of roles in the development life cy-
	cle, starting with the original application in FODA-domain anal-
	ysis and scoping, but also including design and representation of
	product line architectures and evolution-helping to see what fea-
	tures are available, which new features should be added and where,
	and which existing features might need to be retired.

Table 2 .

 2

		1: Comparison of variability modeling approaches
	Approach/	Tool	MDE	Derivation	Dimensions
	Tool	Support	Assets
	Pure	I	Y	A , Ba n dC	M
	Gears	I	Y	A , Ba n dC	S
	FeatureIDE	A	N	Aa n dB	S
	FMP	A	N	Aa n dB	S
	FMT	A	Y	Aa n dB	S
	VELVET	-	N	-	M
	RequiLine	B	N	A and B	S
	XFeature	B	Y	A	S
	CVMTool	B	N	A and B	M
	CVL	B	Y	A,B and C	M
	FAMILIAR	A	N	A	M
	FORM	A	N	Aa n dB	S
	COVAMOF	B	N	A and B	M
	Comper	A	Y	A	M
	Invar	A	N	A	M
	Captain	A	N	A	S
	ToolDAy	B	N	A, B and C	S

study how we ensure the consistency of the variability model when it crosscuts several phases of the system development process. It is necessary to notice that this capability does not ensure that the approach consistently manages variability across all dimensions, but it gives initial modularization.

  [ALHM + 11], Alfeérez et al. applied VCC4RE (for Variability Consistency Checker for Requirements) to verify the relationships between a feature model and a set of use scenarios. Zhang et al. [ZMP12] developed a simulator for deriving product models as well as a consistency checker. Svendsen et al. present an approach for automatically generating a testing oracle for train stations expressed in CVL [SHMP11].

  ).getReference()

	8 9	if (( f .getFinalState() .contains(p) && f. getInitialState () .equals(p1)) || ( f .getFinalState() . contains(p1) && f. getInitialState () .equals(p))) return false ; } / * Rule 2: Pointing an ObjectExistence to an initial state is forbidden. * / else if (e.isInstanceOf[ObjectExistence]){ e.asInstanceOf[ObjectExistence].getOptionalObject().foreach(p=> {if (f. getInitialState () . equals(p.getReference())) return false ;})}}) return true}

  Listing 5.3: Object Substitution generating a sosie.

	"m5_rate", filter, rateUnit, durationUnit);
	this.directory = directory;
	this.locale = locale;
	this.clock = clock;
	}
	//class com.codahale.metrics.CsvReporter, line 135
	private CsvReporter(MetricRegistry registry,
	File directory,
	Locale locale,
	TimeUnit rateUnit,
	TimeUnit durationUnit,
	Clock clock,
	//substitution
	MetricFilter filter) {
	super(registry, /** nodeType: class
	spoon.support.reflect.code.CtLiteralImpl
	"csv-reporter" **/

  5 -we seek to achieve a reasonable statistical relevance.Data: P , a program to transformResult: values for the dependent variables of Table5.2.3 1 V P ={the four kinds of V P } 2 E={elements in the AST of P } 3 while resources_available do

	4 5 6 7 8	randomly select vp 2 V P randomly select compatible e 2 E P 0 apply T hvp, ei to P if compile(P') = true then if test(P') = true then
	9 10	store P 0 as a sosie else
	11 12	store P 0 as a variant end
	13	else
	14 15	store P 0 as a counterexample end
	16 end Algorithm 1:

Table 5 . 3 :

 53 Results for the operators.

		candidate	trial	%trial margin of error compile compile% sosie sosie%
	Link Existence	11248	7247	64.43	0.70	856	11.81	539	7.44
	Link Substitution	14869609	85459	0.57	0.40	3851	4.51	3572	4.18
	Object Existence	626258	79913	12.76	0.30	17559	21.97	6994	8.75
	Object Substitution 14706362886 203566 <0.01	0.20	18776	9.22	12127	5.96
	Total	14721870001 376185 <0.01	0.20	41042	10.91	23232	6.18

Table 5 .

 5 4: Global results for the 15 first and the 15 last transformations ordered by compilation %.

	operator	AST element candidate	trial %trial compile compile% sosie sosie%
	OE	AnnotationType	46	23	50.00	23	100.00	19	82.61
	OE	Continue	124	31	25.00	31	100.00	9	29.03
	OE	ForEach	888	330	37.16	325	98.48	47	14.24
	OS	SuperAccess	60348	189	0.31	186	98.41	183	96.83
	OS	ThisAccess	5790636	2282	0.04	2203	96.54	2006 87.91
	OE	SuperAccess	456	86	18.86	83	96.51	24	27.91
	OE	While	609	95	15.60	88	92.63	18	18.95
	OE	For	3461	251	7.25	230	91.63	60	23.90
	OE	Break	1008	121	12.00	110	90.91	74	61.16
	OE OperatorAssignment	1825	153	8.38	137	89.54	53	34.64
	OE	If	12859	2175	16.91	1851	85.10	587	26.99
	OE	Annotation	3802	903	23.75	699	77.41	655	72.54
	OE	Throw	3092	523	16.91	370	70.75	150	28.68
	OS	Annotation	3150678	1591	0.05	1019	64.05	980	61.60
	OE	Synchronized	95	27	28.42	16	59.26	1	3.70
	. . .	. . .	. . .	. . .	. . .	. . .	. . .	. . .	. . .
	OS	Parameter 172555379 13594	0.01	12	0.09	12	0.09
	OE	Method	18906	3998	21.15	3	0.08	3	0.08
	OE	Parameter	28701	4494	15.66	2	0.04	1	0.02
	OE	Catch	602	218	36.21	0	0.00	0	0.00
	OE	Class	2477	309	12.47	0	0.00	0	0.00
	OE	Enum	61	4	6.56	0	0.00	0	0.00
	OE	Interface	671	52	7.75	0	0.00	0	0.00
	OS	Break	409674	192	0.05	0	0.00	0	0.00
	OS	Case	2035772	530	0.03	0	0.00	0	0.00
	OS	Catch	51158	650	1.27	0	0.00	0	0.00
	OS	Continue	4554	27	0.59	0	0.00	0	0.00
	OS	Do	916	8	0.87	0	0.00	0	0.00
	OS	Field 13019255	3639	0.03	0	0.00	0	0.00
	OS	LocalVariable 162483228 6604	0.00	0	0.00	0	0.00
	OS	Throw	2361268	1270	0.05	0	0.00	0	0.00
	visualization tool we built to achieve this task. First, we provide a global view of the
	input program by packages (see 1 ), within each package we have the classes, which are
	represented as long rectangles with colored lines inside. It is possible to click on those
	rectangles to zoom in the classes (see 2 )

Table 5 .

 5

		2		µ ME
	if	47.44	6.89 26.32 0.01
	loop	54.90	7.41 34.16 0.02
	invocation	13.31	3.65 12.29 0.00
	read	2.22	1.49	6.57 0.00
	write	41.50	6.44 24.42 0.01
	new 111.49 10.56 33.90 0.01
	exception	40.81	6.39 16.62 0.02

5: Variance( 2 ), standard deviation( ), mean(µ) and margin of error (ME) for the compilation% of the 7 groups of Figure

5

.2.

http://www.wired.com/2012/12/automotive-os-war/

http://www.thalesgroup.com/

According to[START_REF] Whittle | The state of practice in model-driven engineering[END_REF], DSLs can be used extensively and in a "quick and dirty" way, with DSLs and their generators being developed in as little as two weeks.

http://www.omgwiki.org/variability

http://www.omg.org/mof/

CVL uses the term materialization to refer to the derivation of a model. Also, a selected/unselected feature corresponds to a positively/negatively decided VSpec. We adopt the well-known vocabulary of SPLE for the sake of understandability.

The Meta-Object Facility (MOF) is an OMG standard for modeling technologies. For instance, the Eclipse Modeling Framework is more or less aligned to OMG's MOF.

realization layer in the current CVL specification

https://github.com/diverse-project/kcvl

Using metamodel as synonym of abstract syntax is one definition in the community. For some researchers, "metamodel" is sometimes referred to abstract syntax plus static semantics.

www.omgwiki.org/variability/

http://groovy.codehaus.org/

http://metrics.codahale.com/

The complete list of program elements can be found in the Spoon API: http://spoon.gforge.inria.fr/mvnsites/spoon-core/apidocs/index.html

CPU: Intel Xeon Processor W3540 (4 core, 2.93 GHz), RAM: 6GB

example, when selecting a class in a model, a slicer is able to return all the other classes associated to the selected one.

The main drawback of this extension mechanism is the opacity of the OVP itself. No CVL checker can ensure the correctness of the variability model and it becomes complex to understand the expected behaviour of a variability realization model. 4.1 shows a comparison of the three extension mechanisms provided in our CVL implementation to support the customization of the CVL semantics for a domain model. We could observe that the second mechanism is generally the best to specialize the CVL semantics for a specific metamodel. Indeed, it does not change the CVL semantics but it only refines the semantics of removing, adding or substituting an element. Opaque Variation Point is often useful even if we loose the ability to understand the materialization and therefore of analyzing the CVL realization model. Besides, it is currently missing in CVL the notion of Opaque Variation Type to ease the reuse of Experimenting CVL variation points with Java program constructs experiment. The number of non-compilable, compilable and programs with preserved behaviour is dependent on the operator and the program element 3 used in the program transformation. We perform the experiment for a controlled set of 8 input programs.

Synthesis

Hypotheses

Following, we enumerate the hypotheses we want to test against the results of our experiment, also explaining their respective motivations. They were categorized within three types, regarding: the overall safety of CVL applied to Java code (H 1 ), the safety of specific transformations (H 2 ), and the safety of transformations with respect to the types of operators and program elements (H 3 ).

• H 1 : It is easier to randomly produce incorrect programs than correct ones.

Testing H 1 : is the first lead to understand that the derivation operators are prone to generate wrong products; a user without the necessary knowledge of the domain language (imitated by the random nature of the transformations) is more likely to design wrong product lines than correct ones.

• H 2 a : There are transformations that will always lead to counterexamples (There exists T = hO, Ei that has a counterexample percentage equal to 100%).

If validated, H 2 a can be the basis for identifying transformations to be always avoided; we can imagine for instance using these transformations as antipatterns to detect errors at design time.

• H 2 b : There are transformations that will always lead to variants or sosies ( There is T = hO, Ei that has a variant percentage or sosies percentage of 100%).

In the same way of H 2 a, H 2 b can help to collect "good" transformations and perhaps be the basis of recommendations to the designer.

• H 3 a : Object Substitution is more prone to generate wrong programs than Object Existence (the counterexample percentage for transformations with O =

Perspectives

In this section, we present some long-and short-term ideas for research around the contributions of this thesis.

Advanced use of counterexamples

This thesis strongly advanced on the counterexamples identification and generation; we also showed initial ways to exploit these valuable artifacts. However, we believe that the counterexamples have an enormous potential still to be explored. We envision that they can feed more sophisticated mechanisms, like Recommender Systems [START_REF] Resnick | Recommender systems[END_REF], Expert Systems [START_REF] Joseph | Expert systems[END_REF] or Machine Learning approaches that identify, correct or forecast possible antipatterns.

Intelligent IDEs for MSPL Engineering

Concretely, these advanced techniques could be integrated into more Intelligent IDEs for constructing software and systems product lines. Nevertheless, there is still a lot to improve on product line adoption, and one of the reasons is because variability is not an evident concern in today's IDEs. Some works, based on feature-oriented software development, try to address this issue, by bringing the preoccupation of features into the code. However, we are still far from integrated development environments that can handle the full product line life cycle. The ideal IDE should facilitate the seamless integration among variability models and any other artifacts of the organization (textual requirements, code, models, etc.), in a way that a valid configuration would always lead to a safe combination of any artifacts -The Safe Derivation of Anything.

To achieve The Safe Derivation of Anything, variability information must be correctly linked to the artifacts, and the semantics of these links should be fitted to each kind of artifact (so far, what we have been advocating in this thesis). We do not believe that an universal product line language, with a common algebra for combining assets, would do the job; new languages and new kinds of artifacts are always being incorporated into complex systems development. We believe that our counterexamples generation approach is a first step to overcome this, opening avenues to even more automated approaches than the one presented in this thesis.

When designing an SPL in this IDE, the engineer would be helped on the definition of features and realization links. For example, if existing source code or models were available, recommenders would suggest features corresponding to parts of code or model elements responsible to a functionality of the system. The link would be made automatically, considering all the other artifacts that are related to that feature and the way they are related (semantics of the relationship). When configuring a product, the List of Figures

Abstract

Systems Engineering is a complex and expensive activity in several kinds of companies, it imposes stakeholders to deal with massive pieces of software and their integration with several hardware components. To ease the development of such systems, engineers adopt a divide and conquer approach: each concern of the system is engineered separately, with several domain specific languages (DSL) and stakeholders. These languages are built within a set of dedicated representations to analyze an area of expertise, and the current practice is to rely on the Model-driven Engineering (MDE) paradigm to construct them.

On the other hand, systems engineering companies also need to construct slightly different versions/variants of a same system; these variants share commonalities and variabilities that can be managed using a Software Product Line (SPL) approach. A promising approach is to ally MDE with SPL -Model-based SPLs (MSPL) -in a way that the products of the SPL are expressed as models conforming to a metamodel and well-formedness rules. The Common Variability Language (CVL) has recently emerged as an effort to standardize and promote MSPLs; it is our adopted language for constructing MSPL.

Engineering an MSPL is extremely complex to an engineer: the number of possible products is exponential; the derived product models have to conform to numerous wellformedness and business rules; and the realization model that connects a variability model and a set of design models can be very expressive specially in the case of CVL. Managing variability models and design models is a non-trivial activity. Connecting both parts and therefore managing all the models is a daunting and error-prone task. Added to these challenges, we have the multiple different modeling languages of systems engineering. Each time a new modeling language is used for developing an MSPL, the realization layer should be revised accordingly.

The main objective of this thesis is to assist the engineering of MSPLs in the systems engineering field, considering the need to support it as earlier as possible and without compromising the existing development process. To achieve this, we provide a systematic and automated process, based on CVL, to randomly search the space of MSPLs for a given language, generating counterexamples that can server as antipatterns. We then provide ways to specialize CVL's realization layer (and derivation engine) based on the knowledge acquired from the counterexamples.

We validate our approach with four modeling languages, being one acquired from industry; the approach generates counterexamples efficiently, and we could make initial progress to increase the safety of the MSPL mechanisms for those languages, by implementing antipattern detection rules. Besides, we also analyse big Java programs, assessing the adequacy of CVL to deal with complex languages; it is also a first step to assess qualitatively the counterexamples. Finally, we provide a methodology to define the processes and roles to leverage MSPL engineering for new DSLs in an organization.