Professor, M Alain Abran

Full Professor, M Alexander Verbraeck

Donna H Mme

Rhodes

Professeur, ISAE-SUPAERO M Rob Vingerhoeds

Examinateur Karama Mme

Kanoun

Assistant Professor, M Silverio Martinez-Fernandez

Mme Claude

M Philippe

Currently employee of the young Renault subsidiary, Renault Software Lab (formerly INTEL), located in Toulouse and focused on onboard software development, I seized a rare opportunity that is offered to me by this situation to achieve, with the downstream from my hierarchy, an important development in my career, namely to move from the managerial sector to the expertise one. Within this new entity, I must indeed evolve to become the specialist on Qualimetry. The objective is a scientific and technical challenge, and also constitutes a strategic industrial issue; it is to be able to quantitatively assess the quality of software development, from requirements, through models, code, to the maturity of the software product. Certainly, despite the existing models and quality measurement solutions, the technical optimization and correction activities linked to the integration and qualification, then acceptance and maintenance phases still represent 65% of the total cost of a project; nonquality costs 5% of turnover (source AFQP 2017 [1]). Therefore, it is essential for the industry to be able to invest in research on this subject in order to improve control, productivity, and quality not only for developers but also for the entire organization. The evaluations to be carried out thus relate to the product, here the embedded software, but also to the development processes of this software whose performance is to be estimated. Completing a doctoral thesis on these subjects will allow me to consolidate and develop my expertise, to establish my legitimacy as an expert and to promote this scientific field within the company. The ISI team at LAAS specializes in product-process-project modeling and is already conducting research in the field of software qualimetry, which is why I naturally built my research project with this team.

The Qualimetry science introduced by G. G. Azgaldov in 1968 [2], is an exciting multidisciplinary scientific field that combines technical and non-technical aspects, quantification through numerous metrics, analyzes and decisions thanks to statistics and data mining, methods, processes, etc. Scientific literature confirms that the subject is vast and indicates to us the need to develop an optimized solution for the embedded software, allowing to connect the two poles: adhesion and evaluation. In addition, this field links a variety of roles and responsibilities which are in line with my background and professional experience because I have been able to take on the role of developer, developer manager, program manager, test architect and validation team manager.

"Dubium sapientiae initium" (Doubt is the origin of wisdom) -René Descartes, Source: Meditations on First Philosophy "One thing I have learned in a long life: that all our science, measured against reality, is primitive and childlike -and yet it is the most precious thing we have" -Albert Einstein, Source: American Ins tute of Physics Our research specifically addresses automotive systems (or software, depending on the choices made above), and in the context of its development, we therefore focus on the quality of embedded software in automotive vehicles.

Following an exploratory study of the literature in the field of quality models for embedded software, it is clear that there is an abundance of these models, but that there is no unified and operational solution that currently meets our needs. Our problematic of applying qualimetry essentials to the development of embedded software is therefore shifting towards reinforcement and unification of the activities to define, evaluate, control, and predict the quality of automotive embedded software.

To deal with it, we first explore the concepts of quality and qualimetry -the science of quality quantification -and establish a state of the art of quality modeling for software, including embedded software. The result of this study allows to synthesize the knowledge behind these complex concepts. It also makes it possible to conclude on the choice of qualimetry as the most relevant approach for this reinforcement and unification of the activities related to the quality modeling of embedded software. This conclusion is motivated not only by both theoretical and applied treatment of quality quantification, but also by the existence of a scientific community (e.g. civil engineers, economists, car manufacturers, architects) actively contributing to qualimetry.

Then, having noticed the existence of a similarity between the forms of evolutions, or adaptations, in quality models and in biology, during our exploration of quality modeling, we continue our study by rightly considering biology as a source of inspiration for solution guidance in our research.

We thus create a classified collection of more than 450 quality models for the software, which then shifts our problematic towards the choice of a quality model for embedded software in cars among this plethora of models. To make this model selection, which we will call the reference model, we analyze and take into account the constraints coming from standards, regulations, the automotive industry and stakeholders. Then, and after having introduced the concept of polymorphism in quality modeling as a built-in evolution and adaptation mechanism of quality models, the answer we bring to the problematic is to define and demonstrate concretely a methodology for adapting and operationalizing this reference model for embedded software in automotive vehicles.

Finally, as a consolidation of this answer, we determine a unified form of the reference quality model not, as suggested by Zouheyr Tamrabet et al [6], as a single quality model for software products, but as a meta-model serving as an aggregator of quality models for software products. The end result is the proposal of the genome of the software quality model that we model in the shape of 7 chromatids.

Résumé

Aujourd'hui, lorsqu'une entreprise conçoit, développe et fabrique des biens ou des services, elle doit non seulement viser un niveau de qualité élevé pour que ses produits satisfassent les clients, mais également se conformer à de nombreuses normes et réglementations. C'est particulièrement vrai pour les systèmes de transport pour lesquels nous pouvons citer quelques normes et directives de référence : la norme ISO 26262 [3] traite de la sécurité fonctionnelle des véhicules routiers et couvre le système, le matériel et le logiciel, l'ARP4754 [4] fournit des directives pour le développement des avions civils, et la DO-178C traite de la sécurité des logiciels [5] dans l'aéronautique. En outre, ces directives de sécurité imposent à l'entreprise d'être à la pointe de la technologie en matière de processus et de méthodes, lors de la conception et du développement d'un nouveau véhicule. Aussi, pour s'assurer d'un niveau de qualité élevé tout en étant en conformité avec ces normes et réglementations, il est nécessaire que toutes les exigences de qualité qui en découlent soient traduites au sein d'un modèle qualité. Ce modèle est la clef de voûte de la définition, de l'évaluation, du contrôle, et même de la prédiction de la qualité du système.

Nos recherches adressent spécifiquement les systèmes (ou logiciels, selon les choix faits plus haut) automobiles, et dans le contexte de leurs développements, nous nous concentrons donc sur la qualité des logiciels embarqués dans les véhicules automobiles.

À la suite d'une étude exploratoire de la littérature des modèles qualité pour le logiciel embarqué, force est de constater l'abondance de ces modèles, sans qu'il n'existe aucune solution unifiée et opérationnelle répondant actuellement à notre besoin. Notre problématique d'appliquer les caractères essentiels de la qualimétrie au développement du logiciel embarqué se déporte ainsi vers le renforcement et l'unification des activités de définition, d'évaluation, de contrôle et de prédiction de la qualité des logiciels embarqués dans les véhicules automobiles.

Pour la traiter, nous explorons en premier lieu les concepts de qualité et de qualimétrie -la science de la quantification de la qualité -, et établissons un état de l'art de la modélisation de la qualité pour le logiciel, incluant le logiciel embarqué. Le résultat de cette étude permet de synthétiser les connaissances qui se cachent derrière ces concepts complexes. Il permet aussi de conclure sur le choix de la qualimétrie comme l'approche la plus pertinente pour ce renforcement et unification des activités liées à la modélisation de la qualité des logiciels embarqués. Cette conclusion est motivée non seulement par un traitement à la fois théorique et appliquée de la quantification de la qualité, mais aussi par l'existence d'une communauté scientifique (par exemple, ingénieurs civiles, économistes, constructeurs automobiles, architectes) contribuant activement à la qualimétrie.

Puis, après avoir remarqué l'existence d'une similitude entre les formes d'évolutions, ou d'adaptations, dans les modèles qualité et dans la biologie, durant notre exploration de la modélisation de la qualité, nous poursuivons notre étude en considérant à juste titre la biologie comme source d'inspiration pour l'orientation de solutions dans notre recherche.

Nous créons ainsi une collection classifiée de plus de 450 modèles qualité pour le logiciel, ce qui déplace alors notre problématique vers le choix d'un modèle qualité pour le logiciel embarqué dans les véhicules automobiles parmi cette pléthore de modèles. Pour réaliser cette sélection de modèle, que nous dénommerons modèle de référence, nous analysons et prenons en compte les contraintes provenant des normes, des réglementations, de l'automobile et des parties prenantes. Ensuite, et après avoir introduit le concept de polymorphisme dans la modélisation de la qualité comme mécanisme d'évolution et d'adaptation intrinsèque des modèles qualité, la réponse que nous apportons à la problématique se résume à définir et démontrer concrètement une méthodologie d'adaptation et d'opérationnalisation de ce modèle de référence pour les logiciels embarqués dans les véhicules automobiles.

Enfin, en guise de consolidation de cette réponse, nous déterminons une forme unifiée de modèle qualité de référence non pas, comme le suggèrent Zouheyr Tamrabet et al. [6], en tant que modèle qualité unique pour les produits logiciels, mais en tant que méta-modèle servant d'agrégateur de modèles qualité pour les produits logiciels. Le résultat final est la proposition du génome du modèle qualité du logiciel que nous modélisons sur la forme de 7 chromatides.

List of Figures

List of Tables

Chapter I. Introduction

This thesis took place at LAAS -CNRS laboratory, in the ISI team via an industrial partnership with Renault Software Labs, a subsidiary of Renault SAS company. This research project, founded by Renault Software Labs, had its prequel in Renault several years before the creation of this subsidiary in July 2017. However, since the industrial research problematic (i.e., define and evaluate automotive embedded software quality) remained still valid and open, and for the sake of consolidating the new company's expertise, it was decided to foster the training through research approach against that problematic. In the following paragraphs we are going to review successively the global context, the problematic, the thesis objective with the expected results and the dissertation plan.

Research, focused on systems and software, is performed in an international research laboratory jointly with strong relationships with the industry.

LAAS, Laboratory of Analysis and Architecture of Systems, is an own research unit of CNRS. Over its 52 years of history 1 , it has settled close and strong relationships with regional, national, and international industries, sometimes resulting in the creation of common laboratories (e.g., the common laboratory between LAAS and Lacroix company created on end of 2016 [7]). The scientific research in LAAS is organized around 6 departments, composed themselves from 3 to 7 teams. And even if each of them has its own field of expertise, these teams are all working on a wide range of systems: "integrated systems, embedded systems with real time and safety requirements, distributed systems, mobile systems, autonomous and robotics systems, micro and nano systems, biological systems" [START_REF]LAAS -Laboratory presentation[END_REF]. Thus, as a member of the ISI research team ("Ingénierie Système et Intégration" in English "Systems Engineering and Integration"), part of the RISC group ("Réseaux, Informatique, Systèmes de Confiance" in English "Trustworthy Computing Systems and Networks"), my research is currently focused on systems engineering applied to embedded system and software, which are both aligned with the company domains.

An industrial context of a distributed company whose business is the development of embedded software in the automotive sector.

Today, when a company designs, develops and manufactures goods or services, it must not only target a high level of quality for the products to satisfy customers, but also comply with many standards and regulations. This is particularly true with transportation systems where we can name few famous standards and guidelines: the ISO 26262 [3] addresses the software functional safety in automotive, the ARP4754 [4] provides guidelines for the development of civil aircrafts, and the DO-178C addresses software safety [5] in aeronautics. Furthermore, these safety guidelines impose to the company to be at the state of the art for processes and methods, when designing and developing a new vehicle. So, in order to take into account these obligations from standards while targeting at the same time a high level of quality for the products to satisfy customers, the international car manufacturer Renault SAS, initiated an internal project called SOQUAL (i.e., SOftware QUALity) in 2015. As the project name indicates, the point of attention is set on the software because the quality and customer satisfaction company department identified the software as one of the most problematic elements in a car since software is especially developed by suppliers and only a minority of the company employees is familiar with software concepts. It should be noted also that software becomes more and more important, functionally, in terms of volume and criticality for vehicles [9], which therefore reinforces the general need to pay attention to this element. Nevertheless, at the time Renault Software Labs joined Renault SAS, only 3 main indicator definitions (i.e., coverage, completeness, and consistency indicators), a delivery checklist and a document listing some software measurements were the results of this project.

An arrival that changes the rules in the company embedded software development model and quality, but a remaining problematic.

With the arrival of Renault Software Labs on July 2017, bringing on board software experts over all activities, from specification to validation, and from methods to tools, Renault SAS makes clear that software development is considered as one of the company strategic activities since then. In this context of software activity All these four research questions can be summarized under the thesis subject "Study of Qualimetry essential applied to embedded software development" and they cadence the chapter sequence of this dissertation (cf. Figure 1). •In this chapter deals about the current industrial context, the vehicle and embedded software, and the quality modeling for embedded software problems we aim to solve, aligned with company needs. •Chapter 3, Research Methodology •During our research we didn't follow a straight line, and we had to reconsider our research approach at a certain time. So here we explain the different steps of our research flow, the analysis technological locks and decision we made (e.g., we decide to reuse, adapt quality model rather than create a new one: in fact when we look for quality model, we can notice that most of cases are reusing, changing or being closed to what have been done rather than a complete disruption).

Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials? Research question 1 Considering the set of software quality models, how to identify and decide which quality model is the most suitable for embedded software? Research question 2

•Chapter 5, Quality Model Classification and Selection

•Following previous section, we understand that quality model is a pivot concept, there is no reference list of software quality models. So, if possible, how to classify and select quality model. This is completed with a systematic literature review and creation of a quality model landscape cartography.

Considering a quality model for a software product, how to operationalize it? Research question 3 •Chapter 6, Quality Model Operationalization

•Here, we investigate how we can move from theory to practice with quality model, what issues prevent the quality model operation (development and use of quality model), and then review practical resolution of these issues, with some processes for polymorphic quality model tailoring. •Chapter 7, Put into Practice •We apply now what we have seen above to solve the company needs against a concrete example of 3 ECUs and one transverse function. We use a survey based on a reference quality model (i.e., from standard), identify consensus, build polymorphic quality models, and connect with metrics, Can we have a unique reference quality model for software product? Research question 4

•Chapter 8, Meta-Model: Software Quality Model Genome •We analyze this key question which lead us to rather have a meta-model as a quality model aggregator for software product. Furthermore, we benefit from genetic knowledge to create a meta-model that can used as the beginning quality model to construct polymorphic quality model, considering that there are variations of same concept over multiple quality characteristics, for example ; we are avoiding to discard contribution on quality models Closure statements

•Chapter 9, General Synthesis and Research Perspectives •This last chapter wraps-up this dissertation and discuss on some openings.

Chapter II. Context and Problems

Introduction

Precisely understanding our current context and our object of investigation are fundamental, otherwise we have no guarantee to look for the right answer to our problematic related to quality definition, assessment, and control.

The purpose of this chapter is thus to first review the industrial context, the automotive industry and more particularly the automotive embedded software, understanding the specificities of vehicle platform development and the standards that must be considered. For instance, a vehicle system follows a development life cycle of five years which is much longer than smartphone or computer system where the development life cycle is on a six months cadence.

As a second step, an enquiry on the current state of the art on quality modeling for embedded software is performed, identifying 33 distinct contributions in embedded system and software quality domains, covering a period going from 1999 to 2020. This literature review aims to confirm not only the existing solutions but also the actual gaps, or road-blockers, that prevent to solve our problematic.

Last, a discussion and critic analysis on this chapter ensure a proper overhaul and boundary identification of the research content and findings.

A complex industrial research context with a need of quality

As we already highlighted, the context of this research is an industrial one, and more specifically the automotive industry one. Indeed, our mother company -Renault SAS-designs, develops, and produces automotive vehicles.

To achieve such development and productization, the company is structured into multiple engineering departments called Direction Engineering Alliance (DEA) (e.g., DEA-S for systems engineering department, DEA-L for software engineering department, DEA-M for mechanical engineering department, DQ-SC for quality and customer satisfaction department) where there exist many different job types with their specific vocabulary, process and tools. Therefore, this situation often makes it complex to interconnect people, processes, and tools.

Regarding the product, from a systems engineering point of view, a vehicle platform is a complex system [13], itself part of a system of systems (e.g., one use case that the Architecture Reference for Cooperative and Intelligent Transportation [14] addresses is where multiple cars are connected within a road system and its infrastructures). Furthermore, a vehicle platform can be considered as one generic complex system from which several vehicle variants such as mini-compacts, crossovers, super cars, vans, sport utility vehicles, convertibles, etc. are derived. As Fairley [9] described, this system is made of sub-systems classified under domains (e.g., chassis, body, infotainment, X-by wire, powertrain) connected together through a gateway and internal car networks. These subsystems are themselves composed of many distinct parts (e.g., electronic control units (ECU), electrical wires, mechanical elements) whose number and complexity depend on the type and version of a car. For instance, a premium car may have more than 60 distinct ECUs while a low-cost car has around 30 ones, and we see in Crolla et al. [START_REF] Crolla | [END_REF] (cf. Figure 2) that the overall number of ECUs is constantly growing over time. So, to produce high volumes of those parts, the company must rely on suppliers. However, the part specification and assembly remain of the responsibility of Renault SAS, and the company must be extremely cautious in their specification to ensure that during multi-system assembly, all separately developed parts fit and work well together. Unfortunately, suppliers are usually working at their own cadence and sometimes release ECUs (both hardware and software) that works fine alone but not all together.

Like any transportation systems (e.g., car, airplane, train, trucks, construction equipment vehicle), the overall product life cycle is also longer compared to other industries such as consumer electronics (e.g., computer, smartphone, tablet) where the design, development and delivery cadence is about 6 months. For a car, it usually takes five years since the initialization of the project to the serial production in the company factories. Moreover, the overall car operation lifetime is about 20 years, which means that during 20 years since the first produced car, the company must keep all materials required for any car reparation, corrective maintenance, or inquisition (in case of accident with injury or death). This also means that quality of the produced car is fundamental, and has some strong economic impact, particularly in the case where quality level is not sufficient to handle such long lifetime period. In addition to this requirement on quality, the company has legal responsibility for any produced vehicle which are reflected in standards (e.g., security and safety [3], safety in autonomous vehicle context [START_REF]:2019 Road vehicles -Safety of the intended functionality[END_REF], cybersecurity [17]) and regulations (e.g., European control on homologation [START_REF] Caudet | Agreement on Commission proposal to tighten rules for safer and cleaner cars[END_REF] linked to the tighten rules for safer and cleaner cars, the European Union General Data Protection Regulation [START_REF]Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC[END_REF] for personal data protection). All those requirement assessments are centralized and tracked by the DQ-SC department which uses customer satisfaction as main quality driver in addition to the compliance to these regulations and standards.

Coming back to our current problematic, the focus is set on the software embedded in the embedded systems that are ECUs.

Prior to Renault Software Labs integration, that embedded software was considered only as a subpart of the systems engineering deliveries. Thus, embedded software, developed by suppliers, was delivered as part of ECUs. Furthermore, embedded software quality was previously tracked as part of system quality through SIL (simulation in the loop) or HIL (hardware in the loop) intersystem tests. Since then, the company priorities evolved with the current age of industrial revolution (i.e., the age of software and digital [START_REF] Perez | Technological Revolutions and Financial Capital: The Dynamics of Bubbles and Golden Ages[END_REF]). These priorities are then put on car electrification, connected car and services, and justify the need of recognizing the importance of software in front of systems. Indeed, with connected services for example (e.g., Firmware Over The Air (FOTA)), software is also offboard (i.e., not in an ECU), and by consequence the software becomes highly configurable or changeable, at the opposite to the hardware. In addition, company internal teams develop either models or embedded software (e.g., FOTA source code) that are delivered to suppliers for integration in their own embedded software. The responsibility and process of development is consequently more complex and previous company approach doesn't fit anymore.

Fortunately there is a standard for automotive software development process: Automotive -SPICE [START_REF]13 / Automotive SIG[END_REF] (A-SPICE).

A-SPICE focuses on automotive software development 2 , management and support processes, and improvement and assessment of the process capability level. It is neither a product quality assessment process, nor a product quality control process, but its guidelines recommend the use of ISO/IEC/IEEE 25010 [START_REF]13 / Automotive SIG[END_REF] 3 software product quality model (i.e., product view) and quality-in-use model (i.e., user view) to support quality assessment and control activities. Consequently A-SPICE guidelines must be followed for the development of all ECU embedded software. Note, ISO/IEC/IEEE 25010 is part of the ISO/IEC 250nn series [25], named System and software Quality Requirements and Evaluation (SQuaRE) which aim to cover quality requirement definition and evaluation.

To summarize, the automotive industrial context jointly with the vehicle as complex system, the development model with suppliers, and the current standard and regulation requirements are raising the overall complexity of our problematic to define, assess, control, or predict embedded software quality.

Moreover, compare to pure academic research, the research performed within an industrial context requires to address a concrete problem that a company has, and consequently the company expects to be able to use and apply these research findings. In other words, in this type of context, one unavoidable aspect of the solution under investigation is the operational aspect.

In our research work we must thereby address any issues that prevent practice of theory, which is not at all a straightforward matter when dealing with quality because quality is an elusive target as rightly highlighted Kitchenham and Pfleeger [26].

Therefore, in this context it is important to have a unified, operational, and appropriate way to define, assess, control, or predict quality of embedded software.

Quality modeling applied to embedded software development

In section 2, we learnt about the complexity of the industrial context of this research work with a clear need of quality for the developed embedded software. The goal of this section is to assess whether we already have a viable solution to our current problematic. In the negative case, we should be able to identify the gaps that we have then to address. So, after a clarification on our problematic about quality modeling applied embedded software development, we perform a state-of-the-art analysis on quality modeling during the development of the embedded system or software. This focus can be on process, specific development stages and/or product.

Today, there exists a myriad of embedded systems, more or less complex, from smart watch to printer, from internet of thing to car, rocket, or airplane. They are not only part of our everyday life but also are the depiction of the current technological revolution: "the age of software and digital" [START_REF] Perez | Technological Revolutions and Financial Capital: The Dynamics of Bubbles and Golden Ages[END_REF]. To succeed in this revolution, it is of first importance to define, assess, control, or predict the quality level of the software embedded in such systems.

This is achieved by applying properly quality modeling in embedded software development. Thus, quality modeling consists in the build, reuse, or adaptation of a quality model composed of quality characteristics / subcharacteristics, sometimes named attributes, together with metrics with the aim to define, assess, control, or predict quality [START_REF] Wagner | Software Product Quality Control[END_REF] of a specific object of interest. Quality characteristics / sub-characteristics represent the qualitative side of the model while metrics represent the quantitative one. However, we remark that metrics are often missing at least partially from published quality models, preventing the quality models to be operational. Fortunately, one way to operationalize such a model is to use the Goal-Question-Metrics (GQM) paradigm [START_REF] Basili | Goal Question Metric Approach[END_REF]. Indeed, to measure quality characteristics / sub-characteristics, GQM starts by assigning goal to each characteristic/ sub-characteristic. Then the goals are derived into sets of questions, themselves completed with the proper metrics to answer to these questions. Unfortunately, the problem of defining quality characteristics / sub-characteristics is not solved with GQM.

As opposed to traditional computing systems shown in Figure 3, embedded systems have specific runtime constraints (e.g., safety, security, limited resources, real-time) and a specific application domain [START_REF] Jeong | A Quality Model of Lightweight Component for Embedded System[END_REF]. Both must be taken into account in the quality modeling applied to the embedded system life cycle stages [13], and which include software development. These specificities are expressed via quality requirements, characteristics / subcharacteristics, attributes, or again quality aspects, and synthetized into quality model.

Thus, to investigate the research question associated to our problematic, "how quality modeling is applied to embedded software", we performed a review of the current state of the art in this field. To be more precise, we performed an exploratory literature review [30], a method close a systematic mapping study [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF],to seek what already exist on "quality model" and "embedded software", or "embedded systems" concepts over the software engineering online digital libraries 4 and the scholarly literature search engine, Google Scholar.

We took 1968 as starting year for our investigation because this is the publication year of the two first published quality models in software engineering, term which emerged only few years before (1965) [START_REF] Berkeley | The Computer Directory and Buyer's Guide[END_REF]: the quality assessment model of Rubey and Hartwick [START_REF] Rubey | Quantitative measurement of program quality[END_REF] and the reliability prediction model of Shooman [START_REF] Shooman | Probabilistic reliability: an engineering approach[END_REF].

We found 33 main relevant research works published over the period from 1999 to 2020, and organized around four research focuses:

quality model creation or use -for 42.4% of the studies, quality characteristics or/and quality model identification -for 30.3% of the studies, reliability growth model creation or use -for 15.2% of the studies, miscellaneous work related to quality modeling in embedded software -for 12.1% of the studies.

The following sections deep dive over each research focuses.

- [START_REF]Part1: Quality Model[END_REF] and was composed of 7 quality characteristics, 29 sub-characteristics and 46 metrics. The authors decomposed subcharacteristics into the ones observable on "runtime" (e.g., stability) and the ones observable during "life-cycle" (e.g., testability). They also added marketability quality characteristic because they jugged about its importance for a certification process and the credibility it could bring to the component customers.

Completing this work linked to embedded software component, Carvalho and Meira [37] refactored CQM to came up with Embedded Component Quality Model (ECQM) and built a quality verification framework within the scope of certification. The authors aligned their work on ECQM with ISO / IEC 25000 SQuaRE [25], keeping the same 7 quality characteristics than CQM but with only 26 sub-characteristics, and used GQM to identify the corresponding set of metrics. Regarding the new quality verification framework, Carvalho and Meira introduced the Embedded software component Maturity Model, a maturity model similar to Capability Maturity Model for software [38]. Thus, depending on environment, safety; security, economic and domain potential damage and risk, the right maturity level for each component is selected, and then the evaluation technique(s) for each quality characteristics is applied accordingly. We note that the overall framework process lacked maturity even if the authors highlighted that it was used against Brazilian industry without any further detail.

In parallel, Choi et al. published [START_REF] Choi | Practical S/W component quality evaluation model[END_REF] (2008) the Samsung software Component Quality evaluation Model (SCQM) to evaluate embedded software components for Digital TV systems. This in-house quality model was based on quality model definition from IEEE 1061 [37] (cf. Figure 4 showing the corresponding ontology where quality is defined as one or more quality factors, themselves refined into quality sub-factors and finally into metrics), and several well-known quality models such as ISO / IEC 9126, McCall [41] & Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF]. The authors identified a list of 8 quality characteristics jointly with 22 sub-characteristics but provided only few metrics as example to illustrate their work. During the creation of SCQM, the authors used expert Delphi method to identify strongness relationships (i.e., strong versus weak) between characteristics and sub-characteristics. Consequently, they organized the sub-characteristics into two categories, common (i.e., invariant) and variance (i.e., depending on component quality requirements), where "variance" finally represents the candidate sub-characteristics used to tailor the SCQM depending on the quality requirements of the targeted Digital TV Systems.

Figure 4 -Ontology of quality model definition from IEEE 1061 [40] On another type of embedded system, Peper and Schneider were investigating the matching of quality of ambient intelligence systems from a quality of service point of view [START_REF] Peper | On runtime service quality models in adaptive ad-hoc systems[END_REF] (2009). Indeed, this embedded system, composed of software components, is an ad-hoc computer system (i.e., adaptive, distributed systems, acting like ad-hoc network) and therefore it provides not only services but also uses services from other ad-hoc systems.

Those services must be negotiated between service provider and the client in term of provided versus requested functionality and quality. So, the authors suggested a service quality reference model for the matching quality problem between service provider and client components: they modeled quality (e.g., with a video system: video size, video rate, audio rate) as discretized into small number of intervals or values, defined per feature which can be similar, or shared, over other features.

Over a series of studies where the quality modeling apex was the DeLone & McLean success model [START_REF] Delone | Information systems success: The quest for the dependent variable[END_REF], Jeong et al. aimed to adapt this model from its original scope, information system, to the software of embedded systems. They chose this model because of its frequent use by large audience, and the fact that it shows factors that influence organization. In the first study [START_REF] Jeong | A Quality Model of Lightweight Component for Embedded System[END_REF] (2012), the authors noticed that embedded systems, as opposed to desktop systems, deliver functionalities for specific domain, and also that their software can be considered as a lightweight component software. So, the authors replaced the three original quality input factors (information, system and service quality) of DeLone & McLean success model by a 5 quality factors (information, system, function, efficiency and maintenance quality) with the corresponding sub-factors, linked to lightweight component software. They continued in the next study [START_REF] Jeong | A Design of Software Quality Model for Embedded System[END_REF] (2012) by regrouping the success model factors under three categories: system design, system delivery and system outcomes. By system design, they considered requirements of embedded system that a system designer must take into account such as interface, data flow, or quality of services, and redefined three quality input factors, with their own sub-factors: system quality, information quality and operation quality. The authors moved use and user satisfaction factors under system delivery, and individual and organizational impact under system outcomes. With the next study [46] (2012), Jeong et al. used ISO / IEC 9126 quality model to strengthen the quality input factors of the success model. They defined four criterions and sub-criterions. We retrieved system quality and information quality from DeLone & McLean success model but also two new ones: software process quality, representing a large part of ISO / IEC 9126 subcharacteristics, and security quality. Finally, with the last study [START_REF] Jeong | The Practical Quality Model for Embedded System and Software[END_REF] (2013) of this series, Jeong took into consideration an additional aspect of embedded software: software as a service (SaaS) with the cloud computing environment. Therefore, he identified and completed sub-criterions of system quality, information quality, software quality and security quality based on attributes for SaaS in cloud computing. Unfortunately, in none of these studies, the authors neither gave details on their construction choice, provided metrics, nor explained how to operate the model.

From another perspective Mayr et al. [48] (2012) investigated the creation of a code based quality model for embedded systems (i.e., ESQM). They noticed that existing solutions didn't provide any quality model that can be easily operated for assessment or control activities. So, in order to create such model with enough operationalization, they defined a four stages process jointly with the use of 336 metrics. They chose C and C++ code metrics evaluated with static analysis tools because their assumption is that embedded software is mainly written in these two languages. The first stage of this process is the quality requirement elicitation. They start from the embedded system and software characteristics to identify and group requirements linked to quality. In the second stage, these requirements are organized hierarchically, and code metrics are identified and assigned to each of them. With the third stage, they aim to associate these requirements and metrics to a higher level of quality characteristics, which they called "quality aspect", targeting high level management. The quality aspect is achieved thanks to a mapping between these requirements and metrics with ISO / IEC 25010 quality view. To consolidate the model, they introduced "factor" concept, and more particularly "product factor", which is similar to Dromey's quality carrying properties [START_REF] Dromey | A Model for Software Product Quality[END_REF], used as the central element connecting requirements, quality aspects and measures. The purpose of the last stage is to finalize the quality model operationalization with threshold definition and weighted aggregation, based on Trendowicz et al. [50] software quality assessment method named SQUAD, a 5 steps approach using multi-criteria decision analysis. We noticed also that each stage last step consists in a quality gate validation where few "experts", with a focus on completeness and validity quality factors and model, review the stage result. Finally based on their feedback, some adjustments are performed before moving to next stage. So, even if this approach is interesting with the operationalization assessment, similar to Trendowicz et al. method, the generalization to any embedded system is not straight forward and obvious since it started from the assumption of software is written in C or C++ and then favored code metrics. Moreover, the overall process is quite heavy to deploy, dependent of the set of selected "experts".

Another attractive contribution is the creation, by On 2015, Al-Sarayreh [START_REF] Al-Sarayreh | A Quality Requirements Safety Model for Embedded and Real Time Software Product Quality[END_REF], introduced a generic safety requirement model for any embedded and real-time software product where safety aspect is mandatory. This requirement model is based on 8 entity types which can be used as a basis for quality measurement, especially functional size. We note that the author didn't refer to any software related safety standard like ISO 26262 where some specific software functional metrics depending on safety level (e.g., ASIL for automotive), for instance, can be found. Moreover, that generic safety requirement model is not a quality model even if it can be considered as input for a quality model.

One year later, Silva and Vieira [START_REF] Silva | Software for Embedded Systems: A Quality Assessment Based on Improved ODC Taxonomy[END_REF] introduced an innovative approach for quality modeling of satellite systems. They extended Orthogonal Defect Classification (ODC) [54] to cover embedded system, and more particularly critical ones with safety consideration. ODC is an efficient root cause analysis (i.e., it is 10 time faster compared to traditional root cause analysis) which allows to classify defects over three opener section attributes (i.e., defect removal activities, triggers, impact) and five closer section attributes (i.e., target, defect type, qualifier, age, source), and then get measures of quality linked to these attributes. Thus, the authors extend these attributes to include embedded system support (e.g., with safety), and their corresponding quality assessment was based on defect classification attribute measures and not from a quality model other than defect based.

Next, on 2017, Garces et al. [START_REF] Garces | Quality attributes and quality models for ambient assisted living software systems: A systematic mapping[END_REF] achieved a systematic mapping study on software of Ambient Assisted Living (AAL) for disabled or elder people. Due to its technical nature, an AAL is like an embedded system and its purpose falls under one the following scopes:

independent living, health and care, occupation in life, recreation in life.

So, the goal of that literature review was to identify quality attributes and quality models defined, used, or assessed for AAL software. Over the 27 studies they found, Garces et al. retrieved only one quality model, OptimAAL based on ISO / IEC 9126 and ISO / IEC 25010, composed of reliability, availability, safety, integrity and maintainability, but this model was neither suitable for quality assessment, nor quality prediction of AAL software. In parallel, they collected 97 quality attributes where the most important ones were security, freedom from risk, usability, reliability, adaptivity, availability, fault tolerance and performance efficiency. To structure these quality attributes into a quality model, the authors defined a mapping algorithm to map these attributes against first ISO / IEC 25010 quality models and then against ISO / IEC 9126 quality models and metrics, adding in passing adaptivity quality characteristic to ISO / IEC 25010 existing quality characteristics. Their reinforced this result by defining a taxonomy based on the four AAL scopes. The resulting quality model was the preliminary version of the authors' Quality Model for Ambient Assisted Living Systems (QM4AAL) [START_REF] Garcés | A Quality Model for AAL Software Systems[END_REF]. Nevertheless, the authors highlighted the need to get industry involved to increase the maturity of the quality attributes and models, especially that out of all these 27 studies, only one was evaluated in industrial context.

The last relevant contribution with regard to "quality model creation or use" theme is the definition of a Software User Review Defect Corrective Model (SURDCM) from Kasiviswanathan and Ramalingam [START_REF] Kasiviswanathan | Development and application of user review quality model for embedded system[END_REF] on 2020, exercised against a 3D-drawing application. In this paper, the authors are considering that a high-quality product means that that product very few bugs, and consequently one of the major areas which requires strong attention to produce quality product is requirement analysis: good quality requirements are essential. So, SURDCM is done via a process starting from "customer requirement" up to "deployment and maintenance". The idea is to perform first an analyze with Kano model [58] (i.e., satisfaction versus unsatisfaction based on observer, or customer perception) to understand the impact of each requirement against customer. Then, based on this analysis, completed with a correlation coefficient of customer satisfaction (i.e., extent of satisfaction and extent of dissatisfaction) and Saaty's Analytic Hierarchy Process (AHP) [START_REF] Saaty | The analytic hierarchy process-what it is and how it is used[END_REF], the requirement prioritization is finalized. Indeed, AHP is an expert analysis process which evaluates pairs of alternatives (in this case the requirements) based on criteria (in this case they are performance, design, security, and usability) to determine the most important alternatives thanks to eigenmatrices, values, vectors and consistency index. The requirements are then designed and implemented relatively to their priority. The last SURDCM step is a Software Failure Mode Effects Analysis (SWFMEA) [60] which is triggered once the code is generated and entered the testing phase. SWFMEA is a failure mode analysis of the impact of both functional and non-functional software failures which helps to find any required corrective or evolutive action on requirements. So, in this contribution, the requirement quality evaluation is not achieved via a quality model with metrics but rather through a complex process whose purpose is to enhance requirements depending on failures.

We learnt with these studies related to "Quality model creation or use" that there is no obvious nor universal solution for modeling quality of embedded software. Some studies focus on requirements while some others on architecture or implemented code, but there is also no quality model usable during all development stages. Moreover, only few contributions investigate the quality model reuse, or the operationalization by providing metrics and a process, often complex to deploy, for objective, or subjective, quality evaluation.

b. Quality characteristics or/and quality model identification studies "Quality characteristics or/and quality model identification" is the second principal research focus of the studies we identified. This corresponds implicitly to a subset of our main research question because we are identifying what model or quality characteristics is belonging to embedded software.

The first and earliest contribution on this research focus was done by Wijnstra [START_REF] Wijnstra | Quality attributes and aspects of a medical product family[END_REF] on 2001. He looked for identifying the quality attributes and aspects of medical systems which are composed of devices made of hardware and embedded software. He remarked that quality attributes can be derived into system, hardware, mechanical and software aspects, from an architecture point of view, and by consequence, quality attributes and aspects are both complementary. Thus, Wijnstra identified a list of seven quality attributes for embedded system in medical domain (reliability, safety, functionality, portability, modifiability, testability, serviceability) with three specificities belonging to modifiability for medical (configurability, extensibility and evolvability) and completed with various aspects that must be took into account: for instance, operational, start-up, shutdown, error handling, graceful degradation. Finally, even if there is a clear quality attributes for embedded systems with relevant explanations, neither quality model, nor metrics are given in this contribution which foster its replication or operationalization.

One year later, Purhonen [START_REF] Purhonen | Quality attribute taxonomies for DSP software architecture design[END_REF] sought to identify quality attributes for digital signal processing embedded software in wireless system. The author successfully identified three main quality attributes (performance, cost and variability) which he derived from quality goals coming from either standards, customers or development source. Furthermore, he used Barbacci et al.'s taxonomy [START_REF] Barbacci | Quality Attributes[END_REF] to classify these quality attributes. This taxonomy is organized over 3 factors which are stimulus (i.e., events causing architecture response or change), response (i.e., measurable or observable quantities to assess architecture) and architectural (i.e., parameters that define architecture decision). Then, despite that software development for digital signal processing for wireless systems is quite similar to some other embedded software developments, the resulting quality model in this contribution covers only three quality attributes, without any metrics, and unfortunately the use of a taxonomy for its refinement was not sufficient to generalize it to any embedded software.

Another relevant contribution in the field is the investigation done by Akerholm et al. [START_REF] Akerholm | Quality attribute support in a component technology for vehicular software[END_REF] (2004) to spot the most important quality attributes for component-based software embedded into vehicular systems (i.e., cars, construction equipment vehicles, heavy trucks, trains). The authors' methodology can be summarized over three main steps. The first one was to build a list of relevant, to vehicular systems, quality attributes collected from existing literature. These quality attributes are mainly associated to non-functional properties, also called extrafunctional properties, or -illities. The second step consisted in interviewing experts from several companies to sort that list, creating groups from the most important ones to less important ones:

safety, reliability, predictability usability, extendibility, maintainability, efficiency, testability, security and flexibility.

In the last step, the authors used Larsson's classification [START_REF] Larsson | Predicting Quality Attributes in Component-based Software Systems[END_REF] against these 10 quality attributes to complete their description. This classification is based on the categorization of on how a quality attribute could be facilitated by component technology, and where support of a quality attribute should be implemented. Thus, the classificationbased description can be one or a combination of: directly composable, architecture related, derived attribute, usage dependent and system environment context. This list of quality attributes cannot serve as it is for embedded software quality evaluation, but rather as a starting point for quality model definition of vehicular systems.

On 2008, Paulitsch et al. [START_REF] Paulitsch | Non-functional Avionics Requirements[END_REF] aimed to analyze the aerospace domain to build a list of non-functional requirements for avionic embedded systems. These non-functional requirements are assimilated to embedded system quality characteristics, or attributes, and therefore must be reflected in both hardware and software architecture of such systems. Thus, the authors came up with a list of 10 requirements related to dependability, performance, development, and operation (i.e., availability, diagnosis, integrity, maintenance, obsolescence, temporal performance, testing, safety, schedulability, and security). Paulitsch et al. indicated also that this list is not exhaustive but together with their analysis and explanations, they have provided a basement to be used and integrated in further work related to non-functional requirements in avionic context.

At the same time, Sherman [START_REF] Sherman | Quality attributes for embedded systems[END_REF] shared a list of 30 quality attributes for embedded systems, where few of them were specific physical attributes (e.g., weight, physical size). What makes this study interesting is the approach that the author used to establish this list. Indeed, he identified quality attributes based on the evaluation of 11 embedded system architecture trade studies made with Architecture Tradeoff Analysis Model (ATAM). These ATAM studies were performed with the main system, hardware, and software stakeholders to review architecture solutions and then decide which architecture design tradeoff to choose. So, Sherman used those architecture decisions to find which quality attributes were cited, and consolidate their knowledge with their citation number, the number of studies where they were referred, and whether or not their definition changed between studies (this happened for two third of the attributes).

To continue on embedded system architecture, Guessi et al. [START_REF] Guessi | Architectural description of embedded systems: a systematic review[END_REF] (2012) led a systematic literature review [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF] to identify quality requirements and constraints for reference, and software, architectural description of embedded systems. The authors found a set of 12 architecture concerns, declined into quality requirements: adaptability, correctness, dependability, fault-tolerance, interoperability, knowledge reuse, maintainability, performance, power consumption, reliability, safety, and timing. In addition, they remarked the existence of architecture information types (components, interactions, interfaces, synchronization, timing, configuration, constraints, dataflow, processes, threads, connectors, concurrency, abstraction layers, concepts, design decision, events, states, behavior) that they decided to associate with these quality requirements. They completed their survey with an analyze of the most frequent architectural language description (e.g., the two main ones are the Unified Modeling Language (UML) and Architecture Analysis and Design Language (AADL)).

We note that based on Guessi et al.'s contribution, we have an easier way to find and associate metrics to quality requirements through these types of information. We depict our proposal in Figure 5 where types of information act as the central joint between quality requirements and metrics. Indeed, one or more types of information (e.g. interfaces, timing, dataflow, events, states) can be associated to a quality requirement, and each type of information can be measured thanks to one or several metrics.

Figure 5 -Use of type of information as a helper to find and associate metrics to quality requirements, based on Guessi et al. [START_REF] Guessi | Architectural description of embedded systems: a systematic review[END_REF] Another relevant systematic literature review is the one from Oliveira et al. [69] (2013). Their research objective was to get a panorama about quality models and quality attributes defined, assessed, or used for embedded system. The authors identified 11 studies where 27.3% of them proposed a quality model while the other 72.7% concentrated only on quality attributes. Among all the information they extracted from their analysis, we learn that 54.5% of studies were developed from document analysis (e.g., systems requirements), 36% from personal experience and 27.3% from literature reviews. Also, 45.4% of the study contributions were evaluated through academic or expert opinion, 27.3% via their application in embedded system, and the remaining 27.3% gathered no clue on an evaluation method. In term of quality attributes, the authors identified 18 attributes (% of presence in the studies):

-91%: maintainability, reliability, -64%: security, safety, functionality, efficiency, portability, testability, -45%: performance, usability, -36%: availability, extensibility, reusability, cost, -27%: fault tolerance, recoverability / repairability, interoperability, flexibility.

The authors found also that defining quality model and attribute is complex task since it requires multiple sources of information and there was still need for a quality model to be widely accepted or adopted. In fact, there was no strong proposal from any of those studies, and only two studies showed some industrial evidence of the use of their quality model and attributes.

In the continuity of the study series, we saw in the section a, where the assumption was completion of DeLone & McLean success model, Jeong et al. [START_REF] Jeong | An ANP-Based Practical Quality Model for a Secure Embedded System with Sensor Network[END_REF] (2014) search identified what they considered as the most important quality characteristics, as well as their weights, for secure embedded system. They used the Analytic Network Process (ANP) method to compute the value of the degree of influence for each quality characteristics. AHP and ANP are two closed methods, both based on the evaluation of pairwise importance relation between each characteristic, and then on eigenmatrix, values and vectors to determine importance of quality characteristics. However, AHP assumes the independence of the factors used for decision making which is the opposite behavior the authors looked for. We note that surprisingly the authors didn't include any security related characteristic for such secured embedded system, but rather they tried to see which characteristics may be the most important in this context.

Bianchi et al. [START_REF] Bianchi | Quality Attributes of Systems-of-Systems: A Systematic Literature Review[END_REF] (2015) contributed as well to this topic but from a slightly distinct system perspective: they focused on system of systems. And because a system of systems often includes embedded systems (e.g., internet of objects / things, automotive, avionic, medical), we decided to include their contribution. So, over a systematic literature review, the authors aimed to determine what quality attributes are associated to a system of systems mainly due to its specific architecture, and to the interdependencies or interoperability relationship between the different systems that compose it: for instance, safety or reliability are complex to determine since any systems can impact in different ways and multiple workarounds can exist also. In their survey, Bianchi et al. retrieved from the literature a list of 56 quality attributes where the five most cited are security, interoperability, performance, reliability, and safety. Regarding their application domains, the-most frequent ones are military, IT systems, smart grid, and automotive. The authors consolidated their survey by determining the amount of these quality attributes which are present in ISO / IEC 25010, actual standard reference for system and software quality model. They figured out that 48% of them are not present in this standard, nevertheless these missing attributes are not the major cited ones, and it appeared that complex interdependencies of attributes are hardly achievable with a hierarchical structure such as ISO/ IEC 25010 quality models, but rather with a general graph structure, for instance.

To complete the quality characteristics or/and quality model identification studies, Zouheyr Tamrabet et al. [6] (2018) did a survey on quality attributes and quality models for embedded software. The authors started with a description of what an embedded system is, and then clarified software quality concept, including quality attribute and model. Next, they established three groups of the most represented quality attributes in literature:

standard characteristics: reliability, usability, maintainability, portability, performance, functionality, sub-characteristics: availability, fair tolerance, interoperability, adaptability, recoverability, reusability, testability, specific attributes: efficiency, safety, flexibility.

In their conclusion, they stated that there were neither consensus on a list of quality attributes, nor on quality model for embedded systems, and therefore in their future work, they aim to propose a generic quality model "that encompasses relevant quality attributes in order to define the quality of embedded software".

Over these 10 studies related to quality characteristics or/and quality model identification, we remark that the authors use either quality attribute or quality characteristic for the same concept but, in all cases, these characteristics are non-functional quality characteristics, and they are mainly associated to requirements, architecture design or constraints.

Moreover, none of these surveys introduce any metrics for quality characteristic, which are essential from an operational aspect.

Finally, like Zouheyr Tamrabet et al., we can state that we didn't notice any consensus on quality attributes and quality model for embedded system and software through those contributions.

Furthermore, from what we saw, we may find a subset of quality attributes (e.g., maintainability, reliability, safety, security, and testability) which are common to all embedded system or software. However, each embedded system or software case is different, and that variety, or variant, must consequently be reflected in the quality model and characteristic solution. Therefore, we may have some doubts on the feasibility to build a generic quality model or come-up with a generic list of quality characteristics.

c. Reliability growth model creation or use studies

Our third group of studies is on "Reliability growth model creation or use" and thus consist in the stream of work about the creation or use of reliability growth model. The objective of this type of model is to determine an implicit or statistical quality model to evaluate and predict quality of embedded software prior to its operation, for example. It is a complementary approach to the previous group of studies.

With Khoshgoftaar and Allen [START_REF] Khoshgoftaar | Predicting fault-prone software modules in embedded systems with classification trees[END_REF] (1999) and Khoshgoftaar et al. [START_REF] Khoshgoftaar | Predicting Fault-Prone Modules in Embedded Systems Using Analogy-Based Classification Models[END_REF] (2002) studies, we have an illustration of some of the research performed Khoshgoftaar et al.. They explored the construction and use of reliability growth models to predict which software modules for telecommunication embedded system were the most fault prone. The developed quality models were statistical models built from that telecommunication system historical data, and their inputs were software process, product, and execution metrics. Unfortunately, the models cannot be applied to other embedded systems because of the specific historical data set used to determine the statistics models.

Regarding reliability evaluation for another type of telecommunication system, He and Li [START_REF] He | Software reliability analysis on embedded system based on SFMEA and SFTA model[END_REF] (2012) developed an alternative approach to determine the software reliability of voice over internet phone. Their inspiration came from safety analysis. Indeed, they achieved software reliability analysis and improvement based on a sequential combination of Software Failure Tree Analysis (SFTA) and then Software Failure Model and Effects Analysis (SFMEA). SFTA is used to retrieve the top events that cause failures as well as their related basic events through a failure tree. SFMEA is the application of Failure Model and Effect Analysis (FMEA), originally developed by US Military, to software domain since 1979. FMEA and therefore SFMEA are an analysis of the cause of failure mode for top and basic events. So even if He and Li solution was not a reliability growth model, the overall behavior of their approach is quite identical to evaluate and then increase the reliability of embedded system.

In 2016, J. Liu et al. [START_REF] Liu | The Evaluation of the Embedded Software Quality Based on the Binary Code[END_REF] introduced a reliability growth model combining reverse engineering approach. That reliability model aimed to evaluate embedded software quality of electric smart meter. The author innovation was to reverse engineer binaries (i.e., compiled and lined code, as opposed to high-level language source code) in order to reconstruct the software program into a high-level language. Then they evaluated the reliability of that reconstructed software, based on utilization-oriented fault model and the reconstruction of the software control flow graph. This contribution is similar to Goel-Okumoto software reliability model (i.e., G-O model) [START_REF] Goel | Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures[END_REF] (1979) but the authors highlighted that their model was simpler to use. The obvious drawbacks are that the deployment of that solution is complex, it requires to have binaries artifact and the reverse engineering software code result is subject to interpretations which may differ from original software code.

The contribution of S. Juneja et al. [START_REF] Juneja | Reliability Modeling for Embedded System Environment compared to available Software Reliability Growth Models[END_REF] (2019) is different from the above research works. The authors proposed a basic random model as reliability growth model for general embedded system. They considered both hardware and software failure occurrences to elaborate their model and once completed, they compared this model against five widely used software reliability growth models. The comparisons were done thought simulations of model failure rate with Matlab tool. As expected, their proposed model gave the best result when predicting embedded system reliability. The best result meant that model had the lowest failure rate because the authors characterized embedded systems as mission critical ones and implicitly, assumed that such systems were developed to have the highest reliability than other software systems (e.g., desktop computer, server). However, this mission critical assumption is accurate for a subset of embedded systems but not for all embedded systems (e.g., smartphone, smartwatch, printers, camera, tablet).

To sum up about reliability growth model, these studies are important because they show a complementary view on quality modeling for embedded software. The reliability evaluation and prediction are achieved in most of the cases with implicit or statistical models.

Unfortunately, these types of model prevent the direct model utilization to other embedded software because the model was elaborated from a specific historical data set.

Finally, concerning the other minor approach which involves reliability analysis process, even if the process can be generalized to any embedded software, that solution does not correspond to neither quality model, nor quality characteristics.

d. Miscellaneous work related to quality modeling in embedded software

Our intent within this last group of research works is to collect some further studies that may of an interest in quality modeling for embedded software, despite the fact that they don't propose any quality model or quality characteristic.

The first research study, driven by Rohloff et al. [START_REF] Rohloff | Quality Measures for Embedded Systems and Their Application to Control and Certification[END_REF] (2006), introduced a practical solution to get real-time feedback to control system quality and produce evidences for certification. That solution made of real-time utility quality measures, was elaborated specifically for distributed real-time embedded systems, with a focus on adaptive reflective middleware systems. The authors considered two aspects here, real-time and quality of produced data. That last aspect is key for certification evidences and an enhancement of that study could be to attach theses quality measures to a "utility" quality model.

With this other survey of Oliveira et al. [79] (2008) on space system design, the authors investigated the potential existence of relationship between metrics, more precisely between software quality metrics (e.g., lines of code, McCabe complexity, nested block depth, efferent coupling) and embedded software or system metrics (e.g., memory usage, performance and energy consumption metrics measured in a worst case scenario and also in a best case scenario). The leading idea behind this comparison is to find correlations between these two sets of metrics in order to support architecture trade-off decision between performance, maintainability, and component reusability, to shrink time to market.

The last two studies of this group are concerning the same embedded system, automotive.

In 2012, Stürmer and Pohlheim [START_REF] Stürmer | Model Quality Assessment in Practice: How to Measure and Assess the Quality of Software Models During the Embedded Software Development Process[END_REF] created an approach to measure and assess model-based design software. The authors highlighted that embedded software development evolved to include frequently model-based design. We remark that this statement is also aligned with ISO 26262 [5] and DO-178-C [7], for instance. So, to evaluate quality, the authors defined a quality model to aggregate measures from model analysis, issue tracking, test management and requirement management. They also included assessment of quality operations (e.g., testing, reviews) for all the generated development artifacts, and not only from model or code. There was no reference to quality characteristic likes in legacy quality models such as McCall, Boehm, ISO / IEC 9126 but rather the quality model in this study was a reference to the artifact that were analyzed and assessed. In other words, the author's quality model reflected the degree of success of the quality operation and not quality characteristic.

From another research perspective, Bouquet et al. [START_REF] Bouquet | Model Quality Objectives for embedded software development with MATLAB and Simulink[END_REF] (2018) introduced model quality objective concept. The model quality objective was an adaptation of software quality objective concept for model-based design, frequently used in embedded system development. The authors described 16 model quality objectives that were declined into requirements to ensure a high level of quality of embedded software which are developed through model-based design.

These four contributions taught us few additional considerations that we must consider for quality modeling applied to embedded software.

At first, we have to pay attention to quality of data, especially when generating the mandatory evidences for certification.

We must also investigate potential correlations between metrics and quality characteristics to support trade-off decisions (e.g., on architecture design).

And finally, an embedded software is not only a matter of code implementation, but as well from model-based design.

e. Limits to existing quality modeling applied to embedded software development

Over the 33 studies of this exploratory literature review, we saw a diversity of approaches and research works, summarized in Table 2. We observed that this diversity is the reflection of the myriad of existing or possible embedded systems and software, each of them having its own specificities and requirements. We remarked also that often, quality characteristics for embedded software are non-functional quality characteristics which are mainly expressed or associated to quality requirements, architecture design or constraints.

Nevertheless, among all embedded software, and thanks to systematic literature review studies, it is possible to identify a limited set of shared properties, or quality characteristics likes maintainability, reliability, safety, security, and testability.

Moreover, as Zouheyr Tamrabet et al. stated, there is no consensus on the adoption of quality attributes and quality model for embedded system and software, despite the existence of standard such as ISO / IEC 25010 which is by construction the result of an international work and collegial decisions. But this is not a surprising finding because the embedded software variety, or variant, must be retrieved also in the development of quality model and quality characteristics.

So, there is no right and unique solution yet to our question "how quality modeling is applied to embedded software". Another limit that was not resolved with these research studies is the operationalization of quality model but, nonetheless, we found some elements of answer in few of these studies. For example, Arhens et al. [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF] highlighted this operation aspect by emphasizing the importance of metrics, and aggregation of metrics together with quality characteristic and quality model. We note also that most of these surveys didn't introduce any metrics for quality characteristics. A second operational blocking aspect, happening in quality assessment, control, or prediction activities, is the objectivity versus subjectivity of the quality evaluation. Indeed, the specification of threshold values used in decision making is often not obvious, and consequently decision in those activities may be subjective. An alternate solution is to use relative threshold or evaluate quality against a reference, or previous embedded software; for example, when improving the quality of a software over multiple consecutive releases. In that case we are able to make objective quality evaluation and decision, but its usage cannot be generalized to any decision making.

Next to that exploratory literature review and analysis of the main limits, we have enough elements to detail our research questions.

Research questions

In the previous section 3, through an exploratory literature review, we analyzed 33 research contributions related to quality model and quality characteristics for embedded software, or systems, and conclude that there does not exist a generally adopted solution to model quality of embedded software. However, to support our company critical needs, we have to define, assess, control, and even predict embedded software quality as well as the quality of its development. In addition, the resulting quality indicators must be robust to support embedded software variants, which are the usual case in automotive domain, constraints coming from standards and regulations, and mandatory evolution accordingly to the development life cycle stages.

We should also manage an overall complexity due to: -An automotive system is a complex system to build with multiple elements that require alignments and interoperability, -Systems and software interface require alignment between process, technology, protocols, teams, and tools, -Alignment over the complete vehicle, architectures, many distinct jobs, and specific vocabulary, -Safety compliance leads to complex and costly tasks (e.g., ASIL D recommend strongly to perform Modified Conditions / Decision Coverage assessment which is costly and complex analysis), -Mandatory continuous software evolution required to avoid obsolescence (first law of Lehman [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF]), -Evolving standard and regulations to support, -Internal development process execution is complex due to sub-systems split over many teams and suppliers, -Difficulty to have objective quality assessment; for instance, management of suppliers with offshore tracking where quality reporters tend to escape strong engagement by reporting an average quality level.

Therefore, in this context, it is important to have a unified and right way to define, assess, control, or predict embedded software quality.

If we take Automotive-Spice, the guidance is to use ISO/IEC 25010 for quality model. Nevertheless, this standard lack of metrics, for example, and require a customization of quality model per ECU or transverse software to be operationalized. This statement is aligned with Wagner et al. survey [83]. Indeed, in this survey Wagner et al.

found that only 28% of the survey participant companies use quality model from standards and over them 71% customize them.

So, obviously a quality model coming from standard should not be used as it is. We note that such quality models are often too generic because they aim to cover a wide range of cases. These models are regularly ambiguous in their quality characteristic definitions, with a risk to be misunderstood and then hard to use, and they usually require customization. From another perspective, if we look for quality models for software, we can find more than 450 more or less distinct quality models.

Thus, it is not trivial to select one knowing that usually they are very case specific, hard to reuse or adapt. We remark that creating a new quality model for automotive embedded software can drive us to that same situation: a new specific quality model that could be hard to reuse or adapt. Thus, there seems to be a discrepancy between to be at the state of the art, with a collection of more than 450 models, and the fact that we must consider standard like ISO / IEC 25010 despite its issues.

Finally, the solution should consist in a trade-off between these two states and consequently, we should consider an alternate approach like qualimetry. This guides us to the following research questions:

Research Question 1 Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials?

Our first step is undeniably to understand and synthetize with clarity and unambiguity the concepts of quality, quality modeling and qualimetry.

Once their main characteristics described, we should be in a position where we will be able to confirm or deny that qualimetry is the right approach to use for quality quantification of automotive embedded software with regards to other approaches.

The goal is to collect and consolidate an exhaustive list of quality models for software by proceeding in a systematic literature review as detailed by Kitchenham and Charter [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF].

We are not limiting the research time range, but from our early exploratory literature review, the oldest publications of software quality models we found were the quality assessment model of Rubey and Hartwick [START_REF] Rubey | Quantitative measurement of program quality[END_REF], and the reliability prediction model of Shooman [START_REF] Shooman | Probabilistic reliability: an engineering approach[END_REF].

So, we suppose that our oldest finding won't be older than 1968, or at least than 1965, year during software engineering concept emerged.

Research Sub-question 2b

Considering a set of quality models, how to classify these quality models, what are the methodology, the criteria, and the characteristics to use?

Then, to analyze, get some fruitful benefits and make decisions based on that exhaustive list of quality models, we must structure its knowledge.

Therefore, we are going to apply the right taxonomy, defining an appropriate set of criteria, or taxons, to index and classify these models.

The expected result from this research sub-question, in addition to that classification methodology and its criteria, is a cartography depicting the current panoramas of quality models for software.

Research Sub-question 2c

Considering at least two quality models, how to compare together quality models, and can we define a reliable distance formula between quality models?

Through this sub-question, we aim to clarify the relevant characteristics and criteria to compare quality models together, keeping in mind that we are looking for the most appropriate quality model for our needs, from a classified list of quality models.

The choice or definition of a formula to evaluate the distance between two quality models can also serve for decision purpose, for example.

Research Sub-question 2d

What is the most appropriate quality model is for embedded software in automotive? This is the conclusion step of our research question 2.

From the classified list of software quality models we build, the key information we extracted, the comparison criteria we identified, and our knowledge about automotive embedded software, we are now able to conclude on which quality model is the most appropriate one for embedded software in automotive.

Research Question 3

Considering a quality model for a software product, how to operationalize it?

The quality model identification, or creation, corresponds to the first phase in quality modeling: the quality definition.

In the next phases, we operate the quality model. This operationalization enables its quality definition completion, quality evaluation activities (e.g., assessment, control, or prediction), and quality model evolution or adaptation management. However, there exist challenges and issues that may prevent the development or use of quality models as Thapar et al. demonstrated [11].

So, through this third research question, we set our focus on identifying potential issues that may prevent quality model operations, on finding practical solution(s) to these potential issues and then, on applying our findings against our needs for automotive embedded software. The application success to this real-world use case is the proof that we have answered to our problematic. We note that the operationalization aspect is also fundamental for our research work to fulfill our company expectations (see Chapter II.2).

The following research sub-questions address that problem decomposition:

Research Sub-question 3a

What are the main challenges and the issues that prevent operationalization of quality model?

As we could expect, this first research sub-question explores the main operational challenges and issues that prevent quality model development and use. Their synthesis is going to be reflected in a list of challenges and issues with their own descriptions.

Research Sub-question 3b What are the practical solutions to those challenges and issues?

For each of the listed operational challenges and issues, we target to find practical solutions to all of them. Practical solution means that the solution not only solve the linked challenge or issue, but also it must be usable and deployable. Therefore, general statement or theorical solution alone should be prohibited for those solutions.

Research Sub-question 3c What is the process to ensure quality model operationalization?

Over the two previous research sub-questions, the main operational challenges, and issues, which prevent development and use of quality model, are listed, and associated at least to one practical solution.

In order to finalize a proposal to solve our third research question, a process should be elaborated, integrating the practical solutions to those operational challenges and issues, to ensure quality model operationalization.

Research Sub-question 3d

What is the practical answer to our needs on automotive embedded software case?

The last step of our research work is to exercise our findings against the real-use case coming from our industrial needs. The success of this application is key because it should assess the correctness and relevance of our research work.

Therefore, through this sub-question, we apply our practical solution to automotive embedded software development, taking into account any necessary automotive specific requirements (i.e., related to variants, complexity or constraints from standard, regulation, and stakeholders).

The result should be the development and use of the right operational quality model.

Research Question 4

Can we have a unique reference quality model for software product?

The successive responses to theses first three research questions, including their respective research sub-questions, allow us to answer to our original needs to have a unified, operational, and appropriate way to define, assess, control, or predict quality of embedded software.

However, with this final research question, we would like to go one step further. Indeed, Zouheyr Tamrabet et al. [6] concluded their study by willing to propose a generic quality model "that encompasses relevant quality attributes in order to define the quality of embedded software".

So, we expect to assess whether a unique reference quality model for software product is more appropriate than a meta-model as quality model aggregator for software product., and afterward define how to build and initiate the construction of either this unique reference quality model or this metamodel.

Research Sub-question 4a

Is it possible to have a unique reference quality model for software product, or instead should we have a meta-model?

Based on our research work, our findings, and a relevant analysis, we target to find and demonstrate the proper answer to that research sub-question. Then, depending on the answer result, we select the correct research sub-question set.

Case 1: Answer is a "unique reference quality model" for software product is possible

Research Sub-question 4b

What is the construction algorithm for such unique reference quality model?

To elaborate and build such quality model, we design the unique reference quality model construction algorithm.

Research Sub-question 4c

What is the first result of the unique reference quality model construction?

We initiate the build of the unique reference quality model. Hence, we execute the construction algorithm, provide the intermediary construction step results to facilitate its comprehension, and deliver the first quality model result.

Case 2: Answer is a "unique reference quality model" for software product is not possible and therefore a "meta-model" is the preferable solution

Research Sub-question 4b What is the construction algorithm for such meta-model?

To elaborate and build such meta-model, we design the meta-model construction algorithm.

Research Sub-question 4c What is the first result of the meta-model construction?

We initiate the build of the meta-model. Hence, we execute the construction algorithm, provide the intermediary construction step results to facilitate its comprehension, and deliver the first meta-model result.

Threats to validity and discussions

The purpose of this last section is to discuss, as needed, about what has been covered in the previous parts and, above all, review threats to validity of this chapter content.

Regarding our analysis and synthesis of our complex industrial research context with a need of quality, we chose to provide a high-level overview. In fact, a deeper analysis and description bring certainly much more details, but the risk is to lose from our mind the most important aspects that characterize our research context and therefore diverge from our original intention. However, at a later stage of this dissertation, we are going to push further some analysis and then detail more precisely, for instance, automotive embedded systems.

On the methodology for the appraisal of the state of the art on quality modeling for embedded software, we decided to perform an exploratory literature mapping instead of a systematic literature review, described in Kitchenham and Charter's guidelines [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF]. The reason is directly linked to their definitions. Indeed, on the one hand, the exploratory literature mapping aims to support the elaboration of unanswered research questions by collecting theory, method, or empirical evidence on a specific topic. On the other hand, a systematic literature review follows a clearly defined protocol (e.g., acceptance or rejection criteria to consider a literature contribution) and relies on a strict analysis and synthesis of the identified research contributions, to answer to a properly defined research question. By consequence, since we were looking for the current state of the art to set and refined our research questions, the exploratory literature mapping appears to be the obvious methodology to apply in this chapter.

Furthermore, with the literature review, we enlarged our research scope. First of all, we extended our scope from automotive embedded software to any embedded software, with the goal to avoid narrowing too much the literature findings and allowing us to learn from other embedded software cases. Then we encompassed embedded system to embedded software research scope because there are both closely related: systems requirements are derived to hardware requirements and to embedded software requirements. Thus, the inclusion of quality modeling for embedded system in our literature review is relevant. In addition, we decided to accept reliability growth model as a valid result. This particular aspect of quality modeling is complementary. Indeed, it covers quality prediction aspect, is often based on statistical or implicit model using historical data -as opposed to hierarchical quality models (e.g., ISO /IEC 25010 quality models)-, relies on various sources of metrics even other than software metrics (e.g., process metrics), and it depicts objective quality evaluation (e.g., comparison of the quality model result over successive software release).

At last, when we elaborated our research questions, we took some assumptions. Qualimetry is our privileged approach for quality modeling, mainly because it is the science of quality quantification, without being restricted to a specific domain, and consequently covers theory and applied aspect of quality quantification. We avoid limiting quality characteristics, or requirements, to non-functional quality, despite the existing overlaps between "quality requirements" and "non-functional requirements" that we noticed in the literature survey related to embedded software. Thus, we decided to not use quality attributes found in our survey to build a new quality model, but rather to construct on top of existing research contributions and quality models, fostering the reuse of many valuable achievements.

Chapter III. Research Methodology

Introduction

Over Chapter II, we explored our industrial context and the problematic related to quality modeling within the context of automotive embedded software development. We also performed an exploratory literature review about this problematic, analyzing many valuable contributions but also highlighting different gaps. That analysis allowed us to define a series of four main research questions that we refined whenever it was necessary.

The purpose of this chapter is to pursue our study by explaining the research methodology we applied to address these four research questions. It is therefore a pivot chapter since it structures the rest of the dissertation. Indeed, our research work didn't follow a straight line, and we had to change the direction of our research approach at a certain moment.

So, this chapter give the overall picture of our research flow, explaining the different steps, the technological locks, the decisions we made and gives hints about our contributions. For example, when we looked for quality models, we noticed that most of the contributions reused and changed slightly what was previously done rather than creating a complete disruption in the research. Thus, we decided to reuse and adapt quality model rather than creating a new one but with some disruption created from biology knowledge and analogy. Figure 7 depicts the summary of our research flow.

Initial research methodology: qualimetry, classification and decision

During our initial analysis of the problematic, we identified three research questions (cf. Table 3). We thought about the fourth one (see Chapter II.4) only during our second analysis, at a later stage of the research. Thus, starting with the first research question (i.e., "Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials?") as our primary step, we seek to understand and learn accurately the complex concepts of quality, quality modeling, and qualimetry. We also seek to assess that qualimetry is the right path to solve our problematic, considering that it fosters both quality quantification theory and practice, for example. And since qualimetry is often misunderstood, we build a synthetic vision depicted by the "House of Qualimetry" (cf. Chapter IV.6.a).

Moreover, during our investigation on quality modeling, we notice a recurrent fact. Effectively, quality models are often more or less similar, and the variants for objects of interest (from a quality modeling point of view) is handled like the customization with Horgans' essential views of quality characteristics [START_REF] Horgan | An essential views model for software quality assurance[END_REF], the commonality versus variation in quality characteristics of Choi et al. [START_REF] Choi | Practical S/W component quality evaluation model[END_REF], or variation in metrics with FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF], to cite few.

So, the global observation is that there is no generalized method to manage those variants on quality modeling. Nevertheless, in object-oriented programming and genetic there is a simple way to manage variety via polymorphism which represents multiple variations or evolutions of a same object, or gene.

We consequently introduce the polymorphism concept applied to quality modeling (see Chapter IV.6.b) to enable built-in adaptation and evolution of quality modeling objects (e.g., quality model, quality characteristic, measurement).

We pursue our quest of knowledge then on quality model for software and rapidly, our findings tend to demonstrate that many quality models exist. For example in Oriol et al. study [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF], the authors indexed up to 51 distinct quality models, building a genealogical tree of quality models highlighting parent links between them. Unfortunately, we fail to find an exhaustive reference list of software quality models.

Then, we naturally move to the second research question (i.e., "Considering the set of software quality models, how to identify and decide which quality model is the most suitable for embedded software?") that we split into two complementary fields (see Figure 6): theoretical field and practical field.

Our focus in the theoretical field encompasses the research from a theory perspective while in the practical field, the research embraces a practical perspective.

Theoretical field

Thus, in theory field, we start to identify the most appropriate methodology, categories, and criteria to classify or index quality models. We find few studies about classification with some categories (e.g., Thapar et al. [11] classification criteria relied on operational issue number ; Kläs et al. [START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF] defined a classification scheme based on quality model characteristics) but none of them are unified. So, recalling the "genealogical tree of quality models" shown by Oriol et al., we push that reasoning further and consequently arrive at the conclusion that we should use cladistic to classify the variety of quality models (see Chapter V.4.a). Cladistic is usually used to classify species, but it can also be used to organize and simply classify a large list of quality models, highlighting some specific connections between those models.

Next to the classification is the decision methodology. Indeed, the purpose of this second stage of the theoretical field is to identify criteria, characteristics and / or formula to compare together quality models and finally decide on which quality model(s) fits best the requested needs.

About the comparison, a distance formula is a relevant approach to measure how closed quality models are together. A quality model is generally composed of different level of quality characteristics. So, we took the assumption that the distance formula in our case should focus on these characteristics, and since they are expressed in natural language, we may rely on string metrics.

However, we prefer the Nei and Li formula from genetic [START_REF] Nei | Mathematical model for studying genetic variation in terms of restriction endonucleases[END_REF] (see Chapter IV.6.c) rather than the frequently used string metrics such as Hamming's distance [87] (i.e., general sequences of symbols difference distance), Levenshtein's distance [START_REF] Levenshtein | Binary Codes Capable of Correcting Deletions, Insertions and Reversals[END_REF] (i.e., string edition distance ; Hamming's distance generalization), Damerau-Levenshtein's distance [START_REF] Damerau | A Technique for Computer Detection and Correction of Spelling Errors[END_REF] (i.e., string transformation distance ; extension of Levenshtein's distance and distance used also in biology as measure of variations on DNA sequence), Jaro's distance [90] (i.e., string similarity distance), Jaro-Winkler's distance [START_REF] Winkler | String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage[END_REF] (i.e., string similarity distance ; extension of Jaro's distance), Gordieiev et al. 's cumulative characteristics matching based distance [START_REF] Gordieiev | Evolution of Software Quality Models in Context of the Standard ISO 25010[END_REF], [93] (i.e., string semantic distance), Natural Language Processing (NLP) based distance [94] (i.e., string lexical distance), or even other similarity distances like Jaccard's distance [START_REF] Jaccard | Distribution de la Flore Alpine dans le Bassin des Dranses et dans quelques Régions Voisines[END_REF] (i.e., general sample similarity distance based on sample attributes). The main reason of this choice is that compared to the other formulas, the genetic approach introduces a statistical notion that we can use to represent the statistical presence of quality characteristics.

Practical field

Parallelly to the theoretical field, the practical field has also two stages synchronized with their corresponding theoretical stage. The first one is related to the application of the classification methodology on a case study. Our original case study is the embedded software but as we noticed above (cf. Chapter II.3), the set of embedded software quality model is quite small (i.e., 17 models) and therefore, we decide to extend the scope to software quality models. So, we perform a systematic literature review (see Chapter V.2) on software quality models, and during this study, we remark that we get a collection of more than 450 quality models. Many of them are justified variations of some previous quality models. The second stage takes as input the output of this first stage (i.e., the classified set of software quality models) and applies the decision methodology in order to identify the most appropriate quality models to apply to our company use cases.

Finally, the result of the second stage of practical field must be deployed and operationalized at our company level in order to answer to third research question (i.e., "Considering a quality model for a software product, how to operationalize it?") and also to our problematic. The complete breakdown of this research flow is summarized over the Figure 6.

Nevertheless, the decision task is tricky because quality model differences are sometimes subtle, and there exists many criteria that we could use with no guaranty to make a right choice, especially from a qualitative point of view. And in case of decision mistake, the risk impact may be reduced, if those decision criteria are properly chosen. However, decision action means that we discard many valuable contributions that could be very beneficial for the solution at the end.

Thus, if we want to minimize proactively the impact of a wrong, not totally accurate decision, or even of a partial solution, we must bring some nuances in the decision result and, integrate some of the potentially discardable contributions to our quality model solution prior to its deployment.

In short, we must reconsider our original strategy about research approach, and update accordingly our heuristic. 3. Realigning our methodology: from theory to practice So, instead of directly deploying the selected quality model to the company, we realign our research strategy to address operationalization aspect of quality model and proactive integration of some quality model valuable contributions, that a decision result could discard but should not afford to ignore.

To do so, we investigate how to go from the theory to the practice in quality modeling, reviewing the key theory concepts and the practical contributions in that domain. Thus, to answer to our third research questions (i.e., "Considering a quality model for a software product, how to operationalize it?"), we must work on these two aspects.

First, we identify the potential issues that may prevents quality model operations (i.e., development and use).

We also find practical solution(s) to these potential issues, identifying the most important contributions in quality modeling, and assess their respective similarities or alignment.

Second, we integrate these practical solutions through a consolidated process for the development and use of quality models.

So, with further details, we begin by looking for papers dealing with quality model operationalization, or issues preventing their use or development. For instance, Thapar et al. [11] identified 9 issues that prevent development and use of quality models, Abran et al. [96] highlighted three main harmonization issues with ISO-based quality models, Kläs et al. [START_REF] Kläs | Adapting Software Quality Models: Practical Challenges, Approach, and First Empirical Results[END_REF] investigated software quality model adaptation, and Wagner et al. [START_REF] Wagner | Quality Models in Practice: A Preliminary Analysis[END_REF] were proceeding on a survey of quality models in practice among four software companies in Germany.

Based on the review and consolidation of the findings from these retrieved papers, we build a first list of issues.

Next, we proceed on a further analysis and raise our list to a total of 16 issues that prevent development and use of quality models (see Chapter VI.2). To continue, we address one by one these issues to find some practical solutions over the 50 years of research contributions in quality modeling (see Chapter VI.3). Once each issue has been associated to at least one solution, we design two processes to take implicitly benefit of these key contributions and to foster their usage.

The first process (see Chapter VI.4.a) focuses on the development or adaptation of quality models, emphasizing their reuse. Indeed, even if we select a quality model for a subjective reason (e.g., use of standard quality model for compliance to standard), this process allows to integrate contributions from other quality models, and to be aligned with key stakeholders and any constraints. The second process (see Chapter VI.4.b) is dedicated to quality model use and call the first process in is early stage.

Finally, to close the loop from the theory to the practice, we must successfully apply our findings against a real use-case.

This real use case is naturally linked to the embedded software from our automotive industrial context: three ECU embedded software and one transversal embedded software. In addition, the quality model selection criteria have to be aligned also with our industrial context, and therefore to the guidelines from A-SPICE standard which focuses on automotive development processes.

Thus, we select the two recommended quality models of ISO / IEC / IEEE 25010 (i.e., system and software product quality model, and quality-in-use model), and apply our development process for quality model to them. The result (see Chapter VII.5) is three polymorphic quality models with their characteristics and sub-characteristics weights, and their "mother", or common, quality model that could be used by default for a new ECU or architecture, for example. Regarding their associated metrics, they are linked to a scorecard, where each polymorphic quality model is used to compute quality product indicator.

So, we demonstrate that our new approach allows us to start from a subjective selection, arrives to an objective and operational solution, and consequently, assesses the correctness of our answer to our company problematic.

Last step: construction of a meta-model

As we highlighted previously, at this stage of our research we have answered to our original problematic. However, during our studies we noticed that there existed many quality models but no unique reference quality model. Moreover, we subjectively selected two quality models from ISO standard as our reference models, like the 71% of the 28% of the companies that Wagner et al. studied [83] and which use and customize quality models from standard. So, to go one step further, we aim to address that last research question "Can we have a unique reference quality model for software product?".

Zouheyr Tamrabet et al. [6] finished their study with the opening to propose a generic quality model "that encompasses relevant quality attributes in order to define the quality of embedded software". Unfortunately, this quest is hardly reachable, and even if we succeed on getting such unique and generic quality model, that solution may become either less satisfactory, because of the constant evolution of our real-world resulting in constant evolution of most of systems and software (e.g., Lehman's laws [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF], [99], [START_REF] Arch | Lehman's Laws of Software Evolution and the Staged-Model[END_REF] are depicting that evolution aspect), or unsuitable, because of the wide diversity of both systems and software we have nowadays. So, the preferable approach for this research question is to build a meta-model as quality model aggregator.

This meta-model can serve as an advice quality model where the main quality perspectives (e.g., Garvin quality perspectives [101]) are present and where we have the notion of likelihood to have a quality characteristic or sub-characteristic as well as consideration to their nuances which are present in existing quality models (e.g., resource utilization, resource behavior and utilization of resource ; instalability and deployability). That description is comparable to what we can find in genetic, more particularly with the genome.

Therefore, strong from this analogy, we decide to construct a meta-model which is the genome of software quality model. After defining the meta-model ontology, we initiate its construction based on eight quality models.

To select this set of quality models, we take the quality models from Thapar et al. study [11] with the least operational issues that we complete with quality models from standard or widely used. We also pay attention to the fact that those selected models must have definitions for all their quality characteristics and subcharacteristics. At last, we limit the selection results to eight quality models since the purpose is to initiate the meta-model construction. The selected quality models are ISO/IEC 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF], Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF],

McCall [41], Alvaro [START_REF] Alvaro | A Software Component Quality Model: A Preliminary Evaluation[END_REF], FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF], Kalaimagal Q'FActo 12 [START_REF] Kalaimagal | Q'Facto 12: an improved quality model for COTS components[END_REF] and Bawane [START_REF] Bawane | A Novel Method for Quantitative Assessment of Software Quality[END_REF]. The genome builds finally result in six distinct quality perspectives, with one of them (i.e., supportability) that can be interpreted as a polymorphic variation of one of the other five quality perspectives (i.e., product revision).

Those quality perspectives are:

-General Utility, -Product in Use, -Product Operation, -Product Revision, -Product Transition, -Supportability, -Software Product.

Threats to validity and discussions

Like in previous chapter, this section goal is to review the chapter content, and particularly the threats to validity.

The main treat is with regards to our decision to change the research approach. Undoubtedly, we could proceed further on the creation of an oracle to predict the best quality model for quality definition and assessment, continuing our investigation of a sharpener list of quality model selection criteria, and then use experimentation to validate the oracle prediction results. However, proceeding on such selection is only one side of the problematic and certainly results in discarding many valuable and complementary contributions in quality modeling. Indeed, during our research investigation on quality model, we noticed that there were many valuable contributions that it would be interesting to take advantage of. And moreover, we noticed that many published quality models are not operationalized yet. Thus, operational criteria should be included also as decision criteria. Nevertheless, such inclusion may consequently reduce even more the pool of candidate quality models to only few, but unfortunately with no guaranty that, from a qualitative point of view, the decision result is satisfying. So, the real problem is not to find the best quality model but rather to make a subjective selected quality model (i.e., selected based for some reasons such as criteria matching, use of standards, company forcing a solution, to cite few) to become the right and optimum objective quality model for our needs. This is the main motivation of our research methodology realignment.

Regarding our source of inspiration to elaborate solutions, we can obviously find many clues of research direction in software engineering since its research domain is very active. However, sometimes we have to think out of the box to tempt to bring some disruption. Like Oriol et al. [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] who relied on genealogical tree to represent kinship line of quality models for web-service, our choice is to refer to biology, making analogy between our research cases, technical locks and what we can find in biology.

Therefore, we borrow some solution from biology like cladistic to classify quality models, but also from a subbranch of biology: the genetic. We start with polymorphism, concept present also in object-oriented programming, and the degree of polymorphism, or variety, formula that includes likelihood on the elements considered for the measurement, as opposed to regular software engineering distance formulas where that notion is missing. Then, the comparison of a quality characteristic with a specific DNA sequence (i.e., a gene), and thus a quality model with a chromatid (i.e., a chromosome is composed of two identical chromatids), allowed us to confirm the use of the degree of polymorphism in our case, and to construct a meta-model: the Software Quality Model Genome.

All those research results affirm our choice of considering biology as one of our research drivers.

In conclusion, the doubt we had with regards to our initial research methodology, and which caused the methodology realignment, together with our inspiration from biology were finally a salvation to solve successfully our problematic while bringing some disruption.

Chapter IV. Quality, Quality Modeling and Qualimetry

Introduction

The purpose of this chapter is to answer to the first research question, and therefore explore and understand accurately the complex concepts of quality, quality modeling and qualimetry.

Research Question 1 Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials?

Indeed, quality definition evolved since the ancient philosophers on the 5 th century B.C., and today a standardized definition is hopefully available. Nevertheless, there are still some further subtilities to capture with quality like, for example, the notions of quality perception with Kano's model [58], Garvin's quality perspectives [101] and quality dimensions [START_REF] Garvin | Competing in the Eight Dimensions of Quality[END_REF], Horgan et al.'s essential views [START_REF] Horgan | An essential views model for software quality assurance[END_REF], lagging or leading indicators [START_REF] Roedler | Systems engineering leading indicators guide -version 1.0[END_REF], [START_REF] Roedler | Systems engineering leading indicators guideversion 2.0[END_REF].

Regarding quality modeling concept resulting in quality model, it is pivotal for quality definition and evaluation activities. We remark that, whatever the type of quality models is, those models can be perceived as composed of dependent (i.e., computed from observation) and independent (i.e., observed) variables, themselves consequently associated to either indicators, quality characteristics, sub-characteristics or measures.

So, we consolidate its knowledge thanks to a review of the key contributions in quality modeling. About the last concept, we have to get to know qualimetry which is in fact the science of quality quantification, and more particularly what the essential characteristics of that science are.

Finally, with this tripartite knowledge (i.e., quality, quality modeling and qualimetry), we are able to affirm which quality modeling or quantification solution is the right one for our problematic case, parallelly contributing to the resolution of the technical locks we encountered. [START_REF]IEEE Standard Glossary of Software Engineering Terminology, IEEE Std[END_REF] which is as well similar to part 1 and 2 definition of ISO/IEC/IEEE 24765. Additionally, to this quality definition, there are some further nuances and concepts around quality to understand in order to capture the very essence of quality. We cover them in the next sub-sections.

The essence of quality

a. Integral quality

The first further nuance about quality is the concept of "integral quality". This concept mainly comes from the socioeconomic field [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], [START_REF] Lobanov | The Basic Concepts of Qualimetry[END_REF]. There are debates about integral quality [START_REF] Lobanov | The Basic Concepts of Qualimetry[END_REF], nevertheless, this concept is important as its joints quality with cost. This last one may vary for same type of object of interest, depending on project model, for example. Thus, the integral quality corresponds to the sum of quality with cost effectiveness (cf. equation ( 1)). The cost effectiveness is described by the object properties that are linked to the input capital for production and consumption of that object. This nuance with quality is important because selecting one or the other concept can lead to different results in quality related activities. -Must-be / basic: these are the basic mandatory customer expectations or requirements. Their absence5 creates dissatisfaction while their presence won't create any satisfaction. Moreover, it is not required or expected that they are asked by customer (e.g., this is the case of product legacy features). -Attractive / exciting: these are not expected by customer. Their presence creates satisfaction while their absence won't create any dissatisfaction (e.g., this is the case with product innovation or disruptive features). -One-dimensional / required: These are the new customer expectation or requirements. Their presence creates satisfaction while their absence creates dissatisfaction. Satisfaction and dissatisfaction are proportional to the performance of implemented requirements (e.g., the new planned product features). -Indifference area: In this area neither absence, nor presence of customer expectations or requirements don't significantly influence customer satisfaction or dissatisfaction. Moreover, the quality perception evolves with time because of the evolution of customer expectations: an "attractive" product feature may become a "required" product feature after a certain number of product releases, and even later, may become a "basic" feature.

Integral
As example to illustrate this evolution, we consider the creation of the first mobile-phone and then a new generation of it. When creating the first mobile phone, the basic feature was "to emit and receive phone calls", the required feature was "to be mobile" and the attractive feature was "to have call log". In the next generation, moving one step to our today' smartphone, the basic features became "to emit and receive phone calls" and "to be mobile", the required features were "to have call log" and "to send and receive SMS/MMS", and finally the attractive feature was "to take photos to attach to MMS". We note that this behavior is what Lehman characterized previously with his laws of software evolution [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF]. We have the same behavior with quality, and therefore, when dealing with quality, we must take into account these perception impacts and time evolution like in von Dran et al. [START_REF] Von Dran | Quality Websites: An Application of the Kano Model to Website Design[END_REF] or Liang et al. [START_REF] Liang | A Study on Using the Kano Two-Dimensional Quality Model to Evaluate the Service Quality of Government Websites[END_REF]. However, through his study [START_REF] Han | Evaluating perceived and estimated data quality for Web 2.0 applications: a gap analysis[END_REF], Han highlighted that these considerations on perceived quality are not obvious in quality evaluation activities because of the existence of a gap between perceived and measured data quality.

c. Quality perspectives and views

Through the description of the perceived quality, we could notice that quality is also a matter of point of view. Indeed, in 1999, Horgan et al. [START_REF] Horgan | An essential views model for software quality assurance[END_REF] -Transcendental view: quality as an ideal that cannot be defined but can be recognized, -User view: quality as a user's expected needs, -Manufacturer view: quality as conformity to specifications, regulations, standards, involving optimization during production and operation, -Product view: quality as the internal characteristics of a product that influence its external ones, -Value-based view: quality as how much a customer is willing to pay for the quality outcomes.

These five perspectives of quality are distinct and can be used alone or combined. As an example, a company which wants to optimize the maintenance cost should focus on the manufacturer quality perspective and may use a quality model accordingly to this perspective to predict its product quality during the development stages.

d. Quality dimensions and characteristics

Then in 1987, deepening his previous analysis, Garvin proposed eight critical dimensions of product quality [START_REF] Garvin | Competing in the Eight Dimensions of Quality[END_REF]. These are performance, features, reliability, conformance, durability, serviceability, aesthetics, and perceived quality. These quality dimensions must be aligned with the targeted quality perspective(s) to be used accurately. With this proposal, he illustrated the elicitation of quality requirements [START_REF]Part1: Quality Model[END_REF] from stakeholders.

Moreover, we noticed that Garvin indifferently used quality dimension or quality category to express the same concept. Over our literature studies, we also noticed the same behavior of using different terms for describing the same concept. The consequence is a mix of vocabulary, sometimes slightly antinomic between contributions. For example in Boehm et al. [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF], ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF] or ISO/IEC 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], the authors deal with multiple levels of quality characteristics and metrics, on their side, McCall et al. [41] consider perspectives, quality factors, quality criteria, and metrics, Dromey [START_REF] Dromey | A Model for Software Product Quality[END_REF] use quality attributes.

For all these sample cases, the targeted contribution is the creation of a quality model.

So, in order to avoid any confusion with regards to the terminology, here are the definition of the main keywords:

-Attribute: "Inherent property or characteristic of an entity that can be distinguished quantitively or qualitatively by human or automated means" (from ISO/IEC 25000 [25]), -Quality criteria (or quality standards): "the parameters established or adopted by an organization to measure the compliance of its products, services and processes to a certain defined standard" (from IAEA glossary terms [START_REF]Glossary of Terms used in the Planning and Design of the IAEA Technical Cooperation Programme[END_REF]),

-
-Quality model: "defined set of characteristics, and of relationships between them, which provides a framework for specifying quality requirements and evaluating quality" (from ISO/IEC 25000 [25]),

-Quality objective: "result to be achieved related to quality" (from ISO/IEC 9000 [START_REF] Antman | From Aristotle to Descartes a Brief history of quality[END_REF]),

-Quality requirement: "need or expectation related to quality, that is stated, generally implied or obligatory" (from ISO/IEC 9000 [START_REF] Antman | From Aristotle to Descartes a Brief history of quality[END_REF]),

-Indicator: "measure that provides an estimate or evaluation of specified attributes derived from a model with respect to defined information needs" (from ISO/IEC/IEEE 15939 [START_REF] Von Dran | Quality Websites: An Application of the Kano Model to Website Design[END_REF])

-Metric: "measurement scale and the method used for measurement" (from ISO/IEC 14598-1 [START_REF] Von Dran | Quality Websites: An Application of the Kano Model to Website Design[END_REF] and ISTQB Glossary [111]). -Quality measure: "derived measure (i.e., measure that is defined as a function of two or more values of base measures (i.e., measure defined in terms of an attribute and the method for quantifying it), from ISO/IEC/IEEE 15939 [119]) that is defined as a measurement function of two or more values of quality measure elements" (from ISO/IEC 25021 [121]) -Quality measure element: "measure defined in term of a property and the measurement method for quantifying it, including optionally the transformation by a mathematical function" (from ISO/IEC 25021 [121]) -Property to quantify: "property of a target entity that is related to a quality measure element and which can be quantified by a measurement method" (from ISO/IEC 25021 [121])

Finally, we build an ontology depicted by Figure 9, taking profit of these definitions. That ontology summarizes in one drawing the links between all these concepts.

Thus, to define, evaluate or predict quality, first we must enumerate and define accurately the corresponding quality objectives. These objectives are then refined into quality requirements and quality criteria, or standards. They also serve to identify the proper quality perspectives or views we aim our quality to focus on. Next, the criteria guide the determination of the necessary indicators set. That set is used jointly with the quality perspectives to construct the right quality models.

Quality models are directly elaborated from the set of quality characteristics compliant, not only to the quality model properties (e.g., type, perspective, lagging or leading indicator) but mostly to the quality requirements. At last, these quality characteristics are decomposed into quality sub-characteristics, attributes, sub-attributes, and metrics, as needed. The last complementary aspect to consider about quality is whether we deal with lagging or leading indicators.

The lagging indicators focus on the past performance, what already happened, and therefore measure the consequence (e.g., test results). They are easy to measure but hard to improve. The leading indicators focus on the current or future performance, prediction and then measure the causes. They are composed of characteristics, a condition which are both analyzed regularly or on-demand to predict a behavior with a certain level of confidence and a lap range (see Figure 10). They are hard to measure but relatively easy to improve (e.g., monitor test development skillset of a team). Nevertheless, that difficulty can be skirted thanks to the work lead by Roedler and Rhodes [START_REF] Roedler | Systems engineering leading indicators guide -version 1.0[END_REF], completed by Roedler et al. [START_REF] Roedler | Systems engineering leading indicators guideversion 2.0[END_REF], where a set of 18 leading indicators (e.g., requirements trends, interface trends, technical measurement trends, systems engineering staffing & skills trends, schedule and cost pressure) is fully described and explained.

Knowing whether quality indicators happen in lagging and/or leading space is crucial in activities depending on quality, including quality modeling and evaluation activities. This certainly influences the resulting quality model since it may be sub-optimal or inefficient. All these further knowledges and nuances are keys to capture the essence of quality and for succeeding in the modeling of quality, perceived or not.

The essence of quality modeling

Research Sub-question 1b What is the essence of quality modeling and particularly in software field?

The premises of quality modeling can be found through plethora of examples often associated to quality quantification. For instance, back to 1910's, the mechanic and shipbuilder A. N. Krylov defined quality measurement in a scientific manner to find the best warship design [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF].

From the previous Chapter IV.

a. Quality model

As we have seen above in Chapter IV.2.d, the current international ISO standard for software product quality requirements and evaluation (SQuaRE), ISO/IEC 25000 [25], defines a quality model as a "defined set of characteristics, and of relationships between them, which provides a framework for specifying quality requirements and evaluating quality".

Based on this definition, we notice that a quality model is used for multiple purposes such as quality requirement specification or quality evaluation. In 2009, Deissenboeck [START_REF] Deissenboeck | Software quality models: Purposes, usage scenarios and requirements[END_REF] classified quality models into three main categories: definition, assessment and prediction. This is the concentric DAP (i.e., Definition, Assessment, Prediction) classification (see Figure 11) where the further we move from the center, the more advanced our model is.

At the center of the DAP classification, we found the first type of usage for a quality model. This is the set of definition models which corresponds to "specifying quality requirements" through a set of quality characteristics and sub-characteristics linked together. The goal is then to identify, refine and organize all quality characteristics required for the quality 6 objectives, and then quality requirements, related to the focused object of the quality modeling activity. Figure 12 shows an example of such definition model ; this is the system / software product quality model from ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]. We note, for instance, that "Functional suitability" quality characteristic is refined into three quality sub-characteristics: functional completeness, functional correctness, and functional appropriateness.

The second type of usage is the assessment, or control. Control activity is close to assessment one, except that control is an assessment performed regularly. So, in order to assess quality, the definition model must be extended with measurement information. The current SQuaRE standard ISO/IEC 25020 [119] recommends to associate at least one software quality measure to each quality sub-characteristic. These software quality measures are then structured under a method to perform the measurement and the elements to measure with their properties, or attributes (cf. Chapter IV.2.d). For example, to measure the "Maturity" sub-characteristic of "Reliability" characteristic, we use the mean time between failure (i.e., MTBF) as the measurement method. The corresponding formula is defined as the ratio of the number of failures per duration period. Consequently, the measure elements are "number of failures" and "duration period", and the associated properties to quantify are then "failures" and "duration". This example is depicted in Figure 13. We remark that similarly than in mathematics, physic or statistics, the properties to quantify, or to observe, can be seen as independent variables -if the independency with the other independent variables is verified -and the measurement methods as dependent variables, outcomes of some combination of those independent variables.

This observation leads us to the last level in the DAP classification: the prediction models. As the model name indicates, the purpose of this type of model is to predict quality and its construction requires to have previously defined the quality (i.e., identify the quality characteristics, sub-characteristics and their relationship) and also the way to evaluate them. In general, this type of model cannot be used as a definition or assessment model because of this model construction often results in statistics or implicit model (e.g., regression analysis over historical data [START_REF] Khoshgoftaar | Predicting software development errors using software complexity metrics[END_REF]). Equation ( 2) illustrates a prediction model for software fault from Khoshgoftaar and Szabo [START_REF] Khoshgoftaar | A Poisson Regression Model of Software Quality: A Comparative Study[END_REF]. The authors elaborated a Poisson regression (i.e., named preg by the authors) model, based on the principal components derived from a set of observed software measures.

𝑭𝒂𝒖𝒍𝒕 𝒑𝒓𝒆𝒈 = 𝒆 𝟎.𝟏𝟗𝟑𝟗 𝟏.𝟏𝟐𝟒𝟖 * 𝑷𝑪 𝟏 𝟎.𝟐𝟐𝟕𝟕 * 𝑷𝑪 𝟐 ( 2
)
where 𝑃𝐶 = 𝑤 𝜂 . 𝜂 + 𝑤 𝜂 . 𝜂 + 𝑤 𝑁 . 𝑁 + 𝑤 𝑁 . 𝑁 + 𝑤 . 𝐿𝑂𝐶 + 𝑤 . 𝑋𝑄𝑇 and 𝑃𝐶 = 𝑤 𝑉 ( ) . 𝑉 (𝐺) + 𝑤 𝑉 ( ) . 𝑉 (𝐺)

PC1 is the principal component 1 and corresponds to a weighted 𝑤 sum of η1 (i.e., number of unique operators), η2 (i.e., number of unique operands), N1 (i.e., total number of operators), N2 (i.e., total number of operands), LOC (i.e., lines of code), and XQT (i.e., number of executable statements). PC2 is the principal component 2 set as a weighted sum of V1(G) (i.e., McCabe's cyclomatic number) and V2(G) (i.e., extended cyclomatic number, V1(G) + the number of logical operators (see paper [START_REF] Khoshgoftaar | A Poisson Regression Model of Software Quality: A Comparative Study[END_REF] regarding weights and further details). Additionally, to these three types of model, there is one last model to consider in the DAP classification. This is the ideal model, or multi-purpose model. Its particularity is to cover the three levels of DAP. Based on Wagner experience [START_REF] Wagner | Software Product Quality Control[END_REF], this type of model is rare and one example we can cite is Kitchenham's COQUAMO [START_REF] Kitchenham | Towards a constructive quality model. Part 1: Software quality modelling, measurement and prediction[END_REF].

COQUAMO is a constructive quality model and therefore, this model has the transversal capability to address definition, assessment, and prediction purpose. Nevertheless, the main difficulty with this quality model is around the unclear relationship with measurements. And as Wagner [START_REF] Wagner | Software Product Quality Control[END_REF] pointed out, on 1997 Kitchenman et al. finally concluded that "there were no software product metrics that were, in general, likely to be good predictors of final product qualities" [START_REF] Kitchenham | The SQUID approach to defining a quality model[END_REF] which calls into question COQUAMO as prediction model.

In parallel to the quality model purpose, with DAP classification, there exist several types of model representation: hierarchical, statistic/implicit, and meta-model.

Figure 12 illustrates the hierarchical model representation. In fact, instead of hierarchical, or tree, graph, we should rather speak about oriented graph (e.g., Boehm' s quality model [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF], [START_REF] Boehm | Characteristics of Software Quality[END_REF] presents some cycles like "Asis Utility", "Reliability", "Robustness /integrity" and "Human engineering" and therefore this model is not by definition a tree or hierarchical graph ; cf. Figure 14) which is a more generic graph representation, even if in most of the cases, the quality model is an hierarchical graph (e.g., ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF] or ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]). Nevertheless, it is possible to remove the cycles from the graphs by splitting and repeating some graph elements, resulting finally in a hierarchical graph. For instance in Figure 14, we can break the highlighted cycle by splitting and repeating the lowest level elements "Robustness /integrity", having one "Robustness /integrity" for "Reliability" and another for "Human engineering". The main advantage of this type of representation is its human comprehension, readability. Indeed, not only the information organized hierarchically relies on natural language, but also the hierarchical graph reflects the relationship between its elements in an obvious manner. However, quality evaluation requires further completion of the model, including measurement and aggregation method.

The next type of model representation is the statistics or implicit model. Those models are defined through mathematical model or function (e.g., Khoshgoftaar and Szabo's model [START_REF] Khoshgoftaar | A Poisson Regression Model of Software Quality: A Comparative Study[END_REF] shown in equation ( 2)). The principal interest of this type of model is the readiness to be used either for assessment or prediction purpose. This is due to the mathematical description which uses measurements as inputs. The main drawback is the complexity to understand or interpret the model, and the difficulty to retrieve the quality characteristics and sub-characteristics behind the model.

The last type is the meta-model. A metal-model describes the general concepts, depicting a view that shows the different these key concepts, their properties, their relationships (sometimes including description of relationship type) and their cardinality. This is similar to a UML class diagram or to an ontology (e.g., Figure 9). Thus, by conception a meta-model is a good candidate to become a multi-purpose quality model, but it requires to be applied against the object of interest in order to be used either for definition, assessment or prediction. To summarize, aligned to its purpose and representation, a quality model corresponds to the abstraction of quality of interest via the identification of independent and dependent variables related to quality, and completed with their mathematical or statistical relationships.

The independent variables are the measurements of quality characteristics (e.g., the eight critical dimensions of Garvin [START_REF] Garvin | Competing in the Eight Dimensions of Quality[END_REF]) while the dependent variables are the aggregated or combined results for some specific quality perspective (e.g., Garvin's five quality perspectives [101]).

The next fundamental element to proceed from a quality model definition to quantification, assessment, and quality control, deals with those independent variables. This is the measurement.

b. Measurements

Historically, there are three may streams of measurement theory [START_REF] Hand | Statistics and the Theory of Measurement[END_REF]. The "representational measurement" theory, initiated with the work of von Helmholtz in 1887 [START_REF] Helmholtz | Epistemological Writings, The Paul Hertz/Moritz Schlick centenary edition of 1921[END_REF], focuses on setting a relationship between objects and number systems via equivalence classes. The second theory, the "operational measurement", was introduced by Bridgman in 1927 [START_REF] Bridgman | The Logic of Modern Physics[END_REF]; its focus is put only on operations used to proceed to measure, neglecting relationships between equivalent measured objects. The last stream is not a single theory but rather a sort of "melting pot" of various other theories [START_REF] Diez | A Hundred Years of Numbers. An Historical Introduction to Measurement Theory 1887-1990 Part I: The Formation Period. Two Lines of Research: Axiomatics and Real Morphisms, Scales and Invariance[END_REF], [START_REF] Diez | A Hundred Years of Numbers. An Historical Introduction to Measurement Theory 1887-1990 Part II: Suppes and the Mature Theory and Uniqueness Representation[END_REF], minor in front of representational and operational theories.

In most of these theories, major or minor, we can find a common underlying factor: the scale. We cover "scale" concept in the Chapter IV.3.c.

In parallel to these stream of measurement theories, the scientific study of measurement, called metrology [START_REF] Bipm | The international vocabulary of metrology-basic and general concepts and associated terms (VIM), 3rd edition JCGM 200[END_REF], establishes convergences related to measurement in human activities through three overlapping active topics: definition, practical realization and traceability. The definition activity of units of measurement drives especially the definition unifications of unit with standard (e.g., international system of units (SI)) and vocabulary (e.g., the international vocabulary of metrology [START_REF] Bipm | The international vocabulary of metrology-basic and general concepts and associated terms (VIM), 3rd edition JCGM 200[END_REF] which is written in both English and French). Concerning the practical realization of these units of measurement, this activity aims to address calibration and uncertainty aspect of measurements, including with measuring devices. At last, the traceability activity focuses on bridging the practical realized measurements to the reference standards.

Another aspect linked to measure is the measurement properties. For instance, in Schneidewind's methodology to validate software metrics [START_REF] Schneidewind | Methodology for validating software metrics[END_REF], the author proposed six validity criteria applicable to software metrics for assessment, control or prediction: association, consistency, discriminative power, tracking, predictability, and repeatability. In his analysis of software measurements thanks to metrology [START_REF] Abran | Software Metrics and Software Metrology[END_REF], Abran focused on the design validation of software measures, including measurement methods, and highlighted key measurement properties such as maturity, precision, uncertainty, calibration and traceability. In health related field [START_REF] Mokkink | The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes[END_REF], it is frequent to use measurement reliability, measurement validity, construct validity, responsiveness for measurement tools or interpretability property, for example. Among all these properties, there are in fact two main critical properties to qualify a measure, encompassing the instrument and method used to perform it. These properties are reliability and validity of the measure. The reliability means how repeatable over time the measurement is providing consistent results (i.e., with same exact condition and same exact way to measure, measurement values must be equal). Part of reliability is measurement error and internal consistency. Concerning validity, this property means whether or not measured value is measuring what we are expecting to measure; it includes content validity, construct validity and criterion validity. The Figure 15 gives a good visual explanation of impact of these two measurement properties, based on the analogy of "target shooting". Here, our "sniper" is trying to hit the target shooting several times in the same conditions. If all shootings are in the target, result is valid, and if shootings are close together, we can conclude that the shootings are repeatable over time and therefore result is reliable. Starting from left to right of Figure 15, on the most left target, we no-ice that the shootings are both spread (i.e., not quite repeatable), inside and outside the target (i.e., they are not valid shootings). On the second target, the shootings are still spread but now all of them are inside the target; we can conclude that the shootings are valid but not repeatable, or reliable. With the third target, all shootings are grouped but some of them are outside; we understand that shootings are repeatable over time but not valid. In the last case not only, shootings are close together and they are all inside the target; the result is therefore valid and reliable. Therefore, we must be in that last case when proceeding with measurement.

Finally, we note that despites which definition is used in any of those theories or activities, "size, amount, or degree" and "standard units" are direct references to the scale theory of measurement, or scale types.

c. Scale types

Scale is essential when we measure quality attributes or properties, with the goal of either assessing that the measurements are matching some criteria (e.g., being above a certain threshold) or controlling quality by comparing their differences over a certain period, or even predicting what will be quality measures. It enables measurement interpretation, standardization, and comparability by describing mathematically types of element with their corresponding operators, properties, functions, permissible statistics, and distributions.

In 1946, S. S. Stevens 7 introduced and published a theory of scales of measurements [START_REF] Stevens | On the Theory of Scales of Measurement[END_REF] which is part of the representational measurement theory. This theory is still widely used by scientist nowadays, despite criticism from statisticians [START_REF] Velleman | Nominal, Ordinal, Interval, and Ratio Typologies are Misleading[END_REF]. Stevens categorized data into four typologies or scales: nominal, ordinal, interval, and ratio. The 'nominal' scale represents the group of data related to labels or type of numbers and words or letters (e.g., {"Red", "Green", "Blue"} or {"True", "False"}). It can be used for classification and membership assessment. The 'ordinal' scale corresponds to the data set characterized by order, rank, and therefore allows comparison and sort of elements: for instance, it is possible to tell that a quality result is better or worse to another one. The third scale, 'interval', is extending the field of possible operations one step further by adding difference and affinity operations between values to be meaningful. Thereby, distances between measures are used to take action and decision. For example, man can tell that a quality result is not only better or worse than previous one but also about how much it progresses in term of difference. And last scale is 'ratio'. At that level, and as its name is indicating, the ratio -or magnitude -between measurement values is relevant and useful. It has also the important property of having a unique and non-arbitrary zero value (i.e., Kelvin temperature scale is a good illustration of a ratio scale since its zero absolute corresponds to -273.15° Celsius, while Celsius temperature scale is an interval scale).

Table 4 summarizes for each scale its associated measure properties, the permissible statistics, the mathematical operators, and group. It is then possible so see that some operations or manipulations are permitted for a specific scale but not for another scale. As illustration of this fact can be found with ISO/IEC 33020 [START_REF] Abran | Software Metrics and Software Metrology[END_REF].

Indeed, this standard describes a process measurement framework for assessment of process capability, and for example, there are two process capability rating tables, one based on 'nominal' scale (see Table 5) and the second one based on 'ordinal' scale (see Table 6).

It obviously appears that it is possible to compare together measurements in 'ordinal', knowing which one is greater than the other. In 'nominal', this type of operation is not possible, nevertheless we can classify or verify the group membership. Consequently, an accurate scale selection is primordial to operate, process and manipulate appropriately the measures.

In addition to the Stevens' four scales of measurement, there exist several other alternate scales such like Chrisman [START_REF] Chrisman | Rethinking levels of measurement for cartography[END_REF] who proposed an extension of the original Stevens' scale list in 1998, shown in Figure 16. That extended typology is interesting since it is not diverging from Stevens' one. It includes therefore six new scale types that are useful in some particular cases (e.g., gradation of membership is key in fuzzy set theory, log interval is widely use in stock market graphics, cyclical ratio is suitable for times and angles).

Rating Meaning Description

N Not achieved There is little or no evidence of achievement of the defined process attribute in the assessed process.

P Partially achieved

There is some evidence of an approach to, and some achievement of, the defined process attribute in the assessed process. Some aspects of achievement of the process attribute may be unpredictable.

L Largely achieved

There is evidence of a systematic approach to, and significant achievement of, the defined process attribute in the assessed process. Some weaknesses related to this process attribute may exist in the assessed process.

F Fully achieved

There is evidence of a complete and systematic approach to, and full achievement of, the defined process attribute in the assessed process. No significant weaknesses related to this process attribute exist in the assessed process. To conclude on this part, it is important to select, apply and use the right measurement scale for each property, attribute, and object where we want to apply metrics. Scale brings mathematical and statistical "tools" on measures, rating (e.g., ISO / IEC 33020 [START_REF]-Information technology -Process assessment -Process measurement framework for assessment of process capability[END_REF]) allowing discretization, sampling (i.e., scale change) and validity and reliability of measure such as in [START_REF] De Souza | Psychometric properties in instruments evaluation of reliability and validity[END_REF]. Knowing limits and recommendations from statisticians [START_REF] Velleman | Nominal, Ordinal, Interval, and Ratio Typologies are Misleading[END_REF], but also the fact that it is still widely adopted by scientists, we can safely consider Stevens' typology as the main scale typology to use. And, when it is required to bring more details, accuracy and control on measurement levels, Chrisman's extended types are offering a good practicable complement.

d. Aggregation

Aggregation consists in operations with the objective to combine elements together, going from a large set of elements to smaller set of elements. In quality modeling, the aggregation is implicit with statistical quality model (e.g., equation ( 2)), but for the other types of quality model, this is an important operation because it allows to join properly quality measures, attributes, and characteristics for different purposes, thanks to the correct aggregation operators.

According to Wagner [START_REF] Wagner | Software Product Quality Control[END_REF], there are five major aggregation purposes to consider: assessment, prediction, hot spot identification, comparison and trend analysis. For assessment, the objective is to assess the aggregation results likes for a product quality assessment, where the measures of its quality attributes are aggregated to define the product quality measure. Prediction purpose is identical to assessment but with a scope of prediction, and frequently use of statistic model. About Hot spot identification, the intent is to focus on specific area of interest, or properties instead of all system or software. It is used for improvement. Comparison aims to compare sets of aggregated elements (e.g., systems, software components). Regarding the last purpose, trend analysis, the goal is to focus on the evolution over time of some specific, or all, set of elements of interest.

Knowing the aggregation purpose is important but not sufficient to clarify the objectives behind the data aggregation. In fact, we must also identify which aggregation operator category we need before selecting the correct aggregation operator(s). Wagner enumerated seven categories: grouping, rescaling, cluster analysis, central tendency, dispersion, concentration, ratio and indices [START_REF] Wagner | Software Product Quality Control[END_REF]. The goal of grouping operators is to regroup elements together into sets, and therefore create groups. Concerning rescaling, this is close to the grouping category. Indeed, the first step for this kind of operators is to group elements together following a certain scale, and then change that scale. For example, we start with ordinal (e.g., Table 6) and then move to nominal (e.g., Table 5). Cluster analysis operators are closed to rescaled group, but the clusters are identified with a more thin and refined analysis (e.g., K-means algorithm). The central tendency category corresponds to determine an "average" of elements. Such operators can be mode8 , median, arithmetic mean, geometric mean, or harmonic mean. The dispersion operators qualify the scattering of inputs. In this category we have variation ratio (e.g., for hotspot identification), maximum, minimum, range, median absolute deviation, variance, standard deviation.

From an opposite perspective, concentration operators (e.g., Lorenz curve -gives graphical hint of the concentration-, Gini coefficient) focus on finding the strongest data cluster (i.e., closest data together). Finally, the ratio and indices operators are binary operators because they are defined as the ratio of two measures, where the ratio divider can be also a reference index value instead of a measure). Thus, this category is composed of fraction measures, relation measures or indices.

In addition to the aggregation operators indicated in the above category descriptions, it is possible to find many other aggregation operators. We are not detailing them here but rather initiate their exploration, first by citing the mathematical and behavioral properties that defined them and second, by providing a catalog of the main existing operators. Definition and applicability of these properties and operators are detailed in Detyniecki [START_REF] Detyniecki | Fundamentals on Aggregation Operators[END_REF] or Wagner [START_REF] Wagner | Software Product Quality Control[END_REF]. So, the properties that defined, or characterized aggregation operators are:

 Mathematical properties: boundary conditions, Monotonicity (non-decreasing), continuity, associativity, symmetry, bisymmetry, absorbent element, neutral element, idempotence, compensation, counterbalance, reinforcement, stability for a linear function, invariance.  Behavioral properties: decisional behavior, interpretability of the parameters, weighs on the arguments.

The catalog of the main existing aggregation operators is described in Table 39 in Annex 1. We remarked that some operators are weighted (e.g., weighted mean). These weight factors depict the fact that characteristics are of relative important to the other characteristics. Moreover these weight factors are often determined thought surveys [START_REF] Wagner | Software Product Quality Control[END_REF], [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF], or expert-nonexpert-hybrid methods from qualimetry [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]. Finally, despite this list of the main aggregation operators, respecting the mathematical and behavioral properties, there is an alternate aggregation approach to include to complete that list. Indeed, in 1997 Dujmovic and Bayucan introduced the "Logic Scoring of Preference" method and the "Continuous Preference Logic" operators [START_REF] Dujmovic | A Quantitative Method for Software Evaluation and its Application in Evaluating Windowed Environments[END_REF] to evaluate quality software products -in their particular case, it was windowed environment software products. The main idea, shown in Figure 17 and by the equations (3), is to specify the relationships (i.e., the "Continuous Preference Logic" operators) between the entries Ei and the output E(r), relying on some trade-off between conjunction (i.e., low entry values influence output the most), neutral and disjunction (i.e., high entry values influence output the most). Then the result of the "Continuous Preference Logic" operators gives the parameter r which is then injected in the weighted power mean of weighted entries (i.e., Ei) to determine the output result E(r).

𝑬(𝒓) = ( ∑ 𝒘 𝒊 . 𝑬 𝒊 𝒏 𝒊 𝟏 ) 𝟏 𝒓 𝐟𝐨𝐫 𝒓 ≠ +∞ 𝐚𝐧𝐝 𝒓 ≠ -∞ 𝐦𝐚𝐱(𝑬 𝟏 , ⋯ , 𝑬 𝒏 ) 𝐟𝐨𝐫 𝒓 = +∞ 𝐦𝐢𝐧(𝑬 𝟏 , ⋯ , 𝑬 𝒏 ) 𝐟𝐨𝐫 𝒓 = -∞ (3)
where 0 ≤ 𝐸 ≤ 1, 0 < 𝑤 < 1 for 𝑖 ∈ [1; 𝑛] and 𝑤 = 1 e. Threshold

To complete our overview of the fundamental elements that characterized the essence of quality modeling, we end with threshold concept. This concept is closely related to decision making and quality criteria, or quality standards where compliance criteria rely on the comparison between measurements and threshold value (e.g., whether measurements are below, on, or above threshold value). Thus, threshold is mandatory to assess, control or even predict the compliance of products, services, projects, and processes, but it is optional when the quality modeling purpose is only to create, or manage, definition quality model.

Even if a threshold can be perceived as a level that may trigger decisions or actions, we found six distinct categories of threshold from academic and industrial domains.

 Acceptance, or acceptable: this is the worst but still acceptable value of a measurement ; this category corresponds to the most frequent use case of threshold and can be linked to risk management (i.e., this is the minimum value from which risk is manageable) (e.g., Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], ISO/IEC 25022 [145]),  Forecasted: this is the estimated or predicted value of a measurement ; this category is used within a safety or a process maturity context, for instance (e.g., ISO 26262 [3], ISO/IEC 9126-4 [START_REF]-Information technology -Process assessment -Process measurement framework for assessment of process capability[END_REF], ISO/EC 25040 [START_REF]-Information technology -Process assessment -Process measurement framework for assessment of process capability[END_REF]),  Opportunity: this is the minimum value of measurement from which opportunities (e.g., in economics, health, environmental) can occurs; it is often associated to management of risk and opportunity (e.g., ISO/IEC 25022 [145]),  Rejection: this is the least worst but still rejected value of a measurement ; it is an alternate category to the acceptable one (e.g., Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]),  Reference: this is the best value of the measurement currently achieved during evaluation period ; the reference is usually the best world-wide achievement and may be used as target (e.g., Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]),  Target: this is aimed value of the measurement; this category is between, or identical to, the acceptance and the reference ones(e.g., Wagner [START_REF] Wagner | Software Product Quality Control[END_REF], Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], ISO/EC 25040 [START_REF]-Information technology -Process assessment -Process measurement framework for assessment of process capability[END_REF]).

Another interesting aspect of threshold is the simultaneous use of several thresholds from same or different categories. For instance, the use of two acceptance thresholds defining a range of acceptable values, or the use of acceptance, target, and reference, or again acceptance with opportunity (cf. Figure 18). Our final highlights on threshold are regarding two specific difficulties that may occur during threshold elaboration. Firstly, it is sometimes hard to specify objective threshold but, as Arhens et al. [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF] shown, it is possible to get around this obstacle with the use of relative threshold (e.g., reference against a previous release, results to see progress against it). Secondly, the relationship of quality characteristics to each other may influence threshold values. Khaddaj and Horgan [START_REF] Khaddaj | A Proposed Adaptable Quality Model for Software Quality Assurance[END_REF] established quality characteristics relationship to each other by means of a factor polarity profile (see Table 7).

This polarity profile identifies the dependency links between characteristics or sub-characteristics. There are three types of polarity profile relationship: direct (i.e., if a characteristic A is enhanced, the related characteristic B is likely to be enhanced), neutral (i.e., if a characteristic A is enhanced, the related characteristic B is unlikely to be enhanced) and inverse (i.e., if a characteristic A is enhanced, the related characteristic B is likely to be degraded). For instance, an increase in the security quality characteristic may improve, or degrade the overall performance quality characteristics. This example illustrates the inverse relationship. Further-more, the relationship behaviors of polarity profiles are applicable as well to the thresholds or levels of quality assigned to each quality characteristic.

In conclusion, to define the essence of quality modeling: namely, the abstraction of quality of interest via the identification of independent and dependent variables related to quality (e.g., quality characteristics and subcharacteristics), completed with their mathematical and statistical relationships.

⌀ ↷ ⇅ = = = = = = Security (S) ↷ ⌀ ⇅ = = ↷ = = = Cost/Benefit (CB) ⇅ ⇅ ⌀ ⇅ ⇅ ⇅ ⇅ =  = Timeleness (T) = = ⇅ ⌀ = = = =  = Correctness (C) = = ⇅ = ⌀ = = = = Maintainability (M) = ↷ ⇅ = = ⌀ = = = Reliability (R) = = ⇅ = = = ⌀ = = LDF LDF1 = = = = = = = ⌀ = … ⌀ LDFn = = = = = = = = ⌀ Legend: ⌀ None (same characteristic)
⇅ Direct (if criterion A is enhanced, then criterion B is likely to be enhanced) = Neutral (if criterion A is enhanced, then criterion B is unlikely to be enhanced)

↷ Inverse (if criterion A is enhanced, then criterion B is likely to be degraded)

Key contributions to quality modeling of software

To consolidate quality modeling theory, a narrow set of the most well-known, significant, or influential quality models or contributions to quality modeling of software, is frequently cited in papers or used as a basis. In the following content, after providing a timeline with the most significant contributions that we found in that field (see Figure 19), we give details about each of them.

-1965 -Software Engineering concept [START_REF] Berkeley | The Computer Directory and Buyer's Guide[END_REF]: even if Alan Turing proposed the first theory about software, and more precisely about algorithm, on 1936 [START_REF] Turing | On Computable Numbers, with an Application to the Entscheidungsproblem[END_REF], the "software engineering" concept only emerged in the 1960s. The first publication that used this term, was "The Computer Directory and Buyer's Guide" of June 1965 [START_REF] Berkeley | The Computer Directory and Buyer's Guide[END_REF]. This is the first key contribution of the timeline.

- In addition, Kano et al. found that the perception of quality evolve with the time, similarly to Lehman's laws of software evolution [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF], and therefore, quality definition and evaluation must be adapted accordingly. This model can also be used to identify the most important product attributes from a customer perception. This model covers product, user and value-based quality perspectives.

-1987 -Kitchenham's COQUAMO [START_REF] Kitchenham | Towards a constructive quality model. Part 1: Software quality modelling, measurement and prediction[END_REF]: This research work was performed within ESPRIT 9 project for reliability and quality of European software. Thus, in this context Kitchenham defined the COnstructive QUAlity MOdel (i.e., COQUAMO) approach, a 'do your own' quality model quite similar to the Boehm's constructive cost model, also known as COCOMO, approach [START_REF] Boehm | Software Engineering Economics[END_REF]. She then used it to create a quality model based on Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF] -1994 -the Goal-Question-Metric (GQM) paradigm and quality model of Basili et al. [START_REF] Basili | Goal Question Metric Approach[END_REF]: The Goal-Question-Metric (GQM) paradigm and quality model of Basili et al. was first cited in 1984. It is frequently used as a common process for creating quality models. Its process flow starts by identifying quality goals, followed by the related requirements needed to meet those goals, and finally completed by a set of metrics to monitor the extent to which the requirements have been met. The quality perspective focused with this approach and the resulting quality models cover product, manufacturer, and user views.

-1995 -Dromey's "5 steps" approach and quality model [START_REF] Dromey | A Model for Software Product Quality[END_REF]: After noticing that quality assessment differs for each product and consequently requires a process to handle this behavior, Dromey defined a 5-step process to evaluate and construct a hierarchical quality model based on ISO/IEC 9126. His approach is crucial because it allows quality model verification by establishing criteria for deciding whether to include software properties as an integral part of a model, and a way of assessing whether the model is complete or not. The quality perspectives addressed here are user and product views.

-1999: Horgan et al.'s essential views model [START_REF] Horgan | An essential views model for software quality assurance[END_REF]: During his PhD on "Construction of a Quality Assurance and Measurement Framework for Software Project", Horgan noticed a specific behavior related to quality characteristics, or factors. Indeed, he found that some factors were key, that is to say, almost invariant whatever the project, product or stakeholders were (e.g., some quality factors that are always requested or required) and some factors were merely local to a project, product or stakeholders (e.g., a quality factor specific to a project and inadequate for another one). Horgan named them as the "Key Quality Factors" and "Locally Defined Factors". He merged them under the essential views model which corresponds to both product and user quality perspectives. McCabe's cyclomatic complexity, Halstead's metrics, function points, use case points), but also addressed specifically two issues: "how to correctly design software measures, and how to recognize if a software measure is well designed, and worth using as a basis for decision -making". To do so, the author used the International Vocabulary of Basic and General Terms in Metrology [START_REF] Bipm | The international vocabulary of metrology-basic and general concepts and associated terms (VIM), 3rd edition JCGM 200[END_REF] as the foundations of his studies on software metrology, software measure design and measurement methods, and then illustrated the COSMIC [158] design as an practical example of the lesson learns with those analyses Finally, through this work, Abran aimed to avoid the main practical pitfalls with software measure use in the industry by setting up a trust relationship between software measure use and industry practitioner and managers.

-2011 -the ISO/IEC 25010 quality model [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]: Subsequent to the ISO/IEC 9126 standard, ISO/IEC 25010 [13] was initiated at the end of 2007 and the result was published In 2011. This is the current official standard for systems and software-quality models -with a wider scope than ISO/IEC 9126; consequently, it is actively and widely used. At the current time, ISO/IEC 25010 consists of two hierarchical quality model parts: a systems/software product quality model, with 8 quality characteristics and 31 quality subcharacteristics, and a quality-in-use model, with 5 quality characteristics and 9 quality subcharacteristics. This standard is supplemented with a data quality model, ISO/IEC 25012, for which there are 15 "data quality characteristics that are required or evaluated from inherent and/or systemdependent points of view" [START_REF] Diez | A Hundred Years of Numbers. An Historical Introduction to Measurement Theory 1887-1990 Part II: Suppes and the Mature Theory and Uniqueness Representation[END_REF]. Suggested metrics are given by the ISO/IEC 2502n series. Like its predecessor, the quality perspective scope of ISO/IEC 25010 is based on a user and a product view.

-2013 -Wagner' software product quality control [START_REF] Wagner | Software Product Quality Control[END_REF]: on 2011, Wagner started to synthetize and consolidate in a book the knowledge about software quality product. He integrated many research contributions in that field, particularly thanks to his participation as the project leader of Quamoco project [START_REF] Wagner | The Quamoco Product Quality Modelling and Assessment Approach[END_REF] in which he accumulated experience. This project focused on quality models and quality evaluation, and hence the book content covers those aspect. Indeed, the author began with miscellaneous definitions and terminologies related to quality, then continued with knowledge about quality models and software measures, description of quality planning and control, and ended the book by sharing and describing six practical experiences Wagner had during the 10 years prior to this book. So, the compiled knowledge about quality model and evaluation, complete with a 10 years' experience sharing makes this book a key contribution.

-2015-Azgaldov et al. ABC of Qualimetry [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]: Through this book in English, Azgaldov et al. extended the diffusion to qualimetry knowledge to a wider audience and continue to contribute to this science. We note that this book is a key contribution because, many contributions to that science are in Russian or address to some specific domain such as socioeconomics, and hopefully this book opens a bridge to other discipline, such as software engineering. This is the last key contribution of the timeline.

Qualimetry: the science of quality quantification

Research Sub-question 1c What is qualimetry, and is this approach the right one for our needs?

Qualimetry, from the Latin qualis "of what kind" and the Greek μετρεω "to measure", can be described as the science of method and problem solving for quality quantification of any kind of object such as service, product, people, project or process [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF].

The concept of quality quantification is not recent as was indicating G.G. Azgaldov [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], the founder of qualimetry; Diez in [START_REF] Diez | A Hundred Years of Numbers. An Historical Introduction to Measurement Theory 1887-1990 Part I: The Formation Period. Two Lines of Research: Axiomatics and Real Morphisms, Scales and Invariance[END_REF] also clearly emphasized this fact, citing works done by Helmholtz in 1887 or by Campbell in 1920. However, until late 1960's the quantification of quality was exclusively done for only one specific type of object at a time, primarily product oriented one and without direct or explicit reuse or generalization over other similar objects. On the beginning of 1968, in former U.S.S.R., a scientist group of interest around problems linked to quality quantitative evaluation and control, published a common summary paper of their workshop [2]. The force of this group was due to the fact that its members were coming from a large variety of domain horizons (e.g., economists, architects, civil engineers, car makers) and shared the same concerns to unify theories and practices used for quality quantification. It was the ignition of an international discussion that led to the birth of qualimetry, a new scientific discipline, during 1968.

In addition, Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF] demonstrated that qualimetry was not only a scientific discipline but a real science by itself, reminding what Plato said on the 5th century B.C. "Exclude from any science mathematics, measure and weight, and it is left with very little". In 1981, that science was split into two distinct disciplines: theoretical qualimetry [START_REF] Florenskii | Some Remarks on Product Quality Assessment[END_REF] (i.e., focusing on problems and method issues, with a mathematical view to object to evaluate) and applied qualimetry [START_REF] Azgaldov | Applied qualimetry: its origins, errors and misconceptions[END_REF] (i.e., application of qualimetry to evaluate type of objects that were not evaluated before). We remark also that qualimetry relies on domain-independent concepts, or foundations, and therefore can be applied to any domains.

In order to quantify quality, Azgaldov et al. defined a general quality modeling process via a quality assessment algorithm [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]. This algorithm is composed of 9 sequential steps that rely on Quality Evaluation Method (QEM) design, planification and execution. A QEM is a method used to define and assess quality, including definition of evaluation context, quality characteristics, their measurable properties, or indices, their weight factors, to cite some of them. There exists three distinct QEM: rigorous, short-cut, approximate. A rigorous method aims to be exhaustive in the quality evaluation, to reduce error and to obtain the most reliable results, but it is expensive in term of resource and time (e.g., detailing with a high precision details quality characteristics and subcharacteristics, using all available measures to evaluate the quality). At the opposite, short-cut QEM tends to have narrow quality evaluation, less accurate results, but still acceptable, and therefore is cheaper in term of resource and time (e.g., applying ISO/IEC/IEEE 25010 with only one metric per quality characteristics). The last QEM, approximate, is between rigorous and short-cut. Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF] highlighted that short-cut is the most used internationally. Also, the team which is the source of information for QEM values can be either formed by expert, non-expert, or hybrid (i.e., mix of expert and non-expert) from the domain where the quality evaluation is required. In addition, the data origination used by qualimetry is a hierarchical quality model and to support the analysis and the model elaboration, a set of about 30 quality model tree derivation rules have been defined. The 16 principal rules are listed in Table 8, for general rules, and in Table 9, for specific rules, while their details are available in Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]. Specific rules to be used if the amount of information obtained in a quality assessment can be reduced thought the use of the rank scale 13 Exclusion of equally expressed properties when the rank scale is admissible 14

Truncated tree when the rank scale is admissible Specific rules to be used if the amount of information obtained in a quality assessment may/may not be reduced by more precise methods

15

Incomplete tree when a short-cut assessment of quality is admissible 16

Complete tree when exact quality assessment alone is admissible After this introduction and global outline of qualimetry, we must address the second part of our research question: "is this approach the right one for our needs?". To recall our needs, we look for a unified, operational, and appropriate way to define, assess, control, or predict quality of embedded software, including handling of reuse, variant and respect of standards and regulations (cf. Chapter II.2). So, to answer to that question, we propose to compare with other existing approaches and then determine which one is the optimum one. We have identified eight main distinct streams of approaches for quality model development and use: Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], with qualimetry, Basili [START_REF] Basili | Goal Question Metric Approach[END_REF], with GQM paradigm, Dromey [START_REF] Dromey | A Model for Software Product Quality[END_REF], with the 5-step approach, Gilb [START_REF] Gilb | Principles of Software Engineering Management[END_REF], with the principles of software engineering management, Khoshgoftaar and Munson [START_REF] Khoshgoftaar | Predicting software development errors using software complexity metrics[END_REF], to cover stream of statistic models, ISO / IEC 250nn [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], [121], [123], [145], [147], [START_REF]-Systems and software engineering -System and software product Quality Requirements and Evaluation (SQuaRE) -Measurement of system and software product quality[END_REF], [165], to cover stream of hierarchical quality models (including standard), Kitchenham [START_REF] Kitchenham | Towards a constructive quality model. Part 1: Software quality modelling, measurement and prediction[END_REF], [START_REF] Kitchenham | The SQUID approach to defining a quality model[END_REF], with COQUAMO and SQUID, and Wagner [START_REF] Wagner | Software Product Quality Control[END_REF], with QUAMOCO. We may note that there exist other works quite similar to these quality modeling approaches, nevertheless, these eight streams are a good synthesis of the current distinct approaches, and their main characteristics, from a quality modeling perspective summarized in Table 10. Moreover, these approaches can be regrouped into three categories: specific quality model solution, generic quality model solution and general methods.

First, the specific quality model solutions which have a narrow focus. This is the case with Khoshgoftaar and Munson [START_REF] Khoshgoftaar | Predicting software development errors using software complexity metrics[END_REF] where the authors showed how to develop and use statistic or implicit quality models. This type of approach is often applied to a specific assessment or prediction case, and therefore it is hardly generalized or reused. Another type of specific solution is the one from Gilb [START_REF] Gilb | Principles of Software Engineering Management[END_REF] who proposed some principles of software engineering management to have a good software quality. He considered that quality requirements come from the required operational properties of software product, or from its maintenance environment. His solution finally is very specific and address neither development, not use of quality model. The second category is the generic quality model solution. We choose ISO/IEC 250nnn [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], [121], [123], [145],

[147], [START_REF]-Systems and software engineering -System and software product Quality Requirements and Evaluation (SQuaRE) -Measurement of system and software product quality[END_REF], [165], the current standard for hierarchical quality modeling on software product, as the representative solution here. Indeed, it provides a good illustration of existing contributions to create and publish other hierarchical quality models, and that can be found in literature in general. ISO/IEC 250nnn suffers from being too general since its quality characteristics / sub-characteristics definitions are relatively ambiguous (e.g., Abran et al. [96]) with the objective to cover a maximum of cases. In addition, this standard requires not only customization (e.g., Wagner et al. [83]), but also a method to complete, for example, aggregation and weight factor part.

The third and last category encompass the general methods for development and/or use of quality model. Dromey [START_REF] Dromey | A Model for Software Product Quality[END_REF] 5-steps approach, is a method which focus on the quality model development and more particularly on the assessment of the built quality model. Regarding Basili's GQM paradigm [START_REF] Basili | Goal Question Metric Approach[END_REF] (generalization of McCall Factor/Crieria/Metric [41]), the approach is performed over 3 stages, starting from a series of goals, which are split into a proper set of questions and where set of metrics, associated to these questions, are answering to the goals (e.g., Shepperd [START_REF] Shepperd | Early life-cycle metrics and software quality models[END_REF]). This is a general method with a wide scope, but it doesn't pay attention to some details such as weight factors, aggregation or even operationalization. GQM result is subjective because of its process flow starting from goals. Kitchenham [START_REF] Kitchenham | Towards a constructive quality model. Part 1: Software quality modelling, measurement and prediction[END_REF], [START_REF] Kitchenham | The SQUID approach to defining a quality model[END_REF] proposed methods for 'do your own model' with COQUAMO and SQUID. Thus, the approaches focus on only on quality model creation. Extending this work through QUAMOCO research project [START_REF] Wagner | The Quamoco Product Quality Modelling and Assessment Approach[END_REF], Wagner [START_REF] Wagner | Software Product Quality Control[END_REF] consolidated in a book a complete set of knowledge accompanied by methods to develop and use software product quality model. At last, Azgaldov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF] represent the general scope of qualimetry approach, quality assessment algorithm that includes characteristic and property identification, tree construction, weight factors aggregations, and therefore covering both development and use of quality model too.

In conclusion, by analyzing all the cited approaches together, and then against our needs, two optimum approaches stand out: -Wagner and his book, which is a consolidation of knowledge and experience in quality modeling, -Azgaldov et al. and qualimetry, which is the science of quality quantification, merging methods and approaches from various domains. Both Wagner's approach and qualimetry appear to be exhaustive and complete compare to the other approaches, dealing with theory and applied aspect. Nevertheless, compared to Wagner's approach, qualimetry theoretical and applied aspect are actively developed, supported by a large community of scientists from a variety of fields (e.g., architecture, economy, civil engineering, factory), and also addressing a larger scope of object to quantify, even if Wagner focused on software product which is well aligned with embedded software.

Consequently, qualimetry is the optimum approach for a unified, operational, and appropriate way to define, assess, control, or predict quality of embedded software.

However, we noticed that we could improve qualimetry with the following contributions.

6. Contributions to Qualimetry a. The "House of Qualimetry"

The first contribution to qualimetry is linked to its comprehension. In fact, this relatively young science, which has a large scope, is not widely used even in software engineering and in systems engineering. Indeed, we encounter only specific applied qualimetry case studies which are mostly decorrelated from theoretical qualimetry. So, in order to leverage qualimetry to a large range of audience, foster its accurate understanding, and ensure that no major knowledges beneath it are eluded or forgotten, we propose a synthetized view of this science through the "House of Qualimetry" and its 6 pillars, depicted by Figure 20. We note that by consequence, this synthetic view answers to the first part of our research questions Q1 (i.e., "What are Qualimetry essentials which make Qualimetry a right answer to quality quantification?"): the qualimetry essentials. We initiated the elaboration of this synthetic view by exploring and analyzing qualimetry, on both aspects, theorical and applied one. Then, we used our knowledge and experience with regards to quality modeling in software engineering and systems engineering to determine the major qualimetry themes. In addition, we voluntarily limited their numbers because from experimental psychology, as Azgadnov et al. pointed out [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], we learn that the number of units of operational information an individual can handle simultaneously is limited to an average of 7±2. The result by Argotti et al. [START_REF] Argotti | Quality quantification in Systems Engineering from the Qualimetry Eye[END_REF] is shown in Figure 20 and an overall explanation is given below with a global summary in Figure 21.

To describe the structure of these concepts, we make an analogy with architecture, borrowing Doric architecture vocabulary. As a science, qualimetry relies naturally on two interlaced and complementary disciplines: theoretical [START_REF] Florenskii | Some Remarks on Product Quality Assessment[END_REF] and applied qualimetry [START_REF] Azgaldov | Applied qualimetry: its origins, errors and misconceptions[END_REF]. These two disciplines are combined into an entablature which relies on two architraves: "quality model" and "measurement". "Quality model" covers the identification, organization, and representation of the relevant quality characteristics while "measure" covers the evaluation, manipulation, and control of them. Furthermore, each of these two architraves is relying on a set of three pillars, all settled on a basement reflecting the object(s) of interest (i.e., the one(s) that is (are) aimed to be quality quantified).

i. The "Quality Model" pillars While the first pillar (i.e., "object analysis") is the major one, the other two are also mandatory in order to achieve the right quality model.

 "Object analysis" pillar: This pillar gathers the necessary knowledge and activities to understand, identify and organize the relevant quality goals, perspectives and characteristics linked to the analysis of our object of interest (i.e., the one that it aims to have its quality quantified). Thus, we first define the purpose of our analysis, aligned with the DAP classification (see Figure 11); we then analyze our object of interest in order to identify the quality objectives, perspectives, characteristics, subcharacteristics; attributes… that are relevant to us; finally, we decide how we are going to organize all this data. We can note that quite often the data organization is achieved via a hierarchical structure (i.e., tree structure) but can be also achieved through a statistical model.

 "Derivation rules" pillar: Here, the focus is with regards to global and specific qualimetry rules (see TABLE 8 and TABLE 9) to help support the analysis and optimize the design of the organizational data structure. For example, maximum tree height, division by equal characteristic, branch a tree until only simple or quasi-simple characteristics remain at its top tier.

 "Weight factors" pillar: Regularly forgotten, even in standards such as ISO/IEC/IEEE 25000 series where it is reduced to few words, the weighting factors are critical because they reflect the importance of quality characteristics among the same level of quality characteristics. ii. The "Measurement" pillars

As was the case for the previous set of pillars, these thee pillars are all mandatory in order to proceed accurately on measurement taking, even if the "theories of measurement" pillar represents the main one.

 "Theories of measurement" pillar: This pillar is composed of three main streams of measurement theories. [START_REF] Diez | A Hundred Years of Numbers. An Historical Introduction to Measurement Theory 1887-1990 Part I: The Formation Period. Two Lines of Research: Axiomatics and Real Morphisms, Scales and Invariance[END_REF], [START_REF] Diez | A Hundred Years of Numbers. An Historical Introduction to Measurement Theory 1887-1990 Part II: Suppes and the Mature Theory and Uniqueness Representation[END_REF]: operational measurement (i.e., how to operate or use the measure), representational measurement (i.e., how to represent the measure) and "various minor" theories. In a sense this is a fundamental pillar as it is bringing together all scale, mathematical and statistical tools for our measurements.

 "Aggregations" pillar: The aim is to deal with the way of combining (i.e., mean, median, variance and more [START_REF] Detyniecki | Fundamentals on Aggregation Operators[END_REF]) together either all or a subset of the measurements depending on their purpose [START_REF] Wagner | Software Product Quality Control[END_REF]. The aggregated measurements can either be weighted or un-weighted.

 "Thresholds" pillar: This pillar is associated with the measure of the ability to assess, control, or predict and therefore make the correct decision. In general, man is using two types of thresholds: acceptance and target. Acceptance is often confused with the rejection threshold even though they are not the same: the acceptance threshold is the worst-case threshold level that may be accepted, it lies just above the best case reject level. In fact, we counted six types of threshold exist as follows (refer to Chapter IV.3.e): rejection, acceptance, target, reference, forecasted and opportunities. Target corresponds to the threshold we are aiming for whereas reference corresponds to the reference value used in the industry or in the community at the time when the measurement is taken. Opportunities threshold corresponds to the level from which economical, health or environmental opportunities can occur. And lastly, forecasted allows to express estimated or predicted threshold, often used within a safety or a process maturity context.

Finally, by mean of the "House of Qualimetry", we popularize a complex concept with a synthetic view easy to remember, visualize, use, and understand. The Figure 22 illustrates an example of the application of two of three "quality model" pillars against "software product" as object to quality quantify, demonstrating that could elaborate the same quality models defined in the ISO/IEC/IEEE 25010 standard.

b. Polymorphism applied to quality model

Research Sub-question 1d How to unify diversity and time evolution in quality modeling?

Within the goal to answer to this research sub-question 1d, the second contribution aims to overcome a lack in quality modeling, and therefore in qualimetry. Indeed, even if there already exist some elements of solution, there is no generic mechanism to link, unify and homogenize quality modeling, during development, use and reuse of quality model. This is key for similar objects (e.g., variants of product) that are candidate to be quality quantified. For instance, in 1999 Horgan et al. [START_REF] Horgan | An essential views model for software quality assurance[END_REF] identified the Key Quality Factors as the quality factors that are invariant over similar product or project, but not equal, and the Locally Defined Factors as the quality factors that must be customized to represent the differences between similar product or project: for instance different releases of a smartphone series, variants of a car (e.g., same type of car but different options likes in premium versus low-cost, or different car types such as convertible, mini, sport, van for instance). Another example Grady and Caswell's FURPS model [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] where depending on the development process stage, metrics are adapted to current stage (cf. example extracted from Grady and Caswell's book and shown over Table 11). Fortunately, in genetic (e.g., Joron et al. [START_REF] Joron | Heliconius wing patterns: an evo-devo model for understanding phenotypic diversity[END_REF]) as well as in program-oriented object (e.g., Cardelli and Wegner [START_REF] Cardelli | On Understanding Types, Data Abstraction, and Polymorphism[END_REF]), we already have a mechanism for this type of behavior: the polymorphism concept. So, we propose to integrate the concept of polymorphism to quality modeling, quality model, and by extension we contribute to qualimetry.

Figure 23 -Polymorphism mechanism showcased with butterfly analogy: example of a generic butterfly and its variety of butterflies with heredity links (source: [START_REF] Meyer | Repeating Patterns of Mimicry[END_REF] & [START_REF] Joron | Heliconius wing patterns: an evo-devo model for understanding phenotypic diversity[END_REF])

Polymorphism means a multiplicity of morphs, or representations, which depends on phenotypic variety and / or temporal aspects. To explain how the polymorphism mechanism can be applied to quality models, we rely on an analogy with biology, substituting "butterfly" for "quality model". So, to describe the first aspect linked to phenotypic variety, we start with how a butterfly can be commonly perceived. A "generic" butterfly can be defined based on common characteristics found in any butterfly, such as a trunk, two antennas, two wings covered with colored scales, three pairs of thoracic legs, a body divided into head, thorax and abdomen … In the real world, this "generic" butterfly does not exist, but there are many different variants of it. And even though they are distinct from each other, all of them inherit from the traits of this "generic' butterfly. In Figure 23 our "generic" butterfly is symbolized by a "black butterfly" and on its immediate right, we see some of the varieties inherited from it. This illustrates one single level of polymorphism; however, we may have multiple levels of polymorphism and this is what the arrow of "phenotypic variety" is pointing out, showing an example of variety of butterflies with heredity links.

The second aspect of polymorphism is a temporal one. Continuing with the "butterfly" analogy, over its life, a butterfly, independently of its variant, evolves from eggs to caterpillar to chrysalid to new-born butterfly and when its wings are dry, to a fully-fledged (operational) butterfly flying away (see Figure 24). We remark that both phenotypic variety and temporal aspects demonstrate unification and homogenization properties. This, if we now transcribe polymorphism from butterfly to quality modeling, a polymorphic quality model means:

1. For the similar type of objects (i.e., phenotypic variety aspect), referring to Cardelli and Wegner [START_REF] Cardelli | On Understanding Types, Data Abstraction, and Polymorphism[END_REF] and Cook et al.

[172] regarding polymorphism, we may have a. Ad hoc polymorphism can be understood as apparent common quality model characteristics, or "interface". Nevertheless, it is defined by two type of behaviors: overloading (e.g., same characteristic name but different sub-characteristics or metrics depending on a context) and coercion (e.g., forcing a sub-characteristic to abort or to behave like its parent characteristic). For instance, ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF] or ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF] mark the beginning of ad hoc polymorphism because they are expected to cover a large spectrum of cases and their customization demonstrates coercion and overloading. FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] and FURPS+ [START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF], for example, demonstrate ad hoc polymorphism behavior with the overloading of the metrics used.

b. Universal polymorphism can be understood as variations with heritage between quality models. It is defined by three types of behaviors: inheritance (e.g., creating a new quality model from an existing quality model, keeping its characteristic definition and implementation; this is also called sub-classing), overriding (e.g., replacing the sub-characteristics or metrics of a characteristic in a new inherited quality model), and extension (e.g., a new inherited quality model has some extended new characteristics to cover specific quality objectives). For example, Horgan et al. [START_REF] Horgan | An essential views model for software quality assurance[END_REF] with the Key Quality Factors vs Locally Defined Factors and Khaddaj and Horgan [START_REF] Khaddaj | A Proposed Adaptable Quality Model for Software Quality Assurance[END_REF] with their adaptable quality model reflecting both ad hoc and universal polymorphism behavior (e.g., overriding of quality subcharacteristics related to a stakeholder view)

2. For objects over their project or product life cycle (i.e., temporal aspect), for example, a quality model can evolve, change (e.g., in the design phase we have a different focus than in the maintenance one). This is the case with the FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] and FURPS+ [START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF] models since the measures of characteristics evolve along with the product life cycle stages. McCall's model [41] is another example because of its three software product activities (i.e., product operation, product revision and product transition).

To sum up, we propose the polymorphism mechanism to consolidate qualimetry, and therefore enabling built-in adaptation and evolution in quality modeling. This concept is transparent, unify many existing contributions in quality modeling domains, and foster development, use and reuse of quality model.

Furthermore, it unifies diversity and time evolution in quality modeling by using natural links between similar, or identical, objects and their evolution over the life cycle stages.

c. Quality model distance formula

Research Sub-question 2c

Considering at least two quality models, how to compare together quality models, and can we define a reliable distance formula between quality models?

Continuing the strengthening of qualimetry, our next contribution to quality modeling is devoted to determining a reliable distance formula between quality models, and consequently, addressing the second part of this research sub-question 2c.

A distance formula is certainly a relevant approach to measure how closed quality models are together.  what the risks linked to such change are, considering not only that "small distance = low risk" and therefore "large distance = high risk", but also when a distance is small, change may be discarded, while a large distance reinforces the necessity to apply this change,  which areas are the most impacted, and where more changes in quality modeling happen, using the distance on each quality characteristic for instance,  how much work and resource it is going to cost to replace all, or partially, current quality model,  where quality quantification, assessment, control, and prediction are changing,  how deep the validation or evaluation path is changed, allowing to capture different types of issues possibly never found before and discarding other areas and paths, Quality model changes can also occur due to modification or evolution of targeted product, or to the stages in the product life cycle. Thus, this formula can be used to support decision, and to control change or update of quality models, including the case of polymorphism. It fosters split of quality model changes into reasonable change increments, from an agile and risk point of view.

Before deciding which formula dovetails with measuring the distance between quality models, we must distinguish two types of computing distance to explore.

The first type deals with the quantitative aspect of quality models. Thus, the idea is to determine a distance between quality models from the results they give when they are exercised during quality evaluation activities. This is for instance the case of Khoshgotaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF]. With the aim to find which software quality model, among a set of statistic models, has the best cost benefit value, the authors performed a cost benefit analysis based on the quality model results obtained over multiple software releases. So, here the distance formula was the profit evaluation (i.e., "Profit = [Benefit resulting of quality model use (e.g., early fix of fault prone components)] -[Cost of development and use of quality model]"). Another example is the case of Juneja et al. [START_REF] Juneja | Reliability Modeling for Embedded System Environment compared to available Software Reliability Growth Models[END_REF] on software reliability growth models comparison. The quality model distance was the estimated failure rate for a set of reliability growth models, thanks to a simulation against a specific use case. However, in both cases, this type of distance approach requires a non-negligible effort to deploy and use quality model against specific use case set, and the resulting distance values may be consequently neither valid, nor reused for a different use-case set.

The second kind of computing distance examines the qualitative aspect of quality models (e.g., quality characteristics and sub-characteristics). Indeed, if we refer to the ontology shown by Figure 9, a quality model is generally composed of different levels of quality characteristics. In this case, we take the assumption that the distance formula focuses on these characteristics, often expressed in natural language with words, or character strings. So, the most frequent and well-known word-based distances are:  Hamming's distance [87] is a widely used distance between two general sequences of symbols. The distance represents the number of symbols which differs between these two sequences (see Annex 2 and Table 40).

 Levenshtein's distance [START_REF] Levenshtein | Binary Codes Capable of Correcting Deletions, Insertions and Reversals[END_REF] can be considered as a generalization of Hamming's distance but on strings. It measures the difference between two strings by counting the number of edit operations (i.e., deletion, insertion, replacement) required to go from one string to the other (see Annex 2 and Table 41).

 Damerau-Levenshtein's distance [START_REF] Damerau | A Technique for Computer Detection and Correction of Spelling Errors[END_REF] is an extension of Levenshtein's distance and is also used in biology as a measure of variations on DNA sequence. This distance measures the difference between two strings by counting the number of transformation operations (i.e., string character edition, with deletion, insertion, and replacement, completed with transposition of two adjacent string characters) required to go from one string to the other. The additional operation (i.e; transposition) allows to compute a distance without being too much impacted by human typo (see Annex 2 and Table 42).

 Jaro's distance [90] is a measure of string similarity which is often used in string duplicate detection (see Annex 2 and Table 43).

 Jaro-Winkler's distance [START_REF] Winkler | String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage[END_REF] is a variant of Jaro's distance, well adapted for evaluating similarity of short string since it includes a notion of string prefix. Moreover, the Jaro-Winkler distance is normalized and therefore a distance value of 0 means no similarity, while 1 means the two strings are identical (see Annex 2 and Table 44).

An alternate group of approaches to all these sequence-, or string-based distances is the similarity distance over sample set. We can cite as principal example Jaccard's distance [START_REF] Jaccard | Distribution de la Flore Alpine dans le Bassin des Dranses et dans quelques Régions Voisines[END_REF] (see Annex 2) which is a general sample diversity distance based on sample attributes (i.e., the greater the distance value is, the lowest is the similarity).

The idea is to evaluate how much the intersection of the sample sets is covering their union. If the intersection and the union are superposed, then the sample sets are identical. Hence, these string, sequence, or sample set similarity distance formulas are powerful tools to detect matches between any level of quality characteristics, with a certain level of robustness. Unfortunately, they are manipulating quality characteristics only from a character string set point of view, without any attention to lexical or semantic nuance. For example, productivity is a synonym of efficiency, and thus term relatively closed lexically and semantically. Furthermore, any of the previous distance formula applied to them shall indicate an opposite verdict: mismatch of the two words. Consequently, these distances don't dovetail with a proper distance between quality models.

Apropos semantic and lexical inclusion in distance formula, Motogna et al. [94] combined some natural language processing measures to perform comparison of quality models through similarities analysis. The authors noted that each quality model characteristic is defined not only by a set of words (i.e., name of the characteristic), but also a description and a set of sub-characteristics themselves defined by a set of words (i.e., name of the subcharacteristic) and a description. So, in their analysis, the authors determined the matching characteristics of two quality models by taking the average of characteristic word set similarities (e.g., with Jaccard' similarity distance) and lexical characteristic similarities (e.g., Mihalcea et al.'s "corpus-based and knowledge-based measures of text semantic similarity" [START_REF] Mihalcea | Corpus-Based and Knowledge-Based Measures of Text Semantic Similarity[END_REF]). To be more precise, the lexical similarity evaluation collected the semantic equivalent list of words that matches the characteristic related word stems thank to WordNet lexical database [START_REF] Miller | WordNet: A Lexical Database for English[END_REF] available online at Princeton University "About WordNet" 11 . Nevertheless, their methodology didn't allow to determine a global distance between two quality models but rather locally to characteristics whenever a similarity can be calculated. With this latest research work, we have a real distance for quality models. It handles quality model structure, relationship between its elements (i.e., characteristics and sub-characteristics) and semantic nuances between them. Nonetheless, there are few flaws that should be addressed. For instance, the authors assumed that the weight of characteristic similarities and the one of sub-characteristics similarities are equal. From our point of view, and from Motogna et al. one too, this is not a correct assumption because it is not taking into account the fact that a characteristic is already defined by its sub-characteristics, and therefore the characteristic similarity evaluation has to include that impact. The distance formula should be also more generic (e.g., distance between any two quality models).

𝑑 = 𝐶𝑀𝑀 + 𝑆𝑀𝑀 , ( 4
)
where However, we remark that all the distances described above are mainly dealing with similarity and not with variety to compare together quality model characteristics.

To handle properly variety, especially with a large set of quality models, we must have a statistical notion about frequency or presence of quality characteristics in quality model, for example. This is not the case of any of the above distances. Hopefully, with polymorphism in genetic, there is a well-known formula to compute the degree of nucleotide diversity, also known as the degree of polymorphism. This is the Nei and Li formula [START_REF] Nei | Mathematical model for studying genetic variation in terms of restriction endonucleases[END_REF] (1979) depicted by equation 5.

This formula is made for DNA sequences from alleles and linked with the existence of single nucleotide polymorphism (i.e., variation of nucleotide at a specific DNA location for more than 1% of a population) [176]. Nevertheless, we can see an analogy between DNA sequences and quality characteristics sequences both of them coding behaviors (cf. Chapter VIII.2). So, in our case the πij represents the proportion of lexically and semantically different quality model elements (e.g., characteristics, sub-characteristics, attributes), and the xi the frequency of the i th quality model or characteristic sequence. In a case of quality model sequence, for instance, that frequency indicates the likelihood to find a quality model among the other models that are part of our sample set. In the same way for characteristic sequences, that frequency indicates the likelihood to find a characteristic among the other characteristics that are part of our sample set. For instance, if a characteristic recurrently appears in quality models, its probability is 1. If half of the time the characteristic is present and the other half is another close (i.e., not disjoint) characteristic, then their respective probabilities are 0.5.

𝜋 = 𝓍 𝓍 𝜋 (5)

where 𝜋 proportion of different nucleotides between sequences 𝑖 and 𝑗 𝓍 is the estimated frequency of the 𝑖 sequence in the population 𝓍 is the estimated frequency of the 𝑗 sequence in the population About lexical and semantic match, we consider three cases, identical, similar, or gaps (i.e., disjoint) and therefore the calculation of the corresponding πij is given by equation 6. Moreover, we use the unbiased estimate of π (cf. 12, Table 13 and Figure 25, proves us right: for instance "Maintainability" characteristic is present on both models but their respective sub-characteristics are identical only at 28.57 %. So, taking the assumption that the characteristics are already identical at 50% (i.e., weight of characteristic with regard to its sub-characteristics) conducts to an incorrect result.

𝜋 = 𝑑 𝑠𝑒𝑞 , 𝑠𝑒𝑞 (6)
where

To conclude on the quality model distance, our contribution reuses a proofed formula directly associated to polymorphism, or variety concept. Through is application against the two consecutive standards of software quality model, we assess that our formula is applicable and gives accurate results. We remark finally that our approach is close to Hamming's distance (i.e., number of symbols which differs between these two sequences), Indeed, the aim of a measurement process is not only to proceed on, or collect, measure but also to record and analyze the results, control quality, help on decision making, including doing some predictions and communicating the results to the right stakeholders. ISO/IEC 25040 standard [147] defines a coarse and linear evaluation process (see Figure 26) which define the main tasks that must be achieved for measurement, beginning by establishing the evaluation requirements and ending with the conclusion of the evaluation. This standard contains the same issue that we can detect with the rest of ISO/IEC 250nn standard series: it is not precise enough, willing to cover all case for computer system and software quality evaluation, and consequently, requires interpretation and strong complement.

Hopefully we may rely on practical work carried out for software related decision makers by, McGarry et al. [START_REF] Mcgarry | Practical Software Measurement: Objective Information for Decision Makers[END_REF] (see Figure 27) which introduces a process articulated around four activities: establish and sustain commitment, plan measurement, perform measurement and evaluate measurement. We note that this approach has been adopted by INCOSE, for example, for the Systems Engineering measurement primer and the technical measurement guide [177]. We can also find the measurement process introduction carried out by Miller et al. [START_REF] Miller | Systems Engineering Measurement Primer: a Basic Introduction to Measurement Concepts and Use for Systems Engineering[END_REF], which has a scope of systems engineering and the process published by Dekkers et al. [START_REF] Dekkers | Measures and Measurement for Secure Software Development[END_REF], a US-CERT team on secure software development.

Moreover, we recall that in automobile -our current industrial context -, A-SPICE [START_REF]13 / Automotive SIG[END_REF] guidelines points out a measurement process called MAN.6, part of management processes, but don't provide it. Instead, A-SPICE enumerate some recommendations linked to measurement process such as the type of process outputs for measures definition and collection, data and reports, some categories of measures to collect (i.e., product quality, field, project, risk, service level, process, personnel performance) and some base practices. Thus, based on all these measurement processes, and in order to be aligned with company requirements, we consolidate a measurement process based on A-SPICE MAN.6 process guidelines, McGarry et al. measurement process mode with regards to the core measurement activities, ISO/IEC 25040 for evaluation plan, inclusion of quality model and polymorphism in quality modeling which requires to be cadenced by the systems or software development life cycle. Our proposal of measurement process (cf. Figure 28) is therefore articulated into three sequential phases: Initial, Planning and Execution.

i. Initial phase

The purpose of this phase is to understand, identify and collect both requirements and context linked to measurement goals and activities. That phase is performed over three tasks which can be realized in parallel. The first task focuses on the identification and enumeration of all measurement objectives, taking measurement requirements as inputs. The second task is dedicated to the measurement context definition which can be understood as defining the scope, the boundaries, the dependencies, and the environment linked to measurement activities. The last task of the initial phase relates to process improvement. In the first iteration of these three phases, we may not yet have any lessons learned or post-mortem data from previous measurement activities to take into account, however, with time, we will be able to integrate this data in order to improve our current process. The different outputs of these three tasks will be merged and used as inputs to the second phase which is planning.

ii. Planning phase

During this phase, we transform the requirements, context and process improvement into an evaluation plan (see Annex 4), criteria and statistical and/or qualitative techniques to be ready for the execution of that plan.

Since that plan must be aligned with the system development life cycle [13], [180], this one is also one input of the planning phase tasks. So, we start to transform measurement requirements and context into the quality model and measurement specifications. Once this has been done, we must plan for their treatment. First by planning for their collection and storage, where processes change, tools and training may be required, then for their analysis procedure and criteria, or thresholds, to apply assessment, control, and prediction. The final task of this phase is the synthesis and organization of all outputs from these three previous tasks into one critical document: the evaluation plan.

iii. Execution phase

The last phase of our process corresponds to the execution of our evaluation plan which is aligned with the system development life cycle phase. The main task here is a loop to collect measurement data at the frequencies defined in the evaluation plan. Each time data is collected, it needs to be stored as well as analyzed and assessed. The results, containing analysis synthesis, predictions, recommendations, and conclusions, are generated under various forms -graphical dashboards, analyst summary, detailed results, and reports-which are then communicated to the stakeholders, for example, development teams, program managers and any key decision makers.

Figure 28 -Our measurement process proposal articulated over three phases and cadenced with SDLC

To conclude on this new process definition achievement, if we do an analogy, for instance, between ISO/IEC 25040 process (cf. Figure 26) and our proposal (cf. Figure 28) we can clearly conclude that the standard process is a sub-part of our proposed process. Indeed, "establish the evaluation requirements" is a subset of our initial phase, both "specify the evaluation" and "design the evaluation" are included into our planning phase, and "execute the evaluation" with "conclude the evaluation" are also a subset of the tasks of our execution phase.

If we compare with McGarry et al. process (cf. Figure 27), our process corresponds to the "core measurement activities". Our initial phase is located just before the "plan measurement" of McGarry et al. process, collecting requirements, goals, and improvement actions from the "technical and management processes", "establish and sustain commitment" and "evaluate measurement". The planning and execution phase are aligned respectively to the "plan measurement" and "perform measurement" activities.

So, our measurement process is directly compatible with these two well-known measurement processes.

Furthermore, our contribution takes benefit of qualimetry, putting quality modeling activity at the center of the measurement strategy, and remind that this process must be cadenced, like a processor, with systems or software development life cycle, impacting measurement tasks (e.g., change of measurement context, objectives and requirements; see FURPS example from TABLE 11) and critical for handling accurately polymorphism behavior.

7-Threats to validity and discussions

Over this chapter we have deep dived into the concepts of quality, quality modeling and qualimetry, with the purpose to understand them enough to assess which research and technology approaches are the optimum one to answer to our main problematic: have a unified, operational, and appropriate way to define, assess, control, or predict quality of embedded software. However, there are several threats that may affect the validity of our analysis and contributions.

So, in the concept exploration, our intention was to clarify these three concepts and their main vocabulary, ensuring to have a strong grasp on them rather to be exhaustive and lost ourselves in tiny details. We rely also on standards as much as possible because they are reflecting international discussions and agreements between academic and industrial experts. And in automobile domain, for example, it is critical to rely on standards. In addition, we made the choice of focusing on quality modeling and qualimetry instead of considering only applied aspect of quality through the current usage of continuous, automated, and agile process, all of them often neglect quality requirements, related to quality characteristics (see ontology of Figure 9), in favor to fast delivery [START_REF] Behutiye | Management of quality requirements in agile and rapid software development: A systematic mapping study[END_REF], for example. The reason behind this choice is that only quality modeling has the strength to structure, homogenize and get control on the quality, especially when dealing with a diversity of product composed of embedded software and systems.

On the key contributions to quality modeling of software, we didn't perform a systematic literature review to retrieve these key contributions. Rather, we took advantage of the knowledge that we incrementally built during this thesis thought an exploratory literature review, on quality modeling applied to embedded software (see Chapter II.3), and a systematic literature review, on software quality model (see next 0.2). By consequence, we didn't subjectively limit the number of key contributions, but took the research contributions in quality modeling that we noticed to be the most disruptive ones, or the most recurrent and cited ones. However, we have hesitated about the inclusion of some contributions since we had to deal with more than 50 years of continuous research works. For instance, Boehm's COCOMO [START_REF] Boehm | Software Engineering Economics[END_REF] is often referred in quality modeling contributions. Nevertheless, COCOMO is a cost model and not a quality model. Thus, its inclusion is not appropriate and could generate some confusion causing misinterpretation of the concepts. Another example is COSMIC [158]. This functional size measurement standard can be used to improve quality of requirements, or in the case of non-functional requirements 13 which can be declined into functional requirements, they can be measured with COSMIC [START_REF] Abran | COSMIC Measurement Manual for ISO 19761 -Part 2: Guidelines[END_REF]. Nonetheless, we decided to exclude COSMIC from our current list of key contributions because it is independent to technology and quality considerations.

Once this list of key contributions achieved, we summarized them through a graphical timeline (cf. Figure 19) which is an interesting medium for knowledge sharing. In fact, we used graphical medium to leverage some complex concept and ensure people remind the principal information. This is the case for "The House of Qualimetry" (cf. Figure 20) for the science of quality quantification, and for polymorphism with the butterfly analogy (cf. Figure 23, Figure 24).

Concerning the derivation rules from qualimetry, we cited only the most important ones that are part of the ABC of qualimetry book [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], but unfortunately, we were not able to retrieve the full set of 30 derivation rules pointed out by the authors. So, we trust the authors about the importance of these rules with regard to the missing ones, but potentially there is some specific ones that we may miss, and which may impact our case for quality modeling of embedded software.

On the scale topic, even if Stevens' scale is well known and still widely used, it is subject to some criticism, such as Velleman et al. [START_REF] Velleman | Nominal, Ordinal, Interval, and Ratio Typologies are Misleading[END_REF] claiming that "Good data analysis does not assume data types" and therefore scale type to be used must follow the analysis. In other words, we must know what we are planning to do first and how to use what we are planning to collect in order to select right scale and data type (for property, attribute or object). Moreover, some extension of Steven' scale exists. We can cite for example Chrisman [START_REF] Chrisman | Rethinking levels of measurement for cartography[END_REF]

Lastly, we would like to remark that two of our contributions get their inspiration from nature, and more precisely from genetic: this is the polymorphism concept and degree of polymorphism. The polymorphism is present also in software engineering (i.e., in object-oriented programming) but with the genetic analogy we were able to go further on that concept, and even explain it easier. For our last contribution on measurement process, we didn't aim to replace existing measurement processes, but rather clarify and adapt it to foster polymorphic quality model development and usage. As we noticed too, our process can fit in the existing process ones, such as McGarry et al.'s measurement process model [START_REF] Mcgarry | Practical Software Measurement: Objective Information for Decision Makers[END_REF] (see Figure 27).

Chapter V. Quality Model Classification and Selection

Introduction

Following previous Chapter IV, we understand that quality model is a pivot concept in software quality quantification. Unfortunately, it appears also that there is an abundance of software quality models (e.g., [START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF]- [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF]) while there is no obvious consolidated reference list of such models (see Chapter II) from which we could select the most appropriate model to solve our problematic about quality modeling for embedded software. In addition, identifying the most appropriate model among a pool of models requires to classify them and define a set of criteria for decision. This classification gives the hints as well to initiate the resolution of the research question 4 (i.e., "Can we have a unique reference quality model for software product?").

Thus, these dilemmas can be summarized via the research question 2 and 4a: with regard to software quality model evolution against ISO/IEC/IEEE 25010 and their study [START_REF] Gordieiev | IT-oriented software quality models and evolution of the prevailing characteristics[END_REF] (2018) on prevailing quality characteristics for IT-oriented software quality models, Buglione [START_REF] Buglione | Some thoughts on quality models: Evolution and perspectives[END_REF](2015) investigation on quality model evolution and perspectives, Motogna et al. [94] (2019) software quality models evaluation based on natural language processing, and the systematic mapping study on software quality assessment model done by Yan et al. [START_REF] Yan | Software quality assessment model: a systematic mapping study[END_REF] (2019).

Research
In addition, we notice that the studies related to software quality models rely in general on small sets of up to ten quality models (e.g., [START_REF] Gordieiev | Evolution of Software Quality Models in Context of the Standard ISO 25010[END_REF], [94], [START_REF] Jamwal | Issues & Factors For Evaluation of Software Quality Models[END_REF]- [START_REF] Polillo | Quality Models for Web [2.0Sites: A Methodological Approach and a Proposal[END_REF], [START_REF] Miguel | A Review of Software Quality Models for the Evaluation of Software Products[END_REF], [START_REF] Buglione | Some thoughts on quality models: Evolution and perspectives[END_REF], [START_REF] Tervonen | A Unifying Model for Software Quality Engineering[END_REF]) with some minor exceptions: To build such consolidated list, we need also to understand how quality models are identified, compared, and classified, if there is a core subset of quality models, or rather a unique reference, that could be considered as basis for any related work on quality models, and what motivates their creation.

So, we reformulate and synthetize our goals into 3 research questions (i.e., RQ) with their rationale:

-RQ1 -Considering software scope, what is the set of existing quality models? (i.e., research subquestion 2a)

Coming-up with a consolidated list of the main quality models in the software domain could serve as the reference starting quality model source of information, fostering academic or industrial survey work related to quality modeling. This list must be achieved without any time limitations on published year, and once the first iteration is achieved, it should be further maintained and completed to stay valuable.

-RQ2 -Considering a set of quality models, how to classify these quality models, what are the methodology, the criteria, and the characteristics to use? (i.e., research sub-question 2b)

Identifying quality model classification and comparison methodologies, categories, and criteria teach us not only how to handle collection of quality models, but also how quality models are chosen and compared by our peers. Moreover, answering to this question is fundamental to have an unambiguous characterization of quality models which is mandatory to proceed properly on any type of operations (e.g., design, update, comparison, assessment) with or on them.

-RQ3 -Is it possible to have a unique reference quality model for software product? (i.e., first part of research sub-question 4a)

We already know that many quality models exist, nevertheless we don't know if they are finally closed to a unique reference quality model for software product. Knowing if such unique reference exists is key to frame efficiently any quality modeling activities, and the classified quality model collection is the proper material requires to answer to that question.

In order to address these three research questions, we conduct a systematic literature review following Kitchenham and Charter's systematic literature review guidelines [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF]. The systematic literature review process and its stages are detailed in Figure 29. Overall, we apply the same search strategy than Adewumi et al. [START_REF] Adewumi | A systematic literature review of open source software quality assessment models[END_REF] and Yan et al. [START_REF] Yan | Software quality assessment model: a systematic mapping study[END_REF], both based on Kitchenham and Charter's guidelines too. Likewise, we rely on the same five digital libraries, or electronic database sources (cf. webofknowledge.com

The electronics database source exploration and search are realized via search queries. These search queries, also named search strings, are composed of words to look for, combined with operators (e.g., OR, AND, NEAR, NOT), sometimes with some wildcard characters (i.e., "?" replaces one alphanumeric character, or "*" replaces zero to multiple alphanumeric characters) and often with indication to fields of narrow search interest in the database (e.g., title, abstract, keywords, date, author). Moreover, each online database search engine has its own query syntax and restriction (e.g., restriction on the number of allowed wildcards in search query). For instance, to search over title and abstract field, with Scopus we must use "TITLE-ABS ()" while with ACM digital library, we must use "Title:" and "Abstract:".

So, to create the right search strings from the research questions, independently to any search engine syntax, we structure our "words" around three main concepts: domain of interest, object of interest and type of research work. In this survey the domain of interest is the "software" domain. The objects of interest are the objects closed or associated to "quality model" concept and therefore we can find "quality model", "quality factor", "quality characteristic" and "quality ontology". For the type of research work, our focus must be aligned to the survey concept since we are performing a systematic literature review. We identified then "survey", "study", "analysis", "review", "mapping", "comparison", "challenge", "evolution" or "taxonomy". In addition to these words, we completed that list with some frequent acronyms (e.g., quality model as QM), plural forms and use of proper wildcards. The right search queries for each online database source can be then derived from the information summarized in Table 15 and are given in Table 16. ACM digital library ("software*" SW) AND ("quality model*" QM "quality ontology" "quality ontologies" QO "quality factor*" QF "quality characteri*" QC) AND acmdlTitle:(survey surveys study studies analysis review reviews "compar*" classification classifications evolution evolutions taxonomy challenge challenges mapping mappings) IEEE Xplore (("software*" OR SW) AND ("quality model*" OR QM OR "quality ontology" OR "quality ontologies" OR QO OR "quality factor*" OR "quality characteri*") AND (("Document Title":"survey") OR ("Document Title":"surveys") OR ("Document Title":study) OR ("Document Title":studies) OR ("Document Title":analysis) OR ("Document Title":"review") OR ("Document Title":"reviews") OR ("Document Title":"compar*") OR ("Document Title":"classification") OR ("Document Title":"classifications") OR ("Document Title":"evolution") OR ("Document Title":"evolutions") OR ("Document Title":taxonomy) OR ("Document Title":"challenge") OR ("Document Title":"challenges") OR ("Document Title":"mapping") OR ("Document Title":"mappings")))

Scopus ( TITLE-ABS-KEY ( ( "software*" OR sw ) ) AND TITLE-ABS-KEY ( ( "quality model*" OR qm OR "quality ontology" OR "quality ontologies" OR qo OR "quality factor*" OR qf OR "quality characteri*" ) ) AND TITLE ( ( survey OR surveys OR study OR studies OR analysis OR review OR reviews OR compar* OR classification OR classifications OR evolution OR evolutions OR taxonomy OR challenge OR challenges OR mapping OR mappings ) ) ) AND ( LIMIT-TO ( SUBJAREA , "COMP" ) OR LIMIT-TO ( SUBJAREA , "ENGI" ) ) AND ( LIMIT-TO ( LANGUAGE , "English" ) ) Springer ("software*" OR SW) AND ("quality model*" OR QM OR "quality ontology" OR "quality ontologies" OR QO OR "quality factor*" OR QF OR "quality characteri*" OR QC) AND (("Document Title":"survey") ("Document Title":"surveys") OR ("Document Title":study) OR ("Document Title":studies) OR ("Document Title":analysis) OR ("Document Title":"review") OR ("Document Title":"reviews") OR ("Document Title":"compar*") OR ("Document Title":"classification") OR ("Document Title":"classifications") OR ("Document Title":"evolution") OR ("Document Title":"evolutions") OR ("Document Title":taxonomy) OR ("Document Title":"challenge") OR ("Document Title":"challenges") OR ("Document Title":"mapping") OR ("Document Title":"mappings"))

Web of Science (ALL=(SW or software) AND TS=("quality model*" OR QM OR "quality ontology" OR "quality ontologies" OR QO OR "quality factor*" OR QF OR "quality characteri*") AND TI= (survey OR surveys OR study OR studies OR analysis OR review OR reviews OR classification OR classifications OR evolution OR evolutions OR "compar*" OR taxonomy OR challenge OR challenges OR mapping OR mappings )) AND LANGUAGE: (English) Thus, once any search query is performed over one of the sources main electronic databases, we obtain a raw result of candidate documents. That raw result contains both relevant and irrelevant documents with our research questions, and therefore must be screened through a process of selection. Our process of study selection, shown in Figure 29, is a sequence of four filtering stages during which we assess the relevance of each collected documents with respect to our research questions. In addition, the document relevance done in each filtering stage relies on a set of inclusion and exclusion criteria, defined in Table 17, and like Oriol et al. [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF], we are setting minimum document quality by assessing the ranking level of the corresponding journal or conferences.

Inclusion criteria Exclusion criteria

Figure 29 -The distinct stages of the systematic literature review process for study selection, with selected document numbers and publication years

The four screening, or filtering, stages are incremental, each stage taking as inputs the outputs of previous one, refining selection to the most relevant documents thanks to these inclusion and exclusion criteria. Furthermore, the effort to perform each selection stage increase incrementally as well. This behavior is explained by the fact that the more refined the list of selected documents is, the more thorough the analysis of the documents must be.

With the first filter, we have a very large number of potential documents to crawl and therefore, the first selection pass must be performed efficiently. Thus, the decision is made by considering only document type, language, and its accessibility. The second stage focus on title and duplicate document removal. The next stage needs more time investment on each document because the focus is on document abstract and therefore, it required to have each document abstract to be read. Note, paper abstracts are always available online. At the fourth and final stage, it is required to access, or download the full document and complete the last selection stage based on the document content. Hopefully at this stage, the number of candidate documents is reduced: in our current systematic, we started from a collection of 3,234 documents and reduced up to 151 documents at the input of the last stage, which represents a screening of 95.33%.

The result of this document cascade filtering is a list of 121 relevant documents (i.e., 39 articles from journals and 82 papers from conferences) covering a period between 1979 and 2019, and the major difficulties we faced during this systematic literature review were:

-Incorrect reference or pointer to retrieve the correct papers: sometimes there were some mistakes on the referenced paper, which was not pointing to correct quality model, -Inaccessibility to publications: some published documents were not accessible or downloadable (e.g., old books), -Bad association or wrong focus with quality model scope: o A publication describes quality activity that is not associated quality model or modeling: for example, a QMS framework is not quality modeling with quality model, o A model is not modeling quality: for instance, COCOMO [START_REF] Boehm | Software Engineering Economics[END_REF] is a for cost modeling but not for quality modeling, -Difficulties in understanding contributions: for example, complex papers referring to specific technical knowledge, or with unclear assumptions, -Difficulties in the identification of quality model characters: for instance, retrieving which statistical method is used for a quality model, or its quality perspective.

Although the review collection and methodology were rigorous, we looked for potential biases due to the online database, the search query results or the filtering stages. So during this systematic literature review, we noted that several relevant papers, we previously found either through our exploratory review (see Chapter II.3) or some previous investigations, were missing. Consequently, we performed a complementary literature exploration (i.e., considering results from previous exploratory review and manual searches) finding 15 additional papers, four conference papers and 11 journal ones.

The result of the entire review process is a total of 136 publications done between 1979 and 2019, and summarized into Table 45 of Annex 5. Almost two third (i.e., 64%) of the 136 publications comes from conference while the source of the remaining 36% papers is journal (see Figure 30). Furthermore, most of these studies (i.e., 79.41%) were performed in an academic context, without any concrete practical consideration from industry partners whose are the primary demanders of these study results (see Figure 31). As depicted in Figure 32, 65.94% of the selected study contributions were done during the last decade, and this period includes the years with the most important number of selected studies: in 2014 we have 16 selected papers, in 2016 12 papers, and in both 2013 and 2018 11 papers. Therefore, we can infer that quality model study continues currently to be an active topic, and even get stronger interest also over the last decade. In parallel, we remark that beside one contribution from Mohanty [START_REF] Mohanty | Models and Measurements for Quality Assessment of Software[END_REF] in 1979, no paper prior to 1994 passed our systematic review. From this systematic literature review, 136 study papers, published between 1979 and 2019, were qualitied and reviewed in order to address our three research questions.

Moreover, we remarked, based on these papers, that quality model study continues currently to be an active topic, even getting a stronger interest over the last decade, and is mostly lead by the academic community.

In the next following sections, taking advantage of that selected literature, we analyze it to find answers to the research questions, starting with software quality model classification aspect since this is linked to the topics and methodologies used in these papers.

Software quality model classification

Research Sub-question 2b

Considering a set of quality models, how to classify these quality models, what are the methodology, the criteria, and the characteristics to use?

In order to answer to this research question, we start our analysis of the 136 papers collection by identifying what kinds of study related to quality models each paper can be associated with. Knowing this information is crucial to learn how information related to quality models are exploited. Indeed, while a comparison study aims to identify and use relevant elements to classify and compare together quality models, a quality model creation study generally sets its focus on identifying limitations or gaps against a specific research case, and then addresses them. Thus, this analysis allows us to extract 65 types (see Table 46 of Annex 5) that we succeed to regroup into seven distinct types of study. These types are:

-Analysis or survey on specific aspects or elements related to quality model: the study aims to investigate a specific topic related to quality, quality characteristics or quality models.

-Comparison of quality models or characteristics: the purpose of such study is to identify a set of quality models and then uses some criteria to compare together the selection of quality models; the conclusion is often to identify the best quality model among that set.

-Creation of new model: the goal of this kind of survey is to reply to a need by creating a new quality model either from an analysis on a context or domain, and / or from existing quality models or quality characteristics.

-Evaluation of quality models or characteristics: this type of study is to perform against some specific use cases to evaluate how well a quality model perform; it can reveal not only the quality model advantages but also its disadvantages.

-Quality model customization: starting from a quality model, this kind of study describe the customization work done to adapt and customize a quality model to answer to some specific needs.

-Quality model improvement: the study object here is to improve some specific aspects of a quality model to answer to some predefined or analyzed constraints or limitations.

-Systematic mapping or literature review: the aim of this type of study is to address some specific research questions related to quality model in order to acquire and build further knowledge from existing literature and research work.

The mapping of the 136 papers against each of these seven types (cf.

Type of study Study Ids

Analysis or survey on specific aspects or elements related to quality model SLR-S01, SLR-S03, SLR-S04, SLR-S05, SLR-S06, SLR-S13, SLR-S14, SLR-S16, SLR-S17, SLR-S26, SLR-S32, SLR-S42, SLR-S61, SLR-S62, SLR-S64, SLR-S65, SLR-S68, SLR-S69, SLR-S71, SLR-S74, SLR-S75, SLR-S80, SLR-S86, SLR-S87, SLR-S88, SLR-S89, SLR-S100, SLR-S101, SLR-S102, SLR-S107, SLR-S108, SLR-S110, SLR-S113, SLR-S119, SLR-S120, SLR-S128, SLR-S129, SLR-S135

Comparison of quality models or characteristics SLR-S19, SLR-S25, SLR-S41, SLR-S44, SLR-S49, SLR-S50, SLR-S53, SLR-S66, SLR-S67, SLR-S70, SLR-S77, SLR-S81, SLR-S92, SLR-S93, SLR-S94, SLR-S96, SLR-S99, SLR-S109, SLR-S114, SLR-S115, SLR-S122, SLR-S126, SLR-S127, SLR-S136

Creation -Against specific quality model or characteristics: under this category, the goal is to compare or classify quality models against a specific quality model (e.g., the quality models evaluation against ISO/IEC/IEEE 25010 of Motogna et al. [94]) or some specific characteristics such as the evolution of quality characteristics linked to mobile software of Gezici et al. study [START_REF] Gezici | Internal and external quality in the evolution of mobile software: An exploratory study in open-source market[END_REF], for example.

-Context / purpose / scope / viewpoint: the aim here is to rely on several usual aspects of a quality model; these are the context (e.g., from Yan et al. [START_REF] Yan | A Systematic Mapping Study of Quality Assessment Models for Software Products[END_REF] academic, industrial, or both), the purpose (e.g., definition, assessment, prediction, multi-purpose from Deissenboeck's DAP [START_REF] Deissenboeck | Software quality models: Purposes, usage scenarios and requirements[END_REF]), the scope composed of the object of interest (e.g., web-service in Oriol et al. [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF], open-source in Adewumi et al. [START_REF] Adewumi | A systematic literature review of open source software quality assessment models[END_REF] or in Petrinja et al. [START_REF] Petrinja | Comparing OpenBRR, QSOS, and OMM Assessment Models[END_REF]) and the quality focus described in the CQML classification scheme of Kläs et al. [START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF] (e.g., general, specific, defects, maturity, cost), and the stakeholder viewpoint depicted by the essential views of Horgan et al. [START_REF] Horgan | An essential views model for software quality assurance[END_REF].

-Description: either a brief or complete description of quality models is performed, often including some historical motivation and context with authors, the main characteristics of the quality model, the application domain (e.g., IT, transportation, medical, socio-economic from Fath-Allah et al. [START_REF] Fath-Allah | A Comparative Analysis of E-Government Quality Models[END_REF]), the quality perspectives (e.g., the five quality perspectives of Garvin [101]: user, product, manufacturer, transcendental, value-based).

-Evaluation / results / benefits vs limitations: the object of this category is to compare or classify quality models based on the exercise and evaluation results of the model applied to sample use cases, or to analyze the model in order to identify its benefits and limitations.

-Formalism: the focus is put on the type of formalism used to model the quality ; we can rely on Wagner's guidelines [START_REF] Wagner | Software Product Quality Control[END_REF]: meta-model, hierarchical, and statistical or implicit.

-None: neither comparison, nor classification criterion is used; this concerns only one paper related describing a statistical quality model creation that is used to analyze and predict software defects in Hitachi company; we notice no reference to other similar work from the authors.

-Quality characteristics / sub-characteristics / metrics / Tools: the classifications or comparisons are made between the presence and the enumeration of quality characteristics, sub-characteristics, metrics and / or tools associated to quality model.

-Quality model parent: this criterion spots the attention on the quality model parent(s) when the quality model under focus is inheriting from one or several anterior quality models; Thapar et al. [11] (see Figure 35) qualified that relationship as basic versus tailored quality models ; the tailored quality models are issued from basic quality models.

Classification or comparison criteria Study Ids

Against specific quality model or characteristics SLR-S16, SLR-S41, SLR-S47, SLR-S57, SLR-S60, SLR-S76, SLR-S86, SLR-S95, SLR-S99, SLR-S103, SLR-S106, SLR-S109, SLR-S110, SLR-S111, SLR-S112, SLR-S115, SLR-S125, SLR-S133, SLR-S135, SLR-S136

Context / purpose / scope / viewpoint SLR-S03, SLR-S13, SLR-S15, SLR-S21, SLR-S40, SLR-S44, SLR-S51, SLR-S52, SLR-S54, SLR-S58, SLR-S59, SLR-S61, SLR-S62, SLR-S68, SLR-S69, SLR-S75, SLR-S78, SLR-S79, SLR-S80, SLR-S82, SLR-S84, SLR-S85, SLR-S89, SLR-S90, SLR-S91, SLR-S93, SLR-S94, SLR-S97, SLR-S100, SLR-S102, SLR-S105, SLR-S111, SLR-S118, SLR-S120, SLR-S121, SLR-S131, SLR-S135 Description SLR-S06, SLR-S07, SLR-S17, SLR-S26, SLR-S36, SLR-S45, SLR-S51, SLR-S56, SLR-S59, SLR-S63, SLR-S64, SLR-S66, SLR-S69, SLR-S70, SLR-S72, SLR-S75, SLR-S78, SLR-S81, SLR-S84, SLR-S89, SLR-S91, SLR-S92, SLR-S94, SLR-S96, SLR-S100, SLR-S103, SLR-S108, SLR-S110, SLR-S112, SLR-S113, SLR-S114, SLR-S115, SLR-S118, SLR-S120, SLR-S121, SLR-S122, SLR-S124, SLR-S126, SLR-S131, SLR-S132, SLR-S133, SLR-S135

Evaluation / results / benefits vs limitations SLR-S02, SLR-S04, SLR-S10, SLR-S12, SLR-S14, SLR-S17, SLR-S19, SLR-S20, SLR-S23, SLR-S25, SLR-S27, SLR-S29, SLR-S30, SLR-S32, SLR-S33, SLR-S35, SLR-S42, SLR-S43, SLR-S45, SLR-S46, SLR-S47, SLR-S48, SLR-S49, SLR-S50, SLR-S53, SLR-S55, SLR-S59, SLR-S65, SLR-S67, SLR-S69, SLR-S72, SLR-S73, SLR-S77, SLR-S83, SLR-S85, SLR-S90, SLR-S92, SLR-S96, SLR-S115, SLR-S122, SLR-S123, SLR-S124, SLR-S129, SLR-S130 Formalism SLR-S10, SLR-S12, SLR-S14, SLR-S17, SLR-S20, SLR-S24, SLR-S29, SLR-S33, SLR-S45, SLR-S66, SLR-S69, SLR-S70, SLR-S82, SLR-S83, SLR-S85, SLR-S107, SLR-S108, SLR-S111, SLR-S112, SLR-S116, SLR-S117, SLR-S123, SLR-S126, SLR-S134 None SLR-S09 Quality characteristics / sub-characteristics / metrics / Tools SLR-S01, SLR-S05, SLR-S08, SLR-S11, SLR-S16, SLR-S18, SLR-S22, SLR-S23, SLR-S25, SLR-S26, SLR-S28, SLR-S30, SLR-S31, SLR-S34, SLR-S37, SLR-S38, SLR-S39, SLR-S41, SLR-S43, SLR-S44, SLR-S45, SLR-S46, SLR-S47, SLR-S50, SLR-S53, SLR-S57, SLR-S58, SLR-S61, SLR-S65, SLR-S66, SLR-S67, SLR-S68, SLR-S69, SLR-S70, SLR-S71, SLR-S75, SLR-S77, SLR-S80, SLR-S81, SLR-S83, SLR-S84, SLR-S85, SLR-S86, SLR-S87, SLR-S88, SLR-S89, SLR-S90, SLR-S91, SLR-S92, SLR-S93, SLR-S94, SLR-S95, SLR-S96, SLR-S97, SLR-S98, SLR-S99, SLR-S100, SLR-S101, SLR-S103, SLR-S105, SLR-S106, SLR-S107, SLR-S109, SLR-S110, SLR-S111, SLR-S112, SLR-S113, SLR-S116, SLR-S117, SLR-S118, SLR-S120, SLR-S121, SLR-S122, SLR-S123, SLR-S124, SLR-S125, SLR-S126, SLR-S127, SLR-S128, SLR-S129, SLR-S131, SLR-S133, SLR-S134, SLR-S135, SLR-S136

Quality model parent SLR-S77, SLR-S79, SLR-S82, SLR-S90, SLR-S93, SLR-S117

Once again, the criteria categories (cf. Figure 36) for quality model classification or comparisons are not evenly distributed. Unsurprising, with 32.8% the predominant category relates to quality characteristics / subcharacteristics / metrics / tools because those elements are the ones which composed a quality model. Moreover, description, context / purpose /scope /viewpoint, and formalism, with respectively 16.22%, 14.29% and 9.27%, complete the portrayal of a quality model. Nevertheless, the second most important category doesn't belong to them: this is the evaluation / results / benefits vs limitations. Therefore, we can conclude that after using quality model elements such as quality characteristics for instance, researchers prefer to construct their classification or comparison study on empirical or theorical evaluations and determine quality model benefits and limitations. So, based on these findings, five complementary themes emerge to describe and classify quality models. We have id, bibliographic, definition, scope and structural. Figure 38 provides the full overview of these five themes together with their refined distinct elements to practically describe, organize and classify quality models.

First, id is similar to an id card. It regroups the quality model name, its author(s), publication year, and its pedigree.

Under pedigree, we include the quality model parents used to inspire, create, or customize the quality model.

Bibliographic theme integrates the reference source, availability and accessibility, and its importance in front of community thanks to citation counters. We have 3 citation counters: from the original source (i.e., publisher counter) when available, from study papers (i.e., how many times the quality model is cited over the papers issued from a systematic literature review) and from Google Scholar (https://scholar.google.com/). We note that from Snyder's analysis on citation counters [START_REF] Snyder | Definition of 'Taxon', plural 'taxa[END_REF], Google Scholar appears to provide a little bit optimist results but they are closer to the reality than publisher ones. Indeed, publisher metrics often do not count for reference in papers, thesis, or another scientific document outside the publisher scope. Therefore, we decided to use Google Scholar as our counter reference but also keep the publisher citation metrics to strengthen the validity of our citation metrics. Moreover, by using Google Scholar citation information independently of the publishers, we have citation information generate in a consistent manner which brings confidence when we compare, in a relative way (e.g., as ordinal), papers based on their citation results. Figure 37 illustrates the differences in citation metrics depending on the metric source and highlighted by Snyder.

Next theme is the definition of three main elements that characterized a quality model. We have the basic or tailored nature of a model as defined by Thapar et al. [11] (cf. Figure 35) and then relayed by Miguel et al. [START_REF] Miguel | A Review of Software Quality Models for the Evaluation of Software Products[END_REF], to which we can also attach standard nature (e.g., ISO/IEC 9126 or ISO/IEC/IEEE 25010). We recall that Wagner et al. [83] concluded that 28% of companies in their survey use quality model standards, even if 79% of them are finally customized. The second element is the quality model purpose. This is aligned with the Deissenboeck et al.

's DAP classification [START_REF] Deissenboeck | Software quality models: Purposes, usage scenarios and requirements[END_REF] whose valid purposes are definition, assessment, prediction and multi-purpose. Model formalism is the third and last main characterization elements. It indicates whether the model is defined by a meta-model, a hierarchical model, or by an implicit or statistical model (see Wagner [START_REF] Wagner | Software Product Quality Control[END_REF]). To complete the quality model characterization, the last theme is the structural one. This corresponds to the quality model constituent elements and their relationships. Consequently, we have first the vocable that used in the quality model (e.g., characteristics, attributes, factors, metrics), the characteristic / sub-characteristic list and definitions, the metrics list and definitions, and both qualitative and quantitative relationship (e.g., aggregation, correlation between metrics and quality characteristics) found also in Kläs et al. 's CQML [START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF]. Note that the characteristic / sub-characteristic and metrics could be either not defined, direct (i.e., available) and defined, direct and partially defined, indirect (i.e., available elsewhere, e.g. in another quality model) and defined, or indirect and partially defined. The notion of "direct" versus "indirect" was introduced and used by Thapar et al. [11] and Fath-Allah et al. [START_REF] Fath-Allah | A Comparative Analysis of E-Government Quality Models[END_REF]. Concerning the notion of "defined" versus "not defined", we rely on the work done Oriol et al. [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] bringing more nuance with their rating Y (i.e., explicitly defined), Y+ (i.e., explicitly defined and contains subdivisions), P (i.e., partially, not explicitly defined but a quality attribute or metric can support that definition), P+ (i.e., partially, not explicitly defined but several quality attributes or metrics can support that definition), and ND (i.e., not defined).

Finally, the result of the assembly of these five themes together with their characterization elements, shown in Figure 38, covers all the classification or comparison criteria that were extracted from the systematic literature review of 136 papers for a period from 1979 to 2019. This means that this consolidated result is therefore enough for classifying software quality models.

Nevertheless, extracting all information required by these classification quality model elements is the preliminary step in their classification. Next section addresses their use and organization to achieve the software quality model classification. In Chapter V.3, we identified 20 software quality models classification elements that we organized into five distinct themes. Consequently, a straightforward and valid classification methodology for software quality models can be achieved by categorizing software quality models into groups that share the similar classification element values. This kind of classification methodology is called a taxonomy, and its definition is "a system for naming and organizing things, especially plants and animals, into groups that share similar qualities" [210].

Nevertheless, the number of combinations to explore for this classification approach is huge. Indeed, regarding definition theme, we have 36 possible combinations (i.e., 3 nature possibilities x 4 purpose possibilities x 3 formalism possibilities). Likewise, the numbers of combinations for bibliographic, scope and structural theme are respectively 4, at least 3,240 and at least 100. So, the total number of all possible combinations to explore is at least 46,656,000 (i.e. 4 x 36 x 3,240 x 100), if we apply a taxonomy based on these elements. Note, we don't include id theme in this calculation since the only relevant element to consider here is the quality model pedigree, and its cardinality is difficult to evaluate, or guesstimate without collecting software quality models. However, knowing that there exists at least more than 46 millions of combination to explore is enough to conclude that this method is not efficient for our needs.

Thus, instead of dealing only with a taxonomy, we can also take benefit of the timeline and evolutionary knowledge contain in software quality model pedigree and which can be depicted likes the genealogical tree of quality models achieved by Oriol et al. [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF].

Therefore, our classification method proposal becomes a composition between the use of taxa (i.e. "taxonomic categories, as species or genera" [211]), likes in taxonomy, but also, with the consideration of the timeline and evolutionary knowledge and reflected into homologies (i.e., "a fundamental similarity based on common descent"

[212]). In other words, our proposal is to rely on cladistic, which is the "classification of organisms based on the branching of descendant lineages from a common ancestor" [213], as classification methodology. Furthermore, this proposal is directly aligned with the polymorphism mechanism introduced in Chapter IV.6.b integrating both the phenotypic variety (cf. Figure 23) and the evolution over time (cf. Figure 24).

When applying cladistic, the result is a cladogram as shown in Figure 23 for instance. The cladogram specifies degree of kinship relations between elements -in general organisms-which are then grouped into clade or taxon. Moreover, a cladogram is different from a genealogical tree because it does not show how ancestor and descendants are related together, but rather uses common ancestors and the study of shared characters to classify the taxa which is more relevant for classifying software quality models.

We applied on the software quality model classification elements the compatibility cladistic analysis from Estabrook et al.

[214] where the objective is to optimize the element grouping (i.e., economy of hypotheses) thanks to the maximum mutually compatible characters. So, we identified five taxa whose definitions come directly from the definition, scope, and structural themes of the software quality model classification elements and are organized from the most generic characters (i.e. with less distinct cases) to the most specific one (i.e. with many distinct cases). We start with definition, considering "formalism" and "purpose" to describe the characters considered in the first taxon. We use the two elements together since they are both closely related. For instance, a meta-model formalism is often associated to definition purpose, while a statistic formalism is often used for prediction purpose.

The next level of taxon is about structural theme with again two closely related elements that are "elements (vocable)" and "qualitative & quantitative relationship". This theme can be perceived as derived, or specialized, from the definition-based taxon, especially from the "formalism" element point of view. For the remaining three taxa, they are all linked to the scope theme. The most generic one is the "quality perspective" classification element. We voluntary keep this element alone to define a taxon because it corresponds to the general perspectives that Garvin expressed about quality [101]. Then, two complementary and related viewpoints define the penultimate taxon. These are the "quality focus" and "stakeholder viewpoint". Consequently, the final taxon composed of the "domain" and "object of interest" elements which are the closest elements together compare to the other scope elements.

As we highlighted, to complete the cladistic approach we must specify a homology in addition to these taxa. The homology supports the identification of common ancestors and the evolutionary arrangement through a timeline of evolution. This homology is defined by three classification elements: "publication year" and "pedigree" from Id theme, and "nature" from definition theme.

The resulting software quality model clade is described in Figure 39, and integrates homology and taxa descriptions.

To proceed on software quality model classification, once software quality models are collected, their classification elements must be filled, or completed, and then applied against our proposed software quality model homology and taxa to generate the corresponding cladogram. The second contribution we achieve thanks to the systematic literature review is the elaboration of a consolidated software quality model list, answering consequently to the research sub-question 2a.

During the reading of the 136 retained papers, we first collected information (see Table 46 of Annex 5) about object of interest, domain and quality focus as specified in the taxa of the software quality model clade, described in Chapter V.4.a. Then, we look for software quality models refer in each of these papers. To do so, we follow snowballing approach as explained in Wohlin's guidelines [START_REF] Wohlin | Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering[END_REF], [START_REF] Wohlin | Second-Generation Systematic Literature Studies Using Snowballing[END_REF]. The main approach idea is to collect further data by reviewing the reference papers as well that are cited in each the original systematic literature paper selection. Thus, we retrieved software quality models not only from the reference papers cited in each of the 136 papers, but also subsequently from the cited papers in the reference papers.

To include any identified software quality model into our collection, we aimed to retrieved and access to the source document, not only to confirm the relevance of the investigated contribution as software quality model but also to retrieve taxa related information likes quality perspective, pedigree, publication information (i.e., publication year, author(s), citation counters, publisher reference), purpose and formalism. In the case where the reference document was found but was not in English, or the reference was not accessible or retrievable, we exclude it. This heavy literature review and investigation has resulted in the first list of 492 distinct software quality models, covering the period going from 1968 to 2019, for a total of 51 years. The complete enumeration of these 492 software quality models is available in Table 48 In parallel, we keep tracked of which software quality models were found or linked with each of the 136 study papers, with a maximum of 48 quality models for Oriol et al systematic mapping [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF]. That information is aggregated into TABLE 47. As we can see in Figure 42 which represents the normal distribution of the number of retrieved quality models per study paper, there is an average (i.e., μ, value of gaussian distribution center) of 9.5 quality models found per study paper, with a standard deviation (i.e., σ) of 7.4. We recall that with a normal law distribution, a range of 2 times σ before and then after μ value includes 95.4499736% of samples.

The data drawn in Figure 41 come from Google Scholar citation counter, as explained in Chapter V.3. However, we noticed that for few software quality models, this type of citation counter didn't provide any answer. This is for instance the case with standard quality model such as ISO / IEC 9126. Hopefully, the data that we generate from the mapping of retrieved quality models against each study paper can fill these gaps, and ensure that the citation results are collected homogenously over the 136 study papers, which we cannot assess for data from Google Scholar counter. The result is shown in Figure 43, where we succeed to get citation data for all the 492 software quality models. We observe also that 7.11% of the 492 soft quality models (i.e., 35 over 492) represents 44.38% of the cited quality models in these studies. Figure 44 is a zoom on these 35 most cited software quality models (i.e., they are cited more than 5 times over the 136 study papers). To answer to both research sub-question 2d and 4a, we must take profit of the underlying knowledge resulting from this unique collection of 492 software quality models by building a software quality model landscape, and then assess if there is a convergence to a unique reference quality model for software product, and which is the most appropriate quality model to select for embedded software in automotive.

First, the 492 software quality models categorization against the Deissenboeck's DAP classification [START_REF] Deissenboeck | Software quality models: Purposes, usage scenarios and requirements[END_REF] shows, via Figure 45, that a majority (i.e., more than 58%) of published software quality models are assessment models, followed by the prediction quality models with almost 28% of the models. This teaches us that the main objective for modeling quality is put on the quality assessment and then on its prediction. This statement sounds logical because we must know how to assess or control before being able to predict.

The next aspect of this landscape is regarding the formalism used for software quality model and illustrated by Figure 46: the overall result is similar to the DAP classification one. Indeed, the principal formalism is the hierarchical one, with around 63% of the cases, and, with about 33%, we find the statistic, or implicit models. We could expect this kind of proportion because in most of the prediction times, a statistic, or implicit model is construct, relying on historical data and specific expert knowledge to predict quality. Concerning the hierarchical quality models, their usages are widely spread since this is the easiest formalism to understand and explain. We remark that meta-model formalism represents only 3.45% of the cases while we have a bigger proportion (i.e., 10.14%) for definition model. The reason of this difference is due to the fact that definition models are either described through meta-model and hierarchical model. Going one step further in the statistic or implicit formalism comprehension, we aimed to understand and retrieve the main applied methods. We found 29 data mining methods, going from regression to classification, from fuzzy logic to Bayesian or neural networks. We noticed that more than one method was often used together to strengthen the prediction, but in our collected data, we counted only the predominant one for each prediction software quality model.

So, with 17.58%, the statistical methods are the most applied type of data mining methods. There are often based on various statistical analysis of historical data. The second most used method (i.e., 11.52%) is the logistic regression due to its particular 'S' curve result. Its popularity sounds to recognize a better-adapted predictor for software quality compared to classical regression analysis, used in 6.06% of the cases. Equally with 9.09% of usage, fuzzy logic and neural network are the two next frequent methods, but their implementations are more elaborated compare to regression methods. Another interesting recurrent (i.e., 8.48%) used method is the capture -recapture approach. This is a statistical inference method usually used in ecology. Its main principle is to consider only a subset of the studied population to infer, or predict, some specific result against the entire population. Moving on with the next less frequent data mining methods, we retrieve the regression analysis with 6.06% likes the Bayesian network which can associated to a hierarchical model, and the following three methods, classification, classification tree and genetic algorithm, that are applied each one in 3.03% of the statistic or implicit quality model cases.

About the 19 other data mining methods, their usages occur in some minor cases. The complete result of the 29 method is enumerated through Figure 47.

To continue the construction of the software quality model landscape, we look for the quality perspectives, or views, as described by Garvin [101], considered in the 492 quality models. The accumulated data demonstrate an equi-distribution among three perspectives (cf. Figure 48): user, product, and manufacturer. Nevertheless, the finer details of this distribution per software quality model reveal a more nuanced landscape (see Figure 49). In fact, while we still have a single quality perspective in 61% of software quality models, for almost 39% of the cases, software quality models integrate a mix of two or three quality perspectives. The single manufacturer perspective is a little bit apart from the rest of the result since it counts for 31.44% of software quality models. This result is explained by the fact that prediction quality models usually address a manufacturer quality perspective. Regarding software quality model scope depicted by Figure 50, the main concern that emerges from the 492 quality models is with regards to the product scope alone. This scope happens in more than 79% of these 492 models, proving that quality for software is first a matter of product quality. The second most frequent scope is the quality related to service software with only a little bit more than 12%, and then we found process, 3.45%, or product and process, 3.25%. Despite that remaining scopes cover a very small minority of the concerns, this doesn't mean that they are insignificant or should be discarded. On the contrary, they highlight future directions to investigate. Consequently, the second key nuance against Thapar et al's postulate is the distinction between three type of quality models rather than two: we have basic, tailored, and standard quality models.

Through the analysis and construction of this landscape, we see the main tendencies that characterize software quality models, including their evolution since 1968. Therefore, published software quality models focus principally on quality assessment, and then on prediction. They are commonly hierarchical except for prediction where the adopted formalism is statistic or implicit. The quality perspectives are equally distributed over manufacturer, user, and product perspectives, but the scope is usually about product. Finally, the software quality model evolution is articulated around three periods: up to 1990, we have the basic quality model period, from 1990 to 2003, the transition period, and since 2003, we are in the quality model tailoring period.

Coming back to the research sub-questions 2d (i.e., "What is the most appropriate quality model is for embedded software in automotive?") and 4a (i.e., "Is it possible to have a unique reference quality model for software product, or instead should we have a meta-model?"), from the landscape knowledge, it appears that there is not one distinct quality model emerging from these 492 software quality models, especially because the quality model depends on various variables such as the DAP, the quality perspective, or the object of interest, to cite a few of them.

Moreover, as we saw in our systematic literature review, the quality model comparison studies represent 16.67% of the 136 study papers (see Figure 34), with the latest study dealing on this topic published in 2019: Motogna et al. [94]. This is a good indicator highlighting that the quest to find the best quality model -and not a unique oneis still an active topic.

Thus, the answer to 4a is undoubtedly that it is not possible to have a unique reference quality model for software product, and, with this list of 492 quality models, we can also conclude that having a unique quality model covering all software product cases is an elusive target.

Regarding 2a, again from both the systematic literature review, combined with snowballing, and the exploratory literature review (cf. Chapter II.3), no quality model for embedded software in automotive clearly emerged.

Nevertheless, if we refer to Figure 44, two of the three most cited software quality models among all the 492 models are the standard quality model ISO / IEC 9126 and ISO / IEC / IEEE 2501. And since we are in the quality model tailoring period, an appropriate approach is similar to Arhens et al. [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF] who customized ISO / IEC 9126 to address their needs for a software quality model for automotive software architecture.

In conclusion, the answer to 2a is that there is no quality model ready yet for embedded software in automotive, but rather, we select the latest standard, ISO / IEC / IEEE 25010, and then customized it to generate the most appropriate model to reply to our needs in automotive domain.

Threats to validity and discussions

This chapter described a quite long work of investigation performed on quality models in current literature. This investigation was possible thanks to the online available libraries that allowed us to retrieve and get access to a plethora of candidate papers during our systematic literature review. Furthermore, the snowballing adjunction to the review has assured us a successful harvest of 492 quality models. Nevertheless, this lengthy investigative work and the corresponding contributions may have been impacted by several threats.

Firstly, concerning literature bias, we didn't really consider the grey literature (i.e., the material and research published outside the traditional commercial academic publishers). We use Google Scholar as a complementary source to find some further candidate study papers, but in a minor way, and certainly not as Haddaway et al. [START_REF] Haddaway | The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching[END_REF] described about the role of Google Scholar in evidence reviews and grey literature searching. Rather, we decided to use traditional commercial academic publishers because of the mandatory peer reviews to publish papers through this type of media channel, and which automatically bring a certain level of scientific peer recognition of the contribution. On the other hand, during our paper screening, we didn't retain any study paper in the range of 1980-1993 (see Figure 32). At last, this was not a blocker to our survey, but as a potential improvement to fill that gaps, the grey literature could support us to identify any additional suitable contributions within that range.

The 136 study papers were the first elements of our quest for quality models. Thus, for each reference about quality models in those papers, we looked for retrieving the original reference papers. However, sometimes it was not possible to retrieve or access to all the documents, even, the ones recommended by the authors and consequently we missed some models. For example, this is the case with April et al.'s study [START_REF] April | Software Maintenance Maturity Model (SMmm): the software maintenance process model[END_REF] where some of the quality models indicated by the authors were not available anymore. So, our choice was to exclude by default any quality model for which we couldn't get the original publications otherwise we could not confirm that the model is really a quality model, and then collect multiple data about them for a later usage and the software quality model landscape creation.

In addition, each time we found an original reference publication about quality model, we scrutinized the referenced papers to find further quality models. This was the snowballing part in our systematic literature review. We include also as valid quality model definition, the results of surveys where we have list of quality attributes due to the nature of this type of quality model where the aim is mainly to describe or define the key quality characteristics, or attributes. This the case for instance of Åkerholm et al. study [START_REF] Akerholm | Quality attribute support in a component technology for vehicular software[END_REF] on the most important quality attribute for vehicular software.

At the opposite, we exclude models for cost prediction (e.g., COCOMO [START_REF] Boehm | Software Engineering Economics[END_REF]), for measure (e.g., COSMIC [158] on functional size, IEEE 1045:1992 [212] on productivity, IEEE 1061:1998 [40] on metrics methodology) even if studies like the Kläs et al. CQML classification [START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF] include them as valid quality model. We decided to not include these models because they don't directly define, assess, control, or predict quality but they may influence quality. Moreover, we didn't include quality model for information quality (e.g., Knight and Burn IQIP quality model [START_REF] Knight | Developing a framework for assessing information quality on the World Wide Web[END_REF]) since their focus is on data quality rather than software quality, despite the fact that quality of data may be linked also to the quality of software. This is the same for Delone and McLean IS success model [START_REF] Delone | Information systems success: The quest for the dependent variable[END_REF] which includes quality aspect, but this model aims to predict the effect of Information System against the organization performance and not on software quality.

Another potential threat is about the analysis performed on each quality model, and more particularly on statistical or implicit quality models. It appears sometimes that the data mining method combine different approaches together and therefore it could be challenging to identify accurately the right main method. Moreover, our analysis was manual and because of the high volume of data to analyze, possibly some mistakes could happen. We trust that their number is minimum since we crossed check the extracted from multiple perspective (e.g., from other authors, during a second reading or data collection).

Lastly, concerning the appliance of our cladistic-based classification method, we didn't use all the defined taxa.

The reasons are the manual treatment of a high volume of data and the lack of proper tool to support and handle that work. Hopefully, we dispose of all the software quality model reference sources and, once an adapt tool is available, we can perform the full classification with all taxa and homology (see Figure 39). Furthermore, this tool should provide an online portal with free access to the quality model list, the classification elements, and allow to the community to contribute such an open-source project to maintain and consolidate that list. We must gather as well this cladistic method as an open-source structure jointly with the corresponding manipulation library otherwise this research work will become deprecated and unused.

Chapter VI. Quality Model Operationalization

Introduction

As we saw in our study on quality modeling applied to embedded software development, in Chapter II.3, often quality model solutions are not, or cannot be operational and use as they are defined. The quality model operationalization requires a particular attention that is usually neglect but it is possible to find contributions where one of the focus is to achieve an operational quality model. This is the case, for example, of Ahrens et al. study [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF]. Thus, it follows the purpose of this chapter, summarized through the research question 3:

Research Question 3 Considering a quality model for a software product, how to operationalize it?

By operation, we aim the development and use of quality model. So, to answer to this question, we must first understand what type of issues, if any, can prevent development and use of quality model. We are rephrasing this under the sub-questions 3a: What are the main challenges and the issues that prevent operationalization of quality model?

Once the issues and the challenges identified, we then need to identify the proper solutions to resolve, or at least to work-around, them. Nevertheless, we must avoid too general solutions which won't bring any solutions but rather bring more questions or issues. So, we must identify practical solution (i.e., solutions related "to actual experience or to the use of knowledge in activities rather than to knowledge only or ideas" [227]) either from the reuse of existing academic or industrial solutions, or from the creation new and tailored solution. This leads us to our next research sub-question 3b: What are the practical solutions to those challenges and issues?

Finally, the final step is to have a repeatable and systematic way to operationalize, based on the findings of 3b, a software quality model. To achieve repeatable and systematic behavior here, we must define a process that handle the operationalization (i.e., development and use) of a software quality model. Therefore, we land the research sub-question 3c: What is the process to ensure quality model operationalization? In the following sections, we are addressing each of these questions sequentially because of their respective dependencies: 3c depends on 3b results, which depends on 3a results.

Operational challenges and issues with quality model

Research Sub-question 3a What are the main challenges and the issues that prevent operationalization of quality model?

In 2012, Thapar et al. performed a comparative study on quality models with the aim to identify "what challenges are posed by quality models" [11]. They first identified a group of 24 quality models that they considered significatively representative for the purpose of their study. They categorized the quality models that emerged until 2001 as basic quality model, and the later quality model as tailored quality models because these models were basic quality models that were modified or completed to answer to the increase of software industry needs on quality evaluation and improvement. We note that the authors' basic quality model set was made of five quality models that we also referred to as key contributions to quality modeling of software: Boehm's quality model [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF], McCall's quality model [41], Dromey's quality model [START_REF] Dromey | A Model for Software Product Quality[END_REF], FURPS' quality model [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] and ISO/IEC 9126 quality model [START_REF]Part1: Quality Model[END_REF]. Then, Thapar et al. investigated the main obstacles, or challenging issues which impede the development and practical use of quality models before applying these findings to compare the 24 models together. Thus, the authors identified 9 main potential issues. These are:

 Association: The software development process and quality model are not, or not sufficiently, formally associated together, thus affecting the implementation of quality characteristics and measures. For example, in agile (i.e., incremental and iterative software development aligned with the agile manifesto [START_REF] Beck | Manifesto for Agile Software Development[END_REF]) and rapid (e.g., Scrum, Extreme Programming) software developments, often being customers centric, mis-association or lack of involvement with quality modeling may results, for instance, in late consideration of quality characteristics and requirements, in business value priority versus quality, in lack of awareness quality characteristics and requirements by customer or DevOps team.

 Evolution: Limited evolution of quality models may lead to them becoming obsolescent, prevent their being adapted or their scaling during the development process. This is often the case statistic quality models, for example, since they are designed from specific data set. Another example is Dromey's quality model which is quite abstract and limited within its four quality characteristics (i.e., correctness, internal, contextual, and descriptive) to evolve easily.

 General: Quality characteristics and sub-characteristics of quality models are too general to be applicable as is, or too specific to be applicable to multiple use cases. This is often the case with standards (e.g., ISO/IEC/IEEE 25010) which tend to cover a large set of possibilities; their drawback is that their quality characteristics and sub-characteristics are too general. Wagner et al. conducted a survey [83], [START_REF] Wagner | Quality Models in Practice: A Preliminary Analysis[END_REF] on quality model in international German companies and they found that 79% of the companies that rely on quality models taken from standards have customized these models to use them.

 Guidelines: Quality model guidelines or documentation are frequently not complete enough to enable stakeholders, including software architects and developers, to understand or use the specified quality model. For instance, the Garcés et al. quality model for ambient assisted living systems [START_REF] Garcés | A Quality Model for AAL Software Systems[END_REF] is not sufficiently detailed to be used because the authors explained only its main definition and principles. At the opposite, McCall [41] and ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF], [146], [START_REF]Software engineering -Product quality -Part2: External Metrics[END_REF], [156] quality models are two examples described with enough definition, details and context explanation to be deployed and use in software development.

 Maintainable: Quality models which cannot be maintained regularly may greatly impede development due to having to fix defects (e.g., wrong quality sub-characteristic decomposition, or incorrect metrics) or integrate new requirements. We saw in Chapter IV.2.b with Kano's model [58],quality perception like stakeholder quality requirements changes over times and therefore quality models require regular changes to avoid becoming quickly obsolete or reflect a wrong view of current quality. Moreover, under the maintainable scope we consider another aspect not covered by Thapar et al. This is linked to configuration management and versioning of quality model. Indeed, transportation system regulation (e.g., automotive, aeronautic) requires having mechanism to keep and recover configuration elements of produced systems. This includes not only documentation, source code, test material, development tools, to cite few, but also quality modeling material.

 Risk-Driven: The risk-driven aspect is limited or missing in the quality model, and therefore makes early risk mitigation difficult to achieve. In development, for example, this is particularly important with the case of quality characteristics describing non-functional requirements (e.g., usability, security, reliability, or performance efficiency). Indeed, they are often deferred at a later stage of the development process, and consequently their achievements may be in difficulty if the related quality modeling is not connected with risk management.

 Stakeholders: Insufficient involvement, or participation of stakeholders in quality model development, quality evaluation and quality framework challenge the quality model buy-in by stakeholders, as well as the correct quality perception and expectation of final customer. In Rapid development (e.g., extreme programming), for example, customer works closely with developers to foster communication, feedback, and shorten loopback on what is being developed to successfully match customer expectations.

 Subjective evaluation: Lack of objective quality evaluation, due to a lack of sufficiently detailed and complete metrics, results in a less than objective assessment, as Ahrens et al. highlighted also in their survey [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF]. This is also the case where the quality assessment chain, including metrics, is partially or not automated. Indeed, automation allows systematic reproducibility of outcomes, increasing the objectivity. At the opposite, a lack of automation is a source of preface to subjectivity. However, subjective evaluation happens in Rapid software development too, where continuous integration and delivery rely on automated building, deployment, and testing capability. The cause of this subjectivity is the time constraint with short iterations which result in limited testing (i.e., in assessment or control activity), for example [START_REF] Behutiye | Management of quality requirements in agile and rapid software development: A systematic mapping study[END_REF].

 Validation fairness: The idea behind this quality modeling challenge is the fact that we cannot be judge and party. Thus, once a quality model is developed, it must be validated by independent experts -ideally-, otherwise the lack of independency in the quality model validation shall result in biased quality assessment, control, or prediction.

In Thapar et al. studies, the 24 quality models were thus compared to each other based on the number of issues found in each model. The study concluded that at least three issues -not always the same ones -were found in each of these quality models, and five of them only had three issues. The short list of those with only three issues includes ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF], GEQUAMO [START_REF] Georgiadou | GEQUAMO-A Generic, Multilayered, Customisable, Software Quality Model[END_REF], Bawane [START_REF] Bawane | A Novel Method for Quantitative Assessment of Software Quality[END_REF], Alvaro [START_REF] Alvaro | A Software Component Quality Model: A Preliminary Evaluation[END_REF] and Kalaimagal [START_REF] Kalaimagal | Q'Facto 12: an improved quality model for COTS components[END_REF].

Although the authors detailed for each issue its rationale and an idea of solution, they did not evaluate the relative importance of the issues, nor did they put forward practical solutions for meeting the corresponding challenges. So, taking the data of Thapar et al., we analyzed the frequency of occurrence of the 9 issues with this set of 24 quality models -see Figure 52. We found that the three most frequent issues (risk-driven, association and validation fairness) accounted for ~49% (cf. Figure 52) of the issues and, when we extended that list with the next three most frequent issues (guidelines, evolution and maintainable), we found ~86% of the challenges. These results mean that by solving risk-driven, association and validation fairness issues, ~49% of the challenges that impede the development and use of quality models can be met; further, by also addressing the next three challenges, ~86% in total of those issues were covered. But, according to the analysis of general, subjective evaluation and stakeholder, these issues did not seem to occur often, which is surprising because the two last issues occur more frequently in reality.

Figure 52 -Frequencies of the 9 main issues identified by Thapar et al. [11], which impede the development and use of quality models

According to our readings and experience, these 9 potential issues are not the only main ones to be taken into account. Indeed, when modeling quality, it is critically important that all new or reused terms, concepts and associated definitions be unambiguous, agreed upon and understood by all stakeholders.

Moreover, attention needs to be paid to redundant or duplicated elements because they usually result in rephrasing, thus increasing the risk of inconsistency and making knowledge more complex. These are terminology and redundancy issues. There are references to them in some of the harmonization issues with ISO-based quality models that were identified by Abran et al. [96]. So, to summarize:

 Terminology: Lack of unambiguity, consistency, agreement and understanding for reused terms, concepts, and their definitions. This issue occurs particularly when standard glossaries (e.g., ISO/IEC/IEEE 24765 International Standard Systems and software engineering-Vocabulary [START_REF] Han | Evaluating perceived and estimated data quality for Web 2.0 applications: a gap analysis[END_REF], the international vocabulary of metrology [START_REF] Bipm | The international vocabulary of metrology-basic and general concepts and associated terms (VIM), 3rd edition JCGM 200[END_REF]) are ignored in favor of proprietary definitions or company jargon.

 Redundancy: Redundant or duplicated elements in quality model that usually result in rephrasing. For example, over the ISO/IEC SQuaRE series, there are duplication of terms and definitions mainly defined in the ISO/IEC 25000 [25] and ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF] which is "contrary to the ISO practice" as Abran Nevertheless, duplication and rephrasing can be sometime saving. In Boehm's quality model [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF] some quality sub-characteristics are linked to two or three quality characteristics (e.g., accessibility is linked to efficiency, human engineering and testability ; see Figure 14). Duplication of these sub-characteristics surely helps to have a proper description of them based on the characteristic contexts.

Within a predictive quality modeling scheme, Khoshgoftaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF] performed cost-analysis on several predictive quality-modelling software. They showed that accuracy and reliability -based on the effectiveness and efficiency of quality models -are significant factors or issues as well.

 Accuracy: Lack quality model of accuracy, including in repeatable prediction results, where the quality model results are not in the defined precision range and therefore produce invalid results.

 Reliability: Lack of quality model effectiveness or efficiency causing diverged results over repeatable uses of quality model within the same conditions (cf. reliability in measurement in Figure 15).

Synthetizing both our research and our practical experience in quality modeling, we identified three other issues which also affect the development and practical use of quality models.

First is their maturity. Maturity is not only a matter of age or youth of the quality model. It is a factor which combines the intensity and history of model use, the development context (research vs. academic vs. industrial), its application for an actual use case, its review history, its evolution, and revision history.

The next issue is expertise in quality modeling knowledge and activities. Without expertise, or expert people the development and deployment of quality model is uncertain or at least shall suffer from lack of properly answering to quality requirements. As well, a loss of expertise, or familiarity, due to a loss of competency or an unmastered complexity increase in quality modeling leads to the same consequence. This issue can be therefore assimilated to the fifth Lehman's law of software evolution, "Conservation of Familiarity" [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF].

Moreover, the importance of quality modeling may be underestimate or even overlooked. This leads us to the final issues: neglect.

The neglect issue is the most critical one. Indeed, if quality modeling or quality model is purely and simply neglected, ignored or underestimate then no quality modeling can happen. This is therefore the most critical and first issue to solved.

 Maturity: Lack of quality model fully grown, developed, reused, and consolidate. A good illustration of mature quality model is the ISO/IEC/IEEE 25010 quality models. These matured models are not only evolution of the ones from ISO/IEC 9126, but also the results of the ISO document life cycle process, involving international academic and industrial participants.

 Expertise: Lack of expertise in quality model development or use may inhibit quality modeling, evaluation plan, measurement, quality assessment, control, or prediction activities. Likewise, without security or safety expert, for instance, security or safety quality requirements implementation, testingmanual or automated-and compliance to standard are extremely difficult tasks.

 Neglect: Lack of consideration of quality model, neglecting consciously or unconsciously quality modeling activities is the most important threat to quality modeling. In Behutiye et al' systematic study on the management of quality requirements in agile and rapid software [START_REF] Behutiye | Management of quality requirements in agile and rapid software development: A systematic mapping study[END_REF], the authors identified several causes of the neglect of quality requirements, and consequently quality characteristic and modeling, by agile software development teams. We can cite for example those teams focusing on shorten delivery time (i.e., implementation and validation of quality requirements may be time consuming) and on functional requirements, using of existing internal infrastructure, or waiting for a request or need to proceed on the quality requirements.

To conclude our exploration and analysis on issues preventing operational quality model development and use, we end up with a list of 16 potential issues that are summarized in TABLE 20, by alphabetical order.

Practical solutions to the operational issues

Research Sub-question 3b What are the practical solutions to those challenges and issues?

In previous section, we have identified 16 obstacles that prevent operation of software quality model (i.e., development and use). Thus, in the following paragraphs, we are exploring how to resolve or workaround practically these obstacles whenever it is possible, addressing then the research sub-question 3b: "What are the practical solutions to those challenges and issues?".

By practical solution, we mean solutions related to experiences, real situations or actions that are possible to reproduce, reuse or deploy. Thereby, we aim to avoid enumeration of hints or ideas like in Thaphar et al. study [11], which lead to inefficient resolutions because they require a non-negligible effort to make them in practice, even when these ideas are good sense.

Hopefully, in sections from Chapter IV.3 to Chapter IV.6, we already explored quality modeling and qualimetry technologies. So, we decided to take advantage of them since they are well documented, studied, and proven practice.

 Association: To address this issue, the practical solution must aim to foster the association between quality modeling and software development process. Grady and Caswell's FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] and then FURPS+ [START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF] quality modeling approach is a good illustration of what must be achieved to meet this challenge. Indeed, they took into consideration the software development life cycle in the definition and instantiation of their quality model, keeping for the same quality characteristics over the different development life cycle stages but they associated a specific set of metrics to each stage. The interest with that approach is that quality modeling remains constantly aligned with the current software development process stage. An example is shown in TABLE 11.

 Accuracy: As we indicated previously this type of issue is similar to the accuracy measurement problem exposed over Figure 15. So, the accuracy issue needs to be addressed by measuring, analyzing, and then making the right adjustments and enhancements to quality models. Such practical solutions may be inspired by the method of Khoshgoftaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF] with regard to accuracy aspect. The authors used test data to evaluate the accuracy of their model, and after analyzing the results, they made decision on the adjustments to propagate to their quality model.

 Evolution: Qualimetry offers a solution for meeting the evolution challenge. Its polymorphism mechanism [START_REF] Argotti | Quality Quantification Applied to Automotive Embedded Systems and Software[END_REF] (see also Chapter IV.6.b) enables built-in adaptation and evolution in quality modeling, allowing smooth and natural evolution of quality modeling, deriving new quality models from previous ones without breaking existing overload or inheritance behaviors. Furthermore, the polymorphic quality model simplifies the building and using of quality models independently from their general characteristics and the quality modeling target.

 Expertise: The practical solution for this issue is first linked to people management. Thus, recruiting and developing people expertise in quality modeling is fundamental as well as retaining the talented people [START_REF] Brooks | No Silver Bullet Essence and Accidents of Software Engineering[END_REF] in that domain. An organization should be able to count on some experts that can drive the quality modeling strategy, even on data science, and disseminate the knowledge over the organization. The second element is to control or maintain the mastery of the quality modeling overall complexity. This means that evolving development and use in quality modeling must be achieved with the respect of the fifth Lehman's law, "Conservation of Familiarity" [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF]. Moreover, if we follow the Agile Manifesto [START_REF] Beck | Manifesto for Agile Software Development[END_REF], "Continuous attention to technical excellence and good design enhances agility". So, quality modeling expertise cannot be skipped even in the modern software development methodology like Scaled Agile Framework (SAFe) [START_REF] Scaled Agile | SAFe: Team and Technical Agility Team[END_REF] where cross-functional agile teams concentrate all type of expertise around the customer.

 General: Likes for evolution the polymorphism mechanism provided by qualimetry allows to derive polymorphic quality models from general quality model or quality characteristics that are overly general, independently of the quality modeling target. An example of practical build of polymorphic quality model is given in next Chapter VII.5.

 Guidelines: Concerning guidelines issue, the solution is to follow the ISO 9001 recommendation or guidelines [233] stating that it is critical to document everything with the right level of detail, for instance. Moreover, the documentation must be versioned and easily accessible to the entire organization using the dedicated and appropriate document management tool.

 Maintainable: To strengthen the maintainable aspect of quality model, the Miyoshi & Azuma [234] recommended that the number of key factors, or characteristics should be kept between three to eight. We find the same guidelines in qualimetry, where Azgadnov et al. [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF] referred, without citing it, to Miller' study in experimental psychology [START_REF] Miller | The magical number seven, plus or minus two: some limits on our capacity for processing information[END_REF] about the human capacity for processing information. Thus, the recommended number is "7±2" and it is nicknamed "the magic number" or "Miller's law". In both cases, we noted that these numbers are quite similar. Additionally, compliance with tree derivation rules [38] (cf. TABLE 8 and TABLE 9) for controlling the analysis of quality characteristics, their organization and the quality model complexity, formalized in qualimetry, fosters a high level of maintainability in the case of hierarchical models. In parallel to these quality model attributes, a proper level of technical documentation, aligned also to the Guidelines issue, is an important factor influencing the maintenance capability such as documented source code can do.

 Maturity: Due to the nature of maturity, this issue deserves a specific focus since it evolves over time, and depends greatly on the changes, or decisions made in the quality modeling which should thus be done carefully, while anticipating the next steps as in a game of chess. However, by applying a capability maturity model such as CMM [START_REF] Paulk | Capability Maturity Model for Software v1.0[END_REF], CMMi [START_REF] Chrissis | CMMI for Development: Guidelines for Process Integration and Product Improvement[END_REF], [START_REF]CMMI V2.0, model at-a-glance[END_REF], or A-SPICE [START_REF]13 / Automotive SIG[END_REF]to quality modeling activities (i.e., as a process, and to quality model characteristics, sub-characteristics and metrics), we can assess, control, and enhance maturity, and consequently address this issue. Since our industrial context in automotive, we recommend A-SPICE and its 5 capability levels shown in Figure 53. Highest level means highest maturity, and in our case, level 0 indicates that there is no quality modeling performed.  Neglect: This issue is complex to solve without a strong and unwavering support to quality modeling, or qualimetry, from the academic and industrial leaders. Therefore, role modeling, showcasing of big win examples thanks to quality modeling (e.g., industry success in certification, academic student evaluation), and proper explicit communication to software engineering and systems engineering teams are the right tools for practical solution. One of the Agile Manifesto principle [START_REF] Beck | Manifesto for Agile Software Development[END_REF] is "Continuous attention to technical excellence and good design enhances agility". De facto, applying it means that even quality modeling shouldn't be neglected in software development, especially with agile and Rapid software development methodology where quality requirements are often neglected or overlooked, and consequently  Redundancy: Following the ISO practices which advise to avoid duplication, it obviously makes sense to be careful, rigorous and keep in mind this advice to limit redundancy when developing and using of quality models. Moreover, factorization of concepts, terms, quality modeling elements (e.g., quality characteristics, perspectives, metrics, quality model) jointly with references to standards such as the international vocabulary of metrology [START_REF] Bipm | The international vocabulary of metrology-basic and general concepts and associated terms (VIM), 3rd edition JCGM 200[END_REF], for example, decrease the risk that redundancy issue occurs.

 Reliability: The reliability issue resolution is close to the accuracy one. Thus, it needs to be addressed by measuring, analyzing, and then making the right adjustment and enhancements to quality models. Khoshgoftaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF] method with regard to reliability aspect can also be considered. However, there is a difference with the practical solution for accuracy. Indeed, to address reliability issue, we should rely on automation to ensure repeatable quality model results.

 Risk-driven: Inspiration for resolving risk-driven issues can be found by combining ISO/IEC 25010 [22] and ISO/IEC 25022 [START_REF] He | Software reliability analysis on embedded system based on SFMEA and SFTA model[END_REF]: the "freedom from risk" quality characteristic, its sub-characteristics (both defined in ISO/IEC 25010) and their measurements (defined in ISO/IEC 25022) demonstrate how the risk aspect can be included in quality modeling. The ISO/IEC 25022 standard is of major importance because it describes how the risk is managed with two quality threshold levels. This is risk versus opportunity associated with the level of quality (see Figure 18). If the measured level of quality is below the lowest threshold, the risk is unacceptable, or not manageable. If the measured level of quality is between these two thresholds, then the risk is acceptable or manageable. Finally, if the quality level is above the highest threshold, it is providing a development opportunity. In fact, by generalizing the use of these two quality threshold levels instead of a single quality threshold level (e.g., acceptance threshold), we automatically encompass the risk aspect in our quality modeling.

 Subjective evaluation: Regarding subjective evaluation, a complete metrics specification with enough detail to apply them is required, and ideally most of them should be automated. We can use Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF],

McCall [41]or ISO/IEC 9126-2,3,4 [START_REF] Garvin | Competing in the Eight Dimensions of Quality[END_REF], [START_REF] Roedler | Systems engineering leading indicators guide -version 1.0[END_REF], [START_REF] Roedler | Systems engineering leading indicators guideversion 2.0[END_REF] as beginning source of information for metrics. Then we have to select and apply correctly aggregation operators. There are many different types of aggregation operators [START_REF] Detyniecki | Fundamentals on Aggregation Operators[END_REF] (e.g., arithmetic mean, weighted mean, symmetric sum, k-order statistics) and their choice depends on the purpose for which they are intended, according to Wagner [START_REF] Wagner | Software Product Quality Control[END_REF] (e.g., assessment, prediction, comparison, hotspot identification and trend analysis). Chapter IV.3.d and TABLE 39 from Annex 1 give further details on aggregation topic. An alternate aggregation approach is the "Logic Scoring of Preference" of Dujmovic and Bayucan [START_REF] Dujmovic | A Quantitative Method for Software Evaluation and its Application in Evaluating Windowed Environments[END_REF] shown in Figure 17 and by the equation 3.

The main idea is to specify the relationships between the inputs and the output based on trade-off between conjunction, neutral and disjunction relationships. Concerning the inclusion of quality characteristics and sub-characteristics relationship, Khaddaj and Horgan [START_REF] Khaddaj | A Proposed Adaptable Quality Model for Software Quality Assurance[END_REF] established quality characteristics by means of a factor polarity profile (see TABLE 7) with direct, neutral and inverse relationships. From another perspective, Perry's quality control checklists [START_REF] Perry | Effective Methods for Software Testing[END_REF] can be adapted to have objective check procedures to quality modeling activities. Finally, to shorten continuous integration, testing and delivery, agile and Rapid software development method encourage automation which contributes as well as reducing subjectivity in the evaluation with repeatable and constant method to evaluate quality.

 Stakeholders: For this issue, all stakeholders must be involved as early as possible to build or customize a quality model that takes all stakeholders' visions into account in accordance with Horgan's views [START_REF] Horgan | An essential views model for software quality assurance[END_REF]: "Key Quality Factors" are those quality factors which are relatively invariant per project, i.e., product and stakeholders, while "Locally Defined Factors" are the quality factors that change, depending on the project, product, or stakeholders. Over the "Quality Assessment Algorithm" of qualimetry [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], expert and non-expert stakeholders participate in quality characteristics, sub-characteristics, and weight factor determination through meeting sessions or surveys. Thus, since the stakeholders are part of the quality model development, their vision is shared and aligned with the resulting quality model.

 Terminology: Like for redundancy, to reduce terminology issue, it is also vital to rely on terminology standards likes the international vocabulary of metrology [START_REF] Bipm | The international vocabulary of metrology-basic and general concepts and associated terms (VIM), 3rd edition JCGM 200[END_REF], ISO/IEC/IEEE 24765 International Standard Systems and software engineering-Vocabulary [START_REF] Han | Evaluating perceived and estimated data quality for Web 2.0 applications: a gap analysis[END_REF], or the international software testing qualifications board vocabulary [111], to cite few. These standards ensure that a set of common vocabulary and concepts are clearly specified and shared broadly. Concerning other than standard vocabulary or concepts used in quality modeling during development or use, the solution consists in ensuring compliance of the terminology with the "five C's" of requirements engineering (i.e., correctness, completeness, consistency, conciseness, and clarity) [9].

 Validation fairness: Resolution of validation fairness issue may be achieved thanks to qualimetry science and its "Quality Assessment Algorithm" [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]. This algorithm guides quality model developers and helps them assess quality models by upstream involvement of experts and non-experts through technical, expert, and steering groups. A particular attention must be made with regard to the independency of involved parties. This validation with expert can also be completed with a joint validation between empirical results (e.g., results from the use of quality model to assess quality of a certain product) and expert (e.g., results from validation expert who assesses quality of the same product that empirical one). This approach was applied by to validate the QMOOD quality model they built [START_REF] Bansiya | A hierarchical model for object-oriented design quality assessment[END_REF].

So, we identified and proposed a complete set of practical solutions for the issues that may prevent the operationalization of quality models. Figure 54 gives an overview of those practical solutions mapped to the corresponding issues. Next section focuses on the construction of two processes that take advantage of these solutions to guide efficiently development and use of quality models in quality evaluation.

Operational Contributions

Research Sub-question 3c What is the process to ensure quality model operationalization?

After the analysis and finding of 16 issues that may prevent development and use of quality models, and then the identification of practical solutions for each of them, we now explore how we can take benefit of them during quality model operations. Thus, the result of this exploration responds to 3c: "What is the process to ensure quality model operationalization?".

We remark that the definition and deployment of such process linked to quality model development and use is aligned to the process maturity capability level 2 (i.e., Managed) expectations (see Figure 53). Moreover, the tailoring of that process to specific projects raises the process maturity to capability level 3 (i.e., Established).

a. Quality model operational use: The "Quality Thermometer"

The purpose of this process is to cover the operational development and use of quality models in a context of monitoring project progress and status, including product quality.

There are various framework solutions for the visualization and monitoring of project progress and status such as dashboards or scorecards [START_REF] Solomon | Dashboards Vs Scorecards -An Insight[END_REF].

A dashboard usually displays key performance indicators graphically, via graphics, gauges, or meters that are updated frequently (i.e., hourly, daily), but users must interpret the data.

A scorecard consists in metrics, targets, and trends, showing how the measures performed against the targets. The scorecard is usually updated on a weekly, monthly or quarterly basis, and like the balance scorecard [START_REF] Kaplan | The Balanced Scorecard-Measures that Drive Performance[END_REF], it facilitates the alignment of company strategy and project objectives.

In our case, we decided to define and use a project scorecard with the necessary details to accurately monitor status and progress for project, process and product. In addition, the division of our project scorecard (cf. Figure 55, Table 49 and Annex 7 for further details) into 6 main blocks was the result of two brainstorming sessions organized with our main stakeholders. Each block addresses a specific status area of the project.  Project dimensions: details about project and product sizes; these are used for the purpose of normalization, weighting and/or for monitoring complex systems.

 Project indicators: a combination of metrics from a project (e.g., sprint or feature completion, scope creep [START_REF]Project Management Institute, A Guide to the Project Management Body of Knowledge (PMBOK Guide) -Sixth Edition[END_REF]), process (e.g., lead and cycle time) and product (e.g., software product quality).

 Quality performance indicators: a synthesis of the conformity to A-SPICE processes and their workproducts.

 Safety & regulation performance indicators: a synthesis of the conformity to safety and regulation processes and their work-products, with regards to applicable standards.

 Internal and external software metrics: a collection of internal and external software metrics linked to some quality characteristics and sub-characteristics of quality model; this collection is made up of the most frequent metrics used by the development, verification, and validation teams.

In the project indicators block, we can see the reference to software product quality which is computed thanks to a quality model, and a set of internal and external software metrics. We also note in passing that this quality model and the required metric set depend on both the project scope and the current project milestone, or life cycle stage.

However, even though the quality model usage appears to be transparent, hidden behind the unique software quality product indicator in our current case, its use must be framed by a formal process in order to be clear and consistent. As a result, we defined a seven-step process that is symbolized under the shape of the "quality thermometer" for the project (shown in Figure 56), as "taking the temperature" of the software product quality.

The description of this process is intended to:

1. Identify the project characteristics: We capture the current project and product definitions, the current targeted milestone, the ype of indicators (i.e., leading vs. lagging) and which quality perspective(s) we aim to consider between product, user, manufacturer.

2. Define and customize the Quality Model: We build, or customize the right quality model, in conjunction with polymorphism to match the project characteristics (e.g., type of product, project, and stage in the product life cycle). Since the development of quality model requires multiple steps to integrate properly practical solutions for association, evolution, general, maintainable, stakeholder involvement, and terminology, we decided to build an independent and complementary process for quality model development. The detail of this process is given in next Chapter VI.4.b.

3. Define the metrics and update the scorecard: We first identify the software metrics set associated with the quality model characteristics and sub-characteristics, starting from the internal and external software metrics pre-listed in the scorecard, completing this list as needed. This association is done accordingly to the same approach than Grady and Caswell did with FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] and then FURPS+ [START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF], taking into account development life cycle impact. Afterwards, we update the scorecard block for internal and external software metrics to reflect the selected metrics set.

Define relationships and aggregation:

During this stage we identify the weights and the relationships between quality characteristics, sub-characteristics, and metrics. We rely on Logic Scoring of Preference -combinations of conjunction and disjunction relationships between inputs and outputs-, polarity profile, and aggregation operators.

Collect software metrics:

We now have all the elements we need to start measuring the software product quality by collecting the selected software metrics -from stage 3. We apply as necessary the polymorphism behavior regarding evolution linked to time, or more precisely, aligned to the product development life cycle stages. Moreover, we dispose of historical metric data allowing to perform any data analysis or graphical display at that step.

Build quality indicator:

To compute the quality indicator, we aggregate these metrics, quality characteristics, and sub-characteristics based on the defined quality model -from stage 2-, then weights, and relationships -from stage 4.

Figure 56 -Inheriting of the practical solutions to the operational issues, the "Quality Thermometer" process for project 7. Apply threshold with color coding and report indicator: Finally, we report the quality indicator on the project scorecard. We also use color coding associated with the range breakdown in ISO/IEC 33020 [START_REF] De Souza | Psychometric properties in instruments evaluation of reliability and validity[END_REF] (see Figure 57) with the percentage of this quality indicator over the targeted threshold. This allows us to give simple and direct visual feedback on the currently achieved quality level. Epilogue: Once this quality indicator has been reported, we are ready to loop, proceeding again -until it is required -in stage 5, collecting new software metrics, building a new quality indicator, reporting it, and so forth.

Note that the scorecard gives an instantaneous view of the project status, and that it must thus be completed with a set of trend charts, displaying dynamic views, showing trendlines -linear or not-, that may be useful for prediction. In addition, we use two levels of thresholds based on ISO/IEC 25022 [START_REF] Chrisman | Rethinking levels of measurement for cartography[END_REF] (cf. Figure 18)risk/opportunities -to indicate when quality is at a risk level and thus manageable or not, and when we move to an opportunity.

Finally, over this process a certain number of pratical solutions have been explicitly or implicitly included to address association (i.e., at stage 3 and 5 levels), evolution (i.e., at stage 2, 3, 4, 5 and 6 levels), general (i.e., at stage 2, 3, 4, 5 and 6 levels), risk-driven (i.e., at stage 3, 5,6 and 7 levels), subjective evaluation (i.e., at stage 3 level) and maturity (i.e., at process level) issues.

Moreover, the main benefits of this approach are that it offers synthetic homogeneous views between projects, whether this be for supplier or internal development teams, as well as bridging different disciplinary teams thanks to a common vocabulary from polymorphic quality models.

The next section describes the quaility model build process, completing the "Quality Thermometer" process for project.

b. Quality model operational development: a 6 stages process

The purpose of this process is to cover the development aspect of the quality model operationalization. In order to build the right quality polymorphic model, addressing then general and evolution issues, we must follow a sequence of six stages as summarized in Figure 60: identify the origin of the quality model, build a survey item list, organize the survey, analyze the survey results, construct quality models and determine which common quality model to use as a basis.

1. Identify the origin of the quality model: In order to initiate the development of a quality model that can be required by a project (i.e., in the scope of the "Quality Thermometer" process for project), for instance, we begin our process by identifying coarsely a quality model to serve as starting model. Indeed, with more than 50 years of active research contributions in quality modeling, there exists many valuable quality models.

Therefore, our assumption is that by fostering reuse of quality model rather than creating from scratch a new one, it allows us to better take advantage of existing contributions. So, by default we consider as original model current standard for systems and software product: ISO/IEC/IEEE 25010 quality models that cover both product and user views of quality and are conform to A-SPICE. However, this origin can be different. For example, if a company has in-house quality models, it is recommended to use them as origin.

Regarding common and specific product characteristics, the aim is to associate each of them with some quality characteristics from the original quality model if possible, otherwise to complete them. For example, "safety" can be associated with "freedom from risk" and also "security", "human-machine interface" with "satisfaction" and "usability". Moreover, we model them by applying the ad hoc and universal polymorphism mechanism to the quality model.

Nevertheless, such analysis and association are not straightforward. They require a large spectrum of knowledge including the project, systems or software product, quality characteristics and quality modeling. Furthermore, the stakeholders, who depend on that quality model, must buy-in to the resulting quality model. We solved that requirement by involving the main experts and stakeholders in the quality modeling activity, and more particularly by getting them to identify quality characteristics and sub-characteristics thanks to a survey, as our next building stage.

Build the survey item list:

The survey is divided into 4 stages. The first one introduced the objective of the survey and collected the contributors' roles and projects. In term of contributor audience, we should consider our main stakeholders likes project managers, architects, verification and validation leaders, and assurance quality engineers, for instance.

For the second stage, we asked to the survey participants to rank the importance of each of the quality characteristics -from the quality model(s) identified as origin and from product characteristics -on a scale from -2 ("not important at all") to +2 ("extremely important") based on their technical knowledge and their own vision of the product and project quality. We chose to use a 5 points Likert's scale [START_REF] Likert | A Technique for the Measurement of Attitudes[END_REF] (e.g., Figure 58) because usually all the stakeholders are already familiar with it. This scale is symmetric with the central value (i.e., point three here) considered to be the neutral value, and Likert's items can be assigned to values (e.g., 1 to 5, or -2 to +2) to facilitate their processing and analysis. The survey's third stage also focused on ranking by order of importance, but rather on all the quality sub-characteristics of the quality model(s) identified as origin and the and the product subcharacteristics. We separated the assessment of quality characteristics from that of the subcharacteristics to capture the stakeholders' perceptions of them. For instance, if a quality characteristic was ranked -2 and all its sub-characteristics were ranked +1 or +2, it means that this quality characteristic was misunderstood or was wrongly ranked. Another example could be, if one of the sub-characteristics of a quality characteristic was highly ranked compared to the other sub-characteristics, then it became the determinant sub-characteristic. Furthermore, we added a sixth ranking choice, +3 ("I don't know") because there were non-experts participating in the survey. This is because they might not have an opinion, or correctly understand some quality sub-characteristics which are in fact more subtle than quality characteristics. With this sixth ranking choice we avoided them biasing the overall survey results by selecting another default answer. The last stage consists in an open question, offering the survey participant to suggest some missing characteristics and sub-characteristics, and share additional comments that may influence quality modeling. Note that the survey elements (e.g., questions, answer proposal, descriptions) must rely as much as possible on standard glossary and metrology vocabulary to avoid any confusion or rephrasing in terminology.

3. Organize the survey (or hybrid method): This approach is similar to the qualimetry hybrid method [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF], where a panel of domain experts and non-experts is asked to evaluate the values of a set of quality characteristics and sub-characteristics. So, during this stage each of the targeted participants are contacted and asked to answer the survey based on their own understanding and perception of the quality. Consequently, we incite the stakeholders to participate in the quality model development, ensuring to get their implicit buy-in on the resulting quality model.

4.

Analyze the survey results: Once the survey had been answered and its data collected, we analyze the data, our fourth building stage, by verifying the participant agreement consensus, or interrater reliability, based on Cohen's kappa ⲕ [START_REF] Cohen | A Coefficient of Agreement for Nominal Scales[END_REF] (see equation 12) in the case of two raters (in this case the stakeholders who contributed to our survey) for a set of answers and Fleiss' kappa ⲕ [START_REF] Fleiss | Measuring nominal scale agreement among many raters[END_REF] (see equation 13) for three or more raters (see Chapter VII.4 for examples and explanation on the calculations of Cohen's kappa ⲕ and Fleiss' kappa ⲕ).

For each project, we checked whether Fleiss ⲕ was available for all roles and if it was at least greater than 0.4 (i.e., Moderate from Table 21). We thus selected a data set based on the highest Fleiss ⲕ for each project, or product, and all roles, or where a role had the highest ⲕ. When this was not sufficient, we used Cohen ⲕ as a decision criterion.

If no kappa showed at least a fair agreement, then we looked for consensus at the specific role level and all projects, or products. In case of poor of slight consensus between participants, we perform another survey round, restarting at stage 2 of the process but with highlighting the areas of divergences and areas of convergences on quality characteristics and sub-characteristics.

𝜅𝑎𝑝𝑝𝑎 = 𝜅 = 𝑝 -𝑝 1 -𝑝 ( 12
)
where 𝑝 = 61 assumes that at least two raters participated to the survey for each project. Moreover, it integrates the fact that a quality model can be composed of sub-quality model corresponding to a specific perspective. For example, with ISO/IEC/IEEE 25010, we can distinguish two quality perspective: system/software product quality and quality in use (i.e., user).

Then, for each project, and each specific perspective, we checked whether Fleiss ⲕ was available for all roles (i.e., all raters for the project or ECU) and if it was at least greater than 0.4 (i.e., Moderate). We thus selected the data set based on the highest Fleiss ⲕ for that project, or ECU, and all roles. If no Fleiss ⲕ is at least moderate for all roles, we look for result for specific role. Otherwise, we used Cohen ⲕ as a decision criterion starting from all roles and then specific role result, if any. If no kappa showed at least a moderate agreement, then we first extend our analysis to all quality perspective for the current project or ECU, applying the method describe above. If we still don't succeed to find a proper Fleiss or Cohen's ⲕ then we look for consensus at all projects, or ECUs, level, starting from the current quality perspective with all roles and applying again our analysis heuristic. Finally, in the case where no consensus can be found at project, role, and quality perspective level, we must conduct a new survey. Construct the quality models: Once the data set had been selected, to build our polymorphic quality models we determined the most important characteristics and sub-characteristics using the scale set value set: {-2; -1; 0; +1; +2}.

First for all characteristics and sub-characteristics, we determine their importance value by calculating the mean values of their ranking values from "not at all important" (i.e., -2) to "extremely important" (i.e., +2). For example, two raters answered "not at all important" and "extremely important" for a characteristic, its importance value is: ( ) ( ) = 0.5. So, all characteristics, with an importance value at least very important (i.e., +1), are included in the polymorphic quality model. Regarding their subcharacteristics, we apply the same rule and therefore include only sub-characteristics that are have an importance value of at least very important (i.e., +1).

Furthermore, for characteristics where the importance is less than +1, we look for the influence of their sub-characteristics. Indeed, if a characteristic is rejected during first step but has extremely important sub-characteristics, then the characteristics should be finally included. To do so, we compute a new characteristic importance value by considering for a half the current importance values of the characteristic and combined with the importance values of its sub-characteristics s (i.e., complete influence of the sub-characteristics like Gordieiev et al. [START_REF] Gordieiev | Evolution of Software Quality Models in Context of the Standard ISO 25010[END_REF]), and the second half the average value of its sub-characteristics importance value that equal or greater than +1. Once a previous rejected characteristic is finally included, we apply the include its quality sub-characteristics with an importance value of at least +1. In the case where we have more than two levels of characteristics, we replicate this mechanism to the different level of quality model hierarchy.

Calculation explanation and example are provided in Chapter VII.5.a. During this construction, we take care to follow the Miller's law and the tree derivation rules whenever they apply, to facilitate the maintainability of quality model. In parallel, the polymorphic quality model construction prevents the evolution and general operational issues.

6.

Determine the common quality model: After we model quality for several products, for our final building stage we take the mathematical intersection of these quality models to generate a common quality model. This last model will then be used as the basis for other products for which we don't already have a quality model. It can be also used for the next generation of the same products and also as a basis for building a new architecture: quality requirements drive systems and software feature requirements, and then the corresponding architecture requirements.

To conclude on this contribution of the 6-stages process for building quality models, even if this process can be used independently to the "Quality Thermometer" process, the two processes address two complementary operational aspect: development and use of quality model.

In addition, the stages of this process include practical solutions to prevent the evolution, general, maintainable, stakeholder and terminology operational issues. We notice that the elaboration of such process contributes also to reach at least a maturity level 2 (i.e., managed) for quality modeling development.

Finally, rather than a creation of a new quality model that may be optimum only for a particular project and product, we based our quality model development strategy on the reuse of existing, and valuable, quality models.

"Quality thermometer" and "6-stages" process comparison against current ISO/IEC standards

In the previous section of this chapter, we described two processes which addresses the development and the use of quality models. During these process elaborations, we referred to and embedded a certain number of operationalization concepts coming from standards (e.g., ISO 9001 [233], ISO/IEC 25022 [145]) but without using any of them as the backbone of our processes. And yet, we found that most of the documents in the ISO 250nn series already address various aspects of operationalizing the quality models contained in ISO/IEC 2501n (i.e., ) ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], ISO/IEC TS 25011 [START_REF]-Information technology -Systems and software Quality Requirements and Evaluation (SQuaRE) -Service quality models[END_REF] and ISO/IEC 25012 [159]), particularly on the quality requirements and model evaluation. Similarly, ISO 15939 addresses such aspects but from a measurement process perspective.

Consequently, the objective of this section is to perform a comparison to verify the relevance of the two proposed processes against the current relevant standards which are  ISO/IEC 250nn series [25], the system and software quality requirement and evaluation standards, that define an evaluation process using software quality models and measurements,  ISO/IEC/IEEE 15939 [119], the measurement process standard.

The result of the comparison analysis is summarized through Table 22 and Table 23.

First, we note that in the "Quality Thermometer" and "6-Stages" process, any quality models can be used as entry model while for ISO/IEC 250nn, the aim is to use the standard quality models defined in ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF] (i.e., systems and software product quality and quality in use models), ISO/IEC TS 25011 [START_REF]-Information technology -Systems and software Quality Requirements and Evaluation (SQuaRE) -Service quality models[END_REF] (i.e., service quality models) and ISO/IEC 25012 [159] (i.e., data quality model).

Furthermore, quality model customization is natively included in our proposed approach which is not the case with the ISO/IEC 250nn, where deletion or addition of quality characteristics is not advised. This customization aspect is enabled thanks to the built-in evolution mechanism of polymorphism for product and life cycle. About the standards, that evolution aspect is lighter. Indeed, there are only 3 levels (i.e., internal, external and in use) of product quality life-cycle phase with a measurement focus of ISO/IEC 25020 [123] (i.e., the measurement reference model and guide) and the optional life cycle applicability criteria of ISO/IEC/IEEE 1593.

Regarding relationships, aggregations, and weight factors, again, they are both embedded and refined into the two proposed processes, especially their determinations are included in the "6-Stages" process. On their side, the studied standards assume that aggregation should be achieved through measurements functions and therefore, they do not address or cover them. This is the same case with weight factors where they are considered optional and manually defined without any further detail. In consequence, the result is subjective for the standard approach while we succeeded to setup an objective one in our processes.

Concerning reporting, both ISO/IEC 25040 [147] (i.e., evaluation process) and ISO/IEC/IEEE 15939 suggest some informative evaluation guidelines which is less effective and consistent than the scorecard based solution of the Quality Thermometer". But since the "Quality Thermometer" is close to ISO/IEC 25040, this process is therefore compatible with both ISO/IEC 250nn and ISO/IEC/IEEE 15939, and then it can benefit from them, and vice-versa.

However, even if we found several absent or limited concepts in the current standard compared to our two proposed processes, we remarked that we missed the traceability with requirement which is highlighted in ISO/IEC 25030 [247] (i.e., software product quality requirements). So, we will have to take into account and integrate this traceability concept not only in the "Quality Thermometer" process but as well in the "6-Stages" process. To conclude, this comparison confirms that, even if both ISO/IEC 250nn series and ISO/IEC/IEEE 15939 standards already cover some operationalization aspects for quality model and evaluation, the "Quality Thermometer" and the "6-stages" process are relevant and bring additional values by filling gaps in the existing standards.

Threats to validity and discussions

The main motivation behind the study described in this chapter is about how to develop and use quality model from a practical point of view. We observed that over the eight main approaches to support quality model development and use (see Chapter IV.5 and Table 10), qualimetry is the most appropriate approach to proceed for quality modeling. Moreover, as we highlighted previously (e.g., via the "House of Qualimetry", Figure 20), qualimetry covers both theoretical and applied aspect. Unfortunately, this applied aspect focuses more on applying qualimetry to evaluate type of objects that were not evaluated before avoiding some pitfalls in qualimetric analysis or quality evaluations [START_REF] Azgaldov | Applied qualimetry: its origins, errors and misconceptions[END_REF], [START_REF] Azgaldov | The Theory and Practice of Product Quality Assessment. Essentials of Qualimetry[END_REF], rather than on challenges or issues that may prevent applying qualimetry, or quality modeling.

We found only few contributions about the quality model operational issues in the literature likes Thapar et al. [11], Ahrens et al. [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF], Abran et al. [96], or Khoshgoftaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF]. Nevertheless, Thapar et al. study was the most exhaustive exploration on this problematic, identifying a total of 9 issues that prevent quality model development or use. In their study, the authors describe each issue and gave an idea of solution about it. However, these solution hints were not practical solutions and the main authors' focus was to identify the best quality model over a list of 24 models using number of issues per model as comparison criteria.

Our survey explored further that problematic, identifying a total of 16 issues and consolidating for the first time not only findings from previous contributions but also proposing three new ones. On the one hand, we succeeded to find for each of these 16 issues some real practical solutions, based either on previous research and industrial works, or our personal experience. On the other hand, we didn't proceed on an additional analysis about the frequency, the importance, or the relevance of the issues to each other.

In Chapter IV.6.d, we defined a measurement process (cf. Figure 28) aligned to other measurement processes such as McGarry et al. [START_REF] Mcgarry | Practical Software Measurement: Objective Information for Decision Makers[END_REF] and ISO/IEC 25040 [START_REF] Dujmovic | A Quantitative Method for Software Evaluation and its Application in Evaluating Windowed Environments[END_REF]. That process is designed more for expert audience. With the "Quality Thermometer" process, we do not redefine another measurement process; in fact, this new process is a practical stepped approach for operational (i.e., development and use) quality modeling. This process is designed for a wider audience, especially to get buy-in from stakeholders. Thus, both processes are complementary and can be aligned together: "initial phase" with "stage 1", "plan phase" with "stage 2, 3 and 4", and "execution phase" with "stage 5, 6 and 7". Moreover, the "Quality Thermometer" process is generic enough to be applied to "take the quality temperature" of project, products, process, resource, supplier, organization… Finally, we choose this name and graphical representation split into seven stages (i.e., applying Miller's law) to ensure that people keep in mind this key process.

Regarding the 6-stages process for quality development model, we decided to foster the reuse of previous quality models rather than creating new one from scratch because we do not want to discard any valuable existing contributions, give to stakeholders the perception that we reinvent the wheel for the umpteenth time, and many quality models get their inspiration from other previous models (i.e., as we are proposing to do here). Then, the construction, or model customization, starts from the selection of quality attributes that are the most relevant according to the opinion of the interviewed stakeholders, present or not in the proposed quality model. The selection of existing quality attributes joint with their weight factors form a set of polymorphic quality models that derived from a mother quality model, itself derived from the chosen quality model in stage 1. The survey together with the selection of quality attributes is not innovative, however the automatic construction of their corresponding weight factors and creation of polymorphic quality models with their mother model are innovative contributions.

Finally, although we have not mentioned quality perception in the development and use of quality model. This concept is implicitly managed through objective quality evaluation and through quality modeling evolution managed with polymorphism (i.e., evolution of the quality perception over time).

Chapter VII. Put into Practice

Introduction

Through the previous chapters we cleared our thesis subject "Study of Qualimetry essential to embedded software development" from a general perspective of theory and practice. We studied current contributions on quality modeling applied to embedded software, acknowledging that none of them was answering properly to our needs (see Chapter II). Then we reviewed theory about quality, quality modeling, qualimetry, confirming that qualimetry is the right approach to our problematic before contributing to its consolidation (cf. Chapter IV). Next, we explored the broad set of existing software quality models (cf. Chapter V) and finally investigated how to go from theory to practical operation of quality modeling, the result being the proposal of two processes for development and use of quality models (see Chapter VI).

Since now we have all the key findings to practically perform quality modeling, we are ready to exercise them to answer to our company needs, "applying quality modeling to embedded software", which are also summarized in our research sub-question:

Research Sub-question 3d

What is the practical answer to our needs on automotive embedded software case?

Consequently, this chapter is about a demonstration that our approach, detailed in the previous chapters, successfully answers the company needs on automotive embedded software. Thus, we select four distinct and representative software elements from the automotive real-world (i.e., three ECU embedded software and one transverse software function) against which we apply quality modeling to develop the proper quality models for these software elements.

In the following sections, we begin by the analysis of our automobile real-world use case. Then we apply the 6stages process to construct the embedded software polymorphic quality models, calculate their weight factors and connect their quality sub-characteristics to a proper set of metrics. Lastly, we generate the quality model that is common to these polymorphic quality models.

Quality modeling on a real-world use case: embedded software for the automotive industry

From a systems engineering point of view, a car is a complex system [13], itself part of a system of systems (e.g., multiple connected cars within a road system and its infrastructures). Furthermore, a vehicle results from a massproduced of a vehicle platform instantiation. Consequently, a vehicle platform is one designed and developed generic complex system from which a set of vehicle variants (e.g., mini-compacts, crossovers, super cars, vans, utility vehicles, convertibles, etc.), with a set of available options (e.g., color, enhanced multimedia features, leather seats) are derived.

This complex system is composed of more than 40 systems, themselves decomposed in several sub-systems (ECU). To give an idea, a vehicle accounts for more than 60 ECUs combining hardware and embedded software. Therewith, it is important to note that each ECU has:

a set of common characteristics with other ECUs: e.g., it performs diagnostic functions and has a connection interface, power, limited resources, -a set of specific characteristics: e.g., human-machine interface, communication and safety features, responsiveness, -a specific context: e.g., door control, engine control, telematics, seat control, air conditioning.

As a matter of fact, we note that each ECU has a design specification, architecture, terminology, and quality quantification which vary more or less with regards to the other sub-systems. This complexity therefore impacts the quality modeling of our real-world use case, a selected subset of embedded software a vehicle-platform.

We chose to limit our real-world use case to a subset of four high priority embedded software projects: three ECUs (In-Vehicle Infotainment (IVI) ECU, In-Vehicle Communication (IVC) ECU and Advanced Driver-Assistance Systems (ADAS) ECU) and one ECU-transverse embedded software (Firmware Over The Air (FOTA)).

Their primary description and characteristics are the following:

-IVI: This ECU is responsible for the infotainment (e.g., radio, audio-Bluetooth, CD, DVD), navigation system and is the principal human-machine interface to control the vehicle (e.g., air conditioning, ambient light control, smartphone pairing, rear view camera). Thus, the performance efficiency is not as important as quality in use aspect which must include efficiency, effectiveness, and satisfaction (e.g., quality characteristics such as pleasure or comfort). Indeed, since human-machine interface is key for this ECU, the right performance criteria must be relying on the user perception of the performance rather than pure processing time for instance. This ECU is part of the vehicle infotainment domain and mainly rely on a class D network (speeds over 1 Mbit/s) because of the high volume of video data transmitted from rear-camera or DVD, for instance. We note that network classification into Its real-time functionalities (for instance, rear view camera for park assistance, lane departure detection, night vision assistance for pedestrian recognition) require processing high volumes of data, and consequently its processing performance and network bandwidth needs are high. In addition to the performance quality characteristics, a particular attention is required on security and safety, included into freedom from risk quality characteristic, because ADAS interacts with many critical vehicle systems (e.g., engine, braking system, steering) and a failure at its level can cause serious accident with injury. Like IVC, there is no direct interaction with the user (i.e., there is no human-machine interface in the ECU).

-FOTA: FOTA is a transverse embedded software, responsible for updating software and firmware. It may also be used to change a car configuration or enable / disable remotely some functionalities. This is for instance the case with Tesla which disabled remotely the autopilot feature on used Model S car, because the company considered that the new car owners didn't pay for that feature [START_REF] Statt | Tesla remotely disables Autopilot on used Model S after it was sold[END_REF]. However, this OTA technology is not yet widely used in automotive domain, but it is progressively adopted by car manufacturers since 2012 (see Figure 62). FOTA is a strong tool to reduce or prevent the car obsolescence like what is done with evolving and corrective software maintenance on smartphone, tablet, and computer (e.g., the regular operating system updates of Microsoft Windows, Linux distributions and Mac OSX) since several decades. The corrective maintenance focus on a continuous quality increase, with performance increase, security, and bug fixes. The evolving maintenance addresses the first Lehman's law [START_REF] Lehman | Programs, Life Cycles, and Laws of Software Evolution[END_REF], "Continuing Change": "an Etype 15 system must be continually adapted or it becomes progressively less satisfactory". Hence, FOTA provide the support to adapt software solution to customer needs, and to push innovation or new content (e.g., navigation maps) towards vehicles. FOTA working principle is depicted by the example of So, thanks to these four overviews, we saw that the embedded software of the three ECUs and the FOTA transverse feature have their own specificities but as well a set of commonalities and links which can be retrieved through Figure 63.

In addition, in the automotive domain, software engineering must comply with regulations and standards such as A-SPICE [START_REF]13 / Automotive SIG[END_REF]. A-SPICE focuses on software development and management processes and improvement and assessment of the process capability level. A-SPICE is neither a product quality assessment process nor a product quality control process, but its guidelines recommend using the ISO/IEC 25010 [22] 16 software product quality model (i.e., product view) and quality-in-use model (i.e., user view). Some associated metrics samples are available from ISO/IEC 25022 [START_REF] He | Software reliability analysis on embedded system based on SFMEA and SFTA model[END_REF] for quality-in-use and ISO/IEC 25023 [START_REF] Horgan | An essential views model for software quality assurance[END_REF] for software product quality. Thus, we must include this recommendation in the quality modeling of our real-world use case.

Initiating quality model construction via the 6-stages process

If we refer to Chapter VI.4.b and the 6-stages process, to initiate the construction of a quality model for the realworld use case that is introduced in previous Chapter VII.2, the first stage is the identification or selection of a reference model.

Because of our industrial automotive context, and as we saw in above, A-SPICE guidelines must be followed for the development of all ECU embedded software. This means that all software may have one unique and identical quality model, ISO/IEC/IEEE 25010. Nevertheless, even though the ISO/IEC/IEEE 25010 quality model is a general quality model, it does not take into consideration the specificities of each ECU: i.e., a specific context, but also common and specific characteristics. This standard quality model must therefore be customized for each ECU in order to be used properly. Note that the customization behavior has also been confirmed by a study on the use of quality models done by Wagner et al. in 2009 [98] and finalized in 2012 [83]. Indeed, this study emphasized that 28% of the companies interviewed use quality models taken from standards (e.g., CMM, ISO/IEC 9126) and that 79% of these 28% customize these standard quality models.

So, we assume that our original model comes from ISO/IEC/IEEE 25010 (cf. Figure 64 and Annex 8). This model covers both product and user quality perspectives and conforms to A-SPICE. Indeed, the product quality perspective is defined by the system / software product quality model.

This model is structured into 8 quality characteristics themselves refined into 31 quality sub-characteristics. One of these characteristics, the "functional suitability", is dedicated to the functional quality requirements, while the other seven are for non-functional quality requirements (i.e., performance efficiency, compatibility, usability, reliability, security, maintainability, and portability).

The measures associated to this quality model are both internal and external measures. The internal measures are the measures done directly on the software without any need to run the software. These are for example, static code analysis, including McCabe cyclomatic complexity of model and code [START_REF] Mccabe | A Complexity Measure[END_REF], functional requirement implementation coverage. The external measures correspond to the measures performed dynamically against the running software, and therefore requires a proper execution environment: built and lined software deployed on hardware with an operating system and all necessary data for its execution. As to user quality perspective, it is covered by the quality in use model with a set of five quality characteristics and 9 sub-characteristics. Furthermore, the associated quality in use measures are performed specific contexts of use and therefore require the complete system, simulated or not, as prerequisite of their realization. The Figure 65 summarizes the relationship between the quality models, the quality properties, the measures and an automotive sample. We note the influence and dependency links between the different quality properties, including the process quality properties coming from the software development process, for example. Regarding common and specific characteristics of ECUs, the aim is to associate each of them with some quality characteristics from the original quality model. For example, "safety" can be associated with "freedom from risk" and "security", "human-machine interface" with "satisfaction" and "usability". In addition, we model them by applying the ad hoc and universal polymorphism mechanism to the quality model (refer to Chapter IV.6.b).

The second and third steps of our 6-stages process are the construction and deployment of a survey. We remind that the survey purpose is to involve the main experts and stakeholders in the quality modeling. So, we chose to limit our survey to a subset of four high priority projects (i.e., corresponding to our real-world use case): IVI, IVC, ADAS and FOTA. Our contributor audience was made up of our main stakeholders: project managers, architects, verification and validation leaders, and assurance quality engineers who are involved in those embedded software projects.

As we described previously in Chapter VI.4.b, the survey is organized into four parts and uses Likert scales [START_REF] Likert | A Technique for the Measurement of Attitudes[END_REF] for the enumeration of the possible answers (see Annex 9 for a copy of the survey). In the first part, we remind to the participant the objective of the survey usage which is to be able generate right quality model, and aligned A-SPICE, to qualify SW product quality of a variety of vehicle projects. We also ask to select their role and project in which they are involved (cf. Figure 66). In the case of multiple role or project involvement, we asked to the participant to answer several times the questionnaire.

Figure 66 -Survey extract: participant role and project

The next part of the survey consists in the evaluation by order of importance each of the 13 ISO/IEC 25010 quality characteristics. Thus, for each of these quality characteristics, we ask to the participants to rank with a five points Likert's scale (see Figure 58 and Figure 67) the importance of the characteristics thanks to their technical knowledge and own vision of the ECU and project quality. Note to avoid confusion or interpretation, the quality characteristic definitions are documented in a wiki page and accessible to any participants. The survey construction doesn't require any advanced functionality from a tool form. Therefore, we decided to build the survey with Microsoft Form because this online tool is freely usable and accessible in our company, it is easy to use, provide short report on survey responses and allows to export the survey responses in a Microsoft Excel file. After the survey implementation in Microsoft Form, we invited a total of 25 participants to contribute to the survey.

Survey result analysis

Once the survey had been answered and its data collected (cf. Annex 10 for the survey responses), we began analyzing the data, our fourth building stage, by noticing a total of 48% participation (i.e., we collected 12 responses out of 25 invitations) and an average response time of 16 minutes and 23 seconds, with a rather scattered standard deviation of 7 minutes and 26 seconds (if we exclude the exception from one stakeholder who spent 3 hours and 22 minutes to complete the survey; after a quick inquiry, we figured out that that stakeholder was interrupted during the survey and left the survey session open). So, since the survey was made of 16 questions, we can conclude that it took about 1 minute to answer to a question and the standard deviation taught us that for around half of the participant they were familiar to these quality characteristics.

Regarding the ranking responses, both quality characteristics and sub-characteristics were homogenous (cf. Figure 70) with a pick, respectively at 46.15% and 38.49%, of quality characteristics and sub-characteristics that were judged as extremely important. Almost none of them were considered as not important at all, and interestingly 12.30% of the sub-characteristics were no tanked by participant. We also remarked that no additional quality characteristic or sub-characteristic have been suggested via the survey open question. the quality in use model on IVC for any project managers. We noticed that only 2 raters provide answers for IVC. Thus, we have to use Cohen's kappa ⲕ to determine the level of consensus regarding these quality characteristics. The response counts are given in TABLE 24, however since we are using Cohen's kappa, we must rather extract and use the ranking response matches between all possible response combinations. These response match data are given in TABLE 25. Note the green cells highlighting perfect match. We remind that the Cohen's kappa ⲕ equation is 𝜅 = . So, the calculations are then:

𝒑 𝒆 = 1 𝑁 𝑛 𝑛 = 1 5 (0 * 0 + 0 * 0 + 1 * 3 + 1 * 0 + 3 * 2) = 1
0 0 0 0 0 Somewhat Important 0 0 0 0 0 Important 0 0 1 1 1 Very Important 0 0 0 0 0 Extremely Important 0 0 0 0 2 - Fleiss' kappa ⲕ example:
The sample set for this example is obtained for the quality characteristics of the system/software product quality model on IVC for any roles. The response counts are given in TABLE 26. Moreover, since we consider any roles, we have 4 raters (i.e., 𝑛 = 4) for IVC, and thus we can use Fleiss' kappa ⲕ to determine the level of consensus regarding these quality characteristics (i.e., subjects; 𝑁 = 8). The levels of importance used in their ranking correspond to the categories are (i.e., 𝑘 = 5). We remind that the Fleiss' kappa ⲕ equation is 𝜅 = . So, the calculations are then: 61, for each project, we checked whether Fleiss ⲕ was available for current quality perspective and all roles. If it was at least greater than 0.4 (i.e., Moderate). We thus selected the data set based on the highest Fleiss ⲕ for each project, or ECU, and all roles, or where a role had the highest ⲕ. When it was not sufficient, we used Cohen ⲕ as a decision criterion. If no kappa showed at least a moderate agreement, then we looked for consensus at the specific role level, all quality perspectives and all projects, or ECUs. For instance, for IVI ECU, project managers had the highest ⲕ. For ADAS ECU, we took assurance quality engineers for all projects, or ECUs. In case where no consensus can be found at project or role level (i.e., no kappa greater or equal to 0.4 for any roles for the project and then for any roles when considering all projects together), we conduct a new survey. The complete results of our kappa-based data analysis are summarized in Table 27.

𝑷 𝒆 = 𝑝 = 1 8 × 4 (0) + 1 8 × 4 (1) + 1 8 × 4 (2 + 3 + 3 + 1) + 1 8 × 4 (1 + 3 + 2 + 2 + 1 + 3) + 1 8 × 4 (1 + 3 + 2 + 2 + 1 + 3) = 1 + 9 +
The result for sub-characteristics shown here excludes answers such as "I don't know" simply because the overall ⲕ values with and without this sixth rating were respectively 0.058276 and 0.120398: including these answers reduces the agreement between raters. In addition, we noted that no architects and only one validation leader contributed to our survey. This explains the cells where there is no ⲕ value. Furthermore, ⲕ values, 0.122846 and 0.120398, obtained for all projects and roles reveal that there is a slight agreement over all projects and between all parties. Consequently, the quality view is different for each combination of project and roles, hence the necessity to create a set of polymorphic quality models.

Furthermore, on both quality characteristics and sub-characteristics, Assurance Quality engineers were quite well aligned over all projects, with a Fleiss ⲕ on characteristics of 0.444330 (i.e., Moderate agreement) and on subcharacteristics of 0.577705 (i.e., Moderate agreement, but close to substantial agreement), while project managers were aligned per project only: their Fleiss ⲕ is from to 0.328165 (i.e., Fair agreement) to 0.433414 (i.e., Moderate agreement).

Contributions

Based on the survey data results and our analysis with Cohen and Fleiss ⲕ, we are able to finalize the elaboration of the 3 polymorphic quality models, their weight factors, the metrics and the quality model which common (i.e., which is inherited by the 3 polymorphic quality models). These contributions are the evidences to confirm that the approach that we have defined in the previous chapters is practically working and giving us right solution.

a. Importance values and weight factors

This is the first step of the construction because we are proceeding on some computation to determine what characteristics or sub-characteristics is kept based on the analysis of the survey results. So, these computations give us automatically the importance of each of these characteristics / sub-characteristics and by consequence, they represent their weight factors once normalized.

For example, we determined for IVC ECU that we have rater consensus on quality characteristics between all raters. The corresponding answers are therefore extracted from the survey for IVC ECU: We recall, for instance, that for the "Usability" characteristic the current importance value is 0.25, 0.38889 is the "Usability" sub-characteristics mean importance value and two of these sub-characteristics are very important (i.e., +1 value in the 5-point Likert' scale). Thereby, we found that only "Portability" characteristic received enough influence from its sub-characteristics to be finally included. To summarize all the IVI ECU quality characteristics with their final importance values and inclusion / exclusion decision. Note, the included quality characteristics are the ones defining the IVI ECU quality model, and since their total number is seven, we respect the Miller's law [START_REF] Miller | The magical number seven, plus or minus two: some limits on our capacity for processing information[END_REF]. To continue, the quality characteristic and sub-characteristic importance values are not only allowing us to determine which of the quality characteristics and sub-characteristics must be taken into account to compose our quality model, but these values provide implicitly the weight factors of the quality characteristics and subcharacteristics. Indeed, by definition these values reflect the importance of a characteristics among the others.

IVI

Quality perspective

We have the same behavior with the importance values and the sub-characteristics. Therefore, to obtain the corresponding weight factors, considering for example a sum aggregator operator, the remaining step is simply to norm the importance values. So, we first determine the total sum ∑ of the included characteristic importance value, per quality perspective, and lastly divide each of these importance values by ∑ . The sum of the resulting weight factors gives 1.

Continuing with our example, we have: Then, even if we considered four sub-systems in our realworld use case from automotive, we noticed from the survey result analysis that we come up with three distinct groups of results. These groups are unique for IVI, IVC but common for ADAS and FOTA. Furthermore, to build the corresponding three quality models, we use the existing hierarchical links between quality perspectives (i.e., "system / software product quality" and "quality in use"), quality characteristics, and quality sub-characteristics. These links can be retrieved directly from the reference quality model selected in the first stage of the 6-stages process; in our current example, this is the ISO/IEC/IEEE 25010 quality model. Regarding the IVC embedded software quality model, Figure 72 depicts this hierarchical quality model with its weight factors. This model is structured over two quality perspectives, seven quality characteristics and 12 quality sub-characteristics. Thus, this model covers 53.85% of the quality characteristics of ISO/IEC/IEEE 25010 and only 28.57% of its quality sub-characteristics. We remark that functional suitability, compatibility reliability and effectiveness are the most key quality characteristics for the IVC ECU. Finally, with these three quality models, we see evidences that each embedded software has specific quality characteristics, and thus quality requirements, that must be took into account during development. Moreover, even if we note a certain level of similarity coverage between these quality models with regards to their quality characteristic and sub-characteristic, their instantiations through weight factors clearly differ, showing that the quality evaluation with these models is objective. Concerning ISO/IEC/IEEE 25010 standard, the three achieved quality models also demonstrate the needs of tailoring standard quality models because with these model adaptations, almost 60%, and down to 29%, of the standard quality sub-characteristics are envisaged for our realworld use case quality models. At last, each of the quality model sub-characteristics are completed by a proper basic set of measures borrowed from ISO/IEC 25023 [START_REF] Florenskii | Some Remarks on Product Quality Assessment[END_REF] (i.e., measures of system / software product quality), and ISO/IEC 25022 [START_REF] Chrisman | Rethinking levels of measurement for cartography[END_REF] (i.e., measures of quality in use) to strengthen the operationalization of these three quality models. This basic set of measures is described in Annex 12.

c. The polymorphic quality models

We reach our final building stage: the emphasis of polymorphism behavior with the construction of a common quality model from which the other quality models of our real-world use case are finally derived.

To proceed, we take the mathematical intersection of these constructed quality models. This mathematical operation gives a model with all the quality characteristics and sub-characteristics present in our constructed quality models. The result is shown in Figure 74. This common model is made of six quality characteristics and 9 quality sub-characteristics. Moreover, this model has no weight factors since it can be considered as the common "ancestor" of the quality models of the embedded software from our real-world use case, and not one specific quality model instantiation.

We remark also that it is possible to create a secondary common quality model, which derived from that common quality model and be the common "ancestor" of IVI, and ADAS & FOTA quality models. The construction is therefore similar to the common quality model but the intersection is reduced to only to these subset of quality models (cf. Figure 75). Behind these "ancestor" relationships is the concept of polymorphism (refer to Chapter IV.6.b and Figure 23). The Figure 76 describes the polymorphism tree structure with the polymorphism five quality models: two common quality models and three embedded software quality models. To conclude on this section, these two common quality models and the polymorphism tree structure representation showcased the interest and how polymorphism is concretely applied.

They can serve as the basis for other ECU embedded software for which we don't already have a quality model. They can be also used for the next generation of the same ECUs and as a basis for building a new architecture: quality requirements drive systems and software feature requirements, and then the corresponding architecture requirements.

Threats to validity and discussions

The achievements covered by this chapter successfully demonstrate the validity of the quality modeling operationalization we have built and detail in previous Chapter VI. Moreover, by constructing quality models against embedded software real-world use case, not only we answered to the company needs, but also, we concretely proved what polymorphic quality model concept is not a theoretical concept and is an efficient quality modeling tool. Nevertheless, we notice several questionable and perfectible contribution elements.

First, our methodology relies on the reuse of quality models rather than creating a new model from scratch. Indeed, by following the reuse approach, we benefit of more than 50 years of valuable contributions in software quality models, including quality model analysis and consolidation studies that tend to mature some quality models. This is the case, for example, of ISO/IEC/IEEE 25010 which is at the center of attention of many surveys (e.g., Gordieiev et al. studies on quality model evolutions compare to ISO/IEC/IEEE 25010 [START_REF] Gordieiev | Evolution of Software Quality Models in Context of the Standard ISO 25010[END_REF], [START_REF] Gordieiev | IT-oriented software quality models and evolution of the prevailing characteristics[END_REF]). In addition, the selection of the reference model is certainly perfectible. We didn't define a specific process for this selection but instead, we consider our real-world use case constraints from automotive industry to be at the state of the art when developing a vehicle. So, we relied on A-SPICE and ISO/IEC/IEEE 25010, because we didn't find any specific and unique quality model for automotive in our exploratory and systematic literature review (see chapters Chapter II.2 and 0.2).

Another element which requires some further attention is the number of survey participants which are missing for some categories. For example, none of the invited software architects and only one validation leader replied to the survey. The main drawback is that it can weaken our results and certainly the quality model buy-in of software architects. However, we got strong support from project managers and assurance quality engineers, and as we indicated previously, these results constitute a starting point and the quality models must continue to evolve along to the projects and the platform vehicles. So, the next evolutions are obvious: we must first involve and get contributions from software architect and validation leaders, and then perform the quality modeling over to entire vehicle platform.

Regarding the calculated Kappa, a particular attention needs to be payed to Kappa interpretation because it appears that there is no universal agreement on the meaning of the table from Landis and Koch [START_REF] Landis | The Measurement of Observer Agreement for Categorical Data[END_REF]. This is mainly due to the fact that Kappa interpretation is subjective and depend on the number of categories for instance. In addition, to explore further our Kappa based analysis, we may calculate Kappa values for each characteristics and sub-characteristic rather than calculating one Kappa value for all characteristics and one for all sub-characteristics per quality perspective. The idea behind this refinement is to look for finer groups of characteristics or sub-characteristics where consensus between raters exists.

The basic set of metrics, given in Annex 12, should be consolidated, or refactored to take into account the project milestone or development life cycle likes in FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] and FURPS+ [START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF], and therefore the temporal polymorphism applied to measures (see Chapter IV.6.b). Here our goal was to demonstrate the practicability of the solution. Moreover, concerning the thresholds, beside some internal company ones (e.g., no critical bug is allowed to pass a milestone, or a certain number of volatile and non-volatile memory footprint is allowed), we adopt the same approach than Ahrens et al. [START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF] where the aims is to do better for each subsequent releases: continue to improve through relative thresholds.

In conclusion, this chapter emphasizes the benefits to develop operational quality models, as well as the practicality of the solution against real-world use cases. Our approach allows to homogenize quality models and vocabulary over company engineering domains (e.g., systems, software, electrical, mechanical), keeping the relationship between the vehicle platform elements, and between the projects together. It is also a unique and simple way to determine weight factors jointly with quality model. Indeed, this is not only a matter of conducting and using a survey, but rather benefiting of the survey results to determine which characteristics and subcharacteristics are the most meaningful and contributing for the project under consideration. Finally, our approach is a breakthrough by reinstituting quality modeling activity as the backbone of quality evaluation, particularly compare to the current company solution, or even with usual Agile or Rapid development methodology, that relies on measurement based on tools, quality assessment via a checklist and customer issue reports or feedback.

Chapter VIII. Meta-Model: Software Quality Model Genome

Introduction

In Chapter V, the systematic literature review on software quality models and their classification allowed us to answer to the research sub-question 4a "Is it possible to have a unique reference quality model for software product, or instead should we have a meta-model?". The conclusion was that there is no unique reference quality model for software product. Indeed, we retrieved a huge number of software product quality models either for quality definition, assessment, or prediction, and each of these quality models has a good reason to exist and to be different from the other models.

Moreover, the current standard ISO/IEC/IEEE 25010 is defined as the reference of system and software product quality model, but as the Wagner et al.' survey [83] pointed out, the standard quality models are usually customized due to the particularity of the products, projects or company requirements. Thus, because of this customization necessity, we cannot conclude that even the current standard can be considered as the unique software product quality model.

Consequently, the purpose of this chapter is to explore the alternate case of a meta-model as the preferable solution instead of a unique reference quality model for software product.

As we did when we proposed polymorphism concept for quality model, we continue to take benefit of genetic knowledge for the creation of a meta-model. We choose genetic since we notice an analogy between quality model variations of the same quality concepts through different quality characteristics, and DNA variations between alleles of same gene. This meta-model can be then used as the beginning quality model of polymorphic quality model construction and offers a mechanism to include any quality models.

So, to elaborate such meta-model, we must respond to the research sub-question 4b:

Research Sub-question 4b What is the construction algorithm for such meta-model?

Once answered, we are ready to execute this construction algorithm and initiate the meta-model construction, delivering finally the first metal model results. This step is summarized over:

Research Sub-question 4c What is the first result of the meta-model construction?

Motivation and analogy with genetic

Our main motivation behind the elaboration of a meta-model is to avoid the comparison and selection of a quality model among a set of candidate quality model, resulting on discarding many potential valuable contributions for the benefit of one. Instead, we prefer to privilege union of quality models to build a quality meta-model, and then get benefit of all research works rather than selecting a subset of them. Our credo is "all research, academic and industry work, should count".

During our exploratory literature mapping and systematic literature review, we noticed that frequently same quality concepts were defined more or less similarly between distinct contributions. The nuances were mainly on the wording, definition, interpretation, or on the sub-elements (e.g., distinct set of sub-characteristics for same characteristic in different quality models). For instance, in Boehm's quality model [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF], maintainability is refined into modifiability, testability and understandability. In ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF], it is defined by analyzability, changeability, maintainability compliance, stability, testability. And in ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], it corresponds to the following sub-characteristics: analyzability, modifiability, modularity, reusability, testability. We remark that over these three variations of the maintainability characteristic, only testability is common to these three models. We note also that modifiability is also present in Boehm's model and ISO/IEC/IEEE 25010, and both ISO/9126 and ISO/IEC/IEEE 25010 share analyzability sub-characteristics.

So, our objectives are to identify not only convergences but also exceptions of quality perspectives, characteristics, and sub-characteristics (e.g., same concept but different wording or set of sub-characteristics) and represent these variations with statistics (e.g., probability to have a specific variation). For instance, in the previous case with ISO/IEC 9126, ISO/IEC/IEEE 25010 and Boehm's quality models, the probability to find testability sub-characteristic for maintainability is 100%, for modifiability two third (i.e., 66.67%) and for analyzability also two third (i.e., 66.67%). This type of variations jointly with likelihood consideration is frequent in genetic. We can cite for example Nei and Li's formula [START_REF] Nei | Mathematical model for studying genetic variation in terms of restriction endonucleases[END_REF] (see Chapter IV.6.c) to evaluate the degree of polymorphism or variety between several alleles, the non-parametric linkage technic [START_REF] Haseman | The investigation of linkage between a quantitative trait and a marker locus[END_REF] used for genetic disease identification.

Therefore, our observations make us propose the Genetic-Quality analogy, and more precisely the analogy between DNA sequences -sequence of nucleotides-, which encode genetic characters through protein information, with sequences of quality characteristics and sub-characteristics, which implicitly encode quality characters and requirements.

Thus, since in genetic a chromosome is composed of two identical copies of DNA sequences called chromatid, by applying this analogy, a quality perspective can be assimilated to a chromatid while two copies of this quality perspective correspond to a chromosome (see Figure 77). To go further, in genetic too, a gene is a subset of a DNA sequence "controlling the development of particular characteristics" [256] and starts at a specific location called locus. This DNA sequence subset is unitary composed of nucleotides located at specific sites. Consequently, by using our analogy Genetic-Quality, we associate a gene with a quality characteristic, starting at a specific location (i.e., locus) in the sequence of quality characteristics and sub-characteristics of a quality perspective. As to the nucleotide sites, they correspond to these quality perspective characteristics and sub-characteristics. In addition, a gene (e.g., a gene responsible for the eye color) in a pair of homologue chromosomes is represented by two alleles (i.e., one for each chromosome). These alleles can be identical (e.g., same eye color), or different (e.g., different eye color), but in any case, each of the allele variations has a certain probability to exist. We have the same mechanism in quality domain, where variations of quality characteristic and sub-characteristics have their own likelihood to exist. Finally, we note that "a gene is said to be polymorphic if more than one allele occupies that gene's locus within a population"[257], retrieving the polymorphism behavior for quality model we introduced in Chapter IV.6.b. The Figure 78 illustrates these analogy details and TABLE 28 summarizes the terminology analogy.

Figure 78 -Genetic-Quality analogy: detail overview with locus, sites, genes and alleles (genetic terminology is in green, quality terminology is in purple)

Figure 79 illustrates and fosters the comprehension of the proposed Genetic-Quality analogy through an example where this analogy is applied against the current standard ISO/IEC/IEEE 25010 quality models. In this example, the two quality perspectives, software product quality and quality in use, are assimilated to two distinct chromosomes. We notice the sequence of characteristics (i.e., genes) starting at specific locus and composed of sequences of sub-characteristics themselves located at specific sites on those chromosomes. At last, we consolidate the description of this Genetic-Quality analogy proposal by detailing the associated ontology (cf. Figure 80). This ontology described the relationship with their numeration between each concept used in this analogy and will serve to structure our meta-model construction. Furthermore, we decided to reuse the quality aligned genetic terminology (see Table 28) for conciseness purpose. Thus, a chromosome can have none or many polymorphic chromosome variations and is composed of exactly two chromatids (i.e., quality perspective). Each chromatid is itself composed of a sequence of DNA (i.e., sequence of quality characteristics and their quality sub-characteristics). Moreover, and as we saw above, a DNA sequence can be decomposed into subsets of genes, starting at specific locus in that sequence. Genes and DNA sequences can be also defined as sequences of sites (i.e., quality characteristics and sub-characteristics) which can be themselves consider as other variations of sites. Finally, a gene can have none to many variants, called allele, constituted of sites and each allele is exactly one polymorphic variation of a gene.

Software quality model genome meta-model construction algorithm a. Construction methodology

Based on the Genetic-Quality analogy described in previous section, we aim to elaborate an algorithm to construct a meta-model for the whole group of genes, also known as genome in genetic, of a set of software quality models.

Our construction methodology is organized around 3 stages. We start with a set of quality models as the sources of the meta-model elements, then extract and prepare groups of quality characteristics from these models, before ending with the genome construction. The Figure 81 shows the 3 stages with the associated seven steps. The first step consists in the identification of the quality models used as source for the meta-model construction.

In fact, we plan to use their characteristics, sub-characteristics, and all sub-sequent level of characteristics, as entries for the meta-model site and gene definitions. We remark the constraint for including a quality model in that source set: to be qualified, the quality model must be clearly defined, with all clear relationship between its elements (e.g., characteristics with their sub-characteristics) and all element definitions. Once the selection is done, the next step is the quality model detail collection. Thus, all quality model structures, with their quality characteristics, quality sub-characteristics and definitions are extracted to initiate the meta-model construction.

During the third step, for each selected quality model quality characteristic groups are enumerated. We defined a quality characteristic group as a quality characteristic that owns at least two sub-characteristics. In a case where a quality characteristic has only one sub-characteristic, we named it orphan characteristic group and we consider the name of this quality characteristic to have a likelihood of 50% for the quality characteristic name and 50% for the quality sub-characteristic name. Through the next step, these quality characteristic groups are compared together in order to identify the distinct ones. Note, during the comparison only the quality characteristic name is used and not yet the sub-characteristics which may be different since they come from distinct quality models. The last step of the quality characteristic groups stage (cf. Figure 81) focuses on reducing the number of distinct quality characteristic groups by matching the similar ones, limiting potential redundancy with quality characteristics. The match criteria are based on quality characteristic group name synonyms, their definitions, and their list of sub-characteristics. For example, the "functionality" from ISO/IEC 9126 quality model [START_REF]Part1: Quality Model[END_REF] matches the "functional suitability" from ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF], the "efficiency" from Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF] and ISO/IEC 9126 quality models matches the "performance efficiency" from ISO/IEC/IEEE 25010. After this fifth, we have collected all the necessary elements to compute quality model genome (i.e., whole genes and their respective chromatids).

Consequently, over the sixth the quality characteristic genes are calculated from the reduced and distinct quality characteristics groups. So, by computing each gene we mean to identify for each quality characteristic all possible quality sub-characteristics which are linked to the characteristic, with their respective likelihood to happen. The different possible set of sub-characteristics from the quality models can be seen as the set of sites forming gene alleles (see previous Chapter VIII.2 and Table 29 for an example on "portability" quality characteristics) from each quality model source where it exists. Therefore, after identifying matching sites between alleles, for each distinct site (i.e., sub-characteristic), we take the most dominant name as site name, determine statistics about the probability to have this site (i.e., number of alleles where we find the site over the number of considered alleles) and the frequency related to each similar name we find in the site (e.g., all identical names over considered alleles give 100%, but if half of them are similar, we may have 50% or less: see result sample on "portability" quality characteristics in Table 30). Note, group characteristic likelihood is computed from the number of quality model sources which reference its quality characteristic. Over the seventh and final step in the meta-model construction, we regroup quality characteristic genes together based on their quality characteristic and sub-characteristic relationship to build chromatids. A chromatid is identified as a top-level gene, that is to say, it is not a child of another gene. So, for each identified chromatid, we take its genes, sub-genes, and descent to sites in order to combine their probability values and list all sites (cf. example depicted by Figure 82).

Moreover, to optimize the number of those sites, we try to merge identical sites together by applying the following rules:

1-if inside a gene, two sites are identical, or similar (i.e., slight variations), and are not part of any subgenes, we replace these two sites by only one site which has a likelihood equal to the sum of the probability values of these two sites (with a maximum value of 1 or 100%), 2-if inside a gene, two sites are identical, or similar (i.e., slight variations), and one of them is part of a subgene, replace the two sites by only the sub-gene; then sum their probability values (with a maximum value of 1 or 100%),

3-if inside a gene, two sites are identical, or similar (i.e., slight variations), and the two sites are part of two distinct sub-genes, replace the two sites by To apply concretely this construction methodology, we create a software quality model genome meta-model construction algorithm. It is organized around the three stages and their steps seen in the methodology (cf. Figure 81).The main construction algorithm is defined in the next three solid line boxes (i.e., one per construction stage) and should be used sequentially. Moreover, to facilitate its applicability, we also describe three functions that require a particular attention: the extraction of quality characteristic group elements, the estimation of distance between two groups of quality characteristics and the creation of a gene node graph. Each of these three functions are put in a dashed box.

Each comment in the algorithm starts by a # character, and the main variables are:

-Chromatids: list of identified chromatids -Distance_Acceptance_Threshold: Maximum acceptance threshold to accept that two characteristics are lexically and semantically closed, -

Genome_Metamodel: Software quality model genome meta-model (i.e., the result meta-model), -Grp: Groups of distinct and reduced quality characteristics used to build chromatid and software quality model genome meta-model, -Grp_Graph: graph containing all characteristics groups with their sub-characteristics (i.e., their children), -

Pool_of_QM: pool of quality model sources to merge, -Push: function to put new element on the top of the destination structure, -QM_Grp: Groups of quality characteristics with their sub-characteristics extracted from the quality model sources.

In the first stage, we assumed that quality model sources are already identified, and their data (i.e., quality characteristics, sub-characteristics, their definition, their relationships) are available in loadable structures (e.g., quality model data store in files). Thus, the data initialization is achieved by loading the data from all quality model sources into the data structure "Pool_of_QM".

With the second stage, the aim is to extract and optimize all the quality characteristic groups "QM_Grp" from the quality model sources "Pool_of_QM". As indicated in the methodology, this optimization consists in limiting the number of distinct quality characteristic groups by regrouping the similar ones. So, we crawl each quality characteristic group from the quality model sources and verify if any similar quality characteristic group exist in these current data. If this is the case, we keep only one instance of these data to avoid redundancy. However, before any removing action, we merge the quality characteristic variations (e.g., "functionality" versus "functional suitability") and their respective likelihood to happen into the kept instance.

While the first two stages were responsible to extract and prepare the data related to quality characteristic groups, the genome real meta-model construction is performed in the third and last stage. It is organized in three phases. The first one is to structure the genes resulting from the groups of distinct and reduced quality characteristics, "Grp", under a graph "Grp_Graph". The graph nodes (i.e., vertices) correspond to each quality characteristics from "Grp" and their edges are the children relationship between theses quality characteristics and their sub-characteristics which are also considered as part of the graph nodes. Next phase is to find all chromatids "Chromatids". Thus, for all nodes in the graph, we check if that node has a parent. If this node has none, it is a chromatid. And finally, for each chromatid, we build list of from descendent gene sub-graphs and add the result to our software quality model genome meta-model "Genome_Metamodel" The extraction and optimization feature are performed through the "Extract element of QM_Grp_Set to be mapped to Grp" function. This function is straightforward. Indeed, it goes through all the elements for current quality model group set QM_Grp_Set and look for which one of these elements has the closest distance to the quality characteristic groups Grp. If there is one element found, it is removed from the quality model group set QM_Grp_Set and the found element is returned. When two elements are distinct, or too far from each other, the distance function (i.e., Get distance between Ref_Grp and Tgt_Grp) gives a value of infinity. Therefore, if all elements of the quality model group set QM_Grp_Set are distinct or too far from the quality characteristic groups Grp, no result is found because the "If Tmp_Distance < Candidate_Distance" comparison will be always false (i.e., this comparison becomes "If infinity < infinity").

#
Regarding the distance function, "Get distance between Ref_Grp and Tgt_Grp", we first test if the two characteristic "words" (i.e., "Ref_Grp" and "Tgt_Grp") are identical. In the positive case, the distance is equal to 0. Otherwise, we verify if the two characteristics "Ref_Grp" and "Tgt_Grp" are closed synonym, by are verifying their lexical and semantic distance.

For this task we use both WordNet lexical database [START_REF] Miller | WordNet: A Lexical Database for English[END_REF] -available online at Princeton University 11 , and the online semantic Altas (i.e., http://www.atlas-semantiques.eu/) allowing to determine multiple levels of synonym constellations (cf. example in Figure 83 of three synonym constellations for "efficiency" word). A synonym constellation is a group of synonyms semantically close based on cliques (i.e., minimum semantic units with very fine granularity).

So, if "Tgt_Grp" is within the "Ref_Grp" synonym constellations, or in the contrary case, if the intersection of "Ref_Grp" and "Tgt_Grp" synonym constellations is not empty, then the distance result is the minimum constellation distance. Finally, to avoid too long distance, the distance result is compared to a threshold value set by user to acknowledge range of valid distance values. However, we consider that if this distance is higher to this threshold, but both "Ref_Grp" and "Tgt_Grp" definitions and sub-characteristics are matching (see Motogna et al. [94]) then we accept this distance. In all other cases, the distance is fixed to infinity to reflect the disjunction.

The "Get Graph Node for i -th element in Grp" function create a graph node, or vertex, for the i th element of the groups of distinct and reduced quality characteristics, "Grp". A graph node is made of the current "Grp" index i, and the children list, if any exist, of this i th element of "Grp". The children retrieval is achieved by crawling all elements of "Grp" and then verifying if they are one of its sub-characteristics. # Finally, assess that the distance is not too high, so require user to acknowledge (above process is automatic)

#
If Distance > Distance_Acceptance_Threshold

Software quality model genome meta-model construction a. List of quality models

As indicated in Chapter VIII.3, the preliminary assumption of the software quality model genome meta-model construction algorithm is the existence of a set of software quality models to be used for the meta-model build. Indeed, the algorithm generate a list of "quality" genes from the quality model characteristics and subcharacteristics, identify the corresponding "quality" chromatids and finally synthetize a software quality model genome meta-model resulting of these input quality models.

Moreover, these quality models must have their quality characteristics and sub-characteristics, together with their definition and relationships, clearly defined to be exploitable. Note, as Oriol et al. study [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] highlighted, definition completeness of quality models is often partial or incomplete. In their survey on quality models for web-services, they found that only 51% of them had their definitions 100% complete.

So, considering Thapar et al. [11] study conclusion with regard to the challenges and issues in software quality model development and use, we select four of the five quality models which have only three identified issues: Alvaro [START_REF] Alvaro | A Software Component Quality Model: A Preliminary Evaluation[END_REF], Bawane [START_REF] Bawane | A Novel Method for Quantitative Assessment of Software Quality[END_REF], ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF], and Kalaimagal's Q'FActo 12 [START_REF] Kalaimagal | Q'Facto 12: an improved quality model for COTS components[END_REF]. We reject GEQUAMO quality model [START_REF] Georgiadou | GEQUAMO-A Generic, Multilayered, Customisable, Software Quality Model[END_REF] from this selection since we were not able to retrieve all details (i.e., definitions and all subcharacteristics) about this quality model. Furthermore, we complete that selection list with four additional and widely used software quality models for which reference papers with quality model details, including definitions, are available. These four quality models are Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF], McCall [41], FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF],and ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF].

Thus, we identified a total of eight software quality models (i.e., Boehm We remark that these quality models total 396 quality characteristics and sub-characteristics. This corresponds to an average of 49.5 quality characteristics and sub-characteristics per quality model, with a minimum of 30 with FURPS quality model and a maximum of 86 with Kalaimagal's Q'FActo 12 quality model. Note, we don't need to necessarily have distinct quality models here. Indeed, ISO/IEC/IEEE 25010 is the evolution of ISO/IEC 9126, and consequently are not "purely" distinct, but it is valid to consider these two quality models as well in the genome construction, because the evolution is a natural variation mechanism of a genome.

b. List of genes with their variations

Before generating the list of quality characteristic genes from these eight selected software quality models, there are several intermediary steps to complete as shown in Figure 81.

Firstly, we must enumerate the groups of quality characteristics from these quality models. We qualified as group of quality characteristics, a quality characteristic that owns at least one sub-characteristic. Thus, we are able to retrieve 103 quality characteristic groups over the eight quality models, which corresponds to an average of 12.875 groups per quality model, and with the same two models for the extrema: five groups for FURPS and 25 for Kalaimagal Q'FActo 12. Moreover, a part of these 103 groups are identical and therefore can be clustered into 55 distinct quality characteristic groups.

The complete group list per quality model is described in Table 32. The two left columns summarize the name of the 55 distinct quality characteristic groups and their number of occurrences over the eight models. For example, "Efficiency" is used in six of these eight quality models (i.e., Boehm, McCall, Alvaro, ISO/IEC 9126, Bawane and Kalaimagal Q'FActo 12) while "Compatibility" is defined only in Kalaimagal Q'FActo 12 and ISO/IEC/IEEE 25010. The "Portability" group is indicated twice at this step because we retrieve this quality characteristic as quality characteristic and quality sub-characteristic in Kalaimagal Q'FActo 12 model. In the next step, this redundancy is removed, counting for only one group for that model. Finally, these 43 quality characteristic groups can be transformed into quality characteristic genes with their own quality characteristic sites (i.e., equivalent to quality sub-characteristics as shown in Chapter VIII.2) and their probability to happen.

Thus, the gene sites result from the aggregation of the quality sub-characteristics of the current quality characteristic genes, regrouping the lexically and semantically similar sub-characteristics, or sites, together like for the genes. An example with regards to "Portability" gene is described by Table 29 and Table 30, and the complete list of genes with their sites, including their lexical and semantic variations, is given into Table 50 and Table 51 of Annex 14.

The likelihood of a gene is determined by the number of its occurrence over the total number of merged quality models -in our current case this is 8. So, for "Adaptability" gene, which is present only in Alavaro's quality model, the probability to happen is 0.125 (i.e., = 0.125). About "Portability" gene, found in all the selected quality models except FURPS, its probability is 0.875 (i.e., = 0.875). Concerning the site probability, the calculations are identical and an example related to "Portability" gene is given in Table 30.

The final result is a list of 43 genes split into two parts:

-A list of the 27 genes resulting from a single quality model (cf. Table 34). The likelihood of these gene sites is therefore 1 since each gene is only present in one quality model. There is a total of 81 sites for these 27 genes, which means an average of 3 sites per gene with a minimum of 2 sites (e.g., "Adaptability" gene) and a maximum of 9 sites for "Supportability" gene. To facilitate identification in later usage, we identify these genes with the prefix "A" and a number from 1 to 27.

-A list of the 16 genes resulting from more than one quality model (cf. Table 35). There is a total of 128 sites, and consequently an average of 8 sites per gene, with a minimum of 2 sites for "Compliance" gene and a maximum of 15 sites for "Testability" gene. To facilitate identification in later usage, we identify these genes with the prefix "B" and a number from 1 to 16. 0.

c. Relationship links between genes and chromatid identification

Following the identification, construction, and definition of the 43 quality characteristic genes, the next step in the construction of the software quality model genome meta-model is to look for any chromatid. A chromatid corresponds to a quality perspective and can be retrieved as a top-level gene (i.e., the gene is not a child, or site, of another gene).

So, to determine which gene, if any, is a top-level gene, we must find first which genes are sites of other genes, and finally if a gene is not belonging, or linked to any other gene(s), we can conclude that it is a top-level gene, or chromatid. To retrieve these gene relationship links, we look for quality characteristics that are quality subcharacteristics in other quality characteristic(s), taking into account also the fact that a characteristic, or subcharacteristic, can vary in its naming (see Table 50 and Table 51 of Annex 14).

The result of this analysis, synthetized in Table 36, shows 7 chromatids: A08 "General utility", A16 "Product operation", A17 "Product revision", A18 "Product transition", A24 "Supportability", B08 "Product in use", and B12 "Software product". Note, in "found in gene(s)" column of Table 36, there are few genes in parenthesis. The parenthesis indicates that the relationship between the two genes (i.e., gene as site and gene as parent gene) require to take into account the context of the parent gene. For example, for gene A20 "Safety in use", we can find it as a site of gene B08 "Product in use". However, in "Product in use" gene the corresponding site to "Safety in use" is site 4 "Safety" (see Table 51 from Annex 14). Thus, the parenthesis points out that a context nuance must be considered carefully when using the "Safety in use" gene to detail "Safety" site.

A visual representation of Table 36 is performed through the oriented graph representation shown by Figure 84.

Each node is one of the 43 genes, the root nodes with a chromosome symbol are the chromatids, and the arrows depict the links between genes. The beginning of each arrow represents the gene as site, and the dotted arrows signal that the parent gene context has to be considered (i.e., the dotted arrows are equivalent to the parenthesis highlight). The remaining task to finalize the software quality model genome meta-model is the construction of each these seven chromatids elaborated from this graph representation and the full gene details from Table 34 and Table 35.

Contributions: The 7 Chromatids of SW Quality Model Genome

Research Sub-question 4c What is the first result of the meta-model construction?

This section is the concluding construction step of the software quality model genome meta-model. From the selected set of eight quality models, we successively extracted 43 quality characteristic genes with their own sites, or quality sub-characteristics, calculated their likelihood values, determined the relationship links between all genes and identified seven chromatids.

Thus, this final step consists in detailing each of these seven chromatids that constitute the software quality model genome meta-model.

From the complete linked gene map shown through Figure 84, we extract a subset of this map by isolating only the genes that are directly or indirectly linked to a specific chromatid. Once this sub-map done, we generate the detailed chromatid site sequence, including probability values, by enumerating each gene sites starting from the chromatid up to the furthest linked genes (see example in Figure 82 about a site enumeration beginning for "Supportability" chromatid). Figure 85 illustrates that final chromatid creation chain. However, during the enumeration task of the site sequence, there are three aspects to be caution when replacing a site by the corresponding gene and its sites For example, in gene A24 "Supportability", its site 4 is "Maintainability", but this site can be replaced by gene B05 "Maintainability" which possesses itself 10 sites.

The first aspect concerns the site likelihood values. In our example, when site 4 of A24 is replaced by a detailed version made of the 10 sites of B05, each likelihood values of these 10 sites are combined, or multiplied by the likelihood of site 4 (e.g., site 1 "Changeability" of B05 is 0.875 and in A024 gene scope this value becomes 0.875 *1.0 (from site 4)= 0.875). In the case of more levels of site-gene, we propagate the calculation over all the levels.

The second aspect is about the optimization of the number of identical sites. Indeed, again with our example, in gene B05, its site 5 is "Modularity" which is also present as site 7 of gene B14 "Testability" which replace "Testability" site 2 in gene B05. Consequently, we sum the two probability values with a maximum of 1, paying attention that we must first combine likelihood values of site 2 of B05 with site 7 of B14 (i.e., site replaced by a gene with its site and therefore it requires to combine together the likelihood values).

The third aspect is related to the identical site optimization number. In fact, two identical sites sometimes cannot be merged into one site because the result will be an incompatible gene overlap. Furthermore, since one of the expected results for chromatid is to be able to localize each gene which composes the chromatid, and not only to list all the distinct sites that compose the chromatid, a gene must be localized accurately at a specific locus but also its sequence must be contiguous. Therefore, if a merge of two identical sites may result in the failure of one of these two conditions, then the two identical sites must remain unmerged. Note, the site optimization may result in the overlap of some genes whoever, we impose that the genes directly linked to the chromatid must not be overlapped to retrieve the exact main quality characteristics of the chromatid. Table 37 gives an example of two site sequence results for gene B05 "Maintainability". The first one, on the left, is the sequence obtained by replacing each site by the corresponding gene and it sites. The second one, on the right, is the same sequence but optimized to reduce the number of similar sites. The quality coverage is identical, and we remark that, for instance, we reduced the number of "modularity" sites from five occurrence to only two. By applying that final construction step against each of the seven chromatids, we are providing the sub-map of chromatid linked genes and their corresponding site sequence, including likelihood values, in the following seven sub-sections.

a. "General Utility" Chromatid (A08)

General utility perspective "reflects the actual uses to which evaluation of software quality would be put. In general, when one is acquiring a software package, one is mainly concerned with three questions:

-How well (easily, reliably, efficiently) can I use it as is?

-How easy is it to maintain (understand, modify, and retest)?

-Can I still use it if I change my environment?"

(Source : Boehm et al. [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF]).

This chromatid is composed of 27 quality characteristic genes linked together as depicted by Figure 86, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 114 quality characteristic sites disclosed in Figure 87. The product operations perspective identifies "quality factors that influence the extent to which the software fulfils its specification" (source: http://www.sqa.net/softwarequalityattributes.html and McCall et al. [41]).

This chromatid is composed of 17 quality characteristic genes linked together as depicted by Figure 88, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 85 quality characteristic sites disclosed in Figure 89. The product transition perspective identifies "quality factors that influence the ability to adapt the software to new environments" (source: http://www.sqa.net/softwarequalityattributes.html and McCall et al. [41]).

This chromatid is composed of 10 quality characteristic genes linked together as depicted by Figure 92, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 43 quality characteristic sites disclosed in Figure 93. Supportability perspective can be defined through a set of questions "Is it testable, extensible, serviceable, installable, and configurable? Can it be monitored? How will system be extended? Who maintains the system?" (source: Grady and Caswell [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF]).

This chromatid is composed of 15 quality characteristic genes linked together as depicted by Figure 94, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 72 quality characteristic sites disclosed in Figure 95.

Note, this chromatid is close to "product revision" chromatid. Indeed, they are around 89% similar and thus "Supportability" chromatid is a polymorphism variation of "product revision" chromatid, and vice-versa. Product in use perspective "is the degree to which a product or system can be used by specific users to meet their needs to achieve specific goals with effectiveness, efficiency, freedom from risk and satisfaction in specific contexts of use" (source: ISO/IEC 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]).

This chromatid is composed of 11 quality characteristic genes linked together as depicted by Figure 96, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 34 quality characteristic sites disclosed in Figure 97. Software quality degree to which a software product satisfies stated and implied needs when used under specified conditions (software quality has the same meaning as software product quality; (source: ISO/IEC 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]).

This chromatid is composed of 32 quality characteristic genes linked together as depicted by Figure 98, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 195 quality characteristic sites disclosed in Figure 99 and Figure 100. Figure 100 -"Software product" chromatid quality characteristic sequence with its 32 genes and the 195 sites, including sites likelihood (part 2 of 2)

Threats to validity and discussions

Through this eighth chapter, we detail the creation of a meta-model resulting from the aggregation of eight existing software quality models. This aggregation is not a direct merge of these quality models but rather uses an analogy with genetic to take into account not only the likelihood for each quality characteristics and subcharacteristics to happen or exists in a particular quality perspective, but also to consider that a quality characteristic, or sub-characteristic, may have some variations in their definition and naming, with a certain level of probability too. The contributions are an analogy definition between quality modeling and genetic, a construction algorithm and the first meta-model of software quality model genome made of seven quality chromatids with their quality characteristic sequences, including their variations defined in Annex 14, and their likelihoods to happen in each chromatid genes.

The overview shown in Figure 101 points out the complexity and length of each of these seven chromatids, as well as the existence of two polymorphic chromatids: production revision and supportability. Moreover, during the meta-model elaboration, we noticed a certain level of subjectivity in grouping together quality characteristics, or sub-characteristics, considering that they are potentially variations from each other. Indeed, if they are similar (i.e., their names are identical, direct synonym, or have an identical definition) or are distinct (i.e., their names are different, not synonym, and have disjoint definition), the grouping decision is thus obvious and objective. Otherwise, the decision may be more or less subjective. This subjectivity impacts the confidence of our meta-model, and therefore, to estimate the degree of subjectivity of the meta-model, we calculate the ratio between the characteristic and sub-characteristic associations we have made subjectively over the total number of characteristics and sub-characteristics. We find that the overall degree of subjectivity of the meta-model is 20.65%, with supportability chromatid as the least subjective (i.e., 5.56%) and product transition as the most subjective one (i.e., 34.88%). The calculation results are given in TABLE 38. To add further quality characteristic nuances, include more valuable quality modeling contributions, and thus consolidate these meta-model results, we must continue to integrate more software quality models, extending the diversity scope to other software quality models (e.g., web-site [START_REF] Polillo | Quality Models for Web [2.0Sites: A Methodological Approach and a Proposal[END_REF], [258], web-services [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF], COTS [START_REF] Rawashdeh | A New Software Quality Model for Evaluating COTS Components[END_REF], [START_REF] Alves | Using goals and quality models to support the matching analysis during COTS selection[END_REF], open-source [START_REF] Adewumi | A systematic literature review of open source software quality assessment models[END_REF], [START_REF] Petrinja | Comparing OpenBRR, QSOS, and OMM Assessment Models[END_REF], [START_REF] Adewumi | A Review of Models for Evaluating Quality in Open Source Software[END_REF], IT software [START_REF] Gordieiev | IT-oriented software quality models and evolution of the prevailing characteristics[END_REF]). Note, the quality models that are candidate to be integrated must have their definition with a proper completeness level. We aim "Y", "Y+" and possible "P+" levels, if we use Oriol et al. classification definition completeness level [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF]:

-"Y (Yes): The (sub)characteristic is explicitly defined in the quality model.

-Y+ (Yes+): The (sub)characteristic is explicitly defined in the quality model and contains further subdivisions.

-P (Partially): The (sub)characteristic is not explicitly defined, but the quality model has a quality attribute or metric which can be classified into this (sub)characteristic.

-P+ (Partially+): The (sub)characteristic is not explicitly defined, but the quality model has several quality attributes or metrics which can be classified into this (sub)characteristic.

-ND (Not Defined): The (sub)characteristic is not defined, neither its quality attributes nor metrics."

Regarding use of this meta-model, it was not the purpose of this chapter. However, the meta-model can be used either as the input reference quality model of the 6-stages process for quality model development (see Chapter VI.4.b), with quality characteristics and sub-characteristics likelihood emphasizing the most important ones, or as is, with the likelihood values as weight factors, or again to develop a customized software quality model jointly, or not, with GQM paradigm [START_REF] Basili | Goal Question Metric Approach[END_REF] for example.

Another kind of usage is the detection of characteristics that prevail to a specific domain, like Gordieiev et al. [START_REF] Gordieiev | IT-oriented software quality models and evolution of the prevailing characteristics[END_REF] who were looking for the most important quality characteristics for IT-oriented software quality models based on expert review and assessment. The analogy with genetic offers an alternate approach to resolve this problematic. Indeed, the identification of prevailing quality characteristics for a specific domain can be solved similarly to the genetic disease identification thanks to non-parametric linkage technic [START_REF] Haseman | The investigation of linkage between a quantitative trait and a marker locus[END_REF]: genes at specific set of locus are looked for in a sample set, combining both healthy and sick samples; if a correlation is found between a set of those locus and the disease under investigation, then the disease is a genetic one and is associated to this specific set of locus. Nevertheless, a study must be performed to confirm the benefit of this genetic approach on sample sets, for instance, for web-services [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] and open-source [START_REF] Adewumi | A systematic literature review of open source software quality assessment models[END_REF], [START_REF] Petrinja | Comparing OpenBRR, QSOS, and OMM Assessment Models[END_REF], [START_REF] Adewumi | A Review of Models for Evaluating Quality in Open Source Software[END_REF] software quality models.

Finally, through a unique and innovative approach relying on an analogy with genetic, we successfully construct a metal-model against the software engineering domain Nevertheless, there is no restriction to apply such construction of quality model genome meta-model to any types of domain (e.g., architecture, socio-economic, systems engineering) which confirms that this achievement is another original contribution to qualimetry, the science of quality quantification.

concept [START_REF] Berkeley | The Computer Directory and Buyer's Guide[END_REF] to 2015 with Azgaldov et al. and the ABC of Qualimetry [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]. Next, we looked for Qualimetry, acknowledging that we had the correct comprehension of the young science of quality quantification, before demonstrating that the right approach for our needs is Qualimetry (i.e., research sub-question 1c).

During this investigation, we noted that Qualimetry was often misunderstood. Therefore, we start to contribute to that science by popularizing it, summarizing its major concepts under a synthetic view: the "House of Qualimetry" and its 6 pillars. We also remarked that it was possible to unify diversity and time evolution in quality modeling (i.e., research sub-question 1d) by finding our inspiration in genetic, and thus introducing the concept of polymorphism (i.e., ad hoc, universal and temporal polymorphism) in quality modeling. To complete this contribution, we proposed and proved that using a genetic diversity-based formula [START_REF] Nei | Mathematical model for studying genetic variation in terms of restriction endonucleases[END_REF] was more appropriate for comparing quality models together (i.e., research sub-question 2c) than Hamming's distance, for example, and proposed a new measurement process cadenced with system and software life cycle to integrate temporal polymorphism.

Research Question 2

Considering the set of software quality models, how to identify and decide which quality model is the most suitable for embedded software?

The focus of Chapter V was to deep dive in the literature to retrieve existing software quality models, and then determine which quality model could be selected to serve our needs with embedded software (i.e., research question 2). To carry out this undertaking, we conducted a systematic literature review where we identified and analyzed 136 study papers published during a period from 1979 to 2019. The result of this review, combined with snowballing approach, as described by Wohlin [START_REF] Wohlin | Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering[END_REF], [START_REF] Wohlin | Second-Generation Systematic Literature Studies Using Snowballing[END_REF] and which consists in exploiting each referenced papers as additional source of study papers, was the retrieval of 492 software quality models from 1968 to 2019 (i.e., research sub-question 2a). This software quality model list is a unique contribution since it represents a collection of 10 times the maximum we found in published papers: Oriol et al. [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] enumerated 48 quality models linked to web-services. Note, in Kläs et al. [START_REF] Kläs | Adapting Software Quality Models: Practical Challenges, Approach, and First Empirical Results[END_REF], the authors claimed that they have provided a classification for about 80 quality models, nevertheless we failed to retrieve that list of quality models, even in the referenced papers of that study or in the authors publication.

Strong of the systematic literature review results and speaking about classification, our next contribution was to propose the usage of cladistic as classification method for the software quality models (i.e., research sub-question 2b). For that reason, the classification scheme was made of 20 software quality models classification elements organized over five themes (i.e., id, bibliographic, definition, scope and structural), and then declined into software quality model clades: homology-based (i.e., similarity related to shared ancestry) and taxa (i.e., conceptual entities).

Although we started to use a subset of these taxa to classify these 492 quality models, they were enough to succeed on depicting a software quality model landscape. We found that these models were designed principally for quality assessment and then for prediction, they are usually hierarchical except for prediction, where statistic or implicit formalism is better adapted, with a scope often put on product, and a quality perspective equally distributed over manufacturer, user, and product perspectives. In addition, our contribution on software quality model landscape rectified Thapar et al. [11] postulate about quality model evolution (i.e., basic quality models before 2000 and tailored quality models since 2000). Indeed, we showed that this evolution is articulated around three periods: up to 1990, we have the basic quality model period, from 1990 to 2003, the transition period, and since 2003, we are in the quality model tailoring period.

At last, the conclusion of this chapter confirmed the inadequacy to have a unique reference quality model covering all software product cases (i.e., research sub-question 4a), and suggested the selection, as well the customization, of the latest quality model standard, ISO / IEC / IEEE 25010, to generate an appropriate model for embedded software in automotive domain (i.e., research sub-question 2d)

Research Question 3 Considering a quality model for a software product, how to operationalize it?

In Chapter VI, our aim was to investigate the transition from quality model theory to practice, and more particularly about the quality model operationalization (i.e., research question 3). This operational aspect is critical to develop and deploy quality model against real word use case, or to succeed in replicating and benefiting of quality model studies.

So, during our study, we identified a list of 16 distinct challenges or issues that prevent development and use of software quality models (i.e., research sub-question 3a), and we succeeded afterwards to associate practical solutions (i.e., solutions related to experiences, real situations or actions that are possible to reproduce, reuse or deploy) to each of these 16 challenges (i.e., research sub-question 3b).

The consolidated synthesis of these issue identification and resolution was achieved through the proposal of two complementary processes (i.e., research sub-question 3c):

-The "6 stages" process focuses on quality model operational development, with an analysis algorithm based on survey, Fleiss and Cohen Kappa ; this algorithm, used for quality model construction, takes into account constraint, stakeholder point of view and allows to determine automatically the quality characteristics and sub-characteristics weight factors.

-

The "Quality Thermometer" process focuses on quality model operational use; therefore, it includes the "6stages" process since one of the early stages of quality model usage concerns the quality model development.

The innovative parts of these two process contributions are the transparent encapsulation of the practical solutions and the use of polymorphism concept.

Next to the thinking and proposals about theory to practice transition, Chapter VII reflected the put into practice of our findings and contributions against our real-world use case: embedded software for the automotive industry (i.e., research sub-question 3d).

Thus, we decided to apply them against a subset of the vehicle embedded software (i.e., three electronic control units -IVI, IVC, ADAS-with their own embedded software and a transversal embedded software functionality -FOTA). The result was the creation of three distinct polymorphic quality models with their respective weight factors. We noticed in the result the existence of two levels of polymorphic quality model inheritance, and a joint quality model for ADAS and FOTA. Furthermore, all the construction steps, detailed in this chapter, can serve as proofed guidelines to perform quality modeling against any software or systems.

Finally, this success on quality model operational development for a real word use case from automotive domain, not only allowed us to answer to the company needs, but also demonstrated the merits and relevance of our findings and contributions.

Research Question 4

Can we have a unique reference quality model for software product?

As a subsidiary chapter, since we already answered to the company demand, Chapter VIII objective was to go one step further in the exploration of a reference software quality model. Indeed, Chapter V concludes on the inadequacy to have a unique reference quality model covering all software product cases and accordingly we could elaborate rather a quality meta-model, gathering knowledge from existing quality models, which could be used a basis for developing new quality model.

Continuing with genetic, we found that a certain level of analogy could be achieved between DNA sequences and quality characteristics and sub-characteristics sequence. Moreover, likes in DNA sequences, variation of quality characteristics and sub-characteristics may exist with a certain level of likelihood, recalling the polymorphism concept. So, the basis of our quality meta-model contribution relied on this analogy which was also captured into a meta-model ontology.

After the detailed design of the meta-model construction algorithm (i.e., research sub-question 4b), we selected a set of eight existing software quality models to initiate the creation of the meta-model first version (i.e., research sub-question 4c). The result of this unique and final contribution is the software quality genome composed of 7 chromatids: general utility, product operation, product revision, product transition, supportability, product in use and software product.

The thesis research work and achievements are summarized also in the global synthesis done in Figure 102.

Research Perspectives

This comprehensive and in-depth research study on software quality models is the beginning of an exciting but hectic journey in the field of Qualimetry. The resulting research perspectives shed light on the first directions this journey should take. There are three of them, namely: valorization, consolidation, and exploration.

-Valorization perspectives: the intend behind these perspectives is not only to share more widely our research findings and contributions with the academic and industrial community, but also to improve our proposal through feedback and measurement of effectiveness, as well as to promote its appropriation and adaptation, through a strategy of deployment in company accompanied by training and tooling. The timeframe for this kind of perspective is mainly between short-term and mid-term period.

Thus, one way to achieve the information sharing is to rely on literature media. We planned to gather the following finding and contribution into several research paper submissions: Furthermore, and as we already raised in Chapter V.5, an online portal tool must be created to spread the sharing of the 492 software quality model collection and enable the collaboration for their use, completion and maintenance. The motivation behind is to allow the academic and industrial community to collaborate on this collection and avoid that this list becomes obsolete within the coming years.

o
An alternate way to enhance the value of our research results is to industrialize, scale and deploy against actual production systems the thesis contributions. However, if we measure the technology maturity of our thesis achievements using the Technology Readiness Level (TRL) [START_REF]Technology Readiness Levels Handbook for Sapce Applications[END_REF] scale, we are currently reaching level 4, that is to say technology has been validated in laboratory environment, while industrialization, scaling and deployment mean a level of 9. Hopefully, we can use the TRL to guide us on the path of the technology readiness.

-Consolidation perspectives: this second kind of research perspectives lead us to continue the consolidation of our current findings and contributions both from a research and development point of views. The timeframe for this type of research perspective is at least mid-term. The following paragraphs briefly describe the primary research and development directions.

Regarding the quality model collection and classification, the consolidation means that we have to create the right tool and data model to gather, store and classify properly at minimum the 492 found quality models. Then, we should be able to perform full classification based on cladistics and takes all benefits from these related contributions.

On the meta-model side, that research perspective indicates that we must first complete the implementation of a tool for the automatic meta-model construction, and next, integrate more quality models in the metamodel during its construction. One of the expected results linked to more quality model inclusions in the metal-model is to strengthen the convergence of the most relevant and important quality characteristics and sub-characteristics. A further meta-model enhancement will be the integration of metrics.

Concerning quality model development consolidation perspective, we aim to use the meta-model as the referenced quality model, foster quality model reuse through polymorphism, and build a tool to automate the "6-stages" process for quality model development, including metrics consideration. In parallel, the practical quality modeling for an entire complex system such as an entire vehicle should be handled.

Similarly, to consolidate quality model usage with the "Quality Thermometer" process, the tooling aspect must be addressed and should cover at least: the transparent use of polymorphic quality model, automated operational deployment and execution of quality models and their metrics, online dashboard and scorecard with data mining capability to enable prediction and prescription.

-Exploration perspectives: the objectives of such research perspectives are to explore, and study open problems related to Qualimetry, quality model or modeling. Consequently, the corresponding timeframe basis for that research work is long-term. A certain number of open problems has been already captured.

The first problem is about the assessing, or predicting, the value brough by quality model development and use. This is a recurrent question often coming from company leaders to accept Qualimetry activity cost. However, this question remains unanswered despite few research studies such as Khoshgoftaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF] whose initiate the construction of quality modeling activity cost-benefit model based on the assumption of quality model reuse over multiple software releases, Porta [START_REF] Porta | Towards a Model for Cost-Benefit-Analysis of Quality Assurance in the Automotive E/E Development[END_REF] with a survey on cost-benefit -analysis model for quality assurance, or the integral quality composed of quality and cost effectiveness (see Chapter IV.2.a).

A second type of problem is the formal definition and generalization of thresholds to assess, control or predict objectively that a quality level of a product, for instance, is good. Unfortunately, we usually have neither universal (i.e., commonly agreed) acceptance, reference, nor target threshold. One way to go around that problem is to define a target, or an acceptance threshold based on previous results obtained from identical product, like a previous software release. Thus, we remove the problem by considering only the progress compare to previous achievements. Nevertheless, the original problem remains intact even if there are few minor industrial attempts like the 15 acceptance ranges from the automotive HIS source code metrics [START_REF] Kuder | HIS source code metrics, version 1.3.1[END_REF].

The third category of problem relates to the modeling generalization of quality trajectory and its velocity.

One parallel problem is about the discontinuity that may exist between assessment and prediction model with identical scope and quality perspective but with distinct model formalisms. Damerau-Levenshtein's distance [START_REF] Damerau | A Technique for Computer Detection and Correction of Spelling Errors[END_REF] similar to Levenshtein's distance but includes the additional operation: transposition of two adjacent string characters. This chapter should provide the purpose of the quality evaluation plan, introducing the context and situation of the evaluation.

Organizations involved in the evaluation, such as the independent evaluation organization, product developers and acquirer's organizational units.

This chapter should provide information products expected from the evaluation.

Performance Processing Unit load

Percentage of current Processing Unit (i.e., CPU, GPU, DSP, NPU) Load when running processing unit load test scenario. Assessment must be done accordingly to project target.

Volatile Memory Footprint

Maximum volatile memory (i.e., RAM, Cache) footprint of the software with respect to available volatile memory. Assessment must be done accordingly to project target.

Non-Volatile Memory Footprint

Maximum non-volatile memory (i.e., ROM, Flash) footprint of the software with respect to available non-volatile memory. Assessment must be done accordingly to project target.

 Performance Efficiency

Quality Sub-Characteristic Definition Time Behaviour Degree to which the response and processing times and throughput rates of a product or system, when performing its functions, meet requirements.

Resource Utilization

Degree to which the amounts and types of resources used by a product or system, when performing its functions, meet requirements.

Capacity

Degree to which the maximum limits of a product or system parameter meet requirements.

 Compatibility

Quality Sub-Characteristic Definition

Co-existence

Degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other products, without detrimental impact on any other product.

Interoperability

Degree to which two or more systems, products or components can exchange information and use the information that has been exchanged.

 Usability

Quality Sub-Characteristic Definition

Appropriateness Recognizability Degree to which users can recognize whether a product or system is appropriate for their needs.

Learnability

Degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use.

Operability

Degree to which a product or system has attributes that make it easy to operate and control.

User Error Protection

Degree to which a system protects users against making errors.

User Interface Aesthetics

Degree to which a user interface enables pleasing and satisfying interaction for the user.

Accessibility

Degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use.

 Reliability

Quality Sub-Characteristic Definition Maturity Degree to which a system, product or component meets needs for reliability under normal operation.

Quality Sub-Characteristic Definition

Fault Tolerance Degree to which a system, product or component operates as intended despite the presence of hardware or software faults.

Recoverability

Degree to which, in the event of an interruption or a failure, a product or system can recover the data directly affected and re-establish the desired state of the system.

 Security

Quality Sub-Characteristic Definition Confidentiality Degree to which a product or system ensures that data are accessible only to those authorized to have access.

Integrity

Degree to which a system, product or component prevents unauthorized access to, or modification of, computer programs or data.

Non-Repudiation

Degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later.

Accountability

Degree to which the actions of an entity can be traced uniquely to the entity.

Authenticity

Degree to which the identity of a subject or resource can be proved to be the one claimed.

 Maintainability

Quality Sub-Characteristic Definition Modularity Degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components.

Reusability

Degree to which an asset can be used in more than one system, or in building other assets.

Analysability

Degree of effectiveness and efficiency with which it is possible to assess the impact on a product or system of an intended change to one or more of its parts, or to diagnose a product for deficiencies or causes of failures, or to identify parts to be modified.

Modifiability

Degree to which a product or system can be effectively and efficiently modified without introducing defects or degrading existing product quality.

Testability

Degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be performed to determine whether those criteria have been met.

 Portability

Quality Sub-Characteristic Definition Adaptability Degree to which a product or system can effectively and efficiently be adapted for different or evolving hardware, software or other operational or usage environments.

Installability

Degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified environment.

Efficiency

Resources expended in relation to the accuracy and completeness with which users achieve goals.

Satisfaction

Degree to which user needs are satisfied when a product or system is used in a specified context of use.

Freedom from Risk

Degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment.

Context Coverage

Degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified.

o Quality sub-characteristic definitions  Effectiveness <no further sub-characteristic>  Efficiency <no further sub-characteristic>

 Satisfaction

Quality Sub-Characteristic Definition Usefulness Degree to which a user is satisfied with their perceived achievement of pragmatic goals, including the results of use and the consequences of use.

Trust

Degree to which a user or other stakeholder has confidence that a product or system will behave as intended.

Pleasure

Degree to which a user obtains pleasure from fulfilling their personal needs.

Comfort

Degree to which the user is satisfied with physical comfort.

 Freedom from Risk

Quality Sub-Characteristic Definition Economic Risk Mitigation Degree to which a product or system mitigates the potential risk to financial status, efficient operation, commercial property, reputation or other resources in the intended contexts of use.

Health and Safety Risk Mitigation Degree to which a product or system mitigates the potential risk to people in the intended contexts of use.

Environmental Risk Mitigation

Degree to which a product or system mitigates the potential risk to property or the environment in the intended contexts of use degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment.

 Context Coverage

Quality Sub-Characteristic Definition

Context Completeness

Degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in all the specified contexts of use.

Flexibility

Degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in contexts beyond those initially specified in the requirements degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified. The capability of the software product to provide the right or agreed results or effects with the needed degree of precision.

Adaptability

The capability of the software product to be adapted for different specified environments without applying actions or means other than those provided for this purpose for the software considered.

Analyzability

The capability of the software product to be diagnosed for deficiencies or causes of failures in the software, or for the parts to be modified to be identified.

Attractiveness

The capability of the software product to be attractive to the user.

Changeability

The capability of the software product to enable a specified modification to be implemented.

Co-existence

The capability of the software product to co-exist with other independent software in a common environment sharing common resources.

Effectiveness

The capability of the software product to enable users to achieve specified goals with accuracy and completeness in a specified context of use.

Efficiency

The capability of the software product to provide appropriate performance, relative to the amount of resources used, under stated conditions.

Efficiency compliance

The capability of the software product to adhere to standards or conventions relating to efficiency.

Fault tolerance

The capability of the software product to maintain a specified level of performance in cases of software faults or of infringement of its specified interface.

Functionality

The capability of the software product to provide functions which meet stated and implied needs when the software is used under specified conditions.

Functionality compliance

The capability of the software product to adhere to standards, conventions or regulations in laws and similar prescriptions relating to functionality.

Installability

The capability of the software product to be installed in a specified environment.

Interoperability

The capability of the software product to interact with one or more specified systems.

Learnability

The capability of the software product to enable the user to learn its application.

Maintainability

The capability of the software product to be modified. Modifications may include corrections, improvements, or adaptation of the software to changes in environment, and in requirements and functional specifications.

Maintainability compliance

The capability of the software product to adhere to standards or conventions relating to maintainability.

Maturity

The capability of the software product to avoid failure as a result of faults in the software.

Operability

The capability of the software product to enable the user to operate and control it.

Portability

The capability of the software product to be transferred from one environment to another.

Portability compliance

The capability of the software product to adhere to standards or conventions relating to portability.

Productivity

The capability of the software product to enable users to expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use.

Recoverability

The capability of the software product to re-establish a specified level of performance and recover the data directly affected in the case of a failure.

Reliability

The capability of the software product to maintain a specified level of performance when used under specified conditions.

Reliability compliance

The capability of the software product to adhere to standards, conventions or regulations relating to reliability.

Replaceability

The capability of the software product to be used in place of another specified software product for the same purpose in the same environment.

Resource utilization

The capability of the software product to use appropriate amounts and types of resources when the software performs its function under stated conditions.

Safety

The capability of the software product to achieve acceptable levels of risk of harm to people, business, software, property, or the environment in a specified context of use.

Satisfaction

The capability of the software product to satisfy users in a specified context of use.

Security

The capability of the software product to protect information and data so that unauthorized persons or systems cannot read or modify them, and authorized persons or systems are not denied access to them.

Stability

The capability of the software product to avoid unexpected effects from modifications of the software.

Suitability

The capability of the software product to provide an appropriate set of functions for specified tasks and user objectives.

Testability

The capability of the software product to enable modified software to be validated.

-Alvaro quality model [START_REF] Alvaro | A Software Component Quality Model: A Preliminary Evaluation[END_REF] (2010)

This model is composed of 6 quality characteristics 23 quality sub-characteristics 48

attributes

This model uses terminology from ISO/IEC 9126, and ISO/IEC/IEEE 25010

Terminology Definition

Accessibility degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use.

Accountability degree to which the actions of an entity can be traced uniquely to the entity.

Adaptability degree to which a product or system can effectively and efficiently be adapted for different or evolving hardware, software or other operational or usage environments.

Analyzability degree of effectiveness and efficiency with which it is possible to assess the impact on a product or system of an intended change to one or more of its parts, or to diagnose a product for deficiencies or causes of failures, or to identify parts to be modified.

Appropriateness Recognizability

degree to which users can recognize whether a product or system is appropriate for their needs.

Authenticity degree to which the identity of a subject or resource can be proved to be the one claimed.

Availability degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use.

Capacity degree to which the maximum limits of a product or system parameter meet requirements.

Co-existence degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other products, without detrimental impact on any other product.

Comfort degree to which the user is satisfied with physical comfort.

Compatibility degree to which a product, system or component can exchange information with other products, systems, or components, and/or perform its required functions, while sharing the same hardware or software environment.

Confidentiality degree to which a product or system ensures that data are accessible only to those authorized to have access.

Context Completeness degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in all the specified contexts of use.

Context Coverage degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified.

Economic Risk Mitigation degree to which a product or system mitigates the potential risk to financial status, efficient operation, commercial property, reputation or other resources in the intended contexts of use.

Effectiveness

accuracy and completeness with which users achieve specified goals.

Efficiency resources expended in relation to the accuracy and completeness with which users achieve goals.

Environmental Risk Mitigation degree to which a product or system mitigates the potential risk to property or the environment in the intended contexts of use degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment.

Fault Tolerance degree to which a system, product or component operates as intended despite the presence of hardware or software faults.

Flexibility degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in contexts beyond those initially specified in the requirements degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified.

Freedom from Risk degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment.

Functional Appropriateness degree to which the functions facilitate the accomplishment of specified tasks and objectives.

Functional Completeness degree to which the set of functions covers all the specified tasks and user objectives.

Functional Correctness degree to which a product or system provides the correct results with the needed degree of precision.

Functional Suitability degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions.

Health and Safety Risk Mitigation

degree to which a product or system mitigates the potential risk to people in the intended contexts of use.

Installability degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified environment.

Integrity degree to which a system, product or component prevents unauthorized access to, or modification of, computer programs or data.

Interoperability degree to which two or more systems, products or components can exchange information and use the information that has been exchanged.

Learnability degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use.

Maintainability degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers.

Maturity degree to which a system, product or component meets needs for reliability under normal operation.

Modifiability degree to which a product or system can be effectively and efficiently modified without introducing defects or degrading existing product quality.

Modularity degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components.

Non-Repudiation degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later.

Operability degree to which a product or system has attributes that make it easy to operate and control.

Performance Efficiency performance relative to the amount of resources used under stated conditions.

Pleasure degree to which a user obtains pleasure from fulfilling their personal needs.

Portability degree of effectiveness and efficiency with which a system, product or component can be transferred from one hardware, software or other operational or usage environment to another.

Reliability degree to which a system, product or component performs specified functions under specified conditions for a specified period of time.

Replaceability degree to which a product can replace another specified software product for the same purpose in the same environment.

Resource Utilization degree to which the amounts and types of resources used by a product or system, when performing its functions, meet requirements.

Reusability degree to which an asset can be used in more than one system, or in building other assets.

Satisfaction degree to which user needs are satisfied when a product or system is used in a specified context of use.

Security degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization.

Testability degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be performed to determine whether those criteria have been met.

Time Behaviour degree to which the response and processing times and throughput rates of a product or system, when performing its functions, meet requirements.

Trust degree to which a user or other stakeholder has confidence that a product or system will behave as intended.

Usability degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.

Usefulness degree to which a user is satisfied with their perceived achievement of pragmatic goals, including the results of use and the consequences of use.

User Error Protection degree to which a system protects users against making errors.

User Interface Aesthetics degree to which a user interface enables pleasing and satisfying interaction for the user. Ce travail a donné lieu à de multiples contributions, partiellement valorisées sous la forme de deux articles présentés en conférences internationales (cf. Argotti et al. [START_REF] Argotti | Quality quantification in Systems Engineering from the Qualimetry Eye[END_REF], [START_REF] Argotti | Quality Quantification Applied to Automotive Embedded Systems and Software[END_REF]). Sa contribution majeure au corpus des connaissances scientifique est l'étude exhaustive et approfondie des modèles qualité existants dans la littérature prérequis incontournable pour aller plus loin dans des propositions conceptuelles et méthodologiques consolidées.

Synthèse Générale

Dans le Chapitre I, nous posons la problématique de recherche de thèse, « Etude des éléments essentiels de la Qualimétrie appliquée au développement de logiciels embarqués », que nous avons reformulée en « renforcer et unifier la définition, l'évaluation, le contrôle ou la prédiction de la qualité des logiciels embarqués », au regard de notre contexte industriel. Nous avons entamé son analyse en identifiant les quatre questions de recherche suivantes :

Question de recherche 1

La Qualimétrie, en tant que science de la quantification de la qualité, estelle la bonne approche et quels sont les éléments essentiels de la qualité et de la Qualimétrie ?

Question de recherche 2

Considérant l'ensemble des modèles qualité pour le logiciel, comme identifier et décider quel modèle qualité est le plus approprié pour le logiciel embarqué ?

Question de recherche 3 Comment opérationnaliser un modèle qualité pour un produit logiciel ?

Question de recherche 4 Peut-on avoir un modèle qualité de référence unique pour les produits logiciels ?

Nous avons ensuite approfondi cette analyse dans le Chapitre II. Nous avons remarqué que le contexte industriel automobile, conjointement avec le véhicule en tant que système complexe, le modèle de développement avec les fournisseurs et les exigences actuelles en matière de normes et de réglementations, augmentent la complexité globale de notre problématique. Par conséquent, dans ce contexte, il est essentiel de disposer d'une méthode unifiée, opérationnelle et appropriée pour définir, évaluer, contrôler ou prévoir la qualité des logiciels embarqués.

Afin de vérifier si une telle solution unifiée, opérationnelle et appropriée pour la qualité des logiciels embarqués existe déjà, nous avons effectué une analyse documentaire exploratoire sur « la manière dont la modélisation de la qualité est appliquée aux logiciels embarqués ». Nous avons constaté l'existence d'une myriade de systèmes et de logiciels embarqués possibles, chacun d'eux ayant ses propres spécificités, caractéristiques de qualité et éventuellement une diversité de modèles qualité. Il est donc apparu qu'il n'y avait pas encore de solution juste et unique à notre question.

En conséquence, nous avons affiné ces quatre questions de recherche en 15 sous-questions de recherche, puis nous avons détaillé notre méthodologie de recherche au Chapitre III. Dans ce chapitre, également, nous avons expliqué le réalignement de notre méthodologie de recherche en soulignant non seulement les difficultés dans la sélection d'un modèle qualité approprié pour les logiciels intégrés, mais aussi les conséquences d'une telle sélection en écartant de nombreuses contributions précieuses.

Ensuite, nous avons abordé ces questions de recherche dans les Chapitres de IV à VIII.

Question de recherche 1

La Qualimétrie, en tant que science de la quantification de la qualité, estelle la bonne approche et quels sont les éléments essentiels de la qualité et de la Qualimétrie ?

Ainsi, dans le Chapitre IV, nous avons exploré l'essence de la qualité (i.e., sous-question de recherche 1a), et la modélisation de la qualité en particulier dans le domaine des logiciels (i.e., sous-question de recherche 1b). Nous avons défini, clarifié ces connaissances et les concepts associés (par exemple, la qualité perçue, les perspectives de qualité, les dimensions et les caractéristiques de la qualité, le modèle qualité, les mesures, l'échelle), et conclu cette exploration par la construction de la première chronologie des contributions clefs à la modélisation de la qualité des logiciels, allant de 1965 avec la première apparition du concept de génie logiciel [START_REF] Berkeley | The Computer Directory and Buyer's Guide[END_REF] à 2015 avec Azgaldov et al. et l'ABC de la Qualimétrie [START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF]. Ensuite, nous avons cherché la Qualimétrie, reconnaissant que nous avions une bonne compréhension de la jeune science de la quantification de la qualité, avant de démontrer que la bonne approche pour nos besoins est la Qualimétrie (i.e., sous-question de recherche 1c).

Au cours de cette enquête, nous avons constaté que la Qualimétrie était souvent mal comprise et nous commençons donc à contribuer à cette science en la vulgarisant, en résumant ses concepts majeurs sous une vue synthétique : la « Maison de la Qualimétrie » et ses 6 piliers. Nous avons également remarqué qu'il était possible d'unifier la diversité et l'évolution temporelle dans la modélisation de la qualité (i.e., sous-question de recherche 1d) en trouvant notre inspiration dans la génétique, et donc en introduisant le concept de polymorphisme (c'està-dire le polymorphisme ad hoc, universel et temporel) dans la modélisation de la qualité. Pour compléter cette contribution, nous avons proposé et prouvé que l'utilisation d'une formule basée sur la diversité génétique [START_REF] Nei | Mathematical model for studying genetic variation in terms of restriction endonucleases[END_REF] était plus appropriée pour comparer des modèles qualité ensemble (i.e., sous-question de recherche 2c) que la distance de Hamming, par exemple, et nous avons proposé un nouveau processus de mesure cadencé avec le cycle de vie du système et du logiciel pour intégrer le polymorphisme temporel.

Question de recherche 2

Considérant l'ensemble des modèles qualité pour le logiciel, comme identifier et décider quel modèle qualité est le plus approprié pour le logiciel embarqué ?

L'objectif du Chapitre V était de plonger en profondeur dans la littérature pour récupérer les modèles qualité des logiciels existants, puis de déterminer quel modèle qualité pourrait être sélectionné pour répondre à nos besoins avec les logiciels intégrés (i.e., question de recherche 2). Pour mener à bien cette entreprise, nous avons procédé à un examen systématique de la littérature, dans le cadre duquel nous avons identifié et analysé 136 documents d'étude publiés au cours d'une période allant de 1979 à 2019. Le résultat de cette revue, combiné à l'approche en « boule de neige », telle que décrite par Wohlin [START_REF] Wohlin | Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering[END_REF], [START_REF] Wohlin | Second-Generation Systematic Literature Studies Using Snowballing[END_REF] et qui consiste à exploiter chaque document référencé comme source supplémentaire de documents d'étude, a été la récupération de 492 modèles qualité de logiciels de 1968 à 2019 (i.e., sous-question de recherche 2a). Cette liste de modèles qualité des logiciels est une contribution unique puisqu'elle représente une collection dix fois supérieure au maximum que nous avons trouvé dans les articles publiés : Oriol et al [START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] ont énuméré 48 modèles qualité liés aux services web. A Noter que dans Kläs et al [START_REF] Kläs | Adapting Software Quality Models: Practical Challenges, Approach, and First Empirical Results[END_REF], les auteurs affirment avoir fourni une classification pour environ 80 modèles qualité, mais nous n'avons pas réussi à retrouver cette liste de modèles qualité, même dans les articles référencés de cette étude ou dans la publication des auteurs.

Fort des résultats de la revue systématique de la littérature et parlant de classification, notre contribution suivante a été de proposer l'utilisation de la cladistique comme méthode de classification pour les modèles qualité des logiciels (i.e., la sous-question de recherche 2b). Pour cette raison, le schéma de classification a été constitué de 20 éléments de classification des modèles qualité des logiciels organisés en cinq thèmes (i.e., id, bibliographique, définition, portée et structure), puis décliné en cladistique des modèles qualité des logiciels : homologie (i.e., similarité liée à une ascendance commune) et taxons (i.e., entités conceptuelles).

Bien que nous ayons commencé à utiliser un sous-ensemble de ces taxons pour classer ces 492 modèles qualité, ils ont suffi pour réussir à représenter un paysage de modèles qualité logicielle. Nous avons constaté que ces modèles ont été conçus principalement pour l'évaluation de la qualité, puis pour la prédiction, ils sont généralement hiérarchisés, sauf pour la prédiction, où le formalisme statistique ou implicite est mieux adapté, avec un champ d'application souvent mis sur le produit, et une perspective de qualité également répartie entre les perspectives du fabricant, de l'utilisateur et du produit. En outre, notre contribution sur le paysage des modèles qualité des logiciels a rectifié le postulat de Thapar et al. [11] Après la conception détaillée de l'algorithme de construction du méta-modèle (i.e., sous-question de recherche 4b), nous avons sélectionné un ensemble de modèles qualité de logiciels existants pour lancer la création de la première version du méta-modèle (i.e., sous-question de recherche 4c). Le résultat de cette contribution unique et finale est le génome de la qualité logicielle composé de 7 chromatides : utilité générale, fonctionnement du produit, révision du produit, transition du produit, supportabilité, produit en cours d'utilisation et produit logiciel.

Les travaux de recherche et les réalisations de la thèse sont également résumés dans la synthèse globale réalisée à travers la Figure 103.

Perspectives de recherche

Cette étude complète et approfondie sur les modèles qualité des logiciels est le début d'un voyage passionnant mais trépidant dans le domaine de la Qualimétrie. Les perspectives de recherche qui en résultent éclairent les premières directions que ce voyage devrait prendre. Elles sont au nombre de trois, à savoir : la valorisation, la consolidation et l'exploration.

-Perspectives de valorisation : l'intention derrière ces perspectives est non seulement de partager plus largement nos résultats et contributions de recherche avec la communauté académique et industrielle, mais Une autre façon de valoriser les résultats de nos recherches est d'industrialiser, de mettre à l'échelle et de déployer les contributions de la thèse par rapport à des systèmes de production réels. Cependant, si nous mesurons la maturité technologique de nos réalisations de thèse en utilisant l'échelle du niveau de préparation technologique (TRL) [START_REF]Technology Readiness Levels Handbook for Sapce Applications[END_REF], nous atteignons actuellement le niveau 4, c'est-à-dire que la technologie a été validée en laboratoire, tandis que l'industrialisation, la mise à l'échelle et le déploiement signifient un niveau 9. Nous espérons pouvoir utiliser le TRL pour nous guider sur la voie de la préparation technologique.

Figure 103 -Synthèse générale des travaux de recherche et des réalisations de la thèse -Perspectives de consolidation : ce deuxième type de perspectives de recherche nous amène à poursuivre la consolidation de nos résultats et contributions actuels, tant du point de vue de la recherche que du développement. Ce type de perspective de recherche s'inscrit au moins à moyen terme. Les paragraphes suivants décrivent brièvement les principales orientations de la recherche et du développement.

En ce qui concerne la collecte et la classification des modèles qualité, la consolidation signifie que nous devons créer l'outil et le modèle de données adéquats pour recueillir, stocker et classifier correctement au moins les 492 modèles qualité trouvés. Ensuite, nous devrions être en mesure d'effectuer une classification complète basée sur la cladistique et de tirer tous les bénéfices de ces contributions connexes.

Pour le méta-modèle, cette perspective de recherche indique que nous devons d'abord achever la mise en place d'un outil pour la construction automatique du méta-modèle, et ensuite, intégrer davantage de modèles qualité dans le méta-modèle au cours de sa construction. L'un des résultats attendus liés à l'intégration d'un plus grand nombre de modèles qualité dans le métal-modèle est de renforcer la convergence des caractéristiques et sous-caractéristiques de qualité les plus pertinentes et les plus importantes. Une autre amélioration du méta-modèle sera l'intégration de mesures.

Au sujet de la perspective de consolidation du développement du modèle qualité, nous visons à utiliser le méta-modèle comme modèle qualité référencé, à encourager la réutilisation du modèle qualité par le biais du polymorphisme, et à construire un outil pour automatiser le processus « en 6-étapes » pour le développement du modèle qualité, y compris la prise en compte des métriques. En parallèle, la modélisation pratique de la qualité pour un système complexe entier tel qu'un véhicule entier devrait être traitée.

De même, pour consolider l'utilisation du modèle qualité avec le processus « Thermomètre de Qualité », l'aspect outillage doit être abordé et devrait couvrir au moins : l'utilisation transparente du modèle qualité polymorphe, le déploiement opérationnel automatisé et l'exécution des modèles qualité et de leurs métriques, le tableau de bord en ligne et la carte de pointage avec capacité d'exploration de données pour permettre la prédiction et la prescription.

-Perspectives d'exploration : les objectifs de ces perspectives de recherche sont d'explorer et d'étudier des problèmes ouverts liés à la Qualimétrie, au modèle qualité ou à la modélisation. Par conséquent, la base temporelle correspondante pour ce travail de recherche est à long terme. Un certain nombre de problèmes ouverts ont déjà été saisis.

Le premier problème concerne l'évaluation, ou la prévision, de la valeur apportée par le développement et l'utilisation de modèles qualité. C'est une question récurrente qui vient souvent des chefs d'entreprise pour accepter le coût de l'activité de Qualimetry. Cependant, cette question reste sans réponse malgré les quelques études de recherche telles que Khoshgoftaar et al. [START_REF] Khoshgoftaar | Cost-benefit analysis of software quality models[END_REF] qui ont lancé la construction d'un modèle coûts-avantages de l'activité de modélisation de la qualité basé sur l'hypothèse de la réutilisation du modèle qualité sur plusieurs versions de logiciels, Porta [START_REF] Porta | Towards a Model for Cost-Benefit-Analysis of Quality Assurance in the Automotive E/E Development[END_REF] avec une enquête sur le modèle d'analyse coûtsavantages pour l'assurance qualité, ou la qualité intégrale composée de la qualité et de la rentabilité (voir Chapitre IV).

Un deuxième type de problème concerne la définition formelle et la généralisation de seuils pour évaluer, contrôler ou prédire objectivement qu'un niveau de qualité d'un produit, par exemple, est bon. Malheureusement, nous n'avons généralement pas de seuil d'acceptation, de référence ou d'objectif universel (c'est-à-dire convenu d'un commun accord). Une façon de contourner ce problème consiste à définir une cible, ou un seuil d'acceptation, sur la base de résultats antérieurs obtenus avec un produit identique, comme une version précédente du logiciel. Ainsi, on élimine le problème en ne considérant que les progrès par rapport aux réalisations antérieures. Néanmoins, le problème initial reste intact même s'il y a peu de tentatives industrielles mineures comme les 15 plages d'acceptation des mesures du code source du HIS automobile [START_REF] Kuder | HIS source code metrics, version 1.3.1[END_REF].

La troisième catégorie de problèmes ici est la généralisation de la modélisation de la trajectoire de qualité et de sa vitesse. Un problème parallèle concerne la discontinuité qui peut exister entre le modèle d'évaluation et le modèle de prédiction avec une portée et une perspective de qualité identiques mais avec des formalismes de modèle distincts.

1 . 2 .

 12 Introduction .............................................................................................................................................. 2. A complex industrial research context with a need of quality .................................................................. 3. Quality modeling applied to embedded software development .............................................................. a. Quality model creation or use studies .................................................................................................. b. Quality characteristics or/and quality model identification studies ..................................................... c. Reliability growth model creation or use studies ................................................................................. d. Miscellaneous work related to quality modeling in embedded software ............................................ e. Limits to existing quality modeling applied to embedded software development .............................. 1. Introduction .............................................................................................................................................. The essence of quality ............................................................................................................................... a. Integral quality ...................................................................................................................................... b. Perceived quality .................................................................................................................................. c. Quality perspectives and views ............................................................................................................ d. Quality dimensions and characteristics ................................................................................................ e. Lagging and leading indicators .............................................................................................................. 3. The essence of quality modeling ............................................................................................................... a. Quality model ....................................................................................................................................... b. Measurements ......................................................................................................................................

Figure 1 -

 1 Figure 1 -Dissertation chapters mapped against the 4 research questions ..........................................................

Figure 2 -

 2 Figure 2 -Growing ECUs number per year for luxury cars (source Crolla et al. [15], chapter 14, figure 2) ...........

Figure 3 -

 3 Figure 3 -Embedded systems in contrast to other computing systems (source Lepistös [34]) .............................

Figure 4 -

 4 Figure 4 -Ontology of quality model definition from IEEE 1061 [40] ....................................................................

Figure 5 -

 5 Figure 5 -Use of type of information as a helper to find and associate metrics to quality requirements, based on Guessi et al. [68] ............................................................................................................................................

Figure 6 -

 6 Figure 6 -Overview of our early research flow breakdown ...................................................................................

Figure 7 -

 7 Figure 7 -Overview of our research flow breakdown: in red the technological locks, in green our contributions .......................................................................................................................................................................

Figure 8 -

 8 Figure 8 -Kano's model of perceived quality [58] ..................................................................................................

Figure 9 -

 9 Figure 9 -Ontology of main quality keywords .......................................................................................................

Figure 10 -

 10 Figure 10 -Leading indicator composition, Roedler et al. [106] ............................................................................

Figure 11 -

 11 Figure 11 -The DAP classification introduced by Deissenboeck et al. 2009 [120] .................................................

Figure 12 -

 12 Figure 12 -Example of a definition model: the ISO/IEC/IEEE 25010 System / Software product quality model [23] ................................................................................................................................................................

Figure 13 -

 13 Figure 13 -Example of an assessment model: Maturity sub-characteristics measurement extension of ISO/IEC/IEEE 25010 System / Software product quality model [23] .............................................................

Figure 14 -

 14 Figure 14 -Boehm's quality model (1976) [42] with an example of cycle in the graph structure .........................

Figure 15 -

 15 Figure 15 -Illustration of the concept of reliability and validity of measures [135] ..............................................

Figure 16 -

 16 Figure 16 -The Chrisman's scales [139], an extension of the Stevens' scales (in boldface) ..................................

Figure 17 -

 17 Figure 17 -"Continuous Preference Logic" operators and corresponding weighted power mean parameter used in the "Logic Scoring of Preference" of Dujmovic and Bayucan [141] ...........................................................

Figure 18 -

 18 Figure 18 -Threshold for risk acceptance and opportunities: from ISO/IEC 25022 [140] theory to practice ........

Figure 19 -

 19 Figure 19 -Timeline of the key contributions to quality modeling of software .....................................................

Figure 20 -

 20 Figure 20 -The "House of Qualimetry" and its 6 pillars .........................................................................................

Figure 21 -

 21 Figure 21 -The "House of Qualimetry" and its 6 pillars in a nutshell .....................................................................

Figure 22 -

 22 Figure 22 -Example of a result from the application of the two of the three pillars of quality model architrave against "software product": the ISO/IEC/IEEE 25010 quality models [23]. ...................................................

Figure 23 -

 23 Figure 23 -Polymorphism mechanism showcased with butterfly analogy: example of a generic butterfly and its variety of butterflies with heredity links (source: [167] & [165]) ..................................................................

Figure 24 -

 24 Figure 24 -Polymorphism mechanism showcased with butterfly analogy: example of temporal evolution (source: [168]) ...............................................................................................................................................

Figure 25 -

 25 Figure 25 -Example of some differences between ISO/IEC 9126 & ISO/IEC/IEEE 25010 ......................................

Figure 26 -

 26 Figure 26 -Software product quality evaluation process defined by ISO/IEC 25040 [144] ...................................

Figure 27 -

 27 Figure 27 -Measurement process model of McGarry et al. [154] .........................................................................

Figure 28 -

 28 Figure 28 -Our measurement process proposal articulated over three phases and cadenced with SDLC ...........

Figure 29 -

 29 Figure 29 -The distinct stages of the systematic literature review process for study selection, with selected document numbers and publication years ....................................................................................................

Figure 30 -

 30 Figure 30 -Ratio of the two types (i.e., conference and journal) of selected studies ............................................

Figure 31 -

 31 Figure 31 -Ratio of context (i.e., academic, industry, both) of the selected studies .............................................

Figure 32 -

 32 Figure 32 -Selected conference and journal paper distribution over publication year .........................................

Figure 33 -

 33 Figure 33 -Citation numbers of systematic literature review qualified papers per year; top bar graph, cumulative citation number, bottom bar graph, normalized citation number per number of papers .........

Figure 34 -

 34 Figure 34 -Distributions of studies per type of study ..........................................................................................

Figure 35 -

 35 Figure 35 -Basic vs. Tailored quality model categorization (source: Thapar et al. [11]) ......................................

Figure 36 -

 36 Figure 36 -Distributions of study classification or comparison criteria categories .............................................

Figure 37 -

 37 Figure 37 -Comparison done by Snyder [207] on citation metrics difference between Google Scholar, Web of Science publisher and the reality ................................................................................................................

Figure 38 -

 38 Figure 38 -The software quality model classification elements organized over five themes: id, bibliographic, definition, scope and structural ...................................................................................................................

Figure 39 -

 39 Figure 39 -Software Quality Model Clade based on a homology and five taxa...................................................

Figure 40 -

 40 Figure 40 -The 492 created and published software quality models per formalism and year wise ...................

Figure 41 -

 41 Figure 41 -The normalized citation numbers of the 492 published software quality model papers year wise ..

Figure 42 -

 42 Figure 42 -The normal distribution related to the number of quality models per study paper ..........................

Figure 43 -

 43 Figure 43 -Cumulative number of quality model citations in the 136 study papers year wise ...........................

Figure 44 -

 44 Figure 44 -The 35 most cited (i.e., cited more than 5 times) software quality models in the 136 study papers, order by chronologic order ..........................................................................................................................

Figure 45 -

 45 Figure 45 -DAP type distribution of the 492 software quality model samples ...................................................

Figure 46 -

 46 Figure 46 -Formalism type distribution of the 492 software quality model samples .........................................

Figure 47 -

 47 Figure 47 -Insight on the main prediction method distribution of the 492 software quality model samples ....

Figure 48 -

 48 Figure 48 -Main quality perspectives, distribution of the 492 software quality model samples ........................

Figure 49 -

 49 Figure 49 -Nuance in quality perspectives, distribution of the 492 software quality model samples ................

Figure 50 -

 50 Figure 50 -Scope distribution of the 492 software quality model samples .........................................................

Figure 51 -

 51 Figure 51 -Comparison between creation of new quality models and derived quality models over the 492 software quality model samples ..................................................................................................................

Figure 52 -

 52 Figure 52 -Frequencies of the 9 main issues identified by Thapar et al. [11], which impede the development and use of quality models...................................................................................................................................

Figure 53 -

 53 Figure 53 -The A-SPICE capability levels of process maturity [21] ......................................................................

Figure 54 -

 54 Figure 54 -Mapping of practical solutions against the 16 issues preventing development and use of quality models .........................................................................................................................................................

Figure 55 -Figure 56 -

 5556 Figure 55 -The software project scorecard including indicators and metrics for project, process, and product Figure 56 -Inheriting of the practical solutions to the operational issues, the "Quality Thermometer" process for project .........................................................................................................................................................

Figure 57 -

 57 Figure 57 -Visual display of quality level done via color coding associated to range decomposition coming from ISO/IEC 33020 [138] .....................................................................................................................................

Figure 58 -

 58 Figure 58 -Example of a 5 points Likert's scale ....................................................................................................

Figure 59 -

 59 Figure 59 -Example of a 6 points Likert's scale ....................................................................................................

Figure 60 - 6 -

 606 Figure 60 -6-stages process for quality model development ..............................................................................

Figure 61 -

 61 Figure 61 -Algorithm of our analysis based on Fleiss and Cohen's kappa ...........................................................

Figure 62 -

 62 Figure 62 -Over-The-Air adoption timeline of major automobile manufacturers (sources: [247], [248]) .........

Figure 63 -

 63 Figure 63 -Example of FOTA working principle with IVC, IVI and ADAS ECUs .....................................................

Figure 64 -

 64 Figure 64 -The ISO/IEC/IEEE 25010 quality models: "System / Software product quality model" and "Quality in use model" ...................................................................................................................................................

Figure 65 -

 65 Figure 65 -Example of a mapping between quality models, properties, measures and an automotive systems and software (source and inspiration from ISO/IEC/IEEE 25010 [23] & ISO/IEC 25030 [251]) ...................

Figure 66 -

 66 Figure 66 -Survey extract: participant role and project ......................................................................................

Figure 67 -

 67 Figure 67 -Survey extract: the ranking choice of quality characteristics .............................................................

Figure 68 -

 68 Figure 68 -Survey extract: the ranking choice of quality sub-characteristics ......................................................

Figure 69 -

 69 Figure 69 -Survey extract: the final open question of the survey .......................................................................

Figure 70 -

 70 Figure 70 -Response distribution per order of importance ranking ....................................................................

Figure 71 -

 71 Figure 71 -The resulting IVI embedded software quality model; numbers in parenthesis are characteristic /subcharacteristic weight factors .......................................................................................................................

Figure 72 -

 72 Figure 72 -The resulting IVC embedded software quality model; numbers in parenthesis are characteristic /subcharacteristic weight factors .......................................................................................................................

Figure 73 -

 73 Figure 73 -The resulting ADAS & FOTA embedded software quality model; numbers in parenthesis are characteristic /sub-characteristic weight factors ........................................................................................

Figure 74 -

 74 Figure 74 -The common quality model from which the other quality models are derived ................................

Figure 75 -

 75 Figure 75 -The secondary common quality model from which IVI and ADAS & FOTA quality models are derived .....................................................................................................................................................................

Figure 76 -

 76 Figure 76 -The polymorphism tree structure with the five polymorphic quality models ...................................

Figure 77 -

 77 Figure 77 -Genetic-Quality analogy: global overview with chromosome, chromatids, and DNA sequence (genetic terminology is in green, quality terminology is in purple) ...........................................................................

Figure 78 -

 78 Figure 78 -Genetic-Quality analogy: detail overview with locus, sites, genes and alleles (genetic terminology is in green, quality terminology is in purple) ......................................................................................................

Figure 79 -

 79 Figure 79 -Genetic-Quality analogy: an example based on ISO/IEC/IEEE 25010 quality models ........................

Figure 80 -

 80 Figure 80 -Genetic-Quality analogy: Meta-model ontology ................................................................................

Figure 81 -

 81 Figure 81 -The seven steps in the software quality model genome meta-model construction algorithm .........

Figure 82 -

 82 Figure 82 -Example of "Supportability" chromatid computation with the main gene and some of its sub-genes (testability, adaptability, maintainability, changeability and reusability); sub-genes are site names with color background .........................................................................................................................................

Figure 83 -

 83 Figure 83 -Example from the online Semantic Atlas: three synonym constellations for "efficiency" word as quality characteristic (source: http://www.atlas-semantiques.eu/) ...........................................................

Figure 84 -

 84 Figure 84 -Links between the 43 genes; chromatids are identified by and dotted arrows indicate a relationship link that requires to take into account parent gene context ..................................................

Figure 85 -

 85 Figure 85 -From the 43 linked genes to a specific the chromatid linked genes and the corresponding site sequence......................................................................................................................................................

Figure 86 -

 86 Figure 86 -The 27 genes of "General utility" chromatid, with their respective links ..........................................

Figure 87 -

 87 Figure 87 -"General utility" chromatid quality characteristic sequence with its 27 genes and the 114 sites, including sites likelihood .............................................................................................................................

Figure 88 -

 88 Figure 88 -The 17 genes of "Product operation" chromatid, with their respective links ....................................

Figure 89 -

 89 Figure 89 -"Product operation" chromatid quality characteristic sequence with its 17 genes and the 85 sites, including sites likelihood .............................................................................................................................

Figure 90 -

 90 Figure 90 -The 15 genes of "Product revision" chromatid, with their respective links .......................................

Figure 91 -

 91 Figure 91 -"Product revision" chromatid quality characteristic sequence with its 15 genes and the 96 sites, including sites likelihood .............................................................................................................................

Figure 92 -

 92 Figure 92 -The 10 genes of "Product transition" chromatid, with their respective links ....................................

Figure 93 -

 93 Figure 93 -"Product transition" chromatid quality characteristic sequence with its 10 genes and the 43 sites, including sites likelihood .............................................................................................................................

Figure 94 -

 94 Figure 94 -The 15 genes of "Supportability" chromatid, with their respective links ...........................................

Figure 95 -

 95 Figure 95 -"Supportability" chromatid quality characteristic sequence with its 15 genes and the 72 sites, including sites likelihood .............................................................................................................................

Figure 96 -

 96 Figure 96 -The 11 genes of "Product in use" chromatid, with their respective links ..........................................

Figure 97 -

 97 Figure 97 -"Product in use" chromatid quality characteristic sequence with its 11 genes and the 34 sites, including sites likelihood .............................................................................................................................

Figure 98 -

 98 Figure 98 -The 32 genes of "Software product" chromatid, with their respective links .....................................

Figure 99 -

 99 Figure 99 -"Software product" chromatid quality characteristic sequence with its 32 genes and the 195 sites, including sites likelihood (part 1 of 2) .........................................................................................................

Figure 100 -

 100 Figure 100 -"Software product" chromatid quality characteristic sequence with its 32 genes and the 195 sites, including sites likelihood (part 2 of 2) .........................................................................................................

Figure 101 -

 101 Figure 101 -Overview of the software quality model genome meta-model composed of 7 chromatids ...........

Figure 102 -

 102 Figure 102 -General synthesis of the thesis research work and achievements ..................................................

Figure 103 -

 103 Figure 103 -Synthèse générale des travaux de recherche et des réalisations de la thèse ..................................

Figure 1 -

 1 Figure 1 -Dissertation chapters mapped against the 4 research questions Consequently, the detailed plan of this document, shown in TABLE1, follows our logical reasoning, starting with introductory statements, detailing context, problems and our research approach, then addressing sequentially each research question, before ending with our closure statements, summarizing the research achievements, contributions and the opening research topics.

Figure 2 -

 2 Figure 2 -Growing ECUs number per year for luxury cars (source Crolla et al. [15], chapter 14, figure 2)

Figure 3 -

 3 Figure 3 -Embedded systems in contrast to other computing systems (source Lepistös [35])

Figure 6 -

 6 Figure 6 -Overview of our early research flow breakdown

Figure 7 -

 7 Figure 7 -Overview of our research flow breakdown: in red the technological locks, in green our contributions

Figure 8 -

 8 Figure 8 -Kano's model of perceived quality[58]

Figure 9 -

 9 Figure 9 -Ontology of main quality keywords e. Lagging and leading indicators

Figure 10 -

 10 Figure 10 -Leading indicator composition, Roedler et al. [106]

Figure 11 -

 11 Figure 11 -The DAP classification introduced by Deissenboeck et al. 2009[START_REF] Deissenboeck | Software quality models: Purposes, usage scenarios and requirements[END_REF]

Figure 12 - 23 ]Figure 13 -Figure 14 -

 12231314 Figure 12 -Example of a definition model: the ISO/IEC/IEEE 25010 System / Software product quality model [23]

Figure 15 -

 15 Figure15-Illustration of the concept of reliability and validity of measures[START_REF] De Souza | Psychometric properties in instruments evaluation of reliability and validity[END_REF]

Figure 16 -

 16 Figure 16 -The Chrisman's scales [142], an extension of the Stevens' scales (in boldface)

Figure 17 -

 17 Figure 17 -"Continuous Preference Logic" operators and corresponding weighted power mean parameter used in the "Logic Scoring of Preference" of Dujmovic and Bayucan [144]

Figure 18 -

 18 Figure 18 -Threshold for risk acceptance and opportunities: from ISO/IEC 25022 [140] theory to practice

Figure 19 -

 19 Figure 19 -Timeline of the key contributions to quality modeling of software

- 2001 -

 2001 McGarry et al. practical software measurement[START_REF] Mcgarry | Practical Software Measurement: Objective Information for Decision Makers[END_REF]: McGarry et al. constructed a practical guide related to measure and based on the experience. They describe a measurement information model to help on the definition and implementation of metrics, and a measurement process to guide the planning and execution of measurement activities. In addition, in both model and process, they included management concepts to ensure that the requirements and rationales for decision makers were took into account in their definition (e.g., measurable concept, reason behind the metric) and risk was reduced during their implementation. At last, this body of knowledge, on practical software measurement, highlighted the importance of user feedback in measurement design. Hence, this contribution is a key support material about measurement for software quality modeling. -2010 -Abran's software metrics and software metrology [136]: In the same line of work of McGarry et al., Abran not only analyzed knowledge about the most popular software measures in the industry (e.g.,

TABLE 8 - 3 Preference 7 Division by an equal characteristic 8 Functional orientation of property statements 9 Necessary

 83789 GENERAL RULES FOR QUALITY MODEL TREE DERIVATION until only simple or quasi-simple properties remain at its top tier Indifference of Properties in a Group 4 Exhaustive consideration of the Application features of an object 5 Exclusion of reliability 10 properties (i.e., this must be part of integrated quality index => Kuse) 6 Structural rigidity of the primary tiers of a tree General sub-tree rules and sufficient number of properties in a group 10 Reference number of purpose properties within a groupTABLE 9 -SPECIFIC RULES FOR QUALITY MODEL TREE DERIVATION Id Rule Specific rules for the application of the expert method to weight factor 11 Random order of properties in a group 12 Minimum number of properties in a group (maximum ≤ 9 )

Figure 20 -

 20 Figure 20 -The "House of Qualimetry" and its 6 pillars

Figure 21 -

 21 Figure 21 -The "House of Qualimetry" and its 6 pillars in a nutshell

Figure 22 -

 22 Figure 22 -Example of a result from the application of the two of the three pillars of quality model architrave against "software product": the ISO/IEC/IEEE 25010 quality models [23].

Figure 24 -

 24 Figure 24 -Polymorphism mechanism showcased with butterfly analogy: example of temporal evolution (source: [171])

Figure 25 -

 25 Figure 25 -Example of some differences between ISO/IEC 9126 & ISO/IEC/IEEE 25010 d. Measurement process As we detailed with qualimetry in sections Chapter IV.5 and Chapter IV.6.a, a proper quality model is one side of the quality quantification problem. The other side concerns the measurement of the quality characteristics and especially all measurement process activities.

Figure 26 -Figure 27 -

 2627 Figure 26 -Software product quality evaluation process defined by ISO/IEC 25040 [147]

Figure 30 -Figure 31 -

 3031 Figure 30 -Ratio of the two types (i.e., conference and journal) of selected studies

Figure 32 -Figure 33 -

 3233 Figure 32 -Selected conference and journal paper distribution over publication year

Figure 34 -

 34 Figure 34 -Distributions of studies per type of studyNow that we have learnt the tendency of the different kinds of study that these 136 papers cover, our next step is to find which classification or comparison criteria elements have been used. So, similarly to the type of study analysis, we extracted from the papers, 114 distinct criteria combinations that we clustered into 8 unique criteria categories (see TABLE19):

Figure 35 -Figure 36 -

 3536 Figure 35 -Basic vs. Tailored quality model categorization (source: Thapar et al. [11])

Figure 37 -

 37 Figure 37 -Comparison done by Snyder [209] on citation metrics difference between Google Scholar, Web of Science publisher and the reality Fourth, scope theme describes the quality model focus and its applicability areas. Thus, we have: -the quality perspective as introduced by Garvin [101] ((i.e., user, product, manufacturer, transcendental, and value-based), -the quality focus likes in Kläs et al.'s CQML [10] (e.g., defects, maturity, general, functionality), -the quality model domain used also by Fath-Allah et al. [208] (e.g., IT, transportation, medical, socioeconomic), -the quality model context (i.e., academic, industrial, or both) that we can find also in Yan et al. [206], -the object of interest (e.g., web-service (Oriol et al. [12]), open-source (Adewumi et al. [189], Petrinja et al. [207])) -the stakeholder viewpoint, identical to Horgan et al.'s essential views [84], and which we retrieved in the Kläs et al. 's CQML classification scheme [10] (e.g., sponsors, users, developers).

Figure 38 -

 38 Figure 38 -The software quality model classification elements organized over five themes: id, bibliographic, definition, scope and structural 4. Contributions a. Cladistic as Classification Method of Software Quality Models

Figure 39 -

 39 Figure 39 -Software Quality Model Clade based on a homology and five taxa b. The first list of 492 software quality models Research Sub-question 2a Considering software scope, what is the set of existing quality models?

of Annex 6 .

 6 We remark that 2008, 2009 and 2011 are the most productive years in term of software quality model creation and publication. Moreover, the last two decades concentrate 71.95% of the quality model production, particularly during the range 2000-2009 which represents 40.65% of the quality model production over the 51 years period. The details are shown in Figure 40. Note, the hierarchical formalism the most frequent one.Regarding the associated normalized citation counters (i.e., for each year, this is the number of citation of the quality model reference papers per the number of quality models for that year), Figure41highlights the fact that despite a low software quality model production rate before 2000, this period contains the most cited contributions, even with a peak due to two widely cited contribution in 1984: Kano et al.'s quality model[58],and Basili and Weiss GQM quality model.[START_REF] Basili | A Methodology for Collecting Valid Software Engineering Data[END_REF].

Figure 40 -Figure 41 -

 4041 Figure 40 -The 492 created and published software quality models per formalism and year wise

Figure 42 -Figure 43 -Figure 44 -

 424344 Figure 42 -The normal distribution related to the number of quality models per study paper

Figure 45 -Figure 46 -

 4546 Figure 45 -DAP type distribution of the 492 software quality model samples

Figure 47 -Figure 48 -

 4748 Figure 47 -Insight on the main prediction method distribution of the 492 software quality model samples

Figure 49 -Figure 50 -

 4950 Figure 49 -Nuance in quality perspectives, distribution of the 492 software quality model samples

Figure 51 -

 51 Figure 51 -Comparison between creation of new quality models and derived quality models over the 492 software quality model samples

Figure 53 -

 53 Figure53-The A-SPICE capability levels of process maturity[START_REF]13 / Automotive SIG[END_REF]

Figure 54 -

 54 Figure 54 -Mapping of practical solutions against the 16 issues preventing development and use of quality models

Figure 55 -

 55 Figure 55 -The software project scorecard including indicators and metrics for project, process, and product Thus, the main project scorecard blocks are:  Project id: main identification details about the project, the current milestone or also its safety, or Automotive Safety Integrity Level (ASIL) since we are working in the automotive domain (in aeronautics we could use the Design Assurance Level (DAL).

Figure 57 -

 57 Figure 57 -Visual display of quality level done via color coding associated to range decomposition coming from ISO/IEC 33020 [141]

Figure 58 -

 58 Figure 58 -Example of a 5 points Likert's scale

Figure 59 -

 59 Figure 59 -Example of a 6 points Likert's scale

Figure 61 -

 61 Figure 61 -Algorithm of our analysis based on Fleiss and Cohen's kappa 5.Construct the quality models: Once the data set had been selected, to build our polymorphic quality models we determined the most important characteristics and sub-characteristics using the scale set value set: {-2; -1; 0; +1; +2}.

Figure 62 -

 62 Figure 62 -Over-The-Air adoption timeline of major automobile manufacturers (sources: [251], [252])

Figure 63 :

 63 we have the corrective or evolving packages, generated on the company servers, that are received wirelessly thought the IVC ECU and dispatched to the target ECUs. Nevertheless, there are many challenges when developing and using FOTA for automotive systems: o The multiplicity of system, software architectures and configuration to manage remotely, o The multiple geographical locations to consider, with their own regulations, network capability and infrastructures, o The compliance on cybersecurity, safety, and regulation requirements to achieve, o The size of data to be transferred: from kilobytes to gigabytes.

Figure 63 -

 63 Figure 63 -Example of FOTA working principle with IVC, IVI and ADAS ECUs

Figure 64 -

 64 Figure 64 -The ISO/IEC/IEEE 25010 quality models: "System / Software product quality model" and "Quality in use model"

Figure 65 -

 65 Figure 65 -Example of a mapping between quality models, properties, measures and an automotive systems and software (source and inspiration from ISO/IEC/IEEE 25010 [23] & ISO/IEC 25030 [254])

Figure 67 -

 67 Figure 67 -Survey extract: the ranking choice of quality characteristics The survey's third part also focused on importance ranking, but rather on all the 40 ISO/IEC 25010 quality subcharacteristics. Their definitions are also documented and accessible to any participants. Moreover, we introduced a sixth choice in this part to give flexibility to the participant as explained in Chapter VI.4.b (see Figure 69).

Figure 68 -

 68 Figure 68 -Survey extract: the ranking choice of quality sub-characteristics Last part of the survey proposes to the participants to share their vision on quality and complete these quality models through an open question shown in Figure 69.

Figure 69 -

 69 Figure 69 -Survey extract: the final open question of the survey

Figure 70 -

 70 Figure 70 -Response distribution per order of importance ranking Next, we verified the participant agreement consensus, or inter-rater reliability, based on Cohen's kappa ⲕ [243] in the case of two raters (in this case the stakeholders who contributed to our survey) for a set of answers and Fleiss' kappa ⲕ [244] for three or more raters. The following calculations exemplify both kappa computations on two sample sets extracted from the survey responses. -Cohen's kappa ⲕ example: The sample set for this example is obtained for the quality characteristics of

  75 + 1.25 + 1.5 + 1.5 + 1.048611 = 𝟕. 𝟎𝟒𝟖𝟔𝟏𝟏𝟏𝟏 ∑ = 1.5 + 1.25 = 𝟐. 𝟕𝟓 And the resulting weight factors: Annex 11 contains the importance values, weight factors and the quality characteristic and sub-characteristic selection results for our real-world. b. Three quality models for four sub-systems: IVI, IVC, ADAS & FOTA Based on the work done in the previous step with importance values and weight factors, we know what the characteristics and sub-characteristics we have to keep.

  Therefore, the IVI embedded software quality model is represented jointly with its weight factors by Figure71, and is composed of two quality perspectives, 9 quality characteristics and 24 quality sub-characteristics. This model covers 69.23% of the quality characteristics of the selected reference quality model (i.e., ISO/IEC/IEEE 25010) and 57.14% of its quality sub-characteristics. The most important quality characteristics highlighted by this IVI quality model are functional suitability, reliability and customer satisfaction.

Figure 71 -

 71 Figure 71 -The resulting IVI embedded software quality model; numbers in parenthesis are characteristic /sub-characteristic weight factors

Figure 72 -

 72 Figure 72 -The resulting IVC embedded software quality model; numbers in parenthesis are characteristic /sub-characteristic weight factors The last embedded software quality model illustrated by Figure 73 covers both ADAS and FOTA embedded software quality model. Indeed, the analysis results for ADAS and FOTA showed rater consensus at higher level of project scope rather than specifically to either FOTA or ADAS, where rater agreements were slight or poor. This hierarchical quality model is organized over two quality perspectives, 10 quality characteristics and 25 quality sub-characteristics. Consequently, this model uses 76.92% of quality characteristics and 59.52% of quality subcharacteristics of ISO/IEC/IEEE 25010. Functional suitability, performance efficiency, effectiveness, and efficiency are the most critical quality characteristics to consider for ADAS ECU and FOTA.

Figure 73 -

 73 Figure 73 -The resulting ADAS & FOTA embedded software quality model; numbers in parenthesis are characteristic /sub-characteristic weight factors

Figure 74 -Figure 75 -Figure 76 -

 747576 Figure 74 -The common quality model from which the other quality models are derived

Figure 77 -

 77 Figure 77 -Genetic-Quality analogy: global overview with chromosome, chromatids, and DNA sequence (genetic terminology is in green, quality terminology is in purple)

Figure 79 -

 79 Figure 79 -Genetic-Quality analogy: an example based on ISO/IEC/IEEE 25010 quality models

Figure 80 -

 80 Figure 80 -Genetic-Quality analogy: Meta-model ontology

Figure 81 -

 81 Figure 81 -The seven steps in the software quality model genome meta-model construction algorithm

  a. the merge of the two sub-genes, summing their probability values, if the gene joint is possible without breaking any other existing gene joint, b. otherwise use site variation to name them and refactor each sub-gene into distinct sites, During merge, or refactor, we must try to reduce the number of sites by A-Merge identical sites including their probability value, to reduce redundancy, B-Integrate sites not belonging to sub-genes into sub-genes, whenever it is possible, C-Optimize merge between sub-genes by maximizing sub-gene overlap, reducing the number of sites through consideration to name variations (e.g., "simplicity" with "self-descriptness") Moreover, if a quality characteristic loop exists (e.g., quality characteristic A point to quality characteristic B, which point back to characteristic A), we must use chromatid context and common sense to remove the most irrelevant relationship. The Chapter VIII.4 and Chapter VIII.5 illustrate in details against the first metal-model construction that methodology.

Figure 82 -

 82 Figure 82 -Example of "Supportability" chromatid computation with the main gene and some of its sub-genes (testability, adaptability, maintainability, changeability and reusability); sub-genes are site names with color background

#

  Give a last chance through a test checking the definition of characteristics Tgt_Grp If Not (Ref_Grp can be mapped to Tgt_Grp based on their definitions and subcharacteristics) Distance  infinity Return Distance # Create a graph node for the i-th element of Grp: the current element i, with children list, if any Get Graph Node for i -th element in Grp Grp_Element  Grp [i] Grp_Children  empty # crawl all Grp elements to verify which ones are sub-characteristics of the current element Loop: j from 1 to size (Grp) if (j != i) and (Grp[j] is in sub-characteristics of Grp_Element) Grp_Children  push j End loop Return (i, Grp_Children)

Figure 84 -

 84 Figure 84 -Links between the 43 genes; chromatids are identified by and dotted arrows indicate a relationship link that requires to take into account parent gene context

Figure 85 -

 85 Figure 85 -From the 43 linked genes to a specific the chromatid linked genes and the corresponding site sequence So, the final expected result for each chromatid is a complete detailed sequence of sites with their likelihood values and the localization of each gene, and any sub-sequent levels of genes, composing the chromatid.

Figure 86 -

 86 Figure 86 -The 27 genes of "General utility" chromatid, with their respective links

Figure 88 -

 88 Figure 88 -The 17 genes of "Product operation" chromatid, with their respective links

Figure 92 -

 92 Figure 92 -The 10 genes of "Product transition" chromatid, with their respective links

Figure 94 -

 94 Figure 94 -The 15 genes of "Supportability" chromatid, with their respective links

Figure 96 -

 96 Figure 96 -The 11 genes of "Product in use" chromatid, with their respective links

Figure 98 -

 98 Figure 98 -The 32 genes of "Software product" chromatid, with their respective links

Figure 101 -

 101 Figure 101 -Overview of the software quality model genome meta-model composed of 7 chromatids

Figure 102 -

 102 Figure 102 -General synthesis of the thesis research work and achievements

oooo

  Systematic literature review with snowballing resulting in a unique list of 492 software quality model, Software quality model classification based on cladistic, o Software quality model landscape and correction of Thapar et al.'s postulate, o From theory to practice analysis resulting in 16 operationalization challenges and their practical solutions, The processes to develop and use quality model, encapsulating practical solutions, and exemplified against an example from automotive, Polymorphism quality model in practice, Software quality model genome meta-model, including the construction algorithm, and the 7 chromatids of the first meta-model result.

Finally, the lastooo

  set of problem encompasses modeling of contemporary quality area of interest and where some research studies have been initiated but not yet solved the problem. We can cite for example: Software greenness and sustainability, Software aging and obsolescence, Quality data for connected software systems and services.AnnexesAnnex 1. Catalog of the main existing aggregation operators (from Chapter IV.3.d)

Mode 1

 1 𝑀 (𝑥 , 𝑥 , … , 𝑥 ) = 𝑥 𝑎𝑛𝑑 𝑛 = max(𝑛 , 𝑛 , … , 𝑛 ) where (𝑛 , 𝑛 , … , 𝑛 ) are frequencies of(𝑥 , 𝑥 , … , 𝑥 ) Median 𝑀 . (𝑥 , 𝑥 , … , 𝑥 ) 𝑀 . (𝑥 , 𝑥 , … , 𝑥 )| Minimum ∀𝑥 , 𝑦 ≤ 𝑥 ⟹ 𝑦 = min(𝑥 , 𝑥 , … , 𝑥 ) Maximum ∀𝑥 , 𝑦 ≥ 𝑥 ⟹ 𝑦 = max(𝑥 , 𝑥 , … , 𝑥 ) 𝑤 ( ) -𝑤 ( ) ). min (𝑥 ( ) , … , 𝑥 ( ) where 𝑤 ≥ 0 , 𝑤 = 1 𝜎 ( ) and 𝑤 ( ) ≥ 𝑤 ( ) ≥ ⋯ ≥ 𝑤 ( ) , 𝑤 ( 𝑤 ( ) -𝑤 ( ) ). max (𝑥 ( ) , … , 𝑥 ( ) where 𝑤 ≥ 0 , 𝑤 = 1 𝜎 ( ) 𝑎𝑛𝑑 𝑤 ( ) ≥ 𝑤 ( ) ≥ ⋯ ≥ 𝑤 ( ) , 𝑤 ( ) = 0 Range 𝑅(𝑥 , 𝑥 , … , 𝑥 ) = max(𝑥 , 𝑥 , … , 𝑥 ) -min(𝑥 , 𝑥 , … , 𝑥 ) Variance 𝜎 = 𝑆 (𝑥 , 𝑥 , … , 𝑥 ) sum 𝑆(𝑥 , 𝑥 , … , 𝑥 ) = 𝑓 𝑓(𝑥 ) where 𝑓 is a strictly monotone continuous function and 𝑓(𝑥) + 𝑓(1 -𝑥) 𝑤 ( ) ≤ 𝑤 ( ) ≤ ⋯ ≤ 𝑤 ( ) Degree of maxness (or orness) 𝑚𝑎𝑥𝑛𝑒𝑠𝑠(𝑤 , 𝑤 , … , 𝑤 ) = 𝑤 . 𝑖 -1 𝑛 -Degree of dispersion 𝑑𝑖𝑠𝑝𝑒𝑟𝑠𝑖𝑜𝑛(𝑤 , 𝑤 , … , 𝑤 ) = -𝑤 . ln(𝑤 ) , 𝑥 , … , 𝑥 ) = max min 𝑥 ( ) , 𝜇 𝐶 ( ) where 𝑥 ( ) ≤ 𝑥 ( ) ≤ ⋯ ≤ 𝑥 ( ) Discrete Choquet Integrals 𝐶ℎ𝑜𝑞𝑢𝑒𝑡 (𝑥 , 𝑥 , … , 𝑥 ) = 𝑥 ( ) -𝑥 ( ) . 𝜇 𝐶 ( ) where 𝑥 ( ) = 0 and 𝑥 ( ) ≤ 𝑥 ( ) ≤ ⋯ ≤ 𝑥 ( ) Fusion operators Bayesian fusion 𝑃(𝑥|𝑥 , 𝑥 ) = 𝑃(𝑥 |𝑥). 𝑃(𝑥 |𝑥). 𝑃(𝑥) 𝑃(𝑥 , 𝑥 ) Probabilistic fusion: prioritized conjunction 𝜋 ∧ = min 𝜋 , max 𝜋 , 1 -ℎ(𝜋 , 𝜋 ) where ℎ(𝜋 , 𝜋 ) = 𝑠𝑢𝑝 min(𝜋 (𝑥), 𝜋 (𝑥)) Probabilistic fusion: prioritized disjunction 𝜋 ∨ = max 𝜋 , min 𝜋 , ℎ(𝜋 , 𝜋 ) where ℎ(𝜋 , 𝜋 ) = 𝑠𝑢𝑝 min(𝜋 (𝑥), 𝜋 (𝑥)) 𝑥 , … , 𝑥 ) = 𝑇(𝑥 , … , 𝑥

Quality

  Sub-Characteristic DefinitionReplaceabilityDegree to which a product can replace another specified software product for the same purpose in the same environment.-Definitions of the quality in use model oQuality characteristic definitionsQuality Characteristic Definition Effectiveness Accuracy and completeness with which users achieve specified goals.

site 1

 1 Risk of software Risk of software = 100% site 2 Commercial risk in use Commercial risk in use = 100% site 3 Risk to the operation in use Risk to the operation in use = 100% site 4 Risk to the public in use Risk to the public in use = Synthèse de la Thèse en Français L'objectif de la thèse nous a conduit à définir un cadre théorique pour la supervision et le pilotage des processus d'ingénierie et de développement de produits par la qualité.

ooo

  aussi d'améliorer notre proposition par le retour d'expérience et la mesure de l'efficacité, ainsi que favoriser son appropriation et adoption, à travers une stratégie de déploiement en entreprise accompagné par de la formation et un outillage. Ce type de perspective se situe principalement entre le court et le moyen terme.Ainsi, une façon de réaliser le partage de l'information est de s'appuyer sur les médias littéraires. Nous avons prévu de rassembler les résultats et contributions suivants dans plusieurs documents de recherche : La revue systématique de la littérature avec un « effet boule de neige » aboutissant à une liste uniquede 492 modèles qualité des logiciels, o La classification de modèle qualité des logiciels basée sur la cladistique, o Le paysage du modèle qualité du logiciel et la correction du postulat de Thapar et al, o De la théorie à l'analyse de la pratique : 16 défis d'opérationnalisation et leurs solutions pratiques, o Les processus de développement et d'utilisation de modèle qualité, résumant des solutions pratiques et illustrées par un exemple tiré du secteur automobile, Le modèle qualité polymorphe en pratique, Le méta-modèle du génome du modèle qualité logicielle, y compris l'algorithme de construction, et les 7 chromatides du premier résultat du méta-modèle. En outre, et comme nous l'avons déjà évoqué dans le Chapitre V.5, un outil de portail en ligne doit être créé pour diffuser le partage de la collection de 492 modèles qualité des logiciels et permettre la collaboration pour leur utilisation, leur achèvement et leur maintenance. L'objectif est de permettre à la communauté universitaire et industrielle de collaborer sur cette collection et d'éviter que cette liste ne devienne obsolète dans les années à venir.

TABLE 1 -

 1 DETAILED DISSERTATION PLAN ..........................................................................................................................

TABLE 2 -

 2 SUMMARY OF STUDY KEY POINTS .....................................................................................................................

TABLE 3 -

 3 OUR FIRST THREE RESEARCH QUESTIONS .............................................................................................................

TABLE 4 -

 4 SUMMARY OF THE 4 SCALES OF MEASUREMENTS DEFINED BY S. S. STEVENS [139] .....................................................

TABLE 5 -

 5 NOMINAL SCALE-BASED RATING, ACCORDING TO ISO/IEC 33020 [136] ................................................................

TABLE 6 -

 6 ORDINAL SCALE-BASED RATING IN PERCENTAGE VALUES, ACCORDING TO ISO/IEC 33020 ...........................................

TABLE 7 -

 7 EXAMPLE OF KHADDAJ AND HORGAN [148] RELATIONSHIP CHART USED FOR POLARITY PROFILE ....................................

TABLE 8 -

 8 GENERAL RULES FOR QUALITY MODEL TREE DERIVATION ........................................................................................

TABLE 9 -

 9 SPECIFIC RULES FOR QUALITY MODEL TREE DERIVATION .........................................................................................

TABLE 10 -

 10 COMPARISON OF EIGHT MAIN DISTINCT APPROACHES SUPPORTING QUALITY MODEL DEVELOPMENT AND USE .................

TABLE 11 -

 11 EXAMPLE OF DISTINCT MEASURABLE OBJECTIVES USING FURPS QUALITY MODEL FOR EACH LIFE CYCLE PHASE (SOURCE: GRADY AND CASWELL [85], FIGURE 11-7,

PAGE 161)...............................................................................................

TABLE 12 -

 12 EXAMPLE OF LEXICAL AND SEMANTIC COMPARISON RESULT BETWEEN TWO CHARACTERISTICS FROM ISO/IEC 9126

AND ISO/IEC/IEEE 25010 ........................................................................................................................................ TABLE 13 -INTERMEDIATE CALCULATION LINKED TO PREVIOUS EXAMPLE ................................................................................ TABLE 14 -LIST OF ELECTRONIC DATABASE SOURCES .......................................................................................................... TABLE 15 -SEARCH QUERY ELEMENTS: KEYWORDS VS. OPERATORS VS. SEARCH FIELDS .............................................................. TABLE 16 -THE FIVE DIGITAL LIBRARY SEARCH QUERIES ...................................................................................................... TABLE 17 -INCLUSION AND EXCLUSION CRITERIA RELATED TO OUR SEARCH STRATEGY ............................................................... TABLE 18 -MAPPING OF THE 136 PAPER STUDIES AGAINST THE SEVEN TYPES OF STUDY ..........................................................

TABLE 19 -

 19 MAPPING OF THE 136 PAPERS STUDIES AGAINST THE EIGHT CATEGORIES OF CLASSIFICATION OR COMPARISON CRITERIA

TABLE 20 -

 20

ALPHABETICALLY SORTED LIST OF MAIN POTENTIAL ISSUES THAT CHALLENGE DEVELOPMENT AND USE OF QUALITY MODELS .....................................................................................................................................................................

TABLE 21 -KAPPA INTERPRETATION (SOURCE LANDIS AND KOCH [245]) ............................................................................. TABLE 22 -COMPARISON BETWEEN THE "QUALITY THERMOMETER" PROCESS AND THE CURRENT RELEVANT STANDARDS: ISO/IEC 250NN SERIES AND ISO/IEC/IEEE 15939 ...........................................................................................................

TABLE 23 -

 23 COMPARISON BETWEEN THE "6-STAGES" PROCESS AND THE CURRENT RELEVANT STANDARD: ISO/IEC 250NN SERIES AND ISO/IEC/IEEE 15939 ......................................................................................................................................

TABLE 24 -

 24 EXTRACTED NUMBER OF SURVEY RESPONSES PER QUALITY IN USE QUALITY CHARACTERISTICS AND IMPORTANCE FOR IVC AND PROJECT MANAGER ROLE .............................................................................................................................

TABLE 25 -

 25 THE ASSOCIATED RANKING RESPONSE MATCHES BETWEEN ALL POSSIBLE RESPONSE COMBINATIONS; THE GREEN CELLS INDICATE PERFECT MATCH ..................................................................................................................................

TABLE 26 -

 26 EXTRACTED NUMBER OF SURVEY RESPONSES PER SYSTEM/SOFTWARE PRODUCT QUALITY CHARACTERISTICS AND IMPORTANCE FOR IVC AND ANY ROLE ...................................................................................................................

TABLE 27 -

 27 SURVEY DATA ANALYSIS WITH COHEN Κ AND FLEISS Κ. COLORED CELLS HIGHLIGHT Κ BASED CHOICE FOR EACH ECU; GRAYED CELLS HIGHLIGHT AT LEAST MODERATE AGREEMENT .....................................................................................

TABLE 28 -

 28 GENETIC

-QUALITY ANALOGY: TERMINOLOGY SUMMARY ....................................................................................

TABLE 29 -ALLELE AND SITE EXAMPLE FOR "PORTABILITY" QUALITY CHARACTERISTIC .............................................................. TABLE 30 -GENE COMPUTATION EXAMPLE FOR "PORTABILITY" QUALITY CHARACTERISTIC .......................................................

TABLE 31 -

 31 OVERVIEW OF THE EIGHT SELECTED QUALITY MODELS FOR SOFTWARE QUALITY MODEL GENOME META-MODEL CONSTRUCTION ................................................................................................................................................

TABLE 32 -

 32 THE 55 DISTINCT QUALITY CHARACTERISTIC GROUPS FROM THE EIGHT SELECTED SOFTWARE QUALITY MODELS .............

TABLE 33 -

 33 THE 10 MERGED QUALITY CHARACTERISTIC GROUPS (GREEN BACKGROUND CELL INDICATES A MERGED ENTRY) .............

TABLE 34 -

 34 LIST OF THE 27 SINGLE QUALITY MODEL GENES ................................................................................................

TABLE 35 -

 35 LIST OF THE 16 MULTI-QUALITY MODELS GENES ...............................................................................................

TABLE 36 -

 36 THE RELATIONSHIP LINKS BETWEEN THE QUALITY CHARACTERISTIC GENES ..............................................................

TABLE 37 -

 37 EXAMPLE OF GENE 05 "MAINTAINABILITY" DETAILED WITH ALL ITS SUB-GENES AND SITES: ON THE LEFT SIDE, THE DIRECT DETAILED SITE SEQUENCE, ON THE RIGHT THE DETAILED AND OPTIMIZED SITE SEQUENCE WITH SAME QUALITY COVERAGE ......

TABLE 38 -

 38 DEGREE OF SUBJECTIVITY FOR EACH CHROMATID OF THE SOFTWARE QUALITY MODEL GENOME META-MODEL ..............

TABLE 39 -

 39 CATALOG OF THE MAIN EXISTING AGGREGATION OPERATORS, BASED ON DETYNIECKI [143], WAGNER [27] AND DUJMOVIC & BAYUCAN [144] ...........................................................................................................................................

TABLE 40 -

 40 EXAMPLES OF HAMMING'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS .................

TABLE 41 -

 41 EXAMPLES OF LEVENSHTEIN'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS ..............

TABLE 42 -

 42

EXAMPLES OF DAMERAU-LEVENSHTEIN'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS .....................................................................................................................................................................

TABLE 43 -

 43 EXAMPLES OF JARO'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS .........................

TABLE 44

 44

-EXAMPLES OF JARO-WINKLER'S DISTANCE COMPUTATION WITH L = 2: RED CHARACTERS INDICATE DIFFERING CHARACTERS ..................................................................................................................................................................... TABLE 45 -LIST OF FILTERED PUBLISHED STUDIES RESULTING THE SYSTEMATIC LITERATURE REVIEW ............................................ TABLE 46 -MAIN RAW RESULTS FROM THE SYSTEMATIC LITERATURE REVIEW ........................................................................ TABLE 47 -MAPPING SYSTEMATIC LITERATURE REVIEW STUDIES WITH QUALITY MODELS ......................................................... TABLE 48 -THE 492 DISTINCT QUALITY MODELS, FROM 1968 TO 2019 .............................................................................. TABLE 49 -DEFINITIONS OF THE DIFFERENT ELEMENTS OF THE PROJECT SCORECARD ............................................................... TABLE 50 -LIST OF THE 27 SINGLE-QUALITY MODEL GENES WITH THEIR POSSIBLE VARIATIONS .................................................. TABLE 51 -LIST OF THE 16 MULTI-QUALITY MODELS GENES WITH THEIR POSSIBLE VARIATIONS .................................................

TABLE 1

 1

, follows our logical reasoning, starting with introductory statements, detailing context, problems and our research approach, then addressing sequentially each research question, before ending with our closure statements, summarizing the research achievements, contributions and the opening research topics. Introductory statements •Chapter 1, Introduction •Chapter 2, Context and Problems •Chapter 3, Research Methodology

TABLE 1 -

 1 DETAILED DISSERTATION PLAN

	Introductory statements
	•Chapter 1, Introduction
	•This is the general dissertation introduction, introducing the main research problematic, questions,
	and dissertation plan.
	•Chapter 2, Context and Problems

•

  Chapter 4, Quality, Quality Modeling and Qualimetry •This chapter aims to understand what is behind quality, quality modeling (including characteristics/sub-characteristics, attributes, metrics; with few quality model example), qualimetry, what are the essential characters, introduce some new concepts such as polymorphism that extend quality modeling.

  Ahrens et al.[START_REF] Ahrens | Objective evaluation of software architectures in driver assistance systems[END_REF] (2013), of a quality model, including its metrics, for objective evaluation of automotive software architecture. To define such quality model, the authors relied on general reference models such as McCall, Boehm, and ISO / IEC 9126 (see Annex 13 for details about these quality models), as well as on some architecture quality characteristics. The result is a quality model dealing with operating conditions (i.

e., conformity, functionality, reliability), production costs (i.e., efficiency) and development costs (i.e., portability, mutability, reusability). Moreover, the authors proposed to close the gap between informal, qualitive evaluations methods, and objective quantitative measurement by comparing quality model result against a previous, or reference architecture result. Therefore, unlikely most of published contributions which have either quality model or metrics, Ahrens et al. 's contribution is an operational quality model that must be used to compare software architecture against a previous, or reference one to improve it, but unfortunately it cannot be used to assess automotive architecture quality.

TABLE 2 -

 2 SUMMARY OF STUDY KEY POINTS

	Id Reference	Year	Type of work stream	Embedded System Domain	Embedded Software Domain	Study relies specifically on
	S01	[72]		Reliability growth model creation or	Telecommunication systems Software module	Classification And Regression Trees
				use			algorithm
	S02	[61]		Quality characteristics or/and quality	Medical	Component-Based Software	-
				model identification		Development
	S03	[73]		Reliability growth model creation or	Telecommunication systems Software module	Case-based reasoning model
				use
	S04	[62]		Quality characteristics or/and quality	Wireless system	Digital Signal Processing	-
				model identification		software, Commercial Off-The-
						Shelf
	S05	[64]		Quality characteristics or/and quality	Vehicular (i.e., cars,	Component-Based Software	ISO/IEC 9126
				model identification	construction equipment	Development
					vehicles, heavy trucks, trains)
	S06	[78]		Miscellaneous related work: real-	Distributed real-time	Adaptive reflective middleware	-
				time utility measure to control	embedded systems	software
				system and use them as evidence for
				certification
	S07	[36]		Quality model creation or use	Generic	Embedded software	CQM model, ISO/IEC 9126 model
						component, Component-Off-
						The-Self; Component-Based
						Software Development
	S08	[66]		Quality characteristics or/and quality	Aerospace, Integrated	Commercial Off-The-Shelf	Multiple Independent Levels of
				model identification	modular avionics		Security/Safety, ARP4754, ARP4761,
							DO178B, model-based analysis
	S09	[39]		Quality model creation or use	Digital TV	Software component,	IEEE 1061, McCall model, Boehm
						Commercial Off-The-Shelf, open	model, ISO/IEC 9126 model
						source
	S10	[67]		Quality characteristics or/and quality	Generic	Architecture	Architecture Tradeoff Analysis Model
				model identification			(ATAM)
	S11	[79]		Miscellaneous related work:	Design of space system	Generic	-
				investigate the existence of
				relationship between metrics
	S12	[37]		Quality model creation or use	Generic	Embedded software	CQM (S07), ISO /IEC 25010 model,
						component	CMM
	S13	[43]		Quality model creation or use	Ambient intelligence systems Ad-hoc computer systems	-
	S14	[29]		Quality model creation or use	Generic	Lightweight embedded	DeLone & McLean success model
						software component
	S15	[45]		Quality model creation or use	Generic	Software component	S14, DeLone & McLean success model
	S16	[46]		Quality model creation or use	Generic	Software component	DeLone & McLean success model,
							ISO/IEC 9126 model
	S17	[80]		Miscellaneous related work: Creation	Automotive	Generic	ISO 26262
				of an approach to measure and
				assessment of model-based design
				software
	S18	[74]		Reliability growth model creation or	Voice over Internet Phone	Generic	Software Failure Tree Analysis,
				use			Software Failure Model and Effects
							Analysis
	S19	[48]		Quality model creation or use	Generic	Generic	Systematic literature review, S02, S05,
							McCall model, Boehm model, ISO/IEC
							9126 model, ISO/IEC 25010 model,
							SQUAD

TABLE 3 -

 3 OUR FIRST THREE RESEARCH QUESTIONS

	Research Question 1	Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials?
	Research Question 2	Considering the set of software quality models, how to identify and decide which quality model is the most suitable for embedded software?
	Research Question 3	Considering a quality model for a software product, how to operationalize it?

  Kano et al. summarized their findings over the Kano's model depicted in Figure8. The two orthogonal axes of this model are the level of customer expectation, or requirements implementation, versus the level of customer satisfaction or dissatisfaction. The main elements of the Kano's model are:

	Quality = Quality + Cost Effectiveness	(1)
	b. Perceived quality
	Recalling the knowledge quest of the ancient Greek philosophers, particularly the summary from Aristotle (i.e.,
	"The body is called white because it contains whiteness"), and the second point of IEEE Standard glossary
	definition of quality (i.e., "Customer or user needs or expectations"), we discern that quality is also a matter of
	perception or interpretation. On 1984, Kano et al. [58] investigated how customers perceive quality, and
	remarked that customer satisfaction and dissatisfaction are dissymmetrical. They noticed, for instance, that if the
	existence of a feature can create a customer satisfaction, its absence doesn't automatically create a
	dissatisfaction, and vice-versa.

TABLE 4 -

 4 SUMMARY

	Number of cases
	Mode
	Contingency correlation
	χ 2

OF THE 4 SCALES OF MEASUREMENTS DEFINED BY S. S. STEVENS [139] Scale Measure properties Mathematical operators Mathematical group structure Permissible Statistics Nominal Classification, membership =, ≠ Permutation group 𝒙 =f(𝒙) f(𝑥) means any one-toone substitution Grouping (unordered)

TABLE 5 -

 5

NOMINAL SCALE-BASED RATING, ACCORDING TO ISO/IEC 33020

[START_REF] Abran | Software Metrics and Software Metrology[END_REF]

TABLE 6 -

 6 ORDINAL SCALE-BASED RATING IN PERCENTAGE VALUES, ACCORDING TO ISO/IEC 33020

	Rating	Meaning	Range values
	N	Not achieved	0 to ≤ 15% achievement
	P	Partially achieved	> 15% to ≤ 50% achievement
	L	Largely achieved	> 50% to ≤ 85% achievement
	F	Fully achieved	> 85% to ≤ 100% achievement

TABLE 7 -

 7 EXAMPLE OF KHADDAJ AND HORGAN [148] RELATIONSHIP CHART USED FOR POLARITY PROFILE

							Criterion A
			Key Quality Factor (KQF)			Locally Defined Factor (LDF)
		U	S	CB	T	C	M	R	LDF1	…	LDFn
		Usability (U)
	Criterion B	KQF

  and McCall models. So, she identified the main quality drivers, or factors, fromBoehm and McCall [41]models. She finally limited to five main factors for user perspective and three for manufacturer one. Therefore, the quality perspective addressed by COQUAMO are the user and manufacturer ones.

-1987 the FURPS

[START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF]

& 1992 FURPS+

[START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF]

quality models: The FURPS

[START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF]

quality model was built by

[START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF]

. This model is interesting in that it is adapted to the different stages of a software life cycle without modifying its 5 characteristics (Functionality, Usability, Reliability, Performance, Supportability). This is achieved by fitting the measures used for each characteristic to the current stage. In 1992 a new model based on FURPS was created: FURPS+

[START_REF] Grady | Practical Software Metrics for Project Management and Process Improvement[END_REF]

. This new model is an extension of the original model, which specifies constraints with respect to design, implementation, interface and physical properties and conditions. Both models cover a user and a product quality perspective.

-1988 -Humphrey's maturity model [153]: Humphrey's maturity model is the first published maturity model, and it corresponds to a framework for process maturity. This model is key because it is the original foundation of many other maturity model likes the Capability Maturity Model (CMM) of the Software Engineering Institute (SEI) [154] on 1991, or Automotive-SPICE [21] on 2001, for example. This model represents the manufacturer quality view. -1990 -Khoshgoftaar and Munson model [124]: Khoshgoftaar and Munson model represents one of the earliest starting point of a long series of research studies and fruitful contributions lead by Khoshgoftaar on prediction of error-prone quality models for telecommunication systems software. During their researches, they explored a wide range of likelihood methods to elaborate their quality models. To cite few of them, they investigated regression analysis, discriminant analysis, neural network, logistic regression, classification tree, genetic algorithm, analogy-based classification, and Poisson regression. It appears that the resulting quality models were specific to their study case and therefore couldn't be reused as it for other type of systems or software. Obviously, they are focusing on manufacturer quality perspective. -1991-the ISO/IEC 9126 quality models [24]: The ISO/IEC 9126 quality model was first published in 1991, with a final release in 2001. This hierarchical quality model is the first standardized quality model. It got its inspiration from the Boehm and McCall models. ISO/IEC 9126 is a hierarchical quality model composed of two quality views (internal & external quality, quality-in-use), 10 quality characteristics and 27 quality sub-characteristics. The suggested metrics are provided by ISO/IEC TR 9126-2 (external metrics) [155], ISO/IEC TR 9126-3 (internal metrics) [156] and ISO/IEC TR 9126-4 (quality-in-use metrics) [146]. Its status as an official standard quality model made it a reference that is widely used as a basis for many quality models. Obviously, this quality model is based on a user and a product quality perspective.

TABLE 10 -

 10 COMPARISON OF EIGHT MAIN DISTINCT APPROACHES SUPPORTING QUALITY MODEL DEVELOPMENT AND USE

ISO / IEC 250nn [23], [121], [123], [145], [147], [164], [165]: Stream of Hierarchical Quality models (and standard)

TABLE 11 -

 11

			Investigation / Specification		Design		Implementation		Testing		Support
	Functional		# target uses to		% spec included in		% designs included		% features tested at		Known problem
			review spec or		design		in code		alpha sites		reports
			prototype		# changes to spec		# code changes		% user		Sales act. Reports
			% grade on report		due to design		due to omissions		documentation tested		(esp. lost sales)
			card from user		requirements		discovered		against product		User surveys
			% features		# users to review		% features		# target alpha		Internal HP user
			competitive with		change if needed		removed		customers		surveys
			other products				(reviewed by
			# interfaces with				original target
			existing products				user)
	Usability		# target uses to		% grade of deign as		% grade by other		# changes to product		# user
			review spec or		compared to		lab user		after alpha test		misunderstandings
			prototype		objectives		% grade by		% grade from usability
			% grade on		# changes to		product marketing		lab testing
			documentation		prototype manuals		documentation		% grade by test sites
			plan by target user		after review

EXAMPLE OF DISTINCT MEASURABLE OBJECTIVES USING FURPS QUALITY MODEL FOR EACH LIFE CYCLE PHASE (SOURCE: GRADY AND CASWELL [85], FIGURE 11-7, PAGE 161)

  Indeed, such tool helps us to compare and classify quality models, to estimate and explain what the impacts and consequences are to change, update or adapt current quality model or to apply one quality model instead of another one. Furthermore, the consequences are directly linked to what we aim to do with quality model. For instance, let say that a company is currently using ISO/IEC 9126 standard and decides to be compliant with latest available standard, which is ISO/IEC/IEEE 25010. Then this distance will help to understand and estimate:

  Gordieiev et al. determined that the distance between ISO/IEC/IEEE 25010 and ISO/IEC 9126 gives 73.75% of similarity.

	Another relevant contribution on semantic distance for quality model comparison is the research work done by
	Gordieiev et al. 's [92], [93]. They compared four quality models against a reference quality model, ISO/IEC/IEEE
	25010, by using a cumulative semantic matching-based distance (cf equation 4). The authors decomposed every

quality model into a set of model elements (i.e., characteristics and sub-characteristics) and a set of relationship between model element together. Then, the distance calculation is an aggregation of the semantic match result of each quality characteristic, and their sub-characteristics, with the ISO/IEC/IEEE 25010 ones. For example,

  equation 7) to get normed distance result (i.e., between 0 and 1).At last, to validate that our contribution to quality model distance formula is practicable and give right result, we decided to apply it against ISO/IEC 9126 and ISO/IEC/IEEE 25010 because we have a reference result with Gordieiev et al. So, we assume that we have two quality model sequences and the frequency of each sequence are equal (i.e., 50 % for ISO/IEC 9126 and 50 % for ISO/IEC/IEEE 25010). We identified: 52 distinct subcharacteristics, 31 unique, 8 similar (i.e., close but not identical: for instance, "Modifiability" versus "Changeability"), and 13 identical. This lands us to calculate, with equations 6 and 7, a diversity degree of 67.31% (cf. Annex 3 for the computation details) which also indicates that ISO/IEC 9126 and ISO/IEC/IEEE 25010 are similar12 at 32.69%. We remark that distance result differs from Gordieiev et al. who found 73.75% of similarity between these two models. The main reason why the results diverge is because of Gordieiev et al. assumed that weight of characteristic similarities and the one of sub-characteristics similarities are equal, which is not our assumption. The example in Table

TABLE 12 -

 12 EXAMPLE OF LEXICAL AND SEMANTIC COMPARISON RESULT BETWEEN TWO CHARACTERISTICS FROM ISO/IEC 9126 AND

	ISO/IEC/IEEE 25010
	ISO/IEC/IEEE 25010 (Sequence 1)	ISO/IEC 9126 (Sequence 2)
	Functional suitability
	Functional completeness	Functionality Compliance	Similar
	Functional correctness	Accuracy	Similar
	Functional appropriateness	Suitability	Similar
	-	Security	Gap
	-	Interoperability	Gap

TABLE 13 -

 13 INTERMEDIATE CALCULATION LINKED TO PREVIOUS EXAMPLE

	Gap	Similar		Identical	Total	πij
	Functional suitability	2	3	0	5	0.7000
	Maintainability	4	1	2	7	0.6429
	SUM	6	4	2	12	0.6667

  who proposed an extension of the original Stevens' scale list in 1998. That extended typology is interesting since it is not diverging from Stevens' one and includes therefore 6 new scale types that are useful for some particular cases (e.g., gradation of membership is key in fuzzy set theory, log interval is widely use in stock market graphics, cyclical ratio is suitable for times and angles …).

		0, +∞[	∀𝑥, 𝑦, 𝑧 ∈ 𝑋
	𝑑(𝑥, 𝑦) ≥ 0	𝐍𝐨𝐧 -𝐧𝐞𝐠𝐚𝐭𝐢𝐯𝐢𝐭𝐲	(8)
	𝑑(𝑥, 𝑦) = 0 ⟺ 𝑥 = 𝑦	𝐈𝐝𝐞𝐧𝐭𝐢𝐭𝐲 𝐨𝐟 𝐢𝐧𝐝𝐢𝐬𝐜𝐞𝐫𝐧𝐢𝐛𝐥𝐞	(9)
	𝑑(𝑥, 𝑦) = 𝑑(𝑦, 𝑥)	𝐒𝐲𝐦𝐦𝐞𝐭𝐫𝐲	(10)

Regarding our quality model distance formula based on Nei and Li's formula

[START_REF] Nei | Mathematical model for studying genetic variation in terms of restriction endonucleases[END_REF]

, we assumed implicitly that our distance, described by equations 6 and 7, is a metric from a mathematical point of view. To demonstrate this fact, a distance is a metric (i.e., in the metric space) if it respects the mathematical properties shown in 8, 9, 10 and 11

[183]

. The non-negativity (

8

) is due to the fact that both equation 6 and 7 are positive equation (i.e., cannot provide any negative results). The identity of indiscernible (

9

) is directly derived from equation 7 which gives 0 only if x and y semantically match. Symmetry property

[START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF]

is verified since both matching and summation, composing equation 6 and 7 are commutative operations. About the subadditivity

(11)

, we won't demonstrate it, but our distance formula satisfies also that property. By satisfying these properties we can claim that our distance is a metrics in a mathematical sense. Conversely,

Gordieiev et

al. distance, equation 4, is not a metric. For instance, 𝑑 (𝑥, 𝑥) ≠ 0 which invalid identity of indiscernible property. 𝑜𝑛 𝑎 𝑠𝑒𝑡 𝑋, 𝑑 𝑖𝑠 𝑎 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑠𝑢𝑐ℎ: 𝑑: 𝑋 × 𝑋 → [𝑑(𝑥, 𝑦) ≤ 𝑑(𝑥, 𝑧) ⊥ 𝑑(𝑧, 𝑦) 𝐒𝐮𝐛𝐚𝐝𝐝𝐢𝐭𝐢𝐯𝐢𝐭𝐲 𝐨𝐫 𝐭𝐫𝐢𝐚𝐧𝐠𝐥𝐞 𝐢𝐧𝐞𝐪𝐮𝐚𝐥𝐢𝐭𝐲

  Question 2 Considering the set of software quality models, how to identify and decide which quality model is the most suitable for embedded software?Research Sub-question 2a Considering software scope, what is the set of existing quality models? During our early investigation on quality modeling and model, we noticed a plethora of existing systematic mappings or reviews with regards to the domain of software quality model. To name a few, we have for example, Kläs et al.[START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF] (2009) CQML classification scheme for comprehensive quality model landscape, R. S. Jamwal and D. Jamwal[START_REF] Jamwal | Issues & Factors For Evaluation of Software Quality Models[END_REF] (2009) with their 9-steps approach to evaluate software quality models, the comparative studies on software quality models done by AL-Badareen et al.[START_REF] Al-Badareen | Software Quality Models: A Comparative Study[END_REF] (2011) and Ahmad et al.[START_REF] Ahmad | A Comparative Study of Software Quality Models[END_REF] (2013), Thapar et al.[11] (2012) on the challenges linked to the development of standard software quality models, Polillo[START_REF] Polillo | Quality Models for Web [2.0Sites: A Methodological Approach and a Proposal[END_REF] (2012) about quality models for web 2.0 sites, the reviews of Adewumi et al.

	Research Sub-question 4a	Is it possible to have a unique reference quality model for software product, or instead should we have a meta-model?

This chapter is organized around the refinement of research question 2 as followed.

As we highlighted in Chapter I and Chapter III, our research strategy aims to take advantage of existing quality models, selecting the most suitable quality model, if any, for automotive embedded software, instead of creating a new model. So, considering software scope, which encompasses embedded software one, we must retrieve and enumerate the published software quality models thanks to a systematic literature review, addressing consequently our research sub-question 2a: "considering software scope, what is the set of existing quality models?" Once this enumeration done, we must classify these software quality models, preparing the set of models to be used for quality model selection purpose, for instance. So, the classification requires to identify the right methodology and criteria which we rephrase into the research sub-question 2b: "Considering a set of quality models, how to classify these quality models, what are the methodology, the criteria, and the characteristics to use?".

Finally, the classified collection of software quality model together with the classification criteria foster the elaboration of a quality model landscape complementary to Kläs et al. one

[START_REF] Kläs | CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes[END_REF]

, expanding the knowledge on the current research contributions done on software quality models, and supporting the selection of the most appropriate quality model for embedded software in automotive (i.e., research sub-question 2d).

2. Systematic literature review on software quality model

[START_REF] Adewumi | A Review of Models for Evaluating Quality in Open Source Software[END_REF]

(2013) and

[START_REF] Adewumi | A systematic literature review of open source software quality assessment models[END_REF]

(2016) on software quality models within the context of open source software, Oriol et al.

[START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF]

(2014) mapping of quality models for web-services, Miguel et al.

[START_REF] Miguel | A Review of Software Quality Models for the Evaluation of Software Products[END_REF]

(2014) review on software product quality models, Gordieiev et al.

[92] (2014)

  Kläs et al. [10] classified 22 quality models, from a publication range from 1978 to 2009, Thapar et al.[11] analyzed a reference set of 24 quality models from 1977 to 2011, Oriol et al.[START_REF] Oriol | Quality Models for Web Services: A Systematic Mapping[END_REF] collected and exploited 51 quality models published from 2002 to 2012, and Yan et al.[START_REF] Yan | Software quality assessment model: a systematic mapping study[END_REF] found 31 quality models that were published between 1998 and 2015. Note, the quality model sets from these last four studies are partially overlapped and cover a publication period going from 1977 to 2015.Consequently, we can state that current available quality model collections aren't unified and should be consolidated at least for the published ones.

Table 14 )

 14 than Yan. et al. rather than Kitchenham and Charter recommendation ones, even if the focus of both is on software engineering. We made this choice because of the following five criteria:

	-	(1) source is a recognized and relevant reference in software engineering domain with a large covering
		period,

-(2) source is a collection of relevant conference papers, books, and journals, -(3) their online search engine accepts extended search query, -

(4)

query results are freely downloadable from our institution, -

(5)

referenced documents are downloadable from the source (i.e., source is not only an aggregator engine).

TABLE 14 -

 14 LIST OF ELECTRONIC DATABASE SOURCES

	Online source name	Online location
	ACM digital library	dl.acm.org
	IEEE Xplore	ieeexplore.ieee.org
	Scopus	www.scopus.com
	Springer	link.springer.com
	Web of Science (also known as ISI Web of Knowledge)

TABLE 15 -

 15

	Concept	Keywords	Operator	Fields
	Domain of interest	software*	OR	Any
		SW
			AND
	Object of interest	quality model*	OR	Any
		QM
		quality ontology
		quality ontologies
		QO
		quality factor*
		QF
		quality characteri**
		QC
			AND
	Type of research work	analysis
		challenge
		challenges
		classification
		classifications
		compar*
		evolution
		evolutions
		mapping
		mappings
		taxonomy
		review
		reviews
		study
		studies
		survey

SEARCH QUERY ELEMENTS: KEYWORDS VS. OPERATORS VS. SEARCH FIELDS

TABLE 16 -

 16 THE FIVE DIGITAL LIBRARY SEARCH QUERIES

	Online source name	Search query

TABLE 17 -

 17 INCLUSION AND EXCLUSION CRITERIA RELATED TO OUR SEARCH STRATEGY

Table 18 )

 18 reveals that they are not evenly distributed as depicted by Figure34. Three types gather 81.25% of the studies. The most frequent study type concerns the creation of quality models, with 38.16% of the papers. This result reflects the fact that rather than reusing or customizing a quality model (i.e., 3.47% of the papers), researchers tend to create new quality models. Then, with respectively 26.39% and 16.67%, we have the analysis or survey on specific aspect, and comparison of quality models. TABLE 18 -MAPPING OF THE 136 PAPER STUDIES AGAINST THE SEVEN TYPES OF STUDY

TABLE 19 )

 19 :

TABLE 19 -

 19 MAPPING OF THE 136 PAPERS STUDIES AGAINST THE EIGHT CATEGORIES OF CLASSIFICATION OR COMPARISON CRITERIA

  To conclude this section, the 15 most cited software quality models from 1698 to 2019, and counting for 33.36% of the total of citations, are:-ISO / IEC 9126[START_REF]Part1: Quality Model[END_REF]: published in 1991 and cited in 66.91% of the study papers (i.e., 91 times), -McCall (also known as FCM or RADC)[41]: published in 1977 and cited in 42.65% of the study papers (i.e., 58 times),

	-	ISO / IEC / IEEE 25010 [23]: published in 2011 and cited in 40.44% of the study papers (i.e., 55 times),
	-	Boehm [128]: published in 1978 and cited in 33.82% of the study papers (i.e., 46 times),
	-	Dromey [49]: published in 1995 and cited in 27.94% of the study papers (i.e., 38 times),
	-	FURPS [85]: published in 1987 and cited in 19.85% of the study papers (i.e., 27 times),
	-	Quality Model for Object-Oriented Design (QMOOD) [218]: published in 2002 and cited in 13.24% of
		the study papers (i.e., 18 times),
	-	Capability Maturity Model Integration (CMMi) v1.1 [219]: published in 2002 and cited in 11.76% of the
		study papers (i.e., 16 times),
	-	Basili (also known as GQM) [217]: published in 1984 and cited in 10.29% of the study papers (i.e., 14
		times),
		Research Sub-question 2d	What is the most appropriate quality model is for embedded software in automotive?
		Research Sub-question 4a	Is it possible to have a unique reference quality model for software product, or instead should we have a meta-model?

-SPICE (ISO / IEC 15504) [220]: published in 1993 and cited in 10.29% of the study papers (i.e., 14 times), -CMM v1.0 from SEI team [154]: published on 1991 and cited in 9.56% of the study papers (i.e., 13 times), -QUAMOCO [160]: published in 2012 and cited in 8.82% of the study papers (i.e., 12 times), -Boehm [42]: published in 1976 and cited in 8.09% of the study papers (i.e., 11 times), -Basili -Briand -Melo (also known as QCM) [221]: published in 1996 and cited in 7.35% of the study papers (i.e., 10 times), -SQO-OSS [222]: published in 2008 and cited in 7.35% of the study papers (i.e., 10 times).

c. Software quality model landscape and the selection question

  et al.[96] pointed out. Indeed, ISO/IEC/IEEE 25010 and ISO/IEC 25012[START_REF]Software engineering -Product quality -Part2: External Metrics[END_REF] duplicate terms and definitions from ISO/IEC 25000, ISO/IEC 25020 [121] duplicates terms and definitions from ISO/IEC 25000 and ISO/IEC/IEEE 15939[START_REF]Glossary of Terms used in the Planning and Design of the IAEA Technical Cooperation Programme[END_REF], ISO/IEC 25021[START_REF] Wagner | The Quamoco Product Quality Modelling and Assessment Approach[END_REF] duplicates terms and definitions from ISO/IEC 25000, ISO/IEC 25010, ISO/IEC 25020 and ISO/IEC/IEEE 15939, … Additionally we remark that some duplications are performed jointly with an adaption of the original definition (e.g., it is written in ISO/IEC 25021 that definition of external measure of software quality is adapted from ISO/IEC/IEEE 25010 definition).

TABLE 20 -

 20 ALPHABETICALLY SORTED LIST OF MAIN POTENTIAL ISSUES THAT CHALLENGE DEVELOPMENT AND USE OF QUALITY

	MODELS

TABLE 21 -

 21 KAPPA INTERPRETATION (SOURCE LANDIS AND KOCH [245])

	Kappa ⲕ	Strength of Agreement
	< 0	Poor
	0.01 -0.20	Slight
	0.21 -0.40	Fair
	0.41 -0.60	Moderate
	0.61 -0.80	Substantial
	0.81 -1.00	Almost Perfect

Figure 60 -6-stages process for quality model development So, our analysis depicted by the algorithm in Figure

TABLE 22 -

 22 COMPARISON BETWEEN THE "QUALITY THERMOMETER" PROCESS AND THE CURRENT RELEVANT STANDARDS: ISO/IEC 250NN SERIES AND ISO/IEC/IEEE 15939

	"Quality Thermometer" process	ISO/IEC 250nn & ISO/IEC/IEEE 15939
	Any quality models as entry with customization	ISO/IEC 2501n
	Built-in evolution with polymorphism for	Product Quality Life-cycle Phase (Measurement Focus): 3 levels
	product and life cycle	-internal/external/in use -ISO/IEC 25020
		Life cycle applicability -ISO/IEC/IEEE 15939
	Aggregation method and relationship definition	Hierarchical relationship -ISO 2501n
		Aggregation via measurement functions -ISO/IEC/IEEE 15939,
		ISO/IEC 2502n
	Scorecard for consistent reporting	Informative evaluation report guidelines -ISO/IEC 25040,
		ISO/IEC/IEEE 15939
	Close to ISO/IEC 25040 : compatible with ISO/IEC
	250nn and ISO/IEC/IEEE 15939
	TABLE 23 -COMPARISON BETWEEN THE "6-STAGES" PROCESS AND THE CURRENT RELEVANT STANDARD: ISO/IEC 250NN SERIES
	AND ISO/IEC/IEEE 15939
	"6-stages" process	ISO/IEC 250nn & ISO 15939
	Any quality models as entry with	ISO/IEC 2501n
	customization
	Built-in evolution with polymorphism for	Product Quality Life-cycle Phase (Measurement Focus): 3 levels -
	product and life cycle	internal/external/in use -ISO/IEC 25020
		Life cycle applicability -ISO/IEC/IEEE 15939
	Automatic weight determination	Optional manual weight -ISO/IEC/IEEE 15939
	Objective stakeholder consensus via	Subjective validation and approval of quality requirements -
	computation	ISO/IEC 25030
	Missing traceability with requirements	Traceability with quality requirements -ISO/IEC 25030

  This ECU is responsible for vehicle telematics based on wireless communication (i.e., Bluetooth, Wi-Fi, 3G and 4G/LTE). It has no direct interaction with user (i.e., there is no. human-machine interface on IVC) but it is connected directly to IVI system by a class D ethernet network. Therefore, IVC is part of the vehicle infotainment domain. Moreover, cyber security must be addressed carefully for this ECU since it is the wireless remote access point to the vehicle network.

		4 classes
		was defined in 1994 by Society for Automotive Engineers (SAE) : SAE j2056 [249], and is based on
		transmission speed and network performance.
	-	IVC:

-ADAS: The advance driver assistance systems is responsible for assisting driver for safer driving and easier parking. We can distinguish four groups of functionalities: o Longitudinal control systems: e.g., anti-lock braking systems (i.e., ABS), adaptive cruise control (i.e., ACC), hill descent control, o Lateral control systems: e.g., electronic stability control (i.e., ESC), lane centering, o Alert systems: e.g., automotive night vision, lane departure warning system (i.e., LDW), blind spot monitor, o Park assist systems: e.g., parking sensor, automatic parking.

TABLE 24 -

 24 EXTRACTED NUMBER OF SURVEY RESPONSES PER QUALITY IN USE QUALITY CHARACTERISTICS AND IMPORTANCE FOR IVC

	25	(3 + 6) =	9 25	= 𝟎. 𝟑𝟔

TABLE 25 -

 25 THE ASSOCIATED RANKING RESPONSE MATCHES BETWEEN ALL POSSIBLE RESPONSE COMBINATIONS; THE GREEN CELLS

		INDICATE PERFECT MATCH
			Project Manager 1
		Not Important at All	Somewhat Important	Important	Very Important	Extremely Important
		Not Important at All
	Project	Manager 2

TABLE 26 -

 26 EXTRACTED NUMBER OF SURVEY RESPONSES PER SYSTEM/SOFTWARE PRODUCT QUALITY CHARACTERISTICS AND IMPORTANCE FOR IVC AND ANY ROLE

	12 + 10 32	=	326 1024

TABLE 27 -

 27 SURVEY DATA ANALYSIS WITH COHEN Κ AND FLEISS Κ. COLORED CELLS HIGHLIGHT Κ BASED CHOICE FOR EACH ECU; GRAYED CELLS HIGHLIGHT AT LEAST MODERATE AGREEMENT

			All roles	Assurance quality engineer	Project Manager	Validation Leader	Architect
			All ECUs [ⲕF] 0.122846 [ⲕF] 0.444330	[ⲕF] 0.313977	ꓱ1	ø
	quality characterisitics	All Sys/SW product	IVI [ⲕF] 0.291964 IVC [ⲕF] 0.488525 ADAS [ⲕC] 0.025 FOTA ꓱ1 All ECUs [ⲕF] 0.126425 IVI [ⲕF] 0.284314 IVC [ⲕF] 0.572111 ADAS [ⲕC] 0	ꓱ1 ꓱ1 ꓱ1 ꓱ1 [ⲕF] 0.412305 ꓱ1 ꓱ1 ꓱ1	[ⲕF] 0.433414 [ⲕC] 0.356436 ꓱ1 ø [ⲕF] 0.286275 [ⲕF] 0.429208 [ⲕC] 0.368421 ꓱ1	ø ꓱ1 ø ø ꓱ1 ø ꓱ1 ø	ø ø ø ø ø ø ø ø

[ⲕF] ... Fleiss' Kappa ꓱ1 no Kappa (one rater) [ⲕC] ... Cohen's Kappa ø no Kappa (no rater)

  Chapter VI.4.b explanations on the calculation of the new importance value of previously excluded characteristics, we have for the seven characteristics of our example the following new inclusion / exclusion analysis results:

	Now, following Characteristics						New importance value	Final Decision
	Usability	0.5 ×	0.25 + 0.38889 2	+ 0.5 ×	1 + 1 2	= 𝟎. 𝟔𝟓𝟗𝟕𝟐𝟐𝟐	Excluded
	Security		0.5 ×	0.5 + 0.8 2	+ 0.5 ×	1.3333 + 1.3333 + 1 2	= 𝟎. 𝟗𝟑𝟔𝟏𝟏𝟏	Excluded
	Maintainability	0.5 ×	0.25 + 1.2 2		+ 0.5 ×	1.3333 + 1.3333 + 1 + 1 + 1.3333 5	= 𝟎. 𝟗𝟔𝟐𝟓	Excluded
	Portability	0.5 ×	0.75 + 0.77778 2	+ 0.5 × (1.3333) = 𝟏. 𝟎𝟒𝟖𝟔𝟏𝟏𝟏	Included
	Efficiency	0.5 ×	0.75 + 0.75 2	+ 0.5 × (0.75) = 𝟎. 𝟕𝟓	Excluded
	Freedom From Risk 0.5 ×	0.75 + 0.8889 2	+ 0.5 ×	1 + 1.3333 2	= 𝟎. 𝟗𝟗𝟑𝟎𝟓𝟓𝟔	Excluded
	Context Coverage	0.5 ×	0.5 + 0.16667 2	+ 0.5 × (0) = 𝟎. 𝟏𝟔𝟔𝟔𝟔𝟕	Excluded
								Raters
	Quality perspective	Characteristics		Rater 1	Rater 2	Rater 3	Rater 4
		Functional Suitability	Extremely Important Extremely Important Very Important	Extremely Important
	System / Software product quality	Performance Efficiency Very Important Compatibility Very Important Usability Important	Very Important Extremely Important Very Important Very Important Extremely Important Somewhat Important Important Extremely Important Extremely Important
		Reliability		Very Important	Extremely Important Very Important	Extremely Important

TABLE 28 -

 28 GENETIC-QUALITY ANALOGY: TERMINOLOGY SUMMARY

	Genetic domain	Quality domain
	Chromatid	Quality perspective
	Chromosome	Two identical copies of a quality perspective
	DNA sequence	Sequence of quality characteristics and sub-characteristics
	Sites	Quality characteristic and sub-characteristics
	Locus	Location of a quality characteristic in a quality perspective
	Gene	Quality characteristic sequence of sub-characteristics at a specific locus
	Polymorphic gene	Existence of more than one variation at a specific locus of quality
		characteristics and sub-characteristics
	Alleles	Variations of quality characteristics and sub-characteristics at a specific locus
		with their own probability

TABLE 29 -

 29 ALLELE AND SITE EXAMPLE FOR "PORTABILITY" QUALITY CHARACTERISTIC

		Gene: Portability	Allele 1	Allele 2	Allele 3	Allele 4	Allele 5	Allele 6	Allele 7	Allele 8
	site Adaptability	Adaptability	Adaptability			Adaptability		Adaptability	Adaptability
	site Instalability	Instalability	Instalability			Deployability		Instalability	Instalability
	site Replaceability	Replaceability	Replaceability			Replaceability		Replaceability Replaceability
				Portability
	site Conformance		Compliance						Conformance
					Self-
	site Modularity			containedness	Modularity
		Machine			Device	Machine
	site	independence			independence	independence
	site Self-Descriptiveness				Self-Descriptiveness			Portability
		Software system				Software system
	site	independence				independence
	site Co-existence		Co-existence
	site 10 Reusability					Reusability

TABLE 30 -

 30 GENE COMPUTATION EXAMPLE FOR "PORTABILITY" QUALITY CHARACTERISTIC

		Gene: Portability
	site 1	Adaptability	Adaptability = 100%
	site 2	Instalability	Instalability = 80%	Deployability = 20%
	site 3	Replaceability	Replaceability = 100%
	site 4	Conformance	Conformance = 50%	Portability Compliance = 50%
	site 5	Modularity	Modularity = 50%	Self-containedness = 50%
	site 6	Machine independence	Machine independence = 50%	Device independence = 50%
	site 7	Self-Descriptiveness	Self-Descriptiveness = 50%	Portability = 50% [i.e., Portability documentation = 25%, Portability complexity = 25%]
		Software system
	site 8	independence	Software system independence = 100%
	site 9	Co-existence	Co-existence = 100%
	site	Reusability	Reusability = 100%

  Get distance between Grp and Tmp_Grp # If the distance is lower than previous one (note if distance is always infinity, # then elements are considered too far from each other: don't take them) Remove # Compute distance between two groups of characteristics: Ref_Grp and Tgt_Grp Get distance between Ref_Grp and Tgt_Grp Distance  infinity # Test first if the two characteristic "words" are identical Get constellation closest distance of Tgt_Grp in Const_Ref_Grp Else # Otherwise check the intersection of Tgt_Grp and Ref_Grp synonym constellations is not empty

	If Ref_Grp = Tgt_Grp
	Distance  0
	Else
	# Otherwise check if Tgt_Grp is within the Ref_Grp synonym constellation
	Cons_Ref_Grp  Get synonym constellation for Ref_Grp from
	(https://wordnet.princeton.edu/ and
	http://www.atlas-semantiques.eu/)
	If Tgt_Grp is in Cons_Ref_Grp
	Distance  Const_Tgt_Grp  Get synonym constellation for Tgt_Grp from
	(https://wordnet.princeton.edu/ and
	http://www.atlas-semantiques.eu/)
	If (Const_Ref_Grp ∩ Const_Tgt_Grp) is not empty
	Distance  Get constellation closest distance of (Const_Ref_Grp ∩
	Const_Tgt_Grp)
	If Tmp_Distance < Candidate_Distance
	Candidate_Grp  Tmp_Grp
	Candidate_Distance  Tmp_Distance
	Candidate_Element_Id  i
	End loop
	# We found a candidate element, so remove it from QM_Grp_Set
	If Candidate_Element_Id != 0

Find the closest element of QM_Grp_Set to Grp, if any, and remove it from QM_Grp_Set Extract element of QM_Grp_Set to be mapped to Grp Candidate_Grp  empty Candidate_Distance  infinity Candidate_Element_Id  0 # Crawl all elements in QM_Grp_Set to find the closest element to Grp if it exists Loop: i from 1 to size (QM_Grp_Set) Tmp_Grp  Get element i from QM_Grp_Set # Determine the distance between these two elements Tmp_Distance  element Candidate_Element_Id from QM_Grp_Set Return Candidate_Grp

Figure 83 -Example from the online Semantic Atlas: three synonym constellations for "efficiency" word as quality characteristic (source: http://www.atlas-semantiques.eu/)

TABLE 31 -

 31 , McCall, FURPS, ISO / IEC 9126, Alvaro, Bawane, Kalaimagal's Q'FActo 12, and ISO / IEC / IEEE 25010) to construct the first software quality model genome meta-model. The quality model details are provided in Annex 13 while Table 31 is an overview of these models. OVERVIEW OF THE EIGHT SELECTED QUALITY MODELS FOR SOFTWARE QUALITY MODEL GENOME META-MODEL

			CONSTRUCTION
	Quality model	Reference	Date Quality model element numbering and type
	Boehm	[42]	1976	3	high-level qualities
				7	intermediate-level qualities
				15 primitive quality characteristics
	McCall	[41]	1977	3	Perspectives
				11 quality factors
				23 quality criteria
	FURPS	[85]	1987	5	Components
				25 Sub-components
	ISO/IEC 9126	[24]	1991	2	quality perspectives
				10 quality characteristics
				31 quality sub-characteristics
	Alvaro	[36]	2010	6	quality characteristics
				23 quality sub-characteristics
				48 attributes
	Bawane	[103]	2010	2	quality perspectives
				11 quality characteristics
				28 quality sub-characteristics
	Kalaimagal's Q'FActo 12	[102]	2010	12 quality factors
				30 quality criteria
				44 quality measures
	ISO/IEC/IEEE 25010	[23]	2011	2	quality perspectives
				13 quality characteristics
				42 quality sub-characteristics

TABLE 32 -

 32 [START_REF] Garces | Quality attributes and quality models for ambient assisted living software systems: A systematic mapping[END_REF] DISTINCT QUALITY CHARACTERISTIC GROUPS FROM THE EIGHT SELECTED SOFTWARE QUALITY MODELS

	Total	Quality Characteristic Name	Boehm [42]	McCall [41]	FURPS [85]	ISO/IEC 9126 [24]	Alvaro [36] Bawane [103]	Kalaimagal Q'FActo 12 [102]	ISO/IEC/IEEE 25010 [23]
	1 Adaptability					Adaptability
	1 Changeability					Changeability
	2 Compatibility							Compatibility	Compatibility
	1 Compliance					Compliance
	1 Context coverage								Context coverage
	1 Controllability							Controllability
	1 Correctness		Correctness
	6 Efficiency	Efficiency	Efficiency		Efficiency	Efficiency	Efficiency	Efficiency
	1 Fault-tolerance					Fault
							Tolerance
	1 Freedom from risk								Freedom from risk
	1 Flexibility		Flexibility
	5 Functionality			Functionalit	Functionali	Functionality Functionality Functionality
					y	ty
	1 Functional suitability								Functional
										suitability
	1 General Utility	General Utility
	1 Generality							Generality
	1 Hardware/Software							Hardware/Softwa
		Independence							re Independence
	1 Human engineering Human
			engineering
	1 Instalability							Instalability
	1 Integrity (security)		Integrity (security)
	2 Interoperability		Interoperability					Interoperability
	1 Learnability							Learnability
	7 Maintainability	Maintainability Maintainability		Maintainab	Maintainability Maintainability Maintainability Maintainability
						ility
	1 Maturity					Maturity
	1 Modifiability	Modifiability
	2 Operability					Operability		Operability
	1 Performance			Performanc
					e
	1 Performance								Performance
		efficiency								efficiency
	7 Portability	Portability	Portability		Portability Portability	Portability	Portability	Portability

TABLE 36 -

 36 THE RELATIONSHIP LINKS BETWEEN THE QUALITY CHARACTERISTIC GENES

	Gene ID	Quality Characteristic Name	Found in gene(s)
	A01	Adaptability	A24, B07
	A02	Changeability	B05
	A03	Context coverage	B08
	A04	Controllability	B11, B14
	A05	Correctness	A16, B03
	A06	Freedom from risk	B08
	A07	Flexibility	A03, A17
	A08	General utility	none
	A09	Generality	A07, B10
	A10	Hardware/Software Independence	B07, B10
	A11	Human engineering	B16
	A12	Instalability	B07
	A13	Learnability	B16
	A14	Maturity	B09
	A15	Modifiability	B05
	A16	Product operation	none
	A17	Product revision	none
	A18	Product transition	none
	A19	Resource behavior	B01
	A20	Safety in use	(B08)
	A21	Satisfaction	B08
	A22	Self-Contained	B03, (B07)
	A23	Suitability	B03
	A24	Supportability	none
	A25	Test documentation	B14
	A26	Time behavior	B01
	A27	Traceability	A05, B14
	B01	Efficiency	A16, B08, B12, B16
	B02	Fault tolerance	B09
	B03	Functionality	B12
	B04	Interoperability	A18, B03, B10, B12
	B05	Maintainability	A08, A17, A24, B12
	B06	Operability	B16
	B07	Portability	A08, A18, B12
	B08	Product in use	none
	B09	Reliability	A16, B12, B16
	B10	Reusability	A18, B05, B07, B12
	B11	Security	A16, B03, B12
	B12	Software product	none
	B13	Compliance	B03, (B01, B05, B07, B09, B16)
	B14	Testability	A17, A24, B05
	B15	Understandability	B05, B16

TABLE 37 -

 37 EXAMPLE OF GENE 05 "MAINTAINABILITY" DETAILED WITH ALL ITS SUB-GENES AND SITES: ON THE LEFT SIDE, THE DIRECT DETAILED SITE SEQUENCE, ON THE RIGHT THE DETAILED AND OPTIMIZED SITE SEQUENCE WITH SAME QUALITY COVERAGE

	Detailed gene B05 "Maintainability"	Detailed and optimized gene B05 "Maintainability"
	Genes	Sites	Genes	Sites

TABLE 38 -

 38 DEGREE OF SUBJECTIVITY FOR EACH CHROMATID OF THE SOFTWARE QUALITY MODEL GENOME META-MODEL

	Chromatid	Number of subjective sites	Number of sites	Percentage of subjectivity
	General utility	23	114	20.18%
	Product operation	23	85	27.06%
	Product revision	4	69	5.80%
	Product transition	15	43	34.88%
	Supportability	4	72	5.56%
	Product in use	6	34	17.65%
	Software product	52	195	26.67%
	Overall	127	615	20.65%

TABLE 39 -

 39 CATALOG OF THE MAIN EXISTING AGGREGATION OPERATORS, BASED ON DETYNIECKI[START_REF] Detyniecki | Fundamentals on Aggregation Operators[END_REF], WAGNER[START_REF] Wagner | Software Product Quality Control[END_REF] AND DUJMOVIC & BAYUCAN[START_REF] Dujmovic | A Quantitative Method for Software Evaluation and its Application in Evaluating Windowed Environments[END_REF]

	Group of aggregation operators	Aggregation operator	Formula

TABLE 41 -

 41 EXAMPLES OF LEVENSHTEIN'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS

	)	. 𝑆(𝑥 , … , 𝑥 )
	where 𝛾 degree of compensation
	𝑇 is a t -norm and 𝑆 is a t -conorm

TABLE 42 -

 42 EXAMPLES OF DAMERAU-LEVENSHTEIN'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS

	Example 1 Distance	Example 2	Distance	Example 3	Distance	Example 4	Distance
	solar		solar		acknowledgement				123456AD90
	vs.	1	vs.	7	vs.			3		vs.	4
	polar		solarization		akcnowlegdemnt				124357AC97
	Jaro's distance [90]			𝑑	(𝑥, 𝑦) =	1 3	𝑚 |𝑥|	+	𝑚 |𝑦|	+	𝑚 -𝑡 𝑚

TABLE 43 -

 43 EXAMPLES OF JARO'S DISTANCE COMPUTATION: RED CHARACTERS INDICATE DIFFERING CHARACTERS

	Example 1 Distance	Example 2	Distance	Example 3	Distance	Example 4	Distance
	solar		solar		acknowledgement		123456AD90
	vs.	0.86667	vs.	0.80556	vs.	0.97778	vs.	0.78333
	polar		Solarization		akcnowlegdemnt		124357AC97
				𝑑	(𝑥, 𝑦) = 𝑑	(𝑥, 𝑦) + 𝑙. 𝑝 1 -𝑑	(𝑥, 𝑦)
	Jaro-Winkler's distance [91]	where 𝑥 and 𝑦 are 2 strings
				𝑙 is the prefix length, with 𝑙 ≤ 4
				𝑝 is a coefficient to foster strings with prefix. Winkler suggested 𝑝
					= 0.1

TABLE 44 -

 44 EXAMPLES OF JARO-WINKLER'S DISTANCE COMPUTATION WITH L = 2: RED CHARACTERS INDICATE DIFFERING CHARACTERSThis chapter should define all responsibilities associated with the implementation of the Plan. This includes system(s) and/or software quality requirements specification, all data collection, analysis tasks, implementation of other supporting requirements, reporting, follow up and similar requirements. [ISO/IEC 25001]

	Example 1 Distance	Example 2	Distance	Example 3	Distance	Example 4	Distance
	Solar		solar		acknowledgement		123456AD90
	vs.	0.89333	vs.	0.84444	vs.	0.98222	vs.	0.78766
	polar		Solarization		Akcnowlegdemnt		124357AC97
	Jaccard's distance [95]			𝑑	(𝑥, 𝑦) =	|𝑥 ∪ 𝑦| -|𝑥 ∩ 𝑦| |𝑥 ∪ 𝑦|
					where 𝑥 and 𝑦 are 2 sample sets

  This chapter should address questions such as accuracy and precision level, dedicated labor effort , repeatability and generalization, or any acceptable vs non acceptable compromises/ trade off This chapter should provide a clear statement about the objective(s) of the evaluation and the intended application of the system(s) or software. This can be stated in terms of business needs. However, they should be useable for the purpose of specifying quality requirements and setting quality objectives and respective criteria. [ISO/IEC 25001] This chapter should provide quantifiable quality objectives (target values), which are verified against values measured at interim or final phases of the project development. [ISO/IEC 25001]This chapter should provide statements of the quality characteristics (e.g. ISO/IEC 25010) resulting from the specification of system(s) or software quality requirements, which support the objectives prescribed in 5.2.1 " Evaluation Objectives ". NOTE: The stated quality objectives may be both product and process oriented. The purpose of this plan is to address the product quality objectives only. [ISO/IEC 25001] This chapter should define the measurements, which are planned to be carried out and cover required scope of quality evaluation. The chapter should indicate at what phase(s) of the development cycle these measurements are to be carried out, what evaluation process should be applied (from ISO/IEC 25041), how often they should be repeated, what techniques or tools should be used to aid data capture and analysis, and what actions should be undertaken if there are divergences from the stated objectives. [ISO/IEC 25001] This chapter should provide a clear plan of activities with milestones and stated deliverables. [ISO/IEC 25001] This chapter should define how data is to be analysed, what, if any, statistical methods are to be employed and what presentation techniques are to be used. It should make references to previously stated responsibilities, supporting tools and forms. It should also state how the information is to be integrated into the progress tracking process or into the product acceptance process. [ISO/IEC 25001]. With regard to ISO2626: for ASIL A and greater we must have process improvement. Therefore forecasted values (e.g. at the end of the development) should also be given, considering the ratio obtained when comparing the planned measurement values to the current values. This chapter should include requirements not covered previously, e.g. it can include the following information: [ISO/IEC 25001] Describe or provide requirements and references for the supporting tools. This can include guides for the use of databases, spreadsheet and statistical packages. [ISO/IEC 25001]Refer to applicable standards and supporting guides. Describe their use and benefits relevant to the systems and software product quality requirements and evaluation processes (e.g. ISO/IEC 25000; ISO 9001; ISO/IEC 90003).Include evaluation and measurement procedures for the effective quantitative assessment of the systems or software product suppliers. This can cover the number of released copies, current error status, surveys about post installation support performance, statistics about past and current users' satisfaction, management [ISO/IEC 25001]

	SLR-S14 SLR-S15 SLR-S16 SLR-S17 SLR-S18 SLR-S19 SLR-S20 SLR-S21 SLR-S22 SLR-S23 SLR-S24 SLR-S25 SLR-S26 SLR-S27 SLR-S28 SLR-S04 SLR-S05 SLR-S06 SLR-S07 SLR-S08 SLR-S09 SLR-S10 SLR-S11 SLR-S12 SLR-S13 SLR-S14 SLR-S15 SLR-S16 SLR-S17 SLR-S18 SLR-S19 SLR-S20 SLR-S21 SLR-S22 SLR-S23 SLR-S24 SLR-S25 SLR-S26 SLR-S27 SLR-S28 SLR-S29 SLR-S30 SLR-S31 SLR-S32 SLR-S33 SLR-S34 SLR-S35 SLR-S36 SLR-S37 SLR-S38 SLR-S39 SLR-S40 SLR-S41 SLR-S42 SLR-S43 SLR-S44 SLR-S45 SLR-S46 SLR-S47 SLR-S48 SLR-S49 SLR-S50 SLR-S51 SLR-S52 SLR-S53 SLR-S54 SLR-S55 SLR-S56 SLR-S57 SLR-S58 SLR-S59 SLR-S60 SLR-S61 SLR-S62 SLR-S63 SLR-S64 SLR-S65 SLR-S66 SLR-S67 SLR-S68 SLR-S69 SLR-S70 SLR-S71 SLR-S72 SLR-S73 SLR-S74 SLR-S75 SLR-S76 SLR-S77 SLR-S78 SLR-S79 SLR-S93 SLR-S94 SLR-S107 SLR-S108 SLR-S132 SLR-S119 SLR-S133 SLR-S120 SLR-S80 SLR-S81 SLR-S95 SLR-S109 SLR-S121 SLR-S134 SLR-S82 SLR-S83 SLR-S85 SLR-S86 SLR-S87 SLR-S88 SLR-S89 SLR-S90 SLR-S91 SLR-S92 SLR-S104 SLR-S105 SLR-S106 SLR-S131 SLR-S118 SLR-S130 SLR-S117 SLR-S129 SLR-S99 SLR-S103 SLR-S116 SLR-S128 SLR-S102 SLR-S115 SLR-S127 SLR-S101 SLR-S114 SLR-S126 SLR-S100 SLR-S112 SLR-S113 SLR-S125 SLR-S84 SLR-S96 SLR-S97 SLR-S98 SLR-S110 SLR-S111 SLR-S122 SLR-S135 SLR-S124 SLR-S136 SLR-S123	2 Objectives 2.1 Evaluation Objectives 2.2 Quality Objectives 2.3 List of priorities 2.4 Level of evaluation and compromises This chapter should prioritize the above characteristics and should provide a supporting rationale for these priorities. [ISO/IEC 25001] 3 Evaluation Elements 3.1 Systems and software quality requirements and applicable quality characteristics 3.2 Evaluation design 4 Execution Plan 4.1 Evaluation planning and execution 4.2 Using and analyzing data 4.3 Reporting This chapter should define all relevant reporting requirements. [ISO/IEC 25001] 5 Other Requirements 5.1 Techniques and methods employed 5.2 Supporting tools 5.3 Supporting trainings 5.4 Relevant standards and guides 5.5 Suppliers' evaluation Provide a full description (or references to other material) of the techniques and methods used, (e.g. method for sizing; development maturity assessment; inspection method for error detection; defect removal model for predicting error rates). [ISO/IEC 25001] [ISO/IEC 25001] Describe or provide requirements and references for the supporting trainings Annex A -List of point of contacts Name Position This chapter should enumerate all involved points of contact Organization e-mail [271] Scopus J 19 Khoshgoftaar T.M. Allen E.B. Jones W.D. Hudepohl J.P. Cost-Benefit Analysis of Software Quality Models https://www.scopus.com/inward/record.uri?eid=2-s2.0-0038138707&doi=10.1023%2fA%3a1016621219262 &partnerID=40&md5=e236c7fd7357975ed5580c280 1139eb4 [272] IEEE Xplore C 14 Hong Zhu Yanlong Zhang Qingning Huo S. Greenwood Application of hazard analysis to software quality modelling https://ieeexplore.ieee.org/stamp/stamp.jsp?arnum ber=1044544 [273] IEEE Xplore C 5 C. V. Ramamoorthy Evolution and evaluation of software quality models https://ieeexplore.ieee.org/stamp/stamp.jsp?arnum ber=1180850 [200] Scopus J 228 Briand L.C. Wüst J. Empirical studies of quality models in object-oriented systems https://www.scopus.com/inward/record.uri?eid=2-s2.0-77957159740&doi=10.1016%2fS0065-2458%2802%2980005-5&partnerID=40&md5=1256de9aefbf034462dacc600 7f42689 [274] Web of Science J 17 Mendoza, LE Griman, AC Perez, MA Rojas, T Evaluation of environments for portals development: A case study https://www.tandfonline.com/doi/abs/10.1201/107 8/43200.19.2.20020228/35141.7 [275] IEEE Xplore C 22 T. M. Khoshgoftaar E. Geleyn L. Nguyen Empirical case studies of combining software quality classification models https://ieeexplore.ieee.org/stamp/stamp.jsp?arnum ber=1319084 [276] IEEE Xplore C 47 Yi Liu T. Khoshgoftaar Reducing overfitting in genetic programming models for software quality classification https://ieeexplore.ieee.org/stamp/stamp.jsp?arnum ber=1281730 [277] Web of Science C 5 Oh, J Park, D Lee, B Lee, J Hong, E Wu, C Certification of software packages using hierarchical classification https://link.springer.com/chapter/10.1007/978-3-540-24675-6_17 [278] Scopus J 32 Côté M.-A. Suryn W. Laporte C.Y. Martin R.A. The evolution path for industrial software quality evaluation methods applying ISO/IEC 9126:2001 quality model: Example of MITRE's SQAE method https://www.scopus.com/inward/record.uri?eid=2-s2.0-17444388547&doi=10.1007%2fs11219-004-5259-6&partnerID=40&md5=09ec05c221610b4c799ab611 1def6568 [148] Manual search J 24 S. Khaddaj G. Horgan A Proposed Adaptable Quality Model for Software Quality Assurance https://www.semanticscholar.org/paper/A-Proposed-Adaptable-Quality-Model-for-Software-Khaddaj-Horgan/815debd01444997d7d20656204be767d7296 d660 [279] IEEE Xplore C 6 Q. Zhang J. Wu H. Zhu Tool Support to Model-based Quality Analysis of Software Architecture https://ieeexplore.ieee.org/stamp/stamp.jsp?arnum ber=4020069 [280] Scopus J 26 Mishra A. Mishra D. Software quality assurance models in small and medium organisations: A comparison https://www.scopus.com/inward/record.uri?eid=2-s2.0-33845719178&doi=10.1504%2fIJITM.2006.008710& partnerID=40&md5=42359aa8dd07631d8be7f435c7 e1f634 [281] Scopus C 11 Jung H.-J. Jung W.-T. Yang H.-S. A study on the standard of software quality testing https://www.scopus.com/inward/record.uri?eid=2-s2.0-33745922863&doi=10.1007%2f11751632_113&part nerID=40&md5=535bc40cf6bbac584b698ede7532b4 c5 [282] Scopus J 0 Wang C.-T. Lo C.-C. Jean T.-F. Probabilistic models for software quality analysis https://www.scopus.com/inward/record.uri?eid=2-s2.0-33746862135&doi=10.1080%2f10170660609509329 &partnerID=40&md5=41cc8711f0c0c0eba87345026e dcf678 [283] Web of Science C 0 Ruiz, Julian Calero, Coral Piattini, Mario Web metrics selection through a practitioners' survey https://www.semanticscholar.org/paper/Web-metrics-selection-through-a-practitioners'-Ruiz-Calero/cf756d83396f40005ff49d4ffbfe3f467812472c Investigation on software evolution & reuse impact on quality quality model result: w/ and w/o reuse industry & academic telecommunication reuse, maintenance telecommunication survey about quality and quality model quality characteristics academic not defined quality in general software survey about challenges main description industry information technology (IT) quality in general information system creation of new model main description industry & academic accounting and commercial management maintenance, cost accounting and commercial management creation of new model product and process metrics industry & academic telecommunication reliability, fault prediction telecommunication creation of new model none industry not defined cost, defect software creation of new model Statistical approach (i.e. construction method or formula) industry & academic telecommunication reliability, fault prediction telecommunication creation of new model quality characteristics and attributes academic university usability, reliability, functionality, efficiency website creation of new model Statistical approach (i.e. construction method or formula) academic telecommunication fault prediction telecommunication Exploratory study purpose / scope academic not defined user perception website Survey on quality models role in business process reengineering Statistical approach (i.e. construction method or formula) industry & academic telecommunication reliability, cost, accuracy, return on investment telecommunication Quality model construction and development methodology purpose / scope academic not defined safety information system, website Survey on quality from different point of view link to maturity and reliability characteristic academic not defined reliability, maturity, maintainability software Empirical study of quality models in OO systems detailed description, formalism, statistical / implicit method, model evaluation, dependent & independent variables, benefits vs limitations academic object-oriented systems reliability, maintainability, fault prediction object-oriented software creation of new model quality characteristics and sub-characteristics industry commercial business functionality, usability, efficiency business web portal development environment Comparative evaluation quality model result academic telecommunication reliability telecommunication, wireless Improvement of genetic programming models for QM Statistical approach (i.e. construction method or formula) academic telecommunication reliability, fault prediction telecommunication creation of new model purpose / scope academic telecommunication certification, process Commercial-Off-The-Shelf (COTS), telecommunication creation of new model structure, quality characteristics and sub-characteristics, and metrics industry & academic not defined quality in general software creation of new model characteristics and limitations academic not defined global quality (maintainability, usability, cost/benefit, security, reliability, timeliness, correctness), local quality software creation of new model type of model (hierarchical vs relational) academic not defined safety, risk, quality hazard e-commerce comparison of quality models benefits vs limitations, main characteristics academic not defined general quality, process software Study of standard quality models description, metrics academic not defined quality standard, reliability software creation of new model quality model result academic not defined complexity, reliability, cost information system Improvement of quality model metrics metrics academic not defined usability, cost web system creation of new model Statistical approach (i.e. construction method or formula) industry & academic military reliability, fault prediction telecommunication creation of new model characteristics and limitations academic government, aeronautic, space functionality, reliability, usability, efficiency, maintainability, manageability COTS creation of new model characteristics academic not defined reliability, surface(literal) & conceptual understandability requirements Study between some quality model metrics metric results academic not defined stability software developed iteratively / agile Predictive model evaluation Statistical approach (i.e. construction method or formula) and evaluation result academic aeronautic, space fault prediction, performance software creation of new model quality characteristics academic not defined maintainability, reliability, supportability software architecture, self-healing system creation of new model metrics result academic air traffic management maintainability software creation of new model description academic not defined flexibility, complexity software creation of new model quality characteristics academic not defined quality in general software creation of new model quality characteristics academic international web/online course sharing quality information sharing system, website creation of new model quality characteristics academic not defined functionality, usability, maintainability Free/Libre open source software Tailoring of quality model scope industry & academic application performance management evolution, maintainability, reliability software comparison of quality models evolvability sub-characteristic industry & academic industrial systems evolvability software survey on quality models in practice interviewee results industry & academic software development quality in general software creation of new model characteristics vs object-oriented patterns academic not defined design (expandability, simplicity, reusability), implementation (learnability, understandability, modularity), runtime (generality, modularity at runtime, scalability, robustness) object-oriented software comparison of quality models Purpose, quality focus, viewpoint, quality factors, validation factors, relationships (quantitative & qualitative) academic not defined defect, maturity, effectiveness, reliability, quality in general software creation of new model description, type of models, practice and impact on quality goals, tool, demo/empirical approach industry & academic not defined correctness, completeness, consistency, comprehensibility, confinement, changeability model-based software development creation of new model characteristic, limitations academic product lines, IT infrastructure evolvability software creation of new model perception, interpretation, survey to map quality characteristics / sub-characteristics with conceptual quality characteristics/sub-characteristics (semantic, pragmatic, syntactic quality) academic naval aeronautical and astronautical reliability, conceptual model qualitysoftware creation of new model limitation against directive to compute aggregate indices (from metrics and quality characteristics) industry IT systems quality in general software comparison of quality models statistical comparison of quality model application against same set of software systems academic not defined maintainability, correctness open source software comparison of quality models benefits vs limitations from experimental use, main characteristics academic not defined functionality, usability, general quality Free/Libre open source software, web-client Method to customize quality model purpose / scope, description academic Software Factory Data Warehouse general quality, process software Method to customize quality model Scope, purpose, viewpoint, focus, context academic not defined perceived consistency, perceived appropriateness, perceived efficiency, completeness, correctness, efficiency software comparison of quality models characteristics and overall weighted sum based on those characteristics (weight set by experts) academic not defined quality in general software creation of new model Check point and threshold purpose industry not defined quality in general Ada software creation of new model Operationalization limitations between quality attributes and measurements academic not defined quality in general java software, open source creation of new model Description academic not defined quality in general web-based applications Systematic review on quality attributes and metrics Against ISO 25010 quality characteristics and ISO 9126 metrics academic not defined quality in general software product lines creation of new model purpose, scope, characteristics academic not defined internal and external quality, quality in use web [2.0] sites creation of new model description, scope, purpose, limitations against embedded systems industry & academic embedded systems quality in general embedded systems software creation of new model Mapping to International Software Benchmarking Standards Group (ISBSG) data collection questionnaires academic software engineering functionality, reliability, maintainability, satisfaction, productivity software Study of benefit to use quality model Scope, quality characteristics industry & academic traffic control system structuredness, traceability, productivity, maintainability software Study of architecture patterns impact on quality attributes scope academic not defined performance, dependability, costs, security, usability, quality of use Service-Oriented Architecture (SOA), service-based application creation of new model description academic Communication systems functionality, reliability, efficiency, usability, portability, hardware standard embedded systems software survey on development of maturity models description academic not defined process software survey on quality characteristic importance for quality model coverage completeness of quality characteristics, sub-characteristics academic E-type systems (evolutionary) quality in general E-type software (evolutionary) comparison of quality models description, quality characteristics and structure academic not defined quality in general software comparison of quality models category (basic, tailored), challenges and issues academic not defined quality in general component based software development Survey on quality models and quality attributes scope, quality characteristics, level of evidences academic embedded systems quality in general embedded systems software Survey on service quality description (i.e., includes quality model as one mean of description) description, scope, completeness of metrics and details, formalism, association with assessment guidelines academic web-based services quality of services software service, software-support (i.e., infrastructural) service comparison of quality models description, quality characteristics and structure academic not defined product quality, process quality software survey on sustainability metrics metrics type, quality characteristics academic not defined sustainability green software creation of new model description, limitations industry & academic university community activity, maintainability, reliability, release activity open source, web application, software repository creation of new model impact on metrics academic software crowdsourcing platform efficiency, cost, platform quality factor, project quality project crowdsourcing-based software Survey on quality model metrics normalization impact relationship with metrics to build quality model industry & academic telecom infrastructure, games, hydraulic control quality in general software Survey on quality model for Legacy Information System assessment description, scope/purpose, characteristics academic Legacy Information Systems (LIS) quality in general information systems software creation of new model difference, limitations against a specific quality model industry & academic IT systems maintainability C# software, IT systems software comparison of quality models benefits vs limitations, main characteristics, quality model origin academic not defined quality in general open source software creation of a reuse model purpose, scope, description academic not defined quality in general, reuse software classification of program analysis rules into quality model scope, origin or use of previous quality model industry Industry, Healthcare, Energy, Infrastructure and Cities sectors internal software quality software code Statistical analysis of metrics interactions in quality model purpose, scope, relationship between academic not defined quality in general software comparison of quality models purpose, scope, metrics, characteristics, quality dimensions, quality model origin academic government quality in use, quality of services websites, web-portal, services comparison of quality models and consolidation of quality characteristic definition description, scope, characteristics, quality model type between basic, tailored and open source academic not defined quality in general software Study of quality models for impact analysis Approach (i.e., quantification method), quality characteristics academic software product line maturity code cloning in object-oriented software Survey on machine learning quality models Description, approach / modeling method academic not defined quality prediction, reliability faut-prediction software comparison / evaluation of quality models quality characteristics, specific metrics relying academic Green IT greenness, reliability IT software, service survey on quality in model-driven engineering quality definition categories (e.g., quality model for MDWE, framework, quality in academic not defined quality in general model-driven engineering creation of new model scope, purpose, brief academic software-intensive sustainability software description systems model transformation), Systematic review of software description, quality academic not defined quality in general agile software development author references Survey on quality models and quality attributes, and creation of new quality model description, purpose, scope, type, quality characteristic mapping academic not defined quality in general embedded software quality aspects with regards to characteristics / sub-agile characteristics, link with agile software development Systematic mapping of software most frequent academic not defined quality in general software characteristics and metrics to build quality model comparison of quality models complete description, presence of quality characteristics academic not defined product quality, process quality Systematic review on quality characteristics, creation of new quality model quality characteristics for systems of systems, belonging to ISO 25010 industry & academic systems of systems quality in general software on characteristics presence aggregation for greenness and reliability creation of new model description, scope, quality characteristics industry & academic Information Systems, financial sector general quality, process information systems software product quality assessment models occurrences of metrics, quality characteristics, aggregations and evaluation method, software Systematic mapping study on software product quality evaluation scope, approach, origin of previous quality model academic not defined product quality, process quality software creation of new model structure, approach, limitations, characteristics academic Information Systems, Web-based information systems quality in general software architecture description, characteristics commerce application creation of new model purpose, scope, approach, limitations, characteristics academic Scientific computing infrastructures reliability distributed software survey on quality models for websites against ISO 9126 quality characteristics quality characteristics mapping against ISO 9126 quality characteristics academic not defined quality in general websites survey on software selection based on quality model quality characteristics academic government organization, IT systems system quality, information quality and service quality. open source software survey on complexity and completeness quality model evolution against ISO 25010 quality characteristics quality characteristics, specific metrics relying on characteristics presence aggregation academic not defined quality in general software survey on software product quality evolution and perspective description, purpose, perspective / viewpoint, Systematic review on quality models purpose, structure, quality model origin, quality characteristics, completeness of quality model definition academic not defined quality of services websites, web-portal, services creation of new model description, quality characteristics academic not defined usability software systems comparison of quality models description, limitations, quality characteristics academic not defined quality in general software creation of new model Quality model operationalization: support of tool, automation, and reconfiguration academic Open source project analysis and comparison open source software creation of new model scope, quality in use quality characteristics academic not defined quality in use Software (FLOSS) Mapping against maturity process quality model process area academic IT systems, quality model goal, characteristics, scope / coverage, meta-model element Systematic mapping study on description, quality industry not defined quality in general, meta-model software government process software quality characteristics medical application) creation of new model description, purpose, quality characteristics, academic ambient assisted living (AAL) system quality in general AAL software, embedded software modeling of quality models automobile, nuclear power plants and various robotic from MATLAB systems (aircrafts, Free / Libre and Open Source evaluation method), creation of new model comparison of results academic embedded systems, reliability, reliability growth model embedded software, software Systematic mapping of software quality measurements Origin model (ISO 9126, ISO 25010, McCall, combined), quantification / characteristics Approach (i.e., academic business of information and technology quality in general software business analysis on quality characteristics requirement engineering & business analysis on quality business analysis (BA) requirement engineering & influence of Engineering (RE), measurement academic not defined quality in general software comparison / evaluation of quality models quality characteristics, specific metrics relying on characteristics presence aggregation for greenness and reliability characteristics academic not defined greenness, reliability software viewpoint, measurement model method, tool Survey on influence of quality characteristics, academic Requirement quality in general software creation of new model description, quality characteristics with regards to usability academic not defined usability software Systematic mapping of software product quality assessment models most frequent occurrences of quality characteristics, assessment / validation approach quality academic not defined quality in general software learning systems frequency, perspective survey on quality models of e-quality indicator academic e-learning systems quality in general e-learning software, website Study of model libraries evolution purpose, scope academic not defined syntactic, semantic, pragmatic, emotional UML model libraries for software comparison / evaluation of software quality description of two ISO 9126-based framework, results against real use-case academic mobile device, mobile network product quality, quality in use, quality of service mobile application software component, citation occurrence total and year-wise software comparison of quality models most well-known / cited quality model for academic not defined quality in general software component, component model-based Analysis of quality model use most well-known quality model academic small IT quality in general IT software comparison / evaluation of quality models description, comparison of quality model results against real use-case industry & academic sustainment management systems maintainability, reliability, security open source software, commercial software in C# in quality model elaboration index quality models techniques method, metrics, dataset used survey on software product quality evolution and perspective description, purpose, stakeholder, perspective / academic not defined quality in general software used for internal quality purpose, quality characteristics mapping against ISO 25010, approach / modeling method comparison of maintainability description, approach / academic not defined maintainability object-oriented software Survey on software quality attributes description, quality blockchain, web) characteristics academic not defined quality in general cloud computing, web-based application (WBA) occurrences in IT technologies virtual reality, augmented reality, artificial intelligence, creation of new model Occurrence of ISO 25010 quality characteristic academic IT, service-based systems (internet of things, green IT, IT software, service creation of new model purpose, scope, academic b2c, mobile quality in general mobile software applications comparison of quality models description, limitations, quality characteristics academic not defined quality in general software component creation of new model purpose / scope, characteristics academic industry, nuclear facilities, flight control system maintainability software systems creation of new model most well-known quality model, maintainability sub-metrics and statistical techniques author, year, citation), characteristics academic web server maintainability open-source, software characteristics Exploratory study of measure against feature models description, quality characteristics related to maintainability academic software product line maintainability, variability feature model, model based designed Systematic literature review on quality model related to open-source software scope, purpose, quality characteristics mapping against ISO 25010, approach / modeling method academic not defined quality in general open source software (OSS) Systematic mapping study on brief description (title, academic not defined internal quality object-oriented software Comparison of quality models, description, quality academic e-commerce quality in general tools websites, web-portal creation of new quality model characteristics, assessment result against use case creation of new model architectural quality characteristics academic systems of systems quality in general, performance, Study on quality in software evolution and creation of new model description, scope, quality characteristics linked to evolution of academic mobile applications, open-source market success, evolution, cohesion, (in)stability, portability, understandability mobile software, open-source software (OSS), object-oriented software quality architecture, design characteristics videogames creation of new model description, limitations, quality academic serious game, education, quality in general programmed serious game ISO 25010 quality characteristics security models processing on quality characteristics against portability, usability, maintainability, compatibility, evaluation types, robustness comparison / evaluation of quality Natural language academic not defined performance efficiency, reliability, software

TABLE 47 -

 47 MAPPING SYSTEMATIC LITERATURE REVIEW STUDIES WITH QUALITY MODELS

	Study Id	Number of quality model	Quality model names
	SLR-S01	11	-Rubey -Hartwick
			-Shooman
			-Jelinski -Moranda
			-Schick -Wolverton
			-Littlewood -Verall
			-Musa
			-Mohanty -Adamowicz
			-Funami -Hastead
			-Klobert
			-Mc Call (FCM)
			-Schneidewind77
	SLR-S02	2	-Khoshgoftaar -Pandya -More
			-Khoshgoftaar -Lanning -Pandya
	SLR-S03	7	-Card -Agresti
			-Gaffney -Durek
			-Agresti -Evanco
			-Process Maturity Framework
			-CMM v1.0 (SEI)
			-Koltun -Hudson
			-SPC Reuse Capability
	SLR-S04	8	-Henry -Kafura
			-Card -Agresti
			-Shepperd
			-Henry -Wake
			-Zage
			-Khoshgoftaar -Lanning -Pandya
			-Khoshgoftaar -Lanning
			-Khoshgoftaar -Allen -Kalaichelvan -Goel

TABLE 49 -

 49 DEFINITIONS OF THE DIFFERENT ELEMENTS OF THE PROJECT SCORECARD Current workweek and year under the form "ww'yy" (where "ww" and "yy" are respectively week and year with 2 digits). e.g.: week of 1st January 2019 is 01'19 Current sprint or PI target workweek and year under the form "ww'yy" (where "ww" and "yy" are respectively week and year with 2 digits). e.g.: week of 1st January 2019 is 01'19 Current SW Gate target workweek and year under the form "ww'yy" (where "ww" and "yy" are respectively week and year with 2 digits). e.g.: week of 1st January 2019 is 01'19Feature NbTotal number of features targeted by the project (including the ones already completed within the project scope)Task NbTotal number of tasks, completed or not, within the project scopeLines of CodeTotal number of lines of code done within the project scope Project Indicators Risk Current global project risk level done by risk manager, or project manager. Possible values: low, med, high Trend Current global project trend compares to previous week and done by project manager. Possible values: , ,  Completion Percentage of completed sprint or PI scope targeted by current target; may be linked to current project management approach Scope Creep Percentage of changes, continuous or uncontrolled growth in a project's scope (different from feature creep), at any point after the project begins. This metric can be achieved using variation metrics Lead time Mean time of entire process crossing (e.g., from to requirement specification to code in use); may depend on project development methodology Fix response time Mean time to fix defect (cycle time) Issues out of delay rate Percentage of issues that are not addressed for more than a certain fixed delay: 15 days SW Product Quality Consolidated percentage of SW product quality. This should be computed (weighted mean if weights are defined) based on proper quality model and all measured quality characteristics/sub-characteristics Automation rate Cumulated percentage of spec (SRS, SAD) and code covered by automated tests Coverage (1) At ECU level, Coverage reflects if a strategy has been decided or not (value=100% or 0%). At higher levels, Coverage reflects the ratio of underlying ECUs for which a strategy has been decided. (3) At lower levels, Coverage reflects the ratio of applicable and planned Gate criteria for current SW Gate Completeness Ratio of evaluated outcomes (green, orange, red) or criteria with regards to. expected outcomes or criteria Consistency Ratio of non-failing outcomes (green, orange; i.e., good quality level, action in place or derogation of evaluated criteria) with regards to expected outcomes or evaluated criteria Percentage of aggregated (i.e., can be either weighted or unweighted mean) "Percentage of Software Unit covered by Software Unit tests", "Percentage of SAD covered by Integration tests" and "Percentage of SRS covered by Qualification tests" Traceability Percentage of End to End (SRS to qualification test) traceability Percentage of SCDR (i.e., System Control Design Review) #AE reviews completed (i.e., evaluated with chairman decision) considering that SCDR number is directly linked to every approved software change request or package of change request according ECU development process Percentage of SCDR #0 reviews completed (i.e., evaluated with chairman decision) considering that SCDR number is directly linked to every approved software change request or package of change request according ECU development process Percentage of SCDR #2 reviews completed (i.e., evaluated with chairman decision) considering that SCDR number is directly linked to every approved software change request or package of change request according ECU development process Percentage of SCDR #3 reviews completed (i.e., evaluated with chairman decision) considering that SCDR number is directly linked to every approved software change request or package of change request according ECU development process Percentage of SCDR #4 reviews completed (i.e., evaluated with chairman decision) considering that SCDR number is directly linked to every approved software change request or package of change request according ECU development process Input Requirement Number of open Input requirement K1 bugs and K2 bugs. These bugs are directly linked to requirements input (i.e., STRCOmp) coming from system SRS Number of open Software Requirements K1 bugs and K2 bugs. These bugs are directly linked to SRS SAD Number of open Software Architecture K1 bugs and K2 bugs. These bugs are directly linked to SAD SW Rule Violations Number of open Software Rule K1 bugs and K2 bugs. These bugs are directly linked to Software Rule violations detected when applying and executing MXAM on model or Static Analysis on source code (e.g., with QAC or CodeSonar tools). Number of open Software K1 bugs and Software K2 bugs not listed in the other categories (i.e., Software Input Requirement, Software Architecture, Software Rule Violations and Legal violations) IP Scan Violations Number of open IP Scan K1 bugs and K2 bugs. These bugs are directly linked to IP Scan violations detected when applying and executing IP Plan.

	Scorecard Category	Name	Definition
	Name		Project name
	Current WW'Y
	Target WW'Y
	Project
			Name of current targeted Software Gate. Possible values: SW_G1, SW_G2,
	SW GATE		SW_G2, SW_G3, SW_G4, SW_G5, SW_G6, SW_G7, SW_G8, SW_G9, SW_G10,
			SW_G11
	SWG Target WW'Y
	Load level	Load level of project: GM (Grand Mother), M (Mother), B (Brother) & C (Child)
	Project Dimensions

  Pass rate: percentage of passed Software Development Tool tests over planned25 Software Development Tool tests,SW UnitRun rate: percentage of executed Software Unit tests over planned25 Software Unit tests, Pass rate: percentage of passed Software Unit tests over planned 25 Software Unit tests, Regression rate: percentage of currently failing Software Unit tests, which previously passed, over Software Unit tests that passed, SW Component Run rate: percentage of executed Software Component tests over planned 25 Software Component tests, Pass rate: percentage of passed Software Component tests over planned25 Software Component tests,23 Open bug.: other than closed and verified bug 24 K1 and K2 represents bug criticality25 Planned: with regards to current Software Gate scope.[K1/K2]24 RegulationNumber of open Regulation related K1 bugs and K2 bugs. Doesn't include Safety related bugs. percentage of executed Safety related software Qualification tests over planned25 Safety related software Qualification tests, Pass rate: percentage of passed Safety related software Qualification tests over planned 25 Safety related Software Qualification tests, Regression rate: percentage of currently failing Safety related software Qualification tests, which previously passed, over Safety related software Qualification tests that passed Regulation Run rate: percentage of executed Regulation related software Qualification tests over planned 25 Regulation related software Qualification tests, Pass rate: percentage of passed Regulation related software Qualification tests over planned 25 Regulation related software Qualification tests, Regression rate: percentage of currently failing Regulation related software Qualification tests, which previously passed, over Regulation related software Qualification tests that passed Quality Sub-Characteristic Definition Functional Appropriateness Degree to which the functions facilitate the accomplishment of specified tasks and objectives.

		Run rate:
	Safety
	Test Rate
	[Run/Pass/Reg.]
		Run rate: percentage of executed Software Development Tool tests over
	SW Dev Tool	planned 25 Software Development Tool tests,
	Test Rate
	[Run/Pass/Reg.]

  Annex 13. Details about the eight selected software quality models for the software quality model genome meta-model construction -Boehm's quality model[START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF] (1976)

	Terminology Terminology	Definition Definition
	Accuracy
			3	high-level qualities
	This model is composed of	7	intermediate-level qualities
			15 primitive quality characteristics
					1.333333333	Included	0.333333333
		Fault Tolerance		0.666666667	0.666666667	Excluded
		Recoverability		1	1	Included		0.25
		Confidentiality		1.333333333	1.333333333	Included
		Integrity		1.333333333	1.333333333	Included
	Security	Non-Repudiation		0	0	Excluded	Excluded characteristic
		Accountability		1	1	Included
		Authenticity		0.333333333	0.333333333	Excluded
		Modularity		1.333333333	1.333333333	Included
		Reusability		1.333333333	1.333333333	Included
	Maintainability	Analysability		1	1	Included	Excluded characteristic
		Modifiability		1	1	Included
		Testability		1.333333333	1.333333333	Included
		Adaptability		1.333333333	1.333333333	Included		1
	Portability	Installability		0.666666667	0.666666667	Excluded	1.333333333
		Replaceability		0.333333333	0.333333333	Excluded
		Usefulness		1.333333333	1.333333333	Included	0.444444444
		Trust		1.666666667	1.666666667	Included	0.555555556
	Satisfaction	Pleasure		-0.666666667	-0.666666667	Excluded	3
		Comfort		-0.666666667	-0.666666667	Excluded
	Effectiveness	Effectiveness		1.5	1.5	Included	1.5	1
	Freedom From Risk	Economic Risk Mitigation Health and Safety Risk Mitigation		1 1.333333333	1 1.333333333	Included Included	Excluded characteristic

  concernant l'évolution des modèles qualité (c'est-à-dire les modèles qualité de base avant 2000 et les modèles qualité adaptés depuis 2000). En effet, nous avons montré que cette évolution s'articule autour de trois périodes : jusqu'en 1990, nous avons la période des modèles qualité de base, de 1990 à 2003, la période de transition, et depuis 2003, nous sommes dans la période de personnalisation des modèles qualité. connaissances des modèles qualité existants, qui pourrait être utilisé comme base pour développer un nouveau modèle qualité.En continuant avec la génétique, nous avons constaté qu'un certain niveau d'analogie pouvait être atteint entre les séquences d'ADN et les caractéristiques de qualité et la séquence des sous-caractéristiques. En outre, comme dans les séquences d'ADN, des variations des caractéristiques de qualité et des sous-caractéristiques peuvent exister avec un certain niveau de probabilité, ce qui rappelle le concept de polymorphisme. Ainsi, la base de notre contribution au méta-modèle qualité reposait sur cette analogie qui a également été reprise dans une ontologie du méta-modèle.

On May

The software development process is based on V-model[22].

In the previous Automotive-SPICE version, up to 2.5, the recommendation was to use ISO/IEC 9126[START_REF]Part1: Quality Model[END_REF].

Ieeexplore, ACM digital library, Springer, Scopus, and Web of Science

When we indicate presence, requirement must be implemented and working as expected, with absence it is when feature is missing or doesn't work as expected.

Perceived and not perceived quality.

Mode determines the value which occurs the most often; It is also the only way to determine an average with nominal scale.

European Programme for Research and Development in Information Technologies: https://cordis.europa.eu/project/id/300

Reliability include storability, faultless operation, maintainability and durability ; Published via a Russian regulatory documents (GOSTs) decree[START_REF] Azgaldov | The ABC of Qualimetry, toolkit for measuring the immeasurable[END_REF].

WordNet Princeton University 2010: https://wordnet.princeton.edu/

Comparison between these two quality models are also available in Annex A of ISO/IEC/IEEE 25010[START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]

Document focus is on software domain, Sources of quality model citation are not provided Document is referencing multiple quality models Document is not only citing, enumerating quality models Document contains a study, analysis, survey, or comparison on quality models, with a minimum or argumentation Document is not a poster, cover, course, conference, or paper review Document contains some description and details on enumerated quality models Document is not a retracted publication Document is from ranked 14 conferences, or journals Full document is not accessible or downloadable Document is not in English 14 Ranking is obtained via Core Computing Research & Education Portal: http://www.core.edu.au/conference-portal

System to address real-world activity

In the previous Automotive-SPICE version, up to 2.5[START_REF] Automotive | Automotive SPICE Process Assessment, version 2.5[END_REF], the recommendation was to use ISO/IEC 9126[START_REF]Part1: Quality Model[END_REF]

https://ieeexplore.ieee.org/abstract/doc ument/268955

https://ieeexplore.ieee.org/abstract/doc

https://ieeexplore.ieee.org/abstract/doc ument/1215871

https://onlinelibrary.wiley.com/doi/abs/ 10.1002/smr.366

https://ieeexplore.ieee.org/abstract/doc ument/5440283

A line may contain zero (i.e., comment), one or more statements.

Enfin, la dernière catégorie de problèmes englobe la modélisation du domaine d'intérêt contemporain de la qualité et où certaines études de recherche ont été lancées mais n'ont pas encore résolu le problème. Nous pouvons citer par exemple : o L'écologie et la durabilité des logiciels, o Le vieillissement et l'obsolescence des logiciels, o Les données de qualité pour les systèmes et services logiciels connectés.

Acknowledgments 3 | P a g e Je dédie cette thèse à ma femme, Estelle, à mes filles, Alwena, Loanne et Erell, À mes parents, Joëlle et Gérard, à mon frère Xavier, à toute ma famille Ainsi qu'aux gens que j'aime.

Qu'à travers ce travail, ils y voient un reflet de ma pensée et de moi-même !

Acknowledgments

During the years of my thesis, I was able to meet and have the support of a number of people, from academia and industry. It would obviously be too long to list all these people, and even if I do not quote them here, I do not forget them.

First of all, I would like to thank Mrs Claude BARON, University Professor at INSA of Toulouse, responsible for the ISI team at LAAS-CRNS, and Mr Philippe ESTEBAN, Lecturer at the University Paul Sabatier of Toulouse. Thanks to their welcome, supervision, support and encouragement, these three years of thesis could be carried out under the best conditions in spite of many pitfalls encountered. I also thank them very much for their kindness and their

My early career, as a research and development engineer in the Computer Science and Applied undoubtedly reinforced my deep interest in research. This is the reason why I completed my training with a DEA in the GRAVIR laboratory (Joint Research Unit between CRNS, the National Polytechnic Institute of Grenoble, INRIA and the Joseph Fourier University). Then I started a doctoral thesis in one of the laboratories of the "Center for Research and Education in Optics and Lasers" of the University of Central Florida -UCF-in the United States, accumulating a grant from the "National Institute of Health" and one from Lockheed-Martin , before making the choice to join an American start-up.

Availability 0.125 site 10 Locability

Availability

Degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use.

Availability

1.333333333

Accessibility Code possesses the characteristic of accessibility to the extent that it facilitates selective use of its parts.

(Examples: variable dimensioned arrays, or not using absolute constants.) Accessibility is necessary for efficiency, testability, and human engineering.

Accountability

Code possesses the characteristic of accountability to the extent that its usage can be measured. This means that critical segments of code can be instrumented with probes to measure timing, whether specified branches are exercised, etc. Code used for probes is preferably invoked by conditional assembly techniques to eliminate the additional instruction words or added execution times when the measurements are not needed.

Accuracy

Code possesses the characteristic of accuracy to the extent that its outputs are sufficiently precise to satisfy their intended use. Necessary for reliability Usability (As-is utility) Code possesses the characteristic of usability to the extent that it is reliable, efficient, and human engineered.

Augmentability

Code possesses the characteristic of augmentability to the extent that it can easily accommodate expansion in component computational functions or data storage requirements. This is a necessary characteristic for modifiability.

Communicativeness

Code possesses the characteristic of communicativeness to the extent that it facilitates the specification of inputs and provides outputs whose form and content are easy to assimilate and useful.

Communicativeness is necessary for testability and human engineering.

Completeness

Code possesses the characteristic of completeness to the extent that all its parts are present, and each part is fully developed.

Conciseness

Code possesses the characteristic of conciseness to the extent that excessive information is not present.

Consistency

Code possesses the characteristic of internal consistency to the extent that it contains uniform notation, terminology, and symbology within itself, and external consistency to the extent that the content is traceable to the requirements.

Device independence

Code possesses the characteristic of device independence to the extent it can be executed on computer hardware configurations other than its current one. Clearly, this characteristic is a necessary condition for portability.

Efficiency

Code possesses the characteristic of efficiency to the extent that it fulfills its purpose without waste of resources.

Human engineering

Code possesses the characteristic of human engineering to the extent that it fulfills its purpose without wasting the users' time and energy or degrading their morale. This characteristic implies accessibility, robustness, and communicativeness.

Legibility

Code possesses the characteristic of legibility to the extent that its function is easily discerned by reading the code. (Example: complex expressions have mnemonic variable names and parentheses even if unnecessary.) Legibility is necessary for understandability.

Maintainability

Code possesses the characteristic of maintainability to the extent that it facilitates updating to satisfy new requirements or to correct deficiencies.

Modifiability

Code possesses the characteristic of modifiability to the extent that it facilitates the incorporation of changes, once the nature of the desired change has been determined. Note the higher level of abstractness of this characteristic as compared with augmentability.

Portability

Code possesses the characteristic of portability to the extent that it can be operated easily and well on computer configurations other than its current one.

Reliability

Code possesses the characteristic reliability to the extent that it can be expected to perform its intended functions satisfactorily

Robustness / Integrity

Code possesses the characteristic of robustness to the extent that it can continue to perform despite some violation of the assumptions in its specification.

Self-containedness

Code possesses the characteristic of self-containedness to the extent that it performs all its explicit and implicit functions within itself. Examples of implicit functions are initialization, input checking, diagnostics, etc.

Self-descriptiveness

Code possesses the characteristic of self-descriptiveness to the extent that it contains enough information for a reader to determine or verify its objectives, assumptions, constraints, inputs, outputs, components, and revision status. Commentary and traceability of previous changes by transforming previous versions of code into non-executable but present (or available by macro calls) code are some of the ways of providing this characteristic. Self-descriptiveness is necessary for both testability and understandability.

Structuredness

Code possesses the characteristic of structuredness to the extent that it possesses a definite pattern of organization of its interdependent parts.

Testability

Code possesses the characteristic of testability to the extent that it facilitates the establishment of verification criteria and supports evaluation of its performance.

Understandability

Code possesses the characteristic of understandability to the extent that its purpose is clear to the inspector. Availability Availability = 100% site Availability Availability = 50% (1/2) Time among failures = 50%

List of Acronyms

To calculate the quality characteristics importance value from these answers, and then determine which of them are included or rejected (i.e., we remind that inclusion criteria is "importance value ≥ +1"), we apply the rules explained in Chapter VI. 4.b together with the 5 point Likert' scale described in Figure 58. So, over the 13 quality characteristics, we found that six of them must be included in the quality model. We proceed the same for the quality sub-characteristics to calculate the importance values and the decision to include or exclude them. Next is to evaluate another time all excluded characteristics, taking into account the influence of their sub-characteristics. In our example we have 7 characteristics excluded and therefore candidate for this further consideration. We detail below the case for the "Usability" characteristic.

Its following sub-characteristics and their importance value computed as above: Secondly, the similar quality characteristic groups are merged together in order to avoid, or at least to reduce, the lexical and semantic overlap between quality characteristic groups. This behavior is not only aligned with the concept of word constellations but also with the gene variations of same genetic character. For example, "Efficiency" and "Performance efficiency" have similar definition and sub-characteristics. Therefore, we can conclude that they both cover the same quality characteristic concept and be regrouped then under "Efficiency".

The regrouping results are shown in Table 33, and its consequence is a decrease of 12 distinct quality characteristic groups, moving from 55 to 43 distinct quality characteristic groups for the same quality modeling coverage. Moreover, 27 over the 43 groups (i.e., 62.28%) come from only one quality model, and 10 over the 43 groups (i.e., 23.26%) are quality characteristic groups found at least in four of the eight quality models, with three of these groups (i.e., "Efficiency", "Reliability" and "Usability") present in the eight quality models.

TABLE 33 -THE 10 MERGED QUALITY CHARACTERISTIC GROUPS (GREEN BACKGROUND CELL INDICATES A MERGED ENTRY)

Total Quality Characteristic Name

Boehm [START_REF] Boehm | Quantitative Evaluation of Software Quality[END_REF] McCall [41] FURPS [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF] Alvaro [START_REF] Alvaro | A Software Component Quality Model: A Preliminary Evaluation[END_REF] Bawane [START_REF] Bawane | A Novel Method for Quantitative Assessment of Software Quality[END_REF] Kalaimagal Q'FActo 12 [START_REF] Kalaimagal | Q'Facto 12: an improved quality model for COTS components[END_REF] ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF] 2

c. "Product Revision" Chromatic (A17)

The product revision perspective identifies "quality factors that influence the ability to change the software product" (source: http://www.sqa.net/softwarequalityattributes.html and McCall et al. [41]). This chromatid is composed of 15 quality characteristic genes linked together as depicted by Figure 90, subset of the graph shown in Figure 84, and its complete "DNA" sequence made of 69 quality characteristic sites disclosed in Figure 91.

Chapter IX. General Synthesis and Research Perspectives

The objective of the thesis conducted us to define a theoretical framework for the supervision and piloting of engineering processes and product development through quality. This work has led to multiple contributions, partially valued in the form of two international conference papers (cf. Argotti et al. [START_REF] Argotti | Quality quantification in Systems Engineering from the Qualimetry Eye[END_REF], [START_REF] Argotti | Quality Quantification Applied to Automotive Embedded Systems and Software[END_REF]). Above all, it was highlighted the need to have an exhaustive and in-depth study of existing quality models in the literature in order to be able to go further in consolidated conceptual and methodological proposals, and before being able to consider any application.

The purpose of this final thesis chapter is to conclude the three research work years synthetized in this document. Consequently, in the next two sections, we are successively reviewing a synthesis of the research path together with the related contributions, and then explore the resulting main research perspectives.

General synthesis

In Chapter I, we posed the thesis research problematic, "Study of Qualimetry essentials applied to embedded software development", that we rephrased into "strengthen and unify the definition, assessment, control, or prediction of the embedded software quality", with regard to our industrial context. We initiated its analysis, identifying the four following research questions:

Research Question 1

Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials?

Research Question 2 Considering the set of software quality models, how to identify and decide which quality model is the most suitable for embedded software?

Research Question 3 Considering a quality model for a software product, how to operationalize it?

Research Question 4 Can we have a unique reference quality model for software product?

We deepened then this analysis through Chapter II. We remarked that the automotive industrial context jointly with the vehicle as complex system, the development model with suppliers, and the current standard and regulation requirements are raising the overall complexity of our problematic. Therefore, in this context it is critical to have a unified, operational, and appropriate way to define, assess, control, or predict quality of embedded software.

In order to verify if such unified, operational and appropriate solution for quality of embedded software already existed, we performed an exploratory literature review about "how quality modeling is applied to embedded software". We found the existence of a myriad of possible embedded systems and software, each of them with their own specificities, quality characteristics, and possibly a diversity of quality models. So, it appeared that there was no right and unique solution yet to our question.

Consequently, we refined these four research questions into 15 research sub-questions and then detailed our research methodology in Chapter III. In this chapter, furthermore, we explained our research methodology realignment highlighting not only the difficulties in the selection of a proper quality model for embedded software, but also the consequences of such selection in discarding many valuable contributions.

Afterwards, we addressed these research questions from Chapter IV to Chapter VIII.

Research Question 1 Is Qualimetry, as the science of quality quantification, the right approach and what are quality and Qualimetry essentials?

Thus, through Chapter IV, we explored the essence of quality (i.e. research sub-question 1a), and quality modeling particularly in software field (i.e., research sub-questions 1b). We defined, clarified these knowledges and the related concepts (e.g., perceived quality, quality perspectives, quality dimensions and characteristics, quality model, measurements, scale), and concluded this exploration with the build of the first timeline of the key contributions to software quality modeling, going from 1965 with the first appearance of software engineering Annex 3. Example of distance calculation: diversity or polymorphism degree applied to ISO/IEC/IEEE 25010 and ISO/IEC 9126 quality models

This annex presents the calculation of quality model distance based on degree of polymorphism, described in Chapter IV.6.c and defined by the equations 6 and 7, between the two consecutive standards for quality model in software engineering: ISO/IEC 9126 [START_REF]Part1: Quality Model[END_REF] and ISO/IEC/IEEE 25010 [START_REF]:2011 -Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models[END_REF]. We note that a comparison between these two quality models are also available in the Annex A of ISO/IEC/IEEE 25010.

To initiate the calculation, we have to identify the sequences, usually linked to alleles in genetic field, and their frequencies. So, in the current case we consider two sequences, each of them directly associated to either ISO/IEC 9126 or ISO/IEC/IEEE 25010. Moreover, we assume that both model frequencies are identical, that is to say there is equal chance to use for software one quality model or the other. Thus, both frequencies are equal to 50%.

Next step is to enumerate the two sequences together (i.e., list their quality characteristics and quality subcharacteristics) and identify whether lexically and semantically they are identical, similar, or different. In this last situation, we mark this as a gap like gap in a DNA sequence. So, the enumeration combined with difference results are: Functional Suitability Degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions.

Performance Efficiency

Performance relative to the amount of resources used under stated conditions.

Compatibility

Degree to which a product, system or component can exchange information with other products, systems or components, and/or perform its required functions, while sharing the same hardware or software environment.

Usability

Degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

Reliability

Degree to which a system, product or component performs specified functions under specified conditions for a specified period of time.

Security

Degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization.

Maintainability

Degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers.

Portability

Degree of effectiveness and efficiency with which a system, product or component can be transferred from one hardware, software or other operational or usage environment to another.

o Quality sub-characteristic definitions  Functional Suitability Quality Sub-Characteristic Definition Functional Completeness Degree to which the set of functions covers all the specified tasks and user objectives.

Functional Correctness

Degree to which a product or system provides the correct results with the needed degree of precision. The purpose of this basic set of measures is to enable the use operations of the real-world use case quality models. This set should be reviewed, updated, and completed as needed, especially to integrate the temporal polymorphism behavior due to the evolution over the development life cycle stages. These measures are extracted from ISO/IEC 25023 [START_REF] Florenskii | Some Remarks on Product Quality Assessment[END_REF] for system / software product quality measures, and from ISO/IEC 25022 [START_REF] Chrisman | Rethinking levels of measurement for cartography[END_REF] for quality in use measures. Further details are available in the standard documentation.

-System / Software product quality perspective This model is composed of 3 perspectives 11 quality factors 23

quality criteria

Terminology Definition

Access audit Those attributes of the software that provide for an audit of the access of software and data.

Access control

Those attributes of the software that provide for control of the access of software and data.

Accuracy

Those attributes of the software that provide the required precision in calculations and outputs.

Communication commonality

Those attributes of the software that Interoperability provide the use of standard protocols and Interface routines.

Communicativeness

Those attributes of the software that provide useful inputs and outputs which can be assimilated.

Completeness

Those attributes of the software that provide full implementation of the functions required.

Conciseness

Those attributes of the software that provide for implementation of a function with a minimum amount of code.

Consistency

Those attributes of the software that provide uniform design and implementation techniques and notation.

Correctness

Extent to which a program satisfies its specifications and fulfills the user's mission objectives.

Data commonality

Those attributes of the software that provide the use of standard data representations.

Efficiency

The amount of computing resources and code required by a program to perform a function.

Error Tolerance

Those attributes of the software that provide continuity of operation under nonnominal conditions.

Execution efficiency

Those attributes of the software that provide for minimum processing time.

Expandability

Those attributes of the software that provide for expansion of data storage requirements or computational functions.

Flexibility

Effort required to test a program to insure it performs its intended function.

Generality

Those attributes of the software that provide breadth to the functions performed.

Instrumentation

Those attributes of the software that provide for the measurement of usage or identification of errors.

Integrity (security)

Extent to which access to software or data by unauthorized persons can be controlled.

Interoperability

Effort required to couple one system with another.

Machine independence

Those attributes of the software that determine its dependency on the hardware system.

Maintainability

Effort required to locate and fix an error in an operational program.

Modularity

Those attributes of the software that provide a structure of highly independent modules.

Operability

Those attributes of the software that determine operation and procedures concerned with the operation of the software.

Portability

Effort required to transfer a program from one hardware configuration and/or software system environment to another.

Reliability

Extent to which a program can be expected to perform its intended function with required precision.

Reusability

Extent to which a program can be used in other applications -related to the packaging and scope of the functions that programs perform.

Self-Descriptiveness

Those attributes of the software that provide explanation of the implementation of a function.

Simplicity

Those attributes of the software that provide implementation of functions in the most understandable manner. (Usually avoidance of practices which increase complexity.)

Software system independence

Those attributes of the software that determine its dependency on the software environment (operating systems, utilities, input/output routines, etc.)

Storage efficiency

Those attributes of the software that provide for minimum storage requirements during operation.

Testability

Effort required to modify an operational program.

Traceability

Those attributes of the software that provide a thread from the requirements to the implementation with respect to the specific development and operational environment.

Training

Those attributes of the software that provide transition from current operation or initial familiarization.

Usability

Effort required to learn, operate, prepare input, and interpret output of a program.

-FURPS quality model [START_REF] Grady | Software Metrics: Establishing a Company-Wide Program[END_REF] (1987)

This model is composed of -ISO/IEC 9126 quality model [START_REF]Part1: Quality Model[END_REF] (1991)

This model is composed of 2 quality perspectives 10 quality characteristics 31

quality sub-characteristics

Time behaviour

The capability of the software product to provide appropriate response and processing times and throughput rates when performing its function, under stated conditions.

Understandability

The capability of the software product to enable the user to understand whether the software is suitable, and how it can be used for particular tasks and conditions of use.

Usability

The capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions.

Usability compliance

The capability of the software product to adhere to standards, conventions, style guides or regulations relating to usability.

-Bawane quality model [START_REF] Bawane | A Novel Method for Quantitative Assessment of Software Quality[END_REF] (2010)

This model is composed of 2 quality perspectives 11 quality characteristics 28 These gene and site variations are determined by gathering the quality characteristics and sub-characteristics from the quality models identified in the first step of the meta-model construction. This regrouping is done based on lexical and semantic such as in Motogna et al. study [94]. An example around "Portability" gene is shown in TABLE 29 and TABLE 30. The complete gene and site enumeration with their related possible variations and corresponding likelihood is given by TABLE 50 andTABLE 51. In TABLE 50, each gene is present only in one quality model. Thus, their sites contain only one possible variation in each gene context. Nevertheless, we considered that a gene can be part of another gene even resulting from several quality models and then seen also as a site. Consequently, that gene can have variations. For instance, "Changeability" gene is resulting from only one single quality model. However, as a site "Changeability" can be retrieved in "Maintainability" gene.