Process-oriented Risk Assessment Methodology for Manufacturing Processes Evaluation

Keywords: Risk assessment, process modeling, objectives modeling, discrete event simulation and decision-making support system

A process-oriented quantitative risk assessment methodology is proposed. Risks involved in a process and the corresponding risk factors are identified through a novel objectiveoriented risk identification approach and analyzed qualitatively in the Process FMEA. Critical risks with high RPN (risk priority number) values identified in the FMEA are incorporated in the process model for further quantitative analysis in the simulation environment. To this end, risk measures of the critical risks are determined for which data is collected during simulation execution. Using the proposed methodology as a decision-making tool, alternative scenarios are developed and compared with each other as well as with two fictive situations to elicit normalized values for the risk measures employing value function approach. To ease the decision-making process in case of multiple risk measures, a global risk indicator is developed by aggregating the individual normalized risk measures through a 2-additive Choquet Integral operator. The global risk indicator is then used to rank the alternative scenarios on the basis of desirability. The methodology is illustrated with a case study issued from parts manufacturing but is applicable to a wide range of other processes.

Introduction

In the process context, risk stems from uncertainty regarding the ability of a process to deliver "the value proposition"and the consequences thereof. The value proposition refers to the implicit or explicit promise a company makes to its stakeholders to deliver a particular combination of values [START_REF] Martinez-Hernandez | Understanding value creation: The value matrix and the value cube[END_REF]. To make sure the promise would be delivered; process risks should be modeled and assessed to account for the uncertainties and their consequences on the "value proposition". Risk management is carried out in the same spirit to protect and maximize the process value for its stakeholders (Shah et al. 2012;[START_REF] Sienou | A method for integrated management of process-risk[END_REF]).

To deal with risk in product/system/project context, various tools and techniques have been developed so far. An exhaustive review of these tools and techniques can be found in the literature [START_REF] Tixier | Review of 62 risk analysis methodologies of industrial plants[END_REF][START_REF] Moeller | COSO Enterprise Risk Management: Understanding the New Integrated ERM Framework[END_REF] and risk management standards [START_REF] Knight | AS/NZS ISO 31000: 2009-the New Standard for Managing Risk[END_REF] ISO/IEC31010:2009). These techniques have different goals; some of them are used to evaluate multiple scenarios, depending on which risky event occurs such a decision tree, strength, weakness, opportunity and threat analysis (SWOT), SWIFT Analysis and What-If Analysis (Cagliano et al., 2015). Other techniques, instead, focus on the origins and implications of the risk events. Commonly used techniques in this category are: failure modes and effects analysis [START_REF] Braaksma | A quantitative method for Failure Mode and Effects Analysis[END_REF][START_REF] Case | A diagnostic service tool using FMEA[END_REF], fault tree analysis [START_REF] Cheng | Application of fault tree analysis to assess inventory risk: a practical case from aerospace manufacturing[END_REF][START_REF] Limnios | Fault trees[END_REF], event tree analysis [START_REF] Aven | Risk analysis: assessing uncertainties beyond expected values and probabilities[END_REF]. In addition to these analytical techniques, Monte Carlo simulation provides a statistical approach towards risk assessment [START_REF] Lee | Risk modelling and assessment for distributed manufacturing system[END_REF][START_REF] Moghaddam | Supplier selection and order allocation in closed-loop supply chain systems using hybrid Monte Carlo simulation and goal programming[END_REF][START_REF] Mun | Modeling Risk: Applying Monte Carlo Simulation, Real Options Analysis, Forecasting, and Optimization Techniques[END_REF].

These techniques have their strengths and weaknesses and are aimed at different application domains. For instance, FMEA is often used in aeronautical, automotive and other industries producing machinery, electronic or electro-mechanical components [START_REF] Su | A systematic methodology for the creation of Six Sigma projects: A case study of semiconductor foundry[END_REF]. Hazard and Operability (HAZOP) technique, however, is commonly applied in the chemical industry [START_REF] Dunjó | Hazard and operability (HAZOP) analysis. A literature review[END_REF][START_REF] Labovskỳ | HAZOP study of a fixed bed reactor for MTBE synthesis using a dynamic approach[END_REF]. Furthermore, Hazard Analysis and Critical Control Points (HACCP) is developed specifically for the beverage and food industry [START_REF] Pierson | HACCP: Principles and Applications[END_REF][START_REF] Sun | A review of the needs and current applications of hazard analysis and critical control point (HACCP) system in foodservice areas[END_REF]. Fault tree and event tree techniques have been applied widely in the nuclear power plants and also in the chemical industry [START_REF] Groso | Risk analysis in research environment[END_REF]. Similarly, the CREA (Clinical Risk and Error Analysis) technique is developed for medical domain [START_REF] Trucco | A quantitative approach to clinical risk assessment: The CREA method[END_REF].

Irrespective of the application domain, often the risk analysis methods are developed to model and analyze risks from a subsystem perspective. A system is reduced to subsystems or component parts and each part is then dealt independently. For example, in FMEA only one component is considered at a time and the rest are assumed to work perfectly. Therefore, the method is not suitable to model critical combinations of component failures (Aven, 2008, p. 64).

Same is the case with many of the risk assessment techniques [START_REF] Groso | Risk analysis in research environment[END_REF]. However, a failure may not happen because of one risk event but a combination of mutually inclusive risk events may lead to one or more failures [START_REF] Shah | Process-Oriented Performance Evaluation: Concepts and Methodology: Value and risk based approach[END_REF].

To address this issue, one way is to approach the risk assessment problem via process modeling and simulation. Because processes are everywhere no matter what organization or scenario one chooses. They manifest whenever a service or product is delivered. Among the many benefits of process-based initiatives are process improvement, process analysis for possible problems and issues and performance measurement [START_REF] Indulska | Business Process Modeling: Current Issues and Future Challenges[END_REF]. But, the risk management discipline and thus the techniques thereof have been developed separately from the operational concern of the business processes (i.e. manufacturing, sales, inventory management etc.). Although risk is the element that can affect (business) process negatively, it is still managed independently [START_REF] Shah | Process-Oriented Performance Evaluation: Concepts and Methodology: Value and risk based approach[END_REF]. A sufficient link is missing between process and quantitative risk models [START_REF] Scott | Managing risks in enterprise systems implementations[END_REF]. To better manage risk of a process, it should be integrated, and evaluated dynamically (through simulation) within the process to account for the uncertainties and chain of events intertwined with activities of a process.

Recently, some research work has been carried out to either treat process risk or integrate risk in the process perspective. Two streams of studies have emerged: the management of risk in (business) processes [START_REF] Cope | Incorporating risk into business process models[END_REF][START_REF] Neiger | Supply chain risk identification with value-focused process engineering[END_REF][START_REF] Sienou | A method for integrated management of process-risk[END_REF][START_REF] Suriadi | Current research in risk-aware business process management: overview, comparison, and gap analysis[END_REF] and process-based risk management [START_REF] Kayis | A Risk Mitigation Methodology for New Product and Process Design in Concurrent Engineering Projects[END_REF][START_REF] Cha | Process-oriented approach for validating asset value for evaluating information security risk[END_REF][START_REF] Kusiak | Risk assessment of process models[END_REF][START_REF] Sackmann | A Reference Model for Process-Oriented IT Risk Management[END_REF][START_REF] Yu | A Business Process-based Risk Evaluation Framework[END_REF]. The former is called risk-aware business process management and the latter is process-oriented risk management. In any case, this convergence of risk management and process management is a positive development for maximizing the value of the process. It is not surprising given the tendency of process based organization design in the last two decades [START_REF] Aguilar-Savén | Business process modelling: Review and framework[END_REF][START_REF] Dumas | Fundamentals of business process management[END_REF][START_REF] Hammer | Reengineering the corporation: A manifesto for business revolution[END_REF].

Furthermore, the importance of such a research has also been confirmed in a number of studies [START_REF] Becker | Exploring the status quo of business process modelling languages in the banking sector-an empirical insight into the usage of methods in banks[END_REF][START_REF] Chapman | Simple Tools and Techniques for Enterprise Risk Management[END_REF][START_REF] Rikhardsson | Business process risk management and internal control: A proposed research agenda in the context of compliance and ERP systems[END_REF]. Nevertheless, the research and practice of risk management in association with processes is still very limited and requires further exploration. The proposed techniques have not yet made inroads into the industry despite the fact that process based management and frameworks are widely practiced nowadays (e.g.

supply chain operations reference model, SAP ERP system).

To advance the theory of risk in the process context, this study proposes a process-based risk management methodology. The central theme of the study is to provide methods and techniques for risk assessment and management for process-based organizations. The methodology draws on, for the most part, the risk modeling concepts in the process context for the risk measurement model from the literature [START_REF] Larson | Managing design processes: A risk assessment approach[END_REF] and extends it to integrate risk identification, evaluation and decision-making mechanisms to better manage risk of industrial systems.

RISK MEASUREMENT MODEL

In the process perspective, uncertainty lies "internally" and "externally" to the activity in the form of risk factors. External risk factors can be machine breakdown, unreliable supplier, natural disaster etc. Internal risk factors of an activity is associated to the uncertainty regarding the estimation of its work content [START_REF] Salah | On the fallacy of averages in project risk management[END_REF]. For instance, assigning an activity (operation) having low capability index to meet a specific objective (e.g. specification of a product) or underestimating activity duration. Whether external or internal, a risk factor can trigger the occurrence of any undesirable events that ultimately affect the activity progress towards objective(s) attainment (cf. Figure 1).

Risk factor

Risk event Objective Causes Influences

1*… 1*… 1*… 1*…
Figure 1: Risk factors and objectives relationship [START_REF] Vernadat | VR-PMS: a new approach for performance measurement and management of industrial systems[END_REF] It is therefore essential to identify and then assess the impact of risk events originating from the risk factors (both external and internal) on the defined process objectives. For this purpose, risk 𝑅 , in the first place, is parameterized in terms of risk scenario 𝑆 , likelihood 𝑃 and consequence 𝐶 as expressed by Equation 1:

𝑅 = (𝑆, 𝑃, 𝐶) (1)
Next, it is modeled and assessed in the activity environment. The underlying assumption for activity-based risk assessment is that each activity in a process is exposed to risk factors RFi, ( 𝑖 = 1, … . , 𝑛) and thus can trigger one to many risk events, which in turn can affect the fulfilment of process objectives (as illustrated in Figure 2). The global risk for activity 𝑖 due to risk event 𝑗 is given by Equation 3:

R i = ∑(P ij × C 𝑖𝑗 𝑜𝑏𝑗 ) J j=1 (3)
Where, R i is the risk magnitude of activity i However, all risk events are not equally important in any given scenario. Some risks are more important than others depending on what objective(s) they are influencing. Therefore, a weighting factor or importance index 𝑑 𝑖𝑗 is introduced to model the importance of the risk events in the scenario 𝑆. So, Equation 3 can be rewritten as:

R i = ∑ 𝑑 𝑖𝑗 (P ij × C 𝑖𝑗 𝑜𝑏𝑗 ) J j=1 (4)
Therefore, the global risk of the process path 𝑃 𝑘 is given by Equation 5:

R(p k ) = ∑ R i ∀ i∈p k = ∑ ∑ 𝑑 𝑖𝑗 (P ij × C 𝑖𝑗 𝑜𝑏𝑗 ) J j=1 ∀ i∈p k (5)
Where k = 1, .2,…K, representing a process in the network of processes (scenarios)

In the case of multiple sub-processes (AND, OR and XOR junctions in the context of IDEF3), special cases apply.

In the case of multiple parallel sub-processes (AND junction), since all successor activities immediately after the junction are performed, so the probability 𝑃 𝑟 (𝑃 𝑘 ) of occurrence of each path set 𝑃 𝑘 is one, so Equation ( 5) is applied to all possible path set (parallel sub-processes) in order to calculate the process risk of the scenario understudy.

For mutually exclusive alternative sub-processes (XOR Junction), since only one subsequent sub-process after branching can be performed, the summation of all probabilities of occurrence of the sub-processes therefore should be equal to 1. So the probability of the path set (or scenario) in the mutually exclusive setting is such that:

∑ 𝑃 𝑟 (𝑃 𝑘 ) K k=1 = 1 (6)
So, the expected risk of the process 𝑃 made of 𝐾 path sets is:

𝐸(𝑅 𝑝 ) = ∑ 𝑃 𝑟 (𝑃 𝑘 ) K k=1 ( ∑ ∑ 𝑑 𝑖𝑗 (P ij × C 𝑖𝑗 𝑜𝑏𝑗 ) J j=1 ∀ i∈p k ) (7)
Regarding mutually inclusive alternative sub-processes (OR Junction), the probability (𝑃 𝑟 ) of any process 𝑘 after branching is unconditional and is calculated differently. If the number of alternative sub-processes (𝑁) after branching is greater than 2, then there are 2 𝑁-1 combinations of subsequent sub-processes. Let us assume that 𝑈 𝑗 is the set containing sub-processes in combination 𝑗 and 𝑀 is the total number of all possible combinations thus the probability of performing only one subsequent sub-process in a combination 𝑗,𝑃 𝑟 (𝐶 𝑗 ) is calculated from Bayes' formula in Equation( 8)

Pr(𝑈 𝑗 ) = ∏ 𝑃 𝑖 ∀ 𝑖 ∈𝐶 𝑗 × ∏ (1 -𝑃 𝑖 ) ∀ 𝑖∉ 𝐶 𝑗 1 -∏ (1 -𝑃 𝑖 ) 𝑁 𝑖=1 (8)
So the process risk for the whole process becomes;

𝐸(𝑅 𝑝 ) = Pr(𝑈 𝑗 ) * ( ∑ ∑ 𝑑 𝑖𝑗 (P ij × C 𝑖𝑗 𝑜𝑏𝑗 ) J j=1 ∀ i∈p k ) (9)
It is highly likely that a single scenario have both the conjunctive and disjunctive (inclusive and exclusive) branching and therefore a combination of Equations ( 5), ( 7) and ( 9) are used to model the risk of the whole scenario.

RISK ASSESSMENT METHODOLOGY

Having defined the mathematical model for risk measurement in process setup, a risk assessment methodology is presented in this section. The proposed methodology relies on modeling, simulation and decision-making approaches and establishes a risk management process consisting of the following three phases:

1. Context establishment 2. Risk modeling and analysis

Risk analysis via simulation

Risk monitoring and control phase is not included in the methodology because the current study focus is solely on the risk assessment. The assessment phase consists of risk identification and risk evaluation. The risk assessment phase is supported with process analysis technique such as simulation. Each phase of the methodology is detailed in the following subsections.

Context establishment phase

Context is about defining external and internal parameters to be considered when managing risk and setting the scope and risk criteria for the risk management policy (ISO Guide 73:2009). In the framework of the current methodology, the context establishment phase identifies the application domain, the stakeholders of the process, their roles and responsibilities as well as the upper and lower bounds for each stakeholder's expectations and concerns. Moreover, the focus of the risk assessment is also determined in the context establishment phase; whether the assessment is about a project or a business activity, merger or acquisition or overseeing a new production facility (Chapman, 2006, p. 109).

In the framework of the current study, the context for risk management is set up using process models which are further enriched with statements about assets (name, value, type, and risk tolerance), stakeholders and relations between assets and stakeholders in terms of interest including stakeholders appetite of each stakeholder [START_REF] Sienou | A method for integrated management of process-risk[END_REF].

Risk modelling and analysis phase

To account for risks in processes, the current methodology proposes a two pronged strategy:

objectives-driven risk identification, and activity-based approach to risk assessment. The objective-driven approach is important because risk cannot be ascertained fully if objectives are unclear as risk by definition is the effect of uncertainty on business objectives (ISO 31000:2009).

However, risk evaluation without (temporal) contextual information is incomplete and the process models enriched with additional statements are therefore required to establish the context.

3.2.1

Objectives-oriented risk identification The general approach to risk identification starts with objectives identification (COSO, 2007).

Relying on this theory, an objective-driven approach to risk identification is to find an assertion that obstructs the objectives attainment. For instance, if the global objective of a process is to satisfy customer order, then the corresponding global risk would be: "failure to satisfy customer order". Many possible dimensions of the global risk are identified by decomposing it into lower level risks while employing principles of Value Focused Thinking (VFT), a decision-making methodology developed by Keeney (1992).

Within the VFT framework, an objective is "a statement of something that one wants to strive toward" (Keeney, 1994). Keeney categorizes objectives into fundamental and means objectives.

The former concerns the ends that the decision-makers (DM) value in a specific decision context, whereas the latter refers to the "methods to achieve fundamental objectives". For detail review and applications of VFT, the interested reader is referred to (Parnell et al. 2013).

In the context of risk management, the fundamental objectives of the VFT are "risk objectives" whereas, the means objectives will be termed "risk factors". Because, the theme of the risk management is to "minimize certain risk" and here the minimization has a preference orientation so it is an objective per VFT technique. Also, similar to means objectives, which are the ways to achieve the fundamental objectives, risk factors are the means that determine the likelihood of a risk occurrence. A chain/network of risk factors contributes to risk objectives in the risk hierarchy. For the sake of simplicity, risk objective is simply called risk in the current study.

To apply VFT for risk identification and structuring, the principle of "specification" is used.

Specification generally aims to break down an objective into logical parts (Keeney, 1996, p. 68).

A simple question: "What do you mean by that" is sufficient to specify, refine and characterize the upper level risk into lower level. This decomposition mechanism transforms global risk into more measurable risks while keeping in view the objectives of the process understudy. For example, the question "what do you mean by failure to satisfy customer order?" brings forward lower level risks such as technical performance risk, incomplete order fulfillment risk, schedule risk and cost overrun, which correspond to quality, reliability, on-time delivery and cost objectives of the process. To refine further, schedule risk means delay in order processing, manufacturing and logistic or distribution. Each part (tasks) of the process (order satisfaction) has its own deliverables and associated risks. It is noteworthy that each risk on the lower level is a dimension or specification of the upper level risk and assumed to holds the axiom of preferential independent [START_REF] Kirkwood | Preference Conditions for Multiattribute Value Functions[END_REF]. That is risk of order processing is independent of logistic risk and also of manufacturing related risks. But all risks when added together contribute to scheduling risk. Figure 3 shows a risk hierarchy for customer order satisfaction process.

Failure to satisfy customer order

Performance risk

Schedule risk Cost overrun

Logistics risk

Mfg. delay Order processing delay The risk hierarchy and risk factor networks are sufficient to map all the relevant risks and their causes in any given scenario. However, the identified risks with their risk factors cannot be assessed because they lack the contextual information. For this purpose, they are further investigated in the (business) process environment where they may happen.

Incomplete order delivery

Process-based risk analysis

The identified risks of the risk hierarchy are assessed using the FMEA technique. For this reason, the required contextual information is obtained from the process model. Because the process model contains sufficient information regarding the execution of the activities and their environment, they are therefore consulted when analyzing a particular risk. For instance, without knowing the type of machine/tool and operator to be employed to machine a critical feature of a mechanical part, it is difficult to know whether the risk may be triggered or not and, hence, risk assessment cannot be performed. It is therefore essential to identify those activities responsible for (in the case of internal risk factors) or prune to (in the case of external risk factors) the identified risks. To this end, a risk/activity matrix is developed which links the risks of the risk hierarchy to the activities of the process model (cf. Table 1). For instance, 𝑅 5 represents the "serviceability risk", a subset of performance risk and is associated to 𝐴𝑐𝑡𝑖𝑣𝑖𝑡𝑦 1 and 𝐴𝑐𝑡𝑖𝑣𝑖𝑡𝑦 3 (facing and tapering operations respectively, see (Shah, 2012, p. 242)). Assume 𝑅 1 to be a manufacturing lead time risk and linked with 𝐴𝑐𝑡𝑖𝑣𝑖𝑡𝑦 2 and 𝐴𝑐𝑡𝑖𝑣𝑖𝑡𝑦 3 due to process novelty and complexity which results into a manufacturing delay.

Similarly, all risks of the risk hierarchy are linked to responsible activities of the process scenario. In the next step, they are analyzed qualitatively using Process FMEA technique. Table 2 shows a subset of schedule risk analyzed in the FMEA. Here, risk event of the manufacturing lead-time risk is analyzed and RPN is calculated. The objective is to determine whether it is worth further analysis or not. To analyze risk events of the process model quantitatively, the risk measurement model described in section 2 is employed in simulation experiment.

Risk analysis via simulation phase

Quantitative risk evaluation requires quantitative data regarding risk event parameters which can be obtained from the historical records employing statistical analysis (Ahmed et al. 2007).

However, in the a priori evaluation of processes, such data do not usually exist. If they exist, they are either not enough or not in the shape to be used. In such circumstances, simulation is the right tool that can generate sufficient data regarding risk parameters that the simulation model can use later to estimate risk measures. To carry out simulation experiments for this purpose, input data for simulation are generated.

Input data gathering:

The input data required for risk analysis in the simulation environment can be divided into three categories: functional data (activities), parameters to make the conceptual process model executable and risk evaluation methods. The former data are obtained in the context establishment phase by process modeling of the scenario using IDEF3 method. For the parameters, it is advisable to divide them into several categories such as run parameters (job arrival law, arrival type: batch or single entity arrival, warm-up period and so on) process plan parameters (i.e. operation times) and objectives related parameters (i.e. order quantity) as described in (Shah et al., 2012;Shah, 2012). Concerning the evaluation methods (see Table 3), they are formula required to analyze the scenario. For computing the risk of cost overrun, an upper bound (UB) for the cost is defined. Cost values falling beyond the upper bound will trigger risk event; however, its impact is a function of 𝑥 i.e.

deviation of the outcome from the upper bound. The more the value deviates, the higher is the risk. It may be a linear or non-linear function to a point beyond which the impact can grow exponentially. Same is the case for the schedule risk. A customer can wait for an order to arrive up to certain time beyond that either the penalty applies (a function of 𝑥; where 𝑥 is delay time after due date). For the quality risk, product specifications are important. Tolerances are defined and any feature beyond tolerance may impact the proper functioning of a part or product as a whole and hence the improper tolerance can trigger a performance or technical risk.

Simulation experimentations and output data collection:

Having defined the input data, the risk aware process model (conceptual model) is transformed into a simulation model. The simulation parameters and evaluation method are fed into the simulation model. In the probabilistic setup, the simulation model generates random data for each parameter and estimates risk measures of interest (cf. Table 3). They are used in the risk evaluation phase to pass on judgment on each risk measure w.r.t the defined risk criteria.

In the case of multiple risk measures of heterogeneous nature (cost vs. time), it is advisable to develop a commensurate scale defined on the interval [0,1]. The evaluation, then, of different risk measures will make sense. Multi-criteria decision making techniques can be employed for the purpose.

Risks evaluation and aggregation via multi-criteria decision-making phase

Risk evaluation means estimating significance or judging the acceptability of the risk. The significance or judgment of a risk can only be established when compared against its target value, upper and lower bounds. In addition, risk measures obtained from simulation experiments are heterogeneous in nature and most often large in numbers. Therefore, it is desirable to consider multiple criteria decision-making technique(s) which develops a normalized value function (also called utility function) for each risk measure. This is carried out by comparing each risk measure against its UB and LB. It also set the stage for aggregation of the normalized risk measures (henceforth, risk expressions) to facilitate decision-making process.

In the multi-criteria model of the current methodology, risk measures are normalized using the MACBETH method. Determination of interaction among risk measures is carried out using Choquet capacity while aggregation is performed using the 2-additive Choquet integral.

MACBETH MACBETH (Measuring Attractiveness by a Categorical Based Evaluation TecHnique

) is a multi-criteria decision analysis approach used to determine value functions as well as aggregated ones while comparing different scenarios (Bane e Costa and Vansnick, 1999). Similar to value functions, risk functions set reflects decision-makers' (DMs) preferences or judgments of risk criteria with reference to fictive situations. Two situations are compared pairwise for a risk criterion and thus ordinal information is obtained, which is then transformed into cardinal information through "difference of attractiveness". The preference through difference of attractiveness is quite natural to decision-makers who usually rely on verbal levels of attractiveness such as {null, very weak, weak, moderate, strong, very strong, extreme} (Bana e [START_REF] Bana E Costa | MACBETH -An interactive path towards the construction of cardinal value functions[END_REF]. Details about the MACBETH method application can be found in the literature [START_REF] Cliville | Quantitative expression and aggregation of performance measurements based on the MACBETH multi-criteria method[END_REF].

In the current study, the MACBETH procedure is used to develop risk expressions 𝑟 𝑖 (normalized value of individual risk of 𝑅 𝑖 ) by solving inter criteria commensurability issue and to map them on a [0, 1] scale. Because MACBETH relies on the weighted mean to aggregate risk expressions, which is often not the case in real-life examples where criteria may interact, the Choquet integral has been chosen as the operator for the aggregation of risk expressions. It can handle interdependencies among different risk functions through Choquet capacity.

Choquet Integral (CI)

To aggregate the risk functions 𝑟 𝑖 , the following 2-additive Choquet integral mathematical model is used [START_REF] Grabisch | A review of methods for capacity identification in Choquet integral based multi-attribute utility theory:: Applications of the Kappalab R package[END_REF]:

C u (x) = ∑ v i r i - 1 2 n i=1 ∑ I ij |r i -r j | n i=1 (10)
Where 𝐶 𝑢 models vectors of risk expressions 𝑟 𝑖 , 𝑣 𝑖 denotes a Shapely index with ∑ 𝑣 𝑖 = 1 𝑛 𝑖=1

that represents importance of risk 𝑟 𝑖 relative to all other risks expressions and 𝐼 𝑖𝑗 , interaction between risk expressions (𝑟 𝑖 , 𝑟 𝑗 ), ranging in [-1,1]. To calculate the unknown CI parameters (𝑣 𝑖 & 𝐼 𝑖𝑗 ), two fictive situation are considered besides alternative processes and are ranked pairwise and individually (Clivillé et al., 2007;[START_REF] Shah | Process-Oriented Performance Evaluation: Concepts and Methodology: Value and risk based approach[END_REF]. A preference ranking of these situations along with the strength of preferences gives a system of equations whose solution determines the Choquet Integral parameters.

In such preference modeling, the alternatives satisfy one or two risk expressions simultaneously.

The situations where only one 𝑅 𝑖 = 1 and all others are equal to zero, the aggregated value is as follows:

R 𝑎𝑔 𝑖 = 𝑣 𝑖 - 1 2 ∑ 𝐼 𝑖𝑗 𝑛 𝑗=1 𝑗≠𝑖 (11)
The aggregated value of the situations where one 𝑅 𝑖 =0 and all other equals to 1 will be as follows:

R 𝑎𝑔 𝑖 = 1 -𝑣 𝑖 - 1 2 ∑ 𝐼 𝑖𝑗 𝑛 𝑗=1 𝑗≠𝑖 (12)
In brief, the decision-making techniques are used to solve the following problem

𝑉(𝑅) = 𝐹(𝑣(𝑟 1 ) + 𝑣(𝑟 2 ) … 𝑣(𝑟 𝑛 )) (13)
Where, 𝑟 ∈ 𝑅 An attribute or risk measure of set 𝑅(risk set from the risk hierarchy)

APPLICATION

The methodology is illustrated on a case study about a manufacturing company that designs and fabricates product on make-to-order (MTO) basis. The reference product for this case study is a mechanical locator (Figure 6), a work holding device used for centering of job on a machinetool. To illustrate the use of the proposed methodology within the company, a manufacturing scenario is defined.

Axle

Case study: Manufacturing Scenario

The company under study receives an order for 200 high quality mechanical locators from a customer with a lead time of two weeks (10 working days). The price of the product is kept at $14 /per unit. So, the most critical objective in the scenario is technical performance of the product, measured in terms of satisfaction index (q) (Anselmetti, 2008).

The company has sufficient resources at its disposal, therefore parts: axle, body and cap are machined at the facility while springs and bolts are purchased from the market. We assume that all raw materials and the purchased parts are available whenever needed.

Failing to satisfy the customer order will cause penalty cost. A product having satisfaction index below 0.8 are rejected. In case of delay after the tolerance period (2 days), the company has to pay $2/unit time tardiness up to 5 working days for each product, beyond this period the order is cancelled and backlog cost of $10 per unit is paid. In addition, upper bound cost is set to $18.)

Having described the manufacturing scenario, the proposed risk assessment methodology is then applied to determine the level of risk involved in the scenario.

Application of the methodology to the mfg. scenario

To assess risks involved in the manufacturing scenario, the company generates manufacturing process plans for the reference product. For this purpose, the methodology described in [START_REF] Sormaz | Generation of alternative process plans in integrated manufacturing systems[END_REF] have been adopted. To develop alternative process plans, the product is decomposed into geometrical features. To manufacture each feature, process candidates are selected using inquiries to the appropriate knowledge base of the manufacturing processes. Once the process plans are generated, they are then modelled using IDEF3 modelling method.

To identify risks and the corresponding risk factors, objectives-oriented risk identification approach is used. A global risk: "failure to satisfy customer order" for the scenario is fixed and decomposed into schedule risk, cost overrun and performance risk. Since critical objective for the customer order fulfillment is the technical performance of the product, therefore, this facet of the risk is further decomposed into more refined facets. In the second step, risk factors for each identified risk are determined. The identified risks and risk factors are then structured into risk hierarchy and risk network respectively similar to Figure 3 and Figure 4. To determine risk parameters for the risk events, they are first contextualized using activity/risk matrix. Once linked, they are then analyzed in the process FMEA High RPN risks are integrated into the process plan model for further analysis in the simulation environment. To calculate the probability of a risk event, it is sufficient to observe and record at a critical activity (whose risk is of concern) in the simulation model; the number of times the activity objective is not achieved divided by the total number of observations made. For determining the impact of a risk event, an impact function 𝑦 = 𝑓(𝑥) is calculated. For example, in case of delay, the formula: 𝑦 = 𝑎. 𝑒 𝑏𝑥 is employed where a and b are the coefficients and 𝑥 is the dependent variable indicating the range which exceeds the due date. Similarly, the cost impact function is calculated assuming a quadratic function (i.e. per unit cost overrun 𝑥 has an impact 𝑥 2 ). For the quality failure impact, the severity scale of the FMEA is used.

For quantitative analysis of the risk events, the risk aware process plan models are transformed into executable simulation models. They are then simulated using the discrete event simulation software: Rockwell ARENA v.13.5. Ten independent replications for each process plan were run. In each replication run, process plan and scenario related parameters were loaded into the simulation model and risk measures estimates are obtained as presented in Table 5. bounds for each measure are defined. (In the case of cost, SIB "Smaller Is Better" means the lesser the cost, the better the utility and for the quality, NIB "Nominal Is Better" means the target value or nominal is preferable). So "Good" situation means SIB for cost and schedule while NIB for quality. Next, process plans and the two fictive situations are ranked in the desirability order along with strength of preferences (ℎ 𝑖 , 𝑖 = 1 𝑡𝑜 6) as follow:

𝑅 1 ⇒ 𝐺𝑜𝑜𝑑 > 1 𝑃𝑃1 > 2 𝑃𝑃2 > 5 𝑃𝑃3 > 1 𝑁𝑒𝑢𝑡𝑟𝑎𝑙 𝑅 2 ⇒ 𝐺𝑜𝑜𝑑 > 1 𝑃𝑃3 > 4 𝑃𝑃2 > 5 𝑃𝑃1 > 1 𝑁𝑒𝑢𝑡𝑟𝑎𝑙 𝑅 3 ⇒ 𝐺𝑜𝑜𝑑 > 3 𝑃𝑃3 > 1 𝑃𝑃1 > 2 𝑃𝑃2 > 1 𝑁𝑒𝑢𝑡𝑟𝑎𝑙
The preference modeling information is next transformed into risk expressions using MACBETH procedure. The resulting risk expressions are then populated in Table 6. 12)are put in Equation ( 10) and global risk for each process plan is calculated ( Table 7). 7, it can be concluded that manufacturing process plan 1 involves the least risk (0.53) followed by process plan 3 (0.56) and process plan 2 with a maximum risk score of 0.59.

Furthermore, the technical performance being the critical risk in the scenario carries more weight with a Shapley index of (0.48) followed by schedule risk (0.34) and cost overrun (0.18).

Similarly, the interaction among schedule & cost, schedule & performance, and cost & performance risks are respectively 0.09 , 0.13 and 0.10 . A comparatively high positive interaction effect is observed between schedule and quality (performance) risks (i.e. 0.13), this is because quality failures lead to manufacturing delay.

CONCLUSIONS AND FUTURE WORK

A process-oriented risk assessment methodology for manufacturing processes has been presented. The methodology establishes a risk identification and process and provides tools and techniques to develop a global risk indicator that can be used a decision-making tool to rank alternative manufacturing processes on the basis of desirability. The methodology is tested with a case study which supports both its conceptual and operational validity. Moreover, the current study stresses on the need to identify major or global risk and then decompose them using specification principle of the Value Focused Thinking technique (VFT). The decomposition of global risk into lower level risks and then identification of the risk factors and linkage of the latter with the former provides a risk structure for the scenario understudy. The risk structuring alone provides an insight and can help risk manager while devising mitigation plans.

In authors' view, risk identification is the most crucial part to any risk management activity, however it is mostly ignored in many risk assessment techniques. Brainstorming and historical data are the tools mostly used for risk profiling but a systematic approach to risk identification is missing in the literature. The current study proposes the risk identification process using objective-driven approach and integrates it with other assessment techniques-process model, FMEA and simulation techniques. For evaluation and ease of decision-making, the risk measures are normalized and aggregated. Although long the methodology is reasonably thorough and handy regarding assessment of risks in the process context.

To further enhance the applicability of the methodology, it will be extended to model the causality relationship between risk factors and risk events via Bayesian network. In addition, cost effective risk mitigation strategy will be incorporated. A risk acceptability zone vis-à-vis value (objectives) will also be defined to make the proposed methodology more robust for decisionmaking.

Figure 3 :Figure 4 :

 34 Figure 3: Risk hierarchy"What causes the risk event to occur?" brings forward the factors for each risk in the lower tier of the risk hierarchy (cf. Figure3). For instance, the manufacturing delay is a fundamental risk but can be triggered by frequent quality failures. So, quality failures are the means (contributing factor) to the manufacturing delay. Other risk factors can be disruption (machine breakdown) which is further related to other risk factors such as improper machine maintenance, overrunning machine capability, ignoring warning signals etc. such are the factors which form chain/network and contribute to machine breakdown. However, machine breakdown is not a fundamental risk per se but a factor to major risk such as manufacturing delay which affects the customer satisfaction objective. In a similar way, risk factors are identified for cost overrun and technical performance. Once done, a network of risk factors is formed and is linked to the objective risk as shown in Figure4

FigureFigure 5 :

 5 Figure 5. 5 (pb)

  𝑣(𝑟) Risk function (risk expression)𝑉(𝑅) Global risk function (global risk indicator)𝐹Aggregation operator (CI) 𝑣(𝑥) or simply 𝑥 𝑖 are determined using the MACBETH procedure while the operator 𝐹 (Choquet Integral in the current study) is employed to aggregate the risk expressions to form global risk indicator.

  Figure 6: Mechanical Locator

;

  L.A. Shah*; A.Etienne*; A.Siadat*; F.B.Vernadat** *LCFC, Arts et Métiers ParisTech; **LGIPM, University of Metz

Table 1 :

 1 Activity/risk matrix

		𝑅 1	𝑅 2	𝑅 3	𝑅 4	𝑅 5	𝑅 6
	Activity 1					
	Activity 2			
	Activity 3					

Table 2 :

 2 Process FMEA for Schedule Risk

	Process	Failure mode	Causes	Effects	P C D	RPN
	Activity i	Duration	Process	Uncertain lead	5	7	6	210
		estimation error	-novelty	time
			-complexity
	By analyzing qualitatively risk events in the PFMEA Table, the most critical ones are then
	integrated into the process model for quantitative analysis in the simulation environment as
	shown in

Table 3 :

 3 Evaluation Methods

	Risk Measures	Calculation Method

Table 4 :

 4 Table (cf. Table 4) Excerpt of PFMEA for quality risks[START_REF] Shah | Process-Oriented Performance Evaluation: Concepts and Methodology: Value and risk based approach[END_REF]

	Processes Requirements	Failure	Causes	Effects	P C D RPN
			mode
	Tapering operation (F2, Axle)	Coax. ⌀ .04mm (req. S9)	Axle cone piece(WP) not locate the work	Improper allocation assembly tolerance	Unable to hold the WP rightly	5 7 6 210
	Parting	Tolerance	Poor	Malfunctioning	Improper assembly	8 5 5 200
	F2 (Cap),	interval 0.04 of	length of	of the	tolerance allocation,
	Facing F7	Plan A w.r.t Plan	the parts	mechanical	wrong machine set
	(Body)	B (req. S1)		locator	up

Table 5 :

 5 Simulation Results for manufacturing process plans Schedule risk (𝑅 1 ) Cost overrun(𝑅 2 ) Since risk measures obtained from simulation experimentations are simply numbers they carry no clear meaning. They are therefore normalized for solving the issue of commensurability among risk measures. For this purpose, first, two fictive situations representing upper and lower

	Performance risk (𝑅 3 )

Table 6 :

 6 Risk expressions (normalized risk measures from MACBETH) Risk expressions 𝑟 𝑖 as well as the CI parameters (𝑣 𝑖 & 𝐼 𝑖𝑗 ) identified via Equations (11) & (

		𝑅 1 expression	𝑅 2 expression	𝑅 3 expression
	Mfg. process plan 1	0.12	0.98	0.66
	Mfg. process plan 2	0.38	0.57	0.78
	Mfg. process plan 3	0.88	0.14	0.56

Table 7 :

 7 global risk indicator for manufacturing process plans Global Risk 𝐼 𝑖𝑗 & 𝑣 𝑖 for risk measures

	𝐼 𝑖𝑗	𝑣 𝑖


Thibault Béziers

La Fosse

Jean-Marie Mottu

Massimo Tisi

Gerson Sunyé

Source-Code Level Regression Test Selection: the Model-Driven Way

Keywords: Regression Test Selection, Regression Testing, Model Driven Engineering, Execution Trace

published or not. The documents may come   L'archive ouverte pluridisciplinaire

Introduction

Regression testing (RT) is an important step in the software development lifecycle. It ensures that code updates do not break the functionalities that have already been suc-cessfully tested. However, the size of regression test suites tends to grow fast [START_REF] Yoo | Regression testing minimization, selection and prioritization: a survey[END_REF] when an application is evolving; thus considerably increasing the testing cost.

According to various studies, up to 80 % of testing cost is related to regression testing, and more than 50 % of software maintenance cost is dedicated to testing [START_REF] Engström | A qualitative survey of regression testing practices[END_REF]. The process of source code compilation, load, and test execution is commonly called a build. When a build is over, a result, successful or not, is returned to the developer. According to this result, the developer will either go to another task or correct the code that has regressed. This is especially true in the context of Continuous Integration (CI), where the regression testing takes place on a separate server [START_REF] Gousios | An exploratory study of the pull-based software development model[END_REF]. During the time of the build, the developer does not know yet if she needs to correct the code (the wait could be long: e.g., 25:33 min including 06:04+18:11 min testing time for the build 372209560 of the Google Guava project1 ). Thus, reducing this duration would improve the development productivity.

Reducing the cost of regression testing by only running a specific subset of test cases is the purpose of Regression Test Selection (RTS). Most of the large variety of RTS techniques are based on change impact analysis [LR03, LHS + 16]. When an application under development is being modified, it might be unnecessary to run all the test cases, especially the ones that are not impacted by changes in the source code. For instance, existing RTS techniques such as Ekstazi [START_REF] Gligoric | Practical regression test selection with dynamic file dependencies[END_REF] are able to reduce the regression test time for the Google Guava project2 to an average of 45 %. A standard RTS approach usually involves three phases:

• (C) Collection of the dependencies between code and test cases.

• (A) Analysis of the changes to select impacted test cases.

• (E) Execution of the selected test cases.

The benefit of running less test cases during phase (E) could be counterbalanced by the overhead introduced by the phases (C) and (A). Some existing approaches such as FaultTracer [START_REF] Zhang | Faulttracer: a change impact and regression fault analysis tool for evolving java programs[END_REF] have shown a RTS time that is longer than the execution of all tests. For that reason, several RTS approaches named offline calculate the phase (C) beforehand. Hence, when the user starts a build, the dependencies are already available for performing phases (A) and (E) with a positive time gain [START_REF] Zhang | Hybrid regression test selection[END_REF].

The overhead introduced by the phase (A) depends on how the dependencies are computed during the phase (A) and analysed during the phase (C). It mainly depends on the precision. A RTS technique is said to be precise if all selected test cases are affected by the changed code [START_REF] Kumar | Regression test selection on system requirements[END_REF] (i.e., no useless test case is ran on unchanged code). Being precise is not mandatory and computing the dependencies at a really fine grain induces a significant overhead [START_REF] Gligoric | Practical regression test selection with dynamic file dependencies[END_REF]. The balance between the precision and its induced overhead should be considered when developing and using RTS techniques.

The precision of RTS techniques depends on the granularity when considering source-code updates. It could distinguish file, class, method, or statement updates. For instance, a class-level update summarises all the modifications inside a class whereas a statement-level update considers modifications of a single statement. The usage of statement-level updates is more precise but induces more overhead than file-level updates. This is the reason why existing RTS approaches [START_REF] Gligoric | Practical regression test selection with dynamic file dependencies[END_REF] have shown faster end-to-end 3 results using file-level changes, despite selecting more tests. Nevertheless, L. Zhang shows that depending on the file updated and the case-study, it could be worth to consider finer method granularity [START_REF] Zhang | Hybrid regression test selection[END_REF]. Therefore, by designing a modeldriven RTS approach, we can provide modularity and allow the tester to manage the granularity and optimise the precision.

Independently from the precision, RTS techniques must be safe, meaning that they should select every test case that is impacted by changes in the code [START_REF] Rothermel | Analyzing regression test selection techniques[END_REF].

While most of the related work on RTS is dedicated to ensuring precision or performance of language-specific and RTS dedicated tools, in this paper we focus on providing a solution that leverages Model-Driven Engineering (MDE) for improving RTS in terms of its reusability and interoperability. In the past decade, MDE has shown that the use of models improves the development in terms of productivity, quality, and reuse [START_REF] Whittle | The state of practice in model-driven engineering[END_REF]. The accuracy of model-driven program analysis techniques has proven this approach to be suitable for tracing down the execution of programs [START_REF] Bruneliere | Modisco: a generic and extensible framework for model driven reverse engineering[END_REF][START_REF] Winkler | A survey of traceability in requirements engineering and model-driven development[END_REF] and can benefit to RTS. First, the modularity of MDE allows for a configurable approach where the RTS precision can be set as required. Second, regression testing is one step among all the software development steps, MDE aims to prevent each step to be independent by sharing the models. Therefore, we propose a model-driven RTS approach that collects the execution traces in an impact analysis model. That model is used all along the RTS and then it can be exchanged, completed, and reused for different purposes (e.g., debugging, performance analysis, optimisation). Such model can be connected to the other models of the MDE environment, allowing for instance to trace regressions from the requirements model or UML models, etc. Finally, regressions often happen because of changes in different artefacts around the code, such as resources, configuration files, or data files. The holistic view fostered by MDE would enable to address all these problems in a uniform way based on the impact analysis model.

The risk of applying MDE in RTS is its potential performance overhead, in particular for building the model [START_REF] Selic | The pragmatics of model-driven development[END_REF]. In this paper we show that this cost can be lightened by developing an offline model-driven RTS technique: costly impact analysis model creation is anticipated. Our resulting tool significantly reduces the execution time of regression testing in our case studies, by up to 32 %. While state-of-the-art dedicated RTS tools may have better raw performance, we argue that the reusability of our computed models makes our solution especially valuable in MDE environments.

In this paper, we propose a model-driven approach for a highly precise and safe RTS, using statement-level impact analysis, in order to select test methods impacted by code changes. An impact analysis model is built offline. When a user requests a build, this model is queried to select all the tests impacted by changes. Finally, this reduced set of test cases is executed, thus accelerating the regression testing time. The approach is based on a customisable model storing the execution traces of the tests. If the fine-grain accuracy of this model may be slowing down the RTS, it is a powerful asset for other software engineering tasks, hence improving its reusability.

To evaluate this approach we aim at answering three research questions:

• RQ1: Does our MDE RTS approach allow for a safe and precise RTS?

• RQ2: Is our MDE RTS approach efficient enough to significantly reduce the regression testing time?

• RQ3: Does our MDE RTS approach offers a better modularity and reusability than existing approaches?

The remainder of this paper is organised as follows: Section 2 presents the related work, Section 3 describes a motivation example, then Section 4 introduces our approach, followed by an evaluation driven by several experimentations in Section 5. Finally, Section 6 discusses our proposal and Section 7 concludes this paper and presents future work.

Related Work

RTS has been extensively studied, leading to multiple systematic studies and surveys [ERS10, YH12, GHK + 01, LHS + 16]. In the following we describe a few representative examples of state-of-the-art tools. The approaches for RTS can be classified according to their granularity.

Coarse-grained regression test selection. Several popular RTS frameworks are coarse-grained, considering file or class granularity. They result in less overhead than a finer-grained one, and tends to be more scalable for bigger systems. Ekstazi has a 3-phases approach: test selection using file-dependencies [START_REF] Gligoric | Practical regression test selection with dynamic file dependencies[END_REF], test execution, and new dependencies collection, using bytecode instrumentation. Ekstazi is safe for both code changes and file-system changes. It is among the most efficient state-of-the-art RTS solutions, with an average time reduction of 47 %.

Fine-grained regression test selection. Orso et al. present a RTS technique that uses two levels of impact analysis [START_REF] Orso | Scaling regression testing to large software systems[END_REF]. The first level is coarse-grained, and accounts for relationships between classes, by generating a graph representing inheritance and references. This phase allows for change impact computation at declaration level. The second level generates control-flow graphs at statement level, for each revision of the code. Coverage information is gathered during the execution of tests, and a coverage matrix is generated. Such matrix highlights the code statements that are executed by a specific test. Finally, by comparing control-flow graphs of two revisions of the same source code, changed statements are gathered, and the test cases impacted are fetched using the coverage matrix. The empirical evaluation shows that running the fine-grained analysis produces a high overhead.

Zhang et al. present FaultTracer, a RTS tool using Chianti's change impact analysis [START_REF] Zhang | Faulttracer: a change impact and regression fault analysis tool for evolving java programs[END_REF] for a method-accurate test selection. It generates a set of extended control-flow graphs, and collects the dependencies at the edge-level by running test methods. Tests impacted by changes are then computed using those graphs. Despite showing accurate results, FaultTracer produces a high overhead, and thus computing the impact analysis is sometimes longer than executing all the test cases.

Lingming Zhang proposes HyRTS, an hybrid RTS combining both method granularity and file granularity [START_REF] Zhang | Hybrid regression test selection[END_REF]. Using both the cost-effectiveness of file granularity, and the accuracy of the method-level RTS, HyRTS outperforms Ekstazi and other state-of-the-art RTS techniques.

Our model-driven approach is modular and can be set to consider a fine-grained impact analysis as accurate as the state-of-the-art fine-grained RTS techniques or a coarser-grained granularity. This results in the possibility offers to the tester to manage the precision of RTS to get a faster test execution, at the expense of a much slower impact analysis, including the model-management overhead. However, differently from related work, we propose a model of execution trace of the code. This model is persisted, customisable, and available to be reused by other model-driven tools for other types of analysis. Finally, we compute this model beforehand, in order to effectively accelerate the regression testing duration when triggered by the user. While we are not aware of any MDE-based tool for RTS of general-purpose languages, there is some recent work on MDE-based management of execution traces, especially for DSLs [START_REF] Bousse | Advanced and efficient execution trace management for executable domain-specific modeling languages[END_REF][START_REF] Hegedus | Back-annotation of simulation traces with change-driven model transformations[END_REF]. We differ in addressing by similar techniques the traces of general-purpose programs. Analysis of execution traces is a very active research area in software engineering (e.g., recent works are [START_REF] Alimadadi | Inferring hierarchical motifs from execution traces[END_REF][START_REF] Goldstein | Experience report: Log-based behavioral differencing[END_REF]) and process engineering (e.g., [START_REF] Jagadeesh | Process diagnostics using trace alignment: opportunities, issues, and challenges[END_REF]). We believe that a model-driven management of traces can enhance the interoperability of such solutions.

Running Example

Figure 1 presents an example that is used to define the context of our model-driven approach for RTS. Figure 1a and Figure 1b define a first and a second revisions of a Java program, respectively. They are verified using the JUnit testing framework.

Granularity

Methods

Class Method Statement

test_mA_0 x x x test_mA_1 x x test_mB x x x test_mC x
Table 1 -Selection of test methods depending on the granularity of source-code updates

In the first revision, R 1 , the test class TestCA verifies that the class CA works as expected. Four test cases are implemented: test_mA_0, test_mA_1, test_mB, and test_mC. The first two tests cover the two branches of the if condition in mA. The third one tests the method mB, and the last one tests the method mC.

The source code modifications of revision R 2 update several lines. The first change occurs at line 6, where the statement inside the else branch is updated. The second change occurs at line 8, by renaming method mB to mD. Finally, the last modification considers the new method name mD inside the corresponding test case.

Orso et al. define two different kinds of program changes, statement-level changes and declaration-level changes [START_REF] Orso | Scaling regression testing to large software systems[END_REF]. A statement-level change can be either a modification, deletion, or addition of executable statements, such as lines 6 and 27 changes. A declaration-level change is a modification of a signature, such as line 8 change. It could be method name modifications, method additions or deletions, variable type changes, or any kind of signature changes.

Depending on the granularity of the change impact analysis, different test sets might be selected for a re-run, as illustrates Table 1. Applying RTS with Class granularity selects four test cases, with Method granularity three, and with Statement granularity only two, thus improving the precision. For instance, a Method granularity would consider the entire method mA() changed, and would select test_mA_1() even if it is not impacted. To improve the precision, Statement granularity analyses the impact of each statement: test_mA_1() is not impacted since it doesn't run line 6. Therefore, the test execution time is reduced but the overhead is increased.

Next Section, we present a model-driven approach that is able to select impacted test cases at the most precise Statement granularity. Thereafter, since our approach provides a persistent impact analysis model, it will be up to the tester to choose the granularity to balance overhead/precision w.r.t. its case study.

Approach

Our model-driven RTS approach performs the three RTS phases relying on impact analysis model:

• (C) Computation of the impact analysis model.

• (A) Analysis of the impact analysis model to select the impacted tests.

• (E) Execution of the impacted tests.

Journal of Object Technology

Computation of the impact analysis model

Several properties of a program (correctness, robustness, safety, dependencies, etc.) can be analysed either statically, i.e., by examining the source code of a program, or dynamically, i.e., by analysing the execution of this program. Combining static and dynamic analysis is often interesting, since the data gathered dynamically can be used to add more precision to the static part. Indeed, considering all the possible execution behaviours of a program is not reasonable for several kinds of analysis [START_REF] Michael | Static and dynamic analysis: Synergy and duality[END_REF].

The MoDisco framework is designed to enable static program analysis in MDE [START_REF] Bruneliere | Modisco: a generic and extensible framework for model driven reverse engineering[END_REF]. MoDisco creates a model of the source code, usable for further processing, by modeldriven reverse engineering. The model is easily accessible to any kind of external modelling tool, providing support for many possible scenarios, such as static measurement, modernisation, understanding. The MoDisco source-code model contains a full model-driven representation of the source code, and a bidirectional transformation between model and code.

RTS requires to add dynamic aspects to the static structure and dependency information already retrieved by MoDisco. Adding dynamic traces turns the static model of MoDisco into an impact analysis model, serialising how test cases execute elements of the source code, and helping RTS identifying test cases impacted by source code changes.

Building the impact analysis model is a costly operation. Nonetheless, considering the building of this model as the dependency computation in a standard RTS approach (phase (C)), this model can be computed offline. Hence, when a build is triggered by the user, after a pull request for instance, this model can be immediately analysed to select the test cases impacted by code changes.

The approach we propose to build the impact analysis model consists in a succession of several steps as shown in Figure 2 and as detailed in the next sections. The first step consists in building a static model from the source code, by reverse engineering using MoDisco. The second step involves the instrumentation of the code. In the third step, executing this instrumented code on all the tests produces execution traces. Finally, the fourth step requires parsing those traces, creating references in the model, that show which elements of the source code are executed by a specific test.

Reverse Engineering

To generate the static model of the source code, MoDisco uses a visitor-based approach, navigating through the abstract syntax tree (AST) of the source code. In the case of Java, this model conforms to the MoDisco Java metamodel, is persisted in XML files, and loaded on-demand in the other steps of our process.

Figure 3 shows an excerpt of the model MoDisco would generate using the source code of Figure 1a available in the AST, organised in a containment structure, from packages down to expressions. For instance, Statement objects contain line and column numbers, expressions, references to declarations, and so on.

In the next steps we complete this model with dynamic data, gathered when executing an instrumented code.

Instrumentation

The most common approach for tracing the execution of a program is by instrumenting it. In the case of Java test cases this can be done for instance using bytecode instrumentation (BCI) or source code instrumentation (SCI). Both approaches have their advantages and drawbacks. BCI is usually faster, because SCI needs to recompile the instrumented code before being able to use it. However BCI may not be as accurate as SCI [HGB + 17].

For instance, the Java Code Coverage Framework JaCoCo4 uses BCI [SM08] through the ASM library5 . It iterates over lines of code using the LineNumberTable attribute available in Java compiled classes. Probes are added between each line of code, and hence, when a probe is executed, the line of code is considered as covered. Indeed, BCI can be used to know whether a specific instrumented line of code is executed, but not exactly when a statement is executed. Usually, there is only one statement per line of code, but specific cases might happen. For instance, considering a Java ternary operator which contains a if-true expression and an if-false expression at the same line: independently of the boolean condition, a standard BCI would consider both as executed, whereas a SCI is able to distinguish which one is executed.

In our approach we use SCI to precisely track the execution of every element on the MoDisco metamodel. MoDisco builds a model at the source code level, and thus, matching the elements of this source code with the model requires less calculation, and is more precise by using an instrumentation at the same abstraction level. Since this step is done offline, duration of the recompilation required by SCI is not an issue.

The source code is not instrumented in the same way considering test classes or System Under Test (SUT) classes. In the SUT classes, all the statements are instrumented, in order to determine if they have been called by a specific test method. Before each statement, a probe is added. Hence, if this probe is executed, then the statement following the probe is considered as executed too. In the test classes, the source code is instrumented at the method level. A probe is added at the beginning of each test method, monitoring the test case currently executed. This allows our approach to trace down all the SUT statements executed by each test method. We implement the instrumentation by leveraging the Spoon [PMP + 16] framework.

Full test set execution

To produce the initial set of execution traces, all the test cases must be executed once, at the beginning of the process (e.g., when the code is first imported). After the reverse engineering step we query the MoDisco model to gather the list of test cases (using Eclipse OCL queries6 ). Methods having the JUnit 4 @Test annotation are considered as test cases, such as methods with the Test prefix, in classes extending the TestCase class of JUnit 3. Once all the test cases are gathered, the execution of the injected probes will produce traces, indicating the exact statements, methods, and classes that have been executed for each test case. These execution traces (Figure 2) are serialised in a simple text file. For instance, executing the instrumented test method test_mA_1() of Figure 1a, would produce the execution trace of Figure 4.

Injecting traces in the model

Once all the test cases have been executed, the trace file can be analysed, in order to complete the MoDisco model of code structure with dynamic information: The classes and methods are queried in the model using their qualified names, and the statements are queried using their positions in the compilation unit. Once those elements have been found, a bi-directional reference is added between the test case, and the executed program elements. In Figure 5, this reference is the one labeled as impacts. For the remainder of this paper, we call this model Impact Analysis Model.

Finally, the Impact Analysis Model is persisted in order to be used as soon as a new version of the code needs to be analyzed, to select the test cases impacted.

Analysis of the impact analysis model to select the impacted tests

This second RTS phase selects the test cases to run, based on the impact analysis model in our model-driven approach. This phase is online, starting when the user launches the regression test activity, e.g., after pushing a new version of the code on a VCS.

Once the build starts and before anything else, the new revision is compiled. If the compilation fails, no tests are executed and an error report is given to the developer. If the code compiles successfully, then the RTS can be performed. The two revisions of the source code are compared, in order to locate the changes. To do that, a VCS comparison algorithm is used (we use the one implemented in JGit 7 ).

Running such algorithm on two different revisions of the code produces a set of DiffEntries. They are not only produced from Java files, but from any kind of files contained in the project, such as configuration files, and compiled sources. When a DiffEntry concerns a Java file, the impact of source code changes on test cases is computed using the Impact Analysis Model. Considering non-Java files is part of our future work. The changed statements are gathered in the model and the impacted test cases are selected, using the trace reference. For instance, if we focus on the Figure 5, revision R 2 in Figure 1b modified the ExpStatement line 6. Using the trace reference in the model, test_mA_1 is considered as impacted, and is selected to be run.

As shown in Figure 5, the trace references are precise, linking statements and test methods. Moreover, by querying the model we can derive the impact at coarser levels of granularity. For instance in Figure 5, the ExpStatement line 6 is contained inside the method mA, itself contained inside the class CA. Hence, a coarser-grained analysis (i.e. method or class granularity) can determine that modifying the method mA or the class CA would impact test_mA_1, and so on. Implementing a faster hybrid visitor for computing a coarser-grained impact analysis, such as the one proposed by Lingming Zhang [START_REF] Zhang | Hybrid regression test selection[END_REF] is possible, thanks to the reusability of our model.

DiffEntry considers a block of a few lines of text, and is classified according to the type of update (Modification, Insertion, Deletion). In the following we detail the behaviour of the test selection by type of update.

Modification

First, modifications introduced by developer can be either statement-level changes or declaration-level changes. If the update is a statement-level change, then the impacts can be computed with a statement-granularity. If it is a declaration-level change, then the impacts are computed at a coarser granularity (Method, or Class), depending on the change.

Second, those changes are either in the SUT, or in the test cases. When a test case is modified, it is immediately selected to be re-executed. However, the approach is more specific for SUT modifications: In case of statement-level changes, the modified lines are parsed, in order to get a set of modified statements. Those statements are queried in the model, and using the trace reference, all the test cases that executed this specific statement are selected for a new run.

The approach differs in case of declaration-level changes: If a method declaration is modified, by either renaming it, or changing the signature, this method is queried in the model, using its old-version signature. Then, all the test cases that executed this method are gathered, using the trace reference, and selected for a re-run. Furthermore, the same reasoning is applied with declaration-level changes at class level. Thus, when a class signature is modified, the old version of the class is queried in the model, in order to get the test cases impacted, using the trace relation.

Selecting the tests impacted by the modification at different granularities (e.g., Package) is also possible, by applying the same approach, thanks to the modularity provided by that model-driven approach.

Insertion

When the change is an insertion, several cases are possible:

First: Any test-related insertion results in the selection of the test case. Those can be either a new test, or an existing test with new lines of code.

Second: When new lines are added in an existing SUT method (i.e. statementlevel changes), the modified method is fetched inside the Impact Analysis Model, using an OCL query. And thus, impacted test cases are obtained using the impacts reference between this method and the test cases.

Third: When new classes or methods are added. If the method was not present in the previous revision, then it is not present in the model either, and hence, no impacts can be directly computed from it. However, adding such method can have impacts on the existing SUT. Figure 6 describes a situation where the insertion of a new method has impacts on the test cases. This case can be solved by checking by OCL if such method overrides a method of a superclass. For instance, if a method m() is added in class C, this approach would check if m() exists in superclass SuperC. If it does, then all the test cases executing C.m() would be selected for an execution, using the trace reference of SuperC.m() in the Impact Analysis Model. This scenario adds a minor overhead, but is necessary to ensure the safety of our RTS approach.

Deletion

In the same way as in the other changes, deletions can be at the statement-level, or declaration-level, which implies two different behaviours.

First, when a line of code is removed at the statement level (i.e.), inside a method, this method is fetched by querying the Impact Analysis Model. Finally, its impacts on the test cases are obtained using the trace reference. Test cases impacted are then selected to be run again.

Second, when an entire method, which does not override any method from a parent class, is removed, then the test cases calling it have probably been modified too. Indeed, removing a non-overridden method from a class, but not the calls to this method would produce compilation issues. The same reasoning works at the class-level. If a deleted parent class is used in the test cases, then those tests have to be updated too.

Otherwise, if the deleted method overrides a method from a superclass, then the old version of this method is fetched from the model, and all the tests executing it are selected, using the trace references.

To conclude the Section 4, we notice that we successfully design and implement a model-driven RTS approach that can select impacted test cases based on a model. The current version of our prototype is available on GitHub8 . Thanks to that model, the granularity of the selection could be easily adapted according to the needs. Existing RTS techniques using variable granularities show excellent results [START_REF] Zhang | Hybrid regression test selection[END_REF]. As shown in Section 4.2, it is possible to get the impacted test cases at several granularities (statement, method, class), depending on the change type (statement-level, declaration-level). Thus, in order to accelerate the RTS, it is possible to limit the impacts to a certain granularity.

Evaluation

This section presents an experimental evaluation of our approach. First, we present the environment setup and then the experimental workflow, followed by a presentation of our results.

Setup

All the experiments are executed on an Intel Core i5-7200U CPU (2.50GHz), 8GB of RAM, running with Ubuntu-16.04, and using Java 1.8.0 51. The four projects presented in Table 2 are used for our evaluation. The column First commit corresponds to the first Git revision used, LOC approximates the lines of Java code available in each project, and the last two columns list the number of system under test classes and number of test classes, during the first commit. Those projects are available on GitHub9 .

Those four frameworks have been chosen because they answer to several criteria that were mandatory for the current state of our prototype: using the git VCS, Java, the Maven dependency manager, and having test suites running with the JUnit testing framework. Since all those tools are popular in the development community, restricting the usage of our prototype to those does not significantly hamper its practical usefulness.

Workflow

We used the following workflow to run the experiments. First, a specific starting revision is determined. This can be a previous tag in a Git history, for instance. This revision is called R 1 . Then we define a variable Step, representing the number of commits between each computation of the RTS. For instance, with Step = 2, the Regression Test Selection would be computed between R 1 and R 3 . Instead of applying the RTS at each commit, being able to merge multiple commits at once offers a more accurate representation of a development lifecycle. In fact, developers either push multiple commits at once, or use pull requests, to merge several commits from other branches on the VCS. We then define another variable, M ax representing the maximum amount of commits to analyse. For instance, with M ax = 50, the last revision that could be analysed would be R 50 .

The experimentation starts with the revision R 1 . Since no model can be found, all tests are executed. Then the impact analysis model is built offline, thus computing the dependencies at the statement-level. Once this is done, the revision R 1+Step is cloned, and the RTS is applied between R 1 and R 1+Step , using the impact analysis model previously generated. Hence, the subset of tests is executed. The impact analysis model is built, from the revision R 1+Step , and the next revision R 1+2Step is cloned, and this goes on until reaching R M ax . Each time, both the subset of tests and all tests are executed with the same technique, in order to compare the results.

For the experimentations, the M ax value is set to 100, and Step is set to 5. Those values have been arbitrary chosen and we assume that the quantity of revisions covered is relevant enough for our evaluation.

Results

Figure 7a presents the results of applying our model-driven approach to RTS to a pool of 100 commits, from the four different projects. The first section of the table presents the quantity of test methods selected by our approach. As the number of tests varies along the commits analysed, average values are used. The second section reports the average compilation and execution times for each project, without RTS. Hence, summing those two values for each project shows the standard build time. The third section presents the average times of the three model-driven RTS activities. The sum of the compilation time with the identification and execution of impacted tests results in the duration of a full build, when using our approach. This model-driven RTS build durations and the standard build durations are compared in the last section of the table, as well as in Figure 7b. Results show that applying model-driven RTS shortens the build time, in all cases, when the impact analysis model is built beforehand. On a bigger project such as AssertJ, the results are more significant, with a RTS build time lasting 67.5 % of a standard build, when only 18.24 % of the tests cases are executed. Comparing modeldriven RTS build times with standard build times on a small project like Javalin shows that 6.31 % of the tests are selected, on an average. However, for a project of small size the impact on the build time is less significant, since selecting and executing the tests takes 90.3 % of the execution time of all tests.

As explained earlier, running the tests, either all tests, or a test selection, includes a compilation time. This is performed using Maven and hence several steps of the Maven build lifecycle have to be executed: validation, compilation, and finally testing. Comparing only the actual test execution times would show a bigger gap between RTS and the execution of all tests, but would not reflect real-life builds.

In our experimentation, selecting the tests cases and running them takes always less time than running all the tests cases. As predicted, a loss of time may occur when the number of tests is low since the run of a test subset will not compensate the overhead of the RTS phase (A). However, testers will consider RTS only when their test suite length starts to be problematic and in that case, our approach is beneficial.

Discussion

In that section, we discussed the approach and the experiments to answer the three research questions.

RQ1: Precision

Precision has been discussed previously in Section 4. We can answer the second part of RQ1 positively since our model-driven approach allows precise RTS. Thanks to the impact analysis model, the precision can be as much finer as Statement granularity. Moreover, the modularity of the approach allows to adapt the precision by analysing differently the impact analysis model.

RQ1: Safety

To answer the first part of RQ1, we consider the safety of our model-driven RTS approach. We should prevent an impacted test not to be selected. Therefore, we consider four risks: miss a source code change, analyse a non up-to-date impact analysis model, consider all the software artefacts, and flaky tests.

All the tests are first executed to generate the impact analysis model and their execution is traced at the finest statement granularity. Hence, if any statement is modified in a future revision, then all the tests that previously executed it can be selected thanks to the trace references.

The test selection requires an impact analysis model, built offline. If the user launches regression testing too frequently, the model of the last revision may not have been completely built. If that happens, the last entirely computed model is used instead. It would be less precise since more tests would be selected (the ones of several iterations), but the safety is ensured. If no model is available, then the entire test suite is executed.

The current state of our tool does not compute the impact of external files changes. Nonetheless, such dependencies could be added in the impact analysis model: e.g., Gregoric et al. describe an efficient approach to collect file dependencies, using BCI and monitoring [START_REF] Gligoric | Practical regression test selection with dynamic file dependencies[END_REF]. All standard library methods able to access files are monitored, and dependencies towards those files are added when accessed at runtime. Enhancing our current SCI to monitor external files accesses is part of our future work. It is another advantage of our model-driven RTS to be able to extend the impact analysis model with more information. Considering that risk, we ensure the safety in the current implementation, by detecting that external files modification (using a diff algorithm) and warning the developer. She can then run all the regression test cases, in order to ensure that nothing has been broken.

Finally, non-deterministic tests, also known as Flaky Tests [START_REF] Luo | An empirical analysis of flaky tests[END_REF], are not covered by our approach. These tests can behave differently between several executions, (depending on multiple parameters: asynchronous calls, concurrency, network, test order dependencies, etc.) and therefore cannot be traced deterministically. The safety risk of Flaky Tests is not only affected RTS but all the testing process and it is up to the tester to annotate them for being processed carefully (e.g., with several execution).

RQ2: Performance

As presented in the Introduction with RQ2, our model-driven approach has to be efficient enough to reduce the regression testing time. Poorly performed RTS steps might produce an overhead so important that selecting and executing a reduced set of tests would take longer than running the entire test suite.

First, our RTS approach is offline. Therefore, we do not take into account the model generation time (RTS phase (C)). This phase is costly when generating an impact analysis model, but it is already performed before the tester launches the regression testing. Second, the test cases selection (RTS phase (A)) is critical, and must be computed in a fast way to prevent a counter-productive overhead. Two expensive activities are processed during the test cases selection:

1. Comparing two revisions of a project, more specifically at a fine granularity (Statement-level).

2. Querying the model for specific elements.

Being efficient on those two points is a major concern. Performance when manipulating models are known issues in model-driven engineering. Experiments presented Section 5 shows that we positively answer RQ2 since selecting the tests cases and running them takes always less time than running all the tests cases. In addition, our model-driven approach allows to configure the granularity and then to balance the overhead depending on the case study (e.g. a case study with few test cases would benefit from a coarser-granularity, while a case study with slow test cases would benefit from a fine-granularity to reject slow and useless test cases).

Our performance (gain of up to 32 %) are less important than the ones of dedicated tools such as the ones of [START_REF] Zhang | Hybrid regression test selection[END_REF][GEM15] (they can gain more than 40 %). However, they will be overtaken by our model-driven approach for further analysis that can reuse the trace models, as discussed next section.

RQ3: Complementary use of the Model

In a MDE environment, our impact analysis model becomes a part of the holistic modelling of all software aspects. Thus, it can be reused to perform heterogeneous analysis on the project.

For example, tracing down the execution of test cases inside a model would provide a better understanding of a test suite, useful while reverse-engineering old code bases. This model could also be used to get an accurate overview of the testing code coverage, and thus significantly improve the quality of test suites [START_REF] Moonen | On the interplay between software testing and evolution and its effect on program comprehension[END_REF].

Concretely, in our tool, we use the Structured Metrics Meta-model 10 (SMM) for gathering measurements performed during the test execution [START_REF] Béziers La Fosse | Characterizing a Source Code Model with Energy Measurements[END_REF]. SMM enables the modelling of several metrics: lines of code, test execution durations, method complexities, energy consumptions, and so on. By weaving the SMM model along with the model produced during RTS, we are able to know the energy saved by not running the test cases that have not been impacted. Figure 8 shows an example of a SMM usage along with the model produced by our RTS approach. The Measurement elements contain energy consumptions, and refer to the Methods available in the model. ObservedMeasure contains the timestamp of the measures. Finally Observation and SmmModel are root elements in the model. We successfully answer the RQ3 since the impact analysis model can be completed and analysed for other software engineering analysis, such as energy consumption. On the contrary of dedicated tools that are not compatible and require each one to run and to analyse the program, we collect and serialise all the information in one impact analysis model. Created once, we save time, counter-balancing the overhead induced by model manipulations.

Threat to validity

Several threats to the validity of our experimentation have to be considered: Commits analysed: First of all, the amount of selected tests highly depends of the changes made by developers. For instance, the modification added between commits fce43e6 and 4fe13cb on the project AssertJ impacted 3312 existing test methods. Therefore, the results could be radically different using other commits. Nonetheless, by applying our RTS approach on a set of 100 successive commits, we assume that it covers usual development scenarios, such as adding of new methods, or modifying of the core methods of the project.

Single machine: All those results have been computed on a single computer dedicated to this task. Even if a particular attention has been taken to only dedicate the machine to the experimentations, background operations performed on the system may have impacted the execution times reported.

State of the prototype:

Finally, our research prototype's implementation may contain undetected bugs, and thus the results produced may be subject to variations with the upcoming improvements and corrections. Even if this prototype is well tested and its source code has been thoroughly examined (and is open to inspection), not all the results obtained during the experimentation can be manually verified. Especially the safety of the approach, while theoretically valid, has been manually checked only for the small projects. Finally, this prototype is still under development, and so far its performances have been improving at each revision. Scalability: Models are well known for suffering of scalability issues [START_REF] Espinazo Pagán | Morsa: A scalable approach for persisting and accessing large models[END_REF]. In fact, the standard EMF implementation persists models using XML data structure. Those files cannot be partially loaded, and if the model is too large, it might not fit in memory anymore. When the source code is massive (thousands of classes), the model generated by MoDisco can be huge (GBs), which may produce scalability issues. This limitation can be resolved by using scalable persistence layers for models, such as NeoEMF11 , or CDO12 .

Conclusion and Future Work

In this paper, we presented an approach that applies MDE techniques to RTS. We build an impact analysis model, starting from a source-code structural model and enhanced with dynamic information, gathered during test case execution. On top of this model we implement a fine-grained impact analysis, allowing for effective RTS. The benefits of the impact analysis model are twofold: it allows the separation of the impact analysis from the test selection and opens different perspectives for future work.

Figure 1 -

 1 Figure 1 -Two revisions of a program

Figure 2 -

 2 Figure 2 -Steps to compute the impact analysis model

Figure 3 -

 3 Figure 3 -Excerpt of the MoDisco model with Figure 1a as input

  test: TestCA.test_mA_1() sutMethod: CA.mA(int) statement: line 3,col 27 -> line 3,col 38 statement: line 5,col 55 -> line 5,col 59 statement: line 6,col 60 -> line 6,col 75 test: TestA.test_mB() ...

Figure 4 -Figure 5 -

 45 Figure 4 -Execution trace of test mA 1() after instrumentation R1: Model default: Package

Figure 6 -

 6 Figure 6 -Adding a new method impacts an existing test case

  10 https://www.omg.org/spec/SMM/

Figure 8 -

 8 Figure 8 -Excerpt of a SMM Energy consumption model

Table 2 -

 2 Projects used for evaluation

	Framework First commit	LOC	#Classes Tested	#Test Classes
	Javalin	caae71e	2500	13	23
	JSoup	7f8010d	25 000	61	31
	JUnit4	64155f8	100 000	126	174
	AssertJ	cf4d367	250 000	368	1903

https://travis-ci.org/google/guava/jobs/372209560

https://github.com/google/guava

time necessary to compute the set of test cases to run, plus the test execution time.Journal of Object Technology

Journal of Object Technology

https://www.eclemma.org/jacoco/

http://asm.ow2.org/ Journal of Object Technology Source-Code Level Regression Test Selection: the Model-Driven Way • 9

https://projects.eclipse.org/projects/modeling.mdt.oclJournal of Object Technology

https://www.eclipse.org/jgit/ Journal of Object Technology

https://github.com/atlanmod/MDE4RTS

https://github.com/jhy/jsoup, https://github.com/joel-costigliola/assertj-core https://github.com/junit-team/junit4, https://github.com/tipsy/javalinJournal of Object Technology

https://www.neoemf.com/

https://www.eclipse.org/cdo/

Computing the long-running impact analysis offline allows us to rapidly apply the RTS when the user needs it, minimising the waiting time, and hence improving the productivity. Indeed, experiments on four different open-source projects showed that our approach significantly shortens the regression test execution time. Moreover, the proposed impact analysis model can serve as a basis for different software engineering activities: fault localisation, precise code coverage, energy consumption analysis, etc.

While the current version of our prototype provides positive performance results, several points can still be improved. Incrementality should be considered to reuse and complete the model during the RTS phase. Indeed, after each build, our prototype creates the whole model from scratch, using MoDisco. Using diffs to update an existing model according to the new source code version would shorten the impact analysis model building time. We also intend to connect our prototype with a persistence layer, to support larger projects. More experimentations will be performed, especially on projects larger than AssertJ, and accuracy will be compared to state-of-the-art RTS tools, such as HyRTS, Ekstazi and FaultTracer.

Finally we will also focus on completing the implementation of our research prototype, in order to have more precise results. The impact analysis should be improved, in order to also analyse external files other than Java source files. Indeed, modifying a configuration file may impact the test cases execution. This kind of problem threatens the safety of traditional RTS tools, and a MDE-based holistic approach to RTS could provide a natural solution.