I Deniaud

F Marmier

D Gourc

DECISION SUPPORT SYSTEM FOR RISK MANAGEMENT IN COMPLEX PROJECT DESIGN

Keywords: Design project, complex product, risk management, systems engineering

This paper outlines how to make strategic decisions in NPD project according to risk. The complexity of the problems to be solved in NPD depends on both the nature of design problem solving and the difficulty of managing the project (activities, risks). The interactions between product subsystems in NPD often lead to conflicts requiring arbitration between alternative solutions. To give an overview of the links between activities to the Project Manager, we propose a joint process of design, project management and risk management. It takes into consideration the design activities and risk activities to generate a design project planning. During the project design, if different strategies can be used only for solving design problems, different others can help deal with the project risks. All of them lead to different possible scenarios. We present a decision tree to show the decisions steps and possible project scenarios. A generic decision support system is proposed. A case study of a satellite design project is developed to demonstrate the effectiveness of the proposition.

INTRODUCTION

With rapidly changing markets and technologies, today companies are facing the same issue: to offer increasingly complex innovative products. Complexity is difficult to manage both at the product design [START_REF] Hobday | Product complexity, innovation and industrial organisation[END_REF], but also in the organization of the project design and process control. Thus, a growing number of companies must both manage the development of their standard products and launch innovative products continuously. To meet this requirement, they must master two forms of design process: exploration and exploitation [START_REF] March | Exploration and exploitation in organizational learning[END_REF]. The more we go towards an exploration project, the more important the innovation capacity of the company should be. It increases with uncertainty (and therefore risk) on design projects. The effects of risk are measured according to the criteria of classic project management such as cost and time. To manage this complexity we propose a model integrating the risk management in design projects. This paper outlines strategic decisions making in NPD project according to risk. Firstly we characterize different design processes. Then we present different models used to manage design complexity. We show that Systems Engineering models reduce the complexity in product design. A multi-level system decomposition helps to evidence technical or organizational problem. We give a flow chart illustrating the processes leading to arbitration management in design project. To give an overview of the links between activities to the Project Manager, we propose a joint process of design, project management and risk management. A methodology is presented supported by a case study.

PROJECT RISK MANAGEMENT IN PRODUCT DESIGN

Product Design

In the literature different model types are proposed for exploitation or for exploration projects:

Design projects models based on exploitation develop a variant of a known product, using in projects predefined existing skills. Design projects are based on pre-specified models: e.g. the single-step waterfall model which does not present the feed-back possibility [START_REF]Managing the Development of Large Software Systems: Concepts and Techniques[END_REF]; Pahl and Beitz model [START_REF] Pahl | Engineering Design, A Systematic Approach[END_REF] with iterations requiring costly and very long development time [5; 6]. The project is structured in a number of sequential and linear steps [START_REF] Sanchez | Modularity, Flexibility, and Knowledge Management in Product and Organization Design[END_REF].

There are two types of Design models based on exploration:

A. Concurrent models: e.g. Vee initial model [START_REF] Forsberg | The Relationship of System Engineering to the Project Cycle[END_REF] and spiral models [START_REF] Boehm | A spiral Model of Software développement[END_REF]. In this case many researchers think that design is a problem-solving process and the objective is to identify a satisfactory solution [START_REF] Simon | The science of the artificial[END_REF]. It consists of a set of hierarchical problems that are solved in a parallel manner. This is the case when interdependences between design parameters for the system to be designed are decoupled [START_REF] Suh | Complexity in Engineering[END_REF]. Here, an order exists that allows the successive definition of each parameter without ever having to modify those that were previously defined.

B. Emergent models: e.g. innovation models, "chainedlinked model" [START_REF] Kline | An Overview of Innovation, The Positive sum strategy: harnessing technology for economic growth[END_REF]. When the problem to be solved is new and complex, the problem formulation phase is critical. According to Suh [START_REF] Suh | The Principles of Design[END_REF] when a product to be designed is very complex, this sequential model no longer applies because system design parameters are strongly interdependent [11; 14]. This exploratory project is nonstructured: the goal is constructed as the activity progresses during negotiations between participants.

The resolution of a design problem sometimes implies, to overcoming contradictions [START_REF] Altshuller | Creativity as an Exact Science: The Theory of the Solution of Inventive Problems[END_REF]. That means developing new concepts and new technologies in order to create a new framework and a new solution area. Emergent models are based on the dynamic system development method, and innovation-based processes presented in the next section.

Systems Engineering was developed to guarantee that the system matches real needs [START_REF] Liu | Creativity or novelty? : Cognitivecomputational versus Social-cultural[END_REF]. To control exploratory projects, tasks have to be parallelized and use different models (e.g. activity-based model and Vee system model). These models have the advantages of: (1) dividing a system into modules [START_REF] Ulrich | Product Design and Development[END_REF] distributed in different layers: from the system to the components (Figure 1a) [START_REF] Forsberg | The Relationship of System Engineering to the Project Cycle[END_REF]; (2) developing the solution, organizing the project into interrelated subprojects, often represented with the "Vcycle" (figure 1b); (3) transferring the design effort to the front of the design process. This implies that managers and designers become able to guarantee the performance of the design process and to anticipate or prevent major risks of future projects. The nature of the interactions between subsystems often leads to a series of conflicts requiring arbitration. This arbitration process can lead to a redefinition of the initial functions, of the product initial requirements or even to a redesign of the organization (activity structure) of the project design. A design project implies strategic choices.

Figure 2 illustrates the processes leading to arbitration or contradiction management [START_REF] Belleval | A model for the management of design project issues: The case of the microsatellites design[END_REF]. At first, arbitration, where the actors use an existing knowledge base, implies that the solution is already known. Arbitration may thus lead to a compromise situation.

Applying this model means: [START_REF] Hobday | Product complexity, innovation and industrial organisation[END_REF] to verify requirements at different stages of the design process leading to a validated solution;

(2) to identify and analyze in details the contradictions using systems engineering tools; (3) to overcome contradictions when they appear during the design project. Each design solution implies different risks that must be take into account in project management.

Project risk management

In the literature, the risk management methodologies refer to a standard process presenting the well-known steps: risk identification, risk evaluation and quantification, risk mitigation for treatment and/or impact minimization and risk monitoring [START_REF]Project management -guide to the management of business related project related[END_REF], [START_REF]International Standards for Business, Risk management -Principles and guidelines[END_REF]. Tixier et al. propose a classification of sixty two existing approaches [START_REF] Tixier | Review of 62 risk analysis methodologies of industrial plants[END_REF]. They sort methods as being deterministic and/or probabilistic, but also qualitative or quantitative. Several academic research works propose methods to complement the different phases of the global approaches, such as the optimisation of different criteria during the schedule or after the identification phase [START_REF] Kiliç | A biobjective genetic algorithm approach to risk mitigation in project scheduling[END_REF]. In parallel to these global approaches, several authors propose methodologies to manage the risk in projects. Gourc et al. propose a reading grid of the risk management approaches as follows [START_REF] Gourc | Towards a general risk model for piloting goods and service related activities[END_REF]: the symptomatic approach and the analytic approach. The first group of approaches, called risk-uncertainty, is associated with approaches where project risk management is transformed into project uncertainty management [START_REF] Ward | Transforming project risk management into project uncertainty management[END_REF] using the Monte Carlo simulation method [START_REF] Kalos | Monte carlo methods[END_REF]. The second approach family considers risk as an event that can affect the achievement of the project objectives [START_REF] Carter | Introducing RISKMAN: the European project risk management methodology[END_REF]. "Risk can be defined as the combination of the probability of an event and its consequences" [START_REF]Risk management vocabulary: guidelines for use in standards[END_REF]. Risk is described as an event, which has occurrence characteristics and consequence characteristics on the project objectives (impact in the event of occurrence). Pingaud and Gourc [START_REF] Pingaud | Approach of controlling an industrial project by the risk analysis[END_REF] propose a project management approach based on a synchronised process of project schedule and risk management. Nguyen et al. propose Prorisk which can model and evaluate the impact of risks on the project cost and the schedule cost [START_REF] Nguyen | A decision-making tool to maximize chances of meeting project commitments[END_REF]. They define the concepts of risk scenario, treatment scenario and project scenario.

In the project management literature, two themes are wellknown for their reference to the innovation and then for the omnipresence of risk: the project management of new product development (NPD) at an operational level and the portfolio management of NPD at a tactical level. A first definition of portfolio management is given in 1999 [START_REF] Cooper | New product portfolio management: Practices and performance[END_REF]: a dynamic decision-making process that allows the project lists to be always updated. Its construction is mainly difficult due to the omnipresence of the evaluation of the balance benefits versus risks. One of the problems is to be able to evaluate and compare several possible portfolios with a global risk indicator. Risks are also intrinsic in new product development (NPD) in all industries [START_REF] Kwak | Examining risk tolerance in project-driven organization[END_REF]. Thus firms need to take initiatives to reduce risks that are related to NPD. The risk management framework should integrate the three most important risk factors that affect NPD performance: technology, marketing, and organization [START_REF] Doering | Identification and assessment of emerging technologies, Wharton on Emerging Technologies[END_REF].

In NPD management, decision-makers have to choose exclusively one orientation as strategy development according to a global risk level tolerance. As an answer, decision trees (DT) are regularly used in the literature on decision [START_REF] Chiu | An evaluation model of new product launch strategy[END_REF]. It makes it possible to find optimal solution to short time dynamic decision problems [START_REF] Clemen | Making Hard Decisions: An Introduction to Decision Analysis[END_REF]. Based on decision variables, decision tree allows managers to choose one way and to react within this way in front of events. To increase the efficiency of innovative the link between the selection of innovative technological solution and the modification the project structures has to be made.

There is no tools helping the project manager evaluate the solution of the project design problem and its consequences on (1) the planning of the duration of a design project, (2) the risks and their associated treatment strategies.

PROPOSED DEVELOPMENT

In this work, we focus on in the decision making process in design project management and risk management by analysing the consequences of the risk "as an event" in the design project. As observed in this literature review, little account is taken of risk and the strategies to deal with it regarding their repercussions on design project. Our objective is to propose a complete framework helping decision-makers to decide innovation and risk prevention strategy.

Hypotheses

The proposed model is based on three main hypotheses.

-Risk integration into the project management is done regarding the duration and the cost criteria.

-The resource aspects such as the availability of skills are not considered for the moment in the model.

-When the decision of treatment strategy and technological solution has to be taken, characteristic relative to the risks are known. This work does not aim at developing a tool which eases the data-gathering.

At any time, the objectives of the model are (1) to analyse the possible scenarios, (2) to evaluate the global risk level, i.e. the global risk level represents the chance, for the project, to satisfy commitments, (3) to select the best treatment strategies.

Formalization of the decision process in design project

At the first step of the Vee-cycle, the need is analyzed. Different technical solutions are possible answers to the same functional need. The first decision at the end of the need analysis is to select the solution that will be developed during the design project. This decision is represented by the square D1 in the illustration presented in Figure 3. Several outputs are most often the time possible. In the Figure 3, three technical solutions (A to C) are possible.

Figure 3: Link between the V cycle, the decision tree and the synchronized process.

For each one of these solutions, it is possible to draw up a planning using the Project Planning Process (noted PPP) of the synchronized process described in [START_REF] Pingaud | Approach of controlling an industrial project by the risk analysis[END_REF]. Based on the planning the Project Manager (PM) has to organize the treatment of the risks over the design project. In that way, the decision, noted D2

(A to C)
, consists in deciding the preventive risk treatment strategy in the project for the chosen technical solution. The Risk Management Process (RMP) is performed to identify the risk treatment and the PPP is developed a second time to plan the risk treatment tasks as well as to integrate others modifications of the planning. These two decisions are made in the preparation phase of the project.

The PM has also to plan the possible project development.

In that way the chance node E characterizes the possible occurrence of risks. In front of each possible set of occurrences, D3

(A to C) represents the decision of corrective strategy that could be carried on during the whole design project. D3 is planned during the preparation phase in order to meet risk occurrence during the project running phase. D3 supposes to reprocess the RMP and the PPP as well as to integrate new information and to obtain the final planning.

The decision tree, in Figure 4, gives a complementary view of the succession of decisions. The example concerns the choice of the technical solution A in D1. The PPP is then developed. The second decision (D2 A ) concerns the preventive risk treatment strategy. For all decision, the RMP has to be developed followed by a second cycle of the PPP (PPP'). Then, in front of all possible sets of risk occurrence, the last decision (D3 A ) concerns the corrective risk treatment strategy. D3

A gives a planned and global vision of all the possible corrective risk treatments that will be carried out over the whole duration of the design project. A second cycle of the RMP and a third cycle of the PPP are therefore processed (RMP' & PPP").

Data

P

ts (ts=0,...,TS) is a Project associated to a particular technological solution ts, TS being the number of possible technological solutions and then potential projects. Each P ts is described by its tasks Tt ts (t=1... T ts ), T ts being the number of project tasks of P ts . The planning process gives an initial planning Pi ts that does not integrate any risks. A project is also described by its set ER ts of identified risks Ri ts (i=1...n ts ), n ts being the number of identified risks in P ts . Each Ri ts is characterised via the risks management process. A risk Ri ts is also characterized by its period of occurrence, i.e. the tasks during which the risk can occur. It has a probability proba(Ri ts ) (the probability that the event related to Ri ts occurs) and impacts in costs CI(Ri ts ) and/or in duration DI(Ri ts ) on a task. This task can be different from the period of occurrence. These probability and impact are also called initial probability and initial impact. The initial impact allows consideration of the fact that the task is running in a graceful degradation.

A risk scenario ScRs ts corresponds to a combination of the risks that are considered as occurring during a project P ts . It has a probability proba(ScRs ts ).

!"#$%(!"# ! !" ) = !"#$% ! ! !" !" ( ! ! !" ∈ !"# ! !" ) 1 -!"#$% ! ! !" !" ! ! !" ∉ !"# ! !" ! !!! (1)
Each risk can be treated in various ways that can be preventive, corrective or a combination of several actions

!"#$% !"# ! !" = !"#$% ! ! !" (2) 1 -!"#$% ! ! !" 3 !"#$% ! ! !" !"# !" !" (4) 1 -!"#$% ! ! !" !"# !" !" (5) ! ! !" !!"# ! !" ,!"# !" !" !!"# ! !" !,!
The cost of a project scenario is noticed C(ScPp ts ). It includes the cost of the T ts tasks that constitute the initial planning of the project, the ScRs ts and the chosen ScTp ts and ( 6) The Global Cost GC initial (Ri ts ) of the occurring risks that are not treated by the treatment strategies. It includes the cost impact that is composed by a fixed part of the total cost (materials, tools, parts, etc.) and by an indirect cost that depends on the action duration, through the Delay Impact, and the actors charge. [START_REF] Sanchez | Modularity, Flexibility, and Knowledge Management in Product and Organization Design[END_REF] The reduced global cost impact GC reduced (Ri ts ) that is obtained taking into account the different strategies StTij ts applied to treat Ri ts and its reduced repercussions on the project cost and duration. [START_REF] Forsberg | The Relationship of System Engineering to the Project Cycle[END_REF] The cost of the treatment strategies StTij ts that is determined by the cost of the action is composed by a direct cost (materials, tools, etc.) and by an indirect cost that depends on the action duration and on the actors.

! !"# ! !" = !(! ! !" ) ! !!! + !" !"!#!$% ! ! !" (6) !" !"#$%"# ! ! !" !"# !" !" 7 ! !"# !" !" ∈!"# ! !" !!" !" !" (8) ! ! !!"# ! !" !,!
Each ScPp ts can be characterized by a criticity Cr(ScPp ts ). This criticity measure is based on its probability of occurrence proba(ScPp ts ), and a duration and a cost metrics of the project scenario respectively αp ts and βp ts :

∝ ! !" = !"(!"# ! !" ) !"# (!"(!"# ! !" ))
and

β ! !" = !"(!"# ! !" ) !"# (!"(!"# ! !" ))
(p =1,…,P), (ts=0,...,TS) (

then αp ts , βp ts ∈ [0,1].

Where CI(ScPp ts ) and DI(ScPp ts ) are respectively the distance between the Cost and Duration Impacts and the budget and duration thresholds defined in the contractual agreement of the project. max(CI(ScPp ts )) and max(CI(ScPp ts )) the distance of the costly and longest project scenario possible over the whole design project.

The global impact, weighted and normalised, Impact(ScPp ts ) is then obtained through the following formula:

Impact(ScPp ts )=q x αp ts + q' x βp ts [START_REF] Simon | The science of the artificial[END_REF] Where q and q′ (respecting q + q′ = 1) are two coefficients that are chosen by the project manager in accordance with the importance of the duration relatively to the cost.

Then, ∀p, ∀!", Cr(ScPp ts ) = proba(ScPp ts ) × impact(ScPp ts ) (11)

Objectives

Taking decisions in the choice of a technological solution to reach the functional needs is a multicriteria problem. When the project manager makes the decision, the number of criteria used to evaluate the proposal is often reduced to the main ones: the cost, which is a sensitive and finite resource and the duration, which traditionally is a matter of contractual commitment.

RESOLUTION APPROACH

In the preparation phase of a project, the technical orientations and the way of managing risks have to be chosen. It has to be done both for the planning and for the potential to react in front of an event during the running phase. Therefore, our method includes input data provided by the schedule process (management team) and from the risk management process;

The body of the approach is composed of two main phases: (a) the construction of the Decision Tree (DT), from the first decision node to the leaves, (b) the resolution of the DT in a backward way.

(a) The construction of the DT The generation of the DT consists in building all the possible project scenarios and their evaluations for each technological solution studied. The project scenarios are the leaves of the tree. An initial schedule for each technical solution is generated, without integrating the notions of risk and risk treatment. Depending on the difference between the technical solutions, plannings can be more or less different from each others.

It is then necessary to calculate the different risks and treatment scenarios. These scenarios allow the set of the project scenarios to be constructed. Finally, when the project scenarios are known it is possible to obtain their durations and costs. The approach called ProRisk proposed in [START_REF] Nguyen | A decision-making tool to maximize chances of meeting project commitments[END_REF] is then used to generate ES ts . The probability calculation method for each project scenario differs, depending on whether the project scenarios contain a treatment strategy or not. For each project scenario, the calculation the probability, the cost and the duration take into account potential modifications induced by the achievement of treatment strategies at the schedule level. Once the initial schedules adapted, the project scenario duration is computed using the PERT method and the earliest starting dates.

(b) The resolution of the DT

The resolution in the backward way consists in studying the effects of the different decisions from the leaves to the root of the tree (i.e. from D3 to D1 in Fig 4). The use of the knowledge of the tree makes it possible to become proactive.

Step (1) consists in finding for each branch of the DT, the best D3 faced to each ScRs ts . They are selected in order to avoid scenarios that would not be possible in the reality. D3 is made by choosing the corrective strategies that minimize Cr(ScPp ts ) for all the leaves of the tree.

Step (2) D2 consists in deciding the preventive treatment strategy that is the most adequate for each technical solution. It is made for each studied design project knowing the best D3 made for all the branches. D2 consists in avoiding the worst possible cases (project scenarios) as defined by the Savage's criterion often used in decisionmaking theory [START_REF] Petar | Application of sensitivity analysis in investment project evaluation under uncertainty and risk[END_REF]. D2 is made by choosing the preventive strategies that minimize Crmax(ScPp ts |ScTprev ts ), ∀ ts knowing the selections realized in D3

Step (3) D1 is the selection of the best technical solution based on its design project. The project management team wants to maximize the chance of meeting the commitments. D1 consists in selecting the technical solution that maximizes the number of possible ScPp ts . respecting the contracted duration and cost.

A SATELLITE DESIGN PROJECT BASED CASE STUDY

Presentation of the satellite design project

The case study, developed here to illustrate this research work, considered the development of a small satellite for a scientific mission, based on different platform. The platform would carry out a predetermined useful load of scientific / technological instruments. Two technological solutions can potentially meet this need: (A) a classical one using ergol liquid for its propulsion to reach its final orbit consecutive to its launch. (B) an innovative solution, fully electrical with the motor using xenon gas ionised and accelerated with the electrical energy provided by two photovoltaic panels.

The second solution allows gaining the load of the ergols and of the associated structure: about half of the weigh of the final satellite. Thanks to this, it provides for lighter satellites for the same capacity. Therefore, the cost of the launch can be divided by up to two since two satellites can be launched with the same booster rocket. The inconvenient is the six months duration necessary to reach its final orbits due to the low power.

The selected solution would go through the different phases of the Vee-cycle. Table 1 presents the phases of the projects for both solutions. The phase durations are presented in Time Unit (TU) with a fixed rate and their costs in Monetary Unit (MU). Antagonist requirements complicate the project. Per example the need of a high onboard power (>100 W), a low satellite mass (<120 kg), a low volume (<1m3), a life time (>2 years)…

An associated provision for risks makes the contractual commitment for this part of the global satellite project of 37 TU and 44 MU whatever the solution retained. Different risks were identified during the project (Table 2). Their associated cost is a fixed cost that is added to the one based on the new task duration. Possible treatment strategies characterize them (Table 3). The impacts of one risk are judged as ∞ since the costs and delays will continually increase until an action is decided. NC (Not Changed) means that the strategy does not modify the impact of the risk. As an example, the risk R4, expresses the anomaly observed during the material integration on the satellite (error of wiring, systems presenting default...). R4 is relatively probable since all the failures are recorded.

If such a risk occurs, the production is immediately stopped until a strategy is implemented. Then two strategies are possible in both versions of the project: a preventive one and a corrective one. StT41 A and StT41 B , consists in carefully check critical material at the subcontractor plant by participating to the reviews, auditing etc. If it did not suppress the risks, it reduced their probability of occurrence by 10%. The cost of these actions is estimated at 2 MU for an associated duration that is not located on the critical path. StT42

A and StT42 B aims to modify the material or the software when problems are observed. Such a strategy costs 1 MU and makes the satellite unavailable for 1 TU. If (R4) occurs even if a preventive strategy has been carried out, it is still possible to develop the corrective strategy. However, only its duration will be taken into account, since the cost will be supported by the suppliers.

Results and discussion

Table 4 presents the results obtained with this approach. The first column shows the different technical solutions. The second column gives the possible preventive strategies. Ø means that the preventive strategy consists in undertaking no action. The third column presents the number of project scenarios containing the previously mentioned preventive strategies. The column entitled "% Pertinents" refers to the percentage of pertinent project scenarios. Are considered as non-pertinent scenarios, the scenarios in which one or more risks occurred, stopping the project without any corrective strategies despite the presence of possible preventive strategies. We consider that corrective strategies should have been applied to that case. The next column presents the maximal criticity among the pertinent scenarios. Still, among the pertinent scenarios, the last column shows the percentage of scenarios that respect the contractual commitments. The variant that maximizes the project scenario number in the zone of agreement is presented in bold in the last column of Table 4. This result means that by choosing the solution A and by applying no preventive treatment strategy, 66% of the pertinent project scenario respects the contractual commitments. Based on these results, the recommendation to the project manager would be simple: choose solution A and apply no preventive strategy.

The weight of the different criteria could influence the results and an expertise is required for the selection of the used data.

The case study is voluntarily simplified for the sake of the demonstration. The approach is flexible since it can be used in different context.

CONCLUSIONS AND PERSPECTIVES

In NPD the interaction between the subsystems of the product often leads to conflicts requiring arbitration between alternative solutions. To choose the best solution to develop and the best-associated risk treatment strategy in the preparation phase of a project is often tricky. Especially when the project should deliver a product presenting technological innovation. If the benefit of such a solution is easy to qualitatively evaluate, each possible solution potentially developed through the design project generates different plannings and different costs and durations but also different risk levels. To estimate the risk level for each project variant, we propose an approach to model and evaluate the impact of risks on the project cost and the schedule cost. This approach uses the synchronized process principle and integrates the repercussion of the project structure modifications on risks and the global risk level. We use the concepts of risk scenario, treatment scenario and project scenario to characterize and evaluate the project solutions. We illustrate the principles of our approach through a case study from the aerospace industry. This methodology analyses the possible scenarios, evaluates the global risk level and selects the best treatment scenarios at any time. An estimate of the global risk level of each solution can be made and gives a vision of the possible scenarios: from the least to the most probable, from the most disastrous to the most optimistic! A software tool has been developed (Java platform). The main perspectives for this research work will be to examine the influence of previously occurring risks on the probability scenarios, but also to integrate human resources constraints into the model, as well as the limiting availability that can be shared over several projects or skill, since particular actors can be assigned/required for specific tasks.

Figure 1 :

 1 Figure 1:Vee cycle.

Figure 2 :

 2 Figure 2:Exploratory design activity based model.

Figure 4 :

 4 Figure 4: Decision tree in design project A.

  , a being the number of identified treatment actions. A treatment action can be materialized by a task to achieve and it can introduce modification to the WBS.A treatment strategy is a preventive strategy if it contains at least a preventive treatment action. Otherwise, it is a corrective strategy. If the strategy consists in running no action at all, it is noted as being an empty set such as Ø (i.e. graceful degradation). The set of project scenarios ES ts is obtained by combining the set of risk scenario and the set of treatment scenario.

								proba(ScPp
	. ts can be associated to one or more treatment strategies StTij A risk Ri ts (j=1...m ts ), m ts being the number of
	identified strategies for Ri
	The treatment actions can be common to several risk treatment strategies. The set of all the identified StTij ts for a risk Ri ts is written StRi ts .
	Then Card(StRi StRi ts )=m ts	= ts +1.	(Ø,StTi1	ts ,..,StTij	ts ,..,StTim	ts )	and
	A treatment scenario ScTd	ts (d=1...D	ts ) corresponds to a
	combination of the treatment strategies chosen to deal with the different risks of P ts . The set of treatment scenarios is given by: E !"# = StR ! ! !!! . For each P ts , EScT ts may contain a set of preventive treatment scenarios EScTprev ts and corrective treatment scenarios EScTcorrec ts .
	The proba(Ri	ts |StTij

ts . A treatment strategy StTij ts groups a set of treatment actions Aijα (α = 1...a) to avoid or reduce the risk Ri ts ts ) is the probability that the event related to Ri ts occurs knowing that StTij ts (preventive strategy) has been done. This probability, as well as the impacts CI(Ri ts |StTij ts ) and DI(Ri ts |StTij ts ), are then qualified as "reduced". A project scenario ScPp ts (p=1...P) is defined as being a possible project achievement that is built with a risk scenario and treatment scenario (ScPp ts =<Pi ts , ScRs ts , ScTd ts >). ts ) is the probability of a given ScPp ts . It takes into account (2) the probability of the occurring risks (Ri ts ∈ ScRs ts ), (3) the probability that several risks does not occur (Ri ts ∉ ScRs ts ), (4) the probability of the occurring risks (Ri ts ∈ ScRs ts ) knowing that a treatment strategy is developed (StTij ts ∈ ScTd ts ) (5) the probability that Ri ts does not occur (Ri ts ∉ ScRs ts ) knowing that a preventive strategy has been processed and the initial probability has been modified (StTij ts ∈ ScTd ts ).

Table 1 :

 1 Phases of the two projects

			Solution A	Solution B
	General design tasks
			Time Cost Time Cost
	T1 Need analysis	2	2	2	2
	T2	Requirement specification	3	2	3	2
		Functional
	T3	architecture	2	3	2	3
		design
		Physical
	T4	architecture	2	2	2	2
		design
	T5 Detailed design	3	3	4	4
	T6 Production stage	4	4	4	4
	T7 Test	3	5	3	5
	T8 Integration	2	4	3	4
	T9 Verification	2	3	2	3
	T10 Validation	1	1	1	1
	T11 Flight	2	8	4	4
	Total		26	37	30	34

Table 2 :

 2 Risks in the 2 versions of the project

	Risk	Version of	Occurrence	Occurrence	Impact	Strategies
		the project	probability	period	TU/MU	(c: p: preventive) corrective,
	R1 Non accepted component	A	0.1	T5	T5+2 / 2	StT11 A (c)
		B	0.8		T5+3 / 2	StT11 B (p), StT12	B	(c)
	R2 Difficulties of communication	A	0.05	T5	T5+4 / 4	StT21 A (p), StT21	A	(c)
		B	0.3		T5+4 / 4	StT21 B (p), StT22	B	(c)
	R3 Lateness of material reception	B	0.2	T6	T6+2 / 10	StT31 A (p)
	R4 Failure of a component or	A	0.25	T7	∞ / ∞	StT41 A (p), StT42	A	(c)
	software	B	0.35		∞ / ∞	StT41 B (p), StT42	B	(c)

Table 3 :

 3 Available risk treatment strategies

	Strategies	Reduced	Treatme	Reduced
		probability (if	nt action	impact
		preventive)	TU/Mu	TU/Mu
	StT11 A		0.5 / 0.5	T5+0.5 /
				0.5
	StT11 B	0.1	2 / 1	NC
	StT12 B		0.5 / 0.5	T5+0.5	/
				0.5
	StT21 A	0.01	0 / 2	NC
	StT22 A		1 / 1	0 / 2
	StT21 B	0.05	0 / 2	NC
	StT22 B		1 / 1	0 / 2
	StT31 A	0.01	0 / 3	NC
	StT41 A	0.1	0 / 4	NC
	StT42 A		1 / 2	T7+1 / 1
	StT41 B	0.05	0 / 4	NC
	StT42 B		1 / 2	T7+1 / 1

Table 4 :

 4 The results of our approach

	Solution	Preventive strategies			Nbr ScP	% Pertinents	Criticity	% Contract respected
									max
	A	Ø							18	66,6667	0,4609	66,6667
		StT21	A						24	66,6667	0,5205	45,8333
		StT41	A						18	50,0000	0,6063	38,8889
		StT21 A + StT41	A			24	50,0000	0,6801	12,5000
	B	Ø							36	61,7021	0,2045	41,6667
		StT11 B						24	66,6667	0,2184	37,5000
		StT21 B						48	66,6667	0,2980	43,7500
		StT11 B + StT21	B			32	66,6667	0,3193	37,5000
		StT41 B						36	50,0000	0,3243	27,7778
		StT11 B + StT41	B			24	50,0000	0,3477	29,1667
		StT21 B + StT41	B			48	50,0000	0,4699	27,0833
		StT11 B + StT21	B + StT41	B		32	50,0000	0,5054	15,6250
		StT31 B						36	66,6667	0,2617	61,1111
		StT11 B + StT31	B			24	66,6667	0,2799	50,0000
		StT21 B + StT31	B			48	66,6667	0,3804	45,8333
		StT11 B + StT21	B + StT31	B		32	66,6667	0,4083	37,5000
		StT31 B + StT41	B			36	50,0000	0,4138	33,3333
		StT11 B + StT31	B + StT41	B		24	50,0000	0,4444	25,0000
		StT21 B + StT31	B + StT41	B		48	50,0000	0,5985	12,5000
		StT11 B + StT21	B + StT31	B + StT41	B	32	50,0000	0,6446	6,2500


F Marmier

D Gourc

V Robin

S Sperandio

A MODEL OF PROJECT SCENARIO EVALUTION TO MONITOR THE LEVEL OF PROJECT RISK AND ASSESS THE FEASIBILITY OF PLANNING

Keywords: Decision support system, organisation, project planning, resource availability, risk management

The companies are faced with a more and more demanding market. Their ability to win contracts and bring in innovative projects is often based on some specific resources. However, several similar projects calling on the same actors or equipments, can be run in parallel within the organization. Resources are usually limited in quantity. If the needs are simultaneous, arbitration becomes necessary to prioritize projects. Therefore, the project managers must reliably know the resource requirements over the project timeline. However, few guides, books or studies on project management deal with these aspects in risky projects. The aim of this article consist in presenting the decision support system we have implemented to help project managers. The various feasible solutions are generated and modeled in a decision tree, which is connected with a model for scenario evalution. The manager is then able to evaluate, compare and choose the best way to control the level of project risk and feasibility of the planning. He will also have the information needed to negotiate the availability of resources with the various resource managers. Finally, this study discusses the case of a building project for a weather-forecasting station. It illustrates the complexity of the decision making problem and validates our approach.

INTRODUCTION

In the current economic environment, companies must improve their efficiency to maintain profitability. Risks must be well controlled. However, regardless of the size of the organization or subject concerned, the projects face many risks. The more innovative the project, or if the technology area is poorly known, the more uncertain and risky the project is. Thus, the initial project planning can be very far from the achieved one. However it is on the basis of such initial planning that many decisions are made. The obtained time slots are used by project managers to assign tasks to resources and to book technical resources. However, several similar projects, using the same skills or equipment, can be run in parallel within the organization. Resources are generally limited, negotiations often become necessary. It is therefore imperative to know the most reliable possible positioning of the needs over the project duration.

More and more organizations use tools and methodological approaches to manage their projects. They help the project manager to meet deadlines, do not exceed the costs and achieve a high quality deliverables. In this context, special attention is given to methods of project management by decision makers and academics. Professional organizations and standards bodies offer guides and books on project management, best practices, etc. [1, [START_REF] Ipma | Competence baseline[END_REF][START_REF]PMBoK A Guide to the Project Management Body of Knowledge[END_REF]. These documents present the processes needed to manage projects and contain the basic elements used by project managers [START_REF] Turner | The global body of knowledge, and its coverage by the referees and members of the international editorial board of this journal[END_REF]. However, few frameworks permit the integration of the impact of risks and their treatments on project planning. Resource requirements can be changed, time-shifted or suppressed. The project manager has no visibility in the real availability of resources. Therefore, the availability of resources can be out of sync with real needs. This article is particularly interested in approaches that take into account the project risk management. Such approaches aim to anticipate potential events and measure their potential impact on the progress of a project or achievement of objectives. They allow the manager to choose treatment strategies for appropriate risk project with knowledge of their impact. Through the works presented in this article, we help the project manager to perceive the feasibility of planning, taking into account the limited availability of critical resources.

In the first section, the literature on methods of risk management shows, the diversity of existing approaches dedicated to specific areas or generic. Then, after a presentation of the model our methodology, which addresses the complexity of the choice of possible treatment strategies in the face of potential risks and the limited availability of resources, is described. Finally, a case study is detailed and we present the results and conclusions of this research work.

RISK MANAGEMENT IN ORGANIZATIONS

Project risk management and decision making process

In the literature, methods of risk management refer to a standard process with the well-known stages: identification, assessment, quantification, treatment and monitoring of risks [START_REF]Project management -guide to the management of business related project related[END_REF][START_REF]International Standards for Business, Risk management -Principles and guidelines[END_REF][START_REF]PMBoK A Guide to the Project Management Body of Knowledge[END_REF]. Tixier et al. propose a classification of 62 existing approaches [START_REF] Tixier | Review of 62 risk analysis methodologies of industrial plants[END_REF]. Methods are sorted according to the fact that they are deterministic and / or probabilistic, but also qualitative or quantitative. Gourc et al. propose a different reading grid that classify risk management approaches whether they are symptomatic or analytical [START_REF] Gourc | Towards a general risk model for piloting goods and service related activities, Vers un modèle général du risque pour le pilotage et la conduite des activités de biens et de services[END_REF]. The first group of approaches, also called risk-uncertainty is associated with approaches where risk management is transformed into uncertainty management [START_REF] Ward | Transforming project risk management into project uncertainty management[END_REF]. The second group of approaches considers risk as an event that may affect the achievement of the project [START_REF] Carter | Introducing RISKMAN: the European project risk management methodology[END_REF].

Two themes are well known for their reference to innovation and the presence of risk: (1) project management of New Product Development (NPD) at an operational level. It consists of applying project management techniques dedicated to innovative projects. It take into account a global tolerance level of the risk. (2) NPD portfolio management at a tactical level. A first definition of portfolio management is given by [START_REF] Cooper | New product portfolio management: Practices and performance[END_REF]: a dynamic of decision-making allowing the list of projects to be always up to date. In this process, new projects are evaluated, selected and sorted. Past projects may be accelerated, stopped or paused and resource assignments can be changed to build a portfolio balanced in innovative projects, following a balance risk / profitability.

Resource management in projects and organizations

Two main research axis can be found in the literature on the management of resources in project: availability and performance. Often, the project activities to be planned are subject to precedence links and specific resource assignments. So in this context, resource management techniques have to minimize different criteria, such as total duration. This is particularly the case for the problems known as RCPSP [START_REF] Hartmann | A survey of variants and extensions of the resource-constrained project scheduling problem[END_REF]. "Organization based on skills" becomes a new model of management and organization of human resources for companies [START_REF] Lawler | A skill-based approach to human resource management[END_REF]. This management style has an effect on the feasibility and performance of the tasks and thus on the project.

The development of projects and project resources management are influenced by company characteristics, by the network of companies and their interdependencies [START_REF] Robin | Towards an integrated management of engineering design system and enterprise[END_REF]. A decision in one of the projects conducted within the network can affect other projects in the network and thus modify their evolution. Consequently, project performance factors are not the same as those of the organization ones. Local indicators describing the projects and global indicators describing the company as a whole can then help decision makers. They know the status of internal and external organizations in terms of availability of resources and thus the capacity of the company [START_REF] Hicks | A framework for the requirements of capturing, storing and reusing information and knowledge in engineering design[END_REF] and its possible evolution [START_REF] Chen | Strategic human resource practices and innovation performance -The mediating role of knowledge management capacity[END_REF].

In conclusion, no tools exist in our knowledge to help the project manager assessing the influence of risk and its management on the organization of activities and therefore the resources that are required by the project. Availability of resources given by the organization to the project can be unsynchronized with the positioning of the real need in time.

MODEL

The occurrence of risk and the modification of the project planning due to the risk treatment strategies affect schedule. Thus the time period during which the tasks will be processed can change. In order to make sure that the project would be realized, the project manager must ensure that the necessary resources will be available during the requirement periods of the project. He must therefore have a reliable vision of the needs, regarding the potential risks, in order to requests resources to the business managers. Similarly, if trade-offs between projects are required, he will need to argue his need.

Hypothesis

Some resources can be replaced more easily than others (by internal resources or external to the organization). Thus, all resources are not considered as being critical. The model we propose is based on two main assumptions:

The integration of project risk management has effects on the duration of the project and cost criteria. The effects taken into account (edit or delete an existing task or inserting new tasks) affect the project duration and cost.

In case of change or evolution of the data, the approach must be restarted on the basis of the new information. This is particularly the case during the evolution of the project where the approach may be processed before the beginning of each phase (milestones) when the information is more accurate.

At any time, the objectives of the model are to:

1. analyze scenarios and to assess the overall level of risk, ie the chance that the project meets the commitments, 2. select the best treatment strategies for anticipating and responding to a risk occurrence, 3. to book resources at the right time.

Definitions and data

A project is described by PTt tasks (t=1...T), where T is the number of tasks. The planning process provides an initial planning Pi which does not include the risks. A set of critical resources ECR is required for certain tasks.

A project is also described by its set of risks identified ER Ri (i= 0...n), where n is the number of risks. Each Ri is characterized through the process of risk management. A risk Ri is also characterized by its period of occurrence, ie tasks during which the risk may occur. proba(Ri) is the probability that the event related to Ri occurs. Its impact on costs is noted CI(Ri) and on project duration DI(Ri). Impacts can affect a task different from that characterizing the period of occurrence. These probabilities and impacts are also called initial probabilities and initial impacts.

A risk scenario ScRS is the combination of risks that occur during a project P. A project with n risks leads to 2 n risk scenarios. ScRS (S=1,...,2 n ) is a possible implementation risks with k (0≤k≤n). The total number of risk scenarios with k risks is equal to n!/k(n-k)!. The probability is noted proba(ScRs) (the probability that the events in this scenario may occur and other risks do not occur).

𝑝𝑟𝑜𝑏𝑎(𝑆𝑐𝑅

! ) = 𝑝𝑟𝑜𝑏𝑎 𝑅 ! 𝑖𝑓 ( 𝑅 ! ∈ 𝑆𝑐𝑅 ! ) 1 -𝑝𝑟𝑜𝑏𝑎 𝑅 ! 𝑖𝑓 ( 𝑅 ! ∉ 𝑆𝑐𝑅 ! ) ! !!!
Each risk can be treated in different ways: preventive, corrective or combination of both. Several treatment strategies StTij (j = 1 ... m) can be identified for a risk Ri,, m is the number of identified strategies. Treatment strategy StTij is a set of processing actions Aijα (α=1...a) to prevent or reduce the risk, a is the number of processing actions identified. An action can be materialized by a task to be performed that can generate three types of changes:

1. adding a new task, 2. deleting a task and the potential risks associated, 3. modification of an existing task.

A treatment strategy is a preventive strategy if it contains at least one preventive action. Otherwise, it is a corrective strategy. If the strategy consists in performing no action at all, it is noted as an empty set Ø. That means that the task is executed impacted in degraded mode (initial impact). ScPp a project scenario (p=1...P) is defined as a possible achievement of the project including a risk scenario and a scenario of treatment (ScPp=<Pi, ScRs, ScTd>). Its duration is noted by D(ScPp). The set of project scenarios (ES) is obtained by combining all the risk scenarios, and all scenarios treatments. proba(ScPp) is the probability of achievement of a ScPp. It takes into account (1) the probability of the occurring risks (Ri ∈ ScRs), (2) the probability that several risks do not occur (Ri ∉ ScRs), (3) the probability of risks occurrence (Ri ∈ ScRs), knowing that a treatment strategy is implemented (StTij ∈ ScTd) (4) the probability that Ri does not happen (Ri ∉ ScRs), knowing that a preventive strategy has been deployed and that the initial probability was modified (StTij ∈ ScTd).

𝑝𝑟𝑜𝑏𝑎 𝑆𝑐𝑃 ! = 𝑝𝑟𝑜𝑏𝑎 𝑅 ! (1) 1 -𝑝𝑟𝑜𝑏𝑎 𝑅 ! 2 𝑝𝑟𝑜𝑏𝑎 𝑅 ! 𝑆𝑡𝑇 !"
(3)

1 -𝑝𝑟𝑜𝑏𝑎 𝑅 ! 𝑆𝑡𝑇 !" (4) ! ! !!"# ! , !"# !" !!"# ! !,!
The cost of a project scenario is noted by C(ScPp). It includes the cost of the tasks C(Tt) of the initial planning of the project, the ScRp and the ScTp selected and the global cost GC initial (Ri) of the occurring risks arising but not covered by the treatment strategies [START_REF]Project management -guide to the management of business related project related[END_REF]. Latter comprises a fixed part of the total cost (materials, tools, bulks, etc.) and an indirect cost depending on the duration of the action, on the delay and on the actors. It also includes the reduced global cost GC reduced (Ri) that is obtained by taking into account different strategies StTij applied to treat Ri and the reduced impact on the project cost and duration [START_REF]International Standards for Business, Risk management -Principles and guidelines[END_REF]. Finally it also integrates the cost of the treatment strategies StTij that is determined by the cost of the actions [START_REF] Tixier | Review of 62 risk analysis methodologies of industrial plants[END_REF]. This cost consists of direct costs (materials, tools, etc..) and an indirect cost which depends on the duration of action and strategy involved.

𝐶 𝑆𝑐𝑃 ! = 𝐶(𝑇 ! ) ! !!! + 𝐺𝐶 !"!#!$% 𝑅 ! (5) 𝐺𝐶 !"#$%"# 𝑅 ! 𝑆𝑡𝑇 !,! 6 𝐶 !"# !" ∈!"# ! 𝑆𝑡𝑇 !" (7)
! ! !!"# ! !,!

Objectives

During the preparation phase of a project, a project manager must be able to estimate the chances of project success. He must take into account the risk when determining the targets of cost and deadline. Then he has to select risk treatment strategies based on their potential to help reaching the terms of the contract.

The objective of this research is to give the project manager a method to compare different treatment strategies of risk. Risks and strategies affecting the positioning in time of the tasks, the project manager must have an indication of the feasibility of the schedules in relation to the availability of resources.

RESOLUTION APPROACH

A proactive approach

In an industrial context, the management of risks associated with a project is a source of many questions and thoughts since whatever the decision taken, it will lead to risks and associated impacts on the rest of the project.

Our approach aims to help project actors, particularly project managers in their decision making process both during the preparation phase of the project and during the running phase. The body of the approach (figure 1) is composed of two phases:

1. generation of all possible project scenarios and their evaluation, 2. selection the best risk treatment strategies based on their ability to make the project meet the contractual commitments and their feasibility.

In the preparation phase of the project, the risk management mode must be selected. Our approach uses the classical data concerning the tasks, the risks and their associated treatment. These data are supposed to be collected on the basis of expert knowledge involved in the project. Therefore, our method has input data provided by the planning process (management team) and the process of risk management. When the project is launched, our approach can also be used when a event occurs and the project manager needs to assess the impact of the decision he must make.

Generation projects possible scenarios

To assess the possible project scenarios, the management team must first generate an initial schedule "nominal", without integrating the concepts of risk and risk treatment actions. From the initial planning, the experts will define the risk treatments and build all project scenarios. Times and costs will be calculated for each project scenario using the tool ProRisk [Nguyen et al., 2010]. This tool calculates the cost and duration of each project scenario, taking into account possible changes induced by the implementation of treatment strategies. The method of calculation of the probability for each project scenario differs between project scenarios whether they include or not a preventive treatment strategy.

A comparison between the periods of availability of resources (granted by the organization, critical or not) and the periods of resource requirements (tasks of the project), the project leader and the resource manager choose the most favorable scenario, to complete the project. Thus, the initial planning is adapted depending on the scenario studied (modified task duration, tasks added or removed) and the duration of the project scenario is calculated using the PERT method.

Selection of the best treatment strategies

Based on the set of project scenarios, it is possible to adopt an opposite approach to the more conventional approaches (choice of preventive strategies and secondly corrective strategies). Decisions can be taken regarding to identified risks and associated treatment strategies: Decision D1 can be taken, knowing what would be the most appropriate decision D2 for each ScRs (Figure 2).

Figure 2. Decision tree

In this decision tree (Figure 2), the project management team goes through decisions represented by decision nodes D1 and D2 to reach its objectives. The first decision D1 is to select preventive strategy risk treatment. D1 would be performed during the preparation phase of the project and its consequences are included in the initial planning of the project. D2 consists in choosing the corrective actions that will be performed in front of the occurrence of all known events (risk scenario). These events are represented by the event nodes E (also called chance nodes) on the decision tree. Corrective strategies consist in selecting strategies that would avoid the scenarios that are not possible in reality, such as scenarios where the project would be stopped pending corrective action or with a NoGo situation.

The resolution of this tree is therefore in the opposite direction of its construction: D2 for each ScRs, then D1 knowing all D2.

D2 is obtained by minimizing the criticality (details of the calculations are presented below) D1 consists selecting the strategy that avoids the worst possible case (project scenarios) as defined by the criterion of Savage often used in decision theory [START_REF] Petar | Application of sensitivity analysis in investment project evaluation under uncertainty and risk[END_REF]. It consists in minimizing the maximum criticality (also called regret) that is a measure of the regret that the decision maker could have if he had preferred action rather than another.

The criticality calculation is obtained as follows:

Each project P has a set of ScPp and each of them can be characterized by a criticality Cr(ScPp). This measure is based on the probability of occurrence proba(ScPp), and a metric of duration and cost, respectively, αp and βp:

∝ ! = !"(!"# ! ) !"# (!"(!"# ! )) et 𝛽 ! = !"(!"# ! ) !"# (!"(!"# ! )) (p =1…P) (8) et ∝ ! , 𝛽 ! ∈ [0,1].
Where CI(ScPp) and DI(ScPp) are respectively the distance between [cost -duration] and the [budget -thresholds set out in the project contractual agreement]. max(CI(ScPp)) and max(DI(ScPp)) represents the distance from the most expensive and the longest possible project scenario.

The overall impact, and weighted normalized impact (ScPp) is then obtained by the following formula:

𝐼𝑚𝑝𝑎𝑐𝑡(𝑆𝑐𝑃𝑝) = 𝑞×∝ ! + 𝑞′×𝛽 ! ( 9
)
Where q and q' (q+q'=1) are two coefficients selected by the project manager based on the importance of the duration criterion relatively to the cost one.

Then p, Cr(ScPp) = proba(ScPp) X impact(ScPp) [START_REF] Carter | Introducing RISKMAN: the European project risk management methodology[END_REF] D1 is to choose the preventive strategy that is most appropriate. Preventive strategy that minimizes the maximum criticality is chosen based on the selections made in D2. For each ScTprevS the maximum criticality CRmax(ScPp | ScTprevS) is obtained by the ScPp associated with the given ScTprevS having the Cr(ScPp) maximum. Thus, ∀p choose ScTprevS such that min CRmax(ScPp | ScTprevS)

The project management team wants to maximize its chances of meeting the commitments. This methodology is a flexible tool. Results are available before the beginning of the project. In case of change or if new information is available, the approach can be processed again. It can also be restarted at each milestone for the project, for each sub-project or when an event appears or when the context changes.

CASE STUDY

To illustrate and validate our proposals, we propose an example taken from a case study on the construction of a weather forecast station. This project involves different business as earthwork, masonry, metal (mounting a mast / antenna wire), electronics, meteorology, etc.).. Different actors of different sectors are involved in the project. Many project data were collected from participants in the project and particularly the project manager. At the end of our case study, results are discussed with the project manager and the scenarios are compared on the basis of their effectiveness.

Presentation

The project includes 10 phases, for a total of 42 tasks (T1 to T42). The initial planning gives a horizon of 180 UT (Unit of Time). In the context of our example, we will consider two risks that may occur during the development of the task of drilling and the assembly of the antenna (table 1). Different strategies to deal with these risks are studied (Tables 2 and3). Two critical resources are considered (Table 4), they contribute to the achievement of certain tasks and are neither replaceable nor exchangeable. The risk R1 described in table 1 corresponds to the premature wear of the head of the drill. If it is not replaced, the activity slowed. R2 is the risk of injury on site during installation of the metal structure of the antenna.

Its initial impact is an increase in the charge of the other team members and therefore an extension of the task.

Presentation of results

Table 5 shows the different project scenarios possible in this study. It is possible from this table to assess the relevance of the implementation of each corrective strategy for each risk scenario in terms of criticality.

Criticality of zero means that the project complies with contractual commitments.

Analysis of possible decisions

The objective of this research work is to give decision makers a methodological tool to compare the impact of risks and their treatment on the project, but also to have a light on the feasibility with regard on the availability of critical resources.

The proposed approach permits evaluate the relevance of corrective treatments of risk, this whatever the context of preventive treatment implemented. Knowing which corrective treatment will be decided in all possible cases, it is possible to decide what would be the best possible preventive treatment strategy.

For each scenario, information about the feasibility are given. In a first approach it is possible to know whether the schedule is feasible with the availability granted by the organization. If it doesn't correspond, the project manager can specify the dates where critical resources are needed. If all its resources are already preempted for other projects over the periods of requirement, negotiation can be started with the business manager. If the business manager offers different resources with different ranges of availability, the PM can select those that can best ensure the progress of the tasks for which they would be in charge knowing the potential occurrence of risks.

The example is deliberately simple for the demonstration purposes, but the behavior of the proposed approach can be extrapolated to wider problems and situations more representative of actual project.

Risks do not relate directly to critical resources. But the impact of these risks can modify the period of critical resources requirement. Risks relating to resources are then transferred on the tasks, within the framework of this approach.

CONCLUSION

The choice of the best strategy for dealing with the risks of a project is often difficult. When the project aims to deliver a product with technological innovations, specific resources may be required. Resources with particular characteristics are often limited in organizations. It becomes necessary to ensure that they will be available at the right time and will be used when available. We propose an approach to model and assess the impact of risks on the duration and cost of the project. This approach uses the principles of the synchronized project planning and risk management. It is based on the concepts of risk scenario, risk treatment scenario and project scenario to characterize and evaluate the effects of risks on the project. This method evaluates the overall level of risk and select the best risk treatment strategy while analyzing the feasibility. An estimate of the overall risk of the project gives an overview of possible scenarios. We illustrate the principles of our approach through a case study and a software tool has been developed. It is often difficult to compare criteria antagonists such as compliance with contractual commitments and resource overload. The first perspective of this work is to facilitate the integration of the resource overload in the other two criteria (cost and duration). Given that different actors may be required for specific tasks, the second perspective of this research is to examine the influence of the skills required in the project in the model.

Finally

  , several treatment strategies are possible for each risk Ri. Strategies may be common to several risks. The set of the identified to StT ij for a given risk Ri is noted StRi. StR i = (Ø, StT i1 ,.., StT ij .., StT im ) and Card(StR i )= m +1. A treatment scenario ScTd (d=1...D) corresponds to a combination of treatment strategies chosen to deal with the various risks of the project. The entire treatment scenarios is given by: 𝐸 !"# = 𝑆𝑡𝑅 ! ! !!! . For each project P, EScT may contain a set of preventive treatment scenarios EScTprev and / or corrective treatment scenarios EScTcorrec. proba(Ri | StTij) is the probability that the event associated with Ri occurs, knowing that StTij (preventive strategy) was performed. The probability as well as the and impacts CI(Ri | StTij) and DI(Ri | StTij) are then called "reduced probability" and "reduced impact."

Figure 1 .

 1 Figure 1. Proposed approach

Table 1

 1

	: Risks

Table 3 :

 3 Treatment actions

		Predecessors	Successors	Duration	Fixed cost
				(UT)	(UM)
	A 111	Delineate the area	Prepare the core	1	4
	A 121	Core drilling of the area	Study carrots	1	4
	A 131	Core drilling of the area	Study carrots	0.2	24
	A 141	Delineate the area	Prepare the core	1	4
	A 142	Core drilling of the area	Study carrots	0.2	24
	A 211	Assembly of the structure	Equipping the	0.2	4
			structure
	A 221	Assembly of the structure	Equipping the	1	8
			structure

Table 4 :

 4 The critical resources

		Tasks requiring the CR	Beginning of the	Duration of the
			availability period	availability (UT)
	Drum	Make the basement of the	t0	25
		structure
	Steamroller	Steam the access	t0	13

Table 5 :

 5 Set of project scenarios

	N° ScP	Probability	Couple (Risk,Strategy)	Duration (UT)	Cost (UM)	criticality
	1	0,64	(.,ST11)	183	4	0,000
	2	0,56		180	0	0,000
	3	0,24	(R1,.)	195	0	0,000
	4	0,24	(R1,StT12)	185	4	0,000
	5	0,24	(R1,StT13)	181	24	0,000
	6	0,16	(R1,ST11)	183	4	0,000
	7	0,16	(R1,StT14)	186	28	0,022
	8	0,16	(.,ST11) (R2,.)	193	4	0,000
	9	0,16	(.,ST11) (R2,StT21)	183	8	0,000
	10	0,16	(.,ST11) (R2,StT22)	183	12	0,000
	11	0,14	(R2,.)	190	0	0,000
	12	0,14	(R2,StT21)	180	4	0,000
	13	0,14	(R2,StT22)	180	8	0,000
	14	0,06	(R1,.) (R2,.)	205	0	0,030
	15	0,06	(R1,.) (R2,StT21)	195	4	0,000
	16	0,06	(R1,.) (R2,StT22)	195	8	0,000
	17	0,06	(R1,StT12) (R2,.)	195	4	0,000
	18	0,06	(R1,StT12) (R2,StT21)	185	8	0,000
	19	0,06	(R1,StT12) (R2,StT22)	185	12	0,000
	20	0,06	(R1,StT13) (R2,.)	191	24	0,000
	21	0,06	(R1,StT13) (R2,StT21)	181	28	0,008
	22	0,06	(R1,StT13) (R2,StT22)	181	32	0,019
	23	0,04	(R1,ST11) (R2,.)	193	4	0,000
	24	0,04	(R1,ST11) (R2,StT21)	183	8	0,000
	25	0,04	(R1,ST11) (R2,StT22)	183	12	0,000
	26	0,04	(R1,StT14) (R2,.)	196	28	0,007
	27	0,04	(R1,StT14) (R2,StT21)	186	32	0,013
	28	0,04	(R1,StT14) (R2,StT22)	186	36	0,020
	Table 6 shows the dispersion of possible outcomes of the	overload (which characterizes the resource requirement
	decision to implement or not StT11 preventive strategy. It	that are not satisfied) as small as possible. Globally, over
	is possible to observe that the implementation of StT11	all scenarios, the cumulative non satisfied resources
	minimizes the maximum achieved criticality obtained in	needs is minimized with the same treatment strategy.
	the worst case. It also allows to obtain a maximum

Table 6 :

 6 Possible project scenarios sorted by the preventive risk treatment strategies

	StTprev	Nbr of ScP	Nbr of ScP respecting the contract	Cumulated probability, ScP respecting the contract	Minimal criticality	Maximal criticality	Overload	Total overload
	Ø	16	13	2,24	0,0	0,03	8	40
	StT11	12	8	1,68	0,0	0,02	3	24


Pierre Couturier
email: pierre.couturier@mines-ales.fr

Abdelhak Imoussaten

A qualitative method for evaluation in conceptual design

Keywords: Systems engineering, Qualitative evaluation, Multicriteria analysis, Mechatronics 

In systems engineering, evaluation is required each time design choices must be made or justified. Generally, design choices take place according to multiple criteria: some of them may be contradictory. Moreover, since knowledge is still missing at the conceptual stage, it is often difficult to anticipate the consequences of design decisions on the ultimate effectiveness of the final product. An original qualitative evaluation method, taking into account the decision makers' optimistic or pessimistic attitude, and combining a preferences model on the criteria is proposed for the purpose of highlighting the most promising alternative system design solutions. An example in the mechatronics field serves to illustrate our proposals.

INTRODUCTION

In the competitive marketplace, both time and finances are often lacking to finalizing several product concepts and then retaining only the most satisfactory one. Engineers therefore require support in reviewing possible system design solutions (called more shortly 'alternatives' hereafter) and in making and defending the best design choices from the earliest product design stages [START_REF] Hehenberger | Hierarchical design models in the mechatronic product development process of synchronous machines[END_REF]. The design evaluation process however faces a number of challenges, including:

1) Though the initial choices are definitely critical to ensuring a successful design, rating the merit factors of each alternative during the conceptual design stage is generally subject to uncertainty and imprecision due to the lack of knowledge about the system/product behaviour;

2) When designing complex products such as mechatronics ones, assessing the consequences of choices among several alternatives is a critical issue that has still not been resolved effectively to this day [START_REF] Torry-Smith | Mechatronic designstill a considerable challenge[END_REF];

3) Design choices can benefit to some criteria and can damage some others. The challenge then is to identify satisfactory alternatives that achieve an acceptable balance between the technical requirements.

Our proposal seeks to address above issues by extending some qualitative reasoning to determine the level of satisfaction of considered alternatives [START_REF] Imoussaten | Multi-criteria improvement of options[END_REF] [START_REF] Couturier | Tracking the consequences of design decisions in mechatronic systems engineering[END_REF]. After defining in section 2 the role of evaluation within the Systems Engineering (SE) framework, section 3 presents our qualitative approach for evaluation. Based on the relationships between a behaviour model of the system being designed and a decision-making model, a qualitative evaluation taking into account the decision makers' pessimistic or optimistic attitude is developed. Then a satisfaction score is associated to each alternative taking into account a preferences model on the criteria. MAUT multicriteria analysis method [START_REF] Keeney | New Industrial Product Design and Evaluation Using Multiattribute Value Analysis[END_REF] has been chosen since we are interested to evaluate the satisfaction score of each alternative on each criteria (regardless other alternatives) and since we limit the expression of the preferences model to the relative importance of criteria, considered as independent. The method improves goals model reasoning [START_REF] Giorgini | Reasoning with Goal Models, 21st International Conference on Conceptual Modeling (ER02)[END_REF] since conflicts resulting from contradictory qualitative influences on goals can be solved. Moreover, it is proved, in section 3, that the proposed evaluation operators are consistent with the optimistic or pessimistic attitude of the decision makers. Section 4 illustrates our proposals through the design of an autonomous mobile robot.

EVALUATION IN SYSTEMS ENGINEERING

Systems Engineering (SE) is an interdisciplinary and collaborative approach based on the standards ISO/IEC 15288 (ISO/IRC, 2008), ANSI/EIA 632 and IEEE 1220, as well as on conceptual guidelines, e.g. SEBoK (BKCASE, 2012) and best practice guidelines, e.g. INCOSE handbook (INCOSE, 2011) or Nasa handbook (Nasa, 2007). SE promotes vocabulary, concepts and a comprehensive approach that facilitates communication within a design team, regardless of the underlying technological field, and moreover relies on a set of standardized processes and activities that have become widely and successfully applied across various industrial fields (SE Guide book, 2007).

Among the aforementioned SE components, five technical processes run in an iterative manner when engineering a system. These iterative and recursive processes are (SEBok, 2012): Mission analysis, Stakeholders' needs definition, System requirements specification, Logical architecture design, and Physical architecture design. They are supported by three services, namely: System verification, System validation, and System analysis.

According to standards EIA 632, IEEE 1220, SEBok 2012, technical performance assessment and trade-offs studies are necessary activities of System analysis.

Thus this paper addresses the set of evaluation issues 1) to 3) (see section 1) and focuses on evaluating the effectiveness of technical product under design. The proposed evaluation method could also be applied in other SE application domains, including the development or the amelioration of socio technical systems such as manufacturing systems. The limitation of the method is the availability of reliable expertise in the application field.

The next section will present the generic activities associated with the evaluation process in SE.

Evaluation process

In SE, an evaluation is required each time engineering choices must be made or justified, e.g. choices of major operations and solution concepts, research and resolution of inconsistencies between requirements, choice of logical vs. physical architecture.

This evaluation process includes generic activities, as summarized in Fig. 1. Without going into the details (see [START_REF] Couturier | Tracking the consequences of design decisions in mechatronic systems engineering[END_REF]) for more explanation), the main activities composing the evaluation process consist of: defining the evaluation objectives ("why" and "what"), preparing the job ("how"), producing and analysing the robustness of the results ("do" and "check"), and finally delivering outcomes ("conclude"). If the evaluation objectives are not met or if results lack adequate differentiation or show excessive dependence on the evaluation methods, then the evaluation process should return to one of its previous steps. Since the evaluation process constitutes the backbone of the decision-making process, the decision-making model must be specified in the 'How activities'. Such a model takes into account core elements, e.g. evaluation criteria, decisionmakers' preferences, scoring functions used to assign satisfaction levels to system attributes, and aggregation methods chosen to provide a global satisfaction score of alternatives.

Multicriteria analysis in evaluation

Alternatives should be assessed or ranked according to most suitable multicriteria analysis methods. Regarding the evaluation of product effectiveness, three main families of multi-criteria decision-making methods can be distinguished (Fülop, 2005): total aggregation methods (Multi-Attribute utility Theory MAUT [START_REF] Keeney | New Industrial Product Design and Evaluation Using Multiattribute Value Analysis[END_REF] (Fishburn,1982), paired comparison methods (Analytic Hierarchy Process AHP [START_REF] Saaty | On the invalidity of fuzzifying numerical judgments in the Analytic Hierarchy Process[END_REF], and outranking approaches (French methods PROMETHEE, GAIA, ELECTRE I, II, III [START_REF] Roy | The outranking approach and the foundations of electre methods[END_REF].

Each of these methods varieties has advantages and disadvantages and is best suited depending on application cases. Comparing these methods is not an objective of this paper. However, the MAUT framework seems to us best suited for our case. Indeed, we are interested to evaluate the behaviour of each alternative on each characteristics of the system (regardless other alternatives) and we are also potentially interested to assess the evolution of alternatives evaluations when new components are added to it. Thus aggregation operators are needed.

We consider that performance indicators are used to assess an alternative on a criterion. For instance, when designing an autonomous robot, performance indicators such as the 'maximum robot speed reached by the robot on a straight way' or the 'capacity of the robot to turn on itself' can be used to assess a 'mobility' type criterion. The goal to reach by a performance indicator is specified by the performance requirements: For instance, a maximum speed between 0.5m/s and 1.5m/s; a turn capacity between -180° and +180°.

The values taken by an indicator depend on the design choices made by the designers all along the SE process: for instance the number of driving wheels, the type of motors etc. So, the values of the performance indicators generally change until the final product is built. To help designers make the most satisfactory choices all along the SE process, the performance indicators values have to be predicted using some models of the product.

With such models and from the design parameters choices (the inputs of the models) can be predicted the performance indicator values (the outputs of the models). For instance, when designing an autonomous robot, a predictive model could be used to estimate the value of the 'maximum speed of the robot' (output variable) from the number of driving wheels, the type of engine etc. (input variables). In the following section, no distinction is made between performance indicators and 'output variables' nor between design parameters and 'input variables'.

Because of the lack of knowledge before the system is built, predictive models tend to be coarse at the conceptual design stage, whereas at the detailed design stage, more specialized and precise models are taken into consideration in order to derive product feature data.

Let's precise now how predictive models are linked to decision-making models in order to evaluate alternatives.

QUALITATIVE EVALUATION

Predictive and decision-making models

Let's start by considering a situation with m input variables. Each input variable takes its values in the set denoted 𝑋 𝑗 with 𝑗 = 1, . . , 𝑚. Let's denote 𝑋 = 𝑋 1 × 𝑋 2 ×. .× 𝑋 𝑚 the set of all m_tuple𝑠 𝑥 = (𝑥 1 , 𝑥 2 , . . , 𝑥 𝑚 ) where 𝑥 𝑗 ∈ 𝑋 𝑗 is the value of the j th input variable of 𝑥. 𝑋 𝑗 may be a set of nominal, discrete or continuous values. Each admissible element of 𝑋 stands for an alternative.

Let's also consider that the system may be described by n output variables relative to the decision-making criteria. Each output variable takes its values in the set denoted 𝑌 𝑖 with 𝑖 = 1, . . , 𝑛. Let's denote 𝑌 = 𝑌 1 × 𝑌 2 ×. .× 𝑌 𝑛 the set of all n_tuples 𝑦 = (𝑦 1 , 𝑦 2 , . . , 𝑦 𝑛 ), where 𝑦 𝑖 ∈ 𝑌 𝑖 is the value assigned to the i th output variable of 𝑦. 𝑌 𝑖 may be a set of nominal, discrete or continuous values. To each input in 𝑋 corresponds an output in 𝑌 representing the characteristics of interest for the decision makers/designers. Let's indicate this correspondence by the transformation 𝑇 = (𝑇 1 , … , 𝑇 𝑛 ) where 𝑇 𝑖 : 𝑋 → 𝑌 𝑖 , 𝑖 = 1, . . , 𝑛 (Fig. 2). T may be a formal transformation deduced either from physical laws or from available and validated knowledge in the field. The values taken by the output variables must then be interpreted in terms of satisfaction with respect to the decision makers'/designers' goals. The Multi-Attribute Utility Theory allows expressing the degree of satisfaction of an output variable by a real-valued utility function in [0,1]. Let denote 𝑢 𝑖 : 𝑌 𝑖 → [0,1] the utility function for the 𝑖 𝑡ℎ output variable. Thus 𝑢 𝑖 (𝑦 𝑖 ) represents the extent to which the goal associated with the 𝑖 𝑡ℎ output variable is satisfied by the value 𝑦 𝑖 .

Fig. 2: Decision model and predictive model

Then the global satisfaction level of an alternative should be computed using some aggregation operator U (Fig. 2). For example, U can represent the weighted average operator. However, as mentioned above, the predictive models that provide transformation T are typically unknown or incomplete during the conceptual design phase. The scenario is then often played out where experts or senior engineers are asked to advice on alternatives and judge the qualitative effects of design choices on decision makers'/designers' satisfaction. To deal with this kind of uncertainty context, we have relied on the research work presented in [START_REF] Montmain | Multi-criteria improvement of complex systems[END_REF] regarding qualitative analyses.

Qualitative evaluation

The notion behind the qualitative model in [START_REF] Montmain | Multi-criteria improvement of complex systems[END_REF] is that an expert can express the fact that an input variable value will improve (positive effect) or deteriorate (negative effect) the performance of the system under design with regard to a criterion. Moreover, experts are also able to associate a degree of confidence to this fact. Authors state: "j th input variable value may improve (resp. deteriorate) i th output variable with a confidence degree 𝛿 𝑖𝑗 + (resp. 𝛿 𝑖𝑗 -) (see Fig. 3).

We use the same semantic in this paper and consider that 𝛿 𝑖𝑗 We intend to compare the alternatives taking into account both the improvement and the deterioration effects of alternatives on each output variable [START_REF] Giorgini | Reasoning with Goal Models, 21st International Conference on Conceptual Modeling (ER02)[END_REF].

At this aim, we define the impact of an alternative on an output variable as the pair: (confidence degree in amelioration, confidence degree in deterioration).

To assign such impact degrees we introduce two aggregation operators along with two propagation constraints as in [START_REF] Montmain | Multi-criteria improvement of complex systems[END_REF]. Each combination of an aggregation operator with a propagation constraint yields a decision-maker's attitude. The following pessimistic or optimistic attitude of the decisionmaker can serve as a guide.

Fig. 3: Negative and positive effects on output variables Indeed, the estimation of an alternative impact on the output variable naturally depends on the design choices, as well as on the decision maker's decisional behavior: a pessimistic attitude (whereby a risk aversion position will lead to retaining the most highly negative impact degree) vs. an optimistic attitude (whereby risk acceptance will retain the most highly positive impact degree).

Let's denote 𝑎𝑡 ∈ {1,2} the decision-maker's attitude ("1" encodes the pessimistic attitude and "2" the optimistic attitude). For each 𝑎𝑡 and 𝑥 ∈ 𝑋, let's also denote 𝛿 𝑖,𝑥 𝑎𝑡 the impact of alternative 𝑥 on the i th output variable. As proposed above 𝛿 𝑖,𝑥 𝑎𝑡 ∈ QS × QS can be represented by the pair of ordinal values (𝛿 𝑖,𝑥 +,𝑎𝑡 , 𝛿 𝑖,𝑥 -,𝑎𝑡 ), where 𝛿 𝑖,𝑥 +,𝑎𝑡 (resp. 𝛿 𝑖,𝑥 -,𝑎𝑡 ) corresponds to an aggregation of the degrees of confidence in positive (resp. negative) effects.

The pessimistic attitude is modeled by assigning the least positive impact degree as the aggregation of improvement cases (less favorable cases) and the worst negative impact degree as the aggregation of deterioration cases (less favorable cases again). The propagation constraint consists of comparing the least positive impact degree to the worst negative impact degree on the i th output variable, resulting in:

𝛿 𝑖,𝑥 +,1 = { min 𝑗 𝛿 𝑖𝑗 +,1 𝑖𝑓 min 𝑗 𝛿 𝑖𝑗 +,1 > max 𝑗 𝛿 𝑖𝑗 -,1 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 𝛿 𝑖,𝑥 -,1 = { 0 𝑖𝑓 min 𝑗 𝛿 𝑖𝑗 +,1 > max 𝑗 𝛿 𝑖𝑗 -,1 max 𝑗 𝛿 𝑖𝑗 -,1 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (1)
The optimistic attitude is modeled by assigning the highest positive impact degree as the aggregation of improvement cases and the worst negative impact degree as the aggregation of deterioration cases. The propagation constraint consists of comparing the highest positive impact degree to the worst negative impact degree on the i th output variable hence:

𝛿 𝑖,𝑥 +,2 = { max 𝑗 𝛿 𝑖𝑗 +,2 𝑖𝑓 max 𝑗 𝛿 𝑖𝑗 +,2 > max 𝑗 𝛿 𝑖𝑗 -,2 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 𝛿 𝑖,𝑥 -,2 = { 0 𝑖𝑓 max 𝑗 𝛿 𝑖𝑗 +,2 > max 𝑗 𝛿 𝑖𝑗 -,1 max 𝑗 𝛿 𝑖𝑗 -,2 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (2)
It is clear that for a same output variable, both the positive impact degree and the negative impact degree can't be different from 0 at the same time, that is:∀𝑖, min(𝛿 𝑖,𝑥 +,𝑎𝑡 , 𝛿 𝑖,𝑥 -,𝑎𝑡 ) = 0

It can be verified that the positive impact on any criterion according to an optimistic attitude is greater or equal to the positive impact according to a pessimistic attitude: ∀𝑖 𝛿 𝑖,𝑥 +,2 ≥ 𝛿 𝑖,𝑥 +,1

(3) Verification of expression ( 3 In the same way, it can be easily verified that the deterioration impact on any criterion according to a pessimistic attitude is superior or equal to the deterioration impact according to an optimistic attitude:

∀𝑖 𝛿 𝑖,𝑥 -,1 ≥ 𝛿 𝑖,𝑥 -,2 (4)
Let's denote 𝛿 𝑥 𝑎𝑡 = (𝛿 𝑥 +,𝑎𝑡 = 𝐹 1 (𝛿 𝑖,𝑥 +,𝑎𝑡 ), 𝛿 𝑥 -,𝑎𝑡 = 𝐹 2 (𝛿 𝑖,𝑥 +,𝑎𝑡 )) the 'global impact' of the alternative x on the global satisfaction level. Operators 𝐹 1 and 𝐹 2 are chosen not only to avoid risky choices but also to facilitate the ranking of alternatives. For instance, the global impact can be computed by taking 𝐹 1 as "min" operator and 𝐹 2 as "max" operator.

𝛿 𝑥 𝑎𝑡 = (min 𝑖 ( 𝛿 𝑖,𝑥 +,𝑎𝑡 ), max 𝑖 ( 𝛿 𝑖,𝑥 -,𝑎𝑡 )) ( 5)

In section 3.3 we show how importance of criteria can be taken into account so as to smooth veto and dictator effects of some poor but uncertain score through min or max operators.

Introducing a preferences model in the qualitative evaluation

In this first work we limit the expression of preferences model to the consideration of the relative importance of criteria. Let's assume that decision-makers/designers can express the importance of criteria in estimating the global impact of the alternatives. In order to keep the global impact interpretable, criteria importance take values in the scale 𝑄𝑆. with 𝛿 𝑖,𝑥,𝜔 𝑖 +,𝑎𝑡 = max (¬𝜔 𝑖 , 𝛿 𝑖,𝑥 +,𝑎𝑡 ); 𝛿 𝑖,𝑥,𝜔 𝑖 -,𝑎𝑡 = min (𝜔 𝑖 , 𝛿 𝑖,𝑥 -,𝑎𝑡 )

Aggregation operator (6) limits the veto given to each criterion through the "min" and "max" operators of expression (5). Indeed, low weights reduce the penalizing effect of a low improvement impact degree in the min Let us verify, hereafter, that the proposed evaluation operators are consistent with the optimistic or pessimistic attitude of the decision makers.

It can be verified that for any weight (extending the case without any weight), the positive impact on any criterion according to optimistic attitude is equal or greater than the aggregated positive impact according to the pessimistic attitude:

∀𝑖 ( 𝛿 𝑖,𝑥,𝜔 𝑖 +,2 ≥ 𝛿 𝑖,𝑥,𝜔 𝑖 +,1 ) (7)
Verification for expression ( 7 In the same way, it can be verified that for any weight, the negative impact on any criterion according to the pessimistic attitude is equal or greater than the aggregated negative impact according to the optimistic attitude: ∀𝑖 ( 𝛿 𝑖,𝑥,𝜔 𝑖 -,1 ≥ 𝛿 𝑖,𝑥,𝜔 𝑖 -,2 ) (8) It comes, using 𝐹 1 and 𝐹 2 operators that the global positive impact (respectively the global negative impact) according to the optimistic attitude (respectively to the pessimistic attitude) is equal or greater than the global positive impact (respectively global negative impact) according to the optimistic attitude (respectively to the pessimistic attitude):

(𝛿 𝑥,𝜔 +,2 ≥ 𝛿 𝑥,𝜔 +,1 ) and (𝛿 𝑥,𝜔 -,1 ≥ 𝛿 𝑥,𝜔 -,2 ) (9)

In section 5 we illustrate the proposed qualitative evaluation in the case of the design of an autonomous mobile robot.

QUALITATIVE EVALUATION: AN ILLUSTRATION

Context

To illustrate our qualitative evaluation approach we consider the design of a small autonomous mobile robot. It concerns an educational robotic challenge RobAFIS 2013 (Robafis,2013), promoted by AFIS, the French association of Systems Engineering. The purpose of RobAFIS is to better understand and develop the use of systems engineering best practices, as recommended and formalized by AFIS.

In RobAFIS 2013, the autonomous robot had to be built using some provided and imposed materials. It was limited to a 0,3m square cube and it had to achieve the following mission as quickly as possible: to grasp and transport some various coloured spheres between several stock devices spread over a plan playground. Some dark lines were drawn on the ground to guide the robot between stock devices. The robot was broken down into four sub-systems: a gripper device, sensors equipment, a rolling base, a control device. The physical alternatives (the possible robot configurations) depend on the design option of the other sub-systems than the control device that was imposed to the competitors. The solution principles are resumed in Table 1. There are three principles for designing the gripper device, three for the rolling base and two for the sensors equipment. Thus, there are 18 alternatives to compare, corresponding to the 3x3x2 admissible configurations. R3: rolling base with tracks Three independent criteria are retained: the sparsity of used components (Cr1), the robot speed capability (Cr2), the number of successful missions (Cr3). Let consider for this case study the ordinal scale 𝑄𝑆 = {𝑁𝑢𝑙𝑙(0), 𝐿𝑜𝑤 (𝐿), 𝑀𝑒𝑑𝑖𝑢𝑚 (𝑀), 𝐻𝑖𝑔ℎ (𝐻)}.

Table 2 presents, for each of the three criteria, the positive impact (denoted by "+") or negative impact (denoted by "-") of each input variable value with their confidence degrees as provided by the experts.

Table 2. Qualitative evaluation

To illustrate the use of operators introduced in sections 3.2 and 3.3, Let's consider the alternative 𝑥 =[G2,R3,S1],(grey cases in Table 2) and a pessimistic attitude (at=1), then according to (5),( 6) the global impact of alternatives can be computed considering a preferences model or not. So according to (5) the global impact of alternative x is: 𝛿 𝑥 1 = (𝛿 𝑥 +,1 , 𝛿 𝑥 -,1 ) = (min(0, , 𝑀, 𝐿) , max(𝑀, 0,0)) = (0, 𝑀)

Case of same importance of criteria

In the same way for an other alternative x' =[G1,R2,S1], we can compute the global impact: 𝛿 𝑥′ 1 = (𝛿 𝑥′ +,1 , 𝛿 𝑥′ -,1 ) = (0, 𝑀) = (min(𝑀, 0, 𝑀) , max(0, 𝑀, 0))

The eighteen alternatives are then ordered according to the decreasing values of global positive impacts and for ex aequo according to the increasing values of global negative impacts.

The most promising alternatives have then been found to be [G2,R1,S1],[G2,R2,S1],[G2,R2,S2] with for each of them a global impact of (L,0). According to an optimistic attitude, the most promising alternatives have been found to be the same as above but with a global impact (M,0). There is no promising robot with a degree -M on any criterion (cases where experts think with a good confidence that the robot will deteriorate at least one criterion). Robots with R1 and S2 options are not found to be promising configurations too.

Case of preferences model on criteria

Let's consider now that the relative importance of criteria Cr1, Cr2,Cr3 are taken into account and let 𝜔 = (𝐻, 𝐿, 𝐻) the weights vector (i.e. the time spent to achieve the mission is not so important). According to expression (6), the new global score for x'= [G1,R2,S1] becomes: 𝛿 𝑥 ′ ,𝜔 1 = (𝛿 𝑥 ′ ,𝜔 +,1 , 𝛿 𝑥 ′ ,𝜔 -,1 ) = (𝑀, 𝐿) = (min(max(0, 𝑀) , max(𝑀, 0) , max(0, 𝑀)), max(min(𝐻, 0) , min(𝐿, 𝑀) , min(𝐻, 0))) and it appears now that x' is a more promising alternative than x. Indeed: 𝛿 𝑥,𝜔 1 = (𝛿 𝑥,𝜔 +,1 , 𝛿 𝑥,𝜔 -,1 ) = (𝐿, 𝑀) = (min(max(𝑀, 0) , max(𝑀, 𝑀) , max(0, 𝐿)), max(min(𝐻, 𝑀) , min(𝐿, 0) , min(𝐻, 0)))

This can be explained by the fact that the poor score of G1 on Cr2 is now considered not so important in the estimation of the global impact of alternative x ' .

So it has been shown how veto and dictator effects of poor qualitative scores on less important criteria can be smoothed by using a weights vector describing the preferences model of the decision-makers'/designers'.

CONCLUSION

The role of the evaluation process in Systems Engineering is to support designers each time engineering choices need to be made or justified. However it is often difficult to anticipate, during the conceptual design phase, given that the data are quite uncertain or imprecise. The proposed evaluation method aims at achieving this objective by adopting an original qualitative analysis in order to detect the most promising alternative design solutions while taking into account the risk aversion of decision-makers (pessimistic attitude verso optimistic attitude) but also a model of their preferences on criteria. Our proposals have been illustrated through the design of an autonomous mobile robot. This work will be completed in proposing intermediate attitude (between pure optimistic and pessimistic ones) and in introducing levels on degradation and amelioration opinions. The formalism should be also extended to deal with the fusion of experts' opinions in the case where the members of a design group do not necessarily agree. More generally, our research consists in incorporating most of the recent progresses in multicriteria decision aid in the aim of increasing the level of confidence designers can show in the choices they make during the Systems Engineering processes.

Fig

  Fig. 1: Evaluation process

+

  and 𝛿 𝑖𝑗 -take their values in an ordinal scale 𝑄𝑆 containing a zero, top and intermediate values. It can be for example a scale of four values 𝑄𝑆 = {𝑧𝑒𝑟𝑜 = 𝑛𝑢𝑙𝑙, 𝑙𝑜𝑤, 𝑚𝑒𝑑𝑖𝑢𝑚, 𝑡𝑜𝑝 = 𝑠𝑡𝑟𝑜𝑛𝑔}.

  For the 𝐶𝑟 1 criterion "pieces nbr" we have min

  Let denote for 𝜔 ∈ 𝑄𝑆, ¬𝜔 the value corresponding to the reversed scale of 𝜔 (i.e., ¬𝑛𝑢𝑙𝑙 = 𝑠𝑡𝑟𝑜𝑛𝑔, ¬low=medium, …). It is the counter part of 1 -𝜔 on the [0,1] interval scale. Let's denote 𝜔 𝑖 ∈ 𝑄𝑆 the weight of the 𝑖 𝑡ℎ criterion and 𝜔 = (𝜔 1 , 𝜔 2 , … , 𝜔 𝑛 ) the weights vector.

	The importance of criteria are taken into account in changing
	the aggregation operator of expression (5) 𝛿 𝑥 𝑎𝑡 into 𝛿 𝑥,𝜔 𝑎𝑡 :
	𝛿 𝑥,𝜔 𝑎𝑡 = (𝛿 𝑥,𝜔 +,𝑎𝑡 , 𝛿 𝑥,𝜔 -,𝑎𝑡 ) = (min 𝑖	(𝛿 𝑖,𝑥,𝜔 𝑖 +,𝑎𝑡 ), max 𝑖	(𝛿 𝑖,𝑥,𝜔 𝑖 -,𝑎𝑡 )) (6)

Table 1 . Solution principles

 1

	Gripper device Rolling	Sensors equipment
		base
	G1: Fork	R1: Four	S1: two colour sensors
	(taking the	wheels	for following the dark
	sphere from	rolling base	lines, one light sensor for
	below)		recognizing the sphere
			colour
	G2: Lateral	R2: Two	S2: one colour sensor for
	gripper	wheel drive	following the dark line,
	(pinching	and one free	one colour sensor for
	laterally the	wheel	recognizing the sphere
	sphere)	rolling base	colour
	G3: Grapnel
	(taking over
	the sphere)