Mehdi El Amine
email: mehdi.elamine.1@gmail.com

Jérôme Pailhes

Nicolas Perry

Integration of concept maturity in decision-making for engineering design: An application to a solar collector development

Keywords: Decision-making, Set-based Conceptual design, Risk, Performance, Trade-off

The presence of immature concepts makes decision-making difficult in upstream phases of a development project. A company's enthusiasm to improve product performance by exploring novel concepts is hampered by a low maturity that results in an inherent risk of cost overruns and schedule delays. Besides this, decisions taken during conceptual design phases have a critical impact on product life cycle cost. In this context, a useful practice is to develop two or more concepts in parallel and delay decision-making until sufficient knowledge is generated. Even so, a preliminary screening of concepts is usually needed because of the substantial resources and time required for the development activities. In this paper, a concept screening approach is proposed at the conceptual design phase on the basis of two metrics. The first assesses risk related to the non-completion of design criteria while respecting available time (consideration of deadline) and company resources. The second metric assesses the expected performance of novel concepts compared with the existing products. It uses preference functions and generalized ordered weighted averaging (GOWA) operators. Fuzzy logic tools are used in the two metrics to capture and propagate imprecision in the description of design and external environment. In order to prove its value, the proposed approach was applied to the development of a solar collector in an industrial environment.

Introduction

In a broad sense, the ability of decision-makers to choose the best design concept in engineering design is strongly conditioned by two factors: (i) having a clear definition of design requirements, and (ii) being able to evaluate or predict the performance of the proposed design concepts. In reality, it is very difficult to predict accurately the performance of immature concepts and design requirements are usually vague and imprecise in the early phases of a development project (Giapoulis 2000). In addition, life-cycle cost can be influenced up to 70% by decisions taken during early phases of the development process [START_REF] Zimmer | Global aircraft predesign based on constraint propagation and interval analysis[END_REF]. These discrepancies constitute the product development paradox (Ullman 2003;[START_REF] Reich | Preventing Breakthroughs from Breakdowns[END_REF].

In order to address this paradox, many design departments adopt the set-based conceptual design (SBCD) approach [START_REF] Morgan | The Toyota Product Development System: Integrating People, Process and Technology[END_REF][START_REF] Ward | Lean Product and Process Development[END_REF][START_REF] Sobek | Toyota's Principles of Set-based Concurrent Engineering[END_REF]) which consists in developing several concepts in parallel until enough knowledge is acquired to facilitate the selection of the most appropriate one (Fig. 1). This can significantly minimize risk and reduce the cost of regret in later development stages. The practice of SBCD is still not formalized and is still human-dependent [START_REF] Tomiyama | Design methodologies: Industrial and educational applications[END_REF]. Although developing many concepts is effective in minimizing risk, the usefulness of such a practice is limited by the substantial cost and time required for the development activities. Indeed, the step of generating concepts (patent analysis, creativity procedures, existent concepts from previous projects, etc.) usually results in several concepts. Eliminating operations are thus required to reduce the initial set of proposed concepts such that it can be supported by company resources and allocated budgets. Such an elimination process must take into account the global strategy of the company behind the product development. It must also consider the fuzziness in the description of the initial concepts. Many authors recognize the importance of assessing and integrating concept maturity in decision-making [START_REF] Pahl | Engineering design; A systematic approach[END_REF][START_REF] Ullman | The Ideal Engineering Decision Support System[END_REF][START_REF] O'brien | Design maturity assessment for concurrent engineering co-ordination[END_REF]. Within our context, concept maturity is defined according to [START_REF] Dunn | Simultaneous engineering side-by-side to success[END_REF]: "Maturity makes possible a "linear" design process, each stage flows sequentially -no backtracking. Non-maturity results in an "orbital process", a circular sequence of repetition, the total time taken being several times that for the linear process". According to this definition, a concept becomes mature when it requires little or no rectifications. At this stage, the concept reaches its maximum potential. Indeed, once a concept is chosen for development by the company, designers start the embodiment design stage which consists in choosing principal design parameters such as rough arrangements and selections of structural dimensions, materials, components and technologies [START_REF] Pahl | Engineering design; A systematic approach[END_REF]. For an immature concept, these choices are made with insufficient knowledge and, as a result, the requirements are usually not met. The consequences of concept maturity are thus closely dependent on the nature and level of requirements. Some authors define product maturity as the association of knowledge and performance [START_REF] Beth | CMMI, Second Edition -Guidelines for Process Integration and Product Improvement[END_REF]Drement et al. 2013). As shown in Fig. 1, the progress of concept development is accompanied by a narrowing in the set of possible combinations of design parameters (due to finite element analysis, mechanical tests, etc.) until the most suitable combination of design parameters is found. The clarification of design requirements also accelerates the narrowing process, as illustrated in Fig. 1.

Fig. 1 Schematization of global development process in the context of set-based conceptual design

Apart from the risk of non-completion of design requirements, cost overruns and schedule delays are very frequent when developing immature concepts [START_REF] Katz | The Relationship of Technology and Design Maturity to DoD Weapon System Cost Change and Schedule Change During Engineering and Manufacturing Development[END_REF]. This aspect should be taken into account when selecting concepts. It is highly recommended that companies have a concept with a high degree of maturity [START_REF] Katz | The Relationship of Technology and Design Maturity to DoD Weapon System Cost Change and Schedule Change During Engineering and Manufacturing Development[END_REF][START_REF] O'brien | Design maturity assessment for concurrent engineering co-ordination[END_REF]. On the other hand, the desire to be distinguished from the competition and to keep pace with a rapidly changing market, means that decision-makers are encouraged to adopt an innovation strategy by developing novel concepts. Market competitiveness through innovation is the common strategy of developed companies. In addition, the company's long-term sustainability may depend on its innovation strategy [START_REF] Sharif | Innovation strategy, firm survival and relocation: The case of Hong Kong-owned manufacturing in Guangdong Province, China[END_REF]. In summary, the choice of the set of concepts to develop in the context of SBCD is driven by both the need for performance improvement and failure risk limitation.

Decision-making process in engineering design

Generally speaking, the decision-making procedure in engineering design is composed of four steps [START_REF] Collignan | Arc-elasticity and hierarchical exploration of the neighborhood of solutions in mechanical design[END_REF]): definition of candidate solutions, performance assessment, interpretation of performances and synthesis. The misguidance in the final decision is due to the association of uncertainties occurring in each of these four steps (Fig. 2). Understanding these steps and possible uncertainties related to each of them is important to correctly address the issue of aid to decision-making.

(i)

Step 0: definition of candidate solutions. In our context, candidate solutions refer to design concepts.

When the maturity of a concept is low, development engineers deal with an imprecise description of the design (imprecise definition of design parameters) since they have insufficient knowledge to define a precise design description. However, design engineers may have preferences over a range of values a design parameter can take. This can be based on their experience and the collected knowledge on the concept. Fuzziness in the description of design parameters makes it difficult to compare concepts between them.

(ii)

Step 1: performance assessment. Performances of each concept are assessed (mass, maximum stress, carbon footprint, etc.) and are required to verify the satisfaction of initial criteria. These performances are assessed using different kinds of behavior models (finite element models, expert judgments, mechanical tests, etc.) that may be inaccurate and thus induce errors in assessed performances. In addition, the assessment of performances requires information on the external environment and this information also can be uncertain. (iii) Step 2: interpretation of performances. Depending on their objectives and requirements, decisionmakers express their preferences towards each performance assessment. Designers' preferences and requirements may also contain imprecision, or be unclear and vague at the early design phases, which can lead to additional difficulty in design-making in the field of engineering design. In the rest of this paper, acceptance threshold refers to a limit value of performance (set by the decision-maker) that must be satisfied by the concept. Otherwise it can be validated. (iv) Step 3: Synthesis. Interpretations of the different performances are combined by the decision-maker to obtain a global judgement on the concept. This global judgement is necessary to compare many concepts.

In this step, imprecision can affect the relative importance of criteria and the degree of compensation between them.

Fig. 2 Decision-making process in engineering design

Uncertainty in input parameters

In our study, the input data for which uncertainty is considered are those related to design concepts (design parameters) and external environmental (environmental parameters). The uncertainty in these input parameters can be aleatory or epistemic by nature. Imprecision, or epistemic uncertainty, is related to a lack of knowledge and can be reduced by further efforts. Aleatory uncertainty is linked to the intrinsic stochastic nature of physical phenomena and cannot be reduced. Probability distributions are suitable to model aleatory uncertainty due to its probabilistic nature. Since a significant proportion of the uncertainty comes from imprecision during conceptual design phases [START_REF] Antonsson | Imprecision in engineering design[END_REF], only imprecision is considered in our study. Imprecision can be quantitatively modeled using discrete/continuous intervals [START_REF] Kreinovich | From Interval Methods of Representing Uncertainty to a General Description of Uncertainty[END_REF][START_REF] Muhanna | Interval Methods for Reliable Computing[END_REF] or fuzzy sets [START_REF] Wood | Computations with imprecise parameters in engineering design: background and theory[END_REF][START_REF] Wang | Interactive evolutionary solution synthesis in fuzzy setbased preliminary engineering design[END_REF]. These types of modeling are widely used in literature. However, the question is whether it is sufficient to characterize the state of uncertainty in input parameters. According to Krishnan (1996), uncertainty in input parameters in a development project can also be characterized by its rate of variation. He introduced the notion of variation as the probability that information reaches its final value. Since it is difficult to quantify this characteristic objectively, he proposed to use the qualitative scale in Table 1. These levels range from "0.2" for a variation that is stable, to "0.8" that is very unstable, meaning that the probability that the object approaches its final value is very low. Since design engineers have limited time and resources to reduce uncertainty in input parameters, integrating the notion of variation can be of interest.

Table 1 Variation levels of an activity (adapted from Krishnan (1996))

Need for a new approach

The main objective of the new approach is to help decision-makers reduce the set of concepts to develop by eliminating the least interesting ones. In general, when considering decision-making under risk, an important aspect to take into account is the sensitivity of the decision-maker to payoffs and risk. In the context of product development, choosing to develop a new concept is always subject to the risk of not fulfilling the minimum design requirements and/or exceeding the project deadline. Paradoxically, the development of a novel concept is usually motivated by the wish to obtain a high performance compared with existing concepts. Generally, decision-makers seek to achieve a satisfactory trade-off between the risk and the expected performance in their decisions. In the context of set-based conceptual design, where a set of concepts is selected for further development, the nature of this trade-off may be different from one concept to another. This can be explained with a simple example. Assuming that the decision-makers have to choose two concepts for future development, they generally choose at least one with high maturity and a high chance of being feasible at project deadline. Once this choice is made, the decision-maker's acceptance of risk will be much higher for his second choice. Decision-makers must be prevented from developing only risky concepts or developing only concepts expecting a low gain in performance. In order to be able to use this logic of concept selection, where different types of trade-off are used between risk and expected performance, the proposed approach must provide decision-makers with an evaluation of both risk and expected gain relative to the candidate design concept.

As mentioned in Section 1, cost overruns and schedule delays are very frequent when developing immature concepts [START_REF] Katz | The Relationship of Technology and Design Maturity to DoD Weapon System Cost Change and Schedule Change During Engineering and Manufacturing Development[END_REF]. Another important aspect that must be taken into account in the proposed approach is the time available for development and the allocated budget. Design engineers have limited time and resources for increasing the degree of product maturity.

Various concept selection methods exist in the literature. Choosing the most appropriate method is itself a critical decision as different methods can yield different outcomes for the same problem (Guitouni and Martel 1998;Okudan and Tauhid 2008). The choice of the most appropriate method is very dependent on the decisionmaking situation (Roy and Slowinski 2013). The decision-maker must be aware of the underlying principle of the chosen method and verify that it is in accordance with his own decision-making situation. The state of the art proposed in this section aims to verify that the concept selection methods proposed in the literature are appropriate for our decision-making process.

Multi-Attribute Utility Theory (MAUT) is one of the most recognized and accepted methods in decisionmaking. It was initially proposed by [START_REF] Neumann | Theory of games and economic behaviour[END_REF] for economic decision-making. Pahl and Beitz (1984) were the first to incorporate MAUT into a systematic engineering design process. They suggest six steps for applying MAUT in engineering design: (i) design criteria are first established (such as cost, resistance, durability, etc.), (ii) these criteria are rated on a scale of 0-100 such that the sum of the weights for all criteria is equal to 100, (iii) evaluation parameters are then defined to enable quantification of concept comparisons with one another (This corresponds to step 1 in Fig. 2), (iv) scores are assigned as a utility to each of the evaluation parameters (This corresponds to step 2 in Fig. 2), (v) utility theory is used to calculate the concept value, which is the product of the criterion parameter score and the criterion weighting (This corresponds to step 3 in Fig. 2), (vi) concepts are ranked based on overall utility and the concept with the highest utility score is selected. According to [START_REF] Scott | Aggregation functions for engineering design trade-offs[END_REF], the use of a weighted sum in engineering design is not appropriate since the aspect of decision-making cannot be compensated, contrary to decision in the field of economics, where all aspects of decision can always be translated into associated cost. He proposed a set of mathematical axioms that must be respected by an aggregation operator when dealing with an engineering design decision. Pahl and Beitz's (1984) utility theory does not directly accommodate the analysis of imprecisely characterized alternatives. By extending uncertainty representations to model imprecision explicitly, it is possible to apply the principles of utility theory to such problems. This can be done by using discrete/continuous intervals or fuzzy logic. The expected utility for a given concept is thus not known precisely and becomes an interval or fuzzy representation (as shown in Fig. 3). However, this can lead to situations of indeterminacy [START_REF] Jr | Multi-attribute utility analysis in set-based conceptual design[END_REF], meaning that the decision-makers have no rational basis for choosing one concept over another. This situation is illustrated in Fig. 3.

Fig. 3

Two examples of comparing decision alternatives with imprecise expected utilities. Case (a) is indeterminate because of overlapping expected utility bounds. In case (b), there is a clear preference for concept A Based on a fuzzy representation of expected concept utilities (as shown in Fig. 3), [START_REF] Roubens | Some Properties of Choice Functions Based on Valued Binary Relations[END_REF] proposed the metric of weak dominance than can be interpreted, when applied to a given concept A, as the truth value of the statement "Concept is better than , for every concept in the set of proposed concepts". By producing a single value of concept evaluation, it eliminates the problem of indeterminacy and thus facilitates the selection of a concept. However, it does not respond to the recommendations in section 5 since it does not allow risk and expected utility to be assessed separately. Let us consider, for example, the case (a) in Fig. 3. Based on the criterion of weak dominance, concept A and concept B yield the same score when using the metric of weak dominance. However, concept poses a risk of not being feasible since there is a possibility that the utility of will be lower than the acceptable threshold ( ). Consequently, concept could be preferred to concept for a decision-maker who is sensitive to risk. Li et al. (2004) propose a joint probability decision-making technique (JPDM) to select the most appropriate concept. The JPDM technique is based on the multivariate probability theory. The core of JPDM technique is the construction of a joint probability distribution that combines the univariate distributions for each of the criteria. The probability distributions reflect the uncertainty associated with design definition that is due to incomplete knowledge. The generated joint probability distribution serves in conjunction with a criterion value range required by decision-makers (Fig. 3) as a universally applicable objective function. The objective function, called Probability of Success (PoS), constitutes a meaningful metric that prevents the development of a concept with low chance of success. However, the Probability of Success assessment is not enough to compare concepts and thus can lead to unjustified discrimination of promising concepts. As advocated in Section 2, the expected performance must also be assessed. Considering the example in Fig. 3, concepts A and B yield the same ( = = .5). However, concept B is of greater interest than A since it has a higher expected performance.

Fig. 4 Limitations of the JPDM technique

In the framework of utility theory, the use of lottery assessment to establish utility values for each of the proposed concepts is another technique that takes uncertainty into account in the evaluation of concepts (Krantz et al. 1971). Considering that concept A is evaluated against a certain criterion . The performance produced is denoted . This could be, for example, maximum strength, manufacturing cost, carbon footprint, etc. In order to take the uncertainty into account in this assessment, the utility value is established by asking a lottery question: "On a scale from 0 to 1, what is your numerical belief you are indifferent between 1) having or

2) having the best performance with certainty and having the worst performance with certainty 1 -?". By using a lottery assessment, designers' attitude toward risks can be taken into account. Using an aggregation operator, a single value of concept evaluation can be obtained at the end by integration utility value associated with each criterion. However, the principal limit of lottery assessment in our context is that the risk and expected gain are not assessed separately (which is advocated in Section 4). The two aspects are expressed in a single value .

Technology Readiness Level (TRL), a 9-level ordinal scale, related to a 9-level semantic scale [START_REF] Mankins | Technology readiness levels (White paper)[END_REF], is a widely used metric in the aeronautics, aerospace and weapons industry to assess maturity of developed concepts. It allows decision-makers to ensure that a concept has enough maturity to be introduced into the system. [START_REF] Conrow | Estimating technology readiness level coefficients[END_REF] argues that mathematical operations performed on TRL metrics can lead to systematic errors because of their ordinal nature. He proposes a calibration of the TRL scale based on an analytic hierarchical process (AHP) that results in estimated cardinal values being assigned to each TRL scale level, allowing for mathematical analysis using TRLs as an independent variable. It has therefore become possible to aggregate it with other metrics. The state of progress of an innovative technology (that is defined by TRL) can be linked to eliminating imprecision in design definition and a concept with a higher TRL score has a better chance of being feasible. However, TRL remains a coarse metric for the maturity of the concept. It does not allow for a fine assessment on the state of imprecision in design definition. Engler III et al. (2007) proposed to use TRL as a first filter to eliminate concepts when the opportunity space is large, but they recommend using a finer filter later in the development process.

Proposed approach

In this section, an approach is proposed to support concept selection in the context of set-based conceptual design under imprecision. As explained in Section 4, we aim at providing an approach that can evaluate both risk of failure and expected performance relative to each candidate design concept, while taking into account the constraint on deadline and available resources. The proposed approach is thus based on two metrics to support concept selection. The first is the risk criticality index ( ) which evaluates the risk related to the noncompletion of design criteria while respecting available time (consideration of deadline) and company resources. The second metric is the overall desirability index (ODI) which evaluates the expected performance of each concept. The assessment procedure of these two metrics is described in this section.

Each candidate concept (denoted X) is characterized by its own set of design parameters { 1 , … , }.

is the number of design parameters for concept . The external environment is described by a set of environmental parameters { 1 , … , }. is the number of design parameters. Contrary to design parameters, environmental parameters are common to all candidate concepts and they are considered in our approach since they are involved in the behaviors of proposed concepts. The first step of the proposed approach is to characterize the imprecision in the design and environmental parameters. Two types of characterization are required in the construction of the two metrics ( and ). First, fuzzy logic is used to capture the actual state of imprecision in these parameters. Fig. 5 illustrates fuzzy representation for continuous and discrete parameters. The problem with an immature concept is that the design engineers have not sufficient knowledge to define the most appropriate value of a design parameter with respect to design criteria. Nevertheless, they usually have a preference towards a certain range of values. Using fuzzy numbers makes it possible to capture the design engineers' preference on a continuous or discrete set of values on the basis of their knowledge, experience and know-how. When the maturity of the concept is low, these intervals are generally broad. They become increasingly finer as further development activities are performed. Since input parameters (design and environmental parameters) are used in different engineering equations, a propagation technique is needed. In our approach, the Vertex method proposed by [START_REF] Dong | Fuzzy weighted averages and implementation of the extension principle[END_REF] is used as a propagation technique. The second aspect of characterization concerns the variation rate of imprecision in input parameters, already discussed in Section 3. Concerning the determination of variation, which is also very dependent on designer knowledge/experience, we use the scale proposed by Krishnan (1996) and shown in Table 1. The variation index of a given parameter is denoted in the case of environmental parameters and in the case of design parameters of a concept . It is important to note that the probability of information reaching its final value is determined in relation to the project deadline and available resources. Now that the imprecision in input data is characterized, the question that needs to be considered is: what is the risk related to the use of this data? In our context, we are interested in the risk of failing to fulfil design criteria while respecting the available time and company resources. Mapping must therefore be established between the characterization of imprecision in input data and the risk of failing to fulfil design criteria. The procedure used for this purpose is inspired from FMEA (Failure Mode Effects Analysis). This analysis is widely used in industry and thus it has the advantage of being easily understandable and acceptable by decision-makers and design engineers. For a given concept , let denote the assessment of risk related to the noncompletion of criterion i. Three main components that characterize this risk are identified and evaluated: the likelihood of occurrence ( ), the likelihood of detection ( ) and, the severity of not fulfilling the criterion ( ). is obtained by multiplying these three factors (Eq. 1). Finally, the global risk criticality index ( ) related to a concept is obtained by summing all (Eq. 2). is the number of design criteria. In the rest of this section, the three factors , and are presented and the procedure for calculating them is explained. = .

.

(1)

= ∑ =1

(2)

The likelihood of occurrence

Given the imprecision in the description of design and environmental parameters, the performances of a concept cannot be determined precisely. It is therefore difficult for the design engineer to verify precisely if a criterion can be satisfied by a concept. Instead, it may be more appropriate to consider the likelihood of non-satisfaction of a design criterion. Let denote the performance value obtained after the evaluation of a concept X. This value can be, for example, mass, maximum stress, displacement, speed, etc. It must be determined in order to verify that criterion is respected. Let denote the function for assessing performance for criterion as a function of design parameters ⃗ = 1 , … , and development parameters { 1 , … , } such that:

= 1 , … , , 1 , … , (3)
Function can be any procedure to assess performance such as finite element analysis, equations, heuristic methods, expert judgments, experiments on physical prototypes, etc. Using the Vertex method, imprecision in input parameters is propagated through function to performance . Fig. 6 illustrates this procedure. The result is a fuzzy representation of . The likelihood of occurrence corresponding to criterion is calculated as the ratio between the overlap surface (intersection between possible values of and admissible values) and the possible values (Fig. 6). This overlap region is represented graphically in Fig. 5. The α-cut method is used to calculate this value and is explained in Eq. 4, Eq. 5 and Eq. 6. is the acceptance threshold for criterion i.

Fig. 6 Using the Vertex method to map design space to performance space (case of one design parameter and one environment parameter)

ASR = - 1 N ∑ C i N i=1 (4)
With:

C i = { if S i < F g i S i -F g i F d i -F g i if F g i < S i F d i if S i F g i (5)
And:

{ F g i = a + i N × b -a F d i = c + i N × d -c (6)

The likelihood of detection

In addition to the likelihood of occurrence, it is also important for decision-makers to be able to eliminate imprecision in the assessment of performances . Detectability is therefore used in our approach to assess the likelihood that a performance measure will be assessed precisely while taking into account the available time (consideration of deadline) and company resources. To determine the likelihood of detection of criterion i for a given concept X, two details are needed on input parameters (design or environmental parameter): their variation indexes vi, determined using Table 1, and their impact on the output performance. Let denote the impact of input parameter j on criterion i. For the design engineer who uses fuzzy logic in the description of input parameters and output performances, impact measures enable him to determine some information on the coupling between the inputs and outputs of design calculations. These measures can also be used to determine which input parameters the design engineer can change while producing little or no effect on the performance, and which parameters will alter output performance the most. Those parameters with little influence may be fixed to the most-desired value by the design engineer, resulting in a simplification of the design problem. Based on the Vertex method, [START_REF] Wood | Computations with imprecise parameters in engineering design: background and theory[END_REF] proposed a  -level measure to calculate the relative impact of input parameters on output performance. The -level measure is used in this paper to calculate . The likelihood of detection D i X of criterion i for a given concept X is calculated using Eq.

1. = ∑ =1 + ∑ =1 with { ∑ + =1 = (7)
Since are normalized, final detectability will lie between 0.2 and 1 (same range of variation as variation indexes) and its significance can be determined using the semantic scale in Table . 1.

The severity

estimates the severity of the scenario in which criterion i is not met. Severity is estimated using the qualitative scale in Table 2. These levels range from "0.2" for the lowest consequences, to "1.0" for the highest consequences.

Table 2 Suggested rating for the evaluation of severity

Notion of reference concept

In many industries, the design department needs to fix a maturity threshold for a product with a critical role in the global system [START_REF] Katz | The Relationship of Technology and Design Maturity to DoD Weapon System Cost Change and Schedule Change During Engineering and Manufacturing Development[END_REF]Department of Defense 2011;[START_REF] Gao | Giapoulis A (200) Einsatz von Methoden zur Produktentwicklung in der industriellen Praxis[END_REF]. This lets the different stakeholders know whether the concepts of the program have acceptable levels of risk. The Department of Defense, for example, fixes a threshold of TRL=6 (Department of Defense 2011). In our approach, this threshold is defined by two conditions: (i) the RCI of this concept must be lower than 0.05, (ii) a prototype of the concept must be tested in a relevant environment (equivalent to TRL=6) as proof of the validity of behavior models. In this paper, a concept verifying these conditions is called the 'Reference concept'.

Second metric: overall desirability index ( )

In general, the acceptability of risk by decision-makers is very much dependent on the expected gain. In our context, this means answering the following question: if a decision-maker decides to invest resources and time in development activities to address unmet criteria, what could the global performance of the concept could be. The question is particularly pertinent when there is a pre-existing concept that has already been developed and validated. Overall desirability index ( ) aims at exploring the possible performance that can be achieved by candidate design concepts. The calculation procedure for this metric is illustrated in Fig. 8 and includes two different steps, interpretation and aggregation, which are explained below. They correspond to step 2 and step 3 respectively of the decision-making process illustrated in Fig. 2. Step 1 has already been performed (fuzzy assessment of performances ) within the calculation of the first metric.

Interpretation step

As explained in Section 2, this step consists in mapping between performances achieved by the candidate design concept (

) and the decision-maker's preferences. This mapping is performed using the desirability function proposed by Harrington (1965). It associates to each performance variable a desirability index ( between 0 and 1 that expresses the level of satisfaction related to the corresponding design criterion. Usually, design constraints are translated into strict constraints using inequality equations. The desirability functions translate design constraints into soft-constraints, which allow a more realistic definition of the design problem.

Harrington's desirability functions were chosen because of their ease of parametrization and their appropriateness to the three common types of design objectives, which are "bounding low values of is better", "bounding high values of is better", and "closer to a particular target value of is better". The interpretation procedure is illustrated in Fig. 7. The Vertex algorithm is used to propagate imprecision in to desirability index through Harrington's desirability functions ℎ .

Fig. 7

Interpretation procedure for a given design criterion

Aggregation step

Aggregation refers to the process of synthesizing a collection of numerical values into a unique representative or meaningful value in order to come to a conclusion or a decision. In our case, it enables us to obtain a single value that quantifies the general level of satisfaction achieved by a concept in regard to the design criteria. For a given concept , this value is denoted and is obtained by the aggregating of each design criterion .

The aggregation operator used is the Generalized Ordered Weighted Averaging (GOWA) proposed by [START_REF] Scott | Aggregation functions for engineering design trade-offs[END_REF], given in Eq. 8. This operator was chosen since the aggregation strategy is completely adjustable by the choice of: (i) weights expressing the relative importance of criteria, and (ii) parameter expressing the degree of compensation between design criteria. An important property of the GOWA operator is that any solution in Pareto front can be reached by adjusting and [START_REF] Scott | Aggregation functions for engineering design trade-offs[END_REF]). However, one must be aware that the interpretation of the given weights is not absolute but depends on the degree of compensation (expressed by ). In other words, the method used to determine the weights must take into account the value of . In order to respect this constraint, here we use the indifference points method [START_REF] Scott | Using Indifference Points in Engineering Decisions[END_REF], based on the definition of indifference design concepts that elicit the same performance, to determine simultaneously a single value for parameter and for weights .

ODI = √∑ (w DI i X ) n s (8)
Using an aggregation operator enables us to simplify the decision problem since the multi-criteria problem is converted to a mono-criterion problem. In addition, the definition of desirability functions and aggregation operator constitutes a means to capture and capitalize knowledge associated with decision-makers' preferences. This is particularly important in collaborative design since it allows the different stakeholders to have a formal way of communicating about the decision-makers' preference. However, in order to avoid misleading decisionmaking, the parameters feeding the construction of desirability functions and the aggregation operator must be assessed as accurately as possible.

Fig. 8 Evaluation model for Overall Desirability Indicator

In many development projects, a company already has a mature concept resulting from a previous project. In this case, the purpose of developing a novel concept is to achieve a better performance. It is thus appropriate that the aggregation operator considers the reference concept in determining the performance of the novel concepts. Equation ( 8) is thus modified to use the distance from the reference concept. The expression becomes:

ODI = √∑ (w t i X |DI i X -DI | ) n s (9)
is the desirability index of criterion j for the reference concept. is a parameter introduced to take into account the direction of the improvement. t i X =1 when the improvement is positive and t i X =-1 when the improvement is negative.

Presentation of the industrial case

Compared to photovoltaic panels, the major advantage of Concentrating Solar Power (CSP) is the possibility of storing thermal energy, so that electricity can then be produced independently of the presence of the sun and the time of day or night. Studies have found that CSP could account for up to 25% of the world's energy needs by 2050 [START_REF] Jha | Concentrated Solar Power Could Generate 'quarter of World's Energy[END_REF]. It thus represents an important contribution to global energy transition.

The company in which our approach is applied is specialized in designing and manufacturing of complex mechanical subassemblies by the forming and assembly of tubes and sheet metal for the aeronautics and solar energy industries. In our industrial case, we use CSP with linear Fresnel collectors. The main function of the collectors within CSP plant is to concentrate and redirect sunlight onto absorber tubes to heat up the working fluid. A schematization of the collector structure is given in Fig. 9. It is composed of reflective plates (reflective glass or polished aluminum) and a supporting structure whose function is to give the reflective plates support and maintain their shape. A mounting device is placed between the reflecting surface plates and the supporting structure to ensure the connection between them. In our study, only the supporting structure is studied. The cost of raw materials for manufacturing solar collectors represents 50% of investment cost in the case of a CSP plant with linear Fresnel collectors (Kumara et al. 2009). It is thus important to reduce the supporting structure mass (criterion 3). In addition, in order to maintain a good optical performance, and thus a good thermal performance, elastic deformation of the supporting structure must remain as low as possible (criterion 1). Finally, this structure must resist a high wind pressure to be usable for a multitude of implementation sites (criterion 2).

Fig. 9 Schematization of solar collector structure

Compared to other CSP technologies, CSP with linear Fresnel collectors is relatively recent and immature. It was noted in our industrial case that the limited background on this technology leads to difficulties when evaluating and selecting design alternatives during the development process. Another difficulty in our industrial case is that the choice of implementation sites has not yet been made and there is a variety of possible sites in which conditions may vary significantly. The environment parameters, which must be taken into account when developing the supporting structure, cannot be known precisely.

Three different concepts were proposed initially for the design of the supporting structure: truss structure (concept A), tube structure (concept B) and sandwich structure (concept C). Fig. 10 illustrates these concepts and indicates the corresponding design parameters (that need to be defined). Before applying the new approach, an early study was performed in the design department on the development of these three concepts. Given the important issues involved in the development of the supporting structure and the low level of knowledge on the proposed concepts (especially the truss and tube structures), the design department is adopting a set-based conceptual design approach. Therefore, as illustrated in Fig. 11, the three concepts are developed in parallel until enough knowledge is available to facilitate the selection of the most appropriate one. As mentioned in the same figure, five design engineers were assigned to the development of the three concepts. These engineers have a meaningful experience in the development of mechanical products (between 3 and 6 years of experience). The distribution of working time on the three concepts is also mentioned in Fig. 11 (expressed in percentage). This development team is directly headed by a development project manager which also manages the development of the entire solar field.

Fig. 10 Candidate design concepts and the corresponding design parameters

As illustrated in Fig. 11, the development process is composed of several milestones, and at each milestone a set of activities is performed in order to generate knowledge on developed concepts and mature them (definition of design parameters values in adequacy with design criteria). These activities generally consist in numerical calculations, prototyping and mechanical tests. On completion of each milestone, an assessment grid is established to synthesize the performance results achieved by each design concept (considering the latest evolutions in design parameters values). It contains performance assessment . The results are then discussed by the development team (design engineers and development project manager). Next, the development team decides which concepts can be eliminated (go/kill decisions) and defines guidelines for actions to be undertaken for the next milestone. This process is referred as "gate review" and is performed after each milestone (as illustrated in Fig. 11).

Fig. 11 Schematization of industrial development process

At each gate review (end of a milestone), the final go/kill decision for each of the remaining concepts is made by the development project manager, who represents the decision-maker in our industrial case. During the gate reviews, in which both decision-maker and design engineers take part, the assessment grid (as illustrated in Fig. 11) filled by the design engineers is placed at the center of discussions. The assessment outcomes marked in red (Fig. 11) are of particular interest during gate review since they indicate that the corresponding criteria are not met by the current design version (current choices regarding design parameters) and thus rectifications are needed in the design parameters in the next milestone. Using their experience and the expertise acquired during the milestone, each design engineer communicates with the decision-maker on the following: what are the possible rectifications in the current design version which will overcome the unmet criteria (marked in red in Fig. 11)? What is the expected effectiveness of these rectifications? How can the effectiveness of these rectifications be verified? Based on this information, the decision-maker can gradually construct his assessment of the risk of failure related to each concept (the non-respect of acceptance thresholds). The 'strengths' of a concept (assessment outcomes marked in green) are also considered and help the decision-maker to decide whether or not it is worth accepting the risk and continuing the development of the concept. The current industrial practice encounters important limitations, making it difficult to reduce the set of initial concepts and thus causing cost overruns and schedule delays. The main limitations are summarized below.

Limitation 1: At this preliminary stage of the development process, many behavior models (denoted by in our study) are developed and used to evaluate the performance of a concept (denoted by for a concept X).

These models are considered as explicit knowledge as they are expressed and transmitted in formal language. However, they often fail to provide an accurate evaluation of performances and do not encapsulate all the knowledge acquired and/or generated by design engineers. Consequently, decision-making cannot be entirely based on these behavior models. This explicit knowledge must be complemented by tacit knowledge [START_REF] Jasimuddin | The paradox of using tacit and explicit knowledge[END_REF], which is related to the designers' experience and know-how. As explained in the previous paragraph, this tacit knowledge is used at gate reviews but in an implicit manner and without following a rational framework. The effectiveness of the procedure is thus very much dependent on the participating individuals' ability to communicate and transfer the available knowledge assets. This "informal" procedure remains limited because, on the one hand, the decision is made jointly and requires the participation of many stakeholders to provide an overall judgment in order to select the most appropriate concept(s), and on the other hand, this kind of knowledge is difficult to communicate [START_REF] Polanyi | The Study of Man[END_REF]. This usually leads to an overestimate or underestimate of the chance of success and the possible performance achievement for the different concepts.

There is therefore a need to retrieve and integrate tacit knowledge more effectively to help decision-making. Limitation 2: Another limitation arises from the difficulty decision-makers have in analyzing simultaneously a multitude of information supplied by the various design engineers in the project and then synthesizing it in order to establish an overall judgment. As we can see, the final decision in the current practice is made without using a multi-criteria decision aid method. There is a perceived need to synthesize the information provided to the decision-maker on which the decision is based.

Limitation 3: The fourth perceived difficulty is that designers are vague about decision-makers' preferences regardless of the different decision-making criteria. For example, it is not clear if there is significant difference between angular deformations of 0.015° and 0.018° in terms of decision-maker satisfaction. There is a need to pool all preference information available to the different designers and, more broadly, to the stakeholder participating in the development. This will give designers not only a better evaluation of the solution they propose but also a better ability to suggest appropriate rectifications to the product.

Application of the proposed approach

The approach proposed in this paper is experimented in gate review I (Fig. 11) using the knowledge generated in milestone I about the three concepts. First, data related to the decisional model (interpretation and aggregation) are assessed by the development project manager (decision-maker) and two other stakeholders in the development of CSP plant (not directly involved in supporting structure development). These data are: severity factors, parameters of desirability functions, criteria weights and aggregation parameters. The decisional model is thus set up (shown in Fig. 8). Then, the characterization of input parameters imprecision is carried out by the five design engineers since they have the most knowledge on concepts. Therefore, design and environmental parameters are characterized by design engineers in terms of state of imprecision (using preference functions) and variation indexes (using the scale proposed by Krishnan (1996)). For a continuous parameter (eg. a thickness), preference function is established using a several point estimates determined by design engineers. Once the characterization of input data is completed, the values of O i X , D i X and RCI i X (with X=A, B or C) are determined for each concept using the approach described in section 6.1. The results are shown in Table 3. Finally, based on the decisional model and behavior models developed during the first milestone, ODI X are calculated. The final outcomes (RCI X and ODI X ) using the proposed approach are presented in Fig. 12. ODI X are represented by fuzzy numbers to take into account imprecision in design and environment parameters (the grey level indicates the membership degree). Eq. 8 was used as an aggregation operator since none of the proposed concepts respects conditions to be considered as a reference concept. In the following paragraphs, we explain how concepts are selected based on the results of Fig. 12.

Table 3 Detailed results of RCI X calculation

As mentioned before, it is suitable in a product development project to have both mature/secure concepts (high ability to respect acceptance thresholds) and innovative concepts to produce a potentially better performance (eg. breakthrough innovations). However, the product developed in our case is part of a global system (CSP plant) and the failure of supporting structure development or major schedule delays will therefore have severe consequences. Obtaining a sufficiently mature and secure concept is thus a more pressing issue in our industrial case. For this reason, the sandwich structure was chosen first by the decision maker because of its low (the lowest value). The development of the sandwich structure began before the other concepts in the context of a previous project and many validation tests were carried out, including a test in an experimental plant (TRL = 7).

The priority of design department for the next milestones is to continue to investigate this concept until all the design parameters are precisely defined (to obtain the final product). However, although sandwich structure is the safest concept, it does not exhibit a high performance (as is low).

Truss structure is a novel concept for the company. Given the high importance of the material cost criterion, the low mass of the truss structure makes it very attractive. This interest is manifested by the high range of (reaching up to 0.86). However, the angular deformation and wind resistance criteria could not be satisfied for the first versions that were tested. Rectifications must be introduced to the design parameters to satisfy all criteria.

of the truss structure yields the highest value (meaning that the development is risky). The acceptance of a high for the case of the truss structure is justified by both the high range of and also by the existence of a very mature/secure concept (sandwich concept). The truss structure was also selected for development.

Compared to the development of the sandwich and truss structures, the development of the tube structure is much less interesting. This is because the range of achieved is close to that of the sandwich concept while the is relatively high (high risk). Therefore, based on the results obtained from the two proposed metrics, it was decided that the development of the tube structure should be interrupted.

Fig. 12

Results of ODI X and RCI X metrics calculation for each concept Despite some difficulties (mentioned in the paragraphs below) during the implementation of the new approach within the industrial organization, empirical observations reveal that the main limitations of the current industrial practice (mentioned before) were addressed. In order to address Limitation 1 on integrating tacit knowledge into the decision-making process, preference functions were used in the new approach to include the judgments of design engineers in regard to the relevance of design parameters values. These functions are established subjectively, based on engineers' experience and know-how. The most predominant observation made by design engineers Such a formalization procedure (use of preference functions) contributes to a more effective integration of engineers' is the effectiveness of the new method to integer tacit knowledge into the decisionmaking process. Indeed, it was noted that this knowledge becomes easier to communicate between design engineers, who are closely involved in the development activities, and development project manager, who is the main actor involved in decision-making. In addition, the new approach allows combining this tacit knowledge with explicit knowledge (represented by behavior models ). During the implementation of the new approach, it was also found that the meaning of preference functions was easy to understand by design engineers. However, certain difficulties were encountered when generating point estimates, mainly because of the high number of points and the divergences of some views (when many engineers are involved). However, it was noted that the use of preference functions was appreciated by design engineers as it is a mean to express and integer more effectively their personal reasoning. In addition, the meaning of these functions was easy to understand by design engineers.

Concerning Limitation 2, the use of aggregation operators was of a great help for decision-maker since they are able to synthesize a multitude of information in a single value. They found it easier to deal with single value instead of multitude and heterogeneous information (performance variable ) when selecting concepts. In addition, the parameters feeding the aggregation operators are a means of formalizing the tradeoff procedure that must be used consistently with the decision-maker's preferences. Instead of dealing with a multitude of information, decision-maker relies only on two metrics (RCI X and ODI X ) when selecting concepts.

Another key advantage of the new approach addresses Limitation 3 mentioned above. For design engineers participating in the experiment, Concerning Limitation 3, the definition of desirability functions and aggregation operator are a means of capturing, organizing and capitalizing on information associated with decision-maker's preferences. This iswas particularly appreciated in our collaborative design project since it allowed the different stakeholders to have a formal way of communicating about the decision-maker's preferences. It was recognized by the different design engineers participating in the experiment that a better understanding of decision-maker's preferences leads not only to better decisions but also a better capacity to propose relevant solutions throughout development process. However, in order to avoid misleading decision-making, the parameters that feed the construction of desirability functions and aggregation operators must be assessed as accurately as possible. In our industrial case, some difficulties were encountered when parametrizing desirability functions and aggregation operators because preferences are not only detained by development project manager but by other stakeholders in the project (not directly involved in supporting structure development). In addition, some discrepancies were noted between the different stakeholders questioned about their preferences.

Conclusion and future developments

The choice of concept in product development processes has a critical impact on overall life-cycle cost. Supporting this choice by adequate approaches is thus of critical importance. Design processes are constrained by the need for performance improvement and failure risk limitation of the product. In this paper, each of these two considerations is represented by an adequate metric (called RCI and ODI) to perform elimination operations for set-based conceptual design in the light of imprecise input data. In order to take into account the constraint on available time (consideration of deadline) and company resources, the calculation of RCI considers not only the state of imprecision of input parameters, but also the variation rate of imprecision. The performance metric (ODI) proposed in this study uses preference functions and an aggregation operator to model decision-maker preferences. It can be adapted to take into account a pre-existing concept that is sufficiently mature. It is left to the decision-maker to choose a trade-off between the two metrics, according to his risk aversion as well as the culture and regulatory framework of the company.

The application of these metrics to our industrial case allows the decision-maker to have a more transparent and straightforward assessment of failure risk and expected range of performance. The experiment also reveals the ability of the new approach to capture and capitalize knowledge associated with decision-makers' preferences, which is particularly important in collaborative design. However, the procedure for calculating RCI and ODI involves complex mathematical calculations which may be beyond the scope of a design engineer's conventional tasks. This difficulty can be partially overcome by developing software tools that can be integrated into product digital mock-up (DMU). In spite of the difficulties encountered during the implementation of the new approach and the time spent on the determination of the different parameters, all stakeholders involved in the development project admit the ability of the new approach to anticipate the less relevant concepts (tube concept in our case) which means that efforts and resources can be focused on the most promising concepts, thus contributing to a better chance of success and a better quality product.

In a real development project, the developed component is usually connected with other components within the global system. In this case, decisions related to the studied component (concept selection and design parameters selection) impact not only the performance of this component but also the performance of other components in the system. In future developments, interactions between components will be taken into account and we will consider the impact of local decisions (decisions related to individual components) on the achievement of global objectives (eg. the energy efficiency of the CSP plant).

Fig. 5

 5 Fig. 5 Use of fuzzy logic to represent preference on input data. (a) case of continuous set. (b) case of discrete set 6.1. First metric: Risk Criticality Index ( )

Table 3 Detailed results of RCIX calculation Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org)

Performance assessment of concepts

Step 1: Performance assessment

Step 2: Interpretation of performances

Step 3: Synthesis

Proposed concepts

Imprecisions in design parameters

Uncertainty in environmental information + errors due to low accuracy of behavior models Imprecision in designer's preferences and requirements Inaccuracies in relative priorities of criteria and degree of compensation between them

Decisionmaking process

Sources of imprecision

Choosing the best concept

Combine different interpretations to compare solutions

Expression of designer's preferences on performance assessment Very unstable: The probability that an object approaches its final value is close to zero.

0.6 Unstable: The probability that an object approaches its final value is low.

0.4

Moderately unstable: The probability that an object approaches its final value is moderately high.

Stable:

The probability that the object approaches its final value is high.