Ludovic-Alexandre Vidal
email: ludovicalexandre.vidal@centraliens.net

Franck Marle

Jean-Claude Bocquet

Using a Delphi process and the Analytic Hierarchy Process (AHP) to evaluate the complexity of projects

Keywords: Project, Complexity, Delphi process, Evaluation, Multi-criteria, Analytic Hierarchy Process (AHP)

Project complexity is ever growing and needs to be understood, analysed and measured better to assist modern project management. The overall ambition of this paper is therefore to define a measure of project complexity in order to assist decision-making, notably when analysing several projects in a portfolio, or when studying different areas of a project. A synthesised literature review on existing complexity measures is firstly proposed in order to highlight their limitations.

Then, we identify the multiple aspects of project complexity thanks to the construction and refinement of a project complexity framework thanks to an international Delphi study. We then propose a multi-criteria approach to project complexity evaluation, underlining the benefits of such an approach. In order to solve properly this multi-criteria problem, we first conduct a critical state of the art on multi-criteria methodologies. We then argue for the use of the Analytic Hierarchy Process. In the end, this tool permits to define a relative project complexity measure, which can notably assist decision-making. Complexity scales and subscales are defined in order to highlight the most complex alternatives and their principal sources of complexity within the set of criteria and sub-criteria which exist in the hierarchical structure. Finally, a case study within a start-up firm in the entertainment industry (musicals production) is performed. Conclusions, limitations and perspectives of research are given in the end.

Introduction

A project is a temporary and unique endeavour undertaken to deliver a result. This result is always a change in the organization, whatever it is in its processes, performance, products or services. This transformation consists then in a gap between a start and a final state. Time and resources are consumed to produce results, which may be deliverables and/or performance improvement and/or resource improvement (skills, knowledge). Each project is unique because there is always at least one of the following parameters that changes: targets, resources and environment. As projects became more and more present into organizations, and as they had bigger and bigger amounts at stake, it became impossible to let them live without specific and rigorous methodology. As a consequence, project management was created as a formalized and structured methodology. It is usually admitted than modern project management appeared during World War II and was initially dedicated to big military and construction projects.

For all practical purposes, lots of studies have been done, based on statistical calculations or surveys. Limits and lacks have been detected in research as well as in industry about the project predictability, since usual parameters (time, cost and quality) are clearly not sufficient to describe properly the complete situation at a given time. As a whole, the conclusion of these studies is that current methods have shown their limits, since they cannot face anymore the stakes of ever growing project complexity. For instance, as noted during discussions with consulting practitioners, in the case of oil industry, it is clear that engineering projects today are larger, involve more sophisticated technology and are organised with a higher number of contractors and partners compared to 40 years ago. As a whole, project complexity results in damages or failures for the projects. In other words, project ever growing complexity is an ever growing source of project risks.

Identifying existing project complexity sources and levels of project complexity has thus become a crucial issue in order to assist modern project management. After a review of the literature on project complexity in part 2, this paper aims at proposing an innovative measure of project complexity which incorporates the multiple aspects of this project characteristic. In order to do so, a methodology is exposed in part 3 and followed in the next paragraphs. Conclusions and research perspectives about this issue are finally given in part 9.

Measuring project complexity: a literature review

But before choosing a suitable project complexity measure, one must be able to define a list of criteria that can be used to assess if it is good or not. [START_REF] Latva-Koivisto ; ) Latva-Koivisto | Finding a complexity measure for business process models -Research report[END_REF] proposes a first list in a research report which includes parameters such as reliability, ease of implementation and intuitiveness for instance. As far as this research work is concerned, a literature review on existing project complexity measures (or complexity measures which could be adapted to project management) was performed.

A focus was made on project complexity in terms of project systemic complexity, and not of algorithmic complexity when solving some issues about project management, such as the sequencing and scheduling problem (Akileswaran and al., 1983). The works of Edmonds (Edmonds, 1999), [START_REF] Latva-Koivisto ; ) Latva-Koivisto | Finding a complexity measure for business process models -Research report[END_REF]) and Nassar and Hegab [START_REF] Nassar | Developing a complexity measure for schedules[END_REF] were crucial sources to generate this list of about fourty complexity measure indicators. For instance, in his Ph.D. thesis, Edmonds (Edmonds, 1999) identified formulations and measures of complexity, working on a large scope of fields and applications. As for him, [START_REF] Latva-Koivisto ; ) Latva-Koivisto | Finding a complexity measure for business process models -Research report[END_REF] reviewed complexity measures to assess the structural complexity of business processes. He argued that the complexity of business processes could be assessed through the conversion of process charts (composed of activities, dependencies, information flows, material flows and control flows) to graphs, giving the example of the resourceconstrained project scheduling problem. If interested, one should directly refer to these three references for more information on complexity measures and formulations. Then, the obtained list of possible complexity measures was refined thanks to the previously introduced criteria. Four specific complexity measures are given here since among the most appropriate ones for a use in project management.



The Coefficient of Network Complexity (CNC) defined by Kaimann [START_REF] Kaimann | [END_REF] applies to both PERT (Program Evaluation and Review Technique) and precedence networks. In the case of PERT networks, the CNC is equal to the quotient of activities squared divided by events. The CNC, thanks to an intuitive definition is a good complexity measure to catch the structural complexity of systems that are modelled thanks to graphs. However they take redundant arcs into account.



The cyclomatic number defined by Temperley (Temperley, 1981) gives the number of independent cycles in a graph. The equation calculation of the cyclomatic number is equation (1). S is the cyclomatic number, A is the number of arcs, N is the number of nodes. S = A -N + 1

(1)



The traditional static entropic measurement of complexity by the Shannon information (Shannon, 1951) is based on the probability of receiving a message, as shown by equation

(2) where p(ni) is the probability of receiving a message ni. The Shannon information is also a complexity measure since information and disorder are strongly related. Sha = -Σ log2 (p(ni))

 Arguing that complexity measures such as CNC are imperfect since they take redundant -4 -arcs into account and therefore show that the system is more complex than it actually is, Nassar and Hegab [START_REF] Nassar | Developing a complexity measure for schedules[END_REF] define a measure for project schedules. This measure gives the degree of interrelationships between the activities in a schedule. This complexity measure is the following equation ( 3) for an Activity On Node project network.

Cn = 100 × (Log(a/(n-1))/Log[(n 2 -1)/4(n-1)])% if n is odd

Cn = 100 × (Log(a/(n-1))/Log[n 2 /4(n-1)])% if n is even (3)

Limits of existing project complexity measures

Existing measures have shown their limits for several reasons. First, some limits have been highlighted about the reliability of such measures. Indeed, some counterexamples were found: some graphs and networks were sharing the same CNC but were very different considering their easiness to be managed. One of the reasons for this lack of reliability is that these measures mainly refer to a single aspect of (project) complexity, notably in terms of interdependencies.

Second, these measures are often non intuitive for the final users and thus give results which are difficult to communicate on. These mathematical formulations do not permit a reference to real project complexity factors: both the identification of important complexity sources and possible actions for complexity handling/reduction are not facilitated. Moreover, such measures are sometimes difficult to calculate for non-skilled users, which make it all the more complex to perform and analyse them. For instance, in the case of the Shannon number, both difficulties are encountered for all practical purposes.

Finally, these measures mainly refer to a model of the project system. Indeed, measures such as the CNC, the cyclomatic number or the one proposed by Nassar and Hegab refer in essence to an existing network or graph. Such graphs are specific models of the project system, which restrict the view and understanding of project complexity. For instance, a project can be modelled thanks to different WBS (Work Breakdown Structure), PERT networks or Gantt charts, depending on the detail level, willingness of the project manager, etc… Applying such measures to these kinds of elementary models of the project systems cannot properly account for a measure of project complexity since they are in essence relative to the model. As a consequence, in order to overcome the limits of existing measures, this paper aims at defining an index which is as far as possible:

 Reliable, meaning the user can be confident with the measure.

 Intuitive and user-friendly, meaning it should be easily computed and implemented, and that users must understand why it assesses project complexity.

 Independent of the project models, so that the measure is an evaluation of project complexity and not an evaluation of the complexity of a given project model.

 Able to highlight project complexity sources when building up the measure, so that the user can analyse more properly project complexity and thus make his decisions with a better vision of the problem.

Research methodology

In order to do so, we propose to carry out the following methodology:

 Performing a broad state of the art to understand the multiple aspects of project complexity through the construction of a project complexity framework (Part 4).

 Identifying the most suitable multi-criteria decision methodology for project complexity evaluation and understanding its principles (Part 5)

 Refining the project complexity framework thanks to an international Delphi study in order to permit the construction of a hierarchical structure (Part 6).

 Proposing a project complexity index and studying its sensitivity (Part 7).

 Testing the whole on a case study (Part 8).

4. The multiple aspects of project complexity

Systems thinking

In order to understand and handle project complexity, this paper claims for the use of systems thinking to facilitate the management of projects. Basically, our systems thinking-based approach is notably based on or at least consistent with the works of (Boulding, 1956), [START_REF] Bertalanffy | [END_REF], (Simon, 1981), [START_REF] Le Moigne | La théorie du système général[END_REF], (Penalva, 1997), [START_REF] Heylighen | Complexity and Philosophy -Complexity[END_REF], [START_REF] Vidal | Modelling project complexity -International Conference on Engineering Design[END_REF], or [START_REF] Vidal | Modelling project complexity -International Conference on Engineering Design[END_REF]). This paper considers the following definition of a system.

Definition -adapted from [START_REF] Vidal | Modelling project complexity -International Conference on Engineering Design[END_REF]:

A system can be defined as an object, which, in a given environment, aims at reaching some objectives (teleological aspect) by doing an activity (functional aspect) while its internal structure (ontological aspect) evolves through time (genetic aspect) without losing its own identity.

According to this definition, a project can be undoubtedly considered as a system. Indeed, it possesses the four aspects listed above. A project exists within a specific environment and aims at reaching objectives given this context (teleological aspect). A project has to accomplish a set of activities using some methods and methodologies (functional aspect). A project has an internal structure composed of human resources, material resources, deliverables, tasks, information systems, etc… (ontological aspect). Finally, a project evolves through time, via resource consumption, product delivery, members" changes and gain of experience, without losing its own identity (genetic aspect). In the systems thinking vision, the project system evolution is to be considered with the assumption that future is under perpetual construction (Prigogine, 1996), which excludes the use of analytical tools. In order to provide innovative practical tools for complex project management, this paper is to claim for the use of a shared epistemology which permits to define, when necessary, methods and tools based on analytical decompositions, but which use or are at least compatible with the systems thinking-based vision of projects as four aspect entities.

Figure 1. A systems thinking-based approach to describe projects

Project complexity

Research works on the concept of complexity have been conducted for years. There are historically two main scientific approaches of complexity [START_REF] Schlindwein | Human knowing and perceived complexity: implications for systems practice -Emergence: complexity & organisation[END_REF]. The first one, usually known as the field of descriptive complexity, considers complexity as an intrinsic property of a system, a vision which incited researchers to try to quantify or measure it. An example of this vision is the work of Baccarini (Baccarini, 1996). He considers project complexity through the concepts of technological complexity and organisational complexity. He regards them as the core components of project complexity which he tries to describe exhaustively. The other one, usually known as the field of perceived complexity, considers complexity as subjective, since the complexity of a system is improperly understood through the perception of an observer. Both approaches can apply to project complexity and project management complexity.

The difficulty is that there is actually a lack of consensus on what project complexity really is. As [START_REF] Sinha | A complexity index for the design process -International Conference on Engineering Design[END_REF] underline it, "there is no single concept of complexity that can adequately capture our intuitive notion of what the word ought to mean". Complexity can be understood in different ways, not only in different fields but has also different connotations within the same field [START_REF] Morel & Ramanujam ; ) Morel | Through the looking glass of complexity : the dynamics of organizations as adaptive and evolving systems[END_REF]. However, some other works (Edmonds, 1999), (Marle, 2002), [START_REF] Vidal | Project complexity understanding and modeling to assist project management -PMI Research Conference[END_REF] led us to propose the following definition for project complexity (this definition is to kept in mind throughout the paper).

Definition: Project complexity is the property of a project which makes it difficult to understand, foresee and keep under control its overall behaviour, even when given reasonably complete information about the project system. Its drivers are factors related to project size, project variety, project interdependence and project context.

The reader may note that every aspect of systems thinking is part of the overall behaviour of the project system. That means that, according to this definition, project complexity is the property which makes it difficult to understand, foresee and keep under control any of these aspects.

As a whole, when reviewing the literature, a very large number of possible project complexity factors can be identified thanks to an approach based on the four principal aspects of systems thinking [START_REF] Vidal | Project complexity understanding and modeling to assist project management -PMI Research Conference[END_REF].

However, the point is that speaking in terms of teleological, genetic, functional and ontological aspects of project complex is not the easiest manner to communicate about complexity in real projects. Neither is it an adequate decomposition to see the concrete phenomena behind these notions. We thus claim for a gathering of these factors into four more intuitive groups (Figure 2).

These groups, which are closely related to the four aspects of systems thinking, are all necessary but non-sufficient conditions for project complexity.

The first group gathers the factors that are relative to the size of the project system. The second one gathers those that are relative to the variety of the project system. These two first groups globally correspond to the ontological aspect of the project system. The third one gathers those that are relative to the interdependencies and interrelations within the project system, which corresponds to some extent to the functional pole of the project system. Finally, the fourth one deals with the context-dependence of project complexity, which mainly corresponds to the teleological and genetic poles of the project system.

Figure 2. Drivers of project complexity

The gathering of the identified project complexity factors into these four distinct groups makes more meaning both for direct industrial use (as these denominations make more sense for fieldwork) and for academic establishment (since these denominations have widely been used in research articles for instance).

Project size

Project size is to be defined as a whole as the sizes of elementary objects which exist within the project system. These sizes are likely to be assessed thanks to appropriate quantitative measure (for instance time scale, cardinal scale, etc…). This aspect of project size (which is somewhat close to the ontological aspect of project complexity in terms of number of identified elements) then appears to be a necessary condition for project complexity which makes sense. Indeed, recent papers notably state that any organisational system should be over a minimum critical size to be considered as a complex system (Corbett & al.,2002).

Project variety

Project variety is to be defined as a whole as the diversity of elementary objects which exist within the project system. This aspect of project variety (which is somewhat close to the ontological aspect of project complexity in terms of diversity of identified elements) indeed appears to be a group which makes sense. Indeed, as mentioned by Sherwood and Anderson (Sherwood Jones & Anderson, 2005), "diversity relates closely to the number of emergent properties". Moreover, as underlined by Corbett and al. (Corbett & al.,2002), "the one thing that comes through loud and clear is that complexity is tied up with variety, be it in the world of biology, physics or manufacturing".

Project interdependence

Project interdependence is to be defined as the existence of relationships between elementary objects within the project system. This aspect of project interdependence is somewhat close to the functional aspect of project complexity in terms of interactions between elements to execute the project. It then appears to be another category which makes sense. As underlined by several project management researchers and practitioners, interdependencies and all the notions related with them such as interactions, interrelationships or interfaces, are likely to be the greatest drivers of project complexity.

Besides, Rodrigues and Bowers (Rodrigues & Bowers, 1996) explain that "experience suggests that the interrelationships between the project"s components are more complex than is suggested by the traditional work breakdown structure of project network", suggesting that traditional project management tools cannot be sufficient to catch the reality of project interdependence. This seems all the more problematic since "there is a complete interdependence between the components of the complexity: each element will depend and influence on the others" (Calinescu & al., 1998).

Project context-dependence

Project context is defined here as what refers to the environment within which a project is undertaken. This aspect of project context-dependence (which is somewhat close to the teleological and genetic aspects of project complexity) indeed appears to be another category which makes sense. First, Chu and al. (Chu and al., 2003) underline that contextuality is an essential feature of complexity, considering it as a common denominator of any complex system.

The context-dependence of project complexity is also stressed by Koivu and al. (Koivu and al., 2004) who notably insist on the fact that "the context and practices that apply to one project are not directly transferable to other projects with different institutional and cultural configurations, which have to be taken into account in the processes of project management and leadership".

First version of a project complexity framework

As a whole, this literature review and proposed classification permits to build a project complexity framework (Figure 3). It aims at being a reference for any project manager to identify and characterize some aspects of the project complexity. One can then understand more efficiently the stakes of project complexity management. Even though we had the ambition to be quite exhaustive, exhaustiveness remains a utopia when dealing with complexity. Some others project complexity factors are likely to be added to this framework.

Knowing that, we insist on the fact that this project complexity framework is a descriptive vision of complexity: for all practical purposes, the perceived complexity of the real system throughout this framework is finally the one that is being managed. Moreover, one should keep in mind that this framework is a form of consensus on project complexity and that complexity cannot in essence be managed and handled through a generic consensus. This framework should as a consequence be considered as a basis to understand better complex projects and particularly identify the principal sources of complexity within a given project.

5. Using the Analytic Hierarchy Process (AHP) to assess project complexity

Multi-criteria decision methodologies

In general, decision-making is the study of identifying and choosing alternatives based on the values and preferences of the decision-maker. Making a decision implies that some alternatives are to be considered, and that one chooses the alternative(s) that possibly best fits with the goals, objectives, desires and values of the problem.

A deeper look at the literature reveals that the problem of selecting the appropriate method appears itself to be a multi-criteria problem: this multiple criteria in the case of project complexity evaluation are synthesized in Figure 4. We do not use any of existing methods to solve it, but we follow the forthcoming methodology to select the method used in our study: we first conducted a literature review in order to define the requirements of the method that could enable us to develop a good tool for project complexity evaluation. This literature review is notably based on the works of Gershon (Gershon, 1981), Deason (Deason, 1984) and Tecle (Tecle, 1988). It permits to build up Figure 5 and to identify the Analytic Hierarchy Process (AHP) as probably the most suitable methodology for this issue.

Requirements Description of requirements

Multi-Criteria

The method should be capable to compare alternatives regarding multiple criteria of different nature.

Handle qualitative criteria

The method should be able to handle qualitative criteria in addition to quantitative ones.

Prioritise criteria

The method should enable the user to prioritise the criteria, since they have different influences.

Evaluate a discrete set of alternatives

The method should be able to search for the best alternative among an initial discrete set of known alternatives.

Rank alternatives

The method should not only give the most complex project within the portfolio but also prioritise the projects functions of their complexity level Rank alternatives according to a cardinal scale

The method should rank alternatives according to a cardinal scale. This cardinal scale is to be used afterwards to build up the relative complexity measure we propose.

Reliable

The method should give a reliable result to be eligible for decision-making support.

Computable

The method is to be computable to enable quick calculations on computers Show great user-friendliness

The method should be user-friendly: this notably includes both the facts that no special/demanding skills should be necessary to perform the process and that results should be understood and handled easily.

Give autonomy

Users (mainly project managers) should be autonomous and should possibly suggest or do modifications.

Evolving

Modifications need to be easily implemented.

Adapted to project environment

The method should be adapted to project environment decision processes (Le Cardinal, 2000), (Jankovic, 2006) and characteristics (constraints, skills, information systems, need for reactivity, …)

Figure 4. Requirements for a multicriteria method to be used for project complexity evaluation

Identified multi-criteria methods are elementary methods, multi-criteria optimisation ones, outranking ones, or single synthesis criterion approach ones. They were assessed during the literature review regarding the identified requirements. The five first criteria are evaluated on a Boolean scale which permits to say if these criteria are respected or not by the method. These criteria are required for the goal which is pursued in this study. As a consequence, when a method is assessed 0 on one of these criteria, further evaluation of the method is not performed and the method is screened out. Then, the set of the six last criteria are evaluated on a 5-level Likert scale. Evaluations of the five first criteria of this set are mainly performed thanks to a state of the art which is notably based on (Gershon, 1981), (Deason, 1984), (Tecle, 1988) and (Al-Shemmeri and al., 1997). Evaluation of the sixth criteria (adapted to project environment) is notably based on a survey of scientific databases (ISI, etc…) to assess the use of these methods in the project management literature. A distance is then defined as a comparison in absolute value with the ideal method which would be noted 5 on every criterion of this set. The two best scores are obtained for the Analytic Hierarchy Process (AHP) and the PROMETHEE methodologies. Regarding the issue of project complexity evaluation, preference is notably finally given to the AHP, because of its numerous applications in the project management context which were found in the literature (Al-Harbi, 2001). For instance, in (Ahmad and Laplante, 2006), the AHP is used to select the most appropriate software project management tool. The authors argue that "the AHP provides a flexible, systematic, and repeatable evaluation procedure that can easily be understood by the decision-maker in selecting the appropriate software project management tool".

Other applications particularly consider the issue of project evaluation or selection in the case of project outsourcing (Bea and Lloveras, 2007) or project portfolio management (Liang, 2003).

Indeed, in (Alhazmi and McCaffer, 2000), the development of the project procurement system selection model thanks to the AHP is presented. Another example can be found in the works of Simpson and Cochran for construction project prioritisation [START_REF] Simpson | An Analytical approach to programming construction projects[END_REF], who however argue that "the AHP methodology is applicable to problem sizes from order 2 to about order 15 [and that] if a large number of projects is to be considered, some means is required to reduce the number of candidate alternatives". Last but not least, another example in the field of project management is the work of Gourc (Gourc, 2006) who uses the AHP for project risk analysis and assessment, under the assumption that project risks have multiple aspects. Finally, the reader should note that the AHP also has many applications in different contexts which all underline the user-friendliness and intuitiveness of the methodology [START_REF] Lin | Improving AHP for construction with an adaptive AHP approach -Automation in Construction[END_REF], (Gerdsri and Kocaoglu, 2007), (Chiu and Chen, 2008). This makes it both a very generic and project context-friendly method.

The Analytic Hierarchy Process (AHP)

The Analytic Hierarchy Process (AHP) was developed by Thomas Saaty [START_REF] Saaty | A scaling method for priorities in hierarchical structures[END_REF], (Saaty, 1980), (Saaty, 1990). It is a multi-criteria decision-making method which permits the relative assessment and prioritization of alternatives.

Characteristics

As underlined by Saaty (Saaty, 1981) and exposed in Fumey (Fumey, 2001), the AHP permits to integrate both quantitative and qualitative aspects of decision -making, which makes it an efficient and effective method under complex contexts, as synthesized in Figure 6.

The AHP is based on the use of pairwise comparisons, which lead to the elaboration of a ratio scale. The AHP uses a hierarchical model for the decision problem, consisting of an overall goal, of a group of alternatives, and of a group of criteria which link the alternatives to the goal.

Pairwise comparisons are classically carried out by asking how more valuable an alternative A is to criterion c than another alternative B. Saaty scales can transform these judgements into numerical values. As shown hereunder on Figure 7, pairwise comparisons constitute in the end square matrices, the values of which are between 1/9 and 9. The diagonal elements of the matrix are equal to 1 while the other ones verify two conditions:

 The i-j th element is equal to the comparison between element i and element j regarding the considered criterion.

 For i different from j, the i-j th element is equal to the inverse of the j-i th element Here, the multiple aspects are not few in number and implementing it could quickly be impossible in fieldwork. That is why we called for a refinement of this framework thanks to an international Delphi study which was carried out on academic and industrial practitioners.

                          1 ... /

The Delphi process

The Delphi methodology [START_REF] Linstone | Project Selection by Using Fuzzy AHP andTOPSIS Technique[END_REF], which was originally developed in the 1950"s, is a systematic and interactive method which relies on a panel of independent experts. It is a very flexible tool which permits to reach a consensus, through the collection of experts" opinions on a given issue during successive stages of questionnaire and feedback. Direct confrontation of the experts, whose anonymity is kept at every stage of the study, is avoided [START_REF] Okoli | The Delphi method as a research tool: an example, design considerations and application[END_REF]. As mentioned in [START_REF] Skulmoski | The Delphi Method for Graduate Research -The Journal of Information Technology Education[END_REF], " the Delphi method is well suited as a research instrument when there is incomplete knowledge about a problem or phenomenon". It has proven over the years to be a very popular tool for framework building, forecasting, issues prioritizing, decision-making, etc… The Delphi method has been used for several studies in the field of industrial engineering and project management, which encouraged us in our research work. For instance, Schmidt and al.

used the Delphi method in order to build up a list of common risk factors in software projects [START_REF] Schmidt | Identifying software project risks: an international Delphi study[END_REF]. Our research methodology is based on a two-round Delphi process (see Figure 8). The Delphi survey was conducted thanks to blind copy electronic mail sending to international academic and industrial experts in project management in order to save time and expenses for both the surveyor and the experts. The questions were formulated thanks to a 5-level Likert scale, in order to express the importance of the contribution of a given factor to project complexity. The minimum of contribution is level 1 and the maximal contribution is level 5. The Likert scale leaves the possibility to answer "do not know" and "do not want to answer"). Furthermore, participants could leave commentaries and questions at any moment on any point of the Delphi questionnaire in order to generate some discussions about it or to suggest other potential project complexity factors. At each round, a little more than three weeks were left to the panelists to answer the survey.

The statistical analyses of round 1 and round 2 correspond to the results expressed in the discussions paragraph. It must be noted that the results of round 1 and round 2 are the same since the synthesis and proposition which was done after round 1 satisfied all experts, reaching global consensus at this stage. No change was as a consequence done between the answers of round 1 and round 2. Only some commentaries and suggestions appeared during round 2.

Panel selection and survey scales definition

The Delphi survey group size appears to be very different in the literature. However, it is often recommended to have a group between 9 and 18 participants in order to draw some relevant conclusions and avoid at the same time difficulty to reach consensus among experts. We argue, such as in [START_REF] Okoli | The Delphi method as a research tool: an example, design considerations and application[END_REF], that an experts categorization should be made properly before undertaking the Delphi survey in order to build up the most representative panel. As for them, Skulmoski and al. require different aspects for the participants to be selected in the Delphi survey panel [START_REF] Skulmoski | The Delphi Method for Graduate Research -The Journal of Information Technology Education[END_REF]):

 Sufficient knowledge and experience about the survey issues,  Capacity, willingness and time to participate,  Good communication skills.

-17 -Our prospective panel was constituted of 38 experts, 19 of them being industrial practitioners and 19 being academics, and at the same time 19 being men, 19 being women. Of those 38 solicited experts, 18 actually participated to the study from the beginning to the end, 10 of them being academics and 8 being industrials, and at the same time 10 of them being women, and 8 being men. Academics were notably identified thanks to their publications regarding project complexity in the Web of Science and specialized conferences or journals (International Journal of Project Management, PMI Research Conference, etc…). Industrial practitioners were identified thanks to the browsing of some professional social networks (Linkedln), the identification of some project managers of large firms websites, and the identification of project managers whose education was followed in some high standard schools, universities and institutions. We thus consider that the overall results are going to be relevant since the interrogation of 18 experts permits to trust them.

In order to do comparisons and generate discussions during the next section, we also study separately men, women, academics and industrials. Even though the suggested minimum quota of 9 experts is not reached for men and industrials (8 for each category instead of 9), we will consider the results as relevant.

Results of the Delphi study and refinement of the framework

Mean standard deviation of the answers is 0.682, which makes it a satisfying consensus for (also notice that all standard deviations are less than 1). Average values obtained for the project complexity factors lie between 2.278 and 4.889, which permits to identify significant differences between the factors. In the end, only the factors getting a score higher than 4.500 are kept in the refined project complexity framework. The obtained framework, which is constituted of only 17 criteria, can be found hereinafter in Figure 9. The reader is to note that more detailed analysis of this Delphi study make the point of ongoing publications. Sub-criteria then correspond to the factors which exist in the refined framework. Default values for the criteria weights come from the Delphi study. They can be kept, but we want to leave the users the possibility to assess by themselves the criteria and sub-criteria weights thanks to the whole AHP process. Moreover, the opportunity to add the criteria which were eliminated between the original and the refined version of the framework should also be left. New criteria or new values should also be possibly added in order to be more consistent with the project context the user is working in.  A matrix C comparing the four criteria to one another (size, variety, interdependency and context-dependence), the eigenvector of which permits to identify the relative weights of these categories W1, W2, W3, W4.

 For each j from 1 to 4, let Kj be the number of sub-criteria corresponding to criterion Cj.

Each matrix SCj, of size Kj* Kj, permit to compute Kj weights . However, new sub-criteria might be included in the hierarchy or some existing sub-criteria might be deleted from the hierarchy in order to cling better to a specific industrial context. That is why we choose to keep this generic formulation here.

 Then, T comparison matrices are built up to compare the alternatives regarding each subcriterion. This permits to define for each evaluated alternative Ai a set of T scores

) ( j ki b .
In the end, the overall score of project Ai can be formulated as:

     4 1 1 ) ( ) ( ) ( j K k j ki j k j j b w W i S
Given the score obtained with these calculations on the set of alternatives, a relative measure of project complexity is proposed. Once again, we insist on the fact that alternatives can be projects in a multi-project environment, possible future projects compared to former ones, areas of a given project or project possible future scenarios in a mono-project environment, etc… Let Si be the priority score of alternative Ai obtained thanks to AHP calculations (0 ≤ Si ≤ 1). We propose that the relative complexity of alternative Ai, given the specific context of the set of alternatives, can be expressed as the following ratio

1 0 )) ( max( ) (     i i CI i S i S CI
A relative project complexity scale between 0 and 1 can thus be built thanks to this method. This index indeed permits to classify projects / project scenarios / project areas according to their global score regarding the main project complexity sources. This scale thus permits to give a relative indicator of project complexity, relative since it is related to the initial set of alternatives. But this indicator does not depend on the models of the projects, as the only expert evaluation of the projects regarding sub-criteria is needed. Subscales can then be defined in the same manner to focus on specific aspects of project complexity and highlight how a project is complex regarding interdependencies or context for instance.

Sensitivity of the measure

As underlined by some works (Isaacs, 1963), (Erkut and Tarimcilar, 1991), (Bayazit, 2005), (Cheng and al., 2007), sensitivity analysis in multi-criteria decision-making, and notably when using the AHP, should be performed in order to study the robustness of a choice. Sensitivity analysis indeed permits to understand the consequences of a change in the weights of criteria and sub-criteria.

For instance, decision makers are likely to change their opinion about (sub-)criteria over time because of an evolving context. In the case of multiple decision makers, disagreements when performing the AHP evaluations may involve future changes or some confidence intervals for the definition of the weights. In the end, whatever the reasons of the doubts, sensitivity analysis improves the credibility and reliability of the AHP model and results. In this paper, we propose to study the robustness of the results obtained thanks to the performing of suitable gradient sensitivity analyses. The reader will note that some of them are notably possible thanks to the Expert Choice software.

Overall gradient analysis

Calculations and formulations for a first overall gradient sensitivity analysis are proposed. In order to do, S(i) is to be rewritten in the following manner

        T k ki k j K k j ki j k j b w b w W i S j 1 4 1 1 ) ( ) ( ) ( , Where ) 4 ( ) ( ) 4 ( 4 4 3 ) 3 ( ) ( ) 3 ( 3 3 2 ) 2 ( ) ( ) 2 ( 2 2 1 ) 1 ( ) 1 ( 1 1 3 2 1 3 2 1 2 1 2 1 1 1 , & , ) 1 ( , , & , ) 1 ( , , & , ) 1 ( , , & , 1 , i K K K k ki K K K k k i K K k ki K K k k i K k ki K k k ki ki k k b b i w W w K k K k b b i w W w K k K k b b i w W w K k K k b b i w W w K k k                                       
Moreover, we know that the following condition is respected on the overall weights wk

   T k k w 1 1
As suggested in (Erkut and Tarimcilar, 1991) with calculations on cases with T = 2 or T =3, we propose that S(i) can be considered as a linear function of a given weight under certain conditions.

Let us suppose that a weight wk can vary from 0 to 1 and that the ratios of the other weights are fixed (to keep the same proportion that is given after performing pair-wise comparisons). The gradient sensitivity can be studied in order to analyze in a first manner the sensitivity to the variation of wk. In order to facilitate the notations, let us suppose that one wants to study the case of w1. Let the ratios of other weights be: ,..., ,..., ,

        Then, 1 .... ) ( .... ) ( ) ( ..., , 1 .... ... , 1 .... ... 2 1 2 2 2 1 1 4 4 2 4 2 1 1 3 3 1 1                                 T T l T k l T l T l T l T l T k T k w w
) ) ( 1 ( ) 1 ( 2 1 2 1          T m l T m l T p w w , And ) ) ( 1 ( ) 1 )( ( ), 1 ( 2 , 2 1 2 1 2 ) 1 (                  T m l T m l l T k l k p w p w T k k
As a whole, for any i, we can rewrite the score S(i) as Meaning that S(i) can be rewritten as an expression which is linear in w1, This permits to identify the gradient sensitivity of each S(i) regarding the variation of w1. If this gives interesting information, we need to have a closer look at the sensitivity analyses since the weights wk do not correspond directly to the evaluations of the decision-makers. That is why the next section focuses more directly on the variations on criteria weights W1 to W4 and on subcriteria weights

) ) ( 1 ( ) 1 ( ) ) ) ( 1 ( ) 1 )( ( ( ) ( 2 1 2 1 1 1 2 2 1 2 1 2 ) 1 ( 1 1                             
) ) ) ( 1 ( ) ) ( 1 ( ) ( ( ) ) ) ( 1 ( ) ) ( 1 ( ) ( ( ) ( 2 1 2 1 2 2 1 2 2 ) 1 ( 2 1 2 1 2 2 1 2 2 ) 1 ( 1 1                                               
) ( j k w .

Deeper gradient sensitivity analysis

Regarding criteria weights

We know that

   4 1 1 j j W By rewriting           4 1 1 ) ( ) ( 4 1 1 ) ( ) ( ) ( ) ( j K k j ki j k j j K k j ki j k j j j b w W b w W i S
For each j and i, let Fj(i) be

   j K k j ki j k j b w i F 1 ) ( ) ( ) ( ) (
We can conduct an analogous approach (as in III.4.2.), meaning that if we want to study the variation on the evaluation of W1 for instance, with the ratios of other weights fixed

4 3 2 3 2 1 & W W p W W p  
As a whole, we can rewrite

2 1 2 4 3 2 2 2 1 2 1 2 4 3 2 2 2 1 1 1 1 ) ( ) ( ) ( ) 1 ) ( ) ( ) ( ) ( ( ) ( p p p i F i F p i F p p p p p i F i F p i F p p i F W i S            ,
That means that S(i) can be considered as such expressions which is linear in W1., or in any weight Wj. In order to study visually this sensitivity of S(i) regarding the variation of the criteria weights Wj, we propose to draw different synthetic graphs. The first gathers the variations of a given score S(i0) regarding the variations of the criteria weights. Let S(i0) be the score of project i0.

As shown on Figure 11, a graph can be drawn to understand this linear sensitivity of S(i0)

regarding the variation of W1, W2, W3 or W4.

This graph (as well as the analytical formulations) permits to compare the sensitivity to any criteria weight variation. This also permits to show the lower and upper bounds of the scores that can be reached for a given project if a criteria weight varies from 0 to 1, with the ratios of the initial evaluation kept identical. This is all the more interesting when two projects initially get two close scores: studying the possible variations of their scores S(i) permits to highlight the robustness of their ranking regarding project complexity evaluation. -24 -

Another interesting graph which can be built up can be seen on Figure 12. This graph shows the possible variations of the scores of all evaluated projects S(i) regarding the variation of a given criteria weight Wj0. This gives particular insights as the variations of the scores are compared. Interestingly, by studying first the close neighbourhood of the initial evaluation, one can analyse the robustness of the initial ranking. Moreover, one can underline when and how the initial ranking of the projects is going to change thanks to the crossings of the lines on the graph. This point is interesting since it permits to see the level of variation which is needed to have a change in the initially obtained ranking. In other terms, this permits to stress how robust the initial ranking is by defining the interval of possible variation of Wj0, where the ranking is kept the same.

Regarding sub-criteria weights

Here, we want to understand the sensitivity of the scores S(i) regarding the variations of subcriteria weights. In order to so, we fix a certain sub-criterion weight 5)

S(4) -25 - ) ( ) ( 2 1 ) 2 ( ) 2 ( 2 4 2 1 1 ) ( ) ( 4 1 1 ) ( ) (               K k ki k j j K k j ki j k j j K k j ki j k j b w W b w W b w W i S j j
By performing analogous calculations as in the former paragraphs, we can rewrite

)) ( ) ( ( ) ( ) 2 ( 1 2 4 2 1 1 ) ( ) ( i i w W b w W i S j j K k j ki j k j j          
Where

) (i  and
) (i



can be calculated similarly as in III.4.2. by replacing T and K2. As a whole, this permits to show that S(i) can be considered as linear in

) 2 ( 1 w
. That is why we suggest the drawing and analysis of similar graphs than those built up in III.4.3.i.

Case study 8.1. Introduction

The case study takes place within a start-up firm, the main activity of which is the production of stage musicals in France. Staging musicals or theatre plays are definitely projects, as underlined by Lehner (Lehner, 2009): it refers to all artisitical, technical and organizational processes which permit to stage a musical. Project start corresponds to the idea and choice of the show to stage.

Project end is generally considered as the first performance or the first week of performances (the activities which follow are often considered as the core day-to-day running activities of a theatre production firm). In the case studied, the firm has a portfolio of 7 projects of musicals to be staged in 2010-2012. The managers of the firm, who have both artistic and industrial backgrounds, recently launched their activity. They are assisted by several possible investors and partners.

They wonder which show(s) they should produce first.

As for them and as for us, project complexity appears here as one of the criteria which should be considered before making a decision on this issue. Other criteria may notably be linked with the overall project performance regarding the values creation processes of the project, notably in terms of profit, image, etc…. A global project selection process may then be defined after this study in order to include all these factors in a multi-criteria approach to select the best project.

We focus only on project complexity evaluation for the moment. Currently, in the context of this firm, the importance of project complexity as a factor for selection is all the more true than the lack of experience in start-up firms implies even greater difficulties to properly handle project complexity. As a consequence, a proper evaluation of relative project complexity appears to be really necessary. Some project complexity aspects are very present or specific to this sector, and as an introduction to this case study, we need to describe this specific context.

 Since they progress in the cultural sector, such projects are very likely to face scarcity and problems of matching incomes and expenses. The struggle for income also means that most long-term planning must be kept open for very swift changes as new orders and constraints are likely to appear. Incomes are also very difficult to foresee in France since French audiences are not used to musicals (for the staging of musicals has been rarer than in other countries for decades).

 Project management techniques are generally very new in this sector, and the majority of stakeholders may not be used to them. This lack of experience and maturity may imply some difficulties when dealing with project complex issues. Lack of experience is also present in this case since these projects occur within a start-up firm: people have very little experience working together for the majority of them.

 Logistic and technical constraints are much higher in the sector of musicals than in any other artistic domains. First, three artistic disciplines and thus three cultures (comedy, singing, dancing) have to coexist. Moreover, technological complexity is generally greater than in other artistic domains since musicals" sets and direction often imply more elaborated technological tools and mechanisms for staging.

In Figure 13, the reader can find a brief description of the 7 projects. Cast, creative team and project team size are notably given hereunder. Note that global budget is a first assessment of the overall budget of the project and that project duration corresponds to the estimated work before staging. The achievement of the first performance is considered as part of the project. Project 1 is the production of a French adaptation of a Broadway musical, whereas projects 2 to 7 are original creations. Projects 1, 2, 6 and 7 include some special effects, notably pyrotechnical ones for some of them. Projects 1 to 6 require detailed work on costume and set design, notably with advanced mechanical structures and machinery for some of them.

Results

We carried out our research thanks to interviews of some possible future project team members (5 participants) following the AHP evaluation process, given our hierarchical structure. These people were asked to perform an a priori ranking of the projects in terms of complexity. This a priori ranking was necessary to highlight the possible differences between their initial perception and the results obtained. Then, as mentioned before, specific advanced Saaty scales were elaborated with the interviewees in order to perform pairwise comparisons with less subjectivity.

For instance, if we note NS(i) the number of stakeholders for a given project i, the advance Saaty scale built in this case for the number of stakeholders criterion was the following (Figure 14).

Building up such advanced scales permit greater consensus when performing the study in group and facilitates communication on the results of the study when performed by a single user. As said before, we left the users the possibility to follow the entire AHP process, meaning that the relative weights of each criterion and sub-criteria obtained thanks to the Delphi study were not given to the users. For instance, as shown on Figure 16, project interdependencies notably appear as the greatest source of global project complexity, since they globally account for more than 55%

of the final sum score of the project in our evaluation.

S iz e V a r ie t y In t e r d e p e n d e n c ie s C o n t e x t T o t a l

P r o je c t 1 0 ,0 2 9 0 ,0 5 6 0 ,1 4 2 0 ,0 6 4 0 ,2 9 2 P r o je c t 2 0 ,0 3 1 0 ,0 1 9 0 ,0 9 6 0 ,0 1 7 0 ,1 6 2 P r o je c t 3 0 ,0 1 7 0 ,0 1 7 0 ,0 5 8 0 ,0 1 2 0 ,1 0 5 P r o je c t 4 0 ,0 0 4 0 ,0 0 7 0 ,0 2 4 0 ,0 0 6 0 ,0 4 0 P r o je c t 5 0 ,0 0 7 0 ,0 0 9 0 ,0 2 6 0 ,0 1 4 0 ,0 5 5 P r o je c t 6 0 ,0 5 1 0 ,0 3 8 0 ,1 8 5 0 ,0 3 3 0 ,3 0 8 P r o je c t 7 0 ,0 0 3 0 ,0 0 6 0 ,0 2 5 0 ,0 0 5 0 ,0 3 8 T o t a l 0 ,1 4 2 0 ,1 5 1 0 ,5 5 6 0 ,1 5 1 1 ,0 0 0

Figure 16. Relative weights of the seven projects

In the end, final results are obtained and permit to realise a ranking of projects according to a complexity scale / index (from 0 to 1), as shown on Figure 17. It can be noted that two projects (P6 and P1) appear to be much more complex than the others, then Project 2 appears to be significantly more complex than the other ones. First, the existence of a numerical relative evaluation of project complexity within a project portfolio appears to be promising since it permits to know which projects are to be the most complex ones, but also how complex projects are. Future research works are carried out to set out a methodology which could define a threshold value (in the context of the studied project portfolio) over which projects could be rejected (notably given the experience and project maturity of the firm). Moreover, this global ranking according to the relative project complexity index we propose is all the more interesting in this case study since the employees who were interviewed had made an a priori ranking which was different. In that a priori ranking, P3 was ranked second and P7 was ranked third, whereas P6 was only placed fourth. With this numerical assessment of complexity and this ranking given, discussions were held with the participants, and communication around the notion of complexity was facilitated.

They started to share their experience on complexity factors and realized that the difference with the a priori ranking they had done was mainly due to some communication and psychological barriers they had. For instance, P7 was a priori ranked by them third because the majority of them did not know where to find the skills and competence for the design of a specific special effect. Four of them had thus ranked this project as one of the most complex ones (two of them had even ranked it as the most complex). But when performing the pair-wise comparisons, the fifth employee, who had ranked a priori P7 as the less complex, said that he had already worked with such special effects and knew who could design them easily. The others changed their minds.

Such example in our case study proves both the necessity to facilitate and promote -30 -communication in order to manage complex projects more efficiently and the benefits obtained with the project complexity refined framework and index we propose.

Figure 18. Specific graphical comparison of projects 1 and 6.

Finally, as shown on Figure 18, we claim for the use of specific numerical and graphical comparisons of projects which obtain close scores. For instance, in this case, project 6 obtains a global score of 0.308 when project 1 obtains a score of 0.292, which makes in the end a small difference (around 5%). In order to assist the decision-makers with their decisions, a closer look is to be done over the two projects. When realizing that the score of interdependencies (the main factor) is 0.185 for project 6 and 0.142 for project 1 (difference of around 25%), the people which were interviewed in this case study definitely evaluated project 6 more complex that project 1.

Indeed, when analysing closer why such a difference was obtained, the participants underlined notably a greater specifications interdependence and interdependence of information systems for project 6. These specific interdependencies seemed all the more difficult to handle for the participants, which led them to the conclusion to reject project 6 at the time of this study. More precise comparisons can even be performed when descending to the level of sub-criteria and comparing projects on 0 to 1 relative subscales, as shown on Figure 19 with specifications interdependence sub-criterion. When W3 is inferior to 0.435, then project 1 is the most complex. Project 6 is the most complex in the other case, and thus notably when W3 is 0.556 (corresponding to the initial evaluation of the members). The obtained ranking may in the end give quite good confidence in the analysis made for decision-making (regarding a change in W3) since no change in the ranking occurs if W3 is kept between 0.435 and 0.778. In other terms, there is no change in the ranking unless a variation of 21.7% downwards and 39.9% upwards of W3. However, these sensitivity analyses are based on gradient calculations which assume that the ratios of other weights are kept constant. In order to be more precise, the weight spaces should be explored. As underlined by Erkut and Tarmicilar (Erkut and Tarmicilar, 1991), hypervolumes of convex subsets of the weight space (corresponding to the dominance of a project) should then be calculated, notably thanks to the works of Cohen and Hickey (Cohen and Hickey, 1979).

Conclusions and perspectives

As a whole, this paper elaborates an AHP-based methodology and measure to evaluate relative project complexity. The works proposed here answer the problem which was set after the literature review on existing (project) complexity measures. Indeed, as shown theoretically and validated with a first case study, the project complexity index proposed here permits to overcome to a great extent the limits of existing ones as it is:

 Reliable, since the final users are confident with the results, measures and scales which are proposed. First, during the case study, no numerical result was ever challenged by the participants of the case study. On the contrary, the results permitted to identify more precisely project complexity sources which were consciously or unconsciously felt without being clearly mentioned or stated.

Furthermore, the reliability of the results was also underlined when comparing with the a priori ranking which was made. The obtained results permitted to underline what they had forgotten in their a priori ranking and all the users agreed in the end on the ranking obtained by the complexity index proposed in this work.

 Intuitive and user-friendly. The users understand the construction of the measure and scales, and why they do measure project complexity or the level of complexity regarding a given criterion, sub-criteria, or set of (sub-)criteria. This results in the end in a facilitation of communication on project complexity and project complexity factors. Finally, the measures and scales were easily computed and permitted rapid calculations and quick changes, which was an important requirement for an industrial use.

 Globally independent of the project models which are used for project management. Indeed, no reference to project management tools or models (WBS, PERT networks, Gantt charts, risk lists, etc…). was ever made during the construction of the measure or during the case study. This means that none of these models is needed as a reference to assess project complexity through this method. However, a limitation exists on this point. The refined project complexity framework is the basis of the measure. But, one should not forget that this framework is in essence a specific model, not of a project, but of what project complexity stands for. As a consequence, the measure which is proposed refers theoretically to this project complexity model. However, the final user is free to add some aspects (project complexity factors) in the AHP hierarchical structure proposed in this study. The methodology which is proposed here is thus quite generic with the project complexity model used, as long as it can be modelled as a hierarchical structure of project complexity factors.

 Able to highlight project complexity sources when building up the global complexity scale and the subscales. As shown by the case study, these scales enable the user to address many issues regarding decision-making and project complexity:

o Project prioritization within a portfolio in order to focus on the most complex projects (the ones where more complexity-related management methods and tools are needed).

o Project areas (for instance thanks to a classical WBS decomposition, or according to geographical areas) prioritization in order to focus on the most complex areas of a given project (the ones where more complexityrelated management methods and tools are needed).

o Comparison of present projects with past projects.

o Project scenario prioritization in the case of mono-project management.

o One-to-one detailed comparison of two projects which exist in the same portfolio in terms of project complexity.

o Global identification of the principal project complexity sources within a given project / a given portfolio / any set of projects.

However, some limitations do appear in this work and offer perspectives for future research on project complexity evaluation.

 First, the Analytic Hierarchy Process has received some criticisms on the fact that rankings can vary when adding or subtracting an alternative to the set of alternatives on which the study is performed [START_REF] Holder | Some comments on the analytic hierarchy process[END_REF]. We thus recommend the users to give specific attention to the step when the set of alternatives to be compared is selected. First, all alternatives in this set should strictly correspond to the final selection or comparison objective which is addressed. Second, projects on which the final users have few information or data may not be selected first as the quality of pair-wise comparisons may be considerably reduced. Finally, even with these recommendations, we need to underline that as for any decision-

Figure 3 .

 3 Figure 3. First version of the project complexity framework

Figure 5 .

 5 Figure 5. Critical analysis of multi-criteria decision methodologies

Figure 6 .

 6 Figure6. Using the AHP under complex contexts (Fumey, 2001) adapted from(Saaty, 1981)

Figure 7 .

 7 Figure 7. AHP pairwise positive reciprocal comparison matrices

Figure 8 .

 8 Figure 8. Conduction of the Delphi study according to a two-round process.

  The questionnaire was introduced by a page explaining, such as in(Bryant and Abkowitz, 2007), the overall purpose and structure of the survey as well as the experts anonymity conditions at each stage of the study. The questionnaire was divided into eight sections, following the structure of the first version of the project complexity framework: SIZE-ORG, SIZE-TECH, VAR-ORG, VAR-TECH, INT-ORG, INT-TECH, CONT-ORG, CONT-TECH.

Figure 9 .

 9 Figure 9. Refined project complexity framework 6.5. Building up the hierarchical structure With the refined project complexity framework, an AHP hierarchical structure is to be built according to Figure 10. The overall goal (objective) is the ranking of alternatives. First level criteria (intermediary goals) correspond to the four groups of project complexity factors, that is to say project size, project variety, project interdependencies and project context-dependence.

Figure 10 .

 10 Figure 10. AHP hierarchical structure to assess project complexity

  Kj. Kj can easily be identified thanks to the reading of the refined framework. In the end, it gives the

Figure 11 .

 11 Figure 11. Sensitivity analysis of score S(i0) regarding the variations of any criteria weight Wj

Figure 12 .

 12 Figure 12. Sensitivity analysis of any score S(i) regarding the variations of criteria weight Wj0

  can rewrite the scores S(i) in the following manner

Figure 13 .

 13 Figure 13. Brief description of the seven projects

Figure 15 .

 15 Figure 15. Overall criteria and sub-criteria weights: project complexity factors comparison

Figure 17 .

 17 Figure 17. Relative project complexity index in the case study

Figure 19 .

 19 Figure 19. Building up comparison subscales on sub-criteria

2.1. Existing project complexity measuresSeveral authors tried to define complexity measures in order to explain project failures, to identify intricate situations, to understand better project complex phenomena and to help decision-making. Indeed, such a measure is notably to assist decision-makers before engaging their projects / portfolios into too complex situations since too early decisions when facing complex and uncertain situations often fail to deliver the targeted performance.

Number of stakeholders criterion Corresponding Saaty scale

If NS(i)-NS(j) = 0 then contribution to complexity is equal Aij = 1

If NS(i)-NS(j) ≤ 2 then contribution to complexity is moderately more important for project i Aij = 3

If NS(i)-NS(j) = 3 then contribution to complexity is moderately more important for project i Aij = 5

If NS(i)-NS(j) = 4 then contribution to complexity is moderately more important for project i Aij = 7

If NS(i)-NS(j) ≥ 5 then contribution to complexity is moderately more important for project i Aij = 9 The methodology and measure proposed in this research work proved to be helpful in this case study. First, as shown in Figure 15, a first table is to be built to analyse the situation of the firm regarding project complexity. The relative weight of each sub-criterion can be evaluated, which gives information on where projects are likely to be more complex within the firm. Project managers should pay particular attention to the project complexity factors which get the best relative score (last column in figure 15). On the contrary, some aspects of project complexity (low scores) may potentially be neglected at a first sight. This piece of information thus permits one to concentrate more efficiently on the principal factors of project complexity under a given firm and project environment.

confident with the decisions they make. For instance, we notably studied the possible variations of the seven project scores regarding any criteria weight variation. The one related to W3

(interdependencies) can be found hereunder in Figure 20 and is built up thanks to following equations corresponding to the sensitivity to W3 variation. 4) S( 5) S( 6) S( 7)

making process and tools, great caution and awareness should be taken when making the final decision (significant gaps should exist between scores and results may not be too sensitive).

 Second, despite the sensitivity analyses which are proposed here, uncertainty in the judgment of the users is not much taken into account with this methodology.

To address this issue, we plan to introduce a fuzzy AHP-based method with triangular fuzzy numbers in future research works. Such applications of this method in project management can notably be found in (Mahmoodzadeh and al, 2007).

 Finally, future research is going to explore the possibility to extend this model to an ANP (Analytic Network Process) model. Indeed, Taslicali and Ercan [START_REF] Taslicali | The analytic hierarchy & the analytic network processes in multicriteria decision making: A comparative study[END_REF] say that their results suggest that "the ANP model represents reality as well as reliability better than the AHP model" due to the better integration of the interactions which exist between criteria. However, "the managerial implications of the execution of ANP and AHP are factors that vary from organization to organization". The AHP seems to be an easier methodology which may be accepted and understood better by managers. We think that in our case, exploring the possibility of using the ANP may be interesting since in essence, the criteria and sub-criteria of our structure are not independent.

Building up an ANP network structure for project complexity evaluation may then be interesting as it includes interdependence and feedback. However, the number of required judgement elicitations is likely to increase and become tougher when dealing with the interrelation between criteria or sub-criteria.

Acknowledgements

The authors would like to thank the employees of the start-up firm for their support to this work.


Member, IEEE Gaël Pillonnet

Modeling and Efficiency Analysis of Multi-Phase Resonant Switched Capacitive Converters

Keywords: DC-DC converter, switched-mode power supplies, resonant. I

This paper presents an analytical method to evaluate pertinent data of the resonant capacitive switching converter especially the voltage gain and power efficiency. Instead of long transient simulation time, the proposed model uses frequency decomposition to speed-up computation. This method is valid for N-phase operation and extends the recently published studies on this promising topology outside zerocurrent/voltage switching conditions. Thanks to this tractable expression, we also reveal the intrinsic efficiencies over the voltage gain of 2-and 3-phase structures working at the resonant frequency in step-down operation. These results help to gain better understanding of multi-phase operation and encourage additional studies to use the full capability offered by the resonant switched capacitor converter especially for power onchip integration.

INTRODUCTION

The Switched Capacitor Converter (SCC) approach has been well-studied in the literature over the past two decades [START_REF] Sanders | The Road to Fully Integrated DC-DC Conversion via the Switched-Capacitor Approach[END_REF]. The main advantage is to allow a highly integrated power supply compatible with IC technology for granular power management. However, the SCC topology suffers from inherent low efficiency when the output voltage is not a fraction of the input voltage [START_REF] Makowski | Performance limits of switchedcapacitor DC-DC converters[END_REF]. The Resonant Switched Capacitor Converter (RSCC) has been described in recent years by [3]- [START_REF] Lin | Parametric study of a resonant switched capacitor DC-DC converter[END_REF]. The main benefits of this structure are to limit the flying-capacitor charging loss of the SCC counterpart and to operate under soft switching conditions (i.e. zerocurrent/voltage switching ZCS/ZVS) thus reducing the switching loss. Promised power-density is achieved by [START_REF] Kesarwani | 4.5 A 2-phase resonant switched-capacitor converter delivering 4.3W at 0.6W/mm2 with 85% efficiency[END_REF] to design an on-chip point-of-load voltage regulator. In the literature, only 2-or 3-phase RSCCs have been described and modeled under ZCS/ZVS operation. This paper extends the model to N-phase RSCCs without any switching condition by an alternative approach. Then, we explore its potential benefit outside ZCS/ZVS operation especially for the wide-range lossless regulation, one of the key issues in SCC and RSCC structures. First, we detail the frequency-domain analysis of the N-phase RSCC providing pertinent data such as the conversion ratio and power efficiency which do not need a large processing capacity. Based on the proposed model, we explore 2-and 3-phase operation. This paper shows that the 3-phase is able to reach better efficiency compared to the SCC counterpart in step-down configuration under the same ideal conditions.

II. N-PHASE RESONANT SWITCHED CAPACITOR CONVERTER MODEL

The N-phase RSCC formed by a j-switch network SW j and RLC tank is shown in Figure 1. The inductor is much smaller than in a conventional inductive-based converter because the main energy is stored in the capacitor [START_REF] Sano | Performance of a High-Efficiency Switched-Capacitor-Based Resonant Converter With Phase-Shift Control[END_REF]. The N-phase, denoted ϕ i , is completed in one resonant period T = ∑ T ∅i N i=1 = 2π√LC called a cycle. A switching sequence (one cycle) is illustrated in Fig. 1 [START_REF] Pascal | Efficiency comparison of capacitor-, inductor-and resonant-based converters fully integrated in CMOS technology[END_REF]. There are four possible switch states called S k shown in Table 1 for each phase based on the switch configuration of a 2:1 SCC (K in and K out coefficients are used later). To study the achievable efficiency of an N-phase RSCC in this letter, the assumptions for the proposed model are i) negligible on-state resistance of the switches, ii) no parasitic component in the RLC tank e.g. no bottom plate, iii) the period of the N-phase called a cycle is equal to the resonant frequency, iv) the quality factor of the RLC tank is sufficiently high, v) the load is modeled as a voltage source (the bypass capacitor C o is sufficiently large) and vi) steady-state operation. The effect of the control scheme, the interleave configuration and the switching loss are however beyond the scope of this letter. Considering the assumptions described above and by using Fourier transform, the frequency decomposition of the voltage across the RLC tank V RLC can be expressed as:

V in C gnd V out L R R o C o gnd V RLC i RLC SW1 SW2 SW3 SW4 T 1 T 2 T N-2 T ϕ 1 ϕ 2 ϕ N-1 ϕ N V RLC T N-1 i in i out i RLC (t) = V RLC (t) Z = a 0 Z(0) + ∑ a n cos nwt + b n sin nwt Z(nw) +∞ n=1 under iv) ≈ assumption a 1 cos wt + b 1 sin wt R (1)
where the Fourier coefficient a 1 and b 1 are equal to:

a 1 = 2 T ∫ V RLC (t) cos wt dt T t=0 = 2 T ∑ ∫ V RLC,ϕi cos wt dt T i+1 t=T i N-1 i=0 (2) b 1 = 2 T ∫ V RLC (t) sin wt dt T t=0 = 2 T ∑ ∫ V RLC,ϕi sin wt dt T i+1 t=T i N-1 i=0 ( 3
)
where N is the number of phases, T i is the i th switching instant between the i th and i+1 th states, T is the resonant period within which the cycle is completed (T ϕN =T), and V RLC,ϕi is the voltage across the RLC tank in the i th switching phase (defined in Table 1).

The averages of the input and output currents are then given by:

〈i in 〉 = 1 T ∑ ∫ K out,ϕi i RLC (t)dt T i+1 T i N-1 i=0 (4. a) 〈i out 〉 = 1 T ∑ ∫ K in,ϕi i RLC (t)dt T i+1 T i N-1 i=0 (4. 𝑏)
where K in,ϕi and K out,ϕi are the coefficients in the ith phase defined in Table 1.

By combining (1-4), the voltage gain and power efficiency are expressed using the design variables T i , R and R o so that:

α = V out V in = R o 〈i out 〉 V in (5. 𝑎) η = P out P in = R o 〈i out 〉 2 V in 〈i in 〉 (5. 𝑏)
The achievable efficiency given by (5.b) does not depend on the switching frequency. This result is also confirmed in the following section in ( 8) and ( 9) for 2-and 3-phase operation. The switching frequency implying switching loss (neglected here) only sets the LC tank value, then the RSCC power density. An increase in frequency, reduces the LC value and size but also increases the switching loss thus decreasing the efficiency. Therefore, we are in line with the scope of this letter which focuses on the theoretical achievable efficiency of Nphase RSCC under ideal conditions i.e. ideal switches.

The method could be extended to N-harmonic decomposition if the quality factor is high enough to neglect the harmonics as shown in [START_REF] Kesarwani | Resonant Switched-Capacitor Converters for Chip-Scale Power Delivery: Design amp; Implementation[END_REF]. For the clarity of this paper, the next results are given using only the fundamental decomposition but the complete expression was calculated using third harmonic extension to check their effect.

III. DERIVATION OF METHOD TO 2-PHASE OPERATION

To illustrate the simplicity of the model derivation, we propose a calculation for a 2-phase 2:1 RSCC. The first and second phases are S 1 and S 2 , respectively, as in 2:1 SCC. The Fourier coefficients of the i RLC are calculated from ( 2) and (3) so that:

a 1 = 2 T (∫ (V in -V out ) cos wt dt T 1 T o =0 + ∫ V out cos wt dt T 2 =T T 1 ) = V in π (1 -2α)s 1 (6) b 1 = 2 T (∫ (V in -V out ) sin wt dt T 1 T o =0 + ∫ V out sin wt dt T 2 =T T 1 ) = V in π ((2α -1)c 1 + 1 -2α)
where c i = cos(wT i ) , s i = sin(wT i ), α = V out V in

Then, the input and output currents are found from (4) and using the i RLC expression, we obtain:

〈i out 〉 = 1 T (∫ i RLC dt T 1 T o =0 + ∫ -i RLC dt T 2 =T sw T 1 ) = 1 πR (b 1 + a 1 s 1 -b 1 c 1 ) 〈i in 〉 = 1 T ∫ i RLC dt T 1 T o =0 = 1 2πR (b 1 + a 1 s 1 -b 1 c 1 ) (7)
Using ( 5) and the previous results (6-7), the voltage gain and power efficiency are thus:

α = V out V in = s 1 2 +c 1 2 -2c 1 +1 2s 1 2 +2c 1 2 -4c 1 +2+π 2 R R o < 0.5 (8. a) η = P out P in = 2α < 1 (8.b)
The numerical resolution of equation ( 8) has been verified against full circuit simulation under the same conditions and shows less than 1% variation. The power efficiency expressed in ( 8) is linearly proportional to the voltage gain as previous work has already described [START_REF] Cervera | A High-Efficiency Resonant Switched Capacitor Converter With Continuous Conversion Ratio[END_REF]. Figure 2(a) shows the best efficiency for a 2-phase RSCC and the associated optimal switching instant T 1 . For example, if R<<R o and T 1 =T/2, the voltage gain is equal to 0.5 and the process is theoretically lossless (η=1). These results prove that the maximal efficiency of a 2-phase RSCC is achieved when α=0.5 as in a classical SCC. A lower voltage gain implies a lower efficiency equal to the 2:1 SCC counterpart. Expression (8) also highlights the effect of the parasitic resistance R compared to the load resistor R o which limits the achievable efficiency. In Fig. 2(a) the ratio R o /R is fixed at 10 and 100 to illustrate the tank resistor effect. The highest efficiency in an RSCC is limited only by the resistor ratio instead of the inherent charging-loss (proportional to C×F factor) in an SCC. The efficiency is insensitive to the LC value and resonant frequency as long as the conduction and switching losses are not taken into account in our analysis (see the aforementioned assumptions). Fig. 2(b) shows the time-domain waveforms of the RLC tank and output for a 0.3 voltage gain. The proposed model highlights an unexpected switching sequence. For example, the converter works in hardswitching conditions to reduce the loss, and the output current i out is sometimes negative to average the output charge transfer in the resonant period T.

The {S 1 ,S 2 } RSCC cycle, also called 2:1 operation, cannot achieve flatness efficiency versus voltage gain as is mathematically proven in [START_REF] Ioinovici | Comments on ldquo;Unified Analysis of Switched-Capacitor Resonant Converters rdquo[END_REF] even if all components are ideal (no on-state resistance, no resistance in the LC tank). We have also analyzed other possible cycles limited to 2 phases. In this case, the cycle could be equal to {S k ,S k' } where {k,k'}ε [START_REF] Sanders | The Road to Fully Integrated DC-DC Conversion via the Switched-Capacitor Approach[END_REF][START_REF] Makowski | Performance limits of switchedcapacitor DC-DC converters[END_REF]3,[START_REF] Cervera | A High-Efficiency Resonant Switched Capacitor Converter With Continuous Conversion Ratio[END_REF] means 42 different cycles. We have used a symbolic computation software (Matlab associated with a symbolic toolbox) to solve equations (1) to (5) for each cycle. Unfortunately, there is no configuration where the efficiency is not linearly proportional to the voltage gain in step-down configuration.

IV. ANALYSIS OF 3-PHASE OPERATION

To alleviate the efficiency dependence on the voltage gain, some papers [START_REF] Cervera | A High-Efficiency Resonant Switched Capacitor Converter With Continuous Conversion Ratio[END_REF], [START_REF] Kesarwani | Resonant Switched-Capacitor Converters for Chip-Scale Power Delivery: Design amp; Implementation[END_REF] propose to add an extra phase. However, adding more phases implies more switching losses but it almost disengages the efficiency from the voltage gain. Here, we propose to study this 3-phase operation by keeping the cycle time equal to the resonant period (where the presented method is valid). Under this constraint, we have analyzed the {S 1 ,S 2 ,S 3 } sequence for a 3-phase RSCC. In this particular sequence, the voltage gain and power efficiency of the cycle are given following the same method as in 2-phase i.e. by:

α = 2s 1 2 + 2c 1 2 -s 1 s 2 -c 1 c 2 -3c 1 + c 2 + 1 4s 1 2 + s 2 2 + 4c 1 2 + 2c 2 -4c 1 c 2 -4c 1 + 2c 2 + 1 + 2π 2 R R o (9) η = 2π 2 R R o α 2 (1 -2α)(s 1 2 + c 1 2 ) + α(s 1 s 2 + c 1 c 2 ) -(3α -2)c 1 -αc 2 -α + 1
Even if these analytical equations do not directly show the efficiency dependence on the voltage gain as in [START_REF] Ioinovici | Comments on ldquo;Unified Analysis of Switched-Capacitor Resonant Converters rdquo[END_REF], their numerical resolution helps to rapidly find the optimal efficiency by exploring the design free variables T 1 and T 2 from 0 to unity where T 2 >T 1 (T 3 =T=1 as a cycle is completed within the normalized resonant period). Figure 3(a) gives the best efficiency for a 3-phase RSCC and the associated optimal switching instants. The association of ideal 1:1 and 2:1 SCCs where their efficiencies are equal to α and 2α, respectively, is also drawn as a dashed line. The result is computed in a few seconds on a laptop, many decades faster than full transient simulations. Full circuit simulations have also been performed to compare the results with the analytical expressions. The mismatch is always less than 1%. Figure 3.b plots the timedomain waveforms for a 0.3 voltage gain at the optimal switching instants {T 1 ,T 2 } given by Fig. 3(a). As for 2-phase operation, the optimal sequence on the one resonant period leads to unexpected behavior in the time-domain.

As can be seen from Fig. 3, the 3-phase RSCC with {S 1 ,S 2 ,S 3 } cycle maintains a better efficiency compared to the 2-phase SCC in step-down configuration (α<1). Even if all components are ideal, the simulation shows that it is not possible to maintain a constant efficiency over the voltage gain by keeping the cycle frequency at that of resonance. We have also taken into account the second and third harmonic components in the previous calculation but we have observed a slight difference when R<<R o .

The RSCC in hard switching condition but the overall RSCC efficiency is maximized compared to other possible switching instants under the assumptions described in Section I. One limitation of the proposed operation is to create some EMI issues due to a current spike problem [START_REF] Shoyama | Evaluation of Conducted Noise of Resonant Switched Capacitor Converter[END_REF] and additional switching loss at high switching operation.

V. CONCLUSION

In this paper, we give a generic method to analyze any cycle sequence of a multi-phase RSCC. We also address the inherent power efficiency limits of 2-and 3-phase RSCCs operating at the resonant frequency outside soft-switching conditions. The model is based on frequency-domain decomposition especially allowing fast exploration of its inherent efficiency capability. We have demonstrated the model by 2-and 3-phase converters to find the tractable analytical expressions of their voltage gain and power efficiency. Excellent agreement was found between the proposed model and the classical transient simulation. The fast resolution of the expression allows us to determine the best switching instants to reach the highest efficiency for any voltage gain in a step-down configuration. Thus, we confirm a more constant efficiency RSCC profile over a wide range of voltage gain compared to a conventional SCC when a third phase is added. Although the discussion has been applied to 2and 3-phase resonant converter efficiency, the method is valid regardless of tank impedance, other switch structure or for evaluating other pertinent data in the circuit. The model could also be extended to outside resonant frequencies by taking into account the other harmonics but the analytical expressions are less tractable. However, the model has fundamental limitations based on the assumptions given in section I. These limitations and extension to 4 phases will be the direction of further research.

Fig. 1 .

 1 Fig. 1. Resonant switched capacitor converter.

Fig. 2 .

 2 Fig. 2. (a) Efficiency v. voltage gain of a 2-phase RSCC {S1,S2} at resonant frequency, (b) time-domain waveforms at a 0.3 voltage gain.

TABLE 1 .

 1 THE FOUR SWITCH STATES OF AN RSCC.

	State	VRLC	SW1	SW2	SW3	SW4	𝐾 𝑖𝑛 =	𝑖 𝑖𝑛 𝑖 𝑟𝑙𝑐	𝐾 𝑜𝑢𝑡 =	𝑖 𝑜𝑢𝑡 𝑖 𝑟𝑙𝑐
	S1	Vin-Vout	ON	OFF	OFF	ON	1		1
	S2	Vout	OFF	ON	ON	OFF	0		-1
	S3	0	OFF	ON	OFF	ON	0		0
	S4	Vin	ON	OFF	ON	OFF	1		0