Camille Jean

Marija Jankovic

Julie Stal-Le Cardinal

Jean-Claude Bocquet

Predictive modeling of telehealth system deployment

Keywords: Decision support, Simulation, Telehealth 1

come

INTRODUCTION

Healthcare systems are facing increasing demographic and economic challenges. Healthcare service demands are growing due to an aging population and a rise in the number of people afflicted with chronic diseases. According to the data of the National Institute of Statistics and Economic Studies (INSEE), the percentage of the population over 60 years of age in France will grow from 23.1% to 32.8% by 2035, a projected increase of 7 million people, bringing the total population in that age category to about 21 million [START_REF] Brutel | Projections démographiques pour la France, ses régions et ses départements (horizon 2030-2050[END_REF]. This trend coincides with current global trends. The elderly often suffer from chronic polypathologies requiring long-term treatment. However this increase in the occurrence of chronic non-communicable diseases, such as heart disease, stroke, cancer, chronic respiratory disease and diabetes, is not only associated with an aging population. Other risk factors associated with contemporary urban life have been identified, such as poor diet, physical inactivity, tobacco use and alcohol or pollution (WHO, 2000). An immediate consequence of the increased prevalence of chronic disease is growth in healthcare service demand.

2 Despite this increased demand, France is facing a shortage of general practitioners and specialists. Moreover, medical "desertification" is a major problem in many country regions. Several factors explain this evolution: an inadequate practitioner replacement rate (the numerus clausus for determining the number of students that are allowed to proceed to the second year of medical studies is too low to compensate for retirement) [START_REF] Doan | Demographic forecasts of medical workforce supply in France (2000-2050)[END_REF], feminization of the profession, increased medical specialization, and the pursuit of a more comfortable life (i.e., more urban) by medical professionals. The combined impact of these factors is a decrease in physician density that will reach a minimum in 2020. Several policies have been put in place in order to bring physician density back to its current level by 2030 [START_REF] Drees | La démographie médicale à l'horizon 2030 : de nouvelles projections nationales et régionales[END_REF].

Despite interest in the potential opportunities and benefits of telehealth [START_REF] Davalos | Economic Evaluation of Telemedicine: Review of the Literature and Research Guidelines for Benefit-Cost Analysis[END_REF], no studies to date have predicted the influence of large-scale integration of telehealth into the existing healthcare system. This remains a main challenge for decision makers who must allocate and manage resources in order to maximize the potential benefits and impact of integration. Given the inherent complexity of the telehealth system (i.e., many different actors, hidden system interfaces, technology management), predicting possible system utilizations, as well as possible impacts on existing healthcare services becomes quite difficult. In order to bridge this gap, we propose to develop an overall telehealth integration model with regard to its sustainability: economic aspect (different healthcare system related costs), social impact (medical time availability) and environmental impact (patient transport). The proposed parametric model is also scenario-based approach identifying several underlying hypothesis regarding telehealth system integration. The results underline the necessity to carefully plan telehealth system deployment in order to ensure healthcare system sustainability and efficiency.

Therefore, in the section 2 we address the utilization of simulation in healthcare modelling and current models related to telehealth. Section 3 gives details of the French telehealth model and in particular "Télégeria" project [START_REF] Espinoza | Déploiement de la télémédecine en territoire de santé. Télégéria, un modèle expérimental précurseur[END_REF]. Section 4 and 5 give an overview of the research design as well as different data gathered that are suited for telehealth system integration modelling presented in section 6. Different underlining scenarios as well as simulation results are detailed in section 9 and 10.

In the end, we discuss general trends and conclusions. LITERATURE REVIEW OF HEALTHCARE SIMULATION APPROACHES Simulation is the process of developing a model of a real system and conducting experiments with this model for the purpose either of understanding the behavior of the system or of evaluating various strategies [START_REF] Shannon | Systems simulation: the art and science[END_REF]. Simulation methods that are used to support decision making in other sectors, such as manufacturing or defense, have been used previously to help understand and solve healthcare problems. In their literature review, Brailsford and colleagues (2009) categorized more than 342 studies related to simulation and modeling methods used in healthcare engineering such as discrete event simulation, system dynamics, Monte Carlo simulation and Markov chains. Jun et al (Jun et al. 2009) have identified and categorized 8 methods they consider frequently used: agent based simulation, discrete event simulation (DES), gaming simulation, hybrid simulation, inverse simulation, Monte Carlo simulation, real time simulation and system dynamics (SD). Discrete event simulations and system dynamics have been found to be the two most simulation methods utilized in healthcare engineering [START_REF] Brailsford | An analysis of the academic literature on simulation and modelling in health care[END_REF].

DES models the operation of a system as a discrete sequence of events over time. Each event occurs at a particular point in time and represents a change in the state in the system (Zeigler et al., 2000). DES has been used for numerous studies in healthcare engineering. It has been used to track the spread and containment of hospital acquired infections [START_REF] Hagtvedt | A Simulation Model to Compare Strategies for the Reduction of Health-Care-Associated Infections[END_REF], plan for disease outbreaks [START_REF] Aaby | Montgomery County's Public Health Service Uses Operations Research to Plan Emergency Mass Dispensing and Vaccination Clinics[END_REF], determine bed requirements [START_REF] Griffiths | A simulation model of bed-occupancy in a critical care unit[END_REF], investigate emergency departments [START_REF] Paul | A Systematic Review of Simulation Studies Investigating Emergency Department Overcrowding[END_REF], and determine appropriate ordering policies in the blood supply chain [START_REF] Katsaliaki | Using Simulation to Improve the Blood Supply Chain[END_REF]. It is also used for planning, scheduling, reorganizing and managing healthcare and hospital services related to communicable diseases, bio-terrorism, screening, and costs of illness; comparing alternative healthcare interventions; evaluating policy and strategy [START_REF] Fone | Systematic review of the use and value of computer simulation modelling in population health and health care delivery[END_REF][START_REF] Katsaliaki | Applications of simulation within the healthcare context[END_REF]; reducing risk, operating costs, lead time, and capital cost; facilitating faster plant changes; and improving customer service [START_REF] Robinson | SimLean: Utilising simulation in the implementation of lean in healthcare[END_REF].

System dynamics is an approach used to understand the behavior of complex systems over time. It was developed by Jay Forrester at MIT in the mid-1950s (Forrester, 1961). This approach addresses internal feedback loops and time delays that affect the behavior of the entire system. It is described as an analytical modeling tool based on a general systems theory approach [START_REF] Bertalanffy | Théorie générale des systèmes[END_REF][START_REF] Le Moigne | Le "Pacte territoire-santé[END_REF]. The methodology uses computer simulation models to relate the structure of a system to its behavior over time. Using SD allows us to see not just events, but also behavior patterns representing time trends [START_REF] Homer | System dynamics modeling for public health : Background and opportunities[END_REF]. Understanding patterns of behavior instead of focusing on day-to-day events can offer a radical change in perspective. The main objective is to show how a system's own structure is related to its successes and failures. Several researchers [START_REF] Erdil | Modeling the dynamics of electronic health records adoption in the U.S healthcare system[END_REF] have used SD to examine the adoption of electronic health records and the impact of telehealth innovation on the overall system. [START_REF] Erdil | Modeling the dynamics of electronic health records adoption in the U.S healthcare system[END_REF] proposed a model for the adoption of electronic health records (EHR) in the United States. The challenge in EHR adoption is related to the one addressed in this research: addressing increasing patient demand while maintaining healthcare costs. SD was proposed as a way of understanding the complexity of EHR integration. The model captures the major variables that have been identified, such as the affected population, cultural barriers, industry pressure, market maturity and usage performance. These variables constituted a framework with the aim of identifying policies that would accelerate the process of EHR adoption. [START_REF] Morecroft | Explaining Puzzling Dynamics: Comparing the Use of System Dynamics and Discrete-Event Simulation[END_REF] compared these two approaches. They argued that DES is particularly suited to operational-level problems such as transactions, processing, and the flow of individual entities. In contrast, SD is often used to assess aggregated high level implications, especially when considering the impact of policy and strategic decisions. The choice between these two approaches can be based on the level of detail needed in problem solving: a close-up resolution in DES yielding individual entities, attributes, decisions and events, versus a distant resolution in SD capturing homogenized entities and continuous policy pressures. In their study on the use of SD in healthcare between 1965 and 2004, [START_REF] Koelling | Health Systems : A dynamic system-benefits from system dynamics[END_REF] found that SD is used mostly to study international, national and regional strategy and policy implications (76% of the articles). Like [START_REF] Jones | Understanding Diabetes Population Dynamics Through Simulation Modeling and Experimentation[END_REF], we agree that SD is an adapted approach that is well-suited to supporting healthcare professionals in their reasoning and decision making. In fact, SD and its visual nature encourage debate among involved stakeholders on the impact of the underlying structure on overall system behavior, which leads to the creation of new policies (Taylor and Lane, 1998;Zare Mehrjerdi, 2012).

As discussed previously, simulation has been widely used in healthcare engineering. In this research study, the focus is on telehealth system integration. We have found very few studies related to this subject. The work of [START_REF] Bayer | Assessing the impact of a care innovation: telecare[END_REF] is one of them. The authors present an interesting study about the implementation of home telecare in the UK. They highlighted the fact that most studies about telehealth services consider either clinical or economic impacts, thus failing to evaluate the overall system. In order to help decision makers gain a greater understanding of telehealth implementation, the authors proposed a model that would consider delays between different patient categories. The authors concluded that it may be more beneficial to focus on somewhat frail patients rather than the very frail. This is mainly due to the fact that a system evolves gradually, and investments in new home telecare technology should be made accordingly because people have a limited ability to cope with radical changes in the way they receive care. However, our work is not related to telecare (home healthcare) but telehealth (access to a medical expert in healthcare establishments) and does not concern the same material, human resources and perimeter.

The research studies discussed in this section illustrate some of the approaches used in healthcare modeling and impact investigation. However, in the case of telehealth, we found no research tackling related issues associated with long-term telehealth integration, with regard to its sustainability (economic aspect, environmental aspect and social aspect). Therefore, in order to support decision makers, we propose a parametric scenario-based model integrating long-term exogenous factors such as demographic patient information and the availability of specialist practitioners with the aim of predicting possible impacts over the next 20 years. The objective of this study is also to create a shared vision of the telehealth system amongst different stakeholders in order to ensure its deployment.

TELEMEDICINE MODEL IN FRANCE: TÉLÉGERIA CASE STUDY

Previously discussed challenges in healthcare system, combined with reduced allocations of public funding to the healthcare system due to the global economic crisis, have fostered a sense of urgency around innovation in medical system organizations. One of them is telehealth which consists in the use of information and communication technology to deliver health and social care [START_REF] Barlow | Implementing complex innovations in fluid multi-stakeholder environments: Experiences of "telecare[END_REF].

The main idea is to provide remote access to healthcare services, thus bridging physical distance between patients and healthcare professionals. Telehealth is based on inter-professional collaboration and necessitates establishing collaboration rules and best practices. In France, two main applications of telehealth are defined by law (Legifrance, 2010): (1) teleconsultation is the framework for consulting a non-local doctor (a healthcare professional may be present with the patient to assist the doctor during the teleconsultation) and (2) teleexpertise involves soliciting the opinion of one or more colleagues for knowledge acquisition or training purposes.

Télégéria project (Figure 1) is one of the experimental projects with aim of testing telehealth approaches; and mainly consists of teleconsultation and teleexpertise. This model has been chosen as one of the references in the French healthcare system. It has been deployed in 3 hospitals (Geogres Pompidou Hostpital, Vaugirard Hospital and Julie Siegfried EHPAD). It is the largest telehealth implementation in France. Over 36 months, more than 1,500 telemedicine sessions integrating 21 medical specialities were conducted. Telehealth stations may consist of a main screen and two related screens: one connected to radiology and medical records, and another one connected to biomedical equipment for exchanging medical information (i.e., hand camera electrocardiograph, spirometer, dermatoscope, otoscope, or ultrasound). Secure networks are mandatory in order to protect transmitted data [START_REF] Espinoza | Déploiement de la télémédecine en territoire de santé. Télégéria, un modèle expérimental précurseur[END_REF].

Figure 1: Télégeria project and mock-up

Based upon this experimentation and observations related to this project, the aim of this research is to investigate potential large-scale deployment involving 200 healthcare establishments that are identified as future constituents of the telehealth network of the Picardie region. This network should decrease the number of laborious inter-healthcare establishments transports, reduce waiting time with a better coordination between health professionals, strengthen prevention, and improve access to health services in rural areas.

RESEARCH PROTOCOL

The goal of this model is to help healthcare managers understand the hidden impacts of several strategies of telehealth integration and to investigate their potential performance. The main difficulty associated with system modeling and investigating possible behaviors is building a model that is reliable and easy to understand. Therefore, our research protocol is divided in 5 phases:

(1) Problem definition: We organized a working group comprised of doctors, medical staff and healthcare managers in order to define the direct impacts of telehealth integration, healthcare performance indicators, the time span and the geographic area;

(2) System modeling: Based on their feedback, a telehealth system integration model was proposed and discussed within the working group. The underlying hypothesis and related variables were defined in this phase;

(3) Quantitative data collection: Only parameters with data from reliable sources were included in the model. Data were collected from French administration reports and on the most significant telehealth experiment in France at the time, Télégéria;

(4) Scenarios making: Once the system structure was validated and the variables were defined, different telehealth implementation scenarios and their potential impacts over time were discussed and defined with the working group;

(5) Results and analysis: Scenarios were investigated in order to predict the future behavior of the system with respect to the impacts defined in step 1.

Even though this process seems to be sequential, in reality several feedback loops occurred when the working group expressed the need to integrate or remove telehealth impacts, change variables and even to clarify or change the research questions.

PROBLEM DEFINITION

The working group

The aim of this work was to investigate the possibility of telehealth integration on a national level. In order to identify system limits and potential integration problems, a group of stakeholders were identified to work on this issue. Our group was comprised of medical professionals associated with the leading telehealth experiment in France, Télégéria [START_REF] Espinoza | Déploiement de la télémédecine en territoire de santé. Télégéria, un modèle expérimental précurseur[END_REF], and healthcare managers from the Picardie region. Moreover, government policy makers were identified and contacted for specific needs during the design work. In the restricted working group 12 stakeholders were involved, including 5 practitioners and 2 medical staff associated with telehealth, and 5 healthcare managers.

Group brainstorming

During this first phase, the working group highlighted many quantitative and qualitative impacts of telehealth. The objective was to organize the proposition and to set the probable impacts and system parameters for the model.

For example, the members of the working group highlighted the fact that institutions and their practitioners are in considerable need of telehealth services. Indeed, it can help their patients avoid long transports and associated risks, as well as provide easier connections with medical experts.

Moreover, the group mentioned that for specialized practitioners, telehealth might be a way to save medical time by delegating activities and creating a more effective patient triage through increased use of teleexpertise. They also emphasized that telehealth might be a way to save costly patient transportations and to reduce bottlenecks in emergency departments. They also identified a potential medical development activity due to easier access to medical specialists.

Direct impacts retained

In view to possible telehealth integration, the working group identified five major impacts: the number of admissions to emergency departments, the number of transports, the number of specialized consultations due to better patient triage, the number of specialized consultations due to better territorial access, and the average duration of consultation. All other qualitative or quantitative impacts related to medical quality (such as the average length of stay (ALOS) of a hospitalization) were withdrawn due to a lack of consensus. These impacts are considered to be inputs in the proposed system:

1) Impact on the number of admissions to the emergency department: The working group posited that telehealth may help reduce visits to the emergency department by anticipating medical problems, especially for older people. In fact, sometimes, the risks associated with moving a patient are considered to be higher than the potential benefits. Therefore, older patients are rarely transported to see a specialist practitioner for prevention. This results in visits to the emergency department when problems become serious. A decrease in the number of admissions to the emergency department will help solve problems such as overcrowding and cost challenges;

2) Impact on the number of transports: The group highlighted the fact that using telehealth instead of moving patients or healthcare professionals would help to save costs associated with transports;

3) Impact on the number of specialized consultations due to better patient triage: The members of the group emphasized that the use of teleexpertise will help prioritize high-risk patients for earlier access. For example, sending skin images for dermatology consideration will help prioritize patients who are more at risk, and avoid inadequate or unnecessary consultations.

They also noted the fact that teleexpertise takes less time than face-to-face consultation; 4) Impact on the number of specialized consultations due to better territorial access: The group emphasized the fact that putting telehealth stations in the territory may increase the number of medical consultations by facilitating access to medical specialists; 5) Impact on the average duration of consultation: The group noticed that teleconsultations are shorter than regular consultations with specialized practitioners because specific tasks are delegated to medical staff. This will increase the amount of medical time available.

Performance indicators retained

Several aspects of system performances have been discussed as important for telehealth system integration:

1) Economic aspect, whether telehealth implementation increases or decreases the cost of the healthcare system. The related indicator is the total healthcare cost 2) Environmental aspect, whether telehealth implementation increases or decreases the CO2 equivalent rejected. The related indicator is the Carbon Dioxide Equivalent emitted

3) Social aspect, whether telehealth implementation increases or decreases medical time available making medical expertise available without delay. The related indicator is the total medical time available

Telehealth will also have an impact on medical quality (differences between the medical outcomes of telehealth consultations and normal consultations). Although some of the issues have been discussed, it has been decided not to integrate them at this stage as further medical investigation is needed with this regard.

Therefore, in this model, we assume that a face-to-face doctor-patient consultation is equivalent to a teleconsultation in terms of quality of medical care.

Time span and geographic perimeter

The French government has decided to focus on several strategic regions. One of them is the Picardie region and this study has been in particular considering this region with its specificities. The time span that has been chosen is 20 years, as it is the maximum time span in the literature for demographic projection.

SYSTEM MODELING

In order to investigate telehealth system integration, a parametric scenario-based model has been proposed (see Figure 2). This model integrates possible telehealth deployment hypothesis that are represented at the very top of the model, and three system performance aspects (economic, social and environmental). Due to the number of variables in this model, we propose to discuss in detail different causal relationships that have been taken into account.

Figure 2. Telehealth system integration model.

Calculation of the five impact rates over time

The scenarios presented in section 7 provide the maximum rate of each direct impact retained. This maximum is reached when all the medico-social establishments of the region have telehealth stations.

As telehealth implementation is made gradually, the direct impact rates need to be calculated over time (top of Figure 2). In order to do so, the maximum rate of each direct impact is multiplied by the impact rate over time.

This impact rate over time is delayed 18 months from the overall usage rate of telehealth stations over time to take into account the phenomenon of inertia of the healthcare system. This value was quantified by the group members based on their experiences with change management.

The overall usage rate of telehealth stations over time represents the speed of telehealth implementation in the region. It starts at "0" (no telehealth stations) and ends at "1" (all medico-social establishments have stations).

Overall usage rate of telehealth stations over time

Total healthcare cost

Number of admissions to the emergency department

Number patient transport

Number of specialist practitioners in the territory

Number of people in the territory

Total patient transport cost Total emergency admission cost

Number of consultation and teleconsultation

Total

Calculation of the total medical time available

The bottom right of Figure 2 represents the total medical time available calculation. This indicator represents the capacity to offer specialized care to patients. Its value is calculated by taking the difference between the total number of hours offered by medical specialists and the total number of hours needed by patients (sum of teleexpertise, teleconsultation and consultation time).

Parameters, such as teleconsultation duration, consultation duration, teleexpertise duration, and duration allocated by the specialist practitioners for consultations are quantified in section 7.

Calculation of the Carbon Dioxide Equivalent emitted

The bottom middle of Figure 2 represents the Carbon Dioxide Equivalent emitted calculation. This indicator is directly correlated to the number of patient transports.

Parameters, as the average patient transport distance per km, and the gram of Carbon Dioxide Equivalent emitted per km per year are quantified in section 7.

Calculation of the total healthcare cost

The bottom left of Figure 2 represents the total healthcare cost calculation. This indicator verifies whether telehealth implementation will increase or decrease the total healthcare cost of the healthcare system. Its value corresponds to the sum of the telehealth station cost (material, implementation, maintenance and work force), emergency cost, transport cost and medical cost.

Parameters such as the initial cost of a telehealth station, the annual cost of a telehealth station, emergency admission cost, transport cost and consultation cost are quantified in section 7.

Overall model

Figure 2 represents our final model to predict the impacts of telehealth integration and help decision makers. It includes the two exogenous variables that change over time: the number of people in the territory and the number of practitioners in the territory.

The number of stations in the territory over time, the number of admissions to the emergency department, the total number of transports, the total number of consultations and teleconsultations and the total number of teleexpertise sessions, and their respective costs and durations, are calculated by multiplying the terms that influence each of them (Figure 2).

As telehealth is a complex system, the variables are interrelated. For example, the average number of specialized consultations per patient per year has an influence on total medical costs, but also on total France; over the past 36 months, more than 1,500 telemedicine sessions in 21 medical specialities were conducted. Even with this high number, the data available were limited and not as high quality as would be desirable because of the small scale of the project compared to a territory implementation.

However, this is the most reliable data in France.

Cost inflation over the 20-year time horizon was not taken into account. In fact, the simulations were run ceteris paribus (all other things being equal). This enabled fair comparison of scenarios to ensure valid results and proper recommendations. Tables 1 and2 quantify the exogenous variables and model parameters with their data sources.

Exogenous variable

Number of people in the territory

In the Picardie region, there were 1,950,000 people in 2010 and a linear projection predicted the population would reach 2,050,000 people in 2030.

Figure 3 shows this increase over time.

(INSEE 2010)

Table 1: Dynamic exogenous variables changing all along the 20 time year's span

The trends related to these exogenous variables are given in Figure 3 and4. Other parameters considered in this model are given in Table 2. For each of them, a quantification is given as well as the source of information. Table 2: Parameters for the 20-year time horizon

Variables

SENSIBILITY ANALYSIS

In order to investigate the robustness of the model, a sensitivity analysis has been conducted with regard to five variables defining proposed scenarios. The change has been defined as uniform distribution with the variation of 10% depending on the case on each of the variables. The effects have been observed on the specialized medical time available, CO2 emissions equivalent and total system cost. Results are given in We can note that the variables that have the greatest influence on the cost of the system are in descending order: (1) number of specialized consultations due to better territorial access, (2) number of admissions to the emergency department, (3) number of specialized consultations due to better patient triage, (4) number of patient transports. The impact on the average duration of consultation has no influence on the cost indicator.

Concerning the Carbon Dioxide Equivalent emitted, we can note that the number of patient transports, number of specialized consultations due to better patient triage and number of specialized consultations due to better territorial access are linearly correlated.

Concerning, the number of specialized consultations due to better territorial access have been found to have a large influence on the total medical time available indicator. However, whatever the change in input variables, the overall relative comparison of scenarios stay the same.

TELHEALTH DEPLOYMENT SCENARIOS

The research objective of this model is to support policy makers understand different impacts of telehealth implementation strategies and to identify those that can contribute greatly to system performance represented as indicators (section 4.4). Four scenarios were proposed by the working group based on their past experiences and possible degree of telehealth system integration (Table 4):

(1) A statu quo scenario : telehealth system is not deployed at all;

(2) Teleconsultation scenario: Teleconsultation are implemented between different institutions in the region. Teleconsultation may cause a reduction of transport and of the average duration of consultations. However, an increase in the number of consultations is expected due to a better territorial access ;

(3) Teleexpertise scenario: Teleexpertises concerns having an expert opinion between medical institutions. Wanted impact can be a reduction of the number of emergency admissions and the number of specialist consultations through better triage ;

(4) Teleconsultation and teleexpertise scenario: Teleconsultation and teleexpertise between different institutions in the region are implemented. It is hoped that a reduction of the transport, of the average duration of consultations, of the number of emergency admissions and of the number of specialist consultations through better triage. However, this reduction will be offset by a rise in the number of consultations due to a better territorial access. As possible deployment hypothesis, a 4 year period of telehealth stations deployment has been considered, i.e. time necessary to equip all 200 hospitals needed has been considered to be 4 years (Figure 5). Figure 5 represents the overall usage rate of telehealth stations over time and the impact rate over time delayed with an average of 18-month. Concerning the economic aspect, one can observe in Figure 6 that in the statu quo scenario the total healthcare cost increase continuously. In all other scenarios, in which telehealth stations are implemented in the region, the value of this indicator is less than the reference status quo scenario where telehealth is not deployed. Impact rate over time : Teleexpertise Overall usage rate of telehealth stations over time : Statu Quo Overall usage rate of telehealth stations over time : Teleconsultation

6 6 6
Overall usage rate of telehealth stations over time : Teleexpertise

7 7 7
Overall usage rate of telehealth stations over time : Teleconsultation + Teleexpertise 8 8

as the population is increasing. One can observe that overall healthcare cost increase in period between 2014 and 2018. This is due to the costs related to the acquisition of the telehealth stations and not being able to use their maximum capacities. However, all scenarios show a decrease of overall costs after that period. It can be observed that, from 2020 Scenario 2 (teleconsultation) saves 1% per year, Scenarios 3 (teleexpertise) saves 1.2% per year and scenario 4 (teleconsultation and teleexpertise) saves 3.1% per year compared to the statu quo scenario. It seems that the most interesting scenario is scenario 4 where both teleconsultation and teleexpertise are deployed. Total healthcare cost : Statu Quo

1 1 1 1 1 1 1 1 1 1 1 Total healthcare cost : Teleconsultation 2 2 2 2 2 2 2 2 2 2 2
Total healthcare cost : Teleexpertise

18

(teleconsultation and teleexpertise) it is reduced by 18.4% per year compared to the statu quo scenario.

The decrease of Carbon Dioxide Equivalent emitted in scenarios 2 is due to the reduction of patient transport, the decrease in scenario 3 is due to a better prevention with a higher triage of patient and the decrease in scenario 4 is due to both reduction of patient transport and better prevention with a higher triage of patient. Concerning the social aspect, the total medical time indicator is estimated by calculating the difference between the total number of medical specialists hours available (supply) and the total number of needed hours with regard to the population (demand) (section 6.2). There are several causes for negative values in this analysis (Figure 8). One is due to the replacement rate for medical specialists.

This shortage of medical specialists is something that has been discussed by the French government and is related to the decision about the numerous clausus. In this case, what is observed also in real situation is an increasing delay for specialist examinations due to this shortage. As this number has been rectified we can observe a general increasing tendency after 2024. In Figure 8 Total CO2 equivalent : Teleexpertise

Discussion

In this work, we investigated the effects of several potential telehealth implementation scenarios defined based on discussions with healthcare managers. With regard to the simulations presented in sections 10, Scenario 3 (teleexpertise) will yield the best outcomes related to the total medical time available and scenario 4 (teleconsultation and teleexpertise) will yield the best outcomes with regard to total healthcare cost and Carbon Dioxide Equivalent emitted.

Saving medical time of specialist practitioners is judged as a priority by the French Ministry of Health (Ministère des affaires sociales et de la santé 2012). Therefore it seems that the most promising possibility is to put the priority in teleexpertise development. Nevertheless, deploying teleconsultation and teleexpertise is also a possibility. It will be less influential on saving time, but more concerning the economic and environmental aspects. CONCLUSIONS, GENERALIZATION AND FUTURE WORKS Healthcare systems are facing intense challenges to maintain quality and fairness. In the face of severe budget restrictions, solutions must be found to solve the supply-demand problem of healthcare as exemplified by some regions in France, where the number of specialized practitioners is decreasing while the population continues to grow. Telehealth is seen as a possible way to address these challenges. It provides remote access to healthcare services, thus bridging the physical distance between patients and the healthcare professionals best suited to helping them.

Different strategies of implementation can be applied. Our study simulated 4 different implementation scenarios (statu quo, teleconsultation, teleexpertise and both) in order to understand which one will yield the best outcomes as defined by experts in the field. We organized work sessions with a group of practitioners, medical staff and healthcare managers in order to identify the direct impacts that telehealth implementation will have on the system. Five potential impacts were retained in the model:

the number of admissions to the emergency department, the number of transports, the number of specialized consultations due to better patient triage, the number of specialized consultations due to better territorial access, and the average duration of consultation. To identify the influence of an implementation strategy on the healthcare system, three indicators were selected: economic aspect (total healthcare cost), environmental impact (Carbon Dioxide Equivalent emitted) and social impact (total medical time available). The perimeter of this study was the Picardie region and a 20-year time horizon.

We chose to use a parametric scenario-based model based on design principles that enables formal computer simulations of complex systems to be built and used in the design of policies aimed at increasing the effectiveness of organizations. Furthermore, we chose it because its visual nature facilitated understanding of the model in discussions with the expert group.

The results of these simulations compare strategic telehealth implementation scenarios. They take into account the number of telehealth stations and their effective use over time, and provide decision support in order to reduce healthcare costs, reduce Carbon Dioxide Equivalent and save valuable medical time in this region. Based on the results presented, teleexpertise scenario will yield the best outcomes to save medical time available and teleconsultation and teleexpertise scenario will yield the best outcomes to save healthcare cost and Carbon Dioxide Equivalent emitted. As the French Ministry of Health gives priority to save medical time of specialist practitioners, our recommendation is to make a priority of developing teleexpertise. Nevertheless, deploying teleconsultation and teleexpertise

  patient transport costs and on the total medical time available and on the Carbon Dioxide Equivalent emitted . QUANTITATIVE DATA COLLECTION Rigorous data collection provided the quantitative parameters of the model. Most of the data were extracted from official reports of French Agencies. The other values are based on the existing telehealth experiment at Georges Pompidou Hospital. It is the largest telehealth implementation in

Figure 3 :Figure 4 :

 34 Figure 3: Number of specialists in the Picardie region, calculated from (DREES 2009) and (CNOM 2009)

Figure 5 :

 5 Figure5: Overall usage rate of telehealth stations over time(left) and impact rate over time(right)

Figure 6 :

 6 Figure 6: Total healthcare cost evolution over 20 years (in euros)

Figure 7 :

 7 Figure 7: Carbon Dioxide Equivalent emitted over 20 year period (in gram of CO2e)

  reinforce the problem of supply demand. Only Scenario 3 (teleexpertise) and Scenario 4 (teleconsultation and teleexpertise) are better, with the first one the best.

Figure 8 :

 8 Figure 8: Total medical time available over 20 year period (in hours)

medical cost <Time> Duration allocated by specialist practitioners for consultation Average number of specialized consultations per patient per year Consultation and teleconsultation cost Emergency cost Patient transport cost Expected rate of avoided patient transports over time Expected rate of avoided emergency department admissions over time Expected rate of patient triage through teleexpertise use over time Expected decrease on the average duration of consultation Average number of visits to the emergency department per patient per year Expected growth rate of consultations due to a better territorial access over time

	<Time>
				Impact rate
				over time
	Expected rate of	Expected rate of	Expected rate of	Expected growth rate of	Expected rate of the
	avoided emergency	avoided patient	patient triage through	consultations due to a	average consultation
	department admissions	transports	teleexpertise use	better territorial access	duration
	Maximum number of telehealth stations in the territory	Ratio of patients transported for a consultation			Ratio of teleexpertise use that avoids a consultation
	Number of stations in the territory				Number of teleexpertise
	Initial cost of a					Teleconsultation
	telehealth station					duration
						Consultation
						duration
	Annual cost of a telehealth station	Teleexpertise cost			Teleexpertise duration
	Total telehealth station cost	Average patient transport distance
			per km	Total teleexpertise	Total consultation
		gram of Carbon Dioxide Equivalent emitted per		Total duration allocated by specialist practitioners	duration	and teleconsultation duration
		km per year		for consultation
		Carbon Dioxide	Total medical time
		Equivalent emitted	available

Table 3

 3

		Imput sensitivity range (uniform law)	Impact on the total healthcare cost	Impact on the total Carbon Dioxide Equivalent emitted	Impact on the total medical time available
	Number of admissions
	to the emergency	0% to -10%	0% to -3,5%	0%	0%
	department
	Number of patient transports	0% to -10%	0% to -1.9%	0% to -10%	0%
	Number of specialized
	consultations due to	0% to -10%	0% to -3.3%	0% to -10%	0% to 37,3%
	better patient triage
	Number of specialized
	consultations due to	0% to +10%	0% to 7.1%	0% to +10%	0% to >100%
	better territorial access
	Average duration of consultation	0% to -10%	0%	0%	0% to 23,5%

Table 3 :

 3 Sensitivity analysis

Table 4 :

 4 Scenario quantification

  The general tendency is overall cost increase in time in all scenarios

									Selected Variables
	1								8 7	7 6	8 2	4 3	8 7 6	4 3 2	7 6	8 2	4 3	8 7 6	3 2	6 4
	1								4 3 2
	0.8
	0.8							6
	0.6
	0.6
	0.4
	0.4							4
	0.2 0.2					8 7 6
							3 2
	0
	0	2010	3 2 1	7 6 5 4 2012	8 2 1	5 4 3 2014	1 2016	5	1 2018	5 2020	1	5 2022		1 2024	5	1 2026	5	2028	1	5 2030
									Time (Year)
	Impact rate over time : Statu Quo	1		1			1		1		1			1
	Impact rate over time : Teleconsultation 2		2			2		2			2			2

  , as in previous figures, the statu quo scenario is presented by the first function. We can see that Scenario 2

							Total CO2 equivalent
	3.228 B
	3.101 B	3 2	4 2	4 3	4 3 2	3 2	4	3	3	3	3	3		3	3	3		3	3	3
	2.974 B
								4
	2.848 B
									4
	2.721 B
	2.595 B 2010		2012		2014	2016		2018	4 2020	4	2022	4	4 2024	4	2026	4	4 2028	4	2030
										Time (Year)
	Total CO2 equivalent : Statu Quo	1		1	1	1	1	1		1	1	1		1	1
	Total CO2 equivalent : Teleconsultation 2	2	2	2	2	2		2		2	2	2	2

Taylor, K., Lane, D., 1998. Simulation applied to health services: opportunities for applying the system dynamics approach. J Health Serv Res Policy 3, 226-232. WHO, 2000. WHO | The world health report 2000 -Health systems: improving performance. Zare Mehrjerdi, Y., 2012. A System Dynamics Approach to Healthcare Cost Control. International Journal of Industrial Engineering & Production Research 23, 175-185. Zeigler, B.P., Kim, T.G., Praehofer, H., 2000. Theory of Modeling and Simulation: Integrating Discrete Event and Continuous Complex Dynamic Systems, 2nd Edition. ed. Academic Press Inc.

LIST OF ACRONYMS

CO2e

Carbon dioxide equivalent


Yun Ye
email: yun.ye@ecp.fr

Dr Marija Jankovic
email: marija.jankovic@ecp.fr

Gül E Kremer
email: gkremer@psu.edu

Jean-Claude Bocquet
email: jean-claude.bocquet@ecp.fr

Managing uncertainty in potential supplieridentification

Keywords: complex system design, modularity, potential supplier identification, uncertainty management

Managing uncertainty in potential supplier identification

As a benefit of modularization of complex systems, Original Equipment Manufacturers (OEMs)can choose suppliers in a less constricted way when faced with new or evolving requirements. Before a supplier is selected for each module, a group of potential suppliers should be identified in order to control the uncertaintyalong with other performance measures of the new system development. In modular design,because suppliers are more involved in system development, the potential supplier identification should be combined with potential architecture identification to manage uncertainty as well as requirements satisfaction level bypotential suppliers. In this paper, we propose an Architecture &Supplier Identification Tool (ASIT), which generates all possible architectures and corresponding suppliers based on new requirements through matrixmapping and propagation. Using ASIT, the overall uncertainty and requirements satisfaction ofgenerated architectures can be estimated. The proposed method aims at providing decision support for the early design stageand thereby helping companies to have an integrated view of suppliers and system architectures to better satisfy customer requirements and reduce risk.

Introduction

In order to reduce complexity and increase manageability of complex systems, one of the principles used in systems engineering is to cluster system elements into larger chunks [START_REF] Chiriac | Level of Modularity and Different Levels of System Granularity[END_REF]; this is known as modularization.For diminishing or managing time-to-market and cost, the design and manufacturing of modules is often outsourced to different suppliers. The supplier companies and the OEM together form an extended enterprise [START_REF] Nguyen | System engineering for collaborative data management systems: Application to design/simulation loops[END_REF].

In modular design,interfaces shared among modules in a given system architecture are usually specified and standardized [START_REF] Ro | Modularity as a Strategy for Supply Chain Coordination: The Case of U.S. Auto[END_REF],so that changes in one module of the system do not require changes in other parts of the system [START_REF] Hoetker | Do modular products lead to modular organizations?[END_REF]. This affords OEMthe ability to choose suppliers more freely vis-à-vis the evolving system requirements.

Before choosing suppliers for a new system, the OEM needs to first identify a group of potential suppliers, let them submit proposals, and then choose a suitable supplier for each module after negotiation.Normally, OEMs tend to use those suppliers with which they have a prior history of cooperation,since past interactions usually improve communication between buyer and suppliers [START_REF] Levinthal | History Dependence in Professional Relationships : Ties That Bind[END_REF][START_REF] Singh | Precarious collaboration: Business survival after partners shut down or form new partnerships[END_REF]. This leads to faster, lower cost procurement and more successful system development [START_REF] Hoetker | How much you know versus how well I know you: selecting a supplier for a technically innovative component[END_REF]. However, modules from prior suppliers maynot always satisfy all buyer requirements. In such situations, the OEM needs to find new suppliers of suitable new modules and technical capabilities.New suppliers and new modules add uncertainty due to various reasons (e.g., supplier"s capabilities to cooperate well with the OEM, technological uncertainty ofnew modules, and the uncertaincompatibilitybetween modules).

Uncertainty can impact decision-making as attested by several studies. For example, [START_REF] Janssen | The effect of modelling expert knowledge and uncertainty on multicriteria decision making: a river management case study[END_REF] assessed the influence of presenting data with or without the uncertainty information on decision-making. A statistically significant shift in preferences was observed when uncertainty information was presented. Uncertainty integration was also found important in system architecture generation [START_REF] Marie-Lise | Product architecture generation and exploration using bayesian networks[END_REF].In complex system design, suppliers are more and more tightly integrated intodesign process [START_REF] Le Dain | Measuring supplier performance in collaborative design: proposition of a framework[END_REF]. Consideration of interfaces [START_REF] Tripathy | Organizing Global Product Development for Complex Engineered Systems[END_REF], cost reduction [START_REF] Nepal | Matching product architecture with supply chain design[END_REF], platform policy [START_REF] Zhang | Simultaneous configuration of platform products and manufacturing supply chains[END_REF] andnew technology integration [START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF]require integrating suppliers startingin early design stages. However, very few methods considereduncertaintywhile integrating assessment of supplier capabilities in system architecture generation. By proposing an Architecture & Supplier Identification Tool (ASIT),we respond to the need forcontrolling overall system uncertainty when combining architecture generation and supplier identification.

In this paper, section 2 addresses different concepts of modularity as well as approaches that are specifically designed for supplier selection. Section 3 discusses different types of uncertainty, and argues forthe need to integrate uncertainty information in early design. An overall ASIT process is presented and discussed. In section 4,a case study on powertrain design is used to illustratethe ASIT. In order to studyifthe consideration of uncertainty changes choices made in supplier identification, we also compare ASIT with Concept Selection Method (CSM) by [START_REF] King | Development of a Methodology for Concept Selection in Flexible Design Strategies[END_REF]. CSM is a well-known deterministic approach for concept evaluation that does not consider overall uncertainty. The difference in results along with use contextsof these two approaches is discussed in section 5.

Finally, we providea discussion and our conclusions in section 6 and 7, respectively.

Background

Modularity in complex systems

A complex system is a system with numerous components and interconnections, interactions or interdependencies that are difficult to describe, understand, predict, manage, design, and/or change [START_REF] Magee | Complex system classification[END_REF]. Systems and complex systems can be decomposed into different levels of modules, and the number of modules increases as the grain size of modules decreases [START_REF] Chiriac | Level of Modularity and Different Levels of System Granularity[END_REF]. An example of decomposition of a vehicle system is shown in Figure 1.In this context, a module is defined as a chunk if it is tightly coupled within and loosely connected to the rest of the system [START_REF] Gershenson | Product modularity: Definitions and benefits[END_REF]. Normally, different levels of modules are also systems or complex systems themselves.Therefore, the method proposed in this paperapplies for systems and complex systems at any level of theirdecomposition. For example, the case study illustrates method application on a powertrain, which is a complex system, and alsoa first level module in Figure 1.

Modularity in buyersupplier relations

According to [START_REF] Fixson | The power of integrality: Linkages between product architecture, innovation, and industry structure[END_REF], there is a substantial literature streamsuggesting that many products are becoming more modularover time; andthe modularity ofproducts leads to modularity of organizations [START_REF] Garud | Managing in the Modular Age: Architectures, Networks, and Organizations[END_REF]. For example, in their empirical work [START_REF] Ro | Modularity as a Strategy for Supply Chain Coordination: The Case of U.S. Auto[END_REF]found that the emergence of modularity in product designis changing the structure of the extended enterprises in American auto industry.The traditional U.S. supplier management model is as shown on the left of Figure 2. According to [START_REF] Ro | Modularity as a Strategy for Supply Chain Coordination: The Case of U.S. Auto[END_REF], in the traditional supplier management model, the parent department (e.g., chassis department) is further divided into more specialized functional departments (e.g., suspension, steering and braking). Each of the functions is undertaken by an OEM release engineer who manages the first-tier suppliers. In this case, the OEM directly interacts with their suppliers.

The desired form of the U.S. supplier model is called "the systems integrator model" [START_REF] Ro | Modularity as a Strategy for Supply Chain Coordination: The Case of U.S. Auto[END_REF], which is shown on the right of Figure 2. In this form, a lead supplier manages and coordinates the design and assembly of large-scale modules and systems across a number of other suppliers.In this case, the OEM needs to communicate only with the integrator suppliers, i.e. OEMis concerned about the high level modules (e.g., chassis, powertrain, etc.). The integrator suppliers work more independently in this case, and the structure of the extended enterprise is more loose and flexible, implying the formation of a "modular organization".

Figure 2 Influence of modularity on supplier management model (adopted from [START_REF] Ro | Evolving Models of Supplier Involvement in Design: The Deterioration of the Japanese Model in U.S. Auto[END_REF] The ease of reconfiguration of organizational actors in modular organizations allows "modular innovation", by which firms improve their products by incorporating improvements in various product modulesthat may occur at different rates for different modules [START_REF] Langlois | Firms, Markets and Economic Change: A dynamic Theory of Business Institutions[END_REF]. It also allows a firm to select the best supplier for a given module at a given time [START_REF] Garud | Technological and organizational designs for realizing economies of substitution[END_REF]. The proposed supplier identification tool in this paper assumes that the context in which the tool is used reflects the above mentioned buyersupplier relations and that the tool is proposed as a decision support tool for the said context. [START_REF] Petersen | Supplier integration into new product development: coordinating product, process and supply chain design[END_REF] demonstrated that "A careful and complete analysis of potential suppliers, leading to the selection of a supplier with the right capabilities and culture to work on the project was positively associated with effective decision making by the project team during the new product development process". There are hundreds of prior works concerningsupplier identification and selection. Most of the supplier selection methods are provided under the traditional product development decision making process, i.e., first the product architecture is fixed by the OEM, then production/manufacturing method is decided; based on these decisions, suppliers are selected [START_REF] Nepal | Matching product architecture with supply chain design[END_REF]. In these methods, product architectures are fixed before supplier selection. The supplier selection is usually based upon financial and managerial criteria such as quality, cost, delivery and performance, etc. In early design, however, such data is not necessarily available and is also uncertain. Reviews of supplier selection criteria can be found in [START_REF] Ha | A hybrid approach to supplier selection for the maintenance of a competitive supply chain[END_REF], [START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF] and [START_REF] Ye | Main Factor Identification for Early Negotiation in Product Design[END_REF]. The publishedsupplier selection methods under this context are oftenMulti-Criteria Decision Making (MCDM) methods using mathematical, statistical, artificial intelligent or a combination of these methods. Surveys of these methods can be found in various publications such as byde [START_REF] De Boer | A review of methods supporting supplier selection[END_REF], [START_REF] Ha | A hybrid approach to supplier selection for the maintenance of a competitive supply chain[END_REF] and [START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF].

Supplier identification and selection methods

Because of the emergence of modularity, suppliers are more involved in the product design phase. Therefore, companies started to consider supply chain issues during product development.For example, studies were carried out for matching product development and supply chain design. [START_REF] Ülkü | Matching Product Architecture and Supply Chain Configuration[END_REF] studied the matching between product modularity leveland the supply chain configurations, i.e. the buyer-supplier collaboration level during product design. [START_REF] Pero | A framework for the alignment of new product development and supply chains[END_REF]studied how new product development and supply chain variables were related to each other;they found that innovativeness had a strong effect on supply chain complexity and matching product features with supply chains improved performance.Some methods are also provided to address product development and supply chain issues simultaneously. [START_REF] Lamothe | An optimization model for selecting a product family and designing its supply chain[END_REF] proposed a mixed integer linear programming model to helpchoose product family variants in a way that the operating cost of the supply chain delivering the product is optimized.More specifically, we have foundthree studies that consider product design and supplier selection conjointly. [START_REF] Zhang | Simultaneous configuration of platform products and manufacturing supply chains[END_REF] developed a mixed integer linear programming model to support product platform design. The main objective was to balance the commonality and variety of the product platform. The suppliers were considered simultaneously with product platformto reduce cost. [START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF] proposed a graph theory based method considering product design and supply design simultaneously. In their work, product functions, assembly issues, and supply chain performance wereconsidered in early product design stage. The main objective was to optimize product cost and lead-time. [START_REF] Nepal | Matching product architecture with supply chain design[END_REF] proposed a fuzzy logic based framework to tackle product design and supply chain design at the same time. Their objective was to minimize the total supply chain costs, and maximize total supply chain compatibility.Therelevant state-of-the-art in concurrent product and supply chain design is summarized by [START_REF] Gan | Concurrent Product -Supply Chain Design: A Conceptual Framework & Literature Review[END_REF].

As it can be seen above, among existing studies, [START_REF] Zhang | Simultaneous configuration of platform products and manufacturing supply chains[END_REF], [START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF][START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF][START_REF] Nepal | Matching product architecture with supply chain design[END_REF] addressed product architecture generation and supplier selection simultaneously. All of these consider the cost issue as their main objective. [START_REF] Chiu | An Integrative Methodology for Product and Supply Chain Design Decisions at the Product Design Stage[END_REF]Okudan (2011) also considered lead-time, and[START_REF] Nepal | Matching product architecture with supply chain design[END_REF] tackled supply chain compatibility issue. However, none of the existing works considered overall uncertainty,whichisan important issue in early design. Because of the frequent high level innovation integration and uncertainty in early complex system design, we address this gap.

Proposition for SupplierInformation Integration

Uncertainty sources in supplier identification

De [START_REF] De Weck | A Classification of Uncertainty for Early Product and System Design[END_REF] defined uncertainty as "an amorphous concept that is used to express both the probability that certain assumptions made during design are incorrect as well as the presence of entirely unknown facts that might have a bearing on the future state of a product or system and its success in the marketplace. [START_REF] Funtowicz | Science for the post-normal age[END_REF] stated that the uncertainty comprises information about the simplifications made during the translation of a natural system into a model. Many previous works classified uncertainty for early product and system design (e.g., [START_REF] Clarkson | Design Process Improvement: A Review of Current Practice[END_REF][START_REF] Mcmanus | A framework for understanding uncertainty and its mitigation and exploitation in complex systems[END_REF][START_REF] De Weck | A Classification of Uncertainty for Early Product and System Design[END_REF]. The risk management in early design was also investigated by [START_REF] Lough | The risk in early design method[END_REF][START_REF] Van Wie | An analysis of risk and function information in early stage design[END_REF], [START_REF] Altabbakh | Variations in Risk Management Models: A Comparative Study of the Space Shuttle Challenger Disaster[END_REF] and others.In the context of this work, we consider the underlying uncertainty of choosing new suppliers and new modules during supplier identification.

We identified three sources of uncertainty inusing new suppliers and modules:

(1)uncertainty related to suppliers" capabilities to cooperate well with the OEM, (2) the probability that a module can be successfully developed, and (3) the compatibility between the modules. For example, supplier A may be able to provide a module B which potentially satisfies the requirements well. However, in reality, the supplier A may not be able to cooperate well with the OEM, and thus the module B may not be successfully developed.

Moreover, even though the module B is developed, it may not be compatible with other modules. The uncertainty of all these issues should be considered when reviewing the high satisfaction score of supplier A.

Architecture and Supplier Identification method

In order to integrate previously discussed system architecture uncertainties conjointly with supplier capability relateduncertainties, we propose an Architecture & Supplier Identification Tool (ASIT). ASIT is a matrix-basedmethod containing information related to requirements, functions, modules, suppliers, and uncertainties. The main objective is to support decision making of the design team in architecture generation and supplier identification.Figure 3 presents an overview of the ASIT, which contains four phases that are automated by a MatLab program. Therefore, project documents regarding the requirements, functions, and modules usually exist; thus, design information is captured and reused.This information capture and reuse is often facilitated through software (e.g., DOORS). However,various types of data are rarely stored in one place. The idea of ASIT is to store critical, high-level data (pertaining to functions but also requirements, modules, and other types of information) on previous projects within amatrix system.The matrix system is composed of aDesign Structure Matrice (DSM) and sixDomain Mapping Matrices (DMMs), as shown in Figure 4. When starting a new project, usually the project manager organizes a one to three day workshopto discuss innovation integration, different system architectures, as well as other constrains.These workshops are attended by experts of different domains in order to cover overall system knowledge.With the support of the matrix system in ASIT, the expertscan choose the adequate existing requirements from the list, and if necessary add new requirements to it. Based upon the requirement-function relations stored in matrix 1, the existing functions related to the defined requirements can be found. Fundamentally, this is a cognitive phase tackling new and existing requirements, where experts will discuss and allocate them to existing functions or create new functions. The functions, in this context, can be seen as translations of requirements to technical language, describing what the module/system should do from a technical point of view. Experts also discuss module typesthat are needed based on functions,and relations between new functions and module types.Matrices 1 and 2 can be updated after these discussions. The main difficulty inthis phase is the expression of requirements and functions due tovarious semanticpossibilities. Here, we assume that designers/engineers are able to define and use a shared language and understanding. Clearly, semantic consistency in reference to functions, etc. is needed.

After the update of matrices 1 and 2, the ASIT can automatically point to (calculate) unsatisfied functions by existing systems. In phase one (using matrices 1,2 and 7), how well OEM"s existing products (e.g., existing powertrains) satisfy the requirements can be estimated.

Subsequently, the functions that cannot be satisfied are identified based on requirement-function relations in matrix 1. In phase two, new modules that can potentially satisfy the unsatisfied functions and new suppliers are found externally, or proposed by experts, and thereby updating matrices 2, 3, 4, 5, and 6. Then,ASIT can automaticallygenerateall possible architectures based on the functionmodule relationsprovidedin matrix 2. In phase three, uncertainty of generated architectures is calculatedbased on uncertainty of modules, compatibility between modules, and uncertainty of suppliers" capabilities. The needed information is stored in matrices 4, 5, and 6.The requirements satisfaction by generated architectures is calculated. Finally, in phase four, using the uncertainty threshold and the requirements satisfaction threshold defined by the experts, the generated architectures are filtered by the ASIT to identify potential architectures and corresponding suppliers.

As explained above, information stored in the matrix system comes from two sources:

(1) information estimated by experts, and (2) information from existing products.For expert estimation, a group of experts work together to provide estimation by using predefined levels (as shown in Figure 5 and6). The information on existing products is considered already stored in the matrix system;because after each project, the related data in the matrix system is updated based on project outcomes. Experts need to provide four types of information during the process of the ASIT. The first type is of percentages used in matrix 1, representing the level a function fulfills a requirement. The second type is about satisfaction levels used in matrix 2, representing how well a module satisfies a function. The third type is probabilities as defined inFigure 6,describinguncertainties in matrices 4, 5, and 6. The last type of information is binary, and is used to define whether one element belongs to another element (matrices 3 and 7) .

The satisfaction levels used in the ASIT are defined as "interval scales" [START_REF] Stevens | On the Theory of Scales of Measurement[END_REF], so that operations such as addition, subtraction, multiplication by a real number are meaningful. Ten levels (1-10) are used for representing satisfaction, "1" is defined as "very inadequate solution", and "10" is defined as "ideal solution". The unit of measurement is 1/10 of the satisfaction difference between "1" and "10". The descriptive meanings for the satisfaction levels are adapted from Fiod-Neto and Back"s parameter value scores (Fiod-Neto and Back, 1994, pp. 35-45), and are shown in Figure 5. "0" is used to represent "the module does not provide the function". During workshops, experts use the linguistic terms in Figure 5 to provide their estimations, then the linguistic terms are quantified using 1-10 scale equivalents.

Figure 5 Linguistic terms for satisfaction levels (adopted from (Fiod-Neto and Back, 1994))

Experts always use linguistic terms to provide their estimation of probabilities [START_REF] Meyer | Eliciting and Analyzing Expert Judgment: A Practical Guide[END_REF]. Many previous works provided natural language terms associated with probabilities (e.g., Boehm, 1989, p. 133;Conrow, 2003, pp. 491-513;Hamm,1991;[START_REF] Lichtenstein | Empirical scaling of common verbal phrases associated with numerical probabilities[END_REF][START_REF] Moore | The business of risk[END_REF]. However, in these previous works, the proposed probability-related terms were different [START_REF] Hillson | Describing probability: The limitations of natural language[END_REF]. Therefore, based on linguistic terms listed in works of [START_REF] Hillson | Describing probability: The limitations of natural language[END_REF] and [START_REF] Halliwell | Linguistic probabilities: theory and application[END_REF], we propose a list of linguistic terms as shown in Figure 6. The experts provide their estimations using these linguistic terms.

Figure 6 Linguistic terms for probabilities ASIT leverages the use of matrices as a database form in early design for modular complex systems; this has two main advantages.First is that because only the first tier suppliers are considered, the number of module is limited, and thus the usage of matrices is practical. The explicit form of matrices makes the relations between elements clear, facilitating comprehension and communicationbetween experts. In addition, the storage of two dimension matrices does not require specific techniques. This flexibility enables companies to continue using tools that they are familiar with. Standardization in terms of the vocabulary used while describing requirements, functions, etc. will ensure consistency; and it is assumed that company-centric standard vocabulary will be used while filling the ASIT matrices.

The main mapping flow in ASIT is requirementfunctionmodulesupplierrelations as shown in matrices 1, 2 and 3. There are prior works that also use matrix systems, such as the Quality Function Deployment (QFD),the concept selection method [START_REF] King | Development of a Methodology for Concept Selection in Flexible Design Strategies[END_REF], the architecture generation method [START_REF] Bryant | A Computational Technique for Concept Generation[END_REF], and the multiple-domain design scorecard method [START_REF] Jankovic | Multiple-domain design scorecards: a method for architecture generation and evaluation through interface characterisation[END_REF].In QFD, the customer requirementsdesign requirementsparts designmanufacturing processbuild quantity relations areused to transform customer desires into design characteristics for each stage of the product development [START_REF] Rosenthal | Effective Product Design and Development: How to Cut Lead Time and Increase Customer Satisfaction[END_REF].In the methods developed by [START_REF] King | Development of a Methodology for Concept Selection in Flexible Design Strategies[END_REF] and [START_REF] Bryant | A Computational Technique for Concept Generation[END_REF], the functioncomponentrelations are used to generate all possible concepts.InJankovic et al. ( 2012)"s method, the functionstructureperformance relations are used to generate the six design assessment cards.Across all of these matrix-based methods, ASIT is the first one to incorporate supplier information and uncertainty.Our approach in potential supplier identification also features a "variable" view of the design (i.e., design is not fixed). In addition, uncertainties related to various systemsupplier aspects are considered.

Implementation

Case Study Description

In this paper, we focus on the powertrain design for a motor vehicle.Due to innovation integration as well as fuzziness in early design [START_REF] Marie-Lise | Product architecture generation and exploration using bayesian networks[END_REF][START_REF] De Weck | A Classification of Uncertainty for Early Product and System Design[END_REF],a vehicle is usually decomposed into two or three levels of subsystemsat this stage. The powertrain is one of the high-level subsystems that can be further decomposed. The main objective in designing a powertrain is to provide adequate propulsion with minimal use of fuel while emitting minimal hazardous by-products or pollutants.For the sake of simplicity, only gas engine and hybrid engine architecturesare considered in this case study.

A powertrain is a system of mechanical parts in a vehicle that first provides energy, then converts it in order to propel the vehicle. In a traditional gas-engine vehicle, the engineprovides power converting from other sources of energy. The transmission then takes the power, or output, of the engine and, through specific gear ratios, slows it and transmits it as torque. Through the driveshaft, the engine"s torque is transmitted to the final drive (wheels, continuous track, etc.) of the car. In hybrid electric vehicles, besides the four modules mentioned above, batteriesprovideelectricalenergy, and electric motors are used to transform electric energy into torque. Therefore in this study, we consider six types of modules including engine, battery, transmission, electric motor, driveshaft and final drive. A powertrain system of a modern automobile is shown in Figure 7.

Figure 7Powertrain system of a modern automobile [START_REF] Akehurst | Lean Powertrain Development[END_REF] In general, when considering couplings that exist within apowertrain system as well as new architectures that are emerging due to new technologies (e.g., hybrid and electric vehicle), the powertrain can be considered as a complex system. [START_REF] Michelena | A Hypergraph Framework to Optimal Model-Based Partitioning of Design Problems[END_REF] also stated that "in practice, this task is completed incrementally by trial and error and is costly and time consuming." Further, as a system of variables to be optimized it is overwhelming; [START_REF] Wagner | A general decomposition methodology for optimal system design[END_REF]showed through tested mathematic models that a powertrain system design can have 87 design relations, 127 variables, and 57 degrees of freedom.However, inthis case study, not all subsystems and technologies are considered in order to simplify the explanations.

Due to the increasing demand of lower emissions and higher fuel efficiency, the OEM plans to design a new powertrain for their motor vehicle to better satisfy market needs.In this case study,the new powertrain needs to satisfy 6requirements (requirements (1) to ( 5 gases (e.g., CO2) it emits.), and (6) Rechargeable by external electric power (This requirement indicates that the OEM would like to develop a plug-in hybrid electric vehicle using rechargeable batteries, the new trend in the market). These requirements can be satisfied by certain functions (e.g., transform energy to torque), andeach of the functions is satisfied by one or more modules [START_REF] Ulrich | Product design and development[END_REF].For the new powertrain development, the OEM expects the optimum performance of the system, but at the same time, the uncertainty of the system development has to be controlled.

Phase I

For the new powertrain design, a list of requirements is defined by a group of experts. The aim of this phase is to use ASIT to calculate how OEM"s existing powertrains satisfy these requirements, and which functions are not satisfied, pointing to the need for new module development.

Existing information is stored in the ASIT matrix system. When starting a new project, the matrices M 1 and M 2 need to be updated by experts with new requirements and functions.

Identification of requirementsrequires experts first to choose appropriate existing requirements, and then add new requirements to the list, if necessary. By using the requirementfunction relations in matrix M 1 , functions that satisfy these requirements are allocated. Experts allocate newly identified requirements to existing functions or add new functions. With new requirements and functions added, the requirementfunction relations in matrix M 1 , and function satisfaction by modules in matrix M 2 are estimated by experts. The updated matrix M 1 is shown in Figure 8, where the requirement "rechargeable by external electric power" is a new requirement, and the function "accept recharge" is a new function.

Figure 8 Requirementfunction relations

The updated matrix M 2 is shown in Figure 9, where a new function is added, and module types needed are also identified by experts.

Figure 9 Function satisfaction level by modules

After M 1 and M 2 are updated, ASIT leverages information from M 1 , M 2 and M 7 (see Figure 10) to estimate requirements satisfaction by existing systems. The M 7 is an excerpt ofOEM"s existing powertrains.In this case study, the OEM successfully developed two types of powertrains in the past (i.e., regular gas engine powertrain and hybrid, as shown in matrix M 7 inFigure 10), which are used as foundationsfor the new product development. In the matrix M 7 ,"1" represents "the module belongs to the architecture", and "0" represents "the module does not belong to the architecture".

Figure 10Composition of existing powertrains

In order to propagate the function satisfaction by modules (Figure 9) to the function satisfaction by architectures, the composition of the existing powertrains (matrix M 7 , Figure 10) is considered. How a system satisfies a function depends on the capability of its relevant modules. When there is only one module in the product that is designed to satisfya function, the satisfaction level of the function by the product is consideredthe same as the satisfaction level of the function by the module. When there are multiple modules satisfying the function, the satisfaction level is defined as the average of satisfaction levels of the modules. For example, the gas powertrain has only one module (the engine 1) fulfilling the function "provide power". Therefore, if the "engine 1" satisfies the "provide power" function at level 8, the gas powertrain should also satisfy this function at level 8, as shown in matrix M fun-arch in Figure 11.The satisfaction levels here represent "how good a module is with specific regards to a function" qualitatively. The average of satisfaction levels is a simplification adopted in this work; the weights of modules for satisfying a function can also be considered.

Figure 11 Function satisfaction level by existing products

In order topropagate the satisfaction of functions to the satisfaction of requirements, the requirementfunction relations (M 1 , Figure 8) are used. Numbers in this matrix represent the percentage that a function satisfies a requirement. The sum of each row of the matrix can be greater or equal to 0 and smaller or equal to 1, since the requirements may be only partly satisfied.For propagating the satisfaction of functions to the satisfaction of requirements, we use the formula:

1 req arch fun arch M M M    
The requirements satisfaction of the existing powertrains is shown in Figure 12. We define that the level 5 represents the "satisfactory solution", and we further define that a requirement is unsatisfied if its satisfaction level is lower than 5 by at least one architecture. Therefore, the requirements "0-60 mph time", "low greenhouse gas emission", and "rechargeable by external electric power" are unsatisfied.Using M 1 inFigure 8, one can see that the requirement "0-60 mph time" is related to the function "provide power"; the requirement "low greenhouse gas emission" is related to the function "respect environment";

and the requirement "rechargeable by external electric power" is related to the function "accept recharge". Then,by using M 2 in Figure 9, one can see that the satisfaction of these three functions depends on the engine and the battery. Therefore, new engines and batteries,which can potentially satisfy these functionsneed to be identified.

Phase II

The objective of this phase is to find/propose potential new solutions for unsatisfied functions by experts, and then use ASIT to generate all possible architectures.After searching fornew modules provided either by new or existing suppliers,two new engines and two new batteries are found.Both of the two engines are from new suppliers; one of the new batteries is from a new supplier, while the other one is from an existing supplier of the OEM. Now the matrix system needs to be updated with new modules and new suppliers. Function satisfaction levels of the new modules in matrix 2, compatibility between modules in matrix 4, uncertainty of modules and suppliers in matrices 5 and 6 are provided by a group of experts, and recorded in the matrix system.

The updated function satisfaction levels of modules are as shown in M" 2 inFigure 13.

It can be seen that the two new engines perform well for functions "respect environment" and "economize fuel", but not as much for "provide power" in comparisonto existing modules.

The two new batteries perform well in satisfying "provide power" and "accept recharge" but for other functions they do not show much advantage.Based on modules shown in M" 2 in Figure 13, all possible architectures are generated. As indicated by experts during phase 1 when identifying module types, the powertrain of a plug-in hybrid electric vehicle needs an engine, a battery, a transmission, an electric motor, a driveshaft and a final drive. Therefore, by taking one module from each type of modules mentioned in M" 2 , all possible architectures are generated (see Figure 14).

Phase III

The objective of this phase is to use ASIT to calculate uncertainty and requirements satisfaction for all possible architectures.The generated possible architectures are of different satisfaction levels and uncertainty.The uncertainty of each module (represented in matrix M 5 in Figure 15), the compatibility between modules (M 4 in Figure 15), and the uncertainty of suppliers" capabilities (M 6 in Figure 15) need to be considered for calculating the uncertainty.

The matrix M 4 shows interface compatibilities between modules due to innovation integration. We define that "not compatible" is equal to "0", "perfectly compatible" is equal to "1", and a number between "0" and "1" represents the probability that the two modules work well together. Compatibility betweenmodules in existing productsisdefined as "1", whileother compatibilities are between 0 and 1. M 4 is symmetrical, and the elements describing the relations between modules, which satisfy the same function, do not need any interpretation.The matrix M 5 represents the uncertainty of modules. Similar to the definition of compatibility, we define "not mature at all" as "0" and "mature" as "1", a number between "0" and "1" represents the probability that the module can be developed successfully by the company. Similarly, the matrix M 6 represents the probability that a supplier and the OEM can work well together.The matrix M 3 represents the relations between modules and suppliers, where the number "1" represents that the supplier provides the module.

Since module uncertainty, supplier uncertainty and compatibility between modules can all be consideredinprobabilistic terms, we define the uncertainty of an architecture as the product of all its modules" uncertainties, its suppliers" uncertainties and the compatibilities between the modules. This definition anchors on the independence of probabilities. The matrices M 3 , M 4 , M 5 and M 6 are shown in Figure 15. M" 2 (in Figure 13) and the matrix M 1 (inFigure 8). In this case, we assume equal importance of the requirements (this assumption can be changed if needed), anoverall requirements satisfaction score is obtained for each of the possiblearchitectures by calculating the average of all requirements satisfaction scores for the architecture, as shown by the equation below: Bigger the number is, greater the level of confidence we have for the architecture. The uncertainty result obtained here cannot be translated directly to the possibility of success of an certain architecture. The "satisfaction" represents the satisfaction level of the requirements by an architecture. Bigger the number is, better the architecture satisfies the requirements.

Phase IV

The aim of this phase is to use ASIT to filter possible architectures by their uncertainties and the requirement satisfaction levels. The thresholds are provided by experts.

The OEM tends to keep the architectures with the best performances whilerejecting highly uncertain architectures in view of theuncertaintyrelated to the project. For this project, the uncertainty threshold is set to 0.02, and the satisfaction threshold is set to 5. Thus,all architectures with uncertainty lower than 0.02and satisfaction level lower than 5 are rejected. score for each architecture. ASIT and CSM do not entirely consider the same information;the inputs and outputs of these two methods are represented in Figure 18. The CSM and the ASIT use different scales for inputs. The CSM requires that "the total score for all modules with respect to each function to equal 1.0", and the compatibility between two modules is represented using a 0 -2 scale. In order to allow the comparison, the inputs of the two methods are normalized.

Figure 18 Main differences between the CSM and the ASIT In CSM, overall scoresfor each architecture are calculated and ranked(see figure 19).

In ASIT, only the architectures with requirements satisfaction and uncertainty above the thresholds are identified, with a set of suppliers contributing to a given architecture design (Figure 19). One can see on the left side of the table that when compatibility is considered as part of the performance, the architecture 2 receives a very high score.This is because this architecture is an existing architecture, and thus the"compatibility performance" is very high.Therefore, when adding the "function satisfaction performance" and the "compatibility performance" together, this architecture receives a good performance score although the "function satisfaction performance"alone is not good enough.We can also see thaton the left side of the table, the architecture 12 is ranked as the third best architecture. However,it isnot among the remainingarchitectureswhen considering overall uncertainty, since its uncertainty is lower than the set uncertainty threshold (0.02). Further analysis reveals that the module "battery 3" in the architecture 12is of an uncertainty value 0.2,which means that although this module can potentially provide very good performance, its development is estimated to be very uncertain, and the probability that its supplier works well with the OEM is low (0.3). The same situation can be found for other architectures such as architectures 10 and11.Uncertainty consideration implies thatcertain potential architectures (5,9,10,11,12) are eliminated, resulting in the elimination of supplier 5providing battery 3 used in architectures 9,10,11,12.

As explained before, the battery 3 provides very good performance; however, its uncertainty is very low.

Discussion

We have seen that the consideration of overall uncertainty influences the result of potential supplier identification. This is because the suppliers,which have potentially high performance but also high uncertainty, are excluded based on the risk that the OEM is willing to take on for the project. In financial terms, return is always accompanied by risk. High return options usually also have high risk. That is why the return/risk trade-offs are necessary when making financial decisions.Using the corollary in engineering design, the concepts with better performances might also have higher uncertainty. Therefore, we propose to consider both performance and uncertainty when making decisions in architecture generation and potential supplier identification.

However, there are several limitations in this work. Firstly, as an initial step towards introducing uncertainty to supplier identification combined with architecture generation, the sources of uncertainty considered in this work may not be exhaustive. Although the information used is mostly from expert estimation, we have not considered the subjectivity in expert estimation. The sensitivity of ASIT regarding this type of uncertainty should be investigated in future works to verify the robustness of this tool. Secondly, with specific regards to performance, only requirements satisfaction is considered in this paper. Many other types of performance are also important in the supplier identification, however (e.g., sustainability,product cost and lead-time,etc). We believe that the feasibility of getting these types of information in early complex system design stage needs to be considered. Thirdly, the weights of requirements, and the importance of modules for satisfying a function (when the function is satisfied by more than one module) are considered to be equal in this paper. It might be fruitful to explore using varying weights. Moreover, several studies pointed out the need for investigating the impact of preference aggregation andcollaborative expert estimation. We believe that this is an important issue and it should be tested within an industrial setting. The work of [START_REF] Clemen | Combining Probability Distributions From Experts in[END_REF] and [START_REF] Keeney | The foundations of collaborative group decisions[END_REF] set a good basis for future research in this aspect.Regarding the validation of the proposed tool, it will be necessary to test the tool in an industrial environment in the future.

Conclusions

Potential supplier identification is the phase of preparing supplier candidates for supplier selection by the OEMs. Because of the use of modular design in complex systems, the suppliers are more and more involved in system design, which makes the technical ability of suppliers more important to better satisfy system requirements. However, using novel architectures and suppliers with potential better performance often causes higher uncertainty.

In this paper, we proposed an Architecture & Supplier Identification Tool (ASIT), which uses both requirement satisfaction and uncertainty threshold to filter possible architectures and suppliers. The uncertainty related to suppliers" capabilities to cooperate well with the OEM, the technological uncertainty in new modules, and the uncertainty of compatibility between modules are considered.To the best of our knowledge, the ASIT is the first supplier identification tool that combines architecture generation, and controls the overall uncertainty during this decision-making.By comparing toa method, which does not consideruncertainty using a case study of powertrain design,it is shown that considering uncertainty impacts the result of the supplier identification, and the uncertainty should be considered independently from the performance.

Figure 1

 1 Figure1Partial decomposition of a vehicle system (adopted from Van Eikema Hommes (2008))

Figure

  Figure 3Overview of the ASIT Due to uncertainty management, complex systems are rarely designed from scratch.

Figure

  Figure 4The matrix system used in the ASIT

  ) are adapted fromMichelena and Papalambros (1995)), including: (1) fleet averaged corporate average fuel economy (CAFE) standard (Violation of this standard results in proportional fines.), (2) acceleration time (This directly relates to customer perceived performance.), (3) Cruising velocity at grade (Relates to the speed at which vehicle can climb a 6% grade in forth gear.), (4) The 0-60 mph time (This requirement relates to average speed vehicle acceleration over the speed range of the engine.), (5) Greenhouse gas emissions (This measure shows a vehicle's impact on climate change in terms of the amount of greenhouse

Figure 12

 12 Figure 12 Requirement satisfaction of existing products

Figure 13Function -

 13Function Figure 13Functionmodule relations with new modules

Figure 14

 14 Figure 14 Generated possible architectures

Figure 15

 15 Figure 15 Uncertainty information



  The obtained uncertaintiesand satisfaction levelsare presented in Figure16.Considering the lack of precision in expert estimation, only two decimal numbersare kept for the results.The "uncertainty" represents the overall uncertainty level of an architecture.

Figure 16

 16 Figure 16 Uncertainty and requirements satisfaction estimation for possible architectures

Figure 19

 19 Figure 19 Comparison of results between CSM and ASIT

Figure 1 Figure 2 Figure 6 Figure 15

 12615 Figure 1 Partial decomposition of a vehicle system (adopted from Van Eikema Hommes (2008))

After filtering, 3out of the 12 generated architectures remain(architectures 6, 7, 8), as shown in Figure 17, for final consideration.

Figure 17Uncertainty and satisfaction filtering of possible architectures

The composition of the architectures 6, 7, and 8 can be found in Figure 14, and the suppliers for the modules are recorded in matrix M 3 in Figure 15. Since the modules "engine 1", "battery 1" and "battery 3"do not belong to any of the three selected architectures, these three modules are deleted from the initial list. With regard to suppliers, only suppliers that are contributing to selected modules are kept for further consideration. That is why the supplier 5 is not considered further.

Finally, for battery, transmission, electric motor, driveshaft and final drive, only one supplier remains; for the engine, three potential suppliers are identified for further negotiation.

Comparison

There are several possibilities for comparing the ASIT method to others, includingthe method proposed [START_REF] Bryant | A Computational Technique for Concept Generation[END_REF], change propagation methodproposed by [START_REF] Clarkson | Predicting Change Propagation in Complex Design[END_REF], and risk management in early designproposed by [START_REF] Lough | The risk in early design method[END_REF]. However, in order to investigate how consideration of uncertainty changes supplier identification, we need to compare the ASIT to a similar matrix-based method that does not consider uncertainty. The Concept Selection Method (CSM) proposed by [START_REF] King | Development of a Methodology for Concept Selection in Flexible Design Strategies[END_REF] is a wellknown matrix based approach allowing ranking different conceptswith consideration of function satisfaction. The consideration of function satisfaction is rare in many complex system generation approaches, and that is why CSM was chosen for the comparison.

The CSMuses two matrices to represent function satisfaction by modules and compatibility between modules, respectively.For eacharchitecture, the summation of function satisfaction and the product of the compatibility score are multiplied, providing an overall Suppliers with potentially high performance may alsohave high uncertainty. The utilization of the ASIT in supplier identification has potential in balancing the return and the risk for the OEM, optimizingidentified suppliers.