Tone Lise Dahl
email: tone.lise.dahl@sintef.no

Manuel Oliveira
email: manuel.oliveira@sintef.no

Emrah Arica
email: emrah.arica@sintef.no

Evaluation of Augmented Reality in Industry

Keywords: Industrial Augmented Reality, Evaluation, Guidelines

Augmented Reality (AR) is seen as a key technology for the development of smart manufacturing. Despite the many possibilities and affordances of this emerging technology, it is a fairly new technology in industry without widespread adoption. Research indicates that there are many affordances of using this technology in industry as well as some challenges. However, there seems to be a lack of research on how to evaluate AR in industry. Based on literature and a case study, we propose guidelines for evaluating AR based on identified dimensions.

Introduction

Augmented Reality (AR) refers to a real-world environment enhanced with computergenerated information such as sound, video or graphics. AR can be delivered by using smartphones, tablets, head-mounted-displays (HMDs) and projectors [START_REF] Uva | Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations[END_REF] [START_REF]XR PRIMER 2.0: A STARTER GUIDE FOR DEVELOPERS[END_REF]. The last few years, augmented reality technology (AR) has arisen in manufacturing [START_REF] Mourtzis | Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling[END_REF] and is considered one of the nine enabling technologies that will power the transformation supported by Industry 4.0 initiative [START_REF] Davies | Industry 4.0: Digitalization for Productivity and Growth[END_REF][10] [START_REF] Zolotová | Smart and cognitive solutions for Operator 4.0[END_REF]. Although industrial augmented reality (IAR) is one of the key pillars of the industry 4.0 enablers, adoption levels remain low in industry [START_REF] Masood | Adopting augmented reality in the age of industrial digitalisation[END_REF]. There also seems to be a lack of research about how to assess IAR in real industrial environments, namely the manufacturing shop floor. The lack of literature on the subject is the genesis for the creation of practical guidelines to evaluate IAR by distilling the lessons learnt from the evaluation studies conducted in the HUMAN research project over a period of two years. This paper is structured with a theoretical background about AR and evaluation of IAR, followed by a description of the HUMAN project and lessons learnt from the evaluation activities focused on the use of an IAR solution to cognitively augment the worker on the shop floor. The paper then distils the lessons into a set of guidelines to assist in the evaluation of IAR in real manufacturing work environments.

Theoretical Background

In this section, we present three categories based on literature to consider when evaluating industrial augmented reality: lab or industrial environment, safety and operability challenges and usability and interface.

Evaluating AR in industry

Lab or industrial environment Evaluating AR in industry can be conducted in a lab or in a real industrial environment. When searching for literature, most of the studies we found was conducted in a lab and there seems to be few studies conducted on the shop floor. A survey shows that most of the technical studies have been carried out only in laboratory settings, without implementing the AR system in a real context [START_REF] Bottani | Augmented reality technology in the manufacturing industry: A review of the last decade[END_REF]. Most implementations of AR for assembly seems to be carried out in laboratory settings, and AR studies targeting ergonomics issues have previously been entirely carried out in laboratory settings, with real case applications still lacking in the literature. Laboratory studies typically represent a proof-of-concept of the AR applicability and should be seen as the first step for the development of a scalable solution [START_REF] Bottani | Augmented reality technology in the manufacturing industry: A review of the last decade[END_REF].While evaluations conducted outside the lab considers the state in which the current system is, evaluations in real environments is important because the system should ultimately be used in the real industrial context [START_REF] Fite-Georgel | Is there a reality in Industrial Augmented Reality?[END_REF].

Safety and operability challenges

When evaluating AR in a real industrial context, there seems to be some safety and operability challenges that might present a major obstacle, such as working in construction sites [START_REF] Chalhoub | Exploring the performance of an augmented reality application for construction layout tasks[END_REF]. Changes in worker´s workload, time of the day, and location of the room are examples of unquantifiable factors that might skew the findings in the process. In addition, gathering data on an active job site for a pilot study presents potential safety and financial risks for the contractor when using unproven and untested technologies. Also, tracking the environment and accurately displaying the content requires well-lit areas and is very sensitive to heavy shadows [START_REF] Chalhoub | Exploring the performance of an augmented reality application for construction layout tasks[END_REF].

Usability and interface

Usability is an essential variable of all Machine-Interface (HMI) when complexity and number of functions increase. If a technology is not regarded as easy to use and easy to understand, it will not enhance the user´s performances and process at hand and will not be accepted by the users [START_REF] Holm | Adaptive instructions to novice shop-floor operators using Augmented Reality[END_REF]. User interface is described as "does not require special knowledge to use, user friendly and easy to use, with minimum interactions required", and has been identified as one important feature for the future adoption of AR [START_REF] Stoltz | Augmented Reality in Warehouse Operations: Opportunities and Barriers[END_REF]. A consequence of evaluating AR with operators is that they might spend more time to complete the task in total, especially in the first steps, because they don't have previous experience with the technology and therefore might need time to understand instructions given in the AR-system [START_REF] Mourtzis | Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling[END_REF]. When assessing usability, it is important that usability is measured relative to the chosen group of users within the specified scope of tasks [START_REF] Holm | Adaptive instructions to novice shop-floor operators using Augmented Reality[END_REF].

The System Usability Scale (SUS) has been used to measure usability [START_REF] Holm | Adaptive instructions to novice shop-floor operators using Augmented Reality[END_REF], others have added questions to a questionnaire based on NASATLX [START_REF] Mourtzis | Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling[END_REF] and formulated a hypothesis and conducted a post-experiment questionnaire to evaluate ease of use, satisfaction level, and intuitiveness for each instruction mode [START_REF] Uva | Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations[END_REF]. The ultimate desired state of an IAR system that prove its usability, is if the system was deployed to the end-users and is being used without requiring constant presence of the developers. It is also important evaluate if the AR system is developed with input of industrial partners. This is an important factor as it usually introduces a level of reality to a prototype [START_REF] Fite-Georgel | Is there a reality in Industrial Augmented Reality?[END_REF].

HUMAN Project

The HUman MANufacturing (HUMAN) project was an European H2020 project aimed to develop a platform that is contextually aware of the factory and the human operator to support the operator in performing their tasks with the desired quality, whilst ensuring their well-being. The development of the solutions was permeated with co-creation principles, engaging with different stakeholders in manufacturing companies from inception to deployment, where evaluation played a crucial role in collating feedback and validation. An outcome of the HUMAN project was a fully integrated industrial AR system that aims to increase manufacturing productivity whilst providing support for quality assurance (KIT-AR).

KIT-AR

Three distinct end-user organizations contributed to the co-creation process that led to the development of KIT-AR: Airbus, Comau and Royo Group. As shown in Figure 1, KIT-AR technology consists of four interactive system modules.

The evaluation studies

The evaluation studies conducted at the three end-users, had the following lifecycle stages: data collection, technical testing, UX testing formative evaluation study and a final study. For the purpose of training the reasoning of the KIT-SMART modules, data was gathered from multiple sensors and the KIT-AR solution. This data gathering happened at shop floor level with the operators. It usually came together with the technical testing. The particular instance of the KIT-AR solution was tested with operators performing their everyday tasks. Preliminary tuning of KIT-AR occurred during all interactions with initial feedback being obtained from operators. Normally, technical testing went together with data collection in the same visit. A particular session to assess the UX of the KIT-AR solution, used a mix of quantitative and qualitative methods and tools. The aim was to improve iteratively the user interface until one can focus on the utility of the solution. With a formative evaluation study, the KIT-AR service was evaluated from all aspects, including the perception of utility. In-depth discussion to expand the initial responses was carried out to understand the underlying motivation and explore potential alternatives to inform the subsequent development cycle. At this stage, the key assumptions of the final study were validated, namely the KPIs. The final stage of evaluation corresponded to the assessment of utility of the KIT-AR solution. The chosen KPIs were measured to determine the impact assessment.

Lessons learnt

As a result of the evaluation process in each of the three end-user organizations, several lessons were distilled and are succinctly summarized in this section.

Operators availability and scheduling

A fundamental reality of conducting evaluation studies in an industrial environment is that production will not change due to evaluation. The traditional approach is to reduce the impact in production and setup an offline area to setup the experiment for evaluation. However, the human resources are limited, and adequate planning is necessary to ensure that sufficient operators are made available for the evaluation study, with the company incurring costs for the operator's time taking part. Another shortcoming of offline evaluation is the difficulty to infer transfer of the results into production. When considering evaluation in production, scheduling becomes even more stringent as the evaluation study cannot determinately impact on the production output as measured by the KPIs. In addition, the nature of the task needs to be able to accommodate the overhead of the evaluation activity to avoid introducing unnecessary stress to the operator or risk exceeding the tolerance of the operator and thereby introducing a negative bias in the evaluation results. The access to the operator population is severely limited on two accounts: motivation of operators and the production planning. Although operators are interested in new ways of doing work more efficiently, they are less interested in taking part of evaluation studies. This has an impact on operators' approach of using novel tools that are not easily integrated into existing work practices without an adequate change management process. The production planning imposes constraints on the availability of operators, thus making quantitative studies extremely difficult to carry out. Consequently, one approach used in HUMAN was to conduct qualitative studies with real operators and complement the population of potential participants with students or workers with different roles. This approach requires an experimental design that attempts to characterize the differences between the different subject populations, thus increasing the credibility of combining the evaluation results from both populations. Ultimately, one must plan for small number participants and consider the fusion of evaluation results from operators and additional participants of different type. It is clear that the company cost of the operator's time, and the impact of using the AR solution, needs to be considered in the evaluation study.

Utility

Traditionally, evaluation studies are conducted in a session-based manner in well-controlled conditions and with hands-on support from researchers. The real assessment of the utility of an AR solution is when the corresponding AR device is left over a period of time within easy access of the operators, who then have the option to either use the solution or not. Consequently, the operator will only use the solution if it deemed of value, with clear benefits in terms of productivity and/or quality by means of doing less mistakes. In addition to monitoring of the usage of an AR solution, one can complement with relevant tools such as diaries [START_REF] Bolger | Diary Methods: Capturing Life as it is Lived[END_REF].

Lack of control

A key distinguishing aspect of field evaluation versus a laboratory setting is the almost excruciating lack of control on the work environment. A manufacturing company will not modify the work environment for the benefit of the study, in particular when considering the study in production and not an offline setting. As part of the evaluation methodology adopted in HUMAN, extensive technical testing was carried out to validate the usage of technology considering the constraints of the work environment, such as:

• the production environment had significant impact on the use of wireless connectivity, which enforced edge computing; • the HoloLens device (version 1) is highly susceptible to the amount of luminosity that had an impact on the rendering of holograms and consequently requiring redesign of the instruction sets in some cases; • The seasonal ambient temperature of the work environment may have an impact on the correct functioning of the IAR device. In any case, the potential acceptance of any deviations to the work environment requires careful negotiation with different stakeholders in the manufacturing company beyond the expected operator and managers, extending to include social actors such as trade unions. Under no conditions should an evaluation study compromise the health and safety regulation of the work environment.

Quality of user interface and content

The quality of the user interface has significant impact on the evaluation results and consequently, it is highly relevant to have appropriate tools to assess the usability with the purpose of ensuring the focus is on the utility of the solution. The expectations of operators in a production environment are high and there is less tolerance for poor usability, thus the recommendation is to iteratively improve the solution until a usability tool such as SUS [START_REF] Lewis | The Factor Structure of the System Usability Scale[END_REF] indicate that acceptable UX has been achieved. However, even then, one should design the effectiveness of the interface design, possibly experimenting different variants of the user interface. In addition to the quality of the user interface, one needs to consider the quality of the content of the AR solution being used. Consequently, it is recommended that the instructions are created by a product engineer or individual with appropriate expertise and experience. An alternative is to involve experienced operators to co-create the instruction sets. However, irrespective of how content is created, it is recommended to conduct heuristic evaluation on the quality of the content.

Onboarding

The introduction of a new technology, such as AR, requires learning of the solution to enable operators to build the appropriate mental model. This onboarding must be considered when conducting the evaluation to eliminate the learning effect as it will distort the results otherwise and possibly introduce bias in the analysis and interpretation of the collated data.

Evaluation Guidelines

Based on the lessons learnt in the HUMAN project, combined with the limited knowledge reported in literature concerning IAR evaluation in real work environments, a set of four guidelines for evaluating IAR is proposed in the following subsections:

Experimental Design

A fundamental premise of evaluation of IAR in real manufacturing work environments is the fact that the experimental design must accommodate the constraints of the industrial environment in terms of:

• Technical. The characteristics of the work environment may negatively impact the correct operation of the IAR solution. Consequently, it is necessary to accommodate adequate testing and piloting of the experimental design. • Cost. There is always a cost to the company to run the evaluation with potential risk to loss of productivity, exacerbating the cost further. It is necessary to negotiate with the company the financial implications and possibly modify the experimental design to accommodate the cost impact. • Participant population. The workers are an extremely scarce resource that impose constraints on number of participants affecting the experimental design. Start the discussion with production managers early to identify the actual number of participants available. Consider the fusion of qualitative results of studies involving workers with quantitative results from studies involving non-workers. • Production. The production planning dictates when evaluation activities can take place and the health/safety regulations cannot be compromised. Consider the need to conduct the evaluation study over a long enough period to accommodate the production schedule.

• Return on Investment (ROI).

There is a cost of conducting evaluation on the shop floor. Understand the needs of the company and manage their expectations concerning the outcome of the evaluation. Strive to associate evaluation goals to the business as it contributes to an understanding of potential ROI. Consequently, it is necessary to plan in advance for the evaluation and negotiate with the relevant stakeholders, from management to team leaders.

Content Quality

An IAR solution provides digital overlay of knowledge to workers and traditionally assessing the impact of the quality of the content is disregarded when conducting evaluation studies. Consider the following two measures:

• Co-design the instructions with the workers and relevant stakeholders. This reduces the impact of poorly designed content; • Conduct heuristic evaluation of the content quality with trainers/experienced worker. This measure may contribute to the improvement of the quality of the content or to understand the limitations of the final instance of the content.

Interface and usability

It is fundamental to ensure that the User eXperience (UX) is well designed so usability is not a barrier to utility. Consequently, one should consider the following measures:

• Strive to improve the UX until the law of diminishing returns applies. This may involve changes to the development of the solution or content workarounds; • Assess the impact of the different devices on the UX, both from the constraints on the interface and the ergonomics; • Understand the impact of the UX on the outcome of the evaluation results by doing a small pilot and combining it with an heuristic evaluation by UX experts; • It is fundamental to have onboarding of the operators in the use of IAR as they require a learning period to acquire the mental model of the system.

Execution

The most well-planned evaluations are subject to the uncertainties of production realities and as such may disrupt the evaluation activities. As such, it is necessary to be able to adapt, but most importantly use every opportunity to gather data and conduct analysis incrementally to best understand how to make necessary changes to the evaluation activities to mitigate the negative impact of disruption.

Conclusion

The presented guidelines are not exhaustive, neither comprehensive, but are an initial approach that builds upon the experience of two years in conducting iterative evaluation studies using IAR. The aim is to provide insights on how to improve evaluation activities in real industrial work environments towards establishing a good practice of research on the shop floor.

Figure 1 .

 1 Figure 1.KIT-AR system

Acknowledgements

The authors would like to thank the operators that took part of the evaluation activities using the KIT-AR solution. This work was partially funded by the Norwegian Research Council (#28026) and the H2020 HUMAN project (Grant Agreement nº: 723737).


Iztok Palčič
email: iztok.palcic@um.si

Simon Klančnik
email: simon.klancnik@um.si

The Use of Organizational Innovation Concepts in Manufacturing Companies

Keywords: Organizational Innovation Concept, Manufacturing Company, Advanced Manufacturing Technology, European Manufacturing Survey, Slovenia

The main purpose of this paper is to map the adoption of selected innovative organizational concepts in manufacturing companies. A further objective of our research is to find out how different organizational concepts are represented in different manufacturing company's types and how they are related to the use of advanced manufacturing technologies. Based on a sample of 118 Slovenian manufacturing companies collected through the 2018-19 European Manufacturing Survey edition, results give information about the most used organizational concepts, their utilization degree, planned use and average year of implementation. In addition, results give evidence that the use of selected organizational concepts is very positively related to the use of specific advanced manufacturing technologies.

Introduction

The importance of innovation to competitiveness has been acknowledged for many years now. Unfortunately, the term innovation is still sometimes more synonymous with product innovation or technical innovation than process or organizational or managerial innovation. Organizational innovations are often neglected in innovation theory [START_REF] Gallego | Organizational innovation in small European firms: A multidimensional approach[END_REF]. Keupp et al. [START_REF] Keupp | The Strategic Management of Innovation: A Systematic Review and Paths for Future Research[END_REF] made an extensive literature review on innovation management, where analysing more than 342 articles, they found out that there is a low number of articles dealing with the field of non-technical innovation, while the number of articles in the field of technical innovation is much higher. Although organizational innovation is considered an important contributor and trigger to the product innovation, and ultimately to company success, organizational or non-technical innovation needs further and deeper understanding [START_REF] Kocijančič | Impact of innovative organisational concepts adoption on manufacturing firms' performance[END_REF]. Many studies analyse specific organizational innovations. On the other hand, there is a lack of papers that try to identify and measure the organizational innovation in companies [START_REF] Armbruster | Organizational innovation: The challenge of measuring non-technical innovation in large-scale surveys[END_REF][START_REF] Damanpour | Footnotes to Research on Management Innovation[END_REF].

Therefore, our paper contributes to analysis of specific organizational concepts in manufacturing companies from different perspectives.

Organizational innovation does have an impact on company output, first directly and second, through its interrelationship with technical innovation [START_REF] Gallego | Organizational innovation in small European firms: A multidimensional approach[END_REF]. Accordingly, several studies exist that emphasize the interdependence of technological and management innovations and their joint, complementary influence on organizational outcomes [START_REF] Damanpour | Footnotes to Research on Management Innovation[END_REF]. These studies usually focus on organizational (and technical) innovation and its impact on innovation and financial performance. On the other hand, there are not many studies that look for relationship between specific organizational concepts implementation and the use of specific advanced manufacturing technologies (AMT).

The main purpose of this paper is to map the adoption of selected innovative organizational concepts in Slovenian manufacturing companies and the relationship with selected AMT. The structure of the remainder of the paper is as follows. Section 2 gives a brief description of organizational innovation. Section 3 describes used research methodology. Section 4 comprises the results of the analysis and discussion, while the conclusions are presented in section 5.

Organizational innovation concepts

Management innovations, also called organizational, administrative, and managerial innovations, are non-technological innovations that have been conceptualized in contrast to technology-based product and process innovations and pertain to new organizational structures, administrative systems, and management practices [START_REF] Damanpour | Footnotes to Research on Management Innovation[END_REF]. Birkinshaw et al. [START_REF] Birkinshaw | Management innovation[END_REF] defined management innovation as 'the generation and implementation of a management practice, process, structure, or technique that is new to the state of the art and is intended to further organizational goals'. OECD Oslo Manual from 2005 [START_REF]Oslo manual. Guidelines for collecting and interpreting innovation data. 3rd edn[END_REF] defines an organizational innovation as the implementation of a new organizational method in the company's business practices, workplace organization or external relations.

There are several ways to differentiate organizational innovation. The first possibility is into structural organizational innovations and procedural organizational innovations. Structural organizational innovations influence, change and improve responsibilities, accountability, command lines and information flows as well as the number of hierarchical levels, the divisional structure of functions (R&D, production, human resources, financing, etc.), or the separation between line and support functions. Procedural organizational innovations affect the routines, processes and operations of a company. Thus, these innovations change or implement new procedures and processes within the company, such as simultaneous engineering or zero buffer rules. Organizational innovation can be further differentiated along an intra-organizational and interorganizational dimension. While intra-organizational innovations occur within an organization or company, inter-organizational innovations include new organizational structures or procedures beyond a company's boundaries. These comprise new organizational structures in an organization's environment, such as just-in-time processes with suppliers or customers, supply chain management practices with suppliers or customer quality audits. Intra-organizational innovations may concern particular departments or functions or may affect the overall structure and strategy of the company as a whole. Examples for intra-organizational innovations include the implementation of teamwork, quality circles, continuous improvement processes, certification of a company under ISO 9000, zero-buffer principles, environmental audits etc [START_REF] Armbruster | Organizational innovation: The challenge of measuring non-technical innovation in large-scale surveys[END_REF].

Scholars have started emphasizing that, in order to capture the full benefits of innovation, technological innovation needs to be combined with management innovation [START_REF] Hollen | The Role of Management Innovation in Enabling Technological Process Innovation: An Inter-Organizational Perspective[END_REF][START_REF] Damanpour | Managerial innovation: Conceptions, processes, and antecedents[END_REF][START_REF] Damanpour | Combinative effects of innovation types and organizational performance: A longitudinal study of service organizations[END_REF]. Historically research on innovation types has followed a technological imperative. It assumes that companies mainly organize their innovation efforts through R&D activities and has thus focused on a narrow definition of product and process innovations associated with the R&D function in manufacturing organizations [START_REF] Damanpour | Combinative effects of innovation types and organizational performance: A longitudinal study of service organizations[END_REF]. The sociotechnical system theory challenged the technological imperative and argued that changes in the technical (operating) system of the organization should be coupled with changes in the social (administrative) system in order to optimize organizational outcome [START_REF] Damanpour | Organizational innovation and performance: the problem of Organizational Lag[END_REF][START_REF] Cummings | Management of Work: A Sociotechnical System Approach[END_REF][START_REF] Trist | The Social Engagement of Social Science: A Tavistock Anthology[END_REF]. Therefore, our research looks for relationship between technical and organizational innovation concepts.

About European Manufacturing Survey

The research data was collected using the EMS, a Consortium of Research Institutions coordinated by the Fraunhofer Institute for Systems and Innovation Research -ISI, the largest European survey of manufacturing activities. The survey's questions deal with manufacturing strategies, application of innovative organizational and technological concepts in production, cooperation issues, production offshoring and backshoring, servitisation, and questions of personnel deployment and qualification. In addition, data on performance indicators such as productivity, flexibility, quality and returns is collected. The survey takes place every three years.

The responding companies present a cross-section of the main manufacturing industries. Included are producers of rubber and plastics, metal works, mechanical engineering, electrical engineering. The survey is conducted among manufacturing companies (NACE Revision 2 codes from 22 to 32) having at least 20 employees. Our research is based on EMS data from a Slovenian subsample from the year 2018/19 round. We sent 778 questionnaires and received 118 responsesa 15% response rate. In our 2018/19 subsample, manufacturing companies from NACE divisions 22, 25 and 28 are represented most widely with around 25% of companies from NACE 25, around 20% from NACE 28, and around 16% from the NACE 22 division. We classified manufacturing companies in three size classed based on the number of employees. The largest share of respondents is from medium sized companies (around 43%), followed by small companies (32%) and large companies (25%).

We present the results of our research with the use of descriptive statistics and linear regression model to analyse the dependency between organizational concepts and AMT.

4

Results and discussion

Organizational concepts in EMS

In EMS 2018, we have divided 11 organizational concepts used in manufacturing companies into 3 groups: Organization of production (OP -4 concepts); Management/controlling (MC -5 concepts) and Human resources (HR -2 concepts).

For each organizational concept, we have asked for the following information:

 Use of organizational concept (yes/no).  Use planned in the upcoming period of three years.  Year in which organizational concept has been used for the first time in factory.  Extent (level) of actual utilization compared to the most reasonable potential utilization in the factory: Extent of utilized potential "low" for an initial attempt to utilize, "medium" for partly utilized and "high" for extensive utilization. High utilization of an organizational concept means, for example, that it involves at least 70% of employees in the company. As we can see from Table 1 both concepts from "Human resources" group have very high adoption rate in manufacturing companies. The age of Industry 4.0 and fast technological pace require that all employees acquire new knowledge and skills in different areas. They are also becoming more and more priceless and scarce, therefore companies must introduce specific instruments to promote staff loyalty (e.g. attractively designed responsibilities, offering learning opportunities, flexible working hours, childcare etc.). Quality standards, such as ISO 9001, have always had a high adoption rate, as they are prerequisite to conduct business in global economy. The pressure of environmental challenges in front of us requires also introduction of specific certified environmental management systems (e.g. EN ISO 14001). As many companies introduced specific organizational concepts in the last few years, the average year of first introduction of organizational concepts differs from previous EMS rounds. Except for Stand-ardized instructions for work, all other three organizational concepts from "Organization of production" group are among the youngest, as well as Visual management from the "Management/controlling" group.

The column "Planned use" presents share of companies that do not possess a specific organizational concept, but are planning to introduce it in the period from 2019 to 2021. We can see that the share of companies that are planning to invest in organizational concepts in the next 3 years is quite high. More than 40% of companies that do not use display boards in production is planning to introduce them by the year 2021. That environmental issues are important is clearly presented also by the fact that one third of companies without environmental systems will introduce them by the end of 2021. CIP and SMED methods are also planned in one third of companies that at the moment do not use them.

Next, we looked at the extent of potential use or actual utilization (low -1, medium -2, high -3). Both quality and environmental standards have a very high share of companies who claim a high use of the standard (around 80% claim to use them to full potential and with average grade of 2,8 for all analysed companies that use them). As expected from the previous research standardized instructions for work are also used at a high level. There is an increase in share of companies that use visual management at the high level comparing to previous EMS rounds. This means that different display equipment in production is becoming widely used and at the same, it delivers required results. Comparing to previous EMS rounds there is also an increase in the share of companies that use continuous improvement methods to more full potential. It can be seen that both concepts from "Human resources" are often used to a very low potential (grade around 2,1). Pull principle, SMED and value stream mapping methods have the lowest average potential use grade of around "2".

The concept "Certified energy management system (e.g. EN ISO 50001)" has a very low adoption rate (around 15%). Therefore, we have excluded this organizational concept in further analysis.

Organizational concepts and advanced manufacturing technologies

In In this paper, we are focusing on Industry 4.0 specific AMT from the digital/smart factory area, industrial robots and additive technologies (we are skipping energy efficient technologies). Table 2 presents the share of AMT and ICT use. When we looked at the use of both types of discussed robots together, we found that at least one type of robots is present in 64% of companies, making industrial robots the most widely used technology from our list. If we combine both types of additive manufacturing technologies, we can observe that at least one of them is used in around one third of the companies. In this paper we are skipping the more in-detail discussion of AMT and ICT use as the focus is on organisational concepts. Our goal was to look for relationship between selected organizational concepts and the use of described AMT. We analysed the following six organizational concepts: Standardized and detailed work instructions (SWI); Measures to improve internal logistics (WMS); Setup time or change-over reductions methods (SMED); Pull principle (KANBAN); Visual management (VM) and Methods of assuring quality in production (Q).

We have used linear regression method to look for relationship between a dependent variable (each organizational concept) and independent variables (each AMT). Table 3 presents (Pearson) correlation coefficients for selected organizational concepts and AMT from the group "Digital factory" and Table 4 presents results for the groups "Automation and robotics" and "Additive manufacturing technologies". Our regression procedure aggregates the empirical results based on the percentage of statistically significant regression coefficients (p < 0.05). Table 3 and 4 show that in general all six analysed organizational concepts are positively and significantly correlated with majority of AMT. The organizational concept that illustrates work processes and work status (Visual Management) is positively and significantly correlated with all AMT. Digital solutions on the shop floor, ERP, EDI and systems for automation and management of internal logistics are positively correlated with all analysed organizational concepts.

Table 5 presents additional results of linear regression analysis, where we present results for the whole model for each selected organization concept. The model is not significant only for the first organizational concept Standardized instructions for work (Sig. F change is 0,522). For all other organizational concepts, the model is significant, meaning that AMT are relevant predictors for organizational concept use.

Conclusion

Since there is not much less research in the area of non-technical innovation than in the area of technical innovation, this paper gives emphasis on this type of innovation. We have analysed how widely selected organisational concepts are used, what is the extent of their potential use, what differences exist in the use and extent of use of these concepts and which are the characteristics of the adopters (company size, technological intensity, OEM/supplier etc.). There is a big difference in the exploitation of organizational concept potential use between analysed concepts. The ones with earlier average year of introduction have higher level of exploiting the potential use. On average, the dispersion of organizational concepts is higher in larger companies, suppliers and within companies from medium or high technological sectors.

The last finding is proved by the analysis of the relationship between organizational concepts and AMT. Our research showed that the use of several organizational concepts, that deal with organization or/and management and controlling of production, are strongly and positively correlated with AMT from "Digital factory" area, robots and additive manufacturing technologies. We can argue that the use of these AMT is not possible without carefully planned and implemented organizational concepts.

Our research has, therefore, practical and managerial implications. Our results show that the use of specific AMT clearly positively affects the need to implement organizational innovation concepts and vice versa. This is a clear message to managers that technical innovation and the use of AMT requires also organizational concepts to support the potential of AMT.

Table 1 .

 1 Characteristics of used organizational concepts.

	Organizational concept	Group	Share [%]	Year of intro-duction	Planned use [%]	Level of use
	ISO 900x	MC	83,9	2003	15,8	2,80
	Training of employees	HR	83,9	2005	26,3	2,08
	Standardised instructions for work	OP	79,7	2002	20,8	2,55
	Instruments for promoting staff loyalty	HR	67,8	2005	18,4	2,10
	CIP, TPM, TQM, Six Sigma	MC	59,3	2009	29,2	2,25
	Display boards in production (Visual Manag.)	MC	52,5	2012	41,1	2,44
	EN ISO 14001	MC	43,2	2007	34,3	2,78
	Value Stream Mapping	OP	37,3	2010	16,2	1,98
	SMED	OP	36,4	2011	34,7	1,98
	Production controlling following pull principle	OP	33,9	2011	20,5	2,03
	Energy management system (EN ISO 50001)	MC	15,3	2014	17,0	2,37

Table 2 .

 2 AMT adoption in Slovenian manufacturing companies

	AMT	Share [%]
	Digital factory
	Mobile/wireless devices for programming and controlling facilities (e.g. tablets)	32,2
	Digital solutions to provide drawings, work schedules or instructions on the shop floor	54,2
	Software for production planning and scheduling (e.g. the ERP system)	62,7
	Digital exchange of product/process data with suppliers / customers (EDI)	51,7
	Near real-time production control system (MES)	39,8
	Systems for automation and management of internal logistics (WMS, RFID)	20,3
	Product-Lifecycle-Management-Systems (PLM) or Product Data Management (PDM)	19,5
	Virtual reality, or simulation for product design, or product development	38,1
	Artificial Intelligence (Deep Learning, Machine Learning or Neural Networks)	5,1
	Automation and robotics
	Industrial robots for manufacturing processes (e.g. welding, painting, cutting)	50,0
	Industrial robots for handling processes (e.g. assembling, sorting, packing processes, AGV)	35,6
	Additive manufacturing technologies
	3D printing technologies for prototyping (prototypes, demonstration models, 0 series)	32,2
	3D printing technologies for manufacturing of products, components and forms, tools, etc.	23,7

Table 3 .

 3 Organizational concepts and digital factory technologies

	Org. con.	Mobi. device	Digital solut.	ERP	EDI	Con-trol	Inter. logist.	PLM, PDM	Virt. real.	AI
	SWI	+0,078 +0

,170 +0,220 +0,186 +0,067 +0,203 +0,142 +0,137 +0,021

	VSM	+0,294 +0,251 +0,232 +0,395 +0,303 +0,438 +0,373 +0,224 +0,141
	SMED +0,270 +0,236 +0,293 +0,344 +0,463 +0,317 +0,472 +0,348 +0,226
	Kan-ban	+0,273 +0,155 +0,182 +0,262 +0,185 +0,305 +0,280 +0,138 +0,160
	VM	+0,255 +0,217 +0,320 +0,270 +0,461 +0,227 +0,382 +0,152 +0,220
	Q	+0,165 +0,209 +0,289 +0,304 +0,357 +0,247 +0,364 +0,259 +0,192

Table 4 .

 4 Organizational concepts and, robots and additive manufacturing technologies

	Organizational concept	Manufac. robots	Assembly robots	3D pro-toyping	3D manu-facturing
	Standardised work instructions	+0,042	+0,112	+0,168	+0,133
	Warehouse Management Systems	+0,210	+0,305	+0,369	+0,353
	SMED	+0,123	+0,357	+0,307	+0,406
	Kanban	+0,251	+0,328	+0,388	+0,316
	Visual Management	+0,170	+0,246	+0,219	+0,251
	Quality in Production	+0,104	+0,291	+0,275	+0,219

Table 5 .

 5 Results of the regression analysis

	Organizational concept	R	R Square	Adjusted R Square	Std. Error of the Estimate	R Square Change	Sig. F change
	Standardised work instruc.	,323	,105	-,007	,40573	,015	0,522
	Warehouse Man. Systems	,575	,331	,247	,42144	,331	0,000
	SMED	,632	,399	,324	,39731	,399	0,000
	Kanban	,499	,249	,155	,43701	,249	0,003
	Visual Management	,547	,299	,211	,44537	,299	0,000
	Quality in Production	,486	,237	,141	,45716	,237	0,006
	Predictors: (Constant), Manufac_3D, Internal_log, Virtual_reality, Robots_manuf, AI, ERP, Robots_assem,
	EDI, Control_system, Mobile_dev, PLM_PDM, Digital_sol, Prototype_3D


Jörg Brünnhäußer
email: joerg.bruennhaeusser@ipk.fraunhofer.de

Sonika Gogineni

Jonas Nickel

Heiko Witte

Prof Rainer Stark

Assembly issue resolution system using machine learning in aero engine manufacturing

Keywords: Machine learning, data-driven service, issue resolution

Companies are progressively gathering data within the digitalization of production processes. By actively using these production data sets operational processes can be improved, hence empowering businesses to compete with other corporations. One way to achieve this is to use data from production processes to develop and offer smart services that enable companies to continuously improve and to become more efficient. In this paper, the authors present an industrial use case of how machine learning can be implemented into smart services in production processes to decrease the time for resolving upcoming issues in manufacturing. The implementation is carried out by using an assistance system that aids a team which attends to problems in the assembling of turbines. Therefore, the authors have analyzed the assembly problems from an issue management system that the team had to resolve. Subsequently three different approaches based upon natural language processing, regression and clustering were selected. This paper also presents the development and evaluation of the assistance system.

Introduction

An increasing amount of data is stored within enterprise systems and databases of many companies. By introducing Internet of Things (IoT) concepts more and more data throughout the whole product lifecycle are made available every second. This includes the production process where due to the advancing digitalization almost every step within that process leaves data behind. A key aspect for many manufacturing companies is to take advantage of these data to increase competitiveness through machine learning (ML), Smart Services and Data-Driven Applications [START_REF] Stich | Digitale Dienstleistungsinnovationen. Smart Services agil und kundenorientiert entwickeln[END_REF].

The application of ML on existing data is one approach to improve manufacturing processes. This paper shows on an example how one can set up a Smart Service with Machine Learning based on already available data and put it into practice. In the best case it can save time and increase the efficiency, hence saving costs. This is also the goal of the COCKPIT 4.0 project (funded by the Berlin Senate, the State of Branden-burg and the European Regional Development Fund -ERDF 1.8/03) in which a consortium of Rolls-Royce Germany, Fraunhofer IPK and the BTU Cottbus are working together to address different challenges in the context of digitalization. This paper is about one work package of the COCKPIT 4.0 project which aims to improve the efficiency of the gas turbine assembly at the Rolls-Royce site in Dahlewitz, Germany. The goal of the work package is to connect various data sources with semantic technologies [START_REF] Gogineni | Semantic Assistance System for Providing Smart Services and Reasoning in Aero-Engine Manufacturing[END_REF] and support the resulting semantic network with machine leaning techniques to assist in solving production issues efficiently. The learned models enable the creation of an assembly issue resolution system which will aid the fitters to fix manufacturing problems faster. The resulting assistance system improves problem identification, resolution and prevention by the fitters, shop floor management and assembly manufacturing engineering.

Problem description

Modern aerospace products are not only highly complex but are also supplied by an equally complex and globally distributed supply chain. While this approach enables both a cost-effective and flexible product development and production, it also increases the risk of supply chain disruptions and part supply issues for the engine assembly process [START_REF] Treuner | A survey of Disruptions in Aviation and Aerospace supply chains and Recommendations for Increasing Resilience[END_REF]. Even though many of these risks can be managed upstream of the final engine assembly, it is necessary to quickly react to missing parts and other assembly issues in the build line in order to avoid build stops and delivery delays. Rolls-Royce Germany has established a small, dedicated "Voice of the Fitter" (VoF) team that focuses on resolving any assembly issues as quickly as possible. As the issues might arise from a number of sources, e.g. supplier production issues, delays in logistics or customs, transport or handling damage, the team needs to access seven data sources and uses different strategies to resolve the issues. Hence, a lot of time is spent on identifying the right data source, accessing the data source, finding the right information, assessing the data, taking a decision and implementing suitable action. In such cases, contextualized data sets assist the users in addressing some of the above tasks such as identifying, accessing, finding and assessing the right data [START_REF] Gogineni | Semantic Assistance System for Providing Smart Services and Reasoning in Aero-Engine Manufacturing[END_REF]. In addition, ML can be used in enhancing those tasks and also in supporting decision making. This results in saving time for the VoF team.

In this paper, the creation of a smart, ML enabled assistance system that connects all data sources in one location and provides context-related information and resolution strategies to the VoF team is developed. The resulting system can subsequently be expanded to support related tasks in the company, e.g. concession assessments for nonconforming parts.

State of the art

ML is an emerging technology which enables powerful applications and handling of complex data. Production systems are providing a lot of use cases for ML. Wuest et al.

[4] outlined the reasons for the increasing importance of ML and mentioned the challenges as well. However, some advantages mentioned are the capability "to handle high-dimensional, multi-variate data, and the ability to extract implicit relationships within large data-sets in a complex and dynamic, often even chaotic environment" [START_REF] Wuest | Machine learning in manufacturing: advantages, challenges, and applications[END_REF].

There are already an abundance of applications such as the estimation of the manufacturing costs of jet engine components [START_REF] Loyer | Comparison of Machine Learning methods applied to the estimation of manufacturing cost of jet engine components[END_REF], the prediction of basic parameters for turbomachinery design based on the load level [START_REF] Azzam | Application concept of artificial neural networks for turbomachinery design[END_REF] or approaches to do predictive maintenance [START_REF] Susto | Machine Learning for Predictive Maintenance: A Multiple Classifier Approach[END_REF]. There are also several papers for different systems which support production tasks. In this work [START_REF] Belkadi | Intelligent assistant system as a context-aware decision-making support for the workers of the future[END_REF] an assistance system for operational workers is created which uses simulation and Proper Generalized Decomposition algorithms to provide decision support. Additionally another decision support system [START_REF] Kocsis | Case-Based Reasoning system for mathematical modelling options and resolution methods for production scheduling problems: Case representation, acquisition and retrieval[END_REF] uses Case Based Reasoning for scheduling tasks. However, there seems to be no suitable approach which addresses the resolution of single and diverse assembly problem in combination with ML.

Solution definition

The VoF use case was one of several use cases collected in the business understanding phase of the Cockpit 4.0 project. The use case was further detailed by interviews with stakeholders involved in VoF. Opportunities for machine learning were identified as part of the requirements capturing during the interviews. Followed by the next phase of the CRISP-DM [START_REF] Wirth | Crisp-dm: towards a standard process modell for data mining[END_REF] process the actual data was fetched and reviewed. The source data for this use case consists of two tables. In order to aid the Voice of the Fitter team the authors considered three approaches based on the available data as useful:

1. Natural language processing (NLP) based approach: The engineering goal was to identify cases with similar descriptions and similar comments, which might be useful for investigating new VoF case. The available data is not labeled which prevents supervised learning. Therefore, NLP could be applied to show similar cases. 2. Case length prediction: The goal here was to generate a model that could predict how long an incoming case will take to be resolved. This information could be helpful to judge the priority. For instance, if a single case would take very long, measures could be taken to reduce the time span like escalating the case or getting support from other departments. The start and end date of each case is available so the duration of every case can be calculated and supervised learning can be applied. 3. Clustering: Unsupervised learning could be used to separate the data into clusters.

Those clusters could provide the user with 'similar' cases which can be taken as starting point for their research on how to resolve the current issue

Design and development of the prototype

The approaches discussed in the previous sectors are further detailed in this section. This is done by explaining the data science aspects of the approaches followed by the implementation in the prototypes architecture.

Data science

NLP based approach. To select an optimal algorithm to find similar cases, three different algorithms were tested. The first one chosen was the commonly used Jaccard coefficient [START_REF] Niwattanakul | Using of Jaccard Coefficient for Keywords Similarity[END_REF], which is an index for the similarity of two sets by the technique known as shingling [START_REF] Manna | Web Documents Similarity Using K-Shingle Tokens and Min-Hash Technique[END_REF]. This approach does not take into account the meaning of the words. The second approach was based on word2vec [START_REF] Mikolov | Distributed Representations of Words and Phrases and their Compositionality[END_REF]. Word2vec is trained [START_REF] Somani | Smart systems and IoT : innovations in computing[END_REF] on a large data set like the Google News corpus and calculates vector representations for each word. The vectors for semantic similar words should also be similar. In contrast the self-trained model was trained with a data set of the VoF case descriptions in order to cast the engineering vocabulary and abbreviations into the model. The calculation of the distance was done by the word mover's distance [START_REF] Kusner | From Word Embeddings to Document Distances[END_REF].

All models were tested with the same data set and subsequently some examples have been evaluated by an expert. The results provide information about the performance of the approaches to find a similar VoF cases. The evaluation revealed that the self-trained model performed best. This seems reasonable because the data set contains many individual words that are not in common usage. Jaccard delivered the second best result, so the pre-trained model from Google was the least accurate. However, this evaluation was just a first test and further research needs to be conducted.

Case length prediction.

Initially the label 'duration in hours' was calculated from the start and the end date to be able to train a regression model. After the data preparation of the two tables mentioned in the previous chapter the data was split into a training and a test data set. The evaluation makes it necessary to test the models with unknown data which was not involved in the training. For this purpose the test data set was used. The training data set was used for the training of several different models like linear regression, lasso regression and support vector regression which was taken from the scikitlearn library [START_REF] Buitinck | API design for machine learning software: experiences from the scikit-learn project[END_REF]. Additionally, a model tree (a decision tree with different models on each leaf [START_REF]Github Model Tree project[END_REF]) was trained. The best model type for the leaf models have been estimated with grid search. For the evaluation the model made predictions of the VoF case length based on the test data set as input. Those predictions can be compared to the actual values of the duration and the root-mean-squared error can be calculated which gives a metric to evaluate the models (Table 3). In this case the Model Tree approach (with SVR models on each leaf) scored best.

Clustering. The authors compared several combinations of feature reduction methods and clustering algorithms. A subset of six configurations as shown in Table 4 was chosen from the best performing algorithms based on several scores. The subset includes centroid based k-means clustering and a density based DBSCAN clustering. Additionally HDBSCAN was tested as proposed by [START_REF] Campello | Density-Based Clustering Based on Hierarchical Density Estimates[END_REF], a combined approach between hierarchical and density based clustering.

The best results were reached with a k-means clustering. The best performing dimensionality reduction method was UMAP which is used for non-linear dimensionality reduction introduced by McInnes et al. [START_REF] Mcinnes | UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction[END_REF]. The Silhouette score was proposed in [START_REF] Rousseeuw | Silhouettes: A graphical aid to the interpretation and validation of cluster analysis[END_REF]. It measures the separateness and tightness of the clusters found by a clustering and can take values between -1 and 1, whereby a value close to -1 implies that the points are misclassified as, the distance towards the centers of the clusters are larger than the distances to the center of the closest neighboring cluster. A score around 1 implies that the result is very well-classified. The Calinski-Harabasz score was proposed in [START_REF] Calinski | A dendrite method for cluster analysis[END_REF]. It minimizes within-cluster distances and maximizes between-cluster distances, therefore, compact, well-separated clusters get a better score. The larger the Calinski-Harabasz score of a clustering is, the better is the clustering [START_REF] Calinski | A dendrite method for cluster analysis[END_REF]. The Davies-Bouldin score was presented in [START_REF] Davies | A Cluster Separation Measure[END_REF], unlike the other scores it is not maximized, but minimized. It calculates the average similarity of the rated clusters with their most similar cluster [START_REF] Davies | A Cluster Separation Measure[END_REF].

For the application the Kmeans1-clustering was chosen based on the performance metrics introduced above and the intended use. Kmeans2 had the best Silhouette score and Davies-Bouldin score, but the three resulting clusters were not practical for the considered use case. The performance of Hdbscan1, Hdbscan2, Hdbscan3 and Dbscan was worse than that of Kmeans1. After examining whether the contents of the clusters produced by Kmeans1 are sound and realistic it was chosen as the final clustering approach. Furthermore, seven clusters were indeed the number expected by experts. For UMAP the implementation by [START_REF] Mcinnes | UMAP: Uniform Manifold Approximation and Projection[END_REF] and for HDBSCAN the one by [START_REF] Mcinnes | hdbscan: Hierarchical density based clustering[END_REF] was used. All other implementations were based on [START_REF] Buitinck | API design for machine learning software: experiences from the scikit-learn project[END_REF].

Implementation of models into a Prototype

The prototype for visualizing the ML results to the VoF team was designed based on a microservice architecture. The advantage is the separation of every functionality into a single, atomic web service. The technology stack consists of the Java Spring Boot framework for the backend and vue.js for the frontend. The gateway-service hosts the frontend and serves as controller between the user and the different services. These services take care of the database access or the filtering of the data. For the voice of the fitter use case the user can see open and closed cases on a 3D model of an engine (see Fig. 1), furthermore, filter options are available in order to handle a larger set of cases. If a case is selected a box pops up with information about the case.

If it is an open case the first ML model comes into play which predicts the duration. This gives the user an estimate of how long such a case usually takes, and therefore how critical this case might get. The clustering appears on the right hand side after a click on the "Clustering" button. The individual clusters consist of at least 100 cases.

In order to show the user the most suitable case where the NLP approach is used. The cases from the same cluster are sorted by the textual similarity of their case description.

Conclusion and outlook

This paper briefly summarizes the approach and a use case to use ML and generate smart services, which helps reduce time of issue resolution in manufacturing. The significance of this work is also to show how to improve production processes with Machine Learning based on existing data in practice. VoF team and their assembly issue resolution was selected as the initial use case to be enhanced with a smart service and machine learning. Three different approaches of the application of ML were investigated: a NLP approach for text similarity, a prediction of the case duration with regression and a clustering of all the cases. This was further integrated into a prototype system to assist the team. The ML models and the prototype were presented and discussed with the experts from Rolls-Royce Germany. This prototype can be used as an example of how existing data can be turned into a smart service which can improve certain processes. The results show the potential for time savings in solving assembly problems. However, the exact time savings still need to be investigated in further work. In addition more data from enterprise systems and databases will be added to the prototype which will allow new approaches for ML. Finally the usability and the usefulness of the prototype and the ML models will be evaluated to assess how those models are assisting in resolving assembly issues.

Fig. 1 .

 1 Fig. 1. Screenshot of the current version of the prototype. 1./3. shows the position of the NLP and the clustering. The duration prediction is shown on position 2.

Table 1 :

 1 Excerpt of VoF table with an exemplary problem case

vof_en- gine_type vof_part_num- ber vof_prob- lem_ident vof_part_descrip- tion vof_organisa- tion vof_respon- sible_id

	BR725	FW85439	Missing com-	BLADE,HPT 2	Production	83
			ponent

Table 2 :

 2 Excerpt of VoF comments table with comments from VoF teamOne table stores the recent VoF cases (see Table1). The second table consists of comments (see Table2) which are linked to the first table. One case can have zero to These comments describe different things like if a part needs to be reworked, if a part which was missing is ordered or is stuck in delivery. Furthermore, it is noted in the comment table if a VoF case is escalated to a higher level of management.

		vof_case_desc	vof_case_	vof_o-		vof_close	vof_custo	vof_cate-
	id	ription		state	pen_date		_date	mer	gory	vof_priority
		LAUNCH STOP
		due to short-		03.04.2017	10.04.201
	41	age blades		Closed	07:42		7 12:25	x0036	Delivery	Build Stop
	vof_case_id	vof_comment		vof_com-	vof_com-	vof_com-	id
						ment_up-	ment_up-	ment_type
						date_time	date_user
	1041		11off in transit, expected	05.04.2017	x0030	Comment	41
			on dock today 05/04	11:23
			around noontime

Table 3 :

 3 Different trained models are compared regarding the root-mean-squared error

	Model type	RMSE [in hours]
	Linear Regression	46.02
	Lasso Regression	45.83
	Support Vector Regression (SVR)	45.15
	Model Tree with (SVR)	43.54

Table 4 :

 4 Table shows the evaluation metrics of the different clustering. One of the HDBSCAN cluster always consist out of outliers.

		Clustering	Di-	Sil-	Calinski-	Da-	Num-
			mension	houette	Harabasz	vies-	ber	of
			Reduction			Bouldin	Clusters
	Hdbscan1	HDBSCAN	UMAP	0.6	3472	1.24	7
	Hdbscan2	HDBSCAN	UMAP	0.55	3907	1.02
	Hdbscan3	HDBSCAN	UMAP	0.62	9937	1.02	7
	Dbscan	DBSCAN	UMAP	0.62	4308	0.36	8
	Kmeans1	K-Means	UMAP	0.64	13732	0.39	7
	Kmeans2	K-Means	UMAP	0.7	9542	0.33	3


Frederick Birtel
email: frederick.birtel@fir.rwth-aachen.de

Achim Kampker

Volker Stich

First Steps to the Digital Shadow of Maintenance Services' Value Contribution

Keywords: Industrie 4.0, digital shadow, maintenance services, maintenance value contribution, intelligent maintenance systems

Industrie 4.0 is said to have major positive effects on productivity in manufacturing companies. However, these effects are not visible yet. One reason for this is the lack of understanding of maintenance services as a crucial value contributing partner in production processes, although scientific literature already highlighted the importance of indirect maintenance costs. In order to retrieve the unused potential of maintenance services, a digital shadow in form of a sufficiently precise digital representation is required, providing a data model for the value of maintenance actions so that asset and maintenance strategies can be optimized later on. Using case study research for process manufacturers, the first research contribution of this paper consists of 21 value contributing elements being identified. The second contribution is a reference processes model, showing seven major process steps as well as the required intra-organization interaction on an information technology system level. Therefore, it provides the base for the missing data model shaping the targeted digital shadow of maintenance services' value contribution.

Introduction

The advance of the digital interconnectedness in industrial companies during the past years, also referred to as smart factories or Industrie 4.0, has increased the awareness of manufacturing companies for their own digital transformation in order to keep up with the global competition. However, that interconnection between cyber-physical systems and people, combined with an unlimited access to real-time data [START_REF] Lee | Industrial AI. Applications with sustainable performance[END_REF][START_REF]Industrie 4.0 Maturity Index. Die digitale Transformation von Unternehmen gestalten. acatech STUDIE[END_REF] that could drastically accelerate critical entrepreneurial decision making [START_REF] Porter | How Smart Connected Products Are Transforming Competition[END_REF], has not led to an increase in productivity across those companies so far [START_REF] Rammer | Produktivitätsparadoxon im Maschinenbau[END_REF], as often Industrie 4.0 use cases do not become successful business cases especially in maintenance departments.

One reason for that are still dominating isolated information technology (IT) systems within the companies, which prevent information getting from the information source to a decision-making stage, so that also intelligent maintenance applications or assistance systems [START_REF] Lee | Industrial AI. Applications with sustainable performance[END_REF] cannot fulfill the promised potential [START_REF] Bienzeisler | Industrie 4.0 Ready Services[END_REF]. Another reason is the lack of understanding of maintenance services as a crucial value contributing partner in manufacturing companies' production processes. Value thereby represents the positive change of a manufacturing company's balance sheet in general or even the increase in profit (EBIT) or return rates (e.g. ROCE). This results in investments for maintaining the assets, e.g. maintenance personnel and equipment, being held back. Additionally, a regular adjustment of the cost-optimal mix of maintenance strategies consisting of reactive, preventive and even predictive measures according to the asset condition and risk assessment are not executed effectively [START_REF] Pawellek | Integrierte Instandhaltung und Ersatzteillogistik. Vorgehensweisen, Methoden, Tools[END_REF]. Both factors may not result in an increase of direct maintenance costs from spare parts, labor and external service providers in the short term, but they lead to a significant increase of indirect maintenance costs. These are the result of, for instance, lost contribution margins due to unplanned downtimes [START_REF] Milojevic | Digital Industrial Revolution with Predictive Maintenance. Are European businesses ready to streamline their operations and reach higher levels of efficiency? CXP Group[END_REF], subsequent contractual penalties or even loss of customer orders [START_REF] Kuhn | Nachhaltige Instandhaltung. Trends, Potenziale und Handlungsfelder nachhaltiger Instandhaltung[END_REF]. Thus, entrepreneurial decisions, e.g. choosing the right maintenance strategy mix and making sustainable asset investments, are made on an incomplete information basis [START_REF] Lundgren | Quantifying the Effects of Maintenance a Literature Review of Maintenance Models[END_REF].

The main reason for that lack of information or visibility is not technological but organizationally driven [START_REF] Roy | Continuous maintenance and the future -Foundations and technological challenges[END_REF]. Currently, there is no approach available to identify that information for costs or effects in manufacturing companies on an information system level, which is required to determine the value maintenance services can add to the company's balance sheet. So, in terms of the concept of Industrie 4.0, a digital shadow is required in order to retrieve the full potential of a company´s digital transformation [START_REF] Harland | Gestaltung des Digitalen Schattens für Instandhaltungsdienstleistungen im Maschinen-und Anlagenbau[END_REF].

2

Theoretical Background

The Digital Shadow as Part of the Internet of Production

The digital shadow represents "an integrated database that generates a sufficiently precise digital representation of the whole enterprise in real-time", primarily on a process level [START_REF] Schuh | The Digital Shadow of Services: A Reference Model for Comprehensive Data Collection in MRO Services of Machine Manufacturers[END_REF] and therefore must be differentiated from the digital twin as a simulationcapable digital representation of a physical asset [START_REF] Uhlemann | The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4[END_REF]. In that context, the digital shadow serves as a transition layer within the 'Internet of Production' providing a domain-specific data model connecting different IT systems [START_REF] Schuh | Change Request im Produktionsbetrieb[END_REF]. Thus, the digital shadow does not represent an optimizing application in the first place but forms the required basis for a variety of analytical applications (e.g. maintenance/asset investment strategy) improving the management's data based decision making [START_REF] Bauernhansl | WGP-Standpunkt Industrie 4.0[END_REF]. The concept of the digital shadow was applied to production environments in the first place in order to increase the visibility in factory and production planning [START_REF] Bauernhansl | The Digital Shadow of production -A concept for the effective and efficient information supply in dynamic industrial environments[END_REF]. It was then adopted to industrial services and provided a reference process model as initial point for a sufficient data model which focused on the service execution level only [START_REF] Harland | Gestaltung des Digitalen Schattens für Instandhaltungsdienstleistungen im Maschinen-und Anlagenbau[END_REF]. However, a comparable approach defining value contributing elements, a reference process model as well as a data model for the value of maintenance services is still unavailable.

Requirements for a Digital Shadow for Maintenance Services

The digital shadow itself does not optimize decision making for maintenance services, but it serves as a tool to provide the required visibility. In order to develop it, three major steps are required [START_REF] Schuh | Change Request im Produktionsbetrieb[END_REF]. First, it is required to identify and clarify elements representing the value contribution of maintenance services and to link them to specific information [START_REF] Harland | Gestaltung des Digitalen Schattens für Instandhaltungsdienstleistungen im Maschinen-und Anlagenbau[END_REF]. Second, the elements and referred information have to be linked to a reference process, fulfilling the purpose of reutilization [START_REF] Schütte | Grundsätze ordnungsmäßiger Referenzmodellierung. Konstruktion konfigurations-und anpassungsorientierter Modelle[END_REF]. Based on the reference process and its steps, a data model can be derived completing the definition of the digital shadow [START_REF] Schuh | The Digital Shadow of Services: A Reference Model for Comprehensive Data Collection in MRO Services of Machine Manufacturers[END_REF]. This paper contributes to the identification of the value contribution elements/information and the resulting reference process model while the development of the data model for the required time series observation of the value elements as well optimized decision making based on that digital shadow are explicitly not subject of this paper (see level three and four in Fig. 1).

Fig. 1. Concept of the digital shadow of the value contribution of maintenance services

In scientific literature, value contributing elements of maintenance services have already been subject of discussions [START_REF] Lorenz | Wertorientierte Gestaltung der betrieblichen Instandhaltung, 1st edn. Schriftenreihe Rationalisierung[END_REF], though that previous work is generally incomplete in two specific areas. First, the value contributing elements are considered from a direct and indirect cost perspective, respectively, while positive effects of maintenance actions on the company value are not included [START_REF]Wertschöpfendes Instandhaltungs-und Produktionsmanagement. Erfolgreich durch Innovationen in Management und Technologie. Praxiswissen für Ingenieure : Instandhaltung[END_REF]. Second, the determined elements are incomplete as they are restricted to a close maintenance perspective only and do not take into account effects appearing in other organizational units of the company (e.g. loss of customer orders monitored in the sales department). This is enhanced by the effect that value contributing elements additional to known maintenance-related downtime costs become visible with a time delay (e.g. shortened asset lifetime). Therefore this paper points out the value contributing elements particularly considering intra-organizational boundaries as well as time delays.

In order to form the targeted data model for the digital shadow of maintenance services' value contribution, a reference process model is required containing core functions (process steps), cross-section functions (e.g. reports, contracts) and a data management (e.g. resources, employees) [START_REF] Kallenberg | Ein Referenzmodell für den Service in Unternehmen des Maschinenbaus[END_REF]. Regarding the reference process and the related process steps, a variety of models is already described. Most of them refer to or are similar to the previous work of German standardization initiatives (e.g. DIN31051) [START_REF]DIN 31051: Grundlagen der Instandhaltung[END_REF]. However, those process models do not consider the existing intra-organizational exchange of data and information required to determine the value contribution of maintenance services leading to an insufficient data management. Thus, this paper introduces a reference process model focusing on the core process steps which provide interfaces to the value contributing elements and referring information across the organization and information systems. Therefore, this paper contributes the first two levels of the concept of the digital shadow as summarized in Fig. 1.

Methodology

In order to address the given problem of defining elements and a reference process model for the digital shadow of maintenance services, the work in this paper is based on the approach of case study research [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. Following its eight steps, this method ensures the practical applicability of the qualitative results later on.

For this paper, we chose cases from process manufacturers. Here, the added value of maintenance services in that industry becomes clear because unplanned downtimes, for instance, have an immediate impact on the production process, thus increasing the potential of the value maintenance services can add to the balance sheet of a company. Additionally, the value contribution is expected to be high due to the loss of contribution margins due to production and quality losses [START_REF] Parida | Maintenance Productivity and Performance Measurement[END_REF]. Other structuring factors were the numbers of employees indicating the size of the companies as well as the complexity of assets used within the manufacturing process. The cases and qualitative results were recorded in form of triangulation [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. The identification and assessment of the value-adding elements, referred information and the reference process model were jointly modeled by experts from the selected companies, following a semi-structured interview. Moreover, the data was supplemented by internal company data, e.g. operating instructions, balance sheets and a comprehensive literature analysis. Seven case studies, as shown in Table 1, were analyzed, but no significant addition was made to the examined contributing elements, information and process steps after the fifth case study, because the findings were repetitive. Thus, the termination criteria (eighth step of the case study research approach) were met.

4

Modeling and Results

Identifying and Evaluating Value Contributing Elements

Following the methodology of case study research for process manufacturers, 21 elements were identified to have an impact on the value contribution of maintenance services. Those elements were then classified into two groups. The first group comprises all the elements which contribute a positive value to the company and should therefore be maximized, such as output and product quality increase. The other group contains elements causing additional costs that consequently influence the companies' value negatively (e.g. loss of contribution margins or contractual penalties due to missed delivery dates). In order to identify the most relevant elements out of the two groups, the elements were evaluated qualitatively according to their effect size regarding the value of the company and their significance, meaning the extent to which they can be influenced by maintenance measures directly or indirectly as shown by the framework downtime consequential costs [START_REF] Pawellek | Integrierte Instandhaltung und Ersatzteillogistik. Vorgehensweisen, Methoden, Tools[END_REF].

Those elements with a great effect size and high significance to maintenance actions were identified as most relevant while other elements were excluded from further modeling (e.g. commissioning costs and image loss). The results of all identified and evaluated elements are shown in Fig. 2 were the subject of the subsequent design of the reference process model.

Fig. 2. Case study results for value contributing elements

Modelling of a Reference Process

In the next step, the selected elements were first broken down to their information level required to retrieve them on IT system level (s. example information in Fig. 3), also following the approach of case study research and literature review. Then, the elements and referred information were transferred into a reference process model. Fig. 3 shows the seven process steps as well as the involved departments of a company in which the relevant value contributing elements and related information are monitored (e.g. sales, purchasing and logistics). The reference process model considers them as an information source that provides the basis used to assign the elements to maintenance action as part of the shown process steps (s. examples in Fig. 3), although many value contributing elements do not occur within the maintenance departments directly but delayed. Furthermore, the process model's data management identifies all involved information technology (IT) systems, such as the computerized maintenance management system (CMMS), the enterprise resource planning (ERP), the customer relationship management (CRM), the warehouse management (WMS) or the manufacturing execution system (MES) (s. Fig. 1). Consequently, the process model allows the distinct assignment of a value contributing element and their related information (s. Fig. 2) to maintenance actions within a specified period of time. Therefore, it provides a useful basis for the central data model shaping the targeted digital shadow of maintenance services' value contribution Fig. 3. Extract from the reference process for maintenance services within the relevant stakeholder environment

Conclusion and Outlook

This paper demonstrates that intra-organizational visibility is required to uncover timedelayed value contributing effects of maintenance actions in form of a digital shadow of maintenance services' value contribution. Using the research approach of case study research 21, elements are shown and organized by the size of their effect (impact) and the significance of maintenance services for their appearance (s. Fig. 2). Besides the dominating elements of contribution losses due to loss of production or quality, further elements (e.g. efforts for re-planning maintenance, contractual penalties) have been identified, representing an extended scientific understanding of maintenance services' value contribution. Moreover, the paper demonstrates that the information referring to the qualitative evaluated elements are monitored in different departments as well as different IT systems within a company. The presented core function of the reference process model with its seven process steps allows to connect maintenance actions with the value contributing elements and information. It sets up the basis for the data model as last key element of the digital shadow.

Hence, future research can build on the contributed model in order to describe how different IT systems on a database level can interact so that domain-specific requests can be answered with real-time data and information using entity relationship models, for instance. Furthermore, design recommendations are needed for the implementation of a digital shadow in practice. In this context, future work may also specify the identified value contributing elements on an existing KPI (key performance indicators) level so that current IT systems and management models can be adapted to the presented concept more easily. Thus, finally utilizing the digital shadow (s. Fig. 1), e.g. to choose the correct maintenance strategy mix as well as asset management strategy in respect of the whole asset life cycle, could become the key to retrieving the full potential of existing and future Industrie 4.0 solutions for maintenance services in manufacturing companies.

Table 1 .

 1 Overview of selected cases

	No.	Industry	Employees	Asset complexity
	1	Steel	> 10,000	high
	2	Textile	ca. 500	medium
	3	Pharmaceutical	> 10,000	high
	4	Automotive supplier	ca. 2,000	medium
	5	Food	ca. 1,800	low
	6	Food	ca. 1,200	low
	7	Paper	ca. 300	medium

size of the effect high low low high significance ofmaintenance services

	(re-)scheduling maintenance	overtime	special shifts	loss of contribution margins (loss of	: positive value contribution
	program					production)	: negative value
	space less storage			increasing	loss of contribution output quantity asset time of the increasing longer usage	contribution
	smoothed	product quality		margins (loss of
	employee workload		(re-)scheduling production	quality) increased spare
				program	parts costs
		additional		increased
	costs recommissioning	transport costs	reduced spare parts inventories	contractual service provider costs	increased need for warehouse staff
					penalties
						loss of
						order/customer
		image loss			new customer
					acquisition