Gabriel Da Silva
email: gabriel.leal@list.lu

Serapião Leal

Wided Guédria
email: wided.guedria@list.lu

Hervé Panetto
email: herve.panetto@univ-lorraine.fr

Interoperability Assessment: A Systematic Literature Review

Keywords: Collaborative ecosystems, Enterprise interoperability, Interoperability assessment, Assessment criteria, Systematic literature review, Comparative analysis

The development of Interoperability is a necessity for organisations to achieve business goals and capture new market opportunities. Indeed, interoperability allows enterprises to exchange information and use it to seize their shared goals. Therefore, it should be verified and continuously improved. This is the main objective of the Interoperability Assessment (INAS). Indeed, such an assessment aims at determining the strengths and weakness of an enterprise in terms of interoperability. Many surveys and reviews have been proposed in the literature to analyse the existing INAS approaches. However, the majority of these reviews are focusing on specific properties rather than a general view of an INAS. Therefore, this paper proposes a systematic literature review of INAS approaches. The objectives are to identify the relevant INAS approaches and to compare them based on a holistic view based on their similar and different properties (e.g. type of assessment, the used measurement mechanism, and the addressed interoperability barriers). A bibliometric analysis of the selected INAS approaches is also conducted with a discussion of their advantages and limitations.

Introduction

In certain cases, companies are adapting themselves and participating in collaborative networks for responding to new business opportunities, while seamlessly following the dynamics of their environment (e.g. new competitors, new regulations and customers growing needs) [START_REF] Agostinho | Towards a sustainable interoperability in networked enterprise information systems: Trends of knowledge and model-driven technology[END_REF], [START_REF] Romero | Collaborative networked organisations and customer communities: value co-creation and coinnovation in the networking era[END_REF], [START_REF] Leal | Towards a Meta-Model for Networked Enterprise[END_REF]. Admittedly, the collaboration (i.e. sharing assets, pieces of knowledge and core competencies) between stakeholders (e.g. businesses, public administration and customers) allows value co-creation and nourish innovative ideas [START_REF] Romero | Collaborative networked organisations and customer communities: value co-creation and coinnovation in the networking era[END_REF]. For instance, an analysis performed by PricewaterhouseCoopers (PwC) company on behalf of the European Commission [START_REF] Vaughan | Assessing the size and presence of the collaborative economy in Europe -PwC UK Report[END_REF] estimates that at least 275 collaborative platforms have been founded in Europe and that this collaborative economy generated revenues of nearly €4billons within Europe in 2015.

In this collaborative context, the Interoperability is a prerequisite that must be satisfied [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]. Predominantly, it refers to the ability of systems to exchange information and use the information that has been exchanged [START_REF][END_REF]. Focusing on the enterprise context, the interoperability refers to the "ability of interaction between enterprises" [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]. Indeed, interoperability can happen between different levels of the interacting enterprises (e.g. processes, services and data) and their related enterprise systems (e.g. data storage devices, software applications, etc.) [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF], [START_REF]Interoperability Developments for Enterprise Application and Software (IDEAS) -roadmaps[END_REF], [START_REF] Berre | The ATHENA Interoperability Framework[END_REF]. As soon as this ability is not attained, it becomes a problem that must be solved [START_REF] Naudet | Towards a systemic formalisation of interoperability[END_REF]. Indeed, interoperability problems can influence drastically the performance and the outcomes of business networks. For example, a study made by the U.S. Department of Commerce Technology Administration in 2004 estimates a cost of U$15.8 billion related to the inadequate interoperability between systems in the U.S. Capital Facilities Industry [START_REF] Gallaher | Cost analysis of inadequate interoperability in the US capital facilities industry[END_REF]. The West Health Institute estimated in 2013 a potential of U$ 30 billion addressable waste per year related to the lack of interoperability across segments of healthcare in the U.S. [START_REF]The value of medical device interoperability[END_REF]. Finally, a 2015 report by PwC, commissioned by the Global System for Mobile Communications Association (GSMA) [START_REF] Pwc | Realising the benefits of mobileenabled IoT solutions[END_REF], [START_REF]Digital Healthcare Interoperability[END_REF] estimates that digital health could save €99 billion in healthcare costs to the European Union Gross Domestic Product due to interoperability improvements.

Therefore, for avoiding interoperability problems and for developing such ability between systems, enterprises should be aware of their strengths and weaknesses concerning interoperability. Thus, enterprises could benefit from an Interoperability Assessment (INAS) [START_REF] Panetto | New perspectives for the future interoperable enterprise systems[END_REF]. It includes the identification of potential problems and possible related solutions. Indeed, such an assessment determines their as-is state, and it provides a roadmap toward the to-be state. In other words, an INAS supports enterprises in planning personalised transformations and reaching better situations.

Numerous INAS approaches were defined during the past three decades for addressing one or more layers of interoperability (e.g. organisational, semantic and technical [START_REF]European Interoperability Framework -Implementation Strategy[END_REF]) such as the Levels of Conceptual Interoperability Model [START_REF] Tolk | The Levels of Conceptual Interoperability Model[END_REF], the maturity levels for interoperability in digital government [START_REF] Gottschalk | Maturity levels for interoperability in digital government[END_REF], the method for measuring supply chain interoperability [START_REF] Chalyvidis | A method for measuring supply chain interoperability[END_REF], the methodology for prior evaluation of interoperability [START_REF] Camara | Methodology for Prior Evaluation of Interoperability[END_REF], etc.

Surveys and reviews have been proposed in the literature to analyse existing INAS approaches and compare the types of INAS they are dealing with (e.g. focusing on the maturity of systems, compatibility between systems or systems' performance) [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF], [START_REF] Yahia | Contribution à l'évaluation de l'interopérabilité sémantique entre systèmes d'information d'entreprise: Application aux systèmes d'information de pilotage de la production[END_REF], the measurement mechanisms (e.g. qualitative and quantitative measures) [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF], [START_REF] Yahia | Contribution à l'évaluation de l'interopérabilité sémantique entre systèmes d'information d'entreprise: Application aux systèmes d'information de pilotage de la production[END_REF], [START_REF] Ford | A survey on interoperability measurement[END_REF] and the covered layers of interoperability [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF], [START_REF] Yahia | Contribution à l'évaluation de l'interopérabilité sémantique entre systèmes d'information d'entreprise: Application aux systèmes d'information de pilotage de la production[END_REF], [START_REF] Cornu | Customizable Interoperability Assessment Methodology to support technical processes deployment in large companies[END_REF], [START_REF] Ford | A survey on interoperability measurement[END_REF], [START_REF] Rezaei | Interoperability evaluation models: A systematic review[END_REF], [START_REF] Panetto | Towards a classification framework for interoperability of enterprise applications[END_REF].

However, existing surveys do not consider a holistic view of the INAS. For instance, Cestari et al. [START_REF] Cestari | Interoperability Assessment Approaches for Enterprise and Public Administration[END_REF] focus on maturity models for the public administration domain. Nevertheless, the authors do not discuss the measurement mechanisms or the interoperability layers covered by the reviewed models. Guédria et al. [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF] address different INAS application domains (e.g. health and military) but focus only on interoperability maturity models. Ford et al. [START_REF] Ford | A survey on interoperability measurement[END_REF] consider both qualitative and quantitative measurement mechanisms but do not explicitly differentiate the types of assessment that are being adopted by the reviewed approaches. Further, to the best of our knowledge, the current INAS surveys are explicitly defining neither a process for selecting papers nor the criteria for comparing the selected INAS approaches.

Therefore, the objective of this paper is to select relevant INAS approaches based on a set of criteria and compare them. For this purpose, section 2 studies the main interoperability layers based on existing interoperability frameworks; as well as the INAS related works for identifying the properties of such an assessment. Section 3 defines a search process for selecting papers. It includes the definition of the steps to be followed, the keywords, the libraries to be adopted, the paper's inclusion, and exclusion criteria. The selected articles and the analysis considering their year of publication and research domains are also presented. Based on that, section 4 defines a set of comparison criteria and analyses the selected INAS literature. Section 5 discusses the findings of the comparative analysis and highlights INAS approaches' limitations. Research perspectives for each identified limitation are also presented. Finally, section 6 concludes the paper.

Research Context

This section presents the central concepts of the Enterprise Interoperability domain. We also describe the importance of INAS approaches and their properties.

Interoperability Frameworks

Several frameworks have been proposed in the literature to describe the Interoperability field. The primary purpose of an Interoperability framework is to provide an organising mechanism so that knowledge of Enterprise Interoperability can be expressed in a more structured manner [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]. Such structures shall strive to connect to all aspects of inquiry (e.g. problem definition, literature review, data collection and analysis) and can act like maps that might give coherence to conceptual theory development and empirical inquiry [START_REF] Jardim-Goncalves | Systematisation of Interoperability Body of Knowledge: the foundation for Enterprise Interoperability as a science[END_REF].

For instance, the ATHENA project provides the ATHENA Interoperability Framework (AIF) [START_REF]A4.2: Specification of Interoperability Framework and Profiles, Guidelines and Best Practices[END_REF] and an associated reference architecture for capturing the research elements and solutions to interoperability issues. The European Interoperability Framework (EIF) [START_REF]European Interoperability Framework -Implementation Strategy[END_REF] describes the different interoperability levels and focus on the interoperability between public entities from various government around Europe. It also provides a collection of recommendations for developing the interoperability between local and international public administrations. Further, the INTEROP Network of Excellence (INTEROP NoE) project defines the Framework for Enterprise Interoperability (FEI) [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF], later becoming the standard ISO 11354:1 [START_REF]2011 Advanced automation technologies and their applications --Requirements for establishing manufacturing enterprise process interoperability --Part 1: Framework for enterprise interoperability[END_REF]. FEI highlights the barriers that might be encountered within the enterprise's concerns regarding interoperability. The Classification Framework for Interoperability [START_REF] Panetto | Towards a classification framework for interoperability of enterprise applications[END_REF] proposes a classification of the different types of interoperability associated with systems' models. Finally, the reference model for sustainable interoperability in networked enterprises [START_REF] Jardim-Goncalves | A reference model for sustainable interoperability in networked enterprises: towards the foundation of EI science base[END_REF] provides formal methods categorised in interoperability practices layers. For the interested reader, more detailed reviews on enterprise interoperability frameworks can be found in [START_REF] Jardim-Goncalves | Systematisation of Interoperability Body of Knowledge: the foundation for Enterprise Interoperability as a science[END_REF] and [START_REF] Chen | Architectures for enterprise integration and interoperability: Past, present and future[END_REF].

For this research work, we consider four main layers of interoperability as described in the EIF [START_REF]European Interoperability Framework -Implementation Strategy[END_REF], which are: The semantic layer of interoperability subsuming the information syntactic (which concerns the information format to be exchanged) and information semantics (which ensures the meaning of the information). The technical layer covers the applications and infrastructures linking systems and services. It includes interface specifications, interconnection services, data integration services, data presentation and exchange, and secure communication protocols. The organisational layer refers to the way in which systems align their processes, responsibilities and expectations to achieve commonly agreed goals. Finally, the legal layer englobing legislations issues involving the alignment of higher enterprise functions or government policies, usually to be expressed in the form of legal elements and business rules.

Interoperability problems arise when two or more incompatible systems are put in relation [START_REF] Naudet | Towards a systemic formalisation of interoperability[END_REF]. In general, these incompatibilities or mismatches are related to the interoperability layers. For example, two enterprises can have different management styles, which can lead to problems related to the organisational layer. Companies can also employ different concepts and representations for expressing the same meaning, what can cause problems related to the semantic layer.

Thus, for describing these incompatibilities, we consider the three interoperability barriers defined by FEI [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]. The conceptual barriers refer to the modelling at the high level of abstraction such as the models of a company. The technological barriers concern over the lack of a set of compatible standards to allow using heterogeneous computing techniques for sharing and exchanging data between two or more systems. Finally, the organisational barriers regard the incompatibilities of organisation structure, business rules and management techniques implemented in two interoperating enterprises. Note that, legislations incompatibilities are also included in this last barrier.

Moreover, there are various views of the enterprise where interoperations can take place [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]. For describing these different viewpoints, we consider the interoperability concerns proposed by FEI [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]. The business concern refers to work in an orchestrated way at the levels of the organisation despite, for example, the different modes of decision-making, and methods of work, legislation, culture and commercial strategies. The process concern aims at making various processes work together. The services concern aims at identifying, composing and operating together various applications as well as systems' interfaces. Finally, the data concern refers to finding and sharing information coming from different databases, and which can furthermore reside on distinct devices with different operating systems and databases management systems.

Note that each interoperability concern is related to all interoperability layers, consequently with their associated barriers. Table 1 presents the cross section between the Interoperability Layers/Barriers and Concerns.

Table 1. Examples of the Interoperability barriers related to the enterprises concerns. Adapted from [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF].

Interoperability barriers and layers

Conceptual (Syntactic and Semantic) Technological

Organisational (and Legal)

Interoperability concerns

Business

Interoperability Assessment

For avoiding potential problems and for better support of enterprise collaboration, the interoperability between enterprise systems needs to be continuously improved [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF], [START_REF] Ford | A survey on interoperability measurement[END_REF]. To improve such ability, organisations should be conscious of their current situation, regarding interoperability. This, in particular, is the objective of an INAS. Indeed, assessing the enterprises' systems ability to interoperate is frequently the initial step toward a new collaboration development (e.g. the creation of a new network, the arrival of a new member) or an improvement program (e.g. reducing the negative impacts caused by interoperability problems or future transformations). In the following sections, we present the different properties of an INAS.

Types of assessment

There are three distinct types of interoperability [START_REF]Deliverable DI.3: Enterprise Interoperability Framework and knowledge corpus[END_REF]: The potentiality assessment which appraises the interoperability of a system towards its environment. The objective of this analysis is to evaluate the potentiality (also called maturity) of a system to adapt and to accommodate dynamically to overcome possible barriers when interacting with a potential partner. The compatibility assessment evaluates the interoperability between two known systems before or after any interoperation. The most crucial task is to analyse the current state of both concerned systems to identify the conflicts that cause or may cause problems.

Finally, the performance assessment evaluates the interoperations during the run-time. It considers the costs induced by implementing interoperable applications, the duration between the time at which information is requested and the time at which the requested information is used and the quality of the exchange, the quality of use and, the quality of conformity.

Measurements mechanisms

Regarding the measurement mechanisms, there are two main types [START_REF] Ford | A survey on interoperability measurement[END_REF], [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF]: The Qualitative measures are mainly subjective. In most cases, this kind of measure uses a rating scale composed of linguistic variables (e.g. "Good", "Optimized" and "Adaptive") for qualifying a system. It is mostly used by the maturity models, which are approaches designed to assess the quality (i.e. competency, capability, level of sophistication) of a selected domain based on a more or less comprehensive set of criteria [START_REF] Bruin | Understanding the Main Phases of Developing a Maturity Assessment Model[END_REF].

The quantitative measures define numeric values to characterise the interoperability. In general, the rating scale is from 0 to 100%. For example, some approaches use equations for determining the interoperability based on the "real / expected" ratio [START_REF] Yahia | Formal measures for semantic interoperability assessment in cooperative enterprise information systems[END_REF], [START_REF] Neghab | Performance evaluation of collaboration in the design process: Using interoperability measurement[END_REF], the interoperation performance indicators [START_REF] Blanc | Evolution management towards interoperable supply chains using performance measurement[END_REF], [START_REF] Camara | A methodology for the evaluation of interoperability improvements in interenterprises collaboration based on causal performance measurement models[END_REF], and others. These measures are commonly applied to compatibility and performance assessments.

Coverage of interoperability layers and concerns

A definite number of criteria should be satisfied to deliver a higher quality of interoperability. To categorise the interoperability criteria, one has to associate them with the different interoperability layers and concerns, which are described by the adopted interoperability framework (e.g. FEI and EIF). This association allows the identification of the barriers that such criteria are related. It is also crucial to have an understanding of the relations between the evaluation criteria from different layers, to support the identification of influences on the overall system if any criterion is not achieved.

Thus, the coverage of criteria from multiple interoperability layers and their interdependencies are necessary when conducting an INAS as they provide a holistic view on what to evaluate and what are the potential impacts of a negative evaluation.

Systematic Literature review

The systematic literature review (SLR) presented in this research paper is based on the guidelines defined by Kitchenham [START_REF] Kitchenham | Procedures for Performing Systematic Reviews[END_REF]. The objectives of an SLR are to identify any gaps in current research in order to suggest areas for further investigation and to provide a background in order to position new research activities appropriately [START_REF] Kitchenham | Procedures for Performing Systematic Reviews[END_REF]. The main advantages are that such review is undertaken following a predefined search strategy and presents evidence concerning the data sources, the selection and analysis criteria.

The procedure of this SLR is the fowling: (i) Define the paper selection process (including search questions, keywords, sample sources and the paper selection steps); (ii) Present the papers selection and the bibliographic analysis (i.e. points out the year of publication, the publishers and the addressed domains); (iii) Select INAS approaches (iv) Define comparison criteria and compare the selected approaches; Finally, (v) Report and discuss findings.

In next section, the paper search process and results are presented. The comparison is done in section 4.

Paper selection process

First, we define questions for supporting and directing the papers selection. The questions are:

 What are the papers proposing approaches for assessing interoperability and identifying potential barriers or negative impacts within a network of systems?  Where these papers are published (e.g. journals, conferences)?  What are the addressed domains (e.g. manufacturing, healthcare)?  When such papers have been proposed? Next, we perform the papers sampling, i.e. we identified relevant papers from the related literature. To cover the overall relevant studies, we search for papers by querying four digital libraries: ScienceDirect, Taylor & Francis Online, Springer and Web of Science.

The keywords are defined based on an iterative process, which is described as follows. First, we query the digital libraries with the keyword "Interoperability Assessment". A total of 135 papers are identified. From these papers, we extract the most used keywords (i.e. repeated more than five times) and the most repeated terms (i.e. repeated more than fifteen times) in their titles and abstract. To do so, we download the papers metadata (i.e. title, year of publication, authors, abstract and keywords) in .RIS format. Next, we perform a data mining on the extracted metadata in order to identify the relevant keywords using VOSviewer software [START_REF] Van Eck | Software survey: VOSviewer, a computer program for bibliometric mapping[END_REF]. VOSviewer is used to construct and visualise co-occurrence networks of important terms extracted from the metadata. After the co-occurrence analysis, two more keywords are defined: "Interoperability Maturity Model" and "Interoperability Evaluation".

In the next step, we query the four digital libraries again with these two new keywords. We identify 88 and 81 papers related to Interoperability Maturity Model and Interoperability Evaluation, respectively. Further, we extract the metadata from these new 169 papers. Before performing another co-occurrence analysis, we exclude the metadata from the redundant papers. Consequently, the new analysis considers 263 papers, i.e. the 135 from the previous analysis more 128 papers specifically related to the two new keywords.

We define four more keywords and terms based on the analysis, which are Interoperability Measurement, Interoperability Analysis, Interoperability Methodology and Interoperability Performance Evaluation. Further, we identify 56 papers related to Interoperability Measurement, 134 papers associated to Interoperability Analysis, 57 papers to Interoperability Methodology and finally, 8 papers related to Interoperability Performance Evaluation.

Finally, we perform a last co-occurrence analysis considering the total of non-redundant papers, which is equal to 418. Figure 1 and Figure 2 present the most occurred keywords and key terms from the identified papers. As additional search strategy, we also include 29 papers using the "snowball sampling" technique [START_REF] Wohlin | Guidelines for snowballing in systematic literature studies and a replication in software engineering[END_REF] whereby we consider the referrals of assessment approaches made by experts, as well as the most cited papers in the existing INAS surveys and reviews. A total of 447 publications are identified at the end of this sampling phase.

Furthermore, the selection of the papers to be analysed and compared is done in two steps. In the first one, we apply for each one of them, the inclusion and exclusion criteria corresponding to the step St1 as described in Table 2. In this step, we only consider the metadata of the papers. The second step includes the reading of the full-text of the selected papers. To select which papers are considered, we apply the criteria related to the step St2 as described in Table 3.

Table 3. The Inclusion and exclusion criteria for step 2

Step Inclusion criteria Exclusion criteria

St2

In the case where the paper does not include the term "interoperability", it should addresses the interaction and connectivity among systems, focusing on the exchange and sharing of information

Paper presenting at least one of the key concepts (interoperability, enterprise interoperability, etc.), but not considering the term "assessment" (and its variants) Paper proposing a methodology, a method or model for assessing interoperability and also proposing measurement mechanism

Once the papers are selected, we classify them by year, the type of publication (e.g. journal article, conference proceedings, etc.) and the addressed domain (e.g. military, industry, etc.). The paper selection process is depicted in Figure 4.

Paper analysis and results

The initial search reveals 418 references from the digital libraries, and 29 papers based on the snowball sampling. From the 447 considered papers, we apply the inclusion and exclusion criteria from Step St1, as described in Table 2. Therefore, we first exclude those papers that are not available, not written in English and papers that are reviews, surveys and comparative analysis of existing INAS approaches. The resulting number of considered papers are 419 in this phase.

Moving forward, we analyse the rest of the papers, considering their title, abstract and keywords. The number of considered papers drop to 139 in total. Moreover, after reading and analysing the full text of the remaining papers, we select 72 of them. Table 4 shows the results from different phases of the selection process. Considering the domains addressed by the analysed papers, 41 of them focus on the Industry domain (including manufacturing supply chains, service providers, etc.), 16 consider the Military domain and 6 papers address Information Technology (IT) systems without considering a specific domain. Finally, 9 papers cover other domains such as health, public administration and crisis management. Table 5 shows the papers classification according to their addressed domains.

Table 5. Selected papers and their associated domains.

Domain Reference

Military [START_REF] Tolk | The Levels of Conceptual Interoperability Model[END_REF], [START_REF] Hamilton | An Interoperability Roadmap for C4ISR Legacy Systems[END_REF], [START_REF] Wang | The Levels of Conceptual Interoperability Model: Applying Systems Engineering Principles to M&S[END_REF], [START_REF] Tolk | Supporting Network Enabled Capability by extending the Levels of Conceptual Interoperability Model to an interoperability maturity model[END_REF], [START_REF] Clark | Organisational Interoperability Maturity Model for C2[END_REF], [START_REF] Fewell | Organisational Interoperability: Evaluation and Further Development of the OIM Model[END_REF], [START_REF] Fewell | Evaluation of Organisational Interoperabiity in a Network Centric Warfare Environment[END_REF], [START_REF] Kingston | An Organisational Interoperability Agility Model[END_REF], [START_REF]C4ISR, Levels of Information System Interoperability (LISI)[END_REF], [START_REF] Ford | The Interoperability Score[END_REF], [START_REF] Ford | Measuring system interoperability[END_REF], [START_REF] Ford | A General Method of Measuring Interoperability and Describing Its Impact on Operational Effectiveness[END_REF], [START_REF] Leite | Interoperability Assessment[END_REF], [START_REF] Amanowicz | Military communications and information systems interoperability[END_REF], [START_REF] Lavean | Interoperability in Defense Communications[END_REF], [START_REF] Mensh | The Quantification of Interoperability[END_REF] Industry [START_REF] Chalyvidis | A method for measuring supply chain interoperability[END_REF], [START_REF] Camara | Methodology for Prior Evaluation of Interoperability[END_REF], [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF], [START_REF] Cornu | Customizable Interoperability Assessment Methodology to support technical processes deployment in large companies[END_REF], [START_REF] Yahia | Formal measures for semantic interoperability assessment in cooperative enterprise information systems[END_REF], [START_REF] Neghab | Performance evaluation of collaboration in the design process: Using interoperability measurement[END_REF], [START_REF] Camara | A methodology for the evaluation of interoperability improvements in interenterprises collaboration based on causal performance measurement models[END_REF], [START_REF] Daclin | Writing and verifying interoperability requirements: Application to collaborative processes[END_REF], [START_REF] Guédria | A Maturity Model for Enterprise Interoperability[END_REF], [START_REF] Guédria | A Maturity Model Assessing Interoperability Potential[END_REF], [START_REF] Guédria | Enterprise Interoperability Maturity: A Model Using Fuzzy Metrics[END_REF], [START_REF] Guédria | Maturity Model as Decision Support for Enterprise Interoperability[END_REF], [START_REF]SCAMPI Upgrade Team, Standard CMMI® Appraisal Method for Process Improvement (SCAMPISM) A, Version 1.3: Method Definition Document[END_REF], [START_REF]CMMI Product Team, Capability Maturity Model® Integration for Services, Version 1.3[END_REF], [START_REF]CMMI Product Team, Capability Maturity Model® Integration for Acquisition, Version 1.3[END_REF], [START_REF]CMMI Product Team, Capability Maturity Model® Integration for Development, Version 1.3[END_REF], [START_REF] Chapurlat | Interoperability Constraints and Requirements Formal Modelling and Checking Framework[END_REF], [START_REF] Mallek | An Approach for Interoperability Requirements Specification and Verification[END_REF], [START_REF] Mallek | The application of interoperability requirement specification and verification to collaborative processes in industry[END_REF], [START_REF] Mallek | Enabling model checking for collaborative process analysis: from BPMN to 'Network of Timed Automata[END_REF], [START_REF] Yahia | Semantics enactment for interoperability assessment in enterprise information systems[END_REF], [START_REF] Fang | A service interoperability assessment model for service composition[END_REF], [START_REF] Chen | Barriers Driven Methodology for Enterprise Interoperability[END_REF], [START_REF] Daclin | Methodology for Enterprise Interoperability[END_REF], [START_REF] Daclin | Developing enterprise collaboration: a methodology to implement and improve interoperability[END_REF], [START_REF] Chalyvidis | Using Supply Chain Interoperability as a Measure of Supply Chain Performance, Supply Chain Forum[END_REF], [START_REF] Maheshwari | Reconceptualizing measuring, benchmarking for improving interoperability in smart ecosystems: The effect of ubiquitous data and crowdsourcing[END_REF], [START_REF] De Soria | Developing an enterprise collaboration maturity model: Research challenges and future directions[END_REF], [START_REF] Alonso | Enterprise Collaboration Maturity Model (ECMM): Preliminary Definition and Future Challenges[END_REF], [START_REF] Cuenca | Structural elements of coordination mechanisms in collaborative planning processes and their assessment through maturity models: Application to a ceramic tile company[END_REF], [START_REF] Camara | A Methodology for Interoperability Evaluation in Supply Chains based on Causal Performance Measurement Models[END_REF], [START_REF] Campos | Maturity Model for Interoperability Potential Measurement[END_REF], [START_REF] Cornu | Interoperability Assessment in the Deployment of Technical Processes in Industry[END_REF], [START_REF] Li | MFI Based Interoperability Measurement of Business Models in Service-Based Enterprises[END_REF], [START_REF] Sseggujja | Towards Domain Ontology Interoperability Measurement[END_REF], [START_REF] Daclin | Enterprise interoperability measurement -Basic concepts[END_REF], [START_REF] Leal | Using Formal Measures to Improve Maturity Model Assessment for Conceptual Interoperability, On the Move to Meaningful Internet Systems[END_REF], [START_REF] Leal | An approach for interoperability assessment in networked enterprises[END_REF], [START_REF] Leal | Towards a Semi-automated Tool for Interoperability Assessment: An Ontology-Based Approach[END_REF], [START_REF] Leal | Assessing Interoperability Requirements in Networked Enterprises: A Model-Based System Engineering Approach[END_REF] IT System (No specific domain) [START_REF] Rezaei | An interoperability model for ultra large scale systems[END_REF], [START_REF] Rings | A generic interoperability testing framework and a systematic development process for automated interoperability testing[END_REF], [START_REF] Bhuta | A Framework for Identification and Resolution of Interoperability Mismatches in COTS-Based Systems[END_REF], [START_REF] Basson | Qualitative Evaluation of Manufacturing Software Units Interoperability Using ISO 25000 Quality Model[END_REF], [START_REF] Vito | Assessing interoperability of access equipment for broadband networks[END_REF], [START_REF] Saturno | Evaluation of Interoperability between Automation Systems using Multi-criteria Methods[END_REF] Others (e.g. Crisis Management, Public Administration, e-Health) [START_REF] Riz | Interoperability Assessment in Health Systems Based on Process Mining and MCDA Methods[END_REF], [START_REF] Bharambe | Use of geo-ontology matching to measure the degree of interoperability[END_REF], [START_REF]Health Information Systems Interoperability Maturity Toolkit: Users' Guide[END_REF], [START_REF]Health Information Systems Interoperability Maturity Toolkit: Assessment Tool[END_REF], [START_REF] Knight | Maturity model for advancing smart grid interoperability[END_REF]; [START_REF] Da Silva Avanzi | A framework for interoperability assessment in crisis management[END_REF], [START_REF] Gottschalk | Maturity levels for interoperability in digital government[END_REF], [100], [START_REF]Interoperability Maturity Assessment of a Public Service: IMAPS v1.1.1 User Guide[END_REF] Next section presents the INAS approaches derived from the selected 72 papers.

INAS selection approach and results

While studying the selected papers from section 3.2, we observe that some of them are addressing the same approach. Considering this, we identify 38 assessment approaches based on the 72 considered papers. For conducting the comparative analysis, we select only the ones that are demonstrated or evaluated through a real or illustrative application.

Indeed, these approaches provide more information about its applicability, usefulness and effectiveness. From the 38 identified INAS approaches, 22 of them have at least one associated publication where the approach is applied to a real case. Table 6 and Table 7 present the selected 22 approaches and their 49 related publications. The approaches that are not considered are reported in Table A1 (see Appendix 1).

Comparative analysis of the selected INAS approaches

In this section, we define the comparison criteria. It is followed by the comparative analyses considering the defined criteria.

Defining the comparison criteria

The first criterion that we consider in this analysis is the application of the INAS approach. It supports identifying which type of systems are assessed and in which cases the approaches can be applied. Hence, we classify an approach based on two types of assessed system: Non-Human Resources subsuming hardware and software (e.g. Manufacturing Executing Systems (MES) and Healthcare Information Systems (HIS)) and Entities including all human and non-human resources (e.g. enterprises, hospitals and governmental departments).

We also identify if the approach can be of General Use (i.e. any type of entity or non-human resource can be considered) or is for Specific Use (i.e. only a certain type of system can be considered e.g. only government entities or only HISs). Next, we highlight if the INAS approach is demonstrated using a Real Scenario (i.e. based on real-world entities and resources) or based on abstract and Illustrative Examples.

The second criterion regards the type of assessment. This criterion is selected for identifying the types of assessment addressed by the INAS approaches. Therefore, we classify and compare the selected INAS approaches according to the three types of assessment described in section 1.2.4: Potentiality, Compatibility and Performance. Besides comparing the current state of the art regarding this criterion, this analysis provides us an insight into the evolution and importance given for each one of the considered type of assessment.

The third criterion refers to the coverage of interoperability layers/barriers. This criterion is essential as it supports the verification of INAS approaches dealing with one or more layers and associated barriers of interoperability. To our best knowledge, almost all of the previous literature reviews explore this criterion on INAS. However, it is worth noting that this criterion is not always defined based on the same nomenclature. For example, in [START_REF] Ford | A survey on interoperability measurement[END_REF], the authors consider seven interoperability layers (or "types"): the technical, conceptual, coalition, programmatic, operational, constructive and non-technical interoperability. The authors in [START_REF] Panetto | Towards a classification framework for interoperability of enterprise applications[END_REF] and [START_REF] Yahia | Contribution à l'évaluation de l'interopérabilité sémantique entre systèmes d'information d'entreprise: Application aux systèmes d'information de pilotage de la production[END_REF] consider the three layers defined in EIF: technical, semantic and organisational interoperability. The reviews [START_REF] Cornu | Customizable Interoperability Assessment Methodology to support technical processes deployment in large companies[END_REF], [START_REF] Cestari | Interoperability Assessment Approaches for Enterprise and Public Administration[END_REF] and [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF] address the three interoperability barriers defined in FEI: technological, conceptual and organisational. The review presented in [START_REF] Rezaei | Interoperability evaluation models: A systematic review[END_REF] discuss four layers of interoperability (technical, syntactic, semantic and organisational).

For the purpose of this review, we adopt the barriers defined in the FEI, which are the Conceptual (including the semantic and syntactic barriers), the Technological (including the IT infrastructure and application barriers) and the Organisational barriers (subsuming the organisation structure and legal barriers). We argue that if an interoperability barrier is addressed, the related interoperability layer is also considered (explicitly or implicitly).

The coverage of the enterprise interoperability concerns is the fourth criterion considered in our comparative analysis. The considered concerns are the Business, Process, Service and Data concerns as defined in FEI. This criterion is relevant for studying the systems and their relations regarding different enterprise levels. It is also useful for identifying if the concerned INAS approaches are also considering the alignment of their addressed enterprise levels.

The fifth comparison criterion concerns the type of measurement mechanism used by the INAS approaches. Such criterion helps us to classify the approaches whether they are using Qualitative, Quantitative mechanism or both of them. It supports the understanding of how approaches are rating evaluation criteria and how to interpret the results.

The sixth criterion refers to the provision of best practices. Best practices are proven guidelines, recommendations or processes that have been successfully used by multiple enterprises [102]. These practices do not describe "which" solutions or "how" to implement solutions, but rather "what" should be done, in broad terms, to improve the system's interoperability [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF].

The seventh comparison criterion is the provision of a computer-mediated tool, whether the tool being automated or semi-automated. In general, Computer-Mediated Tools support different processes (including an assessment) by automatizing certain activities (e.g. rating calculation, data storage, etc.), consequently reducing time and improving the process performance. [START_REF] Krivograd | Development of an Intelligent Maturity Model-Tool for Business Process Management[END_REF], [START_REF] Alalwan | An Ontology-based Approach to Assessing Records Management Systems[END_REF]. Therefore, this criterion is relevant for classifying the INAS approaches as manual-conducted or computer-mediated approaches.

Table 8 presents the comparison criteria adopted in this paper and on the other INAS reviews and surveys.

Table 8. The comparison criteria applied by the different reviews and surveys in the literature

Review

Type of application

Type of assessment

Coverage of barriers

Coverage of concerns

Measurement mechanism

Provision of best practices

Provision of supporting tool (Ford et al. 2007)

[22] - X X - X - - (Panetto 2007) [25] - X X - - - - (Cornu et al. 2012) [23] X X X - - - - (Yahia 2011) [21] - X X - X - - (Cestari et al. 2013) [26] - - X - - - - (Rezaei et al. 2014) [24] - - - X - - - (Guédria et al. 2015) [20] - X X X - - - This review X X X X X X X
In the following sections, we present the comparison according to each defined criteria. Note that the term "assessor" used in this paper is based on both ISO 9000 [START_REF]Quality management systems --Fundamentals and vocabulary[END_REF] and ISO 33001[102] standards. It refers to the person with the demonstrated personal attributes and competence to conduct and participate within an assessment [102], [START_REF]Quality management systems --Fundamentals and vocabulary[END_REF].

Analysing the selected INAS approaches

Hereinafter, the 22 selected INAS approaches are described with a focus on the defined comparison criteria. An ID is also given for each one of these approaches for facilitating their identification during the comparative analysis. The results of the comparative analysis are presented in the following section.

Approach A1: The levels of conceptual interoperability model

The Levels of Conceptual Interoperability Model (LCIM) [START_REF] Tolk | The Levels of Conceptual Interoperability Model[END_REF], [START_REF] Wang | The Levels of Conceptual Interoperability Model: Applying Systems Engineering Principles to M&S[END_REF], [START_REF] Tolk | Supporting Network Enabled Capability by extending the Levels of Conceptual Interoperability Model to an interoperability maturity model[END_REF] is a maturity model assessing the semantic and syntactic divergences between systems. In other words, LCIM assesses the Compatibility of two Entities targeting the Conceptual barriers within the Data interoperability concern. LCIM provides descriptions of each of their seven defined maturity levels and the requirements that should be satisfied to achieving a given level. The assessment is mainly done based on the assessors' expertise and judgement using a Qualitative measurement mechanism. It can also be seen as a guidance model to prescribe and guide the interoperability design and implementation for the concerned systems [START_REF] Wang | The Levels of Conceptual Interoperability Model: Applying Systems Engineering Principles to M&S[END_REF]. This model proposes a set of prescriptive requirements that can be seen as Recommendations for achieving the desired maturity level. It also suggests engineering approaches for reaching the defined recommendations.

This maturity model can be applied to different situations (i.e. General Use). An Illustrative Example of the assessment of one system using the High Level Architecture (HLA) standard [106] and other system using the Base Object Models (BOM) standard [START_REF]2006: Standard for Base Object Model (BOM) Template Specification[END_REF] is given in [START_REF] Wang | The Levels of Conceptual Interoperability Model: Applying Systems Engineering Principles to M&S[END_REF].

Approach A2: Maturity levels for interoperability in digital government

The authors in [START_REF] Gottschalk | Maturity levels for interoperability in digital government[END_REF] define five maturity levels for assessing the interoperability in digital governments. This model addresses the Potentiality and Compatibility assessments of governmental Entities, covering all the three interoperability barriers (Conceptual, Technological and Organisational) and three interoperability concerns (Business, Process and Data).

Descriptions of each maturity level is given. Assessors are free to use their judgement for qualifying the interoperability and for determining the entities' maturity level. It also provides Recommendations for public administrations to improve their potential interoperability and discusses the relevance of two public entities to achieve together a higher level of maturity. This maturity model can be applied mainly to public administration entities (i.e. Specific Use). An Illustrative Example of the application of the model is presented based on the Norwegian Police and Customs departments.

Approach A3: A generic interoperability testing framework

The generic interoperability testing framework defined in [START_REF] Rings | A generic interoperability testing framework and a systematic development process for automated interoperability testing[END_REF] enables automated interoperability testing between at least two Non-Human Resources. It is mainly based on message checks, which assess the compliance of messages exchanged between the considered systems. In other words, it focuses on the Compatibility assessment of two systems regarding the Data concern. It evaluates the Technological (by verifying if the systems are connected and capable to exchange data) and the Conceptual interoperability (by verifying if the format of the message is compatible).

In order to assess the concerned systems, the framework defines a "Test Coordinator" architecture. This architecture provides the guidelines to connect the considered system and guidelines to design the functions for the message checks. This generic interoperability-testing framework can be applied to different systems that can be connected through a communication path (e.g. internet and local architecture network). This framework has been demonstrated in a Real Scenario focusing on Internet Protocol Multimedia Subsystems. The details of this scenario are given in [START_REF] Rings | A generic interoperability testing framework and a systematic development process for automated interoperability testing[END_REF].

Approach A4: The organisational interoperability maturity model

The organisational interoperability maturity model (OIMM) [START_REF] Clark | Organisational Interoperability Maturity Model for C2[END_REF], [START_REF] Fewell | Organisational Interoperability: Evaluation and Further Development of the OIM Model[END_REF], [START_REF] Fewell | Evaluation of Organisational Interoperabiity in a Network Centric Warfare Environment[END_REF], [START_REF] Kingston | An Organisational Interoperability Agility Model[END_REF] defines five maturity levels describing the ability of organisations to interoperate. OIMM aims at assessing the Compatibility of at least two Entities, regarding the Organisational and Conceptual barriers, with a focus on the Business concern. OIMM provides descriptions of each of their five maturity levels. Sets of questions are defined and associated with each of these levels for assessing them. Based on their expertise and judgement, assessors qualify the entities interoperability and determine their maturity level.

This maturity model was initially proposed to be used on the assessment of military organisations. However, OIMM's authors argued that such a model could be applied to different contexts (i.e. General Use). A Real Scenario based on the International Force East Timor military coalition focusing on the interaction between the United States Joint Forces Command and Australia is presented in [START_REF] Fewell | Organisational Interoperability: Evaluation and Further Development of the OIM Model[END_REF].

Approach A5: Writing and verifying interoperability requirements

In the publications [START_REF] Chapurlat | Interoperability Constraints and Requirements Formal Modelling and Checking Framework[END_REF], [START_REF] Mallek | An Approach for Interoperability Requirements Specification and Verification[END_REF], [START_REF] Mallek | The application of interoperability requirement specification and verification to collaborative processes in industry[END_REF], [START_REF] Mallek | Enabling model checking for collaborative process analysis: from BPMN to 'Network of Timed Automata[END_REF], [START_REF] Daclin | Writing and verifying interoperability requirements: Application to collaborative processes[END_REF], the authors propose and develop an approach for defining and verifying interoperability requirements. Such an approach focuses on the verification of requirement that two Entities should comply before interoperating. It also considers the verification of requirements related to the performance of the interaction between entities. In other words, it is an approach addressing the Compatibility and Performance types of assessment.

The forty-five interoperability requirements defined for the compatibility assessment are related to one of the interoperability barriers (Conceptual, Organizational, and Technological) and one interoperability concern (Data, Services, Processes, and Business). The twenty-six interoperability requirements defined for the performance assessment are related to three main factors: the time, quality and cost of interoperations. In order to verify the interoperability requirements (independent of the type of assessment) a computer-mediated tool is proposed. The requirement verifications is mainly based on model checkers. For verifying a-temporal requirements (i.e. requirements that are independent of time), they first transform the requirements into conceptual graphs. Next, they use the COGITANT (Conceptual Graphs Integrated Tools Allowing Nested Typed graphs) tool 1 for performing the requirement verification. For verifying the temporal requirements (i.e. verifiable only at certain stages of the collaboration), they first model the requirements using the Networks of Timed Automata (a behavioural modelling language). Next, they use UPPAAL model checker [START_REF] Behrmann | A Tutorial on Uppaal[END_REF] for performing the requirement s verification. Both model checkers are implemented in the computer-mediated tool developed by the authors. For identifying if requirements are achieved, qualitative rules are instantiated in the tool.

This approach can be used for different entities and contexts. An Illustrative Example regarding a vehicle design and production collaborative process is presented in the paper [START_REF] Mallek | The application of interoperability requirement specification and verification to collaborative processes in industry[END_REF], and another example focuses on the assessment of a drug circulation collaborative process is also described in [START_REF] Daclin | Writing and verifying interoperability requirements: Application to collaborative processes[END_REF].

Approach A6: The maturity model for enterprise interoperability

The Maturity Model For Enterprise Interoperability (MMEI) [START_REF] Guédria | A Maturity Model for Enterprise Interoperability[END_REF], [START_REF] Guédria | A Maturity Model Assessing Interoperability Potential[END_REF], [START_REF] Guédria | Enterprise Interoperability Maturity: A Model Using Fuzzy Metrics[END_REF], [START_REF] Guédria | Maturity Model as Decision Support for Enterprise Interoperability[END_REF], [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF] focuses mainly on the Potentiality assessment of an single Entity. As it is defined based on a systemic approach, the authors argue that it can also be used for the Compatibility assessment.

This model describes five levels of maturity. Each maturity level is an instantiation of the main elements of interoperability with an evolution of the elements regarding the development of the level. Based on the FEI dimensions, it defines twelve areas of interoperability. Those areas represent the crossing between the interoperability barriers and concerns. Each one of the interoperability areas contain the evaluation criteria that should be verified when assessing the maturity level of an enterprise. These areas are named after their associated barrier and concern, e.g. Business-Conceptual and Service-Technological. Table 9 shows the criteria from each area regarding the maturity level 3. The MMEI proposes one criterion for each interoperability area for each maturity level, totalising forty-eight interoperability criteria. For rating these criteria, the model adopts a Qualitative measurement mechanism. It means that, the assessor can rate each criterion using four linguistic variables: Not Achieved (NA), Partially Achieved (PA), Largely Achieved (LA) and Fully Achieved (FA). When there is more than one assessor, the final rating of a criterion is calculated by aggregating the ratings provided by all involved assessors. A Quantitative measurement mechanism, based on the fuzzy sets theory and the Ordered Weighted Average (OWA) aggregation operator [START_REF] Yager | On ordered weighted averaging aggregation operators in multicriteria decisionmaking[END_REF] is provided for translating the linguistic values into numeric values in order to compute, aggregate and calculate the final ratings and maturity levels criteria.

Moreover, MMEI proposes 126 Best Practices. Each practice is associated with an interoperability barrier, concerns and maturity level. These best practices describe "what" should be done to improve a current situation in terms of interoperability. This maturity model can be applied to different situations (i.e. General Use). A Real Scenario based on a company specialised in automobile manufactures with modern wiring harness systems, exclusive interiors and electrical components is detailed in [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF].

Approach A7: Formal measures for semantic interoperability assessment in cooperative enterprise information systems

The formal measures for semantic interoperability proposed by [START_REF] Yahia | Semantics enactment for interoperability assessment in enterprise information systems[END_REF], [START_REF] Yahia | Formal measures for semantic interoperability assessment in cooperative enterprise information systems[END_REF] focuses on the assessment between two cooperative information systems (i.e. Compatibility assessment). This approach provides a Quantitative measurement mechanism for evaluating the Conceptual interoperability of two Non-Human Resources, regarding the Data concern.

For calculating the interoperability between two information systems, this approach defines three main activities. First, one has to identify every concept (mandatory or not) from the two systems' conceptual models. Second, one has to identify the mandatory and non-mandatory semantic relationships with the help a domain expert. These mandatory relationships are those that if not satisfied, interoperability is not fully achieved. The third activity is to calculate the Maximum Potential Interoperability (MPI) and the Minimal Effective Interoperability (MEI).

MPI is reached when all the concepts of one system (even the non-mandatory ones) are instantiated in the other. MEI is reached when only the mandatory concepts of one system are instantiated in the second system. Table 10 describes the formal measures and the meaning of their results. It is worth noting that this approach considers interoperability as non-bidirectional i.e. given two systems A and B and measuring their interoperability level I(x,y) it is structurally coherent to find 𝐼(𝐴,𝐵)≠ 𝐼(𝐵,𝐴). This approach can be applied to different situations (i.e. General Use). It is illustrated in [START_REF] Yahia | Semantics enactment for interoperability assessment in enterprise information systems[END_REF] through a Illustrative Example dealing with a business to manufacturing scenario between an Enterprise Resource Planning (ERP) system and a Manufacturing Execution System (MES) application.

Approach A8: A Framework for Identification and Resolution of Interoperability Mismatches in COTS-Based Systems

The authors in [START_REF] Bhuta | A Framework for Identification and Resolution of Interoperability Mismatches in COTS-Based Systems[END_REF] propose an attribute-based framework for performing an automated assessment of the interoperability between at least two Commercial-Of-The-Shelf (COTS) products. In other words, it deals with the Compatibility assessment Non-Human Resources. The assessment covers the Conceptual and Technological barriers of interoperability and the Service and Data concerns.

This approach develops and provides a Computer-Mediated Tool based on the defined COTS interoperability framework. Such tool is composed of a COTS definition repository (storing generic COTS architectures), an interoperability rules repository (every rule has a set of pre-conditions, which if true for the given architecture and components, identifies an architectural mismatch.) and the interoperability analysis component. For obtaining the analysis results, the assessor enters the considered COTS's information. The tool then uses the COTS definitions and the interoperability rules for identifying potential incompatibilities that the considered COTS may face.

This approach is demonstrated in Real Scenario based on multiple software systems requested by a real-world client. The authors of the approach argue that it is not limited to a single type of COTS and that it can be used for assessing different systems.

Approach A9: A methodology to implement and improve interoperability

The methodology to implement and improve interoperability [START_REF] Daclin | Enterprise interoperability measurement -Basic concepts[END_REF], [START_REF] Chen | Barriers Driven Methodology for Enterprise Interoperability[END_REF], [START_REF] Daclin | Methodology for Enterprise Interoperability[END_REF], [START_REF] Daclin | Developing enterprise collaboration: a methodology to implement and improve interoperability[END_REF] focuses on the interoperability development of enterprises (i.e. it addresses Entities). This methodology is the only one dealing with the three types of assessment: Potentiality, Compatibility and Performance.

Regarding the potentiality assessment, a maturity model containing five levels is defined. The model defines the evaluation of an enterprise potentiality according to the three interoperability barriers defined by FEI that impact the development of interoperability and the levels where interoperability takes place, which is Business, Process, Service and Data. This assessment is based on Qualitative measurement mechanism for determining the enterprise maturity level.

Considering the Compatibility assessment, it proposes a matrix of incompatibilities. Such a matrix has four rows corresponding the interoperability concerns (Business, Process, Service and Data) and six columns based on the three interoperability barriers (Conceptual, Technological and Organisational). These columns are Syntactic, Semantic, Platform application, Communication, Authorities' responsibilities and Organisation. If at least one incompatibility is detected, the coefficient 1 is assigned to the interoperating level and the problem that is considered. Conversely, the coefficient 0 will be applied either when no incompatibility is detected or when the view is not concerned. The set of questions to detect potential incompatibilities is defined according to the needs expressed by partners. The assessors therefore evaluate Qualitatively the defined questions based on their experience and judgment. The total degree of interoperability is given by the sum of the matrix's cells. A compatibility degree equal to twenty-four is the worst situation, as it means that there is at least one incompatibility in each cell. Moreover, Quantitative criteria related to the cost, duration and quality of interoperation is defined by conducting a Performance assessment. The performance criteria are described in Table 11.

Table 11. Interoperability performance criteria. Adapted from [72]

Type of evaluation Details Formula

Cost of data exchange (Cex)

It represents the difference between the initial cost allocated to exchange (Ciniex) and the real cost of exchange (Ceffex) According to the authors, this methodology can be applied to any kind of entities. This methodology have been applied in two Real Scenarios. The scenario regarding a telecommunication company and its dealers is detailed in [START_REF] Daclin | Developing enterprise collaboration: a methodology to implement and improve interoperability[END_REF]. The second scenario detailed in [START_REF] Daclin | Methodology for Enterprise Interoperability[END_REF] corresponds to a carrier and shipper company.

Approach A10: The Interoperability Score

The Interoperability Score (i-Score) [START_REF] Chalyvidis | A method for measuring supply chain interoperability[END_REF], [START_REF] Ford | The Interoperability Score[END_REF], [START_REF] Ford | Measuring system interoperability[END_REF], [START_REF] Ford | A General Method of Measuring Interoperability and Describing Its Impact on Operational Effectiveness[END_REF], [START_REF] Chalyvidis | Using Supply Chain Interoperability as a Measure of Supply Chain Performance, Supply Chain Forum[END_REF] focuses on measuring the interoperability of complex non-homogeneous system networks. It deals with the Compatibility assessment of collaborative Processes established between at least two Entities. The assessment approach considers the Conceptual, Technological and Organisational barriers of interoperability.

This assessment approach proposes a system resemblance matrix for calculating the systems' interoperability. The coefficients in the resemblance matrix represent measures of similarity between systems, based upon system attributes pertinent to interoperability. The cardinal rule to follow is that only functional system interoperability attributes describing what systems do to each other should be used to instantiate systems within the matrix. Their particular mathematic development requires extensive notation and are detailed in [START_REF] Ford | Measuring system interoperability[END_REF] and [START_REF] Chalyvidis | A method for measuring supply chain interoperability[END_REF]. The calculated interoperability between two systems is equal to a positive real number ranging from 0 to 1, where a score of zero indicates no interoperability and a score of one indicates perfect interoperability.

This approach can be applied to different process from different entities (i.e. General Use). Illustrative Examples based on fictional Suppression of Enemy Air Defences (SEAD) systems are presented in [START_REF] Ford | The Interoperability Score[END_REF], [START_REF] Ford | Measuring system interoperability[END_REF], [START_REF] Ford | A General Method of Measuring Interoperability and Describing Its Impact on Operational Effectiveness[END_REF].

Approach A11: Reconceptualising measuring, benchmarking for improving interoperability in smart ecosystems

The authors in [START_REF] Maheshwari | Reconceptualizing measuring, benchmarking for improving interoperability in smart ecosystems: The effect of ubiquitous data and crowdsourcing[END_REF] define a process for measuring and benchmarking for improving interoperability in the smart governments. It focuses on the Potentiality assessment of a single Entity, considering all three interoperability barriers and the four interoperability concerns.

More precisely, this approach defines ten aspects to be considered during the assessment: Semantic, Syntactical, Data linking, Physical, Policy, Enterprise architecture, Business process, Judicial, Governance, and Economical. From these ten aspects, twenty-three evaluation criteria are derived and described.

For measuring the potential interoperability, the approach provides Qualitative measurement mechanisms. For instance, considering the entity to be assessed and the assessment objectives, a questionnaire based on the evaluation criteria should be defined. However, this approach does not provide a standard questionnaire. Therefore, assessors should build their own questionnaires based on their experience and the concerned context. Once the questionnaires are defined, assessors ask the selected employee to categorically specify a numeric value for each of the related questions from 0 (lowest) to 9 (highest). In the end, the interoperability degree is equal to the set of the mean of each criterion.

This approach can be used for different entities and contexts. A Real Scenario regarding the Population Welfare Department (PWD) Government of Sindh in Pakistan is presented in [START_REF] Maheshwari | Reconceptualizing measuring, benchmarking for improving interoperability in smart ecosystems: The effect of ubiquitous data and crowdsourcing[END_REF].

Approach A12: The ultra-large-scale systems interoperability maturity model

The Ultra Large Scale Systems Interoperability Maturity Model (ULSSIMM) [START_REF] Rezaei | An interoperability model for ultra large scale systems[END_REF] defines five maturity levels for assessing the potential interoperability of ultra large scale systems (e.g. health information systems and hospitals itself). This maturity model covers all interoperability barriers (Conceptual, Technological and Organisational) and the four interoperability concerns (Data, Service, Process and Business). Forty-one criteria are defined and related to the maturity levels.

The ULSSIMM proposes a Quantitative measurement mechanism using colours. For instance, a score between 0 and 1 is given for each evaluated criteria. One colour (grey, red, yellow and green) is allocated to each level of interoperability according to the mean of its related criteria (grey [=0], red [<0.4], yellow [>=0.4 and <0.7] and green [>=0.7]). A maturity level is achieved when the allocated colour is green.

This maturity model also provides a solution framework containing Best Practices for improving interoperability. For each maturity level and for each interoperability barrier a set of potential solutions and technologies are suggested for removing the concerned barrier. The ULSSIMM can be applied to different situations (i.e. General Use). A Real Scenario regarding assessment of the Malaysian Healthcare system is detailed in [START_REF] Rezaei | An interoperability model for ultra large scale systems[END_REF].

Approach A13: Assessing interoperability of access equipment for broadband networks

The approach defined in [START_REF] Vito | Assessing interoperability of access equipment for broadband networks[END_REF], proposes a Remote Testing Board (RTB), designed and realized to carry out off-line interoperability tests in smart office devices (e.g. telephones) within a broadband network (e.g. office telephone network). This approach deals with the Compatibility assessment of two Non-Human Resources focusing only on the Technical barriers and the Data concern.

Indeed, the interoperability assessment is done by connecting the concerned smart office devices in the RTB. With the help of a Computer-Mediated Tool, the assessor(s) verifies if the devices can identify each other and if data exchange is possible. For example, the RTB simulates combinations of phone calls directed to and from the telephone line through the office telephone network. A traffic generator is also implemented for customising and testing different phone lines and device communications. The conclusions are given according to the observations, experience of the tester and the defined objectives.

This approach is defined for the specific assessment of broadband networks and their connected devices. A complete application description is given in [START_REF] Vito | Assessing interoperability of access equipment for broadband networks[END_REF].

Approach A14: A framework for interoperability assessment in crisis management

The authors in [START_REF] Da Silva Avanzi | A framework for interoperability assessment in crisis management[END_REF] propose the Disaster Response Management System (DRMS) development cycle framework, which is centred in the Disaster Interoperability Assessment Model (DIAM). DIAM focuses on the Potentiality assessment of a public/private Entities or localities.

The concerned assessment model defines three maturity levels that an entity can achieve: Basic, Intermediary and Advanced. For determining the maturity level, a set of functional and non-functional requirements for crisis management defined in DIAM should be verified. These requirements are related to all interoperability barriers (Conceptual, Technological and Organisational) and concerns (Business, Process, Service and Data).

DIAM provides an Analytic Hierarchy Process (AHP) [START_REF] Saaty | Decision makingthe Analytic Hierarchy and Network Processes (AHP/ANP)[END_REF] to calculate the maturity level of a given entity. An AHP is a multi-criteria decision analysis technique (including Qualitative and Quantitative measurement mechanisms). The AHP architecture is designed as follows: The first layer corresponds to the goal of the interoperability assessment. The second and third layers represent the interoperability concerns and barriers, respectively. Layers two and three are related to the fourth layer, which represents the functional requirements. The fifth and final layer represents the potential interoperability levels. This approach uses the open source Software called Super Decisions2 for implementing their AHP matrix.

Interviews based on the defined requirements should be conducted for gathering relevant information of the assessed entity. From the collected data, pairwise comparisons are conducted in each layer of the AHP matrix using the Super Decisions software. These comparisons intend to identify what are the most relevant concerns and barriers to be addressed. It also identifies how well a requirement is being fulfilled in comparison with the others. In the end, the Super Decisions software automatically generates graphs showing the calculated maturity levels.

In [START_REF] Da Silva Avanzi | A framework for interoperability assessment in crisis management[END_REF], a Real Scenario based on the company responsible for the municipal technology sector of Curitiba (Brazil) is described. Application to other entities e.g. civil defence, firefighters, traffic engineering are also planned.

Approach A15: Levels of Information System Interoperability

The Levels of Information System Interoperability (LISI) [START_REF]C4ISR, Levels of Information System Interoperability (LISI)[END_REF] defines an interoperability LISI maturity model, which considers five increasing levels of sophistication regarding system interaction and the ability of the system to exchange and share information and services. This model can be used for comparing a single system to the LISI reference system model as well as for comparing the desired state of a pair of systems against the LISI reference system model.

In other words, LISI deals with the Potentiality and Compatibility assessment of Entities, focusing on the exchanging and sharing of Data and Services between systems. The proposed model deals with the Technological barriers, but Conceptual issues such as semantics are also considered. For determining the "degree of interoperability" attained by or between systems, a Quantitative measurement mechanism is proposed. It is derived using the Interoperability Questionnaire as the data source and the LISI maturity model as the measurement template. Consequently, the LISI Interoperability Questionnaire forms the bridge between the LISI maturity model and the LISI assessment process.

The LISI identifies for each level of interoperability, a common suite of capabilities across procedures, applications, infrastructure, and data that must be incorporated (i.e. Best Practices) by system developers in order to have a "common-ground" basis for interoperability assurance. A Computer-Mediated Tool is proposed for implementing the interoperability questionnaires and to automatically generate the assessment results (i.e. maturity level determinations and recommendations).

The LISI can be applied to different situations (i.e. General Use). Five Illustrative Examples of LISI application are detailed in [START_REF]C4ISR, Levels of Information System Interoperability (LISI)[END_REF].

Approach A16: Maturity model for the structural elements of coordination mechanisms in the collaborative planning process

The Structural Elements Of Coordination Mechanisms Maturity Model SECM-MM [START_REF] Cuenca | Structural elements of coordination mechanisms in collaborative planning processes and their assessment through maturity models: Application to a ceramic tile company[END_REF] focuses on the maturity of the coordination mechanisms in the collaborative planning process within a business network. It defines five levels of maturity for assessing nine structural elements (e.g. number of coordination mechanisms, information exchanged, information processing) of a given process. Indeed, SECM-MM deals with the Compatibility assessment of two Entities, in terms of the Business and Process interoperability concerns and the related Organisational barriers.

Based on interviews, each structural element is individually assessed by an assessor to determine the level of maturity. When many assessors are involved, the team's assessments are discussed and then a final level is given to each element.

The SECM-MM also includes the Best Practices to be carried out on collaborative planning that must be implemented to reach the highest maturity level in the defined structural elements. This maturity model can be applied to different entities (i.e. General Use). [START_REF] Cuenca | Structural elements of coordination mechanisms in collaborative planning processes and their assessment through maturity models: Application to a ceramic tile company[END_REF] presents the application of SECM-MM in a Real Scenario based on a ceramic tile company.

Approach A17: Evaluation of Interoperability between Automation Systems using Multi-criteria Methods

The authors in [START_REF] Saturno | Evaluation of Interoperability between Automation Systems using Multi-criteria Methods[END_REF] propose a maturity model to evaluate the potential interoperability among systems within an existing automation platform in the Industry 4.0 context. It addresses the Compatibility assessment of Non-Human Resources, focusing on the Technological and Conceptual barriers that can influence the Service and Data interoperability concerns.

This model defines three levels considering automation and information technological requirements, in terms of interoperability. The definition of the requirements has its basis in concepts of Industry 4.0 with orientation to the concept of interoperability between systems. These requirements are instantiated in a AHP matrix [START_REF] Saaty | Decision makingthe Analytic Hierarchy and Network Processes (AHP/ANP)[END_REF] using the Super Decisions software. The AHP architecture is designed as follows:

The first layer in this architecture presents the objective of evaluation. The second and the third layers represent the requirements and interoperability barriers related to the subject of this evaluation (i.e. the assessed systems). The fourth layer represents the interoperability maturity levels. The maturity level is determined based on the requirements pairwise comparisons.

This model had been developed for assessing interoperability of automated systems in the Industry 4.0 context. An Illustrative Example of the application of this maturity model is detailed in [START_REF] Saturno | Evaluation of Interoperability between Automation Systems using Multi-criteria Methods[END_REF].

Approach A18: Performance evaluation of collaboration in the design process: Using interoperability measurement

The authors in [START_REF] Neghab | Performance evaluation of collaboration in the design process: Using interoperability measurement[END_REF] propose a Computer-Mediated methodology for assessing the Conceptual interoperability between systems that have to collaborate in business/design Processes. It deals with the Compatibility type of assessment.

This methodology is divided in two main phases: The first one refers to the process modelling, including all Entities and activities assigned to the concerned process. The second phase is the interoperability assessment. A Quantitative measurement mechanism subsuming two measures is defined for evaluating the semantic and syntax of the data to be exchanged. A Qualitative mechanism is also in place for defining the measures threshold (i.e. what is considered as semantically interoperable). The mathematic development and notation are described in [START_REF] Neghab | Performance evaluation of collaboration in the design process: Using interoperability measurement[END_REF].

Regarding the computer-mediated tool, it is developed based on the Eclipse Modelling Framework (EMF). The authors use three components of EMF: the ECORE for metamodeling the processes to be analysed, the Object Constraint Language for performing the syntax check and the EMF compare for the semantic check.

This methodology can be used to assess different collaborative processes (i.e. General Use). The application of this methodology in a Real Scenario regarding a design process of a mechanical coupling between a propeller and a diesel engine is presented in [START_REF] Neghab | Performance evaluation of collaboration in the design process: Using interoperability measurement[END_REF].

Approach A19: Methodology for Interoperability Evaluation and Improvement

[19], [START_REF] Camara | A Methodology for Interoperability Evaluation in Supply Chains based on Causal Performance Measurement Models[END_REF], [START_REF] Camara | A methodology for the evaluation of interoperability improvements in interenterprises collaboration based on causal performance measurement models[END_REF] propose an approach for the evaluation of interoperability improvements in a networked enterprise based on collaborating Entities Performance assessment. This approach includes an interoperability evaluation framework and an evaluation methodology.

The interoperability framework is composed of three layers. The interoperability investment layer aims to analyse the relationships between elements located in the physical system of networked enterprise. These elements include the interoperability concerns (Process, Service and Data), the Technological interoperability barriers and related solutions. The operational interoperability impact layer subsumes the key performance indicators (KPI) related to the concerned collaborative process (i.e. indicators related to the cost, time and failure reduction in processes). Finally, the tactical and strategic impact layer uses the KPIs related to the enterprises' strategies to evaluate the impact of interoperability on high-level business objectives. The KPIs from the different layers are defined based on the specific context of the networked enterprise and based on the strategic decisions of the concerned stakeholders.

The evaluation methodology describes the main steps to support the INAS as well as explains how to use the defined interoperability evaluation framework. Three blocks of steps are determined: the Configuration Management aiming at modelling the as-is and to-be states of the concerned enterprises and their interactions; the Interface Management aiming to identify interoperability barriers in the as-is state and to propose solutions to remove these barriers; and the Decision Analysis aiming to provide the basis for evaluating and selecting alternatives when decisions need to be made.

According to the authors of this assessment approach, it can be applied to any type of collaborative processes from networked enterprises from different sectors of activity. An Illustrative Example of the application of such approach in a goods entry process between three entities is presented in [START_REF] Camara | A methodology for the evaluation of interoperability improvements in interenterprises collaboration based on causal performance measurement models[END_REF].

Approach A20: Maturity Model for Interoperability Potential Measurement

The maturity model proposed in [START_REF] Campos | Maturity Model for Interoperability Potential Measurement[END_REF] is composed of a methodology and a reference set of evaluation criteria to measure interoperability Potential. It focuses on evaluating the capability of an Entity to interoperate with an unknown partner. The three interoperability barriers and the four interoperability concerns are considered in six enterprise views: Business, Process Management, Knowledge, Human Resources, Information and Communication Technology, and Semantic views.

For each view, a description is provided and a set of evaluation criteria is defined. According to the fulfilment of evaluation criteria, one of the five proposed maturity levels (Isolated, Initial, Executable, Connectable and Interoperable) can be assigned to the concerned view. In order to perform the interoperability assessment, the proposed methodology provides five phases. The Project Planning aims at defining the conceptual aspects of the enterprise in regards to interoperability, taking into account the strategic and cultural goals of the assessed enterprise. The second phase is the definition and classification of collaborations. It aims at studying the organizational structure of the company and the identification of the collaborations that exist between each department for each of the enterprise's processes.

The third phase consists of the measurement and collection of results. The objective of this phase is twofold: first, the design of questionnaires for conducting the assessment is developed. The questions are defined by the assessor(s) according to the defined evaluation criteria for each view and based on the enterprise's current situation studied in the first phase. The second objective of this phase is to assess each identified collaboration identified in the second phase using the defined questionnaires. To complete the questionnaires, Qualitative measurement mechanism (e.g. interviews and group discussions) should be used.

In the fourth phase, a Quantitative measurement mechanism is used for quantifying and aggregating the information gathered through the questionnaires. It allows the analysis and determination of the enterprise's interoperability potential. However, this measurement mechanism is not completely described in [START_REF] Campos | Maturity Model for Interoperability Potential Measurement[END_REF]. Finally, the last phase refers to the proposal of improvements based on the assessment results. However, this approach does not specify the improvement proposals to be adopted.

This maturity model can be applied to different situations (i.e. General Use). A Real Scenario based on a large textile enterprise from Spain is used to demonstrate the maturity model application.

The authors in [START_REF] Cornu | Customizable Interoperability Assessment Methodology to support technical processes deployment in large companies[END_REF], [START_REF] Cornu | Interoperability Assessment in the Deployment of Technical Processes in Industry[END_REF] propose a methodology for INAS regarding the deployment of collaborative processes. This methodology allows the concerned enterprises to select between the Potentiality and Compatibility assessment for evaluating the interoperability between two Entities.

A set of fourteen questionnaires are based on eighty-eight interoperability requirements defined in [START_REF] Cornu | Contribution à la prise en compte de l'interopérabilité pour le déploiement de processus complexes dans une grande entreprise: proposition d'un guide méthodologique outillé pour les processus d'Ingénierie Système[END_REF]. Each question is related to at least one of the interoperability barriers (Conceptual, Technological and Organisational). Besides the Process being the main interoperability concern addressed, the Business, Service and Data concerns are also covered by the proposed questionnaires. These defined questions are yes or no questions, where the "yes" value means that the assessed system fulfils the related requirement(s). At least one Recommendation for improving interoperability is associated to each question.

There are two measurement mechanisms proposed in this methodology. The first one is a Qualitative one, referring to the answer of each question. Indeed, the assessors answer a question based on their experience and best judgment regarding the current situation of the assessed system. Second, a Quantitative measurement mechanism is put in place for "translating" the yes/no answers on numeric values or calculating the result of the assessment. Their mathematic development requires extensive notation and is detailed in [START_REF] Cornu | Customizable Interoperability Assessment Methodology to support technical processes deployment in large companies[END_REF].

The proposed questionnaires were implemented in a Computer-Mediated Tool for ease of use. The tool was designed and developed by the authors. To use the tool, the assessor enters information about the system to be assessed and the answers to the defined questions. The tool computes automatically the assessment results, including the provision of recommendations to the questions that had negative answers. An Illustrative Example of a large company in the field of aeronautics is presented in [START_REF] Cornu | Customizable Interoperability Assessment Methodology to support technical processes deployment in large companies[END_REF].

Approach A22: A holistic interoperability assessment based on requirements interdependencies

The INAS approach proposed in [START_REF] Leal | Using Formal Measures to Improve Maturity Model Assessment for Conceptual Interoperability, On the Move to Meaningful Internet Systems[END_REF], [START_REF] Leal | An approach for interoperability assessment in networked enterprises[END_REF], [START_REF] Leal | Towards a Semi-automated Tool for Interoperability Assessment: An Ontology-Based Approach[END_REF], [START_REF] Leal | Assessing Interoperability Requirements in Networked Enterprises: A Model-Based System Engineering Approach[END_REF] addresses the Potentiality and Compatibility assessment of any kind of Entities (i.e. General Use). It has the objective on detecting and preventing interoperability problems before they occur. It is defined based on the MMEI [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF].

This approach addresses al the interoperability barriers (Conceptual, Technological and Organisational) and concerns (Business, Process, Service and Data). A set of 48 interoperability requirements are defined as well as their interdependencies. Each requirement is related to one interoperability barrier and one concern. Best practices for each one of the defined requirements are also determined. Qualitative measurements mechanisms are used for attributing a linguistic value to each concerned requirement. Quantitative mechanisms are used for computing and aggregating multiple requirements ratings.

This INAS approach proposes a Computer-Mediated Tool for semi-automating the INAS process. Such a tool uses an ontology as the knowledge model with information about interoperability. Based on the information gathered by the assessment team, the proposed tool automatically infer the potential best practices for reducing or removing interoperability barriers caused by the identified non-compliance of interoperability requirements. An assessment report is also generated containing the requirements rating, the potential problems and the recommended practices. This tool is developed by the authors using Java. The embedded ontology is designed using Ontology Web Language [START_REF] Horridge | A Practical Guide To Building OWL Ontologies Using The Protege-OWL Plugin and CO-ODE Tools[END_REF] and implemented using Protégé 5.2 [START_REF] Musen | The protégé project[END_REF]. A Real Scenario based on an active network of enterprises from the marketing and communication domain is presented in [START_REF] Leal | Towards a Semi-automated Tool for Interoperability Assessment: An Ontology-Based Approach[END_REF].

Discussion

The literature review reveals 72 candidate papers, of which 50 are retained. The selected papers propose or improve 22 INAS approaches that are analysed and compared based on seven criteria: the type of application, the type of assessment, the coverage of interoperability barriers, and the coverage of enterprise interoperability concerns, the measurement mechanism, and the provision of best practices and the provision of a computer-mediated tool for supporting the assessment process.

In this section, we provide first a summary of the analysed approaches considering the comparison criteria. Further, we elaborate on the identified limitations and research perspectives.

Among the twenty-two approaches, we identify eighteen addressing the Technological barriers, nineteen dealing with the Conceptual barriers, thirteen approaches assessing the Organisational barriers, and eleven of the studied approaches are addressing the three interoperability barriers. Regarding the interoperability concerns, we identify twelve INAS approaches dealing with the Business concern, fourteen with the Process concern, fourteen with the Service concern, and nineteen addressing the Data concern. As shown in Table 13, the Technological-Data cross-section is the most addressed with seventeen approaches. It is closely followed by the Conceptual-Data cross-section with fifteen approaches. The Technological-Business cross-section is the less addressed, with six approaches.

However, it is important to note that only five of the studied approaches are addressing all interoperability barriers and concerns. These INAS approaches are: the approach for interoperability requirements specification and verification [START_REF] Mallek | The application of interoperability requirement specification and verification to collaborative processes in industry[END_REF], the MMEI [START_REF] Guédria | Maturity model for enterprise interoperability[END_REF], the methodology to implement and improve interoperability [START_REF] Daclin | Developing enterprise collaboration: a methodology to implement and improve interoperability[END_REF], the holistic interoperability assessment based on requirements interdependencies [START_REF] Leal | Towards a Semi-automated Tool for Interoperability Assessment: An Ontology-Based Approach[END_REF] and the framework for interoperability assessment in crisis management [START_REF] Da Silva Avanzi | A framework for interoperability assessment in crisis management[END_REF].

Limitations and research perspectives

Based on the comparative analysis, we discuss the identified limitations as well as research perspectives.

The small number of INAS approaches addressing the performance assessment

We identified that just five approaches are focusing on the interoperability performance assessment. Hence, researcher and practitioners could direct more efforts on developing new INAS approaches containing a set of requirements and interoperability performance indicators. Indeed, an overview of the time, cost and quality of interoperation in real time can be an asset for the stakeholders to take decisions "on the fly". Besides, this type of assessment can be used for validating the implemented interoperability solutions proposed during the potential and compatibility assessments.

The literature regarding Performance Measurement Systems (PMS) [START_REF] Camara | A methodology for the evaluation of interoperability improvements in interenterprises collaboration based on causal performance measurement models[END_REF], [START_REF] Neely | Performance measurement system design: developing and testing a process-based approach[END_REF] and collaboration performance frameworks and metrics [START_REF] Camarinha-Matos | Performance indicators for collaborative networks based on collaboration benefits[END_REF], [START_REF] Westphal | Measuring Collaboration Performance in Virtual Organizations[END_REF], [START_REF] Ramanathan | Supply chain collaboration performance metrics: a conceptual framework[END_REF] can be assets for designing interoperability performance frameworks and for identifying relevant Key Performance Indicators (KPI) for interoperability. A more detailed discussion regarding the association of PMS and the INAS can be found in [START_REF] Da Costa Castro | Performance Measurement Systems for Designing and Managing Interoperability Performance Measures: A Literature Analysis[END_REF].

For example, generic KPI for the conceptual layer could be (1) the percentage of information lost during the exchange between two information systems; (2) the time for exchanging information between two systems; (3) the time for translating the requested information (in the case of the considered system are using different data semantics). Referring to the technological layer, the following KPI could be considered: (1) the percentage of failed connections between two systems; (2) the time for translating the requested information (in the case of the considered system are using different data syntax). Finally, considering the organisational layer: (1) the percentage of the times when the absence of an employee caused any delay in the interoperations, (2) the percentage of the times when interoperations failed because resources (human and non-human) were not allocated.

However, it is important to note that KPI should reflect the reality of the considered context and sectors of activity (e.g. healthcare, manufacturing and financial sectors).

The small number of INAS approaches addressing the multiple interoperability barriers and concerns

We identified that few INAS approaches address multiple interoperability barriers and concerns at the same time. We argue that the application of multiple approaches may cause redundancy and confusion when assessing the same barriers using different metrics and viewpoints. Consequently, few approaches -explicitly or implicitly -address the interdependencies among and between interoperability barriers and concerns. Acknowledging the different dependencies among and between them supports the identification of impacts on the overall system. For example, when implementing a new software application, the enterprise should: verify if the current data format available on their servers are compatible with the new application; verify if employees have the competence and authorisation to use the application; verify if the existing internal and external applications are compatible and connectable with the new one.

In order to establish explicit links among interoperability layers and concerns, the literature from both Enterprise Architecture and Systems Alignment domains are assets. The latter, focus on aligning enterprise systems for achieving cohesive goals across the ICT and other functional organisations (e.g. marketing and human-resources units) [START_REF] Luftman | Assessing It/Business Alignment[END_REF]. For example, Solaimani and Bouwman [START_REF] Solaimani | A framework for the alignment of business model and business processes[END_REF] conceptualise the gap between strategic systems (focusing on the business model of organisations) and operational systems (focusing on business processes) from trans-sector companies. Goepp and Avila [START_REF] Goepp | An Extended-Strategic Alignment Model for technical information system alignment[END_REF] focus on the tactical and operational levels on an enterprise by integrating formally such concerns into the design and development of technical information systems. Castellanos and Correal [START_REF] Castellanos | A Framework for Alignment of Data and Processes Architectures Applied in a Government Institution[END_REF] focus on evaluating the lack of alignment between business process and data models.

Further, the literature of EA is also relevant for gathering insights as an EA can provide a coherent and comprehensive view of the relationships of enterprise systems (machines, human resources, organisation units, etc.) according to the defined business strategy [START_REF] Zachman | A framework for information systems architecture[END_REF], [START_REF] Op't Land | Enterprise Architecture: Creating Value by Informed Governance[END_REF], [START_REF] Tog | The TOGAF® Standard, Version 9.2[END_REF]. Thus, it supports the visualisation and understanding of requirements and constraints from different layers of the enterprise.

The small number of INAS approaches providing guidance for interoperability improvement

We remarked that only seven INAS approaches are conveying any information or guidance to improve interoperability based on their assessment results. Indeed, the provision of best practices can support stakeholders making informed decisions for solving or at least reducing interoperability problems.

To tackle this limitation, researchers could conduct exploratory research for gathering knowledge about interoperability solutions and how to apply it. For example, systematic literature reviews can be done focusing on this topic. International standards addressing aspects of an enterprise/system relevant to interoperability (e.g. standards focusing on data transmission, enterprise architectures and so on) could also be considered. For example, de facto standards such as COBIT 5, CMMI and ITIL can be useful for identifying practices for improving process interoperability in different sectors of activity.

Further, empirical research (e.g. case studies) focusing on enterprises and networked enterprises that successfully implemented interoperable systems could also be performed. The insights as well as the difficulties faced by these enterprises on implementing such systems, can serve for drawing the first draft of best practices. Techniques such as the Delphi method [START_REF] Dalkey | An Experimental Application of the Delphi Method to the Use of Experts[END_REF], group discussion and feedback sections could be done for achieving and validating a consensus. Such discussion can also be useful for validating hypothesis made duringif is the caseexploratory research.

The small number of INAS approaches providing computer-mediated tools

We observed that the majority of INAS approaches is manual-conducted, which is a laborious and time-consuming process and in many times depends on the "subjective" knowledge of experts which can be expensive in time and money when hiring external consultants [START_REF] Alalwan | An Ontology-based Approach to Assessing Records Management Systems[END_REF], [START_REF] Krivograd | Development of a Generic Tool for the Application of Maturity Models--Results from a Design Science Approach[END_REF]. Few of the studied approaches are proposing computer-mediated tools for supporting the assessment and decision-making processes. Indeed, computer-mediated systems for supporting assessment processes enhance a stakeholder's ability to analyse the system's current state and to make improvements [START_REF] Krivograd | Development of a Generic Tool for the Application of Maturity Models--Results from a Design Science Approach[END_REF].

The literature regarding automatic assessment from other domains could serve as the basis for designing and implementing INAS systems architectures. For example, in the Process Assessment domain, Shrestha et al. [START_REF] Shrestha | Innovative decision support for IT service management[END_REF] propose a software-mediated process assessment approach focusing on the evaluation of IT Service Management, Barafort et al. [START_REF] Barafort | A software artefact to support standard-based process assessment: Evolution of the TIPA ® framework in a design science research project[END_REF] propose a software artefact to support standard-based process assessment and Krivograd and Fettke [START_REF] Krivograd | Development of a Generic Tool for the Application of Maturity Models--Results from a Design Science Approach[END_REF] propose an intelligent maturity model tool for Business Process Management.

Further, Knowledge-Based System (KBS) architectures could also be used to building INAS systems. Indeed, a KBS is a software application with specialised problem-solving expertise, where "expertise" consists of knowledge about a particular domain (e.g. interoperability) [START_REF] Power | Specifying an Expanded Framework for Classifying and Describing Decision Support Systems[END_REF]. In general, this "expertise" is stored in a knowledge model. Regarding the potential technologies to be adopted for building the knowledge model, a comparison between ontologies and relational databases are presented on [START_REF] Chandrasekaran | What are ontologies, and why do we need them?[END_REF]. The authors conclude that relational databases require specialisation and integration procedures, and they are one-oriented-purpose, and ontologies provide a restriction-free framework to represent a machine-readable reality. Some examples of ontology-based approaches are the approach to assessing records management systems [START_REF] Alalwan | An Ontology-based Approach to Assessing Records Management Systems[END_REF], the ontology-based system for supporting manufacturing sustainability [START_REF] Giovannini | Ontology-based system for supporting manufacturing sustainability[END_REF] and the ontology-based approach for supporting risk assessment for the intelligent configuration of supply networks [START_REF] Palmer | An ontology supported risk assessment approach for the intelligent configuration of supply networks[END_REF].

Improving the measurement mechanisms: merging qualitative and quantitative measures

In the past contributions i.e. from the early stages of INAS approaches to late early 10', we remarked that few of them were proposing a combination of qualitative and quantitative measures. Indeed, many of the proposed maturity models does not define threshold for delineating the different proposed maturity levels, what can cause ambiguity and misinterpretation when multiples experts are assessing the same system. Further, some INAS approaches based on quantitative measures do not explicit define the meaning of their numeric results.

However, since 2012, researchers are trying to combine quantitative and qualitative measures for providing sound and coherent rating scales. For example, the approaches (A14), (A26) and (A17) are proposing AHP and ANP methods for capturing the insight and qualitative ratings from assessors (i.e. who are evaluating the systems) and transforming them in numeric values. Based on these numeric values, the maturity levels are defined and qualitative descriptions associated. Further, approaches such (A6) and (A18) propose Fuzzy-based techniques for translating the qualitative ratings given by assessors into numeric values.

Therefore, we argue that new contributions should keep proposing this combination of measures for attributing rates in an objective and meaningful manner. Thus, techniques such as the AHP/ANP method [START_REF] Saaty | Decision makingthe Analytic Hierarchy and Network Processes (AHP/ANP)[END_REF] and the fuzzy logic methods [START_REF] Lee | Fuzzy logic in control systems: fuzzy logic controller. I[END_REF], [START_REF] Zadeh | Soft Computing and Fuzzy Logic, in: Fuzzy Sets, Fuzzy Logic, and Fuzzy Systems[END_REF] are assets for improving the measurement mechanisms.

Conclusion

Interoperability is considered as a crucial requirement to be satisfied when companies are pursuing new business opportunities and participating in collaborative networks. In such context, decision makers need a clear and a holistic view of the current state of their ecosystem. In this paper, we highlighted the importance of the Interoperability Assessment (INAS) as a mean to identify potential problems and improvement opportunities. We have compared INAS approaches based on a systematic literature review. The objective of the study was to identify the existing INAS methods, methodologies and approaches to verify how they are dealing with interoperability.

The systematic literature review uncovered 38 relevant INAS approaches. These approaches were classified according to their main domain of application, the year of publication and where they were published. Further, we compared only the 22 INAS approaches that are applied in a real case study or an illustrative example. Next, based on the comparative analysis, we have found five research perspectives related to limitations and improvements. The first one concerns to the lack of INAS approaches addressing the performance. We argue that Key Performance Indicators (KPI) for interoperation are useful for validating the implemented solutions. Thus, the definition of such KPI should be considered in future works.

The second refers to the lack of INAS approaches covering multiples layers and concerns of the enterprise. Consequently, their dependencies are not explicitly defined and formalised. We assert that a holistic overview of the system's interoperability can support stakeholders to better identify problems and to select and prioritise their decisions. Further, the lack of approaches providing best practices for improving interoperability based on the INAS results should be also considered. Such guidelines are essential as it can help stakeholders (e.g. system' engineers and enterprise architects) to design and implement interoperable systems. The lack of computer-mediated tools for supporting the whole or part of the assessment process is also an issue that should be addressed. Finally, research regarding measurement mechanisms based on the merging of qualitative and quantitative measures could also improve the current and future INAS approaches.

Based on the comparative analysis and the discussed limitations, we intend, as future work, to improve the INAS approach proposed and discussed in [START_REF] Leal | Towards a Semi-automated Tool for Interoperability Assessment: An Ontology-Based Approach[END_REF] and [START_REF] Leal | Assessing Interoperability Requirements in Networked Enterprises: A Model-Based System Engineering Approach[END_REF]. It will elaborate the existing dependencies regarding interoperability layers and concerns. It also will include a computer-mediated tool for facilitating the overall INAS process, i.e. it will automatically (i) calculate the interoperability level of an enterprise or between two enterprises; (ii) identify potential interoperability barriers and impacts on different layers and concerns of interoperability; and (iii) provide best practices for reducing the detrimental effects of the identified barriers.

Figure 1 -Figure 2 -

 12 Figure 1 -The most used keywords among the identified papers

Figure 3 -

 3 Figure 3 -The defined keywords

Figure 4 -

 4 Figure 4 -Paper selection process

Figure 5

 5 Figure 5 illustrates the number of papers published per year, from 1996 to 2018. We observe that the number of papers proposing INAS approaches increased in 2009 and 2016.

Figure 5 -

 5 Figure 5 -Number of papers published per year The analysis shows that the publications are divided as journal papers (35%) and conference proceedings (53%). The remaining 12% represents technical reports. The main conferences are the International Conference on Interoperability for Enterprise Software and Applications (3 papers) and the International Command and Control Research and Technology Symposium (3 papers). The main journals are the Computers in Industry (5 papers) and the Enterprise Information Systems (3 papers).Considering the domains addressed by the analysed papers, 41 of them focus on the Industry domain (including manufacturing supply chains, service providers, etc.), 16 consider the Military domain and 6 papers address Information Technology (IT) systems without considering a specific domain. Finally, 9 papers cover other domains such as health, public administration and crisis management. Table5shows the papers classification according to their addressed domains.

  𝐶 𝑒𝑥 = 𝐶 𝑖𝑛𝑖𝑒𝑥 -𝐶 𝑒𝑓𝑓𝑒𝑥 Cost of operation (Cop) It represents the difference between the initial cost allocated to operation (Ciniop) and the real cost of operation (Ceffop) 𝐶 𝑜𝑝 = 𝐶 𝑖𝑛𝑖𝑜𝑝 -𝐶 𝑒𝑓𝑓𝑜𝑝 Duration of data exchange (Tex) It represents the time measurement between the date of the emission of information (partner 1) (Tem1) and the date of reception of the information (partner 2) (Trec2). 𝑇 𝑒𝑥 = 𝑇 𝑟𝑒𝑐2 -𝑇 𝑒𝑚1 Duration of operation (Top) It represents the time measurement between the date of the reception of information (Trec2) and the date of operation (Top2) 𝑇 𝑜𝑝 = 𝑇 𝑜𝑝2 -𝑇 𝑟𝑒𝑐2 Quality of exchange (Qex) It represents the difference between the total number of sendings (Neff) and the number of successful sendings (Nsucc) 𝑄 𝑒𝑥 = 𝑁 𝑒𝑓𝑓 -𝑁 𝑠𝑢𝑐𝑐 Quality of operation (Qop) It represents the difference between the number of requests (Nreq) and the number of receptions (Nrec) 𝑄 𝑜𝑝 = 𝑁 𝑟𝑒𝑞 -𝑁 𝑟𝑒𝑐 Conformity (Qconf) It represents the difference between the total number of receptions (Nrec) and the number of conform receptions (Nconf) 𝑄 𝑐𝑜𝑛𝑓 = 𝑁 𝑟𝑒𝑐 -𝑁 𝑐𝑜𝑛𝑓

Table 2 . The Inclusion and exclusion criteria for step 1 Step Inclusion criteria Exclusion criteria

 2

		Paper written in English	Paper not written in English
		Paper that we have access to the full text	Paper without access to the full text
	St1	Primary study
		Paper establishing a link between "assessment" (and the	Other literature reviews
		variants terms) and interoperability

Table 4 .

 4 The paper selection phases

	Phase

Table 6 .

 6 The selected INAS approaches.

	ID	Name	Acronym	Authors and reference

A1

The levels of conceptual interoperability model LCIM (Tolk and Muguira 2003)

[START_REF] Tolk | The Levels of Conceptual Interoperability Model[END_REF]

,

[START_REF] Wang | The Levels of Conceptual Interoperability Model: Applying Systems Engineering Principles to M&S[END_REF]

)

[START_REF] Wang | The Levels of Conceptual Interoperability Model: Applying Systems Engineering Principles to M&S[END_REF]

,

[START_REF] Tolk | Supporting Network Enabled Capability by extending the Levels of Conceptual Interoperability Model to an interoperability maturity model[END_REF]

)

[START_REF] Tolk | Supporting Network Enabled Capability by extending the Levels of Conceptual Interoperability Model to an interoperability maturity model[END_REF]

Table 7 .

 7 The selected INAS approaches.

	ID	Name	Acronym	Authors and reference
	A9	A methodology to implement and improve interoperability	-
	A11	Reconceptualising measuring, benchmarking for improving interoperability in smart ecosystems	-	(Maheshwari and Janssen 2014) [74]
	A12	Ultra large scale systems interoperability maturity model	ULSSIMM	(Rezaei et al. 2014) [88]
	A13	Assessing interoperability of access equipment for broadband networks	-	(De Vito and Rapuano 2010) [92]
	A14	A framework for interoperability assessment in crisis management	-	(da Silva Avanzi et al. 2017) [99]
	A15 Levels of Information System Interoperability	LISI	(US Department of Defense 1998) [47]
		Maturity model for the structural elements of
	A16	coordination mechanisms in the collaborative	SECM-MM (Cuenca et al. 2013) [77]
		planning process
	A17	Evaluation of Interoperability between Automation Systems using Multi-criteria Methods	-	(Saturno et al. 2017) [93]
	A18	Performance evaluation of collaboration in the design process: Using interoperability measurement	-	(Neghab et al. 2015) [34]
	A19	Methodology for Interoperability Evaluation and Improvement	-	(Camara et al. 2010) [19], (Camara et al. 2012) [78], (Camara et al. 2014) [36]
	A20	Maturity Model for Interoperability Potential Measurement	MM-IRIS	(Campos et al. 2013) [79]
	A21	Customizable methodology to support technical processes interoperability assessment deployment in large companies	-	(Cornu et al. 2012) [23], (Cornu et al. 2012b) [80]
	A22	A holistic interoperability assessment based on requirements interdependencies	-	(Leal et al. 2017) [84], (Leal et al. 2017b) [85], (Leal et al. 2017c) [86], (Leal et al. 2017d) [87]

[START_REF] Daclin | Enterprise interoperability measurement -Basic concepts[END_REF]

[START_REF] Daclin | Enterprise interoperability measurement -Basic concepts[END_REF]

, (Chen and Daclin 2007)

[START_REF] Chen | Barriers Driven Methodology for Enterprise Interoperability[END_REF]

,

[START_REF] Daclin | Methodology for Enterprise Interoperability[END_REF]

[START_REF] Daclin | Methodology for Enterprise Interoperability[END_REF]

,

(Daclin et al. 2016

)

[START_REF] Daclin | Developing enterprise collaboration: a methodology to implement and improve interoperability[END_REF]

A10 The Interoperability Score i-Score

(Ford et al. 2007

)

[START_REF] Ford | The Interoperability Score[END_REF]

,

[START_REF] Ford | Measuring system interoperability[END_REF]

)

[START_REF] Ford | Measuring system interoperability[END_REF]

,

[START_REF] Ford | A General Method of Measuring Interoperability and Describing Its Impact on Operational Effectiveness[END_REF]

)

[START_REF] Ford | A General Method of Measuring Interoperability and Describing Its Impact on Operational Effectiveness[END_REF]

,

[START_REF] Chalyvidis | Using Supply Chain Interoperability as a Measure of Supply Chain Performance, Supply Chain Forum[END_REF]

)

[START_REF] Chalyvidis | Using Supply Chain Interoperability as a Measure of Supply Chain Performance, Supply Chain Forum[END_REF]

,

[START_REF] Chalyvidis | A method for measuring supply chain interoperability[END_REF]

)

[START_REF] Chalyvidis | A method for measuring supply chain interoperability[END_REF]

Table 9 . The areas of interoperability and their evaluation criteria. Adapted from [20]

 9

		Conceptual	Technological	Organisational
	Business	Business models for multi partnership and collaborative enterprise	Open IT infrastructure	Flexible organisation structure
	Process	Meta-modelling for multiple model mappings	Platforms and tools for collaborative execution of processes	Cross-enterprise collaborative Processes management
	Service	Meta-modelling for multiple model mappings	Automated services discovery and composition, shared applications	Collaborative application management services	and
	Data	Meta-modelling for multiple model mappings	Remote access to databases possible for applications, shared data	Personalised data management for different partners

Table 10 . Interoperability conclusions following the values of MPI and MEI. Adapted from [33]

 10

	Type evaluation	of	Interoperability measure	Value	Conclusion
				𝐴𝑙𝑙 𝑖𝑑𝑒𝑛𝑡𝑖𝑓𝑖𝑒𝑑 𝑠𝑒𝑚𝑎𝑛𝑡𝑖𝑐 𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛𝑠ℎ𝑖𝑝𝑠	=0	A is not interoperable with B
	MPI(A,B)		𝑣 𝐴→𝐵 =		𝑓𝑟𝑜𝑚 𝐴 𝑡𝑜 𝐵 𝐴𝑙𝑙 𝑚𝑎𝑛𝑑𝑎𝑡𝑜𝑟𝑦 𝑐𝑜𝑛𝑐𝑒𝑝𝑡𝑠 𝑓𝑟𝑜𝑚 𝐴	<100%	A is partially interoperable with B
					𝑡ℎ𝑎𝑡 𝑠ℎ𝑜𝑢𝑙𝑑 𝑏𝑒 𝑖𝑛𝑠𝑡𝑎𝑛𝑡𝑖𝑎𝑡𝑒𝑑 𝑖𝑛 𝐵	=100%	A is fully interoperable with B
	MEI(A,B)		𝑣 𝐴→𝐵 𝑒	=	𝐴𝑙𝑙 𝑖𝑑𝑒𝑛𝑡𝑖𝑓𝑖𝑒𝑑 𝑚𝑎𝑛𝑑𝑎𝑡𝑜𝑟𝑦 𝑠𝑒𝑚𝑎𝑛𝑡𝑖𝑐 𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛𝑠ℎ𝑖𝑝𝑠 𝐴𝑙𝑙 𝑚𝑎𝑛𝑑𝑎𝑡𝑜𝑟𝑦 𝑐𝑜𝑛𝑐𝑒𝑝𝑡𝑠 𝑓𝑟𝑜𝑚 𝐴 𝑡ℎ𝑎𝑡 𝑠ℎ𝑜𝑢𝑙𝑑 𝑏𝑒 𝑖𝑛𝑠𝑡𝑎𝑛𝑡𝑖𝑎𝑡𝑒𝑑 𝑖𝑛 𝐵	=0 <100% =100%	A is not interoperable with B A is partially interoperable with B but this interoperability is effective. A is fully interoperable with B and this interoperability is effective.

https://cogitant.sourceforge.io/

superdecisions.com

Acknowledgements

This work has been conducted in the context of the PLATINE project (PLAnning Transformation Interoperability in Networked Enterprises), financed by the national fund of research of the Grand Duchy of Luxembourg (FNR), under the grant C14/IS/8329172/R2.

Summary

Table 12 presents summary of the INAS approaches regarding the comparison criteria. The main findings and limitation are discussed hereinafter. The column "approach" identifies the considered INAS approach according to their given ID, i.e. "Approach A1: The levels of conceptual interoperability model" is identified as A1. Regarding the types of assessment, we outline that the Compatibility assessment is the most addressed in the seventeen approaches. It reflects the relevance of understanding thoroughly both systems that need to interoperate. It is related to the fact that most of the enterprises already have a list of primary partners or a desired one. The ten INAS approaches addressing the Potentiality assessment are more diversified comparing the other types. The Performance assessment is the lesser addressed with only three approaches.

Figure 6 to Figure 8 show the evolution of each type of assessment over years. In the following figures, we consider the 49 papers that describe the 22 concerned INAS approaches. Moreover, the number of approaches using Qualitative measurement mechanisms is equal to eighteen. The number of approaches proposing Quantitative measurement mechanisms is equal to thirteen. Among them, eight approaches are combining both types of mechanisms.

Eight INAS approaches are providing Best Practices or guidelines for improving systems interoperability. Indeed, best practices are useful for decision makers in order to design the to-be situation of the system(s) of interest and to implement interoperability solutions. Finally, the results of this review indicate that the majority of the approaches do not have a Computer-Mediated Tool for supporting the assessment process. Only nine approaches propose computer-mediated tools for supporting the INAS process.

Next, we analyse and discuss the coverage of interoperability barriers and concerns by the INAS approaches. This allows us to identify which are the INAS approaches addressing most of the barriers and concerns. Table 13 presents a matrix considering the cross section between the interoperability barriers and concerns. The approach ID is put into a corresponding cell when dealing with the considered barrier and concern. A4), (A5), (A6), (A9), (A14), (A20), (A22) (A5), (A6), (A9), (A14), (A20), (A22) (A4), (A5), (A6), (A9), (A11), (A14), (A16), (A20), (A21), (A22) Process (A2), (A5), (A6), (A9), (A10), (A12), (A14), (A18), (A20), (A22) (A5), (A6), (A9), (A10), (A14), (A19), (A20), (A22) (A2), (A5), (A6), (A9), (A10), (A11), (A12), (A14), (A16), (A20), (A21), (A22) Service (A5), (A6), (A8), (A9), (A12), (A14), (A15), (A17), (A18), (A21), (A22) (A5), (A6), (A8), (A9), (A11), (A12), (A14), (A15), (A17), (A19), (A20), (A21), (A22) (A5), (A6), (A9), (A12), (A14), (A21), (A22) Data (A1), (A2), (A3), (A5), (A6), (A7), (A8), (A9), (A11), (A14), (A15), (A18), (A20), (A21), (A22) (A1), (A2), (A3), (A5), (A6), (A8), (A9), (A11), (A12), (A13), (A14), (A15), (A17), (A19), (A20), (A21), (A22) (A5), (A6), (A9), (A11), (A12), (A14), (A21), (A22) Appendix 1. The excluded approaches.

Table A1 shows the approaches that were not considered in this review. [61], [START_REF]CMMI Product Team, Capability Maturity Model® Integration for Acquisition, Version 1.3[END_REF], [START_REF]CMMI Product Team, Capability Maturity Model® Integration for Development, Version 1.3[END_REF]