Ankica Barisic

Dušan Savić

Rima Al-Ali

Ivan Ruchkin

Dominique Blouin

Antonio Cicchetti

Raheleh Eslampanah

Oksana Nikiforova

Mustafa Abshir

Moharram Challenger

Ankica Barišić

Claudio Gomes

Ferhat Erata

Bedir Tekinerdogan

Vasco Amaral

Miguel Goulao

Systematic Literature Review on Multi-Paradigm Modelling for Cyber-Physical Systems

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

This document reports on the Systematic Literature Review (SLR) of Multi-Paradigm modelling for Cyber-Physical Systems (MPM4CPS) of Working Group4 on Education and Dissemination of the ICT COST Action IC1404. The goal of this SLR is to systematically describe the latest original research studies on MPM4CPS, and in particular the models, meta-models, languages, tools, and processes that enable MPM4CPS.

In this chapter, the report describes the major underlying concepts that support multiparadigm modelling of CPS, explains our research method, and gives an overview of the related work.

In the following chapters, we give a detailed description of our study protocol (chapter 2), of the platform and the process behind the study (chapter 3), and of the study results (chapter 4). The conclusion of the report can be found in chapter 5, followed by Appendix A with the description and the results of the protocol validation survey.

Keywords and definitions

A model is an abstraction of a system that highlights the system's relevant aspects, and is often used in place of the system under development [START_REF] Kühne | Matters of (meta-) modeling[END_REF]). Thus, a model can be considered a simplified representation (or view) of a system. Modelling is a fundamental technique used in various Engineering fields, as well as other areas such as Physics, Mathematics, Biology, Economy, Politics and Philosophy [START_REF] Favre | Megamodelling and Etymology[END_REF]).

The act of modelling involves three distinct concepts [START_REF] Lee | The past, present and future of cyber-physical systems: a focus on models[END_REF]): the subject that is being modeled, the model itself, and the modelling paradigm (also known as the formalism). The subject of modelling is typically either present (or potentially present) in the world -as in the case of scientific disciplines -or is intentionally designed to be deployed in the world -as in the case of engineering disciplines. Models differ in the levels of abstraction. A more abstract model omits more information about the subject being modeled.

Formalisms are mathematical approaches to describing models, consisting of an abstract syntax and a formal semantics [START_REF] Broman | Viewpoints, Formalisms, Languages, and Tools for Cyber-physical Systems[END_REF]). Modelling languages are concrete implementations of formalisms, expressing systems in a formal and precise way by using diagrams, rules, symbols, signs, letters, numerals, and so on. A modelling language may implement more than one formalism by combining their syntax and semantics. Finally, a language is associated with tools to support it, such as parsers, simulators, model-checkers, and code generators. A meta-model is a definition of a modelling language, which should be defined in a more abstract language to describe new modelling languages themselves. The meta-modelling language is a modelling language in its own right, and its meta-model is referred to as a meta-meta-model 1 .

Model-Driven Development (MDD) is a software development approach that uses models as specifications of software and relies on transformations of those models to source code for software development 2 . MDD is successfully applied in software engineering, and it has been advocated as an effective way to deal with software complexity. In MDD, models are first-class artifacts used by the developers at multiple levels of abstraction. The source code for the system's implementation is primarily generated from models, and sometimes can be not edited by 1. Introduction the developers. Therefore, creation, debugging, and testing of models are the primary activities in this approach to software development.

Cyber-Physical Systems (CPS) are systems that integrate computation, networking, and physical processes. The key characteristic of CPS is their seamless integration of both hardware and software resources for computational, communication and control purposes, all of them co-designed with the physical engineered components [START_REF] Lun | Cyber-Physical Systems Security: a Systematic Mapping Study[END_REF]). The term cyberphysical systems was coined in 2006 in a workshop organized by the National Science Foundation3 whose participants were selected experts from the USA and European Union who advocated to use this term in order to emphasis importance the integration of digital computation with physical processes. Over the last decade, CPS have also been referred to as embedded or mechatronic systems.

The intellectual challenge of CPS lies in the intersection (not the union) of the physical and the cyber domains of engineering. CPS combines engineering methods from mechanical, environmental, civil, electrical, bio-medical, chemical, aeronautical, and industrial engineering with the models and methods of computer science [START_REF] Lee | The past, present and future of cyber-physical systems: a focus on models[END_REF]). Therefore, engineering CPS requires trans-disciplinary approaches merging different engineering disciplines. An important challenge of CPS is to conjoin the classic models from computer science (programs, graphs, object models) with the abstractions typically used for physical processes including (differential equations and stochastic processes). Currently, while many research efforts are in progress, there is no common modelling approach yet to all of the involved disciplines in CPS.

CPS technology is typically based on the foundations of embedded systems, sensor technology, and network technology. Depending on the context of a particular CPS, the following requirements may be relevant [START_REF] Liu | Characteristic, Architecture, Technology, and Design Methodology of Cyber-Physical Systems[END_REF]):

• Reliability and fault-tolerance A model of a CPS comprises models of physical processes as well as models of the software, computation, and networks. Physical processes are typically viewed as multiple parallel processes related to mechanical motion, energy exchange, electrical dynamics, and so on. Measuring and controlling the dynamics of these processes by orchestrating actions that influence the processes are the main tasks of CPS (Derler et al. (2012a)). Digital computation and communication is characterized by discretization errors and delays in time, which are major modelling concerns for CPS software and networks.

A common model for CPS is a hybrid system, where physical processes are represented as continuous-time models, and computations are described as discrete-transition models such as state machines, dataflow models, synchronous/reactive models, and/or discrete event models. Continuous-time models are typically analyzed with solvers that numerically approximate the solutions to differential equations (Derler et al. (2012a)). There are many challenges in defining good hybrid systems modelling languages. These challenges include ensuring that models of deterministic systems are indeed determinate. It is also challenging to accurately represent distinct events that are causally related but occur at the same time (Derler et al. (2012a)).

The development activities for modern complex systems, and in particular CPS, encompass multiple technical domains and teams, where each team is using its own set of modelling languages for the aspects of the system that are relevant to that team. Thus, it is required to properly integrate these languages. Indeed, the experience shows that using a single language to cover all domains would lead to very large monolithic languages, not easily customized for the development environment and tools needed by development organizations. These considerations lead to Multi-Paradigm Modelling (MPM) of CPS -a school of thought that advocates the combination of reusable modular modelling languages with different paradigms, using a single monolithic language for the whole system [START_REF] Vangheluwe | An introduction to multi-paradigm modelling and simulation[END_REF]).

MPM proposes to model every relevant part and aspect of a system explicitly, at the most appropriate level(s) of abstraction, using the most appropriate modelling formalism(s). The current approaches to MPM offer somewhat modular and incremental treatment of a single aspect of modelling languages or model operations. However, support for modelling large-scale complex CPS in a modular and incremental fashion, as required for industrial and practical contexts, is not offered so far. The literature review described in the rest of this document aims to survey and assess the state-of-the-art of MPM for CPS.

One of the recent efforts in MPM for CPS was the EU-based ICT COST Action IC1404 Multi-Paradigm modelling for Cyber-Physical Systems In the context of WG1 of this action, Al-Ali et al. (2017b) developed an ontology for the major concepts of CPS and CPS models has been developed by in the report on the the Framework to Relate / Combine Modelling Languages and Techniques (WG 1.2). In particular, this report presents three related ontologies: an ontology of CPS concepts (such as computational and physical components), an ontology of MPM concepts (such as formalism and logic), and finally an MPM for CPS ontology that combines the previous two. This literature review relies on these ontologies for definitions of MPM for CPS concepts.

For definition and interpretation of terms related to CPS and MPM, this literature review relies on three guiding documents, two of which are the deliverables of the aforementioned COST Action. The documents, referred to as D1, D2, and D3, are as follows:

• D1: State-of-the-Art on Current Formalisms used in Cyber-Physical Systems Development (WG 1.1,. This document provides a comprehensive collection of formalisms, languages, and tools used in CPS engineering. This document includes such formalisms as Bayesian networks and hybrid automata.

• D2: A Survey on Concepts, Applications, and Challenges in Cyber-Physical Systems (KSII Transations on Internet and Information Systems, [START_REF] Gunes | A Survey on Concepts, Applications, and Challenges in Cyber-Physical Systems[END_REF]). This document is used as a reference for the definition of domains of application for CPS. The document includes such domains as smart manufacturing and air transportation.

• D3: CPS Profile -Pilot Assessment (WG 4.3, by Goulão et al. (2016a)). This document describes an ontology of expert knowledge in CPS that applies to CPS engineers. The concepts both from formalisms and application domains are used to characterize the knowledge of CPS engineers. The document includes topics such as DSL foundations, Simulink, smart cities, and automotive systems.

More details on how these documents inform our research questions can be found in chapter 2.

Research Method

Systematic Literature Review (SLR) is a research method to obtain, evaluate, and interpret information related to a research question. A SLR provides an objective reliable, rigorous, and methodological manner to conduct some study. In this report, we conduct a SLR of multiparadigm modelling of Cyber-Physical Systems (CPS). This research aim requires accumulating a body of knowledge for various reasons, which are: 1) justifying the basis for future research, 2) learning as much as possible from other domains related to the topic, and 3) providing a basis for other researchers as well as students who consider learning about and contributing to this area.

The research objective of this SLR is to identify and analyze multi-paradigm modelling approaches for CPS. The expected results of the study will give an overview of the existing literature on multi-paradigm modelling of CPS. With this objective, we systematically investigate the research literature of this topic in the period between 2006-2017. The quest is to identify models, and implementations in a model-driven formal way that lends itself to a more systematic development of CPSs. Next is to summarize the state-of-the-art research trends, as well as to categorize proposed approaches, techniques, tools and methods for assessing and improving a multi-paradigm modelling of CPS.

The findings in the literature are to be mapped to the ontologies and their individuals that were introduced in a deliverable of WG1 (Al-Ali et al. ( 2017a)), in addition to identifying the topics that cover the interests of MPM4CPS engineers [START_REF] Goulão | Y1 and Y2 Technical Report WG4 . 3[END_REF]). In context of WG4, it is necessary to map the capabilities that should be mastered by the CPS engineer, in particular: Map of fundamental CPS engineering knowledge; Map MPM4CPS knowledge; and Organize different profiles for the CPS Experts. The all-over result will contribute to construction of a course on MPM4CPS, which is expected to serve for education purpose. [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF] In this SLR, we planned our SLR process as suggested by [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF] (See Figure 1.1).

During pre-planning we identified related studies, relevant keywords and definitions, which we further use to define our protocol. In the planning phase, reported in Chapter 2, the review protocol is developed as well as research strategy. The protocol was validated through a survey which was provided to members of COST action, and through pilot review session with some of reviewers. During conducting phase, which is reported in Chapter 5, the relevant research was identified two fold; (1) through automatic search over digital databases defined in protocol, and , (2) from the SLR on sustainability in modelling of CPS reported in [START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF]. Articles obtained by automatic search passed screening phase, after which reviewers extracted required data. The data was further synthesized and quantitative results are reported in Chapter 4.

Related Work

Multiple systematic literature reviews have been produced on different topics in software engineering, but so far none has been conducted that investigates the multi-paradigm modelling of CPS. An original contribution of our effort is that for the first time in this research field, we have followed an SLR method to be as objective as possible in our selection of primary studies. However, we examine in detail SLR protocol from related work. Related studies provide the necessary instruments to replicate the study, or in our case to extend them to cover new research. [START_REF] Lun | Cyber-Physical Systems Security: a Systematic Mapping Study[END_REF] published Cyber-Physical Systems Security: a Systematic Mapping Study. This study aim at identifying, classifying, and analyzing existing research on CPS security in order to better understand how security is actually addressed when dealing with CPS. Authors empirically define a comparison framework for classifying methods or techniques for CPS security.

From the collected data authors observe that even if solutions for CPS security has emerged only recently, in the last years they are gaining a sharply increasing scientific interest across heterogeneous publication venues. The systematic map of research on CPS security provided here is based on, for instance, application fields, various system components, related algorithms and models, attacks characteristics and defense strategies. Following string was used to obtain a primary studies: (('cyber physical' OR 'cyber-physical' OR cyberphysical OR 'networked control') AND system*) OR CPS OR NCS) AND (attack* OR secur* OR protect*)) [START_REF] Gunes | A Survey on Concepts, Applications, and Challenges in Cyber-Physical Systems[END_REF], presents A Survey on Concepts, Applications, and Challenges in Cyber-Physical Systems. In order to shed some light on the origins, the terminology, relatively similar concepts, and today's challenges in CPS, authors presented a survey on related literature discussing practical applications and dominant research domains. Since CPS is a very broad research area, CPSs span diverse applications in different scales. Therefore, each application necessitates strong reasoning capabilities with respect to unique system-level requirements/challenges, the integration of cutting-edge technologies into the related application, and overall impact on the real world. Authors conclude that existing legacy systems have limited awareness of the CPS requirements, and that revolutionary design approaches are necessary to achieve the overall system objectives. Derler et al. (2012b), focused on the challenges of modelling CPSs that arise from the heterogeneity, concurrency, and sensitivity to timing of such systems. A model of a CPS comprises models of physical processes as well as models of the software, computation platforms, and networks. One of the main challenges is to keep model components consistent and to check for correctness of connections between components. As the model grows, the possibility of error is also growing. In this paper, authors identified three types of errors: 1) unit errors,2) semantic errors,and 3) transposition errors. Also, authors analyzed the state of the art in existing tools and methods.

Introduction

Hehenberger et al. (2016) discussed the importance of design, modelling, simulation and integration of CPS and focused on methods and applications. The authors emphasized that designing CPS requires a multi-disciplinary development process during which designers should focus on integration and interaction physical and computational components. Further, in this paper the authors presented case studies and current best practice from industry.

Al-Ali (2017) conducted a systematic mapping review to find the domains and the challenges related to collective behavior only that industry targets nowadays. The primary studies that they used are EU projects since 2007. The results of the study showed that the most targeted domain is automation and the most addressed challenge is resources. Moreover, domains such as transport, cloud, energy, and smart grids lack variety of challenges, while many other domains do not target any challenges (telecommunication, medical and manufacturing domains).

Protocol

In this chapter we identify the main research questions and we produce a well-defined review protocol that describes in detail the various steps we had to follow. The produced protocol has been further evaluated and refined according to the obtained feedback and is presented in this section in the same form that was used in the conduction phase.

Protocol Definition

Research Questions

The overall objective of our study is to provide an overview of the current state of the art of approaches supporting the modelling of CPS and identify ones which apply a multi-paradigm modelling approach. The main research questions are the following:

• RQ1 Which modelling approaches exist for building CPS?

• RQ2 What are the existing approaches that combine different multi-modelling formalism for CPS?

• RQ3 Which application domains have been considered?

• RQ4 What is profile of person which perform modelling of CPS?

Furthermore, we performed PICOC analysis which specified in detail our search for evidence:

Population: The population is composed of studies in which we found reports about works for modelling CPS and/or approaches for multi-paradigm modelling. No specific industry, system or application domain were considered.

Intervention:

The review searches for reports of methodologies for multi-paradigm modelling, namely focusing on two important factors, which are: Reported for CPS; and Reported for similar products and applicable to CPS. We also search for the methodology/tool/technology/procedure that support the modelling of CPS.

Comparison: Not applicable.

Outcomes: Outcomes should point to techniques, methods and metrics that can be used to address the multi-paradigm modelling of CPS.

Context:All practitioners: Academy and Industry.

The objective of RQ1 is to analyze and classify primary studies, which report on modelling of CPS, and to find among existing approaches if there are ones which take sustainability of CPS into account. It is expected to involve retrieving the type of models used to describe the approach or the system, a list of modelling languages and tools which were developed to support the approach or were used by the reported approach, and determine if they report a systematic process for application of the given approach.

The objective of RQ2 is to analyze and classify among the studies which report on modelling of CPS ones which report multi-paradigm modelling. It is expected to retrieve the information to which extend is MDD approach complete, which part of CPS is modeled, which MPM formalism are used and what are integration mechanisms for the given models.

The objective of RQ3 is to analyze the application domains of modelling approaches for CPS. We find approach to be general if they are applicable to CPS in general, while in most cases we expect to find the domain-specific solutions (e.g. smart grid, intelligent transport).

Protocol

Finally, the RQ4 objective is to extract the profile of the person(s) that perform the modelling for the CPSs. It is necessary to identify if the study report the stakeholders involved in modelling and identify their background knowledge, as well as technical background of the authors.

Defining research questions was started during the 1st STSM while Dusan Savic, visited Universidade Nova de Lisboa, in period from 16th to 25th December 2015 [START_REF] Savic | Definition of the profile of a MPM4CPS course and a systematic literature review on modeling for CPS[END_REF].

After that, during the Vienna Working Group Meeting1 (15-16 April 2016) these research questions was presented to participants.

Data sources and search strategy

We obtained data for this SLR twofold:

-Using automatic search over digital libraries the period from 2006-2017 as it is indicated within related research that interest on topic of CPS development boosted from 2006.

-Manually extracting the primary studies from another SLR reported in [START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF] for period 2011-2017. This study intersect with ours by heaving the same RQ1 and RQ3, however the search string was more wide and included the sustainability concern.

We selected the studies which reported modelling approaches for building CPS (RQ1).

Data sources for automatic search were selected in order to include the most relevant journal, conferences and international peer-reviewed workshops that are concerned with the topic of multi-paradigm modelling or CPS. For the automated we selected the following digital libraries: 1. ACM Digital Library (ACM) 2. IEEExplore (IEEE) 3. Science Direct (SD) 4. Springer Link (SL) 5. Scopus We used the following research string to obtain primary studies: Id Criteria E1 Informal literature (powerpoint slides, conference reviews, informal reports) and secondary/tertiary studies (reviews, editorials, abstracts, keynotes, posters, surveys, books) E2 Duplicated papers E3 Papers that did not apply to research questions i.e. they did not report the method for modelling approach for CPS E4 Papers with the same content in different paper versions E5 Papers written a language other than English E6 Purely hardware, or electrical engineering perspective papers E7 Secondary study inclusion and exclusion criteria are applied during selecting primary studies obtained by automatic search over digital databases (the screening phase) to the titles, keywords and abstract of primary studies which were identified by the search strategy. This studies were temporarily included but might be excluded during the data analysis (the classification phase) when the whole publication had been read.

("multi-paradigm" OR "multi-formalism" OR "heterogeneous formalism" OR "unified modelling formalism" OR "multi-model language") AND ((("cyber physical" OR CPS OR "cyberphysical" OR cyberphysical OR smart) AND system*) OR CPS )AND ("modelling approach" OR "modeling approach" OR "integrate modelling" OR "integrate modeling" OR "model driven" OR "model-driven") AND ("software engineering" OR "software system")

Study selection criteria

Study quality assessment

We reused quality assessment criteria from the related work [START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF]. To have means to reflect a confidence of the reviewer, we used two self-assessment points (see Table 2.3). In a case that a reviewer is not very confident about the paper, the additional reviewer was asked to make revision and the assessment scores will be discussed.

Id

Self-Assessment Criteria Score SA1 Reviewers confidence about content of the study 1 = Very confident 0.5 = Confident 0 = Not very confident SA2 Reviewers confidence about quality of the study

The quality assessment list presented in Table 2.4 has been used to assess the quality of studies after the data analysis. We did not define any exclusion criteria regarding the quality of the study, but we find it meaningful to present statistics and observe the impact of the study. Therefore, the list served as a compliment to extracted data related to the research questions and was used later in result analysis.

To characterize a first criteria (relevance of journal or conference), we decided to use CORE20182 ,3 ranks list. Venues which were not registered at the portal we marked as '0-Not relevant'.

For the second criteria, we took a number of citations reported by Google Scholar4 . * apply for paper published before 2014; while ** for paper published 2014 and after. We did not define any exclusion criteria regarding the quality of study, but we find it meaningful to present statistics at the end and observe if it does make any impact.

Protocol

Id Assessment Criteria Scale

QA1 What is the relevance of the paper according to the conference/journal where it was published?

Data Extraction Form

Data Extraction Form is defined in a way to collect the information in the primary study which can present the answer to the four research questions defined in Section 2.1.1 (See Table 2.5).

For all questions defined in data extraction for, reviewers were instructed that if the answer to the question is 'Yes' it is necessary to write the description in text box 'What it is?'. For questions which have an answer only TextBox (TB) field it was necessary to fill it in. Specific suggestions for filling the TB were given under each question. Reviewers were also provided with following documents to assist them while answering the questions: D1 (Al-Ali et al. ( 2017a)); D2 [START_REF] Gunes | A Survey on Concepts, Applications, and Challenges in Cyber-Physical Systems[END_REF]); D3 (Goulão et al. (2016a)).

For RQ1 Which modelling approaches exist for building CPS?, we are identifying if the paper report modelling approach for building CPS, and if it does we register if it reports a model/metamodel, a tool or a process. The Q1 Does the paper report modelling approaches for building CPS? is answered by selecting Yes/No (Y/N), and the TextBox (TB) is provided so the reviewer can report in more details which modelling approach is provided. It can be any model for any phase of building the CPS. If the answer to this question is NO then the paper should be excluded from the classification so it is not necessary to proceed with the classification. Following question, Q1.1 Does the paper report a model/meta-model?, has Y/N single choice option and a TB. The reviewer is expected to report if meta-model or instance model is presented for approach or for the use case, as well as to specify which kind of model is presented. Further, the Q1.2 Does the paper report a tool/language? has Y/N single choice option an a TB. It is possible that paper reports on (1) tool/language which was built to support the approach OR/AND (2) tool/language(s) which are used in the approach. It was necessary to report on both, by categorizing them with (1)/(2) option. Catalog of modelling tools and languages was given in Section

RQ1 Which modelling approaches exist for building CPS?

Q1 Does the paper report modelling approaches for building CPS? Q1.1 Does the paper report a model/meta-model? Q1.2 Does the paper report a tool/language? Q1.3 Does the paper report the model of the adopted process? RQ2 Which modelling approaches for addressing multi-paradigm modelling of CPS are presented? Q2 Does the paper report a multi-paradigm modelling approach? Q2.1 To which extent the paper present a complete model-driven development approach? Q2.2 Which part of the CPS is modeled? Q2.3 Which formalism(s) is(are) used for modelling the CPS? Q2.4 What is the integration mechanism for the presented models? RQ3 Which application domains have been considered? Q3

Is the approach domain-specific? Q3.1 Which application domain is addressed? RQ4 What can be inferred to be the profile of the person(s) that perform the modelling for the CPS? Q4 Does the paper report the actors/stakeholders involved in modelling of the CPS? Q4.1 Does the paper report the modeler's background knowledge? Q4.2 What is(are) the technical background of the authors? 2.3 and Section 2.4 of document D1 Al-Ali et al. (2017a). Reviewers were also expected to report if the used tool/language already belongs to a list by indicate the section number if it was part of the Catalog. Finally, the Q1.3 Does the paper report the model of the adopted process? has as well Y/N single choice option an a TB. It was expected to report if modelling was performed in a structured way and is that way described (preferably in a formal fashion, using languages such as BPMN or is provided in narrative form (step by step manner)).

For RQ2 Which modelling approaches for addressing multi-paradigm modelling of CPS are presented?, we identify if the paper report approach which fits in multi-paradigm modelling, and identify scope of approach, modeled CPS parts, used formalisms, and integration mechanisms. For Q2 Does the paper report a multi-paradigm modelling approach?, we identify if the approach using different views of same model or provide ones for different models, is it using DEVS, Statecharts, Petri-nets, or other formalisms all together. The question Q2.1 To which extent the paper present a complete model-driven development approach? requires to choose the component of supporting tools: editors, generators and/or transformations to artifacts (prone to be used for simulation platforms, or execution environments). The next question Q2.2 Which part of the CPS is modeled? requires from the reviewer to select certain component of CPS, for which the modelling approach is applied or offered in the research, e.g. Hardware, Network, Environment, or Software. Further question Q2.3 Which formalism(s) is(are) used for modelling the CPS? is mandatory to be filled if the answer to Q2 is YES and requires to select the formalism from the list of formalisms given in Section 2.2. of document D1 Al-Ali et al. (2017a). Reviewers are asked to indicate the name of the formalism and the section. Finally, the question Q2.4 What is the integration mechanism for the presented models? requires to identify is the integration achieved by using for instance mega-modelling approach, models merge, links, or other.

For RQ3 Which application domains have been considered?, we register if the approach is domain-specific, and if it is, we want to know which application domain is addressed. The

Protocol

Q3 Is the approach domain-specific? is answered by selecting Yes/No (Y/N). In our case the approach is general if it is applicable to CPS in general, however sometimes the approach is built for a specific domain (e.g. for smart grid, train transport etc., therefore, in the TextBox reviewer should specify the domain by the exact wording used in the paper for approach specification, or the case studies in which approach was applied (e.g. smart grid, self-driving vehicle etc.) In the text box Q3.1 Which application domain is addressed? the reviewer can select the domain from the following:

• Smart Manufacturing If Other is selected, it is necessary to specify how reviewer would name this new domain category in the separate text box. It is possible that approach can be applicable to several domains, therefore it is possible to make maximum 4 choices, as well as to add new domain suggestions.

Finally, for RQ4 What can be inferred to be the profile of the person(s) that perform the modelling for the CPS?, we explicitly register if the paper reports who is involved in the modelling of CPS and what is the profile. The question Q4 Does the paper report the actors/stakeholders involved in modelling of the CPS? is answered by selecting Yes/No (Y/N) based on the information in the paper about people reported to be involved in the development of approach (analysis, design, implementation, evaluation...) for instance end user of approach, or the CPS system. Reviewer should categorize the participants, by following:

• CPS engineer • CPS user • Domain expert • Evaluation expert • Other
The provided categorization is general and it is possible to select several categories and suggest new ones by selecting the Other. It is possible to choose 3 options, but also suggest additional category in a separate TextBox. The question Q4.1 Does the paper report the modelers background knowledge? is answered by selecting Yes/No (Y/N) and reviewer is asked to report on background knowledge topics and try to relate them to the ontology presented in the CPS Profile study in document (D3) Goulão et al. (2016a). For Q4.2 What is(are) the technical background of the authors? the technical background should be specified for each author from their public profile (e.g. University/Institute official page, Google Scholar, Research Gate, LinkedIn). Reviewers are asked to provide areas of expertise for authors, selecting them from the following:

• Electrical Engineering • Mechanical Engineering • Physics • Software Engineering • Other
The provided categorization is general and it was possible to select several categories and suggest new ones by selecting the Other. It was possible to choose 3 options from drop down menu, but also suggest more in TextBox.

Protocol Validation

This section describes how the proposed protocol has been validated with three techniques:

• Query testing:

• Validation survey:

• Pilot session:

Query testing

The research goal of this SLR (section 1.2) and the concepts behind it (section 1.1) were used as a basis for the creation of a search string in which to identify primary studies. In order to clearly define keywords, we analyzed all of our research questions separately, and divided the keywords into three main groups:

• Cyber-physical systems • modelling approaches for cyber-physical systems • Combination of modelling approaches for cyber-physical systems For each of these groups we created a search sub-string, and then integrated the sub-strings into the final search query.

The first group is consisted of terms referring to cyber-physical system such as cyber-physical system, embedded system, real-time system, hybrid system, sensor networks system, smart system, critical system and we created a first part of the our research string is a combination of the main item.

The second group was included the keywords that referred to modelling and simulation terms (modelling and simulation, model-driven, modelling approach, model-approach) which we obtained the second part from our research string.

Finally, the third group were referred to formalism and paradigm like multi-paradigm, multiformalism, heterogeneous formalism, unified modelling formalism, multi-model language.

In order to find the final, most appropriate combination of these sub-strings that presents our final search query, we defined several versions of query strings and conducted searching process in all the selected databases reported in Section 2.1.2. These versions were evaluated in a survey discussed in the next section.

Testing these search queries was one of the tasks that we performed from April 14-30, 2018 during the 3rd STSM while Ankica Barisic [START_REF] Barišić | STSM Report: Systematic literature review on multi-paradigm modeling for CPS Systems[END_REF]) visited University of Belgrade, Faculty of Organizational Sciences.

The three versions of search queries that we chose are given below: Q1 -"(""cyber physical"" OR ""cyber-physical"" OR CPS OR cyberphysical OR smart OR critical) AND ((""integrate model*"" OR ""composable model*"") OR (""integrate simulation"" OR ""composable simulation"")) AND (""model driven"" OR model-driven OR ""model based"" OR modelbased)". Q2 -("multi-paradigm" OR "multi-formalism" OR "heterogeneous formalism" OR "unified modelling formalism" OR "multi-model language") AND (("cyber physical" OR "cyber-physical" OR cyberphysical OR smart) AND system*) AND ("modelling approach" OR "modeling approach" OR "integrate modelling" OR "integrate modeling" OR "model driven" OR "model-driven") AND ("software engineering" OR "software system").

Protocol

Q3 -"("multi-paradigm" OR "multi-formalism" OR ""heterogeneous formalism"") AND (""Modeling and Simulation"" OR ""Integrate modeling"") AND (""cyber-physical system "" OR ""hybrid system"" OR ""embedded system"" OR ""real-time system"" OR ""smart system"")".

Table 2.6 presents the results of the final set of relevant papers selected from the data sources obtained by the Q1, Q2 and Q3 research queries. Each row shows the number of papers obtained from a particular source that resulted from each search query. From Q1, we got 526 total papers, from Q3 345 total papers, and from Q2 134 total papers.

Next, we discuss the survey in which the protocol was validated and the query was selected.

Validation survey with a COST action members

The purpose of this survey is to evaluate the protocol proposed to conduct a SLR study on Multi-Paradigm modelling of Cyber-Physical Systems.

We performed the internal validation of protocol and announced the survey during a meeting session with our group members during COST Action meeting in Riga on April, 26-27 2018 5 . Before inviting all members of COST action to participate in thus survey, we introduced the participants to the SLR objective. For this purpose, we created a Google sheets survey6 that describes the key aspects of SLR (presented in Section2.1).

For each of these aspects we created a separate Google sheet. The SLR Basic Data sheet contained information about SLR objective, SLR requirements, SLR team and list of defined activities and users who are responsible for these activities. The Research Questions sheet contains information by following PICOC criteria: Population, Intervention, Comparison, Outcomes and Context. The SLR Search Sources sheet contains information about database library, while search keywords and queries are presented in the sheet with the same name. The other three sheets presents inclusion and exclusion criteria, quality assessment strategy and data extraction strategy, respectively. Each question in survey was linked with particular Google sheet.

A survey form was created with several parts. The first part provides general information about the participants and their expertise concerning the SLR. The main part of survey is related to the key aspects of SLR such as Research questions, Search sources, Search keywords and queries, Inclusion and exclusion criteria, Data extraction strategy and Quality assessment strategy that we want to validate.

A total of 10 participants participated in this survey. Five participants chose the answer of I am knowledgeable in SLRs, though not an expert, three of them selected the answer I am a lecturer and/or publisher of SLR, while the remaining two participants chose the answer I am not an expert. My evaluation is that of an informed outsider on the question about their overall expertise concerning the SLRs (see Figure 2.1 for an illustration). Figure 2.2 shows the names of the participants. This query-related section of the survey contains five questions. Four of them are related to the question which search query best fits the objective of SLR, while the last question is openended, and allows participants to give us some suggestions regarding proposed search queries.

For question Q17 : (Figure 2.3) using the Likert 5-point scale, one participant disagreed (point 2), and one neither agreed or disagreed (point 3), three participants agreed (point 4), while 5 participants strongly agreed with this question.

The results for questions Q1, Q28 and Q39 are given in Figure 2.3.

The results in Figure 2.3 show that participants gave the greatest importance to query string Q2 (45), while Q1 and Q3 had the same importance (42). These answers indicate that there is no significant difference in the importance of queries. However, if we look the result for the next question "Which query in your opinion is a best fit for SLR objectives" the difference is more noticeable. The results for this question (see Figure 2.4) shows the participants in survey; 7 participants proposed using Q2, while Q1 proposed 2 participants, and 1 participant proposed Q3.

Protocol

Pilot session with reviewers

After the survey was concluded, we refined the protocol in accordance to the obtained feedback. However, we found it was necessary to provide additional descriptions regarding answering each question related to data extraction. In order to provide comprehensive descriptions, we organized the special session dedicated to this task in context of WG4 during MPM4CPS workshop in Izmir, Turkey10 , which took place between 17 and 19 of September 2018. All meeting participants helped in providing the appropriate descriptions to be provided to reviewers, regarding each question presented in Table 2.5.

As a follow up to the workshop meeting, we organized the pilot session with three reviewers, namely Raheleh Eslampanah, Moharram Challenger, and Mustafa Abshir, who volunteered to perform a pilot session following the protocol. Each reviewer was assigned to read and classify two primary studies. The complete protocol, with a special answer sheet, was provided to reviewers in the form of a Google Sheet document. After completing their assignments, reviewers had a live session with the SLR project lead, Ankica Barisic. During this session, each of the questions was re-evaluated, and reviewers expressed their doubts and eventual problems which they encountered while providing their answers. The completeness of the answers given in text boxes was also evaluated and discussed.

Several necessary clarifications were identified as a result of this session. For instance, it was not trivial for the reviewers to understand that it was necessary to provide the description for the different types of instance models and meta-models which they found in primary studies, as some were provided for a case study which was used to illustrate the approach, while others were provided for approach itself. Further, it was not clear to reviewers that text box descriptions were mandatory to be filled in the case if the Boolean answer to the question is marked with 'Yes'. Finally, one of the reviewers found it confusing which information should be ex- For all questions, if the answer to the question is (YES), it is necessary to write the description in text box 'What is it?'. For questions which have an answer only text box field, it is necessary to fill it in. Specific suggestions for filling the text box are given under each question.

Protocol

Q1

It can be any model for any phase of building the CPS. If the answer to this question is (NO) then the paper should be excluded from the classification so it is not necessary to proceed with the classification. Q1.1 The model/meta-model can be given for the approach or for the use case (or for both). Q1.2 We can find (1) tool/language which was built to support the approach OR/AND (2) tool(s)/language(s) which are used in the approach. It is necessary to report on both, categorizing them with (1)/(2). You can find examples in the document D1 -Section 2.3 and Section 2.4 [This is not a complete list]. If the used tool/language is already belonging to a list please indicate the section. Q1.3 Is modelling performed in a structured way and is that way described (preferably in a formal fashion, using, e.g. such as BPM, but it can be provided also in textual form (step by step manner)). Q2 E.g. using DEVS, Statecharts, Petri-nets, or other formalisms all together (Check Section 2.2 of D1). Q2.1 With supporting tools: editors, generators and/or transformations to artifacts (prone to be used for simulation platforms, or execution environments) Q2.2 Hardware, Network, Environment, Software Q2.3 This question is mandatory to be filled if the answer to Q2 is YES. Check the list of formalisms in D1, Section 2.2. If the used formalism(s) is(are) already belonging to a list please indicate the section. Q2.4 Is the integration achieved by using, for instance, mega modelling approach, models merge, links, other? Q3

In our case the approach is general if it is applicable to CPS in general, however sometimes the approach is built for a specific domain (e.g. for the smart grid, train transport etc.). Q3.1 The description of the domains is given by D2. Please see if the addressed domain fits in some or more offered in the list. If you select 'Other' it is necessary to specify how you would name this new domain category. It is possible that an approach can be applied to several domains. Not only it is possible to choose 3 from the drop-down menu, but also to suggest more in the text box. List

In the text box specify the domain by exact the wording used in a paper (e.g. smart grid) Q4

Are the people reported to be involved in the development of approach (analysis, design, implementation, evaluation...) for instance end user of approach, or the CPS system List

The provided categorization is general and it is possible to select several categories and suggest new ones by selecting the 'Other'. It is possible to choose 3 options from the drop-down menu, but also suggest more in the text box. Q4.1 Background knowledge topics should be based on the ontology presented in the CPS Profile study (D3) Q4.2 The technical background should be found for each author from their public profile (e.g. University/Institute official page, Google Scholar, Research Gate, LinkedIn) List

The provided categorization is general and it is possible to select several categories and suggest new ones by selecting the 'Other'. It is possible to choose 3 options from the drop-down menu, but also suggest more in the text box.

3 Performing the SLR

ReLis platform

An SLR follows a well-defined method to identify, analyze, synthesize, evaluate, and compare all available literature works relevant to a specific research topic [START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF]). The SLR process consists of several discrete activities, of which four are essential:

1. Defining research questions that the SLR is expecting to answer 2. Defining a search strategy (paper selection procedure, resources to be searched, and inclusion and exclusion criteria) 3. Conducting the search to identify the relevant research works 4. Analyzing, synthesizing, and comparing the related work Therefore, the SLR process requires a substantial amount of effort and usually requires several iterations in order to set the process up correctly. In order to support this process, various authors use different tools for this purpose, develop their own tools, or customize generalpurpose platforms (such as spreadsheets) for collecting papers and facilitating the reviewing process.

Motivation and Description of ReLiS

A recent study by [START_REF] Marshall | Tools to support systematic literature reviews in software engineering: A mapping study[END_REF] has provided insight into the current state of research regarding tools to support SLRs in software engineering. They selected 14 papers that describe tools to support SLRs. The results show an encouraging growth of tools to support the SLR process, and most of the tools identified are in the early stages of development and usage (at the time of that study).

The results from another study by [START_REF] Al-Zubidy | Vision for SLR tooling infrastructure: Prioritizing value-added requirements[END_REF] show that, while recent tools cover more of the requirements (activities in SLR), there are a number of high-priority requirements that are not yet fully covered by any of the existing tools.

ReLiS1 is one of the tools not covered in these two studies, perhaps because of its emergence later in 2017. We chose to use the ReLiS platform primarily because it supports the entire endto-end SLR process. It is the only tool that uses a Model-Driven Development approach to adopt a specific SLR. It features a domain-specific modelling editor tailored for researchers who perform SLR and an architecture that enables live installation and deployment of multiple concurrently running projects [START_REF] Bigendako | Automatically Installing and Deploying Tools for Conducting Systematic Reviews in ReLiS[END_REF]). ReLiS is a framework to automatically install systematic review projects on the cloud. Its generic and dynamic architecture allows users to install projects during the process without manual intervention [START_REF] Bigendako | Automatically Installing and Deploying Tools for Conducting Systematic Reviews in ReLiS[END_REF]).Another reason for our use of ReLiS is that it allows researchers to conduct SLR collaboratively.

The main page in ReLiS acts as a navigational hub and presents a user with the SLR-related tasks that this user can perform. A screenshot of the main page is presented in Figure 3.1.

In order to better understand the ReLiS platform and ReLiS process, it is recommended for users to try the platform as the 'demo user', and later create their profile.

The ReLiS platform contains three example projects (Model transformation, Model Transformations for Concrete Problems, and Template-based Code Generation). Before deciding whether to use this platform, a user can see how the process is supported and what they can expect as a result by starting these sample projects. Before starting to use the ReLiS platform for its own SLR, a new user needs to be registered.

ReLiS-supported process

A typical workflow to conduct SLR in ReLiS is described in Figure 3.3. Some of these actions are performed by the user (manually), while some are conducted automatically within the tool.

The user starts by planning the SLR and sets up a project in ReLiS. Setting up a project means defining a configuration model using the appropriate editor. According to the definition, ReLiS platform will automatically install the project on the cloud: a web application on the web server and a dedicated database. In this model, the user defines participants who will conduct the review. Different roles can be assigned (such as reviewers, validators, or project managers) to each participant. Researchers search for primary studies outside ReLiS, and later import them into ReLiS in the appropriate format (BibTeX, EndNote or CSV). ReLiS stores the meta information of each paper, such as title, abstract, author, venue, year and etc, and a link to the full text. As soon as papers are added into ReLiS, each paper can be assigned automatically and randomly to a fixed number of reviewers, after which the participants can start screening and decide which paper to include or exclude. One of the main benefits of using this system is that ReLiS automatically detects conflicting decisions between reviewers of the same paper, and notifies the reviewers that they need to resolve this conflict according to the strategy defined in the protocol.

The configuration model contains a section for quality assessment. Reading this configuration section, ReLiS generates forms for participants with the validator role to assess the quality of each paper, and automatically calculates the score of each study.

In addition to quality assessment, ReLiS generates the data extraction forms from the configuration model. Reviewers read through each paper and fill the online form to classify or highlight relevant information of each retained paper [START_REF] Bigendako | Automatically Installing and Deploying Tools for Conducting Systematic Reviews in ReLiS[END_REF]). Figure 3.3 shows an automatically generated data extraction form. All the extracted data can then be exported to a CSV file to be used in more advanced statistical tools, such as SPSS. ReLiS allows validators to check results of the screening, quality assessment or data extraction. Also, the tool tracks the progress and reports basic statistics for each phase, rendered as tables and plots. ReLiS supports an iterative process, meaning the configuration model can be modified at any time (new participants or new papers can be added, exclusion criteria can be modified and etc).

Roles and Responsibilities

This SLR involved 15 researchers, as listed in Table 3.1. The researchers include members of universities and research institutions and one from a company. The academic participants include PhD candidates, Post-Docs, Lecturers, Professors, and Research Engineers. The researchers participated from 10 different countries, including Portugal, Serbia, Sweden, France, Turkey, The Netherlands, The United States, Belgium, Latvia, and the Czech Republic. The researchers' table includes the abbreviations that will be used to refer to them for the sake of brevity.

Different activities related to various phases of the SLR are assigned to the researchers according to the Table 3.2. The activities cover all the tasks performed in the SLR related to defining the The data extraction, as one of the main activities, was assigned to all researchers involved in the study. Also, most of the participants were involved in the screening phase, another critical task of the SLR, where mainly the inclusion and exclusion of the papers were decided. Ankica Barisic and Dusan Savic, as principal researchers of this SLR, had the highest workload. The tasks are described as follows:

1. Develop Protocol: The SLR started with defining the overall research question and methodology, as described in Section 2. 2. Research Question Definition: The four main research questions, presented in Section 2.1.1 were defined to realize our main objective of finding approaches for developing CPS. [START_REF] Barišić | STSM Report: Systematic literature review on multi-paradigm modeling for CPS Systems[END_REF] was created as a part of the Short-Term Scientific Mission (STSM) awarded to Ankica Barisic in the context of MPM4CPS COST action. 12. Internal Review of technical report: Dusan Savic reviewed the preliminary report. 13. External Review of technical report: The external review of the technical report [START_REF] Barišić | STSM Report: Systematic literature review on multi-paradigm modeling for CPS Systems[END_REF]) was carried out by the STSM committee of MPM4CPS COST action in May 2018. 14. Revise protocol: Based on the comments provided during the review, the protocol was modified. 15. Identify primary research: In this phase, the primary studies were identified using the search quarry over scientific databases. 16. Retrieve primary research: Primary studies were imported to the Mendeley database group. 17. Setup ReLis project: The ReLis project was created and the reviewers were instructed how to create their accounts. We described the ReLis platform in Section 3.1. 18. Import automatic search studies: Primary studies retrieved using the search strings were imported into ReLis. The duplicates were identified and removed. The imported studies are presented in Section 3.3. 19. Screening: In this phase, the reviewers made a decision which of the studies obtained by search strings should be classified. This phase is described in Section 3.3. 20. Import from [START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF]

Screening phase

In this Section, we describe the process and result of the screening phase which was performed by using the Relis platform (Section 3.1). In this phase, the relevant primary studies obtained by the automatic search were selected by reviewers (see Table 3.2) for classification phase where the required data was extracted. The number of retrieved studies from several databases (ACM, IEEE, SD, SL, and SCOPUS) which have been used in this SLR is listed in Table 3.3. The publications are selected by automatic search methods, as explained in Section 2.1.2. The total number of retrieved studies by using automatic search was 211. SCOPUS with 103 studies has the highest and ACM with 4 studies has the lowest number of papers among the used databases. After removing duplicates, 188 primaries studies 4 were imported into the ReLis system on June 29, 2018. The reviewers were assigned screening tasks and instructed to log in to the ReLis platform in which they were expected to read the abstract of around 47 primary studies (for each reviewer) and decide if the study reports on modelling of CPS. Each study was assigned to two reviewers to make a decision regarding if the study should be included or excluded.

Figure 3.4 demonstrates the screening decision by the reviewers and the number of reviewed papers whether included or excluded. In case of conflict, the reviewers had a chance to communicate. If there was a conflict between the reviewers' decisions, the reviewers were notified to discuss the conflicting decision between themselves in such a way that at the final stage, all the conflicts were resolved and no level of conflict was reported. This is also illustrated in Fig. 3.5 which shows there were conflicts at the end. Reviewers were instructed to be more inclusive in this phase, as there was an option to make the exclusion of the study in the next phase.

Data Extraction phase

In this Section we describe how the data extraction process was perform, highlighting the classification preparation, assignments and the classification status.

Classification preparation

For the classification phase, we had two different sources of primary studies6 :

1 Studies which we obtained by automatic search using the search string presented in Section 2.1.2. These studies passed the screening phase and we obtained 92 papers to be reviewed during this phase 2 Studies which were reviewed in [START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF], and which were reported to present a modelling approach for CPS. We obtained 215 studies from this source After the screening phase was finished, we imported 215 studies from the 2nd source, and the ReLis system automatically identified 7 duplicates. There were 300 papers in total to be sent for review.

We created a private shared group in Mendeley library7 , named 'SLR MPM4CPS COST IC1404'. All bibliographic meta-data for primary studies were imported to this group and were set to have the same citation key as in the ReLis system. Reviewers were invited to join this group and were asked to upload the paper's .pdf to its reference.

In the ReLis system, the data extraction form from Section 2.1.5 was generated by using the integrated domain-specific language for specifying forms. The form consisted of the questions and descriptions from data extraction form, as well as from the quality assessment questions.

All the Boolean questions were mandatory to be answered by reviewers before they were able to submit their classification. After submitting the classification, reviewers were able to edit their answers if necessary. They could follow their progress, as well as progress of other reviewers and competition status in the Relis 'Data Extraction' dashboard. However, statistics were presented only for classified papers, meaning that if the paper was excluded in this phase, then it was removed from the overall summary.

Usually, the quality assessment phase is performed before the paper classification, and can serve as an additional criteria based on which the decision is made if the paper will be sent for classification. However, in our case, we decided to merge the quality assessment form to the classification form, because of limitations of the ReLis platform and lack of time to carry out this phase separately.

Classification assignments

Each reviewer was randomly assigned to classify a set of primary studies. Each study was expected to be classified by one reviewer. This phase started on October 22, 2018. The reviewers were provided with the following instructions: For 'Self Assessment' questions -it is necessary to express your own confidence about classification. Note that the papers for which you select '0-Not Confident' option, will need to be assigned to the other reviewer.

'
Finally, we have a Mendeley Group to which you all should receive an invitation to access (https://www.mendeley.com/community/slr-mpm4cps-cost-ic1404/). Although there is an online version, I highly recommend that you download the desktop application. Just remember to SYNC when starting or finishing working in the Mendeley desktop app.

You should find the bibliographic data which is extracted from the databases for the papers assigned to you in folder 'IncludedPapers'. It is necessary to download and attach the publication with data (ADD FILE option). Also, if you notice that some relevant information is missing or is wrong, please correct it. If you are not having the access to the paper which is assigned to you, please register it in sheet 'Missing Articles' of help document: https://goo.gl/4GNd5i For the papers which will be excluded as a secondary study, but they still report on relevant related work, please add the paper to 'RelatedWork' folder in Mendeley.

Here are small videos which show how to start with the classification in ReLis and using the Mendeley desktop app:

ReLis: https://drive.google.com/file/d/1KjJp-VGUG9zmhUC-Hj2pxcE8OiAb5WhX/ view?usp=sharing Mendeley: https://drive.google.com/file/d/13xL0U_A5DMMfHIPXwqob5OSOoj8H7cGc/ view?usp=sharing.

Thank you all for your effort and participation, and feel free to ask any questions if there are some doubts.'

Classification status

According to the ReLis system, as shown in Figure 3.8, 160 papers8 out of 300 have been classified and 81 papers are pending classification. It is worth noting that 59 papers9 have been excluded during the classification phase and they do not count by ReLis system as 'All' and are not presented in individual reviewers progress. The details of classification completion statistics for each reviewer is extracted from the ReLis system, as shown in Figure 3.9. According to this figure, most of the reviewers have finished the classification. It is worth noting that some reviewers did not reach 100% due to unavailability of the full text for some of their assignments (for instance from IGI Global publisher). On another hand, some of the reviewers didn't manage to complete their task due to the relatively short classification period. Altogether, 66% of the papers have been reviewed until now and will be analyzed in the following section. However, the idea is to continue with classifications until its completion later on.

Results

This section discusses the findings of this SLR. We begin with describing the Quality assessment of classified studies for the selected works. After that we present our findings for each research question.

After analysis of the obtained set of classified papers, we identified one duplicate study. Of the obtained 159 papers, 28 papers are registered as obtained by search string, while the rest came from [START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF]. However, as we had duplicate papers, this number is incomplete as the ReLis platform did not allow us to register various search strategies. For the same reason, we are not reporting the number of papers from different search databases.

Quality assessment of classified studies

To begin, Figure 4.1 displays descriptive statistics concerning quality assessment criteria QA1, which reports on the relevance of conference/journal where the paper was published. For this purpose we used the CORE ranking of the journals and conferences obtained from the following links: http://portal.core.edu.au/jnl-ranks and http://portal.core. edu.au/conf-ranks. The venues which reviewers did not find in this ranking list were marked as 'O-Not so relevant'. The highest number of papers belong to the papers that were published on conferences/journals that were not so relevant (46% of the papers). On the other hand, 31% of the papers are

Results

relevant and 23% are very relevant to the domain -indicating that we reviewed 54% of primary studies published in relevant venues, which are ranked as A or B.

In Figure 4.2 we present results related to the relevance of citations, respectively, following the scale defined in Section 2.1.4. For this purpose, we used the number of citations obtained from Google Scholar in November and December of 2018. According to the obtained results, over half of the papers (55%) belong to the group that has a high ranking factor. 33% of the papers belong to the group with medium ranking factor, while only 11% have low importance.

In total, we can conclude that around 55% of primary studies havea very good ranking -meaning that they were published in high ranked venues and are cited.

Furthermore, we highlight the quality assessment regarding the content of the papers, namely the clarity of motivation and research context for the approach presented in the papers, the completeness of the evaluation of the presented approaches, the explicitness of how contributions and future work were stated. We can see in Figure 4.4 that reviewers found that a negligible number of studies have focused on the research context vaguely (only 4 papers). Most convincingly, the largest number of the papers described research context with references (94 papers), meaning that the related work was reported, the lacks and advantages of related work were identified and compared with the proposed approach. Finally, 61 primary studies are classified to describe the research context generally. We judge the rigor of evaluation method used in the primary studies based on their evaluation types. We categorized the evaluation types into four groups: 1) Empirical foundation, 2) Case study, 3) Lessons learned and 4) No evaluation. Figure 4.5 shows the evaluation types used to validate the proposed solution. From the total of 159 papers, more than half primary studies (99 papers) proposed solutions that are evaluated by applying small case studies. 25 papers do not report any evaluation. Furthermore, 23 studies are evaluated empirically, while the remaining 21 papers show the applicability of the proposal using some illustrative examples. In order to make the modelling approaches better understood and accepted, more empirical studies need to be carried out. Finally, almost one quarter of the papers do not present any direction for future work (22%), which is almost the same as the number of papers that emphasized the future directions (21%).

In the remaining papers, more then a half (57%), issues for future work are presented in a generalized manner (Figure 4.7). We can conclude that most of the primary studies motivate their problems and provide the research context. About 80% of the approaches reported by primary studies are evaluated by a case study or empirical foundations. Although in most of cases the contributions are explicit, the future work is either very general or lacking and rarely reported with a concrete road-map. In order to make the review process more objective, the reviewers were asked to make a statement about their confidence about the content and quality of the study (see Figures 4.8 and 4.9). We used this information to decide whether to accept the reviewer opinion or to give the same paper to another reviewer, increasing the overall confidence. Confident response was dominant in both questions (58% and 66% respectively). A negligible number of the responses, approximately 1%, was Not Very Confident. This statistic is important because it can be concluded that the reviewers were appropriately selected. Regarding the modelling approaches for building CPS, the following is observed: most of primary studies report model or meta-model (83%), a large number of papers referred to tools or modelling languages (77%), while less than half of the papers presented the modelling process.

Results

Multi-paradigm modelling approaches

In this section, we present the findings for RQ2: Does the paper report a multi-paradigm modelling approach?

As shown in Figure 4.14, almost half of the papers (43%) support some multi-paradigm modelling approach, while the rest 57% support modelling of a particular part of the CPS (physical, network, communication, or software). Thus, a substantial part of CPS modelling research is involved in MPM.

The list below presents papers that reported a multi-paradigm modelling approach: 2016)). However, there is an approach which uses a language to describe how two models are integrated [START_REF] Sztipanovits | Composition and Compositionality in CPS, in Handbook of System Safety and Security[END_REF]).

Domain-specificity of modelling approaches

In this section, we present the findings for RQ3: Which application domains have been considered? Figure 4.15 shows that over one third of the papers (38%) presented approaches that are domain-specific, while a majority of the papers are domain-independent (62%) -meaning that they are applicable to CPS in general. The list below presents the papers that reported that the proposed modelling approach is domain-specific:

Application domains

According to the characterization of application domain to which the proposed approaches can be/ or were applied, Figure 4.16 shows that approximately the same number of papers address Critical Infrastructure, Intelligent Transportation, Smart Manufacturing and Building Automation domains. Most of the reviewed papers refer to some other domain that is not listed for the reviewers, demonstrating the diversity of CPS domains. We allowed the reviewers to characterize the domain using up to three choices, independently if the approach was reported as domain-specific or not. The reasoning behind this was that even for general approaches, the application is illustrated using one or more domains.

Profiles of stakeholders involved in modelling of the CPS

In this section, we present the findings for RQ4: What is the profile of the person which perform modelling of CPS?

Figure 4.17 shows a concerning fact that only 14% of the papers report the actors/stakeholders involved in modelling of the CPS. We conclude that research papers need to present the expected profiles of engineers or users for their CPS modelling methods.

The list below presents papers that described the actors/stakeholders involved in modelling of the CPS:

Results

Modeler's background knowledge

A similar situation to the above (perhaps even worse) is observed in terms of reporting the modeler's background knowledge. Figure 4.18 shows that a negligible number of papers (only 5%) report the modeler's background knowledge.

Technical background of the authors

During the review process, the reviewers were able to select several categories and suggest new ones by selecting the Other for the technical background of the authors. It was possible to choose 3 options from a drop down menu, but also to suggest more options in a text box. In total, 315 of considered 396 primary studies were reviewed by 15 researchers from different universities and companies. The SLR present the analysis based on a quantitative synthesis of data extracted from 160 primary studies and reports on

• modelling approaches for building CPS by indicating studies which report models and meta-models, tools and modelling languages and reported processes

• multi-paradigm modelling approaches by indicating studies which use a specific kind of modeling formalism

• domain-specificity of modelling approaches by indicating the studies which were applied in certain classification domain

• profiles of stakeholders involved in modelling.

Based on current results, we could observe that the most of the approaches report models or meta-models and modeling tools and languages which were used, however, less then half of them report on the process which is applied during modeling. Surprisingly, it is shown that almost 43% of approaches do report multi-paradigm modeling approach, however, only 23 studies present a complete model-driven development approach. Further, most of the approaches (62%) are proposed for the modeling of any CPS, while the rest were domain-specific. Finally, only 14% of studies report actors or stakeholders which were involved in modeling, and only 5% report on modelers background knowledge.

As a future work, we plan to complete the classification of the remaining 81 primary studies. Further, it will be necessary to perform the extensive descriptive synthesis of the results. We expect to obtain a complete list of the tools, modeling languages and formalisms which were used by the CPS modeling approaches. Next, we expect to obtain insight regarding which part of the CPS modeling tools and formalisms were used. Finally, we expect to obtain more sound classification of the application domain and profiles of stakeholders involved in modeling of the CPS. Section 5 of the survey, "Search Keywords and Queries", asked the participants about three potential queries for retrieving the primary studies. The queries and their results were summarized above, in Section 2.2.1. While all the queries in general were deemed to have somewhat The sixth section, "Inclusion and Exclusion Criteria", contained the question, "Are the inclusion/exclusion criteria complete enough to achieve the study objectives?" and asked for recommendations on improving the criteria ("Write recommendation to add/delete or improve inclusion or exclusion criteria"). The results indicated that the participants considered the criteria complete, as indicated in Figure A.11.

The section "Data Extraction Strategy" contains question "Is the data extraction Form complete enough to achieve SLR objectives?" and asks for recommendations on the improve data extraction strategy. According to the results that are presented on the Figure A.12, the participants found the strategy mostly complete, but in need of some improvement. The provided suggestions helped in making the strategy more comprehensive and reliable. The section eight is "Quality Assessment Strategy", asking the participants of whether the quality assessment criteria were complete.

•

  Safe concurrency and distributed computing • Real-time schedulability • Efficiency of information processing • Autonomous operation. • Physical and cyber security • Dynamic reorganization and adaptation • Interoperability of heterogeneous components and systems

Figure 1 . 1 :

 11 Figure 1.1: Review Process Overview based on[START_REF] Kitchenham | Guidelines for performing Systematic Literature Reviews in Software Engineering[END_REF]

Figure 2 . 1 :Figure 2 . 2 :

 2122 Figure 2.1: What is your overall expertise concerning the SLRs?

Figure 2 . 3 :Figure 2 . 4 :

 2324 Figure 2.3: The validation survey answers for importance and fit assessment of search queries

Figure 3 . 1 :

 31 Figure 3.1: The main page of ReLiS

Figure 3 . 2 :

 32 Figure 3.2: A diagram of the SLR process in ReLiS. Each box represents an SLR activity

Figure 3 . 3 :

 33 Figure 3.3: The data extraction form for this SLR in ReLiS

Activities

  AB DS VA MG AC DB FE BT IR CG MA MC RE ON RA questions and keywords validation: RQ validation was performed at MPM4CPS COST action meeting in Vienna 2 held on April 15-16 of 2016, where all participants discussed and finalized the direction of the SLR. 4. Query Testing: Based on predefined keywords, the researchers constructed several research quarries which were tested over search sources. This activity was described in detail in Section 2.2.1. 5. Define Search String: The queries were refined into several search strings, used to find the publications relevant to the SLR. 6. Define classification scheme: The evaluation criteria, inclusion and exclusion criteria, and quality criteria were defined in this phase and are given in Section 2.1.3 and 2.1.4. 7. Define data extraction form: The data extraction form was defined, including all the subquestions which can help in finding answers to the main research questions, as well as the classifications scales which can help in the characterization of collected data. The extraction form is described in Section 2.1.5. 8. Internal review of protocol: The protocol was evaluated by all members at the MPM4CPS COST action meeting in Riga 3 held on April 25-26 of 2018. 9. Prepare review questionnaire: A survey was prepared in order to have external participants evaluate the protocol. The details of the survey are described in Appendix A. 10. External review of protocol: All members of the MPM4CPS COST action were invited to participate in the survey on April 27 of 2018. The results of this review are presented in Section 2.2.2 11. Create Intermediate technical report: The intermediate technical report

  : Papers from related SLR were analyzed and then added to Mandalay Group, and imported into the ReLis platform. 21. Review of extraction form: The data extraction form was reviewed and validated by participants of the MPM4CPS COST action meeting in Izmir held from September 17-19 of 2018. 22. Pilot session: A preliminary pilot session with 3 participants, as described in Section 2.2.3, involved each researcher carrying out classification on two papers, and then discussing their uncertainties with the data extraction form. Those ambiguous questions were then clarified and revised. 23. Data extraction: All participants took part in classifying the studies and completing the quality assessment and data extraction forms, as reported in Section 3.4. 24. Data synthesis: The results of data extraction were analyzed and summarized, such as in the form of graphs and data tables, as we will show in Section 4. 25. Complete technical report: In this phase, the technical report was written. 26. Review technical report: Finally, the technical report was reviewed and submitted to the WG4 leaders of MPM4CPS COST action.

Figure 3 . 4 :

 34 Figure 3.4: Screening Decision

Figure 3 . 5 :

 35 Figure 3.5: Screening Conflicts

Figure 3

 3 Figure 3.6: Screening Exclusion

Figure

  Figure 3.7: Screening Result

Figure

  Figure 3.8: Classification Status

Figure 3 . 9 :

 39 Figure 3.9: Completion by reviewer

Figure 4 . 1 :Figure 4 . 2 :

 4142 Figure 4.1: Relevance of papers according to the conference/journal where they are published

Figure 4 . 3 :

 43 Figure 4.3: Clarity of the problem description

Figure 4 .

 4 Figure 4.3 shows the results of how clearly the problem of study is described. Almost two-thirds of the selected papers (68%) clearly and accurately describe the problem and 32% of the papers describe problem vaguely. This indicates that reviewers found that all primary studies present the motivation for their approach, and in most of cases this motivation is clear and specific to the problem which the study tries to address.

Figure 4 . 4 :

 44 Figure 4.4: Clarity of the research context

Figure 4 . 5 :

 45 Figure 4.5: Rigor of method evaluation

Figure 4 . 6 :

 46 Figure 4.6: Explicitness of presenting contributions

Figure 4 . 7 :

 47 Figure 4.7: How explicitly are the insights and issues for future work stated?

Figure 4 . 8 :

 48 Figure 4.8: Reviewer confidence about the study content

Figure 4 . 9 :

 49 Figure 4.9: Reviewer confidence about the study quality

Figure 4 .

 4 Figure 4.13: Reported processes

  [START_REF] Basile | A stochastic model-based approach to analyse reliable energy-saving rail road switch heating systems[END_REF];[START_REF] Zhang | Integration design and model transformation for cyber physical systems[END_REF];[START_REF] Fitzgerald | Co-modelling and co-simulation in the engineering of systems of cyber-physical systems[END_REF];[START_REF] Denil | DEVS for AUTOSAR-based system deployment modeling and simulation[END_REF];[START_REF] Herrera | Enhancing analysability and time predictability in UML/MARTE component-based application models[END_REF];[START_REF] Lickly | A practical ontology framework for static model analysis[END_REF] Vara Larsen et al. (2015);[START_REF] Magureanu | Validation of static properties in unified modeling language models for cyber physical systems[END_REF];[START_REF] Bloomfield | Preliminary interdependency analysis: An approach to support critical-infrastructure risk-assessment[END_REF];[START_REF] Cicirelli | Metamodeling of Smart Environments: from design to implementation[END_REF];[START_REF] Gonçalves | Managing CPS Complexity: Design Method for Unmanned Aerial Vehicles[END_REF];[START_REF] Hackenberg | MaCon: Consistent Cross-Disciplinary Conception of Manufacturing Systems[END_REF];[START_REF] Heinze | Application and Evaluation in the Healthcare Domain[END_REF];[START_REF] Bujorianu | A Modelling Framework for Cyber-Physical System Resilience[END_REF]; Mar-A wide spectrum of formalisms is used for modelling CPS, with the following common examples:• Discrete events: Fitzgerald et al. (2014); Denil et al. (2017); Vara Larsen et al. (2015); Sobottka et al. (2017); Ni and Broenink (2014). • Various UML and SysML diagrams: Herrera et al. (2015); Heinze et al. (2012); Tendeloo and Vangheluwe (2017); Zhang et al. (2017b); Hoffmann et al. (2017); Kacem et al. (2017). • Formalisms based on dataflows: Bliudze et al. (2017); Grüttner et al. (2017); Baumgartner et al. (2014); Herrera et al. (2015). • Petri nets: Carnevali et al. (2014); Marrone and Gentile (2016); Tendeloo and Vangheluwe (2017); Eusgeld et al. (2011); King et al. (2014). • Various types of mathematical equations: Hecht (2016); Van Der Auweraer et al. (2013); Bliudze et al. (2017); Grüttner et al. (2017); Sobottka et al. (2017); Carnevali et al. (2014). According to the integration mechanisms, the dominant approach used to integrate the model is semantic anchoring (such as Denil et al. (2017); Herrera et al. (2015); Vara Larsen et al. (2015); Hackenberg et al. (2016); Heinze et al. (2012)) as well as co-modelling and co-simulation (Carnevali et al. (2014); Ni and Broenink (2014); Fernández-Isabel and Fuentes-Fernández (

Figure 4 .

 4 Figure 4.15: Domain-specificity of modelling approaches

  Basile et al. (2016); Casella et al. (2017); Denil et al. (2017); Antal et al. (2017); Asensio et al. (2017); de Farias et al. (2017); Gonçalves et al. (2016); Hackenberg et al. (2016); Heinze et al. (2012); Marrone and Gentile (2016); Shuja et al. (2014); Zhang et al. (2017a); Carnevali et al. (2014); Ciccozzi et al. (2016); Premm and Kirn (2015); Murguzur et al. (2013); Calinescu (2013); Chiaradonna et al. (2012); Woodard and Sedigh (2013); Weissnegger et al. (2016); Alhafidh and Allen (2017); Nejati et al. (2012); Pournaras et al. (2013); Francalanza et al. (2017); Bernard and

Figure 4 . 16 :

 416 Figure 4.16: Reported application domains

Figure 4 . 17 :

 417 Figure 4.17: Reported actors or stakeholders involved in modelling of CPS

Figure 4 . 18 :

 418 Figure 4.18: Reported background knowledge of CPS modelers

Figure 4 .

 4 Figure 4.19 shows the technical background of the authors. Almost half of the authors (48%) are software engineers. The electrical engineers constitute 18.9%, while the authors with mechanical engineering and physics background are far smaller number. This leads us to conclude that the research on CPS is mostly pursued in the fields of computer science, software engineering, and electrical engineering.

Figure 4 . 19 :

 419 Figure 4.19: Reported technical background of the authors

Figure A. 1 :

 1 Figure A.1: Expertise of survey participants in SLRs

Figure A. 2 :

 2 Figure A.2: Responses regarding the SLR goal soundness

Figure A. 4 :

 4 Figure A.4: Responses regarding the SLR research questions covering the objectives

  7, A.8, A.9) show that Query Q1 had the least agreement between the respondents, while Query Q3 had the most agreement. In terms of fit, the participants indicated that Query Q2 was the best-fitting, as shown in Figure A.10.

Figure A. 7 :Figure A. 8 :Figure A. 9 :

 789 Figure A.7: Responses about keyword sufficiency of Query Q1

  Figure A.13 indicates that most participants agreed that the

Figure A. 14 :

 14 Figure A.14: Responses regarding willingness to contribute to this SLR

Table 2 .1: Inclusion criteria

 2 of CPS and which are reported from 2006 till the end of 2017. The Exclusion Criteria are presented in Table2.2. The studies that are not considered could be: part of informal literature, present duplicated work or its extension, or not written in English. The

	Id Criteria
	I1 Publication date from 1/1/2006 -31/12/2017
	I2 Relevance with respect to research questions
	I3 Explicit mentioning of modelling of cyber-physical system
	I4 Papers that report a methodology, metric or formalism for modelling of CPS
	I5 Analysis of relevant application domains for modelling of CPS
	To find the right publications, which answer our research questions, we define the Inclusion
	Criteria as presented in Table 2.1. The criteria are to include peer-reviewed articles, report-
	ing on modelling

Table 2 .2: Exclusion criteria

 2

Table 2 .3: Self Assessment Criteria

 2

Table 2 . 4

 24

: Quality Assessment Criteria (*published before 2014, **published 2014 and after)

Table 2 .

 2

5: Data Extraction Questions

Table 2 . 6 :

 26 Primary studies obtained using automatic search

	Database Q1	Q2	Q3
	ACM	44	4	200
	IEEE	42	0	48
	SD	80	8	25
	SL	294 25	39
	SCOPUS	66	97	33
	Total	526 134 345

Table 2 .7: Data Extraction Descriptions

 2

Table 3 . 1 :

 31 The reviewers of this SLR This table includes a heading column for activities and a heading row for the abbreviations of the researchers first and last name. The activities in which the researcher participated is marked with an 'X' in the table. Some of the activities have been realized in the context of MPM4CPS COST action meetings which are represented in the last column (C).

		Name	E-mail	Institution		Country
	AB	Ankica Barisic a.barisic@campus.fct.unl.pt University NOVA of Lis-	Portugal
				bon
	DS	Dusan Savic	dules@fon.bg.ac.rs	Faculty of Organiza-	Serbia
				tional Sciences
	VA	Vasco Amaral vma@fct.unl.pt	University NOVA of Lis-	Portugal
				bon
	MG Miguel	mgoul@fct.unl.pt	University NOVA of Lis-	Portugal
		Goulao		bon
	AC	Antonio Cic-	antonio.cicchetti@ mdh.se	Mälardalen University	Sweden
		chetti
	DB Dominique	dominique.blouin@	Telecom ParisTech	France
		Blouin	telecom-paristech.fr
	FE	Ferhat Erata	ferhat@computer.org	UNIT	Information	Turkey
				Technologies
	BT	Bedir Tekiner-	bedir06@gmail.com	Wageningen University Netherlands
		dogan
	IR	Ivan Ruchkin	iruchkin@cs.cmu.edu	Carnegie Mellon Uni-	US
				versity
	CG	Claudio	claudio.goncalvesgomes@	University of Antwerp	Belgium
		Gomes	uantwerpen.be
	MA Mustafa Ab-	mustafaxoodiye@gmail.com EGE University	Turkey
		shir
	MC Moharram	m.challenger@gmail.com	University of Antwerp	Belgium
		Challenger
	RE	Raheleh Es-	raheleh.eslampanah@	Izmir University of Eco-	Turkey
		lampanah	ieu.edu.tr	nomics
	ON Oksana Niki-	oksana.nikiforova@rtu.lv	Riga Technical Univer-	Latvia
		forova		sity
	RA	Rima Al-Ali	alali@d3s.mff.cuni.cz	Charles University in	Czech Re-
				Prague		public
	protocol, searching, screening, data extraction, validation, and reporting.

Table 3 .

 3

2: Roles and Responsibilities

Table 3 . 3 :

 33 Retrieved studies using automatic search

	Database Retrieved Studies
	ACM	4
	IEEE	44
	SD	25
	SL	35
	SCOPUS	103
	TOTAL	211

  In the ReLis system, you will find papers which are assigned to you when you select the 'Classify' action from your dashboard. Please note during the classification that we are extracting the information which is reported by authors -so we don't judge what paper claim. For example, if the paper report on modelling we don't judge if it is true or not but we have subquestions and textual notes to stress this. Regarding CPS, if the paper report to model just part of the system it should be also considered -we stress in sub-questions which part of system au-

thors model. For data extraction use the comments which we defined in 'Data Extraction Strategy' sheet of the official protocol (https://docs.google.com/spreadsheets/d/ 1X0B7G1RqNWvsNYILY6UPRPq34V8btbk-EVts-pVltgM/edit?usp=sharing). For all questions (Q1-Q4), if the answer to the question is (YES) it is necessary to write the description in text box 'What it is?'. For questions which have an answer only textbox field, it is necessary to fill it in. Specific suggestions for filling the text box are given under each question. Documents which should be consulted were provided in attachment: D1 (WG1 delivery: https://v2.overleaf.com/7464136733wzrcmhkcfnnd) -Dominique can update us if there is a newer version; D2 (Gunes, Volkan; Peter, Steffen; Givargis, Tony; Vahid, Frank: 'A Survey on Concepts, Applications, and Challenges in Cyber-Physical Systems); D3 (WG4 delivery: https://zenodo.org/record/223900#.W6T04maB1Zo) If you find that answer to our first question is NO, instead of 'Adding Classification' it is necessary to Exclude paper. While Excluding paper, it is mandatory to select the exclusion criteria and ADD justification as a Note. For 'Quality Assessment' questions (marked with QA) we need the following external information to be extracted: 1. Number of citations based on Google Scholar (https:// scholar.google.com) 2. CORE ranking of the journal (http://portal.core.edu. au/jnl-ranks/) OR conference (http://portal.core.edu.au/conf-ranks/). NOTE: if the are papers which you can not find in CORE, please select 0 and ADD paper to the sheet 'Missing Conference/Journal Ranking'https://goo.gl/4GNd5i

NSF Workshop on Cyber-Physical Systems, Austin, Texas, http://varma.ece.cmu.edu/cps/

http://mpm4cps.eu/workshops/16.04.

http://portal.core.edu.au/conf-ranks/

http://portal.core.edu.au/jnl-ranks/

https://scholar.google.pt/

http://mpm4cps.eu/workshops/18.04.

https://docs.google.com/spreadsheets/d/1X0B7G1RqNWvsNYILY6UPRPq34V8btbk-EVts-pVltgM

Does the query S1 have the sufficient keywords to achieve SLR objectives?

Does the query S2 have the sufficient keywords to achieve SLR objectives?

Does the query S3 have the sufficient keywords to achieve SLR objectives?

http://mpm4cps.eu

http://relis.iro.umontreal.ca

X Identify primary research X X Retrieve primary research X Setup ReLis project X Import automatic search studies X Screening X X X X X X X X Import from[START_REF] Barišić | Sustainability in Modelling of Cyber-Physical Systems: A Systematic Literature Review -Intermediate Technical Report[END_REF]) X Review of extraction form X Pilot session X X X Data extraction X X X X X X X X X X X X X X X Data synthesis X X X Complete technical report X X X X X Review technical report X X X X X X X X

http://mpm4cps.eu/workshops/16.04.

http://mpm4cps.eu/workshops/18.04.

Imported library: https://goo.gl/DmJMJs

Excluded papers: https://goo.gl/oggEd8

All papers: https://goo.gl/oA3Jx8

www.mendeley.com

Data Extraction: https://goo.gl/VXj33a

Excluded papers: https://goo.gl/aSbf6n

Acknowledgements

We would like to thank Prof. Paulo Carreira, Prof. Geylani Kardas, Prof. Ivan Lukovic and Prof. Mauro Iacono for their feedback during construction of protocol. We thank Dr. Letitia W. Li for her contribution with classification of primary studies and revision of report. Finally, we thank Prof. Eugene Syriani and Brice Bigendako for their constant support with the ReLis platform.

tracted from what source when they were collecting data on the technical background of the authors.

Once all the concerns were discussed, we updated the descriptions for each affected question. The final forms of question descriptions are presented in Table 2.7.

Modelling approaches for building CPS

In this section, we present the findings for our first research question RQ1: Which modelling approaches exist for building CPS?. The reviewers addressed this question by asking themselves if the primary study reports modelling approaches for building CPS. All the studies for which reviewers found that answer to this question was 'No' were excluded. In total 159 primary studies were classified to report modelling approaches for building a CPS. We can observe that the number of these reports is increasing from 2009 towards the year 2017, for which almost one third of the primary studies were published as shown in Figure 4.10. The selected studies are listed bellow:

A Protocol Validation Survey

This appendix describes the details of the protocol validation survey. A high-level overview is provided above, in Section 2.2.

A.1 Survey Solicitation

The following request to participate in the validation survey was sent to the MPM members of the COST action IC1404 (announcements@mpm4cps.eu) and the department of Computer Sciences and Languages of the University of Malaga (allmembers.mpm4cps@lcc.uma. B. Please answer the survey below.

Please take in consideration that you will be asked in the end to give an estimated time needed to fill this form.

If you have any doubt, feel free to contact me: Ankica Barisic (a.barisic@campus.fct.unl.pt) Dusan Savic (dules@fon.bg.ac.rs)

The survey form "Multi-paradigm modelling for CPSs SLR" contained 9 sections:

1

A.2 Survey Results

Here we describe the responses obtained from 10 participants on the survey. These results were used for protocol validation that was conducted during Working Group Meetings in Riga (26-27 April 2018). The responses are described in the order of sections.

Most questions about the SLR itself are phrased using the Likert 5-point scale: strongly disagree (1), disagree (2), neither agree or disagree (3), agree ( 4), strongly agree (5). Unless otherwise indicated, this interpretation of the horizontal scale applies to the figures in this section.

The first section contains general information and question about participants expertise concerning the SLR. To summarize, the survey indicated that the goals of the SLR were well-motivated, and most of the protocol elements were considered appropriate for those goals. Multiple recommendations of the survey participants helped further improve the SLR protocol.