Catherine Thomas

Sebastian Engell

Chapter 2. Structuring inter-organizational R&D projects: Towards a better understanding of the project architecture as an interplay between activity coordination and knowledge integration

Keywords: projet d'innovation collaboratif, multi-acteur, en amont du marché, architecture, taille, processus, intégration des connaissances, nouvelle technologie, maturité, performance, programmecadre de l'Union research and innovation project, multi-actor, market upstream, architecture, size, process, knowledge integration, innovation output, new technology, maturity, performance, EU Framework Inter-organizational R&D project, Multi-actor, Architecture, Structure, Modularity, Activity coordination, Knowledge integration, Collaboration, Project management multi-actor RDI project, market upstream, size, scale, innovation output, research output, innovation performance, meta-knowledge

Comment améliorer la performance des projets d'innovation collaboratifs : le rôle de l'architecture, de la taille et des processus de collaboration

Résumé :

Des milliards d'euros sont dépensés chaque année par les agences publiques pour financer des projets collaboratifs de recherche et d'innovation avec de multiples acteurs industriels et académiques, générant conjointement de nouvelles connaissances scientifiques et développant des innovations technologiques en amont du marché. Les impacts économiques et sociétaux de ces projets sont largement influencés par la qualité et la quantité des résultats du projet. Lors de la préparation de la proposition de collaboration, l'objectif principal est d'obtenir un financement ; à ce stade de nombreux éléments ne sont pas encore clairs. Compte tenu de l'importance des financements publics investis dans ce type de projets, une meilleure compréhension de leur fonctionnement et des facteurs de succès est nécessaire. Cependant, il manque d'études qualitatives, en profondeur, notamment en raison de la difficulté d'accès au terrain, permettant de fournir des éléments de compréhension sur la mise en oeuvre de ces projets, les résultats produits et les raisons qui expliquent ces résultats.

Ce travail de thèse étudie les projets de recherche et d'innovation (RDI) multi-acteurs, avec pour principale question de recherche : Comment l'architecture, la taille et les processus de collaboration des projets de RDI multi-acteurs influencent les innovations technologiques et la performance globale des projets en matière d'innovation ? Cette question principale se décline en quatre problématiques de recherche: (1) le rôle de l'architecture des projets et ses implications sur la collaboration et donc sur la performance du projet; (2) le rôle des processus de collaboration, leur évolution et les implications sur les innovations technologiques; (3) le rôle de la taille des projets, en termes de nombre de partenaires, sur la performance du projet; (4) une évaluation directe et objective de la performance des projets d'innovation, qui va au-delà des indicateurs 'classiques' des brevets et comptage des technologies innovantes à différents stades de leur développement.

Le manuscrit de la thèse comprend quatre études empiriques, abordant ces problématiques de recherche. Les quatre études sont intitulées comme suit : (1) Structurer des projets de R&D interorganisationnels : vers une meilleure compréhension de l'architecture du projet en termes d'interaction entre la coordination des activités et l'intégration des connaissances ; (2) L'évolution des liens collaboratifs dans les projets multi-acteurs et leur influence sur les outputs ; (3) Est-ce qu'accroitre le nombre de partenaires est toujours positif ? Un examen plus approfondi de la taille et des performances d'innovation des projets de R&D multi-acteurs : preuves empiriques des programmes européens de recherche et d'innovation ; (4) Vers une meilleure compréhension des outputs innovants et de la performance des projets d'innovation collaboratifs.

Les différentes recherches menées sont conduites à partir d'études de cas multiples (de 4 à 6 cas) exploratoires de nature inductive, complétées dans certaines recherches par des données quantitatives. Les données collectées provenaient de deux sources principales: plus de 1600 pages de documents et 54 entretiens semi-structurés avec les acteurs des projets.

Les résultats visent à enrichir notre compréhension des projets multi-acteurs d'innovation collaboratifs, en étudiant notamment le rôle de l'architecture, de la taille et des processus de collaboration sur la performance des projets. La recherche réalisée propose également une méthodologie d'évaluation de la performance de ce type de projets qui complète celles existantes aujourd'hui. Les recherches menées produisent des contributions théoriques dans les champs du management de projet et du management de l'innovation ; elles fournissent également des contributions managériales et des implications pour les politiques publiques.

 Table 1. [START_REF] Chesbrough | Open Innovation: The New Imperative for Creating and Profiting from Technology[END_REF], and more project-organised [START_REF] Tiwana | Do bridging ties complement strong ties? An empirical examination of alliance ambidexterity[END_REF][START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF].

Collaboration influences the potential for knowledge creation, thanks to the variety of knowledge bases and combination possibilities [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF], and provides resource-sharing benefits, through access to knowledge, skills, equipment and other resources of partners, as well as knowledge spillover benefits, through access to information [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF]. Transfer and sharing of knowledge and other assets are especially beneficial when combination, and resource sharing, takes place between heterogeneous partners such as academia and industry [START_REF] Schwartz | Measuring the effectiveness of R&D[END_REF], Belderbos et al., 2004).

It is thus not surprising that public agencies allocate an important, if not the predominant part of R&D funding nowadays to collaborative inter-organisational research, development and innovation (RDI) projects with multiple heterogeneous actors, such as research organisations, technology providers, industrial end users. As an example, the latest Horizon 2020 Framework Programme (FP) has already provided 34,42B€ in 2014-2020 to fund 6288 such multi-actor projects involving 88,694 participants (organisations) all over the world1 (EC, 2021a). These projects are a specific form of open innovation [START_REF] Enkel | Open R&D and open innovation: exploring the phenomenon[END_REF]; they target upstream of the market R&D with the goal of jointly generating knowledge outputs and innovation outputs, often as part of the same project [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF].

These projects address problems which are often ambiguous, uncertain and complex at the same time [START_REF] König | A framework for structuring interdisciplinary research management[END_REF]. Multi-actor publicly funded inter-organisational RDI projects are a propitious setting for the generation of new knowledge and innovations, thanks to the considerable number of actors, diversity of the knowledge base of the actors and the possibilities for knowledge combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]. On the other hand, these projects are difficult to manage [START_REF] Söderlund | Pluralistic and processual understanding of projects and project organizing; towards theories of project temporality[END_REF]Tell, 2011, Lin et al., 2019), due to the involvement of many heterogeneous actors pursuing both shared and own objectives [START_REF] Mannak | A temporal perspective on repeated ties across university-industry R&D consortia[END_REF].

The setting and collaboration processes in this type of projects are not well understood yet [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]. Qualitative studies investigating multi-actor RDI projects are lacking, as the access to the field is difficult [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]: project documents are mostly confidential, consultation of internal project documents or participation in the project meetings is possible only with unanimous authorisation from all project partners because of confidentiality reasons and decentralised decisions making process fixed in the consortium agreement. In addition, as the project work is reviewed by the funding agency, the partners of on-going projects are reluctant to share their concerns that shed light on the collaboration processes other than in face to face interviews with someone they know and trust, with anonymity guaranteed. It is not surprising that with such difficulties to access the field, it has until now remained outside the focus of most of the management and innovation literature. To the best of our knowledge, there is no literature that analyses different multiple multi-actor projects in a multiple case study. However, such qualitative research is needed [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF], to provide 'why' and 'how' explanations and for better understanding of the empirical context. Scholars repeatedly call for research exploring setting, collaboration processes and collective innovation performance of multi-actor projects (e.g. vom [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF][START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF][START_REF] Nepelski | The organisational and geographic diversity and innovation potential of EU-funded research networks[END_REF].

There are several main reasons that make this setting of multi-actor RDI projects particularly interesting for both project management and innovation research: these elements provided motivation for the presented research. First, because the field is large and important: such projects have become an important instrument of RDI funding, and their number is steadily growing. Second, because a while ago, results of such projects were relatively far from the market: projects mostly aimed at generation of research results [START_REF] König | A framework for structuring interdisciplinary research management[END_REF], but nowadays, the focus of programmes is often shifting towards supporting closer-to-market research and developments. This is the case of Horizon 2020, but also of the Horizon Europe Framework Programme, launched in spring 2021: the part of the Programme that will support industrial and closer-to-market research and development was allocated 56% of €95.5 billion, the entire budget of the Programme (Science / Business, 2021). Third, because the current trend in this domain has been towards larger projects, both in total budget and number of partners: as an example, the size of RDI projects funded by European FPs has been more than doubled over the two last decades, from an average of 6.1 per project in FP5 (EC, 2009) to 14.1 in Horizon 2020 [START_REF] Ec | Horizon 2020 Interactive Dashboard[END_REF]. Fourth, because the diversity in such projects is high: multiple partners coming from heterogeneous organisations, such as academia, large firms, technological SMEs, consultancy firms etc., which amplifies the challenges for project management. Fifth, the projects often lack structural flexibility, as the partners and budgets are fixed in the grant agreement, it is difficult to change partners or to drastically revise the workplan: thus, the consequences of the project set up mostly persist until the project end, but on the other side, it is hardly possible to organise well so many organisational actors (often not well known initially) during the proposal phase: so it should be expected that an evolution of the couplings between the organisational actors will happen, but it is not clear when, how, and why this takes place. Also, there is another problem: when studying the performance of RDI projects, scholars usually put the focus on patents and publications, as pointed out e.g. by [START_REF] Hung | Social networks, technology ties, and gatekeeper functionality: Implications for the performance management of R&D projects[END_REF], but scholars also highlighted repeatedly (e.g. Janger et al., 2017, Goldstein and[START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF] that the performance of these projects is multi-faceted and is not adequately reflected by the number of publications and patents (often called knowledge outputs, or research outputs); outputs on the 'innovation side' (new technologies, methods, processes, tools…) shall also be assessed. However, it is not well known how to measure innovation outputs directly and objectively at the end of market-upstream projects, as the innovations have not yet been implemented at the end of the project, the technologies are very specific, and assessment indicators at the development and validation stage of technologies are not discussed much in the literature [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF]. But it is obvious from the sheer size of the public programmes that there is an urgent need to systematically understand and evaluate not only the 'research side', but also, most importantly, the 'innovation side' of the projects, to demonstrate the value for money for the taxpayer. To sum up, from one side, a better understanding is needed about the project setting (collaborative links, size of the consortium), the processes (evolution of the collaborative links), their implications on the project performance, and about the overall innovation performance of the projects.

The remainder of the introduction is organised as follows. Firstly, brief explanation of main definitions used throughout the thesis is provided. Then the theoretical framework of this research is briefly presented; it builds upon the literature on project management and also on innovation research. Then the research question is formulated, the research setting and the research design of the study are explained, followed by the summary of the empirical studies and their contributions.

Key definitions

Several key definitions are used throughout the thesis manuscript. They are explained in each study, and briefly introduced below.

Innovation, since the seminal work of [START_REF] Schumpeter | The Theory of Economic Development[END_REF], has been often understood as the novel recombination of ideas, resources and knowledge for value creation [START_REF] Nonaka | A Dynamic Theory of Organizational Knowledge Creation[END_REF][START_REF] Grant | Towards a knowledge-based theory of the firm[END_REF][START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF]. More recently, innovation has been defined as "a new or improved product or process (or combination thereof) that differs significantly from the unit's previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)" (OECD Oslo Manual, 2018, p. 20).

Project is temporary organisation [START_REF] Lundin | A theory of the temporary organization[END_REF] set up with the goal "to undertake a unique, novel and transient endeavour managing the inherent uncertainty and need for integration in order to deliver beneficial objectives of change" (Turner and Müller, 2003, p. 7). A project combines different elements: actors, groups of collaborating actors, activities, knowledge…, organised into components, which in turn may form modules, i.e. a set of elements or components that are organised in a thread of work for a common purpose, to produce specific outputs. Modularity has been defined as "building a complex product or process from smaller subsystems that can be designed independently yet functions together as a whole" (Baldwin and Clark, 1997, p.86); in the project context, modularity is a way of dividing the project into elements that are connected either by a flow of activities or by a flow of knowledge, or both; modules are characterised by a strong integration of the activities and of the knowledge of the participating partners. Integrative modules, also called interfaces, receive input from two or more other modules, and/or provide input to other modules, and produce own output(s).

The project structure defines the way how the tasks are decomposed and coordinated [START_REF] Mintzberg | The structuring of organizations[END_REF]. In the presence of multiple actors, the structure of interdependencies between actors in an organisational setting [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF], or the way in which the collaboration is arranged [START_REF] Fjeldstad | The Architecture of Collaboration[END_REF], is often called architecture. A tie, or coupling, defines a connection between the elements or groups of elements of the project. The strength of a coupling, weak or strong, defines the degree of inter-dependence between the elements; thus, architecture of the project is defined in terms of strength and intensity of links, or couplings, between the nodes in the network, or actors [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF]. The resilience of a project architecture describes to which extent the project is affected if an element in the project is encountering problems, e.g. not performed or not performed satisfactorily.

A project output is an element of the finality of the project; when innovation outputs are in focus, they were defined as the technological innovations (innovative technologies, methods, procedures, software, deployments, products…) which (1) were generated in the project, (2) have been proven during the project, i.e. for which there is evidence of successful validation, demonstration or application of the technology; (3) were identifiable and (potentially) reportable, e.g. to the funding agency; this definition was adopted to show that non-proven project outputs, such as innovative ideas and other exploratory outputs [START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF] are not included in the study. Innovations outputs may be of different maturity, often assessed using Technology Readiness Level (TRL) scale [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF], showing how ready (mature) technology is on its way towards the final use.

Value is the worth, or usefulness of something in terms of importance; the value of an innovation can be realised and assessed only after its implementation [START_REF] Gault | Defining and measuring innovation in all sectors of the economy[END_REF]. In the project context, value definition embraces benefits, or satisfaction of needs, and costs, or use of resources [START_REF] Laursen | Taking stock of project value creation: A structured literature review with future directions for research and practice[END_REF], and it is closely interconnected with the concept of performance of the project, or project success [START_REF] Green | Value creation in projects: Towards a narrative perspective[END_REF].

Management of collaborative RDI projects 1.3.1 Generation of new knowledge and innovation outputs in interorganisational research and innovation projects

In the research context, new knowledge has been defined as "knowledge that expands existing knowledge, is provable, and is beneficial for a target group" (Mårtensson et al., 2016, p. 598); "provable" means that there is demonstratable evidence about the new knowledge.

Knowledge generation is a vital part of innovation processes [START_REF] Boisot | Integrating modernist and postmodernist perspectives on organizations: A complexity science bridge[END_REF], which involve "the exploration and exploitation of opportunities for new and improved products, systems, processes or services through the generation of knowledge, the transformation of knowledge into artefacts, and the continuous matching of these artefacts to market needs and demands" [START_REF] Pavitt | Innovation processes[END_REF], in [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF]). Knowledge generation often requires exchange and combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF] between the elements of the knowledge base, which in turn depends on a number of factors, including the opportunity for exchange or combination, the anticipation of valuable collaborative outcome, the motivation to be engaged, the combinatory capability of the participants [START_REF] Moran | Theories of Economic Organization: The Case for Realism and Balance[END_REF]. The size and the structure of the knowledge base affects the organisational ability and the potential to create new knowledge (Ahuja, 2000, Yayavaram and[START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF], and organisations increasingly engage into interorganisational projects [START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF][START_REF] Manning | The rise of project network organizations: Building core teams and flexible partner pools for interorganizational projects[END_REF][START_REF] Stjerne | Crossing times: Temporal boundaryspanning practices in interorganizational projects[END_REF]; through such projects, heterogeneous partners coming from different backgrounds jointly create value while reaching their individual objectives (Jones and Lichtenstein, 2009, Dille and Söderlund, 2011, Bakker et al., 2011).

The process of knowledge generation has been undergoing changes over the last decades, and research and innovation have become more open and more collaborative [START_REF] Chesbrough | Open Innovation: The New Imperative for Creating and Profiting from Technology[END_REF][START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF], and more project-organised (Gerwin andFerris, 2004, Tiwana, 2008), as the society has taken the projectification path [START_REF] Midler | Projectification of the Firm: the Renault Case[END_REF][START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF][START_REF] Simard | The utopia of order versus chaos: A conceptual framework for governance, organizational design and governmentality in projects[END_REF]. Collaborative research, development and innovation (RDI) projects bring together actors, often both from academia and from industry, who integrate and apply complementary knowledge and contribute towards creating new knowledge (Okhuysen and Eisenhardt, 2002[START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF][START_REF] Manning | The rise of project network organizations: Building core teams and flexible partner pools for interorganizational projects[END_REF]. During the recent years, RDI projects have become bigger and more complex [START_REF] Steen | Projects and Networks: Understanding Resource Flows and Governance of Temporary Organizations with Quantitative and Qualitative Research Methods[END_REF], they include multiple actors (e.g. [START_REF] Pandza | Collaborative diversity in a nanotechnology innovation system: Evidence from the EU Framework Programme[END_REF][START_REF] Pinheiro | The role of social capital towards resource sharing in collaborative R&D projects: Evidences from the 7th Framework Programme[END_REF]. Publicly funded multi-actor projects have characteristic properties which are different from other inter-organisational projects [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]Lippe, 2015, Bor, 2014) in the following aspects: autonomy and equality of the organisational actors, bottom-up self-organisation in response to competitive calls, considerable numbers of heterogeneous actors, collective responsibilities, high complexity of connections between actors (Fig. 1.1), and lack of structural project flexibility.

Multi-actor RDI projects offer a propitious setting for the generation of new knowledge [START_REF] Johnson | Assessing organizational knowledge creation theory in collaborative R&D projects[END_REF], thanks to the diversity of the knowledge base of the actors and the possibilities for knowledge combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF], but they encounter also many barriers and experience three paradoxes: (1) requirement for freedom due to the research uncertainties versus requirements for structure; (2) requirement for integration of research views and ideas versus heterogeneity of partners and inter-disciplinarily challenges;

and (3) limited management authority versus requirement for commitment of all partners [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]. Collaboration, or joint work to achieve a common purpose, significantly increases the potential for new knowledge generation, thanks to the breadth of the knowledge base [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF], in RDI projects, new knowledge takes form of knowledge outputs and innovation outputs. The quality of collaboration in projects determines the collaborative outputs (e.g. [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. Collaboration does not come by itself, is difficult to organise and manage [START_REF] Swink | Building Collaborative Innovation Capability[END_REF], and it is influenced by several factors [START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF], e.g. the historic experience of collaboration [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF][START_REF] Belderbos | The antecedents of new R&D collaborations with different partner types: On the dynamics of past R&D collaboration and innovative performance[END_REF] or the dynamic project context: "no project is an island" (Engwall, 2003, p.789). Universal collaboration antecedents include mutual trust, commitment, good personal relationships [START_REF] Littler | Factors Affecting the Process of Collaborative Product Development: A Study of UK Manufacturers of Information and Communications Technology Products[END_REF], strong research and innovation capabilities, complementary competences and technologies [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF], and collaboration capabilities [START_REF] Blomqvist | Collaboration Capability -A Focal Concept in Knowledge Creation and Collaborative Innovation in Networks[END_REF].

Scholars also concur that the project structure significantly affects the success of the collaboration [START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF][START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. However, the existing body of literature does not provide answers on how collaborative projects shall be structured to favour knowledge creation in the context of uncertainty, complexity and involvement of multiple autonomous actors.

The role of architecture in multi-actor projects

The structure of a project defines the way how the tasks are decomposed and coordinated [START_REF] Mintzberg | The structuring of organizations[END_REF]. Multi-actor RDI projects usually last several years and thus require a structure [START_REF] Raab | Structure in temporary organizations[END_REF] which provides the framework for collaboration and knowledge integration [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF][START_REF] Lin | Choosing suitable project control modes to improve the knowledge integration under different uncertainties[END_REF] and influences the creation of new knowledge (Schilling and Phelps, 2007). Scholars also concur that the project structure significantly affects the success of the collaboration [START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF][START_REF] Caniëls | Collaboration in project teams: The role of mastery and performance climates[END_REF][START_REF] Rauniar | Knowledge integration in IPPD project: role of shared project mission, mutual trust, and mutual influence[END_REF].

Organisational design explains "what should be the design, structure, or architecture of the organisation" (Burton andObel, 2011, p. 1198). Management scholars almost synonymously use the terms "structure", "design", or "architecture" (Aubry and [START_REF] Lavoie-Tremblay | Rethinking organizational design for managing multiple projects[END_REF] when studying the division and the coordination of activities, but they rarely address the organisational design of the projects, taking it "for granted" (Aubry and [START_REF] Lavoie-Tremblay | Rethinking organizational design for managing multiple projects[END_REF], although a project is also an organisation, a temporary one [START_REF] Lundin | A theory of the temporary organization[END_REF]. Some literature investigated elements of the organisational design and pointed out that structure shapes and stabilises patterns of interactions between participants in projects and reduces the uncertainty for the involved actors (Raab et al., 2009, Söderlund andSydow, 2019), supports governance mechanisms (Van de Ven et al., 2013[START_REF] Miterev | Towards a design for the project-based organization[END_REF], helps to organise the integration of outcomes of different activities [START_REF] Srikanth | Integrating distributed work: comparing task design, communication, and tacit coordination mechanisms[END_REF]Puranam, 2011, Zerjav et al., 2018), influences organisational efficiency, performance and resilience (e.g. [START_REF] Englmaier | Organization Design and Firm Heterogeneity: Towards an Integrated Research Agenda for Strategy[END_REF][START_REF] Burton | The science of organizational design: fit between structure and coordination[END_REF][START_REF] Yi | Differentiation and Integration in Organizational Learning: A Garbage Can Model[END_REF]. It is also known that too much structure may impede the performance of the organisational setting of the project [START_REF] Davis | Optimal Structure, Market Dynamism, and the Strategy of Simple Rules[END_REF], but too little structure may prevent the realisation of the project according to the plan.

In the presence of multiple actors, the way in which the collaboration is arranged [START_REF] Fjeldstad | The Architecture of Collaboration[END_REF], or the structure of interdependencies between actors in an organisational setting [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF], is often called architecture, defined in terms of strength and intensity of links, also called ties or couplings, between the nodes in the network, or actors [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF]. Couplings between the actors are often studied using social network theory, which suggests that the structure and the strengths of the relationships or couplings between the actors in the network explain differences in outcomes for these actors [START_REF] Granovetter | The Strength of Weak Ties[END_REF][START_REF] Coleman | Social capital in the creation of human capital[END_REF][START_REF] Borgatti | Network Analysis in the Social Sciences[END_REF]. Couplings can have different strengths and be present in a variety of patterns which have been studied in the project literature, especially from the angle of processes and structures of activities (how activities are decomposed and integrated:

process architecture framework, process flow diagrams, workflow, e.g., [START_REF] Browning | Managing complex project process models with a process architecture framework[END_REF][START_REF] Caverlee | Workflow management for enterprise transformation[END_REF][START_REF] Shi | A new approach for project scheduling using fuzzy dependency structure matrix[END_REF][START_REF] Bakker | Organizational Effectiveness: Collaboration in an Integrated Project Team[END_REF], and in the innovation and network literature, especially from the angle of the knowledge structure (how the knowledge base is decomposed and integrated, influence of the strengths of ties, e.g. [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF].

Projects requires structuring in order to divide and coordinate the activities, stabilise patterns of social interaction, achieve better collaboration [START_REF] Raab | Structure in temporary organizations[END_REF], and organise knowledge integration [START_REF] Lin | Choosing suitable project control modes to improve the knowledge integration under different uncertainties[END_REF]. In multi-actor RDI projects, both a knowledge structure and a structure of activities are present, and the literature points out that the project actors may depend on each other in terms of workflow (activities) or in terms of knowledge flow [START_REF] Clement | Searching for Structure: Formal Organization Design as a Guide to Network Evolution[END_REF]. In other words, not only workflows but also knowledge flows shall be structured to facilitate collaboration and improve performance of the organisations [START_REF] Rauniar | Knowledge integration in IPPD project: role of shared project mission, mutual trust, and mutual influence[END_REF]. Abundant literature on inter-organisational projects, ecosystems and networks mostly focuses on inter-firm collaboration, in many cases with one focal actor, but how to structure multi-actor publicly funded RDI projects is however not well understood [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF], but very important, as the consequences of structuring usually persist until the project end. There is a call for research to measure the impact of specific influencing factors, such as team structuring, on collaboration for knowledge creation [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF], and a call for research to understand the underlying processes and explain the settings and processes of collaborative research and innovation projects [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF].

The role of collaboration processes and their effect on project performance

The architecture of a project, as described above, reflects only one side of the organisational design: in addition, a dynamic layer of processes is required [START_REF] Lundin | A theory of the temporary organization[END_REF], and both, the architecture ("the thing") and the processes, should be analysed [START_REF] Lavoie-Tremblay | Rethinking organizational design for managing multiple projects[END_REF]. This dynamic layer of processes is inherently related to the dynamics of inter-organisational collaborations (e.g. [START_REF] Majchrzak | A Review of Interorganizational Collaboration Dynamics[END_REF]. It is thus not surprising that dynamics of collaboration has been usually analysed using a process view (e.g., [START_REF] Berends | Introduction: Process views on inter-organizational collaborations[END_REF], investigating how "phenomena emerge, change, and unfold over time", Langley et al., 2013, p. 1).

Extant literature investigated triggers, attributes, conditions of and barriers to interorganisational collaborations at different stages, as well as collaborative outputs and outcomes (see e.g. [START_REF] Salvato | Cooperation across disciplines: A multilevel perspective on cooperative behavior in governing interfirm relations[END_REF][START_REF] Gulati | Meta-organization design: rethinking design in interorganizational and community contexts[END_REF] and studied different types of settings and processes, often taking place sequentially. For example, the birth of business ecosystems has been explained as a collaborative process going from ideation to experimentation, value expansion and reflexion [START_REF] Attour | Naissance des écosystèmes d'affaires: une articulation des compétences intra et inter organisationnelles[END_REF]; collaborative relationships in the inter-organisational projects are often initiated, implemented, adapted, locked-in, maintained, assessed, and ended (e.g. [START_REF] Berends | Introduction: Process views on inter-organizational collaborations[END_REF][START_REF] Sydow | Organizational Path Dependence: Opening the Black Box[END_REF][START_REF] Ligthart | In the Shadows of Time: A Case Study of Flexibility Behaviors in an Interorganizational Project[END_REF]:

this points out to the dynamics of relationships, which could evolve based on contextual conditions, e.g. changed strategic needs of the collaborating organisations, evolving from collaboration to coopetition and competition [START_REF] Majchrzak | A Review of Interorganizational Collaboration Dynamics[END_REF]. Evolution of collaboration may be also influenced by the evolving degree of complexity of relationships between the actors. [START_REF] Wang | Complexity and the functions of the firm: Breadth and depth[END_REF] explained two types of complexity, relational and cognitive, and their 'depth' and 'breadth' dimensions: the former concerns the level of sophistication, while the latter concerns the level of heterogeneity. Both are present in inter-organisational settings and may experience evolution over time, but it is not well understood how these two dimensions of complexity manifest and evolve in multi-actor RDI projects. Scholars therefore called for research to analyse processual features of interorganisational collaborations [START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF].

The topic of relational dynamics between collaborating organisations, or relational embeddedness of interactions, attracted a lot of interest of process researchers [START_REF] Berends | Introduction: Process views on inter-organizational collaborations[END_REF]; often the perspective of the collaborative actor(s) has been used when studying the process and changes (e.g. [START_REF] Escher | Inter-Organizational Collaboration on Projects Supporting Sustainable Development Goals: The Company Perspective[END_REF]. [START_REF] Majchrzak | A Review of Interorganizational Collaboration Dynamics[END_REF] reviewed over 20 longitudinal studies addressing the dynamics of inter-organisational collaboration, analysing causes that led to changes, the dynamics itself, and its implications on the collaborative outcomes. They found that instability, or flexibility, is a distinguishing inherent feature of successful inter-organisational collaboration. In the project setting, literature suggests that flexibility is enabled by trust between project participants, as related to the past experience or expectations for the future, the pressure factor, due to the imminent project end, and the availability of resources [START_REF] Ligthart | In the Shadows of Time: A Case Study of Flexibility Behaviors in an Interorganizational Project[END_REF].

When studying collaboration and its dynamics, the key theoretical concept is collaborative links, or ties, or couplings. The concept of couplings has been extensively used in prior interorganisational studies, but two different conversations -the project literature and the innovation literatureaddressed them in different ways, as described in Chapter 1.3.2. The project literature often studies couplings from the angle of the flow of activities (workflow, e.g., [START_REF] Browning | Managing complex project process models with a process architecture framework[END_REF][START_REF] Caverlee | Workflow management for enterprise transformation[END_REF][START_REF] Shi | A new approach for project scheduling using fuzzy dependency structure matrix[END_REF][START_REF] Bakker | Organizational Effectiveness: Collaboration in an Integrated Project Team[END_REF], or focusing on the relationships within a specific type of coupling. The innovation literature focuses on the organisation of the knowledge base, knowledge integration, diffusion of technologies. For this purpose, social network theory and the concepts of strong and weak coupling have been adopted (e.g. [START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF][START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF], Rost, 2011, among others). The concept of couplings is also used when studying knowledge-based ecosystems with long term goals. Strong and repeated couplings in such ecosystems reflect relational embeddedness and suggest shared interests, accumulation of trust and potential for future collaboration (Van der Borgh, Cloudt and [START_REF] Van Der Borgh | Value creation by knowledge-based ecosystems: evidence from a field study[END_REF].

If one considers collaboration links as a coupling, then the evolution of collaborative links means evolution of couplings [START_REF] Borgatti | Network Theorizing[END_REF], which may affect collaborative innovation outputs, or the success of the project positively, negatively, or neutrally. Extant literature pointed out that cooperation, and notably the alignment of interests, the commitment of the partners and their relationships [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF], are the key determinant of successful collaboration, but less attention was paid to the criticality of coordination [START_REF] Gulati | Meta-organization design: rethinking design in interorganizational and community contexts[END_REF], or the effective alignment and adjustment of the activities, actions and knowledge of the partners (e.g. [START_REF] Lawrence | Differentiation and Integration in Complex Organizations[END_REF][START_REF] Gulati | Meta-organization design: rethinking design in interorganizational and community contexts[END_REF][START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF][START_REF] Okhuysen | Coordination in organizations: An integrative perspective[END_REF][START_REF] Wolbers | Introducing a Fragmentation Perspective on Coordination in Crisis Management[END_REF]. One of the roles of project managers is to optimise collaboration and to maximise the value creation in the project (Söderlund, 2013, Laursen andSverij, 2016); thus, project managers shall monitor and anticipate the evolution of the couplings and take necessary actions, either preventing certain evolutions, e.g. breaks of couplings, or encouraging such evolution, e.g. the establishment of new cooperations that were not planned originally. However, in the context of multi-actor publicly funded RDI projects, it is not known where, when and why reconfigurations of couplings are likely to happen.

This leads to the conclusion that at the intersection of these two literature streams, project studies and innovation studies, lies an interesting theoretical puzzle, which makes the setting of multi-actor RDI projects where couplings are between multiple heterogenous organisations particularly compelling to study how the dynamics of couplings happen within the fixed project boundaries with many constraints, such as time, budget, scope, fixed set of partners, and in the presence of relational and cognitive complexities, and how this influences the collaborative innovation outputs of the projects. To explain this a bit further: first, couplings typically appear in multi-actor RDI projects as planned working relationships; many simultaneous couplings within the project may exist, due to the large setting (see Fig. 1.1) and they may have different levels of priorities for the project actors, but it is not clear how this influences the evolution of the couplings. Second, flexibility seems to be very important: the consortia are often created using an opportunity offered by the funding agency, many actors do not know each other well before the project start, technical and other uncertainties are present to a significant extent in the planning phase (Konig et al., 2013), and thus the partners may often be in the situation described by [START_REF] Huxham | Managing to collaborate: the theory and practice of collaborative advantage[END_REF], that the goals are not fully clear during the set-up of the collaboration and they must be clarified during the collaboration process, when the project implementation starts. In addition, the organisations in the projects considered here are autonomous, and external intervention is limited [START_REF] Bor | A Theory of Meta-Organisation: An Analysis of Steering Processes in European Commission-Funded R&D 'Network of Excellence' Consortia[END_REF]; the absence of external intervention and control instances creates network with a weak centre but conversely, this is precisely what may also open up the possibility of adaptations of the collaborative links. On the other hand, options for flexibility and adaptability are limited: a consortium is set up in the application, the project scope is defined, the collaboration path is laid out, the budget is fixed, and the funding institution monitors the execution according to the plan. The scarce literature on processes in multi-actor RDI projects does not suggest how, if at all, such inter-organisational RDI projects with multiple actors develop over the project life cycle, and how these developments may influence the collaborative innovation outputs.

Also, the processes in the empirical context of multi-actor R&D projects are not well understood. Scholars called for research to clarify how patterns of couplings influence collaborative innovation over time (Michelfelder andKratzer, 2013, Hofman et al., 2016). 1.3.4 The role of project size and its effect on project performance

Positive influences of large groupings of organisational partners have been investigated

extensively in the knowledge management and organisational learning stream of literature. A large knowledge base is generally considered to be beneficial for knowledge generation due to the increase of recombination possibilities [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]. Consequently, an increase of the size and the variety of the knowledge base should be beneficial for knowledge generation and innovation [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]Ahuja, 2008, Cartner et al., 2011). The skills and resources of multiple project partners foster problem solving and are supposed to positively influence the performance of the project (Schilling, 2005). Larger team size allows team members to specialise in different applications and activities [START_REF] Haeussler | Division of labor in collaborative knowledge production: The role of team size and interdisciplinarity[END_REF]. Other positive influences include learning opportunities and increased resource sharing (e.g. [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Gulati | Meta-organization design: rethinking design in interorganizational and community contexts[END_REF][START_REF] Messeni Petruzzelli | University-Industry collaborations and international knowledge spillovers: a joint-patent investigation[END_REF][START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF], although the benefits of size often occur in diffusion, after the outputs have been produced [START_REF] Singh | Lone Inventors as Sources of Breakthroughs: Myth or Reality[END_REF].

Another stream of literature that investigates search strategies of firms for external knowledge and the resulting innovation performance, has investigated the dimension of breadth (size of the search) and has shown that there are decreasing returns for an increased external breadth of search: when multiple sources of knowledge are available, firms should decide on the level of breadth to maximise innovation performance [START_REF] Laursen | Open for Innovation: The Role of Openness in Explaining Innovation Performance Among U.K. Manufacturing Firms[END_REF]. Similar argument was made by researchers that investigate recombination and knowledge interdependency (Fleming andSorenson, 2004, Karim and[START_REF] Karim | Structural recombination and innovation: Unlocking internal knowledge synergy through structural change[END_REF]. [START_REF] Garriga | How constraints and knowledge impact open innovation[END_REF] have demonstrated that while external knowledge increases innovation performance, constraints on resources of the firms has a negative effect. The breadth of the search has an opportunity cost and it takes time and attention away from other innovation activities, thus affecting the innovation performance negatively [START_REF] Dahlander | One foot in, one foot out: how does individuals' external search breadth affect innovation outcomes[END_REF]. This occurs because innovation activities require joint work and significant time spent together, and not just exchange of information [START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Kobarg | More is not always better: Effects of collaboration breadth and depth on radical and incremental innovation performance at the project level[END_REF].

Burt regarded the size of a network as a "mixed blessing" (Burt, 1992, p. 64), and emphasised potential drawbacks of the size in the case of lack of diversity in a network. In the multi-actor R&D project setting, drawbacks of the size with respect to the research and innovation performance exist especially because the partners are diverse and heterogeneous: coordination and cooperation are more complex, difficult and effort-consuming in large interorganisational settings [START_REF] Heidl | The influence of interorganizational embeddedness on multipartner alliance stability[END_REF][START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Li | Top Management Attention to Innovation: The Role of Search Selection and Intensity in New Product Introductions[END_REF][START_REF] Mishra | Collaboration in Multi-Partner R&D Projects: The Impact of Partnering Scale and Scope[END_REF]. Drawbacks may result in organisational inefficiencies of the project and in a decrease of the innovation performance [START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF]. In addition, the larger the number of partners, the larger the risk to get free riders on board, which also negatively influences project performance (Okamuro, 2007, Gibson and[START_REF] Gibson | A Healthy Divide: Subgroups as a Stimulus for Team Learning Behavior[END_REF]. The above-mentioned concerns are even more salient in the multi-actor project setting with multi-dimensional constraints, such as limits of resources, project duration, and project scope. When the project includes multiple actors, the number of knowledge transactions and the collaboration complexity increase [START_REF] Badir | The Formation of Tie Strength in a Strategic Alliance's First New Product Development Project: The Influence of Project and Partners' Characteristics[END_REF] and thus the concerns amplify.

Extant literature investigated the relationships between the project size, in terms of the number of organisational partners (also called scale or breadth) of multi-actor projects and their research and innovation outputs, or performance, but the findings are contradictory and sometimes diametrically opposed. This led to contradictory recommendations to policy makers and programme designers. It is not clear why such tension exists in the literature.

Some scholars found that project performance significantly and positively depend on the number of participating organisations [START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF]. The problem with this study however is that the patenting activities were only analysed for the participating organisations in general, not related to the projects. Thus is equally possible that firms and institutions with a strong patenting activity are more prone to enter large consortia than others, so the claimed causality may in fact not exist. Other scholars drew contradictory conclusions and provided evidence that the increased number of partners-organisations in subsidised R&D projects has a weakly negative effect on the innovation outputs [START_REF] Dyer | Determinants of Success in R&D Alliances[END_REF], no significant effect (e.g. Schwartz et al., 2012, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], or a diminishing marginal effect [START_REF] Lee | Impact of Cooperative R&D Projects on ICT-Based Technology Convergence[END_REF]. [START_REF] Jacob | Analysis of patents, publications and new organisations from FP7-ICT and CIP-ICT projects[END_REF] found that an increase in the number of participating organisations increases the number of publications but has no effect on the number of patents.

It is not clear why such discrepancies exist. The role of additive effect is also not clear: bigger projects may generate more innovation outputs just because they have bigger number of participating organisations. The concern is that the previous research focused mostly on publication and/or patents when considering innovation performance of the project, which this represents just a fraction of the technological outputs in RDI project [START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF], as explained above.

Contradictions in the literature, difficulty in the identification of innovation outputs of the projects and the lack of empirical qualitative research make it difficult to assess why and how, if at all, the size in collaborative RDI projects with multiple actors brings positive, greaterthan-additive effect on the innovation performance of the project, in terms of the full set of research and innovation outputs. Scholars called for future research on the project level which "may be able to go a step further … in an attempt to provide a more complete understanding of the role of scale in collaborative R&D performance" (Spanos and Vonortas, 2012, p. 507).

We have considered different elements of RDI projects from the project studies perspective: architecture, size, processes. But multi-actor collaborative RDI projects have knowledge integration and innovation in focus [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. To investigate these topics, the study draws upon the complementary literature on innovation.

1.4 Knowledge integration, innovation outputs and project performance

Knowledge integration and innovation

Since the seminal work of [START_REF] Schumpeter | The Theory of Economic Development[END_REF], scholars usually view innovation as the novel recombination of ideas, resources and knowledge for value creation [START_REF] Nonaka | A Dynamic Theory of Organizational Knowledge Creation[END_REF][START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF][START_REF] Dibiaggio | Knowledge integration and vertical specialization in the semiconductor industry[END_REF]. New knowledge and technological advancements become innovations when they are implemented (OECD Oslo Manuel, 2018[START_REF] Gault | Defining and measuring innovation in all sectors of the economy[END_REF]. Each step of the innovation process generates new knowledge and creates value [START_REF] Pavitt | Innovation processes[END_REF][START_REF] Eisenhardt | Accelerating Adaptive Processes: Product Innovation in the Global Computer Industry[END_REF][START_REF] Cooper | Winning at new products: Creating value through innovations[END_REF], but not all of these steps lead to successful innovations.

Innovation has been also defined as "a new or improved product or process (or combination thereof) that differs significantly from the unit's previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)" OECD Oslo Manual, 2018, p. 20). Combinatorial innovation [START_REF] Obstfeld | Social Networks, the Tertius lungens Orientation, and Involvement in Innovation[END_REF] requires knowledge integration, and scholars defined knowledge integration "as a process of collaborative and purposeful combination of complementary knowledge, underpinned by specific and focused personal, team and organizational capabilities, a process that usually involves significant element of new knowledge generation" (Berggren et al., 2011, p. 7). The more specialised the knowledge becomes, the more there is a need for dynamic knowledge integration for innovation: this is particularly critical in inter-organisational settings [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF].

(
Knowledge integration depends on the characteristics of the knowledge being integrated (Brusoni andPrencipe, 2013, Johansson et al., 2011), and requires strong internal capabilities for exploiting external knowledge and adequate knowledge integration mechanisms [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF] such as rules, sequencing activities, organisational routines, group problem-solving, formal and informal interactions [START_REF] Grant | Towards a knowledge-based theory of the firm[END_REF][START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF][START_REF] Canonico | The adoption of knowledge integration mechanisms in an interdisciplinary research project[END_REF]. [START_REF] Enberg | Enabling knowledge integration in coopetitive R&D projects-The management of conflicting logics[END_REF] pointed out that among the two dominant approaches in literature on knowledge integration, one puts accent on the need for knowledge integration mechanisms based on frequent interactions and knowledge sharing, and another one emphasises structural mechanisms, attaching lower importance to shared knowledge and joint understanding. Project structuring supports knowledge integration: it helps to deal with interdependencies between activities and components, to organise interactions, to stimulate exchanges when required, or to put constraints on them, for example in case of coopetition in the project [START_REF] Enberg | Enabling knowledge integration in coopetitive R&D projects-The management of conflicting logics[END_REF].

Modularity for knowledge integration

The literature on innovation and modularity often studies the knowledge base and knowledge flows through the network lens [START_REF] Steen | Projects and Networks: Understanding Resource Flows and Governance of Temporary Organizations with Quantitative and Qualitative Research Methods[END_REF], putting the focus on the architecture of the network (e.g. [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF]. Scholars conceptualised the architecture of networks in terms of the nodes that make up the network, the couplings between the nodes, and the resulting pattern of interconnections [START_REF] Ahuja | The Genesis and Dynamics of Organizational Networks[END_REF]. In the context of inter-organisational R&D projects with multiple actors, the interconnections are in terms of patterns of coordinated activities and in terms of patterns of knowledge integration; interconnections of knowledge and activities may be partially independent (Tell, 2011, Brusoni and[START_REF] Brusoni | Making Design Rules: A Multidomain Perspective[END_REF].

The architecture of a complex organisational setting is rooted in the concept of modularity, or decomposability [START_REF] Simon | The Science of the Artificial[END_REF]). Simon's seminal theory of complex system and near decomposability puts in evidence that the structure influences the performance of a system: complex systems with nearly decomposable configurations adapt themselves to the demands of their environment more easily than non-decomposable systems; the level of decomposition depends on the necessity of interactions between components that is necessary to achieve the intended results [START_REF] Simon | The Science of the Artificial[END_REF]. Modularity describes the degree of independence and interdependence of the components of a system [START_REF] Baldwin | Managing in an age of modularity[END_REF], which can also be conceptualised as the presence or absence of couplings [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF]. Schilling (2000) described modularity as a system-related construct which reflects the degree of decomposition and recombination within the system: how tight or flexible are the couplings within the system and to which degree is recombination possible. A modular structure facilitates coordination, helps to organise the knowledge base and the interdependencies [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]Ahuja, 2008, Brusoni and[START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], and helps to manage complexity [START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF]; it also may help to achieve efficiency in integration of specialised knowledge [START_REF] Grant | Towards a knowledge-based theory of the firm[END_REF]. However, a too high modularity has downside effects and is often counterbalanced by integration [START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF], through couplings between the modules.

There is not enough understanding about the implications of the degrees of modularity in a collaborative project setting, and it is even less studied in the setting of publicly funded collaborative research and innovation projects with many partners.

Knowledge integration in inter-organisational structures

Previous research has investigated the characteristics of actors, knowledge, and relevant knowledge integration mechanisms in different inter-organisational settings and contexts. For instance, inter-firm R&D projects are characterised by a relatively limited number of involved autonomous organisations and by a limited (usually up to several years) duration of the project. In such settings, an analysis of the interplay between coordination, or alignment of the knowledge of the firms, and cooperation, or alignment of the interests of the firms and their relationship, is needed for better understanding of knowledge integration, which implies adapted governance mechanisms [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF]. Rotating leadership is an example of a governance mechanism that helps to diversify the access to different knowledge in different phases of the project and to facilitate knowledge integration [START_REF] Davis | Rotating Leadership and Collaborative Innovation: Recombination Processes in Symbiotic Relationships[END_REF]. R&D business ecosystems are often characterised by the presence of a focal, orchestrating firm, multiple partners and relative stability of relationships without fixed durations. [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF] studied couplings and knowledge in such settings, and demonstrated how different types of couplings between the actors are influenced by the features of the problems that require to be addressed, how types of couplings relate to the degree of specific knowledge characteristics, such as uncertainty, complexity, and ambiguity, and how types of couplings may have to evolve over time. The level of ambiguity also depends on the historic collaboration and the context of project preparation [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF], and it echoes the degree of tacitness of knowledge being integrated [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF]:

the higher the ambiguity, the higher the tacitness of knowledge, the more couplings and interactions are required. In innovation collaboration networks, scholars studied the decomposability of the knowledge base with different strengths and densities of couplings, and pointed out a continuum of different types of architectures [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF].

It was found that there is no optimum architecture, as it depends on the objectives of the actors [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF], but the nearly-decomposable architecture combining strong and weak couplings is considered beneficial in terms of the performance of inter-organisational collaborations [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF].

Extant literature on projects and networks mostly deals with inter-firm project and networks, often with one orchestrating firm: although the findings presented above provide directions, they do not provide an answer to the research questions in the context of inter-organisational RDI projects with multiple actors. First, as mentioned before, these projects operate in contexts which are often characterised by complexity, ambiguity and uncertainty at the same time: this requires tight coupling to deal with ambiguity, but also loose coupling to deal with complexity and uncertainty [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], and it is not clear how to combine different types of couplings. In addition, there is no focal firm. Second, there are multiple options of couplings because of many organisational actors, and mechanisms such as rotating leadership [START_REF] Davis | Rotating Leadership and Collaborative Innovation: Recombination Processes in Symbiotic Relationships[END_REF] are hardly feasible at the project level due to the contractual arrangements with the funding agency.

Measuring market upstream innovation outputs and innovation performance of the projects

Extant literature studied research performance since long (e.g. [START_REF] Martin | The evolution of science policy and innovation studies[END_REF][START_REF] Grimaldi | Assessing collaborative, pre-competitive R&D projects: the case of the UK LINK scheme[END_REF][START_REF] Martin | The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster[END_REF], but the understanding of the 'innovation side' of the project performance is lagging behind. Scholars studied implications of size, architecture, dynamics of inter-organisational collaboration on project performance, but this brings up the central question which came up regularly during this research: how to assess the innovation performance at the end of the projects directly and objectively, knowing that innovative technologies are not implemented or offered on the market yet?

The raison d'être of multi-actor RDI projects is often the generation of knowledge outputs (new knowledge, publications, patents…) and of market upstream technological outputs at the same time (e.g. Goldstein andNarayanamurti, 2018, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], while the projects may have their focus more on research or more on innovation [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF]. Technological outputs, often called in the literature innovation outputs (IO), may take different forms: new products, innovative technologies, deployments of innovations at different maturity levels, processes, methods, procedures, software, tools, testing activities, patents (e.g. [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF][START_REF] Nepelski | The organisational and geographic diversity and innovation potential of EU-funded research networks[END_REF]. These diverse outputs are related to the different objectives and timelines of the project actors [START_REF] Mannak | A temporal perspective on repeated ties across university-industry R&D consortia[END_REF].

"Innovation can and should be measured" (Olso Manual, 2018, p.20), but it is difficult to measure the innovative technologies, especially with respect to R&D which is upstream of the market [START_REF] Aguiar | European cooperative R&D and firm performance: Evidence based on funding differences in key actions[END_REF]Gagnepain, 2017, Kim et al., 2017). An objective assessment of IOs at this stage is challenging [START_REF] Henttonen | Searching for appropriate performance measures for innovation and development projects[END_REF], Brown and Svenson, 1988[START_REF] Birchall | Innovation performance measurement: Current practices, issues and management challenges[END_REF]. Scholars, practitioners and policy makers repeatedly question how to assess IOs at the stage when they are not innovations yet (e.g. [START_REF] Dewangan | Towards a holistic enterprise innovation performance measurement system[END_REF]Godse, 2014, Eling and[START_REF] Eling | Managing the Front End of Innovation-Less Fuzzy, Yet Still Not Fully Understood[END_REF]. Different solutions were proposed: some researchers assessed IOs and the innovation performance of the projects from the point of view of the project actors, thus indirectly and subjectively (e.g. [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF][START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF][START_REF] Arranz | Network Embeddedness in Exploration and Exploitation of Joint R&D Projects: A Structural Approach[END_REF], other scholars assessed IOs from the perspective of their innovation potential [START_REF] Nepelsky | Innovation and innovator assessment in R&I ecosystems: the case of the EU Framework Programme[END_REF]Van Roy, 2020, Nepelsky et al., 2019) which necessitates a predictive evaluation by experts, or by looking at their impact a few years after the project end (e.g. von [START_REF] Von Raesfeld | Influence of partner diversity on collaborative public R&D project outcomes: A study of application and commercialization of nanotechnologies in the Netherlands[END_REF], Kim et al., 2017). There is little literature on the assessment of IOs and of the RDI performance at the development and validation stage of the innovation process. [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF], in a thorough analysis of over 250 scholarly papers, did not identify any indicator to assess the IOs at this stage, and called for a deeper analysis of the possibilities for evaluating IOs at the early stage of the innovation process.

These authors also noted that market upstream IOs are traditionally indirectly measured by patents. However, the extant literature highlighted repeatedly (e.g. [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF][START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF][START_REF] Hung | Social networks, technology ties, and gatekeeper functionality: Implications for the performance management of R&D projects[END_REF]) that this lens does not provide the full picture: non-patented innovative technologies are often missing in the indicators [START_REF] Martin | Introduction to discussion paper on 'The sciences are different and the differences are important[END_REF].

The R&D measurement literature suggests that IOs could be assessed by their technical progress [START_REF] Bican | Managing innovation performance: Results from an industryspanning explorative study on R&D key measures[END_REF]Brem, 2020, Chiesa and[START_REF] Chiesa | Searching for an effective measure of R&D performance[END_REF], where technical progress is measured by technical characteristics, but these are specific to each technology, and thus it is hardly feasible to use this indicator in publicly-funded projects that address different technologies. Progress of the technologies at the development and validation stage may also be assessed through their progress in maturity, using the Technology Readiness Level (TRL) scale [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF]. It originates from US space programmes, and has been adapted to other sectors and became an instrument of European innovation policy since 2014 [START_REF] Ec | Horizon 2020 -Work programme[END_REF][START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], however it is interpreted quite differently by different organisations and for different sectors, it is considered as too coarse [START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF], and beyond the defence and space sectors, it is debated controversially [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], Olechovsky at al., 2020).

In a separate literature stream, project studies assess innovation performance and IOs in the project context, from the perspective of project value creation, where value has been defined

as the ratio of benefits to costs (e.g., Laursen andSvejvig, 2016, Martinsuo et al., 2019), and benefit as "improvement resulting from a change… that is perceived as positive by one or more stakeholders" (Laursen and Svejvig, 2016, p. 737). This includes the notions of the quantity and quality of project outputs (e.g. [START_REF] Martinsuo | Evaluating the organizational impact of product development projects[END_REF][START_REF] Hoegl | Financial constraints in innovation projects: When is less more?[END_REF][START_REF] Evanschitzky | Success factors of product innovation: An updated meta-analysis[END_REF][START_REF] Atkinson | Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria[END_REF][START_REF] Andersen | Rethinking Project Management: An Organisational Perspective[END_REF]. In publicly funded projects, cost and time are fixed parameters, so the question is how to assess the quality of the IOs.

A decade ago, scholars already called for research to better understand the performance of collaborative RDI networks and projects (Pandza et al., 2011, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF].

Recent papers continue to highlight the need to investigate other metrics of innovation performance following a case study approach, and going beyond publications and patents [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]: a deeper look and more fine-grained, nuanced view on the innovation outputs will be beneficial for innovation research [START_REF] Collinson | Recombination for innovation: performance outcomes from international partnerships in China[END_REF].

But literature does not propose how to assess innovation outputs directly and objectively at the stage of development and validation process.

Research gaps and research questions

Our understanding of the differences amongst inter-organisational projects is still limited [START_REF] Ahola | So alike yet so different: A typology of interorganizational projects[END_REF] and it is of interest to better investigate collaborative RDI projects with multiple actors, as these projects are not well understood. There are multiple calls for research to explain the setting and processes of collaborative R&D projects, and their influence on collaborative innovations over time. To study the projects focused on innovations, scholars call for a better connection between the research streams of project studies and innovation studies [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018), especially when studying projects that focus on knowledge generation [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF].

In line with these calls, the proposed conceptual framework (Fig. 1.2) grounds the research study in theoretical constructs, builds on both project management and innovation literature, and helps to identify the main research gaps. The overall research gap is related to the lack of understanding of the settings and processes in multi-actor collaborative projects (vom Brocke and Lippe, 2015), and lack of understanding how collaborating couplings and processes influence innovation outputs over time (Michelfelder andKratzer, 2013, Hofman et al., 2016). It is not clear how to assess collective innovation performance in these projects (Spanos andVonortas, 2012, Nepelsky et al., 2018) in an objective and direct way, beyond "classical" indicators of publications and patents [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]. In general, there is lack of qualitative studies allowing to answer the 'why' and 'how' questions, largely due to the difficulty of access to the field.

Consequently, the main research question of the study is: How do the architecture, size, and processes in multi-actor RDI projects influence collaborative innovations and overall innovation performance of the projects? The answers to these questions would also provide directions about how the project management can help to improve the performance of such projects.

The main research question was divided into four individual research questions, as presented below, and then addressed in the four empirical studies presented in Chapters 2 -5.

The architecture of the projects and its implications on collaboration and thus on innovation performance (empirical study #1, Chapter 2):

Research gap: Abundant literature on inter-organisational projects, ecosystems and networks studies separately activity coordination and knowledge integration, and mostly focuses on inter-firm collaboration, in most cases with one focal actor, but it is not well understood how to best set up inter-organisational RDI projects [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF] where both coordination of activities and knowledge integration are required, where they interplay, and what are the implications on the project implementation.

Research questions:

(1) How do activity coordination and knowledge integration interplay to define the architecture of inter-organisational RDI projects with multiple actors? (2) How does the project architecture favour (or disfavour) collaboration and how does it affect the resilience of the project and the requirements for project management?

The evolution of collaboration and implications on collaborative innovations (empirical study #2, Chapter 3):

Research gap: The abundant literature on processes in inter-organisational projects, ecosystems and networks mostly investigated rather flexible inter-firm settings, in most cases with one focal actor (e.g. Johansson et al., 2011, Davis and[START_REF] Davis | Rotating Leadership and Collaborative Innovation: Recombination Processes in Symbiotic Relationships[END_REF]. The processes in the empirical context of multi-actor RDI projects are not well understood: there are multiple calls for research to explain the settings and processes in such projects (Calamel et al., 2012, vom Brocke and[START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF] and how patterns of couplings influence collaborative innovation over time (Michelfelder andKratzer, 2013, Hofman et al., 2016).

Research questions:

(1) When, why and how does the evolution of couplings happen, over the lifetime of inter-organisational RDI projects with multiple actors? (2) How does it influence the collaborative innovations that are generated by the projects? The evolution of couplings is understood as the reconfiguration of the planned couplings between organisations over time.

The relationship between project size (in terms of number of partners) and innovation performance (empirical study #3, Chapter 4):

Research gap: Contradictions in the literature and the lack of empirical research make it difficult to assess whether large collaborative inter-organisational RDI project settings, with high numbers of heterogeneous partners, bring positive, super-additive effect in terms of collective innovation performance of the projects, specifically with regard to new knowledge generation and development of innovations, and how the project design should be adapted to this trend?

Research questions: (1) How, when and why do more actors generate overproportionally more outputs in inter-organisational R&D projects? (2) How can the design of such projects help to take advantage of the variety of partners? In short: is bigger better, and if yes, why, and how can the potential of a larger size be exploited?

Assessment of individual innovation outputs and collective innovation performance of the projects (empirical study #4, Chapter 5):

Research gap: It is not well known how to assess the 'innovation side' of the project performance in market upstream RDI projects (Kim et al., 2017, Aguiar and[START_REF] Aguiar | European cooperative R&D and firm performance: Evidence based on funding differences in key actions[END_REF], and literature does not suggest how to assess innovation outputs (the progress in the development of innovative technologies) directly and objectively at the development stage [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF]: the existing methods are subjective, indirect, or focus on the potential of innovative developments. A decade ago, scholars already called for research to better understand the performance of collaborative R&D networks and projects (Spanos andVonortas, 2012, Pandza et al., 2011). Recent papers continue to highlight the need to investigate other metrics of innovation performance following a case study approach, and going beyond patents [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]. A deeper look and more finegrained, nuanced view on the innovation outputs will be beneficial for innovation research [START_REF] Collinson | Recombination for innovation: performance outcomes from international partnerships in China[END_REF], and their measurement may provide evidence about project performance and thus contribute to reducing uncertainty for policymaking [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF].

Research question: How to assess the 'innovation side' of the collective performance of the projects, objectively and directly, focusing on innovative technologies (innovation outputs) developed in the projects? In the presence of multiple innovation outputs in the projects, this

requests firstly to answer the preliminary research question of how to assess individual innovation outputs. Such projects are "self-organised": the calls define the topic, the challenge, the scope of the project, and expected impact, but do not provide specifications other than minimum requirements, or eligibility, for the collaborative setting. The consortium proposes the collaborative setting for the joint work of partners towards the project objective: the "bottom up" proposal defines the project objectives and its structure, and sets up individual partners' roles, timing of activities, and budgets, the consortium must demonstrate why the proposed work requires joint effort and that the partners are complementary. However, the processing capacities, information and time are limited at the proposal stage and the design is often guided by heuristics and based on incomplete knowledge. In addition, the main goal of the preparatory phase is to prepare a winning proposal in order to obtain funding, there is not enough understanding of the drivers, needs, constraints of project partners as well as of their degree of flexibility. This may lead to promises that later in the execution of the project are hard to meet.

Research setting

The projects have a well-defined architecture that is presented in the proposals. In terms of workflow, they are decomposed into work packages and tasks. "Work packages" mean a major sub-division of the proposed project, providing the logical structure of the project work and the stages in which it is carried out. The proposed collaborative links are based on the project overall objective, interests of the partners, collaborative antecedents, previous patterns of collaborations, and heuristics of experienced team members. The tasks that were negotiated among the organisations that are involved in the project may be redefined during the project, and collaborative links may be redesigned, within the overall constraints of budget and time, but this remains rather exceptional in the context of EU-funded projects that lack structural flexibility: important changes can be done only through formal amendments and approval of the majority of the partners and approval by the funding agency, and in spite of the some budget flexibility, partners rarely wish to reallocate their subventions. Still, as research is uncertain, it may happen that initial plans have to be changed, and collaborative links may evolve or be dropped, with another collaborative link replacing the initial plans.

The projects may develop one or several independent innovative technologies, usually of generic nature, advancing them over various levels, from prototypes to pilot deployments and permanently rolled out innovations. To become productive, technologies are often embedded into an existing environment that consists of specific hardware (e.g. machines, processing equipment, transport units) and IT and automation systems, then they become new elements of operational systems rather than completely new large-scale systems. Many of these innovative technologies are not patented: The whole study is exploratory in nature [START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF]. Exploratory case studies are needed when there is a lack of detailed preliminary research and hypothesis to be tested [START_REF] Mills | Encyclopedia of Case Study Research[END_REF]. We have adopted an inductive multiple case study method [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF] for all studies, complementing it with quantitative evidence in the studies #2, #3 and #4. Multiple case studies provide a stronger base to build theory [START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF], enable comparison and demonstration of replication [START_REF] Eisenhardt | Better Stories and Better Constructs: The Case for Rigor and Comparative Logic[END_REF][START_REF] Bakker | Temporary organizing: Promises, processes, problems[END_REF], decrease distortion which may be present in a single case study and create more robust theory (Eisenhardt andGraebner, 2007, Yin, 2017). The replication logic here is meant as the analysis of each case on its own first, testing the emerging theory in each case separately, and focusing on "hard to measure constructs" (Gehman et al., 2018, p.287). While quantitative studies are looking at a representative sample to make generalisations, the logic of multi-case qualitative studies is to allow analytical generalisation [START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF]: the goal is to identify patterns which could confirm the presence of similar phenomena across cases and to shed light on the mechanisms behind the phenomena, to generalise them further to theory [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. In the study #3, the process approach was adopted [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF][START_REF] Abdallah | Performing Process Research, Standing on the Shoulders of Giants[END_REF][START_REF] Feldman | Making process visible: alternatives to boxes and arrows[END_REF], to illuminate the dynamics of inter-organisational collaborations (e.g. [START_REF] Berends | Introduction: Process views on inter-organizational collaborations[END_REF]. In the study #4, we discussed the results with some informants, and made an illustration of the results: "by seeing a concrete example… the reader has a much easier time imagining how the conceptual argument might actually be applied to one or more empirical settings" (Siggelkow, 2007, p. 22).

Following the guidelines recommended by [START_REF] Avenier | Finding one's way around various methodological guidelines for doing rigorous case studies: a comparison of four epistemological frameworks[END_REF], a case study protocol was developed, including specifications about quantity and variety of data collection. The study comprised several phases of selection of the projects (cases), data collection and data analysis, as described below; in the words of Eisenhardt, it was based on "deep immersion in multiple kinds of data… that help reveal the focal phenomena" (Gehman et al., 2018, p.288).

Although the presentation below is linear by phase, the work was not linear: regular iteration with literature studies was done at different steps, additional interviews were organised to clarify specific elements, etc. Then, depending on the study, additional criteria were applied to narrow down the knowledge base from 11 to 6 cases (study #1), 5 cases (study #4) and 4 cases (study # 2 and #3), to constitute theoretical sampling: the selected cases are "particularly suitable for illuminating and extending relationships and logic among constructs" (Eisenhardt and Graebner, 2007, p. 27). Each of these projects was unique, but the whole sample contributed to theory building, by allowing recognition of patterns of design, collaboration, dynamics of collaboration, processes, outputs, management, and comparison between several cases. For example, for the study #1, all selected cases (1) represented antecedents of collaborative research and innovation (50-60% of the participants in each of selected project had experience of collaboration with some other partners), (2) represented all or most types and sub-types of project architectures identified in the first phase, (3) were at an advanced project stage or already completed.

Data collection

Data for content analysis came from two main sources, documents and semi-structured interviews, however the nature of the main sources was slightly different for the different studies, as explained below.

First, over 1500 pages of project documentation were studied, including approx. 620 pages of project proposals (11 initially selected projects), over 950 pages of public and internal project reports and other documents, such as exploitation plans, approx. 50 pages of meeting agendas of the consortia…. The documents for data collection were slightly different for the different studies. For example, for study #4, detailed data on innovation outputs and about their start and end level in the projects were contained in e.g. exploitation plans, registers of project IPs, periodic reports to the funding agency, and in the EC database of patents and publications (EC, 2021a): this database links generated outputs to the funded projects that generated them.

However, for the first study, such details were not needed and thus only the project proposals and reports were used. In some cases, at least one core project actor, usually the project manager (coordinator) was asked to verify the identification of the project data resulting from the documentation (e.g. status of main outputs; project architecture etc): the use of both primary and secondary sources enables triangulation of the collected information in order to avoid potential interpretation biases [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF].

Second, to complement the data from the project documents, 54 semi-structured interviews were organised in several rounds, in 2018-2021. This also enabled to "catch" dynamics of collaborations over time. The interviews involved 36 informants. We used theoretical sampling [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF] to select the informants, using three criteria: different profiles of the partner-organisations (large industry, technological SMEs, research organisations), different roles of the persons in the projects (project manager, work package leader, contributor), different level of participation of the persons in the EC-funded multi-actor projects (multiple projects, or newcomer). In order to have comparable data, we mostly selected experienced informants: over 80% of the informants were CEOs of technological SMEs, team leaders within large multinational corporations, tenured university professors and research directors. Several informants participated in more than one project from our sample, they gave several interviews with comparative elements (at least one interview per project), adding to this research a comparative multiple-case dimension that so far has been underrepresented in the literature [START_REF] Bakker | Temporary organizing: Promises, processes, problems[END_REF]. The overview of the informants is provided in The interview protocol included 8 groups of questions related to (1) project preparation;

(2) motivation of organisations to participate in the project;

(3) project outputs, their maturity, their benefit; major events or memorable milestones in the projects; (4) project architecture;

(5) interorganisational collaboration, its evolution over time, concerns; (6) knowledge generation; (7) involvement of external parties; and (8) project management, leadership.

A detailed interview guide was developed, and all interviews were recorded. Before the interviews, written information was provided to the potential informants according to European General Data Protection Regulations (GDPR) requirements, to make sure that the interviewees understand the purpose of their involvement, explicitly agree to being interviewed and understand how the collected information will be used. The prior written information included the following elements: (1) names and contact of the investigator and supervisor;

(2) research purpose, length of the research project; and how the results of the project will be used; (3) why the person was invited to the interview; (4) modalities of the interviews, e.g. that taking part in the interview is entirely voluntary, there will be no incentives; the person can withdraw from any question or from the whole interview at any time without giving a reason; (5) durationapproximately one hour; (6) recording and transcript modalities, including period of keeping the recoding and transcripts, and storage modalities; (7) anonymous handling of information collected during the interviewit was specifically ensured that any summary content, or sometimes direct quotations from the interview, that are made available through academic publications will be anonymised, and other information. The informants were asked to sign the consent form to confirm that they understand and agree to the conditions of participation in the interviews. Their written consent was obtained before the interviews and their recording.

Most of the interviews had approx. 1 hour duration, but some additional complementary interviews with the same respondents were shorter (approx. 30 min) and focused on specific issues, to complement the previously collected data, for example on maturity of developed

Data analysis

The data analysis phase included a variety of activities, depending on the focus of the study.

They are presented in more detail in Chapters 2-5. Broadly speaking, firstly the project documents were analysed, for example to identify different project outputs, planned collaborative links, the stability of patterns of collaborating partners throughout the lifetime of the project, organisations contributing to each output, the project architecture etc. Then the explanations provided by the project actors were analysed. The data analysis was based on the abstractive process of open coding [START_REF] Strauss | Basics of qualitative research: Techniques and procedures for developing grounded theory[END_REF]Corbin, 1998, adapted by Gioia et al., 2013).

The Eisenhardt Method (Eisenhardt, 2021) is often used in multiple case studies, and the Gioia Methodology (Gioia et al., 2012), with first order data and second order themes, is considered to be well adapted to single cases. In this study these two methods were combined to analyse and present the structured evidence for the conclusions and to systematise the thinking. We built on the similarities between the methods: deep diving into the phenomena, theoretical sampling, and grounding of the concepts in the data in a plausible and realistic way, and for theory building [START_REF] Gehman | Finding Theory-Method Fit: A Comparison of Three Qualitative Approaches to Theory Building[END_REF]. In short, the approach of [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF] provided the general framework of this research, while the Gioia methodology was mobilised to pass from the data to the concepts, and to highlight the hidden part of the elaboration of the concepts and the interpretation of the investigator.

Most interviews transcripts were firstly coded using the NVivo 12.1 software: a data structure was built by grouping the concepts of 1st order (depending on the study) into aggregate and more abstract themes of the 2nd order, which were combined into several aggregate dimensions. The number of concepts depended on the paper: for instance, in the first study there were 31 concepts of 1st order, 13 aggregate and more abstract themes of the 2nd order, and 6 aggregate dimensions. An example of the data structure can be found in Appendix 2.2.

Coding and analysis were done for each of the projects taken as cases in the relevant study, then the results of the analysis were compared across cases. During the research process, reflections on the theoretical foundations were performed and some new elements were added to the interviews in response to these reflections, e.g. on historic experience [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF] or on cognitive technological distance [START_REF] Nooteboom | Optimal cognitive distance and absorptive capacity[END_REF].

Principal results

The research addresses large publicly funded multi-actor RDI projects, and studies several important elements that influence the collaborative innovations and thus the project performance: the project architecture, the processes in the projects, which lead to the dynamics of the inter-organisational collaborations, and the project size, in terms of the number of organisational partners. The research also suggests how the innovation outputs can be assessed at their early stages and compared in projects of similar domains, in spite of the technological variety of the outputs, and how the innovation performance can be assessed better and more completely for this type of projects.

Each of the four empirical studies addressed one or two interrelated research questions as presented in Chapter 1.5, and provided theoretical and practical contributions. Table 1.3 provides a brief overview of the four studies and their results, and also shows the structure of the thesis manuscript. Comparative inductive multiple case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF] 1. The interplay between activity coordination and knowledge integration defines project architectures.

2. Typology of project architectures: three main types and three sub-types of project architectures.

Architectures influence collaboration, resilience and project management in different way

Published: Klessova, S., Thomas, C., Engell, S. (2020) [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF], process view [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF][START_REF] Abdallah | Performing Process Research, Standing on the Shoulders of Giants[END_REF][START_REF] Feldman | Making process visible: alternatives to boxes and arrows[END_REF] 1. Eight types of reconfigurations of couplings: five in the modules and three at the interfaces between the modules, with systematic and predominantly negative implications on the planned collaborative innovations.

Six disintegrative and two integrative generative

EGOS, July 2020: accepted, track cancelled. Klessova, S., Thomas, C., Engell, S. (2020) Study #1 "Structuring inter-organizational R&D projects: towards a better understanding of the project architecture as an interplay between activity coordination and knowledge integration" identified how the project architecture influences the collaboration and thus the project performance. It provides empirical evidence that the architecture of the projects is an interplay between activity coordination and knowledge integration, and identifies six patterns of project architectures. The workflow-integrated architecture disintegrates the knowledge base, provides a lower collaboration potential and may require high management efforts, while a workflow-decomposed architecture makes project management easy but provides little added value from the inter-organisational setting. Nearly decomposable architectures offer the highest collaboration potential under contingent conditions. The study also provides insights how the different types of architectures influence collaboration, project resilience and project management, and leads to a better comprehension of how activity coordination and knowledge integration simultaneously influence the project execution and set requirements for project management.

Study #2 "The evolution of collaboration in multi-actor R&D projects: Patterns of couplings and collaborative innovations" complemented the study #1 and provided evidence how and why the processes in the projects lead to the evolution of planned collaborative links between organisational partners, and thus influence the collaborative innovations. It uncovered eight types of reconfiguration of couplings, activated by six disintegrative and two integrative mechanisms. Most of these reconfigurations led to negative consequences for the innovations or to innovations created by less partner organisations. The weakening of couplings was much less pronounced within specialised modules (subgroups of closely collaborating partners) within the projects than at the interfaces that were planned to connect the modules. Here, planned strong couplings often degraded into weak couplings or even disappeared over the project lifetime. Based on these findings, the study proposed a process model of the evolution of couplings in multi-actor RDI projects and implications on collaborative innovations. Study #3 "Is bigger better? A deeper look on size and innovation performance of multi-actor R&D projects: Empirical evidence from European Research and Innovation Programmes" added additional light on the implications of the size of the projects, in terms of the number of organisational partners, on project performance. It brought clarification to contradictory literature and provided explanations on when, how and why more organisational actors generate more outputs in such projects. The study provided evidence that over 95% of 185 research and innovation outputs in the studied projects were developed in small stable groups, or individually, independent of the project size: a large knowledge base can be beneficial after the project but was not beneficial within the project. Nevertheless, the generation of metaknowledge, a non-conventional project output not identified in the literature, is a possible benefit of a larger project size in specific settings.

After analysing these results, an additional element of the theoretical puzzle still remained: how to assess the project performance on the 'innovation side', at the end of the projects, in a more objective and direct way, focusing directly on the market-upstream innovation outputs that were developed within the projects? Thus, study #4 "Climbing up the ladder: Towards a better understanding of the innovation outputs and of the performance of market-upstream collaborative research and innovation projects" explored how the innovation outputs and the collective innovation performance in the projects can be assessed in the situation when the project actors generate not only knowledge, but also innovative technological solutions. This study put in focus the progress of the innovative technologies, which until now was not paid sufficient attention in the literature, and proposed a way of how to assess market upstream innovative technologies (innovation outputs) in the projects, assessing the realised (and not potential) value of the innovation outputs for the project actors in more objective way. The study developed a framework to measure the progress in maturity of the innovation outputs on their way to (potential) innovations, from the point of view of technological readiness.

Toward this goal, the existing Technology Readiness Level Scale to measure maturity of new technologies was refined. The study found that, while the absolute amount of efforts significantly differed for each output, the relative increase of the efforts between the start and end maturity levels of the different innovation outputs in the studied projects was relatively similar independent of the path taken, and there is a level when this increase of difficulties and efforts is exponential, and thus risky, especially in collaborative projects. From this, several policy implications have been identified. The study also provided practical contributions for the funding agencies and the project management, including in the private sector.

These four studies enhance the understanding of market upstream RDI projects with multiple actors, their setting and their processes, and contribute both to the project literature and to the innovation literature, further connecting these literature streams.

The remainder of the manuscript is organised as follows. The following chapters 2 -5 present the results of four empirical studies. The thesis concludes with a brief summary connecting the results of the four studies. It summarises the theoretical contributions, as well as the practical implications of the study, the limitations, and future research directions. Then the bibliography is presented, following by the Appendices.

Structuring inter-organizational R&D projects: Towards a better understanding of the project architecture as an interplay between activity coordination and knowledge integration

Introduction

Society has taken the 'projectification' path [START_REF] Midler | Projectification of the Firm: the Renault Case[END_REF][START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF]: organizations increasingly engage into inter-organizational projects [START_REF] Stjerne | Crossing times: Temporal boundaryspanning practices in interorganizational projects[END_REF]. Research, development and innovation (R&D) are affected by this trend: over the last decades, they have become more collaborative [START_REF] Chesbrough | Open Innovation: The New Imperative for Creating and Profiting from Technology[END_REF] and more project-organized [START_REF] Kim | Antecedents of open innovation at the project level: Empirical analysis of Korean firms[END_REF]. Multi-actor publicly funded inter-organizational R&D projects are a propitious setting for the generation of new knowledge, thanks to the diversity of the knowledge base of the actors and the possibilities for knowledge combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]. These projects have characteristic properties which are different from other inter-organizational projects: autonomy and equality of the organizational actors [START_REF] Bor | A Theory of Meta-Organisation: An Analysis of Steering Processes in European Commission-Funded R&D 'Network of Excellence' Consortia[END_REF] and bottom-up selforganization in response to competitive calls, considerable numbers of heterogeneous actors, collective responsibilities, significant public funding, and lack of structural project flexibility [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]. These projects address problems which are ambiguous, uncertain and complex at the same time [START_REF] König | A framework for structuring interdisciplinary research management[END_REF]. Inter-organizational R&D projects are difficult to manage [START_REF] Söderlund | Pluralistic and processual understanding of projects and project organizing; towards theories of project temporality[END_REF]Tell, 2011, Lin et al., 2019), especially in the multi-actor setting.

The project structure defines the way how the tasks are decomposed and coordinated [START_REF] Mintzberg | The structuring of organizations[END_REF], it significantly contributes to the organization of the collaboration [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF] and to the success of the project [START_REF] Caniëls | Collaboration in project teams: The role of mastery and performance climates[END_REF][START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF]. In the presence of multiple actors, the way in which the collaboration is arranged [START_REF] Fjeldstad | The Architecture of Collaboration[END_REF], or the structure of interdependencies between actors in an organizational setting [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF], is often called architecture, defined in terms of strength and intensity of links, or couplings, between the nodes in the network, or actors [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF]. Couplings are often studied using social network theory, which suggests that the structure and the strengths of relationships or couplings between the actors in the network explain differences in outcomes for these actors [START_REF] Granovetter | The Strength of Weak Ties[END_REF][START_REF] Coleman | Social capital in the creation of human capital[END_REF][START_REF] Borgatti | Network Analysis in the Social Sciences[END_REF]. In the context of collaborative research and innovation, dynamic knowledge integration leading into creation of new knowledge has become increasingly important [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF]; and the project structure provides a framework to organize knowledge integration [START_REF] Lin | Choosing suitable project control modes to improve the knowledge integration under different uncertainties[END_REF]. Abundant literature on inter-organizational projects, ecosystems and networks mostly focuses on inter-firm collaboration, in most cases with one focal actor, but it is not well understood how to best set up inter-organizational R&D projects [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF] where the both activity coordination and knowledge integration are required. Their interplay defines the core architectural collaborative patterns in inter-organizational R&D projects, called "project architecture" in this paper.

The present study addresses this research gap, using a theoretical framework which builds on literature on project management and additionally takes into account literature on innovation management. This is in line with the call of scholars for a better connection between the project management and innovation research streams [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018), especially when the focus is on projects aiming at knowledge integration to generate new knowledge [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. We pose two related research questions: (1) How do activity coordination and knowledge integration interplay to define the architecture of inter-organizational R&D projects with multiple actors? (2) How does the project architecture favor (or disfavor) collaboration and how does it affect the resilience of the project and the requirements for project management? In order to provide answers to these questions, an empirical inductive multi-case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF] has been put in place on inter-organizational R&D projects funded by the latest European Union's Research and Innovation Framework Programmes. The on-going Horizon 2020 Programme has provided 27,55B€ to fund 5147 such projects in 2014-2019, with 13.7 partners per project on average (EC, 2020b).

In the first stage of the study, 11 projects were selected and studied using the project documentation simultaneously from two angles: the angle of the activity coordination and the angle of the knowledge integration. In the second part, the set of projects was narrowed to 6 projects with different architectures. 43 semi-structured interviews were organized, coded and analyzed in terms of activity coordination and knowledge integration. The study provides empirical evidence about the interplay between activity coordination and knowledge integration, led to the identification of typologies of architectures with different implications on collaboration, project resilience and project management, and offers heuristics for project managers on how to structure inter-organizational R&D projects with multiple actors.

The study enhances the understanding of complex inter-organizational R&D projects with multiple actors and contributes to the literature on the management of inter-organizational projects and to the connection of project and innovation studies [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018[START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. From a practical point of view, the study provides suggestions to the project managers and funding agencies on how to adequately organize inter-organizational R&D projects with multiple partners.

The remainder of the paper is organized as follows. Firstly, the theoretical framework that guided the study is provided. Then the research setting and the research design of the study are explained, followed by the presentation of results, discussion and limitations of the research. The paper concludes with a summary of the theoretical contributions as well as the practical implications of the study, and future research directions.

2.2 Theoretical framework 2.2.1 Structuring inter-organizational R&D projects: project management perspective

Inter-organizational projects: structuring for activity coordination

Society is characterized by "projectification" [START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF][START_REF] Simard | The utopia of order versus chaos: A conceptual framework for governance, organizational design and governmentality in projects[END_REF]: more and more resources are allocated to projects, "a unique, novel and transient endeavor managing the inherent uncertainty and need for integration in order to deliver beneficial objectives of change" (Turner and Müller, 2003, p. 7). Organizations increasingly engage into inter-organizational projects [START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF][START_REF] Stjerne | Crossing times: Temporal boundaryspanning practices in interorganizational projects[END_REF], which are "so alike yet so different" [START_REF] Ahola | So alike yet so different: A typology of interorganizational projects[END_REF](Ahola, , p.1007); all of them require efficient collaboration [START_REF] Caniëls | Collaboration in project teams: The role of mastery and performance climates[END_REF]. The structure significantly contributes to the success of the collaboration in multi-actor projects [START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF]. Together with the term structure, the terms 'organizational design' or 'architecture' are largely synonymously used in the literature (Aubry and Lavoie-Tremblay, 2018) . Structure is "the sum total of the ways in which (an organization) divides its labor into distinct tasks and then achieves coordination among them" (Mintzberg, 1979, p.2). Architecture can be defined as the way in which a system is arranged [START_REF] Fjeldstad | The Architecture of Collaboration[END_REF] and in an organizational setting the focus often is on the patterns of interdependencies [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF]. Organizational design explains "what should be the design, structure, or architecture of the organization" (Burton andObel, 2011, p. 1198), referring to both the thing, i.e. the resulting organization, and the process to perform the design (Aubry and [START_REF] Lavoie-Tremblay | Rethinking organizational design for managing multiple projects[END_REF]. Often scholars do not address the organizational design/structure/ architecture of the projects, taking it 'for granted' [START_REF] Lavoie-Tremblay | Rethinking organizational design for managing multiple projects[END_REF]. Still, literature points out that structure stabilizes patterns of interactions between participants in projects and reduces the uncertainty for the involved actors (Raab et al., 2009, Söderlund andSydow, 2019), supports governance mechanisms (Van de Ven et al., 2013[START_REF] Miterev | Towards a design for the project-based organization[END_REF], helps to organize the integration of outcomes of different activities [START_REF] Srikanth | Integrating distributed work: comparing task design, communication, and tacit coordination mechanisms[END_REF]Puranam, 2011, Zerjav et al., 2018), and influences organizational efficiency and performance (e.g. [START_REF] Englmaier | Organization Design and Firm Heterogeneity: Towards an Integrated Research Agenda for Strategy[END_REF][START_REF] Burton | The science of organizational design: fit between structure and coordination[END_REF][START_REF] Yi | Differentiation and Integration in Organizational Learning: A Garbage Can Model[END_REF]. Structure also influences project resilience, or capacity to maintain project viability during times of disruptive change;

an important research issue is how structures should be designed to provide resilience of the inter-organizational settings [START_REF] Linnenluecke | Resilience in Business and Management Research: A Review of Influential Publications and a Research Agenda[END_REF]. [START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF] pointed out that most project theories focus on actors; this includes their interrelated activities. Extant literature explains decomposition and integration of activities, using e.g. process architecture framework, process flow diagrams, workflow [START_REF] Browning | Managing complex project process models with a process architecture framework[END_REF][START_REF] Shi | A new approach for project scheduling using fuzzy dependency structure matrix[END_REF]Blomquist, 2012, Bakker et al., 2018). Activities can be organized parallelly, sequentially and with partial overlapping [START_REF] Eppinger | Design Structure Matrix Methods and Applications[END_REF]. The choice of an appropriate structure of activities in a project is a big challenge: there are multiple ways to organize interactions, and as a consequence there may be a reduction or an increase of uncertainty and ambiguity [START_REF] Yang | The impact of uncertainty and ambiguity related to iteration and overlapping on schedule of product development projects[END_REF][START_REF] Lin | An effective approach for scheduling coupled activities in development projects[END_REF]. This challenge is even more salient in large inter-organizational R&D projects which offer even more options to structure human resources, interactions and thus collaboration between actors [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF], but organizational setting and processes in inter-organizational R&D projects with multiple actors are yet not well understood [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF].

Inter-organizational R&D projects: structuring for knowledge integration

Over the last decades, organizations face an increasingly challenging environment: innovation becomes more complex, the time to market is shorter, the competition on the market harder.

In response to these challenges, research and innovation have become more open and more collaborative [START_REF] Chesbrough | Open Innovation: The New Imperative for Creating and Profiting from Technology[END_REF][START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF][START_REF] Lundin | Managing and working in project society: Institutional challenges of temporary organizations[END_REF] and often take place within inter-organizational R&D projects setting [START_REF] Tiwana | Do bridging ties complement strong ties? An empirical examination of alliance ambidexterity[END_REF][START_REF] Kim | Antecedents of open innovation at the project level: Empirical analysis of Korean firms[END_REF].

Inter-organizational R&D projects with multiple actors are a specific category of interorganizational projects: their common purpose, raison d'être, is the generation of new knowledge [START_REF] Manning | The rise of project network organizations: Building core teams and flexible partner pools for interorganizational projects[END_REF] as the result of knowledge integration [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF].

Such projects operate in contexts of simultaneous complexity, ambiguity, and uncertainty [START_REF] König | A framework for structuring interdisciplinary research management[END_REF] and exhibit particular properties: autonomy and equality of partners [START_REF] Bor | A Theory of Meta-Organisation: An Analysis of Steering Processes in European Commission-Funded R&D 'Network of Excellence' Consortia[END_REF], large numbers of partners-organizations [START_REF] Pandza | Collaborative diversity in a nanotechnology innovation system: Evidence from the EU Framework Programme[END_REF][START_REF] Pinheiro | The role of social capital towards resource sharing in collaborative R&D projects: Evidences from the 7th Framework Programme[END_REF], heterogeneity of organizational actors, usually from academia and industry, collective responsibilities, significant public funding and lack of structural project flexibility [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF].

Efficient collaboration, or joint work to achieve a common purpose (new knowledge generation) significantly contribute to the project success. Joint work in early project stages, communication, coordination, and aligned efforts favor collaboration quality in multi-actor projects [START_REF] Caniëls | Collaboration in project teams: The role of mastery and performance climates[END_REF][START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF][START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. Historic experience and project context are other important aspects contributing to collaboration quality: "no project is an island" (Engwall, 2003, p. 789). Project management practice influences collaboration dynamics and needs to be adapted to the different types of projects [START_REF] Kapsali | Systems thinking in innovation project management: a match that works[END_REF].

The project structure in inter-organizational R&D projects with multiple actors exhibits the same characteristics as the structure of other inter-organizational projects; in addition, it has to facilitate knowledge integration and to reduce uncertainties [START_REF] Lin | Choosing suitable project control modes to improve the knowledge integration under different uncertainties[END_REF]. While there are several possible terms to describe the project structure, as explained in section 2.2.1.1, the authors of this paper prefer the term 'architecture' which emphasizes the patterns of interactions between the multiple actors in the project.

Project management scholars pointed out that inter-organizational R&D projects with multiple actors experience three main management paradoxes (vom Brocke and Lippe, 2015):

requirement for freedom due to research uncertainties versus requirements for tight management and structure that reduce uncertainty and help to achieve project outputs; requirement for integration of different research views versus the resulting heterogeneity of partners and inter-disciplinarily challenges; limited management authority versus requirement for integration of results and commitment of all partners. The literature also points out that the project actors may depend on each other in terms of workflow (activities) or in terms of knowledge flow [START_REF] Clement | Searching for Structure: Formal Organization Design as a Guide to Network Evolution[END_REF]; not only workflows but also knowledge flows shall be structured to improve performance of the organizations [START_REF] Rauniar | Knowledge integration in IPPD project: role of shared project mission, mutual trust, and mutual influence[END_REF]. This suggests that the managers of multi-actor inter-organizational R&D projects shall structure both activities and knowledge to facilitate collaboration. How to structure such projects is however not well understood [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF], but very important, as the consequences of structuring usually persist until the project end.

To address this research gap, we pose two related research questions: (1) How do activity coordination and knowledge integration interplay to define the architecture of interorganizational R&D projects with multiple actors? (2) How does the project architecture favor (or disfavor) collaboration and how does it affect the resilience of the project and the requirements for project management?

To fill this gap, scholars call for a better connection between the research streams of project studies and innovation studies [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018), especially when studying projects that focus on knowledge generation [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. In line with these calls, the theoretical framework contributes to better linkages between both streams of studies and builds upon the project management literature (section 2.2.1), and also on the innovation management literature (section 2.2.2).

Structuring inter-organizational R&D projects: an innovation management perspective

Knowledge integration

Knowledge is often created using the mechanisms of exchange and combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]; the activation of these mechanisms is conditioned by the opportunity for exchange or combination, the anticipation of valuable collaborative outcome, the motivation to be engaged, and the combinatory capability [START_REF] Moran | Theories of Economic Organization: The Case for Realism and Balance[END_REF]. However, the combinatory mechanisms are activated only when certain pre-conditions take place, such as reasonable cognitive technological distance [START_REF] Nooteboom | Optimal cognitive distance and absorptive capacity[END_REF] and existence of collaborative links and social relations, influencing the creation of social capital [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]Ghoshal, 1998, Burt, 2004).

Combinatorial innovation [START_REF] Obstfeld | Social Networks, the Tertius lungens Orientation, and Involvement in Innovation[END_REF] requires knowledge integration. Scholars defined knowledge integration "as a process of collaborative and purposeful combination of complementary knowledge, underpinned by specific and focused personal, team and organizational capabilities, a process that usually involves significant element of new knowledge generation" (Berggren et al., 2011, p. 7). The more specialized the knowledge becomes, the more there is a need for dynamic knowledge integration for innovation: this is particularly critical in inter-organizational settings [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF]. Knowledge integration depends on the characteristics of knowledge being integrated (Brusoni andPrencipe, 2013, Johansson et al., 2011), and requires strong internal capabilities for exploiting external knowledge and adequate knowledge integration mechanisms [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF]) such as rules, sequencing activities, organizational routines, group problem-solving, formal and informal interactions [START_REF] Grant | Towards a knowledge-based theory of the firm[END_REF][START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF][START_REF] Canonico | The adoption of knowledge integration mechanisms in an interdisciplinary research project[END_REF]. Knowledge integration is goal-oriented process: in the project context, the goal is to create new knowledge in order to obtain the project outcome. [START_REF] Enberg | Enabling knowledge integration in coopetitive R&D projects-The management of conflicting logics[END_REF] pointed out that among two dominant approaches in literature on knowledge integration, one puts accent on the need for knowledge integration mechanisms based on frequent interactions and knowledge sharing, and another one emphasizes structural mechanisms, attaching lower importance to shared knowledge and joint understanding. Project structuring supports knowledge integration: it helps to deal with interdependencies between activities and components, to organize interactions, to stimulate exchanges when required, or put constraints on them, for example in case of coopetition in the project [START_REF] Enberg | Enabling knowledge integration in coopetitive R&D projects-The management of conflicting logics[END_REF].

Modularity for knowledge integration

The literature on innovation and modularity often studies knowledge base and knowledge flows through the network lens [START_REF] Steen | Projects and Networks: Understanding Resource Flows and Governance of Temporary Organizations with Quantitative and Qualitative Research Methods[END_REF], putting the focus on the architecture of the network (e.g. [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF]. Scholars conceptualized the architecture of networks in terms of the nodes that make up the network, the couplings between the nodes, and the resulting pattern of interconnections [START_REF] Ahuja | The Genesis and Dynamics of Organizational Networks[END_REF]. In the context of inter-organizational R&D projects with multiple actors, the interconnections are in terms of patterns of coordinated activities and in terms of patterns of knowledge integration; interconnections of knowledge and activities may be partially independent (Tell, 2011, Brusoni and[START_REF] Brusoni | Making Design Rules: A Multidomain Perspective[END_REF]. The interplay of these patterns defines the project architecture.

Architecture of a complex organizational setting is rooted in the concept of modularity, or decomposability [START_REF] Simon | The Science of the Artificial[END_REF]). Simon's seminal theory of complex system and near decomposability puts in evidence that the structure influences the performance of a system: complex systems with nearly decomposable configurations adapt themselves to the demands of their environment more easily than non-decomposable systems; the level of decomposition depends on the necessity of interactions between components that is necessary to achieve the intended results [START_REF] Simon | The Science of the Artificial[END_REF]. Modularity describes the degree of independence and interdependence between the components of a system [START_REF] Baldwin | Managing in an age of modularity[END_REF], which can also be conceptualized as the presence or absence of couplings [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF].

Couplings are often studied using social network theory [START_REF] Granovetter | The Strength of Weak Ties[END_REF][START_REF] Coleman | Social capital in the creation of human capital[END_REF], which suggests that the variations in the properties of the networks, such as the structure and the strengths of relationships or couplings between the nodes in the network "account for differences in outcomes for the networks (or nodes)" [START_REF] Borgatti | Network Analysis in the Social Sciences[END_REF]. [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF] described couplings in terms of distinctiveness and responsiveness: "if there is responsiveness without distinctiveness, the system is tightly coupled. If there is distinctiveness without responsiveness, the system is decoupled. If there is both distinctiveness and responsiveness, the system is loosely coupled" (Orton and Weick, 1990, p. 205). In the project context, distinctiveness is about setting up distinct modules, while responsiveness is about maintenance of the coherence between the modules.

A modular structure facilitates coordination, helps to organize the knowledge base and the interdependencies [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]Ahuja, 2008, Brusoni and[START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], helps to manage complexity [START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF], it also may help to achieve efficiency in integration of specialized knowledge [START_REF] Grant | Towards a knowledge-based theory of the firm[END_REF]. However, too high modularity has downside effects, e.g. it encourages specialization within modules, thus creating barriers to collaboration: this is the reason why modularity is often counterbalanced by integration [START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF], through the couplings between the modules, whichespecially when they are not precisely defined initially -require dynamic management to ensure coherence of the system [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF].

Modularity for knowledge integration in inter-organizational structures

Previous research has investigated the characteristics of actors, knowledge, degree of modularity and relevant knowledge integration mechanisms in different inter-organizational settings and contexts. Below, some main findings of prominent scholars about several interorganizational structures are provided.

Inter-firm R&D projects are characterized by a relatively limited number of involved autonomous organizations and by a limited (usually up to several years) duration of the project. In such settings, analysis of the interplay between coordination, or alignment of the knowledge of the firms, and cooperation, or alignment of the interests of the firms and their relationship, is needed for better understanding of knowledge integration: the problems of coordination are related to knowledge characteristics -multi-disciplinarity, interdependence, degree of tacitness…, and thus call for adequate knowledge integration mechanisms, and both coordination and cooperation call for appropriate governance mechanisms [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF]. Rotating leadership is an example of a governance mechanism that helps to diversify the access to different knowledge in different phases of the project and to facilitate knowledge integration [START_REF] Davis | Rotating Leadership and Collaborative Innovation: Recombination Processes in Symbiotic Relationships[END_REF].

R&D business ecosystems are characterized by the presence of a focal, orchestrating firm, multiple partners and relative stability of relationships without fixed durations. [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF] studied coupling and knowledge in such settings, and demonstrated how different types of couplings between the actors are influenced by the features of the problems that require to be addressed, how types of couplings relate to the degree of specific knowledge characteristics, such as uncertainty, complexity, and ambiguity, and how types of couplings may have to evolve over time: "ambiguous problems call for tightly coupled ecosystems; complex problems call for loosely coupled ecosystems; and uncertain problems call for decoupled business ecosystems" (Brusoni and Prencipe, 2013, p. 167). The level of ambiguity also depends on the historic collaboration and the context of project preparation [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF], and it echoes the degree of tacitness of knowledge being integrated [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF]: the higher the ambiguity, the higher the tacitness of knowledge, the more couplings and interactions are required.

In innovation collaboration networks, scholars studied the decomposability of the knowledge base with different strengths and density of couplings, and pointed out a continuum of different types of architectures [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]: in highly decomposable knowledge structures, dense couplings exist within modules but not between modules; in nearly decomposable structures, there are dense couplings within modules and loose coupling between modules; in integrated (non-modular) structures, there are strong and pervasive couplings between groups of elements which are in interactions and interdependent. It was also found that there is no optimum architecture, as it depends on the objectives of the actors [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF], but the nearly-decomposable architecture combining strong and weak couplings is considered beneficial in terms of the performance of inter-organizational collaborations [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF].

Extant literature mostly deals with inter-firm project and networks, often with one orchestrating firm: although the findings presented above provide directions, they do not provide an answer to the research questions in the context of inter-organizational R&D projects with multiple actors. First, as mentioned before, these projects operate in contexts which are characterized by complexity, ambiguity and uncertainty at the same time: this requires tight coupling to deal with ambiguity, but also loose coupling to deal with complexity and uncertainty [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], and it is not clear how to combine different types of couplings. In addition, there is no focal firm. Second, there are multiple options of couplings because of many organizational actors. Rotating leadership [START_REF] Davis | Rotating Leadership and Collaborative Innovation: Recombination Processes in Symbiotic Relationships[END_REF] is hardly feasible at the project level due to the contractual arrangements with the funding agency. Third, to the best of knowledge of the authors, the architecture of couplings and its implications in inter-organizational R&D projects with multiple actors were not studied yet.

Towards a conceptual framework

Our understanding of the differences amongst inter-organizational R&D projects is still limited [START_REF] Ahola | So alike yet so different: A typology of interorganizational projects[END_REF] and it is of interest to analyse different settings of collaborative R&D projects with multiple actors, as these projects offer multiple options to structure activities and knowledge, with different implications which are yet not well understood. [START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF] pointed out that most project theories put the accent on the actors, and most network theories put the accent on the pattern of interdependencies, while none of them shall be put first: both are related and both shall be analyzed together. This duality of patterns is not well studied [START_REF] Pinheiro | The role of social capital towards resource sharing in collaborative R&D projects: Evidences from the 7th Framework Programme[END_REF]. The proposed conceptual framework (see Fig. 2.1) takes the duality of patterns into account, grounds the research study in theoretical constructs and provides a foundation to investigate the research questions defined in section 2.2.1.2: how do activity coordination and knowledge integration interplay to define the architecture of interorganizational R&D projects with multiple actors? How does the project architecture favor (or disfavor) collaboration and how does it affect the resilience of the project and the requirements for project management?

Fig. 2.1. Conceptual framework: the interplay between activity coordination and knowledge integration

In projects, a dynamic layer of processes is required in addition to the structure [START_REF] Lundin | A theory of the temporary organization[END_REF] and both, the architecture and the processes, should be analyzed (Aubry and Lavoie-Tremblay, 2018). This paper puts in focus the project architecture, because the interplay between activity coordination and knowledge integration as well as its implications are not well understood in structurally inflexible projects with multiple actors. The scope of both architecture and processes would be too broad for a single paper.

Research setting

The research setting is defined as inter-organizational R&D projects with multiple actors, that (2007( -2013( ) and Horizon 2020( (2014( -2020)). Although the topic of each project is specific, their setting is similar. The EC calls define the topic, the challenge, and the scope of the projects. The goal of the projects is pre-competitive R&D, the intended outputs is generation of new knowledge which is translated into innovations. The on-going Programme uses the term "research and innovation action" or "innovation action" for these projects, to highlight innovation beyond knowledge generation. The projects usually last three to four years and they are managed (coordinated, in the EU terminology) by a coordinator (project manager in this paper) which can be from industry, from a research organization or from a consultancy.

During the proposal preparation, the applicants agree on the way to organize their collaboration patterns, and they must demonstrate in the proposal why the project requires a joint effort and the presence of each partner. The projects have a well-defined architecture in terms of workflow: they are decomposed into work packages and tasks which provide the logical structure of the project activities. The proposal also describes the collaborative links between the partners, thus identifying the knowledge integration pattern. The project architecture is chosen based on the previous experiences of participants. At the proposal stage, time and available resources usually put tight restrictions on the exchange of information.

There is a great deal of tacit knowledge involved in the projects.

There are three main reasons that make this setting particularly interesting for management research. First, because the field is large and important: inter-organizational R&D projects with multiple actors have become an important instrument of R&D funding, and the number of such projects is steadily growing; the on-going Horizon 2020 Programme has already provided 27,55B€ to fund 5147 such projects involving over 70,000 participants (organizations) all over the world. Second, the number of partners in such projects is quite high, over 13 on average (EC, 2020b). Third, the projects lack structural flexibility, it is difficult to change partners or to drastically revise the workplan. Thus, the consequences of the architectural choices mostly persist until the project end.

The field is difficult to access: consultation of internal project documents by external organizations is possible only with unanimous authorization from all project partners because of confidentiality reasons. As the project work is reviewed by the funding agency, the partners of on-going projects are reluctant to share their concerns other than in interviews with someone they know and trust, with anonymity guaranteed. In addition, partners are geographically distributed across Europe and beyond. Given these difficulties, it is not surprising that this field so far remained largely outside the focus of management research.

Our study was enabled by the fact that the first author was either known by the informants or was introduced to them through a co-author or a common contact, and got access to data, the second author guided the research process and challenged the findings, and the third author has participated in many EU-funded multi-actor inter-organizational R&D projects.

Research design

We have adopted the exploratory inductive multiple case study method [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF].

Multiple case studies usually provide a stronger base to build theory (Yin, 1994, Eisenhardt and[START_REF] Eisenhardt | Theory Building from Cases: Opportunities and Challenges[END_REF], enable comparison and demonstration of replication [START_REF] Eisenhardt | Better Stories and Better Constructs: The Case for Rigor and Comparative Logic[END_REF] and provide a deep understanding of the investigated phenomena and processes [START_REF] Bakker | Temporary organizing: Promises, processes, problems[END_REF]. A detailed case study protocol was developed with two phases, including specifications of the quantity and variety of data collection [START_REF] Avenier | Finding one's way around various methodological guidelines for doing rigorous case studies: a comparison of four epistemological frameworks[END_REF].

Case selection and data collection

Phase one. Identification of project architectures.

For this phase, 11 projects were selected, using three selection criteria: homogeneity, variety and availability of data. Homogeneity: The selected projects (1) are inter-organizational R&D projects with multiple actors;

(2) have durations from 3 to 4 years; (3) have a considerable number of partners, between 10 and 21. Variety: the projects vary in terms of (1) technology maturity (EC, 2014); (2) project advancement: completed or on-going; and (3) their thematic.

We selected projects where it was possible to have access to project data. Each of these 11 projects is unique, but the whole sample enabled us to recognize and compare architectural patterns and to analyze their influence on the project execution.

Names of celestial constellations were given to the projects for their anonymization. The projects were firstly studied in terms of their characteristics, using content analysis of over 1500 pages of project documentation: 624 pages of project proposals, over 900 pages of public and internal project reports, and approx. 50 pages of agendas of plenary meetings.

Following the literature, we conceptualize modularity as the degree of (inter)dependence, or couplings, between the project actors, in terms of knowledge flow or/and workflow. We studied modularity by capturing the couplings (working links) between partners in the project as well as the strength of these couplings and dependencies (activities or knowledge). For example, if academic and industrial actors collaborate throughout a project, they form a module of knowledge and activities with tight couplings. Loose couplings describe weak, punctual or irregular collaborative links: this is, for example, review by one partner of a work done by another partner.

Based on the literature, we adopted the following definitions. A project combines different elements: actors, activities, knowledge, organized into components, which in turn may form modules, i.e. sets of activities organized in a thread of work for a common purpose, to produce specific outputs. Modules are characterized by a strong integration of the activities and of the knowledge of the participating partners. Integrative module(s) receive input from two or more other modules, and/or provide input to other modules, and produce own output(s). A coupling defines a connection between the elements or groups of elements of the project. The strength of a coupling, weak or strong, defines the degree of inter-dependence between the elements. A project output is an element of the finality of the project. The resilience of a project architecture describes to which extent the project is affected if an element is encountering problems, e.g. not performed or not performed satisfactorily.

Previously, scholars conceptualized also interacting activities between partners, such as plenary project meetings, as weak couplings (e.g. Michelfelder andKratzer, 2013, Pinheiro et al., 2016). These activities facilitate knowledge integration, and meetings may be part of the project architecture, however regular plenary meetings are mandatory in all projects in the research setting, and they are not specific to the types of architecture. Thus, for theoretical clarity, this paper focuses on working relations between partners as elements of the architecture; interactive activities are considered as part of the knowledge integration mechanisms.

We used three criteria to identify the project architectures: the finality (how many project outputs are planned or obtained), the decomposition of the project (how many modules are planned, are they running in parallel or sequentially, are they dependent), and the coordination (connectedness between modules, density and strength of integrative modules).

We asked at least one core project actor, usually the project manager, to verify the identification of the project architecture: the use of both primary and secondary sources enables triangulation of the collected information in order to avoid potential interpretation biases [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. Appendix 2.1 provides an overview of the projects and their architectures.

Phase two. Understanding the effect of the project architecture

For this second, explicative phase of the study, following [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF], we selected projects that are comparable and applied two additional criteria to narrow down the knowledge base from 11 to 6 cases that (1) represent all types and sub-types of architectures identified in the first phase, and (2) are at an advanced project stage or already completed. In all selected projects, about half of the partners had historical experience of joint work. We conducted 43 semi-structured interviews in 2018-2019: 39 interviews of about one hour duration each, and 4 complementary shorter interviews, that focused on specific issues that required additional investigations. The interviews involved 35 informants; in each project, representatives of between 35 and 65% of the participating organizations were interviewed.

We used theoretical sampling [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF] to select the informants, using three criteria:

different profiles of the partner-organizations (large industry, technological SMEs, research organization), different roles of the persons in the projects (project manager, workpackage leader, contributor), different levels of participation of the persons in EC-funded multi-actor projects (multiple projects or newcomer). In order to have comparable data, we selected experienced informants: over 80% of the informants were CEOs of technological SMEs, team leaders within large multinational corporations, university professors and research directors.

These informants allocated only part of their activities to the studied projects and thus they were able to see the project in a larger context. 7 informants participated in more than one project from our sample and thus interviews included comparative elements, adding to this research a comparative multiple-case dimension. An overview of the informants is provided in The interview protocol included 8 groups of questions related to (1) project preparation and context;

(2) motivation of organizations to participate in the project;

(3) project overview and outputs; (4) project architecture; (5) inter-organizational collaboration and its evolution over time; (6) new knowledge generation; (7) involvement of external parties, their influence; and

(8) project management. A detailed interview guide was developed, and all interviews were recorded. Before the interviews, information was provided to the informants according to European General Data Protection Regulations, and their consent was obtained about the recording. The fact that the informants knew one or more of the co-authors, either directly or through a colleague, supported establishing of trust since the beginning of the interview and resulted in the collection of a wealth of information.

The interviews resulted into 45.02 hours of recording and 791.2 pages of materials transcribed verbatim (Table 2.2). The results of the interviews were compared with the findings of Phase 1 and confirmed the classification of the project architectures.

Data analysis

The data analysis in phase two is based on the abstractive process of open coding [START_REF] Strauss | Basics of qualitative research: Techniques and procedures for developing grounded theory[END_REF]Corbin, 1998, adapted by Gioia et al., 2013). The interviews transcripts were firstly coded using the NVivo 12.1 software: a data structure has been built by grouping 31 concepts of 1 st order into 13 aggregate and more abstract themes of the 2nd order, which were combined into 6 aggregate dimensions (Appendix 2.2). Then each dimension was matched with different types of architectures identified during the documentary analysis. Coding and analysis were done for each of the 6 cases, and then the results of the analysis were compared across cases.

During the research process, reflections on the theoretical foundations were performed and some new elements were added to the interviews in response to these reflections, e.g. on historic experience [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF].

Results

The findings are presented below in two parts: firstly, the typology of the identified project architectures is presented, with their characteristics and sub-types. Then, the analysis of the implications of different types of architecture is provided.

Identification of three main types and six sub-types of project architectures

To facilitate the explanations, a conceptual representation of the project architecture is shown in Fig. 2.2. It provides an example of a project with four modules: three parallel modules, with small groups of contributing organizational actors following three main consecutive stages of activities (workflow), and one integrative module (green color) going throughout the project, where the whole group of actors contribute.

The results of the first phase put in evidence three main types of project architecture (Table 2.3): workflow-integrated (3 projects), nearly decomposable (6 projects), and decomposed (2 projects). The project architecture is not easily recognizable in the proposals: the workflow and the knowledge flow are often "hidden" in the formal structure of the workpackages and only became visible thanks to the analysis of internal project documents. The workflow-integrated architecture is sequential; its defining features are a dominating collaborative finality and the absence of modularity3 . It has two sub-types, sequential and converging. The sequential sub-type is in fact one big module that runs through the project:

output from one stage is a prerequisite for the work at the next stage, the groups of activities (components) are strongly interdependent. Strong couplings exist within the components where two or more partner contribute. In the converging sub-type, the work is organized in several parallel components; work starts with a joint, integrative activity where most partners contribute, such as the development of specifications for a software platform. However, this integrative activity is only a first step in the process, and the result of this activity does not have its "own life" after the end of the project.

The nearly decomposable architecture comprises three sub-types: weakly coupled, grid, and waterfall. They all consist of modules with strong couplings within them and resulting sets of collaborative outputs. The defining feature of this type of architecture is the presence of some degree of connectedness between parallel modules, integrative module(s) of different intensity, with their own output(s). The knowledge base is strongly integrated in the modules and loosely coupled between the modules. The workflow couplings are sequential in modules and loose between modules. The weakly coupled sub-type is composed of independent sequential modules where a limited number of partners collaborate, combining complementary knowledge bases within each module, and at least one finality per module.

The grid sub-type includes integrative module(s) that interact with other modules regularly at various stages: they are intended to bring actors together at regular intervals to work together throughout the project. The waterfall sub-type includes one important integrative activity which starts at the beginning of the project, then the output of this module is used in a "waterfall" of parallel modules. For the purposes of triangulation, the strength of the integrative module was not only estimated during the interviews, but also measured quantitatively, calculating reported person-month inputs of all partners in the integrative modules comparing to the overall volume of person-months in the project: it was 3.5% in PEGAGUS (weakly coupled), approx. 10% in ORION (grid), and approx. 20% in HERCULES (waterfall). This quantitative data thus confirmed the qualitative feedback that was collected during the interviews.

The decomposed architecture follows a completely modular pattern: similarly to the weakly coupled architecture, the project is decomposed into parallel modules, or sub-projects, often running throughout the project, but there is very little connectedness between modules. Table 2.4 summarizes the main characteristics of the six sub-types of the project architecture. The typology of architectures is conceptually presented in Fig. 2.3. When the project is driven by a well defined main technological output, then it is similar to product development projects, and the project architecture is workflow-integrated. When the outputs are 58 ambiguously defined and numerous, there are several options to structure the project but a nearly decomposable architecture is advantageous, with different strengths of the integrative module(s). In the studied projects, nearly decomposable type of architecture was chosen when several industrial actors wished to develop innovative solutions for their specific needs, or when several scientific actors wished to develop, compare and validate different approaches to similar fundamental problems. In the former case, the relatively strong integrative module provided opportunities to develop the concept and implement it in the modules, allowing for regular feedback loop. In the latter case, the weak integrative module provided opportunities for scientific interactions, linking the modules.

Implications of the types of architectures

During the second phase, we performed a deeper analysis of the effects of the architecture on project resilience, project management requirements and planned collaboration and knowledge integration. The findings show that most partners planned collaborations at the proposal stage, both by planning couplings with known actors, to continue historic collaborations, and by putting forward couplings with new collaborators: The project architecture is crucial for the realization of collaborations: The projects that were studied during the second phase are highlighted by grey color in Appendix 2.1. A summary of the implications of the different types and sub-types of architectures is presented in Table 2.5, then the following sections explain the table.

Workflow-integrated architecture: knowledge-disintegration but a collaborative final output, requiring high managerial effort and strong managerial skills

The workflow-integrated architecture (GEMINI, sequential, and contribute only to very specific tasks, often also limited in time, such as testing of a software.

The architecture facilitates collaboration within the components and between tightly interconnected components. The partners deepen their technical /scientific skills in their domain of specialization; individual work of partners may be used by them after the project end, separately from the main project output. The variety of knowledge integration opportunities is low.

"We firstly discussed about the specification document, this deliverable was done together. And then all partners worked on their side, just to develop things" (ID12_SCORPIUS_cons)

The architecture enforces collaboration of the whole team in terms of finality, in the sense that contributions of different partners are required to produce the main output, which is the result of the joint work. When an output of one group of partners is taken as an input by another partner, then there is strong workflow coupling, but there is no or little knowledge flow coupling. The resilience of the project is low: delay or failure in one of the major sequential components result in a major project shock.

"At the proposal stage, you are concerned about your own workpackage, you are pressed, well you do not pay a lot of attention to another work package. Then the project started, the deliverable of the first step was there, but the responsible for the next step looked at it and said 'okay this is fine but how am I supposed to start my work?'.

Then we realized something is missing, there was the gap, the output of the first workpackage did not become the input of the second workpackage" (ID1_GEMINI_coord).

The converging sub-type is even more risky: there is a high probability that all deficiencies in the previous stages of the project will ultimately harm at the final integrative stage. Thus, workflow-integrated projects require a high managerial effort with very competent monitoring of the work to ensure its timing and quality and, above all, to ensure that it matches with the needs of the subsequent stages. Such projects are suitable for coordination by consultants with strong managerial skills, but require a skilled and recognized technical development leader, usually the one who is behind the project idea. This type of architecture requires particular management attention during the proposal preparation: in case of failure of one partner with important tasks, as happened in one of the studied projects, another partner will have to assume the workload, otherwise the project would fail. The architecture is adapted for less complex projects when there is one main project output; it is similar to product development projects and the study found this type of architecture only in the projects which aims at the development of specific technology, such as new software platform.

Cutting just one essential coupling in a workflow-integrated project presents a major risk for the implementation of the project and puts strong pressure on the project management: continuation of collaboration is required even at high cost for management and for partners.

Thus, workflow-integrated projects are not resilient from the point of view of workflow, but they are resilient in terms of collaboration.

Nearly decomposable architecture: several modules with variety of collaborative outputs; the intensity of integrative module sets requirements for strong scientific/technical competences of the project manager

In the initial sample of 11 projects, the nearly decomposable structure was found in 6 projects.

The modules in all nearly decomposable architectures in the projects studied during the second phase (PEGASUSweakly coupled, ORIONgrid, and HERCULESwaterfall)

involve small groups of partners with complementary knowledge.

The weakly coupled architecture is beneficial to those participants who understand the topics dealt with in other modules without being involved in them, i.e. those with more experience and broad foundations:

"They did a lot of really hard research which I never understood in their presentations… this sounds good but whatever they are doing, I had no idea (smiles)..." (ID8_PEGASUS_ind). "The gluing part there were the universities. Companies always talked with universities. They were not linked between each others in general (ID23_PEGASUS_res).

The grid sub-type, thanks to planned elements of joint work, provides regular opportunities for exchange and knowledge integration, these elements help to get cross-fertilization between the modules, and act as a glue: their loss would drive the project towards full decomposition into independent modules.

The integrative activity in waterfall sub-type of architecture start at the beginning of the project and deals with a "challenge", e.g. a development of a new concept with planned contributions of all or most partners. It helps to deal with ambiguity and requires frequent interactions, especially at the beginning. New knowledge is then further deepened throughout the project thanks to the feedback from specialized modules, where partners work on e.g. new technologies and applications, and the integrative activity leads to a project output that has a "life on its own" after the project end. Thus, the waterfall architecture offers a variety of knowledge integration opportunities and especially favors the development of joint understanding between partners.

"There was from the very beginning a common theme and a common idea that we wanted to pursue. That helped a lot to draw the different ideas together. Some things have to brew for a while, and then all of a sudden the ideas are there and you can implement the ideas"(ID6_HERCULES_ind).

"There was a bigger block of work where an unknown problem had to be solved, a problem where a solution was not known had to be solved by the consortium as a whole. It was a core part of the design. We had to develop this thing, and create a joint understanding. That was an aspect which I have not seen too often in projects, that virtually all partners had to get involved in the creation of something, and this is the creation of concepts and ideas brainstorming about them...And it went through the project from day one to day final" (ID36_HERCULES_compar_res)

Comparing with the waterfall, the weakly coupled and grid architecture did not require knowledge integration at the project level at the beginning, as the concepts used in the projects were known and clear to the actors.

The resilience of weakly coupled and grid sub-types of architecture is relatively high from the point of view of the overall project viability: cutting of one module still allows the project to achieve the larger part of its objectives. But collaboration is much less resilient. For example, competition between actors working in one module suddenly appeared during the course of the ORION project: planned collaboration broke and the planned module disappeared.

The setup of the project was one reason why it was strongly suggested that we should step out. … And (a partner) becoming a competitor, that was something that… I really had no influence on. (ID33_ORION_large ind)

However, although the break in collaboration was regretful, it did not have major influence on the overall project outputs: the module in question was one of many project modules.

The waterfall architecture is more vulnerable at the initial stage than other nearly decomposable architectures, because of the strong links between the initial integrative module and the "waterfall" of further modules: in the studied project, the initial integrative module did not comprise tasks with a high technical risk. Some deficiencies of contributions to the integrative module (e.g. some non-contributing partners) did not prevent the module to continue.

In all studied nearly decomposable projects, the integrative module provided planned knowledge integration opportunities (of different intensity) for different project partners, and strengthened the collaboration in the project in several ways: it brought partners closer together that otherwise would only work in small groups, it widened the horizon of the partners by forcing them to unify concepts and ideas of an overarching nature systematically, and, because of joint work requirement, it 'pushed' for integration of knowledge coming from various backgrounds and applications.

"Most deliverables were not produced just by one partner, but by a group of partners, even if one partner coordinated the deliverable. We contributed to the deliverable, criticized them, gave them ideas, and the other way around, we have got very good input from other academic partners." (ID23_PERSEUS_res).

"It (integrative activity) forces partners to really do something together, not just sit and listen. With the meetings as such, there is a certain randomness and arbitrariness of these interactions. When there are tasks when partners have to work together, the exchanges are much more deeper, richer... If there is a task that you have to do, it is easier to mobilize people."( ID43_ORION_res).

Collaboration and management in weakly coupled projects are easy, as the modules mostly run throughout the project and they have their own leader. The grid and especially the waterfall sub-types require higher managerial efforts and strong scientific or technical competence of the project manager, to keep the project on track. This role is more adequate for an academic or industrial partner who is an expert in the project topic, with strong leadership skills. Significant efforts from the partners are required too to "keep alive" the integrative module.

"Work between partners and across components, this requires quite a bit of effort and coordination, so that it doesn't fall apart. That effort sometimes may be underestimated, managerial efforts but also efforts of the partners." (ID7_ORION_compar_ind).

Decomposed architecture: knowledge-integration within modules but workflowdisintegration between the modules; easy project management but little added value from the large inter-organizational setting

The decomposed architecture significantly limits the collaboration across modules, but it provides a setting for strong collaboration of small groups within specialized modules:

"We did not have scientific partners who could exchange over the boundaries of the modules because the modules were very specialized"(ID15_PERSEUS_res). The topics of the projects were too far apart, people do not have much in common" (ID28_PERSEUS_ind). "It was work on our own, nobody understood what we are doing and nobody was interested in it. It was just for interest of us and our partner" (ID43_PERSEUS_res)

When one partner organization, usually a research organization, provides input to several parallel modules, it may seem that this will help to make the project more integrated.

However, this is not always true: the study found several examples when a research lab employs several staff, each of them working in different modules of the project with different partners on independent topics and with little interaction.

The decomposed architecture is highly resilient at the project level: in case of disruption, only one module will be affected, the other modules continue as planned. However, the modules by themselves are not resilient: the collaboration partners, usually from academia and industry with complementary knowledge, are interdependent during the whole project, and the success of the module critically depends on collaboration quality and on sufficient inputs of all partners involved. The leaders of the disintegrated modules manage the work in their modules. If the manager is a consultant not experienced in the technical domain of the project, then a decomposed project is the easiest type of project to manage.

"It's a very convenient way to run projects because then each industrial partner, each end-user, each technology developer can do what they want to do and that's basically it." (ID4_PEGASUS_res).

The added value of the large inter-organizational setting is low in projects with decomposed architecture:

"If I was a reviewer for proposal, a proposal like this, I would always reject. Because that's not the idea of an EU project…. If you organize the project directly in modules, it is obviously not designed for collaboration" (ID7_ORION_compar_ind).

Perhaps for this reason, the study uncovered that the fact that the project work is done in largely disconnected modules is often hidden in the proposals, by the introduction of "horizontal" workpackages in which "slices" of the modules are formally integrated under a common umbrella. The projects look integrated on paper and there were intentions to integrate knowledge, but the integration did not happen.

Discussion

The study has collected and analyzed rare and rich field data to contribute to project studies [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF] and to bridge between the management and innovation literature [START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018[START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF], as discussed next.

First, the research provided evidence that the interplay between activity coordination and knowledge integration has a structuring effect and defines the architecture of interorganizational R&D projects with multiple actors: both knowledge and activity flow shall be structured [START_REF] Rauniar | Knowledge integration in IPPD project: role of shared project mission, mutual trust, and mutual influence[END_REF], both shape patterns of interactions in the project and decrease uncertainties (Raab et al., 2009, Söderlund andSydow, 2019), and both actors (with their activities) and structures shall be considered in a duality in order to better understand inter-organizational R&D projects, none shall be put first [START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF]. For the first time, different architectures of inter-organizational R&D projects were identified and light was shed on the setting of inter-organizational R&D projects with multiple actors (vom Brocke and Lippe, 2015), using a network lens [START_REF] Steen | Projects and Networks: Understanding Resource Flows and Governance of Temporary Organizations with Quantitative and Qualitative Research Methods[END_REF]. Research explaining the differences between inter-organizational projects is still at its early stage [START_REF] Ahola | So alike yet so different: A typology of interorganizational projects[END_REF]; the study has proposed a typology of project architectures, consisting of three main types and six sub-types. The identified architectures do not depend on the number of partners or the domain of the project.

Second, the study shows that although the optimum architecture depends on the intended innovations and the interests of the partners [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF], for projects with specific characteristics there may be optimum structure. Two elements especially play a role: the characteristics of the final outputs, in terms of their number and the degree of their ambiguity, and the degree of ambiguity of the problem addressed in the project. If there is one main finality, it is well defined and the ambiguity is low, then the workflow-integrated architecture is suitable. If the ambiguity of the outputs is high and the outputs are diverse within an overarching setting, potentially driven by the interests of groups of partners, then the modular architecture (nearly decomposable or decomposed) is suitable, and there are more options to structure the project. If there is high degree of ambiguity at the beginning of the project and joint work is required initially [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF] to reduce it, then the waterfall architecture is the most adequate one. If the ambiguity is low at the beginning of the project, then direct division into distinct modules (e.g. weakly coupled or grid architectures) is possible, with different degree of connectedness, depending on the required level of knowledge integration between modules and knowledge sharing constraints [START_REF] Enberg | Enabling knowledge integration in coopetitive R&D projects-The management of conflicting logics[END_REF]. If there is no need for shared knowledge between the modules, e.g. if the outputs of the projects fall under a common theme but are not related, then the decomposed architecture is suitable.

In the sample of 11 projects analyzed in this paper, the nearly decomposable structure is represented strongly, and most projects are modular: this is not accidental but reflects the needs and the constraints of inter-organizational R&D projects with multiple actors from industry and academia. Extant literature argues that if the uniqueness of projects is low, many elements of the project architectures can be used from one project to another [START_REF] Ahola | So alike yet so different: A typology of interorganizational projects[END_REF].

This study went a step further to show that even when the uniqueness of projects is high, there are typical project architectures which can be used from one project to another. Thus, the study provides heuristics to structure inter-organizational R&D projects.

Brusoni and Prencipe ( 2013) have shown that different levels of complexity, uncertainty, and ambiguity of knowledge call for different types of couplings within an inter-organizational structure and that the couplings may evolve over time. The present study extends the findings in the complex, ambiguous and uncertain context of the projects with rigid structures, where interconnections of knowledge and activities may be partially independent [START_REF] Tell | Knowledge Integration and Innovation: A Survey of the Field[END_REF]: it has been shown that the project architecture can enable the evolution of couplings and it can help to cope with the low structural flexibility.

Third, the study provided insights into the implications of the project architectures. Some of them favor collaboration within the project through integration of workflow, others favor collaboration through integration of knowledge; the waterfall structure help in creating a common base of understanding at the very beginning. The workflow-integrated projects, where partners work for the same finality, provides considerably less opportunities for collaboration than other types of architectures, while decomposed projects consist of largely isolated clusters of knowledge and activities, and the value of the large inter-organizational setting for collaboration is low. Historic experience helps to facilitate collaboration [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF] but may conflict with the R&D nature of the projects, which calls for opening them towards new actors to look for new ideas and approaches, as the study shows. Different types of architecture put different requirements in terms of the role and cost of project management, and the need for scientific or technical skills of the project manager. As previous studies have shown in other contexts [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]Ahuja, 2008, Michelfelder and[START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF], the nearly-decomposable architecture is beneficial for the innovation performance of interorganizational settings. However, this architecture is demanding in terms of management efforts and requires not only management skills, but also specific scientific or technical skills, depending on the strength of the integrative modules. It is difficult to maintain the coupling via the integrative module and to make knowledge integration happen (Tiwana, 2008, Brusoni and[START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], but if the integrative module is maintained, it adds a lot of value to the project, as the study shows. The research has also contributed to a better understanding of how inter-organizational structures should be designed for resilience [START_REF] Linnenluecke | Resilience in Business and Management Research: A Review of Influential Publications and a Research Agenda[END_REF].

Disturbances in workflow-integrated project presents a major risk and this puts strong pressure on the project management to find adequate solutions and practices [START_REF] Kapsali | Systems thinking in innovation project management: a match that works[END_REF].

The research presented here has certain inherent limitations. The empirical evidence is limited to multi-actor inter-organizational R&D projects funded by the European Union Research and Innovation Framework Programmes, which have specificities such as a large number of organizations working together and a rather inflexible structure. It does not include projects of smaller size funded by the national programs. The study covers a certain selection of themes and projects; thus, it is possible that there are additional types of interplay between workflow and knowledge flow, leading to other types of project architectures that have not been found in this study. The paper focused on the project architecture and does not discuss the types of knowledge that is generated in the projects or the knowledge integration processes that operate within the projects. The collaboration quality also depends on the historic experience of the partners, the project context and the dynamics of the participating organizations [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF]. However, these factors play a limited role in the definition of the project architecture and therefore were not included in the analysis.

Conclusion

Contributions

The study aims at developing new perspectives on inter-organizational R&D projects with multiple actors by means of an empirical qualitative analysis of selected projects from the latest European Research and Innovation Framework Programmes. Making use of and bridging the research streams of project management and innovation studies, the study focuses on the interplay between activity coordination and knowledge integration. It suggests a typology of architectures and discusses the implications of each type of architecture on the execution of the projects. The findings show that the workflow-integrated architecture disintegrates the knowledge base, provides a lower collaboration potential and may require high management efforts, while a workflow-decomposed architecture allows easy project management but provides little added value from the inter-organizational setting. Nearly decomposable architectures offer the highest collaboration potential under contingent conditions.

The study provides contributions for theory and practice of project management. On the theoretical level, it leads to a better comprehension of how activity coordination and knowledge integration simultaneously influence the project execution and set requirements for project management. On the practical level, the study provides heuristics on the choice of the architecture and helps to understand the implications of the choice of the architecture. It provides guidance to the project managers, as well as to the funding agencies, about how to adequately structure collaborative R&D projects with multiple partners. Overall, the results of the study enhance the understanding about complex inter-organizational R&D projects with multiple actors, which are networked projects: the network of organizational actors collaborates in the framework of a project with the goal to integrate knowledge and develop innovations.

Future research directions

The findings of the study point to several exciting and relevant questions for future research.

To better understand inter-organizational R&D projects with many actors, it would be of interest to study in more detail the processes that operate within the projects. Couplings can be studied as a structure and as a process [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], and future research could also help to better understand the processes and mechanisms that influence knowledge integration in this empirical context. Also the historic experience of the actors, the project context, the dynamics of the participating organizations [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF], the coordination and cooperation processes as well as the governance mechanisms [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF] shall be taken into account.

The study found that some connections and knowledge integration were planned but did not happen, resulting in a reconfiguration of the couplings. We investigated 4 projects with 54 organizations generating 46 innovations, using data from over 740 pages of documentation and over 33 hours of semi-structured interviews with 24 project actors. The study uncovers eight types of reconfigurations of couplings, activated by six disintegrative and two integrative mechanisms. Most of these reconfigurations led to negative consequences for innovations or to innovations created by less partner organizations. The weakening of couplings was much less pronounced within specialized modules (subgroups of closely collaborating partners) within the projects than at the interfaces that were planned to connect the modules. Here, planned strong couplings often degraded into weak couplings or even disappeared over the project lifetime. Based on these findings, we propose a process model of the evolution of couplings in multi-actor R&D projects and implications on collaborative innovations.

Keywords: multi-actor R&D project, evolution, coupling, module, interface, collaborative innovation, process model, qualitative research

The evolution of collaboration in multi-actor projects: Patterns of couplings and collaborative innovations

Introduction

Collaboration increases the potential for knowledge generation, thanks to the variety of the knowledge base and of the combination possibilities [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]. Over the last decades, research and innovation have become more open, more collaborative [START_REF] Chesbrough | Open Innovation: The New Imperative for Creating and Profiting from Technology[END_REF], and also more project-organized [START_REF] Tiwana | Do bridging ties complement strong ties? An empirical examination of alliance ambidexterity[END_REF]. Also public funding has shifted to support collaborative innovation in the form of inter-organizational R&D projects with multiple actors. In such settings, a group of multiple organizations (actors) works together towards a common objective within the framework of a joint plan, and develop both new knowledge and innovations (e.g. Spanos andVonortas, 2012, Goldstein and[START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF] which are market upstream. These projects often address ambiguous, uncertain and complex problems [START_REF] König | A framework for structuring interdisciplinary research management[END_REF]. Characteristic properties of such projects are bottom up self-organization in response to competitive calls with public funding, requirement for complementarities of actors, usually from academia and industry, collective responsibilities, limited power of the project manager (vom Brocke and Lippe, 2015), and a lack of structural flexibility due to rigid budgets, time frames and consortium membership.

In the context of such projects, collaboration -joint work to achieve a common purposesignificantly contributes to the project success, and goes in line with dynamic knowledge integration, "the purposeful combination of specialized and complementary knowledge to achieve specific tasks" (Tell et al., 2017 p.5). The more specialized the knowledge becomes, the more there is a need for dynamic knowledge integration: this is critical in interorganizational settings [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF]. That is one of the reasons why inter-organizational R&D projects are difficult to manage [START_REF] Söderlund | Pluralistic and processual understanding of projects and project organizing; towards theories of project temporality[END_REF]Tell, 2011, Lin et al., 2019).

In the presence of multiple actors, the way in which the collaboration is arranged, or the pattern of interdependencies between actors [START_REF] Capaldo | Network structure and innovation: The leveraging of a dual network as a distinctive relational capability[END_REF], is often called architecture (e.g. [START_REF] Fjeldstad | The Architecture of Collaboration[END_REF], and is defined in terms of the strength and intensity of couplings, between the nodes in the network, or actors [START_REF] Orton | Loosely coupled systems: A reconceptualization[END_REF]. Couplings relate to flows of knowledge and flows of activities [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF], they can be studied as a structure and as a process and may evolve over time [START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF].

The abundant literature on processes in inter-organizational projects, ecosystems and networks mostly investigated rather flexible inter-firm settings, in most cases with one focal actor (e.g. Johansson et al., 2011, Davis and[START_REF] Davis | Rotating Leadership and Collaborative Innovation: Recombination Processes in Symbiotic Relationships[END_REF]. The processes in the empirical context of multi-actor R&D projects are not well understood: there are multiple calls for research to explain the settings and processes of collaborative R&D projects (Calamel et al., 2012, vom Brocke and[START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF] and how patterns of couplings influence collaborative innovation over time (Michelfelder andKratzer, 2013, Hofman et al., 2016).

The present study addresses this research gap, using a theoretical framework which builds on the literature on project management, and additionally mobilizes innovation studies. This is in line with the call of scholars for a better connection between the project management and innovation research streams [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018), especially when the focus is on projects aiming at knowledge integration to generate new knowledge [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. There are multiple recent calls for research to explain the processes of collaborative R&D projects (vom Brocke and Lippe, 2015) and to improve our knowledge about how patterns of couplings influence the creation of collaborative innovations over time (Michelfelder andKratzer, 2013, Hofman et al., 2016); towards this goal, both the project architecture and the processes should be analyzed (Aubry and Lavoie-Tremblay, 2018). In line with these calls, our interrelated research questions are:

1) When, why and how does the evolution of couplings happen, over the lifetime of interorganizational R&D projects with multiple actors? (2) How does it influence the collaborative innovations that are generated by the projects? The evolution of couplings is understood as the reconfiguration of the planned couplings between organizations over time.

We put in place an empirical inductive comparative multi-case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF], and adopted a process view [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF][START_REF] Abdallah | Performing Process Research, Standing on the Shoulders of Giants[END_REF][START_REF] Feldman | Making process visible: alternatives to boxes and arrows[END_REF] using multiple levels and units of analysis, showing the "when", "why" and "how" evolutions of couplings happened over time in inter-organizational R&D projects with multiple actors funded by the European Union's Research and Innovation Framework Programmes. The latest one, Horizon 2020, has provided 34,42B€ (2014-2020) to fund 6288 multi-actor R&D projects with 14.1 organizations per project on average (EC, 2021a).

We selected and studied 4 projects with 54 organizations and 46 innovations, on the basis of over 740 pages of project documentations and data from 32 semi-structured interviews with planned couplings, on their six integrative and two disintegrative generative mechanisms, leading to different implications on the planned collaborative innovations. All but one of these evolutions lead to negative consequences for collaborative innovations or, at best, to preserving the planned collaborative innovations. We also uncovered that envisioned strong couplings at the interfaces between modules in the projects systematically tend to be weakened during the course of the project, resulting in a downscaling or loss of collaborative innovations relative to what was planned. Based on these findings, we propose a process model of the evolution of the couplings in multi-actor R&D projects.

This research improves our understanding about complex inter-organizational R&D projects and contributes to an ongoing conversation in the project management and innovation literature. It also provides practical implications, suggesting to the funding agencies and project managers how to organize inter-organizational multi-actor R&D projects and how to maintain or even increase the planned innovation outputs.

The remainder of the paper is organized as follows. Firstly, the theoretical framework guiding the study is provided, followed by the research setting and design, presentation of the findings and discussion, theoretical and practical contributions. The paper concludes by a summary of the limitations of this research and future research directions.

Theoretical framework

Innovation and knowledge integration in inter-organizational R&D projects

Since the seminal work of [START_REF] Schumpeter | The Theory of Economic Development[END_REF][START_REF] Schumpeter | The Theory of Economic Development[END_REF], scholars often view innovation as the novel recombination of ideas, resources and knowledge for value creation [START_REF] Nonaka | A Dynamic Theory of Organizational Knowledge Creation[END_REF][START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF][START_REF] Dibiaggio | Knowledge integration and vertical specialization in the semiconductor industry[END_REF]. The innovation process includes the step of generation of new knowledge [START_REF] Boisot | Integrating modernist and postmodernist perspectives on organizations: A complexity science bridge[END_REF], which often requires exchange and combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF] between the elements of the knowledge base, which in turn depends on a number of factors, including the opportunity for exchange or combination, the anticipation of valuable collaborative outcome [START_REF] Moran | Theories of Economic Organization: The Case for Realism and Balance[END_REF], cognitive distance [START_REF] Nooteboom | Optimal cognitive distance and absorptive capacity[END_REF], and others.

Collaboration significantly increases the potential for new knowledge generation, thanks to the variety of knowledge base [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]Ghoshal, 1998, Ahuja, 2000). Thus, over the last decades, research and innovation have become more open and more collaborative [START_REF] Chesbrough | Open Innovation: The New Imperative for Creating and Profiting from Technology[END_REF] and is often organized on a project basis [START_REF] Tiwana | Do bridging ties complement strong ties? An empirical examination of alliance ambidexterity[END_REF], where projectrelated constraints bring additional dynamics to the setting (Lundin andSöderholm, 1995, Gulati et al., 2012). Multi-actor R&D projects offer opportunities for academic and industrial actors to jointly address problems which are ambiguous, uncertain and complex [START_REF] König | A framework for structuring interdisciplinary research management[END_REF] and to work towards generation of both new knowledge and innovations [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF], where innovations are usually market upstream and may take different forms: new products, innovative technologies, deployments of innovations, processes, methods, software, tools, patents (e.g. [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF]. Such projects, which are often publicly funded through competitive calls, are different from contractual partnerships by their diversity of actors, high complexity of connections between actors, autonomy and equality of partners and limited power of the project manager (vom Brocke and Lippe, 2015, [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF].

In the context of collaborative research and innovation, the generation of collaborative innovations requires dynamic knowledge integration, defined as "the purposeful combination of specialized and complementary knowledge to achieve specific tasks" (Tell et al., 2017, p. 5). Knowledge integration requires adequate integration mechanisms [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF] such as rules, sequencing activities, organizational routines, group problem-solving, formal and informal interactions (e.g. [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF]. Knowledge integration in interorganizational settings is critical [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF] and it is an interplay between coordination, or alignment of the knowledge of the organizations, and cooperation, or alignment of the interests of the organizations and their relationship [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF].

There has been a call for future research at the intersection of understanding the underlying processes and explaining the settings and processes of projects that aim at the generation of innovations [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF].

The quality of collaboration in projects, or joint work to achieve a common purpose, determines collaborative outputs (e.g. [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF], it is influenced by several factors [START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF], e.g. the historic experience of collaboration [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF][START_REF] Belderbos | The antecedents of new R&D collaborations with different partner types: On the dynamics of past R&D collaboration and innovative performance[END_REF] or the dynamic project context: "no project is an island" (Engwall, 2003, p.789).

However, new knowledge generation and innovation in multi-actor R&D projects benefit from joint work with new collaborators, without historic experience [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF].

Modularity and couplings as a way to manage project complexity

The potential to generate new knowledge is affected by the structure of the knowledge base [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]). An important concept here is modularity, or decomposability [START_REF] Simon | The Science of the Artificial[END_REF]. Baldwin and Clark (1997, p.86) defined modularity as "building a complex product or process from smaller subsystems that can be designed independently yet functions together as a whole". Simon's seminal theory of complex system and near decomposability put in evidence that complex systems with a nearly-decomposable structure adapt themselves to the demand of their environment more easily than non-decomposable systems. In the organizational setting, modularity reduces the interdependencies between modules or actors, adds robustness and helps to manage complexity [START_REF] Sorenson | Complexity, Networks and Knowledge Flow[END_REF]. On the other hand, too high modularity reduces the potential for collaboration and knowledge generation [START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF] and has other downside effects (Brusoni, 2005, Yayavaram and[START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]. So modularity is often counterbalanced by integration (Brusoni, 2005[START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF]. [START_REF] Siggelkow | Temporarily Divide to Conquer: Centralized, Decentralized, and Reintegrated Organizational Approaches to Exploration and Adaptation[END_REF] studied organizational structures of firms and showed that sequential processes with temporary division and then integration yield the highest organizational performance in the long term. To achieve the planned output thanks to knowledge integration, combinations of strong and weak couplings are required [START_REF] Gulati | Meta-organization design: rethinking design in interorganizational and community contexts[END_REF]. In the inter-organizational setting, strong couplings connect collaborating partners intensely working together (Rost, 2011, Michelfelder and[START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF]. The organizational structures can also be analyzed from the angle of complexity: [START_REF] Wang | Complexity and the functions of the firm: Breadth and depth[END_REF] explained two types of complexity, relational and cognitive, and their 'depth' and 'breadth' dimensions: the former concerns the level of sophistication, while the latter concerns the level of heterogeneity. Both may be present in inter-organizational settings.

These findings are applicable to the project setting too: modularity means dividing the project into specialized modules, in which small groups of partners collaborate with the aim to develop well-defined joint innovations, facing cognitive complexity. The interfaces are transmodular, with larger collaborative groups facing relational complexity. Knowledge integration is especially difficult at the interfaces, if the diversity of accessible knowledge, capabilities and perspectives is high [START_REF] Tiwana | Do bridging ties complement strong ties? An empirical examination of alliance ambidexterity[END_REF]. Couplings reflect the flow of activities and the flow of knowledge between the project actors; at the project level, the pattern of couplings between the modules forms the project architecture; the presence and the intensity of trans-modular interfaces define the types of modular project architecture [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF]. In multi-actor R&D projects, couplings in the modules and at the interfaces represent collaborative work resulting in planned collaborative innovations and thus both are planned as strong couplings during the front-end phase of the project. It is known that strong couplings are unlikely to experience dynamic reconfiguration [START_REF] Borgatti | Network Theorizing[END_REF]. Still, the inherent uncertainty of R&D activities suggests that reconfiguration of couplings may take place: it is intrinsically linked to the evolution of collaboration which affects the collaborative innovation and thus the success of the project [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. In the context of multiactor R&D projects, it is not known where, when and why reconfiguration is likely to happen, and how it influences the planned collaborative innovations.

Project studies and innovation studies are both interested in research on projects in the innovation context, and scholars call for a better connection between the two research streams [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF][START_REF] Davies | When neighboring disciplines fail to learn from each other: The case of innovation and project management research[END_REF], specifically when studying projects with a focus on knowledge generation [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. This connection is salient in the context of multi-actor R&D projects. Scholars concur that a dynamic layer of processes is required [START_REF] Lundin | A theory of the temporary organization[END_REF] and that both "the thing" (structure, architecture) and "the process" should be analyzed (Aubry and Lavoie-Tremblay, 2018); there are multiple recent calls for research to explain the settings and processes of collaborative R&D projects (vom Brocke and Lippe, 2015) and to improve our knowledge about how patterns of couplings influence the creation of collaborative innovations over time (Michelfelder andKratzer, 2013, Hofman et al., 2016). In addition, the process studies call for more complex 'conjunctive theorizing', making analytical distinctions and joining up concepts [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF]Langlay, 2020, Tsoukas, 2017).

The present study was designed to answer these calls and to address the above mentioned research gaps. We pose the following interrelated research questions: 1) When, why and how does the evolution of couplings happen, over the lifetime of inter-organizational R&D projects with multiple actors? (2) How does it influence the collaborative innovations that are generated by the projects? The evolution of couplings is understood in this paper as the reconfiguration of the planned couplings between organizations within the project over time.

Adaptations of topics of work within the same coupling, or other situations when the couplings do not change, are not considered as reconfigurations of couplings in this paper.

Research setting

The research setting is defined as multi-actor inter-organizational R&D projects funded through competitive calls by the European Commission through its Research and Innovation Framework Programmes. These projects often comprise partners from academia, research institutions, and industry, and perform pre-competitive research, development and innovation.

Their specific features are bottom-up self-organization in response to competitive calls requiring European added value of collaboration, large number of geographically distant and heterogeneous organizations with collective responsibilities, and significant budget (von Brocke and Lippe, 2015). The projects usually last three to four years. In the proposal preparation phase, the partners agree on and describe the set-up of their collaboration. The period of proposal preparation is usually short, commonly a few months. If the proposal is funded, the grant agreement is set up on the basis of the proposal with very limited changes.

There are three reasons why this setting is particularly interesting for management and innovation scholars. First, multi-actor projects have become an important way of research and innovation funding. Second, because of the large number of heterogeneous partners involved:

in the period 2014-2020, there were 14.1 partners-organizations per project on average, with high diversity. Third, the relative lack of structural flexibility (the public funds are allocated to each individual partner) brings additional dynamics to the setting.

Research design

The research design is a qualitative inductive multi-case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. We adopted a process view, analyzing changes in couplings over time, and the generative mechanisms behind these changes [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF][START_REF] Abdallah | Performing Process Research, Standing on the Shoulders of Giants[END_REF][START_REF] Feldman | Making process visible: alternatives to boxes and arrows[END_REF], using multiple levels and units of analysis [START_REF] Langley | Strategies for theorizing from process data[END_REF] at the level of coupling, collaborative innovations, and project phases, showing the "when", "why" and "how" evolution of couplings happened over time.

This paper builds on the previous research [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF] which investigated projects with different project architectures. For the present paper, four projects with different types of modular project architectures were selected (Appendix 3.1), using criteria of homogeneity, variety, comparability and availability of data. These projects were comparable in terms of thematic, the modular type of the project architecture, and a level of experience of the participants in such projects. The projects comprised 10 to 17 organizations from academia and industry. 50-60% of the participants in each project had experience of collaboration with some partners. PEGASUS project was research-driven, i.e. driven by the interest of research teams, with long term application of results; the other three projects were industry driven, i.e.

focusing on industrial needs, with short and medium terms application of the results.

The study of the evolution of the couplings required to analyze how the couplings were planned at the beginning of the project, how they changed during the execution of the project, and what the resulting ("lived") couplings were at the end of the project. This paper focuses on strong couplings, i.e. substantial planned collaborations with own innovations. The study distinguishes couplings in modules (small groups of 2-5 organizations) and at the transmodular interfaces (with larger groups of organizations).

In the data collection phase, we collected data using both the project documentation and interviews with project actors. The project documentation included over 740 pages, and allowed us to identify the planned couplings, the collaborative innovations generated by the end of each project, and the contributing partners. Based on this information, the evolution of the couplings (patterns of collaboration) from the planning phase to the end of the project could be traced. A direct comparison between planned and realized innovations in contrast was not possible, as not all innovations were explicitly defined in the proposals. We reviewed and validated the data about the contributors to innovations with the project coordinator or one of the key project partners of each project. Also data from 32 semi-structured interviews with 24 project actors, resulting into over 33 hours of recording and 585 pages of transcripts (Tables 3.1 and 3.2) was used. In each project, representatives of between 40% and 65% of the participating organizations were interviewed. Informants were from large industry, technological SMEs, research entity); over 80% of the informants were CEOs of SMEs, team leaders within large corporations, or tenured university professors. 7 informants participated in more than one project from our sample, and the project manager was the same in 2 projects, thus adding to this research a comparative multiple-case dimension so far underrepresented in the literature [START_REF] Bakker | Temporary organizing: Promises, processes, problems[END_REF]. A detailed interview guide was developed, all interviews were recorded, with prior consent in line with the European General Data Protection Regulations. In the data analysis phase, using the data collected from the project documents, we firstly analyzed the project architectures, i.e. the main pattern of couplings at the project level, and their evolution, i.e. the "what" question. We identified the strong couplings as planned in the project proposals and at the end of the project; by comparison, we tracked the reconfiguration of the couplings in each project, both at the levels of modules and interfaces. Then, from the explanations provided by the project actors and their reasoning, we gained insights about the 'when', 'why' and 'how' the evolution of the couplings happened. The analysis of each project led to the definition of three phases, which were found in each project: Phase 1, the initial explorative steps, was the confirmation of concepts and deeper investigations of the needs and requirements, confirming the feasibility of proposed couplings and collaborative innovations;

Phase 2, the main R&D work, was about the development of collaborative innovations, Phase 3, the consolidation and demonstrations, was the demonstration of these innovations in the applications. The Phases varied in terms of length depending on the nature of the project, research driven or industry driven, and on the previous collaboration experience: for instance, the Phase 1 in PEGASUS took three months, while in PERSEUS and ORION it lasted almost a year. We identified generative mechanisms for each reconfiguration at each stage. The analysis of the interviews was based on the abstractive process of open coding [START_REF] Strauss | Basics of qualitative research: Techniques and procedures for developing grounded theory[END_REF]Corbin, 1998, adapted by Gioia, Corley, and[START_REF] Gioia | Seeking Qualitative Rigor in Inductive Research[END_REF]. The interviews transcripts were coded using the NVivo 12.1 software and a data structure was built. An analysis was done within each case (project), then the results of the analysis were compared across cases. During the research process, some new elements were added to the interviews, in response to the relations to theoretical foundations, e.g. on historic experience [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF]. Thanks to the combination of the interview data and documents, it was possible to identify not only the couplings but also their planned and realized intensity.

Based on these results, we then built a process model of the evolution of the couplings over time and its implications on project-based collaborative innovations.

Results

The results are organized in two parts: descriptive and explanatory. The descriptive part summarizes the identified types of reconfigurations of the couplings across the projects, together with their implications on the planned innovations. In the second, explanatory part, we describe for each project the reconfiguration of the couplings over time, elaborate on relevant generative mechanisms and factors, and propose a process model of the evolution of the couplings influencing innovations in multi-actor R&D projects.

Identification of types of reconfigurations of couplings

Evolution of the architectural patterns.

The studied projects are all characterized by a nearly-decomposable (modular) type of planned architecture (Fig. 3.1a), organized in parallel specialized modules connected by transmodular interfaces, often throughout the project duration, both presented in all project proposals as strong couplings with planned collaborative innovations; the same organizations contributed both in modules and at the interfaces, usually with different roles. There were two kinds of interfaces, the difference was in the way organizations were involved in the work at the interfaces: in the first type of interfaces, larger groups of partners drawn from different modules did joint conceptual work together with the purpose of developing technological foundations, unifying concepts, common approaches. In the second type, work was done by one or few organizations and then applied in different modules. HERCULES was a waterfalltype project: an activity at the interface, with involved all partners, was planned to start at the beginning of the project and to last until its end; its intermediary output was planned to be used in a subsequent "waterfall" in modules, with feedback to the interface. Other studied projects were planned as grid-type projects with strong interfaces (PEGASUS, ORION) or a weak interface (PERSEUS). At the end of the projects, the interfaces had evolved, and thus the project architecture had evolved as well. Table 3.3 shows that in all projects the evolution of the project architecture systematically happened in one direction, towards the weakening of the interface activities:

either due to the loss of contributions from collaborators to planned joint work, or due to the fact that outcomes of the work at the interfaces were used in less modules than anticipated or not at all. One project, PERSEUS, largely disintegrated (Fig. 3.1b): planned couplings at the interfaces between modules did not materialize, the collaborative activities were abandoned. The first result of the study is that the architecture in all four projects underwent an evolution in the negative direction at the level of the interfaces; only in HERCULES (waterfall-type project), the evolution was minor, the architectural pattern did not change.

Types of reconfigurations of couplings.

Couplings were reconfigured in all studied projects (Table 3.4). Some of these reconfigurations were accompanied by a significant downscaling of initially planned couplings. We identified five types of reconfigurations within the modules, and three of the interfaces, this includes reconfigurations which are the result of the response actions.

In At the interfaces, (6) downscaling, or significant degradation of a planned collaboration, was observed in all 4 projects: either planned contributors gradually "retreated" from the collaborative work at the interface or the main partner responsible for the work at the interface continued the work but the results were not connected to the modules as it had been planned in the project. In the extreme case (PERSEUS), downscaling led to a full (7) decomposition of the project into separate modules, see Fig. 3.1b: work at the interface disconnected from the modules. (8) Fluctuation was a result of the response actions to downscaling (HERCULES, ORION): partners decreased their input, but later some of them increased it again, so the collaboration intensity fluctuated over time, but the interface remained in place.

Thus, the second result of the study is that all but one types of reconfigurations observed in the four studied projects pointed in the negative direction (break or weaking of couplings) or, at best, to the restoration of couplings. In 4 projects with 54 participating organizations, only in one case the bottom-up integration within the project was observed.

Implication of the evolution of couplings on the planned collaborative innovations

The four studied projects generated 46 innovations (Appendix 3.2). Reconfigurations of couplings (and thus changes of the collaboration patterns) had different implications on the planned innovations. Negative implications included (1) full loss of planned innovations.

Moderate negative implications were (2) individual innovation instead of planned collaborative innovation;

(3) collaborative innovation as planned, but with less contributors;

(4) downscaled innovation, due to reduced input, but with the same contributors as planned.

Neutral implications were (5) another innovation developed (another option found at the project level), ( 6) preserved planned innovation with the same contributors, thanks to the managerial intervention. Finally, a positive implication was (7) new, unplanned collaborative innovation. Table 3.5, column G provides data on how the evolution of couplings influenced the collaborative innovations in all four studied projects.

In PERSEUS and ORION, after splitting, the expected collaborative innovations did not materialize as planned, but reconfiguration had different effects on the project partners.

Academia and SMEs found individual solutions: the academic partners did research which was not so closely related to the needs of partners but still related to the project topic and objectives. The SMEs invested the funding into own developments related to the project topic.

In both cases, innovations were generated individually. The large industrial end users with whom joint work was planned, advanced alone, if they had capabilities to do so (PERSEUS), or searched for other solutions outside the project (PERSEUS), sometimes accepted the situation (ORION), sometimes exerted pressure on the project partners and project management to find solutions, which led to the restoration of the initial configuration, thus preserving the planned collaborative innovations (PERSEUS).

Consensual refocusing preserved planned collaborations: partners reconfigured couplings according to where they expected the creation of useful innovations, consciously searching for best solutions for them and for the project, overcoming e.g. technical difficulties.

Decomposition at the interfaces of PERSEUS and ORION resulted in the full loss of planned collaborative innovations, but also in the generation of individual instead of planned collaborative ones. In case of downscaling that happened in all projects, the planned collaborative innovations were created, but with less contributors: we consider that this is a moderately negative implication, as less organizations were involved comparing with the plan. One bottom-up integration took place in ORION, which led to the generation of one unplanned collaborative innovation.

The third result of the study is that in the large majority of the cases, the evolution of the couplings overall had predominantly negative implications on the planned collaborative innovations. The study did not find evidence of negative implications of downscaling of the collaborations at the interfaces on collaborative innovations, i.e. when some planned couplings out of many were lost. Bottom-up integration had a positive effect on the number of innovations, but it happened rarely in the studied projects.

Uncovering the generative mechanisms of the reconfiguration of couplings: feedback from project actors

In this section, we firstly explain how the couplings evolved in each project over time, followed by a summary of the processes that are common in all four projects at the interface level. We conclude with a summary of generative mechanisms and factors across the projects.

Evolution of couplings in the projects over time: the "when", "how" and "why". HERCULES (waterfall-type project) mostly was not affected by disintegration, its architecture did not evolve: the main interface survived until the end and resulted in a substantial innovation, even if some planned couplings at the interface were lost. Only some downscaling and fluctuations were observed at the interfaces in HERCULES.

Phase 1 of HERCULES started with an interface activity, which continued throughout the whole project with significant planned inputs from the majority of the partners.

"There was a bigger block of work where an unknown problem had to be solved… we had to develop this thing, and create a joint understanding… virtually all partners had to get involved in the creation of something… the creation of concepts and ideas brainstorming about them...And it went through the project from day one to day final" (ID36_HERCULES_compar_acad)

Later the partners worked predominantly in modules that operated in parallel, were stable and were mostly based on previous collaborations, leading to specific collaborative innovations.

Downscaling of the interface started during Phase 1, with partners not contributing as expected and planned. As the result of the work was vital to the project, the project manager responded to the downscaling, some couplings within the interface were reinforced, the leading role was assumed factually by another partner than the planned one.

Phase 2 continued with similar the changes as introduced in Phase 1 and some minor downscaling at the interface level. The project leader continued to push less contributing partners for more involvement, the fluctuation of the couplings at the interface continued.

"In the very first meeting …, it was apparent that when some of the industrial partners came in, they had their own objectives which only met some of project ideas… At the beginning of the project it seemed for a while that they were just doing their own thing…because it wasn't very much in their interest at the beginning. In the end it turned out, with a bit of pressure I would say from the management team… in the end they did work" (ID6_HERCULES_large ind)

In Phase 3, the main interface was stable: more partners bought into the central ideas than at Phase 2. The interface activity was the "anchor point" of the project and shaped its identity.

"Within the project, I think, there was a real progress … some of the partners tried to develop specific … solutions, others like us, we developed generic solutions…. but there was an anchor point. And we don't have an anchor point like this in ORION… the approaches are totally different (ID26_compar_SME)

There were several reasons for the stability of the couplings, as observed in HERCULES (Fig.

3.2a

): (a) sequential near-decomposability, with an important interface activity starting from the beginning of the project; specific work then happened in the parallel modules but with regular interactions between them, which was facilitated by the medium size of the project;

(b) work at the main interface did not comprise high-risk tasks and had the goal to create a joint understanding and approach between the partners; (c) the proposal remained broad about the activities at the interface, giving room for a variety of contributions especially at the initial stage, (d) the project manager counterbalanced tendencies for downscaling at the interface. "it was a healthy partnership and the expectations I think were met on both sides... the project designed itself: we had four academic groups that were developing along four different lines of research… the design perfectly fit to the interests of the participants" (ID2_PEGASUS_coord)

Phase 1 ran as planned. Bifurcation in modules leading to splitting of two couplings took place during Phase 2: in one case, there was no response action, as it became clear that the hopes for collaboration of the partners would not materialize, one of them pursued mainly own interests, the combination of two different approaches was difficult, and as a result, only light interactions happened instead of intense joint work. In another case, a partner pursued own interests too far and started splitting, then a restoration action was undertaken:

"We had one point in the project were one partner … was going in a direction which was relatively unrelated to the rest of the project, we corrected that, had a discussion, and they refocused that work."ID2_PEGASUS_coord

The need for managerial response action was confirmed by another actor of PEGASUS:

"You cannot really define everything in the proposal. You set up sort of a structure, you have a vision, and then let it run and at some points there will be some clustering of people who have the same minded interests, but at some point, you still need to try to get them to contribute to the whole vision…". ID9_PEGASUS_SME

Downscaling of the interface started in Phase 2, when the R&D work advanced. Although partners confirmed interest at the proposal stage to work at the interface, it did not provide sufficient benefits in the perception of the partners during the project phase. The project manager considered the interests of the companies to be too diverse, and did not undertake responsive actions; the planned solid grid interface became weak:

"It turned out that the assumptions behind the design were between an idealistic okay… some elements that were supposed to be collaborative in the end turned out to be additive… everybody was too busy in solving his or her own problems and so the integrative aspect was like a bit of …how can you say that 'nice to have but not really the essence' (ID2_PEGASUS_coord)

Phase 3 was stable, couplings followed the plans, the previous corrections or downscaling of one major technical interface activity:

"It (NB: interface work) partially indeed happened, let us say fifty percent" (IDsupp5_PEGASUS_coord)

To sum up, the couplings in PEGASUS were established mostly as planned (Fig. 3.2b), collaborative innovations were developed mostly as planned, but the intensity of the collaboration was reduced at the interfaces.

PERSEUS (planned as weakly coupled, de-facto decomposed project) experienced many problems and all types of reconfigurations of couplings except bottom-up integration and fluctuation. As an underlying cause, the proposal had technical flaws: a lack of deep analysis was systematically mentioned by the informants, the descriptions of work remained at a too high level and the technical challenges were not analyzed properly: Other roots of disintegration were, for instance, overestimations of capabilities of partners:

"Well I think we overestimated Partner X. Clearly they … didn't have the resources to work on the different cases, okay, it was seen later on.. They underestimated the technical challenge, or put it the other way, they sell you that they are capable while they didn't even know themselves (ID28a_PERSEUS_large_ind)

During Phase 1, it quickly became clear that technical work in the modules was too demanding or even impossible. These technical difficulties quickly caused abandoning of some planned couplings, or consensual refocusing of other couplings, e.g. when one of the large firms changed priorities, which resulted in the need to adapt the couplings in the modules. In addition, force majeure happened during Phase 1 with the split-up of a large industrial partner; this external pressure led to abandoning, splitting and consensual refocusing of couplings:

"… (partner name) went quickly into internal reorganization… then the pilot was not available anymore, and all the rules changed … yeah it was a surprise" ID31_PERSEUS_res)

Discovery of technical barriers continued during Phase 2, leading to abandoning:

"It became apparent that in some cases it will not work because requirements do not fit the possibilities... And that was after one year. And then it was some kind of head scratching problems …, what direction are we going to go…" (ID31_PERSEUS_res).

Also, cooperation issues caused by selfish interests or shifting needs of large industrials became apparent during Phase 2; together with unrealistic expectations this led to splitting, the dependent partners could not continue the planned work. Downscaling at the interfaces started in Phase 2 and was partly caused by low levels of technological relatedness between the modules, and partly because priorities of organizations were not fully in line with the activity at the interface, they had to deal with other problems in the modules of the project. Organizations retreated, the project disintegrated: the responsibles for the interfaces continued the work alone but it was not applied in the modules.

Organizational factors played a role especially in Phase 1 when planned couplings should have been put in place; personal factors played a major role afterwards, in Phases 2 and 3…

"When I wrap up everything, eighty percent was industrial culture, company culture and personal feelings and personal decisions, and only twenty percent were technical issues. (ID15_PERSEUS_res)

…resulting into splitting, in several cases, and consensual refocusing. These factors played an important role at the interface level and led to the retreat of organizations from the interface.

External pressure from the main organization (research partner), related to technical challenges and the lack of budget, led to the attempt of abandoning another crucial coupling in the modules during Phase 2, but the pressure from both the dependent partner (large industrial) and the project manager activated the integrative mechanism, and at least partly restored the link, avoiding the abandoning of the coupling:

"I had to intervene and threaten them, because many people other partners were depending on their work, and if they stopped working … then we wouldn't achieve our goal. Well the only option I have to make pressure on them" (ID11_PERSEUS_coord)

External pressure from the funding agency, e.g. reporting needs and review, has been generally considered positive integrating factor in PERSEUS (and in other projects), and also helped to maintain the planned couplings during Phase 2 in spite of technical difficulties, and to search for solutions. Phase 3 was characterized by stable couplings: reconfigurations had taken place during the previous phases, including decomposition of the interfaces. Fig. 3.3a

shows how the coupling in PERSEUS evolved over time. ORION (planned as grid, de-facto partial grid) experienced all 9 types of reconfigurations of couplings. ORION also had weaknesses in the proposal, but these were of different nature comparing to PERSEUS: particularly the role of partners was not fully defined, as the challenges were not fully clear in the planning phase and thus not all opportunities for joint work materialized according to the plan. Moreover, due to lack of pre-existing collaborations, capabilities and capacities of partners were over-estimated.

"My assumption was that we would be working with all the use case providers. I think in theory that would have been possible…. But when we were talking about it within the consortium…we very quickly decided that it would be very difficult to try and work with all of the use cases, and that it would be better to focus on one two or three and do something really good there rather than try to … you know spread too wide". ID25_ORION_SME

Phase 1 was dedicated to the confirmation of couplings and some reconfigurations quickly happened. Abandoning and splitting were related to technical barriers… … or because some departments of large firms were not consulted during the proposal phase:

"We have to implement our software and use their data and they don't want us to extract the data from their database… for confidential reasons, and also, yeah, because a software is not approved…. They need to approve our software …. They tried a lot… to give us access to the system, but the IT did not allow. And so we cannot do anything. " (ID38_ORION_SME)

Consensual refocusing massively happened in Phase 1, taking the overall project needs into account and putting higher priorities on some activities, in some cases up to refocusing all efforts from a planned to an unplanned coupling, in consensus with involved parties.

In Phase 2, cooperation issues started to appear, related to unrealistic expectations, technical problems and selfish interests of partners. This initiated splitting, individual work instead of the planned coupling. This is how splitting was explained by the two involved sides:

"They were not interested… We tried to push it at several meetings, they don't want to help us with achieving the project goals. I asked them once, when they gave presentation, what the presentation has to do with the project… We were very disappointed" (ID40b_ORION_large ind)

"They had higher expectations... they had a number of questions that are very complicated to answer and very problem-specific, so you have to know a lot about these problems to actually answer these questions." (supp1_ID37_ORION_SME).

The perseverance of partners played an important role to avoid splitting of couplings: Also in Phase 2, force majeure -appearance of unexpected competition at the level of two large companieshappened. One organization even wanted to leave the project completely, but due to the pressure of the external environment and interventions from both the project manager and internal project staff, this was prevented, and some couplings were restored while other couplings planned for this organization, were abandoned:

"There were troubles down the line because the use case had to change. And although it is frustrating , it did set our work back, there's nothing that anybody could have done about that, nobody knew it was going to happen… that is very unfortunate. ID25_ORION_SME

The end of Phase 2 and especially Phase 3 were characterized by significant reconfiguration at the interfaces level: organizations de-invested from joint activities which did not bring evident benefits for them, or were too complex, and there were too many important issues in the modules to deal with. In other cases, work at the interfaces was used in less modules than anticipated, causing repeated complaints from the interface responsible. The grid lost its planned strength; downscaling or even decomposition of some interfaces were observed, the interface was too weakly planned:

"From the point of view of tackling overarching problems together… it did not work, maybe we could have designed this activity more strongly in the proposal… that probably was a difference between the work in HERCULES and the work in ORION" (ID36b_comparative_acad). … combined with open-minded actors, on the organizational and on the personal level:

"At (partner X), I will have an idea, and they said: well, yeah, that's right. And the other (partner Y) say: yeah (whispering), well, well, I have to check it … some organizational structures in the companies make it more or less helpful. And the main point is personal… it's always the main point. ID26b_ORION_SME

These opportunities for reconfigurations of couplings were only possible when coupled with other factors, such as match of the skills of involved partners, budget availability, and, with regard to the industrial partners, support from decision-makers.

ORION was the only project when unplanned bottom up integration took place in Phase 2:

"It was not planned, but it happened… they sent the paper… this paper came to our attention, as the consortium.

And then I realized, yes, if I want to do something, but of course much more extended and more realistic focus, I need to talk to them." (ID34_ORION_acad)

The study uncovered that when the reconfiguration opportunities were present with positive implications on the collaborative innovations, some industrial partners in ORIONboth small and large firmsdid not seize them: the project participants were either under too much pressure to "have eyes open", did not had enough power within the main organization to put proposals forward, or (in one case) were newcomers to the funding scheme and lacked understanding that adaptations are possible even when the setting is not considered flexible: Reconfigurations in the modules took place also in Phase 3 because previous work in modules did not led to the expected results, and partners searched for additional opportunities.

On the positive side, in two cases in ORION work 'bifurcated' from the modules to the interface, allowing additional generic developments. This did not happen in other projects.

Evolution of couplings in ORION is shown on Fig. 3.3b.

To summarize, the study found six At the level of the interfaces, the most important factor was pressure (of the partner organizations, of the project constraints such as time and budget…), which drives the partners to allocate resources to their main activity that are important for them, consequently partners pay less attention to or abandon contributions at the "nice to have" interfaces. We also observed that in all projects knowledge integration mechanisms were used to a high extent, especially technical exchanges in modules, and according to the project actors, the depth of collaboration significantly influenced the level of the resulting innovations (quality, maturity), but we did not find evidence that they prevented reconfiguration of couplings.

Two generative mechanism activated changes of couplings in a positive direction both in modules and at the interfaces, leading to either reconfiguration or preservation of couplings:

(1) pressure, both internal and external;

(2) proactive action of one of the involved actors, combined with motivation and open-minded attitude of those involved in the couplings, their organisations and the project manager, a relatively broad description in the technical proposal that enabled to react flexibly, and the availability of the right blend of competences required for the reconfiguration to be put in place.

3.5.4 Relational complexity at the interfaces versus cognitive complexity in the modules.

In all projects, the interfaces between the modules were planned and presented as strong couplings in the proposal, with own innovations. As explained above, HERCULES, the waterfall-type project, was the only one in which the main interface was almost not affected by downscaling. In the three other projects, the interfaces systematically downgraded: there were planned strong couplings, for instance first development of innovations and then application in modules, but only the first step happened. In some cases, the developers only were interested in their development, in others it did not work as broadly as assumed or the results simply were ready too late. All in all, the uniting elements detached, and interfaces got significantly weaker, leading to full disintegration of the project with individual partners continuing the work individually (PERSEUS) or to considerable downscaling of the interface (PEGASUS, ORION), with project actors de-investing from joint activities in favour of collaborative innovations in the modules.

One reason for this -even in cases when high technology relatedness was assured between the modules -was the inability of the participants to deal simultaneously with the relational complexity 'in breadth' (interfaces) and the cognitive complexity 'in depth' (modules). It turned out that at the proposal writing stage, large consortia with many organizational partners were created, following the implicit expectation of the call. Large consortia were organized in modules, driven by the interests of small groups of partners. However, in order to demonstrate the value of the partnership (a requirement of the calls), interfaces were proposed, where many partners were expected to contribute: such work at the interfaces generated high relational complexity. At the project implementation stage, the cognitive complexity took its toll: under the pressure from the project constraints, partners prioritized the work and focused on couplings in the modules. As a result, in PEGASUS, the planned output was formally developed at the interface but with reduced investment; in PERSEUS, planned collaborative innovations were not created at the interfaces, in ORION, collaborative innovations at the interfaces were downscaled.

We conclude that under pressure, the activities at the interfaces are the first to be dropped, and the interfaces have higher chances to last until the end of the project and to lead to collaborative innovations if (1) they are planned in the proposal and start from the beginning of the project; (2) significant allocation of inputs is planned for the work on the interface; (3) this work and its result is useful for the whole project and for most partners, and not an artificial add-on or "nice to have" element; and (4) there is strong pressure from the project management to keep the work at the interface alive and sound knowledge integration mechanisms are put in place. This was the situation in HERCULES.

Overall, we found that the more there is pressure for implementations of innovations in industrial environments, the stronger is the tendency to reduce the efforts spent on overarching activities on the interfaces. Close technological relatedness between the modules, a not too large number of organizational partners, significant resources allocated to the interface, and partners who are committed to the project as a whole and not only to their specific elements and are open minded are crucial ingredients to keep the interface alive.

When the projects get under pressure, which almost inevitably happens, downscaling at the interface level and a loss of resulting collaborative innovation seems to be difficult to avoid.

The waterfall structure makes this less likely to happen. Generally, some slack in the proposal to react to unforeseen difficulties and to grasp new opportunities helped to reconfigure couplings and generate more collaborative innovations which speaks in favor of not too rigid planning at the proposal stage. A summary of the findings is reported in Table 3.5. downscaling. Overall, reconfigurations are guided by the willingness to take the project needs into account. The second phase, "the living", i.e. adapting the couplings when needed, is characterized by significant disintegrative mechanisms with cooperation issues and technical difficulties as major drivers: splitting in some modules appears, abandoning may take place; restoration may happen, depending on the attitude and action of management and dependent partners. The interfaces experienced important downscaling due to the inability of actors to deal simultaneously with cognitive and relational complexity; in the extreme case, it leads to full decomposition of the project and loss of planned innovations at this level. Fluctuation of the interface may happen, depending on integrative mechanisms put in place. Bottom up integration, with appearance of new couplings or even modules, generating unplanned collaborative innovations, may appear, however recombination is rare. The last, third phase, stable continuation, is characterized by stability of couplings, although some processes started in the previous phase may continue, especially continuation of downscaling at the interfaces.

"In my experience the proposals are always based on 100% success rates, you say that you're researching and going to develop something to a certain phase, and then the … evaluation is based on the fact that you would … reach all your goals…the system works that way. It's not so bad because in the end you still … do have (innovations) but maybe not all … that was promised when you wrote the proposal." ID31_PERSEUS_res 3.6 Discussion and conclusion

Discussion and theoretical contributions

This research used the empirical context of multi-actor R&D projects [START_REF] König | A framework for structuring interdisciplinary research management[END_REF][START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF], and investigated the evolution of the couplings in terms of their patterns and intensity over time, and its implications on the collaborative innovations.

The architecture of such project has been investigated in [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF] but in addition, the processes in the project shall be better understood (Aubry and Lavoie-Tremblay, 2018, [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. We followed recent calls for relevant research (vom Brocke and Lippe, 2015, Michelfelder andKratzer, 2013, Hofman et al., 2016) and adopted process approach (Abdallah et al., 2019, Cloutier and[START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF]. According to the best of our knowledge, this is the first time when the processes were studied in this empirical context using a multi-case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. The paper contributes to better connecting of project studies and innovation studies [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018), and makes several contributions to the literature, as discussed next.

First, we studied projects based on modular architectures coupled with integration at the interfaces (Brusoni, 2005[START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF] and found eight types of reconfigurations of couplings: five in the modules and three at the interfaces between the modules. In the large majority of the cases, the evolution of the couplings systematically had predominantly negative implications on the planned collaborative innovations. The study shows that bottomup integration thanks to recombination only rarely happened in spite of the fact that a large knowledge base is present in the projects, which is considered beneficial in other contexts (e.g. [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]Ghoshal, 1998, Ahuja and[START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF].

Second, we identified six disintegrative and two integrative generative mechanisms that activate the reconfiguration of couplings. Echoing [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF], we show that knowledge integration issues are critical in such inter-organizational settings, and, as Johansson et al.

(2011) has pointed out, knowledge integration is an interplay between coordination and cooperation. Most of the mechanisms lead to a decrease of planned collaborative innovations, or at best to their preservation. Knowledge integration mechanisms and the depth of collaboration significantly influenced the quality or maturity of the resulting innovations, but we did not find evidence that they prevented the abandoning or splitting of couplings.

Third, we uncovered that three out of four types of architecture underwent an evolution in the negative direction at the level of the interfaces. This is related to many factors, for instance not optimum technological relatedness which favors disintegration. But even if the interfaces draw on modules with high technological relatedness, difficulties to deal simultaneously with both relational and cognitive complexity [START_REF] Wang | Complexity and the functions of the firm: Breadth and depth[END_REF] within the project boundaries were salient, thus actors refocused their efforts from the interfaces to the modules, where couplings are strong and vital for development of innovations [START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF].

Historic experience of collaboration [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF][START_REF] Belderbos | The antecedents of new R&D collaborations with different partner types: On the dynamics of past R&D collaboration and innovative performance[END_REF] helps especially to avoid reconfiguration of couplings in specialized modules. The project constraints such as time, budget, description of work and promised outcomes influence significantly the work at the interfaces. If the interfaces are preserved until the end of such multi-actor R&D project, this would create a more beneficial setting for collaborative innovation [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF]. But such preservation is difficult and requests a lot of efforts from the project manager.

Forth, we uncovered the benefit of the waterfall-type project architecture [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF] from the point of view of innovations at the interface level. Siggelkow and Levinthal (2003, p.650) pointed out the importance of "temporarily divide to conquer": sequential modular decomposition and a modular integrative approach yield the best organizational performance results in the long term. Our study shows that in the multi-actor R&D project context, sequential modular decomposition also yields the best results in terms of preservation of planned innovations at the interfaces, however the order of sequences is different: in our study, integration at the beginning and then decomposition into modules, or "conquer first, then divide", worked best. Having a large variety of actors and knowledge combination opportunities at the beginning of the project has advantages: the recombination potential may more easily meet the conditions that are needed to realize this potential. Such integrative elements, when work at the interface starts from the very beginning, also help in creating common a base of understanding [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF] and a project identity.

Based on these findings, we proposed a process model [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF]Langley, 2020, Feldman, 2016) of evolution of couplings and its influence of collaborative innovations in multi-actor R&D projects. Having said that, generative mechanisms, factors and interactions may be different in the projects, and the pathways that we presented are only examples of options that are possible; there can be others, as pointed out by [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF]. Our findings also show that some structural flexibility exists within this type of multi-actor R&D projects, especially when the description of work leaves room for refinement and modification.

Therefore some slack in the proposal helps to maintain the collaboration pattern in reaction to unplanned developments.

Practical contributions

Our research also provides practical contributions. In order to assure the planned collaborative innovations, reconfiguration in the projects may be necessary, and it shall be managed proactively rather than just "happen": this study shows that some evolution of the couplings, is systematic; the organizational and personal attributes can hardly be changed, thus the managerial goal must be to channel the evolution, maximizing integrative mechanisms and minimizing disintegrative mechanisms. Without interventions, the collaborative innovations systematically decrease compared to the plans or even will not be created at all at the interfaces, while the interface connections are the justification to have multi-actor R&D project and not separated smaller projects with fewer actors, and with the loss of the interfaces, some of the value of the project gets lost too. Also, slack is needed in the project planning, and it should be accepted by the funding agencies. Managers of multi-actor projects shall especially pay attention to the interfaces between the modules from the very early project stages on, if they want to uphold the innovations according to the plan, and to design it carefully. Management especially at the interface level is an important issue also for the funding agencies: keeping an eye at the interface connections will contribute to more successful collaborative multi-actor R&D projects. In turn, this would contribute to improving the impact of public funding for research and innovation.

Limitations and future research

This research is limited to multi-actor collaborative projects funded by the European Union Research and Innovation Programmes, which have specificities such as a large number of autonomous organizations working together. We studied only a selection of projects, with strong applicative dimension and industrial involvement, developing technological innovations, aiming to obtain several main outputs by the project end and with specific nearly-decomposable architectures. The impact of the evolution towards the abandoning of the work at the interfaces was not quantified in the study, as the innovations are not comparable. Also, our typology of reconfiguration may not be complete: other reconfigurations may exist which did not occur in the studied projects.

In terms of future research directions, to advance in the understanding of multi-actor projects it would be important to study in more detail the structuring processes and their roles on the creation of innovations in the projects. It would also be useful to study the effect of the alignment of the interests of the partners and their degree of flexibility on the dynamic reconfiguration and the resulting knowledge creation. Future research could help to better understand why recombination of knowledge is difficult in the setting of such projects and how and why it happens outside the projects after having been triggered within the project.

Is bigger better? A deeper look on size and innovation performance of multi-actor R&D projects: Empirical evidence from European Research and Innovation

Programmes

Introduction

An increase of the size of knowledge base is commonly assumed to be beneficial for knowledge generation and innovation due to novel recombination possibilities [START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Cantner | The effects of knowledge management on innovative success -An empirical analysis of German firms[END_REF]. This also speaks in favor of collaborative R&D projects with multiple organizational actors, or 'networked projects' [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF]. As an example, the size of such R&D projects funded by European Framework Programmes (FP) has been more than doubled over the two last decades, from an average of 6.1 per project in FP5 (EC, 2009) to 14.1 in Horizon 2020, (EC, 2021a). These projects offer to the participating organizations the opportunity to jointly generate research and market-upstream innovation outputs (RIOs) (e.g. Goldstein andNarayanamurti, 2018, Nepelski et al., 2019), which determine innovation performance in such projects. Joint work requires dynamic knowledge integration (Berrgren et al., 2011[START_REF] Tell | Knowledge Integration and Innovation: A Survey of the Field[END_REF][START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF], which is difficult in inter-organizational settings (e.g. [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF][START_REF] Mishra | Collaboration in Multi-Partner R&D Projects: The Impact of Partnering Scale and Scope[END_REF]: "more is not always better" to increase project performance (Kobarg et al., 2019, p. 1).

The extant empirical literature investigated relationships between the project size and performance, but the findings are contradictory and sometimes diametrically opposed: some researchers argue that bigger projects are favorable for innovation outputs and innovation performance [START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF], others found the opposite [START_REF] Dyer | Determinants of Success in R&D Alliances[END_REF], or did not find a significant influence of the project size on project performance at all [START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF].

Past innovative studies predominantly used quantitative approach (Martin, 2016a); qualitative studies are lacking [START_REF] Pandza | Collaborative diversity in a nanotechnology innovation system: Evidence from the EU Framework Programme[END_REF], as the access to the field is difficult (vom Brocke and Lippe, 2015), but needed (Jacob et al., 2019, Goldstein and[START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF] to provide in-depth investigations and "why and how" explanations. In addition, when studying the relationship between project size and quantity of RIOs, scholars usually focus only on publications and patents generated by the projects [START_REF] Hung | Social networks, technology ties, and gatekeeper functionality: Implications for the performance management of R&D projects[END_REF]; innovations, especially those of incremental nature which are not patented, are often not considered [START_REF] Martin | Introduction to discussion paper on 'The sciences are different and the differences are important[END_REF].

With the large amount of the taxpayer money spent to sustain public programmes across the world, a deeper understanding of the relationship of the size of the projects on their innovation performance, measured by the RIOs, and of the related implications, is a sine qua non condition for better understanding of the project innovation performance and for increasing the 'value for money' of the programmes.

In summary, it is not clear whether, and if so, why, the size of collaborative R&D projects with multiple actors brings a positive, more-than-additive effect in the projects. Previously, scholars called "to go a step further … in an attempt to provide a more complete understanding of the role of scale in collaborative R&D performance… and indirect effects of the scale" (Spanos and Vonortas, 2012, p. 507), but this call remained unanswered so far.

This paper aims to contribute to reduction of this research gap. We pose the following research question: Does the project size have a positive effect on the quantity of research and innovation outputs (RIOs), i.e. the collective innovation performance, in collaborative R&D projects with larger numbers of organizational actors, and if yes, and how can the potential of a larger size be exploited? To answer this question, it is analyzed how the RIOs are created, from the point of view of the number of contributing organizations.

Empirically, our research focuses on collaborative R&D projects with multiple actors that were funded by the two recent European Union's Research and Innovation Framework Programmes: FP7 and Horizon 2020. The research design is a comparative inductive multiple case study method [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF], with additional quantitative supporting evidence. Taking a deeper look on each of the 185 RIOs identified in 4 projects of different size (up to 63 RIOs per project), we empirically demonstrate that over 95% of the RIOs have been created thanks to joint work and knowledge integration in small groups of 2-3 collaborating organizations that were stable for most of the duration of the project, or even individually, independent of the project size. This also puts in question the beneficial role of the size of knowledge base of the network [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Fleming | Collaborative Brokerage, Generative Creativity, and Creative Success[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF] for the generation of research and innovation outputs in the multi-actor R&D project context. From the 38 semistructured interviews and their analysis, we uncovered the direct or indirect influences of the project size on the RIOs and found that the main benefit of the project size on the RIOs which had an effect during the project duration, was the creation of meta-knowledge: this nonconventional RIO is more likely to be generated in projects of bigger size comparing with smaller ones and thus speaks in their favor. On the other hand, there are significant negative effects of a larger project size, according to the views of the participants.

The remainder of the paper is organized as following: First the theoretical framework that guided the study is provided. Then the research setting and the research design are explained, the findings are presented and discussed and the limitations of the study are pointed out. The paper concludes by summarizing the theoretical contribution as well as the practical implications of the research and suggesting future research directions. Since the seminal work of [START_REF] Schumpeter | The Theory of Economic Development[END_REF], scholars often describe innovation as the novel recombination of ideas, resources and knowledge for value creation (Nonaka, 1994, Ahuja and[START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF]. New knowledge, which is part of the innovation process [START_REF] Boisot | Integrating modernist and postmodernist perspectives on organizations: A complexity science bridge[END_REF], is often generated using the mechanisms of exchange and combination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF][START_REF] Nonaka | Tacit knowledge and knowledge conversion: Controversy and advancement in organizational knowledge creation theory[END_REF]. The activation of these mechanisms depends on the opportunity for exchange or combination, the anticipation of a valuable collaborative outcome, the motivation to be engaged, and the combinatory capability of the participants. The increase of the size of the knowledge base increase the opportunities for recombination [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]. However, the combinatory mechanisms are activated only when certain pre-conditions are in place, such as technological relatedness and complementarity of the knowledge base (e.g. [START_REF] Nooteboom | Optimal cognitive distance and absorptive capacity[END_REF][START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF][START_REF] Messeni Petruzzelli | University-Industry collaborations and international knowledge spillovers: a joint-patent investigation[END_REF]).

An increase of the size and the variety of the knowledge base should thus be beneficial for knowledge generation and innovation [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF]Ahuja, 2008, Cartner et al., 2011).

Positive influences of large groupings of organizational partners have been investigated

extensively in the knowledge management and organizational learning literature, and include e.g. access to complementary know how, benefit from organizational learning, capability development (e.g. [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Gulati | Meta-organization design: rethinking design in interorganizational and community contexts[END_REF][START_REF] Messeni Petruzzelli | University-Industry collaborations and international knowledge spillovers: a joint-patent investigation[END_REF][START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF]. Perhaps these arguments contributed to the shift of public funding to support multi-actor collaborative R&D projects, which represents a specific, distinct type of open innovation [START_REF] Enkel | Open R&D and open innovation: exploring the phenomenon[END_REF]. Multi-actor R&D projects usually are publicly funded and, similarly to many other projects, they have predefined boundaries in terms of e.g. time, scope, budget, time frames, and largely closed consortium membership [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]. The goal of these project is to generate research and innovation outputs (RIOs) thanks to joint work of participating heterogeneous organizations from academia and industry (e.g. [START_REF] Manning | The rise of project network organizations: Building core teams and flexible partner pools for interorganizational projects[END_REF][START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF][START_REF] Nepelski | The organisational and geographic diversity and innovation potential of EU-funded research networks[END_REF]; these projects produce innovations which are usually market upstream.

Knowledge generation and the need for knowledge integration.

Knowledge integration has been defined as "the purposeful combination of specialized and complementary knowledge to achieve specific tasks" (Tell et al., 2017, p. 5). Knowledge integration requires adequate knowledge integration mechanisms [START_REF] Berggren | Exploring Knowledge Integration and Innovation[END_REF]; the more specialized the knowledge is, the more there is a need for dynamic knowledge integration: this is critical, and more difficult, in inter-organizational settings [START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF], in particular in multi-actor R&D projects. [START_REF] Burt | Structural holes: the social structure of competition[END_REF], in his research on social networks, regarded the size of a network as a "mixed blessing" (Burt, 1992, p. 64), and emphasized potential drawbacks of the size in case of lack of diversity in a network. In the multi-actor R&D project setting, drawbacks of the size on the research and innovation performance also exist because the partners are diverse and heterogeneous. The connecting link between project size and outputs is collaboration:

previous studies established an explicit link between the intensity and quality of collaboration activities within the project and the collaborative outputs, or innovation performance, of the project (e.g. [START_REF] Kobarg | More is not always better: Effects of collaboration breadth and depth on radical and incremental innovation performance at the project level[END_REF][START_REF] Caniëls | Collaboration in project teams: The role of mastery and performance climates[END_REF][START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. Collaboration for knowledge integration is an interplay between coordination, or alignment of the knowledge of the organizations, and cooperation, or alignment of the interests of the organizations, and their relationship [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF]. Both are more complex, difficult and effortconsuming in large inter-organizational settings [START_REF] Heidl | The influence of interorganizational embeddedness on multipartner alliance stability[END_REF]Phelps, 2010, Ahuja, 2000;[START_REF] Mishra | Collaboration in Multi-Partner R&D Projects: The Impact of Partnering Scale and Scope[END_REF]. This may result in a decrease of the innovation performance [START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF]. In addition, the larger the number of partners, the larger the risk to get free riders on board (Okamuro, 2007, Gibson and[START_REF] Gibson | A Healthy Divide: Subgroups as a Stimulus for Team Learning Behavior[END_REF].

Also, historic experience of collaboration contributes to collaboration quality [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF] and reduces the inherent risks of project inefficiencies [START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF]. However, new knowledge generation and innovation in multi-actor R&D projects also benefit from joint work with new collaborators [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF], thus the risks of project inefficiencies and difficulties of knowledge integration increase with an increased number of partners, if the group does not have historic experience of collaboration. Difficulties related to the project management in large inter-organizational settings are well known (e.g. [START_REF] Söderlund | Pluralistic and processual understanding of projects and project organizing; towards theories of project temporality[END_REF]Tell, 2011, Lin et al., 2019).

Finally, in projects with a large number of partners, the number of project activities increases and thus need for synchronization increases too, affecting innovation performance [START_REF] Dahlander | One foot in, one foot out: how does individuals' external search breadth affect innovation outcomes[END_REF]. In such situations, the participants tend to cluster in subgroups (e.g. [START_REF] Gilsing | Exploration and exploitation in innovation systems: The case of pharmaceutical biotechnology[END_REF] which can in turn negatively influence the collaboration, knowledge integration and project performance. Similar to networks and other forms of collaborations, which can have different types of architectures [START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Brusoni | The organization of innovation in ecosystems: Problem framing, problem solving and patterns of coupling[END_REF], multi-actor R&D projects also exhibit different architectures, integrated, nearly decomposable or modular [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF], which may have different implications on collaboration and thus on the project performance [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF][START_REF] Caniëls | Collaboration in project teams: The role of mastery and performance climates[END_REF].

To sum up, the extant literature points out 'pros' and 'cons' of the project size: it can increase the variety of the knowledge base and, under certain conditions, fosters recombination and generation of new knowledge, but at the same time it increases the complexity of the collaboration and complicates knowledge integration. But how these 'pros' and 'cons' interplay in multi-actor R&D projects, and what is the relationship between the project size and the project performance? Literature provides multiple empirical findings, which are briefly summarized in the next section.

Relationship between project size and performance: not conclusive empirical evidence

Quantitative studies analyzing the relationship between project size and performance of projects, are abundantly present in the literature (see [START_REF] Lee | Impact of Cooperative R&D Projects on ICT-Based Technology Convergence[END_REF], but the literature is heterogeneous, as performance is a broad and multi-dimensional construct [START_REF] Kim | Development of an evaluation framework for publicly funded R&D projects: The case of Korea's Next Generation Network[END_REF][START_REF] Vanhaverbeke | Technological Performance and Alliances Over the Industry Life Cycle: Evidence from the ASIC Industry[END_REF]. In multi-actor R&D projects with public funding, the innovations are usually still far from being commercialized within the project timeframe [START_REF] Aguiar | European cooperative R&D and firm performance: Evidence based on funding differences in key actions[END_REF], and performance is often expressed in terms of generated project RIOs [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF]). An overview of the literature findings of some authors who established relationships between the project size in terms of number of organizational partners, and the project performance in terms of outputs of the projects is given in Table 4.1.

The table shows that the findings are contradictory and sometimes diametrically opposed.

Researchers found that an increased number of partners-organizations in subsidized R&D projects has a weakly negative effect on project outputs [START_REF] Dyer | Determinants of Success in R&D Alliances[END_REF], no significant effect (e.g. Schwartz et al., 2012, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], or a diminishing marginal effect [START_REF] Lee | Impact of Cooperative R&D Projects on ICT-Based Technology Convergence[END_REF]. [START_REF] Jacob | Analysis of patents, publications and new organisations from FP7-ICT and CIP-ICT projects[END_REF] found that an increase in the number of participating organizations increases the number of publications but has no effect on the number of patents. [START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF] considered several innovation indicators, including patent count, and provided contradictory evidence, pointing out that "the number of project participants strongly increases all innovation indicators" [START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF](Szücs, , p. 1256)), and concluding that "the success of a project in terms of innovation outcomes strongly depends on the number of participants...." [START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF](Szücs, , p. 1264)).

The non-conclusive empirical evidence offered by (predominantly) quantitative studies lead to contradictory recommendations to policy makers and program designers, and it is not clear why such contrast exists in the literature.

The findings presented above shall be however considered with nuances, as the number of RIOs, or performance, in the projects may be influenced not only by size, but also by other factors, such as project leadership [START_REF] Nishimura | Internal and external discipline: The effect of project leadership and government monitoring on the performance of publicly funded R&D consortia[END_REF], embeddedness of the knowledge base [START_REF] Ahuja | Knowledge structures and innovation: Useful abstractions and unanswered questions[END_REF], diversity of partners [START_REF] Von Raesfeld | Influence of partner diversity on collaborative public R&D project outcomes: A study of application and commercialization of nanotechnologies in the Netherlands[END_REF][START_REF] Coad | Diversity in one dimension alongside greater similarity in others: evidence from FP7 cooperative research teams[END_REF][START_REF] Nepelski | The organisational and geographic diversity and innovation potential of EU-funded research networks[END_REF], differences in institutional cultures and objectives [START_REF] Nepelski | Organizational diversity and innovation potential of EUfunded research projects[END_REF], nature of the project [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF], level of knowledge integration mechanisms [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF], type of the project, exploitative or exploratory [START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF], etc.

Literature directly relevant to multi-actor R&D projects is scarce, but scholars have investigated the relationship between the size of the teams and performance in other contexts, including a literature stream studying the relationship between research performance and the size of research groups, thus considering not the number of organizations, but the number of collaborating specialists within research projects. This stream provides evidence that the research teams are getting larger (Wuchty et al., 2007, Cummings and[START_REF] Cummings | Organization theory and the changing nature of science[END_REF], which is beneficial for research performance [START_REF] Wu | Large teams develop and small teams disrupt science and technology[END_REF]; however, an increase of the size of a research group has a positive influence on the research performance only up to a certain point: Overall, the relationship between scientific outputs (scientific articles and patents) and the project size is inverted U-shaped: from a certain point on, the increase of the number of participants yields decreasing marginal return. However, the authors noted that the results shall be interpreted with caution, as the critical point seems to be approx. 52 organizational partners, which significantly exceed the average project size.

There is significant difference in performance between three groups of the studied projects. For instance, Networks of Excellence (NoE), with large number of partners per project, have performed much better than IPs and STRePs, both in terms of scientific achievements and in terms of generated patents. No statistically significant relationships were found neither for scientific outputs, nor for technological outputs. "It appears that the simple assumption 'bigger is better' in European Union-financed collaborative R&D may need to be reconsidered." (Spanos and Vonortas, 2012, p. 494) Negative /weakly negative relationship between the increase of partners and RIOs [START_REF] Dyer | Determinants of Success in R&D Alliances[END_REF] Quantitative 397 firms involved in 142 contractual subsidized R&D alliances.

The number of partners in the subsidized R&D alliance has a weakly negative effect on patent applications that derived from this alliance. The number of patents per participating firm starts declining when the alliance has three or more partners.

Table 4.1. Overview of selected papers providing empirical evidence about the relationship between the project size and innovation performance, expressed by (some of the) research and innovation outputs (RIOs) of the projects

then no effect or even a decrease in research performance has been observed [START_REF] Von Tunzelmann | The Effects of Size on Research Performance: A SPRU Review[END_REF]. Similarly, [START_REF] Kenna | Managing research quality: Critical mass and optimal academic research group size[END_REF] studied conditions which have a positive influence on research quality, including size of the group, and suggest that there is a linear relationship up to a certain critical point: above this point, large groups fragment and a significant increase in research quality is no more observed. These authors suggest that the optimal size of the research group is slightly bigger than this critical point.

To sum up, using quantitative methods, scholars have already investigated the influence of project size/scale on innovation performance of projects; but the findings are contradictory.

This suggests that investigation of the relationship between the size and outputs will benefit from a deeper look: scholars called for future research in the project context "to go a step further in this regard in an attempt to provide a more complete understanding of the role of scale in collaborative R&D performance" and to investigate "the causal mechanisms through which scale affects (if at all) performance" (Spanos and Vonortas, 2012, p. 507).

The current study aims to contribute to closing this research gap. The following research question is posed: Does the project size have a positive effect on the collective innovation performance in collaborative R&D projects with larger numbers of participating organizational actors, and if yes, how can the potential of a larger size be exploited?

Research setting

The research setting considered here are market-upstream multi-actor collaborative R&D This would suggest a positive role of the project size, as a larger size allows to the project partners multiply combinations in order to get access to different knowledge bases of the participants. Fourthly, the explanations provided by the project actors were analyzed. This was based on the abstractive process of open coding [START_REF] Strauss | Basics of qualitative research: Techniques and procedures for developing grounded theory[END_REF], adapted by [START_REF] Gioia | Seeking Qualitative Rigor in Inductive Research[END_REF]. Two steps were performed in the analysis: (1) factual, to understand who is at the origin of the RIOs, who collaborated with whom, and whether the collaborating group was stable during the duration of the project, and (2) interpretative, to understand how the project size influences the collaboration and IOs. The interviews transcripts were coded using the NVivo 12.1 software, and a data structure (shown in Appendices 4.2 and 4.3) was built by grouping 30 concepts of 1 st order into 18 aggregate and more abstract themes of the 2nd order, which were combined into 6 aggregate dimensions, and compared between the cases. Using the project documentation, 182 RIOs were identified, between 31 and 63 RIOs per project, and classified according to the detailed categories of RIOs (see Table 4.4). It was observed that the RIOs in the category 'innovation' are the 'main RIOs' targeted in the projects, all others RIOs rather are derivates, such as additional knowledge, publication or patents. Although the total number of all RIO is obviously not related to the project size, there is an increase of the number of innovations with the project size, and the largest project, ORION, generated significantly more RIOs, which suggests at least an additive effect (see section 4.5.2.1 for more details). Some RIOs were related, e.g. an innovative technology was tested in the lab (RIO1), then this technology was deployed (RIO2), and a publication was done about it (RIO3), but many other RIOs were not related, although all RIOs were relevant to the overall topic and goals of the project.

Results

The first conclusion of the study is that the studied multi-actor R&D projects generated high numbers of RIOs of different categories, as shown in Table 4.4, there is no direct relationship between the project size and the number of RIOs. In addition, it was observed that RIOs are not necessarily inter-connected or depended on each other.

Categories of research and innovation outputs (RIOs)

Number obtained by the group of partners and was generalized from analytic comparisons and/or multiple evidence following testing, lab experiments, or deployments in operational environments. Meta-knowledge is much more consolidated than exchanged opinions, it helps to increase the certainty about the technical findings, it is often tacit and has strategic importance for the industrial partners.

"We had fragmentary knowledge… and we put together our brains and created something. So the partners had fundamental knowledge and experiences but with this fundamental knowledge and experiences we produce something that was new. And as we produced it jointly, I mean it was a joint intellectual effort. ID3_HERCULES_acad" "There were many people who tried different aspects okay so it was (Partner A), it was (Partner B), it was us to some extent, it was (Partner C), and they applied different methods, and in the end some approaches worked and others worked less well…And that's a sort of meta-knowledge that has been created in the project… it has also some strategic importance… it was an experience that I would not like to miss. Yeah While meta-knowledge is potentially reportable (in project deliverables) and publishable (e.g. in survey papers), it only appeared in the project documentation when it was planned in the initial project proposals. For instance, in PEGASUS and HERCULES, relevant reports to the funding agency were planned in the proposals, but they were not destinated to a wide audience; in ORION, the meta-knowledge was unplanned, and thus it remained tacit. Also, because meta-knowledge may be generated only by the end of the project, the eventual resulting publication would only appear well after the project end, and thus can hardly be accounted in the investigations using the databases with the project outputs. It turned out that meta-knowledge on the scientific/technical level was produced in those projects where small groups of partners worked in parallel on different applications and produced different RIOs, but the technological relatedness between the work was relatively large.

The origin of meta-knowledge, a non-conventional RIO resulting from comparison of knowledge generated by groups working in parallel in the project, suggests that the project size has an important role for the generation of meta-knowledge. However, not only the large size, but also specific project architecture (modular, with parallel work on different RIOs that could be compared to generate meta-knowledge), technological relatedness between the groups working in modules (to make such a comparison of RIOs obtained in the modules meaningful), and mechanisms allowing exchanges and comparison of RIOs are required, in order to generate meta-knowledge. This is the second result of the study.

Collaborating organizations

Number of collaborating organizations per RIO.

Data on 185 RIOs were matched with the data on collaborating organizations contributing to each RIO as documented by the projects ( Table 4.6 also shows that the number of organizations per innovation output remained stable across categories of RIOs, independent of the project size. Publications had 1.40 organizational partners in average across projects, while innovations had 1.95 organizational partners in average.

All studied projects aimed at higher TRL levels and had a strong application dimension requiring specialized knowledge; for this reason, work in small groups was inevitable.

Besides this, in large consortia it was found to be difficult to avoid overlapping of knowledge, resulting sometimes in internal competition between partners with similar competences:

"Partner X tried to work with all industrial companies if possible, so there was some competition… I'm not blaming others, what the academic groups do is they try to exploit the work to get interesting research and publications from it, and of course the areas where you can do this are somehow limited (NB: in the project)… ( ID36a_comparative_ORION_acad) "I think partner A and partner B there is some overlap in what we do, so there is an element of wanting to carve out your section of the project that's separate from theirs. I think partner C and to a certain extent partner D maybe have some competing functionalities..

." ID25_ORION_SME

The third conclusion of the study is therefore that the RIOs are generated relatively independently in small groups of 2-3 organizational partners; the maximum size of the group contributing to RIOs in the studied projects was 5 organizations (but only in a special case, meta-knowledge). This suggests why the number of RIOs in the project, in particular of the innovations, have an approximately proportional relationship with the project size: in larger projects, more small groups will be formed, which in turn generate more RIOs. We call this 'additive effect', in the sense that the large projects simply accommodated more small groups, working on their specific RIOs, with broader collaborations inhibited by the difficulties described in section 4.5.3.1 below.

Stability of the groups of collaborating organizations.

It was found that the sub-groups of collaborating partners, whose work resulted in the generation of RIOs, were very stable in course of the project (except for exceptional cases, e.g. of withdrawal of an organization from a project) and these groups worked largely independently from each other (Appendix 4.5). There was movement (but not recombination) of partners within the small groups, when the innovative technologies which were developed in the projects evolved in their maturity so that further partners were needed for their deployment in the industrial or other operational environment. For instance, in ORION, two innovative technologies (innovative components, RIO1 and RIO2) were produced individually by two organizations, then these two organizations joined forces when the two innovative components were integrated in the system prototype (RIO3, with 2 partners), then the deployment of the system prototype was done in the operational environment (RIO4, with 3 partners). The need for an additional collaborating organization arose due to the need for deployment which was enabled by the 3 rd partner, which led to an increase of the size of the sub-group. From this, also a new product (RIO5) resulted, an enabling technology which can be used more generally. Only one partner is the developer of this product and thus the only contributor to RIO5. In this example, the maximum size of the collaborating group in ORION was 3 organizations, at the deployment phase. Even in the largest project in the study, ORION (17 organizational partners), RIOs were created in the small stable groups of maximum 4 partners (innovations, publications) and 5 partners (meta-knowledge), and the groups and their RIOs were largely independent of each other; i.e. RIOs developed by one group were not required for the development of RIOs of another group.

The fourth conclusion of the study is that the project size does not impact significantly the RIOs because the groups of collaborating partners are stable throughout the project duration, with some variations in the number of partners within the group according to the needs.

Recombination to produce different RIOs could not be observed: RIOs that are related to innovations required joint intense work in small groups, and not so much a broad exchange of information or ideas which could benefit from larger numbers of partners. However, in specific cases explained in section 4.5.1.2, non-planned combination between the partners may additionally take place, which can lead to the generation of meta-knowledge.

4.5.3 Effects of the size of the projects: feedback from the project actors 4.5.3.1. Negative effects of project size: short term effect, within the project.

The interviews provided multiple evidence of a negative influence of the project size on innovation performance within the project, directly or indirectly. Four main blocking points related to the increase of the size of the projects systematically emerged (Appendix 4.2):

1. Lack of synergies, lack of coherence in the project. With the large group of partners and the specific scope of the project, some overlapping of knowledge base resulted and this was not always seen positively. In some cases, internal competition between the partners was also observed.

"There's too many (partners) to build a coherent project, because of the seventeen competing companies... some of them are. I think X and Y there is some overlap in what we do, so there is an element of wanting to carve out your section of the project that's separate from theirs. … The universities obviously they're not competing but companies like S might be interested in doing (work) that the universities want to do with their PhD students, so again there's that element of competition there. I just think this is just too big to be honest".

ID25_ORION_SME"

The lack of synergies and lack of coherence within the larger projects was systematic, and informants pointed out that the presence of a large group of partners working under the same 'umbrella' topic of the project had little beneficial effect on the project work and on the RIOs:

"…the European Commission I think they should fix that. Because you know we get together all these partners, but …. there is no synergy at all". ID28b_PERSEUS_large industry 2. Difficult knowledge integration. Knowledge exchange and knowledge integration was difficult in all studied multi-actor projects. The main reasons mentioned by the project actors were lack of commonalities between groups, too big differences between the approaches used by the groups, and lack of relevant background to exchange and integrate knowledge:: The findings also show difficulties to benefit from knowledge exchange and knowledge integration during the meetings especially in the two largest projects under study, PERSEUS and ORION. There were two "layers" of knowledge exchange and integration in the studied projects: small groups layer (this went quite well in most of the projects and it happened mostly by day-to-day work and exchanges), and 'beyond the small groups' layer at the project level. For the latter, meetings were the prime knowledge integration mechanism. But the plenary meetings are limited in duration, which is relatively similar for smaller and for bigger project, and the duration can hardly be extended: project participants lack availability..

The more organizations are involved, the less time they have to discuss their work at the meetings. Consequently, a clear pattern emerged with participants from larger projects noticing insufficient time for detailed technical discussions at the plenary meetings, and thus reduced opportunities for knowledge sharing and knowledge integration.

"If you have fifteen people, then everyone is able to provide his or her opinion on the topic, here with thirty-five people we do not have time anymore. You just can't manage all the different case studies and concepts in a two days meeting if you have all these case studies to discuss, it's not possible.... ID40b_comp_large industry "Q: How did exchange happen? A: Well, at the bar (Laughing.) Q: At the bar? A: Yeah, well, the social events are very important. Q: Why at the bar and not at the plenary? A: Well, because in the plenary we have a strict plan, so there's time twenty minutes, and coffee break is ten minutes, and in reality it's five minutes. No time for real discussion… we have so many topics ( ID26_comparative_SME)

"With the group of 10 partners like in HERCULES, it was quite often possible to discuss everything (NB: technical topics) at the plenary, which in ORION is not possible." ID7_ORION_comparative_large industry

The projects teams adapted to this and details were discussed in small groups but then the benefit to have larger group of partners for knowledge sharing gets lost.

"At the beginning it was a bit problematic because everyone just gave an overview on the actions taken and there was no time to discuss, and I think we all have to learn that technical discussions are more helpful than just having a work description… We improved that" (NB: by meetings in small groups) ID40a_compar_large industry).

An additional concern with knowledge integration mechanisms was that the applications are by definition specialized, and competitors are not welcome. The prototypes of generic innovations developed in the studied projects were deployed and adapted to the needs of each application. However, the specialization to the applications and the goal of reaching higher technological maturity created a barrier for knowledge integration and knowledge sharing:

"We expected to …throughout the ORION project we starting to embed (our technology) within the use case providers, certainly the ones that we were involved with … we have found that that's not happened at all. … This specificity of the use cases is really a barrier. ID25_ORION_SME

Thus, knowledge exchange and integration mostly happened through the academic partners, however their number in the projects were limited (4 in PEGASUS, 2 in HERCULES, 3 in PERSEUS, 3 in ORION):

"For the technical work the other case study providers were not very helpful. Because they had other use cases which were not comparable to the use cases that we provided, so that was there was no real benefit in seeing their results, there was nothing that we could use for our own use case. But the feedback from the academic partners to our scientific approaches was helpful, because they developed similar things… so there were other perspectives and other opinions that were discussed during the project meetings and gave us new inputs to develop our ideas further. " (ID40A_HERCULES_large ind)

Presence of many organizational actors creates technical barriers to knowledge integration: the more the number of organizations, the more technical barriers are present. For example:

You basically have these different software tools, partners have ecosystem of software tools… it's hard to connect all these tools. I think the tooling creates walls… some tools are licensed, and some tools you have to know quite a bit about the tools, especially academic tools ... This sets limitations... it's sort of ecosystem of twenty thirty different things…. ( supp1_ORION_SME)

"The idea was to use the (technology) at least in different case studies. It was too difficult actually in the end, too ambitious to have links between the case studies, to have the technologies developed for one case study and then using them in a different case study. ID11_PERSEUS_coord

Slower progress and in some cases risk of lack of quality.

The progress was slower at the beginning of the larger projects comparing with the smaller projects: the more the number of organizations, the more time is needed to investigate technical details required for the project work, and the less time remains for the actual development of RIOs:

"ORION went very slowly for the first maybe yeah year and a half…it took quite a while for things to get moving and I think if there'd been fewer people involved it would have got moving more quickly" (ID25_ORION_SME)

Finally, the bigger the project, the bigger is the risk to lack clear responsibilities, and the more room exists for freeriders:

Flaws in the project proposal, as an underlying issue.

In addition, an important underlying issue was the flaws and lack of details in the project proposals, which is used to implement the project. Bigger numbers of organizational partners imply more work during the proposal preparation: the understanding of important details was lacking at the proposal preparation phase in all studied projects. In addition, the proposal text has fixed limited length: with more partners, it is more difficult to write details about the planned work, and in most cases, descriptions were at "high level", leaving ambiguities.

"You don't have a consortium contract, you cannot really talk in detail, you don't have the time… There was no time when we drafted the proposal to really go into requirements and details... it was on a higher level…I don't know how you could improve it actually, it's already so much work to put into the writing proposal already… "(ID31_PERSEUS_res)

Frequently, project actors complained that the projects were too complex, and specifically identified the issue of large size. The large majority of the interviewed participants in the larger projects complained about the increased difficulty of understanding what goes on in the rest of the project and about the lack of cross-fertilization when the project size increased. The mentions of a negative influence significantly intensified with the project size: it was e.g. significantly more salient in ORION (17 partners) than in HERCULES (10 partners), and many partners from HERCULES also took part in ORION, compared the projects, and identified the size of the partnership with its negative implications, as the major difference between the projects (highlights done by the authors). But this statement about the broader technical feedback in a large group was related to the smallest project in the study, HERCULES. The same respondent then indicated concerns with the lack of time at the meetings in ORION (see section 4.5.3.1), and pointed out that technical feedback was mostly received in small workshops, where technical details were discussed:

"Due to the variety of cases, they are not so close to our problems as it was in the HERCULES project, but they are still useful… We had a common workshop with (gives the names of 4 partners) and another workshop with (gives the name of 3 partners)". "(ID40b_ORION-large ind)

Thus, a paradox is that from one side the large size of the project brings more opportunities to get technical feedback, but from another side, such feedback could only be obtained during technical meetings in small groups, thus losing the benefit of the large project size.

In addition, within the project of the largest size, some actors got an opportunity to compare own technologies at a broader scale than what could have been possible in small size projects: 2. Future opportunities, to be used outside the project.

a) Possibility to generate meta-knowledge, to be used outside the project. As described above, technical meta-knowledge has been identified as a non-conventional RIO and a positive implication of the project size could be concluded; it was generated in some projects where small groups worked in parallel and exchanged regularly, but both ORION and HERCULES (the largest and the smallest projects in the study) generated meta-knowledge.

We found that the benefit from technical meta-knowledge in the future, i.e. after the end of the project, was one of the main motivations of the industrial partners to enter the project and one of the most valuable RIOs for them. It increased the certainty about the technical findings in the projects, helped the private firms to decide into which R&D direction to invest, and it helped to avoid costly errors. It also helped the academic groups to decide on future research directions.

"…when we went to the project… we had a suspicion that's an important topic, now we have pretty much certainty that it's something which is very very important and useful… Well when you're in the business like ours, you have to take risks…the more certainty you have, the better it is… The impact is that we are devoting a lot more resources than is funded by a project, and secondly we will continue to devote further resource for developing this technology beyond the end of the project." ID24_ORION_SME b) Bigger number of business and R&D opportunities. A larger consortium gives to the participants bigger numbers of business and R&D opportunities after the project. However, these opportunities may or may not be seized: For almost all informants, the positive implications of the project size thanks to the periodic interactions in large group of partners were acknowledged, but seen as useful only in the long term perspective and outside the project. The fifth result of the study is that the size of the project has a negative effect on the productivity within the project, its positive effect is mostly in terms of learning and future opportunities, which materialize in the longer term and are not specifically related to the innovation performance of the projects. The projects of large size provide conditions to generate meta-knowledge, but this depends on the project architecture and on other factors, such as the technological relatedness between the groups that work in the project, and exchange opportunities between the groups with enough time to discuss technical details.

To sum up, the results of the study show that the RIOs were uniformly created by small groups of organizational partners and these groups were rather stables throughout the project duration. Unplanned recombination of knowledge resulting into joint project output may happen (and thus speaks in favor of larger projects), but it mostly does not happen within the project duration: thus, the effect of unplanned recombination could only very rarely be seen in the RIOs generated in the project. Generally, the actors strongly favored projects of medium size over larger projects.

Discussion and conclusion

Discussion

This study aspires to develop new perspectives on multi-actor collaborative R&D projects by taking a deeper look on the project size, the research and innovation outputs (RIOs) and their relationships in collaborative R&D projects with multiple partners which generated new scientific knowledge and innovations at the same time (König et al., 2013, Goldstein and[START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]. To the best of the knowledge of the authors, this is the first time when the RIOs in collaborative R&D projects with multiple actors, as well as the number of contributing organizations, were analyzed via an explorative multi-case comparative study.

The research makes several contributions to the innovation literature.

First, it adds some clarifications to the contradictory literature that studied the relationships between the project scale, or size, in terms of number of organizations, and the innovation performance (e.g. [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF][START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF][START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF][START_REF] Jacob | Analysis of patents, publications and new organisations from FP7-ICT and CIP-ICT projects[END_REF]. Echoing [START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF], the study shows that across the selected cases the RIOs of the projects are mostly produced thanks to joint work in small groups of organizational partners (mostly 2 to 3 partners), or even individually. The evidence shows that in over 95% of the generated RIOs, the collaborations were stable and happened as planned. It was found that the average size of the collaborating group of partners per innovation is 1.95 organizations. This directly implies that there is high likelihood that larger R&D projects with multiple partners just develop into a 'collection' of a larger number of small groups producing more outputs (so there is no larger-than-additive effect).

Our findings are in line with previous studies which did not find a relationship between the project size and the number of patents [START_REF] Callaert | Analysis of patenting activities of FP7 NMP projects[END_REF][START_REF] Jacob | Analysis of patents, publications and new organisations from FP7-ICT and CIP-ICT projects[END_REF], or between the project size and the number of publications [START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF]: there is most likely no such relationship beyond the simple additive effect. For instance, the findings of Jacob et al.

(2019) that the increase in the number of participating organizations leads to increased number of publications, is most probably due to such additive effect. This also puts in question the beneficial role of the size of the knowledge base [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF]Katila, 2001, Cantner et al., 2011) on the research and innovation outputs in the R&D project context, with limitations of budget, duration and scope: benefits are mostly useful in the future, beyond the project, but then these benefits are not reflected in the RIOs generated in the project. The present study shows that the size of the knowledge base does not play a critical role in the generation of outputs in multi-actor R&D projects with a strong application dimension.

Our analysis contradicts the findings and recommendations of Szücs ( 2018), but it points in the same direction as Arroyabe et al. (2015, p. 629), who found that "a maximum in joint performance is reached when 4 or 5 partners take part in the network, which [START_REF] Hagedoorn | Understanding the Rationale of Strategic Technology Partnering: Interorganizational Modes of Cooperation and Sectoral Differences[END_REF] has denoted as the 'minimum to cooperate". This study found that, out of 185 identified RIOs across 4 projects, the 'maximum to cooperate', i.e. the maximum number of organizational collaborators per conventional RIO was 4, and for meta-knowledge the maximum number of contributing organizations was 5, in just one case. With a bigger number of organizational partners, indicatively more than 8-10, the project splits into sub-projects (or sometimes into individual work by one organization), echoing the finding of e.g. [START_REF] Gilsing | Exploration and exploitation in innovation systems: The case of pharmaceutical biotechnology[END_REF]. There can be cross-fertilization between these sub-groups or modules, but if their number becomes large and the technological relatedness decreases, the participants judged that the downsides outweigh the positive effects.

Second, through the inductive process applied in the multiple cases, we have identified a "hidden" output, which may be created in multi-actor R&D projects of large size: proven meta-knowledge which was generated close to the end of the project. This result also adds to the literature on innovation outputs of large collaborative projects with multiple actors (e.g. Goldstein andNarayanamurti, 2018, Nepelski et al., 2019). In addition, this result is interesting from the point of view of the implications of the project size on the project outputs: meta-knowledge is precisely the output where the large size of the project can play a positive role as additional combinations between the organizational partners can take place and can produce meta-knowledge.

The third finding is about the materialization of the influence of the project size on the project performance. The study investigated some mechanisms through which the project size affects

Practical contributions

Our research has practical implications. It helps the funding agencies and R&D project management to better understand the potential implications of the increase in the project size, which is not necessarily positive above a certain threshold. The study shows that 'bigger' is usually not 'better'; the size of the projects can be increased almost arbitrarily, by adding more small groups of collaborating partners, and by adding partners with small, specific roles.

But then the possible benefits of this variety are likely to not materialize: the increase in the number of the partners creates at best proportional growth, at the expense of a larger coordination effort. An argument in favor of large projects with many partners could be that funding agencies and R&D management want to incentivize generation of meta-knowledge and make it more visible and shared with the community. These insights are of interest to the designers of funding programmes and will contribute to more successful collaborative projects, thus increasing the impact of future public funding for research and innovation.

Limitations and future research

The research presented here has certain inherent limitations. It is limited to multi-actor R&D collaborative projects funded by the European Union Research and Innovation Programmes, which have specificities such as a large number of autonomous organizations working together in a rather inflexible frame of planned work and budget. Only a certain selection of themes and projects was studied here, and the impact of the research and innovation outputs was not included, such as the number of citations of publications and patents, as these data are only available several years after the project end. The results of the study are thus applicable to the projects with specific characteristics. Also, there might be specific cases where large groups of partners are required 'by design' for sequential work: such projects were not included in this study.

To advance in the understanding of multi-actor projects, future research could help to better identify the conditions under which recombination has more chances to happen during or outside the project, or why recombination opportunities do not realize in spite of the presence of multiple partners and the variety and size of knowledge base and regular interactions. We join the call of [START_REF] Kobarg | More is not always better: Effects of collaboration breadth and depth on radical and incremental innovation performance at the project level[END_REF] that an investigation of different combinations of collaboration partners on the innovation performance would be of interest. The architecture of the type of multi-actor R&D projects considered here, and dynamics of couplings leading to collaborative innovation, merits further investigations. Research on knowledge integration mechanisms that are adapted to this type of large inter-organizational project settings would also be of great importance.

Chapter 5. Climbing up the ladder: Towards a better understanding of the innovation outputs and of the performance of market-upstream collaborative research and innovation projects 10

Joint work with Sebastian Engell and Catherine Thomas

Study #4

Abstract. In market-upstream research and innovation projects, it is difficult to assess their innovation performance as the innovative technologies have not become innovations at the end of the projects; the projects aim at their maturation towards implementation and future market success. Assessments are so far done by counting patents which covers only a subset of the contributions of the projects, or subjectively, or by analysing the potential of the innovation outputs. The paper sheds light on the up to now unresolved issue how to assess market upstream innovation outputs in the projects, and thus the performance of the project on the 'innovation side' directly and objectively, at the end of the projects, going beyond patents, through the lens of the advancement of their maturity on the way to potential innovation. An exploratory multi-case study of 5 projects with 67 organizations, funded by the European Union's Research and Innovation Programmes is reported. Using over 1000 pages of documentation and data from 49 semi-structured interviews, 54 market upstream innovation outputs were identified and their advancement of maturity from the starting point to the final result was analysed. Towards this goal, a framework is proposed with a refined technology readiness scale and the scoring method that reflects the required average efforts to progress from one level of maturity to the next. The scores also point to the critical steps in the process of maturation. Using these scores, performance of the projects that pursue different innovations on different levels of maturity can be assessed. The implications of the findings of the study for research and innovation policy and practice are outlined in the end.

Keywords: multi-actor RDI project; market upstream project, innovative technologies; maturation of technologies; innovation output; technology readiness levels; innovation performance 10 The earlier version of this study has been accepted for presentation at the 81st Annual Meeting of the Academy of Management (AOM2021), July 29 -August 4, 2021. In its previous version, it has been reviewed by Research Policy journal, was resubmitted after revisions in April 2021, and is under the second round of review. , 2015[START_REF] König | A framework for structuring interdisciplinary research management[END_REF]. Nowadays, the focus of programmes is often shifting towards supporting closer-to-market research and developments. This is the case of Horizon 2020, but also of the Horizon Europe Framework Programme, launched in spring 2021: the part of the Programme that will support industrial and closer-tomarket research and development was allocated 56% of €95.5 billion, the entire budget of the Programme (Science / Business, 2021). Nonetheless, the funded work has to be market upstream and must not interfere with competition on the market, so the projects do not include the final stages of commercialization but are expected to lead to innovations after the end of the project.

Climbing up the ladder: Towards a better understanding of the innovation outputs and of the performance of market-upstream collaborative research and innovation projects

Introduction

New knowledge and technological advancements become innovations when they are implemented (OECD Oslo Manuel, 2018[START_REF] Gault | Defining and measuring innovation in all sectors of the economy[END_REF]. A common view is to characterize innovation process by phases, or steps, e.g. conceptualization or ideation (coming up with new ideas, investigating fundamental issues and feasibility), development, testing and validation of innovative technologies, and commercialization (e.g. [START_REF] Pavitt | Innovation processes[END_REF][START_REF] Eisenhardt | Accelerating Adaptive Processes: Product Innovation in the Global Computer Industry[END_REF][START_REF] Cooper | Winning at new products: Creating value through innovations[END_REF]. Each step of the innovation process generates new knowledge and creates value, but not all of these steps lead to successful innovations, there usually is a significant attrition on the way. While the first phase is supported by public and industrial basic research funding [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF], and the third phase is usually performed by commercial organizations or consortia, the middle phase is frequently supported by publicly (co)funded market upstream RDI projects such as those funded by the European Commission or national funding agencies in the US (Goldstein andKearney, 2020, Klessova et al., 2020). The current trend in this domain has been towards larger projects, both in total budget and number of partners: as an example, the size of RDI projects funded by European FPs has been more than doubled over the two last decades, from an average of 6.1 per project in FP5 (EC, 2009) to 14.1 in Horizon 2020, (EC, 2021a). The projects considered here develop one or a set of related innovative technologies, termed innovation outputs (IOs) in this paper: new materials, methods, software, devices, processes etc. The partners often collaborate in subgroups that are targeting specific developments [START_REF] Klessova | Research and Innovation Programs as a Mechanism to Support Collaborative Efforts[END_REF].

With the increase in the number of project actors with own objectives [START_REF] Mannak | A temporal perspective on repeated ties across university-industry R&D consortia[END_REF], also the number of the innovative technologies that are pushed forward within a project often increases. The goal and the purpose of the projects is to advance these IOs from a certain starting point towards their commercial exploitation, this advancement is called 'increase of maturity" in this paper. The goal of the actors is that the IOs indeed become innovations that are used internally or offered on the market in the future, after the termination of the project, but this depends also on external factors, such as competing developments, finance, regulations, company strategies, etc. which are beyond scope of the project. A crucial aspect is that within a single project of this kind, the IOs are not developed from scratch to a commercial solution, but the projects advance the technologies from a certain stage of maturity to a higher one, where maturity means how technologically ready (mature) the technology is on its way towards the final use [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF]. The starting points and the end points differ between the projects and also between the technologies developed within the same project.

Given the significant amount of public funding that is invested in this type of projects, there is a strong interest in assessing their performance. Extant literature studied research performance since long (e.g. [START_REF] Martin | The evolution of science policy and innovation studies[END_REF][START_REF] Grimaldi | Assessing collaborative, pre-competitive R&D projects: the case of the UK LINK scheme[END_REF][START_REF] Martin | The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster[END_REF], but the understanding of the performance of such projects on the 'innovation side' is lagging behind.

Such assessment is challenging (Kim et al., 2017, Aguiar and[START_REF] Aguiar | European cooperative R&D and firm performance: Evidence based on funding differences in key actions[END_REF] because each project is specific with respect to their objectives; the IOs are diverse [START_REF] Perkmann | How should firms evaluate success in universityindustry alliances? A performance measurement system[END_REF][START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF][START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF]. Also the efforts required for producing the different IOs can differ largely. Suitable indicators to evaluate project success from the point of view of IOs are not generally agreed upon [START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF][START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF].

Scholars, practitioners and policy makers repeatedly question how to assess IOs at the stage when they are not innovations yet (e.g. [START_REF] Dewangan | Towards a holistic enterprise innovation performance measurement system[END_REF]Godse, 2014, Eling and[START_REF] Eling | Managing the Front End of Innovation-Less Fuzzy, Yet Still Not Fully Understood[END_REF]. Some researchers assessed IOs and the innovation performance of the projects from the point of view of the project actors, thus indirectly and subjectively (e.g. [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], [START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF][START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF][START_REF] Arranz | Network Embeddedness in Exploration and Exploitation of Joint R&D Projects: A Structural Approach[END_REF]). An objective assessment of IOs at this stage is challenging [START_REF] Henttonen | Searching for appropriate performance measures for innovation and development projects[END_REF][START_REF] Birchall | Innovation performance measurement: Current practices, issues and management challenges[END_REF]. From the objective point of view, IOs have been assessed from the perspective of their innovation potential [START_REF] Nepelsky | Innovation and innovator assessment in R&I ecosystems: the case of the EU Framework Programme[END_REF]Van Roy, 2020, Nepelsky et al., 2019), or from the point of view of their impact a few years after the project end (e.g. von [START_REF] Von Raesfeld | Influence of partner diversity on collaborative public R&D project outcomes: A study of application and commercialization of nanotechnologies in the Netherlands[END_REF], Kim et al., 2017). There is little literature on the assessment of IOs and of the RDI performance at the development and validation stage: [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF], in a thorough analysis of over 250 scholarly papers, did not identify any indicator to assess the IOs at this stage, and called for a deeper analysis of the possibilities for evaluating IOs at the early stage of the innovation process. These authors also noted that market upstream IOs are traditionally measured by patents. However, the extant literature highlighted repeatedly (e.g. [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF][START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF][START_REF] Hung | Social networks, technology ties, and gatekeeper functionality: Implications for the performance management of R&D projects[END_REF]) that this lens does not provide the full picture: RDI projects with both academic and industrial actors usually generate not only patents, but also progress in the development of non-patented innovative technologies, which is often missing in the indicators (Martin, 2016a). Also patenting often is not in the focus of the actors, e.g. securing freedom to operate and not sharing details is another possible IP protection strategy.

A decade ago, scholars already called for research to better understand the performance of collaborative RDI networks and projects (Spanos andVonortas, 2012, Pandza et al., 2011), but still scholarly studies on the collective performance of multi-actor projects, using objective data about the IOs, are almost inexistent [START_REF] Nepelski | The organisational and geographic diversity and innovation potential of EU-funded research networks[END_REF]. The two research streams, innovation studies and project studies, while both dealing with innovations and IOs in the project setting, are mostly disconnected, and scholars repeatedly call for their stronger connection [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018[START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. Recent papers also pointed out the need to investigate the performance of publicly funded RDI projects following a case study approach and going beyond publications and patents [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]. Qualitative studies may provide valuable insights [START_REF] Messeni Petruzzelli | Maturity of knowledge inputs and innovation value: The moderating effect of firm age and size[END_REF][START_REF] Arundel | Advancing innovation in the public sector: Aligning innovation measurement with policy goals[END_REF], their "why and how" explanations can help to get a more fine-grained, nuanced view on the innovations, which will be beneficial for innovation research [START_REF] Collinson | Recombination for innovation: performance outcomes from international partnerships in China[END_REF]. But qualitative studies of multi-actor publicly funded RDI projects, and are lacking.

This paper responds to these different calls and presents an exploratory qualitative study of the innovation outputs of several multi-actor RDI projects, aiming at contributing to the understanding of the collective innovation performance in publicly funded projects and the assessment of the individual IOs. It addresses the research question how to assess the collective innovation performance of multi-actor market-upstream projects from the project perspective, at the end of the projects, measuring the innovation outputs not from the point of view of their potential, but objectively and directly, by assessing their progress in maturity achieved during the project. The study uses a theoretical framework which draws upon and integrates innovation studies and project studies. As a basis of this assessment, the TRL ladder is refined and the refined scale is used for the assessment of the progress of the individual IOs. Based on this, also an assessment of the collective innovation performance of the projects is possible.

Empirically, our research focuses on collaborative RDI projects with multiple actors that were funded by the two recent European Union's Research and Innovation Framework Programmes: FP7 and Horizon 2020. The research design is an exploratory comparative multiple case study method [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF], with additional quantitative supporting evidence. We analysed 5 projects with 67 participating organizations on the basis of over 1100 pages of project documentations and data from 49 semi-structured interviews with project actors, resulting into over 46 hours of recording and over 800 pages of transcripts.

Firstly 54 innovation outputs (IOs: innovative technologies, methods, tools, processes…)

were identified in 5 projects, and then the framework was developed to assess their maturity progress on the way to (potential) innovation. We found that the relative increase of the required efforts between the steps was quite similar independent of the specific technology, while the absolute numbers differed significantly. We thus assigned average values of scores to the increments between the levels of the refined scale, to 'normalise' the progression in the maturation of the technologies according to their difficulty. By adding the average scores, the innovation performance of the project can be assessed. We then demonstrate the adequacy and interest of the framework, the policy implications of the study, as well as how it contributes to project management.

The remainder of the paper is organized as follows: we firstly provide the theoretical framework that guided our study, highlighting the research gap. Then we explain the research setting and the research design, present and discuss our findings, and explain the limitations of our research. We conclude by summarizing our theoretical contribution as well as policy and other practical implications of our research, and suggest future research directions.

5.2 Literature review 5.2.1 Assessing collective innovation performance in RDI projects

Innovation outputs in collaborative multi-actor RDI projects, and their progress

Public agencies around the world encourage collaborative research and innovation, and support multi-actor collaborative RDI projects, with considerable numbers of heterogeneous actors, generating new knowledge and innovative technologies at the same time [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]. These projects have characteristic properties, e.g. autonomy, equality and collective responsibilities of the organizational actors, and relative lack of flexibility to shift resources (vom Brocke and Lippe, 2015). The selected consortia of organizations jointly produce a variety of outputs which are related to the different objectives and timelines of the project actors [START_REF] Mannak | A temporal perspective on repeated ties across university-industry R&D consortia[END_REF]: publications, databases, patents, and also innovation outputs (IOs). In this paper, we define the IOs as the innovative technologies, e.g. new materials, methods, procedures, software, deployments, products…) that are advanced in the project providing evidence of successful validation, demonstration or application. The key performance indicators (KPIs) of the public funding programmes refer to the level of advancement of IOs, ranging from the number of testing activities and demonstrated prototypes to the number of deployment of innovative technologies in operational environments and new products close to the introduction to the market (e.g. EC, 2015, Spanos andVonortas, 2012, Kostopoulos et al., 2019). The efforts required for producing different types of IOs differ largely [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], and may be influenced by variety of contextual conditions [START_REF] Perkmann | Academic engagement and commercialisation: a review of the literature on universityindustry relations[END_REF][START_REF] Dietrich | The dynamics of collaboration in Multipartner Projects[END_REF], such as collaboration quality, historic experience of joint work, the project context [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF][START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF], level of knowledge integration mechanisms [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF][START_REF] Berggren | Relating knowledge integration and absorptive capacity: Knowledge boundaries and reflective agency in path-dependent processes[END_REF][START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF], project leadership [START_REF] Nishimura | Internal and external discipline: The effect of project leadership and government monitoring on the performance of publicly funded R&D consortia[END_REF] and others.

Innovation has been defined as "a new or improved product or process (or combination thereof) that differs significantly from the unit's previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)" (OECD Oslo Manual, 2018, p. 20). It requires and creates knowledge to develop ideas, models, technologies, which form the basis of innovations, but to become ' innovation', technological advancements shall be implemented (Oslo Manual, 2018[START_REF] Gault | Defining and measuring innovation in all sectors of the economy[END_REF].

Innovation process is not linear and includes various steps to generate knowledge and to transform it into artefacts [START_REF] Pavitt | Innovation processes[END_REF][START_REF] Cooper | Winning Businesses in Product Development: The Critical Success Factors[END_REF][START_REF] Cooper | Winning at new products: Creating value through innovations[END_REF]. In the middle of the innovation process, in-between idea generation and commercialization, lays the phase of the development and validation of technologies [START_REF] Eisenhardt | Accelerating Adaptive Processes: Product Innovation in the Global Computer Industry[END_REF], potentially leading to innovations. In the development process, the technology is matured, starting from first ideas and lab experiments or software prototypes to demonstrations in the real target environment (e.g. an oil platform or an airplane). This implies changes in complexity (Olechovsky et al., 2020), time needed for maturation [START_REF] Kenley | An Analysis of TRL-Based Cost and Schedule Models[END_REF], implementation environment, costs, risks [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF][START_REF] Webster | Aligning technology and institutional readiness: the adoption of innovation[END_REF][START_REF] Cooper | Winning Businesses in Product Development: The Critical Success Factors[END_REF], knowledge base, socio-technical infrastructure, user relations, market, policies and regulations [START_REF] Nakamura | Multi-level Perspectives with Technology Readiness Measures for Aviation Innovation[END_REF].

The value of an innovation can be realized and assessed only after its implementation [START_REF] Gault | Defining and measuring innovation in all sectors of the economy[END_REF], and scholars analysed different dimensions of such value: scientific, commercial, social value, among others [START_REF] Capaldo | Knowledge maturity and the scientific value of innovations: The roles of knowledge distance and adoption[END_REF][START_REF] Phene | Breakthrough Innovations in the US Biotechnology Industry: The Effects of Technological Space and Geographic Origin[END_REF][START_REF] Cooper | Managing technology development projects[END_REF]. The projects have been conceptualized as value creation processes [START_REF] Winter | Projects and programmes as value creation processes: A new perspective and some practical implications[END_REF][START_REF] Vuorinen | Value-oriented stakeholder influence on infrastructure projects[END_REF][START_REF] Ahola | Purchasing strategies and value creation in industrial turnkey projects[END_REF]. In the project context, value definition embracing benefits, or satisfaction of needs, and costs, or use of resources [START_REF] Laursen | Taking stock of project value creation: A structured literature review with future directions for research and practice[END_REF], and the concept of value creation is closely interconnected with the concept of performance of the project [START_REF] Green | Value creation in projects: Towards a narrative perspective[END_REF].

Assessment of innovation outputs and innovation performance in RDI projects

Performance measurement in RDI settings has attracted the interest of innovation scholars and policy makers since long. In private sector, scholars measure innovation performance in terms of commercial success and financial measures (e.g. Belderbos et al., 2018, van Beers and[START_REF] Van Beers | R&D Cooperation, Partner Diversity, and Innovation Performance: An Empirical Analysis[END_REF]. However, these indicators are not well-suited for publicly funded RDI projects [START_REF] Kim | Development of an evaluation framework for publicly funded R&D projects: The case of Korea's Next Generation Network[END_REF], as they are market upstream. The performance of RDI projects has been assessed so far mostly from the 'research side'(e.g. [START_REF] Grimaldi | Assessing collaborative, pre-competitive R&D projects: the case of the UK LINK scheme[END_REF]Von Tunzelmann, 2002, Fernandes et al., 2017). Assessment of the 'innovation side' of the project performance requires to put the focus on their innovation outputs (IOs), but these are complex and may still be relatively far from the market, so the baseline of the assessment is not clear [START_REF] Perkmann | How should firms evaluate success in universityindustry alliances? A performance measurement system[END_REF] and their value is difficult to assess [START_REF] Aguiar | European cooperative R&D and firm performance: Evidence based on funding differences in key actions[END_REF]. That is why scholars often assess the IOs of such projects subjectively, from the point of view of the project actors (e.g. [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF][START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF][START_REF] Arroyabe | R&D partnerships: An exploratory approach to the role of structural variables in joint project performance[END_REF][START_REF] Arranz | Network Embeddedness in Exploration and Exploitation of Joint R&D Projects: A Structural Approach[END_REF], or by analysing how adequate the technical solution is in comparison to the needs of the users (e.g. [START_REF] Bendoly | System Dynamics Understanding in Projects: Information Sharing, Psychological Safety, and Performance Effects[END_REF][START_REF] Hagedoorn | Research partnerships[END_REF]. Subjective assessments are useful but should be complemented by objective measures [START_REF] Perkmann | How should firms evaluate success in universityindustry alliances? A performance measurement system[END_REF], Nepelsky et al., 2019).

The available objective data to measure performance at the early stages of innovation processes is limited [START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF][START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF]. Innovation studies have become largely econometric [START_REF] Martin | Introduction to discussion paper on 'The sciences are different and the differences are important[END_REF] and therefore are based on indicators of the IOs that are more easily available [START_REF] Grimaldi | Assessing collaborative, pre-competitive R&D projects: the case of the UK LINK scheme[END_REF], mostly numbers of patents [START_REF] Hagedoorn | Measuring Innovative Performance: Is There an Advantage in Using Multiple Indicators[END_REF][START_REF] Van Roy | Technology and employment: Mass unemployment or job creation? Empirical evidence from European patenting firms[END_REF][START_REF] Du | Managing open innovation projects with sciencebased and market-based partners[END_REF][START_REF] Maietta | Determinants of university-firm R&D collaboration and its impact on innovation: A perspective from a low-tech industry[END_REF][START_REF] Soh | When do firms benefit from university-industry R&D collaborations? The implications of firm R&D focus on scientific research and technological recombination[END_REF][START_REF] Agrawal | Putting Patents in Context: Exploring Knowledge Transfer from MIT[END_REF][START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF]. However, the counting of patents alone does not provide the full picture on the IOs of the projects [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF][START_REF] Hung | Social networks, technology ties, and gatekeeper functionality: Implications for the performance management of R&D projects[END_REF][START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]: the patenting strategy of organizations has a strong influence on whether or not patents are applied for and in some sectors, the value of patents may not be considered to be very high, thus the effort that has to be invested is not made. Martin (2016a), when putting forward twenty challenges for innovation studies, pointed out that the existing innovation-related indicators may miss innovation activities with non-patented results. This holds true for many activities of the type of multi-actor RDI projects analysed in this study. Innovation scholars repeatedly call for better measurement of IOs, highlighting that it would provide a better understanding about the (partial) results of innovation process, and better policy learning [START_REF] Nepelsky | Innovation and innovator assessment in R&I ecosystems: the case of the EU Framework Programme[END_REF][START_REF] Godin | National Innovation System: The System Approach in Historical Perspective[END_REF][START_REF] Arundel | Advancing innovation in the public sector: Aligning innovation measurement with policy goals[END_REF][START_REF] Quintane | Innovation as a Knowledge-Based Outcome[END_REF]. Equally important is the motivational and action-oriented role of assessments, as they enable monitoring of on-going initiatives and may help to point out directions for improvements [START_REF] Perkmann | How should firms evaluate success in universityindustry alliances? A performance measurement system[END_REF][START_REF] Neely | The Evolution of Performance Measurement Research -Developments in the Last Decade and a Research Agenda for the Next[END_REF].

Assessment of the IOs can be done from different perspectives, e.g. in terms of inputs (labor, financial resources), the outputs of the process, the market potential of the IOs, the scientific value embedded in the IO etc. (e.g. [START_REF] Capaldo | Knowledge maturity and the scientific value of innovations: The roles of knowledge distance and adoption[END_REF][START_REF] Cooper | From experience: the invisible success factors in product innovation[END_REF][START_REF] Birchall | Capabilities for strategic advantage: leading through technological innovation[END_REF]. It is obvious that any direct measures of IOs at the end of the projects, before the implementation of the innovations, can only indicate their potential in the future or their technical progress in the past. Nepelsky et al. (2019) analyse the collective performance of multi-actor RDI projects through the analysis of the innovation potential of the IOs. These authors use the data of the Innovation Radar (IR) [START_REF] De Prato | Innovation Radar: Identifying Innovations and Innovators with High Potential in ICT FP7, CIP& H2020 Projects[END_REF][START_REF] Nepelski | Organizational diversity and innovation potential of EUfunded research projects[END_REF][START_REF] Nepelsky | Innovation and innovator assessment in R&I ecosystems: the case of the EU Framework Programme[END_REF], an innovation management tool used in EU-funded projects which aims to identify high-potential innovations and innovators, to provide guidance to the project team towards further steps to reach the market [START_REF] Van Roy | Technology and employment: Mass unemployment or job creation? Empirical evidence from European patenting firms[END_REF][START_REF] Ec | Horizon 2020 Interactive Dashboard[END_REF]. The innovation readiness dimension of the IOs (which is one of the dimensions of their quality) is measured through the 'go to the market' lens, and the progress the of IOs within the project is not considered by the IR in detail, as this is not its goal.

In a separate literature stream, project studies assess innovation performance and IOs in the project context, from the perspective of project value creation, where value has been defined

as the ratio of benefits/costs (e.g., Laursen andSvejvig, 2016, Martinsuo et al., 2019), and benefit as "improvement resulting from a change… that is perceived as positive by one or more stakeholders" (Laursen and Svejvig, 2016, p. 737). This includes the notions of the quantity and quality of project outputs (e.g. [START_REF] Martinsuo | Evaluating the organizational impact of product development projects[END_REF][START_REF] Hoegl | Financial constraints in innovation projects: When is less more?[END_REF][START_REF] Evanschitzky | Success factors of product innovation: An updated meta-analysis[END_REF][START_REF] Atkinson | Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria[END_REF][START_REF] Andersen | Rethinking Project Management: An Organisational Perspective[END_REF]. In publicly funded projects, cost and time are fixed parameters, the question is, how to assess the quality of IOs. [START_REF] Kim | Development of an evaluation framework for publicly funded R&D projects: The case of Korea's Next Generation Network[END_REF] studied publicly-funded collaborative RDI projects several years after the project end, and assessed the quality of the IOs by analysing the level of technologies, using the proxy of the patent productivity ratio. This however is not possible when the IOs are not patentable or deliberately not patented, and also not possible at the end of the projects. Quality of IOs in collaborative RDI projects, from the technology perspective, is often assessed from the point of view of comparison between achieved and planned promised outputs [START_REF] Vom Brocke | Managing collaborative research projects: A synthesis of project management literature and directives for future research[END_REF]Lippe, 2015, Perkmann et al., 2011), as each IO is specific. In the context of projects with multiple IOs, this suggests that the quality of each IO must be measured and then be considered for all IOs in the project. In any case, the quality of the results shall be assessed comparing to a certain baseline, or reference values [START_REF] Perkmann | How should firms evaluate success in universityindustry alliances? A performance measurement system[END_REF]. To investigate the questions of the assessment of IOs and the quality level of IOs further, we draw on the literature concerning assessments of technologies at the different stages of the innovation process, within and outside the context of collaborative RDI projects.

Assessment of innovative technologies

Indicators to assess market upstream technologies and their technical progress

Innovations and R&D performance are assessed by most firms [START_REF] Markham | Product Development and Management Association's 2012 Comparative Performance Assessment Study[END_REF][START_REF] Chiesa | Performance measurement in R&D: exploring the interplay between measurement objectives, dimensions of performance and contextual factors[END_REF][START_REF] Lazzarotti | A model for R&D performance measurement[END_REF][START_REF] Ojanen | Coping with the Multiple Dimensions of R&D Performance Analysis[END_REF]. The early stages of innovation process are critical and strongly influence the future success of innovations [START_REF] Markham | The Impact of Front-End Innovation Activities on Product Performance[END_REF][START_REF] Dziallas | How to evaluate innovative ideas and concepts at the front-end? A frontend perspective of the automotive innovation process[END_REF][START_REF] De Oliveira | What do small business owner-managers do? A managerial work perspective[END_REF][START_REF] Kock | How ideation portfolio management influences front-end success[END_REF] mainly because selecting the right ideas and projects is crucial [START_REF] Elhorst | Evaluating proposals in innovation contests: Exploring negative scoring spillovers in the absence of a strict evaluation sequence[END_REF][START_REF] Van Den Ende | The Front End of Innovation: Organizing Search for Ideas[END_REF][START_REF] Hunt | Modelling resource allocation of R&D project portfolios using a multi-criteria decision-making methodology[END_REF][START_REF] Cooper | Maximizing Productivity in Product Innovation[END_REF]. Other researchers focus on the last stage of this process, commercialization, or market launch (e.g. Hittmar et al., 2015, Dewangan and[START_REF] Dewangan | Towards a holistic enterprise innovation performance measurement system[END_REF]. Both directions are not relevant to the assessment of the innovation outputs in the RDI projects considered here, the selection of the "what" to innovate has been done when the proposal is submitted, and the commercialization is outside of the scope of the project itself. Innovative technologies have also been assessed according to their level of innovativeness and creativity, input of collaborators, number of generated ideas and related patents [START_REF] Adams | Innovation management measurement: a review[END_REF][START_REF] Salimi | Evaluating firms' R&D performance using best worst method[END_REF], Chiesa et al., 1996[START_REF] Cooper | Stage-gate systems: a new tool for managing new products[END_REF], 2017[START_REF] Hart | Industrial companies' evaluation criteria in new product development gates[END_REF]. Scholars concur that the assessment of IOs at this phase is complex and challenging [START_REF] Henttonen | Searching for appropriate performance measures for innovation and development projects[END_REF][START_REF] Birchall | Innovation performance measurement: Current practices, issues and management challenges[END_REF][START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF][START_REF] Hart | Industrial companies' evaluation criteria in new product development gates[END_REF], the time lag to the 'tangible' outputs is large [START_REF] Wang | The lag structure of the relationship between patenting and internal R&D revisited[END_REF].

Thus, it is not surprising that methods for the assessment of the progress of the IOs in this stage of the innovation process is largely missing in the literature [START_REF] Aiello | Innovation and productivity in family firms: evidence from a sample of European firms[END_REF]. For instance, [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF], in a thorough literature analysis of over 250 papers published in 1980-2015, did not identify any indicator used to assess the technologies at the development and validation stage, except patents and other IP indicators. Literature in this stream also repeatedly highlights the need for a better understanding of the innovation process (e.g., [START_REF] Birchall | Innovation performance measurement: Current practices, issues and management challenges[END_REF][START_REF] Edison | Towards innovation measurement in the software industry[END_REF], and calls for future research to assess market upstream IOs, with the results being useful for researchers, managers, and policymakers [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF]. These authors also pointed out that, while the scheme and indicators may remain common across sectors, their values shall be defined by each organisation, as individual IOs contain many sector-dependent specificities [START_REF] Schwartz | Measuring the effectiveness of R&D[END_REF].

In New Product Development (NDP) projects in private sector, market upstream IOs are traditionally measure using the three criteria described in Section 5.2.1.2: quality, quantity and cost [START_REF] Ojanen | Coping with the Multiple Dimensions of R&D Performance Analysis[END_REF], Project Management Institute, 2013[START_REF] Markham | Product Development and Management Association's 2012 Comparative Performance Assessment Study[END_REF][START_REF] Henttonen | Searching for appropriate performance measures for innovation and development projects[END_REF][START_REF] Hoegl | Financial constraints in innovation projects: When is less more?[END_REF], Lazarrotti et al., 2011). R&D efficiency measures reflect the ratio of technical progress to costs (e.g. [START_REF] Bican | Managing innovation performance: Results from an industryspanning explorative study on R&D key measures[END_REF], where technical progress is measured as the difference between the past performance (technical parameter) of a technology (product, process) and the current performance, and could be also compared with the parameters of the competitors [START_REF] Chiesa | Searching for an effective measure of R&D performance[END_REF]. This makes the indicator specific to each technology.

We argue that the technical progress, which is the main technological objective of RDI projects, can be measured at the level of each individual IO (new material, new technology, new method, new software, new component, etc.) in terms of the advancement in the maturity of this IO between the end point that is reached within the project and the starting point (the baseline). The concept of "increase of maturity" describes the progress of the development of a technological innovation from the first idea to a product on the market or to an installation in a real operative environment (a production plant, a construction site, a running productive software system). This advancement reflects the quality of the IOs from the project perspective. The most commonly used methodology to assess the maturity of technologies is the scale of technology readiness levels (TRL). Its main characteristics and some shortcomings which are relevant for this research are briefly described in the next section.

Maturity of innovative technologies: Technology readiness level

In order to structure funding schemes, to define handovers between different organizations or consortia at certain levels of maturity of innovative technologies, to manage and reduce technological risks, and to communicate between different actors, NASA formally defined the so-called technology readiness level (TRL) scale in the late 1980s. It consists of 9 levels and describes the maturity of a technology, i.e. how ready (mature) it is on its way towards the final use in flight or space [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF][START_REF] Peters | Technology Maturity Assessments and Confidence Intervals[END_REF]. Handovers between different players and consortia were assumed by the government agencies to happen mostly at TRL levels 3 (from basic research to technology development) and 7 (from prototypes to the development of flight-proven systems) [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF]. These levels were tailored to flight and space programs with very high requirements on the reliability of the innovative technologies and a very high cost of the development of flight-proven systems. [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF]) reports that the cost of moving from TRL 6 to 9 is 90% of the total development cost. This does not include the cost of the developments that were terminated on lower TRL levels because of technical problems or too small expected benefits. However this increment is largely domainspecific [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], Olechovsky et al., 2020). It was also implicitly assumed in the introduction of the TRL ladder in the aerospace and defense sectors in the US that in moving up the TRL ladder, there is a significant attrition: many projects are funded at a relatively low cost at TRL 1-3, fewer on TRL 4-6, and then a significant investment is made into promising technologies for their further maturation [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF]. The TRL scale is standard now in space and defense-related funding and generally accepted in this domain.

Funding agencies and other organizations worldwide adopted the TRL approach and often distinguish between the stages of research, development and deployment, corresponding to TRL1 to 3 (fundamental research and experiments, considered to be the "playground" for research and technology push up to validation in the lab), TRL4 to 6 (technology development and proof of concept in a relevant environment) and TRL7 to 9 (technology maturation, demonstration and deployment [START_REF] Ec | Horizon 2020 -Work programme[END_REF][START_REF] Napp | A Survey of Key Technological Innovations for the Low-Carbon Economy[END_REF]. For instance, following the discussion about the "valley of death" [START_REF] Butler | Translational Research: Crossing the Valley of Death[END_REF] between initial research and technology development and putting the innovations to the market (HLEG-KET, 2011), the EU has defined and generally applied the TRL scale according to Table 5.1 in the Horizon 2020

Programme (EC, 2014) in a wording which is very close to the original definition by NASA; calls often specify the TRL that should be achieved by the end of the projects.

As Héder (2017, p.10) states, in the interpretation of the EU, "Technological Readiness Level is a metric that shows how far a technology is from being ready for use in its intended operational environment" -this is what makes it attractive for the topic of this paper: how can the progress of IOs be measured. It is however questionable that the TRL levels can be defined uniformly across sectors. The TRL ladder must not be confused with the commercial readiness level, it only measures the maturity of the technology and does not take other factors that are necessary for roll-out and introduction to the market into account. In the Australian Commercial Readiness Index (ARENA 2014[START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], all activities up to TRL 7 or even 8 correspond to Commercial Readiness Level 1 ("Hypothetical commercial proposition"). This is why an assessment of the IOs based on the market potential is difficult and highly speculative. The TRL definitions leave a lot of room for interpretations [START_REF] Cornford | Quantitative Methods for Maturing and Infusing Advanced Spacecraft Technology[END_REF][START_REF] Tan | A Probabilistic Approach to System Maturity Assessment[END_REF][START_REF] Peters | Technology Maturity Assessments and Confidence Intervals[END_REF], often leading to subjective, coarse, inconsistent, and inaccurate TRL assessments. A discussion of the difficulties of the transfer of the TRL scale to other sectors can be found in [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF]; other scholars also state that the TRL scale is very domain-specific, does not reflect nonlinear innovation process, and it does not necessarily work with diverse innovation processes and product cycles (e.g. [START_REF] Tan | A Probabilistic Approach to System Maturity Assessment[END_REF][START_REF] Peters | Technology Maturity Assessments and Confidence Intervals[END_REF]. Olechovsky et al. ( 2020) present a study of the adequacy and the deficiencies of the TRL scale mostly from an innovations management point of view by means of interviews with practitioners from a variety of sectors. These authors extracted 15 challenges of TRL implementations, structured into the domains of system complexity, planning and review and assessment validity. As the focus here is on the assessment of the progress of technologies, we only discuss those challenges which are relevant in this context.

The domain of system complexity in particular relates to large systems which are made up of many components. Here the issues of integration and connectivity, maturity of interfaces, lateral effects and overall system readiness level are prominent: the complexity increases with the increase of maturity. This is very context-dependent: it makes a big difference whether a solution consist of a few components or hundreds of them, and whether a failure of one component has a disastrous effect on the functioning of the overall system or not. Issues und the category "Planning and review" are mostly related to innovations management. When taking decisions on whether to progress with a development, it is needed to assess how much effort will be needed on higher TRL levels, and this is not uniformly a certain fixed multiple of the previous effort but depends on the specific circumstances. This issue points to the imprecision of the TRL scale and the subjectiveness of the assessment.

Other studies have shown difficulties to use the coarse TRL scale in multi-actor RDI projects in case of the presence of several innovative technologies of different maturity in the project, and due to the differences in the interpretations of the TRL terminology, especially at the TRL scales between TRL 4 and 7 [START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF]. "Given its use in a diverse range of systems within which technology is being developed, procured, and readied for market, it is surprising that TRLs have had relatively little consideration in the field of innovation studies" [START_REF] Webster | Aligning technology and institutional readiness: the adoption of innovation[END_REF]Gardner, 2019, p. 1231).

To sum up, our literature review suggests that three main concerns shall be addressed when assessing the collective innovation performance in multi-actor RDI projects: (1) The need to assess all IOs that are developed in the projects;

(2) The need to use a fine grained assessment scale, to have clear and as unambiguous levels of technical progress as possible;

(3) The assessment must be based on the difference between the initial maturity level of each IO, i.e.

the baseline, and the final level reached within the project. The link between advancement of maturity (progression between the levels) and project performance is shown in Figure 5.1.

Innovation studies and project studies are both interested in research on innovations and IOs, and scholars repeatedly call for a better connection between the two research streams [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018), especially when studying collaborative settings where knowledge integration is required [START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. This connection is salient in the context of multi-actor RDI projects. In this context, scholars called a decade ago for research leading to a better understanding of the performance of the projects (Pandza et al., 2011, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], but studies objectively and directly assessing the innovative technologies and collective performance of the projects, are still almost inexistent [START_REF] Nepelski | The organisational and geographic diversity and innovation potential of EU-funded research networks[END_REF].

Fig. 5.1. Link between advancement of maturity and project performance

Recent calls go in the same direction, pointing out the needs to investigate additional metrics of performance of the projects beyond publications and patents, using a case study approach [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF], to take into account progress of activities with nonpatented results (Martin, 2016a), calling for research to provide a more fine-grained, nuanced view on innovation outputs of multi-actor projects, which will benefit the innovation research [START_REF] Collinson | Recombination for innovation: performance outcomes from international partnerships in China[END_REF], and highlighting a need for "research on innovation outputs … measurement which will eventually contribute to reducing uncertainty in policymaking" (Janger et al., 2017, p. 40). This exploratory study contributes to the reduction of this research gap, to a better understanding of the collective performance of publicly funded RDI projects, while stimulating further discussions and relevant research developments. We pose the following research questions: how to assess the 'innovation side' of the performance of the projects objectively and directly, first on the level of the innovative technologies (innovation outputs, IOs) and then on the project level.

Research setting

The research setting is defined as multi-actor collaborative RDI projects that target creating There are three main reasons that make this setting particularly interesting for both innovation and project research. First, multi-actor collaborative RDI projects that generate innovation outputs, in addition to new knowledge, have become an important element of RDI funding, and their number is steadily growing. Second, usually the projects lead to the generation of market-upstream innovative technologies, much more nowadays than in the past, but it is not clear how to measure their progress. Third, because funding agencies regularly discuss the need to incentivize "well performing projects", which is obviously not possible without measuring the collective innovation performance during the course and at the end of the projects.

The field is difficult to access (vom Brocke and Lippe, 2015): detailed data on the innovations are not publicly available; consultation of internal project documents by external organizations is possible only with unanimous authorization from all project partners. Given these difficulties, it is not surprising that this field so far remained largely outside the focus of innovation scholars. The authors of this paper were in the privileged position to have full access to the materials of the projects under study and to have trusted work relationships with actors from the projects which enabled them to obtain additional information, including interviews, where needed.

Research design

This research is part of a larger study that takes a deeper look into multi-actor collaborative RDI projects. The research design is exploratory multiple case study of 5 projects [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]. Exploratory case studies [START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF] are needed when there is a lack of detailed preliminary research and hypothesis to be tested [START_REF] Mills | Encyclopedia of Case Study Research[END_REF]. Multiple case studies provide a stronger base to build theory [START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF], to enable comparison and demonstration of replication [START_REF] Eisenhardt | Better Stories and Better Constructs: The Case for Rigor and Comparative Logic[END_REF][START_REF] Bakker | Temporary organizing: Promises, processes, problems[END_REF], and create more robust theory [START_REF] Eisenhardt | Theory Building from Cases: Opportunities and Challenges[END_REF]. While quantitative studies are looking at a representative sample to make generalisations, the logic of multi-case qualitative studies is to identify patterns which could confirm the presence of similar phenomena across cases and to shed light on the mechanisms behind the phenomena, to generalize them further to theory [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF].

In the data collection phase, we selected 5 projects for investigation using 4 selection criteria: homogeneity, variety, comparability and availability of data. Summaries of the selected projects are presented in Appendix 5.1, where names of celestial constellations were given to the projects for their anonymization: HERCULES, PEGASUS, GEMINI, PERSEUS, and ORION. The five projects are characterized by: (1) large diversity of organizations from academia, large industry and technology providing SMEs; (2) strong application dimension;

(3) duration of 3 to 4 years; (4) presence of a strong digital and IT dimension. The projects had between 10 and 17 organizational partners, different sizes of budgets (between 3 and 6 M€) and different focus: some were more research-oriented, others more innovation-oriented.

Then we collected empirical data about the IOs developed in the five projects, using over 330 pages of project proposals and over 780 pages of confidential project reports and other internal documents, e.g. exploitation plans, Innovation Radar documents when available, registers of project IPs, meeting agendas of the consortium, periodic reports. In addition, for triangulation purpose, we also collected data on efforts, as periodically reported by partners (person months spent, according to the timesheets 11 , which corresponds to costs). These data were reported per workpackage; we thus had to reconstruct the effort per IOs with the help of the project partners in some cases. We thus were able to identify the maturity level of the innovations at the end of their development in the projects, first following the TRL scale from the literature (Table 5.1), and the relevant efforts. We only considered the IOs which were available by the end of the project, as reported in the final project documents. We then checked the starting level of each of these IO in the projects, i.e. the baseline. Additionally, we collected data during 49 semi-structured interviews with 31 project actors. In each project, representatives of between 40% and 65% of the participating organizations were interviewed. In order to obtain comparable data, over 80% of the informants were CEOs of SMEs, team leaders within large corporations, or tenured university professors, they were responsibles for specific IOs in the studied projects; additional informants were main contributors to the developed IOs. A detailed interview guide was developed. The interviews captured, among other, the qualitative judgements about the project results and their value, progress in degree of maturity of IO, paths of progression (steps passed or not, reasons, difficulties experienced during the progress of maturation of these technologies…), benefits and importance of progression, relative efforts comparing between the stages. The interviews were organised in several rounds. All interviews were recorded, with prior consent of the 11 We considered person months as minimum and indicative parameter: for instance, industrial partners did not necessarily book efforts of technicians on the project, and this was explained during the interviews, but these staff was involved in the projects, and in some cases to a large extent. Efforts of organisational support was not reported in person month, as it was accounted as "indirect costs", or overheads. In other cases, the project budget was reached, and the effort of the project staff was not recorded completely.

TRL 9actual system proven in operational environment TRL8 -System completed and qualified TRL7 -System prototype demonstrated in operational environment TRL6 -Technology demonstrated in relevant environment TRL5 -Technology validated in relevant environment TRL4 -Technology validated in lab TRL3 -Experimental proof of concept TRL2 -technology concept formulated TRL 1basic principles observed informants, and transcribed verbatim. This resulted into over 46 hours of recording and over 850 pages of transcripts (Table 5.2).

In the data analysis stage, several steps were performed, some in parallel. The first step aimed at understanding the progress of each of the 54 IOs, and whether the IOs can be homogeneously classified using the existing TRL scale. When the levels comprised heterogeneous levels of maturity of IOs (this was visible using documentary analysis or through the interviews), we noticed the need for a finer-grained classification. We also checked the levels of the IOs with the project coordinator and/or one of the key project partners. The interpretative step helped to understand the quality construct, how it is related to the advancement of maturity, and how the project actors viewed the advancement. During this step, the explanations provided by the project actors were analyzed based on the abstractive process of open coding [START_REF] Strauss | Basics of qualitative research: Techniques and procedures for developing grounded theory[END_REF]Corbin, 1998, adapted by Gioia et al., 2013). The interviews transcripts were coded using the NVivo 12.1 software. Appendices 5A and 5B

(supplementary materials) show the data structure and data table for the refinement of the scale (categories and principles), and Appendices 5C and 5D provide the data structure and data table for advancement of IOs (relative efforts, pivotal steps…). In the category 'scale'

(data structure in Appendix 5A), we grouped 15 concepts of 1 st order into 7 aggregate and more abstract themes of the 2nd order, which were combined into 3 aggregate dimensions. In the category 'advancement of IOs' (data structure in Appendix 5C), we grouped 14 concepts of 1 st order into 6 themes of the 2nd order, and 2 aggregate dimensions. Then a fine-grained scale for the advancement of the innovative technologies was developed and augmented by an exemplary scoring system (weighing of the steps between levels), according to the efforts (resources) needed to progress in the maturity of IOs that was made during the project. The result was compared between the cases. As it is generally agreed upon, and also confirmed by our investigations, the necessary efforts to progress increase when progressing to higher levels, so for a fair and unbiased comparison, the increase in the required effort has to be taken into account when comparing different advancement. Such a scoring system is specific for a certain context, and the one presented in the next section was developed for the projects considered here. So the concrete numbers illustrate the general approach but may have to be modified depending on the domain. The average scores were developed using both theoretical constructs, knowledge of the field, and empirical data, i.e. qualitative feedback triangulated with quantitative data for selected IOs. It is worth to note that we took as much care as possible to make sure that the scores reflect the "objective" effort that is needed to progress, i.e. not taking into account contextual conditions related e.g. to collaboration issues or external factors: we particularly looked at the innovative technologies where collaboration was smooth. We complemented the qualitative data by quantitative evidence in three projects (38 IOs): these projects were from a comparable domain with comparable type of the project (research and innovation action funding scheme), also, it was possible to more carefully reconstruct quantitative data in these projects based on detailed reports of resources.

The average scores were applied to all IOs in the largest project, ORION, where practically all levels of maturity of technologies were achieved. We sent the working paper, presenting the fine-grained scale and scores, to the ORION project partners, collected comments, presented and discussed the revised framework and the results at a meeting with 36 representatives of the project partners. The framework was agreed upon by the participants, with some suggestions of adjustments and clarifications. For additional interviews, the materials (scale and average scores based on the previously collected data) were sent to key project partners and discussed with them. The interviews focused on the discussion how the proposed average scores reflect the informants' experience in the studied projects, concerning the effort needed to progress their innovative technology from one level of maturity to the next.

Once the scale and the scoring were available, the assessment framework was applied to measure the advancement of the IOs of all projects and the performance of the projects. Then the results were compared and analysed.

Results

Refinement of the maturity scale

Identification of outputs

We identified 54 IOs that were generated at the end of the five projects under study, and their maturity level, using the conventional TRL scale (Table 5.3). We also identified knowledge (research) outputs. The nature of these IOs differed: these were innovative technologies, methods, processes, tools or models, physical devices, software etc. The spectrum of the IOs was broad, comprising from sensing devices, software, simulation, modeling and model management, new algorithms and their embedding into industrial operations.... The development usually started from existing foundations created by previous work and from different levels of maturity, the goal was to advance the maturity of the technologies.

You write the proposal that way, and the goal is to bring TRLs to a higher level, and make it ready, for example, for implementation … in the proposal phase you describe how you will increase the TRL level… the intention is to develop it further because the goal is to bring it to the market. ID31_PERSEUS_res

In GEMINI, one core software system was developed with several innovative components, using agile development methods. In contrast, ORION led to many independent IOs at various levels, from prototypes to pilot deployments and permanently rolled out innovations. Some of these IOs were not initially planned, they were pursued as opportunities were realized, on the other hand, the progress of others was stopped well before the project end because of technical difficulties or a (predicted) unsatisfactory ratio of efforts needed to complete to benefits. To become productive, all technologies had to be embedded into an existing environment that consisted of specific hardware (e.g. machines, processing equipment, transport units) and IT and automation systems. The interfaces to both kinds of systems were important elements of the development. In the projects considered, the innovative technologies were new elements of operational systems rather than completely new largescale systems12 . For a productive use, the reliability of the solutions needed to be high, but not ultra-high as e.g. if rolled out in cars or aircrafts 13 . Most of the innovative technologies were not patented, only one patent application was identified across five studied projects. One reason for this was explained by a participant: "It's not something that you could patent, the ideas were new but not new in the sense of new molecule … but we realized it's such a large effort to implement it, that being first on the market is far more important than having a patent". ID6_HERCULES_large ind

Refinement of categories of IO (maturity levels)

The straightforward application of the conventual TRL categories to classify the identified IOs was difficult as already suggested by the theoretical framework: the existing categories (shown in bold face below) were not homogeneous and did not fully fit to the studied projects.

The definitions were confusing for the project actors and not specific enough:

"For me, it was an operational environment in the pilot, but it wasn't a system. For him, it was a system but not in an operational environment." ID52_GEMINI_SME

Several elements emerged, showing that differentiation between categories (levels) was needed, comparing to the existing TRL scale (Appendices 5A and 5B):

(1) a clear differentiation of component versus system, as distinguished by the project actors:

"It depended a lot on … the complexity and maturity of the components" ID49a_ PERSEUS_large ind "Our hope is that they will have a new system… and that will use the component that we developed" ID25_ORION_SME

"This was a system, and we had already have components demonstrated in operational environment…separate components. Putting it together is not so simple." ID52_GEMINI_SME

(2) type of the environment, where the project outputs were tested, simulated, validated, demonstrated or deployed; for instance, a digital environment did not fit to the conventional definitions.

"It was a synthetic case … technology was validated in a few synthetic cases… not 100% realistic cases… on own PC." ID51_ORION_SME "A demonstration of the prototype in the relevant environment was achieved, although in the last minute but it was achieved…. we fought very hard to actually have this integrated demonstration… small-scale but real…" ID4_PERSEUS_acad

« The best we got was the demonstration of one of these solutions in a real environment, otherwise for three out of four of the applications it was just demonstration in simulations... It was very nice that we did this real-life demonstration with real data of the development, that was quite an achievement." ID2_PEGASUS_coord

In addition, some categories at the high maturity level of the existing TRL scale required refinements, as they described different achievements:

(3) number of deployed applications (single or multiple applications). For instance, in ORION and PERSEUS, some deployments were done with one application, while in GEMINI they were done in multiple settings.

"The only change that we had was introduced was the third pilot." ID1_GEMINI_coord Because of the nonlinear nature of innovation process, some levels of TRL were not used in the projects: e.g. in ORION, some IOs did not go through the stage of deployment in nearoperational environment…

"TRL level 6, does it exist, I don't know for us. We went directly from five to seven". ID51_ORION_SME

…while in PERSEUS, a component was demonstrated and then integrated into a system only later on, when the output was at higher level of maturity. Also, different entry points were regularly mentioned: development of some IOs started from an advanced stage, others were developed from a quite preliminary initial stage:

"We directly started from the successful demonstration of (partner name)…from the results of (previous project) …and we enhanced this." ID5_ORION_coord "This typical application … that was new and that was the step that we had to make, almost from scratch." ID31_PERSEUS_acad

Refinement of scale: fine-grained categories of maturity levels

Based on interviews and their interpretations, as presented above, we refined the categories of the scale as explained below, to make sure it fits to all 54 IOs. We proceed from the highest to the lowest considered level.

Refinement of the category "System completed and qualified" (TRL8).

The wording here comes from the world of aircraft, spacecraft, also automotive. In the context of the publicly funded RDI projects analyzed in this study, with the characteristics listed above, the highest achieved level of maturity was that of a beta-version (prototype) of a product, or of a significant improvement of an existing product, given for testing to potential users or customers. Another case of a high level of maturity was that the innovative technology was rolled out within the productive environment of one or more project partners, e.g. it is actually used in day-to-day operations in an industrial plant over an indefinite period of time. Both new and improved products and a roll-out in an industrial environment require a high level of robustness and reliability but there is a difference between a product which is given to several external users and an internal solution, where some updates and adaptations are tolerable. Still the requirements in all cases are significantly higher than for a pilot application (see below), in particular there must not be a need for frequent maintenance and the necessary features for long-term productive use must all be present. As mentioned above, in all cases of interest, the innovative technologies have been integrated into a complex environment with which they interacted intensely. Consequently, we refined the category TRL8 into 4 new categories: roll out to one application (TRL8.1), roll out to multiple applications (TRL8.2), prototype of improved own product given for testing to customers (TRL8.3) and prototype of new product given for testing to customers (TRL8.4).

Refinement of the category TRL 7, "System prototype demonstrated in operational

environment". The notion of prototype is vague, prototypes can be anything from a first realization to a serious implementation. We therefore used the term "pilot deployment" which is common in developments for and in industry, in line with the feedback of the project actors.

The understanding of a "pilot" is a realization of the innovative technology that has all key features that are needed to prove its value in the real application environment. The purpose of a pilot is to demonstrate that the expected benefits materialize and that it integrates well into the workflow and the hard-and software environment. Deployment means that the pilot is in operational use for a significant period of time, but while and after gaining experience it may have to be developed further to better meet the requirements of the operational environment, before roll-out. It can also happen that the pilot is not developed further into a permanent solution, e.g. because it did not fully meet the expectations, or the benefits did not warrant the necessary development efforts towards a long-term stable solution, or because there was no convincing concept for continuous maintenance and adaptation. "Pilot deployment" was refined this into two sub-categories: deployment of one pilot (TRL7.1) or multiple pilots (TRL7.2) which requires a higher degree of maturity and comes with a higher effort. Pilots always require an integration effort into the environment where the pilot is deployed.

The differences between the categories "Technology demonstrated in relevant environment" (TRL6) and "Technology validated in relevant environment" (TRL5) are not obvious from the wording. According to Oxford Reference14 , validation means "The act or process of making something valid, ratifying it, or checking that it satisfies certain standards or conditions », other sources define it as a proof of correctness. Demonstration can be understood as showing the working of the technology to the users. If one goes back to [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF] where the TRL scale is discussed in detail, the wording is: TRL 5:

"Component and/or breadboard validation in relevant environment » and TRL 6 :

« System/subsystem model or prototype demonstration in a relevant environment (ground or space) », and in the discussion, the difference between the two categories mostly is in the definition of the «relevant environment ». To clarity the difference between the categories, we use "Prototype demonstrated in operational or near-operational environment", which is understood as the prototype working successfully in a (temporary) installation at an industrial site or in a demonstration setting (pilot plant), or working with real live data from the application in case of software solutions. A prototype is a temporary, simplified realization of the innovative technology (component or system) that does not have to have the full set of features (e.g. it could only have a rudimentary user interface or, if the focus is on the user interface, the data processing may not be fully robust yet). In contrast to the pilot, a prototype installation is done under the assumption that the prototype will be removed and replaced by an operational installation (the pilot) later in the process. Demonstrated means that the main functionality of the prototype was in line with the technical requirements in the operational or near-operational environment. We subdivided this category into "component prototype demonstrated in operational or near-operational environment" (TRL6.1) and "system prototype demonstrated in operational or near-operational environment" (TRL6.2). A system prototype integrates one or more innovative components into a larger system as well as with the environment.

The category "Technology validated in relevant environment" (TRL5) in the original literature differs from TRL 6 mostly in the type of environment where the validation takes place. We precise it here by stating that the prototype is demonstrated in a synthetic, but realistic environment (mock-up), or in extensive computer simulations. This is a standard step in the development process, where, for the first time, a solution is shown to work as a device or a piece of software, under serious realistic testing conditions, e.g. in a test-rig, or coupled to a computer simulation (e.g. hardware in the loop simulation) or in extensive tests. This test in a synthetic environment may come with efforts for a first integration of several components or with the (synthetic) environment. Therefore, this category was split into two sub-categories, "component prototype validated in a synthetic environment" (TRL5.1) and "system prototype validated in a synthetic environment" (TRL5.2).

The categories "Technology validated in lab" (TRL4) and TRL3 (Experimental proof of concept) were kept, only the explanation was revised, to fit it to digital IOs so the word "experimental" was dropped. TRL4 is understood as showing that a first realization of the innovative technology performs as expected in a laboratory setting or with realistic data sets, while TRL3 is understood as providing sufficient confidence that the envisioned technology will be applicable to the problem at hand by means of simple tests or computations. TRL 3 was the lowest possible starting point to generate IOs on higher TRL levels within the studied projects.

Putting the categories together: a fine-grained scale.

We combined the refined categories into a finer-grained multi-dimensional scale (see Appendix 5.2, the new or refined categories are shown in light blue color). Instead of 6 categories of maturity of innovative technologies, the refined scale now has 12 sub-categories, grouped in larger groups such as 'products', 'deployment' etc. The purpose of the refined scale is to identify in more precise way the initial and the final maturity levels, independent of iterations between the levels. Then we classified the 54 IOs according to the fine-grainer scale (Appendix 5.3).

5.5.2 Development of a framework to assess the progress in the maturity of innovation outputs

Climbing up the TRL ladder: average efforts for maturation

We firstly assessed the efforts for maturation, in terms of nature and relative amount, differentiating between the levels. The interviews confirmed that the necessary efforts grew with the increase of the level of maturity of IOs "The academic prototype is ten percent of the work; but getting it into application is ninety percent... For me, the most memorable point (of the project) was to figure out how large the gap between the research and implementation really was…There was a moment when I talk with (multinational firm, project partner), they said okay doing this if we put everything behind, it will take 30 years… This is sort of level… it's at the TRL four or five now". ID9_PEGASUS_SME

Both the average amount of efforts and nature of effort changed with the increase of maturity.

The difference was mostly related to the change of the environment while climbing up the TRL scale, and thus different organisational, technical, but also legal and regulatory challenges, which consumed time, energy and efforts of participants to cope with.

Appendices 5C and 5D show an overview of the data structure and illustrative quotes. The steps from level 6.2 to 7.1 (pilot deployment), and from TRL7.1 to TRL8.1 to 8.4, were found to be particularly demanding in terms of the required time and effort, the amount of efforts significantly increased during each next step.

Step to TRL 7.1 and also to TRL8.1 were considered as crucial steps by the informants for all IOs that reached this level without exception, in terms of resources and benefits. This is due to three main challenges:

Organisational challenges were related to the increase in interactions (both in terms of number of external actors, i.e. those not directly involved in the project, but usually internal to the organizations, and the amount of interactions with them) as well as the increase of the dependence on these actors. This was related to the fact that at high maturity level, technologies must be integrated into existing systems for demonstration and deployment, which request not only new technical challenges, but also interaction with managers, business units, technicians, IT, legal and regulatory departments etc. Legal and regulatory discussions in some cases required significant efforts and led to interventions of other units of the organisations.

Technical challenges were related to additional technical issues appearing or an increasing importance of some aspects with the increase of maturity, such as complex technical integration into existing systems, necessity to deal with technical robustness and reliability, measures for safety, data security, data privacy, data availability, necessity to deal with affordability issues, real time data streaming and others. Although these elements were often taken into account since the beginning, their importance, and thus the amount of efforts increased significantly, also due to unexpected factors, according to the project actors.

Challenges related to the change of temporality of deployments: temporary deployment was done with the intervention of the project partners, but for permanent and multiple roll outs, even in prototype form, there was increased need in training, maintenance and other elements 'to live' with the solution in the long run: "there was no way back", according to one of the informants.

The technologies and development processes were different, and thus the efforts for the different steps and for different IOs were different. For instance, the internal integration (from level 5.1 to 5.2 or 6.2) in some cases, e.g. in GEMINI (pure software development) was much more demanding than the pilot deployment (integration into the operational environment of the end user, progress to level 7.1) while in other cases, the internal integration was less difficult than the pilot deployment. Progressing from level 7.1 (pilot installation in one case) to 8.1 was described as easier if the application is the same, then the progress to multiple rollout required higher additional efforts compared to going from 7.2 to 8.2. Also some pilots were already quite refined (e.g. had an almost perfect user interface in order to get the approval of the end users), while others were preliminary in some aspects for a faster evaluation in the sense of rapid prototyping, thus shifting some effort to the next level.

To sum up, the relevant factors related to the increase of efforts in the changed environments were: (1) increased importance for robustness, reliability, in-depth technical integration; (2) increased importance of interactions between humans and the technical solutions;

(3) increased importance to live with the solution in the long run. We found that the relative increase of the efforts comparing with previous efforts, was comparable in the studied projects, independent of the IOs, on the average, as the additional challenges attached to each IO, were similar: the IO development passes from a protected environment (own computer, lab), to the near-operational environment, then to operational but temporary environment (pilot(s)), and then, for a limited number of IOs, to a permanent deployment in an operational environment of a partner (roll out), or to the operational environment of first users. Additional factors such as e.g. inefficient collaboration, or specificities of the technology, affected the efforts needed for specific IOs, but for a larger set of IOs this averages out 15 .

A scoring system to assess the progress in the maturation

Based on these findings, to 'normalise' the progression in the maturation of the technologies according to their difficulty, and make it comparable, we developed the metric of average efforts per IO and developed a non-linear metric that assigns average scores to the progression from one level of maturity to the next (Table 5.4). These average scores reflect the increment of the required average efforts for the advancement, i.e. the relative increase in the efforts and resources that is required to advance from one level to the next. They do not represent absolute effort. The values of the average scores were derived from qualitative judgements of the project actors and were calibrated using quantitative data about reported resources of partners in the projects (see below). These average scores reflected the "objective" effort that is needed to progress the levels, not taking into account contextual conditions.

The logic of the constructions of the scores for the steps of advancement and an example, in the form of a process of maturation, is visualized in Appendix 5.4. The increments do not increase as sharply as discussed in the TRL literature for aircraft and space applications, because of the lower requirements for reliability at TRL8 in the domain considered here.

During the last interviews with project participants, we collected feedback on the proposed scores, whether they reflect the experience of the project actors in the studied projects. The feedback was that the scores were in line with their experience, with some variations: they were taken into account, and we noted the minimum and maximum scores as proposed by the actors in Table 5.4, based on their experience in the projects. Some illustrative quotes are below. For each IO, the scores from the initial stage to the final stage are summed up, giving the total advancement of this IO during the project.

The logic behind the assignment of the scores is that they should grow proportional to the effort that is required to advance the innovative technologies. This has been validated, to the extent possible, by the quantitative data. We calculated the scores for each IO using projects16 , PERSEUS, HERCULES and ORION; these values were plotted over the scores. In accordance with the literature, the interviews led to the result that the efforts are higher when increasing the level of maturity at higher maturities further, so a curve of the required effort over the increase in maturity would be non-linear. Thus, the scores should reproduce this behavior, so when the efforts are plotted over the scores, a linear relationship should result.

As an example, the dots in Figure 5.2 represent the individual IOs in the three projects. The xaxis are the scores according to the increase in the TRL levels, the y-axis is the estimated input in person-months. The data for PERSEUS and HERCULES on the average follows the postulated linear relationship (Fig. 5.2a). The slope of the linear trend gives the absolute average effort for a score of 1, due to the differences between the projects, these values differ.

For the ORION project (Fig. 5.2b), the trend is linear as well but there are larger deviations of individual IOs. This is due to a number of factors: (1) the difficulty to attach the efforts to the IOs 17 and problems with the reporting (e.g. when the budget was reached, not the full effort was reported by some industrial partners);

(2) variability due to differences between technologies, qualifications of the people involved and efficiency of collaboration 18 . The data confirms the claim that the increase in the required effort is relatively constant while the absolute effort may differ 19 .

Towards an assessment of the innovation performance of the projects

Adding all scores for all IOs gives the overall performance of the project on the 'innovation side', reflecting both the number and the progress in maturity of the IOs. The summary of the results, both in terms of numbers of IOs and scores for their advancement, are shown in Table 5.5. The knowledge (research) output is not discussed in this paper for reasons of space, but they are also shown to have a global view on the project outputs. Table 5.5 shows that the overall performance of the five projects on the 'innovation side' differs significantly when 17 Reporting was done not per IO but per activity. 18 In Fig. 5.2b, circle A shows two IOs where a group of several partners worked on a particularly difficult and challenging problem, circle B shows two IOs developed by single partners. 19 Note that the available data mostly is on advancing the IOs by several levels, corresponding to summing up the respective scores. Nonetheless, as this is a linear transformation, the trend of efforts over scores should be linear. observed with the lens of "numbers" and with the lens of "scores". For example, PEGASUS generated 53 outputs in total, including 7 IOs, and seems to be more performant. However, HERCULES also generated 7 IOs and the advancement of the IOs (innovation performance score) in HERCULES (87) is almost three times higher than the score of IOs in PEGASUS (30): HERCULES has generated not many IOs but several of them reached high progress in maturity. PEGASUS was a project with a stronger orientation towards research and generated significantly more publications than other projects studied here. The difference in the performance of the projects is in line with the judgement of the participants and the formal reviews of the projects by the funding agency: HERCULES and ORION were seen as particularly successful with respect to IOs. In the similar vein, PERSEUS generated twice less IOs than ORION, but the score of PERSEUS on the innovation side is three times less comparing to ORION. This suggests that assessment of the projects only from the point of view of number of outputs is not a good indicator to assess value creation in the project on the 'innovation side'. The progress of the IOs as measured by the scores is a more adequate indicator and it is not proportional to the number of IOs, increases of maturity at higher levels lead to larger scores, in line with the larger effort required.

To summarise, the proposed framework assesses the technological advancement of each IO, through the assessment of the advancement of the maturity of individual IOs. The idea behind the scores is that they should reflect the average 'objective efforts' to go from one level to another, and account for the difference in efforts to advance between different levels. They provide a "normalization": for the same amount of funding, advancement of more IOs can be expected on the lower levels whereas on higher levels, a larger investment is necessary and fewer IOs can be advanced to levels 7 and 8. The fine-grained scale is necessary for an accurate assessment of the IOs. The method values only the IOs which are available by the end of the project, regardless of whether the IO was planned from the beginning. It takes into account the starting points of different IOs, and measures the progress; the paths may be different and some levels are optional. The average scores represent the relative increase of efforts and resources needed for advancement in the studied projects, the absolute effort can differ depending on the technological area. Using the scores, the collective innovation performance of the projects can be assessed. The framework does not assess commercial readiness, which is a different topic that is beyond the scope of this paper.

Discussion

The study enhances the understanding of market upstream innovation outputs and of RDI projects with multiple actors. To the best of our knowledge, this is the first time when a full set of innovation outputs in publicly funded RDI projects with multiple actors was identified via a multi-case study, in line with the recent call of [START_REF] Goldstein | Simultaneous pursuit of discovery and invention in the US Department of Energy[END_REF]. We have collected rare and rich field data and used it to contribute to the advancement of both innovation literature and project literature, contributing to connecting the two fields [START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018[START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF], as discussed next.

First, the study puts in focus the progress of the innovative technologies, which until now was not paid sufficient attention in the literature, and proposes a way of how to assess market upstream innovative technologies (innovation outputs) in the projects. In the innovation context, this may contribute to a better understanding of the innovation process (Birchall et all, 2011), benefit innovation research [START_REF] Collinson | Recombination for innovation: performance outcomes from international partnerships in China[END_REF], and contribute to reducing uncertainty in policymaking [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF]. Literature is scarce on the development and validation stage, and the established indicators are insufficient to measure the innovation outputs at this stage [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF][START_REF] Schwartz | Measuring the effectiveness of R&D[END_REF][START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF].

This holds not only for collaborative multi-actor projects but also for single actor projects. In market-upstream RDI projects, the developed IOs are diverse and specific [START_REF] Perkmann | How should firms evaluate success in universityindustry alliances? A performance measurement system[END_REF][START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF][START_REF] Kostopoulos | Market-, Firm-, and Project-Level Effects on the Innovation Impact of Collaborative R&D Projects[END_REF] and an objective assessment of IOs at this stage is challenging (e.g. [START_REF] Henttonen | Searching for appropriate performance measures for innovation and development projects[END_REF][START_REF] Birchall | Innovation performance measurement: Current practices, issues and management challenges[END_REF]. Until now, they have been assessed from the point of view of their market potential (e.g. [START_REF] Nepelsky | Innovation and innovator assessment in R&I ecosystems: the case of the EU Framework Programme[END_REF], subjectively (e.g. [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], or indirectly, via patent indicators. We consider the IOs directly and objectively.

Second, the study developed a framework to measure the progress in maturity of innovation outputs on their way to (potential) innovations, from the point of view of technological readiness. To appreciate the progress of IOs in a nuanced way, and as one side of the framework, based on the empirical data collected in the multi-case study, we refined the conventional TRL scale [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF] which provides coarse levels of technological maturity [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], Olechovsky et al., 2020[START_REF] Enzing | Ex post evaluation and impact assessment of funding in the FP7 NMP thematic area[END_REF] for the domains of the projects considered here. The other side of the proposed framework is the scoring system assessing the progress of IOs. Literature suggested that progress of the IOs can be assessed in terms of their technical characteristics (e.g. [START_REF] Bican | Managing innovation performance: Results from an industryspanning explorative study on R&D key measures[END_REF]Brem, 2020, Chiesa and[START_REF] Chiesa | Searching for an effective measure of R&D performance[END_REF], but this is technology-specific and does not enable a quantitative comparison. We introduced a baseline, the need of which has been highlighted by [START_REF] Perkmann | Academic engagement and commercialisation: a review of the literature on universityindustry relations[END_REF], and assessed the IOs in terms of advancement of maturity, i.e. the progress between the initial maturity level and the end level reached in the project. With this approach, the issue of specificity of IOs and thus difficulty of their assessment at the earlier stages of the innovation process is avoided, as the construct 'advancement of maturity' is not domain specific and enables to assess any IO from the point of view of its progress on the way to a potential innovation.

Third, the study found that, while the absolute amount of resources significantly differed for each IO, echoing e.g. [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF] and Olechansky et al. (2020), the relative increase of the efforts between the start and end maturity levels of the different IOs in the studied projects was relatively similar independent of the IO or the path taken. This happens because the increase of the efforts was related in all cases to the change of the environment and additional requirements (Olechovsky et al., 2020[START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF][START_REF] Webster | Aligning technology and institutional readiness: the adoption of innovation[END_REF][START_REF] Cooper | Winning Businesses in Product Development: The Critical Success Factors[END_REF][START_REF] Nakamura | Multi-level Perspectives with Technology Readiness Measures for Aviation Innovation[END_REF]. This allowed us to develop the scoring system.

The proposed scores hold for the domain of innovations that are embedded into larger systems and do not require an extremely high level of reliability. Their values will have to be adapted to other domains where the efforts and thus the scores may increase faster or slower towards higher level of maturity.

Forth, the paper contributes to a better understanding of the innovation performance of multiactor RDI projects (Pandza et al., 2011, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], or in other words, the success of such projects [START_REF] Green | Value creation in projects: Towards a narrative perspective[END_REF][START_REF] Winter | Directions for Future Research in Project Management: The Main Findings of a UK Government-Funded Research Network[END_REF][START_REF] Andersen | Two Perspectives on Project management[END_REF][START_REF] Breese | Benefits realisation management: Panacea or false dawn[END_REF], with respect to the innovation side. Other scholars also pointed out (e.g. [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF][START_REF] Hung | Social networks, technology ties, and gatekeeper functionality: Implications for the performance management of R&D projects[END_REF] that the innovation performance in RDI projects is multi-faceted and is not adequately reflected by the number of patents. Our proposed framework brings a project perspective, assesses the realised value of IOs and complements existing views and methods (e.g. Spanos andVonortas, 2012, Nepelsky and[START_REF] Nepelsky | Innovation and innovator assessment in R&I ecosystems: the case of the EU Framework Programme[END_REF]: the assessment of the IOs from the perspective of future potential and realised value are both important, and an assessment of the innovation performance of the project should account for these different dimensions. The framework uses relatively simple measures and available data as called for by [START_REF] Dziallas | Innovation indicators throughout the innovation process: An extensive literature analysis[END_REF] and [START_REF] Kim | Development of an evaluation framework for publicly funded R&D projects: The case of Korea's Next Generation Network[END_REF], among others. When the size of the projects increases, it can be expected that the number, variety and progress of different project IOs increase too. The proposed framework accounts for the advancement of the maturity of the individual innovative technologies and therefore can be applied generally to projects with few or many IOs, including projects within single organisations. We have shown that the number of IOs alone is not a good indicator to assess the project results on the innovation side: the value creation [START_REF] Laursen | Taking stock of project value creation: A structured literature review with future directions for research and practice[END_REF], Martinsuo et al., 2019[START_REF] Martinsuo | Evaluating the organizational impact of product development projects[END_REF][START_REF] Evanschitzky | Success factors of product innovation: An updated meta-analysis[END_REF] is not adequately captured by the number of IOs but by measuring the progress of maturity of the IOs.

A better assessment of the performance of publicly funded projects can also support policymaking practices and policy learning, it is important for designing policy interventions [START_REF] Dziallas | How to evaluate innovative ideas and concepts at the front-end? A frontend perspective of the automotive innovation process[END_REF]Blind, 2019, Gault, 2018) and can help to reduce uncertainties [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF]. This is particularly important with respect to the 'innovation side' of market upstream RDI projects which are not well studied yet. The assessment of the research performance is another, largely complementary topic [START_REF] Martin | The evolution of science policy and innovation studies[END_REF][START_REF] Grimaldi | Assessing collaborative, pre-competitive R&D projects: the case of the UK LINK scheme[END_REF][START_REF] Martin | The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster[END_REF] but outside of the scope of this paper. The concept of "level of maturity" could e.g. also be extended to the different kinds of publications.

Conclusions

Summary of the theoretical contributions

We performed an exploratory multi-case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF][START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF] of several RDI projects with multiple actors that generate a variety of market upstream innovative outputs on their way to (potential) innovations. Responding to multiple calls for relevant research, the study enhances the understanding of market upstream RDI projects with multiple actors and of the IOs developed within such projects and contributes both to the innovation literature and to the project literature, further connecting these literature streams. We also contribute to the literature studying project performance and project success.

We propose an approach to the up to now unresolved issue how to assess market upstream innovation outputs in the projects, and thus the performance of the project on the 'innovation side' directly and objectively, at the end of the projects, going beyond patents, from the point of view of their progress on the way to potential innovation. This is the main theoretical contribution of the paper, which complements existing views and perspectives. Towards this goal, we refined the conventional TRL scale, based on the qualitative feedback of the project actors supported by quantitative evidence, and exemplarily, for the technical domain of the considered projects, allocated scores to the progression from one level to the next which capture the average efforts needed for the advancement of the maturity of each IO. The study found that although the path taken, obstacles, difficulties and thus absolute amounts of efforts are different for each technology, the relative increase of the necessary efforts for maturation is comparable for the different IOs. The same number of market-upstream IOs does not mean the same realised value of these IOs.

Policy implications

The study has several policy implications.

First, policymakers shall clarify what shall be taken as evidence of success of multi-actor RDI projects. At low TRL, publications, patents and reports are relevant, however, at the higher levels of maturity, the study suggests that assessment should focus on the progress of IOs that was achieved during the project. The study shows that that the assessment of the success of the projects is different when considering the number of innovation outputs or the advancement of their maturity with an appropriate scoring system. Therefore the funding agencies should define not only the target level of maturity, as it is often the case nowadays, but also the expected degree of maturation, i.e. the distance between the initial level and the final level.

Secondly, the study shows that the crucial step in the maturation process is to progress to the pilot deployment, or TRL7.1 of the proposed refined scale. A difference of just one level of maturity at the higher TRL levels implies major differences in terms of efforts (including the need for qualified manpower) and dependence from external actors: the target level of maturity in the funding projects should be precisely defined, and the funding level must be consistent with the expected increase of efforts. The paper does not focus on risks, but the increasing need of resources and dependency on collaborators and external factors increases the risk that developments are abandoned. If the funding institutions target higher maturity levels at the end of the projects and significant own contributions by the commercial project partners, abandoning of developments must be expected with the increase of maturity and thus acknowledged, and mechanisms to deal with this risk must be considered. Putting it more generally: what is required to advance the innovation outputs as far as possible? The identification and understanding of the nature of inefficiencies may also help in designing policies.

Third, the policymakers should consider whether the focus of assessments during and at the end of the projects shall be on whether what was proposed initially was realized several years later as proposed, or on what is actually achieved at the end of the project, in terms of the advancement of innovative technologies compared to the relevant baseline, not necessarily comparing to the initial plans. The study suggests the latter option: an assessment of the advancement of the degree of maturity reached by the innovation outputs of the project would help not only to better appreciate the project performance on the innovation side but also stimulate targeting outputs with a high promise for progress to roll-out and prototype products rather than "sticking to the plan". This means that the funding agency should give flexibility to the funded projects, accepting that different IOs than originally anticipated may result and some of the planned ones will have to be given up.

In addition, with the growth of the number of innovation outputs in the projects, resultoriented reporting, i.e. reporting by IO, may help to better capture the progress of the IOs and therefore the performance of the projects. The authors of the study spent significant efforts by reconstructing effort data per IO: reporting was done per workpackage (a workpackage "packages" work by topics), but the development of the IOs was often related to several workpackages. While this may be well suited for projects with limited number of IOs or with IOs at lower maturity stage, the study suggests that reporting per IO (and not per workpackage) would lead to more transparency and a better analysis of the project results, especially in the projects with intermediary and high TRL.

Managerial contributions

The study also has several managerial contributions. The proposed framework enables project managers, both from the funding agencies and the consortia, to measure systematically and more precisely the progress of the innovative technologies in the projects, to assess the 'innovation side' of the project performance, and thus to demonstrate the value for money for the taxpayer or the company. The funding agencies can check the project progress in a more objective way, helping to identify obstacles and ways to address them. The average scoring (innovation performance score) is relative comparing to other indicators, such as costs and time. In relation to the project costs, the method allows assessing project efficiency (total score over budget): the scores reflect the effort, so project with different efficiency would need different amount of funds to reach the same point. The efficiency should not depend on the targeted TRL level, as the scores account for the additional efforts requires at the higher stages, they do not favor big expensive projects. If the absolute number of IOs is looked at, then e.g. 15 IOs would give more points than 5 IOs. But reaching higher levels of maturity for 5 IOs may require more efforts, hence budget, than generating 15 IOs on lower levels. Factors influencing the efficiency of the project work will lead to different ratios of total scores to funding or financial investment and may provide actionable intelligence with regard to e.g.

the organisation of the project, the quality of the collaboration, the motivation of the partners, and the quality of the project management. This helps to answer several questions, for example: (1) For every Euro or Dollar allocated to the project, how much progress on the 'innovation side' of the project are we getting? (2) How does that compare to other comparable projects, e.g. projects funded in the same call? (3) How could the progress be improved, and what are the blocking points where the progress is low?

The results of the study are also useful for project managers. The start and end points of the IOs, project durations and budget are usually available and objective data in all settings, public or private. The project management can set internal goals in terms of funding and advancement, using a refined maturity scale, analyse progress at certain points in the project, and perform corrective actions. Difficulties could be better identified and addressed, if assessment is done timely: they could point to e.g. dependence from external factors, necessity of higher business value compared to the estimated further cost, or contextual conditions.

The proposed method is generic and applicable in different contexts: it does not depend on whether the development was funded publicly or not, and whether it is collaborative or not.

When considering different domains, the scores attached to the increments of the maturity may need to be adapted based on domain knowledge, but the general principle can be transferred. Thus, the results of the study, and notably the assessment method, provide actionable intelligence for the project actors, also in the context of internally funded market upstream RDI projects within firms, including those focused on individual technologies, but from another side: to describe 'where we are ", "where we want to be" and "how it went". In the context of privately funded projects, the method may help to anticipate project planning and implementation, adapt management strategies, and may point out stages requiring particular attention. In the interviews, the project actors from industry repeatedly highlighted such needs for project planning, monitoring and assessment purpose:

"The steps on the way within one of these levels can be quite large to implement and quite expensive…it would be good to distinguish this in order to more accurately describe where you are, where you want to be at a certain point… To support the management process, do we have enough efforts done within this project, how expensive is it up to now and how it could be, this helps". ID53_HERCULES_large_ind funded projects, but we expect the framework to be applicable here as well, as far as projects which are strongly applications-driven are concerned. Also only a limited range of themes was covered: innovations embedded into larger systems with a significant share of algorithms and software development. The refined maturity levels may have to be adapted to other domains, in terms of their labeling, but the overall scheme is considered to be general. The proposed scores are specific for the domain of the projects under investigation and based on a simplified, uniform treatment of the outputs, and do not account for differences between product and process innovations. For the case studies, the scoring was developed and calibrated based on the discussion with the project actors and the quantitative data (efforts).

Due to the type of projects, the scores are biased towards personnel efforts: developments that require a high level of material resources may require different scores comparing to those proposed. Risks and benefits were also perceived to increase with the increase of maturity, but this is not considered in this paper; attrition (developments being stopped despite efforts invested because e.g. they do not meet the expectations, the effort becomes too high, or the business value is not clear) influences the situation because the maturation ended at a certain level. Especially for the assessment within companies, this may be a relevant additional indicator in the assessment of larger units.

The results obtained in this exploratory study are a proof of concept. One of the objectives of this work was to engage the discussion around the topic, stimulate further research developments, and open avenues for exciting future research. Theoretical and empirical studies are needed to better understand and assess the innovation side of the project performance in publicly funded RDI projects, especially for a broader range of sectors. This need will increase in the future, with the trend of the agencies to fund RDI projects which reach high levels of maturity. Complementary to this, the innovations that resulted from the projects should be assessed a certain period of time after the end of the projects.

To advance in the understanding of multi-actor projects, future research can also use this framework to better understand how to improve the advancement of innovative technologies, and to investigate whether causal relationships exist between project parameters, such as scale or number of partners, and innovation performance of the project. Future studies could also investigate the variations of the scores in other domains, the process behind the maturation of the outputs, to better understand the elements which are behind maturation, as well as the contextual conditions which have implications on the efforts and the advancement of the maturity of the IOs. A related topic is to investigate how to quantify and to manage the risks in the process of the maturation of innovative technologies, as also discussed in (Olechowsky et al., 2020). Additional factors, besides technical maturity, such as a measure of disruptiveness, might be needed to be taken into account as well when assessing the innovation potential of the project outputs.

decomposable architectures offer the highest collaboration potential under contingent conditions. The study also shows the benefits of the waterfall and grid sub-types: the interface, or integrative module, provides planned knowledge integration opportunities for the project actors, brings partners closer together that otherwise would only work in small groups, widens the horizon of the partners by forcing them to unify concepts and ideas of an overarching nature, and 'pushes' towards knowledge integration from various backgrounds and applications. The waterfall architecture is more resilient than other types of architecture:

deficiencies of contributions of the project actors did not prevent the activities at the interface to continue.

6.1.2 Empirical study #2: The evolution of collaboration in multi-actor projects: Patterns of couplings and collaborative innovations

The second study built on the findings of the first study, investigating the collaboration processes in more detail. The study provides evidence about eight types of reconfigurations of the planned collaborative couplings; in the large majority of the cases, such reconfiguration systematically led to predominantly negative implications on the planned innovation outputs, or, at best, with the managerial intervention, to preserving the planned outputs. Behind these reconfigurations, there are six disintegrative and two integrative generative mechanisms that activate and influence the evolution of the collaborative couplings. The study also shows that the project constraints such as time, budget, description of work and planned outputs significantly influence the work at the interfaces: the more there is pressure for implementations of innovations in industrial environments, the stronger is the tendency to reduce the efforts spent on overarching activities on the interfaces. The findings point out that envisioned strong couplings at the interfaces between modules in the projects systematically tend to be weakened during the course of the project, and a loss of resulting collaborative innovation outputs or its downscaling seems to be difficult to avoid. The waterfall architecture was more resilient and remained intact, making the losses less likely to happen. Based on these findings, the study proposed a process model of the evolution of the couplings in multiactor RDI projects and its influence on the innovation outputs, with three phases: the "fixing", where reconfiguration of planned couplings occurs, the "living", where significant disintegrative mechanisms materialise, and "stable continuation". The study also demonstrates that not too rigid planning at the proposal stage enabled to react to unforeseen difficulties and helped to reconfigure couplings and generate more collaborative innovations, which happened in the projects with a grid architecture and of large size. The third study took a deeper look on the research and innovation outputs that were generated in the studied projects, and on the way of creation of these outputs, from the point of view of the contributing organisational actors. It investigated how the size has implications on the project performance. The study found that over 95% of 185 research and innovation outputs in the studied projects were developed in small groups that were stable over the project time, or even individually, independent of the project size. This suggests the absence of a relationship between the project size and the number of project outputs, beyond the proportional increase due to the larger number of partners, or additive effect: the bigger the size, the more groups can of course be created, and the more outputs result. Putting this differently, even if the collaboration increases in size, the groups involved in the sub-projects remain of similar size, leading to a "pooling" of different groups and their results under the overall umbrella of the projects: clusters emerge within projects. The study shows the short term negative implications of the project size within the project, creating coordination difficulties, and the long term positive implications of the project size mostly outside the project: in the studied projects, a large knowledge base was seen to be beneficial after the project but not within the project: the benefits of the larger size for exploration purposes are difficult to realise, due to the need to have detailed technical discussions within the larger partnership with limited amounts of time that the partners are able to or willing to allocate to meetings. This suggests that knowledge integration mechanisms shall be carefully designed and adapted to such large projects, which comes at a cost. Nevertheless, the study found that the large project size may have beneficial effect on the number of outputs, if the projects generate meta-knowledge, a non-conventional project output that had not been identified in the literature before and was valued by the project partners: this is a proven and potentially reportable output which would be useful for the community, with positive effect after the end of the project. However, the generation of meta-knowledge requires an adequate project architecture with parallel comparable modules, and more intense coordination.

Bridging the gap between the rigor of theory and the relevance for practice [START_REF] Romme | Making a Difference: Organization as Design[END_REF] is important. In addition to the study #3, and in order to analyse how the results of this study correspond to the practice of the actors outside the studied projects, an on-line survey was conducted in 2019-2020, targeted to the coordinators and participants in EU-funded projects not included in the qualitative study presented in this manuscript. The survey included several questions related to the project size and to the research and innovation outputs, to provide an element of triangulation, to get additional relevant insights, and to give a larger perspective in addition to the results of the study #3. A large proportion of the respondents were experienced participants in EC programmes: 83% of respondents took part in two or more projects. The informants were asked to choose their most recent project and answer questions related to it; not all questions were mandatory, that is why different questions have different numbers of respondents. 128 responses were collected, from them 96 respondents answered the questions relevant to the topic of the study #3.

The results of the survey show that among 128 respondents, over 50% took part in projects with 11 organisational partners and more (Fig. 6.1a), most projects had several main outputs (Fig. 6.1b). 88% of the respondents (99 out of 113 respondents who answered this section)

worked in small groups of partners, from them only 6% started the work in these small groups only in the second part of the project (Fig. 6.1c). Hence these small groups usually collaborated from the early phase of the project to the end, and most of the collaborative work happened in these. These results are in line with the findings of the study #3, showing that the research and innovation outputs are created mostly in small groups; these outputs are not necessarily tightly related, but fit under the common "heading" of the project.

One of the questions in the survey was related to the experience of participants about the optimum size of the consortium for fruitful collaboration. From 98 respondents answering this question, the majority prefer to have 6 to 10 organisations in the consortium (Fig. 6.2); from 76 respondents who worked consortia with 11 and more organisational partners, only 14 answered that this is an optimum size. These results also point to the preference of the project actors towards a smaller project size, and echo the results of the study #3, which shows that the growing size of the project is often seen negatively, and in addition, it had no measurable positive effect on the number of innovations.

6.1.4 Empirical study #4: Climbing up the ladder: Towards a better understanding of the innovation outputs and of the performance of market-upstream collaborative research and innovation projects

The fourth study (Chapter 5) investigated how to assess innovation outputs at the development stage, and developed a framework to measure the progress in the maturity of the innovation outputs on their way to (potential) innovations, from the point of view of technological readiness, directly and objectively. The two sides of the assessment framework include a refined TRL scale for the use in multi-actor market upstream projects, and a scoring system. The scoring is based on the finding that while the absolute amount of efforts significantly differed for each innovation output, the relative increase of the efforts between the start and the final maturity levels of the different innovation outputs in the studied projects was relatively similar independent of the output or the path taken. The findings help to better understand the 'innovation side' of the projects, or the collective innovation performance.

They complement existing perspectives, which analyses the innovation performance of the projects from the point of view of the future potential of innovations, or from the point of view of their impact a few years after the project end, or from the subjective point of view of the project actors.

Linkages between the results of the four studies

Taken altogether, the four studies provide a better understanding the implication of different factorsthe project setting, notably the architecture and the size (number of organisational partners) and the processes -on the project performance (Fig. 6.3).

The study #1 showed that the architecture of the projects with a strong application dimension, i.e. not focused on basic research, with its interplay between activity coordination and knowledge integration, has impacts on the collaboration, and thus on the project performance.

Some types of architecture are not resilient: the breaking down of collaboration patterns leads to the loss of planned outputs, and it requires a lot of effort to maintain collaboration. These results on the architecture are complemented by the process approach used in the study #2, which focused on the evolution of the collaborative couplings, and gave more nuanced results: if the planned coupling breaks and is replaced by another coupling, then there can be a positive effect on the innovation performance of the project. The waterfall architecture and sequentual processing facilitate such adaptive adjustmentsalthough the study shows that such adaptations are rare, they are possible, especially when driven by the interests of the industrial partners. Thus, the critical role of a solid interfaces, as in the waterfall architecture, becomes evident, but such interfaces require management efforts to be maintained, as highlighted in the study #1.

The increase of the size of the projects, in terms of the number of organisational partners, was found to have at best an additive effect on the project performance. The study #3 provided evidence that the partners cluster in small modules around applications, or around specific innovation outputs related to the applications; ; even if the project increases in size, the groups doing sub-projects (modules, in the terminology of this research) remain of similar size.

However, the management of the project becomes much more complicated and requires significantly more effort. An adequate architecture is critical in such projects of the larger size: it can help to mitigate, or even take advantage of the trend of an increase of the project size, which often pushed by the funding agencies, and at the same time it can limit the effort for coordination in such projects. For instance, if a modular architecture is used, and the work in parallel modules is comparable, then the necessary effort for coordination decreases ("the modules managed themselves" , according to the words of one of the informants), and in addition, a specific valuable output could be generated, meta-knowledge. A grid or partial grid architecture can be even more beneficial in such projects of large size, but it is not easy to put in place, and it also requires highly experienced and dedicated coordinator with both technical and management skills. The waterfall' structure with a phase when the project actors develop a joint concept and establish collaboration patterns can greatly help in such large projects, as the study #3 shows. However, to define such an architecture alone is not sufficient: as the study #2 demonstrates, collaboration couplings may evolve over time, and this may be beneficial for the innovation outputs, but it shall not be accompanied by a 'laissez faire' attitude of the project management.

'Beneficial' is a broad notion. The study #4 helped to clarify the concept of innovation performance in the type of projects investigated here, using objective and direct measurements of the innovation outputs at the end of the project, taking into account that the more the advancement of maturity of the innovative technologies developed in the projects progresses, the higher are the necessary efforts to do to obtain these project results. This further accentuates the importance of adequate project architectures. The largest studied project received the biggest innovation performance score, as found in the study #4, and in addition generated meta-knowledge and many other research (knowledge) outputs, as shown in the study #3. This largest studied project provided opportunities for a number of reconfigurations of couplings during the early project phase, as shown in the study #2: this was enabled by the relatively broad technical description of the project, flexibility of the project manager, combined with open-minded actors on the organisational and on the personal level. Taken altogether, and combined with the fact that the project had a large size, with many organisational actors and thus more opportunities for recombination, the slack in terms of the evolution of couplings had a positive effect on the innovation performance.

Theoretical contributions

This research aspired to develop new perspectives on multi-actor collaborative RDI projects, by studying their settings and collaborative processes, and their implication on the project performance. Such projects with many heterogeneous autonomous organisations cumulate tensions and management challenges (vom Brocke and Lippe, 2015), but we know little about how collaboration for knowledge generation and innovation works within the project boundaries, and which role the project setting and the collaboration processes play for the project results. Calamel et al. (2012, p.1) pointed out that "a literature review highlights the need to open up the 'black box' of collaboration within projects". We studied how the project architecture, size, and processes related to the dynamics of the inter-organisational collaboration influenced the innovation outputs and the overall performance of multi-actor market upstream research and innovation projects, and how one could more adequately and precisely assess their innovation performance, regarding the 'innovation side'. The work responded to the repeated calls of both project management scholars and innovation scholars for a better connection of these research streams [START_REF] Geraldi | Project studies: What it is, where it is going[END_REF][START_REF] Sydow | Projects as temporary organizations: An agenda for further theorizing the interorganizational dimension[END_REF], Davies et al., 2018[START_REF] Berggren | The cumulative power of incremental innovation and the role of project sequence management[END_REF]. Despite the fact that the studies had a focus on specific aspects and were addressed either to a project management scholars (studies #1 and #2) or to innovation scholars (studies #3 and #4), they always connected the two theoretical fields. The research makes several contributions to the project management literature and the innovation literature, as discussed next.

The first study (Chapter 2) extended our understanding of two key dimensions which constitute the architecture of multi-actor RDI projects: the structure of the activities and the structure of the knowledge base. In the context of these projects, both the flows of knowledge and of activities shall be structured [START_REF] Rauniar | Knowledge integration in IPPD project: role of shared project mission, mutual trust, and mutual influence[END_REF], both shape patterns of interactions in the project and decrease uncertainties (Raab et al., 2009, Söderlund andSydow, 2019).

However the couplings of the activities can be different from the couplings in the knowledge base, and both dimensions intertwine and define the project architecture. The findings suggests that in order to structure multi-actor RDI projects, the lens of activities and the lens of knowledge integration must be taken into account simultaneously (Sydow and Braun, 2019), as both of them have a structuring effect on the project. The study led to a better comprehension of how activity coordination and knowledge integration simultaneously influence the execution of the projects. Different architectures of inter-organisational R&D projects were identified and light was shed on the setting of these projects with multiple actors using a network lens [START_REF] Steen | Projects and Networks: Understanding Resource Flows and Governance of Temporary Organizations with Quantitative and Qualitative Research Methods[END_REF]. Research explaining the differences between interorganisational projects is still at its early stage [START_REF] Ahola | So alike yet so different: A typology of interorganizational projects[END_REF]; the study proposed a typology of project architectures, consisting of three main types and six sub-types, with different implications on collaboration, project resilience and project management, and offers heuristics for project managers on how to structure inter-organisational RDI projects with multiple actors. Similar to previous studies in other contexts [START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Tiwana | Do bridging ties complement strong ties? An empirical examination of alliance ambidexterity[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF], we found that nearly decomposable architectures are beneficial in the context of collaborative RDI projects, but while they are beneficial for knowledge generation, they lead to challenges for project management. We also provided insights into how different types of project architectures favor or disfavor collaboration within the project [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF], and showed that in some settings, the added value of a large interorganisational setting is low. Finally, the results of the study pointed out how the type of architecture and its resilience put requirements in terms of management role, management cost, and need for scientific/technical skills of the project manager. We showed that while multi-actor R&D projects are unique endeavors, there are typical architectures with specific advantages and drawbacks.

The second study (Chapter 3) extended the findings of the first study using the process view, and investigated the evolutions of the collaboration (patterns of couplings) and their implications on collaborative innovations (Michelfelder andKratzer, 2013, Hofman et al., 2016), using the process view [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF][START_REF] Abdallah | Performing Process Research, Standing on the Shoulders of Giants[END_REF][START_REF] Feldman | Making process visible: alternatives to boxes and arrows[END_REF]. The study investigated projects with modular architectures coupled with integration at the interfaces (Brusoni, 2005[START_REF] Tee | Modular designs and integrating practices: Managing collaboration through coordination and cooperation[END_REF]. The study provides evidence that integration thanks to recombination only rarely happened in spite of the fact that a large knowledge base was present in the projects, which is considered beneficial in other contexts (e.g. [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF]Ghoshal, 1998, Ahuja and[START_REF] Ahuja | Technological acquisitions and the innovation performance of acquiring firms: a longitudinal study[END_REF]. This happened because of the limited structural flexibility of the kind of large multi-actor projects considered here due to multiple project constraints. Even if the interfaces draw on modules with high technological relatedness, the study revealed the difficulties of the project actors to deal simultaneously with both relational and cognitive complexity [START_REF] Wang | Complexity and the functions of the firm: Breadth and depth[END_REF] within the project boundaries. If the interfaces are preserved until the end of such multi-actor RDI project, this creates a beneficial setting for collaborative innovation [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF][START_REF] Rost | The strength of strong ties in the creation of innovation[END_REF][START_REF] Michelfelder | Why and How Combining Strong and Weak Ties within a Single Interorganizational R&D Collaboration Outperforms Other Collaboration Structures[END_REF]. But such preservation is difficult and requires efforts from the project manager. Here, we uncovered the benefit of the waterfall-type project architecture from the point of view of innovations at the interface level. Siggelkow and Levinthal (2003, p. 650) also pointed out the importance of "temporarily divide to conquer": sequential modular decomposition and a modular integrative approach yield the best organisational performance results in the long term. Our study shows that "conquer first, then divide", worked best in the studied setting and helped in creating a common base of understanding [START_REF] Calamel | Inter-organisational projects in French innovation clusters: the construction of collaboration[END_REF]. Having a large variety of actors and knowledge combination opportunities at the beginning of the project has advantages, as the recombination potential may more easily meet the conditions that are needed to realise this potential. Based on these findings, a process model [START_REF] Cloutier | What Makes a Process Theoretical Contribution? Organization Theory[END_REF]Langley, 2020, Feldman, 2016) of the evolution of the couplings and its influence on the collaborative innovations in multi-actor R&D projects was proposed.

The third study (Chapter 4) analysed why and how does the project size matters for the performance of RDI projects with large numbers of heterogeneous partners, and whether there is positive, super-additive effect with regard to the generation of new knowledge and the development of innovation outputs. A qualitative study alone can not fully answer this research question but provides a starting point for larger scale investigations and moreover provides insight into the reasons why the postulated increase in productivity of the projects could not be observed. This is why we opted for a qualitative study: the goal was not only to add more data to the existing set of conflicting findings, but also to identify 'why' and 'how' elements related to the size and project performance [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF][START_REF] Schwartz | What drives innovation output from subsidized R&D cooperation?-Project-level evidence from Germany[END_REF][START_REF] Szücs | Research subsidies, industry-university cooperation and innovation[END_REF][START_REF] Jacob | Analysis of patents, publications and new organisations from FP7-ICT and CIP-ICT projects[END_REF].

Echoing Rost (2011, p.588), who investigated "the strength of strong ties in the creation of innovation" in the empirical setting of German automotive industry, the study #3 showed that the outputs of the projects are mostly produced by joint work in small groups of organisational partners (usually 2 to 3-4 partners), or even individually. Therefore, the size of the knowledge base does not play a critical role in the generation of outputs in multi-actor collaborative RDI projects with a strong application dimension. This implies that there is high likelihood that larger RDI projects with multiple partners develop into an assortment of a number of loosely coupled small groups, hence producing more outputs but that there is no larger-than-additive effect. This also puts in question the beneficial role of the size of the knowledge base [START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF]Katila, 2001, Cantner et al., 2011) on the research and innovation outputs in the RDI project context, with limitations of budget, duration and scope. The results of the study #3 suggest that while most of the gains due to a larger size are additive, on the other hand the increasing need for coordination leads to inefficiencies that hinder productivity. The increase in the number of participating organisations was found to lead to increased numbers of publications in previous studies (e.g. [START_REF] Jacob | Analysis of patents, publications and new organisations from FP7-ICT and CIP-ICT projects[END_REF]; study #3 points out to an additive effect. The results of the study put into question the beneficial effect of more knowledge inputs generating more opportunities for recombination, because the potential for the realisation of such recombination is limited due to fixed time and budget and constraints as e.g. the pressure to realise what was planned both from the participating organisations and the funding agency. This is consistent with a different perspective on organisation, that a larger team size enables team members (subgroups of partners) to specialise in different applications and activities [START_REF] Haeussler | Division of labor in collaborative knowledge production: The role of team size and interdisciplinarity[END_REF], and thus helps to organise a large setting. But then the value of the project size would also be that all these elements and specialists are indeed needed and related to each other, with an overarching goal that requires it, a joint work at the interfaces between groups (modules), with an opportunity to create meta-knowledge. The study also found that the benefits of size mostly occured after the outputs had been produced, e.g. outside the projects, echoing [START_REF] Singh | Lone Inventors as Sources of Breakthroughs: Myth or Reality[END_REF]; the negative influence of the increase of the project size operates within the project, poses multiple managerial challenges (e.g. [START_REF] Söderlund | Pluralistic and processual understanding of projects and project organizing; towards theories of project temporality[END_REF]Tell, 2011, Lin et al., 2019), and inhibits knowledge integration, which indirectly influences the project outputs.

The fourth study (Chapter 5) focused on the assessment of market upstream innovation outputs and of the innovation performance of market upstream RDI projects (Kim et al., 2017, Aguiar and[START_REF] Aguiar | European cooperative R&D and firm performance: Evidence based on funding differences in key actions[END_REF], directly and objectively. This requested firstly to answer the preliminary research question of how to assess individual innovation outputs at their development stage. The literature of R&D assessment neglected this issue until now, and the established indicators are insufficient to measure the innovation outputs at this stage [START_REF] Dziallas | How to evaluate innovative ideas and concepts at the front-end? A frontend perspective of the automotive innovation process[END_REF]Blind, 2019, Schwartz et al., 2011). The existing assessment methods are based on subjective evaluations, or are based only on the count of patents, or focus on the potential of the innovative developments or on their impact. Our study proposed an approach how to assess market upstream innovation outputs at the end of the projects, going beyond patents, from the point of view of their progress on the way to potential innovation. This theoretical contribution complements existing views and perspectives. In the innovation context, this may contribute to a better understanding of the innovation process [START_REF] Birchall | Innovation performance measurement: Current practices, issues and management challenges[END_REF], benefit innovation research (Collinson and Lui, 2019), and contribute to reducing uncertainty in policymaking [START_REF] Janger | The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?[END_REF].

Towards this goal, the study developed a framework to measure the progress in maturity of innovation outputs on their way to (potential) innovations, from the point of view of technological readiness. On one side, the conventional TRL scale [START_REF] Mankins | Technology readiness assessments: A retrospective[END_REF] which provides coarse levels of technological maturity [START_REF] Héder | From NASA to EU: the evolution of the TRL scale in Public Sector Innovation[END_REF], Olechovsky et al., 2020) was refined, based on the qualitative feedback of the project actors supported by quantitative evidence, and on the other side, exemplarily for the technical domain of the considered projects, scores for the progression from one level of maturity to the next were allocated, capturing the average efforts needed for the advancement of the maturity of each IO. The proposed scores hold for the domain of innovations that are embedded into larger systems and do not require an extremely high level of reliability, in contrast to e.g. the aircraft or pharmaceutical sector. Extant literature suggests that progress of the innovative technologies can be assessed in terms of their technical characteristics (e.g. [START_REF] Bican | Managing innovation performance: Results from an industryspanning explorative study on R&D key measures[END_REF]Brem, 2020, Chiesa and[START_REF] Chiesa | Searching for an effective measure of R&D performance[END_REF], but this is technology-specific and does not enable a quantitative comparison. With our approach, the issue of specificity of outputs and thus difficulty of their assessment at the earlier stages of the innovation process is avoided, as the construct 'advancement of maturity' is not domain specific. The study found that although the path taken, obstacles, difficulties, and thus the absolute amounts of efforts were different for each technology in the studied projects, the relative increase of the necessary efforts for maturation was rather similar for the different innovation outputs. The study also contributed to a better understanding of the innovation performance of multi-actor RDI projects (Pandza et al., 2011, Spanos and[START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF]: we showed that the number of innovation outputs alone is not a good indicator to assess the project results on the innovation side: the value creation [START_REF] Laursen | Taking stock of project value creation: A structured literature review with future directions for research and practice[END_REF], Martinsuo et al., 2019[START_REF] Martinsuo | Evaluating the organizational impact of product development projects[END_REF][START_REF] Evanschitzky | Success factors of product innovation: An updated meta-analysis[END_REF] is not adequately captured by the number of innovation outputs but by measuring the progress of the maturity of the outputs.

Managerial contributions

The research presented in this thesis provides practical contributions to project managers, project actors and funding agencies.

First, it highlights how to adequately organise inter-organisational RDI projects with multiple partners. Evolutions of planned collaborations may be necessary and shall be managed proactively rather than 'let it be': the managers can help to channel the evolution, minimising disintegrative mechanisms and supporting the creation of new collaborations. Especially the interfaces between sub-groups are fragile, while these connections are often the justification to perform such multi-actor RDI projects: losses of collaboration at the interfaces imply a loss of value of the project. This also suggests that the interfaces between the groups of project actors (modules) require a careful design at the proposal stage, and that the funding agencies may want to pay attention to these interfaces and how they evolve over the lifetime of the projects.

accepted by the funding agencies. The results also suggest that the funding agency should give flexibility to the funded projects, accepting that different outputs than originally anticipated may result and that some of the planned ones will have to be given up.

Limitations and future research directions

The exploratory research presented here has certain inherent limitations. It is limited to multiactor collaborative projects funded by the European Union Research and Innovation

Programmes, with a large number of autonomous organisations. It does not include multiactor research and innovation projects funded by national programmes which often are of smaller size, similar to the modules in the larger EU projects. Only a certain selection of themes and projects was investigated; thus, it is possible that there are additional types of project architectures and outputs or processes that have not been found in this study. The studies focused on a limited number of elements, and there are also other factors that may provide implications on the performance of the project, such as the historic experience of the partners, contextual conditions, the dynamics of the participating organisations [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF], diversity and other characteristics of the partner organisations etc. The assessment framework developed in the study #4 may need to be adapted to other domains, and additional elements, such as risks, may need to be introduced.

The findings of the four empirical studies point to several exciting future research directions:

(1) those which address the limitations of this research, (2) those which would deepen the understanding of concepts emerging in the studies; (3) those dealing with aspects not included in the studies.

The first avenue is about the extension of the findings to other contexts. This would concern investigations about e.g. additional types of project architectures, analysis of different contexts and domains, including different thematic domains (e.g. manufacturing, automotive, pharmaceuticals…), geographical or institutional diversity of organisational actors, or different settings. For example, the framework to assess the advancement of the innovation outputs, which was developed for projects of a specific domain, engineering with a strong ICT component, may be tested at a larger scale and in projects in other domains. In the study #4, it was postulated that the scoring system might need to be adapted but the adapted TRL scale should be possible to apply with minor modifications in the interpretation of the levels.

The efforts significantly increased too, and the advancement of the technologies may not be continued due to lack of resources and competition with other options. The links between knowledge management and innovation [START_REF] Barbaroux | Knowledge management and innovation: Interaction, collaboration, openness[END_REF] would also benefit from a deeper look. One of the relevant topics for future research would be managerial innovation [START_REF] Birkinshaw | Management Innovation[END_REF], and outputs related to it [START_REF] Damanpour | Managerial Innovation: Conceptions, Processes and Antecedents[END_REF] in this specific empirical setting.

The concept of meta-knowledge that was developed in the study #3 also calls for more indepth research, e.g.: Which mechanisms need to be in place to generate such metaknowledge? How does it depend on the nature of subprojects performed by subgroups, e.g., on their similarity? How important is its planning in the proposal?

It also may be of interest to include elements in the investigations which were not yet studied in this empirical context, but which may influence the project processes and performance. For instance, how do the characteristics of the firms, such as their corporate social responsibilities practices [START_REF] Bocquet | Are firms with different CSR profiles equally innovative? Empirical analysis with survey data[END_REF][START_REF] Bocquet | CSR, Innovation, and Firm Performance in Sluggish Growth Contexts: A Firm-Level Empirical Analysis[END_REF] influence the project processes and the innovation performance of the projects and the advancement of the innovation outputs, both within the project and after the project end? This analysis may include additional factors, e.g.

engagement of project participants.

The third large and exciting avenue for future research is the investigation of new directions that were not included in the studies. Some of them may be relevant to the findings and would extend them: for instance, the 'why' and 'how' of the 'future life' of the innovation outputs after the project end and its relationship to the project results merit further investigations: how do the project outputs advance, or not advance, after the end of the project thanks to the take up of the outputs by the participating organisations, why does this happen or not, how it could be supported e.g. within in the projects, and how the maturation of the innovation outputs could be better facilitated at higher TRL?

Scholars repeatedly noted the need to bring closer together project studies and innovation studies: a step towards this goal was done in this thesis. Another interesting direction, when studying the empirical setting of multi-actor RDI projects, would be to establish a connection between the project literature and the ecosystem literature. Value creation in these projects draws on both the overall project and on the dynamics of the individual actors and their collaborative couplings (partnering), similarly to knowledge based ecosystems [START_REF] Van Der Borgh | Value creation by knowledge-based ecosystems: evidence from a field study[END_REF], and the intra-actor relationship of constructs in the projects is very similar to the constructs in the ecosystem literature, with actors, resources, activities, dependence, risks, value addition, and value capture [START_REF] Talmar | Mapping, analyzing and designing innovation ecosystems: The Ecosystem Pie Model[END_REF]. Could the project setting, with its multiple interdependencies at different levels, be seen as an innovation ecosystem of temporary nature, how could the Ecosystem Pie Model [START_REF] Talmar | Mapping, analyzing and designing innovation ecosystems: The Ecosystem Pie Model[END_REF] be applied to the multi-actor project setting, how could the project actors adapt their behavior or influence the behavior of other actors, and how does this influence the project outputs? How do project actors benefit from being integrated in the 'temporary ecosystem'? How does the knowledge ecosystem within the project evolve to a business ecosystem, with an emergence of entrepreneurial activities (Attour and Lazaric, 2018)?

Another future research direction would be the use of the design research method, to enrich and to confront the results obtained in the present studies. The seminal paper of [START_REF] Romme | Making a Difference: Organization as Design[END_REF] explains the nature of design-science work and the application of design rules to organisation and management studies; the design principles offer a deeper understanding and aim at guiding the development of solutions in specific contexts and settings [START_REF] Romme | Making a Difference: Organization as Design[END_REF][START_REF] Denyer | Developing Design Propositions Through Research Synthesis[END_REF], as each specific situation is unique. There are now sufficient number of investigated elements to start from; the produced knowledge could serve as a foundation to elaborate design rules (Romme, 2003, van Aken and[START_REF] Van Aken | Reinventing the future: adding design science to the repertoire of organization and management studies[END_REF] and test them, in order to produce more nuanced, advanced and rich results. Here, testing means field testing, not a statistical validation. The four empirical studies show the need for design science method in this specific field of multi-actor RDI projects: it would help to decrease the gap between research and practice, respond to the challenges of the real world and help solving practical problems, while being grounded in the scholarly body of knowledge [START_REF] Pascal | Developing a Human-centred and Sciencebased Approach to Design: The Knowledge Management Platform Project[END_REF], it could help to e.g. improve project processes, leading to more positive effect on project performance.

This thesis aimed at developing new perspectives on inter-organisational multi-actor RDI projects, called here 'networked projects'. It opted for a multi-case qualitative study, taking a deeper look at the setting and the processes in these projects, as well as their implications on project performance, and identifying 'why' and 'how' elements. Among other findings, the research results highlighted the evidence of the interplay between activity coordination and knowledge integration in this type of projects; demonstrated their specific architectures, some opening up possibility of adaptive adjustments in addition to the modularisation of the work and outputs; put into question the beneficial effect of the size of the knowledge bases in this specific setting; identified a non-conventional output that was highly valued by the project actorsmeta-knowledge; and provided arguments that interfaces between modular in particular are of critical importance. The research also suggested how the innovation performance can be assessed more adequately in this type of projects, enriched both the project literature and the innovation literature, and put forward a number of suggestions for policymakers: one of the anonymous reviewers of the fourth study pointed out: "the central ideas … should provoke thought in public funding and monitoring of R&D". The findings of this exploratory research lay the ground for deeper investigations to better understand these projects and their performance, contributing to the improvement of the impact of future public funding of research and innovation. Grid

SCORPIUS

PROTOTYPE VALIDATION AND DEMONSTRATION

A technology prototype is an intermediate step in the development process where the essential features are present but the pilot implementation in the real environment needs further improvements. The installation is temporal and provisional. Only the essential features must be present.

TRL6. Prototype demonstrated in operational or near-operational environment

A prototype of an innovative technology is demonstrated in an operational or near-operational environment, or operated with real-time data from the operational environment. Near-operational environment means pilot facility or other setting that exhibits the main features of the operational environment, i.e. the environment where the innovation will operate when deployed as a pilot. Demonstration means that the prototype functions according to the specification when integrated with the real equipment or using real-time data that is provided by the end users (usually project partners). The goal is to get the green light to progress to the pilot deployment.

Demonstration of a prototype in operational or near-operational environment over a sufficiently long period of time to decide on the next steps in the development process.

TRL6.2. System prototype demonstrated in operational or near-operational environment See explanation above.

The system prototype consists of several elements (components) that have to be integrated into a system to perform the demonstration. A system prototype requires efforts to integrate components. Example: Measurement system that consists of hard-and software and is connected to the hardware and IT systems of a pilot facility.

See above: demonstration of system prototype with at least two components Technology prototype validated in the laboratory or testing environment, i.e. the basic functionality is proven under ideal conditions or in simple setups.

Serious tests under laboratory or simulated conditions are successful.

TRL3. Proof of concept

The principles and methods that were proposed are shown to work in principle for the targeted application

Preliminary tests are successful.

Appendix 5B. Refinement of the maturity scale: illustrative quotes (first order themes).

First order themes Illustrative quotes

Components vs system

"Our hope is that they will have a new system… and that will use the component that we developed". ID25_ORION_SME

"This was a system, and we had already have components demonstrated in operational environment…separate components. Putting it together was not so simple."ID52_GEMINI_SME « There is one system … that integrates all together, plus additional supporting components" ID39_GEMINI_large ind Complexity of components vs complexity of system "It was more (work) with system prototype… Because you have all the difficulties and problems with the system environment, with the systems of the customers, with the IT system departments and all these things. You have to do … well, in that phase, walk around to implement that on the customer's side." ID54_ORION_SME "It depended a lot on … the complexity and maturity of the components" ID49a_ PERSEUS_large ind Work in real environment "In real application, the operators and management came back with feedback." ID49_ORION_large ind "It is now used in the real operation of the plant". ID6_HERCULES_large ind Work in realistic (near-real, small scale) but not real environment "A very big element … was access to realistic industrial data" ID24_ORION_SME "To have an end user that can actually provide some insights about how they work and realistic data of their operation was a case that added value to my research. Otherwise, yes, I could go from the literature and take a fictitious case to work with." ID35_ORION_acad "It was one out of two real applications… these two were the first worldwide. Small scale but real (NB: at a pilot plant)" ID4_PERSEUS_acad "I was responsible for the demonstration phase … it was quite amazing to see that the device … was really running in our pilot plant. " ID15_PERSEUS_SME" "We were doing things that we were able to present, case studies were quite realistic". ID23_PEGASUS_acad Work in computer (synthetic) environment, simulation "It was a synthetic case … technology was validated in a few synthetic cases… not 100% realistic cases… on own PC." ID51_ORION_SME "I would like to have more practical results, to apply the project results to the use cases, so to have a higher TRL …not only to test it in simulation ID45_PEGASUS_large ind Deployment of one application/pilot "This was too complex for deployment, we could not advance… the company wanted but then dropped this idea" ID17_ORION_acad "I wanted at the end of the project to not only prove in the pilot but also have it operating." ID28_PERSEUS_large ind Deployment of multiple applications/pilots "In terms of pilots …. One which is actually the pilot itself…the idea was to do certain pilots and then I mean if you are a pilot you would be only concerned about how what are the result that will come out from your own testing. But for (partner X) would be able to derive useful information or useful results that would make substantial change… "The only change that we had we introduced was the third pilot" ID1_GEMINI_coord "The diversity of the different applications that the parties brought in, yeah. The common denominator is that the cases are very different. ID31_PERSEUS_res "There was a non-trivial stage from one to multiple (applications), because with one you can fool yourself that you have a product while really you have a solution for a particular customer" ID51_ORION_SME Innovative feature added to existing own product "As a result we did within ORION … it was bringing in a new dimension…We have added elements … to our technology and this has gone very well for us" ID24_ORION_SME "They want to develop their own tool, to upgrade" ID5_ORION_acad Result of development is completely new product "We developed a new IT platform… That was more than I expected" ID26_HERCULES_SME Temporary deployment, stopped to integrate a feedback "We did two pilots. Every pilot was specific, every application had its specific features… This doesn't mean you can roll out if the next application is different or significantly different". ID49_PERSEUS_large ind "We were able to obtain the demo the 21 of December, the demonstration on the pilot plant" ID30_PERSEUS_acad Permanent deployment (roll out) for internal use "The technology is really close to the market, and they don't have to do a big effort to explore it internally and use it, eventually" . ID34_ORION_acad Some levels are not required "Our products are meant to be offline, so the TRL level is a bit more hazy… (we went) directly from TRL 5 to 7. ID51_ORION_SME Back and forth loops to the previous level "It was an iterative process,… more iterative loops were done sooner than later. ID54_ORION_SME"It worked at TRL 5, proved. But with all the knowledge gained, we concluded that …it's much too costly to scale up. You can't scale it up anymore. Then that part went back to TRL 3, the rest is there at TRL 5". ID55_PERSEUS_res Work builds on previous development "Without the previous work, that would have taken so much time and effort" ID6_HERCULES_large ind "The project with (partner X) has been initiated previously to HERCULES" ID16_HERCULES_acad "If you are almost ready and you finally go for a product that looks very nice, but it might not be such a big step than if you start from TRL 3 and after four years you have something" ID46_ORION_acad Work started from early stages "We developed a new IT platform. We planned to do it to technical readiness level of five …. And at the end, I would say, we reached six or seven." ID26_HERCULES_SME "This was new. Well the XXX technology of course must been done before, but this typical application that was new and that was the step that we had to make. It was pretty risky and that's why also you have this EU funding." ID31_PERSEUS_acad "You have to spend more work… , you have to incorporate operator feedback, because they have to work with the system. And then you go into that iteration loops where you start with something, and for a prototypical implementation that's fine already you have the idea, you have a (result) and job done… Here, you come with a first idea, discuss with the operators and plant management, they come back with feedback, and you have to incorporate the feedback, redesign things, remodel things, come back with a solution and discuss it again. And then you go to these iteration loops, and even if you come with a, let's say, final solution after some time, there will be another feedback, okay, this is what I like, this is what I don't like, this is realistic, this is not manageable. And then again you go to this iteration loop… it's all the effort for the iteration loops …" ID49_ORION_large ind "We had more variety of resources. At the end, it could be, if you put the number of FTEs (NB: full time employees), I needed two FTEs, but you do that in 1 or 2 level higher, and it's still two FTEs. But instead of being two people, it's 20 people. What happened then is that the more people, the more meetings were needed to have to agree and to align. So, it's not only that you compare two FTEs with two FTEs, there were more and more people involved, but every individual has only a small amount of the time that they put in the project." ID28_PERSEUS_large ind_TRL7 "We realised it is too big in terms of number of people that we should involve. And it would not be manageable". ID29_PEGASUS_large ind Increased dependence from external project actors "It was on human resources, man-hours that you are allowed to spend on such a project. And you need to convince people … to spend the man-hours, the engineering hours, to progress on this project." ID49_ORION_large ind "We need other people, and those people don't have the same priorities, things are just extended in time. And this is what happens when you go to higher level, that you need more people, you need them because they have to do something. …to sign a document, or agree on something, or just do a very, a very tiny thing, but we cannot progress until they've done that, and if for them it's not the first priority, which happened many times, then everything is kind of dragging in time. That goes with the effort. ID57a_PERSEUS_large ind Increased IP issues "We realized it's such a large effort to implement it (IP)…" ID6_HERCULES_large ind "In ORION… there were licenses (we used) that are free for academia, for doing research, but as soon as you use that for production, that changed, there is no longer research. Maybe you are not making money out of that intervention incrementally, but the fact is that the technology somehow has been involved in the process of making a product that is now in the market." ID57b_compar_large ind Increased regulatory issues "(during pilot deployment) We introduced … regulators, increased the complexity ID52_GEMINI_SME

"…all the regulatory parts to consider (at high maturity stage)…ID49_PERSEUS_large ind

Complex technical integration into existing systems

The increase of technical difficulty the complexity …we could not evaluate initially. They underestimated the technical challenge (of advancement). Clearly (name of partner) could not … didn't have the resources (to deal with the challenge)"… ID28a_PERSEUS_large ind You have to deal with a lot of small details that in the end determine the success or lack thereof…there was a very high challenge… All this has been developed by (partner name) in a really remarkable effort." ID4_PERSEUS_acad Technical measures for safety, data security …

It was something like double safety… this made everything very unflexible and very, very expensive.

Nothing is allowed to go wrong, because you want to prove this is a safe environment... (in line with) explosion safety requirements. ID15_PERSEUS_acad "It happened in the last month …when we were going to the demonstration in the plant. You don't want to expose the plant to any cyber security attack. So, the precautions there are such that, yeah, we needed to go through a lot of checks and meetings with people to make sure that whatever is going to happen is not posing any cybersecurity risk … " ID57a_PERSEUS_large ind Necessity to deal with affordability (cost/benefits factors)

We did it quite nicely offline, but then, if you have to continuously stream new data and to implement this, this is quite an extensive amount of cloud architecture and data engineering… this is expensive, and I'm thinking from the expenses side, because you need a lot of workforce to do this… The effort to implement such a system does not have the payback. I mean, it works, but it's too expensive to implement... ID53_HERCULES_large ind It worked, but … the threshold was in terms of … you see if you want to implement something of that nature in the real (environment), it's a lot of effort, and for this you have to have a need which is bigger than the cost." ID3_HERCULES_acad Need for training and documentation "The third difficulty is installing and training the pilots. And that requires time … and effort and patience in order to set up the software, train the people, develop the user manuals … in a language that is user-friendly, so the others understand. So that's another difficulty that is there". ID14_GEMINI_SME "So, the main documentation, the main documents… user training slides I already did … I showed him the software with his data, and that's very important if the customer sees its data in the software, because… it makes him to be familiar with the software with his data. ID38_ORION_SME Need for maintenance "It (the prototype) became really integrated with the rest of our software, which means we need to maintain it forever. So, there's no way back. And it's not only effort to implement it, but it's also effort to maintain". ID51_ORION_SME Need for reliability "… have to spend more work to make it a real-world application. You have to work on your availability, you have to work on your reliability…" ID50_ORION_large ind "And of course, reliability was very important… maturity isn't just the technology itself, but also having an environment that supports the technology" ID49_PERSEUS_large ind Protected temporary environment: TRL3 to TRL4

In comparison… the step from 4 to 5 was more expensive than a step from 3 to 4, definitely. ID49_PERSEUS_large ind "You have an algorithm, which is, at least for software, more or less works for some few cases that you're pursuing. And then… (explain all the work which went after this)" ID51_ORION_SME Protected temporary environment: TRL4 to TRL5.1 "We could demonstrate that the (technology) is working…it was after one and half years… this was just a prototype." ID15_PERSEUS_acad It was just installing the system on a virtual machine and letting it a good run, so that was less effort" (comparing with the next steps) ID53_HERCULES_large ind In the prototyping environment, the effort was not so high, maybe 2 (comparing with the previous step). ID54_ORION_SME Semiprotected temporary environment: TRL5.1 to TRL5.2 (system difficulty vs components)

The integration was our weak point… the integration was much more difficult." ID52_GEMINI_SME "The system with about six to seven software components…" ID14_GEMINI_SME It depended a lot on … the complexity and maturity of the components" ID49a_ PERSEUS_large ind Semiprotected temporary environment: TRL5 to TRL6.1 "It was TRL5. But then you want to get from there to the actual implementation… it showed how big actually need the hurdle is, from TRL5… " ID9_PEGASUS_SME "It was three times the effort" (comparing with the previous stage) ID58_ORION_large ind) "What I saw, also in PERSEUS, that when you are at TRL 5 and you have to go to 6-7, there's a huge peak or a gap, there's this valley of death… That's a big step, big investments". ID55_PERSEUS_acad

We did this quite nicely offline, but then, when we had to continuously stream new data and to implement this, this is expensive, and I'm thinking from the expenses side, because you need a lot of workforce to do this. A lot of efforts…. This where we got all these problems, these technical problems that you have to interface with different systems. ID53_HERCULES_large ind "That's also something not only for us in this project, but there were other partners in this project who had the same issue…they needed …the data... that's where the big difficulty is. We needed information about the data. Is it good data, is this sensor working… all the specifics.. That is a lot of work, and I think at some point also there was a phase when (partner X) felt underresourced, they had to do a lot of work to help others get the data they wanted. ID37_ORION_SME

"the best we got to was the demonstration of one of these solutions in a real environment, otherwise for three out of four of the applications it was just demonstration in simulations….It was very nice that we did this real-life demonstration with real data of the development, and that was quite an achievement… to cope with all the dirty details of the real application… ." ID2_PEGASUS_coord

We had all kinds of requirements … it (demonstration) would cost a lot of money, with data from true processes … (explaining hard work behind this)" ID31_PERSEUS_res

We wanted to demonstrate the technology (at TRL6) … but… it was too difficult actually in the end, too ambitious… that would be far too expensive." ID11_PERSEUS_large ind "From where we are now, at the end of the project TRL five, let's say, if you would bring this further to real product, it would take you at least a couple of millions of investments to develop it further…you need a large huge investment now to bring it further to the market... So probably even could be that we need another consortium or another project program, together with related companies, to develop this from TRL five to TRL six". ID31_PERSEUS_res (same was confirmed by the industrial partners) Semiprotected temporary environment: TRL6.1 to TRL 6.2

At this system prototype level, we had bigger efforts… all the difficulties and problems with the system environment, with the systems of the customers, with the IT system departments and all these things… we had to walk around. ID26b_ORION_SME "We had to deal with the end-user perspective and … an operational environment. It was much more challenging, bigger-scale". ID35_ORION_acad "A demonstration of the prototype in the relevant environment was achieved, although in the last minute but it was achieved…. our first experiment failed….we fought very hard to actually have this integrated demonstration… small-scale but real…ID4_PERSEUS_acad

It was more (work) with system prototype… Because you have all the difficulties and problems with the system environment, with the systems of the customers, with the IT system departments and all these things. You have to do … well, in that phase, walk around to implement that on the customer's side. ID54_ORION_SME Non protected temporary environment: TRL6.2 to TRL7.1 (pilot deployment) "It (output) was killed at this step, it was too complex...too much efforts". ID11_PERSEUS_large ind "We demonstrated something at one customer that it works and that it gives a benefit. That was a key stage". ID51_ORION_SME "On month eighteen we started the testing and validation... we have said that we are going to send some developers to each pilot in order to be there for at least one or two weeks until the system works and we don't have issues, so that we can do the next step…"(explains a lot of work). ID13_GEMINI_acad "We had too much manual configuration and work on it, too much efforts" ID53_HERCULES_large ind "And then go to pilot deployment, ouf… all the failures we had before, we could ignore, but at the customer side, it was not acceptable. So, we have to do this kind of efforts to stabilize the system. We had other difficulties … specific versions of other systems to connect… other than we had in the lab... All these things are big efforts. No one sees that, these efforts." ID54_ORION_SME "For a prototypical implementation that's fine already you have the idea, you have a visualization and job done. In real application, the operators and plant management came back with feedback … , and you have to incorporate the feedback, redesign things, remodel things, come back with a solution and discuss it again. And then you go to these iteration loops, and even with the final solution after some time, there was another feedback, okay, this is what I like, this is what I don't like, this is realistic, this is not manageable… it' was all the effort for the iteration loops remodeling, making things operate more smoothly, debug things, problems collecting data and things". ID49_ORION_large ind "It was not a protected environment anymore, where you can design and deploy. You introduce new other complexities, you introduce actors, you introduce contractual and legal stuff that you have to sort out, you introduce support… a lot of new factors…the technical development was only part of the equation…deployment of the pilot was … an organizational challenge too". ID42_GEMINI_SME Non protected extended temporary environment: TRL7.1 to TRL7.2 "(Pilots) were difficult, installing and training…. that required time, effort and patience in order to set up …, train the people… so that the others understand… we introduced some organizational and even legal complexities". ID14_GEMINI_SME "The first pilot is the most expensive one... many issues to resolve, and then you can copy solutions to the next pilots". ID49_PERSEUS_large ind Non protected permanent environment: TRL7.1 to TRL8.1 "Yeah, it's a lot of effort… additional things like documentation, like online help …. The customers want to have that in that phase, OK. So, if you say that is a final, it is rollout, which could be at the site for longer time, more than the project. Then you have to have more efforts to do for that. Maybe 5 (time more) or something like that. ID54_ORION_SME

We have spent… at least two years or so, testing results and improving what we were doing… It has no interest for us at all, this part. But this is very important. It needs to be done in combination with the people in the factory. So, it's a lot of work, in terms of time. It's less interesting, but needs to be done… Yeah, it is a lot of time that needs to be devoted do that. And sometimes, imagine, we received data for ten months of operation. And then, it's very tedious to look at all this information, refine them, move to the data properly fit, and so on. These are tedious works. ID56_HERCULES_acad Non protected permanent extended environment: TRL8.1 to TRL8.2 More people were involved. So, more employees… It's mostly the efforts of other people who were involved. Maybe that is organizational, but it's also technical things to transfer. It's still internal, but the requirements of the customers, it's nearly the same. ID54_ORION_SME

Figure 1

 1 Figure 1.1. Project complexity: mapping the connections in the EC-funded project. ............................. Figure 1.2. Conceptual framework guiding the study. .......................................................................... Figure 1.3. Example of diversity of the studied projects: drivers behind the projects .......................... Figure 2.1. Conceptual framework: the interplay between activity coordination and knowledge integration ............................................................................................................................................. Figure 2.2. Conceptual representation of the project architecture (example of a project with three parallel modules and one integrative module). ..................................................................................... Figure 2.3. Types and sub-types of architectures of the projects included in the study. ...................... Figure 3.1. Types of architectures of the projects included in the study ............................................... Figure 3.2. Evolution of couplings, as observed in (a) HERCULES and b) PEGASUS projects. ....... Figure 3.3. Evolution of couplings, as observed in (a) PERSEUS and (b) ORION projects. .............. Figure 3.4. Evolution of couplings and implications on collaborative innovations, as observed in the multi-actor R&D projects in the study: a process model ...................................................................... Figure 5.1. Link between advancement of maturity and project performance .................................... Figure 5.2. Average total scores for the maturation of the IOs versus absolute efforts for these IOs..Figure 6.1. Results of the on-line survey: work in the projects of large size mostly happens in small groups. ................................................................................................................................................ Figure 6.2. Results of on-line survey: optimal size of the consortium. .............................................. Figure 6.3. Implications of different factors on project performance, and positioning of the four empirical studies. ................................................................................................................................

  Figure 1.1. Project complexity: mapping the connections in the EC-funded project. ............................. Figure 1.2. Conceptual framework guiding the study. .......................................................................... Figure 1.3. Example of diversity of the studied projects: drivers behind the projects .......................... Figure 2.1. Conceptual framework: the interplay between activity coordination and knowledge integration ............................................................................................................................................. Figure 2.2. Conceptual representation of the project architecture (example of a project with three parallel modules and one integrative module). ..................................................................................... Figure 2.3. Types and sub-types of architectures of the projects included in the study. ...................... Figure 3.1. Types of architectures of the projects included in the study ............................................... Figure 3.2. Evolution of couplings, as observed in (a) HERCULES and b) PEGASUS projects. ....... Figure 3.3. Evolution of couplings, as observed in (a) PERSEUS and (b) ORION projects. .............. Figure 3.4. Evolution of couplings and implications on collaborative innovations, as observed in the multi-actor R&D projects in the study: a process model ...................................................................... Figure 5.1. Link between advancement of maturity and project performance .................................... Figure 5.2. Average total scores for the maturation of the IOs versus absolute efforts for these IOs..Figure 6.1. Results of the on-line survey: work in the projects of large size mostly happens in small groups. ................................................................................................................................................ Figure 6.2. Results of on-line survey: optimal size of the consortium. .............................................. Figure 6.3. Implications of different factors on project performance, and positioning of the four empirical studies. ................................................................................................................................

  Figure 1.1. Project complexity: mapping the connections in the EC-funded project. ............................. Figure 1.2. Conceptual framework guiding the study. .......................................................................... Figure 1.3. Example of diversity of the studied projects: drivers behind the projects .......................... Figure 2.1. Conceptual framework: the interplay between activity coordination and knowledge integration ............................................................................................................................................. Figure 2.2. Conceptual representation of the project architecture (example of a project with three parallel modules and one integrative module). ..................................................................................... Figure 2.3. Types and sub-types of architectures of the projects included in the study. ...................... Figure 3.1. Types of architectures of the projects included in the study ............................................... Figure 3.2. Evolution of couplings, as observed in (a) HERCULES and b) PEGASUS projects. ....... Figure 3.3. Evolution of couplings, as observed in (a) PERSEUS and (b) ORION projects. .............. Figure 3.4. Evolution of couplings and implications on collaborative innovations, as observed in the multi-actor R&D projects in the study: a process model ...................................................................... Figure 5.1. Link between advancement of maturity and project performance .................................... Figure 5.2. Average total scores for the maturation of the IOs versus absolute efforts for these IOs..Figure 6.1. Results of the on-line survey: work in the projects of large size mostly happens in small groups. ................................................................................................................................................ Figure 6.2. Results of on-line survey: optimal size of the consortium. .............................................. Figure 6.3. Implications of different factors on project performance, and positioning of the four empirical studies. ................................................................................................................................

  Figure 1.1. Project complexity: mapping the connections in the EC-funded project. ............................. Figure 1.2. Conceptual framework guiding the study. .......................................................................... Figure 1.3. Example of diversity of the studied projects: drivers behind the projects .......................... Figure 2.1. Conceptual framework: the interplay between activity coordination and knowledge integration ............................................................................................................................................. Figure 2.2. Conceptual representation of the project architecture (example of a project with three parallel modules and one integrative module). ..................................................................................... Figure 2.3. Types and sub-types of architectures of the projects included in the study. ...................... Figure 3.1. Types of architectures of the projects included in the study ............................................... Figure 3.2. Evolution of couplings, as observed in (a) HERCULES and b) PEGASUS projects. ....... Figure 3.3. Evolution of couplings, as observed in (a) PERSEUS and (b) ORION projects. .............. Figure 3.4. Evolution of couplings and implications on collaborative innovations, as observed in the multi-actor R&D projects in the study: a process model ...................................................................... Figure 5.1. Link between advancement of maturity and project performance .................................... Figure 5.2. Average total scores for the maturation of the IOs versus absolute efforts for these IOs..Figure 6.1. Results of the on-line survey: work in the projects of large size mostly happens in small groups. ................................................................................................................................................ Figure 6.2. Results of on-line survey: optimal size of the consortium. .............................................. Figure 6.3. Implications of different factors on project performance, and positioning of the four empirical studies. ................................................................................................................................

  Abbreviations

Fig. 1

 1 Fig. 1.1. Project complexity: mapping the connections between 11 partners in the EC-funded project DITOs. Picture taken at the project meeting in 2018 by Dr Christian Nold, UCL, published by Prof. Muki Haklay, University College London, coordinator of DITOs. Used with permission.

Fig. 1

 1 Fig. 1.2. Conceptual framework guiding the study.

  Fig. 1.3. Example of the diversity of the studied projects: drivers behind the projects.

Fig. 2

 2 Fig. 2.2. Conceptual representation of the project architecture (example of a project with three parallel modules and one integrative module). Key to symbols used: 1: set of partners (organizations); 2: project boundary; 3: module; 4: components (with collaborating partners); 5, 6: dependencies; 7: integrative module; 8: main project output. Couplings within the components and intermediary project outputs have been removed for clarity.

Fig. 2

 2 Fig. 2.3. Types and sub-types of architectures of the projects included in the study. Key to symbols used: 1: set of project partners (organizations); 2: component (group of collaborating partners), 3, 4: dependencies, 5: integrative module, 6: main project output. Links at lower level and intermediary project outputs have been removed for illustration clarity.

Fig

  Fig. 3.1. Types of architectures of the projects included in the study. (a) planned architecture; (b) realized architecture -example of decomposition in PERSEUS. Key to symbols used: 1: set of project partners (organizations); 2: module (group of collaborating partners), 3, 4: dependencies, 5: interface, 6: collaborative innovation: project output. Links at lower level have been removed for illustration clarity.

  modules, (1) abandoning, i.e. dropping the planned collaborative work completely (PERSEUS, ORION), (2) splitting, i.e. bifurcation of work, following own paths, resulting in individual work instead of planned collaborations (PEGASUS, PERSEUS, ORION), and (3) consensual refocusing, i.e. consensual reconfiguration of the couplings in the modules, forming another coupling (PERSEUS, ORION) took place. (4) Bottom up integrationor formation of additional, unplanned coupling, was found in one case (ORION). (5) Restoration was a result of response actions of the project management to abandoning (PERSEUS, ORION), or splitting (PEGASUS).

Fig. 3

 3 Fig. 3.2. Evolution of couplings, as observed in (a) HERCULES and b) PEGASUS projects. Red arrow: response action of the project manager

Fig. 3

 3 Fig. 3.3. Evolution of couplings, as observed in (a) PERSEUS and (b) ORION projects. Red arrow: response action of the project manager

  7 disintegrative generative mechanisms and driving factors that activate a negative evolution of couplings, and two integrative mechanisms that activate a positive evolution. Three disintegrative mechanisms were observed in the modules: (1) cooperation issues; (2) technical barriers; (3) external pressure. At the interface level, it was (4) relational and cognitive complexity, leading to the inability of the partners to contribute to different topics in different groups simultaneously. Also there were other factors: (5) actorsrelated attributes, both personal and organizational, not being supportive to the collaboration, and finally (6) flaws in proposal preparation, which activated disintegration, such as illdesigned use cases of industrial partners, unrealistic plans or too high expectations. According to the project actors, some overestimations of the opportunities however is a typical ingredient of proposals for pre-competitive public funding and a certain degree of imprecision helps to adapt to new developments. At the level of modules, the most common factors were: (1) Unexpected technical barriers; (2) Cooperation issues, such as selfish interests, misalignment of interest; (3) Partner-related attributes at the organizational or individual level. "We have talked about organizational issues … very often in all these projects it boils down to individuals. You know, how open how willing they are to work together, exchange ideas" ID24_ORION_SME

  3.5.5 The evolution of couplings in multi-actor R&D projects: a process model Based on the previous findings, we propose a process model of the evolution of couplings, which influences the planned collaborative innovations in multi-actor R&D projects of modular type (Fig. 3.4).

Fig. 3

 3 Fig. 3.4. Evolution of couplings and implications on collaborative innovations, as observed in the multi-actorR&D projects in the study: a process model

4. 2

 2 Theoretical framework 4.2.1 Knowledge generation and innovation performance in collaborative R&D projects: the double role of the size 4.2.1.1. Knowledge generation and the need of variety.

  and consolidated meta-knowledge as a special type of RIO. The interviews revealed that three projects (HERCULES, PEGASUS and ORION) generated a specific kind of RIO, collective technical meta-knowledge, by the end of the projects. Metaknowledge is consolidated, collective inductive knowledge, a result of the synthesis of the project, based upon the comparison of different results, on understanding what works under which conditions, what matters and what not. It is knowledge beyond the state of the art, was

  both new scientific knowledge and technological innovations, funded on the basis of competitive open calls by the European Commission in the context of its Framework Programmes of Research and Innovation, and specifically its LEIT (Leadership in Enabling and Industrial Technologies) Programme (EC, 2020a) in Horizon 2020 and its predecessor, FP7. Such projects in the engineering domain have scientific and technological objectives and are aiming at developing new knowledge and innovative technologies of generic nature that are often tested or deployed in pilot use cases. The innovative technologies may reach different levels of maturity but are still upstream of the market. The projects often build on previous work that has led to different levels of maturity of the technologies, usually starting at least at TRL3 (proof of concept in lab or in computer simulations). Funded projects usually last three to four years and are carried out by inter-organizational consortia with multiple, diverse and geographically distant partners from different countries, including research organizations, large industry and technology providing SMEs.

Fig. 5

 5 Fig. 5.2. Average total scores for the maturation of the IOs versus absolute efforts for these IOs: (a) PERSEUS and HERCULES; (b) ORION. The trendline is shown per project.

  Fig.5b

Fig. 6

 6 Fig. 6.1. Results of the on-line survey: work in the projects of large size mostly happens in small groups.

Fig. 6

 6 Fig. 6.3. Implications of different factors on project performance, and positioning of the four empirical studies.

  Horizon 2020 and its predecessor, FP7. As mentioned in Chapter 1.1, the latest Horizon 2020 FP has already provided several billion of euros in 2014-2020 to fund over six thousand multi-actor projects. The projects have scientific and technological objectives and are aiming at developing new knowledge which results into knowledge (research) outputs, such as publications, patents, databases, and in innovation outputs, such as new technologies, materials, devices, algorithms, software, services, manufacturing processes, etc. The Horizon 2020 Programme uses the term "research and innovation action" or "innovation action" for these projects, to highlight innovation beyond development. Technologies may reach different levels of maturity but are still upstream of the market. The projects must advance the state-ofthe-art, they often build on previous work on different levels of maturity of technologies,

The research setting is defined as multi-actor collaborative RDI projects developing both new scientific knowledge and technological innovations, funded through competitive open calls by the European Commission through its Research and Innovation FP, and specifically through its Leadership in Enabling and Industrial Technologies (LEIT) Programme (EC, 2020a) in usually starting least at TRL3 (proof of concept in lab or computer simulations). Funded projects usually last three to four years and are carried out by inter-organisational consortia with multiple, diverse and geographically distant partners from different countries, including research organisations, large industry and technological SMEs. The projects are managed (coordinated, in the EU terminology) by a coordinator (a legal entity and a responsible person, called project manager in this paper) which can be from industry, a research organisation or a consultancy.

s such a large effort to implement it, that

"It's not something that you could patent, the ideas were new but not new in the sense of new molecule … but we realized it'being first on the market is far more important than having a patent". ID6_HERCULES_large ind 1.7 Research design 1.7.1 Research method

Table 1 .

 1 Table 1.1.

		Informants, number	Interview;
	Profile	Total	Including those	number
			participated in two or
			more projects under
			study
	Research/university	11	2	19
	Industry	23	5	33
	incl. large industry	11	4	18
	incl. tech SMEs	12	1	15
	Consultancy	2	0	2
	TOTAL	36	7	54

1. Overview of informants and number of interviews.

81st Annual Meeting of the Academy of Management (AOM2021), August 2021: peer

  -reviewed, accepted.

			innovations that are		mechanisms that activate the
			generated by the		reconfiguration of couplings.
			projects?
					3. Systematic evolution in the
					negative direction at the level
					of the interfaces.
					4. Process model of the
					evolution of collaborative
					couplings and its influence on
					the collaborative innovations in
					multi-actor RDI projects.
	Chapter 4.	Is bigger better?	Does the project size	Comparative inductive	1. Absence of a relationship
	Empirical	A deeper look	have a positive	multiple case study	between the project size and the
	study #3	on size and	(overproportional)	(Eisenhardt, 1989), with	number of project outputs,
		innovation	effect on the quantity	additional quantitative	beyond the proportional
		performance of	of research and	supporting evidence	increase due to the larger
		multi-actor	innovation outputs		number of partners (additive
		R&D projects:	(RIOs), i.e. the		effect)
		Empirical	collective innovation
		evidence from	performance, in		2. Puts in question the
		European	collaborative R&D		beneficial role of the size of the
		Research and	projects with larger		knowledge base on the research
		Innovation	numbers of		and innovation outputs in the
		Programmes	organisational actors,		RDI project context
			and if yes, how?
					3. Meta-knowledge as a
					specific output, often hidden
					but valued by the project
					partners. Thus, a positive effect
					of size, but depends on project architecture and processes	. The evolution of ties in multi-actor projects:
					4. Implications of the project size: negative: within the project, in the short term; positive: mostly outside the	How dynamic reconfiguration shapes collaborative knowledge creation. Track 34 « New Approaches to Organizing Collaborative Knowledge Creation"
					project, in the long term	EURAM2021 conference, June 2021:
						peer-reviewed, accepted.

R&D Management conference, June 2020: accepted, conference cancelled - COVID-19. EURAM 2021, June 2021: peer-reviewed, accepted. 81st Annual Meeting of the Academy of Management (AOM2021), August 2021: peer

  -reviewed, accepted.

	Chapter 5.	Climbing up the	How to assess the	Indictive multiple case	1. Framework to measure the	Research Policy. Status: revise
	Empirical	ladder: Towards	'innovation side' of	study (Eisenhardt, 1989)	progress in maturity of	A first version of this paper was submitted to
	study #4	a better	the performance of	with additional quantitative	innovation outputs on their way	Research Policy. A revised version was
		understanding	the projects	evidence	to (potential) innovations, from	submitted in April 2021.
		of the	objectively and		the point of view of
		innovation	directly, first on the		technological readiness,
		outputs and of	level of the		directly and objectively. This
		the performance	innovative		includes: (a) refined TRL
		of market-	technologies		(Technology Readiness Scale)
		upstream	(innovation outputs)		for the use in multi-actor
		collaborative	and then on the		market upstream projects; (b)
		research and	project level		scoring system.
		innovation
		projects
					2. While the absolute amount
					of resources significantly
					differed for each IO, the
					relative increase of the efforts
					between the start and end
					maturity levels of the different
					IOs in the studied projects was
					relatively similar independent
					of the IO or the path taken
					3. Better understanding of the
					'innovation side' of the
					projects: collective innovation
					performance, complementing
					existing perspectives
					4. Policy implications

81st Annual Meeting of the Academy of Management (AOM2021), August 2021:

	A previous version of this paper "A
	Framework for Measuring Market-upstream
	Innovations and Innovation Performance in
	R&D Projects" was submitted to AOM2021,
	peer-reviewed, accepted.
	Chapter 6. Conclusion
	Synthesis of the results; summary of theoretical and practical contributions; limitations; future research directions.
	Bibliography
	Annexes
	Table 1.3. The structure of the thesis manuscript.

Table 2

 2 Table 2.1.

		Informants, number	Interview;
	Profile	Total	incl. those participated	number
			in two or more projects
			under study
	Research/university	11	2	17
	Industry	22	5	24
	incl. large industry	10	4	12
	incl. tech SMEs	12	1	12
	Consultancy	2	0	2
	TOTAL	35	7	43

.1. Overview of informants and number of interviews.

  Table 2.2. Overview of interviews per case study.

			Interviewed								Type / sub-type of
	Anonymous name	Partners (organi-zations), number	organizations per case Num % ber	Interviews (including compar.), number	Number of interviews per profile of informants Industry Rese Consul arch tant Large SME	Recor-ding, min.	Trans-ripts, pages	project architecture (de facto)
												workflow-integrated,
	GEMINI	13	6	46	6	1	1	3	1	410	118,1	sequentual
												workflow-integrated,
	SCORPIUS	14	5	35	5	1	1	2	1	298	87,6	converging
	HERCULES	10	4	40	6	3	2	1	0	443	130,3	waterfall
	ORION	17	11	65	11	4	3	4	0	639	201,1	grid
	PERSEUS	15	7	47	8	5	2	1	0	527	145	decomposed
	PEGASUS	12	6	50	7	3	3	1	0	384	109,1	weakly coupled
										2701
					43	17	12	12	2	(45.02 h) 791,2

Type of project architecture Sub-types of project architecture Number of main outputs Structure of activities Knowledge base

				Decomposition of	Integration of
				activities	activities
	Decomposed	N/A	Several (at	Decomposed into	No couplings	Strongly integrated
	(highly		least one	independent	between modules	within modules, no
	decomposable)		per	modules, with		coupling between
			module)	dense couplings		modules
				within modules
	Nearly	Three:	Several (at	Decomposed into	Couplings	Integrated within
	decomposable	weakly coupled,	least one	independent	between modules	modules, nearly
		grid, waterfall	per	modules, with	are not	decomposable in
			module)	dense couplings	pronounced but	integrative module
				within modules	relevant
	Workflow-	Two:	One	Integrated, with	Integrated,	Decomposed; integration
	integrated	sequential		strong couplings	elements are	only happens at the inter-
		converging		between groups of	sequentially	faces and on a lower
				activities	dependent	level if the group of
						partners works together

Table 2

 2

.3. Three main types of architecture in inter-organizational R&D projects.

Table 2

 2

	Low	High	Low, then high at	Preferably
			the last stage

SCORPIUS, converging)

disintegrate knowledge: the partners contribute what they know best, this results mainly in isolated work at the organizational level with occasional synchronization. Some partners may .5. Implications of the types of project architectures on knowledge integration, collaboration, project resilience and project management

  Multi-actor R&D projects are a setting where a network of multiple organizational actors, usually from research and industry, creates a temporary consortium and are funded based upon a proposal to jointly develop new knowledge and innovations. The couplings between the organizations are designed in the proposal, however based on incomplete knowledge. Using a comparative multiple case study of multi-actor projects funded in European Research and Innovation Programs, we investigate why and how these planned couplings, i.e. the collaborations between organizations evolved during the different project phases and what effect this had on the collaborative innovations.

	Chapter 3. The evolution of collaboration in multi-
	actor R&D projects: Patterns of couplings and
	collaborative innovations 5
	Joint work with Sebastian Engell
	and Catherine Thomas
	Study # 2.
	Abstract.

Thus why and how dynamic reconfiguration happens in structurally inflexible multi-actor projects, and what the consequences of this are on new knowledge generation should be investigated. The project size regularly increases: for instance, it has more than doubled in the European Research and

Innovation Programmes during the last 20 years, from 6.1 to 13.7 partners per R&D project on average 4 . Whether larger inter-organizational settings, which are not easy to structure, are beneficial for knowledge generation and how the project management and knowledge integration mechanisms should be adapted to this trend is another promising research direction, which also could be useful for the designers of the research programmes.

Table 3

 3

					Informants, number		Interview;
			Profile		Total	incl. those participated	number
						in two or more projects
						under study
			Research/university	9		2		15
			Industry		15		5		17
			incl. large industry	8			4	10
			incl. tech SMEs		7			1	7
			TOTAL		24		7		32
			Table 3.1. Overview of informants and number of interviews.
	Anonymous project name	Partners (organi-zations), number	Interviewed organizations per case number %	Interviews, informants per case number	Number of interviews per profile of informants Industry Research Large SME	Recording, min.	Transcripts, pages
	HERCULES	10	4	40	6		3	2	1	443	130,3
	PEGASUS	12	6	50	7		3	3	1	384	109,1
	PERSEUS	15	7	47	8		5	2	1	527	145
	ORION	17	11	65	11		4	3	4	639	201,1
										1993
	TOTAL				32		15	10	7	(33.2 hours)	585,5

.2. Overview of interviews per case study.

Table 3

 3

	Reconfiguration of couplings

.4. Reconfiguration of couplings in the projects under study

  "Partners have ecosystem of software tools, we have a software tool… it's hard to connect all these tools. I think the tooling creates walls, and it's not that people don't want to collaborate, it's just that the only thing that they can exchange, is data… and they can't work with each other tools very easily, some tools are licensed, and some tools you have to know quite a bit about the tools... this sets limitations (supp1_ORION_SME)

they did not dig for it… you

"Partner X did not really want to work on the applications (of the partner Y)… realize that the problems they present to you do not actually fit what you're interested in, but then you can dig deeper, you can try to develop something with them where it could fit, okay, or you can go home and say oh the problem doesn't fit. And that's bluntly speaking what (partner X) did." (ID36a_ORION_compar_acad).

  Consensual refocusing of couplings in ORION had positive implications on the project innovations and were possible because of several intertwined elements, such as relatively broad description of technical work in the proposal, giving slack and thus possibility to accommodate reconfigurations, some leeway from project manager…

"The project proposal probably has to create more possibilities than can actually be exploited, because you have to narrow down, you have limited resources. Q: maybe to narrow it already on the proposal stage by digging deeper? A: there are two reasons not to do this, one is that you don't have the knowledge, and the second one is that you have this matrix structure, you have the use cases on the vertical, you have the methodological work on the horizontal, and when you write a proposal you have to make sure that at many of these intersections you have crosses. And clearly some topics are important everywhere, so you cannot write in the proposal 'we will only explore this in two use cases out of five', but in the end it may not be feasible …so if you do it for two, the others may learn from it later. I'm not worried about that." ID36a_comparative_acad

  "I was so busy (NB: with the other project)… since it didn't work out (NB: with planned partner), it was not like, okay, I've got free time available and I'll try to spend it to work in other use cases. Also, to be honest … I have no idea that it is possible. I was like a soldier. There is a list of tasks, and you do it". ID38_ORION_SME

  "When you struggle (NB: with the main work) then you're not so concerned (NB: about other work). And that was the case yeah and then you fight with technical problems…" (ID4_PERSEUS_acad)

"It´s like adding balcony to the house, you know: you can live in a house without a balcony, but it looks nicer… with a balcony (smiles). These were balconies, these tasks… Balconies are nice to sit and have a broader view.

However people are more concerned with digging in the soil to plant... " ID43_ PERSEUS_acad

. Level of interfaces: large collaborative groups

	Type of	Changes in the pattern	Projects and phases where	Action of the partner-	Action of other	Attitude / action	Implication on planned collaborative
	reconfiguration	(configuration) or in	reconfigurations were observed	initiator regarding	involved partner(s),	of the project	innovations in the projects
	of couplings	intensity (strength) of		planned collaboration	and implications	manager
		coupling
	A	B	C	D	E	F	G
	I. Level of project modules: small collaborative groups
	1. Abandoning	Disappearance of	PERSEUS (Phases 1, 2), ORION	Abandoning of the	Informed. No	Intervention or	Negative:
		coupling (pattern)	(Phases 1, 2)	collaborative work	planned R&D	laissez-faire	Loss of planned collaborative innovations
	2. Splitting	Separation (de-merge, or	PEGASUS (Phase 2), PERSEUS	Individual work done	Informed.	Intervention or	Moderately negative: Individual innovations
		splitting) of couplings	(Phase 1 and 2), ORION (Phase	instead of collaborative or	Loss of	laissez faire	instead of planned collaborative innovations
		(pattern)	2)	less collaborators	collaboration
					opportunity
	3. Consensual	Refocusing: replacement	PERSEUS (Phase 1), ORION	Effort is refocused on	Contribute to the	Participates	Neutral: Equivalent outcome on the project
	refocusing	of some couplings by	(Phases 1 and 2)	promising directions. Not	extent possible.	actively.	level.
		others (pattern), often		promising planned topics	Look for other
		accompanied by		may still be dealt with	topics. Re-assign
		downscaling		reduced effort	efforts
	4. Bottom up	Formation of new,	ORION (Phase 2)	Search of new	Search for new	Support,	Positive: Increase of collaborative
	integration	unplanned coupling		opportunities	opportunities.	encouragement, or	innovations unless other promising directions
		(pattern)			Possibly	intervention	are not realized any more
					reallocation of
					resources.
	5. Restoration	Temporary change of	PERSEUS (response to	Reducing efforts, then	Active management	Active	Neutral: Preservation of planned
	(response to	coupling, then restoration	abandoning, Phase 2), ORION	returning to plans	and	management,	collaborative innovations
	abandoning or	(pattern)	(response to abandoning, Phase		counterbalancing.	counterbalancing
	splitting)		2), PEGASUS (response to		Plans stay in place
			splitting, Phase 2)
	II6. Downscaling	Some couplings are	PEGASUS (Phase 2),	Some planned	Laissez-faire.	Laissez-faire	Moderate negative: collaborative
		downscaled (intensity),	HERCULES (Phases 1 and 2),	collaborators retreat (do	Continuation, with		innovations generated as planned but with
		may lead to	PERSEUS (Phase 1, leading to	not provide input)	input from some		lower N° of contributors: ORION (planned
		decomposition	decomposition, see below),		partners		innovation with less contributors);
			ORION (Phases 2 and 3)				PEGASUS, ORION (down-scaled
							innovations); restored (HERCULES)
	8. Decompositi-	Slow degradation of	PERSEUS (full decomposition,	No initiator: all planned	Withdrawal or	Laissez-faire	Negative: loss of collaborative innovations
	on	couplings until full	Phase 2), ORION (partial	collaborators slowly	continuation,		(PERSEUS, ORION). Moderate negative:
		disappearance (pattern,	decomposition of one interface	retreat	individually		Individual innovations instead of planned
		intensity)	Phase 3)				collaborative ones (PERSEUS)
	9. Fluctuation	Yes, but temporary	HERCULES (Phases 1 and 2),	Collaborating partners	Continuation even if	Active	Neutral: Preservation of planned
	(response to	(intensity)	ORION (Phase 2)	silently retreat but re-	some partners	management,	collaborative innovations, with lower N° of
	downscaling)			investing efforts	retreated	counterbalancing	contributors (HERCULES, ORION)

Table 3 .

 3 5. Evolution of couplings (patterns and intensity), implication on involved project actors and on planned collaborative innovations, as observed in the studied project

findings relevant to this paper: the relationship between the project size and project performance, considering (some) research and innovation outputs (RIOs) Positive relationship: the increase of partners positively influences the RIOs

	Authors Main Szücs (2018) Quantitative Multi-actor projects funded by FP7 Method Empirical setting Considered five innovation indicators, including patent count. Strong correlation between the patenting
		"Cooperation" sub-programme:	activity of the participating firms (but not the projects) and the size of the projects, in terms of the number of
		1656 projects with 1,975 firms generating	partners.
		patents.	The number of project participants, especially the number of university participants, positively influences
			the project performance.
			"A standard-deviation-increase in either project participants (about 43 additional organizations) … increases
			all innovation indicators by 18-50%, where the effect is particularly strong on patent count (41-46%)."
			(Szücs, 2018, p. 1264)
	Jacob et al.	Quantitative 2706 projects funded by FP7-funded ICT	An increase in the number of participating organizations increases the number of publications, including (1)
	(2019)	projects launched during the years 2007 to	journal articles; (2) papers originating from conference proceedings; (3) published books, book chapters,
		2013	reviews etc. However, no effect of increase of project size on the number of patents has been found.
	U-

shaped or mixed relationship: sometimes positive, sometimes negative, depending on the project characteristics

	Breschi and	Quantitative 734 EC-funded ICT projects (6 th
	Malerba	Framework Programme), of three types:
	(2011)	IPs, NoEs and STRePs 9

  Project size has an influence on project joint performance. It is different in exploitation and exploration projects. The joint performance of exploration projects positively depends on the number of partners. The joint performance of exploitation projects negatively depends on the number of partners.

			European Framework Programmes by 500
			biotech firms (informants)
	Lee et al.	Quantitative 416 cooperative R&D projects funded by	The number of project partners has an inverted U-shaped relationship with ICT convergence, regarded as
	(2017)		the ICT-based industry convergence R&D	innovative performance of the project: for larger numbers of partners, the size of the partnership has a
			Program, Republic of Korea.	diminishing marginal effect on the innovative performance of the projects.
				ICT-based industry convergence is defined as "technologies that lead to economic, societal, and cultural
				changes by introducing creative values that come from synergic combinations of these technologies » (Lee
				et al., 2017, p.467)
	No relationship between the number of partners and RIOs
	Schwartz et	Quantitative 406 subsidized R&D university-industry	No significant effect of the number of partners in the project on the number of patents and publications
	al. (2012)		collaborative projects, funded by one of	generated by the project.
			the German regional Support Schemes for
			R&D Cooperation. 564 different partners
	Callaert et	Quantitative,	81 projects which had at least one patent	No relationship between project characteristics and patents
	al. (2015)	complement	application, funded by 7 th Framework
		ed by ten	Programme, "Cooperation/ NMP" sub-
		interviews	programme
	Enzing et	Mixed	Projects funded by the EU FP7 NMP	No relationships between the number of partners in the project and publications.
	al. (2015)		thematic area: 508 projects that were
			finished or that had passed the mid-term
	Teirlinck	Mixed	Publicly funded university research assessment of the ; 51 case studies
	and Spanos and	projects funded by the Mobilizing Quantitative Organizations having
	Spithoven Vonortas		Programs (Belgium). Both public-public participated in the EU FP5 and FP6. 7,098
	(2015) (2012)		and public-private types of projects. and 2,921 respondents (two surveys)
			173 projects with 711 organizations
				The large size of the consortium, and thus a wide knowledge range, is not beneficial to use-inspired
				outcomes related to basic research; the number of partners is less important when the project deals with
				applied research.
	Arroyabe et	Quantitative Joint R&D projects (both of exploitation	Perception measure of the joint performance expressed by the firms: the levels of achievement of time,
	al. (2015)		and exploration nature) carried out within	budget and results in the joint project. Joint performance measures the probability of successful performance
				of exploration and exploitation joint R&D projects.

Projects includes the following categories: (1) Pure applied research (i.e. the application of state of the art scientific knowledge in new market domain); (2) Pure basic research , i.e. project resulting in new state-ofthe-art scientific knowledge, and (3) Use-inspired basic research The project resulted in new state-of-the-art scientific knowledge and its application in new market domains. Big number of partners stimulates pure basic research. This suggests that large projects are beneficial for technological breakthroughs and publications.

  Secondly, each RIO was analyzed with respect to the contributing organizations: how many organizations collaborated to develop the different RIOs. Thirdly, we analyzed the stability of collaborative partners throughout the project: whether the collaborative groups change, or new partners are added to the existing collaborations in order to generate RIOs.

	detailed interview guide was developed, and all interviews were recorded, with prior consents
	of informants. In each project, representatives of between 40% and 65% of the participating
	organizations were interviewed. The informants were selected using three criteria: different
	profiles of the partners-organizations (large industry, technological SMEs, research
	organization), different roles of the persons (coordinator, workpackage leader, contributor),
	and different experience of participation of the persons in EC-funded projects (newcomers
	and experienced participants). In order to obtain comparable data, over 80% of experienced
	informants were selected: CEOs of SMEs, team leaders within large multinational
	corporations, or tenured university professors. 7 informants participated in more than one
	project from the sample, adding to this research a comparative multiple-case dimension that
	so far is underrepresented in the literature (Bakker et al., 2016).
					Informants, number		Interview;
			Profile		Total	incl. those participated	number
						in two or more projects
						under study
			Research/university	9		2		15
			Industry		15		5		17
			incl. large industry	8			4	10
			incl. tech SMEs		7			1	7
			TOTAL		24		7		32
			Table 4.2. Overview of informants and number of interviews.
	Anonymous project name	Partners (organi-zations), number	Interviewed organizations per case number %	Interviews (including compar.), number	Number of interviews per profile of informants Industry Research Large SME	Recording, min.	Transcripts, pages
	HERCULES	10	4	40	6		3	2	1	443	130,3
	PEGASUS Programme has already provided 34,42B€ in 2014-2020 to fund 6288 such multi-actor 12 6 50 7 3 3 1 384 PERSEUS 15 7 47 8 5 2 1 527	109,1 145
	projects with 14.1 organizations per project on average (EC, 2021a). The projects have both ORION 17 11 65 11 4 3 4 639	201,1
	1993 (33.2 hours) scientific and technological objectives and aim at developing new knowledge and TOTAL 32 15 10 7	585,5
	innovations. Most of the projects have a strong application element, which means that both an Table 4.3. Overview of interviews per case study.
	explorative and an exploitative dimension are generally present in the same project;
	innovations are often of incremental nature. The data analysis phase includes several steps. Firstly, different types of RIOs were
	identified, based on the categories of RIOs suggested in the literature: this included innovative
	technologies of different degrees of maturity, patents, publications of different types, datasets

projects that are funded through competitive open calls by the European Commission through its Framework Programmes of Research and Innovation. The latest Horizon 2020 Framework This setting is particularly interesting for innovation research: first, because multi-actor collaborative R&D projects have become an important way of research funding, and their number has been growing, and second, because the funding agencies have steadily increased as well as new advancement of knowledge, not published yet but identified and included in the project deliverables. Then the number of RIOs per category and per project was determined.

Table 4 .

 4 4. Number of research and innovation outputs in the studied projects in different categories.

		of RIOs per projects
	HERCULES	PEGASUS	PERSEUS	ORION
	(10	(12	(15	(17
	partners)	partners)	partners)	partners)

  this is new knowledge, you see especially what does not work, and you see that certain approaches are useful, successful, and also I mean you see work which is theoretically interesting." ID36a_comparative_acad_ORION

Table 4 .

 4 Table 4.5, detailed data are shown in Appendix 4.4). 5. Number of RIOs in the studied projects over different sizes of the contributing groups.

		Number of	Meta-	Research and innovation outputs per contributing organizations (org.) (number)
	Project	documented	know-
	name	conventional	ledge
		RIOs		1 org.	Group of 2 org. Group of 3 org. Group of 4 org.	Group of 5 org.
	HERCULES	35	1	14	18	2 + 1 meta-knowledge	1	0
	PEGASUS	53	1	40	9	3	1 + 1 meta-knowledge	0
	PERSEUS	31	0	26	2	1	2	0
	ORION	63	1	31	25	4	3	0 + 1 meta-knowledge
						10 +1 meta-	7 +1 meta-	0 +1 meta-
	Total	182	3	111	54	knowledge	knowledge	knowledge

Table 4 .

 4 5 shows that all 182 conventional RIOs were created by sub-groups of 4 organizations or less. From all these RIOs, 96% were created either individually (61%) or in small groups of 2-3 collaborating partners (30% and 5%, respectively). Even in the biggest project, ORION, where the largest number of RIOs were created, the number of contributing partners per RIO remained small. Table4.6 specifies these results by showing the average and maximum number of organizations by type of RIOs.

				Number of contributing organizations per RIO
	Categories of research and	Average	HERCULES (10	PEGASUS (12	PERSEUS (15	ORION (17
	innovation output (RIOs)	across 4	partners)	partners)	partners)	partners)
		projects*	Average	Max	Average Max Average Max Average	Max
	1. Innovations	1,95	2	3	2	3	1,7	4	2,1	4
	2. Patents	1	N/A	N/A	N/A	N/A	1	1	N/A	N/A
	3. Publications	1.40	1,67	5	1,18	3	1,29	4	1,45	3
	4. Other general advancement of knowledge	2.33	2	2	4	4	1	1	N/A	N/A
	5. Meta-knowledge	4	3	3	4	4	N/A	N/A	5	5
	*calculated only for those projects where the category is present
	Table 4.6. Number of contributing organizations per RIO in the projects under study: average and maximum per
				project.

  "If your aim is to come together and deliver something, you know, not so much about learning generally… then it (project) shall be more focused… then it is clear what each partner needs to deliver…". ID24_ORION_SME There were also partners who actually didn't understand what the project is for. … If I had knew before what would happen, I would kick them out and replace by a different partner. ID11_PERSEUS_coord

group was so large in contrast to HERCULES… it

  was maybe too much to hope for. Too much expectation for this group to work together…. ORION

"What is the big difference (between HERCULES and ORION)? …. Number of partners is many… The problem, because again, because the

didn't fall into place as quickly as HERCULES… and

  it was also because of so many partners very difficult to keep an overview, at least for me… And that's why there is, from a human perspective, an expected tendency to concentrate on the sections that you understand, which as an industrial case study provider is your case study, and as the solution provider is the case study that you're closest to.

And that's a big difference (with HERCULES). That's why ORION fell apart…

	ID6_HERCULES_large industry
	"The consortium (ORION) is bigger (than HERCULES), and due to this we have different challenges. …we have
	different fractions, different groups of companies that work closely together, and the exchange of ideas and
	concepts is more pair… this is different from what was done in HERCULES project. I think it's just the number
	of players." ID40b_compar_ORION_large industry
	4.5.3.2.

Positive effect of project size: long term effect, mostly outside the project.

  Out of 30 informants, only two indicated that the size of the project was beneficial for their knowledge generation and innovation performance within the time span of the project. In the largest project, ORION, one case was identified when unplanned recombination of knowledge indeed happened and resulted into a RIO. The main positive effects of the project size that

were identified by the actors were the following (Appendix 4.3):

1. Broader scope of technical feedback: to be used within the project and outside it. With a larger group, more technical suggestions could be received during the project:

"If you are only looking on the results in your own company, you will lose the big picture, and this is something that that still happens… So it was very helpful to get other opinions and other mindsets than only having the mindset that you get at some point if you are working in a single company for some years.

(ID40a_compar_ HERCULES_large ind)

  But for me it's always interesting to see what they are doing, and how they progress … There are some topics … something where I never thought about it… I definitely gained some questions … afterwards to take a look, and which I probably didn't get if I wouldn't be in ORION" (ID38_ORION_SME)"There is a benefit from general awareness, learning. … I was making distinction between learning and actively doing things together. To do things together by necessity is very different … (ID24_ORION_SME)

	Short term benefits versus long term benefits. In the interviews, several actors made a
	difference between long term benefits (outside the project), and work in the project for the
	purpose to generate RIOs (short term benefit):
	"I don't think you can meaningfully interact with seventeen partners, of course you can go to meetings, you can
	listen to presentations, you may learn something interesting from everybody anybody, but in terms of more
	active interaction… the rest of us are just interacting with a subset of the partners…Seventeen partners it's just
	impossible yeah you know to do something …(ID24_ORION_SME)
	"Of course the size gives more opportunities... Whether people grasp the opportunities is a different question. I
	think especially it is attractive for the technology providers, okay, because the technology providers have sort of
	a bigger set of possible clients whom they can contact and offer and work together…. The opportunities are
	there but it depends then on people and on interests and on conditions and outer pressures" (ID5_ORION_acad)
	c) Broader learning opportunities. Larger inter-organizational settings provided broader
	learning opportunities and broader general awareness. However, this was mostly not directly
	related to the project work:

Q: The fact that there are seventeen partners, how does it help you? A: It is helpful for me only first month, to get an understanding of what is going on outside the world I'm doing… we don't have anything to do with them.

… "You have to work hard to understand to fully understand the problems… to do that, you need to interact a lot with the companies…you have to work with people closely". ID23_PEGASUS_acad

"The project is the mean to have a number of very intelligent people in the room. And if you stick people together for a number of years, then you have the chance to have the lucky event of a brilliant idea from someone that you can then use and develop"(NB: after the project) ID7_ORION_large_industry

Table 5 .

 5

1. TRL scale (EC, 2014)

used for the initial identification of the maturity of IOs.

Table 5 .

 5 3. Maturity level of identified IOs at the end of the projects (conventional TRL scale, see Table 5.1).

	HERCULES PEGASUS GEMINI PERSEUS ORION TOTAL

"

  We did two pilots. Every pilot was specific, every application had its specific features… This doesn't mean you can roll out if the next application is different or significantly different". ID49_PERSEUS_large ind "We were able to obtain the demo the 21 of December, the demonstration on the pilot plant."

	ID30_PERSEUS_acad
	(4) type of product (improved product versus completely new product):
	"We developed a new IT platform… That was more than I expected" (ID26_HERCULES_SME)"It was bringing
	in a new dimension…We have added elements … to our technology and this has gone very well for us."
	ID24_ORION_SME
	(5) type of deployment (temporary or permanent):
	"We implemented a new … system. During HERCULES we implemented that solution and thanks to this we
	were able to run … way more efficient." ID27_HERCULES_large ind
	"We deployed… in June July, and then some updates customizations were done afterwards."
	ID39_GEMINI_large ind

  "The technical development was only part of the equation…for us, at lower TRL 70-80 percent of the problem was technical. But when you go up, then you introduce some organizational and even some legal complexities.. technical stuff went down to 30-40% or even less at this point." ID52_GEMINI_SMEWe had more variety of resources. At the end, it could be, if you put the number of FTEs (NB: full time employees), I needed two FTEs, but you do that in 1 or 2 level (of maturity) higher, and it's still two FTEs. But instead of being two people, it's 20 people. What happened then is that the more people, the more meetings were needed … to agree and to align. So, it's not only that you compare two FTEs with two FTEs, there were more and more people involved, but every individual has only a small amount of the time that they put in the project.

	ID28_PERSEUS_large ind_TRL7
	"When I wrap up everything, 80 percent was industrial … and personal decisions, and only 20 percent were
	technical issues." ID15_PERSEUS_acad

  Table 5.4, and compared the scores with the estimated input in terms of person months per IO from three

				Minimum or
			Score	maximum
	From	To	increment	range as	Remark
			(average)	observed in the
				study
	TRL 3 Proof of concept	TRL 4 (Technology validated in lab or testing environment, e.g. software validated for representative data	1	1	Lowest initial stage for the scale in the studied projects.
	TRL 4	TRL 5.1 (component validated in synthetic environment)	2	2
	TRL 5.1	TRL 5.2 (system validated in synthetic environment)	1 per component	0.5 to 2 (for complex system, equal to the number of components integrated into the system)	5.2 is an optional level. This score is only added if the integration takes place on this level. It can also be accounted for on the next higher level.
	TRL 5.1	TRL 6.1 (Component prototype demonstrated in operational or near-operational environment.	3	2 to 4	This is counted for each component of a system.
				0.5 to 2 per	TRL 6.2 is needed for
	TRL 6.1	TRL 6.2 (System prototype demonstrated in operational or near-operational environment)	Additionally 1 per new component	component	progress to higher levels as the IOs on these levels are always systems. If integration is done on level
					5.2, there is no additional
					score here.
	TRL 6.2	TRL 7.1 (deployment of one pilot)	6	4 to 7
	TRL 7.1	TRL 7.2 (deployment of multiple pilots)	3	2 to 3	7.2 is an optional level
	TRL 7.1	TRL 8.1 (Roll-out of one application)	5	4 to 6
	TRL8.1	TRL8.2 (Roll-out of multiple applications)	3	2 to 4	Level 8.2 can also be
	TRL 7.1	TRL 8.2 (Roll-out of multiple applications)	8	4 to 6	reached going from 7.1 to 8.1 and then to 8.2
		TRL 8.3 (Prototype of improved
	TRL 7.1	product given for testing to	8	4 to 8
		customers)
	TRL 7.1	TRL 8.4 (Prototype of new product given for testing to customers)	12	8 for software
	Table 5.4. Incremental average scores for reaching different stages of maturity of IOs: from TRL3 to TRL8.

  6.1.3 Empirical study #3: Is bigger better? A deeper look on size and innovation performance of multi-actor R&D projects: Empirical evidence from European Research and Innovation Programmes

  *Consolidated and done in a systematic manner (not the raw data) ** Method, tool, algorithm, process, result of testing activities, other. Not yet published and not yet put into use, but identified in the documents and reported ***See Chapter 4 for explanation.

	TRL7.1.Deployment of one	As above, one installation												As above, one installation
	pilot
																					3	3
	2. Patents	New software		One		H2020	ICT		Ended		14		Workflow-integrated, 1 1
	3. Publications												converging
	ANDROME 3.1. Peer reviewed journal paper New		Several H2020 6 2 3	Environ 1	On-11	10	21	1	Waterfall	8	5	2		1	14	7	6	1
	DA 3.2. Book (edited volume) hardware,		1		1	ment		going
	3.3. Dataset*	methods,													1	1
	tools, software New software 3.5. Conference paper with peer 3.4. Book chapter LIBRA review (proceedings)	5 One 14	3 8	2 FP7 6	ICT		Ended 34	28	16	5	Workflow-integrated, converging 1 7 7				28	17	11
	GEMINI 3.6. Technical publication New software		One 1		H2020	ICT	1	On		13		Workflow-integrated, 1 1
	CAPRICOR 4. Other general advancement of New matetials, knowledge**	Several H2020 1 1	Producti	going On-1		12		sequential Decomposed 1	3	3
	NUS TOTAL RIOs, as identified through methods and tools documentation		35	14	18	on 2	1	going 53	40		9	3	1	31	26	2	1	2	63	31	25	4	3	0
	CYGNUS 5. Meta-knowledge***	New software		Several FP7 1	Energy 1	Ended 1		12		Grid	1						1			1
	and hardware New TOTAL, RIOs including meta-LEO knowledge		Several FP7 36 14 18	Producti 3 1	Ended 54	40	15	9	Waterfall 3 2	31	26	2	1	2	64	31	25	4	3	1
		technologies,					on
		products and
		processes

  TRL6.1 Component prototype demonstrated in operational or near-operational environmentThe prototype consists only of one new component. Example: Software that provides one novel function Validation means that the prototype meets the success criteria for moving on to an installation in the operational or near-operational environment and adding additional features. Synthetic environment can mean a test rig or a virtual environment, e.g. a computer simulation that represents (simulates) activities at a high level of realism, and enables extensive testing under realistic conditions.Validation of one component in the synthetic environment, e.g. extensive testing of a piece of software with realistic data.

			See above: demonstration of
			component prototype
	TRL5. Prototype validated in
	a synthetic environment
	TRL5.2 System prototype validated in a synthetic environment	Requires effort to integrate validated components into a system and to validate it.	Prototype can be operated in the synthetic environment and provides the specified functionality.
	TRL5.1 Component prototype		See above
	validated in a synthetic
	environment
	TRL4. Technology validated
	in lab or testing environment
	(computer)

It corresponds to

25,440 unique participants (organisations) in the collaborative RDI projects in Horizon 2020, from 01.01.201401. to 31.12.202001. (EC, 2021)).

Isolated work of partners which does not have important consequences on the project in case of their failure, does not count as a module.

Authors' calculation based on the data provided by the European Commission(2010, 2020).

This study has been peer reviewed and accepted for presentation at the 81st Annual Meeting of the Academy of Management (AOM2021), July 29 -August 4, 2021. It also was peer reviewed and accepted for presentation at the 21 st European Academy of Management (EURAM) Annual Conference, June 16-18, 2021. The order of authors in the paper EURAM is different, because it was only possible to submit one paper per first author/presenter.

This was found in the previous study and used here for comparison.

This study has been peer reviewed and accepted for presentation at the 21 st European Academy of Management (EURAM) Annual Conference, June 16-18, 2021. It also has been peer reviewed and accepted for presentation at the 81st Annual Meeting of the Academy of Management (AOM2021), July 29 -August 4, 2021.

In terms of FP6: IP : Integrated Project; NoE: Network of Excellence; STReP: Specific Targeted Research Projects. NoEs mostly intended to bring together existing research capabilities around a common topic, had important networking component and focused rather on the integration of research capabilities; while IPs and STREPs focused on generation of new knowledge and had defined scientific and technological objectives[START_REF] Breschi | Assessing the scientific and technological output of EU Framework Programmes: evidence from the FP6 projects in the ICT field[END_REF].

Such as e.g. an industrial-scale iron making process based on the use of hydrogen instead of coal.

To achieve ultra-high reliability requires a massive additional effort on higher TRL levels which is not considered here, as this was not part of studied publicly-funded projects.

https://www.oxfordreference.com

See Section 5.4 for explanation on how the input per IO was estimated from the available data, and the reason of choosing three projects

Appendix 4.3. Data structure: an overview. Positive implications of the large size.

Acknowledgments

I have been told that the thesis will be T.H.E.S.I.S -True Happiness Ended Since It Started. For me, it was the other way around: T.H.E.S.I.S -True Happiness Enlarged Since It Started, an immense pleasure to do research work. I did not see these three years passed, despite the extended days and nights. I also had remarkable opportunity to be introduced to knowledge recombination in practice: two open minded supervisors with very different backgrounds, from management science and from engineering science, harmonised in joint cross-disciplinary work. I owe a great

the project budgets, and the number of partners per project has increased too, while it is not clear whether this increase of project size is beneficial for the innovation performance of the projects. The field is difficult to access; it is not surprising that multi-case studies in this field so far to the knowledge of the authors have not been published.

Research design

The research design is a qualitative comparative inductive multiple case study [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF]) that took place in 2018-2020. Multiple case studies usually provide a stronger base to build theory [START_REF] Yin | Case Study Research and Applications: Design and Methods[END_REF], to enable comparison and demonstration of replication [START_REF] Eisenhardt | Better Stories and Better Constructs: The Case for Rigor and Comparative Logic[END_REF][START_REF] Bakker | Temporary organizing: Promises, processes, problems[END_REF], and create more robust theory [START_REF] Eisenhardt | Theory Building from Cases: Opportunities and Challenges[END_REF].

While quantitative studies are looking at a representative sample to make generalizations, the logic of multi-case qualitative studies is to identify patterns which could confirm the presence of similar phenomena across cases and to shed light on the mechanisms behind the phenomena, to generalize them further to theory [START_REF] Eisenhardt | Building Theories from Case Study Research[END_REF].

In the data collection phase, 4 projects were selected for investigation (Appendix 4.1), using 4 criteria: homogeneity, variety, comparability and availability of data. Names of celestial constellations were given to the projects for their anonymization: HERCULES, PEGASUS, PERSEUS and ORION. All four projects: (1) had a strong application dimension; (2) were ambidextrous;

(3) lasted 3 to 3.5 years; (4) had a large diversity of organizations from academia and industry; (5) included about half of the organizational partners with historical experience of collaboration; (6) used knowledge integration mechanisms of regular plenary meetings and small group meetings. The projects had between 10 and 17 organizational actors. HERCULES and ORION were particularly comparable in terms of knowledge integration mechanisms: they built on each other.

Then data about the research and innovation outputs (RIOs) available by the end of the project were collected, using the project documentation, cross-checked the information from EC database of patents and publications linked to the funded projects and contributing organizations (EC, 2021a), and then validated with the project coordinator or one of the key project partners of each project.

Also, data from 32 semi-structured interviews with 24 project participants (Table 4.2), resulting into over 33 hours of recording and 585 pages of transcripts (Table 4.3) was used. A collaboration, knowledge integration and overall the innovation performance of the project [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], and found that the positive effect materializes mostly outside the project, but the negative influence of the increase of the project size operates within the project and inhibits knowledge integration, which indirectly influences project outputs. There is indeed a better chance of unplanned collaboration and new knowledge generation in larger projects, but this is rarely realized within the project and is counterbalanced by the higher effort required for the collaboration and knowledge exchange and integration. On the other hand, the study also suggests that the loss of efficiency is moderate if the consortium splits into small groups. If the consortium gets too large, the benefit of the large knowledge base is mostly lost, as knowledge integration is difficult at the consortium level. This suggests that knowledge integration mechanisms shall be carefully designed and adapted to such projects, which comes at a cost: participants do not form a homogeneous group as they often do in smaller projects. The results also give evidence that with the increase of the size, reasonable technological distance between the groups [START_REF] Nooteboom | Optimal cognitive distance and absorptive capacity[END_REF]) is difficult to ensure.

Summary of theoretical contributions

Our qualitative research complements existing quantitative studies and provides the following three theoretical contributions to the contradictory innovation literature: (1) The relationship between project size and the number of project RIOs (or innovation performance), does not go beyond the proportional increase due to the larger number of partners (additive effect); (2)

There is a possibility of generation of unconventional output, meta-knowledge, in larger projects, if the project is organized accordingly; (3) The negative implications of a large size of the project materialize within the project, in short term. This can be, to some extent, counterbalanced by long-term positive effects outside the project. The findings add qualitative insights to the debate about the implications of the project size on the project performance [START_REF] Spanos | Scale and performance in publicly funded collaborative research and development[END_REF], demonstrating why there is no larger-than-additive relationship between the outputs of the project and the project size. Our findings put in question the beneficial role of the sheer size of the knowledge base [START_REF] Nahapiet | Social capital, intellectual capital, and the organizational advantage[END_REF][START_REF] Ahuja | Collaboration Networks, Structural Holes, and Innovation: A Longitudinal Study[END_REF][START_REF] Fleming | Collaborative Brokerage, Generative Creativity, and Creative Success[END_REF][START_REF] Yayavaram | Decomposability in Knowledge Structures and Its Impact on the Usefulness of Inventions and Knowledge-base Malleability[END_REF] in the R&D project context:

although benefits exist, their effect is mostly relevant outside the project. The conclusions of this work are summarised in the rest of this section. It is organised as following. Firstly the synthesis of the main results is presented, per each empirical study, and these results are integrated. The conclusion then continues with a summary of the theoretical contributions and managerial implications. Section on the limitations of the studies and on future research directions finally concludes the thesis.

6.1 Synthesis of the main results

6.1.1 Empirical study #1: Structuring inter-organisational R&D projects: Towards a better understanding of the project architecture as an interplay between activity coordination and knowledge integration

The first study (Chapter 2) analysed the architecture of the projects and its implications on the collaboration in the projects and on their innovation performance. It provides evidence about the interplay between activity coordination and knowledge integration, which defines the project architectures. The study identified three main types of architectures: decomposed, workflow-integrated, and nearly decomposable, with its three sub-types: weakly coupled, grid and waterfall. They have different implications on collaboration, project resilience and project management. The findings show that the workflow-integrated architecture disintegrates the knowledge base, provides a lower collaboration potential, and may require high management efforts, while a workflow-decomposed architecture, with different modules (sub-groups, or sub-projects), makes project management easier and can be used in projects of the large size, but it provides little added value that is gained from the inter-organisational setting. Nearly

Second, the findings help the funding agencies and RDI project management to better understand the potential implications of an increase in the project size, which was found to be not necessarily positive above a certain threshold. The size of the projects can be increased by adding more small groups of collaborating partners, also by adding partners with small, specific roles, but then the possible benefits of this variety of partners and knowledge bases not likely to materialise: the increase in the number of the partners creates at best proportional growth, at the expense of a larger coordination effort. However, the public agencies may wish to incentivise generation of meta-knowledge and its sharing outside the project. If larger projects are needed for specific reasons, such as for facilitation of project management by the funding agency, then the management structure of the projects and their knowledge integration mechanisms should be adapted to the large setting.

Third, the proposed framework to assess the 'innovation side' of the project performance enables project managers and the funding agencies to measure systematically and more precisely the progress of the innovative technologies in the projects. The funding agencies can check the project progress in a more objective way, helping to identify obstacles and ways to address them, and to compare to other projects, e.g. projects funded in the same call. The project management can set internal goals in terms of funding and advancement, using a refined maturity scale, analyse progress at certain points in the project, and perform corrective actions. Difficulties can be better identified and addressed, when they are identified at an early stages.

The findings also put forward several questions for policymakers. What shall be taken as evidence of success of multi-actor RDI projects? How to plan the required advancement of maturity, and which baseline to define? How to help to advance the innovation outputs as far as possible, within the projects and also outside the projects? How to account for the risk of abandoning collaborative developments at the higher stages of maturity, which must be taken The second large avenue concerns investigating the promising concepts that were put in evidence by the studies, in more detail. Future research could help to better understand the processes and mechanisms that influence knowledge integration in the context of multi-actor RDI projects, investigating in more detail e.g. the implications of the historic experience of the actors, the project context, the dynamics of the participating organisations [START_REF] Engwall | No project is an island: linking projects to history and context[END_REF], the coordination and cooperation processes as well as the governance mechanisms [START_REF] Johansson | Knowledge integration in interfirm R&D collaboration: How do firms manage problems of coordination and cooperation[END_REF]. It would also be beneficial to study in more detail the structuring processes, e.g. to better understand the role of the proposal phase and of the key project persons, as well as the structuring role of industrial use cases. The interviews in this research showed the effect of the alignment of the interests of the partners and their degree of flexibility on the evolution of the collaborative couplings and the resulting knowledge generation: this is also one of the promising directions for future research.

Future research could help to better understand why recombination of knowledge is difficult in the setting of such projects and how and why it happens outside the projects after having been triggered within the project. A variety of knowledge bases requires more attention (Messeni Petruzzelli andRotolo, 2018, Nepelski et al., 2019), specifically when it comes to knowledge acquisition and learning. The multi-actor projects considered here are unique, because of national and institutional diversity (e.g. different countries, small and large companies, universities and research centers), so future studies could investigate the tensions along the industry-university divide, different technological interdependencies among partners, the role of "token partners" etc. For instance: how do projects mitigate coordination risks associated with diversity, how do projects nurture long-term benefits related to diversity, and what is the role of the project leadership, not only the project management but also other key persons. These issues should be better understood in such 'centerless projects' that are implemented in the relative absence of control instances. The role of the contextual conditions on the efforts invested by the project actors, and the implication of the contextual conditions on the innovation processes and on the innovation performance should be better understood:

such contextual conditions may be related to technology, applications, collaboration itself, personal attributes, etc. The contextual dynamics, i.e. the evolution of the influence of the contextual conditions, may go in line with e.g. the changes of the level of risks related to the advancement of the maturation of the innovation technologies: at the level of pilot deployment, the interdependence of the partners significantly increased, but this depended on the type of technology, type of environment (virtual or physical) and on other characteristics.

Appendices

The Appendices 2 to 5 are those which were included in the studies (Appendices 2.1 to 2.2 for the study #2, Appendices 3.1 to 3.2 for the study #3 etc). Therefore, taken together, some of these appendices may be repetitive or similar.

Appendix Example: New feature in a large software system. Could also be a new model library as part of a larger software system that is planned to be marketed commercially.

Same as above for improved product 8

DEPLOYMENT DEPLOYMENT OF INNOVATIVE TECHNOLOGIES IN THE OPERATIONAL ENVIRONMENT.

In contrast to products, deployment refers to the internal use of innovative technologies by project partners by implementing them in their daily operations (operational environment).

Roll out (permanent deployment in operational environment)

Project partners implement the innovative technology from the project for long-term productive use within the organization. The installation must be robust, errors and need for maintenance occur only rarely.

In operation for several months within the duration of the project. TRL8.2 Roll-out leading to multiple applications Several installations of the innovative technology in permanent use.

See above 8

TRL8.1. Roll-out of one application

One installation of the innovative technology in permanent use, typically a continuation of a pilot. See above 8

TRL7. Pilot deployment, or integration into the operational environment

Pilot: Putting the innovative technology to work in the operational environment over a significant period of time to prove its potential and to provide information on further necessary improvements. Still frequent interactions with the developers. Pilots can also be done by technology providers (e.g. project partners) at their customers. Example: Novel measurement device installed temporarily at a production plant.

Pilot means an implementation for validation of the benefits by the end users, and requires integration in their operational environment under their supervision. Must be significantly more stable and user friendly than a prototype.

The technology is deployed temporarily and tested over a significant period of time. The purpose is to demonstrate the usability and the benefits to the end users and to generate feedback on necessary further improvements.

TRL7.2. Deployment of multiple pilots

As above, multiple installations.

As above, multiple installations. 7

Appendix 5.3. Innovation outputs in the projects under study: refined categories, and start/end maturity level.

Appendix 5.4. Illustration of scores: maturation process of IOs.

Supplementary materials

223

Supplementary materials.

Appendix 5A. Data structure overview: refinement of the maturity scale.

Appendix 5D. Climbing up the TRL ladder: illustrative quotes (first order themes).

First order themes

Illustrative quotes Increased interactions with external project actors (managers, business units, users…)

"We iterated what is the real setup of the problem that the industrial partners had, and we were tailoring our algorithms, our developments to that, also getting some feedback" ID20_PEGASUS_acad_TRL4 "We do it as a simulation study" ID5_ORION _acad_TRL5 Billions of euros are spent every year by public agencies to fund collaborative research and innovation projects with multiple actors from industry and academia, jointly generating new scientific knowledge and developing market upstream technological innovations. The economic and societal impacts of these projects are largely influenced by the quality and quantity of the project results. The joint activities are planned during the proposal preparation, when the main goal is to obtain funding; but the details usually are not fixed at this stage yet. Given the significant amount of public funding that is invested in this type of projects, there is a strong interest in a better understanding of their architecture, processes and performance. However, due to the difficulty of access to the field, there is a lack of studies providing deeper insights into these projects.

The thesis work holistically addresses multi-actor research and innovation projects. The main research question is: How do the project architecture, size, and processes in multi-actor RDI projects influence technological innovations and overall innovation performance of the projects? This main question is divided into four research questions, regarding (1) the role of the setting, i.e. the architecture of the projects and its implications on collaboration and on innovation performance; (2) the role of the processes, i.e. the evolution of collaboration and implications on the innovation outputs;

(3) the role of the project size, in terms of number of organisational partners, and its influence on innovation performance; (4) a direct and objective assessment of the innovation performance, beyond the use of 'classical' indicators as e.g. patents, for innovative technologies at different stages of development.

The thesis manuscript includes four empirical studies, addressing these research questions. The titles of the four studies are: (1) Structuring inter-organizational R&D projects: Towards a better understanding of the project architecture as an interplay between activity coordination and knowledge integration; (2) The evolution of collaboration in multi-actor projects: Patterns of couplings and collaborative innovations; (3) Is bigger better? A deeper look on size and innovation performance of multi-actor R&D projects: Empirical evidence from European Research and Innovation Programmes;

(4) Climbing up the ladder: Towards a better understanding of the innovation outputs and of the performance of market-upstream collaborative research and innovation projects.

The research design is an exploratory inductive multiple case study method, complemented in some cases by quantitative evidence. 4 to 6 projects, depending on the research question, have been selected and investigated. Data came from two main sources: over 1600 pages of project documentation and 54 semi-structured interviews with project actors.

The study provides a better understanding of the setting and the processes in these projects, as well as their implications on project performance. It also suggests how the innovation performance can be assessed objectively at the end of the projects, complementing existing methods. The study provides theoretical contributions which enrich both the project literature and the innovation literature, further connecting these streams, as well as managerial contributions, and policy implications.

Keywords : research and innovation project, multi-actor; market upstream, architecture, size, process, knowledge integration, innovation output, new technology, maturity, performance, EU Framework Programme for Research and Innovation