Guidelines on designing, implementing and evaluating open innovation activities

Guidelines on designing, implementing and evaluating open innovation activities

www.inos-project.eu facilitators open the exercise before the sponsor / company / jury present their case, which leads to the formation of students groups / teams, which leads to the formation of students groups / teams, which generate their first ideas for 45 min before the first break, and so on (reproduced from [START_REF] Suominen | Educational Hackathon : Innovation Contest for Innovation Pedagogy[END_REF]. ................. [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF] ... Table 4-2 OIAs strategic objectives, impacts and long-term indicators (adapted from [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF] ........

List of Figures

List of Tables

Executive Summary

The INOS project focuses on integrating open and citizen science in higher education institution (HEI) curricula with the overarching goal of making HEIs key open knowledge and open innovation agents in a changing world.

The project aims to do so through inspiring HEIs' roles as co-creators of innovation communities, generators of skilled human capital and actors of academic open science.

A large corpus of experience and knowledge already exists on open innovation methodologies. However, INOS aims at strengthening them by active learning approaches embedded in the learning design framework [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF], which will be used to design upcoming open innovation activities (OIAs). These OIAs will ideally bring together academic staff and students from different disciplines (with some external stakeholders) to engage them in the transformation of knowledge to innovative solutions. Together, and with external stakeholders, they coreflect, co-develop and apply their knowledge to address a technical challenge or social problem drawn from observation or from previous knowledge. They do so following an iterative methodology, which often involves tangible artefacts prototyped in collaborative spaces (fablab, hackerspace…).

The goal of this document is to help organizers of OIAs with the design and evaluation of their events. The document strives to balance between practical how-tos and general information, mostly borrowed from the literature or from specific examples at the University of Bordeaux. Over 2020-2021, partner HEIs within INOS plan to organize four short (1-2 day) and four long (4-6 month) OIAs bringing together 400 participants, including HE staff and at least 270 students from different disciplines. Short OIAs take the shape of so-called hackathons (which will be explained later), while long OIAs can be thought of as a series of workshops which progress towards the delivery of an innovative solution.

Section 2 deals with the general framing of the OIA, where organizers decide what activity they want to run and set their goals. Section 3 deals with the design of the four phases of an OIA, which follow the structure of problem-based learning [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF]. Section 4 covers the evaluation of OIAs, with a template evaluation survey provided in Annex 1, while Section 5 covers the challenges of engaging specific participants (external stakeholders, mentors, students…) and working across disciplines and across cultures. Section 6 deals with the use of specialized equipment in dedicated innovation spaces, e.g. fablabs. Section 7 deals with the dissemination of the OIA outputs and their impact.

1 Introduction

Open innovation in the context of INOS

According to the European Commission (2016 p. 11), "open Innovation is to open up the innovation process to all active players so that knowledge can circulate more freely and be transformed into products and services". [START_REF] Simeone | Knowledge translation mechanisms in open innovation: The role of design in R&D projects[END_REF] also suggest that "open innovation occurs where knowledge flows beyond the boundaries of a single organization and where a high degree of cross-border organizational collaborations take place: endusers, policy makers, industry and academic institutions work together to advance scientific knowledge or to develop new solutions and prototypes".

Innovation studies, and most particularly the seminal work of Henry [START_REF] Chesbrough | Open innovation: The new imperative for creating and profiting from technology[END_REF], generally consider open innovation (OI) from the point of view of an organization that wants to innovate. The French Standards Organization defines OI as a "process consisting of interactions with the economic, social, cultural and technological environment in order to identify and leverage external resources that are necessary to achieve innovation projects that the organization would not want, or could not perform alone" [START_REF] Afnor | Management de l'innovation -Guide de mise en oeuvre d'une démarche d'innovation ouverte[END_REF]. In their review of the literature, [START_REF] Dahlander | How open is innovation?[END_REF] distinguish between outbound innovation (sharing internal resources with the external environment) and inbound innovation (using external sources of innovation). Inbound innovation is illustrated by businesses that collaborate with students and HEIs to solve their problems.

The point of view of the business sector has been dominating in innovation studies. However, it leaves out several innovation behaviours:

• when no specific organization owns the initiative: the Independent Expert Group on Knowledge Transfer and Open Innovation set up by the DG Research and Innovation has admitted that "the concept of Open Innovation is constantly evolving and is moving from linear, bilateral transactions and collaborations towards dynamic, networked, multi-collaborative innovation ecosystems" (European Commission, 2016). Scholars have called this community-based innovation [START_REF] Shah | Open beyond software[END_REF], or innovation communities (Von Hippel, 2005 p. 93); • when the organization taking the initiative has other goals than innovation, e.g. teaching institutions that must offer students "interactive spaces and virtual infrastructure to promote the multidisciplinary participation of students and considering the participation of external actors from other academic institutions, companies, government and even from society" as described by the University of Monterrey in Mexico [START_REF] Miranda | Open Innovation Laboratories as Enabling Resources to Reach the Vision of Education 4.0[END_REF]. Innovation contests have long been part of higher education curricula, particularly as "an introductory course at university, introducing engineering concepts to freshman or sophomore students" [START_REF] Adamczyk | Innovation Contests: A Review, Classification and Outlook[END_REF]; • when open innovation stems from open science, and is merely a continuation in the field of products and services of a free flow of knowledge: indeed, as soon as a scientific matter produces knowledge and data under open access, it is natural for communities to draw innovative services from it, all the more so when this has a low cost (e.g. in the digital sector). This point also works with citizen science: "Scientists, whether they are in companies or universities, know that innovation needs diversity -the ability to think differently. In the twenty-first century, science and civil society are both faced with the need to address large global challenges, many of which have been identified through collaboration between scientists and NGOs. For too long Citizen Scientists have been seen as a fringe element of science. We should instead regard them as a model for how science can better tackle the problems that we all face." (Stilgoe, 2009 p. 62).

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu OI practiced by HEIs, which is at the heart of the INOS project, takes place under these three influences: as cocreators of innovation communities, as "generators of skilled human capital" (European Commission, 2016 p. 17), and as actors of open science [START_REF] Zourou | Academia permeating society through Citizen Science: use cases of engagement in Higher Education, INOS consortium[END_REF]. It is closer to the definition of OI in the strict sense proposed by Julien Pénin (2008), which emphasizes the importance of the openness of knowledge and technology.

A detour through the lessons of the COVID-19 pandemic feels more than appropriate. In this context of crisis, many initiatives have emerged to provide solutions for caregivers, patients, exposed workers, etc. without concern for market laws and corporate strategies. The essential thing was to call on collective intelligence to halt the crisis and get out of the technical or human deadlocks in which it has placed people. HEIs played a role.

Pondering on these initiatives, Henry Chesbrough (2020) has broadened his own paradigm to include initiatives from both, established economic players (companies, foundations and research organizations) and universities bringing together the efforts of their students and professors to develop a solution (in his example the MIT with a low-cost artificial respirator). He also highlighted user-centered innovation (citizens with a diverse set of skills who organize themselves to develop their model of mask or their recipe for hydroalcoholic gel, doctors who modify the machines at their disposal), of which he makes a lesson for the post-crisis period.

Based on these grounds, INOS' attention is turned to the pedagogical value of OI, for the participants in the activities, as well as in the Open Educational Resources (OER) they can produce.

Collaborative events and challenges

Most OI activities (OIAs) within HEIs fall into either a short (typically two-day) format or a long (over several weeks or months) format. Short events allow mixing up large groups of people who happen to have an interest or be concerned by the issue at hand, without intruding too much on their daily life. Longer formats are more suited to an extended collaboration with a small set of stakeholders, and often mean that students receive university credits for their work.

Collaborative innovation events (CIEs)

As we will see in the following pages, hackathons are the stars of innovation events. Other names may be used that also conjure the idea of intense collaboration, of accelerated time, of creative mindset and of playful approach: code fest, hack fest, idea jam, innovation jam, code jam, hack day… Some authors have proposed the umbrella term Collaborative Innovation Events (CIEs) for this category of activities.

Although compared with marathons (hence the name "hackathon"), these events also have elements of sprints: sprints are a well-known practice in iterative design where a series of successive sprints helps the participants prototype, test, and improve their idea. Therefore, this terminology can also be used.

CIEs, criticized as often as they are praised, are definitely a popular attraction [START_REF] Fabbri | Hackathons & Co : entre fascination et rejet[END_REF]. However, even if "everyone agrees on the value of CIEs in bringing together and bonding diverse actors to facilitate and accelerate the innovation process (...), few in-depth studies over a long period of time manage to show in which way(s) CIEs create value -and what kind of value? -and therefore how to capture it." The guidelines will thus be inspired by results drawn not only from the literature, but also from our own experience, as well as the testimonial of other organizers. CIEs should not be considered silver bullets: "these events are not magic solutions to fully and systematically meet the innovation imperative, nor do they definitively overcome the difficulties inherent in entrepreneurial and innovation activities" [START_REF] Fabbri | Hackathons & Co : entre fascination et rejet[END_REF].

In a recent review of the literature, Nada [START_REF] Endrissat | Hackathons: A field of dreams for 'collaborative innovation'? A review of recent studies[END_REF] notes that "hackathons become a vehicle to push forward enterprise culture and entrepreneurial citizenship". Their expected benefits may not always manifest themselves, but the participants may gain pleasure or professional benefits (e.g. reputation). For students from the undergraduate level, participation in CIEs allows them to apply their knowledge in solving practical problems and to benefit in terms of skills, including cross-curricular skills [START_REF] Kienzler | Learning through inquiry: a Global Health Hackathon[END_REF]. Their satisfaction rate is usually very positive, both in short two-day formats and in longer two-week formats [START_REF] Wang | An Extended Hackathon Model for Collaborative Education in Medical Innovation[END_REF].

How to use this document

In order to help with the implementation of OIAs within INOS and maximize their impact, the guidelines focus on the specific challenges that will be met by organizers. In order to determine which challenges were the most critical and needed to be covered, we surveyed members of the INOS consortium and we appealed to our own experience of organizing OIAs, including one activity which failed at the planning stage and didn't happen. As a general rule, we put ourselves in the shoes of OIAs organizers while writing these guidelines.

The guidelines are intended to be used together with the INOS learning design framework that specifically helps expand and improve the pedagogy of OIAs [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF]. The reason for this is because INOS proposes that the overall educational, scientific, innovative and social impact of OIAs would be optimised if the learning components were grounded in solid pedagogy. All concepts and approaches pertaining to the pedagogical dimension of OIAs, i.e. problem-based learning, are covered by the learning design framework.

Other deliverables will mark the progress of the work package: a compilation of use cases of OIAs run by INOS partners, an internal report on the implementation of OIAs and a short guide on fostering open innovation activities at HEIs.

Framing the activity

When an HEI wants to organize an OIA, they need to design its concept and how it will be run. This section helps organizers frame their activity.

Choosing the topic

Whereas they have traditionally dealt with software engineering and IT challenges, CIEs in particular and OIAs in general can be seen as "an innovative way of approaching a problem that has not been considered before, a reframing of an already existing solution, or a proposal within a non-technical sphere such as public policy" [START_REF] Paul | More open than open innovation? Rethinking the concept of openness in innovation studies[END_REF]. They emphasize that "innovation is about solving complex challenges and developing solutions to these challenges tackling them from three perspectives: business, technology and people" [START_REF] Charosky | Challenge based education: an approach to innovation through multidisciplinary teams of students using design thinking[END_REF]. Therefore, social challenges in general are well suited to OIAs, and organizers may want to focus on the 17 United Nations Sustainable Development Goals (United Nations, 2015).

CIEs often focus on narrow topics that create a bigger sense of ownership and more consistency between projects. The topic can be defined for the event as a whole or within specific tracks. For example, the MIT Clean Earth Hackathon on sustainability challenges had sponsors ask participants to solve their most-pressing sustainability problems, and awarded projects in four categories: natural resource management, environmentallyconscious design, mobility in the modern world, and refueling the next generation (Siderwicz, 2015).

OIA topics may cover, for example:

• lack of safe water and inadequate sanitation (Mahler [START_REF] Levine | Brains collide' during hackathon for climate change, disaster relief[END_REF]; water safety [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF],

• climate change and disaster relief (Mahler Levine, 2011),

• natural conservation [START_REF] Furth | Duke Blueprint Pioneers the Ideathon[END_REF],

• European labour mobility [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF],

• food safety in home delivery [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF],

• creating a literate world [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF].

Some national or international franchises allow an OIA organizer to reuse a specific brand name and communication tools, provided they abide with its rules and topic. Some examples are:

• Earth Hacks <https://www.earthhacks.io/> works with college students to host 48-hour environmental hackathons focused on creating innovative, equitable, and just solutions to the climate crisis; • Green Hackathon <http://www.greenhackathon.com/> is an international series of events to get together to create and implement new ideas, with a "hacker approach", for a more sustainable future • DigiEduHack <https://digieduhack.com/> is a series of online and offline idea-hackathons happening all around the world during two days, focusing on co-creating the future of education in the digital age.

Setting goals

CIEs take a problem-solving approach and their goal is always to "offer solutions" to a challenge. Some HEIs take this approach as part of a "product development" learning track [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF]. The focus is on the delivery of relevant solutions in the form of products or services. However, even the concept for a product or service is an outcome in itself.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

OIAs "can be viewed as an exploration of a problem space and solution space, i.e. within the scope of the challenge, identifying and evaluating alternative problems to be solved and different solutions to address the chosen problem" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF]. In this respect, it is not necessary to define a specific goal: what matters is that participants explore (by iterations) the solution space. It is important to note that these iterations, and the direct contact with users "improves the creative part (needfinding, ideation) but reduces the time for designing complex solutions and the associated learning outcomes" [START_REF] Charosky | Challenge based education: an approach to innovation through multidisciplinary teams of students using design thinking[END_REF]. Therefore, OIA outcomes should not be expected to attain the technical sophistication level of pure engineering education projects.

For the sake of inspiration, here is a list of deliverables borrowed from [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF] and listed in chronological order of their Challenge Based Innovation (CBI) program:

• concept,

• moodboard,

• rough Lego prototype,

• draft business plans,

• financial plan,

• hardware and/or software prototype.

The more advanced the solution, the better the learning, especially when it comes to learning prototyping techniques. However, prototyping must begin as early as possible "and continuous prototyping is considered necessary and beneficial throughout the entire process" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF]. Prototypes are "created to facilitate thinking and knowledge creation, to make concepts concrete, and to help the exploration of numerous possible solutions. They are low-cost representations of the idea: sketches, cardboard models, or rough digital mock-ups, that are created with the purpose of receiving early feedback from the users with minimum investment of resources" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF].

The communication of the solutions is also a deliverable in itself, be it a poster, a pitch, a video or a full-fledged presentation. Priority should be given to the storytelling and visual communication, rather than formal reports more suited to traditional learning designs.

Dealing with innovation artefacts

Even when they are only prototypes, the product and services delivered by OIA participants which materialize in a tangible form are innovation artefacts in their own right. What does it mean for OIA organizers?

Artefacts typically combine hardware (nanocomputer, robotic arm, locomotion unit, sensor, antenna, screen, tactile device…) and software. It should be noted that their respective importance is variable: in their CBI program, [START_REF] Charosky | Challenge based education: an approach to innovation through multidisciplinary teams of students using design thinking[END_REF] observe that the latest editions tended to have solutions based only on software (e.g. data analytics, machine learning, blockchain…) and that the students' preferences were shifting to these fields.

Another section of this document tackles how to work in innovation spaces (e.g. a fablab), which help in delivering hardware artefacts. Software artefacts, on the other hand, are typically setting-and infrastructure-independant. The present section deals with two challenges presented to OIA organizers: how to document the work being done and how to deal with intellectual property issues.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Documentation

Documenting an artefact is critical to help with reproducibility and dissemination. The artefact itself is only a materialization of the documentation: it proves that the idea is feasible, it helps to demonstrate the impact to the prospective users, but it will only have a lasting impact when the documentation exists to back it up.

Documentation is a process rather than a result: it means that the capture of documentation material must be organized in a systematic way as the OIA progresses. It can be continuous, with regular updates covering all steps, or discontinuous, with the benefit of hindsight when a key milestone is reached. It must at least cover the ins and outs of the artefact, and it can optionally cover the thought process leading to the final result.

A typical documentation is a written collection of notes, how-tos, checklists, frequently asked questions (FAQ), perhaps even blueprints and other visuals, which help anyone understand:

1. how the idea is supposed to work, through a plain language description of the general concept and the environment with which it interacts, 2. what is needed to make it work, through a description of technical prerequisites and tips that facilitate operations (things you wish you knew but only found out through trial and error, or after spending considerable amount of time), 3. how it could be adapted to different settings, therefore increasing its potential for reusability.

Written documentation is typically stored alongside the software in code repositories (also called forges, for instance GitHub or its open-source equivalent GitLab). When no forge is involved, the documentation can be compiled within a stand-alone document, or on a webpage. A series of open source tools that mix both worlds have gained momentum: they generate easy-to-use documentation webpages from textfiles, generally written with a specific syntax called Markdown. One can cite Docsify, GitBook, Docusaurus… among others.

Classical websites can be used to convey a complex message more clearly than technical documentation. For instance, a blog can be used for a "live report" on a team progress. However, classical content management systems (e.g. WordPress) are first and foremost publication tools, not always well suited to the challenges of documentation. However, some templates exist that turn WordPress into a full-fledged fablab documentation system, such as Opendoc <https://github.com/l-atelier-des-chercheurs/lopendoc>. Its authors, a group of designers, have also created do•doc, an open and modular tool that allows anyone to capture media (photos, videos, sounds and stop-motion), edit, layout and publish them. It can be connected to a physical device that operates a camera and a microphone, and enables to capture diverse traces from an ongoing activity for reconstruction and creation of narratives <https://github.com/l-atelier-des-chercheurs/dodoc>. The design principle of do•doc, which can be transposed to other tools, is shown below (Fig. 2.1).

Guidelines on designing, implementing and evaluating open innovation activities

www.inos-project.eu

Intellectual property

Ideas, processes, discoveries, concepts… as such cannot be protected and do not need to be treated with special care (except in the case of trade secrets, which are protected by law but would typically not be shared during an OIA). Other products of the human intellect are protected; the intellectual property principles addressed in this section are covered by the Paris Convention for the Protection of Industrial Property and hold for most countries.

What to do with potential names for a product or a service that would arise during an OIA? Should the name be disclosed, the risk is that someone else registers the trademark and prevents you from using the name. However, to do that they need not only to own the trademark but also to actively use it. If not in use (for a period that depends on national regulations), a trademark is lost. The same proviso goes for OIA participants who would register a trademark to preserve their future rights and would be required to actually use it. Therefore, the scenario where your name is exposed for lack of being registered when the OIA finishes is very unlikely.

Copyright protects original works of authorship as soon as they are created. Literary works (including computer software, documentation, databases, website text, blog posts, and e-books) and graphic works (including logos and website graphics) are most likely to be found in OIAs. Authorship goes to the very person who wrote the software or created the logo, not their fellow team members. They are free to do whatever they want with it, including transferring copyright ownership or placing it under a permissive license (e.g. a Creative Commons license for texts or graphic works, or an open-source license for software).

An idea alone is not patentable, but the exact instructions for arriving at the solution to a technical problem are.

(This excludes any organizational, financial… solution that is not considered as technical.) Therefore, OIA organizers should expect to deal with the intellectual property related to these technical solutions through participants' agreements (see below). This is important, but not enough: the technical solution must not have been described in a publication or available to the public before the patent application was filed [START_REF] Hammons | Intellectual property issues for startups participating in entrepreneurship support programs in Wisconsin[END_REF]. Therefore, should OIA participants consider filing a patent application, they are expected to communicate as vaguely as possible about the technical elements of their solution. This is possible because other aspects of the solution can be put to the front, e.g. the "why" and the "what" if not the "how". In the case of HEIs staff participating to OIAs as part of their employment, their invention belongs to their employer when it is related to their normal activity. It should be noted that if a participant is "moonlighting", i.e. they are participating in the OIA outside their normal employment, then the participant's employer might also hold rights to their work should it relate to the business of their employer, or the reasonably anticipated business of their employer [START_REF] Hammons | Intellectual property issues for startups participating in entrepreneurship support programs in Wisconsin[END_REF].

Should a participant team up with start-up employees, the start-up would not hold the right to any works created by the participant [START_REF] Hammons | Intellectual property issues for startups participating in entrepreneurship support programs in Wisconsin[END_REF]. Because the transactional cost of entering into a written agreement is often higher than committing to a hackathon, and because the likelihood of a copyrightable work being produced that a start-up would use is low, it is reasonable to wait until after the OIA to purchase the right to any copyrightable work produced by or with outsiders [START_REF] Hammons | Intellectual property issues for startups participating in entrepreneurship support programs in Wisconsin[END_REF]. However, had the work been placed under a permissive license, it could still be used (but not exclusively) by the start-up.

Please be aware that "ignoring intellectual property altogether may be justified in some situations, but in the usual course it is a risky approach" as it "increases the likelihood of future precarious situations" (O'Leary, 2019). Therefore, two scenarios can be considered where either organizers or participants retain all rights; unfortunately, "ownership interest for an individual creator might result in little incentive to push for something great, but outright ownership might result in little means by which property can be developed or scaled to be of real value to others" (O'Leary, 2019). That's why two other scenarios are often considered: permissive license, and option/right of first refusal.

The Table 1 below aims to assist the decision-making process of OIA organizers by giving insights on each situation; a fully-fledged comparison of the scenarios and example contract language can be found in O'Leary (2019) and its companion repository of participant agreements <https://suffolklitlab.org/research/hackhelp/>: HEIs will generally not seek to bring solutions to the market and might favour the two scenarios where participants retain all rights (and where standard rules for the invention of their staff applies) or the work is under permissive license. As a rule of thumb, the latter "can be a terrific option in many hackathon environments. It tees up the concept of intellectual property for participants, sets expectations, facilitates continued collaboration and improvement to foster (one hopes) better and more useful tools for delivery of legal services, and can be simple enough not to steamroll hackers' innovative spirit and motivation to contribute" (O'Leary, 2019).

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Designing the tasks

According to INOS learning design framework [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF], an OIA is a Problem-Based Learning activity, and therefore should include four learning tasks:

• ideation, • design, • implementation, • communication.
The four tasks are covered in the following sections.

If delivered online, all tasks meet the same challenges where, essentially, participants do not align well because "in remote or virtual environments, open communication is more difficult to facilitate, with cross-talking or an unwillingness to speak up in a remote meeting often cropping up" (Smart, 2020a). Challenges also crop up because of the lack of shared visual space: being able to walk around, visualize the progress of each team on whiteboards and flipcharts, and move between groups and subgroups is a massive bonus to the live meeting setting. Engagement is also trickier when people are at home and can easily be distracted by what's going on in their other screens or in their physical environment (Smart, 2020a). Zoom, which supports breakout sessions, is a preferred tool, but must be combined with other tools such as online whiteboards (e.g. Miro, Mural or GroupMap) and engagement tools (e.g. Kahoot or Mentimeter) during sessions, as well as group chat (e.g. Slack) and document sharing for asynchronous communication between video calls [START_REF] Smart | Survey report: online facilitation and virtual meetings[END_REF]. In addition, it could be interesting to investigate innovative tools that facilitate co-presence of participants and recreate a feeling of shared physical space (e.g. Sococo, VirBELA). Finally, CIEs have specific needs compared to other online events, and solutions like Agorize or Devpost cater for these needs, providing for instance:

• a matchmaking algorithm to form multidisciplinary and complementary teams,

• a public space for holding discussions and sharing information between participants and organizers,

• online mentoring sessions,

• an online voting interface for the organizers,

• a dashboard for challenge statistics.

Ideation phase

Ideation is the "process of generating, developing, and testing ideas" [START_REF] Brown | Design thinking for social innovation[END_REF]. Perfectly associated with co-creation, ideation takes shape thanks to a set of exercises or methods that boost creativity stimulate the imagination and collective invention. As such, it is the starting point of an OIA and the first steps of the participants into the "solution space".

The objective of the ideation phase, according to INOS learning design framework [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF], is threefold:

• Topic exploration, need finding

Topic exploration, need finding

In the Double Diamond design model, this stage is also called Discover (Fig. 3.1). It "consists of learning more about the different variables that affect the problem and its possible solution", "to identify and contextualize the actual problem or opportunity", often by "focusing on users' needs, wants and behavior" [START_REF] Costa | The Double Diamond model: what is it and should you use it?[END_REF]. This may comprise field research to get true first-hand experience of users issues (including interviews, ethnographic and observational studies), as well as desk research (including market research) -both commensurate to the time available.

Defining a problem

The second stage of the Double Diamond model (Fig. 3.1) is meant to filter out the noise and focus on relevant data, anything that helps "identifying bottlenecks or resource waste, seeing hidden opportunities or setting a list of things the design team definitely shouldn't do (called no-gos)" [START_REF] Costa | The Double Diamond model: what is it and should you use it?[END_REF]. It is also meant to reduce the range of problems identified, for instance by surveying users and asking them to prioritize their problems.

Eventually, the context of the project should appear clearly, as well as what the team would like to achieve. This can be conveyed in one or two sentences, for instance: in order to improve safe access to water in rural areas of Ghana, the team would like to tackle the fact that more than 70% of wells are out of function, while avoiding pitfalls in the way the money provided by NGOs is used that are so detrimental to humanitarian support [START_REF] Charosky | Challenge based education: an approach to innovation through multidisciplinary teams of students using design thinking[END_REF]. This is a perfect stepping stone to the next stage in the Double Diamond model (Fig. 3.1): Develop potential solutions, starting with a process of brainstorming, also called ideation.

Brainstorming solution ideas

Contrary to common belief, brainstorming does not have to be an unstructured process and it comprises a wide range of exercises that can be adapted to the progress of the project, the challenges encountered as well as the composition of the team and the personality of each learner. Whereas an unstructured process could block people who are less comfortable speaking out their ideas, facilitated ideation promotes collaborative work in a friendly and educational atmosphere. Indeed, the most important factor to be creative on demand is a relaxed and playful atmosphere [START_REF] Mumford | Creativity syndrome: Integration, application, and innovation[END_REF], especially in the context of active learning.

Active learning is based on the connection of different modes of self-expression or artistic skills, such as drawing.

The modern conceptualization of creativity places the emphasis on a certain kind of thought process, leading to phrases such as "thinking differently" or "thinking outside the box". One way to break out of the box of conventional thinking is to use a range of facilitation methods. The website SessionLab offers free access to a library of facilitation methods, with an easy-to-use search engine and filter: https://www.sessionlab.com/library/idea_generation.

It is worth mentioning that ideation-focused games ("gamestorming", a category used by SessionLab) can provide rich interactions between the participants in a dynamic atmosphere, where the rules of a game ensure that participants progress in an orderly manner and have their fair share of chances to make moves or decisions, until they reach success.

Several ideation sessions could be organized until the ideas are ripe to be considered for the design phase (Fig. 3.2). In the case where (too) many ideas are generated, these iterations can also be used to cluster and organize the ideas around common themes (and removing duplicates), before selecting the best ideas to be taken further based on a set of criteria (for example: relevance, feasibility and novelty) with the use of voting methods (for instance "dotmocracy" where each participant has a fixed number of dots to assign to the ideas in the shortlist and the ideas with the most dots are selected).

Guidelines on designing, implementing and evaluating open innovation activities

www.inos-project.eu

Design phase

The objective of the design phase, according to INOS learning design framework, is twofold:

• Design model(s) • Prototype design(s)
Said models and prototypes are meant to check that the product or service is desirable for the user, viable as a product in the market, and feasible in terms of technology [START_REF] Brown | Design thinking for social innovation[END_REF]. In the Double Diamond model, this still fits in the stage of Develop potential solutions (Fig. 3.1).

Design model(s)

This task is at the heart of problem-solving and design, because it is where the "magic happens" and the so-called "value proposition" emerges. In the example of Ghanean water wells, the task would see students come up with the idea of "a low-cost sensor arrangement that was attached to the well outlet pipe (no need of modifying it), detected if the well was operated through vibration pattern detection, if water was flowing through temperature change detection, and sent an SMS message to a cloud-server that displayed the well status in a synoptic map in the nearby villages and activated a Uber-like network of potential repairers, who would be paid through a NGO when the correct well status was automatically checked" [START_REF] Charosky | Challenge based education: an approach to innovation through multidisciplinary teams of students using design thinking[END_REF].

The same facilitation methods listed by SessionLab can be used to generate "divergent" designs and sketch as many solutions as possible. Solutions can be merged, pitched and then voted (dotmocracy) to select the best design model that will be prototyped.

Prototype design(s)

Prototypes must not be too complex at first, because "the less is invested, the easier it is to modify the direction of the project if the received feedback so requires" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF]. Hence the idea of rapid prototyping: cardboard or Lego models, rough digital mock-ups, or even 3D models obtained with fablab equipment.

"Prototypes are, in fact, primarily seen as a tool for stimulating thinking and exploring ideas, not as representations of the products. They are created to facilitate thinking and knowledge creation, to make concepts concrete, and to help the exploration of numerous possible solutions" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF].

It is hard to imagine a 100% online equivalent, unless prototyping is focused on software; otherwise expectations for this phase should be reduced and prototypes will be limited by the resources that the participants can use in their close environment.

The prototype may also be accompanied by a business model [START_REF] Charosky | Challenge based education: an approach to innovation through multidisciplinary teams of students using design thinking[END_REF].

Implementation phase

The objective of the implementation phase, according to INOS learning design framework [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF], is twofold:

• User testing • Reiterative design
Remember that prototypes are "low-cost representations of the idea (…) that are created with the purpose of receiving early feedback from the users with minimum investment of resources" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF]. This requires usability testing, i.e. observing a user completing a task with the prototype and asking probing questions such as "what are you thinking currently" or "why did you decide to do that?".

The design phase requires continuous testing and feedback, using an iterative loop that can be described as a PDCA cycle (Fig. 3.3). "Reflection on the information gained from testing gives direction for the next direction, i.e. how the idea and the prototype need to be modified" [START_REF] Buck | Accelerating university-industry interaction using the design-sprint for experiencebased learning[END_REF]. Buck and Löffl (2020) provide an example of planning for the Design and Implementation phases: half a day for value proposition design, half a day for rapid prototyping, half a day for testing the hypothesis, half a day for refining the prototype (Fig. 3.4). In their own words: "The teams concretize and revise their solution approaches and transfer them on Wednesday afternoon into paper-based prototypes which they then test with the first scripted customers. These first impressions are used to revise the prototypes. The implicitly processed hypotheses of each prototype are formulated concretely on Thursday and tested in a structured way (using pre-defined test protocols) with real customers. The valuable impressions from real-life testing are invested in the revision of the prototype".

We are now in the Deliver stage of the Double Diamond model, and the final outcome could be as close as possible to a real-world solution (Fig. 3.1).

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Communication phase

The objective of the communication phase, according to INOS learning design framework, is twofold:

• Presentation and discussion of final outputs • Dissemination of final output for real-world application This is consistent with Fig. 3.4 that plans for the so-called final "pitch".

Presentation and discussion of final outputs

It is important that participants do their team proud when presenting their work, and give justice to the work they put in and the journey they travelled together. A critical skill in OIAs, as in any HE curriculum, is the ability to present one's work and convince an audience of its relevance. OIAs typically plan for pitches, often short (a few minutes), where participants deliver a concise presentation of their work and output.

It is recommended to provide a pitch training (Fig. 3.2) and rehearsal to maximize the chances of learning, and successfully delivering a convincing presentation. The pitch happens before a jury, typically comprising the sponsors of the OIA, who can get a sense of the solutions developed as well as select which solution they would like to implement. The motivation is greater for students to present before professionals, especially C-level company representatives [START_REF] Buck | Accelerating university-industry interaction using the design-sprint for experiencebased learning[END_REF].

Pitches often mark the end of the OIA (except when followed by a jury decision and an award ceremony), therefore they are like a climax and the atmosphere can be very festive.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Dissemination of final output for real-world application

In the experience of the DesignShop, a key feature is a monetary prize that makes sure that projects are "ready with extra push for real life/world, sets participants on pipeline of scope of project" and "makes project execution a possibility for everyone, regardless of their monetary status" [START_REF] Artiles | The Education DesignShop: Broadening non-designers' solutions for big issues[END_REF].

The contribution of sponsors can also help bring the projects to the next level. It should be noted that "the motivation of the students is also increased by clearly communicating during the IW, at the latest, when the final event is held, what happens to the proposed solutions that have been developed during the week" [START_REF] Buck | Accelerating university-industry interaction using the design-sprint for experiencebased learning[END_REF].

Even when a real-world application of the project is not being considered, it is important to document it (see section "Dealing with innovation artefacts"). Social media can be used to share this documentation and bring attention to the deliverables, especially if specific hashtags are used to target specific communities.

.

Evaluating the activity

Evaluation is determining. Based on a strong methodology, this stage is of prime importance to the improvement of project management, and to a large extent to policymakers and their understanding of Citizen Science [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF]. However, when time is running out, evaluation is one of the phases of a project which can be sacrificed in favor of the completion of the activity. Indeed, evaluating is a complex task which comprises several steps: 1) defining objectives and goals, 2) planning, 3) conducting and analysing, and finally 4) implementing the results to produce some long-term impact [START_REF] Pettibone | Citizen science for all -A guide for citizen science practitioners[END_REF]. This task still requires a lot of different tools, resources, and a diversity of methods to achieve a successful evaluation.

In the diversity of citizen science projects, the specifics of OIAs enforces the basics of a correct method by bringing to light some meaningful challenges such as optimizing time or monitoring community-level outcomes.

Finding the right evaluation method

In the INOS context, defining goals, especially within the learning design framework [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF], is considered as the first essential step to design, implement and assess a CS activity. Every study agrees on the necessity to codesign and co-define early sustainable outputs. For example, the LDF brings up a learning evaluation design that oversees the activity during the planning phase for problem-based learning projects. Then, when the exploitation stage comes, which can be very short and intense in the example of hackathons, the method selected -clarifying the evaluation team, duration, depth, available resources, methodology, schedule and procedure [START_REF] Pettibone | Citizen science for all -A guide for citizen science practitioners[END_REF]) -needs to consider several aspects to achieve the goals.

Finding learning evaluation methods for short-lived activities

Evaluation has a role in adaptive project management [START_REF] Wright | Evaluating a Citizen Science Research Programme: Understanding the People Who Make It Possible[END_REF]. Many tasks fall to the organizers who have a key role and a lot of charges to deal with: hands-on organization, stakeholder communication, content design and facilitation. Some data is easy to gather with classic evaluation tools (statistical data, data analytics e.g. online activity data, assessment e.g. quiz), but much data is complex to catch because coming from live feedback. If referring to the LDF, OIAs' evaluation establishes goals for continuous improvement (enhancing education value and enhancing openness) with more subtle criteria such as relevancy of the topic or efficiency of the communication between the different participants. Part of the solution may be finding the right time to collect data while not wasting time on organizational tasks.

During short-lived activities, when all the participants are in the same place, special time slots can be devoted to sharing experiences and lessons learnt, thus supporting the social learning process [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF]. For instance, during an innovation workshop (called DesignShop) reported by Jessica A. [START_REF] Artiles | The Education DesignShop: Broadening non-designers' solutions for big issues[END_REF], a 20-minute evaluation of each participant's problem-solving ability was conducted at three different points: Pre-event (before the event began), Post-event (in the immediate moment the event is over while the judges deliberate the winners), and more classically Post-Post-event (in the 2-3 months' timeframe after the event). Survey, focus groups, all tools can be used.

Simplicity is a main success factor (see criteria above). Asking three understandable questions is an option chosen for the DesignShop as well. An open framework is a good way to be reactive during the event, and to be fit to adjust any questions according to the inherent unpredictability of these kinds of events.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

The artefact team based evaluation is challenging

During the OIAs, time is always dedicated to the evaluation of the artefact. Whether in the context of long-term activities or short-term activities, the moment of evaluation comes as the culmination of an intense design process. Thus, it must be well presented and sufficiently transparent for all participants.

In CIEs for instance, a jury usually composed of 3 to 5 people typically judges along four transparent criteria: relevancy, novelty, actionability and impact. Each activity takes place in a specific, complex innovation ecosystem, and each artefact must be evaluated by the right jury to obtain all the support it can get and optimize sustainability and impact -often a more important consideration than the prize. It is important to optimize artefacts' chances to have a life after the event.

According to their implication on the process, especially in the design process, mentors, partners and other ecosystem players can also contribute and help to plan ahead the expected value brought about by the use of the artefacts. This process which can lead to a common and enriched reflection on the indicators (see table of indicators below -"scientific impact").

Finding the right performance indicators

"Despite these contributions to evaluation, citizen science projects currently lack comprehensive evaluation frameworks that would allow for comparability across projects and programmes" [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF].

Comparability is actually one of the key values to consider when building the criteria with relevancy and to enhance the performativity of the evaluation. How to consider indicators which can be on the one hand easy to collect and on the other hand easy to analyse in light of strategic objectives ? How to deal with the different dimensions of OI activities which includes a lot of different overall goals such as innovation, openness and education? In summary, how to create an original framework which can be general enough to embrace all the CS objectives, especially the learning goals, and specific enough to fix a given trajectory which takes account of the internal characteristics? And beyond, which allows comparing data in order to keep a scope for improvement not only for organizers but for policymakers?

Defining the process indicators (feasibility) and the impact indicators

In their article "Toward an open framework", [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF] propose an evaluation grid based on two aspects:

• Outcome-based evaluation, which assesses the overall goals of activities and the benefits to participants and recipients of the results and; • Process-based evaluation, which identifies the operational strengths and weaknesses of activities or programmes.

In other words, the performance indicators measure the outcomes, the impact of the activity over the long-term and the inputs, linked to the feasibility and the process of the activity, over the short-term. The second aspect is at least as important as the first one: each organization has some examples of abandoned projects because of lack of participants or lack of funding.

Inspired by collaborative workshops and feedback, the article highlights three dimensions on CS initiatives which can detail the operational objectives, and help to find the appropriate indicators: 1) the scientific (specifically linked to innovation for OIA), 2) the participant dimension (and notably the learning process) and 3) the socioecological and economic dimension

The principle is that each organization can "tailor" its indicators based on these targets. This approach is interesting because it's taking into account all the stakeholders, and allows the organizer to 1/ prepare the evaluation session and 2/ to save time which is as said before an issue itself (cf. sub-section "finding learning evaluation method for short-lived activities).

Some indicators deal with the question "what have we done?". These indicators (italic) respond to operational objectives laid out at the outset (bold text): "what do we want to do?" The table below retakes some points and inputs of the table Evaluation criteria and supporting questions which can be found in the article and proposes to extend them to the OIAs with some illustrative indicators.

Please note that the table can be modified or enriched according to the nature of the activity.

Create the conditions of the sustainability of collaborations and results

Diversity of the results, number of workable results (according to the objectives)...

Create the conditions of partnerships Number and alignment of the partners, representation of the ecosystem during the event (networking)…

Facilitate the appropriate dissemination of the results

Number of publications in partners network, number of publications in specialized network

It should be noted here that each activity can find or adapt its own management and monitoring tools to fill these indicators.

But, more interestingly, the measure of the impact deals with the question "how have we changed the world?". These indicators (italic) respond to strategic objectives laid out at the outset (bold text).The table below retakes some points and inputs of the table Evaluation criteria and supporting questions which can be found in the article "Citizen evaluation framework" [START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF] Educational Value of Open Science, Citizen Science and Open Innovation Activities" [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF] and finally with an analysis of the "far reaching outcomes".

Please note that the third column will be developed in the next subsection and that the table can be modified or enriched according to the nature of the activity. Linked to sustainable goals, the indicators listed above are more difficult to measure either because these indicators have to be built up or because the data must be built in the long term. Indeed, the outcomes are almost never immediate: to complement the information collected the day of the activity (surveys, livefeedbacks...), the data relating to the impact indicators must be gathered immediately after (data analytics) or long after the activity (surveys, real-time internet monitoring).

Moreover, as said previously, this data has to be compared to become meaningful: this is why it is important to prepare this stage by documenting it (external references, reports on similar activities) and by setting up sometimes collectively strategic goals. For instance, when a local community invests in a hackathon on soft mobility, the problematic "how my activity can promote and develop soft mobility on the territory?" is to be considered.

However difficult to reach, these outcomes can be assessed by facilitating the collect of the data, notably by building one or several communities open to sharing real-time information.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Focus on community's indicators and "far reaching" outcomes

More and more projects are concerned by ecological issues and the reason why participants get involved in these kinds of initiatives is that they want to act collectively and concretely in transformative projects. In the matrix detailed above, the social impact, which corresponds to the capacity of the project to increase civic resilience, although meaningful, seems very difficult to implement. This notion of social impact can be understood by community level outcomes, which are beyond the individual learning outcomes and the programmatic outcomes and so considered as "far-reaching" [START_REF] Jordan | Key issues and new approaches for evaluating citizen-science learning outcomes[END_REF]. As impact, the corresponding indicators have their place in the table below: they can be collected together with the previously mentioned dimensions (innovation and participation) but not only. The collection of the data requires long-term monitoring or an analysis work which can be facilitated thanks to the help of the living community, provided that all parts have the same level of understanding.

One of the lines of thought to better understand these outcomes would be the development of precise indicators to measure the degree of collaboration and the impact of open-innovation on a community and eventually on the society. The upheld idea that tools used by social scientists to assess impacts could be adapted and used to evaluate impacts when it comes to communities is for any reason very relevant, and needs to be pushed forward [START_REF] Jordan | Key issues and new approaches for evaluating citizen-science learning outcomes[END_REF].

In social policies, a community group is defined as "any group of two or more individuals or agencies working together to meet a common need and achieve a common goal. This can range from networking type associations, to community-driven educational programming, to any level of existing groups such as boards, committees, coalitions, and collaborative groups" <https://cals.arizona.edu/sfcs/cyfernet/nowg/comm_index.html>.

Discussion about "university community" is common, but open innovation projects include external partners like companies, associations and sometimes foundations or even public actors. All these participants are expecting some sustainable outcomes, which can lead to some interesting indicators [START_REF] Jordan | Key issues and new approaches for evaluating citizen-science learning outcomes[END_REF] linked to the open-innovation process and to the creation of value for communities, and beyond for the society itself.

The table borrows some outcomes presented in the article "Key issues and new approaches for evaluating citizenscience learning outcomes" [START_REF] Jordan | Key issues and new approaches for evaluating citizen-science learning outcomes[END_REF] and proposes to illustrate these outcomes with examples of indicators which basically answer the question: what is new and what is valuable for society?

Table 4-3 Focus on socioecological and economic dimension and the far-reaching outcomes (adapted from [START_REF] Jordan | Key issues and new approaches for evaluating citizen-science learning outcomes[END_REF] Far reaching outcomes Impact indicators

Increase social capital [START_REF] Adger | Social capital, collective action, and adaptation to climate change[END_REF] and create value New practices, new resources, creation of new organizations or new jobs (companies, associations…), degree of trust in social organizations.

Improve community capacity [START_REF] Donoghue | Social science constructs in ecosystem assessments: revisiting community capacity and community resiliency[END_REF] and strengthen an innovation ecosystem

Diversity of the participants and evolution of the community from an activity to another (increase in the size of women for example), empowerment as an organization / as a citizen and collective intelligence, initiation of new collective projects, sustainability of collective actions and project groups…

Encourage trust between scientist, manager and the public [START_REF] Fernandez-Gimenez | Adaptive management and social learning in collaborative and community-based monitoring: a study of five community-based forestry organizations in the western USA[END_REF] Measure of the openness of results: co-publications, joint projects, public's awareness of the projects' results, media coverage of the results and guidelines…

As observed in the table above, this level of evaluation depends on the sustainability of the projects themselves and the capacity of the organizers to conduct a long-term assessment with all the stakeholders. And the common will to share some assessments, and maybe data from an institution to another, which is basically the INOS community approach. Some indicators could appear to be roundabout. But they are not because the specificity of this framework is that it produces impact indicators creating a dynamic analysis. The other benefit is that it encourages improvement and collaborations between different communities in the same institution, on the same territory or even between universities by creating some references and good practices and by opening up new perspectives for funding.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Engaging participants

In order to work, open innovation must favour the combination of brains and the mixing of different skill sets, and/or even world views. As pedagogical activities, it seems only natural that OIAs bring together students and teaching staff; as civic activities, they also need to consider non-academic stakeholders. Mentors also play an important role in coaching teams and bringing extracurricular value to the process.

This section helps OIA organizers with the challenge of ensuring the participation of these specific communities, and their diversity.

Engaging non-academic stakeholders

Non-academic stakeholders participating in an OIA are typically individuals or organizations that are concerned with the topic at hand and can contribute specific expertise as technical experts or as prospective users or beneficiaries of the innovation outputs. In the latter case, they are involved in the co-design of a solution that addresses the needs of their community.

Non-academic stakeholders can be engaged on an organizational level, when a company, a public agency, a national or local association is interested in an OIA that meets its values and falls within its remit. They can become "sponsors" of the OIA through financial, in-kind and/or human contribution:

• financial contribution: the sponsor covers the expenses of the OIA,

• in-kind contribution: the sponsor gives access to their equipment, venue, etc.,

• human contribution: the sponsor participates (with one or several employees) in the OIA.

A sponsor typically also defines a specific challenge that will be addressed by the participants (see section "Choosing the topic"). However, there doesn't seem to be much literature on how the sponsors follow-up on the outputs: "It would also be interesting to understand how to proceed with the new ideas generated within student innovation contests and what role the inclusion of sponsors into the innovation contest could play" [START_REF] Adamczyk | Innovation Contests: A Review, Classification and Outlook[END_REF].

Incentives

The key to incentivize stakeholders on an organizational level is to reward them with promotion: "you will certainly thank your sponsors, by name, during your opening and closing session, and you will probably want to tweet your thanks too. Beyond that, do you want to give them a time at a podium to speak to your attendees? Or a table in the back to show off their stuff? It's up to you, and you have to strike the right balance between bringing in enough sponsorships [while] not interfering with the goals of your event" [START_REF] Tauberer | How to run a successful Hackathon[END_REF]. However, sometimes the gain in reputation is too low to incentivize and the motivation must be found in a shared, specific connection: an alumni employee, a past successful collaboration, a possibility to use the artefacts as basis for further research and development, a possibility to meet young talent that could be further recruited to work for the sponsor, potential tax deductions, alignment with their existing corporate social responsibility strategies, etc.

On an individual level, non-academic stakeholders must be interested in getting involved with a typically lengthy and risky activity that will put them out of their comfort zone and may not produce actionable results. Here, the key is to build a trust-based relationship that will convince them that participating in the OIA will advance their cause and provide positive externalities: experience intensive teamwork, train their creativity, spot new talents for hire, communicate on what they do, network with stakeholders, and so on. If possible, OIA organizers must profile these individuals based on their technical expertise or their representativeness of the target community.

Engaging teaching staff

In his critical appraisal of hackathons at HEIs, Anthony L. [START_REF] Clary | University Hackathons: Managerialism, Gamification, and the Foreclosure of Creativity[END_REF] states that "university staff and administrators permit and pursue hackathon events because of their utility in preparing students for real world practices and professions". The pedagogical value of OIAs as their perceived benefit holds the key to the involvement of teachers, as OIAs must contribute directly to the students' curriculum in order to be accepted. This comes as no surprise since one of the objectives of INOS is to mainstream open and citizen science (including open innovation) into the HE teaching practice.

Therefore, some outputs from INOS will directly address teachers to help them embrace OIAs. The learning design framework [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF] is already a useful resource to be used by teaching and/or supporting staff to build and demonstrate the pedagogical value of OIAs.

However, the burden of the practical organization of the OIA, which can be quite heavy, must not necessarily lay on teachers. Preparation of hackathons is incredibly time-consuming and requires excellent organizational skills and time. Walter F. [START_REF] Uys | Hackathons as a formal teaching approach in information systems capstone courses[END_REF] has found it valuable to use the services of a professional company to organize the hackathon, in addition to "senior management support", "as academics are not prepared for the kinds of activities that are required".

Engaging students

As it has already been said, OIAs have proved to be inspirational and motivational for most students, who rate them positively: they can apply their skills to real-world problems, they can compete against other teams, their creativity is fostered. The Lappeenranta University of Technology (LUT) in Finland reports that they "work hard to be able to build events that [intrigue] the mind of young [students], something that offers new opportunities in life, something that challenges the skills and maybe even shows new areas of expertise for them. [They pursue] to help participants to learn new things and even maybe find something about their personalities too. [At best] they are able to discover skills, they already had inside of them, but which were still hidden and not in use. By giving the students this sort of challenging setting, which pushes them forward with some positive and safe environment and time stress, it seems that they are able to take a step in [the] next level of knowledge and skills" [START_REF] Happonen | Ideas and experiences from university -industry collaboration: Hackathons, Code Camps and citizen participation[END_REF].

Therefore, students are expected to be easily persuaded to participate. The experience of the University of Bordeaux, as orally reported by some teachers, is that students are more inclined to partake in a CIE in-house, within a HEI setting, than in other CIEs organized by external stakeholders and typically held off campus. The difference is the reassurance that their peers will attend and that their students status will be taken into consideration when judging their work, for instance. This is a good case for HEI-led OIAs that invite contributions from non-academic stakeholders, versus third-party OIAs that invite students.

Incentives

The participation of students is guaranteed when the OIA belongs to their curriculum. Of course, as with any learning activity, their commitment will depend on the perceived benefits of the activity -of which there are many.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

What about voluntary OIAs? To quote [START_REF] Adamczyk | Innovation Contests: A Review, Classification and Outlook[END_REF], "research is necessary that examines what really drives students to participate in an innovation contest, [when] that participation is optional. By understanding these drivers, the structure and operation of an innovation contest could be adapted to the needs of the students and, in that way, better fulfil its aim for enhancing students' technical, design, teamwork and communication capabilities."

Engaging mentors

OIAs can be a tedious process, and facilitation is key to channel the focus and energy of participants. Teachers can be facilitators with regard to the mobilization of students' knowledge and skills, such as "providing feedback and advice during presentations (…) as well as technical advice" [START_REF] Uys | Hackathons as a formal teaching approach in information systems capstone courses[END_REF], and the application of the chosen learning design. However, many CIEs also involve external mentors, typically professionals who have experience in the topic at hand, and possibly have also participated in open innovation, with the objective to emphasize "an apprenticeship model where students are guided by experienced mentors" [START_REF] Uys | Hackathons as a formal teaching approach in information systems capstone courses[END_REF]. However, little is known about the determinants of good mentoring and facilitation [START_REF] Adamczyk | Innovation Contests: A Review, Classification and Outlook[END_REF].

Jessica [START_REF] Artiles | The Education DesignShop: Broadening non-designers' solutions for big issues[END_REF] mentions mentoring coming from sponsors, volunteers, or experts in the community, to which Walter F. Uys (2019) adds senior students. Artiles also mentions facilitation for forming the teams, matchmaking and "finding each other during the event". She further mentions that mentoring or feedback-seeking be made mandatory to the OIA, and part of the scoring rubric. [START_REF] Porya | Hackathon -A method for digital innovative success: A comparative descriptive study[END_REF] list the participation of mentors and the quality of their interactions with the participants (especially to communicate their expertise by answering questions and providing guidance) as one of the six success factors of hackathons. Indeed, according to their case study, "mentors and the role they play [affects] whether or not participants manage to develop the required prototype". Failure occurs when the mentors have enough expertise in the problem area but the communication with participants is short and rarely informative or inspirational enough. Therefore, mentors will need to be committed and display effective interpersonal communication and facilitation skills.

Incentives

Mentors should be chosen wisely, and be onboarded early to make sure that they are a good match. They need to be accessible over the duration of the OIA, from beginning to end. Sponsors, staff or senior students have an incentive to mentor. Other volunteers are typically experts and must be incentivized exactly like non-academic stakeholders, albeit on a smaller level because they will feel less pressure and challenge than regular participants: trust, direct and indirect benefits are key.

Engaging participants from different disciplines

As an organizer of the Seattle Random Hacks of Kindness reports (Mahler Levine, 2011), a nice moment in a hackathon typically occurs when a participant stands up on a chair and asks if anyone has Arduino programming skills, then "someone [jogs] over to them with a huge grin on their face". It is true that "the right [competencies] are needed for the development of the right prototype" [START_REF] Porya | Hackathon -A method for digital innovative success: A comparative descriptive study[END_REF]. The participants' skills can be technical or non-technical, and it is common to assume that anyone has a skill to contribute.

Since CIEs have gone beyond software engineering but mostly retain this component, interdisciplinarity is a prerequisite: technical and engineering skills will be shaped into social and economic views of the challenge, with artistic skills brought into the mix. This is all the more relevant for HEIs that "today there is a big challenge for most of educational institutions to propose multiskilled environment to develop the soft skills of their students.

During their teaching program, it's not easy to [immerse] students to experiment and practice such diversity. Indeed, most of the time, the young people are separate in different schools or [universities] with specific teaching program according to their orientation for engineering, design, marketing, sales, [administration]" [START_REF] Legardeur | The paradox of diversity's influence on the creative teams. Lessons learned from the analysis of 14 editions of 'The 24h of innovation' hackathon[END_REF]. Therefore, interdisciplinarity is most valued in CIEs.

However, is interdisciplinarity a condition for success? [START_REF] Legardeur | The paradox of diversity's influence on the creative teams. Lessons learned from the analysis of 14 editions of 'The 24h of innovation' hackathon[END_REF] delve into the example of the annual "24h de l'innovation" event they organize at ESTIA engineering school (Bidart, France), whose goal "is to foster the sociotechnical practices of the students that are involved in a short but intensive collaborative period with the use of creativity and design tools, marketing and communication methods". Through the retrospective data analysis of 14 editions of this CIE, the authors conclude that "if the team is composed of students with different affiliations, the diversity can increase positively the average performance (i.e. the probability to be awarded by one of the different prizes in our case study). [On the contrary], if the team is mainly composed [of] students [with] the same affiliation, the non-diversity can sometimes increase positively the excellence performance (i.e. the probability to be awarded by the 1st prize in our case study)." In summary, interdisciplinarity increases the average performance while specialty increases the excellent performance.

Specific challenges are posed by the inclusion of engineering students who "are probably the ones that experience the biggest perturbation respect to their previous training" [START_REF] Hassi | Mixing design, management and engineering students in challenge-based projects[END_REF]. The authors insist that such students tend to jump directly to technological solutions in "the low-resolution prototyping phases during needfinding and ideation (two thirds of the project duration), where they are still not needed". During that phase, engineering students also tend to focus on "the technology limitations" even though "disruptive solutions that go beyond the currently possible solutions could appear" during the iterative process. Also, because an OIA is typically driven by teaching staff from a given faculty, they must find a way to connect with other faculty staff in order to invite other participants and make them feel equally welcome.

Ocean i3, a pedagogical innovation project born from the strategic partnership between the University of Bordeaux and the University of the Basque Country (UPV/EHU), is open to all disciplines from both universities. It is a semester-long OIA, whereby students focus on the challenges posed by ocean plastic pollution with the aim to contribute to the reduction of pollution on the Atlantic coast. Students have a background in law, biology, nursing, etc. One trick used by Ocean i3 for building interdisciplinarity is to ask students to draw mind maps aimed at identifying skills and methods from each discipline and how they can be combined to provide solutions to the shared objective. In one instance, law and civil engineering students worked together to identify legal responsibility in the management of rainwater drainage, which was then taken into account in the design of technical solutions such as filters that minimize and measure the output of plastic into the ocean.

Engaging participants from different cultures

In addition to interdisciplinarity, Ocean i3 mixes participants from the intercultural and multilingual Euroregion (French Aquitaine and Spanish Basque Country), using three national and regional languages: Spanish, Euskara (Basque), and French. Since this is an international project, a logical and "efficient" decision would have been to work in English as the lingua franca. However, the organizers chose to promote multilingualism with a focus on social cohesion and intercultural dialogue, seen as an opportunity to develop transversal skills not only linked to the knowledge of languages and communication resources, but also to empathy and the awareness of diverse cultural realities, assuming that languages are key vehicles for cultures.

In order to achieve this, the organizers where supported by researchers from the DREAM -Donostia Research in Education and Multilingualism group from the University of the Basque Country (UPV/EHU), using a participatory In terms of final outputs, Ocean i3's recommendation was to present in the language most familiar to the speaker, and use two additional languages in the supporting documents (including presentation slides).

In this experience, the participants used their entire linguistic repertoire and English was the least used language, considered a last resort when no other language would allow efficient communication [START_REF] Gabbricci | Multilingual communication in a cross-border university project: the case of the Ocean i3 community[END_REF]. The Ocean i3 experience shows that intercultural dialogue (and more specifically multilingualism) can be dealt with as an enabler instead of a barrier between participants, all the more so as it is assumed rather than just accommodated.

6 Planning location, tools and resources

It was already mentioned that the OIA methodology can be supported by dedicated collaborative spaces, especially when they are focused on innovation (fablab, hackerspace, biohacklab…) or creativity (gamelab, design studio…). The goal of this section is to prepare OIA organizers for the challenges of innovation spaces: how to access and use them, how to maximize their potential in terms of specific tools and resources.

Accessing innovation spaces

Whether they are HEI-run or independent facilities, one should be able to find an innovation space within close range. HEI-run innovation spaces are more inclined to students projects but this should not deter you from finding an accommodating space off-campus. Each space has its peculiarities: whether they focus on carpentry or prefer advanced prototyping techniques, whether they are technology-savvy or open to laypeople, whether they focus on science, arts or business… you need to find the differentiating factor of the innovation space that you are interested in. It is recommended to spend time on their website and contact the key people, typically the socalled "fab manager", who can serve as a first point of contact and address your needs. If not, they usually have a good knowledge of the other innovation spaces in the area and can recommend the most suited place.

Each innovation space is different and because of their significance, the remainder of the section will focus on fablabs. We will use the fablab of the University of Bordeaux, called Coh@bit, as our main example. The fablab comprises several rooms that serve different purposes: one room for the noisy devices such as milling machine and laser cutting machine (Fig. 6.1), one room for the computer stations that is also fit for ideation (with large walls and paperboards); one room for 3D printers and their computers (Fig. 6.2); one room for tinkering, storage and meeting table (Fig. 6.3). A fablab may also have a wood workshop, but do not expect a lecture room: they are not part of the typical fablab setup. These rooms often have modularity built-in (especially in terms of seating arrangements), but the equipment is not meant to be moved. For catering, Coh@bit can sit up to 10 people in the lobby, equipped with a kitchenette. A larger group would typically eat outside, either on the university grounds or at the canteen. Some authors highlight the importance of "concentration lounges" to clear the air and avoid the "tiring routine" of the hackathon [START_REF] Gréselle-Zaïbet | Un Hackathon en contexte académique pour apprendre, se transformer et innover[END_REF][START_REF] Suominen | Educational Hackathon : Innovation Contest for Innovation Pedagogy[END_REF]: this can be achieved through simple furniture, some bean bag chairs, a couch.

Using innovation tools

The principle of fablabs is that one learns by doing: there will be no prior training with the tools, no skills required, one learns on the job that is needed to advance their project. The role of the fab managers (and sometimes the volunteers who help them) is to serve as technical advisors for users, help them learn by trial and error, and support them in asking the right questions throughout the digital production line (from 2D or 3D modeling to rapid prototyping) [START_REF] Adam | Des fablabs pour apprendre à innover : rôle des fabmanagers[END_REF]. This is achieved when users know how to use a tool on their own, and even more when they can teach their peers. Unlike "a science lab or a robotics workshop, in which each group works autonomously with one kit", a fablab typically has only one copy of each machine (except for multiple 3D printers): "this generates pressure for productivity and division of labor that could be either productive or disempowering" [START_REF] Blikstein | Digital Fabrication and 'Making' in Education: The Democratization of Invention[END_REF]. Paulo Blikstein argues that it makes "the division of labor […] a crucial enabler for the project to happen" and that "this scheme could easily turn into a disempowering arrangement when students realize that they are too dependent on the facilitators and cannot create the more complex designs by themselves".

Because the supplies used (for laser cutting, for 3D printing…) have a cost, it is expected that users pay attention to the performance of their prototyping and avoid wasting raw material. Before moving to the final material of superior quality, they can test their design with a less expensive material; again, this is done on the job with the target model, not a teaching model. The supplies are typically provided for a fee to the users, except when it is included (within a reasonable limit) in the annual subscription. Also, the scraps can be saved by the fab manager for later use, at no cost. As for the small electronic equipment (Arduino electronic board, RaspberryPi nanocomputer, etc.), it is typically loaned (for testing and experimenting) but not sold, so the user must bring their equipment if they want to keep it.

It is important to realize that fablabs, as most innovation spaces, depend on shared access and shared use.

Privatization will be the exception rather than the rule. Even if you think of your OIA as a close group of participants, it will typically not hurt to have (a small number of) users sharing the tools with you. This can even bring some benefits. There are many examples of projects that were stuck until another fablab user gave useful input. In one instance, after working for almost two weeks where "nothing seemed to work" and "frustration was in the air", a team of students "was desperate, asking everyone in the lab for ideas and help": ."some facilitators volunteered to help and come up with new ideas, and when just about everything seemed to be a failure, one revolutionary idea emerged" [START_REF] Blikstein | Digital Fabrication and 'Making' in Education: The Democratization of Invention[END_REF].

Digital skills also include the use of 2D and 3D design software, or any software that belongs to the digital production line. Such software may be less easy to use, or offer comprehensive functionalities, than its commercial alternative. However, they empower the user and they can be installed on any computer for no fee: this is particularly useful in a pedagogical context, if students want to reproduce a setup at home. The role of fab managers is therefore to explain the benefits of using open source software and to help users, who could be more familiar with commercial software, to get used to their open source alternative and convert their file format between the two. In the case of Coh@bit:

• FreeCAD is the preferred open source software for 3D design, however SolidWorks is installed on four computers (education license for a reduced fee) because it is teached at the University of Bordeaux. 123D Design is also available as it is free (it is proprietary but no longer maintained), and its successor Fusion 360 is free for three years for students and HEIs, • Inkscape is the preferred open source software for 2D design, however CorelDRAW is installed on some computers as a commercial alternative.

If possible, ask what software is available in the innovation space so that you can save your file in the right format and avoid some troubles.

Using open/online resources

Underlying the charter that lays out the rules for a proper functioning of the space, a fablab relies on the open source philosophy: "when users have used free, public time in the fablab they are strongly encouraged to make their designs available to other users so that all can learn from each other" [START_REF] Johns | FabLab Guide -how to set up your lab and maximise its impact[END_REF]. The challenges of documenting one's project have already been discussed in the section "Documentation".

Because of this open source philosophy, fab managers will encourage users to turn to open resources that they can find online to help them during the course of their project. They are numerous, the most popular resources being:

• Thingiverse and GrabCAD for 2D and 3D models shared by users,

• Github for code repositories,

• Instructables (proprietary content) and Hackster (open source content) for how-tos.

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Disseminating the activity

Dissemination is key to connect to the community at-large, to engage beyond the mere participants and to give the best chances to the OIA outcomes to make an impact after the activity. The success of the dissemination depends on:

• the strategic planning which can be defined by the five Ws: who, why, what, where, and when (Fig. 7.1),

• the partners who are invited to participate and who will be asked to contribute to the active and committed dissemination in their own communities, either through the OIA channels or their own channels. Responsibilities must be identified and shared within the organizers, in order to ensure that communication and dissemination are not left behind.

Before the OIA, the right channels must be setup to increase visibility of the OIA:

• website: a website typically displays a strong visual identity for the project. It should contain information about the OIA, the organizers and sponsors, and an updated list of their news and publications. A dedicated website may not last over time, therefore it is also recommended to store critical information on third-party repositories,

• social media: social media (e.g. LinkedIn, Twitter, Facebook…) should be used as soon as possible to promote the OIA and engage the community interested by the event, • print communication such as posters, flyers, brochures… can be displayed and distributed to raise interest to users of a building, a campus, an innovation space… During the OIA, these channels will be leveraged to inform anyone who cannot participate, but has an interest in the topic, help them follow the event and learn from its outputs. It is recommended to capture the highs and lows of teamwork, their emotion and enthusiasm… until the pitches that form the climax of the OIA. Videos are well suited, and Facebook or Twitter livestreams have a good track record of immersing the audience into an event.

Photos are also used to tell a story as it unfolds -and afterwards; wide lens, long lens, depth of field, black and white conversion, in-camera HDR… can be used to highlight specific moments of the OIA [START_REF] Clarke | The Hack Day[END_REF].

After the OIA, partners must continue the dissemination by following-up on the outputs, looking for collaboration opportunities. It is recommended to produce appropriate messages for each stakeholder target group. This phase is much less codified than the previous two and OIA organizers should feel free to experiment with any initiative that they feel can add value to the outputs. One example of large scale dissemination is provided by the European hackathon #EUvsVirus, which organized a matchmaking exercise between the 120 best projects and 458 partners from the public and private sectors, with tremendous results: 1 500 curated meetings generated 2 235 new partnerships (#EUvsVirus, 2020).

Table 7.1 sums up what can be expected from the various stakeholders and the objectives of the dissemination strategy before, during and after the event. [START_REF] Teo | The INOS Learning Design Framework: Fostering the educational value of Open Science[END_REF]. It is proposed as a first step toward the realization of a shared INOS vision on mainstreaming open innovation into higher education curriculum: it is mostly through the OIAs organized over the course of the next academic year that the INOS consortium will build its knowledge base and gather material to inform specific recommendations.

More deliverables, and more dissemination and training activities, will follow to advance this discussion and upskill the HEI community.

Figure 2 - 1

 21 Figure 2-1 Design principles of do doc ...........................................................................................................................

Figure 3 - 1

 31 Figure 3-1 The Double Diamond model has been introduced by the British Design Council in 2005 and has had a lasting impact on the field. It is still used to drive product and service design processes. (reproduced from Costa, 2018) ................................................................................................................................................................................

Figure 3 - 2

 32 Figure 3-2 Example of an ideation process (which includes pitching the ideas) in a hackathon. How to read the figure: facilitators open the exercise before the sponsor / company / jury present their case, which leads to the formation of students groups / teams, which leads to the formation of students groups / teams, which generate their first ideas for 45 min before the first break, and so on (reproduced from Suominen et al., 2018). .................

Figure 3 -

 3 Figure 3-3 The PDCA (which stands for Plan-Do-Check-Act) cycle must be repeated until the problem is solved (reproduced from Tang, 2016) .......................................................................................................................................

Figure 3 - 4

 34 Figure 3-4 Structure of an Ideation Week (reproduced from Buck and Löffl, 2020) ..................................................

Figure 5 - 1

 51 Figure 5-1 Multilingual material prepared and used during Ocean i3 project (reproduced from Gabbricci, 2020) .

Figure 6 - 1

 61 Figure 6-1 Milling machine and laser cutting machine (author: Jean-Baptiste Bonnemaison) ..................................

Figure 6 -

 6 Figure 6-2 3D printers (author: Jean-Baptiste Bonnemaison) ......................................................................................

Figure 6 - 3

 63 Figure 6-3 Tinkering, storage and meeting room (author: Jean-Baptiste Bonnemaison) ...........................................

Figure 7 - 1

 71 Figure 7-1 The five Ws of a dissemination strategy ......................................................................................................

Figure 2 - 1

 21 Figure 2-1 Design principles of do doc

Figure 3

 3 Figure 3-1 The Double Diamond model has been introduced by the British Design Council in 2005 and has had a lasting impact on the field. It is still used to drive product and service design processes. (reproduced from[START_REF] Costa | The Double Diamond model: what is it and should you use it?[END_REF]

Figure 3 - 2

 32 Figure 3-2 Example of an ideation process (which includes pitching the ideas) in a hackathon. How to read the figure:facilitators open the exercise before the sponsor / company / jury present their case, which leads to the formation of students groups / teams, which leads to the formation of students groups / teams, which generate their first ideas for 45 min before the first break, and so on (reproduced from[START_REF] Suominen | Educational Hackathon : Innovation Contest for Innovation Pedagogy[END_REF].

Figure 3 - 3

 33 Figure 3-3 The PDCA (which stands for Plan-Do-Check-Act) cycle must be repeated until the problem is solved (reproduced from[START_REF] Tang | Continuous Improvement 101: The Deming Cycle (PDCA)[END_REF]

Figure 3 - 4

 34 Figure 3-4 Structure of an Ideation Week (reproduced from[START_REF] Buck | Accelerating university-industry interaction using the design-sprint for experiencebased learning[END_REF]

Figure 5 - 1

 51 Figure 5-1 Multilingual material prepared and used during Ocean i3 project (reproduced from[START_REF] Gabbricci | Multilingual communication in a cross-border university project: the case of the Ocean i3 community[END_REF]

Figure 7 - 1

 71 Figure 7-1 The five Ws of a dissemination strategy

Table 2 -

 2

1 Comparison of intellectual property management scenarios. .....................................................................

Table 4-1 OIAs' operational objectives, inputs and short-term indicators (adapted from

Table 4 -

 4

3 Focus on socioecological and economic dimension and the far-reaching outcomes (adapted from Jordan et al., 2012) ......................................................................................................................................................................

Table 7 -

 7 1 Dissemination goals before, during and after the OIA vary according to the target groups .....................

Guidelines on designing, implementing and evaluating open innovation activities

www.inos-project.eu

Table 2 -

 2

		Organizers get all rights	Participants retain all rights	Permissive license	Option/right of first refusal
		Anyone can register the	Anyone can register the	Anyone can register the	Anyone can register the
	Trademark	trademark if it's disclosed but not used	trademark if it's disclosed but not used over a long	trademark if it's disclosed but not used	trademark if it's disclosed but not used over a long
		over a long period	period	over a long period	period
			Copyright transfers could		Participants retain their
	Copyright	Acceptable if organizers commit resources to bring the solutions to the market. Must be well communicated to participants	take place after the OIA if a participant (or the organizer) wants to use the work of others. Complications could arise with regard to co-ownership, therefore	Anyone can use the work. Good incentive for reuse, not even barring private companies (e.g. startups participating in the OIA).	intellectual property but obligate themselves to some extent of future sharing: they grant organizers an option to negotiate a license by the end of a review period
			organizers should add		(e.g. 2 months) or if they

1 Comparison of intellectual property management scenarios.

Table 4 -

 4

	Innovation dimension	Participant dimension	Socioecological and economic dimension
	Meet the interest of the participants Quality of the activity, relevancy of the topic, number of the participants...	Encourage the personal engagement / involvement Target group alignment, degree of involvement	Foster ownership and participation Number of possible patents taken (according to the evaluation jury)...
	Assure the quality of the activity innovation process / in fine of the artefact Measure of the documentation or IP for instance, diversity of funding	Facilitate the synergies between the participants Relevancy of the choice of the mentors, communication and community management tools use and performativity...	Accompany the appropriate reception Participation of the public during the showcase, articles in specialized press around the event...

1 OIAs' operational objectives, inputs and short-term indicators (adapted from

[START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF]

  and proposes to extend them to the OIAs with some illustrative indicators, in light of the work carried out in "The INOS learning design framework: Fostering the

Guidelines on designing, implementing and evaluating open innovation activities

www.inos-project.eu

Table 4 -

 4 2 OIAs strategic objectives, impacts and long-term indicators (adapted from[START_REF] Kieslinger | Evaluating citizen science -Towards an open framework[END_REF]

	Innovation dimension	Participant dimension	Socioecological and economic dimension
	Create new projects and collaborations e.g. service, product, concept, insight, start-up, initiative, joint project... Number of new projects or collaborations (compared to the typology of new projects and number of teams for example),	Develop personal learning and development Degree of learning (comparison between personal learning before the event and skills learnt (according to the participants) after the event), typology of the personal	Increase civic resilience and social / ecological impact Described in Table 4.3 below
	collaborations...
	Create new knowledge resources Number of publications around the activity, typology of the publications analysis on the activity...) (e.g. presentation of the results,	Motivate new skills, empowerment and fun Number of suggestions for improvement thinking in learning tasks discussions, evidence of advanced of the activity, quality of questions and	Build a community around innovation Described in Table 4.3 below
	Develop and enrich the innovation
	processes in the institution Evolution of the stakeholders involved in the institution, implication of the research fields and structures...	Innovative value of the project's outputs Quality of the outputs (life of the product, publications...)	Encourage trust in science Described in Table 4.3 below

Table 7 -

 7 1 Dissemination goals before, during and after the OIA vary according to the target groups This document provides guidance to the implementation of open innovation activities (OIAs) as part of the INOS Erasmus+ strategic partnership. It is a companion to the learning design framework for open science activities

		Pre-event	During event	Post-event
	Problem owners, topic experts, engaged "grassroots" citizens / movements	Be involved Define the relevant topics and themes for the hackathon Determine the problems of real value that can be addressed	Provide inspiration and challenge the relevance and assumptions of the problems and solutions	Are key to helping OIA outcome turn into impactful solutions
		Provide important insights
	Public authorities / services at local, regional or national level, interest groups, NGOs	about existing organizational, business related and institutional settings. and/or provide access to Define a relevant problem	Provide buy-in and context, informed understanding of existing (social) infrastructure contexts as well as relevant political	Participate in the development and incubation of the solutions
		relevant data
	Students and researchers Frame and design the OIA	Participation, ideation, networking and learning	Have the motivation, space and time to continue working on their solutions
		Provide expertise that enhances
		creativity about data	Document the data made
	Open data enthusiasts	acquisition and data	available for the "hack"	Advise on the development
	and data specialists	possibilities.	Assist the participants to	of the solutions
		Identify and assess relevant	access and use the data
		open datasets

Guidelines on designing, implementing and evaluating open innovation activities

Guidelines on designing, implementing and evaluating open innovation activitieswww.inos-project.eu

For LIBER it applies to O3 only as not involved in O4.

Guidelines on designing, implementing and evaluating open innovation activities

www.inos-project.eu research approach to learn, accompany, and guide intercultural and multilingual practices within the Ocean i3 community. Their key messages, which can be applied to other dimensions of intercultural dialogue, were:

1. to deal with and take advantage of the multilingual environment, 2. to protect the rich language diversity of the participants.

Consequently, specific rules have been established to improve operativeness, while protecting linguistic and cultural diversity:

• all local languages are welcome, in addition to English, which is used as the lingua franca,

• the use of minority languages is especially encouraged,

• each person may use the language they are most comfortable with and may alternate between languages if they wish, • the challenge is to co-construct meaning together using all the linguistic resources of the participants:

○ speak at an adequate pace in order to make it easier for the others to understand, ○ when possible, include multiple languages in the supporting material for contributions, ○ indicate if someone doesn't understand the speaker so he or she can repeat, paraphrase, translate, slow down, etc., ○ sit next to someone who speaks other languages than you do, in order to best combine linguistic resources, ○ use simultaneous-spontaneous translation in order to help each other out, ○ switch languages any time you feel the need to do so. Support material had been prepared, such as help cards that participants may use to ask someone to repeat an idea, speak slower, request translation, or report that a concept has not been understood. Innovation spaces exist in other fields than engineering. In the field of health studies for instance, one can mention the Pharmacy Innovation Lab at the University of Pittsburgh, the Health Design Lab at Thomas Jefferson University [START_REF] Aungst | From ideation to practice: How pharmacists and students can leverage hackathons and innovation labs to accelerate innovation in pharmacy[END_REF] or the MakerHealth Space at the University of Texas Medical Branch [START_REF] Whitmer | Makerspaces That Set The Stage For Lifelong Learning[END_REF].

General COVID-19 regulations apply to innovation spaces (physical distancing, limited number of people per room, disinfection of surfaces), as they pose no specific risk.

Fablabs are shared, collaborative spaces that enforce a set of rules (typically a charter) in order to empower users in ensuring that the place runs correctly. These rules, as summarized in a report by the University of Bristol [START_REF] Johns | FabLab Guide -how to set up your lab and maximise its impact[END_REF], are threefold:

• safety: not hurting people or machines,

• operations: assisting with cleaning, maintaining and improving the lab,

• knowledge: contributing to documentation and instruction (fablabs often struggle to achieve comprehensive documentation of their users' projects, see for example the projects gallery of Coh@bit: https://projets.cohabit.fr/redmine/).

Safety and operations, which are at the heart of fablabs, is covered in the next section. Also, fablabs have diverse business models and access is often controlled by a membership or service fee [START_REF] Johns | FabLab Guide -how to set up your lab and maximise its impact[END_REF]. • Please, share any suggestions of improvement for the organisers (practical organisation, time management, access to the resources, choice of mentors, choice of topics...):

• How would you rate your understanding of the activity and its goals?

• How would you rate your degree of involvement in the activity?

• How would you rate your interaction with the rest of the team? 1 2 3 4 5

• How would you rate your access to knowledge, resources and/or data?

• How would you rate your access to training and to the expertise of the other participants and/or mentors?

4 Following the activity

To fill after the activity (right after its completion or later)

57

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu 1 is the lowest and 5 is the highest.

• How would you rate your level of knowledge about the topic? 1 2 3 4

• Please specify:

• How would you rate your empowerment in term of (to fill according to the topic) after the activity?

• Please specify:

• How would you rate your acquisition of new skills (or fill the type of skills: technical, soft, domain expertise, project management, others) after the activity? 1 2 3 4

• Please specify:

• How would you rate your motivation to continue the project? 1 2 3 4

• Please specify:

58

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

• How would you rate the networking dimension of the activity? 1 2 3 4 5

• Please specify:

• How would you rate your understanding of open-innovation and the social impact of citizen science for society? 1 2 3 4 5

• Please specify:

59

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

Annex B -Guidelines for video testimonials

Prepared by Katerina Zourou,Ania Skowron and Giulia Torresin,Web2Learn (W2L) 1 Context

For O3 and O4 citizen science (CS) activities foreseen, partners having the responsibility of carrying them out will produce, with the technical support of W2L, short videos of approximately 1-1,5 minutes to maximize impact of the local activities, to share them broadly with the international community as well as to connect all CS activities of INOS project happening at different locations, under a common identity of the project.

Profile of interviewees and expected result

As citizen science is a multifaceted phenomenon, we opt for one (1) interview with a participant and one (1) interview with a mentor/coordinator of the event, for each of the 2 outputs. Therefore, the final result will be: If a partner wishes, the video with the mentor can take the form of a short recorded online interview that we can carry out (yourself acting as the interviewee and us (W2L) acting as interviewer) through a videoconferencing system.

3 Choice of the event during which the interview will take place

Because in each Output every partner carries out two CS activities, each partner decides which event is more suitable to carry out the recording. It is thus up to the partners, as organizers of the event, to find the most suitable participants from the suitable event. INOS focuses on the potential of CS to Higher Education, so university students as participants seem to be ideal interlocutors. This also allows to overcome the obstacle of participants of minor age.

Duration

The total duration of the video is expected to be around 90 seconds (can be up to 120 seconds). The point of keeping the duration short is a) to adopt the general trend for short videos as a guarantee that the entire video is watched and b) to maximize the overall number of videos views, in a way that connections/transitions happen between all INOS videos gathered by all partners. Through this strategy we ensure a bigger reach: a more diversified audience (from more countries than the local/national community of the interviewees), and a complementarity of points of view (participants/mentors).

The total duration of each video will be slightly bigger than the mere collation of video extracts (around 10 seconds more) because we plan to add pictures and 3 slides, namely a) the INOS project identity b), the event factsheet (cf. point 8 below) and c) a disclaimer slide with all our communication channels.

Content

We opt for common questions to be asked to all interviewees, participants as well as mentors, as a means to familiarize the audience with the wealth of CS projects and all connections between the different CS projects.

QUESTIONS TO THE PARTICIPANTS

• Did the activity [event / project / course] allow you to collaborate with people of different profiles and backgrounds, and to develop a new approach on the subject / challenge? • Did the challenge (or the theme) seem relevant to you as a participant and as a team? Did the activity [event / project / course] and its organization allow you to develop new ideas in an efficient and concrete way? • How are you going to follow-up on the innovation / ideas developed? Does the innovation / idea seem applicable to other fields than those considered during the activity [event / project / course]?

QUESTIONS TO THE MENTORS

• Did the activity [event / project / course] allow for the mixing of participants from different backgrounds and the sharing of different knowledge, thus contributing to an open innovation process? • According to you, was the challenge and more generally the activity [event / project / course] suitable for the co-development of innovations and their practical applications? • Do you think that the innovation [idea / service / outcome] produced meets socio or economic needs, and is it transferable to other fields or applications, other contexts and other people?

6 Recording a video Some basic advice on how to shoot your video 1 :

1 Guidelines adopted from https://www.givegab.com/blog/9-guidelines-for-creating-video-content/

61

Guidelines on designing, implementing and evaluating open innovation activities www.inos-project.eu

• Audio Conditions -Film in a quiet place, where you are not likely to be interrupted.

• Lighting -Film in a bright room, in the shade outdoors, or under cloud cover. Avoid bright sunlight as it creates harsh shadows. Avoid lighting that comes only from directly above. • Camera Orientation -Film with landscape orientation (horizontally) This will be most practical for viewing on all types of devices. • Composition -Having your subject in the centre of the frame, looking directly into the camera, creates a very personal feel and can load your video with emotion. • Camera Angle -Keep the camera just above your subject's eye level, slightly angled down, which is flattering for most people.

7 Images

To make an attractive video we also need pictures from each event. Pictures are ideal especially when we need to remove some small extracts from the video due to factors such as noisy extract, very silent voice, etc. Please take as many pictures as you can, and in case of a person of minor age, we can edit these pictures (by blurring them) if needed afterwards, at editing stage.

8 Event factsheet

We will add one slide with short event identity so we kindly ask you to fill in the following data for each event:

• Title of the event 9

Storage of videos

The final videos, after being edited by W2L, will be stored at each institution's site, according to each institution's data sharing policy/GDPR. We can also foresee that all videos are not hosted by each institution but on the INOS website (depending if other institutional partners are OK with it). In the first case the videos will be internally linked to the project website and can be disseminated by other partners.

10 Language

In case of non-English interview: partners must translate the content of the interview in their language and link it to time stamps (connect the sentences with a time stamp) so that we are able to add subtitles.

11 Informed consent form

Participants and mentors will sign a consent form allowing us to use the recorded video. Because each partner may emphasize different aspects of disclosure and data privacy, each partner should use an informed consent form that aligns to his/her institutional policy. The consent should be given for full exploitation and (re-)use of the entire video. Participants may prefer to give their final opinion about it before public release: we prefer not to add this level of complexity. But if your institutional policy says so then that's fine on our side as well.

An example of an informed consent form, made available by the UK National Agency for Erasmus+, can be found at <https://drive.google.com/drive/folders/1jfpcynXkg-2sm12QNZzl1YM-3870isTe>.

Please make sure you share with W2L both the template of the Informed consent form you will use, plus the signed forms, for monitoring needs by the National Agency. You can place them in this folder, thank you.

12 Responsibilities AAU, LIBER, TU, UBx and UO 2 :

• Identify an informed consent form that fulfils the data security policy at your institution.

• Record the raw videos. In case you have more than one videos with the same interviewee please do share as well (perhaps you needed to do a second video, but in case in the first video there is a piece to exploit, do send it along). For LIBER • Take pictures (we can always blur some faces, for instance from minor participants) • If in another language, transcribe and translate the content of the interview W2L: edit all videos in an engaging format. Also add transitions and intermediary slides, such as a) INOS project identity b), event factsheet like short bullet points and c) disclaimer slide at the end.