Kenneth Maussang -Leader

Hélène Jouguet

Thomas Jouneau

Jean-Francois Martin

Nicolas Larousse _____

F S Sanz

T Holocher-Ertl

B Kieslinger

Olivier Laurent

Yara Carrejo Gironza

Sophie Ancelet

Olivier Armant

Denis Bard

Christian Gonzalez-Laporte

R Bonney

H Ballard

R Jordan

E Mccallie

T Phillips

J Shirk

C C Wilderman

B Ambrose-Oji

A P N Van Der Jagt

Participatory research and data Issues at stake and recommendations coming from examples of participatory research projects

Keywords: participatory research, citizen sciences, data, quality, reliability, reproducibility, credibility, legitimacy, heterogeneity, community, mediation, software tool, trust, open source, quality strategy, fake data, GDPR. v2.9.21-EN

Recommendation 1: To identify the benefit of a participatory approach for answering the project's research question; to define in advance the type of participatory research and the level of the participants' involvement to take into account the associated constraints: co-construction, participatory data analysis, crowdsourcing, manuscript transcription, etc.

Recommendation 2: To refer to an ethics committee prior to the project being implemented.

Recommendation 3: To use a data management plan, starting at the project construction stage in order to clarify and plan for the specifics involved in the

Participatory research and data

Issues at stake and recommendations coming from examples of participatory research projects

Participatory research is "a means of producing scientific knowledge in which non-professional and non-scientific actors -whether individuals or groups -are involved in an active, deliberate way" [1]. The non-scientific participants of a participatory research project can intervene at any stage of the scientific research process: data collection, data analysis, as well as during the construction of the initial research problem.

These non-scientific actors can be groups of individuals (for example in the form of associations or citizens' collectives); primary or secondary school pupils; students; groups of professionals from a given sector, etc... Participatory research can also involve, for example, patient associations in medical research or victims' associations.

Participatory research was initially developed in the fields of agronomy and ecology [1,2], where the contribution of a group of non-scientific participants makes it possible to cover a significant geographical range and increase the volume of observational data produced. Within these research communities, skills have been developed to address the specific issues encountered in this type of project, notably at Cirad [3], Inrae [4], IRD, MNHN and Inserm. However, as a result of the increase in the amount of digital equipment available to the overall population (smartphones equipped with sensors, internet access, etc.), such participatory research projects are now being developed in all disciplinary fields, with a wide variety of types of implementation and operating procedures.

Participatory research is a tool that helps to establish a dialogue between science and society, one that initiates citizens into a scientific approach. It is also a means to draw on collective intelligence to produce knowledge. However, in order to fully benefit from these approaches, it is important to take account of the specifics of this type of knowledge production in order to maintain the quality and reliability of the scientific results.

The aim of this document is to put forward recommendations for all researcher(s) who would like to initiate a participatory research project in whatever discipline. These recommendations include points to which particular attention should be paid and general entry level methodological principles. It should be remembered that the wide variety of types of participatory research projects and disciplinary practices makes it impossible to be exhaustive. For more details on any of the topics discussed here, please refer to the research support services within your own establishment or to the dedicated services such as MNHN's MOSAIC (https://mosaic.mnhn.fr/). This document only covers issues linked to the data from participatory research projects.

Scientific issues at stake and stakeholder motivations

The benefits of a participatory approach should be assessed in terms of the defined research objectives, looking at the quantitative and qualitative benefits on the data. The main motivations for volunteer participants are usually:

-the desire to learn and/or develop skills; -the feeling of being involved in a collective project for a meaningful purpose; -willingness to understand the associated research issues. The data produced have a strong impact on the participants' motivation. This means that it is crucial to enable them to "see" the data and the impact of their contributions to the project. From a scientific point of view, as a secondary objective, participatory research can be a means of teaching people about the scientific approach. It helps to explain how research is carried out, with all the complexity involved in it. The reliability of the data produced is a major scientific issue, as is the credibility of the institution backing the project in terms of its ability to produce high-quality knowledge.

Structuring and steering a participatory research project

The following aspects should be clearly identified: the task in the research process to be assigned to external participants, the stakeholders and their common interests and motives for getting involved. The input of an ethics committee should be sought as early as possible, and the project's Steering Committee should possess a wide variety of skills (legal, information technology, health and safety, communications, etc.). All the potential issues associated with the participatory and heterogenous nature of the community thus formed should be planned for in advance. These include animation and follow-up, differing levels of commitment over time and of attention to detail in the work, the types of documents produced and accessibility in terms of adapting these to the relevant audience. It is recommended to use a data management plan from the design phase onwards. This is even more important than in a conventional research project. The data from the project constitutes a common good shared between the researchers and the contributors, and thereby a lever for motivating the community. The participants' expectations, their role in the research process and the benefits they will receive in return should be clearly and v2.9.21-EN formally established at the beginning. All of the documents provided to the project participants should be written in language that is clear and comprehensible to all. The participants' commitment will depend on their trust in the data treatment process. To gain this trust, it is important to be transparent about the how the data will potentially be processed and used. It is recommended to formally define the terms of participation in the project. A data policy (for the whole data lifecycle) can be communicated to the participants. This is not a substitute for the data management plan, but it provides information on the rules governing the availability and use of the participatory data, expressed in terms that are clear and accessible to the participants.

The participants

The participants in the project form a heterogenous community in terms of their knowledge and skills, their digital literacy, their motivations and commitment to the project. This is reflected in how closely they follow the protocols of the tasks assigned to them. This heterogeneity can take many different forms and is an intrinsic source of additional variability in the data produced. The participants should receive training in order for them to correctly perform the tasks assigned to them. The accompanying materials should be adapted to a heterogenous population of contributors, with materials of different levels of detail and technicality. This heterogeneity can be taken into account via a quality strategy aiming to ensure that the data produced is of high quality. Depending on the project, this strategy can either entail assigning different tasks to different participants in line with their skills or assigning dynamic confidence scores to individuals linked to the improvements in their skills over time. The protocols should be established to suit all of the actors (scientists and participants) and take into account the diversity of knowledge and uses. It is common practice and generally useful to involve third parties (associations, experienced contributors, professional organisations, etc.) to provide facilitation to the community of a participatory research project. However, the motivations and interests of these external actors should not conflict with the objectives and motivations of the researchers. For the project to function correctly, these actors must have (real and/or perceived) legitimacy and be impartial. It is recommended that prior consideration be given to how the work provided by the different parties involved will be recognised when communicating the results, and to make this explicit to the participants when they sign up to the project. Ensuring that non-expert participants are attuned to the culture of research enables them to understand the scientific issues at stake. To do this, it is also worth considering providing training in scientific mediation to the researchers leading the project.

Research data and data produced by non-scientific participants

The data in a participatory research project can be the data shared with the participants as study objects, as well as the data produced by the participants as part of the task delegated to them. The involvement of a large number of people in the research process will intrinsically generate a higher rate of error in the data produced. Assessing the acceptability thresholds for the error rates enables a pertinent use of the data and helps to ensure that the work done by the participants is useful and usable. It is recommended that a strategy be established to assess the quality and reliability of the contributions (beforehand and afterwards) and, if possible, to integrate quantitative indicators to attribute dynamic individual confidence scores. The protocols given to the participants should be assessed and tested before being handed out or may even be co-constructed with participants. v2.9.21-EN Maintaining a link between a dataset and the participant who produced it (while retaining anonymity) makes it possible to construct these indicators and identify the less reliable contributors, including malicious practices where data are deliberately tampered with (fake data). Exclusion rules can be put in place for such participants. There are different levels of openness when managing the sharing of the project data:

-data sharing restricted to the researchers; -data sharing restricted to the participants and the researchers in compliance with regulatory constraints (GDPR 1 , health data); -public open access to the data during or after the project. It is useful to assign a pseudonym or username to the participants when they sign up, to ensure their anonymity in public communications. It is important to give clear guarantees relating to the use of the data, to be explicit as to what will become of them and to define clear rules on sharing and citing them (data sharing plan). The best tool for doing this analysis is the data management plan. When data are made openly accessible, it is important to provide materials that can be understood and accessed by all interested persons -beyond simply the actors involved in the project. The data produced can be a tool to motivate the contributors, notably when a visualisation tool allows them to "see" their contributions. The participants or the general public may express legitimate concerns as regards the credibility of the data produced and the results obtained, so it is important to be as transparent as possible about the way in which the raw data produced by the contributors will potentially be processed, using language that is clear and accessible to all. Real-time visualisation of the contributions improves the transparency of the system (a guarantee of credibility) and helps to develop a culture of high standards among the contributors. A lack of credibility can impact the research team, its home institution and/or the project's funding body. Participatory research can raise problems in terms of personal data in disciplinary fields where such concerns are not normally present.

The tools

These are key factors in the success of a participatory research project. The ergonomics of the software tools used by the participants are very important. The tools should be easy to use by non-scientists in order to reduce the level of input errors. It is recommended to use open-source tools to build a relationship of trust. Developing tools is costly and lengthy, so it is advisable to reuse existing tools, or tools developed for other projects. The tool should preferably be able to automate as much of the data production as possible, especially the associated metadata. It should clearly delimit the potential answers, where this is appropriate to the task in question. It is recommended to use interoperable standards when these exist. However, the inclusion of a free text field (for comments for example) provides the flexibility required for situations that were not anticipated at the design phase, and a means of reporting errors. Providing a variety of tools to collect and process the data (website, smartphone app, paper version, etc.) is one lever for diversifying the participants' profiles and mitigating risk, including malfunctions, obsolescence or poor adhesion in the target community. It is possible to enable contributors to develop their own tool by providing them with an API. The costs of developing and maintaining the tool should be planned for. The software tool should make it possible to save all of the data and keep track of all modifications made to a piece of data (timestamping, historical record, versioning). 1 General Data Protection Regulation 6/63 v2.9.21-EN Data visualisation (and the associated tools) is a major but complex issue when it comes to motivating the contributors and mediating in the community. It is worth providing a wide range of different communication channels to encourage the greatest number of people to get involved (newsletters, websites, forums, webinars, etc.).

Legal aspects

The project's data policy should clearly define the ownership of the data produced during the project and be explicit regarding the role of the participants in the project. It is also important to assess the sensitivity level of the data produced (GDPR, ethics, etc.). It is recommended to refer to the relevant legal services at the project leader's home institution. 9/63 v2.9.21-EN Participatory research is "a way of producing scientific knowledge where non-professional and nonscientific actors -whether as individuals or in groups -participate in an active and deliberate way" [1]. These non-scientific actors can be groups of individuals, members of associations, primary or secondary school pupils, students, or groups of professionals from a given professional branch. Participatory research may involve patient associations in medical research or victims' associations. The non-scientist contributors can be involved at any stage of the research process: defining the research problem, collecting data, analysing data. Participatory research projects can differ greatly in the way in which they to get participants involved. There is a continuum in the degree of participant commitment in the research process, with operational implementation ranging from crowdsourcing to action research. Although there is a range of different definitions and terminologies used for participatory approaches in the context of research projects, reference [1] provides a classification made out of three levels of commitment: citizen science, contributory science and community science.

"There are plenty of different degrees of engagement possible [...], ranging from very small actions to involvement as co-researchers."

Fabian DOCAGNE, DR Inserm "Science et société" service -Inserm In this document, we will use a broad definition, where a participatory research project is a research project in which non-scientists and non-professionals -whether as individuals or in groups -participate in an active, deliberate way at any stage of the research process. We will not consider research projects where the participants are simply passive study objects. To ensure the reliability of the data produced in a participatory research project and of their participatory analysis, the specifics of the operators -i.e. the participantsshould be taken into account, particularly the heterogeneity of their skills and knowledge levels. Failing a closed, robust protocol, a key factor is the training of participants. The scientists should "mobilise disciplinary knowledge, knowledge dialogues and support for change" [1]. It is essential to train the contributors to ensure data quality using detailed, educational protocols, via dedicated training programmes, or by creating participant communities within the project. This can also be a source of motivation for the actors during the project. Also, "empowering the participants during the programme and allowing them to directly visualise the data and research results form part of the general aims of participatory approaches" [1]. A participatory research project is based on a win-win principle, with the v2.9.21-EN research team benefiting from contributors' help to perform a predefined task, with them becoming self-learning communities.

"This system meant that we were able to give citizens a central role in the ongoing research, and to make them actors in the research projects that they had coconstructed. [...] We organised research training programmes on subjects that we co-constructed with the citizens -in the laboratory -that aimed to add value to the data and samples sent to us by other citizens."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project A participatory research project can be a tool to raise awareness of the scientific process or to disseminate scientific culture, but only as a secondary objective. Primarily, it is a research project where a participatory approach is identified as a relevant way of answering a research question. In a national survey led by the "Research Data" college of the french Committee for Open Science [5], leaders of participatory research projects highlighted the benefits of this approach for their data, notably in the following ways:

• increase in the volume of data;

• geographical diversity;

• sociological diversity;

• diversity of experimental conditions. Some respondents also mentioned that participatory approaches made it possible to access data that would otherwise have been hard to reach.

"In some cases, this link with society is also about seeking knowledge in society that is not immediately obvious. This additional expertise can be found in society."

Thomas Lebarbé, Professor at Université Grenoble Alpes v2.9.21-EN

PARTICIPANT MOTIVATIONS

"It is a win-win situation for the researchers and the participants. The participants will become self-learning communities: the more they participate, the better they'll understand the protocol, and the more skills they will gain individually and collectively on an issue. [...] This increase in skills is an inherent part of participatory research projects."

Emmanuelle Gonzalez, MNHN Deputy director of the MOSAIC skills centre "Participating actually enables them to see things they wouldn't otherwise have seen."

Thomas Lebarbé, Professor at Université Grenoble Alpes

A participatory research project must identify sources of motivation for potential participants. Indeed, these people will only get involved if they are going to get something out of it, whatever this may be. The desire to increase skills or acquire new knowledge is an important driver in these projects, but only at the beginning. It is essential to offer other more long-term sources of motivation in order for the participants to continue to commit over time. Another driver is often the sense of contributing to an important mission, a meaningful project. Thus, the research team should provide context and explanations to the participants about the issues addressed by the research question. More generally, the driver can be the satisfaction of participating in building knowledge. However, this very often requires a form of reciprocity, with access to the results. This means that the organisers should carefully plan how they communicate with the participants, in a way that is adapted to the audience and their level of knowledge. Another type of benefit lies in the training the participants receive, which is a form of knowledge transmission. The training on offer can be used when publicising the project as a way of encouraging people to sign up. It is also possible to create a community of participants or a community of users by organising regular meetings (eg. OpenRadiation 2 , by creating a forum, a map of users, etc.). These communities make it possible to maintain links with the participants and to nurture their feeling of belonging to the project and so their motivation. Additionally, through dialogue, these communities can participate in the process of continually improving the training resources for participants, and create a dynamic of exchange between beginners and the more experienced, etc. Finally, the sense of belonging to a community -made up of participants involved in the project -can be a powerful lever for boosting contributor commitment. Here, social media can be powerful tool for nurturing a sense of belonging to this community. v2.9.21-EN "A powerful driving force to create stickiness is the feeling of belonging to a community. [...] We can see this in the comments, there is a momentum in social media, reputations are built there. [...] When it works, this sense of belonging is a powerful lever for loyalty." "There is another driving force, which is commitment to a meaningful project. It is really important to work towards giving sense to a research project, on this link between the contribution made and what it will be used for."

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre A survey on participants' motivations for getting involved in participatory research projects was carried out in 2016 at the Science, Research and Society Forum organised by "Le Monde" and "La Recherche" magazine [6]. It revealed that 51% of the respondents would like to "know how [their] work is used", 32% would like to "be able to meet and discuss with the researchers", 32% would like to "receive a preview of the results of the research carried out" and 24% "receive scientific training". So it appears that communication, as well as interacting with and meeting the researchers are key aspects in participant motivation. In the same survey, only 5% said they would like to "be cited in a scientific publication".

"[be] very transparent about how a publication is done, how it is written. We planned a [videoconference] event for the day we submit the publication, it is a process that people don't know about. [...] The idea is that they try to participate as much as possible in all the stages." Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

INSTITUTIONAL AND SOCIETY MOTIVATIONS

Trust in science and the role of the expert

The French Ministry of Higher Education and Research's "Science With and For Society" action plan includes participatory research actions. Indeed, a participatory research project makes it possible to -actively involve non-scientists in the research process, in contrast to science popularisation approaches where the public remains passive; -get the general public familiarised with scientific methods; -get the general public familiarised with the concepts of error and doubt in science when constructing knowledge; -make visible the invisible actors in research -meaning the staff involved in research aside from the researchers; -reveal all the aspects of researchers' work at all stages of the research project (from initial construction to scientific publication), and not simply the results of the research; -make the general public familiar with the methods of interpreting research results. v2.9.21-EN "Today we have a problem with citizens not trusting science. To rebuild this trust, I am convinced that we need to be highly transparent about the process, and that citizens should be able to interact with it. Indeed, there are knowledgeable citizens who could tell us "it would be better if you corrected this line of code," creating a win-win situation." Sébastien Payan, Professor at Sorbonne Université "they had a laboratory notebook [...] and we explained what this notebook was [...] We clearly explained that if we made a mistake in the protocol, this should never be hidden, and that there is no shame in making mistakes, it was better to report it and that it would be reported in the notebook."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

Making the public familiar with scientific methods helps to improve the relationship of trust within science and in the scientific discourse. The aim is not simply to explain a research subject to the overall population, but to show them how research is done, and the complexity involved beneath. A better understanding of scientific methods and approaches contributes to understanding the scientific controversy which is inherent in knowledge construction, where doubt and error play an important role. These approaches also make it possible to address the question of the role of researchers as expert in society, notably their role as guarantors of the credibility of facts in public discourse.

"introducing the scientific approach to volunteers, trying to make them understand our profession from A to Z. [...] our volunteers are like researchers, they work on almost all of the stages of a scientific experiment." Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

In 2023, an Ifop study for the Reboot Foundation and the Jean Jaurès Foundation revealed an increasing defiance in young people concerning science, but also a form of withdrawal from certain truths for which there is a consensus in the scientific world [START_REF] Tiktok | Génération "toctoc"? Enquête sur la mésinformation des jeunes et leur rapport à la science et au paranormal à l'heure des réseaux sociaux[END_REF]. This study is complementary to the Ifop report on French people's relationships to science [START_REF]Le rapport des Français à la science ET au progrès scientifique[END_REF]. The problem is no longer one of access to information but rather information "fog", because of the increasing number of tools and information channels. Social media now plays an important role in disseminating information, as well as disinformation. How can complexity be considered in a time of immediacy on social media? The issue is no longer providing access to information but making people understand how knowledge is constructed. Participatory approaches form part of the response to this problem. Another issue lies in lay-people's understanding of the role of experts in public debate, and how to identify these experts, whose legitimacy stems from peer recognition and an established scientific community. Gaining an awareness of the culture associated with a scientific approach should contribute to the general public developing a form of critical judgement v2.9.21-EN regarding the information flow to which they are exposed. It should provide an understanding of the role of the expert in transmitting knowledge and information, as well as the role of information professionals (journalists, librarians, etc.). This is how these participatory approaches help people to understand the inherent complexity in the process of constructing knowledge, scientific doubt, error and controversy. The Biden administration (White House Office of Science and Technology Policy -OSTP) announced that 2023 would be the year of Open Science, with a series of upcoming initiatives (https://open.science.gov/). Participatory approaches had already been initiated during the Barack Obama administration in the context of participatory democracy and open government initiatives via the Crowdsourcing and Citizen Science Act in 2016 3 . In 2011, the Obama administration launched the first U.S. Open Government National Action Plan [9]. As part of this, one of the initiatives was to support participatory research (Crowdsourcing and Citizen Science Programs), which was defined as follows:

Public participation in scientific research, one type of crowdsourcing known as "citizen science", allows the public to make critical contributions to the fields of science, technology, engineering, and math by collecting, analyzing, and sharing a wide range of data. The Administration will expand its use of crowdsourcing and citizen science programs to further engage the public in problem-solving.

Within this action plan, the federal agencies supported many participatory research initiatives [START_REF]Designing a Citizen Science and Crowdsourcing Toolkit for the Federal Government[END_REF] and produced resources now available online, such as those of the Department of Agriculture4 and NASA 5 .

Data reliability and institutional credibility

By its very nature, a participatory research project has greater exposure and hence greater visibility. The institution supporting the project will be associated with this project, and any problems regarding data reliability may impact its image and credibility. The reliability of the data produced in a participatory research project is not simply a scientific issue but also an institutional one. It involves the credibility of the institution supporting the project in terms of its ability to produce reliable, high-quality knowledge. It is important to pay special attention to the quality and reliability of the data produced, so that the project is able to carry out accurate qualitative research but also so that the image of the associated institution is not negatively affected (lack of credibility, data distortion, etc.).

THE PROJECT: STRUCTURING AND STEERING

IN THE DESIGN PHASE

Why take a participatory approach?

15/63 v2.9.21-EN "What research objectives is the project aiming to meet, and why is a participatory approach useful? Why do we need to use participation to obtain this type of data?" Romain Julliard, MNHN Professor Director of the MOSAIC skills centre A participatory research project is, above all, a research project with additional specifics and constraints. It is therefore appropriate to question the benefits of a participatory approach to answer the project's research question. The tasks in the research process that will be assigned to external participants should be clearly identified to set out the methods and procedures of participation. Which tasks will be carried out by the participants? What data (and metadata) will the research team need once this task is fulfilled?

Recommendation 1: Identify the interest of a participatory approach to answer the project's research question; define in advance the type of participatory research and the level of the participants' involvement to take their constraints into account: coconstruction, participatory data analysis, crowdsourcing, manuscript transcription, etc.

Understanding the stakeholders and their mutual interests

"What do we give them back, what do we show them? [...] What we give back is a valid yet complicated question. There is no point in giving them raw data, because it can be badly interpreted if it is badly analysed [...] And that is something that should really be co-constructed I think, with the participants right at the start of the projects. We can explain, scientifically, what data we need to get to progress our knowledge, but then the participant could say 'I would be interested in having this or that type of information' and we try to co-construct the indicators from the data. This process, I think, should be put into action and make it possible to build trust."

Sébastien Payan, Professor at the Sorbonne Université When constructing a participatory research project, it is important to understand the stakeholders, their mutual interests in the project and their motivations for getting involved. In particular, the following points should be examined -What are the participants' expectations? -What data is needed to answer the research question? -What will the data from the research project be used for? -Who will be animating the community of participants? -What economic model will this animation use?

Very often, this animation is delegated to a third party such as an association, for example. The interests and motivations of the third party should never conflict with the objectives and motivations of the researchers (see section 4.8, page 31).

v2.9.21-EN "it is important for the steering committee to possess a diversity of skills. For example, having a computer scientist [...], a health and safety officer and a GDPR specialist from the start of the project."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

It is recommended that a wide range of expertise in the project's Steering Committee be included, particularly communication and information systems skills. Depending on the type of project, legal and health and safety expertise can also be relevant (chemical products or biological samples, for example).

"we can't just do what we like with the samples. Yes to citizen help, but in a wellequipped facility so that the work is done under optimal health and safety conditions."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project

Referring to an ethics committee

"If we have location information, this is deemed to be sensitive. We may have to introduce ways of blurring or anonymising the data. I recommend referring the project to a research ethics committee. This gives an outside view in both ethical and legal terms, and provides us with feedback and advice."

Sébastien Payan, Professor at the Sorbonne Université

In some disciplines, the conventional research projects do not encounter certain ethical problems that frequently arise in participatory research projects (personal data, conflicts of interest, etc.), so these researchers are not used to dealing with such issues. If the project holder's host institution has an ethics committee, it should be brought in as early as possible.

Recommendation 2: Refer to an ethics committee before implementing the project.

THE HUMAN RESOURCE IMPACT OF A PARTICIPATORY RESEARCH PROJECT "The workload of all the phases of a participatory research project should be carefully considered."

Marta Severo, Professor at Université Paris Nanterre Coordinator of the "Vitrines en confinement" project Leader of the ANR Collabora project v2.9.21-EN

In a participatory research project, it is important to anticipate the heavy workload involved in the animation, production of educational materials, communication, etc... in addition to the usual activities of a standard research project. There can be a lot of additional work, the amount of which may sometimes be underestimated at the beginning, which can endanger the project's viability and the quality of the resulting scientific outcomes.

"this was the key word in all the interviews we did with researchers [who managed a participatory research project]. They all mentioned the time [that it takes]. They all mentioned trust, and they all talked about time. The notion of time is completely different. Especially the time needed upstream of the projects. During this time, nothing is being produced -in terms of the usual production indicators: publications, patents,... [...] all this upstream time when we have to agree on how the data will be used, to whom it belongs, the type of governance, the level of involvement, the ethical issues, ... for all of these questions, we really need to take the time to sit around a table and come to an agreement, to be sure that everything will then work as it should, so that everyone knows what they have to do in order for the tasks to be truly delegated in both directions. We can't economise on this silent time. [...] That is when is all comes together."

Fabian DOCAGNE, DR Inserm "Science et société" service -Inserm When external and non-scientific actors are included in the research process, more time has to be devoted to the project construction phase, to plan for all of the associated problems that may arise. This planning is essential to ensure that the project runs smoothly, since the involvement of a large number of participants is a constraint to operational management. If the host institution has a specific service for researchers undertaking participatory research, this should be called up as soon as possible during the project design phase. This phase can also involve co-construction where the participants contribute to and facilitate the consideration of the practical constraints involved in carrying out the tasks assigned to them, and which would not necessarily have been identified by the research team. If a project is launched prematurely, this poses a significant risk of not running smoothly afterwards, and consequently a risk of some of the participants becoming less engaged.

ANTICIPATING DIFFERENT LEVELS OF COMMITMENT

Accounting for participant drop out and different levels of involvement

One benefit for the researcher of using a participatory approach can lie in the increased volume of data, because of the high numbers of people involved in the project. However, in all participatory projects, participant commitment declines over time, and some of them will gradually stop contributing at all. In some types of projects, the different tasks can be shared out amongst subgroups of participants. Each subgroup should produce a minimum amount of contributions to obtain statistically significant data. In this case, it is important to anticipate when constructing the project that some of the participants who signed up will then drop out. A wide margin should be allowed between the number of participants v2.9.21-EN assigned to a protocol or a given task, and the number required to obtain sufficient data to be usable. For example, in the case of experimental tasks such as those assigned in the "Derrière le Blob, la recherche" project, a factor of 10 should typically be counted between the number of participants receiving a protocol and the number of data sets actually collected. The greater the investment required to complete the task, the greater the margin to be allowed.

"Volunteers drop out progressively, so that should be taken into consideration. When setting up a participatory research experiment, we should bear in mind that not everyone will be with us until the end, so this needs to be planned for. [...] I planned to send a given protocol to 400 different people, to get at least 50 back in the end. I multiplied by 8, to be sure."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

Sustaining momentum over time

"There are more than thirty scientific cooperation projects to date that have made use of either the expertise, the data, or the samples from CiTIQUE." "60,000 ticks in the collection, 86,000 reports, nearly 300 pupils hosted, 100 citizens [hosted at the laboratory], [...] the success of the project is making some of the actors think about how to sustain it in the long term." Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project A participatory research project involves sustaining a group dynamic to meet the research objectives. The participants contribute to the project by completing the tasks assigned to them, but they also bring their creativity and different perspectives to a research problem. A participatory research project benefits from the participants' emulation and creativity, and from the visibility of the data produced. It opens the way to other research questions and thus new projects to extend and make use of the data produced. Very often, a participatory research project leads to new ideas and new areas of work. The research outcomes of participatory research projects are highly varied: events for the participants, dedicated software applications, educational and training materials, etc... The tools and data produced by a participatory research project can also be used indirectly by the public authorities to assess trends or for monitoring purposes.

"I think that a participatory project tends to keep going, it is hard to put an

"Starting with a project arising from a precise question [...], we have ended up with a project to develop a participatory monitoring and risk prevention platform for ticks. [...] which is under consideration [at national level], which doesn't yet exist but will be based on everything that CiTIQUE has developed. [...] It is a fine note to end on, also in terms of how a participatory research project can contribute to public policy."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project The data management plan is a useful tool to plan for the involvement of non-scientists in the project. It enables the right questions to be asked at the start of the project about the data that will be produced. • What is the scientific robustness of the production protocols.

• What are the minimum requirements in terms of information about the participants to obtain usable statistics? • Is participant anonymity a critical factor?

The data management plan also makes it possible to list the data that is or will be available to the participants and scientists during the project (e.g. raw data, analysed data, photos, sound recordings, etc.), to define the rules for sharing (which data will be accessible and to whom), to decide whether the data will be reused in this project and whether it can be shared with the participants (awareness of ethical issues, cases of personal data for example). The data management plan also makes it possible to define the potential delays relating to making data available (embargos), for example delays needed to validate the data collected (quality control protocol for example), or the existence of constraints due to a partner research project with teams of professionals or a business for example. In the national survey carried out by the "Research Data" college at the Committee for Open Science, 30.7% of the respondents (who were participatory research project leaders) said that they had used a data management plan [5].

"In Bakery, we didn't have this hindsight, we discussed it a lot less. We didn't draw up a data management plan in advance, and I think that this can help to streamline data acquisition and communication."

Delphine Sicard, DR INRAE Coordinator of the ANR "Bakery" project.

It is recommended that a link be maintained between the identity of the contributors and the data they produce, in order to assign dynamic confidence scores to the participants, for example. It is also recommended to save all the history of the data processing and enrichment, with clear versioning, in order to improve the transparency of the process and identify the participants whose contributions are less reliable. These points should be planned for in advance in the data management plan.

Recommendation 3: To use a data management plan, right from the project construction stage to clarify and plan for the specifics involved in producing and analysing data by non-scientists. It is important to be precise and explicit about the roles of the different participants, the stage of the research process that they will be involved in and what information will be available to them. The participants should clearly and unambiguously understand what is expected of them and the associated constraints. They should explicitly commit to performing the tasks assigned to them -including attending training if required. v2.9.21-EN "In a participatory research project, we quickly realised that the main thing was to build the trust of all partners." Delphine Sicard, DR INRAE Coordinator of the ANR "Bakery" project.

"There must be a reciprocal relationship of trust between the project leader and the data producers, and the project objectives must be very clear since that is one of the reasons why the contributors will sign up. There is this trust by the contributor in the project, and in return the researcher must trust in the method."

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre All of the aspects required for the participants to contribute should be clear and explicit [1] • clear missions;

• objectives and/or added value of the participatory approach;

• ability to communicate with the community formed; while checking there are no blocking factors, such as

• overly complex tasks in terms of the initial skills possessed by the participants;

• the overly specialised knowledge required to undertake the tasks requested, requiring an excessive investment in training.

It is possible to produce formal specifications to cover these points, including the objective to be met, the processes and basic tasks, the participant profiles, means of involving them, the stages and synchronisation points of the project and the recruitment process. Once these specifications have been produced, they should be broken down at operational level in terms of the methodology and the technology: which instruments? Which protocols? Which platforms? What level of error tolerance (including malicious behaviour by certain contributors)? What potential biases in the basic tasks? etc...

When the project is presented to the participants, the following items should be made clear • Objectives and scientific problem;

• Hypotheses and study limitations;

• Resources;

• Tasks for participants and their associated constraints;

• Existing data to be used;

• Associated bibliography;

• Expected outcome and impact.

Recommendation 4:

To identify what is expected of the participants and their role in the research process in a clear, formalised manner.

Communication plan

The success of a participatory research project lies in the success of the communication plan in motivating participants and sustaining their commitment and motivation throughout the duration of the project. The driving factors for the contributors may be:

v2.9.21-EN curiosity, the benefits to them in terms of learning new skills, and the sense of being involved in the research process. When designing a participatory research project, it is important to integrate communication as a tool for dialogue throughout all stages of the project [START_REF]Baromètre du numérique -Edition[END_REF]. The data plays an important role in this communication strategy. It is possible to draw up a communication plan and a communication log [START_REF]Baromètre du numérique -Edition[END_REF]. Communication about the project and the role of the non-scientists is also important for participants recruitment.

Recommendation 5: To put in place a communication plan to maintain the participants' commitment. Providing open access to the data produced in the project and communicating the research results can be important parts of this communication.

Press releases and social media are good ways of reaching the greatest number of people to tell them about the project and/or the availability of an application (smartphone or web platform).

3.4.4 Formally define a data policy and the terms of participation in the project.

It is recommended to draw up a data policy (for the whole lifecycle of the data) in the context of the project and the terms of participation in the project to which the participants will give their explicit consent [START_REF]Baromètre du numérique -Edition[END_REF]. This data policy is not a substitute for the data management plan, but it aims to inform the participant via a document that is more accessible to a non-scientist than a data management plan. These documents should be written in language that is clear and accessible to the target population of participants. The rules governing the availability and use of the participatory data should also be presented in a clear and accessible manner. In particular, these terms of participation should precisely describe what is expected of the participants in the project, the intellectual property sharing conditions, how the data produced will be used, whether they will be accessible or not as open data, etc... Finally, these terms of participation should set out the commitments and rights of each party during and after the project. The terms of participation in the project can include a commitment to adhere to a charter. This charter can be written to ensure that the best ethical practices are applied (reporting incorrect use of the data or the instruments) as well as to ensure that the data is reliable (commitment to following the protocols, attending the training sessions, etc.). If the data transmitted by the participants is to be processed, this should be explained to them in an easily understandable way. The commitment of the contributors will depend on their level of trust in the data process. To gain this trust, it is important to be transparent about the how the data will potentially be processed.

Recommendation 6: To formally define a data policy and the terms of participation in the project. These documents should be written in clear, accessible language to a nonscientific audience. The rules governing the availability and use of the participatory data should be presented in a clear, accessible manner.

v2.9.21-EN

THE PARTICIPANTS 4.1 A DIVERSE, NON-EXPERT POPULATION

In a "conventional" research project, a low number of operators are involved in the research process, and they all have scientific expertise in their area of competence (both technical and methodological). Participatory research projects are specific in that they involve a large number of operators in one or several stages of the research process. These large numbers of operators will create additional variability in how the corresponding stage is performed. This variability should be planned for so as to ensure the quality of the whole research process. In general, the majority of non-scientific operators do not have sufficient technical and methodological knowledge to guarantee that the research process will be properly undertaken. Therefore, these effects need to be considered in advance to ensure that a participatory approach is right for the project. Right from the outset, the methodology to be used to form this community of participants needs to be carefully thought through. The tasks assigned to the different non-scientific actors in the project should be clearly identified in order to decide which profile(s) of participants are to be involved professionals in a given trade (e.g. local bakers in the ANR Bakery project); -individuals with a shared characteristic associated with the project (e.g.: chronic disease); -overall population; -primary and secondary school pupils; -communities that have already been formed and identified (e.g. members of an association).

These communities of participants can be more or less homogenous as regards their profiles and skills. It is also possible for the non-scientist participants in the project to be a community of professionals who have a controlled vocabulary and/or significant related technical knowledge. For example, in the ANR Bakery project, bakers were involved in a research project on sourdough biodiversity 6 . Very often however, a degree of variability in the participants should be taken into consideration and planned for. This heterogeneity can be found at several different levels heterogeneity in initial knowledge and skills; -heterogeneity in motivation and involvement in the project; -heterogeneity in digital literacy; -heterogeneity in how precisely the protocol for the task requested is followed.

This potentially multifaceted heterogeneity is an additional intrinsic source of variability in the production of data coming from the task assigned to the participants, which should be planned for and assessed at the start of the project.

v2.9.21-EN

FORMING A COMMUNITY OF PARTICIPANTS

Within the community of participants, it should be possible to accommodate the range of its members, while ensuring the research project objectives are met. Once the community has been identified, it is important to identify the appropriate communication methods, channels and/or relays to get them to commit to the project. The methods chosen to recruit participants should be adapted to ensure that the project runs smoothly and that a group is formed with a well-defined knowledge level and skills. Digital tools can be powerful way of forming communities (especially social media), and also of interacting with them. In 2022, for the whole French population aged 12 years or over, 92% of French people had an internet connection, and 87% had a smartphone [START_REF]Le rapport des Français à la science ET au progrès scientifique[END_REF]. These high levels of connection make these communication channels especially useful when forming and animating a community of participants, by developing either a web platform or a smartphone app. However, it is important to avoid assuming that everyone has a high level of digital literacy, and to remain agnostic regarding the participants' understanding of the various software tools available in order to be as inclusive possible to the greatest numbers of people and not harm the quality and reliability of the associated research outcomes. Schools (primary, secondary) make it possible to form groups of participants with similar profiles which are renewed each year. It is possible to run a joint research and education project in one (or several) partner school(s). It is also possible to form a community of professional but nonresearcher participants. The project can then be constructed with a win-win approach for the two parties in their respective work.

"In projects co-created at a regional level, selecting the group and the participants is critical. The sets of actors must be well understood, and a prior diagnostic study of the regional actors may really help. This issue is frequently raised in work involving co-design and participatory resource management. There is support available at Inrae for research collectives faced with these issues (researcherschools, reflective practice training, animation training, co-development workshops)."

Delphine Mézière, IR Inrae Project manager at Pôle Sciences en Société -Inrae/DipSO For example, in the EcoVitiSol project (on soil quality in viticulture), the researchers give the wine producers personalised feedback on their soil quality. The farmers gift their land and knowledge to the research and in return they are given a personalised soil assessment. With the right means, it may be possible to reach out to "hard-to-reach" audiences, who may have difficulty accessing ways of transmitting scientific knowledge and culture, such as people in hospitals, prisons or retirement homes... It is also possible to select the participants based on their motivation and initial skills in the form of a call for expressions of interest. In this case, the selection process should be carefully planned. It could be based, for example, on the candidates' motivation. They could be asked to fill in an online preregistration form or attend a webinar presenting the project. These stages naturally weed out the simply curious from the highly motivated. It is possible to formally draw up the rules for participating in and contributing to the project, ending with a "signature" (including a digital one) to sign up. This constitutes a kind of pre-selection in order to find the most conscientious and motivated participants. In projects involving a large number of v2.9.21-EN contributors, it is possible to phase the project in several recruitment steps with a gradual increase in the number of participants. The first cohorts can help to finalise the protocols transmitted to the participants, with only a limited number of contributors involved. Once the protocols have been tested, it is then possible to increase the number of participants.

LEVELS OF PARTICIPANT KNOWLEDGE AND SKILLS

The level of knowledge of the participants depends on the kind of participatory project. This should be identified at the start of the project in order to put in place the necessary steps to ensure the smooth running of the project. Depending on their level of knowledge and skills, the participants should form groups or be put into groups in a way that enables them to perform the assigned tasks correctly. Two strategies can be put in place for this purpose

• by establishing a restricted, closed protocol where the operator's initiative is reduced to the basics. In this case, the training is confined to handling the tools to be used to perform the corresponding task or an automation of the majority of the work (automatic metadata collection, autonomous sensors, etc.); • by providing support in the form of training, or detailed and well explained protocols, or a community of users helping to initiate the new participants. This includes training in the technical elements of an experimental procedure, where necessary.

The variation in the participants' knowledge levels may restrict their involvement because of the difficulties of offering suitable training.

Recommendation 7: Offer support and training to the non-scientists that is adapted to their levels of knowledge in order for them to effectively contribute to the tasks assigned to them.

RAISING THE LEVELS OF KNOWLEDGE AND SKILLS OF THE PARTICIPANTS

Training

"Annotation can be very difficult for non-specialists.

[...] We help them so they don't get frustrated." Thomas Lebarbé, Professor at Université Grenoble Alpes

Training can help participants gain the level of knowledge required to be actively involved in the participatory research project. More specifically, the training • provides a means for scientists and participants to interact using shared language and concepts, by preventing any misunderstandings, particularly in terms of vocabulary; • provides the participants with the minimum amount of knowledge required to understand the issues at stake in the projects; • provides the participants with the minimum amount of knowledge required to understand their role in the project; v2.9.21-EN

• provides the participants with the minimum technical skills required to use the tools they will be handling (instruments and/or software); • provides the participants with good methodological practices, where appropriate, to produce reliable data with relevant metadata; • reassures the participants on their ability to correctly perform the tasks assigned to them.

It is important to make accessible training materials available to the participants, and to ensure that the corresponding information is understood.

"Explain what a data distribution is. How, for example, we can sometimes obtain significantly different results with two identical methods, because this depends on the data distribution. [...] These are people with no idea of concepts such as mean, variance and standard deviation. We'll publish a post to explain [that]."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

Training the trainers -trainer communities

It is also possible to train the trainers, meaning that the very active and/or experienced participants in the project can take on the role of trainers for less experienced participants who have joined the project more recently. Assigning dynamic qualitative scores and/or certification badges (such as Open Badges for example) of the participants' knowledge or skills is a means of identifying such experienced people and getting them more actively involved as intermediaries between the researchers and the community of contributors. They then receive indirect recognition of their investment in the project.

Providing support

It is not enough to simply provide training on the tools and tasks assigned to the participants. It is also necessary to contextualise and familiarise the non-scientists with the research subject so they can understand the issues at stake in the project more effectively. This also helps to motivate the participants, who should not be considered as simply a means of implementing the project. The actions they perform in a research problem can only become meaningful if the scientific context is presented in an accessible way. This support can also involve support and/or training for the researchers in science popularisation.

"We often think about training the participants but we shouldn't forget to train the researchers too. [...] It is not always easy to express yourself in a way that is understandable to an audience without their university education [...] It is very important to work on the dialogue between researchers and the rest of society."

Fabian DOCAGNE, DR Inserm "Science et société" service -Inserm

Importance of a wide range of materials

Educational support materials for participants are essential to ensure that they can complete the tasks assigned to them autonomously, with good qualitative implementation.

v2.9.21-EN

The support materials produced should be of high quality and adapted to suit a heterogenous audience of participants. This diversity of the audience makes it important to offer several kinds of materials with varying levels of detail and technicality. This heterogeneity can be taken into account via a quality strategy aiming to ensure that the data produced is reliable and pertinent. Depending on the project, this strategy can either entail assigning different tasks to different participants based on their skills or assigning dynamic confidence scores to individuals linked to the improvements in their skills over time. For example, it is possible to offer two versions of the experimental protocol, one detailed and the other a summary (in the form of a list of bullet points). The participants will also differ greatly in terms of their affinity with the different kinds of material. It is recommended to provide any given piece of information using several kinds of material in order to be as inclusive as possible: written documents, written tutorials, video tutorials, webinars, etc.

"We produced tutorials, either in video form or as slide shows or manuals [...].

For some projects, we organised a half-day training. [...] this mostly serves to reassure the people who attend."

Thomas Lebarbé, Professor at Université Grenoble Alpes

Recommendation 8: To provide a range of different materials to communicate with the candidates -written documents, summaries, checklists, video tutorials, webinars, etc.

CASE OF SCHOOL GROUPS

In the case of participatory research projects with school groups, a range of extra precautions should be taken to ensure that the project is successful. The training can be divided into different levels (teaching staff, pupils, etc.) and can be integrated into an educational project in line with the work of the teachers involved in the projects. This requires some prior preparatory work with the teaching staff. For more information, see reference [1], an in-depth study of this type of project.

"High school pupils are a great target group because the group is renewed every year."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

TASKS ASSIGNED TO THE PARTICIPANTS -PROTOCOLS "If we ask too much, participation goes down. We have to find the right balance between what we need to produce usable results and what the people are willing to provide."

Marta Severo, Professor at Université Paris Nanterre Coordinator of the "Vitrines en confinement" project v2.9.21-EN

Leader of the ANR Collabora project

The protocols that the participants follow should be clear and understandable to all contributors so that they can be correctly implemented (and it is therefore important to accurately assess the level of knowledge of the group formed). The protocols should be established to suit all of the actors (scientists and participants) and take into account the diversity of knowledge and uses. It is possible to find a compromise by setting up a dedicated open working group to undertake a co-construction approach for the protocols [1]. In the specifications, it is important to set out the constraints of the protocol. This document also serves to itemise the tools needed for the participants, including the software tools, and the means of making available the necessary resources, as well as the rules governing how these resources are to be shared.

"People who are familiar with the participants' situations from their own experience should be involved in checking that the contents of the protocol are acceptable and feasible. [...] Ensure that the people who will be signing the protocols have properly understood the instructions. It should be clear to these people, even if French is not their first language, whatever their level of education, even if they are children. There are questions of accessibility for partially sighted people and others."

Fabian DOCAGNE, DR Inserm "Science et société" service -Inserm For example, in the case of a project that involves distributing a sensor to the participants, how should this be done? Should it be made freely available for a certain period or given away for free? How should it be distributed and collected at the end? The protocol should set out clear rules and instructions on the steps to carry out. The parameter to be measured should be reliable and reproducible. The aim is to find a balance between scientific robustness and pragmatism when the protocol is implemented by a diverse group of operators, where there is a degree of disparity in this implementation. It is important to identify the skills that are essential to the implementation and to identify the parts that risk being badly interpreted by non-specialists. It is possible to offer a "simple" protocol to prevent the participants from losing motivation, but with another more detailed protocol also available, with a level of information that matches the participant's level of knowledge, education and involvement in the project. It is also possible in some cases to set a difficulty level for the various tasks assigned to the participants, and to share out the tasks in function to individual levels of competence and understanding (estimated using indicators such as dynamic confidence scores). As the project progresses, identifying outliers and recurring errors can help to detect a problem in the input system or wording of the protocol.

"We used lots of safeguards to ensure that the protocol was as rigorous as possible."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project. v2.9.21-EN

"They [the volunteers] took part in the pilot experiments [...] they helped us to modify the protocol [...], there were things that we didn't consider in terms of conditions at homes [...] and so we were able to improve the protocol."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

The protocols should be tested in advance, for example in the form of beta tests by students or the participants themselves, which is a means of getting them involved in developing the protocols. This can be done by selecting a small cohort of participants who will implement the protocol and help to improve it (via feedback) based on the difficulties they encountered that the research teams had not thought of. Everything that can be framed in the proposed protocol should be framed.

"[...] a great deal of upstream work on the protocols, with blinded validations to check the quality of the results obtained by the participants compared to those obtained by experts [...] which was in fact at the same level of quality."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project

"How to get quality scientific results obtained by citizens in laboratories? For example, in morphological identification [...], I started with the existing identification protocols aimed at scientists [...] and gradually, with the feedback from the citizens, I created an increasingly sophisticated one, and I was able to blind-test it: I compared 200 ticks recorded by citizens with 200 ticks recorded by me in terms of quality of identification: there was in the order of 95% of good matches [the difference was no greater than what is found between two experts]."

Jonas Durand, IR Inrae Responsible for data analysis and enhancement for the "CiTIQUE" project Recommendation 9: To identify whether the tasks performed by the non-scientists require a restricted, closed protocol or the provision of support to enhance skills through training.

Recommendation 10: To assess and test the protocols before giving them to the participants. This prior assessment can be done by students or pools of selected participants. The participants can contribute to co-constructing the protocols.

FACILITATING A COMMUNITY OF PARTICIPANTS

"there must be resources behind [the facilitation]."

v2.9.21-EN

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

"Who is doing the facilitation? How will it be done? What economic model is used for the facilitation?"

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre Animating the community of participants is a key part of a participatory research project. It helps to ensure that the participants remain committed over time. This aspect should also be taken into account to ensure a quality scientific outcome, since it contributes to supporting this non-scientific public in the process of producing knowledge in a broad sense. Communicating, diffusing and enhancing the results are important in sustaining participant commitment to a participatory research project over time. The training materials and protocols should be adapted to the participants' level of knowledge and be distributed over a short timeframe to sustain motivation and commitment. Communicating results is a way of bringing participants together, giving them a sense of co-creating something and being involved in the research process. Generally, this means that open science approaches are natural frameworks for participatory projects. The data should not solely be perceived as raw material for the analysis phase but as a resource to be fully exploited for its potential to communicate and disseminate knowledge [START_REF]Baromètre du numérique -Edition[END_REF].

"participatory research takes time, [...] people want to have genuine contact, see how it works. [...] There must [...] be facilitators available to organise meetings, communicate within the collective and outside of this collective to civil society in general."

Delphine Sicard, DR INRAE Coordinator of the ANR "Bakery" project.

Researchers leading participatory research projects can encounter a range of difficulties in terms of their interactions with the community of participants. One difficulty often encountered is in managing a large number of participants. Very often, these participants want to get involved in a participatory research project to contribute to building knowledge and to come into contact with the world of research and with it, researchers. It takes time to facilitate a community of participants. It is important to foresee the risk of the project researcher(s) being overwhelmed by the demands or of ending up with high human resource investments that are hard to fund. Internet platforms can play an important role in facilitating the community. First and foremost, these tools make it possible to: collect the data; manage its availability in a controlled way; integrate data visualisation tools; publish content that provides information and/or participant training, such as discussion forums and sections for Frequently Asked Questions (FAQ). These platforms serve to make available all of the educational resources needed for the contributors to work autonomously. Thus, such tools can work to easily manage recurrent issues. v2.9.21-EN "all the same, there is still a need to meet up. There have been a few opportunities [...] to join conferences. I think that this is also important for motivation." Nathalie Lambert, social media manager Communications department at CNRS Member of the steering committee of the "Derrière le Blob, la recherche" project.

Experienced participants (who are very active and/or experienced in the project) can also serve as intermediaries to manage the flow of requests for attention and give a real sense of belonging to a community. Assigning dynamic qualitative scores and/or certification badges enables these experienced participants to be identified as intermediaries between researchers and participants. They then receive indirect recognition for their investment in the project. Non-scientific contributors also seek direct contact with the researchers, so this type of organisation should also include events where all of the participants can meet and discuss with the research team.

"In terms of the data, we could envisage a system of accreditation or labelling to give certain people authorisation to access a specific data set. The benefit is that these people could then train others, and that's a lever in terms of HR [...]. The trainers are trained."

Sébastien Payan, Professor at the Sorbonne Université

THIRD PARTY -THIRD PLACE

"Very often, facilitation involves working with other stakeholders."

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre A large number of participants can be managed by involving voluntary third parties (associations, experienced participants, knowledge transfer professionals) and/or by using a tool to facilitate discussions between a large number of speakers. Participatory research implies that the project participants and researcher(s) really do want to meet and interact, beyond simply facilitating the community that has been built. In contrast to the tools and methods described above, these meetings should be an opportunity for an interactive discussion between the researcher(s) and participants, in order to renew and sustain the motivation and sense of belonging. These meetings can be organised via digital videoconferencing tools, or face-to-face, or using a hybrid of the two. University premises (amphitheatres) or the premises of partner associations may be obvious locations for such meetings, but third places for knowledge transfer can also play a role in these participatory research projects, places such as public libraries, media libraries, university libraries, foundations (Maison de la Chimie, etc.) as well as primary and secondary schools, museums and in some cases (clinical research) hospitals, etc. These third places can also serve to distribute equipment for the projects requiring this, or to collect physical samples. v2.9.21-EN Associations also play an important role in many participatory research projects by facilitating the community, or even building this community from their member base. However, it is important to check that the association involved in the project is legitimate, and to be very clear with the (association) stakeholders that when a research question is asked, the results cannot be predicted in advance (this is the very essence of scientific research) and that the result will not necessarily support the cause advocated by the association. Participatory research can also be deployed in the framework of projects in partnership with private sector companies.

"the animators can have interests that are different from those of the researchers'."

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre

Trust is a crucial aspect in sustaining the participants' commitment and interest. In order to guarantee this trust, the participants should be reassured by a guarantee that the researchers leading the project are working independently. If a project is a collaboration with and/or is funded by a private sector firm, it is worth enlisting a trusted intermediary to ensure the necessary guarantees of independence to reassure the participants that their work is independent, and the project conclusions are objective. If a private sector company is funding the project, for example, the trusted third party would be responsible for managing the corresponding funds and would be the contracting party. Thus, although participatory research is the meeting between one or several researchers and a community of non-scientific participants, this relationship should not necessarily be seen as a binary relationship but rather as a triangular one, with the possibility of involving a third place for knowledge transfer, a third party to guarantee independence or a third party to facilitate the community. When a third-party organisation is responsible for facilitating the community, it is important to pay attention to the motivation of this party, which should not conflict with the research objective. It is recommended that it be ensured that the third-party organisation has legitimacy (real and perceived) and is impartial, and that it faithfully represents the target participant community.

Recommendation 11: When a trusted third party is involved in facilitating the community of participants, to ensure that this person's motivation and interests do not conflict with the research process and the aims of the research team.

ACKNOWLEDGING THE PARTICIPANTS' CONTRIBUTION

A lever to ensuring that the participants get involved in a serious, substantial way involves recognising the quality of their work by introducing a form of symbolic reward, and clearly informing them about this when they sign up to the project. It is recommended that it be carefully considered in advance how the participants' work will be acknowledged when communicating the results, including in the form of scientific articles or conferences: citations (named or not) of non-scientific contributors? Acknowledgements? Co-authors? Other? These arrangements should be explicitly communicated to the participants when they sign up to the project. In the case of named citations, their prior consent is obviously necessary. More generally, the returns and benefits of the project should be identified for v2.9.21-EN the participants. They will only fully commit to the project if there is some benefit to them in doing so (cf. section 2.2, page 11).

THE DATA FROM A PARTICIPATORY RESEARCH PROJECT

The data in a participatory research project can be the data shared with the participants as study objects as well as the data produced by the participants through the task delegated to them. Likewise, sharing this data can be designed at several levels: between researchers, with the participants but not openly available, or open to people from outside the project (open data in the sense that are made public).

IMPACT OF A PARTICIPATORY APPROACH

Researchers may express legitimate concerns about the risk of reducing the quality of the data in a participatory research project [1].

"The first [question to ask yourself], is how can you prove to us that the data is reliable?"

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

Various tools can be used to guarantee a rigorous scientific approach. Nine aspects have been identified to guarantee the scientific rigour and quality of participatory approaches [1] 1. Clarity of the research objectives; 2. Construction of a shared problem and language; 3. Quality and transparency of the protocol; 4. Adaptation of the tools and equipment; 5. Reliability and reproducibility of the results; 6. Support for the participants; 7. Respect for scientific ethics; 8. Adaptation of the digital tools to manage the data; 9. Openness and sharing of the results with stakeholder consent.

Participatory research projects, by their nature, benefit from the production of greater amounts of data than a standard project but with the intervention of a large number of nonscientific participants, so it is possible to offset a certain reduction in data quality (greater uncertainty for example) by using statistical tools (mean values, etc.). However, these tools require a certain number of hypotheses to be tested so that the statistical treatments applied are pertinent and able to produce reliable results. For example, several participants can be asked to take equivalent but independent measurements or analyse the same data sets. A statistic on the different contributions then makes it possible to take an average of the errors, providing that these errors are not overly significant. The quantity of data can partially offset the loss in intrinsic quality (notably uncertainty or biased sampling) but for the results obtained by the statistical processes to be reliable, a specifically adapted approach is required.

v2.9.21-EN "On these environmental aspects, the fact of using a sensor or instrument to measure something can modify their behaviour. [...] A bias is introduced by the act of measuring.

Sébastien Payan, Professor at the Sorbonne Université "It is normal to find biases associated with participatory research data, in the same way as they can be found in conventional research data [...], the bias depends on the research question asked."

Jonas Durand, IR Inrae Responsible for data analysis and enhancement for the "CiTIQUE" project "It is not a question of data quality, the issue is whether the data is adapted to the question being asked"

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project For example, we can cite the CiSStats network [START_REF]CiSStats -Statistiques pour les Sciences Participatives / Statistics for Citizen Sciences[END_REF], which brings together applied statisticians, ecological modellers, and stakeholders (associations, natural spaces managers, etc.) who would like to develop statistical methods to make better use of the current and future data sets obtained through participatory research. This data can come from both professional and citizen networks, and have the common feature of being collected by a large number of observers.

"design the protocol well so that it is feasible, and think about the data that will be collected, what will be done with it and who it will belong to." Delphine Mézière, IR Inrae Project manager at Pôle Sciences en Société -Inrae/DipSO

ERROR TOLERANCE

The involvement of a large number of people in the research process will intrinsically generate a higher variability in the data produced, but also introduce a higher rate of error. It is important to assess the acceptable level of error so that the results of the participants' work are usable and can enable the objectives of the research project to be met. For example, in a project using smartphone sensors, a variability in quality must be anticipated depending on the device models (hardware), as well as the conditions of use. A photo can be blurred, with low light levels or low resolution: what impact would this have on data analysis? Is it possible to anticipate this in the protocol given to the participants or by prequalifying the photos with a software tool when they are uploaded? v2.9.21-EN

"The difficulty we have in participatory research projects on air quality and pollutant measurements is that the data is not reliable enough to be used in science today. It is interesting because it is a research subject, but we are not making much progress. There are exceptions [in other projects]."

Sébastien Payan, Professor at the Sorbonne Université It is important to find a means of operational implementation that makes it possible to reconcile the contradictory demands of producing reliable, high-quality and pertinent data in a sufficiently strict working environment, and the demands of sustaining the motivation and creativity of the participants.

"By listening [to citizens], we construct new research questions."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project Systems to manage the errors and uncertainties in the data produced can be applied both before and after the participants' involvement. It is better to focus on an upstream quality strategy: everything that it is possible to do to ensure data quality from the start. If possible, it is a good idea to assess the degree of quality/reliability of the data produced by the participants that can make it possible to obtain useful knowledge in the research process, and to assign dynamic confidence scores to the participants.

Recommendation 12:

To put in place an indicator-based strategy that makes it possible to control the quality and reliability of the participants' contributions beforehand and afterwards.

QUALITY STRATEGY -DATA RELIABILITY

As part of the national survey run by the "Research Data" college at the Committee for Open Science [5], feedback from respondents who had led a participatory research project made it possible to identify different types of strategy used to ensure high-quality data (or high-quality analysis of these data) in a participatory project, some of which had already been identified in reference [1]:

• Produce charters and good practice guides;

• Introduce statistical checks (data triangulation, comparison of observations made by different participants), while setting a statistically acceptable error rate or not; • Assign dynamic confidence scores to the participants that are dependent or not on their skill profiles, which can be automatically deduced from the quality of the responses given previously; • Compare the results from experts with those from participants on a subset; • Rereading by the experts and feedback to the participants (linked to the training); • Define quality criteria;

• Identify suspicious data and introduce expert validation if required;

• Identify the participants who fail systematically (produce suspicious results). • Introduce a system of self-validating the data by the community of users; v2.9.21-EN

• Ensure a sufficient volume of logged data to apply statistical methods;

• Use reasonableness and coherence tests.

In the specific case where the participants undertake the data analysis:

• Propose the same dataset to several participants to compare the results of the analysis; • Propose a known/calibrated dataset (analysed by an expert) to "test" the quality of the participant's work; • Introduce a system of random verification;

• Compare the results from experts with those from participants on a subset; • With transcriptions, compare the results on the same segments of the corpus;

• Analyse the reproducibility of the data analysis (similar data analysed by the same participant).

In the specific situation where participants undertake data collection:

• Cluster the data produced by the participants (mean values, etc.);

• Define quality metrics for data collection using sensors (signal-to-noise ratio for example); In participatory research projects, the quality of the data can be improved by sharing it openly. Alongside the quality control done by cross-checking between the contributors (participant peer review), making their work visible encourages them to take responsibility for the work. The data is shared between the participants, hence a culture of shared high standards is developed among the contributors, reinforced by the fact that the contributions can be visualised in real time. This gives the data visualisation tool a central role.

DATA CREDIBILITY -INSTITUTIONAL CREDIBILITY

The reliability of the data from a participatory research project is ensured by the upstream work on the protocols, the training given to the participants and the quality and variety of the materials and tools made available to interact with the community. However, the reliability of the data is not always sufficient in a project that takes a participatory approach. In a conventional research project, the credibility of the data and thus of the knowledge produced is ensured by the credibility of the research professionals that v2.9.21-EN produced it, as well as by the reputation of the institution associated with and/or funding the project. In participatory approaches, some researchers are legitimately concerned about the quality of the data produced in a situation where the research professionals do not follow the entirety of the knowledge production process. It is up to the project leader to introduce mechanisms that can ensure that the data produced is reliable, as discussed above. Legitimate concerns can also be expressed by the participants or the general public as to the credibility of the data produced and results obtained. It is important to be as transparent as possible about how the raw data produced by the participants will potentially be processed, using language that is clear and accessible to the participants, to build trust. If participants harbour doubts about what will be done with the data and/or whether it will really be used in producing the results, they are likely to drop out of the project. Therefore, it is recommended to  use open source tools, where possible, as guarantees of transparency;  display the participants' contributions in real time -without any delay -where data visualisation tools are used;  save all data (historised if appropriate), but add in comments or labels to any data that is erroneous, incorrectly handled or deemed abnormal according to a pertinent criterion that has been clearly defined and explained to the participants, where this is suitable for the project.

"There are functions to qualify the measurements on the website. They are either normal or abnormal. The criteria have been clearly defined. The measurements can't be changed or deleted, but it is possible to add in a comment."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

While credibility is important to sustain the participants' trust, a lack of credibility can also impact the research team, its home institution and/or funding institution. The credibility of the knowledge produced in a conventional research project is ensured by the professionalism of the professional researchers, which is recognised by their peers, and by their affiliation to an institution which transfers a legitimacy rooted in its reputation . In a participatory project, the data and knowledge produced are shown directly to a nonscientific general public and thereby benefit from increased visibility compared to a conventional research project. If the data is not sufficiently reliable, the reliability and/or relevance of the knowledge produced may be impacted, and it may not even be possible to use the data and/or produce results from it. Once this issue of the results is revealed to the participants and by extension to the general public, this can have a negative impact on the reputation and credibility of the project's researchers and the associated institutions hosting and supporting institutions, funding agencies).

FAKE DATA -TROLLING

The participants can make mistakes when performing the tasks assigned to them. They must be taught about errors in scientific work and how to manage them -mistakes happen, it doesn't matter too much, but the correct action should then be taken, and they should be clearly reported (for example in a comment). It is also worth considering the potential for v2.9.21-EN malicious contributions by people who may want to damage the project for various reasons, by deliberately inputting false or erroneous data.

"if the data is false, there is no way our project can work. We have explained this and published a post about fraud in science. [...] This was taken on board, they understood that idea. [...] And so every time they made a mistake, they reported it."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

To defend the project against this, it is important to maintain the link between the data produced and the participant who produced it. It is possible to identify the participant at fault if a quality strategy has been set up in advance. It is recommended that rules on excluding participants at fault be introduced, or even to co-construct these rules with the participants themselves. The participants should be clearly informed of these exclusion rules (if they exist) when they sign up to the project (in the terms and conditions, Charter of Commitment, etc.). Once the participants at fault have been identified, the corresponding data should be dismissed, but nevertheless saved somewhere (with adapted annotation or separate storage).

"We had virtually no cases of data pollution by trolls. This does exist, but very rarely. [...] It happened during the "transcriptathons" [...]; that was because we set up a reception system in libraries, and there we did get a few data trollsotherwise we had no problems." Thomas Lebarbé, Professor at Université Grenoble Alpes

OPEN DATA

It is generally advised to open up the data since this can motivate the participants, as long as legal and ethical conditions are met [1]. This promotes transparency in the approach and enables people who want to be involved to follow the project's progress. The participants naturally want access to the data resulting from their contributions and the results of the entire collective effort. Opening up data becomes a guarantee of credibility in participatory research. This should be done with materials that are understandable to everyone and accessible to those who are interested -beyond the participants involved in the project -who should be identified (researchers, journalists, decision-makers, communities), and by making use of simple visualisation tools (statistical summaries, tables and graphs).

"There is the reliability of the measurements, but also the credibility of what has been measured and displayed. Very quickly, we decided to upload the data and display it without any prior filtering. That means [...] it was possible to see them in real time." v2.9.21-EN

SPECIFIC CASE OF SENSORS

In the case of participatory research projects involving the use of sensors, it is important to consider these devices and the reliability of the data they produce. It is important to carefully consider the calibration of the sensors used, their continuity over time and their frequency of use. If the sensors are specific and distributed to the participants, it is important to characterise these sensors beforehand, notably their sensitivity dispersion or other relevant parameters, and the overall robustness over time of the data produced.

"All the detectors used were characterised in terms of our reference installations: their response to the dose flow rate and the energy of the particles measured, etc... [...] these sensors are robust, virtually impervious to external parameters and stable over time. That was a deliberate choice."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

In the case of built-in smartphone sensors, it is important to study the robustness of the physical quantity measured beforehand because of the high variability in telephone models and thus corresponding sensor models. Additionally, the environmental conditions they are used in should be assessed (low light, the microphone being partially clogged by dust, the impact of parasitic magnetic fields in magnetic surveys, etc.). For example, a smartphone can be used to collect photos for which most of the useful properties can be obtained in the metadata of the files transmitted. However, measuring sound levels using a microphone or measuring other physical parameters (gyroscope, magnetometer, etc.) is more complicated. The information on the sensor's calibration is not necessarily available, nor are the experimental conditions of the measurements, which can introduce biases. For more information on these problems, please refer to the CAPSA initiative 7 For participatory research projects reliant on sensors, the reliability and uncertainty of the measurements can be assessed at two levels:

it is possible to want to have reliable data at all costs, even if this means targeting the desired information and/or reducing the number of contributions, to achieve usable measurements for an identified research outcome. -another approach consists of collecting data which is less precise for the purpose of defining trends (monitoring, crisis management, etc.).

Analysing data from participatory research projects that use sensors should be done with caution considering the intrinsic biases introduced by a sampling that is determined by the v2.9.21-EN composition of the community of participants. The simple fact that a participant possesses a sensor may change their behaviour -this should be planned for and integrated in the data analysis in line with the research objective.

"It is relatively easy to lock in many of the parameters to obtain a high level of confidence. [...] The most important point is the credibility of the data feedback when using systems with detectors."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project 6 THE TOOLS

IMPORTANCE OF ERGONOMICS

"We did a lot of work on the usability of the tool [...], even for non-specialists because there is a lot to look into: for example, we looked at how the measurement progresses and how it becomes statistically acceptable."

"develop an application where the protocols and usability make the data acquisition more reliable."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project "In the tool, it is important to make space for providing information that was not originally considered [via free text fields], and to modify the tool so that it can search for this information more directly."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project The software tool that the participants will use should consider the fact that they are not scientists, and that they have no contractual commitments. If the tool is not usable by the target population, this can lead participants to lose motivation or reduce the participation potential. It should never be forgotten that the participants have no obligation, other than a moral one, to contribute. It is therefore important to maintain their motivation, notably by providing easy-to-use, pleasant tools for non-scientists. If the budget permits, do not hesitate to employ a UX designer to design the smartphone application or web platform, thereby optimising usability for the project and the target population.

Recommendation 13: To pay particular attention to the ergonomics of the tools for the participants.

v2.9.21-EN

BUILDING TRUST

"To ensure traceability, it is important give a detailed explanation of the whole process -from acquiring the raw data to visualising or analysing it -so that this process is accessible to the participants. That means, above all, working entirely in open source, meaning that all the codes are accessible, so ensuring that the lifecycle of the data is explained and laid out in details somewhere." Sébastien Payan, Professor at the Sorbonne Université It is recommended to use open-source tools. This helps to build a relationship of trust and transparency with the participants. Additionally, developing tools is a costly and lengthy procedure, so it is encouraged to reuse existing tools or adapt tools developed for other projects.

"There is a real need to share in order to avoid reinventing the wheel each time, which is a waste of time and energy. All of the data collection, analysis and visualisation. [...] The question of a charter, paperwork for the participants to sign, the question of prior reflection on the organisation, sustaining participant commitment... if a resource can be identified for each of these items that the project leaders can use, a huge amount of time can be saved."

Sébastien

UPSTREAM TOOLS: DATA EXCHANGE

If participants are involved in data collection, it is best to automate the tool that will produce the data as much as possible, so that the participants' operational freedom is reduced to the essential: date and time stamping, versioning, geolocation (if necessary), participant username, etc. It is recommended to use interoperable standards when these exist. These tools must be easy to use to encourage the greatest number of people to get involved.

"in fact, the greatest difficulty we faced in this project was about data storage, and how to upload the data, how to recuperate the volunteers' data. If a form has to be filled in to transmit the data, the system used should offers the maximum possible fields that are filled in automatically, including boxes to tick or scroll-down lists as often as possible to facilitate the data processing. However, it is useful to provide a free text field (for comments for example), to retain flexibility and find out about situations that were not initially anticipated in the design phase. This free text feedback can also provide a way of updating and improving the tool. The contributors are not professionals, so they may sometimes have doubts or may not be sure of their answers when filling in the form. The tool should therefore enable the participants, when appropriate, to answer "I don't know" in case of doubt -or to not answer at all. The tool should enable every action on a piece of data to be kept track of and be accessible to the researcher, with an appropriate historization and versioning system.

"the questions asked must be sufficiently clear to ensure that there is no ambiguity when they have to give an answer [...] We also gave people the option of answering 'I don't know'. We didn't want to force them to answer if they didn't really know." v2.9.21-EN For example, in the CiTIQUE8 project, about 6% of the contributions were made on paper. One lever to ensure the diversity of participant profiles is to provide a range of different tools, given that there are disparities in digital literacy -as regards both technical skills and habits. It is also possible to give participants the opportunity to develop their own tools by giving them access to the database through an API. Thus, the project can benefit from their creativity, while taking a bottom-up approach to the needs and desires of the community involved in the project. When designing the project, the tool development should include the associated cost in financial and/or human resources terms if it is developed internally, but also for its maintenance, notably for smartphone applications. Diversifying the tools used to collect and process the data can be a lever to manage risk: if one tool malfunctions, becomes obsolete or fails to be taken up by the target community, other tools can serve to maintain participation in other ways.

"to upload the data, there is the smartphone application via an API: the input format on the database is public, so it is possible to upload the data independently of the application. [...] The data can even be entered manually. [ The software tool should enable all data to be saved. If a participant makes an error, this should be tracked (in the form of a comment, for example), but no data should ever be deleted or removed. It is important to teach participants about errors and how these are accounted for in the scientific method. As is the case for the upstream tools, the downstream tools should enable every action on a piece of data to be kept in track of and be accessible to the researcher, with an appropriate historization and versioning system. It should also enable the participants, if appropriate, to answer "I don't know" in cases of doubt -or not answer at all.

"there was a clear expectation and there was not enough feedback to the community itself on what the community was doing." In a project that measures environmental parameters (noise pollution levels for example), when the participants add in a measurement, they should be able to see their contribution immediately on a map available in the application or website, as in the case of the Noise Planet9 project, for example. Data visualisation (and the tools associated with it) is an important but complex challenge in terms of motivating participants and animating the community.

v2

COMMUNICATION AND/OR FACILITATION TOOLS

The community of participants can be diverse, so it is worth providing a wide range of communication channels to encourage the greatest number of people to get involved. The following types of tools can be used:

newsletters; -a dedicated website for the project; -a dedicated web platform (with login/password); -forums; -online data visualisation tools; -webinars.

"There is a third stakeholder who is involved in animating the community. Very often, this is an external partner. [...] The platform also acts as a management tool v2.9.21-EN for this facilitator. There are three issues at stake on the platforms: the user experience (how to make the contributors' lives easier), [...] making the researchers' lives easier in terms of data management (structured, historised) [...] [and] a management tool for the facilitator (user management, news publishing, community management, etc.)"

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre It is also interesting to let the participants comment on the data where possible and appropriate, as a means of reporting errors and facilitating discussions within the community. The comments in the upstream and downstream tools for data collection and processing then become a way of communicating within the community with the data itself as supporting material.

"it is possible to comment on a measurement made by someone else and discuss this with the community."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

LEGAL ASPECTS

The project's data policy should clearly define the ownership of the data produced during the project and be clear as to the role of the participants in the project. The legal considerations in participatory research projects are complex, so it is recommended to refer to relevant legal support services during the project design phase. The specifics of participatory research projects in terms of data rights are also addressed on pages 58 to 62 of reference [START_REF]Droit des données de la recherche -Science ouverte, innovation, données publiques[END_REF]. The participants should have a clear understanding of the rules governing ownership of the data and associated databases:

• Who owns the data collected/produced?

• Who can remove their data from the database?

Recommendation 15: To meet with the relevant departments within your home institution to set out ownership arrangements clearly and formally for the data produced during the project. To inform the participants about their ownership rights for the data produced, where appropriate.

v2.9.21-EN Data -a common good shared between researchers and participants: The data is a lever for interaction with the community. The data in a participatory research project can be the data shared with the participants as study objects as well as the data produced by the participants through the task delegated to them. Opening up the data thus produced contributes to building a relationship of trust with the community of participants, while taking all the required precautions to protect the participants' personal data under the GDPR. It is important to give the contributors clear guarantees on the use of the data, to be explicit as to what will become of them and define clear rules on sharing and citing them. When the data is made openly accessible, it is important to provide materials that can be understood and accessed by all interested persons -beyond simply the actors involved in the project. There are different levels of openness when managing the sharing of the project data:

-data sharing restricted to the researchers; -data sharing restricted to the participants and the researchers in compliance with regulatory constraints (GDPR, health data); -public open access to the data during or after the project.

Reliability, reproducibility -quality strategy: Assessing the acceptability thresholds for the error rates enables a pertinent use of the data and helps to ensure that the work done by the participants is useful and usable. It is recommended to establish a strategy to assess the quality and reliability of the contributions (beforehand and afterwards) and, if possible, to assess the participants based on quantitative indicators in order to attribute dynamic individual confidence scores. These indicators also make it possible to identify the less reliable contributors, including malicious practices where data is deliberately tampered with (fake data). If appropriate, exclusion rules can be put in place for such participants.

The tools: Real-time visualisation of the contributions improves the transparency of the approach and helps to develop a culture of mutual high standards among the contributors. The ergonomics of the tools is important: they should be easy to use by non-scientists to maintain the contributors' motivation and limit input errors. It is recommended to use open-source tools. It is best to automate the tool that will produce the data as much as possible, notably the metadata, and to use interoperable standards where these exist. Providing a range of tools used to collect and process the data (website, smartphone app, paper version, etc.) is one lever for diversifying the profiles of the participants and mitigating risk.

Informing -getting support: The participant should be clearly informed about the ownership of the data produced during the project. The legal considerations in participatory research projects are complex, so it is recommended to refer to relevant legal support services during the project design phase.

APPENDIX -LIST OF RECOMMENDED TOOLS

2. 3 . 2

 32 Open democracy and participatory research -Crowdsourcing and Citizen Science Act(2016)

3. 4

 4 STEERING TOOLS 3.4.1 Data management plan "The data management plan is a very useful tool. It forces us to ask questions beforehand. And, again, it can also be co-constructed." Sébastien Payan, Professor at the Sorbonne Université

3. 4 . 2

 42 Building trust: clearly defining the expectations of the various parties.

Recommendation 14 :

 14 Payan, Professor at the Sorbonne Université "This is an open source and open data project. In terms of the data feedback system, we decided to upload it without any filters, [...] instantly."Jean-François Bottollier-Depois, IRSNOpenRadiation Project Use open source software and prioritise the reuse or adaptation of existing tools for general public use.



  Data management plan  Educational / training materials (with different types of material and different technical levels to be accessible to a wide-ranging public) General terms and conditions if a web platform or a smartphone application is used  Information documents on what is expected of the participants (in the form of a commitment charter, terms of participations, information sheet)  Data policy  Communication plan

  The small farmers, bakers,] truly took part in the research process. That means they co-constructed the research program with us. We designed the experimental protocols together to test the effects of different lands and the varieties of wheat and yeast on sourdough quality."

	1 PARTICIPATORY RESEARCH
	Delphine Sicard, DR INRAE
	Coordinator of the ANR "Bakery" project.

"[

  be used to continue the work and the potential tools to be used beyond the initial project.

	"we also had spontaneous data coming to us right from the start [...]: this was as
	written accounts [...] often letters sent in with the reports [...] people really wanted
	to get involved, share their experiences and their knowledge [...] from the start we
	made preparations [...] to save all of this and number all these resources, and it is
	already being used by colleagues at SHS. [...] This is another resource that we
	hadn't foreseen at the start."
	"Many photos of biting ticks and photos of symptoms were sent in with the bite
	reports. Today my colleagues working in artificial intelligence are using these to
	develop a digital recognition system for ticks, and even for erythema migrans, one
	of the post-tick-bite symptoms, since they need a large volume of photographs for
	the machine learning process."
	Pascale Frey-Klett, DR Inrae
	Coordinator of the "CiTIQUE" project
	end
	to it."
	Marta Severo, Professor at Université Paris Nanterre
	Coordinator of the "Vitrines en confinement" project
	Leader of the ANR Collabora project
	This methodology is specific to participatory research: new questions are generated by the

feedback from participants, and not by taking a top-down approach. This can sometimes lead to totally fortuitous discoveries. Once the participant community has been formed, the question of what to do after the initial project finishes is often raised. This includes the v2.9.21-EN economic model to

  What is the right sample size (geographical, time)?• What rules govern data access?

	• What data is needed?
	• What data already exists?
	•
	20/63	v2.9.21-EN

•

  Analyse the data reproducibility (similar data produced by several participants under similar conditions); • Define coherence indicators if relevant; • Manage missing data; • Multiple transcripts to compare the results of several participants; • Gather metadata associated with the transcripts to test their coherence (date, time, scale of value if appropriate).

	"We use community correction: one person will work on a page, [...] and
	the transcription is only validated once it has been reread and corrected by
	a second user. The first person is then notified of the changes made by the
	second. [...] All changes are tracked."
	Thomas Lebarbé, Professor at Université Grenoble Alpes

  designing a data repository platform.[...] It would have been better to start the project once the platform was up and running."Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project."We insist on historisation. We recommend that the contributors are always given access to the data they produce and that they can modify it [...] rather than only the experts being able to make corrections. And all of this needs to be saved in the memory, every time something happens to the data, whether it is a comment or a modification. If the project leader needs to know, they have access to the entire history of all the actions made on the data: quantifying the learning, the error rate due to data entry, quantifying useful information in terms of quality control, [...]" Romain Julliard, MNHN Professor Director of the MOSAIC skills centre "In cases of crisis [situations], we planned a streamlined operating mode [...] so as to avoid losing uploaded data, since the flow of data is likely to be much higher in times of crisis."

	Jean-François Bottollier-Depois, IRSN
	OpenRadiation Project
	"We also have [participants] who are not part of the digital culture."
	Thomas Lebarbé, Professor at Université Grenoble Alpes

[...]

we had difficulties obtaining 30 TB of storage space, and finding people who had time to v2.9.21-EN get involved in

  ...] We know whether they have has been uploaded via the application or manually." Jean-François Bottollier-Depois, IRSN OpenRadiation Project "We shouldn't just automatically use applications. [...] They need ongoing attention due to changes to iOS and Android, ... and backward compatibility is difficult to manage, as are the Apple or Google updates and new requirements." Jonas Durand, IR Inrae Responsible for data analysis and enhancement for the "CiTIQUE" project 6.4 DOWNSTREAM TOOLS: DATA PROCESSING AND VISUALISATION

  .9.21-EN "we held workshops with representatives of the user communities to find out what they wanted in terms of progressing the functions of the website. [...] What emerged was that they wanted more feedback on what had been done, more discussions within the community. People were expecting more feedback and discussions." Jean-François Bottollier-Depois, IRSN OpenRadiation Project Data processing and visualisation tools can be powerful levers to motivate participants. For this to work, it is important to focus on usability and content. To build a relationship of trust, it is recommended to use open source and real-time data visualisation tools (especially visualisation with maps), wherever possible. "Everything that will happen in terms of quality control between contributors. The first level is relatively passive: we insist on making the data visible and sharing it with other contributors. The data can be seen on the platform. Since the data is immediately shared by peers, the data producers feel a responsibility to produce a high standard of work [...]. The simple fact of sharing the data raises the standard." Romain Julliard, MNHN Professor Director of the MOSAIC skills centre

https://www.openradiation.org/fr/le-contexte, a participatory research project for collaborative measurement of environmental radioactivity levels.

https://www.govinfo.gov/app/details/BILLS-114hr6414ih/

https://www.fs.usda.gov/working-with-us/citizen-science

https://science.nasa.gov/citizenscience

3.1.3 A range of skills in the Steering Committee

https://www.inrae.fr/actualites/biodiversite-levains-pains-qualite

https://caspa.fr/

https://www.citique.fr

https://noise-planet.org/

v2.9.21-EN

v2.9.21-EN

ACKNOWLEDGEMENTS

The authors of this document would like to warmly thank the following people for agreeing to answer their questions: Jean-François Bottollier-Depois (IRSN), Fabian Docagne (Inserm), Jonas Durand (Inrae), Audrey Dussutour (CNRS), Pascale Frey-Klett (Inrae), Emmanuelle Gonzalez (MNHN), Romain Julliard (MNHN), Nathalie Lambert (CNRS), Thomas Lebarbé (Université Grenoble Alpes), Delphine Mézière (Inrae -DipSO), Sébastien Payan (Sorbonne Université, Data Terra), Marta Severo (Université Paris Nanterre), Delphine Sicard (Inrae). v2.9.21-EN

production and analysis of data by non-scientists. participation-scientific-research-defining-field-andassessing-its-potential-informal-science 57/63 v2.9.21-EN

  

General guides

Tutoriels et fiches d'informations-INRAE -Sciences et recherches participatives

https://ist.blogs.inrae.fr/sciencesparticipatives/category/tutoriels-et-fiches-informations/ Vers de bonnes pratiques de recherche participative, Inserm https://pro.inserm.fr/rubriques/recherche-responsable/recherche-participatory/vers-de-bonnespratiques-de-recherche-participatory/

Data Charter for Citizen Science: A basic set of principles to support open and interoperable citizen

science data M. Sterken, R. Capiau. Scivil (2021) https://www.scivil.be/en/book/data-charter-and-guide-citizen-science Data Management Planning for Citizen Science UK Environmental Observation Framework (2020) https://www.ukeof.org.uk/resources/citizen-science-resources/cswg-data-guidance-bookletweb.pdf/view Briefing Paper on Citizen Science Science Europe (2018) https://doi.org/10.5281/zenodo.4925779 Citizen science at universities: Trends, guidelines and recommendations D. Wyler, F. Grey, K. Maes, J. Fröhlich. League of European Research Universities (2016) https://www.leru.org/publications/citizen-science-at-universities-trends-guidelines-andrecommendations , LIBER Citizen Science Working Group (2021) https://doi.org/10./2022/02/social_media_support_citizen_science_april_2014.p df v2.9.21-EN    Sitography Captors and Participative Science https://caspa.fr/ CiSStats -Statistiques pour les Sciences Participatives https://cisstats.mathnum.inrae.fr/ ALLISS -Pour une alliance sciences sociétés https://www.alliss.org/ ParticipARC -recherche culturelle et sciences participatives https://www.participarc.net/

v2.9.21-EN Citizen science and marine conservation: a global review R. Kelly, A. Fleming, G.T. Pecl, J. von Gönner, A. Bonn Phil. Trans. R. Soc. B 375: 20190461 (2020) http://dx.doi.org/10.1098/rstb.2019.0461 Crowdsourcing methods for data collection in geophysics: State of the art, issues, and future directions F. Zheng, R. Tao, H.R. Maier, L. See, D. Savic, T. Zhang et al. Reviews of Geophysics, 56, 698-740 (2018) https://doi.org/10.1029/2018RG000616 Koala Counter: Recording Citizen Scientists' search paths to Improve Data Quality A. Stenhouse, P. Roetman, M. Lewis, L. P. Koh Global Ecology and Conservation, vol. 24, p. e01376 (2020) https://doi.org/10.1016/j.gecco.2020.e01376 Les projets de sciences citoyennes à l'École : quelles pratiques d'enseignement ? S. Perron Review of Science, Mathematics and ICT Education, vol. 15, no. 1, p. 25-43 (2021) https://doi.org/10.26220/rev.3572 http://archive-ouverte.unige.ch/unige:153292 Quantifying data quality in a citizen science monitoring program: False negatives, false positives and occupancy trends S.S. Cruickshank, C. Bühler, B.R. Schmidt Conservation Science and Practice, 1:e54 (2019) https://doi.org/10.1111/csp2.54 Research Data Management Challenges in Citizen Science Projects and Recommendations for Library Support Services J.S. Hansen, S. Gadegaard, K.K. Hansen, A.V. Larsen, S. Møller, G.S. Thomsen, K.F. Holmstrand A Scoping Review and Case Study. Data Science Journal, 20: 25, pp. 1-29 (2021) https://doi.org/10.5334/dsj-2021-025 Taking a 'Big Data' approach to data quality in a citizen science project S. Kelling, D. Fink, F.A. La Sorte et al. Ambio 44 (Suppl 4), 601-611 (2015). https://doi.org/10.1007/s13280-015-0710-4 The Citizen Science Landscape: From Volunteers to Citizen Sensors and Beyond C. L. Catlin-Groves International Journal of Zoology, vol. 2012, p. e349630 (2012) https://doi.org/10.1155/2012/349630 The Critical Importance of Citizen Science Data A. de Sherbinin, A. Bowser, T.-R. Chuang, C. Cooper, F. Danielsen, R. Edmunds, P. Elias, E. Faustman, C. Hultquist, R. Mondardini, I. Popescu, A. Shonowo, K. Sivakumar Front. Clim. 3:650760 (2021) https://doi.org/10.3389/fclim.2021.650760 The use and value of citizen science data in New Zealand M. Peters, C. Eames, D. Hamilton Journal of the Royal Society of New Zealand, vol. 45, n o 3, p. 151-160 (2015) https://doi.org/10.1080/03036758.2015.1051549

research results can be important parts of this communication. the participatory data should be presented in a clear, accessible manner.

v2.9.21-EN "the data is not transformed before being displayed. This helps to build trust. A measurement is a number. But how was it obtained? Has it been corrected? Such questions have a direct impact on trust."

Jean-François Bottollier-Depois, IRSN OpenRadiation Project

If data is made publicly available or shared among the participants, it is important to maintain traceability in the data exportation through the use of licenses. It is highly recommended to apply the FAIR principles [START_REF] Wilkinson | The FAIR Guiding Principles for scientific data management and stewardship[END_REF] to optimise data reuse, and above all, to allow it to be shared.

" [concerning] data quality, there is the aspect of the information they provide and the use that will be made of it. [...] In the first version of the application, all the [users] had access to the tick bite map [...]. We put a stop to this option because it was being badly used, people were taking it as a map of risks [...] but a bite report is not equivalent to a level of risk."

Jonas Durand, IR Inrae Responsible for data analysis and enhancement for the "CiTIQUE" project Data opening should be considered at several levels. In contrast to a "conventional" research project, a participatory research project is structured as a triangular relationship between the project researchers, the participants involved in the project and individuals (both scientific and non-scientific ones) not involved in the project. It is crucial to plan ahead as to what information and data will be made accessible to whom and under what conditions.

INFORMING THE PARTICIPANTS

It is important to be explicit about what will happen to the data. The participants of a participatory research project will be curious and want to know how the data they have produced will be used. Clear rules governing the sharing and citation of the data should be established (data sharing plan). The participants should be clearly informed about the ownership, use and enhancement of the data produced during the projects [1], and the guarantees relating to data use should be explicitly stated. This includes, notably, situations of passive data collection (particularly health data) -which is becoming more common with the growing use of connected objects (IoT) and smartphone applications. It is possible to use such personal data for research purposes, but the participants must be explicitly informed and give their consent. The data management plan is the best tool for thinking about these issues.

LINK BETWEEN DATA AND PARTICIPANTS

The GDPR aspects should be taken into account: it is useful to assign a pseudonym or username to the participants when they sign up, to make it easier to publicly communicate v2.9.21-EN the instructions while maintaining their anonymity. It is also important to maintain the link between the participants and the data they have produced. In order to identify failing participants or assign a dynamic quality score, it must be possible to identify to the participant who produced a certain piece of data. This identification should maintain the participant's anonymity if the data is shared or circulated publicly. This problem can be solved in the majority of cases by creating a matching username (ID#) or pseudonym and a participant's identify. However, particular attention should be paid to participatory research projects where the data and metadata could allow a participant to be identified indirectly. This is notably the case for time-and date-stamped and geolocated data (observation projects with smartphones for example) where a data processing procedure should be applied to ensure anonymity. The data management plan is a good tool to use when planning for these problems.

"we can't write to everyone with a visible list of people's names [to distribute the protocols]. If we were to do it again, we'd come up with a system where people sign up on a platform, and we give them an anonymized name or number [...] and then we could have easily published [online] to distribute the protocols."

Audrey Dussutour, DR CNRS Coordinator of the "Derrière le Blob, la recherche" project.

"In healthcare, we planned to separate the sensitive data and shared data into two different information systems. [...] It is quite revolutionary to share data in medical research. [...] The data is not public but is shared between the contributors, but cannot be associated with a specific person."

Romain Julliard, MNHN Professor Director of the MOSAIC skills centre Research teams from disciplines where managing GDPR aspects is inherent to the research field are familiar with the techniques of anonymising and pseudo-anonymising data. However, the particularity of participatory research projects is that it exposes research teams to personal data issues who do not necessarily have experience of these issues in "conventional" research projects in their field where such problems do not exist. Here, it is important to inform and support these researchers who would not necessarily plan for anonymisation at the start of the project and so run the risk of encountering difficulties in the operational phase. Data sharing can be done at several levels:

closed data that is only shared among the professionals (researchers), notably sensitive or personal data; -closed data that is shared with the participants, but not outside the project; -data divided into sets that are distributed to different subsets of participants in an approach that divides up the tasks separately, or in a quality approach that compares contributions and/or cross-checks these between participants. Each participant only has access to some of the data. A validation process should be introduced before sharing it with all the participants; -open data in the broader sense. v2.9.21-EN Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project "they can say how sure they are [about their answer on the geographical location of the bite]." "The majority of items they had to input were a list of options, i.e., boxes to tick. [...] In the initial phases, it is worth having free text boxes, but these are hard to analyse and clean up."

Jonas Durand, IR Inrae

Responsible for data analysis and enhancement for the "CiTIQUE" project

In projects where the participants have to upload data on a platform, the technical and software aspects become very important. The volumes of the corresponding data should be estimated, and the connection flows anticipated, for example during the sign-up phases of the project or if the results have to be transmitted on specific dates. It is recommended to enlist the help of skilled IT services at the start of the project to anticipate such problems.

If the stages in the project require the participants to connect to a platform simultaneously, it is possible for example to plan for several waves (by sign-up number for example) that are separated over time to limit server overload. In order to encourage the greatest number of participants to get involved, it is important to offer -where possible -several different technical options for participating, for example, a website and a smartphone application.

"The digital gap can introduce a bias into the data collected."

Pascale Frey-Klett, DR Inrae Coordinator of the "CiTIQUE" project "the drawback with paper forms is that they have to be handed in physically. [

..] Around 6% of the reports [were made via paper forms]. [...] Paper forms are generally used by older people [...] That was reassuring, [the fact that] we were able to reach people who would not necessarily have got involved in the project if they didn't have this form of access."

Jonas Durand, IR Inrae Responsible for data analysis and enhancement for the "CiTIQUE" project

Diversifying the tools enables different types of people to get involved. It can be useful, in some cases, to offer the contributors a paper alternative to submit their contribution. Some members of the general public are not inclined to use digital tools (digital gap), and so the paper format provides a greater inclusivity. It allows a marginal section of the population to be involved, and their underrepresentation may have an impact on some projects. However, the data from these paper contributions must be uploaded manually.

v2.9.21-EN

CONCLUSION

A participatory research project aims, above all, to answer a clearly identified research question, by involving non-scientific participants in one or several stages of the process. In a conventional research project, generally only a small number of professionals is involved; they possess the knowledge and skills required to rigorously and reliably carry out the tasks assigned to them. As a result, the data produced can be reproduced by a professional operator with an equivalent level of competence. In contrast, in a participatory research project, a large number of participants will contribute to producing the research data. These participants are not professionals, so there will be variability in the precision with which the assigned tasks are performed; this results from the potential heterogeneity of the community thus formed. This heterogeneity generally takes several forms -diversity of initial skills and knowledge, diversity of motivations and commitment to the project, sociological diversity and diversity in digital literacy skills. This variability can raise legitimate concerns for the researchers regarding the reliability and reproducibility of the research data thus produced. However, by introducing certain methodological precautions, it can help to overcome these problems.

Forward planning: A participatory research project involves non-scientists in potentially high numbers, so the design phase of the project is more important than in a conventional research project. The benefits of a participatory approach should be assessed in terms of the defined research objective, looking at the quantitative and qualitative benefits of the data. Educational resources and information documents should be provided to the participants in language that is clear and understandable to them. The protocols for the tasks assigned to the participants should be assessed beforehand and can even be coconstructed with the participants. The cost of developing and maintaining the software tool used to collect the data should be carefully considered at the start of the project.

Building trust -credibility: Building a relationship of trust with the participants is a way of encouraging long-term commitment. Opening up the data helps to build this relationship of trust, as does transparency about all of the potential treatments applied to the data produced by the participants. Data visualisation is an important issue: the participants must be able to "see" the data and the impact of their contributions. The reliability of the data produced is a major scientific issue, as is the credibility of the institution hosting the project in terms of its ability to produce high-quality knowledge.

Facilitating a community:

The following aspects should be clearly identified: the task in the research process to be assigned to external participants, the stakeholders and their common interests and motives for getting involved. The time-consuming task of facilitating a community is a key factor in maintaining contributor commitment. The participants should receive training in order to correctly perform the tasks assigned to them. The support materials should be adapted to suit a heterogenous population of contributors and written in language that is clear and understandable to them all. It is common practice to involve third parties (associations, experienced contributors, professional organisations, etc.) or third places in animating the community. The interests and motivations of these external actors should not conflict with the researchers' objectives and motivations. For the project to function correctly, these actors must have legitimacy (real or perceived) and be impartial.