Catherine Donnars

Anaïs Tibi

Marc Antoine Caillaud

Regina Dashkina

Agnès Girard

Sophie Leenhardt

Virginie Lelievre

Sophie Le Perchec

Guidelines for the conduct of collective scientific assessments and advanced studies

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

FORWORD

Like any research organisation, the French National Research Institute for Agriculture, Food and Environment (INRAE) has the primary mission of producing scientific knowledge in its fields of expertise: agriculture, the environment, the processing of agricultural products, food and the bioeconomy. Among the many missions resulting from this primary vocation, providing expertise, disseminating scientific culture, and shedding light on public policies and debates are important dimensions of the Institute's activities. Furthermore, society's expectations of INRAE continue to grow as the societal issues raised by INRAE's fields of activity are at the heart of many of the challenges facing contemporary society. The scientific assessments and foresight activities developed by INRA and Irstea since the early 2000s to address these issues are carried out by the Unit for Collective Scientific Expertise, Foresight and Advanced Studies (DEPE), working closely with the executive officer in charge of assessments and support for public policies. The central mission of DEPE is to provide scientific insights to respond to the questions raised by public authorities and society as a whole concerning agriculture and the use of its products, and the environment.

The role of public research organisations in supporting public policy is reflected in various actions, all based on the results of scientific research. Before public policies can be developed and in order to understand the agronomic, biological, environmental, economic and social issues in which public action is intended to intervene, decision-makers and all stakeholders need to be provided with a review of the available scientific knowledge, without overlooking what may be considered as acquired, uncertain, insufficiently documented or incomplete. This is the very principle of the collective scientific assessments conducted by INRAE. Complementary approaches such as advanced studies or foresight studies are intended to deepen or extend these knowledge bases in order to make them even more accessible and usable in the thinking of the various public policy stakeholders. This work requires the definition and implementation of a rigorous methodology shared by all stakeholders in accordance with the National Charter on Institutional Scientific Reports that INRA and Irstea, like their partners in public research, signed in 2010-2011. This methodology aims to ensure the credibility, legitimacy and relevance of the scientific elements provided via this intermediary to the public debate and made available to public and private decision makers.

In addition to the Charter on Scientific Expert Reports, DEPE bases its work on a set of procedures, described in the form of precise and detailed files, but whose format and volume make it difficult to publish in their current form. It has been deemed necessary to make the working methods and rules used to conduct these projects available to the various actors and users of the activities carried out by DEPE (sponsors, partners, scientific experts, stakeholders and various public audiences). A first version of this booklet was published in 2018 to publicise the way in which we conduct this type of complex and sensitive work with all possible rigour and impartiality for a constructive relationship between science and society. A number of reflections within DEPE have resulted in the clarification of certain aspects of the conduct of these delicate projects, and these have been integrated into this second version, which now appears as a new publication.

I would like to thank and congratulate the INRAE DEPE team for the quality of the work carried out over the last few years. Through this sharing of our practices, we hope to improve the understanding of and appreciation for the benefits as well as the limitations of the activities we carry out in this domain.

Philippe Mauguin, INRAE CEO

Introduction: Collective scientific assessments, forecast studies and advanced studies to inform public policy and debate

The mission of INRAE's Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE) is to inform public decision making on complex societal issues and, at the same time, to encourage the Institute to reflect on its own scientific directions.

As developed by INRA since 2000, Collective Scientific Assessments (known by their French initials ESCo) provide public decision-makers and the broader public with the most exhaustive possible review of validated scientific knowledge in response to a complex question. This question is referred to the Institute, and sometimes jointly to its scientific partners (e.g. CNRS, Cirad, Ifremer, Cemagref), in the form of a referral from one or more ministries, agencies, public or semi-public organisations. The referral generally occurs in advance of the implementation of a public policy (possibility of implementing a dedicated policy, adaptation of a regulation, environmental or public health issues, etc.). A committee of scientific experts conducts a multidisciplinary critical analysis of the international academic literature. The aim is to identify the results of research, the outstanding issues, the uncertainties that are still being studied, and to identify the unresolved controversies and gaps in scientific knowledge. A Collective Scientific Assessment thus constitutes an inventory of the knowledge produced by science and identifies future research needs. As conducted by INRAE, it does not go so far as to formulate recommendations to the sponsors.

Initiated a few years earlier at INRA (in 1993), foresight studies seek to inform decision-making with regard to possible futures by developing and exploring contrasting development scenarios at distant horizons, by identifying potential breaks and hypotheses about the future. To be rigorous and credible, it requires a solid foundation of knowledge in the field to be explored and therefore a good understanding of contemporary scientific knowledge and its uncertainties. These are shared within a working group, which is more of a participatory approach and therefore open to stakeholders during the scenario development phase.

More recently (since 2010), advanced studies have been carried out to overcome some of the limitations of the ESCo, particularly when the available scientific literature is insufficient to provide a precise answer to the question asked. These studies draw on sources of information other than academic literature (reports, articles in technical journals, etc.) and involve the processing of data, most often derived from numerical models. The scientific scope of these studies is generally narrower than that of the ESCo.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

10

Public dissemination and debate are essential to the credibility of these assessment activities in the broader sense. This is why, whatever the nature of the project, the results are made public and disseminated as widely as possible in order to inform public decision-making and debate.

While each of these three types of project has its own specific approach, there is a continuum between ESCos, advanced studies and foresight studies. While ESCos always focus on large-scale bibliographic syntheses, they may require the pooling of knowledge from more diverse sources to better contextualise the results acquired and the conclusions to be drawn from results that are otherwise too generic. Two types of study are then positioned as extensions of an ESCo:

• those that integrate data processing with a view to producing original or contextualised indicators from national databases, original simulations carried out using already existing and unmodified models, etc.;

• those which are closer, in terms of their purpose, to the foresight approach and which are based on more or less distant projections of a so-called 'reference' scenario, accompanied by sensitivity analyses via the development of 'alternative' scenarios. For their part, the most recent foresight studies attempt to combine qualitative scenarios for exploring the future with simulations to quantify the consequences.

In addition, ESCos and advanced studies may propose an in-depth and sometimes quantitative analysis of the assembled bibliographic corpus by means of textual analysis, systematic review or meta-analysis methods.

All projects must comply with the four principles of the National Charter for Expertise: competence of experts, impartiality of outputs, diversity of disciplines and approaches, and transparency of methodologies. This publication focuses on the activities of ESCos and advanced studies, whose methodological principles are similar. Foresight studies are the subject of another publication currently in preparation.

This document presents the key elements, in a condensed format, of a guide to procedures for internal use, drawn up by INRA's DEPE and complying with the main specifications of the NF X50-110 standard. It is intended for all audiences interested in the approaches of ESCos and advanced studies, including public authorities, scientists, professionals, community activists, elected officials, etc. This is a second, updated 2023 version (the first in 2018).

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

Table 1 below shows the overall process of an ESCo or advanced study.

The process of examining the ESCo or study project is described in Chapter 1. It begins with the receipt of the request and gives rise to numerous exchanges and discussions within the Institute, with the initial and potential sponsors, and with research partners. The question arises as to INRAE's institutional and scientific positioning with respect to the request, its legitimacy to lead the project, and its scientific interest in the project. The challenge is to choose the most appropriate type of study (between ESCo, advanced study, or even foresight study) and the most appropriate form of management (DEPE, scientific directorate, department, etc.). Depending on the case, INRAE will accept or reject a project, propose reformulations or restrictions on the questions asked, broaden the request, suggest co-management or a scientific partnership, etc.

Two committees are set up during the appraisal phase to support the working group in charge of conducting the ESCo or advanced study: the monitoring committee enables the sponsors to follow the work's progress, and the stakeholder advisory committee informs the actors and stakeholders of the project's progress, as well as gathering their perspectives on the issues associated with the problem under examination. These are essential forums for exchange, which can help to identify useful work (particularly unreferenced reports) or data, and facilitate access to them.

Chapter 2 presents the rationale for setting up the working group. A multidisciplinary committee of scientific experts, specialised in the subject and chosen on the basis of their skills as demonstrated by their publications, is formed. It is led and coordinated by one or more scientific leads and by a DEPE project manager. It also relies on project management and documentary research skills. One of the challenges of leading the committee of experts is to build a shared vision of the questions posed by the 12 sponsors, despite the divergent points of view and arguments that are specific to each scientific discipline. The findings and disagreements must be presented and explained as a result of the ESCo or the study.

Chapter 3 explains the procedures for collecting and analysing links of interest between experts, in accordance with the principles of transparency and impartiality. This includes identifying links of interest at the individual level and analysing links of interest at the level of the expert group.

Chapter 4 deals with the principles for developing and using the bibliographic corpus. The process of collecting, sorting and selecting references, from which the committee of experts will extract the most relevant ones in order to answer the question posed by the sponsors, determines the robustness of the project. Settling on a clear and transparent working strategy is crucial in view of the exponential growth in the number of scientific articles listed in bibliographic databases.

Each project leads to the production of three types of document, detailed in Chapter 5: an extended report (of several hundred pages) which brings together all the critical analyses written by the experts on the basis of the sorted and selected bibliographic corpus; a condensed report (around one hundred pages) intended for decision-makers and, more broadly, for all stakeholders in society concerned and/or interested in the issue (associations, professional organisations, etc.); and a summary report (around ten pages) which communicates the major conclusions of the work to a wider audience. These documents are disseminated at a public feedback symposium, which allows for discussion of the conclusions with stakeholders. The results of the work are also intended to be published in national and international scientific journals to promote their dissemination within the scientific community. The project appraisal and development phase involves discussions with the sponsors, leading to the drafting of the ESCo or advanced study terms of reference and the signing of the agreement between INRAE and its potential partners and the external sponsors. The project officially starts on the date the agreement is signed.

The appraisal first consists of defining the Institute's position with regard to the request (institutional and scientific support), identifying the most appropriate format to respond to the request and the way in which the project will be conducted (section 1.1). It continues with the drafting of the specifications (section 1.2) and the agreement (section 1.3), and the setting up of the committees that will accompany the project (section 1.4). Sections 1.2 to 1.4 apply to cases where DEPE is designated to coordinate the project.

The entire process, which takes six to twelve months, entails many discussions and exchanges between the receipt of the request and the signing of the agreement.

Coordination of the project by DEPE

When INRAE assumes the institutional and scientific leadership of a project, coordination is entrusted to DEPE in the following cases:

• The nature of the request is to shed light on public policies and debates on the basis of syntheses of knowledge, and not on the basis of a collection of expert opinions; • The questions are multidisciplinary, involving both the biological and social sciences; • The academic scientific literature is sufficiently extensive for its analysis to provide an answer to the questions posed.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

1.1.3. Type of project

It is then necessary to define, a priori, the most appropriate form of response (ESCo, advanced study, or even foresight study) to the request. The decision is made on the basis of the following factors:

• whether the scope of the question is rather broad (ESCo) or, conversely, more focused (advanced study); • the coverage of the subject by the academic literature (ESCo) or whether there is a need for extensive use of grey literature (advanced study); • there is an explicit request for the creation of new data (advanced study), which requires simulations, statistical analyses, etc. • there is an explicit request to make projections based on hypotheses of the evolution of the system under study (advanced study).

Development of the request

Developing the project specifications

Once INRAE has decided to respond favourably to the request, a second phase of exchanges with the sponsors is undertaken with the aim of refining the scope of the request and translating it into scientific questions. The outcome of these exchanges is formalised in a set of project specifications.

The exchanges are coordinated by the Director of DEPE. They involve the INRAE Scientific Director who will monitor the project, the representatives of the other institutes involved (if any), and the sponsors. When they are identified, the project leader and the lead scientists also participate³.

This development phase is supported by an initial diagnosis of the available bibliographic material4 . This entails an assessment of the number of existing articles in order to identify:

• questions for which there is an abundance of literature, which may need to be clarified, reframed and circumscribed; • questions on which the literature is limited and which, for lack of sufficient sources, should possibly be excluded from the scope of the project from the outset.

The exploration of the bibliography also aims to identify scientists who publish on the themes covered in the project specifications. These potential experts could be called upon to participate in the project (see Chapter 2).

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

16

In the more specific case of advanced studies, this development phase must also include an initial assessment of the data available as well as the suitability of various processing or simulation methods and tools required for the work to be carried out.

Project specifications

The specifications consist of a document of a few pages drafted jointly by INRAE, its possible partners and the sponsors. The DEPE director coordinates the drafting with the help of the project leader and the scientific leads (if they are identified). The specifications must include:

• the societal, political and regulatory context of the request and the associated issues. e.g. Why has the request been made? How do the sponsors intend to use the results? etc. • the most precise possible description of the purpose and scope of the project (what is included and excluded from the project); • all questions asked by the sponsors, to be examined subject to the existence of scientific literature5 ; • a provisional schedule showing the duration of the various phases of the project • a provisional budget; • the arrangements for monitoring the project (see section 1.4).

These specifications have the status of a 'scoping paper'. At the time of the agreement's signing, they would not have been submitted to the committee of experts since the latter would not yet have been formed. This special status is a compromise that makes it possible to:

• have a stable concept, and therefore start on the basis of an initial document shared by INRAE, its potential partners and the sponsors; • to have a sufficiently precise formulation of the questions, to be able to identify the disciplinary skills to be sought; and • guarantee a certain margin of flexibility for the expert group, whose first task will be to draw up a refined diagnosis of the feasibility of addressing each of the questions posed.

Two pitfalls should be avoided when drawing up the project specifications:

• When the objective of the project is to examine the potential impacts of a phenomenon, the characteristics of the impacts to be included in the scope of the project must be defined. Indeed, an exhaustive list of impacts that might be worthy of analysis according to their nature (environmental, social, agronomic, health, etc.) and their targets (environmental compartments, organisms, etc.) can be a source of misunderstanding when the expert committee is faced with the material impossibility of taking on all the questions posed.

• While it is legitimate for public authorities and stakeholders to want to have a cost-benefit analysis of a set of phenomena with potentially antagonistic effects, such an analysis can only be carried out if it has been the subject of specific scientific work that has already been published.

Drawing up the agreement linking INRAE, its potential partners and the sponsors

ESCos and advanced studies are subject to an agreement between the sponsor(s), INRAE and its possible partners. The agreement defines the purpose of the request made to the Institute, the terms and conditions for conducting the project and the funding allocated by the sponsors to INRAE (and its possible partners) to cover the project's operating budget. The signature of the agreement marks the end of the development phase and the start of the project.

The main points to consider concerning the content of the agreement are:

• The implementation period. This varies from 18 to 36 months from the date of signing and includes further value-adding activities following the feedback symposium (see Chapter 4). • Delivery of intermediate outputs: some payments may be conditional on the delivery of outputs, for example, a progress report. The nature of these intermediate deliverables and their status must be specified in the agreement. They may not concern the preliminary results of the project and may not be distributed beyond the sponsors. • The nature of the relationship between the signatories of the agreement: it does not constitute a commercial relationship, and the various deliverables from the project are public and not the property of the sponsors (see Chapter 4). • The status of the deliverables6 : the leads and scientific experts are responsible for writing the scientific report, while DEPE is responsible for writing the condensed report and the summary report. The condensed report is presented to the sponsors in an almost final draft for verification of its adequacy in relation to the project specifications, but is not subject to their approval in any way. The summary report is also sent to the sponsors for their opinions and suggestions before final validation by the CEO of INRAE and those of its eventual partners. • Monitoring the progress of the project: the role of the committees on which the sponsors are represented is explicitly specified in the agreement (see Section 1.4).

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

18

Establishing the monitoring of ESCos and advanced studies: the monitoring committee and the stakeholder advisory committee

Two committees are established to monitor the progress of the project for the sponsors and to inform the socio-economic stakeholders of progress in the project. These discussion forums can also help to identify additional issues as well as studies (particularly unreferenced study reports) and useful data, and to facilitate access to them.

Monitoring committee

The monitoring committee provides the interface between the working group (see Chapter 2) and the sponsors. It is informed of the progress of the project and of any difficulties that the expert committee may encounter, advises the latter of changes in the political and regulatory context in which the application is made, and takes part in the discussion of the results.

The monitoring committee is composed of representatives of the sponsors, the INRAE executive management as the lead institute (the relevant scientific management and the DEPE director) and the executive management of the potential partners. This committee is set up as soon as the activities begin. Its meetings are chaired by the director of DEPE and prepared with the support of the project leader. It usually meets three times during the course of an ESCo or advanced study:

• When the agreement is signed, the committee discusses and validates the project specifications (see Section 1.2.2). • During the course of the project, the leads and the overall project manager present the progress of the work and any provisional elements. These are not intermediate results, but rather aspects for reflection shared by the working group, which structure the work in progress and may lead to a slight reorientation. The leads and the project manager also report any difficulties that may prompt the monitoring committee to clarify or change certain parts of the questions listed in the project specifications. • Shortly before the public release of the results, near-final versions of the condensed report and the summary report are sent to the members of the monitoring committee for review and comment. The monitoring committee also validates the provisional programme of the feedback symposium on the basis of a proposal drafted by DEPE (see Chapter 4).

The monitoring committee does not validate the content of the ESCo's deliverables or the advanced study. It is consulted for its opinion on the condensed report and the summary report. Its feedback should focus on the clarity of the elements presented and their relevance to the request. The working group remains responsible for following up on the comments and suggestions made.

The monitoring committee applies strict confidentiality rules until the results are published: any document received (draft condensed reports or summary reports, minutes, etc.) cannot be circulated beyond the members who meet in session.

Stakeholder advisory committee

The Stakeholder Advisory Committee is the framework within which stakeholders are consulted about the direction and conclusions of the ESCo or advanced study. It is the forum for expressing stakeholders' concerns and questions about the project.

The composition of the stakeholder advisory committee is proposed by the DEPE director and validated by the monitoring committee (the members of the monitoring committee are themselves members). The stakeholder advisory committee brings together representatives of all stakeholders likely to be interested in the conclusions of the project and to use the results, such as ministry departments interested in the ESCo or the study without being the sponsor, French or European agencies, environmental or consumer associations, local authorities, professional organisations, economic stakeholders in the agri-food sector, scientific interest groups, etc. Each member of the stakeholder advisory committee participates as a representative of the organisation to which he/she belongs and not in a personal capacity.

Like the monitoring committee, the stakeholder advisory committee meetings are facilitated by the DEPE director and prepared with the project leader. The stakeholder advisory committee meets at least twice:

• at the time of the launch of the project, for a presentation to the participants in the demand, involving the issues, the organisation and planning of the project, as well as the framing elements resulting from the reflections of the working group. The objective of this session is twofold: to inform the stakeholders of the project and the methodology on the one hand; and on the other hand to gather their opinions with regard to their respective issues. • at the end of the project, between the final meeting of the monitoring committee and the feedback symposium, for a presentation of the major conclusions of the project. This session allows the results to be shared and the initial reactions of the stakeholders to be collected, which is often useful for preparing the symposium (see Chapter 4). A third meeting can be organised in the middle of the project to report on the progress of the work.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

CHAPTER 2. CONSTITUTION AND ROLES OF ESCO AND ADVANCED STUDY WORKING GROUPS

The working group of an ESCo or advanced study relies on a multidisciplinary committee of 20 to 40 scientific experts specialised in the subject. These experts identify and collect, with the help of information officers, the relevant international academic literature and extract elements that shed light on the questions raised by the sponsors. In the case of advanced studies, they complete this bibliographic synthesis by processing and assembling the data. Finally, they collectively write the scientific report of the project.

The work of the expert committee is co-led by a project leader from DEPE and one or more scientific leads. The leads set the scientific directions of the project, lead the collective production of results, ensure the use of available bibliographic material and knowledge, and draw up the general conclusions. They are responsible for producing the scientific report (see Chapter 5). The project leader is responsible for the overall coordination of the various stages of the project in accordance with the deadlines set and the methods and procedures established by DEPE. He/she is also responsible for the production of the condensed report and the summary report (see Chapiter 5).

The project leader, the scientific leads and the expert committee are supported by information officers, a person responsible for the logistical and financial management of the project, and, depending on the project, one or more project officers (support for coordination, carrying out simulations and data analysis, analysis of technical documents, etc.).

If the analysis requires it, a technical committee can be set up to provide additional non-academic knowledge. It can also be called upon to collect, analyse and interpret data from the grey literature.

Leadership of an ESCo or advanced study

The project leader and the scientific leads together manage the methodological and scientific aspects of an ESCo or study. Mutual trust and a common vision of their respective responsibilities as well as the objectives of the project guide the collaboration between the project leader and the scientific leads.

Guidelines for the conduct of collective scientific assessments and advanced The project leader is a member of DEPE and is responsible for the institutional and functional management of the project. He/she ensures that the deadlines set with the sponsors and the working principles and methods developed by the DEPE are respected.

The production of an ESCo or advanced study depends primarily on the establishment of a sustainable working dynamic. The project leader must contribute to creating and maintaining links between the various members of the working group over time.

Together with the scientific leads, he/she is responsible for:

• identifying the scientific experts (see Section 2.2);

• preparing the expert meetings and contributing to their facilitation;

• proposing frameworks for consideration to the expert committee as the project progresses: analysis framework, report structure, framework for general conclusions, guiding themes for the condensed report, etc. • ensuring that the momentum of the work is maintained from one meeting to the next and that interest in the collective work is maintained; • ensuring that the experts' contributions are written within the set deadlines, and reviewing all the contributions; • checking the quality of the work produced, i.e. that it responds to the questions in the project specifications, that it cites the scientific literature and that it is written in accordance with the procedures.

Although not an expert on the subject, the project leader, with a scientific background, contributes to the development of the results by participating in the compilation of knowledge and the development of the arguments in response to the questions asked. He/she is responsible for writing the condensed report and the summary report.

Scientific coordination: the scientific leads scientifiques

The scientific leads direct the expert committee from a scientific perspective. There are usually two or possibly three, with complementary skills. They are identified on the basis of their scientific achievements and discussions led by the DEPE director with the relevant scientific directors and department heads. Ultimately, the leads are nominated by the INRAE CEO and by his/her counterparts from the partner institutions in the case where the project is carried out by several organisations.

The scientific leads are recognised for their scientific competence, their ability to step back from the questions posed, their broad knowledge of the scientific fields concerned, as well as for their open-mindedness and curiosity and their people skills in leading a group. The acceptance of the leads by the scientific community, and in

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

22 particular by the experts, is essential. Great care must also be taken to ensure that they have no conflicts of interest (see Chapter 3) qwith stakeholders relating to the issue being analysed. Finally, the leads must be reasonably available throughout the project, as their involvement is estimated to be around 30-40% of a full-time position. This estimate varies over time according to the project phase and the number of scientific leads.

The leads are responsible for the project's scientific coordination. To this end, they:

• They ensure the scientific coherence of the arguments developed in each contribution, in each chapter and at the overall level of the report, and make sure that the conclusions are well-founded (which must be supported by the scientific literature and, in the case of studies, by the results of the additional data processing and collation phase).

• They lead the discussion of the experts' individual contributions in order to structure the general conclusions of the work, for which they are responsible for drafting a first version (final chapter of the report).

• They ensure that the experts make a clear distinction in their analyses between knowledge considered to be acquired and stabilised as opposed to more uncertain knowledge.

• They ensure that the report reflects scientific controversies and identifies gaps in knowledge.

• In the case of a study, they define the methodology for processing, assembling data or simulations and coordinate its implementation with the experts concerned.

• They ensure that the experts adopt and adhere to the logic of the summary, structured by the project manager on the basis of the general conclusions of the report.

• Together with the experts, they are scientifically responsible for the content of all deliverables.

The scientific leads also represent the expert group on the monitoring committee, the stakeholder advisory committee and the technical committee if necessary.

Finally, a scientific coordination role continues beyond the public presentation of conclusions at the end of the project, in order to disseminate and promote them as widely as possible. The scientific leads are thus required to present the results of the project to various bodies: scientists (internally and externally), stakeholders, media, professional audiences, associations, etc. They also coordinate the strategy for the academic exploitation of the results, which may take the form of publications and scientific symposia.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

Conducting the ESCo or advanced study: the scientific expert committee

Identification des experts

The experts are scientists identified within French or foreign public research or higher education institutions (researchers, teacher-researchers, engineers). The identification of experts begins at the end of the preparation phase of the project in accordance with four main principles that underpin the project's activities:

• Competence: experts are first selected on the basis of their publications in peer-reviewed scientific journals and on themes consistent with the field of the project. For advanced studies, depending on the nature of the data collection and processing component, their skills in using certain tools or their expertise concerning the necessary data are also considered (use of models, conducting meta-analyses, construction of indicators, etc.). • The plurality of disciplines and approaches: this is reflected in the diversity of the scientific disciplines represented and the diversity of the institutional origins of the experts. Experts not affiliated with INRAE and its partners should, if possible, represent at least 1/3 of the committee in order to avoid a certain institutional homogeneity of approaches. The integration of foreign experts is also desirable in order to go beyond a "Franco-French" approach. • Impartiality: this is assessed at the level of the committee of experts on the basis of declarations of links of interest that each expert is likely to have with different spheres of society and stakeholders affected by the project (see Chapter 3). • Transparency: the principles for setting up the expert committee are presented in this chapter. The mobilisation of experts is carried out in a transparent manner with regard to their scientific hierarchy.

The preliminary exploration of bibliographic databases carried out by information officers during the preparation phase allows for the identification of the authors who publish the most on the themes within the scope of the project. ESCos and advanced studies are, for now, mostly conducted in French. French authors are therefore examined as a priority, as well as foreign authors who speak French or understand French without necessarily writing it, in order to facilitate dialogue within the predominantly French committee. When the sponsors agree, the expert committee may be international, and the entire project is then conducted in English.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

24

Solicitation of potential experts identified from the bibliographic databases is prioritised according to several criteria: • the match between the expert's publications and the topic on which he/she is to be called upon to contribute; • the scope of the topic on which the expert is likely to be competent (bearing in mind that the topic area assigned to a given expert in an ESCo or a study often exceeds that on which they publish); • the expert's experience in writing bibliographic summaries and his/her experience of working collectively; • the availability of the expert (involvement in other projects, responsibilities and assignments, etc.); • the absence of known 'major' links of interest inferred from the expert's membership of certain think tanks, decision-making bodies, etc. (see Chapter 3).

Depending on the nature of the project and the scope of the request, additional skills may be identified. These may include engineers for more technical projects (conducting simulations, developing calculation methodologies, etc.). As they do not necessarily publish in peer-reviewed journals, they do not appear in the preliminary bibliographic exploration. The identification of additional skills therefore also involves the knowledge of internal teams and consultation with the scientific hierarchy. Any identification of additional experts during the course of the project (in the event that a need for additional skills is identified after the project has been launched) must follow the procedure described above.

Ultimately, around twenty experts are usually engaged in an ESCo or advanced study, although some projects may involve larger committees.

Once the list of potential experts has been finalised, the DEPE director seeks the agreement of the management of the experts' home institutions, and each expert is then contacted individually by the project leader and/or the scientific leads. It is important that each expert contacted is aware of the subject and the specific work that will be required of him/her. As soon as the expert confirms his or her participation in the project, two elements formalise his or her commitment:

• the declaration of the links of interest which they may have ( see Chapter 3);

• a letter signed by INRAE's CEO (and if applicable, by the general direction of the partners), which has the status of a mission letter. This letter is copied to the expert's scientific superiors (unit director, head of department and scientific director concerned) to inform them of this engagement.

Guidelines for the conduct of collective scientific assessments and advanced

Roles and responsibilities of experts

In an ESCo, the experts analyse the published scientific literature, extract the acquired, uncertain or controversial knowledge, and detect gaps in scientific knowledge. In advanced studies, in addition to reviewing the scientific literature (certified or 'grey'), the experts conduct data analysis and processing. The experts collectively develop and are responsible for the scientific content of the ESCo or advanced study. Their work requires an openness to the diversity of bibliographic sources and approaches to the collective endeavour. Each expert is responsible for reporting on all perspectives and approaches, even minority ones, as long as they are scientifically sound.

On average, the investment of an expert is estimated at about 15% of his or her working time, with significant variability throughout the project. Each expert is expected to:

• establish, with the support of the information officers, a relevant bibliographical corpus to respond to the issues in question; • read the selected references in full in order to provide answers to the questions posed by the sponsors; • develop, with the other experts, a methodology to process and/or assemble data or even conduct simulations in the case of advanced studies; • write a referenced summary (around 15 pages), thus participating in the collective drafting of the report; • participate in the expert committee meetings (3 to 6 meetings during the project), in the collective discussion of the general conclusions and in the feedback symposium.

Subsequently, the experts participate in the dissemination of the results of the project, and are encouraged to publish them in peer-reviewed journals and further develop them by entering into new collaborations.

When the range of questions posed in the request requires the mobilisation of a large number of disciplinary skills, it is possible to set up expert committees in two circles: a first circle of coordinating experts and a second circle of contributing experts (whose role has already been defined above). In this configuration, the coordinating experts have scientific leadership responsibilities in support of the scientific leads, each coordinating the writing of a chapter of the report. They bring the chapter plan to the contributing experts whose work they coordinate. They are responsible, with the help of the project leader, for bringing together/contacting the contributing experts that they coordinate, to review the progress of their analysis and drafting and to check that the contributions have been properly integrated into the plan. They write an introduction and a conclusion to the chapter that they coordinate.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

26

Support roles

Support roles have several dimensions.

The logistical and financial management of projects is carried out by the DEPE's secretary-managers. These logistical and financial managers organise the material conditions for conducting the work and manage all aspects related to the financing of the project. This work includes the organisation of the symposium.

One or more information officers accompany all phases of the project. DEPE has inhouse expertise in scientific and technical documentation. INRAE information officers and/or those from other organisations are often also involved. The information officers interact with the experts to build up the bibliographic corpus that underpins the project: they develop the bibliographic database search strategy in collaboration with the experts, then provide them with the corpus. At the end of the project, they draw up the final list of references cited in the ESCo or advanced study and conduct a bibliometric analysis.

Finally, depending on the nature of the project and the skills required, one or more project managers, usually recruited on fixed-term contracts, may complete the work group. Depending on the project, they may participate in the general organisation of the project by supporting the project leader in organising meetings, drafting deliverables, etc. They may also provide specific skills to complement those of the expert committee, for example to carry out certain technical/calculative projects to produce quantitative results (preparation and processing of data, conducting of simulations, etc.) and/or to take charge of studies complementary to the analysis of the bibliography (textual analysis of the corpus, analysis of the context of the request based on the technical literature, etc.).

Technical Committee

Some projects may require the involvement of experts who do not belong to public research structures and who, for this reason, cannot be included in the committee of scientific experts. For example, the analysis of the geographical, regulatory, economic or social context in which the questions posed to the experts lie may require the input of technical centres or specific departments of ministries or quasi-public or professional bodies (statistical bodies, experts within agencies such as ADEME, CEREMA, experts from technical institutes, etc.). Another situation that may require the mobilisation of 'technical' experts is the analysis of data not directly accessible to the working group (for example, unpublished results of field experiments).

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

In order to benefit from these skills and/or data while guaranteeing the independence of the expert committee, it is possible to set up a technical committee. This committee provides support to the scientific experts in the discussion of certain choices made by the expert committee, in the interpretation of results from technical and/or field data, in giving an opinion on the choice of situations not described in the bibliography that are interesting to study, and in providing an opinion on the coherence of the work conducted within the 'study' component given their knowledge of the field, etc.

The members of this group participate in a personal capacity and not as representatives of their organisations (institutional representation is part of the stakeholder advisory committee, as described in Chapter 1). Their contributions are formalised in writing, with the expert committee remaining the sole judge of the follow-up to be given to the proposals made by the technical committee. If they are included in the scientific report, the special status of this information in relation to the rest of the elements comprising the report is explicitly stated.

The members of the technical committee do not co-sign the project deliverables and do not take responsibility for its conclusions. The information exchanged between the scientific expert committee and the technical committee remains confidential until publication of the study's results.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

CHAPTER 3. IDENTIFICATION AND ANALYSIS OF THE EXPERTS' LINKS OF INTEREST

Two types of links of interest are likely to introduce bias into the experts' analyses and thus influence the lessons learned and the conclusions conveyed to public decisionmakers: the links between experts and stakeholders in the subject under study, and the scientific links between experts. The principles of impartiality and transparency to which ESCos and advanced studies are subject require that these links be made explicit and that their analysis be transparent.

Considerable attention is paid to the examination of links of interest at the start of the project: first when the scientific leads are chosen, and then when the expert committee is formed.

The approach adopted by DEPE is based on:

• the completion of a standard form by each expert in which he/she indicates his/her links with stakeholders involved in the field under study; • the identification of scientific links between experts and, as a corollary, the assessment of the diversity of scientific approaches within the committee of experts; • an examination of these links by an ad hoc committee under the aegis of the person responsible for ethics at INRAE.

The examination of individual links of interest aims to identify possible conflicts of interest that would lead to the exclusion of the expert concerned. At the level of the committee as a whole, the analysis of links seeks to guard against possible biases tending to orient the collective, to alert to imbalances in the proximity of the experts to certain stakeholders and/or scientific communities working in the field in question. The conclusions of this examination are included in the extended report of the ESCO or advanced study in order to make this information public, thus contributing to the credibility of the study.

Identifying links of interest

As the name suggests, the Declaration of Interests (DLI) is the responsibility of the expert making the declaration. The form used by DEPE is available online at https://hal.inrae.fr/hal-03553051. It is based on the legal framework adopted by the Public Health Code (2012) 7 and the Research Code (2021) 8 . It conforms to the standards of international research organisations. This form is completed by the leads and all members of the working group before the start of the project and ensures transparency regarding links with stakeholders in the field of the ESCo or advanced study over the previous five years. These links may be direct or indirect, and may be in a professional or personal capacity. The expert is expected to take a thorough approach to his or her engagements in relation to the subject matter of the ESCo or advanced study.

The form allows for qualification of the expert's activity: it may be a regular or occasional consultancy activity; the expert may be a member of a board of directors, a steering committee, a steering or evaluation committee, etc. He/she may be a shareholder in a start-up or a company, or hold a patent, etc. Speeches at conferences (or any other event) should also be reported.

Organisations that may be subject to links of interest include companies, associations, trade unions, consultancies, technical institutes, publishers, learned societies, clubs, foundations, local authorities, public and semi-public agencies, etc.

The existence of financial or in-kind support from private bodies is indicated in an appendix which is not made public. This information should include amounts received in the context of research projects or theses (e.g. Cifre contract). The remuneration or compensation allocated to the expert, either directly or indirectly via the institution to which he/she belongs, is also specified.

Research projects involving exclusively public research organisations, as well as assignments with public teaching bodies or agencies, make it possible to identify the scientific communities with which the expert works. Finally, the expert must mention whether any of his close relatives are employees and/or have financial interests in any entity whose corporate purpose falls within the thematic scope of the project.

The DLIs filled in and signed by the experts must be in the possession of the DEPE at the beginning of the project so that they can be examined as soon as possible. DEPE archives all DLIs for a period of 5 years. Subject to regulations (CNIL), they can be viewed on request (with the exception of the financial appendix). Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023 30

Analysing links of interest

The review of links of interest is threefold. The first part deals with individual links of interest. The second considers all the links of interest identified at the level of the expert committee, thus providing a consolidated picture of the risks of bias. The third part maps the scientific networks linking the experts to each other, making it possible to assess the diversity of scientific approaches within the expert committee.

Review of individual declarations of interest

A review panel is called upon as soon as the names of the scientific leads are identified.

It is chaired by the INRAE ethics delegate and consists of at least one member of the INRAE ethics and scientific integrity committee, a representative of the INRAE scientific directorates charged with monitoring the project, the DEPE director and, if the project is conducted in partnership, a representative of each partner organisation. The commission meets twice, once to decide on the leads' declarations and once to examine the experts' declarations. The choice of the leads and the participation of the experts are validated only after examination of their declaration of interest.

Based on the declaration forms provided by the experts, the review panel examines the links that could be detrimental to the balance and independence expected in an expert study. The objective is not to exclude any expert with links of interest with stakeholders, as collaborations with civil society and with professional actors are frequent and encouraged in a targeted research organisation such as INRAE.

Indeed, these links ensure that the expert is familiar with the social, political and economic context of the ESCo or advanced study request.

The panel considers that a conflict of interest exists when the declared links are likely to bias the expert's assessment, influence their freedom of expression or call into question their legitimacy in relation to the expert committee.

A conflict of interest is grounds for removal from an ESCo or advanced study. If in doubt, the commission may ask an expert to clarify his/her declaration. The declaration of interests nevertheless remains the responsibility of the expert.

Analysis of the collective links of interest

While the risks of conflicts of interest are dealt with individually, the committee also looks at the links of interest as a whole. Their diagnosis is based on the analysis provided by the project leader to the panel. Based on the DLIs, he/she draws up an anonymised overview of all of the links of interest in order to present, in a summarised and consolidated manner, the types of stakeholders with whom the committee of experts is in contact, and the respective influence of the stakeholders in these relationships.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

The diversity among the types of stakeholders shows the varied interactions of the experts with the non-academic sphere, which suggests that the expert committee collectively has the means to understand the social and professional issues in the field of the ESCo or advanced study in an open manner. The relative weight of the different categories of actors may highlight either a balance between, or an over-representation of, certain socio-economic sectors in the partnerships. The absence of certain categories of stakeholders may highlight a lack of appreciation or underestimation of certain issues. For example, the absence of consumer associations may suggest that the experts are less aware of demand issues than of supply issues arising from the world of agriculture or agri-food processing.

The number of links per expert gives an indication of the intensity of the experts' involvement in 'field' issues. A high frequency of financial links and their average amount can be an indication of possible precautionary positions taken more or less implicitly towards funders.

Diversity of scientific approaches

The commission also examines the composition of the expert committee and the scientific links between experts. This diagnosis is prepared by the project leader and the information officers involved in the ESCo or advanced study. Its main purpose is to verify the diversity of scientific approaches within the committee.

The description of the committee is based on the following indicators:

• the representation of the different scientific disciplines;

• the representation of experts according to gender and experience;

• the representation of INRAE experts and its partners;

• the representation of international experts.

The analysis of experts' links within the scientific community is documented using textual analysis software applied to experts' publications. These tools make it possible to identify the relationships between authors within the scientific community. The resulting map represents the networks of scientific relationships, whether present within the committee (direct co-publication links between experts) or, more indirectly, via the experts' collaborators (links between co-authors). The diversity of these networks is an indication of the diversity of scientific approaches and schools of thought with which the experts are in contact. The distance between the scientific communities present may indicate divergent theoretical approaches. The arrangements and the respective weight of the subsets may be rather balanced, or on the contrary reveal an over-representation of certain scientific communities, while others will appear to be in the minority or marginal.

The review panel draws its conclusions from these different perspectives. It warns of the risks identified. It may suggest changes in the composition of the expert committee, recommend that additional experts be brought in on an ad hoc basis to compensate for certain shortcomings, complete the stakeholder advisory committee, etc.

A reflective tool for the management of the project

The methodological attention given to revealing links of interest, whether societal or scientific, also contributes to the conduct of the ESCo or the study.

The diagnosis carried out for the review panel is presented at a meeting of the expert committee, so that the experts can integrate it, in a reflective manner, as an element of their own functioning. This mirroring effect has a function of revealing the risks of bias. In a certain sense, the committee becomes aware of its position in the interplay of social and scientific actors. The project leader, the scientific leads and the expert committee can possibly take measures to counteract certain characteristics inherent to the expert group. This can lead, for example, to including stakeholders with whom the experts are not used to collaborating in the stakeholder advisory committee, in order to gather their views.

The conclusions of the analyses can also be communicated to the monitoring committee and the stakeholder advisory committee at the beginning of the project.

Beyond the concern for transparency (the information will be included in the report published at the end of the project), this dialogue serves to raise awareness of the assessment of links of interest and helps to build confidence in the process.

Documentary sources

The methodology of the bibliographic search is as exhaustive as possible, by searching international scientific databases with the help of queries explained in the report. The documentary sources used must be 'certified', i.e. judged to be robust in terms of method (experimental protocol, choice of primary data, etc.) and interpretation of results. Review committees attached to scientific journals guarantee this certification by peer-reviewing manuscripts on scientific criteria.

Academic literature

The international academic literature forms the basis of the ESCo's documentary base. It mainly includes articles from peer-reviewed scientific journals and, to a lesser extent, books and book chapters. International peer-reviewed scientific journals are indexed in international databases (see Box 1). The lists provided by Clarivate Analytics 10 and by the High Council for the Evaluation of Research and Higher Education (HCERES) are a guarantee of reliability.

The major international publishers (see Box 2) of academic books also have peer review committees or quality assessment processes in place for manuscripts.

The importance of international scientific journals differs between disciplines. Academic literature in the social sciences, in contrast to the life and earth sciences, relies significantly on books and journals published at the national level.

CHAPTER 4. COMPILATION AND ANALYSIS OF DOCUMENTARY SOURCES

This chapter presents the methodology for compiling the bibliographic corpus and sets out the principles adopted to guarantee the reliability of the documentary sources 9 on which the results of the ESCo and advanced studies are based. The relevance of the bibliographical references cited in the report determines the credibility of the results.

The ESCo or advanced study report provides a quantitative and qualitative assessment of the final bibliographic corpus.

9 The term "documentary sources" covers all the documents analysed by the experts. The selected sources form a bibliographic corpus. The final corpus includes the "bibliographic references" mentioned in the expert or study report. 10 https://clarivate.libguides.com/ld.php?content_id=48842741. When the available academic scientific literature within the scope of the ESCo or the study is insufficient (poorly contextualised, unsettled or incomplete scientific knowledge), the experts may draw on so-called 'grey' literature11 .

International bibliographic databases

In particular, grey literature includes reports produced by bodies or working groups that carry out expert studies. These include reports from international bodies such as the FAO (HPLE reports) or the work of the European Joint Research Centre (JRC), European or international agencies. 'Grey' literature also includes documents of a much more diverse nature from government bodies (parliamentary and ministerial reports, reports from the Court of Auditors, etc.), the education sector, professional sectors, NGOs, think tanks, etc.

This literature can provide recent information and insights that are useful for supplementing the elements extracted from the academic scientific literature, particularly because it is contextualised and geographically situated. For example, the Agreste summaries produced by the Statistics and Forecasting Service of the Ministry of Agriculture and Food are frequently cited in ESCos and advanced study reports.

The same applies to the literature produced by technical agricultural institutes when there is a need to contextualise the results of experimental or theoretical scientific research.

However, the use of non scientific literature needs to be kept to a minimum for ESCo projects. Grey literature must be validated by the expert committee. It should only be used to illustrate specific questions, and not to support major conclusions. It can be used in a more structuring way in advanced studies which, by construction, deal with subjects less documented in the academic literature.

Body of regulations

However, the use of grey literature should remain ad hoc in the case of an ESCo. Grey literature must be validated by the expert committee. It should only be used to illustrate specific questions and not to support major conclusions of the work. It can be used in a more structuring way in studies which, by their very nature, deal with subjects that are less documented in the academic literature.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

36

Bibliometric analysis published in the report

State of corpus

Exploratory corpus

Initial corpus

Intermediate corpus

Versions 1, 2, 3, 4, etc.

Validated version

Final corpus

Bibliographic queries

Compile the corpus containing all references cited in the report

     
Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

Creation of the bibliographic corpus

The creation of the bibliographic corpus follows a formalised working method based on a search of bibliographic databases using queries, then on successive refinements through exchanges between information officers and the experts. The tracking through the various steps guarantees the transparency of the approach and choices leading to the final bibliographic corpus, which will be cited in the scientific report.

Successive steps

Table 2 presents the general approach, from the first queries of the bibliographic databases to the assembly of the final corpus.

During the initial appraisal of the ESCo or advanced study, the information officers, the project leader and the scientific leads define an initial list of keywords based on the vocabulary of the referral and the main publications to which it refers. The exploratory corpus obtained from the queries permits an assessment of the state of the scientific literature on the subject. This exploration may lead to the refinement of the scientific questions raised by the referral, or even to a narrowing of the scope depending on the quantity of literature to be processed. The exploratory corpus also makes it possible to identify the main authors in the field likely to be called upon to form the expert committee. When the project begins, these initial queries are completed and adjusted based on word lists suggested by the experts according to their themes and disciplines. Particularly relevant articles provided by the experts help to target new keywords. At this stage, the corpus (or the sum of the corpora distributed among experts) is classified as initial, in the sense that it is the result of queries reworked by the entire committee of experts in order to collect as many relevant references as possible for the ESCo or the advanced study.

The experts (or chapter coordinators, depending on the structuring of the project) establish an initial diagnosis of this corpus by highlighting its scope and limits, and make some very general observations about the temporal dynamics, the diversity of research institutions, geographical areas, disciplines and research fields represented. This diagnosis also includes the main features of the strategy identified for selecting the articles to be analysed.

Once this initial corpus has been distributed among the experts, a sorting and refining phase begins. The objective is to select the relevant references in order to best answer the questions in the project specifications. To do this, the experts carry out an initial sorting based on the titles and abstracts of the articles. In the frequent case where the quantity of literature exceeds the analysis capacity of the committee of experts, several strategies aimed at limiting the number of references to be Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

38

analysed are discussed: relate in particular to the nature of the articles (favouring literature reviews and meta-analyses that have already been published and limiting the exploration of primary articles to questions that are not covered by the former) and to the time frame of the bibliographic investigation (depending on the questions and scientific developments, the references may go back 25, 20, 15 or 10 years). In all cases, the ESCo or advanced study must provide an updated overview of current scientific knowledge, i.e. it must shed light on the most recent findings. The experts then gradually refine their selection by reading all the pre-screened references. They amend and complete each sub-corpus through interactions with the other experts and information officers. The refinement of the intermediate corpus thus continues throughout the entire project. The experts may also need to re-interrogate the bibliographic databases using additional queries (more targeted, specific to certain questions, etc.) or to query new bibliographic databases. The corpus thus goes through 'intermediate' states before being finalised.

The final corpus includes all bibliographic references cited in the ESCo or advanced study report.

The main criteria for the selection of references

At the beginning of the project, each expert therefore has a list of references corresponding to the issue(s) he or she is responsible for dealing with. To give a rough idea of the size of the list, an expert generally reviews several hundred publications and selects a range of between 50 and 150.

The following references will be excluded from the corpus:

• not relevant (topic, geographical context);

• methodologically unreliable or not transferable to the context of this project.

The following are not given priority:

• redundant references, in particular certain earlier references whose results are repeated and updated in more recent articles; • primary sources cited in reviews and meta-analysis, the latter being examined as a priority because they already constitute a synthetic analysis of the scientific literature.

Conversely, references not captured by the initial bibliographic queries can be added to the bibliographic corpus:

• references identified in the bibliographies of the publications obtained from the queries; • articles collected by the information officers in the course of the project;

• references known to the experts through their research activities, in particular those from journals not indexed in international databases or from grey literature, as well as references relating to related fields and deemed essential to shed light on the issues under investigation.

Experts are expected to explain their reasons for rejecting or adding references, if possible involving a dual reading of titles, keywords and summaries. These arguments should be validated collectively. In particular, care must be taken to ensure that the process does not eliminate references that would give a complete picture of the scientific controversies on the issue in question. Similarly, additions may tend to broaden the subject beyond the specifications (by going into the field of the expert's research, for example), potentially biasing the answers given to the sponsors. The traceability and explicitness of the choices motivating the list of selected sources are therefore necessary to ensure the credibility of the results of the ESCo or advanced study.

Qualitative and quantitative analyses of the bibliographic corpus

The experts write a general statement on the bibliographic corpus by chapter: main subjects dealt with in the scientific literature, main journals publishing on the issue, most cited authors or institutions, existence of authoritative works in the field, geographic settings of the studies, evolution of the way in which science has dealt with the given questions, identified controversies or gaps, etc.

A rigorous, common reading grid

The critical examination of the documentary sources follows the rules of scientific work. For each article selected, the experts assess the robustness of the methodologies, the suitability of the analytical tools used, and the rigour of the methods of interpretation and discussion of the results. Each expert defines this review framework according to the prevailing standards in his or her scientific discipline. The analysis of grey literature requires increased rigour and collective validation by the committee of experts.

In addition to these standard rules for analysing scientific literature, there is a specific reading grid for expert assessments: experts must clearly distinguish, among the scientific findings, those that can be considered as acquired, i.e. currently established and agreed upon. They must point out the uncertainties and variability that may affect certain results or conclusions. They must explicitly explain disagreements between disciplines and authors. Finally, they must identify the gaps in scientific knowledge in relation to the questions posed by the sponsors. The review of the literature in an ESCo or advanced study is part of a progressive and collective approach to assessing scientific results. In this sense, it differs from systematic review or meta-analysis approaches. Indeed, the latter rely on a process of systematically screening publications using a pre-established analysis grid, which provides guarantees of transparency and rigour. However, the implementation of such an analytical process is generally not sufficient to meet the expectations of an ESCo or advanced study given the broad nature of their scope, questions and criteria for analysis.

Use of bibliometric analysis

The rapid increase in the number of scientific articles published each year increases the difficulty of the corpus selection process as well as the critical synthesis work required of experts. Automatic processing methods can therefore be used to describe and analyse the corpus. The descriptions are based on information referenced in bibliographic databases.

The WoS platform provides tools to analyse the frequency of certain indicators such as the thematic fields of journals, the titles of publications, the authors and their areas of affiliation, institutions, funding sources, etc.

ITextual analyses can be performed for scientific articles by looking at words in the title and abstract, keywords, authors, their institutions of affiliation, etc. and performing statistical analysis. Time sequence analyses offer a retrospective look at how the literature is evolving. The results are usually represented in graphical form, which allows the characteristics of a large number of texts to be quickly visualised.

Using measures of term co-occurrence, it is possible to construct collaborative networks between authors or thematic networks. DEPE is currently using WoS tools and the digital platform CorTexT Manager 12 , which combine language processing, information extraction, complex network analysis and scientometrics.

Bibliometric analysis tools are used for several purposes, depending on the stage of the project. At the start of the project (initial corpus), they are useful for highlighting the main features of the scientific literature on the subject to be dealt with in order to initiate the discussion within the committee of experts. They can also facilitate the distribution of the overall corpus among the experts according to the topics they are to deal with. The analyses can also lead to new bibliographical requests or more detailed investigations on a given aspect. As the work progresses, the bibliographic analyses make it possible to objectify the evolution of the corpus and to identify

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

any selection biases in terms of both themes and communities of authors. Particular attention is paid to self-citations and the relative share of the different networks of authors cited.

At the end of the project, bibliometric analyses applied to the final corpus provide key information on the sources on which the ESCo or advanced study is based. They represent a significant result from the ESCo or advanced study, as they shed light on the current state of scientific knowledge. Conducted by the information officers and included in the final report, the analyses cover the following items:

• the category of documents: academic articles, reviews, conference proceedings, statistics, reports, etc; • the temporal distribution of the cited references (it is expected that they are as up-to-date as possible). This analysis can be further developed by describing the temporal evolution of research questions on the subject; • the allocation between themes and the number of references per chapter;

• the principal authors cited by chapter; • the comparison between the final corpus and the initial corpus, with regard to the main reasons for the orientations, deletions or additions .

Finally, the keywords and/or thematic descriptors of the references are indicated (usually in the appendix of the report) so that the queries can be reproduced.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023 42

Extended report

The drafting of the scientific report is the responsibility of the scientific leads and the experts. The outline of the scientific report is developed collectively by the experts.

Each expert writes a contribution that provides a critical synthesis of the elements that they have extracted from the bibliographic corpus that has been assigned to them.

CHAPTER 5. PREPARATION OF DELIVERABLES AND DISSEMINATION OF ESCO RESULTS

AND ADVANCED STUDIES

ESCo and advanced study deliverables do not include opinions or recommendations, unlike those of risk assessment agencies such as health agencies, which are usually required to issue them. These studies yield three types of deliverables, as shown in Figure 1: 1. the scientific report, which is often extensive (500 to 1 000 pages), brings together all contributions and critical analyses written by the experts on the basis of the bibliographic corpus (Chapter 4) as well as the list of references cited. In the case of advanced studies, it also includes a description of the data processing and assembly methodologies developed by the experts, as well as details of the results of their implementation; 2. the condensed report (approximately 100 pages) presents the main findings and results of the report. In addition to those who commissioned the project (political leaders and decision-makers, ministry or agency officials), it is intended for all societal stakeholders concerned and/or interested in the issue (associations, professional organisations, industry stakeholders, etc.); 3. the summary report (usually around ten pages) communicates the main conclusions of the work more widely.

Deliverables can be checked with anti-plagiarism software.

The results and conclusions of ESCos and advanced studies are always made public, and are discussed with stakeholders at a symposium open to anyone. The condensed report and summary reports are then disseminated. Finally, the scientific exploitation of the results in the form of academic publications is also a major objective of these projects, and is envisaged and planned from the outset of the project.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023 A contribution must explicitly summarise (i) the current state of knowledge, indicating the level of evidence or uncertainty, (ii) the scientific debates identified in the literature, (iii) and the gaps in scientific knowledge in relation to the issue explored. The review concludes with key findings from the analysis and the formulation of possible research and/or data needs. In the case of advanced studies, the bibliographical analysis is accompanied by specific statistical analyses or simulations carried out in addition to the bibliographical analysis. The project leader and the scientific leads coordinate the process of drafting each contribution and ensure, together with the information officers, that each statement is well supported by evidence.

The extended report generally adheres to the following standard structure:

• The introduction, which may be extended by a framing chapter, presents the context of the ESCo or advanced study request and the issues that motivated the request. It also specifies the work process, the composition of the committee, the results of the examination of the links of interest and the highlights of the bibliometric analysis (number of references, most cited journals, etc.). The introduction is usually handled by the project leader and the scientific leads. • The following chapters are organised around the scientific issues addressed by the project. Each chapter begins with a review of the bibliography and concludes with the key aspects of the various contributions to the chapter. Each expert first writes a short summary which is discussed at the expert committee meeting. On this basis, the expert writes a contribution of approximately 10 to 20 pages with explicit links to the references selected to support the analysis. Cross-reading of the contributions by the project Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

44 13 In this case, releasing the English version of the condensed report may prejudice the submission of a scientific paper based on it.

leader,the scientific leads, the other experts and the information officers results in the content being reworked until it is finally formulated. • The conclusions of the report, which generally take the form of a final chapter, offer a transversal overview of the analytical chapters in response to the sponsors' request. The conclusions are therefore expressed in a concise manner (in the order of 10 to 20 pages) and refer to the various sections of the report as necessary. The scientific leads generally draft the first version of the conclusions, which are eventually validated by the entire expert committee. • The appendices include the bibliometric analysis of the cited corpus as well as any other methodological contributions, data or complementary analyses. • The report includes a description of the entire work group (names, contact details, roles): scientific leads, members of the expert committee, possible one-off scientific contributors, the project leader, information officers, managers, and other participants.

There are two ways of citing an extended report depending on the manner in which it was produced:

Condensed report

The condensed report goes beyond simply presenting the general conclusions of the extended report. It also provides an account of the approach and the more sectorspecific conclusions of the analysis. It can also introduce generic elements that are useful to help the non-specialist reader understand the reasoning presented. The condensed report should be readable without the need to refer to the extended report. In fact, it provides an entry point to the extended report, with the reader consulting the sections of the latter that he or she wishes to explore further. The synthesis cites a limited number of bibliographical references.

The condensed report is published in French and English, except in cases where the experts publish a peer-reviewed scientific article, based on the condensed report, in an internationally refereed journal 13 .

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

46

Summary report

As a medium for institutional communication, this short document (about ten pages) presents the issues surrounding the request and the principal findings of the work. The summary report is distributed to symposium participants and journalists. It can be used by INRAE, its partners and the sponsors in their communications.

Its content must be approved by the executive management of INRAE's and its potential partners and by the sponsors (in addition to the experts). The production of this document therefore requires numerous iterations between the various parties. However, the sponsors do not have a 'right of veto'. The President and the Chief Executive Officer of INRAE are responsible for the final validation of the document.

The summary report is written by the project leader in close collaboration with the scientific leads, and makes explicit reference to the extended and condensed reports.

It is always translated into English, edited in this form and posted on the institute's.

The summary report is cited as follows: IINRAE -possible partners (date). Title of the summary report. Summary report of the ESCo/study, INRAE -possible partners (France), xxx pages.

Feedback symposium

The conclusions of an ESCo or advanced study conducted by the DEPE are made public at a symposium (generally organized over half a day or a day). The symposium meets two requirements: to respect the principle of transparency stated in the charter of scientific expertise, and to fulfil INRAE's and its partners' mission to contribute to public debate and participate in the dissemination of scientific culture on the themes that are the subject of these studies.

The program of the symposium is established by the project leader, the scientific leads and the director of DEPE, in collaboration with INRAE's executive management and its possible partners, and the sponsors. The symposium is generally chaired by an external speaker (scientific journalist or scientist from outside the working group).

It is organized around two central sections: (i) presentation of the work program and the results by the project leader, the scientific leads and members of the project team;

(ii) discussion of the conclusions during one or two round tables made up of representatives of stakeholders directly concerned by the ESCo or the study.

The introduction, generally given by the sponsors, is an opportunity for them to explain their expectations regarding the requested project and the issues they are facing. The conclusion is usually delivered by the INRAE President and CEO accompanied by his counterparts from the partner institutions if applicable, and is intended to open the discussion to outstanding questions requiring new knowledge to resolve, and calling for new research directions.

The symposium is open to all, whether face-to-face or remotely, and is free of charge. It is publicised as widely as possible in order to bring together all potentially interested parties: professionals, civil servants from ministries and decentralised government departments, researchers, teachers, students, community organisations, elected representatives, etc. The symposium is videoed, and the videos are posted on the Institute's website a few days after the event. A simultaneous English translation of all discussions is provided and posted on INRAE's English website.

Academic outputs from ESCos and advanced studies

Although posting the results on the INRAE website guarantees their availability to all, it provides only limited visibility to the scientific community. The publication of the condensed report in the form of a book in both paper and digital versions (PDF and e-pub) by Editions Quae offers an opening to publishing platforms. International visibility is essentially achieved through the publication of all or part of the results in international scientific journals, with a preference for open access publications. Several articles can be grouped together in a special issue.

Peer-reviewed scientific publications provide assurance of the scientific quality of the work. This scientific validation is a guarantee of credibility recognized by the sponsors and stakeholders. Experts are therefore strongly encouraged to publish their work in international scientific journals. Although publications are the responsibility of their authors (and as such do not appear contractually in the ESCo and advanced study deliverables), they are the subject of careful thought by the expert committee from the outset of the project.

Archiving of documents from ESCos and advanced studies

In order to ensure traceability of both method and content, INRAE's institutional archiving policy defines the list of documents that will be kept for 20 years at DEPE, before being transferred to the National Archives. The archived documents are presented in Appendix 1.

Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

CHAPTER 6. CONCLUSIONS

For the past twenty years, INRA and IRSTEA, which became INRAE in 2020, have been helping to clarify public policies and public debate by conducting expert assessments at the request of public authorities. These projects are entrusted to a dedicated INRAE department, the Direction of Collective Scientific Assessment, Foresight and Advanced Studies (DEPE), which examines requests, and coordinates the execution of the projects, production of deliverables and dissemination of the results. Although scientific expertise has never been accorded so much importance in public decisionmaking, a distrust of science among part of society is growing in a climate of posttruth. It is therefore essential that the methods used to produce an assessment be transparent, so that everyone can assess the strengths and limitations of a collective scientific assessment. This is the purpose of this document, which is intended for all stakeholders, whether scientists, public authorities or citizens. It should help the reader to understand (i) the examination of the request from the public authorities, (ii) comitology, the rules for setting up committees, (iii) the construction of the scientific knowledge base and the way it is analysed, (iv) the production of the various deliverables and their public release. It highlights the fundamental difference between assessment based on the words of experts and that based on a collective analysis of scientific knowledge as practised in the framework of a collective scientific assessment. This is particularly important in view of the multiplicity of forms of assessment that are developing in society outside research institutions: think tanks, environmental or consumer associations, etc. For its part, collective scientific assessment must (i) establish a relationship with all stakeholders to meet its objective of informing public debate, (ii) respond to questions concerning the impartiality and links of interest of experts, and (iii) deal with the explosion of scientific output. This context requires us to reflect on and constantly update our assessment practices, which is why we are updating this document with the aim of it becoming a reference document in public debate.

APPENDIX 1. ARRANGEMENTS FOR THE RETENTION AND ARCHIVING OF DOCUMENTS FROM DEPE PROJECTS

-Request letter (optional document) -Agreement with the sponsor(s) and its appendices (including the project specifications) -Minutes of the monitoring committee meetings (preparatory documents in the appendix) -Budget -Correspondence with experts:

• Letters of assignment for the scientific leads and the experts • Letters of thanks -Validation of experts:

• Experts declaration of links of interest • Minutes from expert validation committees -Full expert meetings: dossier for each meeting:

• Agenda • Preparatory documents (experts' advance contributions)

• Minutes of the full meeting -Minutes of the stakeholder advisory committee meetings (for some projects) -Minutes of the technical committee meetings (for some studies) -Extended reports -Condensed reports -Summary reports -Bibliographic databases -Final feedback symposium: slideshows of presentations and video footage (produced by the Communication Department) -Press coverage -Follow-up of the promotion of the project (scientific publications, etc.) Retention period: 20 years from the end of the project (5 years for the declaration of links of interest of th experts).

Management of files at the end of the retention period at INRAE:

Transfer to the National Archives, except for the declarations of interest, the preparatory documents and the budget, which are destroyed.

Comments:

The originals of the agreements are on paper

Figure 1 .

 1 Figure 1. Schematic flow of the preparation of the extended, condensed, and summary reports of an ESCo or advanced study

DEVELOPMENT AND MONITORING OF AN ESCO OR ADVANCED STUDY

	ESCos and advanced studies are conducted in response to external requests. This
	demand mainly comes from public bodies (ministry divisions, agencies such as ADEME
	or OFB), but may also come from private bodies (technical institutes, professional
	associations) or those recognised as being in the public interest if the question raised
	concerns public policies.
	ESCos and advanced studies may be conducted in partnership with one or more other
	public organisations with scientific expertise: research organisations, and sometimes
	agencies whose missions include research.

All ESCos and advanced studies conducted by INRA (and later INRAE) since the early 2000s are presented in Appendix 2. Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023 CHAPTER 1. The sponsors finance the operating costs of the project¹; the salaries of the DEPE staff and the experts mobilised remain the responsibility of INRAE or their parent organisation². ESCos and advanced studies conducted by DEPE therefore differ from (full-cost) services that can be commissioned from a consultancy firm. The scope of work for an ESCo or an advanced study is developed jointly by the commissioning parties and INRAE. The results of the ESCo and the studies remain the property of the Institute, and the sponsors cannot restrict their dissemination, i.e. free access to all documents resulting from these activities (Chapter 5).

Receipt of the request by INRAE

	1.1.1. INRAE's institutional support for the project
	Upon receipt of a request to the Institute, an initial phase of reflection is initiated to
	clarify the subject and identify the interest and legitimacy for INRAE to carry out the
	project at the institutional level. The Director of DEPE analyses the request with INRAE
	scientific Directors and, where appropriate, with the representatives of other research
	organisations involved. The following reasons may lead the INRAE general direction
	to decline the project:
	• The subject is already being addressed by another body or organisation,
	and/or does not fall directly within INRAE's scientific fields of competence.
	• The sponsor(s) and INRAE (and any other organisations involved) do not
	reach a satisfactory agreement on the referral;
	• The subject involves questions that have not yet been fully explored by the
	scientific community. The available scientific knowledge is not sufficiently
	complete or robust to be the subject of a relevant synthesis.
	• The request is of scientific interest but of little interest to society as a whole
	(the issues and concerns underlying the request are not motivated by the
	general interest and/or are not the subject of a public policy or debate to
	which an ESCo or advanced study would contribute).

The terms of reference, which are co-constructed and validated by INRAE, its potential partner(s) and the sponsor(s), are attached to the agreement. Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023 14 1.1.

  Unit for Collective Scientific Assessment, Foresight and Advanced Studies INRAE -Unit for Collective Scientific Assessment, Foresight and Advanced Studies

	INRAE -
	studies -2023 -2023	25

of Science™ Core Collection (WoS) : scientific

  online platform launched in 2001 by the CNRS, which provides access to publications from French and foreign teaching and research establishments and public and private scientific laboratories. Access is free, and INRAE has its own HAL-INRAE portal: https://hal.inrae.fr/ information platform managed by Clarivate Analytics (originally by Thomson Reuters). Since 1955, it has referenced more than 20,000 journals, mostly in the fields of science and technology, but with limited coverage of the social sciences. • Scopus : trans-disciplinary bibliographic and bibliometric database launched by the scientific publisher Elsevier in 2004. The first references date back to 1970. Themes include physical, medical and life sciences, as well as social sciences and humanities. database containing Medline, but enhanced with references not yet indexed in Medline https://bib.umontreal.ca/guides/bd/pubmed or https://www.nlm.nih.gov/bsd/difference.html database of journals and books in the humanities and social sciences:

	https://www.cairn.info/
	• CAB Abstracts® : database published by CAB International and specialised in applied
	disciplines relating to the life sciences. More information at:
	https://www6.inrae.fr/reselec/Bases-de-donnees
	2 :

• HAL open archive: • Web • Food Science Source : platform focused on food science and agriculture. • PubMed : • EconLit : database focused on economic literature. • Repec (Research Papers in Economics) : bibliographic database of research in economics: http://www.repec.org/ • CAIRN :

Examples of publishers whose works can be considered as certified academic literature

• Cambridge University Press http://www.cambridge.org • CABI Publishing http://www.cabi.org/ • Wageningen Academic Publishers http://www.wageningenacademic.com • John Libbey Eurotext http://www.jle.com/fr/index.md • Springer http://www.springer.com • Wiley-Blackwell http://eu.wiley.com • Elsevier http://www.elsevier.com/wps/find/authors.authors/bookauthorshome • Quae https://www.quae.com/ 4.1.2. Technical literature

Table 2 .

 2 Stages in the compilation of the corpus bibliographic

	Use of corpus
	Distribution of the corpus
	among experts according
	to disciplines and issues
	addressed
	Bibliometric analysis
	of the corpus of each
	chapter to inform
	the discussion within
	the expert committee
	Qualitative diagnosis
	of the bibliographic corpus
	of the different chapters
	by the expert committee
	Validation of the bibliographic
	corpus of the different chapters
	by the expert committee

  These four dimensions (achievements, uncertainties, controversies, knowledge gaps) are considered necessary by DEPE to fully clarify the state of scientific knowledge

	40	Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023

12 https://www.cortext.net/projects/cortext-manager/

Costs related to staging meetings, publishing documents, organising public feedback symposium and, where applicable, the remuneration of a project manager contracted for the project.

As a result, the funding provided by the sponsors generally represents a maximum of around

30% of the total cost of the project.

Reflections aimed at identifying potential scientific leads can begin quite quickly, as soon as the decision is made to have INRAE lead the project at an institutional level.

Mainly in the Web of Science or Scopus, and if necessary in databases specific to certain disciplines.

The bibliographical exploration carried out at the time of this appraisal phase only provides an overview of the available literature; it does not guarantee the existence of literature on all the issues of concern to the sponsors.

Expertise is provided by INRAE and any partner institutions.

According to AFNOR, grey literature refers to any 'typed or printed document, often of a provisional nature, reproduced and distributed in fewer than a thousand copies, outside the commercial publishing and distribution channels'.

Using plant diversity in agricultural areas to promote natural pest control and protect crops May 2023 Tibi A., Martinet V., Vialatte A., Alignier A., Angeon V., Bohan D.A., Bougherara D., Cordeau S., Courtois P., Deguine J-P., Enjalbert J., Fabre F., Fréville H., Grateau R., Grimonprez B., Gross N., Hannachi M., Launay M., Lelièvre V., Lemarié S., Martel G., Navarrete M., Plantegenest M., Ravigné V., Rusch A., Suffert F., Thoyer S., 2022. Protéger les cultures en augmentant la diversité végétale des espaces agricoles. INRAE. Extended report, 954 p. https://hal.inrae.fr/hal-04127709 Condensed report, 90p. https://hal.inrae.fr/hal-03852213 Summary report, 12 p. https://hal.inrae.fr/hal-03852226

Impacts of plant protection products and biocontrol on biodiversity and ecosystem services

In partnership with Ifremer -May 2022 Leenhardt, S., Mamy, L., Pesce, S., Sanchez, W., Achard, A.-L., Amichot, M., Artigas, J., Aviron, S., Barthélémy, C., Beaudouin, R., Bedos, C., Bérard, A., Berny, P., Bertrand, C., Bertrand, C., Betoulle, S., Bureau-Point, È., Charles, S., Chaumot, A., Chauvel, B., Coeurdassier, M., Corio-Costet, M.-F., Coutellec, M.-A., Crouzet, O., Doussan, I., Fabure, J., Fritsch Nicola Gallai, C., Gonzalez, P., Gouy, V., Hedde, M., Langlais, A., Le Bellec, F., Leboulanger, C., Le Gall, M., Le Perchec, S., Margoum, C., Martin-Laurent, F., Mongruel, R., Morin, S., Mougin, C., Munaron, D., Nelieu, S., Pélosi, C., Rault, M., Sabater, S., Stachowski-Haberkorn, S., Sucre, E., Thomas, M., Tournebize, J., 2022. Impacts des produits phytopharmaceutiques sur la biodiversité et les services écosystémiques. INRAE. Extended report, 1408 p. https://hal.inrae.fr/hal-03777257/ Condensed report, 139 p. https://dx.doi.org/10.17180/gfkj-e861 Summary report, 15 p. https://dx.doi.org/10.17180/hra7-df15

Quality of food of animal origin in relation to production and processing conditions

May 2020 Prache, S., Sante-Lhoutellier, V., Adamiec, C., Astruc, T., Baéza, E., Bouillot, P.-E., Bugeon, J., Cardinal, M., Cassar-Malek, I., Clinquart, A., Coppa, M., Corraze, G., Donnars, C., Ellies, M.-P., Feidt, C., Fourat, E., Gautron, J., Girard, A., Graulet, B., Guillier, L., Hocquette, J.-F., Hurtaud, C., Kerhoas, N., Kesse, E., Le Perchec, S., Lebret, B., Lefèvre, F., Martin, B., Médale, F., Mirade, P.-S., Pierre, F., Raulet, M., Remond, D., Sans, P., Souchon, I., Sibra, C., Touvier, M., Verrez-Bagnis, V., Vitrac, O., 2020. La qualité des aliments d'origine animale selon les conditions de production et de transformation. INRAE. Extended report, 1023 p. http://dx.doi.org/10.14758/m20h-1q76 Condensed report, 112 p. http://dx.doi.org/10.14758/z8q2-ey12 Summary report, 10 p. http://dx.doi.org/10.14758/9q2b-hf73 Guidelines for the conduct of collective scientific assessments and advanced studies -2023 -2023 51 INRAE -Unit for Collective Scientific Assessment, Foresight and Advanced Studies INRAE -Unit for Collective Scientific Assessment, Foresight and Advanced Studies

Can organic farming manage without copper?

January 2018 Andrivon, D., Bardin, M., Bertrand, C., Brun, L., Daire, X., Decognet, V., Fabre, F., Gary, C., Grenier, A.-S., Montarry, J., Nicot, P., Reignault, P., Tamm, L., 2018. Peut-on se passer du cuivre en protection des cultures biologiques ? INRA. Extended report, 185 p. http://dx.doi.org/10.15454/p7ex-0236 Condensed report, 66 p. http://dx.doi.org/10.15454/bd0g-mg26 Summary report, 8 p. http://dx.doi.org/10.15454/34k3-wz34 Artificialized land and land take; drivers, impacts and potential responses In partnership with Ifsttar -December 2017 Béchet, B., Le Bissonnais, Y., Ruas, A., Aguilera, A., André, M., Andrieu, H., Ay, J.-S., Baumont, C., Barbe, E., Vidal Beaudet, L., Belton-Chevallier, L., Berthier, E., Billet, P., Bonin, O., Cavailhes, J., Chancibault, K., Cohen, M., Coisnon, T., Colas, R., CORNU, S., Cortet, J., Dablanc, L., Darly, S., Delolme, C., Fack, G., Fromin, N., GADAL, S., Gauvreau, B., Geniaux, G., Gilli, F., Guelton, S., Guérois, M., Hedde, M., Houet, T., Humbertclaude, S., Jolivet, L., Keller, C., LeBerre, I., Madec, P., Mallet, C., Marty, P., Mering, C., Musy, M., Oueslati, W., Paty, S., Polèse, M., Pumain, D., Puissant, A., Riou, S., Rodriguez, F., Ruban, V., Salanié, J., Schwartz, C., Sotura, A., Thébert, M., Thévenin, T., Thisse, J., Vergnes, A., Christiane, W., Werey, C., Desrousseaux, M., 2017. Sols artificialisés et processus d'artificialisation des sols : déterminants, impacts et leviers d'action. INRA. IFFSTAR. Extended report, 609 p. http://dx.doi.org/10.15454/731a-nn30 Condensed report, 127 p. http://dx.doi.org/10.15454/6snj-zn04 Summary report, 8 p. http://dx.doi.org/10.15454/dqxf-0c26 Eutrophication: manifestations, causes, consequences and predictability ESCo led by CNRS in partnership with IFREMER, INRA and lRSTEA -September 2017 (support from DEPE) Pinay, G., Gascuel, C., Menesguen, A., Souchon, Y., Le Moal, M., Aissani, L., Anschutz, P., Barthélemy, C., Béline, F., Bornette, G., Bourblanc, M., Boutin, C., Chapelle, A., Chauvin, C., Claquin, P., Crave, A., Denoroy, P., Dorioz, J.M., Douguet, J.-M., Doussan, I., Durand, P., Etrillard, C., Euzen, A., Gascuel, D., Gross, E., Hoepffner, N., Humbert, J.F., Lacroix, G., Le Pape, O., Lefebvre, A., Lescot, J.-M., Levain, A., Miossec, L., Moatar, F., Mostajir, B., Pannard, A., Rimet, F., Rossi, N., Sanchez-Perez, J.-M., Sauvage, S., Souchu, P., Terreaux, J.-P., Usseglio-Polatera, P., Vinçon-Leite, B., 2017. Eutrophisation. Manifestations, causes, conséquences et prédictibilité. CNRS. IFREMER. INRA. IRSTEA. Extended report, 144 p. http://dx.doi.org/10.15454/tzr1-6m97 Condensed report, 148 p. http://dx.doi.org/10.15454/z186-0a84 Summary report, 8 p. http://dx.doi.org/10.15454/z186-0a84 Animal consciousness May 2017 Le Neindre, P., Bernard, E., Boissy, A., Boivin, X., Calandreau, L., Delon, N., Deputte, B., Desmoulin-Canselier, S., Dunier-Thomann, M., Faivre, N., Giurfa, M., Guichet, J.L., Lansade, L., Larrère, R., Mormède, P., Prunet, P., Schaal, B., Servière, J., Terlouw, C., 2017. La conscience animale. INRA. Extended report, 165 p. http://dx.doi.org/10.2903/sp.efsa.2017.EN-1196 Condensed report, 120 p. https://www.quae.com/produit/1520/9782759228713/la-conscience-des-animaux Summary report, 8 p. http://dx.doi.org/10.15454/86f6-h087 Roles, impacts and services provided by European livestock production November 2016 Dumont, B., Dupraz, P., Aubin, J., Batka, M., Beldame, D., Boixadera, J., Bousquet-Mélou, A., Benoit, M., Bouamra-Mechemache, Z., Chatellier, V., Corson, M.S., Delaby, L., Delfosse, C., Donnars, C., Dourmad, J.-Y., Duru, M., Edouard, N., Fourat, E., Frappier, L., friant-perrot, M., Gaigné, C., Girard, A., Guichet, J.-L., Haddad, N., Havlik, P., Hercule, J., Hostiou, N., Huguenin-Elie, O., Klumpp, K., Langlais, A., Lavenant, S., Le Perchec, S., Lepiller, O., Letort, E., Levert, F., Martin, B., Méda, B., Mognard, E.L., Mougin, C., Ortiz, C., Piet, L., Pineau, T., Ryschawy, J., Sabatier, R., Turolla, S., Veissier, I., Verrier, E., Vollet, D., Van Der Werf, H., Wilfart, A., 2016. Rôles, impacts et services issus des élevages en Europe. INRA. Extended report, 1 032 p. http://dx.doi.org/10.15454/1vjn-8f61 Condensed report, 127 p. http://dx.doi.org/10.15454/c0hw-k742 Summary report, 8 p. http://dx.doi.org/ 10.15454/90y7-1b24 ADVANCED STUDIES A reference framework of soil quality indicators for the evaluation and implementation of public policies In progress Impacts of production methods of labelled food products on biodiversity In partnership with Ifremer -In progress Advanced study on country-scale monitoring on agri-environmental sustainability Study conducted within the framework of the OECD Temperate Agriculture (TempAG) network -January 2022 Bergez, J.-E., Béthinger, A., Bockstaller, C., Cederberg, C., Ceschia, E., Guilpart, N., Lange, S., Müllerk, F., Reidsma, P., Riviere, C., Schader, C., Therond, O., Van der Werf, H.M.G., 2022. Advanced study on country scale monitoring on agri-environmental sustainability. INRAE. Technical Report , 55 p. https://hal.inrae.fr/hal-03782955

The place of European agriculture in the world by 2050: between climate issues and the challenges of global food security February 2020 Forslund, A., Marajo-Petitzon, E., Tibi, A., Guyomard, H., Schmitt, B., Agabriel, J., Brossard, L., Dourmad, J.-Y., Dronne, Y., Faverdin, P., Lessire, M., Planton, S., Debaeke, P., 2020. Place des agricultures européennes dans le monde à l'horizon 2050 : Entre enjeux climatiques et défis de la sécurité alimentaire mondiale. INRAE. Extended report, 218 p. http://dx.doi.org/10.15454/jh78-yb46 Condensed report, 159 p. http://dx.doi.org/10.15454/pz5b-v806 Summary report, 12 p. http://dx.doi.org/10.15454/sf4h-xa58 Potential of French agriculture and forestry to achieve the carbon storage target of 4 per thousand June 2019 Pellerin, S., Bamière, L., Launay, C., Martin, R., Schiavo, M., Angers, D., Augusto, L., Balesdent, J., Basile-Doelsch, I., Bellassen, V., Cardinael, R., Cécillon, L., Ceschia, E., Chenu, C., Constantin, J., Darroussin, J., Delacote, P., Delame, N., Gastal, F., Gilbert, D., Graux, A.-I., Guenet, B., Houot, S., Klumpp, K., LETORT, E., Litrico, I., Martin, M., Menasseri-Aubry, S., Meziere, D., Morvan, T., Mosnier, C., Roger-Estrade, J., Saint-André, L., Sierra, J., Therond, O., Viaud, V., Grateau, R., Le Perchec, S., Savini, I., Rechauchère, O., 2020. Stocker du carbone dans les sols français. Quel potentiel au regard de l'objectif 4 pour 1000 et à quel coût ? INRA. Extended report, 528 p. http://dx.doi.org/10.15454/nhxt-gn38 http://dx.doi.org/10.15454/nhxt-gn38 Condensed report, 114 p. http://dx.doi.org/10.15454/1.5433098269609653E12 Summary report, 12 p. http://dx.doi.org/10.15454/1.5433045552555942E12 Ecosystem Services from Agricultural Ecosystems -A Contribution to the EFESE Program Octobre 2017 Therond, O., Tichit, M., Tibi, A., Accatino, F., Biju-Duval, L., Bockstaller, C., Bohan, D., Bonaudo, T., Boval, M., Cahuzac, E., Casellas, E., Chauvel, B., Choler, P., Constantin, J., Cousin, I., Daroussin, J., David, M., Delacote, P., Derocles, S., De Sousa, L., Domingues, J.-P., Dross, C., Duru, M., Eugène, M., Fontaine, C., Garcia b, Geijzendorffer, I.R., Girardin, A., Graux, A.-I., Jouven, M., Langlois, B., Le Bas, C., Le Bissonnais, Y., Lelievre, V., Lifran, R., Maigne, E., Martin, G., Märtin, R., Martin-Laurent, F., Martinet, V., McLaughlin, O., Meillet, A., Mignolet, C., Mouchet, M., Nozières-Petit, M.-O., Ostermann, O.P., Paracchini, M.L., Pellerin, S., Peyraud, J.-L., Petit, S., Picaud, C., Plantureux, S., Poméon, T., Porcher, E., Puech, T., Puillet, L., Rambonilaza, T., Raynal, H., Resmond, R., Ripoche, D., Ruget, F., Rulleau, B., Rush, A., Salles, J.-M., Sauvant, D., Schott, C., Tardieu, L., 2017. Les services écosystémiques rendus par les écosystèmes agricoles -Une contribution au programme EFESE--. INRA. Extended report, 966 p. http://dx.doi.org/10.15454/prmv-wc85 Condensed report, 118 p. http://dx.doi.org/10.15454/1h4z-tq90 Summary report, 12 p. http://dx.doi.org/10.15454/mjk0-xf31

Authors :

Catherine Donnars (coord.), Anaïs Tibi (coord.), Marc-Antoine Caillaud, Regina Dashkina, INRAE -Unit for Collective Scientific Assessment, Foresight and Advanced Studies INRAE -Unit for Collective Scientific Assessment, Foresight and Advanced Studies

The project leader drafts the condensed report in close collaboration with the scientific leads and the project managers. The experts amend the proposed plan and then the intermediate versions, and validate the final version. The drafting process begins as soon as the general conclusions of the report are formulated.

A very advanced working version is reviewed by:

• the sponsors, for an opinion on the clarity of the document, its relevance to the questions in the project specifications, and the appropriateness of its content for the purpose of informing public decision-making. The sponsors do not intervene in the content of either the analyses or the conclusions; • INRAE scientific directors who monitor the project, and if necessary, their counterparts in the partner organisations, in order to access the contents as the public presentation of the results approaches, and; • usually, by scientific reviewers from outside the working group in the interests of scientific rigour.

The authors of the condensed report are not compelled to accept the remarks of the reviewers but, if they do, they must justify why.

The project leader, the scientific leads, the members of the expert committee, the information officers and the project managers are the authors of the condensed report, except in special cases. DEPE is responsible for its publication. The condensed report is published by Editions Quae in a version that is often revised in French and/or English (with the possibility of also engaging an English publisher).

The condensed report is cited (general case) as follows: Project leader (coord.), scientific leads (coord.), experts and other members of the working group who contributed to the synthesis in alphabetical order (date). , Babut M., Belliard J., Bernez I., Burger-Leenhardt D., Dorioz J.M., Douez O., Dufour S., Grimaldi C., Habets F., Le Bissonnais Y., Molénat J., Rollet A.J., Rosset V., Sauvage S., Usseglio-Polatera P., Leblanc B., 2016 Use of fertilizing materials of waste origin on soils for agricultural or forestry use: agronomic, environmental, socioeconomic impacts

In partnership with CNRS andIRSTEA -July 2014 Houot, S., Pons, M.-N., Pradel, M., Tibi, A., Aubry, C., Augusto, L., Barbier, R., Benoit, P., Brugère, H., Caillaud, M.-A., Casellas, M., Chatelet, A., Dabert, P., De Mareschal, S., Doussan, I., Etrillard, C., Fuchs, J., Génermont, S., Giamberini, L., Hélias, A., Jardé, E., Le Perchec, S., Lupton, S., Marron, N., Menasseri-Aubry, S., Mollier, A., Morel, C., Mougin, C., Nguyen, C., Parnaudeau, V., Patureau, D., Pourcher, A.M., Rychen, G., Savini, I., Smolders, E., Topp, E., Vieublé, L., Viguié, C., 2014. Valorisation Nitrogen flows associated with livestock farming: reducing losses and restoring balance January 2012 Peyraud, J.-L., Cellier, P., Aarts, F., Béline, F., Bockstaller, C., Bourblanc, M., Delaby, L., Donnars, C., Dourmad, J.-Y., Dupraz, P., Durand, P., Faverdin, P., Fiorelli, J.-L., Gaigné, C., Girard, A., Guillaume, F., Kuikman, P., Langlais, A., Le Goffe, P., Le Perchec, S., Lescoat, P., Morvan, T., Nicourt, C., Parnaudeau, V., Rechauchère, O., Rochette, P., Vertes, F., Veysset, P. , 2012 Herbicide-tolerant plant varieties: agronomic, environmental and socio-ecological effects

In partnership with CNRS -November 2011Beckert, M., Dessaux, Y., Charlier, C., Darmency, H., Richard, C., Savini, I., Tibi, A., 2011 Dietary behaviors and practices: determinants, action, outcomes?

June 2010 Etiévant, P., Bellisle, F., Dallongeville, J., Etilé, F., Guichard, E., Padilla, M., Romon-Rousseaux, M., Donnars, C., Tibi, A., 2010 The Role of French forests and the forestry sector in climate change mitigation: opportunities and deadlocks by 2050

In partnership with IGN -June 2017Roux, A., Dhôte, J.-F., Bastick, C., Colin, A., Bailly, A., Bastien, J.-C., Berthelot, A., Bréda, N., Caurla, S., Carnus, J.-M., Gardiner, B., Jactel, H., Leban, J.-M., Lobianco, A., Loustau, D., Marçais, B., Martel, S., Meredieu, C., Moisy, C., Pâques, L., Deshors-Picart, D., Rigolot, E., Saint-André, L., Schmitt, B., 2017 Ecophyto R&D. Ways to reduce the use of pesticides?

January 2010 Butault, J.-P., Dedryver, C.-A., Gary, C., Guichard, L., Jacquet, F., Meynard, J.-M., Nicot, P. C., Pitrat, M., Reau, R., Sauphanor, B., Savini, I., Volay, T. 2010 Ecophyto R&D. Towards crop systems with low phytosanitary product use. Part 1, Volume VII: Analysis of stakeholders Barbier, J.-M., Bonicel, L., Dubeuf, J.-P., Guichard, L.,Halska, J., Meynard, J.-M., Schmidt, A. 2010 A., Dedryver, C.-A., Reau, R., Volay, T., Butault, J.-P., Darmency, H., Barbier, J.-M., Debaeke, P., Delos, M., Gary, C., Girardin, P., Guichard, L., Meynard, J.-M., Nicot, P., C., Pitrat, M., Rolland, B., Sauphanor, B., Viaux, P., Walter C., 2009. Écophyto ., Gary, C., Guichard, L., Jacquet, F., Meynard, J.-M., Nicot P., C., Pitrat, M., Reau, R., Sauphanor, B., Savini, I., Volay, T., 2010. Écophyto Ecophyto R&D: Ways to reduce the use of pesticides. Summary of the study Butault, J.-P., Dedryver, C.-A., Gary, C., Guichard, L., Jacquet, F., Meynard, J.-M., Nicot, P., C., Pitrat, M., Reau, R., Sauphanor, B., Savini, I., Volay, T., 2010. Écophyto R., Fortino, G., Bintein, Y., Boisset, K., Conteau, C., Coulon, T., Dehlinger, F., Faloya, V., Petit, M.-S., Plénet, D., Lusson, J.-M., Verjux, N., Veschambre, D., Cellier, V., Boll, R., Chanet, J.-P., Boulet, A., Cerf, M., Jeannequin, B., Weissenberger, A. 2009 -P., Delame, N., Jacquet, F., Rio, P., Zardet, G., Benoit, M., Blogowski, A., Bouhsina, Z., Carpentier, A., Desbois, D., Dupraz, P., Guichard, L., Rousselle, J.-M., Ruas, J.-F., Varchavsky, M., 2009. Écophyto Brismontier, E., Nicot, P., C., Pitrat, M., Blancard, D., Bressoud, F., Le Delliou, B., Mazollier, C., Navarrete, M., Roche, G., Taussig, C., Tchamitchian, M., Trottin-Caudal,Y., Villeneuve, F., Wuster, G., Stengel, P., Lapchin, L., Dedryver, C.-A., Volay, T., 2009. Écophyto B., Dirwimmer, C., Volay, T., Sophie Boutin, S., Chaussabel, A.-L., Dupont, N., Fauriel, J., Gallia, V., Lambert, N., Navarro, E., Parisi, L., Plénet, D., Ricaud, V., Sagnes, J.-L., Sauvaitre, D., Simon, S., Speich, P., Zavagli, F., 2009. Écophyto D., Gary, C., Barbier, J.-M., Bernos, L., Clément, C., Constant, N., Deliere, L., Forget, D., Grosman, J., Molot, B., Rio, P., Sauvage, D., Sentenac, G., 2009. Écophyto N., Debaeke, P., Delos, M., Volay, T., Guérin, O., Guichard, L., Guinde, L., Mischler, P., Munier-Jolain, N., Omon, B., Rolland, B., Viaux, P., Villard, A., 2009. Écophyto Institut national de recherche pour l'agriculture, l'alimentation et l'environnement