Sacha Altay
email: sacha.altay@gmail.com

Marlène Schwartz

Anne-Sophie Hacquin

Aurélien Allard

Stefaan Blancke

Hugo Mercier
email: hugo.mercier@gmail.com

Scaling up Interactive Argumentation by Providing Counterarguments with a Chatbot

Discussion is more convincing than standard, unidirectional messaging, but its interactive nature makes it difficult to scale up. We created a chatbot to emulate the most important traits of discussion. A simple argument pointing out the existence of a scientific consensus on Genetically Modified Organisms (GMOs) safety already led to more positive attitudes towards GMOs, compared to a control message. Providing participants with good arguments rebutting the most common counterarguments against GMOs led to much more positive attitudes towards GMOs, whether the participants could immediately see all the arguments, or could select the most relevant arguments in a chatbot. Participants holding the most negative attitudes displayed more attitude change in favor of GMOs. Participants updated their beliefs when presented with good arguments, but we found no evidence that an interactive chatbot proves more persuasive than a list of arguments and counterarguments.

Introduction

In many domains-from the safety of vaccination to the reality of anthropogenic climate change-there is a gap between the scientific consensus and public opinion 1 . The persistence of this gap in spite of numerous information campaigns shows how hard it is to bridge. It has even been suggested that information campaigns backfire, either by addressing audiences with strong pre-existing views [START_REF] Nyhan | When corrections fail: The persistence of political misperceptions[END_REF][START_REF] Nyhan | Effective messages in vaccine promotion: a randomized trial[END_REF] , or by attempting to present too many arguments [START_REF] Cook | The debunking handbook[END_REF][START_REF] Lewandowsky | Misinformation and its correction: Continued influence and successful debiasing[END_REF] .

Fortunately, it appears that in most cases good arguments do change people's mind in the expected direction [START_REF] Guess | Does counter-attitudinal information cause backlash? Results from three large survey experiments[END_REF][START_REF] Wood | The elusive backfire effect: Mass attitudes' steadfast factual adherence[END_REF] . Still, the effects of short arguments aimed at large and diverse audiences, even if they are positive, are typically small [START_REF] Landrum | Examining the Impact of Expert Voices: Communicating the Scientific Consensus on Genetically-modified Organisms[END_REF][START_REF] Dixon | Applying the Gateway Belief Model to Genetically Modified Food Perceptions: New Insights and Additional Questions[END_REF][START_REF] Kerr | Changes in perceived scientific consensus shift beliefs about climate change and GM food safety[END_REF] . By contrast, when people can exchange arguments face-to-face, more ample changes of mind regularly occur. Compare for example how people react to simple logical arguments. On the one hand, when participants are provided with a good argument for the correct answer to a logical problem, a substantial minority fails to change their minds [START_REF] Claidière | Argumentation and the diffusion of counterintuitive beliefs[END_REF][START_REF] Trouche | Arguments, more than confidence, explain the good performance of reasoning groups[END_REF] . On the other hand, when participants tackle the same problems in groups, nearly everyone discussing with a participant defending the correct answer changes their mind [START_REF] Claidière | Argumentation and the diffusion of counterintuitive beliefs[END_REF][START_REF] Trouche | Arguments, more than confidence, explain the good performance of reasoning groups[END_REF][START_REF] Laughlin | Group problem solving[END_REF] . More generally, argumentation has been shown, in a variety of domains, to allow people to change their minds and adopt the best answers available in the group [START_REF] Minson | Two to Tango[END_REF][START_REF] Laughlin | Group problem solving[END_REF][START_REF] Smith | Why peer discussion improves student performance on in-class concept questions[END_REF][START_REF] Mercier | The argumentative theory: Predictions and empirical evidence[END_REF][START_REF] Mercier | The enigma of reason[END_REF] . Even on contested issues, discussions with politicians [START_REF] Minozzi | Field experiment evidence of substantive, attributional, and behavioral persuasion by members of Congress in online town halls[END_REF] , canvassers [START_REF] Broockman | Durably reducing transphobia: A field experiment on door-todoor canvassing[END_REF] , or scientists [START_REF] Altay | Are science festivals a good place to discuss heated topics?[END_REF][START_REF] Chanel | Impact of information on intentions to vaccinate in a potential epidemic: swine-origin Influenza A (H1N1)[END_REF] can lead to changes of mind that are significant, durable [START_REF] Broockman | Durably reducing transphobia: A field experiment on door-todoor canvassing[END_REF] , and larger than those observed with standard messages [START_REF] Minozzi | Field experiment evidence of substantive, attributional, and behavioral persuasion by members of Congress in online town halls[END_REF][START_REF] Chanel | Impact of information on intentions to vaccinate in a potential epidemic: swine-origin Influenza A (H1N1)[END_REF] .

Mercier and Sperber [START_REF] Mercier | The enigma of reason[END_REF] have suggested that the power of interactive argumentation, by contrast with the presentation of a simple argument, to change minds stems largely from the opportunity discussion affords to address the discussants' counterarguments. In the course of a conversation, people can raise counterarguments as they wish, the counterarguments can be rebutted, the rebuttals contested, and so forth [START_REF] Resnick | Reasoning in conversation[END_REF] . When people are presented with challenging arguments in a one-sided manner, as in typical messaging campaigns, they also generate counterarguments [START_REF] Edwards | A disconfirmation bias in the evaluation of arguments[END_REF][START_REF] Greenwald | Cognitive learning, cognitive response to persuasion, and attitude change[END_REF][START_REF] Taber | Motivated skepticism in the evaluation of political beliefs[END_REF] ; however, these counterarguments remain unaddressed. Arguably, the production of counterarguments that remain unaddressed is not only why standard information campaigns are not very effective, but also why they sometimes backfire [START_REF] Trouche | Objective evaluation of demonstrative arguments[END_REF] .

In a discussion, not only can all counterarguments be potentially addressed, but only the relevant counterarguments are addressed. Different people have different reasons to disagree with any given argument. Attempting to address all the existing counterarguments should lead to the production of many irrelevant rebuttals, potentially diluting the efficacy of the relevant rebuttals. This might be why, in a normal conversation, we do not attempt to lay out all the arguments for our side immediately, waiting instead for our interlocutor's feedback to select the most relevant counterarguments to address [START_REF] Mercier | Why don't people produce better arguments? in Cognitive Unconscious and Human Rationality[END_REF] .

Unfortunately, discussion does not scale up well-indeed, it is most natural in groups of at most five people [START_REF] Fay | Group discussion as interactive dialogue or as serial monologue: The influence of group size[END_REF][START_REF] Krems | Why are conversations limited to about four people? A theoretical exploration of the conversation size constraint[END_REF] . Here, we developed and tested two ways of scaling up discussion. The first consisted in gathering the most common counterarguments for a given issue and a given population, and creating a message that rebuts the most common counterarguments, as well as the responses to the rebuttals-as would happen in a conversation. The issue, then, is that many-potentially most-of these counterarguments are likely to be irrelevant for most of the audience. As a result, we developed a second way of scaling up discussion: a chatbot in which participants could select which counterarguments they endorse, and only see the rebuttals to these counterarguments. Studies on argumentation using chatbots or similar automated computer based conversational agents suggest that they can be useful to change people's mind [START_REF] Andrews | Argumentative human computer dialogue for automated persuasion[END_REF][START_REF] Rosenfeld | Strategical argumentative agent for human persuasion[END_REF] , and that asking users what they are concerned about increases chatbots' efficacy by providing users with more relevant counterarguments [START_REF] Chalaguine | Impact of Argument Type and Concerns in Argumentation with a Chatbot[END_REF] .

In the remaining of the introduction, we present the topic we have chosen to test our methods for scaling up discussion, as well as the design of the experiment, and how the different conditions were constructed. Finally, specific hypotheses are introduced.

We choose Genetically Modified Organisms (GMOs) and Genetically Modified (GM) food as a topic for our experiment because, despite the broad scientific consensus on GM food safety for human health [START_REF] Baulcombe | Science Update: A report to the Council for Science and Technology[END_REF][34][START_REF]Genetically engineered crops: experiences and prospects[END_REF][START_REF] Nicolia | An overview of the last 10 years of genetically engineered crop safety research[END_REF][START_REF] Ronald | Plant genetics, sustainable agriculture and global food security[END_REF][START_REF] Science | Statement by the AAAS board of directors on labeling of genetically modified foods[END_REF][START_REF] Yang | Governing GMOs in the USA: science, law and public health[END_REF] , public opinion remains, in many countries, staunchly opposed to GM food and GMOs more generally [START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF][START_REF] Cui | Public perception of genetically-modified (GM) food: a nationwide chinese consumer study[END_REF][START_REF] Gaskell | Worlds apart? The reception of genetically modified foods in Europe and the US[END_REF][START_REF] Scott | Evidence for absolute moral opposition to genetically modified food in the United States[END_REF] . In the United States it is the topic on which the discrepancy between scientists and laypeople's opinion is the highest 1 . In France, where the pilot study was conducted (see the Pilot data section and the Pilot Study section in SI), rejection of GMO is pervasive [START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF] : 84% of the public thinks that GM food is highly or moderately dangerous [START_REF] Irsn | Baromètre sur la perception des risques et de la sécurité par les Français[END_REF] and 79% of the population is worried that some GMO may be present in their daily diet [START_REF] Ifop | [END_REF] . In the United Kingdom, where the pre-registered study was conducted, rejection of GMO is common: 45% of the public thinks that GM food is dangerous (only 25% think that it is not) [START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF] , and 58% of the public does not want to eat this type of food (only 24% wants to) [START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF] . On the whole, British people appear to be largely unpersuaded by the benefits of GMOs [START_REF] Burke | GM food and crops: what went wrong in the UK?[END_REF][START_REF] Poortinga | Public Perceptions of Genetically Modified Food and Crops, and the GM Nation? Public Debate on the Commercialisation of Agricultural Biotechnology in Th UK: Main Findings of a British Survey[END_REF][START_REF] Cordon | GM Crops opposition may have been 'over-estimated[END_REF] . The gap between the scientific consensus and public opinion on GMOs is all the more problematic since GM food and GMOs more generally can not only improve health and food security, but also help fight climate change [START_REF] Ronald | Plant genetics, sustainable agriculture and global food security[END_REF][START_REF] Bonny | Will Biotechnology Lead To More Sustainable Agriculture[END_REF][START_REF] Hielscher | Rationalizing the GMO debate: the ordonomic approach to addressing agricultural myths[END_REF] .

Our goal was thus to test whether rebutting participants' counterarguments against GMOs will lead them to change their minds on this topic. To properly evaluate the efficiency of this intervention, we used the following four conditions. First, as a Control condition, we provided participants with a sentence describing what GMOs are. Given that no persuasion should take place in this condition, any attitude change (measured as the difference between the pre-and post-intervention attitudes) would reflect task demands, and can thus be used as a baseline against which to compare attitude change in the other conditions.

Second, we compared our interventions to one of the most common techniques used to bridge the gap between scientific consensus and public opinion: informing the public of the existence and strength of the scientific consensus (the so-called Gateway Belief Model). Some studies using this Gateway Belief Model have proven effective at reducing the gap between public opinion and the scientific consensus on a variety of topics [START_REF] Ding | Support for climate policy and societal action are linked to perceptions about scientific agreement[END_REF][START_REF] Dunwoody | Using weight-of-experts messaging to communicate accurately about contested science[END_REF][START_REF] Kohl | The influence of weight-of-evidence strategies on audience perceptions of (un) certainty when media cover contested science[END_REF][START_REF] Lewandowsky | The pivotal role of perceived scientific consensus in acceptance of science[END_REF][START_REF] Van Der Linden | The scientific consensus on climate change as a gateway belief: Experimental evidence[END_REF][START_REF] Van Der Linden | Gateway illusion or cultural cognition confusion[END_REF] (although see [START_REF] Landrum | Examining the Impact of Expert Voices: Communicating the Scientific Consensus on Genetically-modified Organisms[END_REF][START_REF] Dixon | Applying the Gateway Belief Model to Genetically Modified Food Perceptions: New Insights and Additional Questions[END_REF] ). This Consensus Condition allowed us to tell whether our interventions could improve attitude change by comparison with a popular messaging strategy.

Third, in the Counterarguments Condition participants were provided with a series of counterarguments against GMOs, rebuttals against these counterarguments, counterarguments of these rebuttals, and so forth (for at most four steps, see how these arguments were created in the Design section). One of these counterarguments mentions the existence and strength of The comparison of these four conditions allowed us to tell whether (i) any of these interventions resulted in attitude change, (ii) whether the attitude change was larger when arguments were provided (i.e. in the Consensus, Counterarguments, and Chatbot Conditions), (iii) whether any argument-driven attitude change was larger when rebuttals to counterarguments were provided (Counterarguments and Chatbot Conditions) and, (iv) whether any rebuttal-driven attitude change was larger when only relevant rebuttals were provided (Chatbot Condition). See Table 1 for an overview of our design.

On the basis of the literature reviewed above, we derived the following hypotheses. First, the literature on the Gateway Belief Model, on the importance of addressing counterarguments, and on the importance of only addressing relevant counterarguments, led to the following hypotheses: Models of attitude change-such as the Elaboration Likelihood Model [START_REF] Petty | The elaboration likelihood model of persuasion[END_REF] -suggest that participants might use the number of arguments as a low level cue that they should change their minds, at least when they are not motivated to process the arguments in any depth [START_REF] Petty | The effects of involvement on responses to argument quantity and quality: Central and peripheral routes to persuasion[END_REF] .

However, it has also been argued that presenting people with too many arguments-even good ones-might make a message less persuasive if the misinformation that the arguments aim to correct is simpler and more appealing [START_REF] Lewandowsky | Misinformation and its correction: Continued influence and successful debiasing[END_REF] , so that more is not necessarily best when it comes to the number of arguments provided. Still, if H6 and H7 proved true, it could be argued that participants use a low-level heuristic in which they are convinced by the sheer number of arguments, instead of being convinced by the content of the arguments. If people use the number of arguments in this manner, it should affect their overall attitudes towards GMOs.

By contrast, if people pay attention to the content of the arguments, the arguments should only change the participants' minds on the specific topic they bear upon, leading us to the following hypothesis: H8: In the Chatbot Condition, participants will hold more positive attitudes after the experimental task on issues for which they have explored more of the rebuttals related to the issue, controlling for their initial attitudes on the issues, the type of issue, and the total (i.e. related to the issue or not) number of arguments explored.

Finally, given that the backfire effect has been observed in several experiments 2,4,59-61 but has rarely, or not at all been observed in several large scale studies [START_REF] Guess | Does counter-attitudinal information cause backlash? Results from three large survey experiments[END_REF][START_REF] Wood | The elusive backfire effect: Mass attitudes' steadfast factual adherence[END_REF][START_REF] Schmid | Effective strategies for rebutting science denialism in public discussions[END_REF][START_REF] Van Der Linden | The gateway belief model: A largescale replication[END_REF][START_REF] Van Der Linden | Exposure to Scientific Consensus Does Not Cause Psychological Reactance[END_REF] , we formulated the following hypothesis: H9: H1, H2, and H3 also hold true among the third of the participants initially holding the most negative attitudes about GMOs. (Note that this criterion is more stringent than an absence of backfire effect, as it claims that there will be a positive effect even among participants with the most negative initial attitudes).

Although it is methodologically impossible to completely disentangle the effects of the mode of presentation (e.g. degree of interactivity) and of the specific information presented, the present experiment provides a test of whether addressing people's counterarguments, in particular by using an interactive chatbot, results in attitude changes that are larger than those obtained with a common messaging technique. From a theoretical point of view, these results help us better understand the process of attitude change, potentially highlighting its rationality. If people are sensitive to the rebuttals of their counterarguments, it suggests that their rejection of the initial argument was not driven by sheer pigheadedness, but by having unanswered counterarguments. From an applied point of view, positive results would provide an efficient and easy to use tool to help science communicators bridge the gap between scientific consensus and public opinion.

Results

In the Control Condition participants read a sentence describing what GMOs are; in the Consensus Condition they read a paragraph on the scientific consensus on GMOs safety; in the Counterarguments Condition they were exposed to the most common counterarguments against GMOs, together with their rebuttal; in the Chatbot Condition they were exposed to the same arguments as in the Counterarguments Condition but through a chatbot (i.e. instead of scrolling, they had to click to make the arguments appear). The effect of the treatments on participants' attitudes towards GMOs is depicted in Figure 2 and Figure 3. GMOs was similar after the treatment and before (i.e. on a four-point Likert scales their attitude did not change by more than one point overall). Among the other participants, green (resp. red) lines represent participants whose attitude toward GMOs was more positive (resp. negative) after the treatment than before. In the box plots, the boxes represent the middle 50% of the scores (the boundaries of the boxes represent the 25 th and 75 th percentile), the lines dividing the boxes are the median, the boundaries of the upper and lower whiskers were determined by multiplying the Inter-Quartile Range by 1.5. Density plots represent the Contrary to H8, participants did not hold significantly more positive attitudes after the treatment on issues for which they had explored more of the related rebuttals (b = 0.006 [-0.02, 0.04], t (898.9) = .09, p = .77, BF10 = 0.09). This effect is negligible as the 90% CI [-0.04, 0.07] falls inside the pre-registered [-0.1, 0.1] interval corresponding to an effect smaller than ß = 0.1. For instance, participants exploring more arguments related to GM food safety did not develop more pro-GMOs opinion on GM food safety, compared to other issues. We also found that H1-3 held true for each question of the GMOs attitudes scale: participants deemed GM food to be safer to eat, less bad for the environment, reported being less worried about the socio-economic impacts of GMOs, and perceived GMOs as more useful after the treatment in the Consensus Condition, Counterarguments Condition, and Chatbot Condition compared to the Control Condition (see the 'Exploratory Analyses' section in SI).

Finally, for three out of the four main arguments on GMOs, the best predictor of whether a participant clicked on a given argument in the chatbot was how negative their initial attitudes were regarding that argument (see the 'Exploratory Analyses' section in SI). This suggests that participants selected arguments that addressed their concerns, instead of arguments that might have reinforced their priors.

Discussion

In this article, we investigated whether addressing many of participants' arguments against GMOs would result in significant changes of mind on that issue. First, despite previous failures to apply the Gateway belief model to GMOs [START_REF] Landrum | Examining the Impact of Expert Voices: Communicating the Scientific Consensus on Genetically-modified Organisms[END_REF][START_REF] Dixon | Applying the Gateway Belief Model to Genetically Modified Food Perceptions: New Insights and Additional Questions[END_REF] , we found that a simple argument pointing out the existence of a scientific consensus on the safety of GMOs led to more positive attitudes towards GMOs (ß = 0.32). Second, we found that addressing many of the participants' arguments against GMOs led to much more positive attitudes towards GMOs (Counterarguments Condition: ß = 0.85; Chatbot Condition: ß = 0.66). These effect sizes compare very favorably to those observed in other interventions, such as 8-10,62,67-69 . After reading the rebuttals against criticisms of GMOs, a large number of participants adopted strongly pro-GMOs views: the number of participants with an average score of at least five (on the one to seven attitude scale) went from 104 to 299 (out of 601), and the number of participants with an average score of at least six went from 15 to 107 (out of 601). A subsequent study found similar results in another domain: a chatbot aimed at alleviating concerns about COVID-19 vaccines generated more positive attitudes towards these vaccines, and higher intentions to vaccinate [START_REF] Altay | Information Delivered by a Chatbot Has a Positive Impact on COVID-19 Vaccines Attitudes and Intentions[END_REF] .

Our results reveal that participants changed their minds more as they spent more time reading counterarguments, and they tended to spend more time when all the counterarguments were available (Counterarguments Condition) than when they were offered the possibility of only selecting the most relevant counterarguments (Chatbot Condition). Moreover, being exposed only to counterarguments participants had selected, by contrast with all the counterarguments, did not make the counterarguments more efficient. It is possible that participants used the sheer number of arguments presented as a cue to change their mind. It is also plausible that, in the case at hand, all the counterarguments presented to the participants were sufficiently relevant that none detracted from the persuasiveness of the whole set, or that participants selected the most relevant via scrolling, and that this selection was more efficient than via clicking. If this is the case, then the main reason for the increased efficiency of the chatbot (controlling for time spent), i.e., that people avoid reading irrelevant arguments, disappears.

This finding has practical consequences: given the available evidence, it is probably best to give chatbot users the option to scroll through the arguments instead of clicking on them, as in our Counterarguments Condition. In the chatbot that we used to inform French people about the COVID-19 vaccines, we gave users the possibility of scrolling through the arguments instead of clicking on them [START_REF] Altay | Information Delivered by a Chatbot Has a Positive Impact on COVID-19 Vaccines Attitudes and Intentions[END_REF] . In that study, users selecting the non-interactive chatbot did so as a complement of the interactive chatbot.

In line with a growing body of literature [START_REF] Wood | The elusive backfire effect: Mass attitudes' steadfast factual adherence[END_REF][START_REF] Swire-Thompson | Searching for the backfire effect: Measurement and design considerations[END_REF] , we found no evidence that participants initially holding more negative attitudes towards GMOs held even more negative attitudes towards GMOs after having been exposed to arguments in favor of GMOs. Instead, we found the opposite pattern: participants initially holding more negative attitudes displayed more attitude change in favor of GMOs. Similar evidence suggest that corrections work best on people who are the most misinformed [START_REF] Bode | Correcting Misperceptions About Genetically Modified Food on Social Media: Examining the Impact of Experts, Social Media Heuristics, and the Gateway Belief Model[END_REF][START_REF] Bode | In related news, that was wrong: The correction of misinformation through related stories functionality in social media[END_REF][START_REF] Vraga | Using expert sources to correct health misinformation in social media[END_REF] and that, in general, those whose attitudes were initially furthest from the facts changed their minds the most towards the facts [START_REF] Altay | Are science festivals a good place to discuss heated topics?[END_REF][START_REF] Altay | Information Delivered by a Chatbot Has a Positive Impact on COVID-19 Vaccines Attitudes and Intentions[END_REF] .

These results are good news for science communicators, showing that participants can be convinced by good, well-supported arguments. Moreover, the initially very negative attitudes of some participants did not prove an obstacle to changing their minds. This should encourage science communicators to discuss heated topics with the public, even with those furthest away from the scientific consensus (see also: [START_REF] Altay | Are science festivals a good place to discuss heated topics?[END_REF][START_REF] Schmid | Effective strategies for rebutting science denialism in public discussions[END_REF] ).

Our study has some important limitations. First, we don't know the impact our chatbot would have in more naturalistic settings. In the present experiments, participants were paid to interact with the chatbot. How many people would spontaneously want to spend five minutes learning about GMOs without being paid? And would people opposed to GMOs be less likely to interact with it? Second, all our dependent variables are declarative, and attitudes don't always translate into behaviors. For instance, we do not know whether the chatbot increase people's willingness to buy and consume GM products. Third, our sample of participants is not representative of the general population. The chatbot might work better on some segments of the population than others, such as young and educated people (although the experimental manipulations should be robust across samples, see [START_REF] Coppock | Generalizability of heterogeneous treatment effect estimates across samples[END_REF] ). All these metrics are key to estimate the chatbot's effects outside of an experimental setting.

Exploratory hypotheses point to two interesting patterns in our data. First, participants behavior in the chatbot was in line with their attitudes, as they selected the issues for which they had the most negative attitudes, thereby exposing them to the most relevant counterarguments. Second, the counterarguments-including simply providing information about the scientific consensus-had effects beyond the specific issue they addressed. While this might suggest that participants were falling prey to a kind of halo effect, it is also possible that participants drew judicious inferences from one set of arguments to others: for example, participants who come to accept that GMOs are safe to eat might also see them as more useful.

At first glance our results might seem to suggest that presenting counterarguments in a chatbot, by contrast with a more standard text, offers little advantage, or might even prove less persuasive. However, it should be noted that even when not presented in a chatbot, the counterarguments were organized according to a clear dialogic structure (the exact same one as the chatbot) which might have facilitated their understanding, and the identification of the most relevant counterarguments. Moreover, it is possible that participants not expressly paid to take part in an experiment might find the chatbot's interactivity more alluring than a standard text. Future experiments should investigate whether that is the case. Another promising avenue for future research is whether the very large effects observed here persist in time (see, e.g. [START_REF] Broockman | Durably reducing transphobia: A field experiment on door-todoor canvassing[END_REF] ).

Methods

Ethics information

The present research received approval from an ethics committee (CER-Paris Descartes; N° 2019-03-MERCIER). Participants were presented with a consent form and had to give their informed consent to participate in the study. They were paid £1.38.

Pilot data

Among 147 French participants who pretested the chatbot we found that: More details about the pilot can be found in the "Pilot Study" section in SI.

Design

To create the Counterarguments Condition, we systematically gathered the most common counterarguments to the acceptance of GMOs, relying on a variety of methods. First, we drew on popular anti-GMOs websites (such as the "nongmoproject.org"), and on the scientific literature on public opinion towards GMOs [START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF][START_REF] Bonny | Factors Explaining Opposition to GMOs in France and the Rest of Europe[END_REF][START_REF] Evenson | Consumer acceptance of genetically modified foods[END_REF][START_REF] Mchughen | Popular misconceptions: agricultural biotechnology[END_REF][START_REF] Parrott | Genetically modified myths and realities[END_REF] . Second, we relied on the expertise of two of the co-authors, who have both participated in public events about GMOs [START_REF] Altay | Are science festivals a good place to discuss heated topics?[END_REF][START_REF] Blancke | Fatal attraction: the intuitive appeal of GMO opposition[END_REF] . Third, we conducted a preliminary study in which we asked participants to rate how convincing and how accurate they found our rebuttals to the most common counterarguments against GMOs.

When the rebuttals were found to be unconvincing, participants were asked to explain what made the rebuttals unconvincing and write any counterarguments that came to their mind that could weaken the rebuttals (participants that found the rebuttals convincing were also asked to explain why they found them convincing). At the end of the preliminary study participants were asked to write if they had any counterarguments against GMOs that had not been raised during the experiment. This ensured that we covered most of the arguments people hold against GMOs and that the rebuttals to these counterarguments were taken seriously.

To develop the rebuttals to the most common counterarguments, we relied on personal communication with an expert on GMOs, on the website "gmoanswers.com," on the scientific literature on attitudes towards GMOs [START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF][START_REF] Bonny | Factors Explaining Opposition to GMOs in France and the Rest of Europe[END_REF][START_REF] Evenson | Consumer acceptance of genetically modified foods[END_REF][START_REF] Mchughen | Popular misconceptions: agricultural biotechnology[END_REF][START_REF] Parrott | Genetically modified myths and realities[END_REF] , the scientific literature on GMOs 36,83-86 , Wikipedia, as well as the publications of scientific agencies [START_REF] Baulcombe | Science Update: A report to the Council for Science and Technology[END_REF][34][START_REF]Genetically engineered crops: experiences and prospects[END_REF] .

The counterarguments and rebuttals were used to build the Counterarguments Condition.

In this condition, participants are presented on the chatbot interface with the counterarguments and rebuttals available on the chatbot. However, participants cannot select the arguments. Instead, they have to scroll to read the counterarguments and rebuttals. The only difference between the Counterarguments Condition and the Chatbot Condition is the interactivity of the Chatbot (i.e. having to click on the counterarguments, seeing the rebuttals appear progressively instead of instantly, and having the option of not displaying at all some rebuttals). We estimated the reading time for the counterarguments and rebuttals (~ 3000 words) to be approximately 11 minutes (for a reading time of 4 words per second [START_REF] Brysbaert | How many words do we read per minute? A review and meta-analysis of reading rate[END_REF] ).

In the Counterarguments Condition, participants were exposed to the most common counterarguments that we gathered against GMOs, as well as the rebuttals of these counterarguments. However, many participants might not share the concerns expressed in some counterarguments, and thus find the rebuttals largely irrelevant. To address this problem, we created a chatbot whose content was identical to the content of the Counterarguments Condition, but in which participants had to select (by clicking on them) the counterarguments against GMOs (or against the rebuttals to their previous counterarguments) they were most concerned about, and they were provided with rebuttals addressing the selected counterargument.

The chatbot was organized as follows. After a brief technical description of GMOs (used in part in the Control Condition), participants were asked if they had any concerns about GMOs, and were given a choice of four counterarguments to select from: "GMOs might not be safe to eat," "GMOs could hurt the planet," "The way GMOs are commercialized is problematic," "We don't really need GMOs." Participants were also be able to select, at any stage, an option "Why should I trust you?," which informed them about who we were, who funded us, and what our goals were (all the materials are available on the Open Science Framework (OSF) at https://osf.io/cb7wf/).

Each time participants selected a counterargument, the chatbot offered a rebuttal.

Participants could select between several counterarguments to these rebuttals, which were addressed by the chatbot as they were selected. In total the chatbot offered 35 counterarguments against GMOs, together with their 35 rebuttals. Participants were not able to write open-ended counterarguments addressed to the chatbot, they were only able to select among the counterarguments offered, to which the chatbot answered with a predefined rebuttal. If the rebuttal exceeded five lines, it was displayed in separate discussion bubbles appearing progressively to give participants the impression that the bot was typing. As an example, here is the text participants saw after selecting the first counterargument that the chatbot presented at each step (sections in brackets did not appear to the participants):

Participant [first counterargument]: GMOs might not be safe to eat.

Chatbot [first rebuttal]: Did you know that the scientific consensus today is that genetically modified products on the market are as safe as non-genetically modified products? Each GMO is heavily tested before being introduced on the market. The testing process takes on average 13 years. Humans have been eating GMOs for more than 20 years and no ill effects have ever been reported. In 2016, an authoritative (and independent) report including more than 900 studies, from The National Academies of Science, Engineering, and Medicine concluded that there is "no substantial evidence of a difference in risks to human health between current commercially available genetically engineered crops and conventionally bred crops."

Participant [follow-up counterargument]: We don't know about the long-term effects.

Chatbot [follow-up rebuttal]: After over 40 years of research we have a good idea of the long-term effects of genetically modified food. On genetically modified corn alone more than 6000 studies have been published in scientific journals. A recent independent review of the scientific literature on long-term effects of genetically modified food concluded that: "genetically modified plants are nutritionally equivalent to their nongenetically modified counterparts and can be safely used in food and feed."

Arguments in favor of GMOs contained hyperlinks to scientific articles, reports from scientific agencies, and Wikipedia pages (which were identical in the Counterarguments text. By doing so we improved the ecological validity of the task, as participants were explicitly given leeway to engage with the arguments to the extent they wished-as they would if they had encountered the arguments in any other setting. Once they finished reading the arguments, participants answered the same questions regarding their GMOs attitudes as before the experimental task. Finally, participants provided basic demographic information (age, gender, education). Since data collection was automatized on Qualtrics and Prolific Academic, that all our statistical analyses were pre-registered, and that there was no subjective coding of the data, the experimenters were not blind to the conditions of the experiments. Participants were not blinded to the study hypotheses. However, since the experiment had a between-participants design, and that most of our hypotheses (except H6,8)

bear on comparisons across conditions, participants should not have been able to infer our hypotheses and act accordingly. See Table 1 for an overview of our design.

[INSERT TABLE 1]

Materials

The neutral GMOs description used in the Control Condition reads as follows:

Genetically modified organisms are plants and animals whose DNA has been modified in a laboratory.

In the Consensus Condition, participants were provided with an account of the scientific consensus accompanied by sources. This account was more detailed than the ones used by most studies highlighting the scientific consensus on GMOs, such as "Did you know?

A recent survey shows that 90% of scientists believe genetically modified foods are safe to eat." [START_REF] Dixon | Applying the Gateway Belief Model to Genetically Modified Food Perceptions: New Insights and Additional Questions[END_REF] The text used in the present experiment was:

There is a scientific consensus on the fact that genetically modified products on the market are as safe as non-genetically modified products. In 2016, an authoritative (and independent) report including more than 900 studies, from The National Academies of Science, Engineering, and Medicine concluded that there is "no substantial evidence of a difference in risks to human health between current commercially available genetically engineered crops and conventionally bred crops." 88% of scientists of the American Association for the Advancement of Science think that GM crops are safe to eat.

All the materials can be found on OSF at https://osf.io/cb7wf/ (in French and in English). The Control and the Consensus Condition were displayed on the survey platform Qualtrics. The Chatbot and the Counterarguments Condition (composed of all the counterarguments and rebuttals available on the chatbot) were displayed on the same custommade website. The only difference between the two conditions were that in the Chatbot Condition participants selected counterarguments, and thus only saw the rebuttals that address these counterarguments, whereas in the Counterarguments Condition participants scrolled through all the counterarguments and rebuttals. Figure 1 offers a visualization of the chatbot's interface.

Sampling Plan

Based on the literature and on our pilot study (see below), we expected the effect of the chatbot on the evolution of attitudes towards GMOs to be large. Previous studies have

shown that learning about the science behind genetic modification technology leads to more positive attitudes towards GMOs (ANOVA , p < .001, η² = .09) [START_REF] Mcphetres | Modifying attitudes about modified foods: increased knowledge leads to more positive attitudes[END_REF] , as does discussion of the scientific evidence on GMOs safety in small groups (ANOVA, p < .001, η² = 0.45) [START_REF] Altay | Are science festivals a good place to discuss heated topics?[END_REF] . In our pilot study, we found a large effect of the chatbot on attitude change (ANOVA, η² = 0.15).

However, because the current pre-registered study we compared the chatbot to controls, were some attitude change occurred, we expected the effect to be smaller (between small and medium instead of large).

We performed an a priori power analysis with G*Power3 76 . To compute the necessary number of participants, we decided that the minimal effect size of interest would correspond to a Cohen's d of 0.2 between two different experimental conditions, since this corresponds to what is generally seen as a small effect [START_REF] Cohen | Statistical power analysis for the behavioral sciences[END_REF] . Based on a correlation of 0.75 between the initial and final GMO attitudes (estimated from the pilot), we needed at least 275 participants per condition to detect this effect, at an α-level of 5%, a power of 95%, and based on a two-tailed test (see the 'Power analysis' section in SI for more details). We expected that approximatively 15% of participants would encounter problems accessing the chatbot's interface (a percentage estimated while pre-testing the chatbot). We planned to exclude these participants. To anticipate the losses in participants unable to access the chatbot, we planned on recruiting 324 participants (275/0.85) instead of 275 in the Chatbot Condition and in the Counterarguments Condition. We planned to recruit a total of 1198 UK participants on the crowdsourcing platform Prolific Academic. Data collection stopped when each condition reached the minimum number of participants required by the power analysis after exclusions (due to inability to access the chatbot).

During the Stage 2 peer-review process, reviewers noticed that our a priori power analysis at Stage 1 had not taken into account our correction for multiple testing and that it was based on a comparison of two conditions instead of four. To enable readers to evaluate the impact of this on our power calculations, we repeated our simulations with fixed sample sizes and a fixed effect size (the a priori defined minimum effect size of interest, i.e., d=.0.2).

These simulations showed that actual power to detect the smallest effect size of theoretical interest was higher than 91% in all conditions (see Supplementary Information for further details).

Participants

Between the 15th of October 2020 and the 26th of October

Analysis plan

All analyses were conducted with R (v.3.6.1) [START_REF] Team | A language and environment for statistical com-puting[END_REF] , using R Studio (v.1.2.5019) [START_REF] Team | RStudio: integrated development for R. RStudio[END_REF] . All statistical tests are two-sided. We refer to "statistically significant" as the p-value being lower than an alpha of 0.05. We controlled for multiple comparisons applying the Benjamini-Hochberg method to H1-8 (which controls for the False-Discovery Rate, and has a less negative impact on statistical power than alternative methods), but not to H9, for two reasons.

First, we had planned on testing H9 only if one of the first three hypotheses were supported.

As a consequence, H9 does not increase the familywise error rate (this is a special case of the closure principle in multiple comparisons [START_REF] Bretz | Multiple comparisons using R[END_REF] ). Second, since H9 was conducted only on a third of the participants, controlling for multiple comparisons would have reduced our statistical power even more. Due to this reduced power and the lack of correction, we were especially cautious in interpreting the results of H9.

All the p-values reported in the exploratory analyses have been corrected for multiple comparisons applying the Benjamini-Hochberg method. This correction included the p-values of the confirmatory analyses (whereas the correction applied to the p-values of the confirmatory analyses did not include the p-values of the exploratory analyses). We used this method to maximize power for the confirmatory analyzes (and conform to the pre-registered plan) while limiting the risk of false positives for the exploratory analyzes.

Given that we used null hypothesis statistical testing, null results were interpreted as the impossibility to reject H0, and as an absence of support for the hypothesis tested, but not as support for H0. Data from previous studies suggested that our experimental design would allow us to test our hypotheses. First, survey data on attitudes about GMOs in the UK, or other European countries, suggested that participants would be far from the ceiling (i.e. being maximally in favor of GMOs) [START_REF] Bonny | Factors Explaining Opposition to GMOs in France and the Rest of Europe[END_REF][START_REF] Bonny | Why are most Europeans opposed to GMOs?: Factors explaining rejection in France and Europe[END_REF] , so that we would be able to observe attitude change towards attitudes more favorable to GMOs. Second, previous studies using consensus messaging [START_REF] Dixon | Applying the Gateway Belief Model to Genetically Modified Food Perceptions: New Insights and Additional Questions[END_REF][START_REF] Kerr | Changes in perceived scientific consensus shift beliefs about climate change and GM food safety[END_REF] suggested that some attitude change should be observed in our Consensus Condition, which could thus be used as a positive control.

We compared participants' attitudes before and after being exposed to one of the four conditions by using a composite measure composed of the mean ratings of the four GMOs attitudes questions. In order for our measures to be more intuitive, we reverse-coded all but one of the questions (the first), such that higher numbers denote a more positive attitude towards GMOs. Time was measured by our custom-made website that provides a precise and reliable measure of the time spent by participants interacting with the chatbot in the Chatbot Condition or reading the arguments in the Counterarguments Condition. To estimate whether an effect was small enough to be considered negligible, we conducted equivalence tests using the "Two One-Sided Tests" (TOST) method [START_REF] Lakens | Equivalence tests: a practical primer for t tests, correlations, and metaanalyses[END_REF][START_REF] Campbell | Equivalence testing for standardized effect sizes in linear regression[END_REF] , which we implemented by computing 90% CI around the estimate of the regression coefficient. To further assist with interpretation, we provided Bayes Factors at Stage 2 for all confirmatory analyses (BF's were not registered at Stage 1). To calculate the Bayes Factors, we used the 'lmBF' function from the 'BayesFactor' R package (Package 'bayesfactor' [START_REF] Morey | Package 'bayesfactor[END_REF] ). The choice of priors and Markov chain Monte Carlo settings were the default settings of the package (as of 2020). The R script used to analyze the data, together with the mock dataset on which the script was tested, are available at https://osf.io/cb7wf/.

According to H1, participants will hold more positive attitudes towards GMOs after the experimental task in the Consensus Condition than in the Control Condition, when controlling for their initial attitudes towards GMOs.

According to H2, participants will hold more positive attitudes towards GMOs after the experimental task in the Counterarguments Condition than in the Control Condition, when controlling for their initial attitudes towards GMOs.

According to H3, participants will hold more positive attitudes towards GMOs after the experimental task in the Chatbot Condition than in the Control Condition, when controlling for their initial attitudes towards GMOs.

H1-3 were tested on the full dataset with one multivariate regression. Attitudes after the experimental task were set as the dependent variable, while attitudes before the experimental According to H6, in the Chatbot Condition, the number of arguments explored by the participants will predict holding positive attitudes towards GMOs after the experimental task when controlling for their initial attitudes towards GMOs (i.e. exploring more arguments should lead to more positive attitude change).

H6 was tested with one multivariate regression among participants in the Chatbot Condition, with attitudes after the experimental task as the dependent variable, and attitudes before the experimental task together with the total number of arguments explored by participants as predictors.

According to H7, in the Chatbot Condition, time spent on the task should lead to more positive attitude towards GMOs after the experimental task than time spent on the Counterarguments Condition, when controlling for their initial attitudes towards GMOs.

H7 was tested with one multivariate regression among participants in the Chatbot

Condition and the Counterarguments Condition, with attitudes after the experimental task as the dependent variable, and using the Time variable, the Condition variable, and an interaction between the Time variable and the Condition variable as predictors. The Time variable was mean-centered to facilitate the interpretation of the regression coefficients.

The four questions measuring attitudes towards GMOs correspond to concerns about GM foods safety, GMOs' ecological impact, GMO's usefulness, and the socio-economic dimension of GMOs. The internal consistency of the scale was higher after the treatment (α = .79) than before the treatment (α = .68), but this effect was mostly driven by the Chatbot (pre:

.67, post: .79) and Counterarguments Condition (pre: .66, post: .81) rather than the Control (pre: .70, post: .71) and Consensus Condition (pre: .68, post: .71). The chatbot menu is also composed of four main counterarguments against GMOs: "GMOs might not be safe to eat," which targets health concerns, "GMOs could hurt the planet," which targets ecological concerns, "The way GMOs are commercialized is problematic," which targets economic concerns and "We don't really need GMOs," which targets the usefulness of GMOs. Each of these main counterarguments is answered by a rebuttal, which can then be answered by several counterarguments, which have their own rebuttals, and so forth. According to H8, on the Chatbot Condition, participants will hold more positive attitudes after the experimental task on issues for which they have explored more of the relevant rebuttals targeted at the issue, when controlling for their initial attitudes on the issues and the total number of arguments explored (not necessarily related to the issue). To test H8, we counted the number of arguments explored in each of the four branches (between zero and nine).

To investigate the relation between the type of arguments that participants explored on the chatbot and attitude change on these particular aspects of GMOs (health, ecology, economy and usefulness), we conducted a linear mixed-effects model with participants as random effect (varying intercepts), attitudes after the experimental task on a specific issue as the dependent variable, and number of arguments explored on the same specific issue, together with attitudes before the experimental task on the same issue, the total number of arguments explored, and the type of issue as predictors.

According to H8, we expect that the number of arguments explored by participants on the topic related to the subscale will be a predictor of more favorable attitudes towards GMOs after having interacted with the chatbot, when controlling for their initial attitudes towards GMOs.

According to H9, H1, H2, and H3, should hold true among the third of the participants most opposed to GMOs.

H9 was tested by conducting the same analysis used to test H1, H2, and H3 (i.e. a multivariate regression with attitudes after the experimental as dependent variable, and attitudes before the experimental task together with condition as predictors-with the Control Condition as the baseline for the variable Condition) among the one third of participants initially holding the most negative attitudes toward GMOs.

We made no predictions regarding gender, education, or other socio-demographic variables. We did not add these variables in the models since their influence should mostly be taken into account when controlling for initial attitudes. For all interpretations, the effects will be characterized not only by their statistical significance (below the 0.05 alpha threshold as specified in the manuscript) but also by their size. For brevity we will only refer here to "significant" and "not significant," but in the final manuscript more attention will be paid to effect sizes. Since we will use two-sided tests, we will be able to interpret effects in the opposite direction of what we predicted.

Tables Table 1. Design Table

To estimate whether an effect will be small enough to be considered negligible, we will conduct equivalence tests using the "Two One-Sided Tests" (TOST) method [START_REF] Lakens | Equivalence tests: a practical primer for t tests, correlations, and metaanalyses[END_REF][START_REF] Campbell | Equivalence testing for standardized effect sizes in linear regression[END_REF] If we find a significant difference in the opposite direction of what we predicted, we will conclude that H7 is not supported and that the opposite of H7 is supported (in this case that not interacting with the chatbot was more efficient at changing people's minds in favor of GMOs).

If we find no significant effect, we will conclude that we did not find support for the hypothesis.

If we find no significant difference, we will conclude that we cannot reject the null hypothesis. Then, using the TOST method, we will compute the If we find a significant difference in the opposite direction of what we predicted, we will conclude that H8 is not supported and that the opposite of H8 is supported.

If we find no significant difference, we will conclude that we cannot reject the null hypothesis. Then, using the TOST method, we will compute the If we find no significant difference, we will conclude that we cannot reject the null hypothesis. Then, using the TOST method, will compute the 90%

Confidence Interval around the regression coefficient of Condition for each hypothesis (i.e. H1-3); if this Confidence Interval includes neither the value of -0.1 nor the value of 0.1, we will declare the effect to be practically negligible.

48

Again, due to the reduced power and the lack of correction to test H9, we will be especially cautious in interpreting the results of H9.

Except for H9 all the hypotheses presuppose that participants will, on average, either not change their opinion between the pre-and the post-treatment questions, or that they will become more favorable towards GMOs (i.e. there is either a significantly positive change, or a significant absence of difference, as tested with an equivalence test). If, on the contrary, we find instead that in one or more condition, participants became significantly less favorable towards GMOs, this would provide evidence for a backfire effect.

  the scientific consensus, as in the Consensus Condition. Comparing the attitude change obtained in the Consensus and the Counterarguments Conditions allowed us to test whether countering participants' arguments, instead of only presenting a forceful argument, was more effective at changing people's minds. Fourth, in the Chatbot Condition, participants could read exactly the same materials as in the Counterarguments Condition, but through a chatbot, enabling them to easily access the most relevant, and only the most relevant, rebuttals to their counterarguments (the workings of the Chatbot is detailed in the Design section; see Figure 1 for a visualization of the chatbot's interface). Comparing the changes of minds obtained in the Chatbot and the Counterarguments Condition allowed us to test whether presenting participants only with the rebuttals that are most relevant for them leads to more ample changes of mind.

Figure 1 .

 1 Figure 1. The beginning of a conversation with the chatbot. The screen-wide dialogue bubbles correspond to past interactions, with the participant's counterarguments in blue, and the chatbot's rebuttals in beige. The right-justified blue bubbles present the participant's choices at this stage of the interaction.

H1:

  Participants will hold more positive attitudes towards GMOs after the experimental task in the Consensus Condition than in the Control Condition, controlling for their initial attitudes towards GMOs. H2: Participants will hold more positive attitudes towards GMOs after the experimental task in the Counterarguments Condition than in the Control Condition, controlling for their initial attitudes towards GMOs. H3: Participants will hold more positive attitudes towards GMOs after the experimental task in the Chatbot Condition than in the Control Condition, controlling for their initial attitudes towards GMOs. H4: Participants will hold more positive attitudes towards GMOs after the experimental task in the Counterarguments Condition than in the Consensus Condition, controlling for their initial attitudes towards GMOs. H5: Participants will hold more positive attitudes towards GMOs after the experimental task in the Chatbot Condition than in the Counterarguments Condition, controlling for their initial attitudes towards GMOs. H6: In the Chatbot Condition, the number of arguments explored by participants will predict holding more positive attitudes towards GMOs after the experimental task, controlling for their initial attitudes towards GMOs (i.e. exploring more arguments should lead to more positive attitude change). H7: In the Chatbot Condition, time spent on the task should lead to more positive attitudes towards GMOs after the experimental task than time spent on the Counterarguments Condition, controlling for their initial attitudes towards GMOs. Participants were given the opportunity to read many more arguments in the Counterarguments Condition and in the Chatbot Condition than in the Consensus Condition.

Figure 2 .

 2 Figure 2. Distribution of participants' attitudes towards GMOs. Density plots

Figure 3 .

 3 Figure 3. Evolution of participants' attitudes toward GMOs. Each dot represents a

  ' attitudes towards GMOs before treatment and after treatment. Control Condition (N = 275): a sentence describing what GMOs are; Consensus Condition (N = 273): a paragraph on the scientific consensus on GMOs safety; Counterarguments Condition (N = 299): a text with the most common counterarguments against GMOs, together with their rebuttal; Chatbot Condition (N = 302). Confirmatory analyses In line with H1, participants held more positive attitudes towards GMOs after the treatment in the Consensus Condition than in the Control Condition (b = 0.37 [0.23, 0.51], t (1144) = 5.36, p < .001, BF10 = 10 4 ). In other words, being exposed to the scientific consensus led to a 0.37point (on 7 point-scale) move towards pro-GMOs opinions. In line with H2, participants held more positive attitudes towards GMOs after the treatment in the Counterarguments Condition than in the Control Condition (b = 0.99 [0.86, 1.12], t (1144) = 14.65, p < .001, BF10 = 10 41 ). That is, scrolling through the chatbot's arguments led to a 0.99-point move towards pro-GMOs opinions. In line with H3, participants held more positive attitudes towards GMOs after the treatment in the Chatbot Condition than in the Control Condition (b = 0.77 [0.63, 0.90], t (1144) = 11.38, p < .001, BF10 = 10 25 ). In sum, interacting with the chatbot by clicking on the arguments led to 0.77-point move towards pro-GMOs opinions. In line with H4, participants held more positive attitudes towards GMOs after the treatment in the Counterarguments Condition than in the Consensus Condition (b = 0.62 [0.49, 0.75], t (1144) = 9.15, p < .001, BF10 = 10 16 ). In other words, scrolling through the chatbot's arguments led to a 0.62-point move towards pro-GMOs opinions compared to exposure to the scientific consensus. Contrary to H5, participants held more positive attitudes towards GMOs after the treatment in the Counterarguments Condition than in the Chatbot Condition (b = 0.22 [0.09, 0.35], t (1144) = 3.37, p < .001, BF10 = 15.32). That is, scrolling through the chatbot's arguments led to a 0.22-point move towards pro-GMOs opinions compared to interacting with the chatbot by clicking on the arguments. In line with H6, the number of arguments explored by participants in the Chatbot Condition predicted holding more positive attitudes towards GMOs after the treatment (b = 0.04 [0.02, 0.06], t (299) = 4.09, p < .001, BF10 = 14.41). That is, each additional argument that participants explored led to a 0.04-point move towards pro-GMOs opinions. Contrary to H7, time spent in the Chatbot Condition did not lead to significantly more positive attitudes towards GMOs after the treatment than time spent on the Counterarguments Condition (b = 0.004 [-0.05, 0.04], t (596) = 0.20, p = .84, BF10 = 0.08). This effect is negligible as the 90% CI [-0.08, 0.06] falls inside the pre-registered [-0.1, 0.1] interval corresponding to an effect smaller than ß = 0.1. Figure 4 offers a visual representation of the interaction. In both conditions, time spent on the task led to more positive attitudes towards GMOs (b = 0.07 [0.05, 0.09], t (596) = 7.54, p < .001, BF10 = 10 12 ; Chatbot Condition: b = 0.07 [0.03, 0.10], t (299) = 3.36, p < .001, BF10 = 19; Counterarguments Condition: b = 0.07 [0.05, 0.09], t (296) = 7.35, p < .001, BF10 = 10 9 ). In other words, the longer participants spent on the chatbot's interface, the more they developed pro-GMOs opinions.

Figure 4 .

 4 Figure 4. Relationship between time spent on the treatment and attitude change in the Counterarguments and Chatbot conditions. The blue and yellow lines represent the regression lines of Attitude change on Time spent in each condition. The shaded area represents the 95% CI. The density plots represent the distributions of Attitude Change and Time Spent in each condition. Counterarguments Condition (N = 299): a text with the most common counterarguments against GMOs, together with their rebuttal; Chatbot Condition (N = 302).

  line with H9, H1-3 held true among the third of the participants initially holding the most negative attitudes towards GMOs (Chatbot Condition: b = 0.93 [0.67, 1.20], t (376) = 6.92, p < .001, BF10 = 10 8 ; Counterarguments Condition: b = 1.35 [1.09, 1.62], t (376) = 9.96, p < .001, BF10 = 10 17 ; Consensus Condition: b = 0.47 [0.29, 0.75], t (376) = 3.33, p < .001, BF10 = 24). That is, being exposed to the consensus, and scrolling or clicking through the chatbot's arguments, led to more pro-GMOs opinions even among the third of participants initially holding the most anti-GMOs opinions. Exploratory analyses To assess whether time spent on the task might explain the greater impact of the Counterarguments Condition compared to the Chatbot Condition, we tested whether participants had spent more time in the former than the latter. They had: Counterarguments Condition, M = 5.74 minutes, SD = 5.63 minutes; Chatbot Condition, M = 3.70 minutes, SD = 2.56 minutes; t (415.48) = 5.73, p < .001. In a regression model without time as predictor, Condition (Chatbot vs Counterarguments) was a significant predictor of attitude change (b = 0.23 [0.07, 0.38], t (599) = 2.83, p = .008), but when adding time spent in the model the effect of Condition was not significant anymore (b = 0.08 [-0.07, 0.23], t (598) = 1.01, p = .37), whereas the effect of time was (b = 0.07 [0.05, 0.09], t (598) = 8.29, p < .001). Then, a mediation analysis (nonparametric bootstrap confidence intervals with the percentile method 65 ) suggested that 65% of the effect of condition was mediated by time ([0.37, 1.84], p = .004), with an indirect effect via the time mediator estimated to be b = .15 [0.10, 0.21], p < .001. However, this should not be taken as proof of causality because non-observed variables could create (or inflate) the correlation observed between time and attitude change 66 . Nevertheless, time remains a credible mediator since it plausibly plays a role in attitude change, and more time spent reading the arguments might translate into greater attitude change. To investigate H9 further, we examined the relationship between participants' initial attitudes, and attitude change. More precisely we tested the interaction between participants' initial attitudes and the experimental condition (with the Control Condition as baseline) on attitude change. By contrast with H9, here all the participants are included in the analysis. We found that, compared to the Control Condition, participants initially holding more negative attitudes displayed more attitude change in favor of GMOs in the Counterarguments Condition (b = 0.30 [0.

  Participants' attitudes toward GMOs became more positive after having interacted with the chatbot (t (69) = 3.68, p < .001, d = 0.28, 95% CI [0.13, 0.44]. (ii) The number of arguments explored by participants significantly predicted a larger shift towards positive attitudes towards GMOs (β = 0.23, 95% CI [0.09, 0.37] t (67) = 3.32, p = .001). (iii) Participants who only provided their attitudes toward GMOs after having interacted with the chatbot did not have significantly different attitudes towards GMOs compared to participants who provided their attitudes toward GMOs both before and after having interacted with the chatbot (t (138.57) = 0.71, p = .48, d = 0.14, 95% CI = [-0.21, 0.44]). (iv) Participants judged the bot as quite enjoyable (M = 60.13, SD = 25.3), intuitive (M = 65.94, SD = 26.45), and not very frustrating (M = 35.39, SD = 31.22) (all scales from 0 to 100).

  task and condition were set as predictors. The Control Condition was set as the baseline for the variable Condition. In other words, Consensus Condition, Counterarguments Condition, and Chatbot Condition, were each compared to the Control Condition. Attitudes before the experimental task was mean-centered, in order to facilitate the interpretation of the intercept, which corresponds to the mean post-attitude for the control condition. According to H4, participants will hold more positive attitudes towards GMOs after the experimental task in the Counterarguments Condition than in the Consensus Condition, controlling for their initial attitudes towards GMOs. H4 was based on the same regression model as in H1-3, we conducted a linear contrast analysis between the Counterarguments Condition and the Consensus Condition. H4 led us to expect that the Consensus Condition would predict less attitude change in the direction of more positive attitudes towards GMOs than the Counterarguments Condition. According to H5, participants will hold more positive attitudes towards GMOs after the experimental task in the Chatbot Condition than in the Counterarguments Condition, controlling for their initial attitudes towards GMOs H5 was based on the same regression model as in H1-3, we conducted a linear contrast analysis between the Chatbot and the Counterarguments Conditions. H5 led us to expect that the Counterarguments Condition would predict less attitude change in the direction of more positive attitudes towards GMOs than the Chatbot Condition.

  less attitude change inIf we find a significant difference in the expected direction, we will conclude that interacting with the chatbot led to more positive attitudes towards GMOs than reading the counterarguments and rebuttals available on the chatbot without being able to interact with it, and that H5 is supported.If we find a significant difference in the opposite direction of what we predicted, we will conclude that H5 is not supported and that the opposite of H5 is supported (in this case that chatbot led to less positive attitude change toward GMOs than reading the arguments available on the chatbot).If we find no significant difference, we will conclude that we cannot reject the null hypothesis. Then, using the TOST method, we will compute the 90% Confidence Interval around the regression coefficient of Condition; if this Confidence Interval includes neither the value of -0.1 nor the value of 0.1, we will declare the effect to be practically negligibleof arguments explored by participants significantly predict more positive attitudes toward GMOs we will conclude that the more arguments participants are exposed to the more they change their minds in favor of GMOs, supporting H6. If we find a significant positive difference in the opposite direction of what we predicted, we will conclude that H6 is not supported and that the opposite of H6 is supported (in this case that being exposed to fewer arguments led to more positive attitude change toward GMOs).If we find no significant difference, we will conclude that we cannot reject the null hypothesis. Then, using the TOST method, we will compute the 90% Confidence Interval around the regression coefficient of number of arguments; if this Confidence Interval includes neither the value of -0.1 nor the value of 0.1, we will declare the effect to be practically negligible. interacting with the chatbot led to more attitude change in favor of GMOs in the Chatbot condition than in the Counterarguments condition, we will conclude that being exposed to only relevant counterarguments is more efficient at changing people's minds in favor of GMOs than presenting them with potentially irrelevant arguments.

90%

  Confidence Interval around the regression coefficient of the interaction between Time and Condition; if this Confidence Interval includes neither the value of -0.1 nor the value of 0.1, we will declare the effect to be practically the same issue, the total number of If after having interacted with the chatbot participants hold more positive attitudes on issues for which they have explored more of the rebuttals related to the issue, we will conclude that participants payed attention to the content of the arguments and probably changed their minds because of arguments' content and did not use a low-level heuristic in which they are convinced by the sheer number of arguments.

90%

  H2, and H3 hold true among the third of the participants initially holding the most negative attitudes about GMOs, we will conclude that we did not find evidence in favor of the backfire effect and that attitude change was not dissimilar between the third of the participants initially holding the most negative attitudes about GMOs and the rest of participants. If participants holding the most negative attitudes about GMOs showed more positive attitude change toward GMOs than the rest of participants H9 will still be supported. On the other hand, if the effect goes in the opposite direction of what we predicted, we will conclude that the opposite of H9 is supported (i.e. it will be evidence in favor of a backfire effect).

Acknowledgements

This research was supported by the CONFIRMA grant from the Direction Générale de L'armement, together with the following grants: ANR-17-EURE-0017 to FrontCog and ANR-10-IDEX-0001-02 to PSL. The first author's PhD thesis is funded by the Direction Générale de l'Armement (DGA). The funders have had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We would like to thank Camille Williams for statistical advice.

Data availability

The data associated with this research, together with the code of the chatbot and the materials, are available on OSF at: https://osf.io/cb7wf/.

Code availability

The R scripts associated with this research are available on OSF at: https://osf.io/cb7wf/.

Condition). At any time, users had the possibility of coming back to the first four basic counterarguments of the main menu, or of exiting the chatbot.

Experimental procedure

Participants were asked to either read a simple explanation of what a GMO is (Control Condition), read a short paragraph on the scientific consensus on the safety of GM food (Consensus Condition), read counterarguments to GMOs accompanied by rebuttals of these arguments (Counterarguments Condition), or explore the same counterarguments and rebuttals by interacting with a Chatbot (Chatbot Condition).

Participants first had to complete a consent form and answer a few questions to measure their attitudes towards GMOs. Participants had to express the extent to which they agreed with the four following statements on a seven-point Likert scale:

-Genetically modified food is safe to eat.

-Genetically modified organisms (GMO) are bad for the environment.

-GMOs are useless.

-I'm worried about the socio-economic impacts of GMOs (on farmers in poor countries, wealth distribution, lack of competition, etc.).

In all analyses (except for H8) these four variables were treated as a single composite variable, that we refer to as "GMOs attitude." Next, participants were presented with one of the four following conditions:

Participants were randomly assigned to one of the four conditions by a pseudo-randomizer on the survey platform "Qualtrics" (i.e. a randomizer that ensures that an equal number of participants is attributed to each condition). In all the conditions, participants were told to spend as much or as little time as they wanted interacting with the chatbot and exploring the

Protocol Registration

The Stage 1 protocol for this Registered Report was accepted in principle on October 8, 2020.

The protocol, as accepted by the journal, can be found at https://doi.org/10.6084/m9.figshare.13122527.v1

The sentence 'Although it is methodologically impossible to completely disentangle the effects of the mode of presentation (e.g. degree of interactivity) and of the specific information presented, the present experiment provides the first test of whether addressing people's counterarguments, in particular by using an interactive chatbot, results in attitude changes that are larger than those obtained with a common messaging technique.' was modified in the stage 1 portion of the manuscript before acceptance at stage 2 on editorial request, in order to comply with journal policies regarding priority claims.

The following paragraph "However these studies remain limited in particular as they did not include control groups comparable to the present control conditions. Instead the chatbots were compared either (i) to argumentation between participants31 (ii) to a chatbot that could not address all counterarguments30 or (iii) to a chatbot that did not take into account users' counterarguments 32. Moreover the robustness of these results is questionable since their design had poor sensitivity to detect even large effect sizes of dz = 0.5 (the study with the greatest sensitivity 32, which recruited 25 participants per condition on average, had no more than 0.71 power to detect large effects (dz = 0.5) with an alpha of 0.05; in the other studies power was even lower: 0.49 31 and 0. 52 30)." was removed from the Stage 2 submission because since in principle acceptance of the Stage 1 protocol studies have been published that are not subject to these limitations [START_REF] Altay | Information Delivered by a Chatbot Has a Positive Impact on COVID-19 Vaccines Attitudes and Intentions[END_REF] .

Author Contributions

Competing Interests

The authors declare no competing interests. If we find no significant difference, we will conclude that we cannot reject the null hypothesis. Then, using the TOST method, we will compute the


est destinée au dépôt

Gain-of-function and loss-of-function GABRB3 variants lead to distinct clinical phenotypes in patients with developmental and epileptic encephalopathies

Nathan Absalom, Vivian Liao, Katrine Johannesen, Elena Gardella, Julia Jacobs, Gaetan Lesca, Zeynep Gokce-Samar, Alexis Arzimanoglou, Shimriet Zeidler, Pasquale Striano, et al.

To cite this version:

Nathan Absalom, Vivian Liao, Katrine Johannesen, Elena Gardella, Julia Jacobs, et al.. Gainof-function and loss-of-function GABRB3 variants lead to distinct clinical phenotypes in patients with developmental and epileptic encephalopathies. Nature Communications, 2022, 13 (1), pp.1822. 10.1038/s41467-022-29280-x. hal-03633628