9 (12 % [6 %; 22 %]) 0 No 17 (22 % [14 %; 33 %]) 0 Unclear 51 (66 % [54 %; 76 %]) 22 (100 %) Intention to share other research data Yes 6 (8 % [3 %; 17 %]) 7 (32 % [15 %; 55 %]) No 14 (18 % [11 %; 29 %]) 0 Unclear 57 (74 % [63 %; 84 %]) 15 (68 % [45 %; 85 %]) Intention to share any other documents Yes 63 (82 % [71 %; 89 %]) 10 (46 % [25 %; 67 %]) No 1 (1 % [0.1 %; 8 %]) 0 Unclear 13 (17 % [10 %; 28 %]) 12 (54% [33 %; 75 %]) Restriction of time for availability Yes 34 (44 % [33 %; 56 %]) 10 (46 % [25 %; 67 %]) No 22 (29 % [19 %; 40 %]) 8 (36 % [18 %, 59 %]) Unclear 21 (27 % [18 %; 39 %] 4 (18 % [6 %; 41 %]) Free access Yes 7 (9 % [4 %; 18 %]) 1 (4 % [0.2 %; 25 %]) No 69 (90 % [80 %; 95 %]) 21 (96 % [75 %; 99 %]) Unclear 1 (1 % [0.1 %; 8 %]) 0 Possibility to use data for any type of purpose Yes 7 (9 % [4 %; 18 %]) 1 (4 % [0.2 %; 25 %]) No 60 (78 % [

Reproducibility in Therapeutic Research -A survey on Data Sharing in the biomedical literature and Clinical Trials in Marketing Authorization

Abstract

Several researchers in biomedicine have described a reproducibility crisis. Various open science practices may maximize reproducibility. This thesis focuses on data sharing and its extent in the biomedical sciences. In the first part, we wanted to explore the implementation of the data sharing policy of the International Committee of Medical Journal Editors (ICMJE), which came into effect in July 2018. Implementation of the data sharing requirements in journal policies was suboptimal for ICMJE member journals and poor for ICMJE affiliated journals. In a second step, we conducted a scoping review to explore the impact of data-sharing initiatives on the intent to share data, actual data sharing, use of shared data, and research output and impact of shared data.

We concluded that there is currently a gap in the evidence base regarding the impact of sharing individual patient data, resulting in uncertainties in implementing current data sharing policies.

Researchers have high intentions to share data but rarely do so.

In the third part of the thesis, the emphasis was on transparency regarding clinical trials in drug regulatory frameworks. We tried to reanalyze 62 studies marked as main trials in marketing authorization applications. Our results showed that individual patient data was available for only 10 of 62 trials (16.1%). The clear message from this research is that clinical trial data for licensed drugs remains inaccessible to the public and the research community. Importantly, re-analyzes of the few trials with available data showed good reproducibility. In the final part, we suggest

ideas on advancing open science methods in drug regulatory contexts.

In summary, we concluded that sharing data in the biomedical literature is substandard. The main factors are the absence of mandatory data sharing policies on journals, publishers, and regulatory agencies. Adequate policies need to be implemented.

Substantial French Summary

En 2016, 1576 chercheurs de disciplines scientifiques ont répondu à une courte enquête par questionnaire sur la reproductibilité de la recherche pour la revue Nature. Quoi qu'il en soit, ce processus ne semble pas si bien se faire en pratique. L'absence de contrôle des biais lors de la spécification des hypothèses, la faible puissance statistique dans la conception de l'étude, le mauvais contrôle de la qualité et le p-hacking dans la collecte et l'analyse des données et les biais de publication dans la diffusion des résultats sont des problèmes majeurs qui rendent le processus d'autocorrection, parfois trop lent, parfois inefficace et qui, in fine renvoie ce processus au second plan dans un contexte ou la recherche est compétitive et l'innovation survalorisée. Ces problèmes pourraient être soulignés par le fait que les résultats positifs ont augmenté de plus de 22 % entre 1990 et 2007.

Des solutions actuelles de développement comme la science ouverte, qui permettrait de restaurer un cycle vertueux propre au raisonnement hypothético-déductif et propice à l'autocorrection.

Parmi les outils de la science ouverte, nous nous intéresserons dans cette thèse à l'intérêt du partage des données.

Premièrement, parce qu'aborder tous les points briserait la portée de cette thèse, et deuxièmement parce qu'en partageant les données, les risques mentionnés dans le modèle hypothético-déductif peuvent être contrôlés ou limités. Un mauvais contrôle de la qualité et un biais de publication peuvent être détectés lorsque les données sont librement disponibles et piratées, et donc indirectement, le HARKing (hypothesizing after results are known) peut être réduit lorsque des ré-analyses indépendantes sont possibles avec un protocole enregistré a priori et des données ouvertement partagées.

Si les problèmes de transparence et de reproductibilité dans la recherche ont trouvé leur chemin très tôt dans des domaines comme la psychologie ou la physique, il a fallu du temps à la recherche biomédicale pour trouver sa voie. Bien que pionnière sur l'enregistrement a priori des essais thérapeutiques, la recherche biomédicale est en retard sur la question du partage des données.

Certains experts voient aussi la crise actuelle un problème systémique, qui peut être évalué en partie en étudiant les politiques de partage des journaux, des financeurs et des autorités sanitaires. Ces intervenants sont en pratique en mesure d'établir les incitatifs pour améliorer la transparence des essais thérapeutiques.

Cette thèse fait partie d'un projet financé par l'Agence Nationale de la Recherche (ANR-17-CE-36-0010-01). L'idée était d'avoir deux doctorants travaillant sur des tâches similaires qui se complètent.

Un doctorant étudie les politiques de partage de données des financeurs des essais cliniques et, dans un deuxième temps, la reproductibilité inférentielle des essais cliniques de phase III que l'on peut trouver sur les principales plateformes de partage de données.

Le deuxième doctorant, l'auteur de cette thèse, se concentre sur la manière dont les politiques de partage de données sont intégrées dans les revues biomédicales. Dans un autre projet, des ré-analyses d'essais cliniques qui ont conduit à une autorisation de mise sur le marché au sein de l'Union européenne sont menées.

Impliqué dans un consortium international, ce projet devrait permettre de clarifier et de mieux comprendre la place actuelle du partage de données dans la recherche thérapeutique.

Dans cette thèse, quatre questions de recherche seront abordées :

1. Est-ce que les revues biomédicales implémentent les politiques recommandées de partage des données ? Et si oui, dans quelle mesure ?

2. Quels éléments supportent le partage de données dans la littérature biomédicale ?

3. Pouvons-nous obtenir les données et reproduire les principaux résultats de 62 essais pivots faisant partie des autorisations de mise sur le marché évaluées par l'agence Européenne du médicament ?

4. Comment intégrer les méthodes de la science ouverte dans l'approbation des médicaments ?

Question 1 :

Est-ce que les revues biomédicales implémentent les politiques recommandées de partage des données ? Et si oui, dans quelle mesure ?

Dans la première partie, nous voulions explorer la mise en oeuvre de la politique de partage de données du Comité international des éditeurs de revues médicales (ICMJE) qui est entrée en vigueur le 1er juillet 2018 par les revues membres de l'ICMJE et par les revues affiliées à l'ICMJE déclarant suivre les recommandations de l'ICMJE.

Une enquête transversale sur les politiques de partage de données en 2018 sur les sites Web des journaux biomédicaux a été menée.

Étaient incluses les revues membres de l'ICMJE et 489 revues affiliées à l'ICMJE qui avaient :

1/ publié un Essai Clinique Randomisé (ECR) en 2018, 2/ un site Web en ligne accessible, et qui n'étaient pas considérées comme des revues prédatrices selon la liste de Beall. Dans un second temps ont été analysés 1/ 100 ECRs pour les revues membres et 2/ 100 ECRs pour les revues affiliées (échantillons tirés au sort).

Le critère de jugement principal était l'existence d'une politique de partage de données (politique de partage de données explicite, aucune politique de partage de données, politique se référant simplement aux recommandations de l'ICMJE) comme indiqué sur le site Web de la revue, en particulier dans les instructions aux auteurs. Pour les ECRs, nous avons décrit l'intention de partager les données individuelles des participants comme exprimée dans la déclaration de partage des données de l'article.

Huit (sur 14 ; 57 %) des revues membres avaient une politique de partage de données explicite sur leur site Web (trois étaient plus strictes que les exigences de l'ICMJE, une était moins exigeante et quatre étaient conformes), cinq (35 %) faisaient simplement référence aux exigences de l'ICMJE. Dans les ECR publiés dans des revues affiliées avec une politique explicite de partage de données, les déclarations de partage de données étaient rares (25 %) et des intentions exprimées de partager des données ont été trouvées dans 22 % (15 % à 32 %).

La mise en oeuvre des exigences de partage de données de l'ICMJE dans les politiques des revues en ligne était sous-optimale pour les revues membres de l'ICMJE et médiocre pour les revues affiliées à l'ICMJE. La mise en oeuvre de la politique était bonne dans les revues membres et préoccupante pour les revues affiliées. Nous suggérons la conduite d'audits continus des politiques de partage des données des revues médicales à l'avenir.

Question 2 :

Quels éléments supportent le partage de données dans la littérature biomédicale ?

Dans une deuxième étape, nous avons mené une « scoping review » pour explorer l'impact des initiatives de partage de données sur l'intention de partager des données, sur le partage réel des données, sur l'utilisation des données partagées et sur les résultats de la recherche et l'impact des données partagées.

Toutes les études portant sur les pratiques de partage de données pour les données des participants individuels provenant d'essais cliniques ont été sélectionnées. Nous avons conclu qu'il existe actuellement un manque de connaissance important concernant la pratique du partage des données issues des ECRs, en particulier en qui concerne l'impact de cette pratique. Il y a de grandes incertitudes sur l'impact potentiel des politiques actuelles de partage des données. Des preuves de haut niveau sont nécessaires pour évaluer si la valeur de la recherche médicale augmente avec les pratiques de partage de données. Nos résultats ont montré que pour seulement 10 des 62 essais (16,1 % ; 8% à 27.7%), les données individuelles des patients étaient disponibles. Dans tous les essais, nous sommes arrivés à la même conclusion en termes de significativité statistique. Cependant, nous n'avons pas encore pu reproduire le critère de jugement principal dans deux essais car les variables correspondantes aux critères de jugement étaient manquantes dans les données reçues. Nous laissant avec huit essais entièrement réalisés et résultant en un taux de reproductibilité de 12,9 % (5.7% à 23.9%).

Les données d'essais cliniques des médicaments autorisés restent largement inaccessibles au public et à la communauté des chercheurs. Lorsque ces données sont accessibles, les analyses sont reproduites dans leur grande majorité.

Question 4 :

Comment intégrer les méthodes de la science ouverte dans l'approbation des médicaments ?

Dans la dernière partie, nous proposons d'aller un peu au-delà de la notion de partage des données et de réfléchir dans quelle mesure la science ouverte pourrait améliorer les pratiques des autorités sanitaires. Avant l'approbation des médicaments, les autorités sanitaires établissent un ensemble de règles a priori selon lesquelles les essais cliniques seront jugés « positifs » ou « négatifs ». Trop souvent, ils enfreignent ces règles post-hoc. Cela donne lieu à des débats controversés quant à savoir si les seuils appropriés de réussite ont été atteints. La voie de la science ouverte que nous proposons pour l'autorisation de mise sur le marché des médicaments -l'approbation enregistrée des médicaments (en anglais : registered drug approval) -vise à adapter le concept de rapports enregistrés (en anglais : registered reports) au processus d'approbation réglementaire des médicaments et d'autorisation de mise sur le marché. Il valorise l'importance clinique des questions de recherche, avec une approbation de principe accordée sur des critères de succès prédéfinis. Ceci est fait avant la collecte des données, empêchant toute altération post hoc des règles et permettant une transparence totale. Le partage des données et de tous les documents liés aux études pivots sera central.

Conclusion

En résumé, le partage des données dans la recherche thérapeutique est sous-optimal. Les politiques de partage, quand elles existent, sont laxistes et mal implémentées. Nous proposons plusieurs pistes pour aller vers un changement.

Bien entendu, nous n'avons exploré qu'une petite partie de la notion de reproductibilité (reproductibilité inférentielle) dans les travaux de cette thèse, à la lumière du partage des données des essais cliniques. D'autres aspects comme la réplicabilité nécessitent d'autres approches. Très récemment, une étude d'envergure a montré que la reproductibilité de la recherche préclinique en biologie du cancer est là aussi décevante.

Ainsi, pour mieux cerner les problèmes de reproductibilité, les efforts de méta-recherche devront trianguler les résultats de différents types de recherche. C'est d'ailleurs, selon Munafò, la protection essentielle face à la mauvaise science. L'idée est d'aborder la même question de recherche sous un angle différent. Chaque approche a ses hypothèses, ses forces et ses faiblesses indépendantes. Une image complète d'une question de recherche est créée en combinant les différents aspects.

Nous avons pris le parti dans cette thèse d'appliquer quelques recettes qui rendent la science le plus reproductible possible :

• Avoir une vision claire de la littérature (scoping review) ;

• Enregistrer nos questions a priori (l'étude sur les autorités sanitaires a été acceptée de principe à BMC Medicine en tant que registered report) ;

• Partage de nos données, nos codes ;

• Trianguler les recherches en collaborant autant que faire se peut à notre échelle.

Néanmoins, le partage de données est une cible mouvante dans un environnement en évolution rapide et on ne peut qu'espérer que la situation s'améliorera avec le temps et que les parties prenantes s'adapteront aux changements suggérés. In 2016, 1576 researchers from science disciplines replied to a short questionnaire survey on reproducibility in research in the journal Nature. 52% of the participants replied that there is a "significant reproducibility crisis," and 38% said there is a slight one. Only 3% indicated that there is not any. Furthermore, between 60 and 90% of the researchers admitted that they encountered difficulties reproducing results of other research teams across different research domains. At the same time, the survey showed that only a small percentage of researchers tried to reproduce findings of others and that it is complicated to publish these reanalyzes [START_REF] Baker | 1,500 scientists lift the lid on reproducibility[END_REF].

List of Tables

List of Figures

FIGURES PRESENTED IN INTRODUCTION

According to the researchers surveyed, selective reporting, pressure to publish novel, significant findings, and low statistical power are the main reasons for the found patterns.

Thus, the question is, is there a real reproducibility crisis? And if so, to which extent?

a) Types of reproducibility

Talking about the reproducibility crisis without terminology is problematic. Every researcher is exposed to these terms; however, the concepts have been used differently due to their philosophical structure [START_REF] Plesser | Reproducibility vs. Replicability: A Brief History of a Confused Terminology[END_REF].

One of the first times the term reproducibility was mentioned was in 1992 by the Stanford geophysicist Claerbout who described the phenomena when "an author attaches to every figure caption a pushbutton or a name tag usable to recalculate the figure from all its data, parameters, and programs " [START_REF] Claerbout | Electronic documents give reproducible research a new meaning[END_REF].

The definition and its use were then redeveloped by many others and amplified. Whereas Claerbout did not describe a situation where other researchers come to the same conclusion using newly collected data, Peng and colleagues defined this as replicability. They state: "Scientific evidence is strengthened when important findings are replicated by multiple independent investigators using independent data, analytical methods, laboratories, and instruments" (4).

Despite early efforts of researchers like Claerbout and Peng, other researchers approached the opposite way of describing these terms. Such, the Association for Computing Machinery (AMC) had the opposite definitions to Claerbout and Peng in terms of reproducibility and replicability but described repeatability as "the measurement that can be obtained with stated precision by the same team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same location on multiple trials" [START_REF]Artifact Review and Badging -Current[END_REF].

In biomedical research, probably the most accepted use of terms are the ones from Goodman and colleagues, which are in line with the definitions of Claerbout and Peng, adding a definition of inferential reproducibility [START_REF] Goodman | What does research reproducibility mean?[END_REF].

The authors distinguish between three different types:

• Methods reproducibility which means to repeat the same analyses as the initial study with sufficient detail about the data and procedures.

For the authors, this kind of reproducibility is not adding additional evidence as no new data is gathered but is only a quality check.

• Results reproducibility which refers to obtaining the same results from the conduct of an independent study whose procedures are as closely matched. In other definitions, this is used as the term replicability.

• Inferential reproducibility which refers to the ability to obtain qualitative similar conclusions from an independent replication of a study or a reanalysis of the original study. For the authors, the most important kind of reproducibility.

Goodman

Claerbout / Peng AMC

Repeatability

Methods reproducibility Reproducibility Replicability

Results reproducibility Replicability Reproducibility

Inferential reproducibility

The conflict of terminology swept over to funding agencies enforcing different research cultures in reproducibility. Whereas in biomedical research, medicine, and epidemiology the terms of Goodman & Claerbout were used, in microbiology, immunology, and computer science, it was the opposite. According to findings from Barba from 2018, the fields of economics and political science don't distinguish between replication and reproducibility [START_REF] Barba | Terminologies for Reproducible Research[END_REF] .

However, after discussion with the National Information Standards Organization, the ACM swapped its definitions in 2020, and now terms are comparable to the ones of Claerbout [START_REF]New Changes to Badging Terminology[END_REF].

After large discussions on this topic, one can only hope that research culture will find consensus with these definitions as they created significant trouble in the past.

b) History of Science in crisis narrative

Years before the above-mentioned survey published in Nature, there were warnings that the current research system is likely to be flawed.

In 2005 this narrative gained strong attention when John Ioannidis published his article "Why most published research findings are false" [START_REF] Ioannidis | Why Most Published Research Findings Are False[END_REF].

Here he explains how, in his perception, research findings are false because of major flaws in methodology, the false and unjustified importance of statistically significant results, and biases such as manipulation in the analysis or selective reporting of findings.

He claims that "a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical models; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance."

Furthermore, as a corollary, he describes the Proteus phenomena, named after a Greek god1 .

This phenomenon is observed when the first published study in a certain field, mostly biased, is pointing to an extreme result. Follow-up studies might be biased against the result of the first one creating debates that receive a lot of attention.

In 2016 & 2017, he enlarged his criticism to specific fields with papers "Why most clinical research is useless" and "The Power of Bias in Economics Research".

In the former, one of his main arguments is that clinicians only read some specific medical journals such as the New England Journal of Medicine or the British Medical Journal. The problem with this is that these reviews only contain a few clinical trials out of all that have been conducted. Although these journals contain, on average, more trials, small trials in these journals tend to be more over-exaggerated than elsewhere. Furthermore, also data sharing is still uncommon in major medical journals. With his point, he argues that problems come from the physicians who are relying too much on certain medical reviews and the journals that are not showing enough transparency [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF].

But Ioannidis went beyond medical research and found that other areas were affected by this crisis as well. He showed that empirical economic research lacked methodology. In his survey of 159 meta-analyses, he found that "nearly half of the areas surveyed have 90% or more of their reported results stemming from underpowered studies" [START_REF] Ioannidis | The Power of Bias in Economics Research[END_REF].

Although his papers drew a lot of attention and created huge awareness on this topic, Ioannidis was by far not the first researcher describing problems in scientific findings.

In his book "Little science big science" from 1963, 42 years before the article of Ioannidis, De Solla predicted that science would reach saturation under its own weight and that science is growing so quickly that it will reach senility [START_REF] Price | Little Science[END_REF].

Eight years later, in 1971, Ravetz published his book on "Scientific Knowledge and its Social Problems." [START_REF] Ravetz | How should we treat science's growing pains? The Guardian[END_REF]. He explains that science is lacking control. According to him, this happened because there was a transformation from little to big science, and science became a marketing product.

In his opinion, this happened for two reasons. Firstly, researchers went from a Gemeinschaft to a Gesellschaft. Two very sociological, qualitative terms deriving from the German language, that are basically describing that a Gesellschaft is exposed to social norms, whereas a

Gemeinschaft is more in the sense of interaction between researchers. A Gesellschaft has become an environment where corruption and falsifications are possible. As a second reason, he sees the industrialization of research itself, whereas the individual researcher is now working in a big team. However, since there is no product to present to customers, quality control is lacking or nonexistent in some areas. Finally, he indicates that science is coming to stagnation and says that under hard job conditions, researchers are under a lot of pressure to publish and gain prestige. Manipulating data to get data published in high-impact journals is the logical consequence.

Another big insight into this issue came when Brian Nosek started the reproducibility study and changed the direction of the discussion by putting a focus on irreproducible results.

In 2015 Brian Nosek and his team of the Open Science Collaboration tried to replicate 100 studies in the field of Psychology, published in the top three journals [START_REF]Estimating the reproducibility of psychological science[END_REF]. In terms of p-values, only 36 studies had the same and were significant as to 97 in the original publication. Even though this study was conducted in the field of psychology, these findings shook the research community and brought up new questions. Now another problem was that reproducibility is not guaranteed even when data is shared.

In an alternative study, Hardwicke and colleagues tried to reach methods reproducibility for 35 papers from the journal Cognition. This endeavor proved to be difficult without assistance from the study authors and was even impossible for over one-third of the articles despite author assistance [START_REF] Hardwicke | Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the[END_REF] . See Figure 1.

Figure 1 Reproducibility rates in articles from the Journal Cognition [START_REF] Hardwicke | Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the[END_REF] 31.5% 37%

31.5%

Reproducible

Reproducible with author assistance

Not fully reproducible despite author assistance

This mediocre reproducibility also has consequences for the researcher themselves. Several surveys were carried out among scientists in biomedicine to see how they feel about research culture in their field [START_REF] Siebert | Overflow in science and its implications for trust[END_REF][START_REF] Tijdink | How do scientists perceive the current publication culture? A qualitative focus group interview study among Dutch biomedical researchers[END_REF]. In conclusion, the general feeling was that they are concerned and have critical views on the quality and validity of scientific research.

c) Skepticism to the crisis narrative

As an integral part of science, there are other opinions, and there is an ongoing discussion about the severity of the crisis. Several researchers are opposed to the idea of a reproducibility crisis.

Penders and Janssens argue that "sloppy" science can be reproduced but is still lacking quality.

At the same time, irreproducible science doesn't mean that it is of bad quality. According to them, there are other causes to the crisis instead of bad quality research. They base their arguments on philosophical and sociological theories [START_REF] Penders | Finding Wealth in Waste: Irreplicability Re-Examined[END_REF].

Already Karl Popper argued that theories are not able to be verified but that one can only disprove a theory. According to his philosophy, irreproducibility is a chance for improving science.

Kuhn has a different approach to the crisis and to science in general. He has a circular evolution in mind (Figure 2). He claims that there is something like normal science that entails prescience. Pre-science being the phase where there is not enough knowledge to fully seize the problems of a certain field and no ability to solve the issues. Normal science is built on the paradigms2 set in the pre-science phase, and the phase where problems are ignored. However, when problems appear there is a drift away from the model, which then, if not responded to, unrepairable or unsolved, will go into crisis. Kuhn says that once the normal science paradigm has entered crisis mode, it needs to run through the whole cycle, resulting in a revolution and a paradigm change [START_REF] Penders | Finding Wealth in Waste: Irreplicability Re-Examined[END_REF][START_REF] Khettab | The Ship Will Never Reach a Safe and an Ultimate Land[END_REF][START_REF] Thwink | Normal Science[END_REF]. The irreproducibility crisis would thus not be a problem but the result of natural development.

Another skeptic of the reproducibility crisis is Daniele Fanelli, who says that reproducibility is not a binary outcome but rather a system-specific observation.

In his piece from 2018, he explains why in his opinion, the "science in crisis" narrative is unjustified and even counterproductive as it might make early career researchers afraid of science and lead to anti scientific movements.

For him, the crisis is not growing as opposed to what some researchers are suggesting [START_REF] Fanelli | Opinion: Is science really facing a reproducibility crisis, and do we need it to[END_REF][START_REF] Fanelli | Why growing retractions are (mostly) a good sign[END_REF] . He goes by saying that small study bias is being made one of the reasons for this supposed crisis. However, this approach might be justified in some cases, such as if there has been careful planning. According to Fanelli, this bias is made up by meta-analysts who do not understand the context of the study. Furthermore, the non-publication of results might be justified as well since there are high costs of publishing included [START_REF] Fanelli | Conceptual challenges concerning re-analysis and replication practices in reproducible research[END_REF][START_REF] Fanelli | A theory and methodology to quantify knowledge[END_REF].

In his belief, this momentum should be described as a new opportunity or a revolution instead of a crisis.

Interestingly in this context is that other papers show higher estimations of reproducibility than the approach of Brian Nosek and his team. In some cases, reproducibility is proven to be as high as 60% across fields [START_REF] Klein | Investigating variation in replicability: A "many labs" replication project[END_REF][START_REF] Camerer | Evaluating replicability of laboratory experiments in economics[END_REF][START_REF] Camerer | Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015[END_REF][START_REF] Friedl | Rethinking research into metastasis[END_REF].

Another viewpoint on this topic is coming from Jamieson, who is claiming that there are three ways of scientific news reporting: 1) good scientists producing breaking inventions, 2) immoral scandals of scientists, and 3) the science is broken narrative. The latter for her is a hoax as not science itself can be broken and is often wrongly negatively taken up by the media, which leads to downsizing science in general [START_REF] Jamieson | Crisis or self-correction: Rethinking media narratives about the well-being of science[END_REF].

Fiske et al. have the same opinion as Fanelli in the sense that there is no crisis, and if at all, that this is an opportunity for science. In 2016 they wrote the following: "First came a few tragic and well-publicized frauds; fortunately, they are rare-though never absent from science conducted by humans-and they were caught. Now the main concern is some well-publicized failures to replicate [...]. All this is normal science, not a crisis. A replication failure is not a scientific problem; it is an opportunity to find limiting conditions and contextual effects. Of course, studies don't always replicate" [START_REF] Fiske | Introduction[END_REF] In a 2019 report titled Reproducibility and Replicability in Science, the National Academy of Sciences study committee also seemed to restrain the discussion of a replication crisis: "The advent of new scientific knowledge that displaces or reframes previous knowledge should not be interpreted as weakness in science" [START_REF]Reproducibility and Replicability in Science[END_REF].

Although these last arguments behold some truth, Sirmine Vazire argues that it is impossible to ask the public for a blank check and to build trust in science when there is no reason to do so. Furthermore, there needs to be a threshold until which errors can be accepted. If mistrust and fraud are normal, it is becoming a serious issue and can be behold against the whole community (32).

As discussed, reproducibility and science-in-crisis narrative remains a highly debated topic that is not only limited to psychology or biomedical sciences anymore but has touched several other research domains.

d) Reasons for crisis

When talking about reproducibility, the article of Munafò et al. Paper from 2017 needs to be brought into the discussion. If the crisis hadn't been debated enough at that point in time, said paper pushed the boundaries in the different research domains and gave a broader overview of the problem.

A team of researchers from different fields of science, including psychology, epidemiology as well as pre-clinical and clinical research, published the "Manifesto for reproducible science" [START_REF] Munafò | A manifesto for reproducible science[END_REF] , where they present several challenges of reproducible science.

In an ideal world, research follows the hypothetico-deductive model: a researcher generates a hypothesis based on existing literature, designs its research, collects data, and analyzes it, interprets the results, decides if his findings are meaningful, and finally publishes it [START_REF] Nola | The hypothetico-deductive method[END_REF].

Unfortunately, things aren't that easy in a real-life setting.

Failure to control the threats in this model is a main part of the reproducibility crisis.

Failure to control for bias when specifying hypotheses, low statistical power in study design, poor quality control, and p-hacking3 in data collection and analyzing and publication bias4 in disseminating the results are major issues (see Figure 3). These issues are underlined by the fact that positive results in published literature have increased over 22%from 1990 to 2007 [START_REF] Fanelli | Negative results are disappearing from most disciplines and countries[END_REF]. Other reasons for the misconduct and biases seem to be systematic. Some say that those in power are not interested in a change. To have a good position at a high-ranked university, publications in high-impact journals are important. However, usually, novel findings are necessary for this, including proven significance. These problems are not new and well known in biomedical research. Altman and Simera are showing that published literature from the early 20th century mentions poor research methodology in medical research [START_REF] Altman | A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR Network[END_REF] . Also, low statistical power in biomedical research has been described on several occasions.

Whereas the accepted threshold for minimum power lies around 80%, Button showed recently that studies in the neuroscience literature have a median statistical power between 8 and 31% [START_REF] Button | Power failure: why small sample size undermines the reliability of neuroscience[END_REF]. A study by Dumas surveyed the statistical power in neurological, psychiatric, and somatic disease and found similar results. 50% of studies had statistical power in the 0 -10% or 11 -20% range [START_REF] Dumas-Mallet | Low statistical power in biomedical science: a review of three human research domains[END_REF].

Publication bias was described early in clinical research by Robert John Simes who found discrepancies in data reported to a cancer trial registry. He compared the data to the published articles on the survival impact of two cancer therapies and found that results for the impact of the therapies either disappeared or were substantially less in the literature [START_REF] Simes | Publication bias: the case for an international registry of clinical trials[END_REF][START_REF] Devito | Publication Bias[END_REF].

A more recent example is provided by Turner and colleagues on depression treatment. They showed that 31% of a cohort of studies for antidepressant drugs registered and reported to the U.S. Food & Drug Administration (FDA) were never published. The literature included 94% positive articles, whereas the analysis of FDA documents only contained 51% positive studies [START_REF] Turner | Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy[END_REF].

As stated above, besides problems Munafò and colleagues also indicate solutions to the problems in actual science [START_REF] Munafò | A manifesto for reproducible science[END_REF].

In their opinion, there need to be several improvements, like pre-registration or improving the quality of reporting to make science reproducible. However, to promote transparency and Open Science, open methods and open data must be given and the new normal.

Small study bias, underpowered sample sizes, statistical misinterpretation, p-hacking, or the publish-or-perish dogma might contribute to this problem. Nevertheless, this thesis will put a focus on the role of data sharing in the light of the reproducibility crisis.

First, because addressing all points would break the scope of this thesis, and second because by sharing data, risks mentioned in the hypothetico-deductive model can be controlled or limited.

Poor quality control and publication bias can be detected when data is openly available and phacking, and thus indirectly HARKing, can be reduced when independent reanalyzes are possible with openly shared data.

Data sharing in biomedical research a) Advantages of Data Sharing

Data Sharing is crucial for reproducibility in research. It is the root of hypothesis planning and the base of research observations. Despite this fact, some researchers are still careful with data sharing. There is massive literature about both sides of the view. Below will follow positive opinions and, in a second step, concerns about data sharing.

Among several reasons, Ross and Krumholz claim that sharing data "maximizes the value of the collected data." Thus, researchers can use the shared data for sequel analyses and provide new insights. Other researchers agree and point out that there is a possibility of doing more digital comparative effectiveness research as clinical trial data becomes more open [START_REF] Ross | Ushering in a New Era of Open Science Through Data Sharing: The Wall Must Come Down[END_REF][START_REF] Butte | Trials and Tribulations-11 Reasons Why We Need to Promote Clinical Trials Data Sharing[END_REF].

A perfect example of this was the SPRINT data reanalysis, an initiative launched by The New England Journal of Medicine trying to value data sharing in clinical trials. Therefore, the SPRINT trial data set, which compared intensive management of systolic blood pressure with standard management and was stopped early, was opened to investigators. The aim was to see if data sharing of a clinical trial data set could yield new findings. And it did [START_REF]About the SPRINT Data Analysis Challenge[END_REF]. Over 100 research teams from academia, industry, and agencies asked for the data and contributed to this project by submitting a new analysis. Three final winners were chosen. Several new clinical conclusions regarding benefits and risks for the different interventions could be drawn from already collected clinical data. This example demonstrates that research collectives can be mobilized to come up with novel findings from existing data sets [START_REF] Burns | Learning What We Didn't Know -The SPRINT Data Analysis Challenge[END_REF].

Another noted argument is that data sharing is helping to save money in the long run as no new initiatives must be formed if the data is available. The same goes for patients. There is no need to recruit more patients to check if the same hypothesis is confirmed. At this point, the Open Science initiatives are a sign of respect for the patients who consented to participate for them.

A further opinion on data sharing is the point of self-correction. Data sharing allows researchers, patients, and the general audience to reproduce the results and thus have more confidence in the results, which is the primary goal of this thesis. This can be extrapolated to research in general, not only the biomedical field.

Examinations are essential and trust enhancing in cases where fraud is supposed. This goes for the critical observation of drugs or other therapies that impact the lives of people. Moreover, this point would help get the reproducibility crisis back in control.

In a recent survey , trialists state that they want to promote the culture of Open Science with data sharing [START_REF] Rathi | Sharing of clinical trial data among trialists: a cross sectional survey[END_REF].

Data sharing is also named for some because of academic benefit and recognition, and indeed a study from 2020 has shown that data sharing goes with higher citation rates [START_REF] Colavizza | The citation advantage of linking publications to research data[END_REF].

Furthermore, patients feel reassured when they know that their participation in a trial has helped to advance scientific discoveries in their field. Only a minor part is unsure or opposed to data sharing. Furthermore, patients affected by rare diseases are more willing to share their data with non-profit organizations [START_REF] Burns | Learning What We Didn't Know -The SPRINT Data Analysis Challenge[END_REF] . Data sharing will never be perfect, and de-identification might happen. However, these risks are lower than imagined, stresses Barth-Jones as he says that identification issues are marginally small compared to the whole data sets out there [START_REF]New Burdens on Medical Research and De-Identified Data from CCPA and Other Looming Legislation -Any Fixes in Sight? Privacy+Security Forum[END_REF]. Also, data sharing has come a long way with guidelines like the HIPAA Safe-Harbor guidelines that make de-identification in 18 points mandatory (52).

As to everything, there is a trade-off in data sharing: the more information shared, the better, but this comes with disclosure problems. However, if too little data is shared, bad decisions or lousy science results.

Another argument for data sharing comes from Ioannidis, who says that errors in datasets are ubiquitous, and we must accept this fact and create a culture of awareness and not of attacks.

Only then a fruitful data sharing culture can grow [START_REF] Ioannidis | Anticipating consequences of sharing raw data and code and of awarding badges for sharing[END_REF].

b) Risks of Data Sharing

Often investigators hesitate to share data since they are afraid that their data is misused. Many fear that secondary analyses might be misused, and external validity might not be given as data is too hard to interpret for outsiders. Concerns for data sharing from investigators' sites are related to appropriate data use, such as misleading secondary analyses. Also, concerns related to publishing their results are typical. Investigators are afraid someone else might take the data and not get credit for it.

Another point is that data sharing is a cost-intensive process. In order to be shared, data must be formatted, checked, re-written. Resources, personal and costs, need to be gathered.

If a researcher wants to share biomedical data with his peers, several other obstacles could be hindering the process. Informed consent forms need to be signed to assure that the patients agree with sharing their personal information. Furthermore, privacy and confidentiality issues need to be organized. This goes especially for rare diseases where patients are eager to share data to advance the healing of their disease [START_REF] Courbier | Share and protect our health data: an evidence based approach to rare disease patients' perspectives on data sharing and data protection -quantitative survey and recommendations[END_REF].

At the same time, there are risks such as identification of patients, which is particularly critical in these groups of patients.

In a survey, half of the patients also said that they are afraid to share their data because it might be shared with third parties or used in a different context that they do not want it to be used [START_REF] Courbier | Share and protect our health data: an evidence based approach to rare disease patients' perspectives on data sharing and data protection -quantitative survey and recommendations[END_REF]. This is presumably one of the reasons why patients attended by rare diseases are more willing to share their data with non-profit organizations. Some argue that if there is an obligation to share data, research teams from low-and middleincome countries that are already disadvantaged will be more penalized by these measurements.

The publishing costs would rise, and the file-drawer problem could become even worse. The latter was the crucial point the 2017 ICMJE data sharing policy was finally not mandatory but instead became a declaration of whether or not data is shared [START_REF] Wager | Sharing data is a good thing. But we need to consider the costs[END_REF].

A team from Clinical Study Data Request, a data sharing platform for clinical trials, wrote that it is an "expensive and resource-intensive task for trial sponsors to provide access to data through this managed access model when it involves secure analysis environments with the licensed software." Furthermore, they argue that over half of the studies were never being requested and that more initiatives needed to happen to foster data reuse [START_REF] Kochhar | Clinical trial data sharing: here's the challenge[END_REF]. This analysis imposes the further question of whether the difficult efforts made for data sharing were unnecessary and are not appreciated by the research community.

c) Importance of Data Sharing during a Global Pandemic/ COVID 19

If the advantages of data sharing have not become clear enough, the pandemic has brought more dynamic into the discussion. When discussing Open Science, the current pandemic is an example of the impact and importance of data sharing.

On Jan 30th 2020, the WHO announced the COVID-19 infection as an outbreak of Public Health Emergency of International Concern [START_REF]Statement on the second meeting of the International Health Regulations (2005) Emergency Committee regarding the outbreak of novel coronavirus[END_REF].

The genome sequence data of the virus was shared quickly through a public access platform.

Thanks to this approach, the international research community could present a vaccine in less than a year (58).

Many are now using the argument that with the COVID-19 Pandemic, data sharing has entered a new phase [START_REF] Moorthy | Data sharing for novel coronavirus (COVID-19)[END_REF].

What followed was a wave of research in the form of clinical trials and other research on the new virus. A recent analysis from August 2021 investigated that there were 210 183 COVID-19-related publications from 720 801 unique authors [START_REF] Ioannidis | The rapid, massive growth of COVID-19 authors in the scientific literature[END_REF] .

A team of researchers found that out of 535 COVID pre-prints, only 21% of authors included availability statements, and even less, 11%, made the data available in external repositories [START_REF] Sumner | Reproducibility and reporting practices in COVID-19 preprint manuscripts[END_REF].

This research was conducted in March 2020 and data mainly came from China.

For an update, Weissgeber and colleagues screened 6,570 COVID-19 pre-prints from medRxiv and bioRxiv posted before July 2020. 13.6% of pre-prints shared data openly, and 14.3% of authors shared their code, showing similarly low rates in data sharing as the first study [START_REF] Weissgerber | Automated screening of COVID-19 preprints: can we help authors to improve transparency and reproducibility?[END_REF].

In a recently published article in BMC Clinical Trials, Li and colleagues surveyed COVID-19 trial registrations and articles for data sharing statements. They screened 924 registrations on ClinicalTrials.gov registered before June 30th, 2020, and COVID-19 publications were searched in May 2020. From the screening they found that 47.6% of trialists declared they were not willing to share. In twenty-six publications, 80.8% of authors did not include a data sharing statement [START_REF] Li | COVID-19 trials: declarations of data sharing intentions at trial registration and at publication[END_REF].

Research activity in biomedical research has risen during the pandemic, but that did not positively correlate with good research quality and data sharing. Such, the pandemic has shed light on the dishonest practices of some researchers.

A prominent example of lack of data sharing when it came to vaccines, was the Sputnik V vaccine.

Several researchers found uncertainties in the data coming from the Phase I / II trial [START_REF] Bucci | Safety and efficacy of the Russian COVID-19 vaccine: more information needed[END_REF].

When the authors published interim results of the Phase III trial, inconsistencies were again found [START_REF] Bucci | Data discrepancies and substandard reporting of interim data of Sputnik V phase 3 trial[END_REF][START_REF] Baraniuk | Covid-19: What do we know about Sputnik V and other Russian vaccines?[END_REF]. Nonetheless, data for this trial will only be available at the end of the study and only after demanding it from the security department of the Russian government. These barriers are diminishing chances to obtain the data as an independent researcher and stand in contrast to transparency [START_REF] Logunov | Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia[END_REF].

However, not only the Sputnik vaccine is lacking transparency.

After new revelations that there were problems during the Phase III trial from Pfizer, data for the respective vaccine will not be accessible until 2025 [START_REF] Thacker | Covid-19: Researcher blows the whistle on data integrity issues in Pfizer's vaccine trial[END_REF][START_REF] Demasi | Investigative journalism[END_REF][START_REF] Tanveer | Transparency of COVID-19 vaccine trials: decisions without data[END_REF] .

In their piece in BMJ Evidence-Based Medicine, Doshi and colleagues show that limited transparency comes with all COVID-19 vaccines. Furthermore, the most used vaccines, Sinopharm and Sinovac, lack openness, and experts are unsure if this will change since the FDA, or the European Medicines Agency (EMA) has not given authorization for these two agents.

In his piece, "COVID-19 and the research scholarship ecosystem: help!", David Moher sheds a negative light on the way data sharing has been handled in the first year of the pandemic [START_REF] Moher | COVID-19 and the research scholarship ecosystem: help![END_REF].

Furthermore, he suggests that publishers and funders should incentivize more open research practices.

Another event that shook the biomedical world during the pandemic was the Surgisphere scandal.

In 2020, Surgisphere claimed to have patient data from the hospitals, from which two articles were produced. In May 2021, an article published in The Lancet demonstrated higher mortality in hospitalized patients infected with COVID-19 and treated with Hydroxychloroquine. Two other articles followed, one pre-print and one article published in The New England Journal of Medicine. The issue with these articles was that the data probably never existed. Researchers expressed their concerns by pointing at high effect sizes in the articles, number of deaths exceeding national registries, and peer-reviewers never seeing raw data [START_REF] Offord | The Surgisphere Scandal: What Went Wrong? : The Scientist[END_REF].

However, not all outcome of COVID-19 was terrible, and more awareness of the importance of data sharing was created. Early in the pandemic, in January 2020, one of the biggest funders, Wellcome trust, enforced the need for data sharing [START_REF](COVID-19) outbreak 2020[END_REF]. Moreover, several initiatives for data sharing came up during the crisis. One was the Data Sharing Working Group that "was convened to facilitate and promote effective, ethical, and equitable data sharing across geographies and disciplines." [START_REF] Fegan | Electronic address Wcco. Solutions to COVID-19 data sharing[END_REF] The actual aftermath of this pandemic regarding data sharing will probably influence research still many years later and will need careful analysis of what achievements were made or ultimately missed.

d) History of Data Sharing in Biomedical Journals

If transparency and reproducibility issues in research found their way early in domains like psychology or physics, it took a longer run in biomedical. Some experts also see the actual crisis in part by the journals that did not act strict enough on these issues for a long time.

Already in 2009, Chalmers and Glasziou estimated that around 85% of research funding in biomedicine was wasted, which could have been avoided. According to them, this is due to irrelevant questions in research, unappropriated designs and methods, unpublished research, and biased or unusable study reports [START_REF] Chalmers | Avoidable waste in the production and reporting of research evidence[END_REF].

Nonetheless, retractions in this field are rare. However, there is a strong positive correlation between the impact factor and retractions [START_REF] Fang | Retracted Science and the Retraction Index[END_REF] .

Reasons for this correlation can only be assumed. One could be that authors might take more risks to falsify or manipulate data to get their results published in a high-impact journal resulting in higher chances of promotion and grant funding. However, another explanation could be that articles in those journals have more visibility, as Ioannidis said [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF] and thus mistakes or irregularities might be detected faster.

Data sharing policies and top journals in biomedicine remain a complex story.

In 2007, Annals of Internal Medicine was one of the first major medical reviews to encourage data sharing but not require it. Since this date, the editors require a reproducible research statement to be included for clinical trials. This reproducible research statement indicates "whether the study protocol, data, or statistical code is available to readers, and under what terms authors will share this information." [START_REF] Laine | Reproducible Research: Moving toward Research the Public Can Really Trust[END_REF].

The BMJ introduced a similar policy in 2009 [START_REF] Groves | Managing UK research data for future use[END_REF], which then became mandatory in 2013 for clinical trials from drugs and medical devices [START_REF] Godlee | The new BMJ policy on sharing data from drug and device trials[END_REF] and finally mandatory for all research in 2015 [START_REF] Loder | The BMJ requires data sharing on request for all trials[END_REF].

In March 2014, data sharing became mandatory for all research articles in the PLOS journals [START_REF]Data Availability[END_REF].

The International Committee of Medical Journal Editors (ICMJE), formerly known as the Vancouver Group, is one of the prominent leaders and predominant policymakers in biomedical research.

In 2016 they discussed the possibility of introducing a data sharing statement in journals following their guidelines, aware of the issues found in biomedical literature. As stated above, this approach was intended to mean mandatory data sharing first but then became a voluntary option in the final statement 2018. Authors should include a data sharing statement in their article and state its availability without making it mandatory [START_REF] Taichman | Sharing Clinical Trial Data -A Proposal from the International Committee of Medical Journal Editors[END_REF][START_REF] Taichman | Data Sharing Statements for Clinical Trials -A Requirement of the International Committee of Medical Journal Editors[END_REF].

In the last decade, much research has been surveying data sharing practices and their status quo in biomedical research.

One of the first extensive studies surveyed 441 journal articles in the biomedical area, published between 2000 -2014. The multicenter research team found that most studies did not share protocols, raw data or disclose funding or potential conflicts of interest [START_REF] Iqbal | Reproducible Research Practices and Transparency across the Biomedical Literature[END_REF].

Rerunning the screening from 2015 -2018 revealed that although there have been improvements, especially in data sharing, more progress is needed [START_REF] Wallach | Reproducible research practices, transparency, and open access data in the biomedical literature, 2015-2017[END_REF].

Even in the BMJ, one of the top medical journals with strong transparency, data sharing was low in articles published from 2009 to 2015. Only 7/157 data sets were accessible when data was asked [START_REF] Rowhani-Farid | Has open data arrived at the British Medical Journal (BMJ)? An observational study[END_REF].

In a study from 2018, Naudet and colleagues tried to see if they could reanalyze primary outcomes from 37 clinical trials published in PLOS Medicine and the BMJ. They received data from 17 (46%) trials and managed to reanalyze 14 (82%) of them successfully. Two contained errors, but similar conclusions were obtained, and for the last one, there was not enough information given to reproduce the analysis [START_REF] Naudet | Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine[END_REF].

In a recent study from Serghiou et al., they screened 2.75 million articles from the biomedical literature from 1959 -2020 from 10 570 different journals to find out how transparency indicators progressed over the years. Data sharing statements were found in only 8.9% of articles and the rates did not increase as much as they did for the conflicts of interest's declaration and funding disclosures [START_REF] Serghiou | Assessment of transparency indicators across the biomedical literature: How open is open?[END_REF].

Malički could prove with his recently published paper that Open Science instructions in journals for authors increased. In the study, he and his team studied instructions for authors from 1987 -2017 and checked how often data sharing was addressed. He showed that from 1991 to 2010, there was an increase in addressing data sharing from 15% to 88% in the top journals [START_REF] Malički | Systematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017[END_REF].

In a survey from 2020, Hamilton and colleagues found that only 15% of medical journal editors stated that their journals had declared mandatory data sharing policies [START_REF] Hamilton | Journal policies and editors' opinions on peer review[END_REF].

Despite several policies, data sharing in journals is still an issue that needs to be addressed and further advanced.

e) Initiatives for Data Sharing and for transparency in biomedical research

Despite problems and issues with data sharing in biomedical research and journals, many initiatives have been started to improve the situation.

Jennifer Miller has created Good Pharma Scorecard, an initiative to survey ethics performance, including clinical trial transparency and data sharing criteria within pharmaceutical companies.

Research shows that around a quarter of the biggest pharmaceutical companies met the data sharing item of the Scorecard. The median company data sharing score was 63%. After giving the companies feedback and the chance to improve their policies to meet this measure, three companies made amendments, raising the percentage of companies in full compliance to 33% and the median company data sharing score to 80%. This also shows that public pressure can help advance data sharing from clinical trials [START_REF] Miller | Sharing of clinical trial data and results reporting practices among large pharmaceutical companies: cross sectional descriptive study and pilot of a tool to improve company practices[END_REF]94).

In addition, there has been a significant shift in data sharing with several platforms coming up. These platforms were launched to act in the calls for improved transparency in clinical trial data sharing and making data available to researchers. Some of the websites provide access to study data without restrictions. However, most of these platforms require the data requestors to submit a formal research proposal before receiving the de-identified IPD.

One of the first was Yale Open Data Access Project (YODA), run by Yale University since 2011. Other platforms emerged quickly, such as Vivli and Clinical Study Data Request (CSDR).

Nowadays, YODA, CSDR, and Vivli are the biggest platforms in terms of clinical trial data sharing; the three together contained around 8000 RCTs in 2020 [START_REF] Vazquez | Characteristics of available studies and dissemination of research using major clinical data sharing platforms[END_REF].

Initiatives to make a sustainable future for IPD data sharing were in need. A stakeholder meeting of more than 40 leaders in institutions that deal with clinical trial data management proposed ten principles and 50 recommendations on how to share data from Clinical Trials properly.

Amongst other things, they emphasize the importance of making data sharing a reality (e.g., through cultural change or academic incentives). Other areas are the consent for data sharing, protection of trial participants, data standards, rights, types and management of access, data management and repositories, discoverability, and metadata. These rules should help to promote data sharing and reuse among researchers (see Figure 4) [START_REF] Ohmann | Sharing and reuse of individual participant data from clinical trials: principles and recommendations[END_REF]. Another solution that was established were Open Science badges. The main goal is to demonstrate that a scientific article applies Open Science practices. If a researcher is sharing his data, a badge will appear on the article, indicating that data is available. Some studies have been conducted to find out the impact of this intervention. There are contrary findings regarding this measure; while an intervention in the journal Psychological Science had a positive effect and data sharing increased by more than tenfold, a study in BMJ Open showed no effect [START_REF] Kidwell | Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency[END_REF][START_REF] Rowhani-Farid | Did awarding badges increase data sharing in BMJ Open? A randomized controlled trial[END_REF]. This contrast might be due to the different research domains, whereas Psychology is more advanced and exposed to Open Science and transparency. Furthermore, patient confidentiality might be more critical in the biomedical area than in psychology.

A similar proposal came from Bielekova and Brownlee, discussing a new set of scores to improve biomedical research. Besides a methods score and a societal impact score, they suggest giving out a reproducibility score to scientific papers (100).

Since research institutions are a part of this crisis, initiatives should not exclude them, and universities should rethink how to act appropriately and hire researchers substantially.

Moher and colleagues have addressed these issues by including other aspects when hiring scientists for promotion. They recommend assessing Open Science practices used by the researcher when it comes to tenure or promotion (101).

In the Hong Kong Principles, a petition for fostering research integrity, researchers are proposing five principles (responsible research practices; transparent reporting; Open Science /open research; valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity) to assess researchers for scientific jobs (102).

Noteworthy, Dan Quintana proposes that undergraduate students reanalyze studies instead of creating new hypotheses. He argues that most traditional undergrad research projects are never published due to low statistical power and unwell formulated research questions. A replication approach could solve these problems and educate students on Open Science practices (103).

Another central role play funding agencies. They have a meaningful impact on data sharing since they make the rules for the researchers, they are giving money. They can decide if the funded research needs to be published in open access and if the data must be shared with the public. One of our studies shows that there is room for improvement. 38% of non-commercial funding and 41% of commercial funders have a data sharing policy (104).

Although journals have already been addressed, new formats and new initiatives have come up.

The following examples focus less on data sharing and more transparency and Open Science in the publishing system.

As such, pre-prints, not peer-reviewed articles and uploaded prior to publication and to peerreview, have become very popular in research culture. This is advantageous for researchers since articles can be shared before official publication and beneficial for readers since pre-print platforms are free to access. On the other hand, there are risks with this format as it is not peerreviewed, meaning that unscientific and even unethical research can be shared with the public pretending to be inclusive of qualitative standards. Although not improving data sharing, this approach still fosters transparency in science.

Another initiative that appears in Chapter IV in this thesis is Registered Report, a new form of publishing made possible by Chris Chambers. Approaches differ in detail from journal to journal, but in general, in the first step, the authors are writing a protocol including introduction and background and presenting the methodological design of the study. This is peer-reviewed, like a regular submission. In a second step, data collection and discussion of results are added.

After this stage, there will be a second peer-review before final publication. If changes in methodological approach apply, rejection is possible.

A more descriptive scheme is pictured below (see Figure 5). Advantages of this new form are control of publication bias, starkened transparency, and a peerreviewed approach that will help to make data analysis easier.

On the other hand, this approach can be time-intense and might not suit every research project.

Furthermore, quality depends on the peer-review and the methodological approach since a project recorded as a registered report itself does not guarantee quality or rigor.

When discussing journals and publishers regarding reproducibility and transparency crisis, some researchers are even arguing for the dissolution of academic publishers.

They argue that large publishing houses possess articles, data, and code and that the power should be given back to the research institutions and to researchers interacting with each other (106).

It will be interesting to see if the initiatives named above will bring change to Open Science and, if so, to which extent and at what pace.

Clinical trial data sharing in Regulatory settings

Despite a clear shift in transparency in biomedical research, the question remains: Do the patterns of irreproducibility and lack of transparency show up in trials for drug approval as well?

One report that raised the alarm in terms of drug approval and transparency was the case of China's State Food and Drug Administration (CFDA). According to a source, 80% of clinical trials were substandard. The agency hired around 300 staff members to run a one-year investigation to ensure the standards are met and to verify dossiers of clinical trials sent to the agency for marketing authorization. After examining data for 1622 drugs, their final report contained that severe issues, including falsifying or fabricating data, were found in the marketing authorization documents (107).

If this report might have raised issues in the Eastern hemisphere, the Western world (especially the USA and Europe) had to fight with their own issues.

One of the first studies that concerned stakeholders was Begley's study describing that only six out of 53 so-called landmark pre-clinical studies were to be reproduced. Although the authors describe that this difference might come from altered techniques and distinctive machines used, researchers found proof in what was long supposed regarding pre-clinical drug research (108)

. Another example confirmed these findings. The results were contradictory to the journal articles in 43 out of 67 oncology and cardiovascular target discovery studies from Bayer (109) .

Aware of the problem, two organizations representing the biopharmaceutical companies, Pharmaceutical Research and Manufacturers of America (PhRMA) and the European Federation of Pharmaceutical Industries and Associations (EFPIA), published a joint statement that intends to foster data sharing. For them, responsible data sharing is a must but needs to be in line with the principles of safeguarding patients' privacy, respecting the integrity of national regulatory systems, and maintaining incentives for investment in biomedical research (110).

Despite this commitment from 2013, low rates for data sharing were found in an audit for clinical trials on medicines sponsored by the pharmaceutical industry published between 1 July 2015 and 31 December 2015 in the top 10 general and internal medicine journals. For only 9/61 (15%) trials, data were available (111).

Moreover, even if data is available, published reanalyzes are still sparse (112).

Nonetheless, some reanalyzes showed different results than the original article. One of the more famous examples is the "Restoring Study 329" by Le Noury et al., which contradicted the initial publication, a trial for depression that was already known to be misreported. The reanalysis found that the harms were much higher in the active drug than in the placebo group and that there was no clinical significance for superiority against placebo (113).

Essentially for the studies mentioned above, the pharmaceutical industry now requests services to replicate scientific studies. Mainly because money is an essential factor when designing a new drug. If neither robustness nor replicability for previous studies is not given, the drug agent in question is not a good investment for the company in the long run (114).

To advance this topic, the EMA released its policies 0043 and 0070. Policy 0043 active from 2010 should make all documents ever received from the agency available on request (115). Non-disclosure of data raises concerns about the standards of research integrity of the European authority. This has been observed for the vaccines developed during the COVID-19 pandemic [START_REF] Tanveer | Transparency of COVID-19 vaccine trials: decisions without data[END_REF]117).

Therefore, rests the question: Are trials that are part of marketing authorization applications reanalyzed by the agency?

An official email exchange with the staff of the agency confirmed that the EMA is not, unlike the FDA, trying to reproduce the results of the data and, if in doubt, is asking for reanalyzes from the sponsor itself (118).

Nonetheless, in a recent survey that compared the drug agencies of three regions, Health Canada, the FDA, and the EMA, the latter is more innovative and transparent than its American opponent, the FDA. Furthermore, the study found that the FDA lags behind the two other agencies in transparency for not publishing their clinical study report proactively (119).

Drug regulation has come a long way regarding transparency. Nevertheless, there is still a long way to achieve full transparency, if ever reached.

Reproducibility in Therapeutic Research Project and Thesis objectives a) Project ReiTheR

This thesis is part of a project funded by the French National Agency for Research (ANR).

The idea is to have two PhD students working on similar tasks that complement each other.

One PhD student is investigating the data sharing policies of funders of clinical trials and, in a second step, the inferential reproducibility of Phase III Clinical Trials that can be found on major data sharing platforms.

The second PhD student, the author of this thesis, focuses on how data sharing policies are embedded in biomedical journals that follow the ICMJE guidelines. In a further project, reanalyzes of clinical trials that led to a marketing authorization inside the European Union are conducted.

Involved in a multi-functional group, this project should give clearness and better understanding to which extent data sharing is advanced in therapeutic research.

b) Objectives of this thesis

In this thesis, four research questions are approached:

1.

Are Data Sharing agreements in journals that follow the ICMJE guidelines part of the author instructions and in the publication outcome? And if so, to which extent?

2.

What is the Status Quo of Data Sharing in biomedical literature?

3.

Can we get the data and reproduce the primary outcomes of 62 pivotal trials that were part of marketing authorizations assessed by the EMA?

4.

How to integrate Open Science into drug approval?

Main Outcome Measures

The primary outcome for the policies was the existence of a data-sharing policy (explicit datasharing policy, no data-sharing policy, policy merely referring to ICMJE recommendations) as reported on the journal website especially in the instructions for authors. For RCTs, our primary outcome was the intention to share individual participant data set out in the data-sharing statement.

Results

Eight (of 14; 57%) member journals had an explicit data-sharing policy on their website (3 were more stringent than the ICMJE requirements, 1 was less demanding and 4 were compliant), 5

additional journals (35%) stated that they followed ICMJE requirements and one (8%) had no policy online. In RCTs published in these journals, there were data-sharing statements in 98/100, with expressed intention to share individual patient data reaching 77/100 [77%; 95% confidence interval: 67% to 85%].

One hundred and forty-five (of 489) ICMJE-affiliated journals (30 % [26% to 34%]) had an explicit data-sharing policy on their website (11 were more stringent than the ICMJE requirements, 85 were less demanding and 49 were compliant) and 276 (56% [52% to 61%]) merely referred to ICMJE requirements. In RCTs published in affiliated journals with an explicit data-sharing policy, data-sharing statements were rare (25%) and expressed intentions to share data were found in 22% [15% to 32%].

Conclusion

The implementation of ICMJE data-sharing requirements in online journal policies was suboptimal for ICMJE-member journals and poor for ICMJE-affiliated journals. The implementation of the policy was good in member journals and of concern for affiliated journals. We suggest the conduct of continuous audits of medical journal data-sharing policies in the future.

Registration https://osf.io/n6whd/

INTRODUCTION

In June 2017, the International Committee of Medical Journal Editors (ICMJE) published a statement supporting data-sharing practices for randomized controlled trials (RCTs). For the ICMJE, "there is an ethical obligation to responsibly share data generated by interventional clinical trials because trial participants have put themselves at risk" with the aim to "maximize the knowledge gained" from these outstanding studies. The ICMJE policy requires a specific data-sharing statement to be included in each newly submitted paper (and pre-specified in study registration) containing clinical trial data, starting from 1 st of July 2018 (120) .

Examples of medical journals having a data-sharing policy before this requirement were few.

In 2007, the Annals of Internal Medicine was the first journal to adopt a policy encouraging data-sharing practices (121). The BMJ adopted a similar policy encouraging data-sharing in 2009 ( 80) and went further by making it mandatory in 2013 for drugs and devices ( 81) and for all RCTs in 2015 [START_REF] Loder | The BMJ requires data sharing on request for all trials[END_REF]. PLOS journals also adopted a strict policy enforcing RCT data-sharing in 2014 (122). No other leading general medical journal has had a specific policy for datasharing in RCTs.

The ICMJE policy could therefore have an impact on biomedical literature as a whole. At the time of the present research, the ICMJE included 2 organizations (the U.S. National Library of Medicine and the World Association of Medical Editors) and 14 journals, including leading medical journals such as The New England Journal of Medicine (NEJM) and The Lancet. In addition, about 5000 affiliated journals follow the ICMJE recommendations (123). As this policy is now in place, it is important to monitor its implementation both in the ICMJE-member journals and in the ICMJE-affiliated journals. It is also important to assess intentions to share data among RCTs published in the journals implementing a data-sharing policy.

METHODS

The protocol was registered before the start of the research on the Open Science Framework (OSF) (https://osf.io/n6whd/). This study was divided into two parts: a survey of journal datasharing policies and a survey of published RCTs.

Journal selection and data extraction

A data extraction sheet and a data extraction explanatory document were developed [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF]. Three

Outcomes describing journal data-sharing policies

Our primary outcome was the existence of a data-sharing policy (specific data-sharing policy, no data-sharing policy or a policy merely referring to ICMJE requirements) as reported on the journal website. This outcome had to be changed from our initial protocol due to non-response to our emails from the sample of ICMJE-affiliated journals, and because some email addresses could not be identified. The change took place before any analysis. For journals mentioning a specific data-sharing policy on their website, the explicit statement and various features of these policies were collected: the start date of the data-sharing policy, the type of policy: ICMJE compliant, more stringent than required by ICMJE or less demanding than required by ICMJE (for instance, less demanding could mean that there was no obligation for a data-sharing statement and more stringent could mean that data was to be shared with other researchers). We also noted whether the policy was limited to clinical trials. Furthermore, the indication of one or more preferred data-sharing platform (and if so, which ones), the existence of any sanctions in case of non-compliance with data-sharing (and if so, what they were). Any existing policy demanding trial registration was also extracted (and if there was one, we noted whether it mentioned prospective registration). The following features of the journals were also extracted: indexed on PubMed, ISSN (print ISSN), the number of issues per year, 2017 Journal Impact Factor (JIF), publisher or publishing group, gender of the editor in chief ("men", "women" and "both genders represented " if the co-editors in chief were men and women), the country of the journal head office, the wealth category of the country where the editorial office is located as defined by the World Bank (125) and the research domain covered by the journal.

Survey of RCTs published in journals with a data-sharing policy

RCT eligibility criteria

Eligible studies were RCTs published after 1 st of January 2019 in a journal with an explicit data-sharing policy reported on its website and submitted after 1 st of July 2018. Any RCTs, including cluster trials and crossover trials, non-inferiority designs and superiority designs, were included. No distinction was made in terms of patients, interventions, comparators or outcomes. We had originally planned to include only Phase-III studies but realized that this information was not always reported in the publications. Consequently, no distinction in terms of study phase was applied.

Search strategy for RCTs published in journals with a data-sharing policy

ICMJE-member journals were contacted to gather the list of RCTs they published after the 1 st of January 2019. This approach was not used for ICMJE-affiliated journals, due to non-response from most of the 14 member journals and because it was not possible to identify an email contact for all these journals. The following search strategy was applied to retrieve all RCTs.

For journals indexed on PubMed/Medline, a search algorithm to identify RCTs was developed with the help of a librarian from Rennes 1 University using the Cochrane sensitivity maximizing approach (126) and adding further key words. The exact filter can be found in the supplementary material [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF]. For journals not-indexed on PubMed, an investigator (MS) screened all articles published after 1 st of January 2019 to identify RCTs.

All identified RCTs published in ICMJE-member journals with a data-sharing policy were assessed in random order using R [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF]. The first 100 that met our selection criteria were included, enabling us to estimate a proportion of 50 % (the worst-case scenario for precision estimates) with a precision (boundaries of the 95 percent confidence interval) of ± 9.8 %. We followed the same approach to include a second sample of 100 RCTs published in ICMJEaffiliated journals.

RCT selection and data extraction

As for the journal selection procedure, a data extraction sheet and a data extraction explanatory document were developed [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF]. Three investigators (MS, LC, JG) had a one-hour explanation and were trained via a pilot data extraction performed on 10 RCTs. For each RCT, two of these investigators independently assessed eligibility (giving reasons in case of non-eligibility) and extracted the characteristics listed below from each included published article. Disagreements were resolved by consensus or in consultation with a third investigator (FN).

Outcomes describing data-sharing statements in published RCTs

For this part of our survey, the primary outcome was the intention to share individual patient data (IPD) expressed by the authors in the data-sharing statement (yes/no/unclear). The latter of the three response options, "unclear", was recorded if the statement was written in a general tone without specifically mentioning that IPD would be available. Secondary outcomes were trial registrations: the existence of trial registration, prospective trial registration and registration of a data-sharing plan. If the trial report mentioned the existence of a data-sharing plan, we checked whether there was an intention to share data or not. For data-sharing under the secondary outcomes, we checked whether a statement was included in the article, whether the statistical code was shared, whether other data sets than IPD were available and if not, whether only parts were available, and lastly whether any other documents were available.

Under the section data accessibility, we checked to see whether there was a time restriction for data access, whether it was freely accessible or with restrictions, whether the data could be used for any type of purpose, and if not, whether there was an aim for data use suggested in the proposal, whether there was a specific access mechanism and whether data requests were reviewed by an independent committee.

Statistical analysis

All outcomes were reported and described by counts, percentages, means (or medians) and standard deviation (or range) with all the corresponding 95 % confidence intervals. If available, verbatim quotes from journal policies were used (qualitative analysis). For ICMJE-affiliated journals, the features of the included journals were compared as part of an exploratory analysis between journals with and without a specific data-sharing policy using univariate logistic regression and multivariate logistic regression (which included covariates identified in univariate analyses at a threshold of p < 0.25). Due to complete separation observed in our data set, the "brglm" package in R was used, implementing the bias-reduction method developed by Firth (127). All analyses were conducted using R version 3.4.1. The supplementary material on the OSF page contains our statistical analysis plan [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF].

Changes to the initial protocol

The definition of our primary outcome for journal policies was changed. Indeed, we had initially planned to contact journals with no explicit policy on their website to ask them about the existence of a data-sharing policy. Due to non-response from some of the 14 member journals, and because it was not possible to identify an email contact for all ICMJE-affiliated journals, we decided to rely only on the information presented on the journal websites. Some minor changes were also made. Our selection criteria were simplified, so that only one RCT in 2018 was necessary (instead of 3 over the last three years as initially planned). No distinction was made for RCTs in terms of clinical phase. Lastly, we added a secondary outcome, whether or not data requests were reviewed by an independent committee.

Patient and public involvement

We had no established contacts with specific patient groups in this project. No patients were involved in defining the research question or the outcome measures, nor were they involved in the design and implementation of the study. There are no plans to involve patients in the dissemination of results, nor will we disseminate results directly to patients.

RESULTS

Survey of journal data-sharing policies

Journal selection and data extraction

Search for and extraction of eligible journals started on 1 st of February 2019, ended with a consensus on July 11 th 2019 and resulted in 14 ICMJE-member journals and 4892 ICMJEaffiliated journals. Of the affiliated journals, 2367 were randomly screened and 1878 (79%)

were excluded, including 745 journals (31% of all screened journals), for which the journal and/or the publisher were listed as "predatory" on Beall's list. 489 ICMJE-affiliated journals were therefore included in analyses as initially planned. The selection process is reported in For the eight journals with specific data-sharing statements, five referred specifically to Clinical

Trial data, and for the three others it was for all research data submitted.

Sanctions were described in two journals, PLOS Medicine and the Annals of Internal Medicine:

possible rejection of the manuscript if the data was not provided.

Except for the Ethiopian Journal of Health Sciences and the Iranian Journal of Medical Sciences, all journals had their editorial office in high-income countries.

ICMJE-affiliated journals

The characteristics of the 489 ICMJE-affiliated journals are also presented in

RCTs published in ICMJE-member journals

Results are displayed in Table 3 for the 100 selected articles. Among these, 30 were from NEJM, 28 from the Lancet, 17 from JAMA, 13 from PLOS Medicine, 5 from the Annals of Internal Medicine, 3 from the BMJ, 2 from the Journal of Korean Medical Sciences and 2 from the German Medical Journal. Characteristics of the data-sharing plans are detailed in Table 4. Of the 77 articles with data-sharing intentions, 7/77 (9%) mentioned access to other data from the study, besides IPD (e.g. data-frame for "unpublished data" / "medical coding dataset" / "non-patient-level data"). 63/77 (82%) mentioned sharing for the following supplementary documents: the study protocol (for 51), the statistical analysis plan (for 37), the informed consent form (for 15), the data dictionary (for 14), and the case report form (for 5). Time restriction for IPD was present in 34/77 (44%) of the data sets for either the start data of sharing, the end date or both. In 2 data-sharing statements, it was clear that data was to be available directly after approval of the drug in the European Union and in the USA. In 28/30 other cases there was an embargo: 8 after two years, 2 after 18 months, 15 after 1 year, one after 9 months, and 2 after 3 months. A restricted access period was specified for 12 datasets: 6 of these specified restricted access for 2 years, three for 1 year, one for 5 years, one for 10 years and for one it was stated that the time would be defined by the committee. 60/77 (78%) data-sharing statements specified that data could only be used for specific reasons: 53 mentioned a scientific aim only, 6 indicated willingness to share data specifically for Meta-Analyses or Individual

Meta-Analyses and one data-sharing statement specified that the aim of the re-use was to be focused on a particular disease (Herpes Zoster). A specific mechanism was detailed in 68/77 (88%) data-sharing statements. 35 only mentioned the need to establish a data-sharing agreement and/or a formal data request, 20 indicated that an e-mail contact was necessary, and 13 mentioned data-sharing platforms.

22/100 (22%) had registered a data-sharing plan on registers such as clinicaltrials.gov. Of these, 14 specified IPD data sharing, 6 did not, and for 2 it was unclear.

RCTs published in ICMJE-affiliated journals

The 100 selected RCTs were from 38 different journals (mean number of RCTs per journal = 11 (±10)). We found 25 RCTs with data-sharing statements. 17 authors/teams (17% [10%; 26%]) declared an intention to share, while the intention was unclear for an additional 5 (5% [ 2%; 12%). The characteristics of the data-sharing plans are detailed in Tables 3 and4. 7/22 (32%) articles with a positive (or unclear) intention to share data expressed in a data-sharing statement indicated that other datasets, besides IPD, would be available. Regarding the sharing of any other documents, authors stated they would share study protocols (for 7 studies), the statistical analysis plan and the study report (for 4), and the case report form (for 2). Time restriction was present for 10/22 (46%) data sets. Three datasets had a limitation for the start date of data-sharing, ranging from 12 months to18 months and up to 3 years. For the end date of data availability, the following time frames were collected: 5 years in two cases, 3 years for one, 2 years for one, 1 year for 3 and 3 months for one. For the question as to whether data could be used for any type of purpose, 14/22 (64%) eligible datasets were only available for specific purposes (i.e. research). For 10 of these cases the scientific aim was mentioned but not detailed, and in 4 statements, no aim was specified at all. A specific kind of access mechanism was cited in 21/22 (96%) statements. Six of them mentioned a data-sharing agreement, one referred to a data platform, and 14 datasets could be requested by e-mail. For the 13/100 (13%)

trials with registration of their data-sharing plan, two planned to share the data, six did not and for five it was unclear.

DISCUSSION

Statement of principal findings

In our survey we found that 57% of ICMJE-member journals had an explicit data-sharing policy on their website and that approximately a third of the ICMJE-affiliated journals had one.

Slightly more than a third of the member journals and most of the affiliated journals (around 56%) referred to the ICMJE guidelines without specifying a specific data-sharing policy. In addition, nearly 60% of the affiliated journals with an explicit policy had a less demanding policy than that required by the ICMJE. In contrast, the former ICMJE policy of trial registration was better implemented, with more than 71% of member journals and 66% of the affiliated journals explicitly requiring it as part of their policies.

For journals with a data-sharing policy, a data-sharing statement was frequent among member journals (98%), with rates of intention to share data of around 77%. These rates are in line with the intention to share previously reported in the Annals of Internal Medicine (128). In contrast, among ICMJE-affiliated journals with a data-sharing policy, data-sharing statements were not frequent (25%) and the intention to share data was only found in 22% of RCTs published in journals with an explicit data-sharing policy. Importantly, the statements often refer to datasharing upon request, and rarely to a specific repository or to fully available datasets. We already know that, even under a strict data-sharing policy such as the policy in place at the BMJ and PLOS Medicine, data availability is suboptimal, even when researchers express an intention to share [START_REF] Naudet | Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine[END_REF]. And indeed, in a recent scoping review (129) we found that while the willingness to share data was generally high across trials, actual data-sharing rates were generally lower. In addition, there was considerable heterogeneity in data-sharing statements, with a focus on IPD data, and with very inconsistent information related to statistical codes and other documents (e.g. the study protocol or the study report) which are key elements for reproducible research [START_REF] Munafò | A manifesto for reproducible science[END_REF]. Our results therefore question whether the new ICMJE policy as implemented by journals adequately supports clinical trial data-sharing, and they underline the need for efforts towards more reproducible research. Although data-sharing is only one aspect among others (e.g. registration / best practices in reporting), without data-sharing, reproducibility is not possible.

Few characteristics were found to be associated with an explicit data-sharing policy. All were related to the publishers, and the World Bank wealth category of the country/region of the editorial offices. As observed in previous research ( 130), a positive association between the JIF and data-sharing was found in univariate analysis, but it did not survive in the multivariate analysis. While the journal JIF is often (incorrectly) thought to be a surrogate for journal quality, our study suggests that professionalism and characteristics of the publisher and the editorial office resources could be better markers of quality and the implementation of reproducible research policies.

Findings in relation to other studies:

A similar survey conducted in 2019 by our team also identified a lack of implementation of basic data-sharing instructions in surgery journals with a JIF over 2. Only 50% of the journals had a data-sharing policy on their website (131), and in general these policies were not as demanding as those required by the ICMJE.

Furthermore, research done by Dal-Ré and Marušić found independently an alike number of predatory journals in the list of journals which claim to follow the ICMJE recommendations [START_REF] Thwink | Normal Science[END_REF].

It is important to note here that almost all the "big" publishing houses have different datasharing policies for their different journals (e.g. the BMJ group has 3 levels of data-sharing policy, and Taylor & Francis have 5 different types) (21; 22). A related analysis was conducted by Mellor on the various data-sharing policies of the 4 big publishing houses, Elsevier, Springer Nature, Taylor & Francis and Wiley [START_REF] Fanelli | Conceptual challenges concerning re-analysis and replication practices in reproducible research[END_REF]. This survey compared the Transparency and Openness (TOP) guidelines with the data-sharing policies of the different journals [START_REF] Fanelli | A theory and methodology to quantify knowledge[END_REF]. Similar definitions to the ones we used to define more or less stringent requirements were adopted and the authors found that most of the basic or level 1 data-sharing policies were not even TOPcompliant. This confirms our impression that even if policies are in place, they are not sufficiently demanding to be liable to change the data-sharing culture. A harmonization needs to be established, and previous successful experiences should be taken into account [START_REF] Hardwicke | Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the[END_REF]132).

Limitations of this study

In our study, we had to rely on online information, as it proved difficult to contact editors and ascertain the existence of data-sharing policies. While all included journals had an electronic format, we cannot exclude that some may have implemented a policy without mentioning its enforcement explicitly on its website. In addition, we are studying a moving target in a changing environment, and it is likely that some journals that had no explicit policy when we performed our search have now implemented one. Repeated monitoring of the implementation of datasharing policies therefore seems necessary. Another limitation was that we did not check specifically whether the datasets were actually made available when the authors indicated availability in the statement, nor did we request any data to ascertain data availability. Data availability rates could indeed be lower than suggested in an intention to share, as observed in the BMJ overall [START_REF] Rowhani-Farid | Has open data arrived at the British Medical Journal (BMJ)? An observational study[END_REF] and more particularly for clinical trial data, even after communicating with the study authors [START_REF] Naudet | Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine[END_REF]. Moreover, it would be interesting to see how many funders or academic institutions really share their data after expiry of the time restrictions indicated.

A further limitation was the language filter. Due to lack of resources we were not able to include every language. This might have caused a bias as for instance Russian journals might have presented a different data-sharing policy from journals that publish in English.

A large range of journals was included, especially in terms of quality. We tried to limit the inclusion of "predatory journals" using Beall's list. In this matter there is no real gold standard, as no exact definition existed when we planned our study. Other lists such as Cabell's blacklist show an overlap with Beall's list (133). Recently, a new definition has been proposed (134) and it could help to better identify predatory journals. On the one hand, we were surprised by the large number of ICMJE-affiliated journals referenced on the ICMJE website and listed in Beall's list. On the other hand, the investigators performing the data extraction had the impression that some of the selected journals had very poor editorial standards (in cases where instructions for authors were not clear and information was not given for all the steps of the editorial process) and could also fit the definition of predatory journals.

Finally, our identification of factors associated with an explicit data-sharing policy is only exploratory. Several unmeasured confounders, for instance the journal's income and/or the numbers of RCTs published by a given journal, could account for some of the associations found. Other unmeasured confounders may exist and great caution is warranted in interpreting these results, which naturally cannot be considered as reflecting any causal relationship.

Perspectives

It appears that data-sharing policies are infrequent and poorly enforced in most ICMJEaffiliated journals. Perhaps the journals do not know how best to implement the policy, or they may be worried they will lose submissions if the policies are implemented. Other explanations could be the costs resulting from the process or the greater labour intensity. It is also possible that some authors, researchers and indeed editors may be opposed to data-sharing policies. In addition, there is no specific enforcement for an affiliated journal to follow the ICMJE guidelines. It can be noted that the ICMJE states on its website that they "cannot verify the completeness or accuracy of this list" and that "there may be some listed journals that do not follow all of the many recommendations and policies in the document" (123). Furthermore, the large proportion of presumed predatory journals we found, as well as the small proportion of journals enforcing the new policy is of concern for the impact and credibility of the ICMJE. We suggest that journals provide audits and feedback (to readers), especially as the number of ICMJE-affiliated journals is growing very fast, with 4725 in November 2018 (135) and already 5504 in November 2019 (7) (+ 16 % in one year). Without such checks, journals with poor editorial practices could present affiliation with the committee as an endorsement of a sort of quality label in biomedical journals, while this is not the case. The ICMJE affiliation could be indeed perceived as a guarantee, since these standards have of course been endorsed by more than three quarters of the most prominent journals in biomedicine, as illustrated by Shamseer and colleagues in 2016 (136).

In addition, continuous audits of journal policies and their enforcement could be used as a better indicator of journal quality than the current exclusive focus on the JIF. There is room for development of new responsible metrics in this area, encompassing other aspects of reproducible research practices, such as registration policies and the use of reporting guidelines (101). And indeed, data-sharing is only one facet of reproducible research policies.

Steps in the right direction have already been taken, such as the uniform guidelines for datasharing in journals that have been developed by the Data Policy Standardisation and Implementation Interest Group of the Research Data Alliance (137). This could help to reach the goal of full transparency and data-sharing for clinical trial results, since the implementation of the current ICMJE policy seems suboptimal.

ABSTRACT

Objectives

To explore the impact of data-sharing initiatives on the intent to share data, on actual datasharing, on the use of shared data and on research output and impact of shared data.

Eligibility criteria

All studies investigating data-sharing practices for individual participant data (IPD) from clinical trials.

Sources of evidence

We searched the Medline database, the Cochrane Library, the Science Citation Index Expanded and the Social Sciences Citation Index via Web of Science, and preprints and proceedings of the International Congress on Peer Review and Scientific Publication. In addition, we inspected major clinical trial data-sharing platforms, contacted major journals/publishers, editorial groups and some funders.

Charting methods

Two reviewers independently extracted information on methods and results from resources identified using a standardised questionnaire. A map of the extracted data was constructed and accompanied by a narrative summary for each outcome domain. Metrics provided by platforms suggest that a large majority of data remains unrequested. When requested, the purpose of the re-use is more often secondary analyses and meta-analyses, rarely re-analyses. Finally, studies focused on the real impact of data-sharing were rare and used surrogates such as citation metrics.

Conclusions

There is currently a gap in the evidence base for the impact of IPD sharing, which entails uncertainties in the implementation of current data-sharing policies. High level evidence is needed to assess whether the value of medical research increases with data-sharing practices. (141) The ICMJE considers that there is an implicit social contract imposing an ethical obligation for the results to lead to the greatest possible benefit to society. The ICMJE proposed to require that de-identified IPD is made publicly available no later than 6 months after publication of the main trial results. This time lapse would be useless for public health emergencies like COVID-19. However, the ICMJE proposal triggered debate, and a large number of trialists were reluctant to adopt this new norm(142) on account of the feasibility of the proposed requirements, the resources required, the real or perceived risks to trial participants, and the need to protect the interests of patients and researchers.( 143)

INTRODUCTION

Rationale

Despite the cultural shift towards sharing clinical trial data and the major commitment of scientific organisations, funders and initiatives, overall there is still a lack of effective policies in the biomedical literature to ensure that underlying data is maximally available and reusable.

The only requirement appears to be a data management plan or a data-sharing plan. A few journals require data-sharing and, for those who do require data-sharing, guidelines are

Intention to share data

There is an intention to share data, expressed by a stakeholder (e.g., sponsor/PI, funder). This can be done by a written data-sharing commitment or by a declaration included in the trial registration. This also includes surveys on attitudes towards data-sharing.

Actual data-sharing

Data is truly made available for data-sharing to secondary users. This is important because there are cases known where the data is offered for sharing but sharing does not take place, as a result of a possible hidden agenda or change in plans.

Use of shared data

Shared data can be used for various purposes. It can be used as background for research, usually not leading to research outputs. This covers use for education, researcher training and understanding of data. Study types that should lead to new research outputs include 1/ validation/reproducibility of results, 2/ further additional analyses (prognostic models, decision-support, subgroup analyses, etc.) and 3/ IPD meta-analyses.

Research outputs from shared data

Research outputs are scientific presentations, reports and publications.

Impact of research output from shared data

Research output from shared data can have an impact on medical research (e.g. development of new hypotheses and methods) and/or medical health (e.g. changes in treatment via guidelines).

Box 1: Definitions used for the 5 outcome domains

In the scoping review only data-sharing of IPD from clinical trials was considered. We defined clinical trials following the ClinicalTrials.gov definition: "a clinical study is a research study involving human volunteers (also called participants) that is intended to add to medical knowledge. There are two types of clinical studies: interventional studies (also called clinical trials) and observational studies. Clinical trial is another name for an interventional study."(149)

We therefore considered any interventional clinical studies (no matter whether they were randomised), and we did not consider studies on data-sharing concerning observational and non-clinical studies (e.g. on genomics) nor different fields outside medicine (e.g. economics).

We included studies that investigated and reported information on current data-sharing practices performed without restrictions in terms of promotional initiatives, type of repository or platform (see Box 2 for definitions) and that promoted data-sharing practices (e.g. at editorial level, at funder level, at research level etc.). We considered many different types of studies (e.g.

experimental studies, surveys, metrics, quality assurance studies, qualitative research, reviews, reports), as the inclusion criteria were not method-specific but rather content-specific.

Initiatives

Major activities of an organization (or a network of several organizations) to actively promote data-sharing in this area (e.g. Pharmaceutical Research and Manufacturers of America

Repository

Large database infrastructures set up to manage, share, access and archive researchers' datasets from clinical trials. Repositories can be specialised and dedicated to specific disciplines (e.g. FreeBird, Biological Specimen and Data Repository Information Coordination Center (BioLINCC) or more general (e.g. FigShare, Dryad).

Platform

A computer environment where researchers can find datasets from clinical trials across different repositories, and where additional functionalities (e.g. protected analysis environment) are provided (e.g. CSDR, YODA, Project Data Sphere, Github).

Box 2: Definitions used for initiatives, repository and platform

The detailed search terms for the MEDLINE databases, the Cochrane Library and the Web of Science databases can be found in Supplementary Material 2. The main search strategy developed by CO, DM und FN was peer-reviewed independently (by a senior medical documentalist, EM who joined the team subsequently) using evidence-based guidelines for Peer Review of Electronic Search Strategies (PRESS).( 150) Discrepancies were resolved between the authors, and EM performed the search. All references were managed and de-duplicated using a reference manager system (Endnote).

On 23/01/2019 (update on 02/11/2020), two researchers (MS and FN) independently searched for relevant pre-prints on OSF PREPRINTS using the search function to find all papers relevant to medicine with the following keyword (trial* OR random*). On 29/01/2019, the two researchers independently searched the proceedings of the three latest International Congress on Peer Review and Scientific Publication reports for relevant abstracts (2009, 2013 and 2017).

Selection of sources of evidence

The selection of sources of evidence was performed by two independent reviewers (CO and FN). Contact with initiatives/platforms/journals/publishers was made by a single reviewer (MS). In case of disagreements, these were resolved by consensus between CO and FN and, when necessary, in consultation with a third reviewer (DM).

Data charting process

We developed a data collection form and pilot-tested it on 10 randomly selected research papers which were later included in our final study. In case of disagreement, these were resolved by consensus and, when necessary, in consultation with a third reviewer (DM).

Data items

For each research paper included according to the selection criteria we extracted: 1/ basic information on the paper (type of study exploring data-sharing practices, authors, year, references, and type of initiative and/or repository and/or platform studied), 2/ information on the material shared (sharing of data, code, programs and material), 3/ whether it reported data about one or more of the five outcomes domains defined box 1, 4/ how these outcome domains were assessed, and 5/ a qualitative description of the main results observed on these outcomes.

For each data-sharing platform, publisher and funder providing detailed research output from shared data, we extracted the following information (authors, date of request, date of publication, type of re-use). We initially planned to describe the scale of re-use in qualitative terms and the observed results of the re-use (i.e. "positive" or "negative" study) but these two characteristics were difficult to extract with very poor inter-rater agreement and we decided not to detail them.

Critical appraisal of individual sources of evidence

The studies included were classified according to study type (e.g. survey, metrics, experimental). Potentially relevant characteristics of studies included with regard to their internal-external validity and risk of bias were not assessed systematically with a specific tool, but explored when one of the two reviewers considered it relevant, and in this case each study was thoroughly discussed between the reviewers.

Synthesis of results

No outcome was prioritized since there was no quantitative synthesis for this study. All outcomes were described separately in sections corresponding to the outcome domain and subsections corresponding to similar types of initiative. Our plan for the presentation of results was specified in our protocol and organized into 1/ different sections corresponding to the key concepts detailed in the data-sharing pipeline (intention to share data, actual data-sharing, results of re-use, output from data-sharing, impact of data-sharing) and 2/ different subsections corresponding to the different contexts and actors involved in the data-sharing pipeline (e.g.

targeted group for intention to share data or type of use for re-use of shared data)). A summary of the data extracted from the papers included was constructed in tabular form with basic characteristics, and was accompanied by a narrative summary describing all results observed in the light of the review objective and question/s. Usually, individual studies were summarized in a short text with descriptive statistics of the main results (numbers, percentages), when appropriate visual representations of the data extracted were provided.

Patient and public involvement

There was no patient or public involvement in this scoping review. The median year of publication was 2018 (range [2001][2002][2003][2004][2005][2006][2007][2008][2009][2010][2011][2012][2013][2014][2015][2016][2017][2018][2019][2020]). The vast majority of these studies were from North America (50, 54%), Europe (16, 17 %)

Collating and summarising the data

Figure 2 shows the proportion of the 93 references exploring each outcome domain. In an effort to create a useful synthesis of results, we collated results on each outcome from each publication and organised them into the pre-specified categories. -Qualitative: research that relies on non-numerical data to understand concepts, opinions or experiences.

-Other: any other research not covered above (e.g. case studies, environmental scans, etc.)"

Critical appraisal of sources of evidence

In general, there was a high risk of bias, especially due to study design (e.g. surveys with low response rates and absence of experimental design). As stated in the methods, this was not assessed systematically. If available, we have tried to present this information in the narrative part of the review.

Results for individual sources of evidence: intentions to share data

Clinical Trialists

Surveys of attitudes

Four surveys investigating intention to share data by trialists reported high data-sharing rates of around 75% or more (see Figure 4). These surveys targeted authors of published trials and in one study reviewers in a Cochrane group (where the majority of respondents had been involved in a randomised controlled trial (RCT)). The studies differed by different estimations of data-sharing rates, different selection criteria and/or survey methods. Response rates were comparable across the surveys (42-58%). Reviewers in the Cochrane IPD meta-analysis group were strongly in favour of a central repository and of providing IPD for central storage (83%)(151). In the survey by Rathi et al., 74% and 72% respectively thought that sharing deidentified data through data repositories should be required and that investigators should be required to share de-identified data in response to individual requests. However, only 18% indicated that they were required by the trial funder to place the trial data in a repository. In this survey, support for data-sharing did not differ on trialist or trial characteristics. (152) Trialists in Western Europe indicated they had shared or would share data in order to receive academic benefits or recognition more frequently than those from the USA or Canada (58 versus 31%).

The most academically productive trialists less frequently indicated they had withheld or would withhold data in order to protect research subjects (24 versus 40% for the least productive), as did those who had received industry funding compared to those who had not (24 versus 43%).( 153) The survey by Tannenbaum, 2018 suggested that willingness to share data could depend on the intended re-use of the data (97% of respondents were willing to share data for a meta-analysis versus 73% for a re-analysis).( 154) For secondary analyses, the willingness to share was largely influenced by respondents' willingness to conduct a similar analysis. In addition, willingness to share was more marked after 1 year than after 6 months. In the fourth survey on trials published in Chinese medical journals, the overwhelming majority (87%) stated that they endorsed data-sharing (155).

Metrics of data-sharing statements in journal articles

Intentions to share data for trialists were less clear for data-sharing statements in published journal articles (although this section is not specific to clinical trials) (see Figure 4). Depending on the journals considered, the rates vary from less than 5 % to around 25%. An analysis of the first year after the Annals of Internal Medicine policies encouraged data-sharing found that data was available without condition for 4%, with conditions for 57%, and unavailable for 38% analysed by disease. For the case of HIV, which was analysed separately, the sharing rate was higher on average (24.5%). In a prediction model, studies that deposit basic summary results on ClinicalTrials.gov, large studies and phase 3 interventional studies are the most likely to declare intention to share IPD data.

Other data sources

A 2015 survey focused on PCORnet (The National Patient-Centered Clinical Research Network), found that a possible barrier toward data-sharing intentions related to how data can be used when shared with institutions that have different levels of experience, and to the possibility of some "competition" between institutions on the marketplace of ideas.( 167)

Experimental studies

Experimental data suggests that estimations of intention to share data could differ depending on the formulation of the request. For instance, a small randomised prospective study conducted in 2001 including 29 corresponding authors of research publications published in the BMJ, explored their preparedness to share the data from their research.( 168) The email contact, randomly allocated, was in one of two forms, a general request (asking if the author would "in general" be prepared to release data for re-analysis) and a specific request (a direct request for the data for re-analysis). Researchers receiving specific requests for data were less likely and slower to respond than researchers receiving general requests. Similarly, in 2019, a randomized controlled trial in conjunction with a Web-based survey included study authors to explore whether and how far a data-sharing agreement affected primary study authors' willingness to share IPD. (169) The response rate was relatively low (21 %) in this study since more than 1,200 individuals were initially contacted and 247 responded. Among the responders, study authors who received a data-sharing agreement were more willing to share their data set, with an estimated effect size of 0.65 (95% CI [0.39, 0.90]).

Authors of published reports on prevention or treatment trials in stroke were asked to provide data for a systematic review and randomised to receive either a short email with a protocol of the systematic review attached ('Short') or a longer email that contained detailed information, without the protocol attached ('Long').( 170) 88 trials with 76 primary authors were identified in the systematic review, and of these, 36 authors were randomised to Short (trials=45) and 40

to Long (trials=43). Responses were received for 69 trials. There was no evidence of a difference in response rate between trial arms (Short vs Long, OR 1.10, 95% CI 0.36 to 3.33).

within the informed consent process was important or very important. A higher percentage of ethnic minority participants was against data-sharing (white, 6%, vs. "other", 13).

In a second survey with a high response rate (79%), 93% were very or somewhat likely to allow their own data to be shared with university scientists and less than 8% of respondents felt that the potential negative consequences of data-sharing outweighed the benefits.( 176) Predictors of this outcome were a low level of trust in others, concern about the risk of re-identification or about information theft, and having a college degree. 93% and 82 % respectively were very or somewhat likely to allow their data to be shared with academic scientists and scientists in for-profit companies. The purpose for which the data would be used did not influence willingness to share data except for use in litigation. However, patients were concerned that data-sharing might make others less willing to enroll in clinical trials, that data would be used for marketing purposes, or that data could be stolen. Less concern was expressed about discrimination and exploitation of data for profit.

In a survey of Italian patient and citizen groups, 280/2003 (14%) contacts provided questionnaires eligible for analysis.( 177) 144/280 (51%) had some knowledge about the IPD sharing debate and 60/280 (42%) had an official position. Of those who had an official position 35/60 (58%) were in favour and 19/60 (32%) in favour with restrictions. 39% approved broad access by researchers and other professionals to identified information.

Other data sources

While consent seems to be a crucial issue for trial participants, an analysis of 98 Informed Consent Forms (ICFs) found that only 6 (4%) indicated a commitment to share de-identified IPD with third party researchers.( 178) Commitments to share were more common in publicly funded trials than in industry-funded trials (7% vs 3%).

Publishers/funders

Publishers

Metrics of data sharing statements and policies

Several studies were found about the intentions (and data-sharing policies) of publishers. Many publishers have developed data-sharing policies (20-75%), however, less than 10% are mandatory (see Figure 4). In a 2009 survey of editors of different member journals of the World 15/115 (13%). Most paediatric journals recommend that authors deposit their data in a repository but they do not provide clear instructions for doing so.

Funders and clinical trial units

Metrics of data sharing policies by funders

Several studies investigated mandatory data-sharing policies of funders. 30-80% of the noncommercial funders provided data-sharing policies, the highest rates were observed in the US.

Only around 10-20% of these policies were mandatory (see Figure 4). In one study 50% of the policy. Among these 9 funders, only one had a mandatory sharing policy and 8 a policy supporting but not enforcing data-sharing. Funders with a data-sharing policy were small funders in terms of total financial volume.

Three studies investigated mandatory data sharing policies among commercial sponsors (see Figure 4). In a 2016 survey, 22/23 (96%) companies among the top 25 companies by revenue had a policy to share IPD. In a second sample of 42 unselected companies, 30 (71 %) had one.

These policies generally did not cover unlicensed products or trials for an off-label use of a licensed product. 52 % of top companies, and 38 in the sample including all companies considered that requests for IPD for additional trials were not explicitly covered by their policy.( 193 In a survey targeting European heads of imaging departments and speakers at the clinical trials in radiology sessions (July -September 2018), the response rate was 132/460 (29%).( 196)

Responses were received from institutions in 29 countries, reporting 429 clinical trials. For future trials, 98% of respondents (93/95) said they would be interested in sharing data, although only 34% had already shared data (23/68). The main barriers to data-sharing were data protection, ethical issues, and lack of a data-sharing platform.

Results for individual sources of evidence: actual data-sharing

Re-users

Studies related to journal articles

Metrics of actual data-sharing

Several studies have been performed investigating data-sharing rates for studies that have been published in journals, the majority with data-sharing policies and high impact (Figure 5). Even with strict data-sharing policies, the data-sharing rates are low or at most moderate, and vary between 10 and 46%, except for one study with a very high data-sharing rate due to a partly preselected sample of authors willing to share their data ( 154). In the 6-year cross-sectional investigation of the rates and methods of data-sharing in 15 high-impact addiction journals that and 2016 was 17/37 (46%) [START_REF] Naudet | Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine[END_REF].

Experimental studies

In a parallel group RCT, an intervention group (offer of an Open Data Badge for data-sharing)

was compared to a control group (no badge for data-sharing).( 198) The primary outcome was the data-sharing rate. Of 160 research articles published in BMJ Open, 80 were randomised to the intervention and control groups, of which 57 could be analysed in the intervention group and 54 in the control group. In the intervention group data was available on a third-party repository for 2/57 (3.5%) and upon request for 32/57 (56.1%) respectively in the control group:

3/54 (5.6%) and 30/54 (56%). Data-sharing rates were low in both groups and did not differ between groups.

Data sharing for IPD meta-analyses

Metrics of data-sharing for IPD meta-analyses

Some examples demonstrate that data availability for IPD meta-analyses is still limited despite the various data-sharing initiatives/platforms (Figure 5). Availability can be increased under specific circumstances, such as the creation of a disease-specific repository for a scientific The data available shows that the use of these platforms has increased steadily since their initiation and that 50% and more of the data requests lead to actual data-sharing. 158) In 50% of cases availability could not be definitely confirmed.

As part of the Good Pharma Scorecard project, data-sharing practices were assessed for large pharmaceutical companies with novel drugs approved by the FDA in 2015, using data from ClinicalTrials.gov, Drugs@FDA, corporate websites, data-sharing platforms and registries (e.g.

YODA, CSDR) [START_REF] Miller | Sharing of clinical trial data and results reporting practices among large pharmaceutical companies: cross sectional descriptive study and pilot of a tool to improve company practices[END_REF]. 628 trials were analysed. 25% of the large pharmaceutical companies made IPD accessible to external investigators for new drug approvals, this proportion improved to 33% after applying a ranking tool.

Non-commercial sponsors

Disease-specific academic clinical trial networks have a long history of IPD sharing, especially US-related NIH institutions. This is clearly demonstrated by the available literature; however, the metrics of data-sharing are not always as transparent as with the industry platforms, and data cannot be structured and documented easily in a table.

In a survey on the use of the National Heart, Lung, and Blood institute Data Repository, access to 100 studies initiated between 1972 and 2010 was investigated.( 221

Results of individual sources of evidence: re-use

Any type of re-use

The majority of research projects using shared clinical trial data are dealing with new research.

This covers studies on risk factors and biomarkers, methodological studies, studies on optimizing treatment and patient stratification and subgroup analyses. IPD meta-analyses were a less frequent reason for data-sharing requests to repositories and only a few have been reported. Re-analyses are only exceptionally applied.

Early experiences with CSDR, involving GlaxoSmithKline trials found low rates of IPD metaanalyses and re-analyses, the vast majority being secondary analyses (studies on risk factors or biomarkers, methodological studies, predictive toxicology or risk models, studies of optimizing treatments, subgroup analyses etc.).( 213) Similar results were found in an update of the analysis.(215)

In the YODA project, which had received 73 proposals for data-sharing as of June 2017 and had approved 65 proposals, the most common study purposes were to address secondary research questions (n=39), to combine data as part of larger meta-analyses (n=35) and/or to validate previously published studies (n=17) (217).

Among the 172 requests to the National Heart, Lung and Blood Institute (NHLBI) data repository with online project descriptions and coded purpose, 72% of requests were initiated to address a new question or hypothesis, 7% to perform a meta-analysis or combined study analysis, 2% to test statistical methods, 9% to investigate methods relevant to clinical trials, and 9% for other reasons.( 221) In only two requests, the available description suggested a reanalysis.

From 2014 to the end of January 2019, 222/473 (46.9%) of the requests to CSDR gained access to the data (in progress and completed).( 210) 90/222 (40.5 %) of the research teams had completed their analyses by January 2018. 41 published at least one paper, and another 28 that were expected to publish shortly.

In the SPRINT challenge, individuals or groups were invited to analyse the dataset underlying the SPRINT RCT and to identify novel scientific or clinical findings.(230) Among 200 qualifying teams, 143 entries were received.

Further additional analyses

There were few indications concerning the exact type of secondary analysis that was performed.

Approved proposals per subject matter are available for the Cancer Data Access system (CDAS), covering two large cancer screening trials (PLCO, NLST).( 227) Of the 199 approved requests to PLCO between November 2012 and October 2016, 84 (42%) were devoted to cancer etiology, 66 (33%) to trial-related screening, 29 (15%) to other areas, 14 (7%) to risk prediction and 6 (3%) to image analysis. Of the 214 approved requests to NLST, 95 (44%) were devoted to image analysis, 90 (42%) to trial-related screening, 14 (7%) to other subjects, 10 (5%) to cancer etiology and 5 (2%) to risk prediction.

IPD meta-analyses

In one study, IPD meta-analyses proved to amount to a small proportion of data re-use. Among the 174 research proposals approved up to 31 August 2017 by CSDR, 12 proposals were IPD meta-analyses, including network meta-analyses.(203) All were retrospective IPD metaanalyses (i.e. none was a prospective IPD meta-analysis).

Re-analyses

A 2014 survey of published re-analyses (112) found that a small number of reanalyses of RCTs have been published (only 37 re-analyses of 36 initial RCTs) and only a few were conducted by entirely independent authors. 35% of these reanalyses led to changes in findings that implied conclusions different from those of the original article for the types and numbers of patients who should be treated.

In the survey of 37 RCTs in the BMJ and PLOS Medicine published between 2013 and 2016, 14 out of 17 (82%, 95% IC: 59% to 94%) available studies were fully reproduced on all their primary outcomes.( 89) Of the remaining RCTs, errors were identified in two, but reached similar conclusions, and one paper did not provide enough information in the Methods section to reproduce the analyses.

Results for individual sources of evidence: output from data sharing

Publications can be considered as the main research output of data-sharing. Publication activity in the re-use of clinical trial data was considered in several studies. Detailed data are available for academic clinical trial networks and disease-specific repositories in the US, some of them already practising data-sharing for a period longer than 10 years. Here, fair to moderate publication output has been observed depending on the individual repository. So far this is not the case for the repositories storing clinical trial data from commercial sponsors, taking into consideration that these repositories were established around five years ago and that there is usually a considerable time lag between request, approval, analysis and publication. Current statistics indicate improvement in publication output with time.

Non-commercial sponsors

In a cross-sectional web-based survey about access to clinical research data from BioLINCC, covering the period from 2007 to 2014, 98 out of 195 responders (50%) reported that their projects had been completed, among which 66 (67%) had been published.( 209 In five studies (Table 2) the number of publications was reported, usually referring to the number of trials included in the repository/platform.

Reference

Commercial sponsors

Various studies explored metrics of both YODA and CSDR (Supplementary Material 4).

Up to 2021, Vivli's website indicates very little published output. We were not able to retrieve published output from NIDDK.

Results of individual sources of evidence: impact of research output

Evidence on the impact of research output from sharing IPD from clinical trials is still very sparse. So far only two studies, with inconsistent results dealing with this issue and focusing only on citation metrics could be identified.

Metrics on citations

One study, already published in 2007, suggested that sharing detailed research data was associated with an increased citation rate.( 231 cumulative citations that ranked in the top 34% normalized for subject category and year of publication, compared to 28.3% for publications based on observational studies and 29% for random samples. The differences were, however, not statistically significant.

Other data sources

In the SPRINT challenge, individuals or groups were invited to analyse the dataset underlying the SPRINT RCT and to identify novel scientific or clinical findings.(230) Among 200 qualifying teams, 143 entries were received. Entries were judged by a panel of experts on the basis of the utility of the findings to clinical medicine, the originality and novelty of the findings, and the quality and clarity of the methods used. All submissions were also open for crowd voting among the 16,000 individuals following the SPRINT Challenge. Cash prizes were awarded, and winners were invited to present their results. 143 entries to the SPRINT data challenge were received.

DISCUSSION

Summary of evidence

There are major differences with respect to the intention to share IPD from clinical trials across the different stakeholder groups. The studies available so far show that clinical trialists and to some extent study participants, as the two main actors of clinical trials, usually have great willingness to share data (60-80%). This is much less pronounced when it comes to data-sharing statements published in journal articles. Depending on the journals considered, the rates vary from less than 5% to around 25%. The situation is even worse when data-sharing plans documented in registries (e.g. ClinicalTrials.gov) are analysed. Here the willingness to share data is between 5 and 10%.

As a consequence, considerable discrepancy between the positive attitude towards data-sharing in general and the intention to do so in an actual study needs to be ascertained. Publishers, enabling the publication of research output from clinical trials and funders/sponsors financing clinical trials, could be major drivers to change the situation. Meanwhile many publishers have developed data-sharing policies (20-75%), but less than 10% are mandatory and have thus not been enforced. There are differences between journals, with some of the high-impact journals being more involved in the data sharing movement than the others (e.g PLOS Medicine, the BMJ, Annals of Internal Medicine). For funders, the situation is similar, but differs between commercial and non-commercial funders. 30-80% of the non-commercial funders provide datasharing policies, with the US and NIH at the front. Only around 10 to 20% of these policies are mandatory. Data-sharing policies have been developed more often in the group of commercial funders (40-95%) but information on the proportion of mandatory policies is lacking. In short, the pressure by publishers and funders to share data is still limited and the situation is only slowly improving. Stronger policies on data sharing that include a strong evaluation component are needed. The situation is better for the pharmaceutical industry, which has not only promoted data-sharing policies in their organisations to a large degree but has also implemented platforms and repositories, providing practical support for the process of data-sharing (e.g. CSDR, Yoda, Vivli).

Several studies have been performed investigating data-sharing rates for clinical studies that have been published in journals. The focus has been on high-impact journals with strict datasharing policies (e.g. PLOS Medicine, BMJ, Annals of Internal Medicine), demonstrating data-sharing rates between 10% and 46%, except for one study with a very high data-sharing rate due to a partly preselected sample of authors willing to share their data. Data availability for IPD meta-analyses is usually limited (0-20%), available only under specific circumstances (Cochrane group, disease-specific repository) and the availability can be increased to 50% and more. A few individual studies describe access to repositories/platforms from the viewpoint of the user, which does not enable identification of a general pattern. Different initiatives and platforms have been implemented for the pharmaceutical and medical device industry to support sharing of IPD from clinical trials (these platforms are now open to academic trials, but this has not been used very often so far). This covers the YODA project, CSDR, Vivli and SOAR (which is now part of Vivli). The data available shows that the use of these platforms has increased steadily since their initiation and that 50% and more of the data requests lead to actual data-sharing. The reasons for not sharing are numerous but data access is rarely denied by the platforms. One of the hurdles to better acceptance of data sharing is the time delay between a request for data sharing and receiving the requested data. This was not systematically investigated in the scoping review, but a few studies have demonstrated that there may be a considerable time lag between initial request and response (200,205) and the time between request and receiving a data sharing agreement (207).

The majority of research projects using shared clinical trial data deal with new research. This covers studies on risk factors and biomarkers, methodological studies, studies on optimizing treatment and patient stratification and subgroup analyses. This is important because new research may be easier to publish in peer-reviewed journals, which is a major driver of academic careers.

So far only some IPD meta-analyses have been planned as part of data-sharing initiatives, and only a few have been reported. There are many hurdles for IPD meta-analyses, including the findability, the accessibility and the re-usability of datasets (F, A and R in FAIR). ECRIN has developed a metadata dictionary (MDR), able to identify clinical studies and data objects related to it (e.g. protocol, DMP, CRF).( 232) This tool allow for identifying studies for which datasets are available and the conditions for access (ECRIN, MDR). Even if IPD datasets are accessible for meta-analyses, the studies are usually distributed across various repositories. This has been demonstrated in several studies in our scoping review. One central repository could simplify the situation, but instead, the number of repositories is steadily increasing.( 139) The situation could be considerably improved with more standardisation and harmonisation of data and procedures and a federating approach between repositories.

repositories, data-sharing of IPD is still very limited. Possible reasons could include the lack of widely accepted repositories for non-commercial clinical trials and insufficient incentives and benefits related to data-sharing. Some investigators may be reluctant to share their data, other may simply not know how to proceed.

We describe secondary analyses as a very popular type of reuse. These analyses are however exploratory and carry a risk of alpha inflation (due to multiple comparisons). Not all results of these analyses have been published. Alpha inflation and selective reporting can be fertile ground for non-reproducible science and this phenomenon surely deserves attention. Improvements could be achieved with a prospective registration of any protocol for secondary data use similar to the trial registries (e.g. ClinicalTrials.gov), a mandatory link between the registration and the original publication or data set and the need to refer to the primary publication or dataset if the re-analysis is published. Existing approaches and tools could then be extended to automatically identify publications related to re-use of data and establish a link to the original work (e.g. see crossmarkcrossref (236) , metadata repository (MDR) developed by ECRIN linking clinical studies with related data objects).( 232) Another possibility could be to set up a register for secondary analyses.

To be widely accepted, research output from shared data should have an impact on medical research (e.g. generation of new hypotheses) and medical health (e.g. changes in treatment via guidelines). Many interventions seek to maximise the benefit of trial data sharing (e.g. use of incentives for clinical trial data sharing, development of infrastructure for data sharing, etc.) but it is paramount that these interventions are evidence based. It is well known that the impact of primary studies on medical research and health often has a considerable time-lag and direct effects are not easy to demonstrate. So it is to be expected that evidence from research output from shared data is even more difficult to demonstrate. In this scoping review, taking into consideration the limited time available for data-sharing activities to generate an impact, no major effects were to be expected. As a consequence, the evidence on the impact of data-sharing is still very sparse. This could mean that it is still too early to measure any impact, or that the impact is very limited. So far, only surrogate measures have been considered (citation metrics) with inconclusive results. It is hoped that in the coming years, more studies with more relevant criteria and metrics will be performed. One option could be to closely follow up the SPRINT challenge, where 143 secondary analyses on a single clinical trial were performed, and it would be interesting to see whether one or more of these secondary analyses really had an impact.

Limitations

Retrieving and synthesizing information for this study proved to be difficult because we operated in a very siloed landscape where each initiative platform operates with its own metrics.

We have tried to be exhaustive by reviewing both the literature and the most important initiatives. However, it was hard to keep the review up-to date as we were studying a moving target in a rapidly changing environment with more and more new initiatives. Some pharmaceutical companies may operate in their own environment and not on larger data-sharing platforms. This makes these activities even more difficult to track. In addition, data-sharing has not had a long history and many of the initiatives and activities were launched in the recent past. Therefore, only a limited research output from data-sharing can be expected so far and indeed, the number of publications is disappointing. It is expected that the number of publications will increase, and indeed we are already seeing this.

CONCLUSIONS

There is currently a gap in the evidence base evaluating impact of IPD sharing, which causes uncertainties in the implementation and adoption of current data-sharing policies. Data-sharing faces many challenges including, for instance, the scepticism of trialists.( 237) There is therefore a need to provide high-level evidence that the value of medical research liable to inform clinical practice increases with greater transparency, and with the opportunity for external researchers to re-analyse, synthesize, or build on previous data. First, a register (such as PROSPERO( 238))

for any secondary use of shared data should be created. The inclusion in such a register could be mandatory for any data-sharing agreement/publication, as for the registration of clinical trials. This register would make it possible to build an observatory of data-sharing practices providing direct feedback, without the present silos we have to face. In addition, a register of this sort could help to prevent any selective publication of secondary analyses. Lastly, we suggest that interventional studies should be run to determine the optimal data-sharing policy and/or incentives that add value to clinical research. We do however need to take into consideration that the experimental studies performed so far were not very conclusive, indicating that experimental studies in this area are very demanding.

The Supplementary material to this study can be found under this link: https://bmjopen.bmj.com/highwire/filestream/218043/field_highwire_adjunct_files/0/bmjope n-2021-049228supp001_data_supplement.pdf Our hypothesis is that for most trials (> 95%) for which we obtain the data, the results observed on the primary outcome would be fully reproducible. However, although we planned one year for data collection, we are aware that after this time some data would still not be available and thus not be re-analyzable. Nevertheless, the worst-case scenario for precision estimates is that 50% of the studies would be analyzable and reproduced.

Outcomes

The Primary outcome is the proportion of studies where the conclusions were reproduced (yes/no; reproduced and reproduced with verification, as defined above). In case of a divergence for two or more co-primary outcomes in the same study (i.e. one analysis is reproduced and not the other(s)) the different co-primary outcomes were described independently but the whole study was considered as not reproduced. All reasons for classifying studies as non-reproducible or not reproduced were described qualitatively using a taxonomy we developed during the research process.

In addition, we described in what way the data-sharing required clarifications for which additional queries had to be presented to the authors to obtain the relevant information, to clarify labels or use, or both, and to reproduce the original analysis of the primary outcomes.

A catalogue of these queries was created, and we grouped similar clarifications for descriptive purposes to generate a list of some common challenges, and to help tackle these challenges preemptively in future published trials.

Concerning secondary outcomes, we described and compared the main outcomes, p-values and effect sizes in the re-analyzes, and the analyses reported in the EPARs, the study reports and the publications, and we described discrepancies. In addition, for each paper we assessed the presence of the following key reporting biases: selective reporting of the primary outcome and "spin" (261).

In case of outcome switching, meaning that a secondary outcome was considered as a primary outcome in the final analysis, both endpoints had to be re-analysed.

To analyse "spin" in the results observed for the primary outcome, we took the definition provided by Yavchitz et al. who described it as being "a specific way of reporting, intentional or not, to highlight that the beneficial effect of the experimental treatment in terms of efficacy or safety is greater than that shown by the results" (262).

The Modalities of data sharing were described by the following categories: the type of datasharing, the time lapse for collecting the data, the reason for non-availability of data, the deidentification of data (i.e. 18 identifiers, as required by the Health Insurance Portability and Accountability Act) (52) and the type of the shared data (here we distinguish "computerized data" which is not formal or ordered, "cleaned data, categorized and ordered" and "analyzable data" meaning ready for analysis) (263).

RESULTS

Study selection

The searches and consensus finished on 27 February 2020 resulted in 317 main studies identified in 176 EPARs. Of these, 25 were excluded (duplicates and studies with no primary endpoint) resulting in 292 single studies that were included in an MAA of a drug with a positive opinion by the Committee for Medicinal Products for Human Use (CHMP) between the 1st of January 2017 and the 31st of December 2019. Of those, 62 were randomly selected (Figure 1)

and respective data was requested from forty different sponsors. All sponsors were contacted via email and data was requested. After exchange with staff, for six datasets on Vivli, and for three on YODA, 2 requests were issued. antidepressant could not be formally tested, the LS means difference was -4.1 [-7.67, -0.49] (nominal 2-sided P value=.027). » and "… Statistical significance was not achieved for the primary endpoint; nevertheless, the treatment effect (Montgomery-Asberg Depression Rating Scale) for both esketamine/antidepressant groups exceeded what has been considered clinically meaningful for approved antidepressants vs placebo [...] This study provides supportive evidence for the safety and efficacy of esketamine nasal spray as a new, rapid-acting antidepressant for patients with treatment-resistant depression."

For 9/10 studies, results reported in the EPAR, the study report, and the publication were identical (Figure 4). In one study (273) , small numerical differences were observed since the statistical approach required by the EMA for the EPAR (ANCOVA) was different from the approach required by the FDA (Mixed Model with Repeated Measures) and reported in the study report and the paper. In some cases, comparisons were not indicated in the paper nor in the study report (as detailed on Figure 4). patients. The absence of a clear randomization list as well as a data dictionary made the reanalysis very challenging. The results of the re-analysis exceeded the fixed threshold for the effect size of the primary endpoint relapse of depressive symptoms (originally -0.45 vs -0.57 in the re-analysis) but this was judged as a minor clinical difference and the study was considered as reproduced.

In studies using mixed models with repeated measures, we used SAS instead of R in order to reach similar conclusions (this was suggested by the sponsor). However, small numerical inconsistencies were present. Again, for three outcomes in those two studies, the re-analyzed effect size crossed the prefixed threshold of 0.10 points in the effect size. However, the referees in charge (FN and JG) concluded that the differences of -1.05 vs -0.82, -0.76 vs -0.58 and -1.31 vs -1.18 on the change in HbA1c did not affect the conclusions of the study demonstrating large effect sizes in reducing HbA1c.

One study did not specify primary endpoints in its protocol but only objectives (274). We double checked reasons for inclusion. Despite being a single-arm safety study, the trial was eligible since it was labeled as a main study in the EPAR and had primary endpoints described on clinicaltrials.gov. The two researchers that were not involved in the study analysis decided to retain the first endpoint (treatment-emergent adverse events) over eleven primary outcomes listed on clinicaltrial.gov was retained for the analysis as it was in line with the study objectives.

DISCUSSION

Main results

Eight out of 62 main trials (12.9 %) used by the EMA in its approval processes were reproduced.

When IPD was available, all reanalyzes largely reproduced the original results. These results are in line with a precedent survey of RCTs published by PLOS Medicine and The BMJ [START_REF] Naudet | Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine[END_REF].

However, lack of IPD availability hampered our reproducibility effort for most of the trials, despite the fact that a large majority of sponsors had a data sharing policy. Certain trials had extension phases, excluding any data sharing in the sponsors view before study completion.

Similar issues regarding timing of release of IPD has recently been described regarding COVID-19 vaccine trials [START_REF] Tanveer | Transparency of COVID-19 vaccine trials: decisions without data[END_REF].

Such delays, similar to an embargo, could impact the possibility for independent researchers to perform timely reanalyzes. Even for trials sharing IPD, time for demands and receival of data was quite long. Another reason for non-availability of sharing was "lack of scientific merit" as judged by the companies' procedures. Interestingly, we decided on purpose to adopt the registered report format for this paper, in order to pre-emptively address this potential concern:

this publication process allowed for a thorough and independent peer review of its "scientific merit" prior data collection and analysis.

If scientific merit of any data re-use is surely important when it comes to responsible sharing of IPD, it is however a subjective and arbitrary notion. Furthermore, there was no agreement on this point regarding our request, as some sponsors, including those with independent procedures (e.g. those sharing on YODA), agreed to share their data. It is likely that sponsors are less inclined to share their data for the purpose of a re-analysis. A survey of trialists suggested that willingness to share data could depend on the intended reuse of the data with 97% of respondents were willing to share data for a meta-analysis vs 73% for a re-analysis (276). One additional explanation could be the fear of data misuse [START_REF] Rathi | Sharing of clinical trial data among trialists: a cross sectional survey[END_REF]. In addition, in the field of clinical trials, there is currently no systematic culture of reproducibility and independent reanalyzes of clinical trials remain sparse in published literature (112).

Limitations

Caution is needed before generalizing these results to other trials. Our results are focused on a very selected sample of trials, i.e. main studies submitted at the EMA. Those studies (mostly from Europe) are larger than the average published RCT in the medical literature (277) and all were sponsored by the pharmaceutical industry. Implementation of data sharing policies, although not optimal is likely better than implementation by public funders (104). In addition, we selected trials labeled as main studies (pivotal trials) in the EMA dossier and other studies could have been selected from the EPAR, i.e. the so-called supportive trials. Although less important, those supportive trials may have different characteristics that the main studies we included.

Low rates of data sharing limited our ability to explore in details other inferential reproducibility issues. We are exploring these questions in a complementary registered report that received in principle acceptance at Royal Society Open Science (278). While this study is ongoing, we already received an agreement for 90 % of 62 studies randomly selected on the main data sharing platforms (Vivli, YODA and CSDR).

Another limitation of our study is that it was restricted to primary endpoints. While primary endpoints are paramount in main trials, other endpoints (e.g. secondary endpoints and/or safety endpoints) could also be of interest to regulators.

Finally, while we tried to ensure as much as possible that the re-analyst was blind to study results, some bias might have applied to the researcher in charge of reanalyzing the data as he was aware that the studies were part of MAAs tend to be significantly "positive", and indeed, all but one of the included studies were "positive" trials.

Perspective

Contrary to the FDA, the EMA is not conducting independent reanalyzes, making reanalyzes by independent researchers even more important. Possibly, for those trials, the application of data sharing policies should not rely only on the sponsor and appropriate policies should be adopted by the regulatory authorities. While EMA has demonstrated openness toward the idea of transparency with its implementation of policies 0043 and the first step of the even more progressive 0070 policy (119), more action is needed to ensure data to be effectively shared.

Phase 2 of the EMA policy 0070 is foreseeing the sharing of IPD but there is no clear timeline yet. Our results support the urgent need to adopt, implement and monitor this policy.

Nonetheless, in terms of transparency, FDA lags behind EMA. Despite a high amount of drug approval, the former agency does not make regulation documents available to the public, hindering open science regarding drug regulation (119).

In addition, efforts toward transparency and data sharing may be incentivized. Success stories like the Good Pharma Score Card show that data sharing rates are rising when sponsors are made aware of inaccessibility (279). We have recently proposed the concept of registered drug

approvals, an open science pathway for drug marketing authorization that may incentives data

sharing among other open sciences practices (280).

Conclusion

Data sharing practices are rare for reanalyzes of clinical trials from authorized medicines in Europe, even for sponsors with data sharing policies. As a consequence, most main studies used in EPARs lack transparency and their results are not reproducible for external researchers, despite their results support decisions that affect millions of people's health across the European Furthermore, interested researchers can contact the corresponding author via mail. IPD of the respective reanalyzed studies cannot be shared directly but should be requested to the corresponding sponsor.

BACKGROUND

Before drug approval, health authorities like the European Medicines Agency (EMA) and the United States Food and Drug Administration (FDA) evaluate findings from the relevant clinical trials to assess the balance between clinical benefit and safety. When requesting marketing authorization for their drug products, pharmaceutical companies are allowed to choose the indication, design the trials, and choose assessments. In the US, pharmaceutical companies and drug manufacturers must submit full trial protocols to the FDA before those trials can begin. In Europe, companies can, at their discretion, obtain prior scientific advice from the EMA. This consultative process between sponsor and regulator is not fit for purpose, as there is, in practice, no clear a priori consensus on the exact criteria that will be applied to adjudicate success.

Although the FDA lays out a set of a priori rules, all too often, it later bends those rules post hoc. For instance, for Esketamine, for treatment of resistant depression, the FDA decided post hoc that a maintenance trial could substitute for a second positive short-term trial (281). Other examples include Nalmefene for alcohol use disorder (approved by the EMA), which was based on a post hoc subgroup analysis of the pivotal trials (282), or Eteplirsen for muscular dystrophy (approved by the FDA) despite a lack of clinical evidence (283).

Even the initial standards agreed upon between the sponsor and regulator can be too lax. Too often, trials ask the wrong question: Trials may explore superiority over an inappropriately weak comparator such as placebo when superiority versus an already approved active comparator would be more clinically relevant (284). Trials can also be underpowered, focus on surrogate markers, or omit clinically relevant outcomes (285). Moreover, the regulator is laissez-faire with respect to trial publication in journal articles, allowing the sponsor to freely choose which findings to include and how to frame them, often diverging starkly from the regulator's reviews. With few stakeholders aware of these reviews, the journal publication, often rife with selective reporting and spin, becomes the most influential source of information.

Consequently, drugs approvals are frequently marred by inaccuracies and contradictions. Some regulators, like the EMA, do not attempt to replicate the sponsor's analysis. Even the FDA, which reanalyzes individual patient data from the sponsor, does not make the data accessible to independent researchers. The combination of controversial approvals and lack of transparency nurtures justified criticism and decreases societal trust in medicine.

AN OPEN SCIENCE PATHWAY FOR DRUG MARKETING AUTHORIZAION

We propose to adapt the concept of "registered reports" to the process of regulatory drug approval and marketing authorization. It may provide an innovative, unambiguous, transparent, and trustworthy research pathway.

Registered reports represent a publishing format premised on "the importance of the research question and on the quality of methodology by conducting peer review prior to data collection" (105). Transposed into the field of regulatory science, in a registered approval (Figure 1), health authorities would be required, a priori, to pose research questions that matter (in terms of patients, interventions, comparator, outcome, and study design) and define adequate criteria for success, with no possibility of bending the rules after data collection. for evidence generation. Comparative effectiveness will be systematically considered, as will the use of core outcome sets (i.e., an agreed-upon standardized list of outcomes to be measured and reported in all trials for a specific clinical area (288). The required number of direct pivotal trials as well as study designs will be set a priori for the research program, including large simple trials and non-ambiguous criteria for success, e.g., 2 positive confirmatory studies with low risk of bias and a prospective meta-analysis. These criteria will not only define the prespecified analyses and criteria for statistical significance, but also the precise criteria for clinical relevance, i.e., a minimal clinically important difference defined on clinically relevant outcomes or net benefit. Health authorities will ensure a thorough peer review process of the protocols.

Following a positive outcome of the peer review process, drugs would be provisionally granted approvals for specific use in the clinical trials of the registered drug approval research program.

In case of any deviations from the protocol, the committee in charge of the registered approval would agree on the best way to handle them before unblinding and statistical analysis.

Subsequently, an approval would be granted for clinical use and marketing authorization provided that 1/ the research program adhered to the registered methodology; and 2/ predefined criteria for success were met. Approval would require that both conditions have been met.

Transparency would be paramount, with sharing of protocols, followed by aggregated data and individual patient data. Last, all output of the research program would "feed" the living meta-analysis without delay so as to inform future registered drug approvals and to ensure the integrity of the entire scientific process from planning to publication of the results and data.

IMPLEMENTATION CHALLENGES

Practical implementation and acceptability of this pathway could be challenging. Owing to the complexity and resources needed for registered drug approvals, a centralized approach would be desirable. This approach requires the endorsement and harmonization of the pathway among the various health authorities, who currently follow distinct procedures. Nevertheless, the EMA and FDA have already initiated collaboration protocols on drugs (289). Joining forces on an initiative that fosters sound science and scientific integrity seems a compelling reason to strengthen such collaborations.

The most obvious obstacle to this proposal is sponsor buy-in. Adopting such a pathway would require major structural changes in drug laws, which would almost certainly be met by heavily financed opposition and lobbying by drug companies. And because much agency funding comes from drug company user fees, sponsors may be reluctant to lose control over the process by which trial results, which they have long regarded as "trade secrets," are disseminated. Other obstacles could include differences in ethics criteria and specialty-specific clinical guidelines across countries. Therefore, a first implementation initiative would be aimed at encouraging sponsor participation. We propose the pathway as optional for selected drugs that may be eligible, akin to a "golden" approval pathway that would be accompanied by additional and specific incentives.

One such incentive for sponsors could be that, through a single application, this process simplifies the process of access to all markets, owing to the potential involvement of an international agency. A more important incentive is that approval via this pathway honors the ethical duty of all stakeholders toward trial participants who altruistically put themselves at risk and can hence strengthen trust in science. Drugs approved via this pathway would thus gain a seal of quality, affording them a competitive advantage in the marketplace, resulting in a financial incentive for the sponsor.

Nevertheless, it is also important that appropriate incentives be allocated to all involved stakeholders, independently of the results and with a particular focus on data generators.

In such a pathway, one may balance the cost of an independent and robust system of evidence generation with the savings generated by ending the continuous flow of costly drugs, with little added value and concrete risks, approved within the current system.

While these challenges are difficult to overcome, the minimum we believe can and should be achieved is that any trial intended to support drug approval should be submitted as a registered report. Such a publication would not prevent the regulatory agency from post hoc rule bending and approving a drug that shouldn't have been approved, but, at least, clinicians, patients, and policymakers would be apprised of the true outcomes of all trials. Compared to trials disseminated through conventional publication pathways, stakeholders would likely find such registered report publications more credible and informative.

CHAPTER VI Discussion and conclusion

Main results & conclusions of the papers

In the first part of this thesis, we showed that 57% of ICMJE-member journals and around a third of the ICMJE-affiliated journals had an explicit data sharing policy on their website.

However, most journals referred to the guidelines without explicitly mentioning a data sharing policy. Furthermore, we found that the publisher is one of the main factors incentivizing data sharing policies.

When we screened 100 articles for each sample, we saw a high percentage of data sharing statements in member journals; yet, in articles of journals claiming to follow ICMJE guidelines, only 25% had a statement, and only 22% eventually intended to share data. Notably, the statements often refer to data sharing on request and rarely to a specific repository or fully available data sets. We already know that data availability is suboptimal even under a strict data sharing policy such as the policy in place at the BMJ and PLOS Medicine, and even when researchers express an intention to share.

What also surprised, was how many alleged predatory journals claimed to follow ICMJE guidelines. They made up around 30% of the excluded journals. This is critical as these journals are prone to not following ethical publishing guidelines and including them probably would have enlarged the percentage of journals and articles without data sharing policy and data sharing statements.

The question arises if the ICMJE guidelines are just a checkbox that researchers claim to follow.

Even excluding predatory journals, low rates of data sharing were observed in clinical trials and statements. So, it is essential to know what steps are necessary to better implant the guidelines.

Parallel to our paper, Danchev and colleagues screened 487 articles published in JAMA, The Lancet, and NEJM between July 1, 2018, and April 4, 2020. They found that 68.6% of the articles declared data sharing. However, less than one percent were de-identified and publicly available, and the remaining were supposedly accessible via a request to authors. Among the articles declaring that IPD would be stored in repositories, only around 20 % deposited data, primarily because of embargo and regulatory approval. The interpretation of their findings confirms ours in the sense that there is a considerable gap between the intention of data sharing and the performance (290) .

In the broader project of ReiTheR, we wanted to explore the extent of data sharing for 78

funders that financed at least one randomized clinical trial between 2016 and 2018. On this approach, we distinguished between commercial and non-commercial funders. Around 40% had a data sharing policy on their website for both types of funders. In the 200 screened articles funded, 100 for each sample, 77%, and 81% respectively for non-commercial and commercial funders detailed a data sharing plan. Two percent and 59% respectively expressed an explicit intention to share data, exposing a lack of sharing by non-commercial funders (291).

This first segment showed that data sharing in clinical trials was low, thus with a scoping review, we wanted to explore the impact of data sharing initiatives on the intent to share data, on actual data sharing, on the use of shared data, and on research output and impact of shared data in biomedical literature. Our results were that the willingness to share IPD from clinical trials is exceptionally high, but the actual data sharing rates are suboptimal. Depending on the journals considered, the sharing rates vary from less than 5% to around 25%. This conclusion derives from the fact that policies of journals are not applied well enough and data, if it is available, remains unrequested many times. Reanalyzes are rare in this context, and secondary use is more critical when data is requested. Finally, studies focused on the real impact of data sharing were scarce and used surrogates such as citation metrics.

As emphasized in the first article, the publishers who oversee the publication process of research output from clinical trials and sponsors financing clinical trials could be the gamechanger in this crisis of non-available data. Despite this fact, the scoping review found that while many publishers had developed data sharing policies (20%-75%), only about 10% were mandatory. In summary, we concluded that data sharing in biomedical literature is substandard. The main factors are the lack of mandatory data sharing policies from journals and publishing houses.

Moving away from journal articles, we wanted to explore the extent of data sharing and reproducibility in drug regulation in a second step.

Planned as a registered report, the third study tried to inferentially reanalyze 62 studies that were included as pivotal trials in Marketing Authorization Applications. We only managed to get data for ten studies (16.1%). Reasons were heterogeneous. Most IPD was not shared because of the ongoing study status. Nonetheless, some companies found no scientific merit in our approach.

With this, the focus of our paper changed, as the low availability of data became the main research output. Nonetheless, results in the form of p-values were reproduced for all ten studies.

Effect sizes varied slightly for some outcomes. The shared data had to be updated for five trials because several items necessary for reproducibility were missing (e.g., test scores to reanalyze the primary outcome).

This study showed that inferential reproducibility was met when data is shared. At the same time, this project has shown that despite efforts on transparency by the EMA and pharmaceutical companies, it is still challenging to obtain all data and obtain full transparency in drug regulation.

But there is hope. A similar project from our team that surveys data sharing from Phase III clinical trials available on data sharing platform reached a high percentage of available studies (278). It would be interesting to see how many of those studies played a role in marketing authorizations. If a high percentage were attained, new questions on disclosing trials that led to marketing authorization would arise.

Similar studies on this topic are missing. Mainly because documents are not publicly made available by agencies and demands for trial data are lengthy and, in most cases, need to be justified, especially if part of a sponsored trial. In the USA these files are less available than in Europe, and Clinical Study Reports are not made publicly available despite the huge impact of one of the biggest regulators in the world.

Trust in drug regulation is thus not reassured and we thought of a way to improve the situation.

In the last piece, an essay, we wanted to stress the importance of new approaches for drug regulation. This approach should make drug marketing more open so that, among other aims, reproducibility checks post hoc would not be necessary.

Before drug approval, health authorities lay out a set of a priori rules by which clinical trials will be judged as "positive" or "negative". All too often, they bend those rules post-hoc. This results in contentious debates as to whether appropriate thresholds for success were met.

The proposed Open Science pathway for drug marketing authorization, registered drug approval, aims to adapt the concept of registered reports to the process of regulatory drug approval and marketing authorization. It values the clinical importance of research questions, with in-principle approval granted on pre-specified success criteria. This is done before data collection, precluding any post hoc rule-bending, and enabling full transparency. This project has won the essay prize from the Health Research Alliance and PLOS, and thus an expanded project in collaboration with the Center for Open Science is on the way.

2. The reproducibility crisis isn't over -will there be another one?

Thinking that discussion about reproducibility and research findings in crisis will be over soon would be an illusion.

Just recently, another extensive study showed that the replicability of preclinical research in cancer biology is substandard. The author team from the Center for Open Science aimed to replicate the results of 193 experiments from 53 high-impact papers. They only managed to repeat 50 experiments from 23 papers (25.9%).

They explain further that they encountered several challenges when collecting the data.

Although this approach, contrary to ours it did not focus on inferential reproducibility, a high percentage of study data was unavailable and effect sizes were in 92% of replications smaller than the original. Likewise, no experiment was described sufficiently to design protocols for replicability. Nonetheless, over 40% of authors helped reanalyze the data (292).

Even though irreproducibility is still existent in biomedical research, with the importance of data sharing being more highlighted, the question arises if there will be another issue.

Some researchers claim that we are not having a reproducibility crisis, but the real issue is illdefined hypotheses.

Thus, the question arises: will we switch from reproducibility to a hypothesis crisis?

Andrew Gilman, a professor in statistics at Columbia University, writes in his online blog about issues of the replication crisis.

There he presents several causes for this crisis. One of them is that "theories are so flexible that just about any comparison can be taken to be consistent with theory. Remember sociologist Jeremy Freese's characterization of some hypotheses as "more vampirical than empiricalunable to be killed by mere evidence" (293).

Also, Scheel explains in her piece "Why most psychological research findings are not even wrong" that most scientific claims in psychology are too vague. She argues that the claims are critically underspecified; thus, trying to reproduce them would not make any sense. For her, researchers must recognize that more reproducibility or rigor in data collection is not the remedy but a more thoughtful reconsideration of the conceptual basis of their hypotheses before trying to test them (294).

Munafò further elaborates this point of moving beyond the focus of reproducibility. The first author of the piece "A manifesto for reproducible science" published in 2017 [START_REF] Munafò | A manifesto for reproducible science[END_REF] claims in a comment that replication of results is not enough (295). He argues that the latter is not helpful and might provoke the contrary intention. Worth mentioning here is the argument by Penders and Janssens, who claim that when sloppy science is reproduced, meaning bad design and methods, it is not helpful to the research community and does not result in knowledge gain [START_REF] Penders | Finding Wealth in Waste: Irreplicability Re-Examined[END_REF].

According to Munafò, essential protection to bad science is the method of triangulation, which was described by Lawlor and colleagues (296) and has the idea of approaching the same research question from a different angle. Each approach has its unrelated assumptions, strengths, and weaknesses. A whole picture for a research question is created when combining the different aspects.

In the opinion piece, Munafò argues that robustness coming from different sources, fixation on replication needs to go away and credit needs to be given for authors who create collaborations and enhance the idea to evaluate ideas from different angles. Results that agree across different methodologies are less likely to be artifacts (297).

According to the authors, triangulation has not yet reached the attention it deserves. In research reigns the paradigm of Popper which describes that theories can never be proved, only falsified.

However, in his opinion, not enough research projects, including replication attempts, are constructed to falsify a theory. He worries that an overemphasis on repeating experiments could provide an unjustified sense of certainty about findings that rely on a single approach instead of the whole picture.

Furthermore, error detection and rectification in academic publishing is a significant issue.

Allison and colleagues write that it is easy to spot errors in articles but hard to fix them (304).

Expressions of concern are complicated to communicate. Often there is no clear guidance on whom to address, whether the editor, the authors, or the journal itself. The resulting problem is that many journals do not care about mistakes. Moreover, even retractions are complicated, although they would be necessary. Instead of retraction, a response letter to the article is published most of the time. This is advantageous for the journal and authors, creating higher citation rates. Another reason for non-retraction is, according to Allison, that journals charge researchers when making others aware of mistakes. Commentaries to flawed research articles need to be paid. This is delicate since the Committee on Publication Ethics, an independent body that provides advice on handling research misconduct, affirms that no one should pay to read retractions. Likewise, detecting errors in post-publication is not as rewarding as publishing an original article.

Another concern they raise is that data is often not available in the journal article. Even if researchers want to reanalyze the data independently, it is often impossible.

This topic has again been stressed with our findings which confirm that data sharing in biomedical journals is suboptimal.

Issues with Regulatory agencies

Additionally, journals play a paramount role in drug regulation since main trials from marketing authorization applications, such as every other extensive trial, should generally be published in medical reviews.

In this context, the general audience assumes that if a regulatory agency detected issues in trial sites, misreporting or selective reporting of findings, this would be reflected in the peerreviewed literature.

However, in the case of antidepressant trials for 12 different drugs, a team of researchers found that out of 74 FDA-registered studies around a third (31%) were not published [START_REF] Turner | Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy[END_REF].

The FDA inspects a hundred clinical trial sites every year and sometimes finds evidence of substantial departures from good clinical practice and research misconduct. However, the FDA has no systematic method of communicating these findings to the scientific community, opening the possibility that research misconduct detected by a government agency goes unremarked in the peer-reviewed literature.

In a cross-sectional survey of available documents from 1998 to 2003 describing the inspection of clinical trials sites, a team of researchers obtained the inspection documents and compared them to published output. Over 50 clinical trials were found where an FDA inspection had stated an issue in terms of research ethics. Twenty-two trials submitted false information and reporting adverse events, protocol violations, and inadequate record keeping was very common.

More surprising was that only three out of 78 publications that contained trials identified by the FDA as a problem site mentioned the conditions found in inspections. No expressions of concern or other comments were mentioned (305).

In an interview, the study's author detailed that the FDA is not feeding journals with this information, nor does it alert the public. Therefore, it is complicated to detect trials implicated in research misconduct. Also, he recommends that the FDA make unredacted information about its findings of research misconduct available and that journals should require authors to disclose any adverse findings during FDA inspections (306) .

After Seife's study was completed, transparency in the FDA enhanced by releasing an "Inspections Classifications Database Search" (307). This list contains inspections for trials sites and includes the final inspection classification for inspections (308).

However, there is room for improvement. An outstanding achievement would be if the FDA included clinical trial numbers (e.g.: NCTs) in the official documents.

In an opinion piece, Dal-Ré and co-authors make several claims for journals and trial regulation.

They agree with Seife on the aspect of linking inspection reports and clinical trial registries.

They argue that when these kinds of documents are available, journal editors could independently assess the quality of reporting of trial elements and consider publication (309).

Furthermore, an advantage would be that researchers who synthesize evidence for treatment could more easily detect the quality of evidence.

Moreover, funders could better manage risks for patients in their field and improve standards when confronted with problems from similar trials. Finally, this documentation could be used by other regulatory agencies for better decision-making.

Even if the points discussed above are valid, we suggest a limitation as the discussion and our results only focus on two agencies. Indeed, a study from 2017 surveying different regulatory agencies has shown that the FDA has the highest budget followed by the EMA and the Japanese Regulatory Body and shows that EMA has the most experts (310).

Nonetheless, it would be of interest to see how other agencies handle research integrity issues, especially those from emerging countries, like China or India. If these countries will be the future tone giving leaders in drug regulation, exclusion of research misconduct must be granted.

In this context, it is worth mentioning that divergent decisions between agencies and issues around regulation persist (311, 312) .

Hence, how will these issues be handled in the future, and will drug regulation become internationally comparable regarding transparency?

Perspectives

If something has become clear throughout this work, it is that publishers and journals need to start improving data sharing.

The global output from our group efforts were synthesized to frame policies to advance data sharing in biomedical journals. We did so by first identifying the problem, then suggesting a change to the ICMJE policy, followed by a proposal for an evaluation component (313).

Firstly, we identified the poor implementation of ICMJE policies by journals following them.

A reproducibility editor could enforce such a change in the policy. The development of a monitoring software tool could help tackle these issues.

Next, we found a suboptimal intention to share data from clinical trials. One possible suggestion would be to make data sharing mandatory, except if significant obstacles exist. As an example serves PLOS who mandates data sharing for their journals (122). Again, a software could help to monitor if implementation were established or not. In the same way, data sharing upon request should not be allowed as it is ambiguous and does not guarantee data sharing.

Likewise, we found a low intention to share data from trials by funders who have registered trials on a clinical trial registry and have a data sharing policy. Thus, monitoring in the form of initiatives like the EU clinical trials tracker and Good Pharma Score card should be widened (279,314).

Lastly, one of the most critical identified problems is that the real impact of clinical trial data sharing is insufficiently documented. The policies of ICMJE and funders for clinical trials should thus be evaluated regularly and emphasize the beneficial use of data sharing. This could be solved by tracking the re-use of shared data and surveying its impact. For this, all stakeholders, journals, policymakers, funders, and researchers must work closely together.

An idea for future research would be to conduct an intervention trial, similar to the study that tested open data badges for journals [START_REF] Rowhani-Farid | Did awarding badges increase data sharing in BMJ Open? A randomized controlled trial[END_REF]. A sample of journals identified as not-following ICMJE guidelines would serve as a starting point. Two groups would be randomized. Whereas in the intervention group editors would be reminded of ICMJE guidelines, in a control group, no contact with editors would happen. The outcome of the intervention would be observed after a pre-fixed time frame.

Conclusion

In our research, we found that sharing data in biomedical literature and disclosure of clinical trial data from Marketing Authorization Applications is still suboptimal.

This lack contributes to the current reproducibility crisis and is mainly driven by journals and publishing houses.

Despite several initiatives in this field, more actions are necessary, more policies need to be implemented, and more monitoring is required.

Data sharing is a moving target in a quickly changing environment and one can only hope that the situation will improve over time and that stakeholders will adapt suggested changes. In a second step, we conducted a scoping review to explore the impact of data-sharing initiatives on the intent to share data, actual data sharing, use of shared data, and research output and impact of shared data. We concluded that there is currently a gap in the evidence base regarding the impact of sharing individual patient data, resulting in uncertainties in implementing current data sharing policies. Researchers have high intentions to share data but rarely do so.

In the third part of the thesis, the emphasis was on transparency regarding clinical trials in drug regulatory frameworks. We tried to reanalyze 62 studies marked as main trials in marketing authorization applications. Our results showed that individual patient data was available for only 10 of 62 trials (16.1%). The clear message from this research is that clinical trial data for licensed drugs remains inaccessible to the public and the research community. Importantly, re-analyzes of the few trials with available data showed good reproducibility. In the final part, we suggest ideas on advancing open science methods in drug regulatory contexts. In summary, we concluded that sharing data in the biomedical literature is substandard. The main factors are the absence of mandatory data sharing policies on journals, publishers, and regulatory agencies. Adequate policies need to be implemented.

  52 % des participants ont répondu qu'il y avait une « crise de reproductibilité importante » et 38 % ont dit qu'il y en avait une « légère ». Seulement 3 % ont indiqué qu'il n'y en avait pas. De plus, entre 60 et 90 % des chercheurs ont admis avoir rencontré des difficultés à reproduire les résultats d'autres équipes de recherche dans différents domaines de recherche. Dans le même temps, l'enquête a montré que seul un petit nombre de chercheurs ont essayé de reproduire les résultats d'autres chercheurs et qu'il est compliqué de publier ces ré-analyses. Selon les chercheurs interrogés, la publication sélective des résultats, la pression pour publier des résultats nouveaux et significatifs ainsi qu'une faible puissance statistique sont les principales raisons des problèmes de reproductibilité. Depuis un influent article de 2005, cette thèse a attiré une importante attention avec la publication par John Ioannidis de l'article «Why most research findings are false ». Il explique pourquoi les résultats de la recherche sont majoritairement faux en raison de défauts majeurs de méthodologie, d'une trop grande importance des résultats « statistiquement significatifs » en présence de biais tels que la manipulation des analyses et la communication sélective des résultats. Il existe bien sûr d'autres opinions et une discussion est en cours sur la gravité et l'existence éventuelle de cette « crise » de la reproductibilité. Plusieurs chercheurs s'opposent à l'idée d'une réelle « crise » de la reproductibilité. Les problèmes de reproductibilité seraient inhérents à la science, et finalement une bonne chose pour avancer. Déjà Karl Popper a soutenu que les théories ne peuvent pas être vérifiées mais que l'on ne peut que réfuter une théorie. En quelque sorte, l'ir-reproductibilité est une chance d'améliorer la science : la science se corrige d'ellemême.

  La base de données Medline, la bibliothèque Cochrane, le Science Citation Index Expanded et le Social Sciences Citation Index via Web of Science, ainsi que les prépublications et les abstracts de l'International Congress on Peer Review and Scientific Publication ont été consultés. De plus, nous avons inspecté les principales plateformes de partage de données d'essais cliniques, contacté les principaux éditeurs et groupes éditoriaux et certains bailleurs de fonds. Deux examinateurs ont extrait indépendamment des informations sur les méthodes et les résultats des ressources identifiées à l'aide d'un questionnaire standardisé. Une carte des données extraites a été construite et accompagnée d'un résumé narratif pour chaque domaine de résultats. Nous avons identifié 93 études dans la recherche documentaire (publiées entre 2001 et 2020, médiane : 2018) et 5 provenant de sources d'informations supplémentaires ont été incluses dans notre scoping review. La plupart des études étaient descriptives et se concentraient sur les premières phases du processus de partage des données. Alors que la volonté de partager les données individuelles des patients issues des essais cliniques est extrêmement élevée, les taux réels de partage des données sont sous-optimaux. L'analyse des pratiques des journaux médicaux suggère une application insuffisante des politiques par les éditeurs. Les métriques fournies par les plateformes suggèrent qu'une grande majorité des données restent non demandées. Lorsqu'elles sont demandées, la finalité de la réutilisation est le plus souvent des analyses secondaires et des méta-analyses, rarement des ré-analyses. Enfin, les études axées sur l'impact réel du partage de données étaient rares et utilisaient des critères intermédiaires tels que les métriques de citation.

  Dans la troisième partie de la thèse, l'accent a été mis sur la transparence concernant les essais cliniques utilisés dans le cadre réglementaire de l'évaluation des thérapeutiques. La transparence et la reproductibilité devraient être des caractéristiques clés des ECRs utilisés pour la prise de décision concernant les autorisations de mise sur le marché de nouveaux médicaments. Nous avons réalisé une étude transversale visant à évaluer la reproductibilité inférentielle de ces ECRs. Deux chercheurs ont identifié de manière indépendante toutes les études pivots (études désignées comme études principales dans les rapports d'évaluation européens) concernant les nouveaux médicaments et les biosimilaires ayant reçu l'approbation de la Commission européenne entre le 1er janvier 2017 et le 31 décembre 2019. 62 de ces études ont été échantillonnées au hasard, et un chercheur a tenté de récupérer les données individuelles des patients pour ces études ainsi que d'autres documents nécessaires pour une ré-analyse, en contactant les sponsors de l'étude. Pour chaque étude, il a préparé un dossier contenant les données, le protocole et les informations sur le déroulement de la conduite de l'étude. Un second chercheur qui n'avait pas accès aux rapports d'étude ou au code, a utilisé le dossier pour effectuer une ré-analyse indépendante de chaque essai.

Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 2 Figure 2 1 .

 12345221 FIGURES PRESENTED IN INTRODUCTIONFigure 1 Reproducibility rates in articles from the Journal Cognition Figure 2 Kuhn cycle Figure 3 Version of the hypothetico-deductive model and its threats in red Figure 4 Principles for clinical trial data sharing Figure 5 Steps of a registered report approval

Figure 2

 2 Figure 2Kuhn cycle[START_REF] Thwink | Normal Science[END_REF]

Figure 3

 3 Figure 3Version of the hypothetico-deductive model and its threats in red (used under CC BY)[START_REF] Munafò | A manifesto for reproducible science[END_REF]

  Publishing houses like Elsevier and Springer Nature made research articles related to COVID-19 available on open access (74, 75) and the WHO Bulletin made data of COVID-19 research articles available (59).

Figure 4

 4 Figure 4Principles for clinical trial data sharing (used under CC BY-NC)[START_REF] Ohmann | Sharing and reuse of individual participant data from clinical trials: principles and recommendations[END_REF]

Figure 5

 5 Figure 5 Steps of a registered report approval (used under by CC-BY ND) (105)

  Policy 0070 went a step further to proactively publish Clinical Study Reports from 2015. A second stage foresees the sharing of Individual Patient Data (IPD) of the respective trials. Due to Brexit and the relocation of the EMA to the Netherlands, additional developments of the second stage have stopped (116).Concerns about decisions of the EMA remain.

  implementation of the International Committee of Medical Journal Editors (ICMJE) data-sharing policy which came into force 1 st of July 2018 by ICMJE-member journals and by ICMJE-affiliated journals declaring they follow the ICMJE recommendations. Design A cross-sectional survey of data-sharing policies in 2018 on journal websites and in datasharing statements in Randomized Controlled Trials (RCTs) Setting ICMJE website; PubMed/Medline Eligibility criteria ICMJE-member journals and 489 ICMJE-affiliated journals that published an RCT in 2018, had an accessible online website and were not considered as predatory journals according to Beall's List. One hundred RCTs for member journals and 100 RCTs for affiliated journals with a data-sharing policy, submitted after 1 st of July 2018.

  investigators in charge of data extraction (MS, LC, HG) had a one-hour training session and completed a pilot data extraction on 10 journals. For each journal, pairs of these investigators independently assessed eligibility (with reasons in case of non-eligibility) and extracted the data on all the outcomes listed below from each included journal. Disagreements were resolved by consensus or in consultation with a third investigator (FN).

Figure 1 .

 1 Figure 1.

Figure 1

 1 Figure 1: Flow chart of the selection and analysis process for journals and article

Results 93 studies

 93 identified in the literature search(published between 2001-2020, median: 2018) and 5 from additional information sources were included in the scoping review. Most studies were descriptive and focused on early phases of the data-sharing process. While the willingness to share IPD from clinical trials is extremely high, actual data-sharing rates are suboptimal. A survey of journal data suggests poor to moderate enforcement of the policies by publishers.

  Data sharing is increasingly recognized as a key requirement in clinical research.(138) In any discussion about clinical trial data-sharing the emphasis is naturally on the data sets themselves, but data-sharing is much broader. Besides the individual participant data (IPD) sets, other clinical trial data sources should be made available for sharing (e.g., protocols, clinical study reports, statistical analysis plans, blank consent forms) to enable a full understanding of any data set. In this scoping review, there is a focus on the sharing of individual participant data from clinical trials.Within clinical research, data-sharing can enhance reproducibility and the generation of new knowledge, but it also has an ethical and economic dimension.(139) Scientifically, sharing makes it possible to compare or combine the data from different studies, and to more easily aggregate it for meta-analysis. It enables conclusions to be re-examined and verified or, occasionally, corrected, and it can enable new hypotheses to be tested. Sharing can therefore increase data validity, but it also draws more value from the original research investment, as well as helping to avoid unnecessary repetition of studies. Agencies and funders are referring more and more to the economic advantages of data reuse. Ethically, data-sharing provides a better way to honour the generosity of clinical trial participants, because it increases the utility of the data they provide. Despite the high potential for sharing clinical trial data, the launch and implementation of several data-sharing initiatives and platforms, and outstanding examples related to the value of data-sharing,(113) to date data-sharing is not the norm in clinical research, unlike many other scientific disciplines.(140) One major hurdle is that clinical trial data concerns individuals and their health status, and as such requires specific measures to protect privacy.To support sharing of IPD in clinical trials, several organisations have developed generic principles, guidance and practical recommendations for implementation. In 2016, the International Committee of Medical Journal Editors (ICMJE), a small group of medical journal editors, published an editorial stating that "it is an ethical obligation to responsibly share data generated by interventional clinical trials because participants have put themselves at risk".

  heterogeneous and somewhat ambiguous.(130) Nevertheless, some innovative and progressive funders (e.g. Wellcome Trust, Bill & Melinda Gates Foundation), and publishers/journals (e.g.Public Library of Science (PLOS) [in 2014],The British Medical Journal (BMJ))[2009][2010][2011][2012][2013][2014][2015], have adopted strong data-sharing policies. As part of a wider cultural shift towards more open science, there have been various attempts to explore how clinical researchers can best plan for data-sharing and prepare their 'raw' IPD so that it becomes available to others(144)albeit often under controlled access conditions rather than simply being publicly available online(145) -and can structure that data to make it FAIR (findable, accessible, interoperable and reusable).(146) Meanwhile several data-sharing platforms and repositories are available and in use to provide practical support for the data-sharing process in clinical research (e.g. Yale University Open Data Access (YODA) launched [in 2011], ClinicalStudyDataRequest.com (CSDR) [launched in 2013], Vivli [launched in 2018]. A considerable number of individual studies have been performed to access and explore the sharing of data from clinical trials under different circumstances and within different frameworks. What is strongly needed is a scoping review providing an overview of the status of implementation of data-sharing as a whole and the implications originating from the available evidence.

(

  PHRMA)/European Federation of Pharmaceutical Industries and Associations (EFPIA), Nordic Trial Alliance, Institute of Medicine (IOM), ICMJE, Research Data Alliance (RDA)).

Figure 1

 1 Figure 1: PRISMA flow diagram (PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-

Figure 3

 3 presents a detailed overview of the different outcome domains and the related outcomes used in the 93 different references included, organised by type of research.

Figure 2 :

 2 Figure 2: Proportion of the 93 references exploring each outcome domain Study designs considered: -Experimental: prospective research that implies testing the impact a strategy (e.g. randomised controlled trial) -Survey: a general overview, exploration, or description of individuals and/or research objects; -Metrics: descriptive metrics from each initiative provided by the initiative;

Figure 3

 3 Figure 3: Outcomes used to assess current data-sharing practices for individual patient data for clinical

Figure 4

 4 Figure 4: Intent to share

  top non-commercial funders had a data-sharing policy but it was found that in only 2/20 cases data-sharing was required. Six funders offered technical or financial resources to support IPD sharing.(190) Trial transparency policies were investigated for 9/10 top non-commercial funders in the US (May to November 2018).(191) 7/9 (78%) funders had a policy for individual patient data-sharing, for 1 it was mandatory. 6 offered data-sharing and 5 monitored compliance. Of 96 responders out of 190 non-commercial funders contacted in France, 31 were identified as funding clinical trials (2019).(192) 9/31 (29%) had implemented a data-sharing

  published clinical trials between 2013 and 2018, none of the 394 clinical trials included shared their data publicly (183). Of 86 responders in a survey targeting the corresponding authors of 619 RCTs published between 2014 -2016 in 7 high-ranking anaesthesiology journals, raw data was obtained only for 24 studies.(159) 62 declined to share raw data. In a study targeting PLOS Medicine and PLOS Clinical Trials publications conducted in 2009, 1/10 (10%) of the data sets was made available after request (197). In articles in Chinese and international journals from 2016, sharing practices were indicated for 29/247 (11%) of the articles.(155) Among the top 10 general and internal medical journals investigated in 2016, IPD was provided after request for 9/61 (15%) of pharmaceutical-sponsored studies (111). For BMJ research articles published between 2009 and 2015, data sets were made available in 7/157 (4%) of the articles(88). For the sub-sample of clinical trials, the rate was higher (5/21 (24%)). Of 317 clinical trials published in 6 general medical journals between 2011 and 2012, 115 (36%) granted access to data(152). The data availability for RCTs published in BMJ and PLOS Medicine between 2013

Figure 5

 5 Figure 5: Actual data-sharing

  ) A total of 88 trial datasets were requested at least once, and the median time from repository availability and the first request was 235 days. Since its inception in 2006 and through to October 2012, nearly 1700 downloads from 27 clinical trials have been accessed from the Data Share website belonging to the National Drug Abuse Treatment Clinical Trial Network (CTN) in the US, with use increasing over the years.(222) Individuals from 31 countries have downloaded data so far. In a case study approach, the data-sharing platform Data Share of the National Institute of Drug Abuse (NIDA) was investigated in detail.(223) As of March 2017, the Data Share platform had included 51 studies from two trial networks (36 studies from CTN and 15 studies from NID Division of Therapeutics and Medical Consequences). From 2006 to March 2017, there have been 5663 downloads from the Data Share website. Of these, 4111 downloads have been from the US. The Project Data Sphere (PDS) is an open-source data-sharing model that was launched in 2014 as an independent, non-profit initiative of the CEO roundtable on cancer.(224) PDS contains data from 72 oncology trials, donated by academics, governments, and industry sponsors. More than 1400 researchers have accessed the PDS database more than 6500 times. As an example, a challenge to create a better prognostic model for advanced prostate cancer was issued in 2014, with 549 registrants from 58 teams and 21 countries. The Immune Tolerance Network (ITN) is a National Institute of Allergy and Infectious Diseases /National Institutes of Health-sponsored academic clinical trial network.(225) The trial sharing portal, which was released for public access in 2013, provides complete open access to clinical trial data and laboratory studies from ITN trials at the time of the primary study publication. Currently, data from 20 clinical trials is available and data for an additional 17 will be released to the public at the timepoint of first publication. So far, more than 1000 downloads have been registered. In the MRC Clinical Trials Transparency Review Final Report (November 2017), the MRC United Kingdom (UK) reported that 24/107 (22%) trials that started during the review period had created a database for sharing. Seven of these datasets (7/24, 29%) had already been shared with other researchers.(226) Of 215 requests submitted for PLCO (Prostate, Lung, Colorectal and Ovarian) cancer screening trial data, 199 (93%) were approved, and for NLST (National Lung Screening Trial) 214 (89%) out of 240 requests.(227) Other stakeholders In a case study about experiences with data-sharing among data monitoring committees, access to five concurrent trials assessing the level of arterial oxygen, which should be targeted in the care of very premature neonates, was investigated.(228) The target of taking all relevant evidence into account when monitoring clinical trials could be only partially reached. One case-study directly addressed the issue of costs. Data from two UK publicly funded trials was used to assess the resource implications of preparing IPD from a clinical trial to share with external researchers.(229) One trial, published in 2007, required 50 hours of staff time with a total estimated cost of £3185, and the other published in 2012 required 39.5 hours with £2540.

  ) Of the 97 respondents who had not yet completed their proposed projects, 81 (84%) explained that they planned to complete their project; 63 (65%) indicated that their project was in the analysis/manuscript draft phase. In a survey targeting European heads of imaging departments and speakers at the Clinical Trials in Radiology sessions (July -September 2018), 23/68 reported that they had already shared data.(196) At least 44 original studies were published based on the data shared by the 23 institutions involved.

Figure 6

 6 presents publication metrics for CSDR (up to 31 August 2019) and YODA (up to 1 st July 2019). Among 88 published papers (62 from CSDR and 26 from YODA), 49 were secondary analyses (42 from CSDR and 7 from YODA), 30 were meta-analyses (13 from CSDR and 17 from YODA), 6 were methodological studies (5 from CSDR and 1 from YODA) and 3 were re-analyses (2 from CSDR and 1 from YODA). The details of these publications are presented in Supplementary Material 5.(212, 215, 217)

Figure 6 :

 6 Figure 6: Temporal trends, number and type of published output from CSDR and YODA (Blue YODA, Red CSDR)

  ) Of 85 cancer microarray clinical trials published between January 1999 and April 2003 41 made their microarray data publicly available on the internet. For 2004 -2005, the trials with publicly available data received 85% of the aggregate citations. Publicly available data was significantly associated with a 69% increase in citations, independently from journal impact factor, date of publication and the author's country of origin. Citation metrics for 224 publications based on repository data for clinical trials in the NHLBI Data Repository were compared with publications that used repository observational study data, as well as a 10%-random sample of all NHLBI-supported articles published in the same period (January 2000 -May 2015).(221) Half of the publications based on clinical trial data had

Figure 1

 1 Figure 1 Flow chart of the selection and analysis process for main trials

  Figure 4 Original study effect size versus reanalyzed effect size in different types of reports

Figure 1

 1 Figure 1 Overview of the registered drug approval pathway

Titre:

  Reproductibilité de la recherche thérapeutique -Une enquête sur le partage de données dans la littérature biomédicale et les essais cliniques dans les autorisations de mise sur le marchéMots clés : Reproductibilité, Transparence, Essais cliniques Abstract : Plusieurs chercheurs ont décrit l'existence potentielle de difficulté de reproductibilité des travaux scientifiques. Diverses pratiques de science ouverte pourraient néanmoins maximiser la reproductibilité. Cette thèse porte sur l'une de ces pratiques, le partage de données, dans les sciences biomédicales. Dans la première partie, nous voulions explorer la mise en oeuvre de la politique de partage de données du Comité international des éditeurs de revues médicales (ICMJE), qui est entrée en vigueur en juillet 2018. La mise en oeuvre des exigences de partage de données n'était pas optimale pour les revues membres de l'ICMJE et médiocre pour les revues affiliées à l'ICMJE. Dans une deuxième étape, nous avons mené une « scoping review » pour explorer l'impact des initiatives de partage de données sur l'intention de partager des données, sur le partage réel des données, sur l'utilisation des données partagées et sur les résultats de la recherche et l'impact des données partagées. Nous avons conclu qu'il existe actuellement un manque de connaissance important concernant la pratique du partage des données issues des essais cliniques, en particulier en ce qui concerne l'impact de cette pratique. Il y a de grandes incertitudes sur l'impact potentiel des politiques actuelles de partage des données. En outre, des preuves de bonne qualité sont nécessaires pour évaluer si la valeur de la recherche médicale augmente avec les pratiques de partage de données. Dans la troisième partie de la thèse, l'accent a été mis sur la transparence concernant les essais cliniques utilisés dans le cadre réglementaire de l'évaluation des thérapeutiques. Nous avons essayé de ré-analyser 62 études considérées comme des essais principaux par l'Agence Européenne du médicament dans les demandes d'autorisation de mise sur le marché. Nos résultats ont montré que les données individuelles des patients n'étaient disponibles que pour 10 des 62 essais(16,1 %). Le message clair de cette recherche est que les données d'essais cliniques pour les médicaments mis sur le marché restent inaccessibles au public et à la communauté des chercheurs. Il est important de noter que les ré-analyses des quelques essais avec des données disponibles ont montré une bonne reproductibilité. Dans la dernière partie, nous proposons d'aller un peu au-delà de la notion de partage des données et de réfléchir dans quelle mesure la science ouverte pourrait améliorer les pratiques des autorités sanitaires. En résumé, le partage des données dans la recherche thérapeutique est sous-optimal. Les politiques de partage, quand elles existent, sont laxistes et mal implémentées. Nous proposons plusieurs pistes pour aller vers un changement.Title : Reproducibility in TherapeuticResearch -A survey on Data Sharing in the biomedical literature and Clinical Trials in Marketing Authorizations Keywords : Reproducibility, Transparency, Clinical Trials Abstract: Several researchers in biomedicine have described a reproducibility crisis. Various open science practices may maximize reproducibility. This thesis focuses on data sharing and its extent in the biomedical sciences. In the first part, we wanted to explore the implementation of the data sharing policy of the International Committee of Medical Journal Editors (ICMJE), which came into effect in July 2018. Implementation of the data sharing requirements in journal policies was suboptimal for ICMJE member journals and poor for ICMJE affiliated journals.

TABLE PRESENTED IN INTRODUCTION Table 1 Comparison of terminologies before harmonization TABLES PRESENTED IN ARTICLES

 PRESENTEDINTRODUCTION1

	Paper 1,

Table 1 Characteristics of journal policies for ICMJE-member and ICMJE- affiliated journals

 1

	Paper 1,

Table 2 Journal characteristics associated with an explicit data-sharing policy

 2

Paper 1, Table

3

Characteristics of all published randomised controlled trials included Paper 1, Table

4 Characteristics

of the data-sharing statements for articles with an intention to share individual patient data (including those with unclear intentions)

	Paper 2, Box 1 Definitions used for the five outcome domains
	Paper 2, Box 2 Definitions used for initiatives, repository and platform
	Paper 2, Table 1 Metrics of CSDR, YODA and Vivli websites	101
	Paper 2,

Table 2 Studies reporting published outputs for non-commercial sponsors 107

 2

Paper 3, Table

1 Summary

of studies for which data was received 131

Table 1

 1 Comparison of terminologies before harmonization, adapted fromPlesser, 2018 (2)

Table 1

 1 The respective adjusted Odds Ratios can be found in Table2.

	Among the 145 journals with a specific data-sharing policy 11 (7% [4%; 13%]) had a more
	stringent policy than that required by the ICMJE, 49 (34% [26%; 42%]) journals were ICMJE-
	compliant and 85 (59% [50%; 67%]) had a less demanding policy than required by the ICMJE
	that did not explicitly require a data-sharing statement. Nineteen (of 145; 13%) journals with
	data-sharing policies referred only to Clinical Trial data while for the rest, the statement
	comprised a more general statement. For 94 (of 145; 65%) journals no start date was found for
	the policy, 25/145 (17%) had a policy starting in early 2018 (January and February) and 26/145
	(18%) had a policy starting on 1 st of July 2018.

. 145 of them (30%, [95% Confidence Interval 26%; 34%]) had a specific data-sharing policy on their website. Two hundred and seventy-six journals (56% [52%; 61%]) merely referred to the ICMJE guidelines, without any specific mention of a data-sharing policy. 68 (14% [11%; 17%]) had no data-sharing policy and did not allude to the ICMJE in their recommendations. In contrast, 178 (37% [32%; 41%]) required prospective trial registration, 142 (29% [25%; 33%]) asked for trial registration without specifications, 114 (23% [20%; 27%]) referred to the ICMJE and 55 (11% [9%; 14%]) did not refer to any trial registration.

Table 2 :

 2 Journal characteristics associated with an explicit data-sharing policy § aOR = adjusted Odds Ratio

	+ Missing data for 2 journals: 1 for journals without explicit data-sharing policy and 1 with explicit
	data-sharing policy
	* Missing data for 21 journals: 4 for journals without explicit data Sharing policy and 17 with explicit
	data-sharing policy
	~ Journals that published over 15 journals in the medical domain; big output > 1000 journals, medium
	output 250-1000 journals, small output < 250 journals in publisher repertoire
	other: Publishers that did publish under 15 journals in the medical domain

Registration of a data-sharing plan

		ICMJE-member journals	ICMJE-affiliated journals
		(n = 100)	(n = 100)
	Data-sharing statement in article	98 (98 % [92 %; 99 %])	25 (25 % [17 %; 35 %])
	Intentions to share Individual Patient
	Data in statement
	Yes	67 (67 % [57 %; 76 %])	17(17 % [10 %; 26 %])
	No	21 (21 % [14 %; 31 %])	3 (3 % [0.1 %; 9 %])
	Unclear	10 (10 % [5 %; 18 %])	5 (5 % [2 %; 12 %])
	Not available	2 (2 % [0.3%; 8 %])	75 (75% [65 %; 82 %])
	Type of Registration
	Prospective	80 (80 % [71 %; 87 %])	50 (50 % [40 %; 60 %])
	Retrospective	20 (20 % [13 %; 29 %])	22 (22 % [15 %; 32 %])
	Unclear	-	28 (28 % [20 %; 38 %])
	Yes	10 (10 % [5 %; 18 %])	8 (8 % [4 %; 16 %])
	Yes, but not in original version	12 (12 % [7 %; 20 %])	5 (5 % [2 %; 12 %])
	No	78 (78 % [68 %; 85 %])	87 (87 % [78 %; 93 %])

Table 3 :

 3 Characteristics for all published Randomized Controlled Trials included

	Almost all the articles (98% [92%; 99%]) had a data-sharing statement. The two articles without
	data-sharing statements were from the Journal of Korean Medical Science and were confirmed
	as having been submitted after 1 st of July 2018. 67% [57%; 76%] of the statements indicated
	an intention to share data while the intention was unclear for an additional 10 (10% [5%; 18%]).

Table 4 :

 4 Characteristics of the data-sharing statements for articles with an intention to share Individual Patient Data (including those with unclear intentions)

  ) A second survey studied data availability for 56 publications reporting on 61 industry-sponsored clinical trials of medications.(111) Of these 61 studies, 32 (52%) had a

	public data-sharing policy/process.
	78 non-commercial funders and a sample of 100 leading commercial funders in terms of drug
	sales having funded at least one RCT in the years 2016 to 2018 were surveyed (15 February
	2019 -10 September 2019).(194) 30/78 (38%) non-commercial funders had a data-sharing
	policy with 18/30 (60%) making data-sharing mandatory and 12/30 (40%) encouraging data-
	sharing. 41/100 (41%) of the commercial funders had a data-sharing policy. Among funders
	with a data-sharing policy, a survey of two random samples of 100 RCTs registered on
	Clinicaltrial.gov found that data-sharing statements were present for 77/100 (77%) and 81/100
	(81%) of RCTs funded by non-commercial and commercial funders respectively. Intention to
	share data was expressed in 12/100 (12%) and 59/100 (59%) of RCTs funded by non-
	commercial and commercial funders. The survey indicated suboptimal performance by funders
	in setting up data-sharing policies.
	Other data sources
	A 2005 survey of 107/122 accredited medical schools in the US (response rate = 88%) explored
	data-sharing in the context of contractual provisions that could restrict investigators' control
	over data in the context of industry-funded trials.(195) There was poor consensus among senior
	administrators in the offices of sponsored research at these institutions on the question of
	prohibiting investigators from sharing data with third parties after the trial is over (41 % allowed
	it, 34 % disallowed it, and 24 % were not sure whether they should allow it).

Metrics of data-sharing policies by CTUs

Among 23 UK Clinical Research Collaboration (UKCRC) registered Clinical Trial Units (CTUs) (response rate = 51 %), 5 (22 %) had an established data-sharing policy and 8 (35%) specifically required consent to use patient data beyond the scope of the original trial (see table).(145) Concerns were raised about patient identification, misuse of data, and financial burden. No CTUs supported the use of an open access model for data-sharing.

Table 1 :

 1 The reasons for not sharing are numerous but data access is rarely denied by the platforms. Our assessment of CSDR, YODA, NIDDK and Vivli websites is presented in Table1. Metrics of CSDR, YODA, and Vivli websites ; NIDDK also provided metrics concerning the number of requests (530) but no other information This was the case despite the time lapse of 18 months since the completion of the drug trials by the company sponsor. Differences across sponsors were observed. Pharmaceutical repositories may cover only part of the trials with commercial sponsors needed for metaanalyses. In a study investigating data availability for industry-sponsored cardiovascular RCTs with more than 5000 patients, performed by a top-20 pharmaceutical company and registered at ClinicalTrials.gov (up to Jan. 2015), only 25% of the identified trial data was confirmed to be available.(

	Platform Metrics date Available	Number of	Number of	Number of	Number of
		studies	requests	requests with	requests with	publications
				data shared	data leading to
					publication
	CSDR	30/11/2020 3008	621	318	59*	79
	YODA	15/11/2019 334	196	173	29	35
	Vivli	02/11/2020 5203	215	123	8	9

*publication anticipated

Metrics of trial coverage for data-sharing

Ethics approval in applications for open-access clinical trial data from CSDR was investigated in a survey.(211) Projects with and without ethics approval were applied to at roughly similar rates

(62/111 and 43/61)

.

RCTs registered in ClinicalTrials.gov, only 512 (53%) were available on CSDR and only 385 (40%) of the RCTs were registered and listed on CSDR with all datasets and documents available.

Table 2 :

 2 Studies reporting published outputs for non-commercial sponsors

		Repository/	No. of trials	No. of published	Assessment
		platform	included in	articles
			repository/platform
	Shmueli-	CTN Data	27 trials	13	2012
	Blumberg,	Share	(1700 downloads)
	2013
	Zhu, 2017	CDAS	2 trials (PLCO,	25% for PLCO	2016
			NLST)	projects, 19% for
			(455 requests)	NLST projects
	Coady, 2017	BioLINCC	100 trials	35% of clinical	5/2016
			(88 requested at	trials at least 1
			least once)	publication 5
				years after
				availability in the
				repository
	Huser, 2018	NIDA Data	51 trials	14	3/2017
		Store
	Pisani, 2017	WWARN	186 trials	18	2016

  European Commission between January 2017 and December 2019 marked as main studies in the European Public Assessment Reports (EPAR). 62 of these studies were randomly sampled. One researcher retrieved the Individual Patient Data for these studies and prepared a dossier for each study, containing the IPD, the protocol and information on the conduct of the study. A second researcher who had no access to study reports, used the dossier to run an independent re-analysis of each trial. All results of these re-analyzes were reported in terms of each study's conclusions, p-values, effect sizes and changes from the initial protocol. A team of two researchers not involved in the re-analysis compared results of the reanalyzes with published results of the trial.The influence of main trials (i.e. evidence used for drug marketing approval) as assessed by the European Medicine Agency (EMA) is paramount. These studies have a major impact on drug Marketing Authorizations and can change the practices of European medical practitioners and the care offered to millions of patients in the European Union. Because of the major financial conflicts of interest inherent in the evaluation of pharmaceuticals (239, 240), stakeholders are typically more confident when the results and conclusions of these studies can be verified. For a long time, however, transparency was lacking and the individual patient data (IPD) and accompanying material (code, protocol, data analysis plan, etc.) to reproduce these analyses were unavailable. An empirical analysis suggests that only a small number of re-analyzes of randomized controlled trials (RCTs) have been published to date; of these, only a minority were conducted by entirely independent authors (112). Data-sharing enabling such re-analyzes is being increasingly mandated in medicine.And indeed, the EMA aimed to pioneer transparency in this field when, in November 2010, it decided to share every piece of documentation received, in the wake of the first version of policy 0043 (115). As part of its transparency policy, the EMA publishes European Public December 2015 in the top 10 journals of general and internal medicine (111). If such low rates of data-sharing were also to be observed for main trials, it would invalidate any efforts towards reproducibility for these important studies.However, the environment for data-sharing is changing fast. And indeed, data-sharing Platforms like ViVli, YODA project or Clinical Study Data Request are more and more widely used. In fall of 2019 these platforms gathered a large number of trials sponsored by the pharmaceutical industry. The three together reached about 8000 RCTs in November 2019(129).Despite this available data, re-analyzes are still sparse. Among the 88 published outputs we identified (before our research) resulting from data-sharing on these platforms, only 3 were reanalyzes: "Restoring Study 329" by Le Noury et al. which contradicted the initial publication, a trial that was already known to be misreported (113), a re-analysis of the TORCH trial suggesting an overestimation of the treatment effect in the original study (243), and the reanalysis of the "SMART-AF" trial which came to similar conclusions to the original study (244).

	1. ABSTRACT Background Transparency and reproducibility are expected to be key features in clinical trials used for decision-making on Marketing Authorizations for new medicines. This registered report introduces a cross-sectional study aiming to assess inferential reproducibility for main trials assessed by the European Medicine Agency (EMA). Methods Two researchers independently identified all studies on new medicines and biosimilars given 2. BACKGROUND Assessment Reports (EPAR) after the European Commission's decision on the specific medicines. These reports include, amongst other documents, results of main trials (241). On and 31 As part of a global research program on reproducibility in therapeutic research (ReiTheR, funded by the French National Research Agency), we designed the present cross-sectional study to assess inferential reproducibility (i.e. when IPD is available, whether qualitatively similar approval by the Results October 2nd 2014 the EMA released its policy 0070 on "publication of clinical data for conclusions can be drawn from a re-analysis of the original trials) for main studies assessed by
	of January 2015 clinical reports on medicines submitted for Marketing Authorization have been 292 main studies in 176 EPARs were identified. For 10/62 of randomly sampled studies, we medicinal products for human use"(116). The agency describes a two-step approach. From 1st the EMA.
	received IPD (16.1% [95% CI 8%; 27.7%]). Median number of days between data request and published. A second step includes the publication of IPD. A date for the implementation of this
	data receival was 253 [Interquartile range 182 -469]. For the ten trials, we identified 27 distinct step still needs to be fixed. However, as a result of Brexit and the relocation of the EMA to the
	outcomes eligible for reanalyzes. Conclusions of 8/62 trial (12.9% [95% CI 5.7%; 23.9%]) were Netherlands, further developments and renovation have been stopped for the moment (117,
	reproduced. The two remaining studies were not reproducible because data concerning one of 242). Efforts are therefore still needed to reach full transparency in the EMA.
	their principal outcomes were not available (deleted as part of the anonymization process).
	On the other hand, biopharmaceutical companies (i.e. Pharmaceutical Research and
	Conclusion Manufacturers of America [PhRMA] and the European Federation of Pharmaceutical Industries
	Despite their results support decisions that affect millions of people's health across the European and Associations [EFPIA]) endorsed a commitment 'to enhancing public health through
	Union, most main studies used in EPARs lack transparency and their results are not responsible sharing of clinical trial data' in a manner that is consistent with 3 main principles:
	reproducible for external researchers. Re-analyzes of the few trials with available data showed safeguarding the privacy of patients, respecting the integrity of national regulatory systems, and
	good inferential reproducibility. maintaining incentives for investment in biomedical research (110). Despite this commitment
	Registration: https://osf.io/mcw3t/ from 2013, an audit found that data availability was reached for only 9/61 (15%) clinical trials
	on medicines sponsored by the pharmaceutical industry and first published between 1 July 2015

  Transparency would be guaranteed, regardless of whether the drug is approved or not. Clinical trial registries, such as ClinicalTrials.gov, could evolve to support the uploading of all these documents. Prospective registration on these public registries is the norm for clinical trials. Moreover, they have evolved to also include trial results, and, thus, are in a privileged position to expand toward incorporating more comprehensive open science tools empowering Findability, Accessibility, Interoperability, and Reusability (FAIR) access to any study related data. Undoubtedly, such a radical push toward transparency in planning, conducting, and reporting research, if promoted by an influential national or transnational regulatory authority, would have profound consequences for the entire field of biomedicine.

Proteus : God from Greek mythology who rapidly metamorphosed himself to very different figures (https://doi.org/10.1016/j.jclinepi.2004.10.019)

Paradigm: Kuhn defines it as "universally recognized scientific achievements that, for a time, provide model problems and solutions for a community of researchers"

P-hacking: when researchers try out several statistical analyses or data eligibility specifications and selectively report those that produce significant results (35) 35.Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The Extent and Consequences of P-Hacking in Science. PLOS Biology. 2015;13(3):e1002106.

Publication bias: failure to publish the results of a study, most likely due to lacking strength of the study findings. This non-publication introduces a bias which impacts the ability to accurately synthesize and describe the evidence (36) 36.Richards GC, Onakpoya IJ. Reporting bias 2019 [Available from: https://catalogofbias.org/biases/reporting-biases/, 36.Ibid.

Matthew Effect derives its name from a verse in the New Testament (Matthew 25:29) which reads, "For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath," and roughly translates to, "Those who are successful are most likely to be given the special opportunities that lead to further success, and those who aren't successful are most likely to be deprived of them."[START_REF]Estimating the reproducibility of psychological science[END_REF] 300. Moss PG. The Mattew Effect 2019 [Available from: https://paulgmoss.com/2019/06/10/the-matthew-effect/. (

Remerciements

Aux membres du jury, je vous remercie pour l'honneur que vous me faites de juger mon travail de thèse.

Acknowledgements:

We would like to thank Chloé Rousseau from CHU Rennes for her statistical advice and Clémence Belvèze from the university library Rennes 1 for her help with the search algorithm.

Acknowledgments: This publication is based on research using data from data contributors Abbvie, that has been made available through Vivli, Inc. Vivli has not contributed to or approved, and is not in any way responsible for, the contents of this publication. This study, carried out under YODA Project #2021-4637, used data obtained from the Yale University Open Data Access Project, which has an agreement with Johnson & Johnson. The interpretation and reporting of research using these data are solely the responsibility of the authors and does not necessarily represent the official views of the Yale University Open Data Access Project or Johnson & Johnson. Damien Bergeat for providing the code for Figure 3. Anne Hespel and Frederic Rimattei for treating the data requests.

Funding: FN received funding from the French National Research Agency for the ReiTheR (Reproducibility in Therapeutic Research) project, that this article was part of. Project Number: ANR-17-CE36-0010. Funding: The project is funded by the Agence Nationale de la Recherche (reference number: ANR-17-CE-36-0010-01) Funding: The project is funded by the Agence Nationale de la Recherche (reference number: ANR-17-CE-36-0010-01). The sponsor had no role concerning preparation, review, or approval of the manuscript.

Availability of data and materials:

The web appendix, the data and code that supports the findings of this study will be available on our link to the Open Science Framework (https://osf.io/mcw3t/) once the article is published.

Furthermore, the pre-registered SAPs for the data analyses and a guide on how to demand IPD from the studies can be found under the link indicated above.

List of Abbreviations

or Portuguese, 4/ had an accessible online website and 5/ were not considered as "predatory" journals according to Beall's list (124).

Search strategy for journals

The ICJME website was consulted to copy the list of all ICMJE-member journals and all 4892 ICMJE-affiliated journals. In cases where an affiliated journal changed its name after registration on the ICMJE website (e.g. Cancer Immunity changed to Cancer Immunology research), we checked whether the new name was also listed on the ICMJE website. If this was the case, the journal was considered as non-eligible and was marked as "discontinued";

otherwise it was included.

All 4892 ICMJE-affiliated journals were assessed for eligibility in random order obtained using the statistical software R [START_REF] Ioannidis | Why Most Published Research Findings Are False[END_REF]. The results of the randomization can be found in the supplementary [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF]. The first 489 journals that met the selection criteria (10 % of affiliated journals) were included, enabling us to estimate a proportion of 50 % (the worst-case scenario for precision estimates) with a precision (boundaries of the 95 percent confidence interval) of about ± 4.5 %.

ICMJE-member journals

The characteristics of the 14 ICMJE-member journals are detailed in Table 1. hundred and ninety-seven RCTs published in member journals were found [START_REF] Ioannidis | Why Most Clinical Research Is Not Useful[END_REF]. We could only confirm for 20 articles that they had been submitted after 1 st of July 2018. For 6 articles without data-sharing statements in the NEJM and the Lancet, we were not sure if they were eligible with respect to the submission date. Authors were contacted and for two we were able to confirm that they had been submitted before 1 st of July 2019. These were replaced, as were the four others where doubt persisted. Among the affiliated journals 722 RCTs were identified and were randomly sorted and assessed for eligibility criteria. Figure1 details the selection process for both ICMJE-member and ICMJE-affiliated journals.

Data-sharing statement:

All data and all materials are available on the OSF website: https://osf.io/n6whd/.

Contributors:

MS, JG, AD, DM and FN conceived and designed the experiments. MS, JG, LC and HG performed the experiments. MS and FN analyzed the data. MS and FN interpreted the results.

MS wrote the first draft of the manuscript. DM and FN contributed to the writing of the manuscript. MS, JG, LC, HG, AD, DM and FN agreed with the results and conclusions of the manuscript. All authors have read them, and confirm that they meet the ICMJE criteria for authorship. All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. MS is the guarantor.

Competing interests:

Objectives

In this scoping review we explored the impact of data-sharing initiatives on the willingness to share data, the status of data-sharing, the use of shared data and the impact of research outputs from shared data.

METHODS

Protocol and registration

The study protocol was registered on the Open Science Framework on September the 12 th 2018 (registration number: osf.io/pb8cj). The protocol followed the methodology manual published by the Joanna Briggs Institute for scoping reviews.(147) Methods and results are reported using the PRISMA (Preferred Reporting Items for systematic Reviews and Meta Analyses) extension for scoping reviews (PRISMA-ScR).( 148)

Eligibility criteria

The following eligibility criteria for studies were used:

All study designs were eligible, including case studies, surveys, metrics and experimental studies, using qualitative or quantitative methods. Only published or unpublished reports (e.g.

pre-prints, congress presentations, non-indexed information such as websites) in English, German, French or Spanish were considered.

We included all studies and reports 1/ providing information on current IPD data-sharing practices for clinical trials and 2/ reporting on one or more of five outcome domains defined according to the data-sharing process presented in Box 1.

Information sources

The identification of studies was performed in two complementary stages: a) A systematic literature search in bibliographic databases (MEDLINE databases, Cochrane Library, Science Citation Index Expanded and Social Science Citation Index).

In (F1000Research)). These journals/publishers were targeted because they had either an early or a robust data-sharing policy (NEJM, Lancet and JAMA had no data-sharing policy before the 2018 ICMJE policy). Some funders (see Supplementary Material 1) were also contacted, and preprints repositories were explored (bioRxiv, PeerJ, Preprints.org, PsyArXiv and MedRxiv.

For the sake of completeness, ASAPbio (Accelerating Science and Publication in biology) and the Center for Open Science were also contacted for the same information, as well as three International Congress on Peer Review and Scientific Publication conference abstracts. In addition, when relevant references were found in various papers these references were included (snowballing searches).

Search

Changes to the initial protocol

We initially planned to contact leading authors in the field to ask whether they were aware of other unpublished initiatives, but this was not done as it was difficult to identify relevant authors. We found relevant references about data-sharing policies including both clinical trials and observational studies, without making a distinction. These references were included in the scoping review and this point was discussed in the text.

RESULTS

Selection of sources of evidence

A total of 3024 records were identified, 3,005 records (1991 + 1014 in the update) were retrieved by database search (2141 without duplicates). An additional 8 records were identified by screening the proceedings of the last three International Congress on Peer Review and Scientific Publication conference abstracts and ten records by snowballing searches. One additional relevant record was identified after screening 630 identified pre-prints. We screened all irrelevant records by title and abstract, leaving 409 possibly relevant references which were eligible for full-text screening. Subsequently, 316 references were excluded, leaving 93 reports that met the inclusion criteria (Figure 1). We inspected websites and when needed contacted (MRC UK)) provided some additional data (online metrics and or data about its policy) (Figure 1) which was extracted in June 2019 and updated in December 2020.

Trial participants

Qualitative studies

Perceptions of trial participants toward data-sharing and their intention to share were explored qualitatively. A systematic review with a thematic analysis of 9 qualitative studies from Africa, Asia, and North America identified four key themes emerging among patients: the benefits of data sharing (including benefit to participants or immediate community, benefits to the public and benefits to science or research), fears and harm (including fear of exploitation, stigmatization or repercussions, alongside concerns about confidentiality and misuse of data), data-sharing processes (mostly consent to the process), and the relationship between participants and research (e.g. trust in different types of research or organizations, relationships with the original research team).( 171) Some qualitative reports provide data on heterogenous samples including patients and various stakeholders from low-and middle-income countries.

In-depth interviews and focus group discussions involving 48 participants in Vietnam suggested that trial participants could be more willing to be involved in data-sharing than trialists.( 172) A similar study on a range of relevant stakeholders in Thailand found that data-sharing was seen as something positive (e.g. a means to contribute to scientific progress, better use of resources, greater accountability, and more output) but it underlined considerable reservations, including potential harm to research participants, their communities, and the researchers themselves.( 173)

In a qualitative study with 16 in-depth interviews, cancer patients currently participating in a clinical trial indicated a general willingness to allow re-use of their clinical trial data and/or samples by the original research team, and supported a generally open approach to sharing data and/or samples with other research teams, but some would like to be informed in this case.( 174)

Despite divergent opinions about how patients prefer to be involved, ranging from passive contributors to those explicitly wanting more control, participants expressed positive opinions toward technical solutions that allow their preferences to be taken into account.

Surveys

Two surveys performed in the US and one in Italy assessed the intention-to-share rates among trial participants (see Figure 4). In one survey with a moderate response rate (47%), 463/799 (58%) patients favored or strongly favored data-sharing, while only 9% were against or strongly against it.(175) Most participants (84%) believed that disclosing the data-sharing plan

Experimental studies

The low data availability for IPD-meta-analyses is underlined by two experimental studies. One experimental study covered the issue of actual data-sharing. In this small randomized prospective study where 29 corresponding authors of original research articles in a medical journal were contacted via two different modes (general versus specific request), only one author actually sent the data immediately in response to a specific request and one author, without caveats, reported willingness to send the data in response to a general request.( 168)

A randomized controlled trial investigated the effect of financial incentives on IPD sharing.( 205) All study participants (129 in all) were asked to provide the IPD from their RCT.

Those allocated to the intervention group received financial incentives, those from the control group did not. The primary outcome was the proportion of authors who provided IPD. None of the authors shared their IPD, whichever the group.

Other data sources

Two studies investigated the completeness of data availability in IPD meta-analyses. Out of 30 IPD meta-analyses included in a survey,(206) 16 did not have all the IPD data requested. The access rate for retrieving IPD for use in IPD-meta-analyses was investigated in a systematic review.(200) Only 188 (25%) of 760 IPD meta-analyses retrieved 100% of the eligible IPDs for analysis and there was poor evidence that IPD retrieval rates improved over time.

Access to repositories/platforms

Only a few studies describe access to repositories/platforms from the viewpoint of the user (Figure 5). Experiences with two major platforms (CSDR, PDS) were reported.( 207) In these very early-phase projects, no data access was possible with CSDR, and faster data acquisition was achieved via the Project Data Sphere. High sharing rates were reported for academic repositories (MRC CTU, BioLINCC). Of 103 requests to MRC CTUs, access was granted in

Repositories/platforms

Commercial sponsors

Metrics of actual re-use

Different initiatives and platforms were initially implemented for the pharmaceutical and medical device industry to support sharing of IPD from clinical trials (these platforms are now open to academic trials but this has not been used very often so far). This covers the YODA project, CSDR, Vivli and SOAR (which is now part of Vivli). For the different platforms and repositories, metrics describing the actual use of data are available (Figure 5). Re-analysis of clinical trial data could help the scientific community to enhance the validity of reported trial results. An illustration is the "restoring study 329" initiative, investigating efficacy and harm of paroxetine and imipramine in the treatment of major depression in adolescence.

The re-analysis reached different conclusions with important implications for both clinical practice and research.( 113) RIAT (Restoring invisible & abandoned trials support center) was initiated as an international effort to tackle bias in the way research is reported with the goal of providing more accurate information to patients and other healthcare decision makers.( 233)

One of the problems that is tackled by RIAT is misreporting (inaccurately or incompletely reported trials). In our scoping review we found that re-analyses are only exceptionally applied.

In one review, the majority of studies was reproduced on all primary outcomes, in another around one third of studies led to changes in findings different from the original articles. It seems that re-analysis is only attractive in a minority of cases deserving major public interest.

Nevertheless, for these cases, repositories holding and sharing IPD could be very useful and speed up the process of data-sharing. It could be of interest to establish a link between RIAT and data-sharing platforms and initiatives.

Publications can be considered as the main output from data-sharing. Usually, there is a considerable time lag between requesting data for re-use, receiving shared data, performing secondary analysis, writing a manuscript and publishing the secondary analysis. This has to be taken into consideration when the publication output of data-sharing initiatives and platforms is analysed. Repositories and platforms mainly devoted to commercial trials have now existed for around 5 years, so only a limited publication output can be expected. Fortunately, these repositories provide detailed metrics for data-sharing requests, including number and type of publications originating from data-sharing. As expected, the number of publications related to Once accepted, the editors undertake to publish the completed study if the protocol is validated even if there are statistically negative findings (i.e. study hypothesis not verified). This approach is expected to reduce issues such as publication bias (246).

Eligibility criteria

EPARs

We collected all EPARs on new authorized human medications, biosimilars and orphan medicines given a positive opinion by the CHMP (Committee for Medicinal Products for Human Use) between 1st January 2017 and 31st December 2019 and approved by the European Commission. We excluded EPARs concerning generics and hybrid medicine. Definitions concerning the different types of drugs can be found in additional file 1 in the web appendix (245). The distinction between new biosimilars, new generics, new hybrid medicine, orphan medicines or new medicines followed the CHMP Meeting Highlights (247).

Main Studies

Pivotal trials are referred to as "main studies" in the different EPARs. Any main study was included, with no distinction in terms of study phase, study type, study design or intervention.

If an indication for a drug has been refused and another indication authorized, we did not consider the main study for the non-authorized indication.

Furthermore, studies with no primary outcome identified were not included and were listed as non-evaluable studies.

Search Strategy

Eligible main trials

Two reviewers (MS, JG) manually extracted all names of the new medicines, biosimilars and orphan medicines approved by the CHMP and enter the information on an Excel Sheet.

Afterwards, a check was performed to verify that the CHMP opinion was adopted by the European Commission (248). Next, the reviewers identified the corresponding eligible EPARs on the EMA website (249) and extracted all main studies reported in these EPARs. In case of disagreement, a third reviewer (CL or FN) arbitrated.

Sample size calculation

A random sample of 62 of these main studies was selected using R (rnorm function) (250). This sample size ensured a precision of ± 12% to estimate our primary outcome (i.e. percentage of reproducible studies, see below for a definition) in the worst-case scenario for precision estimations (i.e. if the percentage is 50%).

Main study document accessibility

For all selected main studies, one reviewer (JG) searched for the EudraCT number and/or the Once the sponsor and the study number were identified, the reviewer contacted the sponsor to collect all of the following main study documents: 1) IPD, 2) data analysis plan, 3) unpublished and/or published study protocols with any date-stamped amendments 4) all the following dates: date of the last visit of the last patient, date of database lock (if available) and date of study unblinding, 5) unpublished and/or published (scientific article) study reports.

To this end the reviewer sent a standardised email (additional file 2), presenting the research project with a link to the pre-registered protocol on the Open Science Framework. In order to improve the return rate, up to 4 emails were sent, the original and 3 reminder emails (with a two-week interval between e-mails).

If we were asked for this information, we indicated that the Data-Sharing of raw data is welcome in form of Study Data Tabulation Model (SDTM) which was created by the Clinical Data International Standard Consortium (CDISC) (255).

In some cases, it was sufficient to contact the sponsor by e-mail, in other cases the sponsor asked us to retrieve the data on a data sharing platform.

In parallel the same reviewer searched these documents on the EMA portal ( 256) and by inspecting the published reports (if available) identified using open trial (257,258). This process is summarised in supplementary Figure 1 in the web appendix.

Data Extraction

The identification of main studies and the following trial characteristics were extracted on an Excel spreadsheet by two independent researchers (JG and FN).

These characteristics included patient characteristics (e.g. percentage of women, mean age of participants, paediatric indication), study design (e.g. endpoint type, description for each primary endpoint) and intervention characteristics (e.g. drug).

An exhaustive list of the trial characteristics extracted can be found in the additional file 3.

Concerning the re-analysis, a first reviewer (JG, PhD Student) collected the information and collated data for the re-analysis. More specifically, the reviewer prepared a dossier with the following information for each study: 1/ the protocol, 2/ all amendments to the protocol (with their dates), 3/ all the following dates: date of the last visit of the last patient, date of database lock (if available) and date of study unblinding, and 4/ the IPD. In case of information was still lacking, the study authors were contacted.

Strategy for re-analyzes:

In case the IPD were not available one year after our initial request, we initially planned to consider the study as non-reproducible (primary outcome of our study). However, we allowed some flexibility regarding deviations to this rule (in terms of delay) during the conduct of the study, since delays were in general longer than initially planned, including from the legal review on our side. We only considered studies as not reproducible when data was not shared entirely to reproduce the primary endpoint.

Based on the dossier prepared by the first researcher, re-analyzes of the primary outcome(s) of each study were performed by a second researcher (MS, PhD student) who had no access to study reports, journal publications, statistical analysis plan, or analytical code, in order to ensure that the analysis was as blind as possible to the primary analysis. In addition, this reviewer was instructed not to try to find these documents or the published report.

For single-blind studies or open-label studies, analyses were performed according to the first version of the protocol, because outcome switching were possible in these studies. For doubleblind studies, all re-analyzes were based on the latest version of the protocol issued before database lock and unblinding. If this information was not available, the date of the last visit of the last patient was used as a proxy.

Although in therapeutic research statistical analysis is fairly simple, in some cases the reanalyzes can involve difficult methodological choices. An independent senior statistician (AR) was available to discuss any difficult aspect or choice in the analysis plan before the re-analysis, so as to choose the most consensual analyses (e.g. Intention to Treat population for a superiority trial).

If insufficient information concerning the main analysis were provided in the protocol, the best practices for clinical research were used, following the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH Guidelines) ( 259).

An analysis plan was developed for each study included and was recorded on the Open Science Framework. In the supplementary material a table is provided with details of what was taken from the ICH guidelines in case of missing information (additional file 4).

Re-analyzes entailed the following different steps: 1/ identification of the primary outcome (and detection of outcome switching), 2/ definition of the study population, 3/ re-analysis of the primary outcome. Any change identified between the first version of the protocol and the version used for the re-analysis of the primary outcome was tracked and described.

Procedure to assess reproducibility

All results of these analyses were reported in terms of each study's 1/ conclusion (positive or negative), 2/ p-value, 3/ effect size (and details about the outcome), 4/ changes from the initial protocol.

These results were first compared with the results of the analyses reported in the EPARs and, if these were not available, with the study reports, and again if not available, with the publications. All results from all available documents were gathered (EPARs, study reports & publications) and were presented in the results section.

Because interpreting an RCT involves clinical expertise, and cannot be reduced to solely quantitative factors, an in-depth discussion between two researchers not involved in the reanalysis (JG and FN), based on both quantitative and qualitative (clinical judgment) factors enabled a decision on whether the changes in results described quantitatively could materialize into a change in conclusions.

If these two reviewers judged that the conclusions are the same, the study was considered as reproduced. If these two researchers judged that the conclusions were not the same, then the researcher in charge of the analysis (MS) was be given the statistical analysis plan of the study and was asked to list the differences in terms of analysis. If he found a discrepancy between the study data analysis plan and his own analysis plan, then he corrected this discrepancy in his analysis (e.g. analysis population, use of covariates). Again, an in-depth discussion between two researchers not involved in the re-analysis (JG and FN) enabled a decision on whether the changes in results described quantitatively could materialize into a change in conclusions, and whether the differences in terms of analytical plan were understandable and acceptable. If these two researchers judged that the conclusions were the same, the study was considered as reproduced with verification.

In case these two researchers judged that the conclusions were not the same or that the change in the analytical plan is neither justified nor desirable, we have foreseen to have a senior statistician performing his own re-analysis. Details on this step can be found in the protocol of the registered report (260). This process is described in supplementary Figure 2.

Data Analysis

We performed a descriptive analysis of the characteristics of the extracted main studies included in the EPARs selected. This included counts, percentages and their associated 95% confidence intervals (CIs).

Effect estimates in the different studies were expressed as standardised mean differences (SMDs) and their associated 95% confidence intervals. For binary outcomes, odds ratios and their 95% CIs were calculated and converted into the standardised mean difference (264).

In order to compare the results of our re-analyzes with the original results, the following steps were implemented: 1/ We compared the statistical significance in the form of the p-value. If different, the results were considered as not reproducible. If not different, 2/ we qualitatively compared effect sizes and their respective 95% CIs. In case of +/-0.10 points difference in point estimates (expressed as standardised mean differences), the difference was discussed with a clinician in order to assess its clinical significance.

All analyses were performed using the open source statistical software R (R Development Core

Team) ( 250) and SAS software™ . The code will be made public on the Open Science Framework, as well as a file summarizing the process to retrieve all data-sets (245).

Changes to the initial protocol:

We set a one-year deadline to obtain data. However, data demands were lengthy, and delays were in some cases produced from our side. Hence, study data that was sent after this date was included in the re-analysis process.

Furthermore, although we said we only will use R as software for data analysis, SAS software was used in two studies due to a more potent approach in mixed model analyses.

For one study we were unable to calculate the Odds Ratio. Starting with the Incidence Rate Ratio, we used Chinn conversion to receive the SMD (265) . This approach is justified in cases when events are rare and the Incidence Rate Ratio can be treated like an Odds Ratio.

Data availability

We received IPD for 10/62 studies (16.1% [95% CI 8%; 27.7%]) from six different sponsors.

(266-275) For these studies, the median number of days before data were available was 253

[Interquartile range 182 -469]. For these ten studies, all sponsors were pharmaceutical companies and all but one of those companies had a data sharing policy on their website. IPD of four studies were provided via data sharing platforms (one was provided by one sponsor on

Vivli and three by a single sponsor on YODA). Three studies were shared via a remote desktop that is monitored by the company in possession of the data. Another three data sets from three different sponsors were sent directly to us. All received IPD was analyzable and deidentified.

For the remaining 52 studies, reasons for unavailability were heterogeneous (Figure 1). The main reason was restriction due to the study status, i.e. extension studies were ongoing (13/52).

Other frequent reasons included confidentiality in 9/52 IPD or lack of scientific merit as judged by the companies' procedures. The existence of possible privacy concerns was put forward for one study as a reason for not sharing data. Of the 52 studies who did not share, 37 (71.2%) belonged to a company that had a data sharing policy (Figure 2).

Study characteristics

Characteristics of the ten studies with available IPD are presented in Table 1. Median sample size was 548 patients [Interquartile range 278 -778]. Three were a single-arm study, one was a two-arm study, four were three-arm and two were four-arm studies. Two involved a noninferiority design and for all ten studies the main articles, the study protocols and the EPARs were retrieved.

Reproducibility

For the ten trials with available IPD, we identified 27 distinct outcomes eligible for reanalyzes (regarding different comparisons and/or different primary endpoints). Detailed results of those reanalyzes are presented in Figure 3. Sixteen reanalyzes (in six studies) were considered as reproduced, seven reanalyzes (in five studies) were considered as reproduced with verification, and four reanalyzes (in two studies) were considered as not reproducible because data was not available for a specific outcome. In these two three-arm studies from the same sponsor, comparing Semaglutide with Sitagpliptine and Placebo, on two primary outcomes (Change in HbA1c and bodyweight), data concerning bodyweight was not available (deleted as part of the anonymization process).

The 52 studies without available data were considered as not reproducible. Therefore, regarding our primary outcome, conclusions of 8/62 trial (12.9% [95% CI 5.7%; 23.9%]) were reproduced (i.e. reproduced and reproduced with verification).

We found no selective reporting of the primary outcome and no change from the original study protocol regarding the primary outcome in these ten studies. We found no selective reporting of the primary outcome and no change from the original study protocol regarding the primary outcome in these ten studies.

Spin was observed in one study ( 273). This study of Esketamine used a hierarchical testing approach and in case the 84mg dose was not positive, testing of the 56 mg dose should not have

List of challenges

Delay in data retrieval

Data demands were long. Interactions with sponsors were, in some occasions, lengthy, especially if several ones were involved on the same data sharing platform. For example, on

Vivli we submitted a data request concerning six studies from three sponsors. Various questions from the different sponsors came up. In one study (NCT00927498), the ownership of data created confusion. The trial data was purchased by Pfizer. However, Vivli informed us that Pfizer was not in possession of the data and referred us back to the original Principal

Investigator of the study who did not have any rights over the data anymore. After clarification via the platform, access from Pfizer was denied for missing scientific merit.

In a final step for the data acquisition process, the legal service of our unit had to confirm the data agreement and this step from our side was also lengthy. For two data sets we exceeded our one-year limit for data retrieval by two weeks. For three additional studies that were requested on YODA, additional 134 days were calculated. In this case, data was not available at the time of our initial request (18/05/2020) but YODA contacted us on (04/03/2021) to indicate that data was now on the platform and could be asked. After a request, we received those data on (14/10/2021). Those studies were included in our analysis as those long delays were considered as minor deviations to our initial protocol.

Incomplete datasets, metadata and further clarifications

In five studies, we had to contact the sponsor/platform to require additional data as data necessary to re-analyze the primary outcomes was missing (in three Esketamine trials and in two Semaglutide trials). We received additional data after 28 days in the case of the Esketamine trials but our request regarding the Semaglutide trials are still pending (data demanded 21/12/2021), hampering reproduction of their outcomes. In addition, data dictionaries were available for 7/10 studies.

Data analysis

In 2/10 cases, while we reproduced the conclusion of the study, we did not define the same analysis population with respectively 303 and 434 vs 297 and 433 patients analyzed in studies Janssen SUSTAIN-1 and Merck Clarity. Among those two studies, the Esketamine study provided by Janssen had a very complex design involving randomized and non-randomized Union. Nonetheless, re-analyzes of the few trials with available data showed good inferential reproducibility.

Declarations:

Ethics approval and consent to participate: Not applicable Furthermore, more than half of those prolific authors were typically members of the journal's editorial board (298).

Their results show that scientific reviews have immense power, that some authors are favored, and that scientific methods are pushed into the background.

Dumas, who wrote her PhD thesis on reproducibility and the role of medical journals, came to a similar deduction. Her conclusion is that journals are a primary factor for the reproducibility crisis (299).

She takes the theory from Merton, who described the Matthew effect for publications 5 . A publication in a prestigious journal brings attention to certain researchers, making it easier to obtain funding for further research. This ends in publishing new results in these same prestigious journals. Furthermore, because of the high impact of the journal, their work is acknowledged by the media and, again, more attention is guaranteed.

Indeed, studies have shown that articles published in journals with a high impact factor are more cited (301,302) and that media coverage of these publications further increased their likelihood of being cited (303).