Patrick Veit-Haibach

Irène Buvat

Ken Herrmann

EJNNMI Supplement "Bringing AI and Radiomics to Nuclear Medicine"

This supplement to the European Journal Nuclear Medicine and Molecular

Imaging (EJNMMI) is dedicated to two of the recently most "hyped" topics in the modern imaging era: artificial intelligence (AI) and radiomics. There are probably not many examples in the history of modern imaging where congress participants have lined up in long queues for sessions about different aspects of AI in imaging and radiomics. Almost every day, we hear news of discoveries using AI algorithms for different applications, and almost as much news of success stories on how radiomics has improved prognostication in a variety of disease conditions, ranging from oncology to vascular, cardiac and brain imaging. Even Wall Street's attention is drawn to the terms "AI" and "medical imaging" (https://www.wsj.com/pro/artificialintelligence ). At the same time, worries and conjecture about the end of diagnostic imaging professionals have been increasing expressed (https://www.diagnosticimaging.com/blog/end-radiologists, https://www.economist.com/leaders/2018/06/07/ai-radiologyand-the-future-of-work ).

Thus, the guest editors and the EJNMMI editorial board are confident that this new supplement focuses on timely topics, both for the hybrid imaging community and for nuclear medicine in general. In fact, in medical imaging, nuclear medicine has the longest history and ongoing efforts towards quantification, standardization and harmonization [START_REF] Aide | EANM/EARL harmonization strategies in PET quantification: from daily practice to multicentre oncological studies[END_REF][START_REF] Kaalep | EANM/ EARL FDG-PET/CTaccreditation-summary results from the first 200 accredited imaging systems[END_REF], whereas those topics are often controversial and not well laid out in morphological and functional radiological imaging procedures (except maybe for CT). Therefore, hybrid and nuclear medicine imaging should certainly be in the driver's seat when it comes to experience in quantification and data-mining (and thus radiomics) applications, as radiomics is based precisely on the extraction and analysis of large numbers of quantitative features from medical images. On the other side, AI, although having been around for quite some decades, has only recently gained momentum in imaging sciences in general with the introduction of new/advanced algorithms, creating a big wave of new (and hopefully clinically useful) scientific efforts in imaging. Although AI can be defined as the ability of a machine to correctly interpret data, to learn from those data, and to use that learning to achieve Starting with Visvikis and colleagues, the first article gives an excellent overview of the basic terminology of AI, machine learning and radiomics [START_REF] Visvikis | Artificial intelligence, machine (deep) learning and radio(geno)mics-definitions and imaging applications[END_REF]. Specifically, different techniques and algorithms are described, along with several of their advantages and disadvantages, as well as some of the most significant (technical) applications. The article also highlights the technical challenges which still remain before further, true clinical integration can be achieved.

Zwanenburg et al. discuss the extremely important yet still underestimated topic of feature robustness, reproducibility and standardization/harmonization in radiomics [START_REF] Zwanenburg | Radiomics in nuclear medicine-robustness, reproducibility, standardisation and how to avoid data analysis traps and replication crisis[END_REF]. These are topics which have haunted the medical imaging community for decades, but huge improvements have now been made, especially in PET (e.g.

the Evaluation and Report Language [EARL] initiative).

The article by Sollini and co-workers provides a systematic analysis of the AI and radiomics literature [START_REF] Sollini | Towards clinical application of image mining: a systematic review on artificial intelligence and radiomics[END_REF]. The authors conclude that there are indeed very promising results available but-considering the large amount of partly spectacular news-that the entire topic might be not as ready as one might think for integration in clinical routine and specifically for phase II/III trials. This view is shared by a number of other authors in this supplement. In addition, the review by Castiglioni and her group of researchers details the entire pipeline for radiomics feature extraction, from acquisition optimization, postprocessing and correction methods to radiomics-based multiparametric decision models [START_REF] Castiglioni | AI based applications in hybrid imaging: how to build small and truly multi-parametric decision models for radiomics[END_REF]. This group, too, emphasizes the need for largescale standardization and harmonization.

Zaharchuk meanwhile focuses strictly on the applications and possibilities arising from deep learning [START_REF] Zaharchuk | Next generation research applications for hybrid PET/MR and PET/CT imaging using deep learning[END_REF]. His article additionally takes a different turn compared to others in this supplement, as he describes how this specific AI technology can improve image quality in almost every imaging modality and how certain imaging modalities can be modulated/synthesized. The latter offers highly original opportunities in research and multimodal imaging in general.

Another interesting take on the topic is provided by Roland Hustinx. In his article, he sheds some light on how the human factor-the physician-is affected by the technologies discussed here [START_REF] Hustinx | Physician centred imaging interpretation is dying out-why should I be a radiologist/nuclear medicine physician?[END_REF]. While he describes a number of "threats" to the radiologist/nuclear medicine physician and cites opinions in the literature which see imaging specialists doomed, he also dissects the realistic opportunities arising from AI technologies. Although he is certain that the integration of radiomics, and specifically AI, will change the landscape and workplace for the imaging specialist profoundly (at least in the long run), he also highlights how it can complement the physician's diagnostic and prognostic abilities.

A very special topic is discussed in the article by Goh and Cook [START_REF] Cook | What can artificial intelligence teach us about the molecular mechanisms underlying disease?[END_REF]. Some of the radiomics parameters are based on very (and maybe even overly) complex equations, which at times makes it difficult to understand how these imaging metrics relate to underlying molecular mechanisms of disease. The article outlines reasons why knowledge of the underlying biological and molecular mechanisms is still important for our overall clinical understanding.

Last but definitely not least is the manuscript from Holzinger, Haibe-Kains and Jurisica [START_REF] Holzinger | Why imaging data alone is not enough-AI-based integration of imaging, omics and clinical data[END_REF]. The authors make a compelling case as to why radiomics or imagingbased omics is not enough for a thorough disease characterization. Multiple other omics are being researched and have shown tremendous value in diagnosis, disease characterization and prognostication. It is time that we as imaging specialists are aware of this potential and try to integrate it, together with the radiomic features, into our clinical and diagnostic pathways.

It certainly remains to be seen how fast AI and radiomics in medical imaging will continue to grow and enter clinical practice. Because of the sheer dominance of these topics in recent years, AI and radiomics will make their way into routine applications at multiple levels of our clinical lives, including acquisition protocol and image quality optimization, workflow improvement, disease detection and classification, and probably reporting. Actually, AI is already operating in current scanners during the process of image acquisition, for instance to optimize patient position or make the most of the detected signal. By virtue of combining radiomics and AI/machine learning in medical imaging, these techniques will undoubtedly improve patient management and foster academic and clinical research interest in the years to come. We as the guest editors believe that AI and radiomics provide more opportunities than threats, and it is our joint responsibility to shape the future and translation of AI and radiomics in nuclear medicine. This requires the sharing of knowledge not only through publications, but also through contribution to databases, enabling collective learning from the large variety of images we work with. Maybe even more importantly, this will require passing around well-documented radiomic and AI models so that they can be tested independently by different groups to gain the level of evidence needed for clinical acceptance.

  specific tasks through flexible adaptation, sophisticated algorithms for the performance of certain tasks, even if not explicitly based on training from examples, are often presented as AI, making AI omnipresent in current medical imaging research. In this EJNMMI supplement, we bring together experience and opinions from explorers around the world and across continents. Several different topics concerning the terminology, the physics and instrumentation background, and possible clinical and research applications are discussed from multiple angles, showing the vast potential for AI and radiomics in hybrid imaging and nuclear medicine. All contributors are respected and experienced clinicians and researchers in their field, providing broad expertise on methodological and clinical applicability.

Acknowledgements

We would like to thank all authors and contributors and additionally would like to extend our gratitude to the reviewers for their help with this EJNMMI supplement.

Compliance with ethical standards

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

Ken Herrmann: personal fees from Bayer, other from Sofie Biosciences, personal fees from SIRTEX, other from ABX, personal fees from Adacap, personal fees from Curium, personal fees from Endocyte, grants and personal fees from BTG, personal fees from IPSEN, personal fees and nonfinancial support from Siemens Healthineers, and nonfinancial support from GE Healthcare, all outside the submitted work.

Patrick Veit-Haibach: personal fees, travel support and IIS grants from GE Healthcare, IIS grants and non-financial support from Siemens Healthineers, IIS grants and non-financial support from Bayer, ISS grants from Roche Pharmaceuticals. All support outside the submitted work.

Irène Buvat: none.


PhD Irène Buvat
email: irene.buvat@u-psud.fr

PhD Fanny Orlhac

The dark side of radiomics: on the paramount importance of publishing negative results

Keywords: Radiomics, Texture, PET, ethics

Over the past few years, we have witnessed an exponential increase of the number of radiomic-related publications. In the PET literature, a PubMed search using "(radiomic OR radiomics OR texture OR textural) AND PET" search criterion yielded 37 publications in 2015 to 110 in 2018. Interestingly, an extensive survey of these publications demonstrates that 94% reported positive or "promising" results involving some sophisticated radiomic features, with large variations in the performance supporting that conclusion. As few as 6% of them clearly concluded at negative results, including the paper by Saadani et al [START_REF] Saadani | Metabolic biomarker based BRAFV600 mutation association and prediction in melanoma[END_REF] published in that issue of the Journal of Nuclear Medicine. These numbers highlight a publication bias well acknowledged in many different research fields, with an on-going debate about the actual false discovery rate in the medical literature [START_REF] Ioannidis | Why most published research findings are false[END_REF][START_REF] Jagger | An estimate of the science-wise false discovery rate and application to the top medical literature[END_REF][START_REF] Chalkidou | False discovery rates in PET and CT studies with texture features: a systematic review[END_REF]. This publication bias has well understood roots. From the editors' standpoint, positive results are often thought to be much more exciting and valuable than negative ones and are more likely to be cited hence favorably increase the journal influence metrics (eg, 2-year impact factor) [START_REF] Duyx | Scientific citations favor positive results: a systematic review and meta-analysis[END_REF]. From the authors' perspective, positive results are more rewarding than negative findings and might better contribute to boosting their careers. As a consequence, negative radiomic results, the dark side of radiomics, currently remain mostly unpublished. Yet, publishing negative results is a must for ethical reasons. Criteria for publishing should be the quality of the study and its statistical power, whatever the outcome. A clinically or biologically relevant question and a methodologically well-designed study should warrant publication, no matter if the null hypothesis is rejected or not. In some instances, negative studies might be even more impactful than positive ones, as they may challenge existing paradigms and invite investigators to focus research efforts on different paths. The conclusion of Saadani et al paper [START_REF] Saadani | Metabolic biomarker based BRAFV600 mutation association and prediction in melanoma[END_REF] is that although BRAF mutation drives the MAPK pathway and glucose metabolism in some cancers, BRAFV600 status could not be successfully detected using radiomic features calculated from 18F-FDG PET/CT in melanoma patients. This observation should be an incentive to further explore the connection between the genetic mutations or pathway alterations and their macroscopic consequences detectable using our in vivo imaging devices. It is reasonable to expect that some biological dysfunctions resulting from mutations will produce a cascade of events that might ultimately yield a signal detectable by our exquisite molecular imaging scanners. However, both the magnitude and the spatio-temporal extent of the biological effect will determine our ability to identify an abnormality from in vivo images using a given radiotracer. Investigations of the relationships between the triggering signal (here a mutation) and the net observable result (here change in tumor glucose metabolism) are absolutely needed for two reasons: first to establish realistic expectations regarding the potential power of radiomic features, second to use radiomic observations as a driver to formulate more precise biological assumptions regarding the underlying processes and subsequently test them. Advancing that field will require the publication of both positive and negative radiomic results.

One could argue that given the overwhelming number of radiomic-related publications reporting positive results, which might be the trees that hide the forest, the publication of negative results in that domain will be practically challenging. What we need are methodologically sound, properly powered and robust radiomic studies addressing a biologically-driven hypothesis and described in such a way that independent investigators can reproduce and confirm the findings. Indeed, replication has often more scientific value than original discovery in the radiomic field. To date, to the best of our knowledge, no PET radiomic model has ever been validated by an independent group since 2014. Even worse, the misleading interpretation of some of the most famous radiomic findings [START_REF] Aerts | Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach[END_REF] has recently been demonstrated [START_REF] Welch | Vulnerabilities of radiomic signature development: The need for safeguards[END_REF], confirming that some radiomic features initially interpreted as biomarkers of tumor heterogeneity were actually surrogates of the tumor volume, as shown as early as 2014 [START_REF] Orlhac | Tumor texture analysis in 18F-FDG PET: relationships between texture parameters, histogram indices, standardized uptake values, metabolic volumes, and total lesion glycolysis[END_REF][START_REF] Buvat | Tumor texture analysis in PET: where do we stand?[END_REF]. These findings suggest that radiomic-related publications should be thoroughly designed with sufficient statistical power, described in such a way that they can be repeated, and that they should also include well-supported interpretation possibly based on dedicated experiments.

Among the methodological arsenal that can be used to chase spurious and confounding effects in radiomic studies, "sham" data and permutation/randomization tests could be used more often. "Sham" data can be obtained from healthy regions or by reshuffling samples in artificial groups or randomly changing voxel values [START_REF] Welch | Vulnerabilities of radiomic signature development: The need for safeguards[END_REF]. Such sham data are useful to check that the findings are present in the real data and not in the sham data. This is thanks to such sham data that the erroneous interpretation of the radiomic features highlighted in ( 6) could be nicely demonstrated [START_REF] Welch | Vulnerabilities of radiomic signature development: The need for safeguards[END_REF]. Randomization/permutation tests also yield a precise estimate of the distribution of a test statistic under the null hypothesis in a non-parametric setting by calculating all possible values of the test statistic under rearrangements of the labels on the observed data points. They are frequently used in genomic data analysis to assign a statistical significance to the findings [START_REF] Dudoit | Multiple hypothesis testing in microarray experiments[END_REF]. Last, statistical reviewing should be systematic as it is effective in improving the quality of published articles.

The validity and soundness of radiomic studies also depend on how the "vibration of effects" is handled. Vibration of effects means that results can differ over a broad range depending on how the analysis is performed. This is observed when many variations of analyses are used, for instance selecting different variables, making statistical adjustments, testing different statistical models, sorting the data differently or using different inclusion and exclusion criteria. The variability of results as a function of the chosen analysis strategy gives an indication about their robustness. Reporting only the analyses that yield a particular result, called the chrysalis effect [START_REF] O'boyle | The chrysalis effect: how ugly initial results metamorphosize into beautiful articles[END_REF], can yield a bias, while consistent conclusions across various methodological approaches, as reported in Saadani et al paper [START_REF] Saadani | Metabolic biomarker based BRAFV600 mutation association and prediction in melanoma[END_REF], provide a persuasive indication of the correctness of the findings.

Radiomic results should be interpreted with great circumspection. Radiomic observations are easy to report given the wide availability of software enabling the calculation of radiomic features and of packages supporting sophisticated statistical analysis and data mining. Yet, radiomics should now go beyond reporting associations and designing predictive models. It should explain what the observed patterns and associations mean from a biological point of view, so as to enhance our understanding of the mechanisms. Dedicated experiments and multi-omics approaches are needed to fully take advantage of the radiomic approach, so that the radiomic phenotype and associated models can be additional exploratory tools to untangle the complexity of the biological processes and of their macroscopic repercussions.

A more stringent selection of radiomic studies should thus be based on tougher editorial standards. More skepticism on the part of referees is needed to avoid the inflation of false positive findings or overzealous interpretation of radiomic results. In that respect, the five aspects discussed above might be systematically considered: biological rationale justifying the study, ability to independently replicate and reproduce the findings, comprehensive control of false discoveries validated by systematic statistical reviewing, investigation of the vibration of the effects, proofs supporting the interpretation of the results. Careful selection of publishable manuscripts based on these criteria will then leave room for publishing radiomic studies yielding negative results and confirmatory radiomic studies, which are both absolutely needed. Dedicated sections in top-tier journals could actually be devoted to important negative and confirmatory studies. Still, it will remain virtually impossible to publish all negative radiomic findings that would be useful to avoid unnecessary and costly repeats of experiments already done. Several practical solutions can be suggested. First, studies could be registered before they start in international registries (radiomic trial registries), so that it would be easier to determine whether a given question has already been or is being addressed by any investigator before investing in that research. Second, negative results could be made accessible in public repositories, such as arxiv.org, and referred to in review papers to facilitate their identification. Last, funding agencies should request the registration of the funded radiomic studies and the access to the results including the negative findings. All these actions together could considerably reduce the number of weekly contributive radiomic studies, clarify the state of the art, balance positive against negative radiomic results, and contribute to a proper identification of radiomic models that are useful either for patient care or for advancing our knowledge regarding how in vivo imaging can probe microscopic biological mechanisms. In addition, establishing and enforcing best practices should help us get continuous support from external stakeholders that provide funding to advance sustainable research in radiomics.

FINANCIAL DISCLOSURE

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.