Ms Myriam Edjali-Goujon

Mr Mr Vincent Edouard Lebon

Mr Loic Duch- Esnay

Feuvret

Guillaume Alexandre Carré

Myriam Klausner

Marvin Edjlali

Jade Lerousseau

Roger Briend- Diop

Samy Sun

Sylvain Ammari

Emilie Alvarez Reuzé

Théo Andres

Stéphane Estienne

Enzo Niyoteka

Maria Battistella

Frédéric Vakalopoulou

Nikos Dhermain

Eric Paragios

Catherine Deutsch

Johan Oppenheim

Charlotte Robert Pallud

Sylvain Reuzé

Jade Briend-Diop

Myriam Edjlali

Johan Pallud

Catherine Oppenheim

Frédéric Dhermain

Alexandre Carré

Enzo Battistella

Stephane Niyoteka

Eric Deutsch

Alix De Causans

Alexandre Roux

Arnault Tauziède-Espariat

Samy Ammari

Edouard Dezamis

Frederic Dhermain

Samir Klausner

Théo Achkar

Théophraste Estienne

Angela Henry

Roger Rouyar

Grégoire Sun

Frédéric Fournier

Eric Dhermain

Charlotte Deutsch

Robert

ADC Apparent Diffusion Coefficient AFT Accelerated Failure Time AI Artificial Intelligence AIC Akaike Information Criterion ALT Alternative Lengthening of Telomeres ANOVA ANalysis Of VAriance ANTs Advanced Normalization Tools ASL Arterial Spin Labeling AUC Area Under the Curve B BA Balanced Accuracy

Keywords: Picture Archiving and processing, radiomics extraction, machine learning, experiments management, and radiomics extraction, machine processing, radiomics extraction, machine learning, experiments management, and data Rad., Radiomics (or handcrafted), Clin., Clinical, NP, used but Not Provided DTI, Diffusion Tensor Imaging, DSC, Dynamic Susceptibility Contrast-Enhanced CNNs, Convolutional Neural Networks, SVM, Support Vector Machine CV, Cross-Validation, LOOCV, Leave-One-Out Cross-Validation, OOB, Out-Of-Bag

My thesis project is a collaboration between different professions, especially radiologists, radiation therapists, computer engineers and researchers. I deeply thank each of them for contributing their expertise to this project and contributing to its success.

Finally, I would like to thank all the members of INSERM U1030 -RaMo-IT, my lab

3.2

Representation of radiomic feature types . . . . . . . . . . . . . . . . . . . . . Edjlali and Charlotte Robert). We would like to acknowledge the participation of the five physicians who were willing to test their performance against the machine learning model. Alix de Causans, and myself wrote the paper. All co-authors reviewed the paper. The era of data-driven precision medicine for cancer . . . . . .

The era of data-driven precision medicine for cancer

Around 400 B.C., Hippocrate compared cancer to a crab ("karkinos" in Greek) as it is characterized by a rounded lesion surrounded by extensions similar to the legs of a crab. Nowadays, in a simple definition, the word "cancer" refers to "a group of diseases involving abnormal cell growth with the potential to invade or spread to other parts of the body [1]." According to current knowledge, all cancers are caused by a combination of environmental and genetic factors [2]. These causal factors may act together, or in sequence, to initiate or promote carcinogenesis.

Carcinogenesis is a term that refers to the underlying factors that cause cancer. Among the models of carcinogenesis that have been proposed, two have been widely cited [3][4][5]. The model developed by Vogelstein and Kinzler stresses that cancer is a genetic disease, i.e., characterized by damaged DNA as a result of a sequence of alterations and leading to the transformation of normal cells into malignant ones [3]. Hanahan and Weinberg's model focuses on the processes occurring at the cellular level that result in malignant tumor development [4,5]. Sustaining proliferative signals, evading growth suppressors, avoiding immune destruction, enabling replicative immortality, tumor-promoting inflammation, activating invasion & metastasis, inducing angiogenesis, genome instability & mutation, resisting cell death, and reprogramming of energy metabolism are hallmarks of cancer in this model. These characteristics confer to malignant tumors important properties in their capacity of invasion and dissemination. This

1 Introduction
hypothesis is based on the dual assumption that carcinogenesis is triggered by abnormal tissue organization and that all cells are naturally proliferative.

The diagnosis of cancer can be accomplished through complementary approaches such as physical examination, laboratory tests, imaging tests, and biopsy. In most situations, a biopsy is the only way to diagnose cancer definitively. A biopsy will provide a biological characterization of the tumor lesion. Once the diagnosis of cancer is confirmed, the extent (stage) of the cancer is determined. The cancer staging allows to determine the treatment options and to provide a survival prognosis assessment. To achieve staging, the "TNM" classification is mainly used in solid tumors. The system is based on the assessment of the extent of the primary tumor (T), the absence or presence of regional lymph nodes (N) and the absence or presence of distant metastases (M) [6].

To treat cancer, various treatments and medicines are available, and many more are currently being researched. The term "local" treatment refers to therapies applied to a particular tumor or region of the body, such as surgery and radiation therapy. Therapies using drugs (such as chemotherapy, immunotherapy, hormone therapy, or targeted therapy) are often referred to as "systemic" treatments because they have the potential to impact the whole body. These various therapies may be administered alone or in combination at different periods. However, the combination of classical techniques with new advanced technologies can constitute a significant but sometimes very complex therapeutic arsenal: it is difficult to judge the best method given the multidisciplinary nature that cancer treatment requires.

Over the past few decades, oncology has made remarkable strides, whether in identifying tumor pathophysiological processes, developing methods to aid in diagnosis, assessing disease extension, evaluating response to therapy, or developing novel therapies. Leveraging new and improved cancer screening practices, earlier detection of disease, more specific and precise staging, and the addition of a greater number of therapeutic treatment options have all had a profound impact on the rate of deaths for the most common cancers. Nonetheless, the mortality rate remains high for many cancers. The number of new cancer cases and deaths in 2020 was estimated at 19.3 million and 10.0 million, respectively, based on the data collected by the International Agency for Research on Cancer (IARC) [7]. Due to the increase in the world's population and the aging of people, IARC predicts that approximately 28.9 million new cancer cases and 16.3 million cancer deaths will be diagnosed each year by 2040 (Fig. 1.1) [8]. Cancer has thus emerged as a critical public health problem, which further justifies and necessitates continuing studies aiming at better understanding the tumor pathological processes as well as developing treatments that are more targeted [7][8][9]. For instance, in an attempt to address this concern, France has adopted a ten-year cancer strategy in 2021 with ambitious objectives.

This strategy notably aims to significantly improve the survival rate of cancers with a poor prognosis and minimize the side effects of treatments [10].

Oncology therapies are undoubtedly moving towards personalized medicine, also known as

1.
1 The era of data-driven precision medicine for cancer precision medicine, intending to characterize each disease phenotype as precisely as possible and allow for the targeting of biological abnormalities unique to each organism. Precision medicine is the polar opposite of the "one-size-fits-all" approach (i.e., the same treatment for all patients), in which treatment or preventive disease methods are based on the model of the average person, with only a minimal consideration for each individual. Panomics brings together multiple "omes" such as the understanding of genome structure ("genomics"), DNA methylation landscapes ("epigenomics"), gene expression ("transcriptomics") and protein expression ("proteomics") [11][12][13]. A broader definition integrates data derived from imaging features, i.e., "Radiomics [14]." All this high-dimensional data has turned cancer management into the "big data" era. In parallel with advances in high-throughput biology, applications of artificial intelligence to biomedical data are booming. This integration of multi-omics data analysis and machine learning has a promising future in discovering new biomarkers [15][16][17].

Imaging data are of paramount importance because they can allow the decoding of tumor phenotypes non-invasively. Thus, they have made possible the emergence of this new field of radiomics, which is of interest throughout this manuscript.

1 Introduction

Decoding tumor phenotype by noninvasive imaging

Due to the complicated nature of many cancers, particularly in situations where a patient shows clinical symptoms and the disease is not apparent, imaging modalities with a threedimensional (3-D) capability such as CT, MRI, and PET have emerged as crucial elements in patient management (see details about medical imaging in 2.2). Imaging is present in almost every step of patient care, including diagnosis, tumor grading, treatment planning, treatment administration, response monitoring, and patient follow-up. Depending on the modality, it can allow non-invasive and repeatable anatomical, functional, or metabolic characterization of the lesions (Figure 1.2). This section discusses the importance of medical imaging in decoding tumor phenotypes via in particular the quantification of intratumoral heterogeneity and introduces radiomics with its application in neuro-oncolongy.

Figure 1.2: Multilevel imaging: anatomical, functional, and molecular imaging. Reprinted from [18] under the Creative Commons Public Domain Mark 1.0.

Intratumoral heterogeneity quantification

The notion of tumor heterogeneity is a multi-scale concept: between tumors of different patients, between a primary tumor and its metastases, within the cells of a tumor, and over time [19]. Figure 1.3 illustrates the different forms of tumor heterogeneity.

Malignant tumors should not be regarded as a homogeneous entity composed of similar tu- mor cells, but rather as a collection of tumor and non-tumor cells, including subclones with their own panel of mutations, partly common and partly proper, in a dynamical system characterized by a rapid evolution [20][21][22][23][24]. There may be temporal variation in terms of morphology, immunophenotypic (protein expression), genomic [25,26], and epigenetic markers [25] within each tumor. Two non-exclusive models are used to explain intratumoral heterogeneity: the cancer stem cell model and the clonal evolution model [27]. The cancer stem cell model suggests a hierarchical organization within tumors where only a small subset of cells defined as cancer stem cells can both self-renew and produce differentiated daughter cells. This source of multiple differentiation potentials maintains intra-tumor heterogeneity. The clonal evolution model follows Darwinian selection mechanisms inducing that there is no hierarchy between different clones, and all cells can contribute equally to the growth of the tumor. Thus, different clonal cells of the tumor give rise to daughter cells that acquire mutations as they multiply.

This leads to a gradual selection of the most aggressive clone capable of surviving in the tumor environment, maintaining intratumoral heterogeneity.

Numerous studies spanning a broad spectrum of cancer types indicate that intratumoral heterogeneity promotes cancer growth, increases treatment resistance, and is pivotal for patient survival [28][29][30][31][32][33]. Therefore, comprehensive characterization of tumor heterogeneity is essential

to optimize cancer management. When the tumor location is identified, biopsies are performed prior to the start of treatment to collect tissue samples, which can provide molecular, biochemical, and even genomic information. Due to the invasive nature of the procedure, it does not allow for complete characterization of the tumor. Indeed, despite the excellent resolution of biopsies at a cellular level, the tumor description is therefore portrayed at the sampling point, and intratumor heterogeneity description may underestimate the tumor genomics landscape [21]. Radiomics circumvents this limitation through the use of medical imaging.

Definition, objectives of radiomics and the trend towards deep learning

Traditional imaging-based evaluation of tumors relies upon qualitative features which are subjective to interobserver variability, such as the distribution of grey level intensities (heterogeneous, homogeneous) within the tumor lesion and the analysis of boundaries and contours (regular, blurred, spiculated...). Quantitative approaches are used for follow-up over time under treatment or for prognostic purposes. To perform this assessment, unidimensional (largest tumor diameter for RECIST criteria) or bidimensional (RANO for the glioblastoma follow-up) measurements are performed on the lesions. These qualitative phenotypic descriptions are generally referred to as "semantic" features. In a complementary fashion, emerging methods such as "radiomic" analysis allow algorithms to digitally decode medical images into quantitative features that can describe the intensity, texture or shape of a tumor. The fundamental assumption of radiomics is that the extracted tumor imaging features underlie the gene expression patterns [34]. Thus, the imaging data are a non-invasive method able to reflect the biological and pathological phenomena of tumors, avoiding biopsy sampling error through a comprehensive evaluation of the three-dimensional tumor bulk.

Radiomics is a sophisticated quantitative medical image analysis technique that was initially developed to decipher the genetic activity of tumors [35,36] and has since been applied to a wide variety of pathologies and imaging modalities. The term Radiomics was pioneered by Gillies et al. in 2010 and then reintroduced by Lambin et al. in 2012 [18, 34]. As shown in Figure 1.4, its use in the scientific community has grown exponentially.

Although there is no agreement on an exact definition, one proposal is as follows:

"Radiomics" designates the computer translation of medical images into high-dimensional objective quantitative data to determine, through subsequent analysis, typically with a machine learning technique, imaging biomarkers that can be used to support clinical decisions.

In recent years, numerous studies have demonstrated the virtues of radiomics in the personalization of patient management in oncology [37,38], particularly concerning diagnosis, classification of tumor subtypes based on molecular profile [34,39], characterization of tumor stages [40][41][42][43], and prediction of response to treatment [43][44][45]. Radiomics, which may be used in conjunction with or as a supplement to traditional biomarkers derived from biopsy One may consider that radiomics has thus become a sub-discipline of AI that deals with the extraction of image features through handcrafted features that correspond to the so-called standard radiomics or deep learning features, and among others, with the generation of prog-

nostic or predictive mathematical models [46] (details on radiomics modeling are given in Chapter 3). Deep learning, a subset of machine learning, involves a particular form of artificial neural network that mimics the human cognitive system. It has the advantage of using a data-driven approach and does not require prior features definition by human experts. Deep learning algorithms, such as convolutional neural networks (CNNs), excel in pattern recognition and discovering complex patterns in imaging data, enabling better performance due to the generated features' effectiveness. For these reasons, deep learning is extensively garnering attention and interest (Fig. 1.4).

Overwiew of radiomics and deep learning applications in neuro-oncology

Malignant brain tumors are classified into primary tumors that start in cells of the brain, represented mostly by gliomas, and metastatic or secondary brain tumors spread from another location (see details about brain tumors in section 2.1). So far, main brain tumor imaging techniques have included CT, multi-parametric MRI, and occasionally PET. Several review articles summarize the current workflow and methods used in feature-based radiomics in neurooncology and provide examples of their clinical applications [47][48][49]. This section discusses some of the work in this field (not an exhaustive summary) applied to gliomas and metastases (Fig. 1.6). Furthermore, we outline the current technical caveats for deploying radiomics based on MRI in a clinical radiation therapy workflow.

Clinical applications

The major clinical applications in neuro-oncology include differential diagnosis, tumor grading and molecular characterization, image segmentation, survival prediction, differentiation of tumor progression from pseudoprogression and local recurrence prediction.

Differential diagnosis

Radiomics has been used to establish challenging differential diagnoses, such as the difference between a single brain metastasis (BM) and a glioblastoma (GBM) in the case of a solitary contrast-enhanced tumor [50,51]. Indeed, these tumors can present similar clinical and imaging characteristics on conventional MRI. The performance showed an accuracy of around 85% for differentiation in the different studies [50,51]. One recent study used deep-learning with a 2D

CNN and showed an accuracy of 84% [52].

Tumor grading and molecular characterization

The world health organization (WHO) grade tumor classification system [53], which strongly relies on molecular markers of the tumor [54], namely isocitrate dehydrogenase (IDH) genotype [55][56][57], O 6 -methylguanine-DNA methyltransferase (MGMT) promoter methylation status [58][59][60], 1p/19q co-deletion status [61][62][63], EGFR expression level [64], H3-K27M status Figure 1.6: Imaging-based radiomics and deep learning tasks in neuro-oncology. Deep learning can be applied to automated tumor segmentation to track tumor volumetry and pattern recognition to conduct various end-to-end classification tasks. Radiomics approaches using engineered features and machine learning-based feature selection have also been applied to radiogenomics classification tasks, differential diagnoses, and diagnosis of early tumor progression. Imaging phenotypes identified using deep learning and radiomics could ultimately be combined with clinical characteristics to assess prognosis and treatment response of individual patients. Reprinted from [48] under the Creative Commons Attribution-NonCommercial 4.0 International. [65], and ATRX mutation [66] (see details about molecular markers in 2.1.2) is currently used to classify tumors and to determine the best potential treatment radiation therapy options for patients with glioma, including concomitant and adjuvant chemotherapy. Tissue samples obtained through tumor resection or stereotactic biopsy are commonly used to diagnose a malignant glioma and determine molecular markers. However, approximately 15% of gliomas are non-resecable, and the risk of morbidity with stereotactic biopsies, particularly in the elderly, is significant [67,68]. Three studies sought to determine the WHO grade using radiomic analysis of features collected from conventional MRI imaging [69][70][71]. Apart from conventional MRI, four groups investigated the extraction of radiomic features from advanced MRI images [72][73][74][75], and one research additionally included PET imaging [76]. The performance of deep learning-based radiomics employing CNN from conventional and advanced MRI imaging was examined in one research [77]. In all of these studies, no single approach was found to be superior. However, WHO grade determination of newly diagnosed gliomas using machine learning approaches achieved an accuracy of around 90%. In terms of determining the molecular characteristics of the tumor, many studies have shown that they may also be deduced non-invasively 1 Introduction from images using machine learning [78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95].

Image segmentation

Gliomas cause various modifications to the surrounding tissue and tend to develop in distinctive patterns. Four sub-regions of the tumor are conventionally identified on the pre-operative images, called "labels" in image processing: necrotic core, enhancing tumor, non-enhancing tumor, and edema. Labeling (delineating) these labels quickly and accurately is critical in many aspects of neuro-oncology, including radiation therapy treatment, image-based monitoring (tumor volumetry and tumor response assessment), and unveiling prognostic factors through the use of radiomics or a deep-learning based approach [47][48][49]. Annually, the Multimodal Brain Tumor Image Segmentation Challenge (BraTS) is held to evaluate the state-of-the-art volumetric segmentation methods of brain tumors using conventionnal multiparametric MRI scans [96,97]. Today's most successful techniques are often built on CNNs [97], which achieve excellent segmentation performances evaluated through Dice similarity coefficients with about 90% of the voxels correctly classified, similar to the performance of trained clinicians [98].

These methods can then be used in radiotherapy planning to delineate gross tumor volume (GTV) and clinical target volume (CTV) [99,100]. Metastases differ from gliomas in that they have a weak infiltrative character, necessitating segmentation based primarily on T1weighted, contrast-enhanced MR images. However, these can be multiple, and delineation can become a laborious task, hence the usefulness of automatic segmentation for their detection and segmentation [101,102].

Survival Prediction

Due to the dismal prognosis of individuals with glioblastoma, the determination of their survival is of particular importance [103]. A bleak prognosis might be related to a strong intratumoral heterogeneity, with the patient showing varying degrees of resistance to therapy throughout time and space. To further highlight the potential of these prognosis models, models' performance including radiomic characteristics were compared to those relying on clinical characteristics alone [104,105]. One study showed that adding radiomic variables extracted from MRI imaging to clinical variables was associated with increasing the C-index metric from 0.64 to 0.70 [104]. A second study improved performance from 0.70 to 0.74 [105]. Additionally, models performed better when perfusion-weighted imaging or diffusion-weighted imaging were employed in conjunction with clinical variables [106][107][108][109]. For example, a combined model (C-index, 0.87) improved prognosis when added to clinical inputs (C-index, 0.72) [108]. Deep learning methods were also exploited in the BraTS challenge to predict survival in 3 different classes (short, medium, and long survivors). The winner of 2019 used a Fully Convolution Neural Network (FCNN) and showed moderate performances with an accuracy of 59% on the validation and 58% on the test [110]. However, this deep learning task is complex and seems prone to randomness and overfitting. Indeed, it happens that the best validation teams end up with lower performances on the test and vice versa.

Differentiation of tumor progression from pseudoprogression

Pseudoprogression is a treatment-related phenomenon shown on MRI of high-grade gliomas in 30% of patients [111]. Thus, an increase in gadolinium contrast on MRI may be misinterpreted as tumor recurrence whereas it is due to radiation-induced necrosis. Diffusion-and perfusionweighted MRI data were included in a radiomic model in one study, and the model performed better than other models based on only diffusion or perfusion MRI, with an area under the curve (AUC) of 0.85 in the test data set [112]. A few studies have applied deep learningbased radiomics to classify progressing lesions. Based upon diffusion weighted image and flair sequence, a CNN achieved an accuracy of 82% [113]. Another study combined standard radiomic features and deep learning features extracted with a CNN from multiparametric MRI (conventional and advanced MRI). Accuracy in the inter-institutional test cohort was 75% [114].

Local recurrence prediction

Recurrence, local in more than 80% of cases i.e., within a distance of 2 cm from the edge of the resection cavity, seems inevitable in patients with glioblastoma due to the very high intratumoral heterogeneity [115][116][117][118][119]. To account for the potential recurrence, the irradiation margins are relatively consequent, resulting in a large volume of the brain being irradiated. In addition, attempts to increase the radiation dose to the target volume more than the current protocol have not yielded favorable results in improving the prognosis of patients [120][121][122][123].

However, there is an untested hypothesis to date, which envisions that if the tumor pathway was predictable and there was a dose-response relationship for glioblastoma, then the ability to predict the specific site of recurrence would be of tremendous benefit. Indeed, regions with the highest risk of recurrence could benefit from a dose painting in radiation therapy and be targeted with higher radiation doses. To reach this aim, radiomics have been used to leverage. A study determined imaging biomarkers that delineate areas of tumor infiltration from conventional and advanced MRI imaging and predict early recurrence in peritumoral tissue with an AUC of 0.84 [124]. Another study developed a radiomic signature of infiltration in peritumoral edema to predict subsequent recurrence in glioblastoma with approximately 90% accuracy [125]. To our knowledge, no study using deep-learning is present in the literature.

Barriers to clinical implementation

Before reaching a clinical deployment, several challenges are encountered: (i) imaging and annotations data are limited, (ii) imaging and annotations data have different qualities levels, (iii) model development strategies need to be carefully crafted? and their interpretability explained.

Small data problem

Machine learning and deep learning models require large amounts of training data so that their performance can be generalized to a wide range of real-world scenarios. An example of this is the ImageNet dataset (which contains more than 14 million photos of natural environments), which has enabled significant advances in AI image processing [126]. Unfortunately, similar large-scale integrated public datasets do not exist in neuro-oncology. For example, in 2020, the largest dataset came from BraTS challenge and counted 369 patients in the training set (high grade and low grade gliomas combined) with multiparametric MRI and their associated annotations [96,97,127]. First, medical data in gliomas is present in lesser abundance, and second, medical data is protected by patient privacy and security rules, i.e., the general data protection regulation (GDPR) in the European Union, which makes it complicated to share.

Moreover, for use in supervised tasks, data requires annotation by medical experts. The cost and time required to annotate the data can hinder it [128]. However, in recent decades, there has been a movement towards the "open source" availability of imaging data, notably through data collections established by the Cancer Imaging Archive (TCIA) [129], including Ivy Gap [130], CPTAC [START_REF] Charron | Image analysis in patients with cancer studied with a combined PET and CT scanner[END_REF], to provide clinicians and researchers with large-scale imaging data of brain tumors.

Preprocessing and feature extraction inconsistencies

There are numerous opportunities for potential errors or inconsistencies to manifest themselves in the extracted features due to differences in scanners and acquisition protocols, impacting the generalizability of models at a multicentric level. Futhermore, image processing steps such as co-registration, spatial resampling, segmentation, intensity normalization, and noise reduction, which are all reflected in the extracted features can be a source of inconsistencies (details in preprocessing step are given in section 3.1.3). When different segmentation techniques are used, or when inter-expert variability is present, small variances in voxel inclusion may result in orders of magnitude discrepancies in the estimated radiomic feature values [132,133]. Aside from that, automated segmentation is prone to errors, particularly in the presence of artifacts, a low signal to background ratio, noise, and/or when the lesions of interest are very heterogeneous [46]. In addition to segmentation, the differences between techniques for intensity processing and noise reduction significantly influence the reliability of radiomic parameters derived from glioblastoma patients [134,[START_REF] Moradmand | Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma[END_REF]. In glioma studies, it is only recently that experiments to better understand the possible effect of preprocessing on radiomic feature extraction have begun to emerge [134][START_REF] Moradmand | Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma[END_REF][START_REF] Tixier | Reliability of tumor segmentation in glioblastoma: Impact on the robustness of MRI-radiomic features[END_REF]. Inter-study heterogeneity in reported findings will continue since defined universal image processing pipelines have not been established. Though, a consensus 1.3 Thesis hypotheses, objectives and outline is beginning to form around image processing best practices [START_REF] Moradmand | Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma[END_REF]. Guidelines in practice may make it easier to replicate clinical results. Ensuring reproducible research means encouraging the community to come up with defined standards that specify all processing steps, including functions, imaging libraries, parameters, and so on, and publicly releasing the source codes.

Model building and interpretation pitfalls

Examining, understanding, and comprehending new datasets should lead to clinically meaningful decision tools. However, the development of a model must follow a precise methodology with ideally training and validation on a large multicentric dataset and testing on an unseen (prospective/external) large multicentric dataset. According to a recent study reporting the performance of artificial intelligence algorithms focused on diagnostic analysis of medical images, only 6% of 516 eligible published studies met the stringent criteria of the use of an external validation [START_REF] Kim | Design Characteristics of Studies Reporting the Performance of Artificial Intelligence Algorithms for Diagnostic Analysis of Medical Images: Results from Recently Published Papers[END_REF]. False correlations may be readily drawn because of the capability of AI in the processing of complicated multidimensional data. When models are highly dimensioned, they tend to "memorize" a particular combination of parameters with which the data being used for training is connected, known as the overfitting issue. In this case, an AI-trained model only learns to handle the training set's prediction problem, but it will fail to anticipate future observations and so generalize effectively. One other stumbling block is the interpretability of machine learning models, commonly known as the Explainable AI Problem (XAI), which refers to opening up black-box models and explaining how the model generates predictions in a way that humans can comprehend [START_REF] Gordon | Explainable Artificial Intelligence for Safe Intraoperative Decision Support[END_REF]. When using artificial intelligence methods in clinical contexts, the interpretability of models is especially essential.

Using a large set of representative samples for training, cross-validation techniques, selection of the most significant features, or integration of domain knowledge and clinically inspired imaging biomarkers help to avoid these pitfalls. A number of recent articles have described useful methods for modeling and evaluating radiomics models [START_REF] Lambin | Radiomics: The bridge between medical imaging and personalized medicine[END_REF][START_REF] Zwanenburg | Radiomics in nuclear medicine: Robustness, reproducibility, standardization, and how to avoid data analysis traps and replication crisis[END_REF][START_REF] Mongan | Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers[END_REF].

Thesis hypotheses, objectives and outline

This thesis focused on different applications of radiomics in brain tumor imaging. Our main primary assumptions were as follow:

• Hypothesis 1 : The human eye has limits, and even when trained to interpret medical images, it under-exploits the data they contained, especially in anatomical MRI.

• Hypothesis 2 : Optimization of specific steps performed as part of the radiomic workflow can improve the robustness of features extracted from the MR images and the predictive and generalizability properties.

• Hypothesis 3 : A tumor segmentation performed automatically can be as qualitative as the one performed by a trained physician, or even better.

• Hypothesis 4 : Using postoperative imaging data coupled with clinical data to predict survival in glioblastoma patients may offer useful prognostic information.

• Hypothesis 5 : Areas of recurrence in glioblastoma patients can be predicted upstream at the voxel scale in anatomical postoperative MRI and could contribute to more targeted therapies such as dose painting in RT and improve the local response.

This work has 7 main objectives focused on anatomical MRI imaging, all of which are in line with the goal of using AI to support clinicians in adapting cancer treatment to every patients and moving toward precision medicine:

• Objective 1 : To determine the impact of image preprocessing, specifically the standardization of images and the grey level discretization step used in radiomics (Chapter 4).

• Objective 2 : To propose a new method of MRI harmonization adapted from ComBat, a method derived from the genomics application but limited for radiomics use due to the need for appropriate statistics and labels corresponding to the "batch" effect (Chapter 5).

• Objective 3 : To develop a model capable of assisting in the differential diagnosis between brain metastases and glioblastomas in case of a solitary lesion based only on T1 MRI with contrast injection (Chapter 6).

• Objective 4 : To develop an automatic brain tumor segmentation tool (Chapter 7).

• Objective 5 : To create the largest cohort ever seen in the literature gathering glioblastoma patients and composed of anatomical MRI imaging with their respective annotations at two time points, i.e., at postoperative time and at recurrence time (Chapter 8).

• This thesis is structured as a collection of eleven chapters. Since this is a paper-based thesis, each chapter is written in a self-contained fashion, and some concepts and references can overlap between chapters. The different chapters have been structured into parts by theme.

The following summarizes the organization of the thesis:

• Chapter 1: Introduction to the concepts of "precision medicine," "radiomics," and "machine learning" in neuro-oncology from a clinical standpoint. -Chapter 5: This second paper on the thematic compares a so-called "optimal" preprocessing to a post-processing correction method, named ComBat, which we have overcome the difficulties of applicability to machine learning.

• -Chapter 8: The first paper answers the need for data in postoperative glioblastoma.

It describes the cohort, including the associated clinical data, the preprocessing steps applied to the images, the label annotation methodology, and the extracted radiomic features.

-Chapter 9: The second paper is the development of an overall survival signature including patient cohort described in the previous chapter, based on the extracted radiomic features and clinical data.

-Chapter 10: The third paper seeks to determine through deep-learning the areas of recurrence in patients with glioblastoma.

• Chapter 11: This chapter summarises the results and perspectives of the thesis.

• Appendix A: List of publications

Brain tumors

This section reminds the structure of the brain before focusing on brain tumors and more specifically in the context of this thesis on GBMs and BMs. According to IARC, 308,102 new cases of brain or nervous-system cancer were registered worldwide in 2020, leading to more than three cases per 100,000 people [1]. The vast majority of those cancers -approximately 80% -are gliomas, the most widespread and lethal of which is GBM.

Structure

Gross anatomy

On average, the adult human brain weights between 1.2 and 1.4 kg and constitutes roughly 2% of the total body weight [2,3]. Although there is considerable individual variation, it occupies a volume of roughly 1260 cm 3 in males and 1130 cm 3 in women [4]. The brain is composed of 3 main structural divisions (Fig. 2.1A): 33 1) Cerebrum, or cortex, is the largest component of the brain and linked to three key functions: sensory, cognitive, and motor. The cerebrum is composed of right and left hemispheres and is also divided into 4 major lobes according to their overlying neurocranial bones: the frontal, parietal, occipital and temporal lobes (Fig. 2.1B) [5].

2) Brainstem is located at the base of the brain and at the apex of the spinal cord. It links the brain to the spinal cord and the cerebellum. It serves as a control center for many involuntary functions, such as heartbeat, respiration, blood pressure, and several other reflexes [6].

3) Cerebellum is a major component of the hindbrain, located near the brainstem. It plays an important role in motor control and is also involved, to a lesser extent, in certain cognitive functions, such as attention, language and the regulation of fear and pleasure reactions [7,8].

Microanatomy

The brain is made up of two distinct cell types. On the one hand, there are nerve cells, which transmit nerve signals, and glial cells, or non-neuronal cells, which protect and feed the nerve cells.

Nerve cells

These cells are also called neurons. Each neuron is endowed to receive information (through its dendrites), integrate it (in the soma), and transmit it (through its axon). The dendrites, which have many ramifications, receive information and deliver it to the cell's body (Figure 2.2). The body of the cell contains the nucleus and various organelles. The axon is a single extension that starts from the cell body and ends, in a terminal arborization that establishes contact with the target cells. The axon is surrounded by a myelin sheath which serves as insulation and facilitates the transmission of nerve impulses. The connection between two nerve cells, i.e., between an axon terminal and a dendrite, is called a synapse.

Glial cells

Unlike neuronal cells, glial cells do not transmit nerve impulses. There are three main types of glial cells: astrocytes, oligodendrocytes, and microglia (Figure 2.

White matter, grey matter and cerebrospinal fluid

Two distinct regions of the CNS (brain and spinal cord) can be identified: white matter and grey matter (Fig.

Glial tumors

Histological and molecular classification of glial tumors

Gliomas originate from glial cells, specifically astrocytes and/or oligodendrocytes (Figure 2.3).

They may be composed of one or both cell forms (mixed tumors, oligoastrocytes). Glial tumors rarely spread beyond the central nervous system.

In the previous WHO classification of the CNS, WHO 2007 [9], tumors were mostly based on a histology definition. Following significant developments in tumor genetics in recent years, the current 2016 WHO classification of CNS tumors is not only based on morphological criteria such as mitoses, microvascular proliferation, and necrosis but also incorporates molecular parameters, resulting in an "integrated diagnosis" in multiple strata [10]. The goal is to define homogeneous tumor subgroups in terms of prognosis and response to treatment since data from the literature show that molecular classification correlates better with clinical outcomes than histological characterization, especially for gliomas [11,12]. Finally, each tumor that has been characterized should be assigned to one of the classification groups. However, if molecular or immunohistochemical studies are non-contributory or impractical, or if a tumor examined does not fall into one of the well-defined categories, so-called "NOS" (not otherwise specified) diagnostic categories are retained in the classification. Table 2.1 shows the WHO 2016 classification of glial tumors. Gliomas are graded according to their growth rate and aggressiveness.

The WHO grading scale is from I to IV. Typically, slow-growing tumors are grade I or II, while fast-growing and more aggressive tumors are grade III or IV.

Molecular genetics of gliomas

A number of molecular markers were used until 2016 in the field of neuro-oncology as predictive markers of survival or response to certain treatments. Thus, in the WHO 2016 classification, these markers are used as criteria for distinguishing between the different groups. This subsection presents the different molecular markers used in the WHO 2016 classification.

IDH1 and IDH2 gene mutations

The mutational status of genes IDH1 and IDH2 encodes two isoforms of the enzyme isocitrate dehydrogenase (IDH) [12]. In order to produce alpha-ketoglutarate, these mutations cause hypermethylation. According to current knowledge, this tumor suppressor gene-inhibiting phenotype stimulates glioma growth.

These types of tumors are referred to as "IDH-mutant" tumors because they are both the product of IDH1 or IDH2 gene mutations. These mutations are present in nearly every grade II (87%) and III (83%) gliomas and in the glioblastomas that derive from it (85%) [13]. When IDH mutated status is identified, the prognosis is favorable for grades II, III, and IV gliomas [14].

1p/19q-codeletion

This is a codeletion of the chromosomal arms 1p and 19q, which corresponds to an unbalanced reciprocal translocation t(1;19)(q10;p10). This codeletion allows the determination of a subgroup of oligodendroglioma patients associated with better prognosis and better chemosensitivity to alkylating agents [15]. The presence of the 1p/19q-codeletion is necessary to define the tumor entity as oligodendroglioma.

ATRX and TERT promoter mutations

Telomere shortening with each cell division results in replicative senescence. Tumor cells maintain their telomere length abnormally, allowing unlimited proliferation. Two different mechanisms are involved in diffuse gliomas and are mutually exclusive: TERT promoter mutations (C228T and C250T) and ATRX mutations (Alpha-Thalassemia / mental Retardation syndrome X-linked) [16,17]. The TERT enzyme is part of the telomerase complex responsible for telomere extension. Two recurrent point mutations in the TERT promoter have been Entries in italics are considered to be provisional. In other words, the WHO working group thought that there was insufficient evidence to identify them as distinct disease entities at this time.

described (C228T and C250T) and increase TERT expression, leading to abnormal telomere lengthening. The ATRX protein belongs to a multi-protein complex involved in telomere maintenance. Mutations in ATRX have been identified in diffuse gliomas and are associated with the alternative lengthening of telomeres (ALT) phenotype, different from that of tumors mutated for the TERT promoter. ATRX protein expression is ubiquitous in normal cells.

Histone H3 gene mutations

These are recurrent mutations (K27M and G34R/V) affecting the H3F3A and HIST1H3B

genes encoding histones H3.3 and H3.1, which are mostly observed in pediatric diffuse gliomas of the midline, brainstem and thalamus [18,19]. These mutations are more seldom seen in adults but most often observed in patients under the age of 50.

MGMT promoter methylation

O 6 -methylguanine transferase (MGMT) is a DNA repair enzyme, particularly involved in repair of damages induced by alkylating agents such as temozolomide (TMZ). Methylation of the MGMT promoter results in decreased MGMT expression and repair of TMZ-induced lesions [20,21]. Methylated MGMT status is a predictive marker of improved response to alkylating agents used in chemotherapy and an independent favorable prognostic biomarker in GBM [22].

Glioblastoma alteration, "IDH-wildtype"

The most common alterations in wildtype IDH GBM are the combination of a gain of 7p and a loss of 10q (80%), and/or an amplification of EGFR gene. These wild-type genetic alterations are very heterogeneous and preferentially affect 3 pathways: receptor tyrosine kinase/Ras/PI(3)K (88%), p53 (87%) and retinoblastoma (77%) [23]. In adult GBMs, high-level genomic amplification (40 %) occurs in the EGFR gene, and may have predictive benefit to immunotherapy treatment [22]. The investigation of this molecular diversity is currently without clinical implications for the patient.

Biopsy, "the integrated diagnosis"

The "integrated diagnosis" is made on biopsy-exeresis fragments (open surgery) or on fineneedle microbiopsies after spotting [24]. These samples are accompanied by information describing the clinical context, i.e., the anatomical location of the lesion and its radiological aspects (notion of contrast heterogeneity and boundary analysis). According to the WHO 2016 classification, a diagnosis requires three steps [10] (Figure 2.5):

1) A morphological analysis of diffuse glioma cells according to histopathological criteria to guide the diagnostic process.

2) A molecular analysis, including at least the search for IDH mutations and 1p19q codeletion (nowadays essential for the diagnosis of oligodendroglioma).

3) The establishment of the grade of malignancy in relation to the histoprognostic criteria specific to each tumor subtype (mitosis, neoangiogenesis, necrosis) and rendering of the integrated diagnosis.

Glioblastoma

In WHO 2016, GBMs are part of the diffuse glioma family grouping all diffusely infiltrating gliomas (whether astrocytic or oligodendroglial) and referred to as a grade IV tumor (Fig.

2.5).

Epidemiology in adults

GBM in adults is the most frequently occurring primary malignant brain tumor accounted for 14.6% of all primary brain and other CNS tumors, 48.3% of primary malignant brain tumors, and 57.3% of all gliomas [26]. GBM is most prevalent between the ages of 45 and 75 with an average age of 60 years [27]. However, GBM can develop at any age, and is slightly more common in men than in women. According to the 2016 WHO classification, GBM can be divided into three groups:

1) GBM, IDH-wildtype, usually corresponds to primary or de novo GBM, developed in a patient who has no known history or histological evidence of underlying lower grade glial lesion (about 90% of cases). Wild-type IDH GBM predominates in patients over 55 years of age [28,29].

2) GBM, IDH-mutant (about 10% of cases), often corresponds to so-called secondary GBM developed from the malignant transformation of a diffuse low-grade glioma and appears preferentially in younger patients [28,29].

3) GBM, NOS, designates cases on which we do not have sufficient pathological, genetic, and clinical knowledge and which should, therefore, be the subject of future study before the classification can be further refined.

Pathophysiology

GBMs consist of a proliferation of malignant glial cells, with nuclear atypia, poorly differentiated, with high cell density, high mitotic activity, necrosis, and microvascular and endothelial proliferation [9,30,31].

A B

Survival and prognostic factors

The prognosis of GBM remains poor due to its rapid progression. Despite recent advances in neuro-oncology, the typical survival time is 12 to 15 months, with a low 5-year survival rate of 3 to 7% [26,27]. Numerous factors influence the prognosis of patients with GBM, including age, Karnofsky performance scale (KPS) status, tumor location, preoperative imaging characteristics of the tumor such as the degree of necrosis and enhancement, and extent of resection [32][33][34][35][36].

Several molecular markers have been identified in GBM as associated with a better prognosis, including MGMT status, IDH status, EFGR overexpression and amplification, mutation of the tumor suppressor and transcription factor p53 and genetic loss of chromosomes (10q deletion) [14,22,32,[36][37][38].

Treatment

The clinical pathway of IDH-wildtype GBM is shown in Figure 2.7.

Newly Diagnosed

Current conventional treatment of de novo GBM combines surgery, external radiation therapy and chemotherapy according to a protocol published by Stupp et al. [39]. When safe and practicable, surgical resection is the primary line of treatment. It enables histological diagnosis and tumor bulk decrease. Although total resection of the tumor at the macroscopic scale is desired, it remains challenging to perform depending on the tumor location. Microscopic complete resection is not attainable due to the infiltrating tumor cells' systematic distant extension. For IDH-wildtype GBM patients < 70 years of age and in good general and neurological condition or IDH-mutant GBM adults, the standard treatment is pursued by concurrent radiation ( 60Gy in 30 fractions) and chemotherapy with TMZ (75 mg/m 2 daily throughout radiation therapy, including weekends) and six cycles of maintenance TMZ (150-200 mg/m 2 , 5 days every 28 days). This protocol was associated with a significant increase in progression-free survival (PFS) (from a median of 5.0 to 6.9 months) and overall survival (OS) (from a median of 12.1 to 14.6 months) with the combined therapy relative to RT alone. In addition, this protocol increased the number of patients alive at 5 years from 1.9% to 10.4% [39]. For IDH-wildtype GBM patients with unfavorable prognostic factors (defined by age and/or KPS), hypofractionated radiotherapy (e.g., 40 Gy in 15 fractions) can be used, which is similar to 60 Gy in 30 fractions [40,41]. Chemotherapy alone with TMZ may be an option depending on the functional status of the patient and the methylation status of MGMT [42].

Recurrent

Recurrence appears to be unavoidable, and very few therapy alternatives are available as palliative care, repeated surgery, re-irradiation, systemic therapies, and combined modality therapy [43,44]. Treatment is determined by the patient's status (age, KPS, MGMT promoter methylation status) and pattern of disease progression [45].

Brain metastases

Previously, we discussed primary brain tumors, i.e., tumors that originate from the brain tissue or its immediate environment. We will now discuss metastatic brain tumors. These are secondary cerebral locations of a primary tumor (for example in the breast or lungs) from which malignant cells have migrated to the brain, usually through the bloodstream. Metastatic tumors are considered cancers and are malignant.

Epidemiology in adults

Brain metastases (BMs) are the most frequent intracranial tumors and account for more than half of all brain tumors [46,47]. They are ten times more common than primary brain tumors [48]. In addition, it is estimated that approximately 20% of patients with cancer will develop BMs [49]. The BM may be the first sign of an undiagnosed cancer, or it may occur years or decades after the primary cancer is diagnosed [50]. In 5-12% of cases [47,[51][52][53] the BM is diagnosed before the primary tumor, and in 10% of cases, the primary is not discovered [54,55]. In addition, 75% of secondary brain lesions are multiple [56]. Although any type of cancer can metastasize to the brain, the three most common primary tumors associated with BMs are lung (20-56%), breast (5-20%), and melanoma (7-16%) [46,47,[57][58][59]. Lung cancer (all subtypes) is the most frequent provider of BMs regardless of gender and is the most common BM in men. The most common cancer to cause BMs in women is breast cancer [57].

43

Pathophysiology

Metastasized cancer passes through the bloodstream and reaches the central nervous system by breaching the blood-brain barrier. Subsequently, the local proliferation of clonal cells brings about the following conditions: invasion, displacement, inflammation, and edema (Figure 2.8).

The distribution in the central nervous system is then proportional to the blood flow with 90% of the distribution being in the supratentorial area and 10% in the posterior fossa [60,61].

However, the location of tumors in the brain tends to be specific to different histological subtypes which exhibit different distributions [61,62]. All other patients are Class II with a median survival of 4.2 months. Further seminal analyses were conducted on prognostic factors for survival [64] as the Rotterdam Score [63], the scoring index for radiosurgery (SIR) [65], the basic score for brain metastases (BSBM) [66], a scoring system proposed by Rades et al. that was updated to incorporate an additional prognostic factor [67,68], the graded prognostic assessment (GPA) [69], and a nomogram tool [70]. Based on these analyses, the following prognostic factors emerged: performance status, age, presence of extracranial metastases, extracranial tumor control (primary/metastases), number of BMs, volume of the largest BM, time from cancer diagnosis to brain radiation and primary tumor site and histology [63,[66][67][68][69][70].

Treatment

Currently, the best treatment for BMs is not fully established [71]. Corticosteroids, radiotherapy, surgical therapy, and radiosurgery are all proven treatments. Additionally, chemotherapy is sometimes beneficial. Each of these treatments offers distinctive advantages, but each also comes with its own negative effects. A multidisciplinary team composed of a neurosurgeon, radiation oncologist, and neuro-oncologist should be included in the design of the treatment plan along with the patient [61].

Corticosteroids are symptomatic care and prevent the development of cerebral edema. Radiotherapy is the preferred treatment for the majority of patients with BMs, owing to the lesions multiplicity. This comprises whole brain radiotherapy (WBRT) and hypofractionated stereotactic radiotherapy (HFSRT) [72]. Radiosurgery (SRS) is an excellent option when the patient has a limited number of metastases. In addition, WBRT can be used in adjuvance with SRS [61,71]. Surgical resection is recommended for patients with solitary BM found in an accessible location or with a tumor of significant size that is causing brain edema or hydrocephalus and preferably with a good performance status [73]. Finally, chemotherapy is rarely used as the main treatment option as chemotherapeutic agents breakthrough the blood-brain barrier poorly [74]. Chemotherapy for BMs might be appropriate in individuals with small, asymptomatic tumors from primary known to be chemosensitive [71].

Tumor response assessment criteria

Response assessment in solid tumors refers to a collection of established guidelines such as the response evaluation criteria in solid tumors (RECIST), WHO, Macdonald Criteria or response assessment in neuro-oncology (RANO), for evaluating tumor burden to provide an objective evaluation of response to therapy (Table 2.2) [75][76][77][78][79]. The assessment of response is based primarily on measurement of visible lesions on imaging (linear, two-dimensional), although other parameters may be taken into account such as corticosteroid use, clinical signs and symptoms. The magnitude of changes that defines response or progression is inherent to each guideline [75][76][77][78][79].

Medical imaging in neuro-oncology

Medical imaging has an essential role in the management of brain tumors (Table 2.3). At baseline, imaging can be used for diagnosis, prognosis or treatment planning purposes including the definition of the location of the stereotaxic biopsy, resection, and delineation and dosimetry stages in radiation therapy. After treatment, imaging is used to quantify the response to treatment and the extent of residual tumor. At follow-up, it is used to monitor tumor progression and to differentiate recurrent tumor growth from treatment-induced tissue changes, such as radiation necrosis. Different types of complementary imaging modalities (CT, PET and MRI) may be used depending on the characterization objectives (Table 2.4). CT scans may identify brain cancers, contribute to planning radiation therapy, and show bleeding or swelling in the brain. PET allows the evaluation of brain tumors characterized by a higher rate of cell proliferation than healthy tissue using specific tracers, which reflect the uptake of amino acids by brain cells. These tracers are 11 C-methionine, 18 F-fluorothymidine (FLT), 18 F-fluoro-ethyltyrosine (FET), and 18 F-dihydroxyphenylalanine (DOPA) [81][82][83]. This section emphasizes the fundamentals principle of MRI physics. Particular attention is given to conventional MRI sequences (T1w, T1w-gd, T2w, T2w-flair) which provide anatomical details and are of particular interest in this thesis. In addition, we briefly discussed about the major advanced MRI techniques used in clinical brain imaging: diffusion-weighted imaging (DWI), perfusion-weighted imaging (PWI), diffusion tensor imaging (DTI) and magnetic resonance spectroscopy (MRS).

These techniques provide information about tumor cellularity (DWI), tissue perfusion and permeability (PWI), white matter invasion (DTI), and biochemical composition or metabolites (MRS). Lastly, we illustrate the contribution of multimodality imaging. is the case for the hydrogen nucleus ( 1 H), phosphorus 31 ( 31 P ), carbon 13 ( 13 C), and many others. The hydrogen nuclei participate in 86% of the chemical composition of the organism and is distributed mainly in the water and lipid molecules. Due to their abundance, they are usually studied in MRI. In their natural state, the intrinsic magnetic moments of a sample of hydrogen protons are randomly oriented in all directions of space (Fig. 2.9A). The macroscopic magnetization vector ( -→ M ) resulting from all the microscopic magnetization vectors ( µ) is therefore null. On the other hand, if this same sample is placed in an magnetic field ( -→ B 0 ), the microscopic magnetization vectors -→ µ or spins are directed in the direction of -→ B 0 , either in the same direction ("parallel") or in the opposite direction ("antiparallel"), resulting in a non null macroscopic magnetization vector -→ M (Fig. 2.9B). More precisely, the spins have a rotational motion and describe a cone around the -→ B 0 axis. This is called spin precession. Their precession angular frequency ω 0 is related to the magnetic field strength B 0 and the specific gyromagnetic ratio γ of the nucleus by the following Larmor relation:

ω 0 = γ -→ B 0 (2.1)
γ depends on the nucleus. For example, the gyromagnetic ratio of hydrogen is 42.6 MHz.T -1 .

This last value allows to express directly the precession frequency in Hertz (s -1 ), also called natural frequency ν 0 = ω 0 2π . From a quantum point of view, the two positions (parallel and antiparallel) are assimilated to two distinct energy levels called spin up (E + = +γhB 0

2

) and spin down (E -= -γhB 0

2

). The Maxwell-Boltzmann statistics defines the population difference between the energy levels:

N + N -= e -γhB 0 kT (2.2)
where N + and N -are respectively the number of spins in the upper and lower states, k is the Boltzmann constant, and T the temperature in Kelvin. According to this law, the number of protons in the low energy state is slightly higher than in the high energy state. This difference, linked to the strength of the magnetic field and the temperature, is about 10 protons out of a total of 1 million protons at 1.5T and at ambient temperature. This difference is sufficient to produce an NMR signal at the tissue scale and will be at the origin of the appearance of a

Excitation phase

The excitation phase allows the flipping of -→ M by an external contribution of energy. This contribution is done by exploiting the phenomenon of resonance which stipulates that to disturb a system in equilibrium; it is necessary to bring it with energy that will move the system from the "low energy" state to the "high energy" state. Thus, an electromagnetic pulse equal to the frequency of Larmor is required. This electromagnetic wave is called a radio frequency (RF) pulse or a rotating magnetic field B 1 . Two distinct phenomena take place simultaneously during the energy input. First, the difference in the number of atoms between the two energy states will be compensated. The particles will indeed absorb the energy necessary to change the direction of their spin and thus pass from the low energy state to the high energy state.

Secondly, the net magnetization -→ M will start to precess around -→ B 0 at the Larmor frequency.

The longer the wave is applied, the greater the angle of "flip" (angle between -→ M and -→ B 0 ) will be. A 90°RF pulse will nullify the longitudinal component -→ M z of -→ M and maximize the transverse component --→ M xy (Fig. 2.10B). All spins are then in phase. A 180°RF pulse, called "inversion pulse," will totally invert the longitudinal component of -→ M . When the RF pulse is stopped, the spins will start to dephase, and the macroscopic magnetization -→ M will return to its initial equilibrium state (before the RF pulse). During its return, the net magnetization -→ M will continue to precess around -→ B 0 . This return to equilibrium is characterized by the simultaneous appearance of two phenomena: a longitudinal relaxation and a transversal relaxation (Fig.

Relaxation phase

From a quantum point of view, the stop of the RF pulse corresponds to the return of particles to the lowest energy state. This return is accompanied by a release of energy transmitted in the thermal form to the surrounding molecular medium. Two relaxation processes exist: is the consequence of the interaction of protons ("spins") between them. These dephases reflect the local field inhomogeneities specific to each tissue. The transverse relaxation is also called T2 relaxation because the transverse magnetization regrowth follows an increasing exponential where the time constant T2 (ms) is characteristic of a given tissue.

In other words, it is the time that corresponds to a 63% decrease of --→ M xy (Fig. 2.11B).

Contrary to T1, the T2 values are independent of the intensity of B 0 . Indeed, the intrinsic inhomogeneities responsible for the spin-spin relaxation (T2) are (almost) not related to the main magnetic field (B 0 ) but depend on the nature of the physicochemical medium studied.

Measurement of the free induction decay

The measurement of the NMR signal is performed with a coil (antenna) using the free induction decay (FID). For this, an echo is emitted at a time TE (echo time) after the radiofrequency pulse in order to induce the spins to return in phase and thus collect a signal of maximum intensity. The signal acquisition is done at this TE time. The sequence composed of the radiofrequency pulse and the echo is repeated periodically with a delay TR (repetition time).

The echo time sets the T2 weighting, while the repetition time sets the T1 weighting.

In summary:

• a short TR and a short TE give a T1 weighted image

• a long TR and a long TE give a T2 weighted image

• a long TR and a short TE give a proton density weighted image An MRI sequence is a particular set of pulses sequences whose parameters (TE, TR, for example) are adjusted to obtain images with a given contrast. Table 2.5 gives the order of magnitude of the TR and TE values for the most common MRI sequences.

Table 2.5: Order of magnitude of Repetition Time (TR) and Echo Time (TE) for the most common MRI sequences for the field strength of 1.5T [88].

Conventional MRI sequences

The conventional MRI protocol in the diagnosis of brain tumors includes (non-exhaustive list) [84,89]:

• T1-weighted (eventually contrast-enhanced): spin echo (SE), turbo spin echo (TSE), gradient echo, three-dimensional (3D) sequences.

• T2-weighted: SE, fast spin echo (FSE) or TSE, and 3D sequences.

• T2-weighted "dark fluid": proton density (PD) and fluid-attenuated inversion recovery (FLAIR).

• Gradient Echo (GRE): GRE T2, T2* GRE, and GRE 3D T1.

• Inversion Recovery (IR): FLAIR, T1 IR, and short-time inversion recovery (STIR).

• Fat Suppression (FS): STIR and T1 FS.

T1-weighted Pre-contrast

This sequence refers to the spin-lattice relaxation time and highlights differences in T1 relaxation times of tissues. Here, the scan parameters are set (short TR/short TE) to minimize T2 relaxation effects. This sequence is interesting for the anatomy visualization because tissues with a high-fat content such as white matter appear light (normal white matter is lighter than grey matter) and water-filled compartments (CSF) dark. On T1w imaging, neoplasms are not especially visible (Fig. 2.12A).

T1-weighted Post-contrast

T1-weighted imaging can also be performed using contrast agents, which are paramagnetic substances containing in most cases gadolinium chelates. They shorten the relaxation time of T1, which will result in a stronger T1 signal. The use of contrast agent enhance visualization and demarcation of the tumor by highlighting the hypervascularization of the disease. This is particularly useful for observing vascular structures and the breakdown of the blood-brain barrier (BBB) [90] (Fig. 2.12B).

53

T2-weighted

This sequence refers to the spin-spin relaxation time and highlights differences in T2 relaxation times of tissues. This allows differentiation of anatomical structures mainly based on T2 values. Here, the scan parameters are set (long TR/long TE) to minimize T1 relaxation effects.

This sequence is described as anti-anatomic because tissues with a high-fat content such as white matter appear dark (normal white matter is darker than grey matter) and water-filled compartments (CSF) light. The tumor area corresponding to the BBB leakage shows edema that appears hyperintense with an intensity similar to that of the ventricles (Fig. 2.12C).

T2 Fluid Attenuated Inversion Recovery

This sequence is analogous to a T2-weighted image, except for the highly extended TE and TR times. Thus, anomalies stay visible while normal CSF fluid is attenuated and darkened. This sequence is extremely sensitive to disease and aids in the separation of CSF and abnormalities, facilitating, for example, the distinctness of the hyperintense edema from the adjacent periventricular parenchyma (Fig. 2.12D) [91,92].

Advanced MRI techniques

To overcome the limitations of conventional MRI, advanced magnetic resonance techniques have been proposed in neuroradiology to assess changes at the microvascular, hemodynamic, cellular, metabolic, biochemical levels of brain tumors [84,93,94]. The most commonly used advanced MR imaging techniques are DWI, PWI, DTI and MRS).

Perfusion weighted imaging (PWI)

This term encompasses three types of sequences allowing to obtain perfusion-weighted images capable of providing insight into the blood perfusion of tissues using hemodynamic changes principle [84,96]:

• Dynamic susceptibility contrast (DSC): gadolinium is injected as a contrast agent and 55 fast-repeat imaging (usually T2 or T2 * ) is performed to quantify susceptibility-induced signal loss [97,98]. Various perfusion parameters can be extracted, such as cerebral blood volume (CBV is the quantity of blood in a given volume in mL/100mg), cerebral blood flow (CBF is the blood flow in brain tissue in mL/100g/min) and mean transit time (MTT is the average time for arteriovenous passage of blood in a given volume in seconds) [99,100].

• Dynamic contrast-enhanced (DCE): measurement of the shortening of T1 relaxation time induced by a bolus of gadolinium contrast agent [98,101]. After sequence acquisition, compartmental model calculations are used to estimate vascular leakage and blood-brain barrier breakdown [94,102,103]. The estimate of the vascular permeability can be given by the transfer constant (k trans ) of contrast agents from the intravascular compartment to the extravascular compartment.

• Arterial spin labeling (ASL): ASL is used to quantify CBV. Arterial blood protons are marked by a radiofrequency pulse; thus it is not necessary to infuse gadolinium as a contrast agent [104].

PWI was used to estimate the grade of tumors in several studies. High-grade gliomas have higher CBV values than low-grade gliomas [105][106][107]. The initial assessment of the relative CBV with a threshold value also allows predicting the time to progression and the clinical outcome [108][109][110][111]. Furthermore, it is used in treatment monitoring for recurrent tumor differentiation from radiation-induced tissue changes (pseudoprogression) [112][113][114][115][116]. Pseudoprogression has lower CBV values, and therefore areas of increased perfusion correspond to tumor recurrence.

Diffusion weighted imaging (DWI)

The theory is based on random motion of water molecules' within various tissues. This "Brownian motion" is dependent on the cellularity of the tissue, the integrity of the cell membranes, and the degree of vascularization. Increases in these parameters result in a restriction of molecular diffusion, which is evaluated by the calculation of an apparent diffusion coefficient (ADC) [117]. The coefficients may then be used to perform voxel-level mapping (ADC map) of water diffusion properties of brain tissue. The difference in ADC values permits the characterization of morphological features, including edema, necrosis, and viable tumor tissue [118]. In addition, it would facilitate the detection of areas of tumor infiltration that are not visible on other MRI sequences [119]. Finally, it would allow greater certainty than conventional MRI to distinguish brain abscesses from cystic or necrotic brain tumors [120]. In adults with diffuse gliomas, ADC values correlate with IDH mutation status and overall survival [121]. Compared to IDH-mutant gliomas, IDH wild-type gliomas had lower ADC values and a shorter overall survival time.

Diffusion tensor imaging (DTI)

This is a kind of DWI that allows to map white matter tractography and characterizes the flow of water molecules in the brain using two parameters: mean diffusivity (MD) and fractional anisotropy (FA), which characterize the directionality of water diffusion [122]. The reduction in FA values surrounding the tumor suggests that tumor components have infiltrated the peritumoral white matter [123]. DTI provides critical information regarding the tract's involvement and displacement related to the tumor location [124].

Magnetic resonance spectroscopy (MRS)

Magnetic resonance spectroscopy (MRS) enables the metabolic characterization of tissues by collecting signals that are produced by the atomic nuclei within molecules. The identification of their presence in the brain is based on the differences in proton resonance frequencies. Among the metabolites observable in clinical spectroscopy, six are of particular importance in tumor pathology [125,126]: N-Acetyl Aspartate (NAA), which is a neuronal marker; creatine (Cr), which is a marker of global metabolism and cellularity index; choline (Cho), which increases in the case of cell proliferation; myo-inositol (MI), a sugar present only in glia, free lipids (Lip), which are a marker of cellular necrosis; and lactate (Lac) which accumulates through increased glycolysis indicating altered metabolism. Different metabolic ratios were found to be relevant for tumor classification and prediction of malignancy [127,128]. (A) Glioblastoma multiforme and (B) intracerebral metastases. Reprinted from [129] under the Creative Commons Attribution 3.0 Unported.

Multimodal imaging

A significant complementarity exists between functional imaging (PET, MRI), which enables the detection of abnormal metabolism within tissue but with limited spatial resolution, and 57 anatomical imaging (CT, MRI), which allows accurate anatomical localization of these abnormalities. In this regard, Beyer created the world's first hybrid PET-CT imager in 2000 [130,[START_REF] Charron | Image analysis in patients with cancer studied with a combined PET and CT scanner[END_REF], and more recently, hybrid PET-MRI systems have been brought to the market to enhance the accuracy of diagnostic procedures [132]. The promise of PET-MRI for monitoring brain cancers has been shown in some studies, utilizing tracers such as FET or Methionine to detect and track the progression of the tumor [133,134]. In neuro-oncology, no research has shown that hybrid imaging is preferable over separate examinations.

Radiomics modeling

Imaging data acquisition

Imaging data is acquired in the department concerned and is subject to institutional proce- Grey-level discretization The region of interest is then segmented either manually, semi-automatically, or automatically. Note that the segmentation step can take place before the preprocessing. For conventional radiomics, handcrafted features (histogram, shape, texture) are extracted from the region of interest after discretizing the grey levels. Machine learning algorithms are then used to determine the clinical outcome of interest. The deep-learning method allows to automatically extract features and determine the clinical outcome of interest, either directly from the previously defined segmentation or by including the segmentation step directly in the process.

Segmentation

In radiomics analysis, segmentation is a critical step in identifying the area of interest for feature extraction. A bidimensional region of interest (ROI) or volume of interest (VOI) can be produced if the contouring has been performed on a single slice or multiple slices, respectively [2,3]. This procedure has traditionally been carried out manually, strongly depending on the knowledge and skills of the expert [4]. Thus, manual contouring is not optimal since it can add variability and bias and affect the reproducibility of predictive models [5]. A technique such as ROI/VOI pertubation can take into account the possible variations during manual contouring and keep only the robust features [6,7]. Thus, morphological operations can consider overestimation (with dilation), and underestimation (with erosion). In addition, manual segmentation remains an acceptable option when a few patients are included in a study, but as the number of patients increases, it becomes unrealistic, even if a highly skilled practitioner does the process. Thus, when the size of the cohorts becomes important, methods involv-ing an operator in a lesser way, such as semi-automatic or automatic segmentation methods, are prefered [8]. A semi-automatic segmentation method combines automated and manual editing to get excellent results with considerable time and cost savings. In this subtype of computer-aided segmentation, automated contouring algorithms such as Clustering [9], Fuzzy Connectedness [10][11][12], Level Set [13], Region Growing [14], Snakes [15,16], or Thresholding [17] are used. With recent advances in deep learning, it has become possible to create fully automated segmentation techniques based on different architectures [18], such as the popular U-Net [19]. However, their performance is questionable, especially in the presence of artifacts, when the signal-to-background ratio is low, when noise is present, when the lesions of interest are heterogeneous or when an adjacent structure of similar contrast is present [20]. A further drawback is that the generalizability of the trained algorithms is presently a significant limitation, and applying these algorithms to a different dataset may result in failure.

Image preprocessing

The image preprocessing is an interchangeable step that also can be performed upstream the image segmentation. An important limitation in MRI imaging relies in the lack of standardization of voxel intensities for inter-and intra-scanner variability, even for the same protocol, same patient, same body region; which are greatly dependent on radio frequency coil nonuniformity, gradient field, magnetic field, and other factors [21][22][23]. This limitation impacts radiomic features [22,24]. Image preprocessing methods such as bias field correction, spatial resampling, skull-stripping, and intensity standardization make quantitative MRI analysis more straightforward and more reproducible. It also facilitates automated techniques such as segmentation and registration [25][26][27]. These methods improve the consistency and comparability of the radiomics findings [28].

Bias field correction

Bias field is known as intensity inhomogeneity or intensity nonuniformity. Bias field corresponds to a low-frequency unwanted signal that blurs MR images and reduces high-frequency MRI contents like contours and edges. This smooth intensity variation in MR images is caused by different factors such as: nonuniform reception sensitivity, inhomogeneous RF, or less important parameters as eddy currents, mistuning of the RF coil, geometric distortion, patient movement [29].

Nonparametric intensity nonuniformity normalization (N3) and its improved version (N4ITK) are reference algorithms for intensity inhomogeneity correction. The method is an iterative algorithm seeking the smooth multiplicative field that maximizes the high-frequency content of the tissue intensity distribution [30]. The following equation is the basis of bias field correction algorithms:

v(x) = u(x)f (x) + n(x) (3.1)
where v is the observed image, u is the uncorrupted image, f is the bias field, n is the noise (assumed to be independent and Gaussian), and x designates the spatial position or voxel.

Assuming a noise-free scenario, the image model becomes:

v(x) = u(x) + f (x) (3.2)
where û = log u. The particularity of N4ITK is that the B-spline fitting is improved compared to N3. Thus for N4ITK, from model of Eq. 3.2, the iterative component of the algorithm is optimized as described by the following equation:

u n = u n-1 -f n r = u n-1 -S * u n-1 -E u | u n-1 (3.3)
where S * {•} is a modified B-spline estimator, and f n r is the estimated residual bias field at the nth iteration.

Voxel size resampling

Spatial resampling can reduce the variability caused by different voxel sizes [31][32][33]. This resampling is generally done in such a way as to preserve isotropic voxel spacing (for example 1mm 3 ) which is necessary for most texture feature classes to become rotationally invariant [34]. Interpolation methods such as nearest neighbor and cubic interpolation are included in the majority of software. Among those methods, one may interpolate the ROI mask using the nearest neighbor method to preserve the 0's and 1's in the interpolated ROI and the imaging volume using cubic interpolation to achieve a smooth interpolated image.

Skull-stripping

Skull-stripping or brain extraction refers to the process of separating the brain from nonbrain tissues in medical images. In brain imaging studies, this processing step is critical for two reasons [35,36]. First, the skull exhibits the highest intensity variations, and second, it allows to define the region in which intensities need to be proceeded by the final harmonization step, which is intensity standardization. For brain extraction, either popular Brain Extraction Tool (BET) [37] based on a deformable surface model or HD-BET [38] based on convolutional neural networks have been proposed in the literature.

Intensity standardization

Different standardization methods have been proposed in the literature to homogenize MRI images intensities before performing quantitative analysis. Among these methods, the Nyùl et al. [39] method and further extended by Shah et al. [40] is based on the creation of a standard histogram from a retrospective database of MRI images, then a transformation of the images to be processed so that their histograms correspond to the standard histogram. The Hybrid White Stripe method developed by Shinohara et al [41] is based on a standardization of intensities in order to reach a standard value of mean and standard deviation of the intensities in the white matter. The Z-Score method is similar to the Hybrid White Stripe method but based on a region of interest (here the brain). These methods, which were often not applied in radiomics studies in 2019 and never been evaluated in the radiomic context, were the subject of a paper presented in Chapter 4. The Nyùl et al., Hybrid White Stripe and Z-Score methods are described in details in the section 4.4.2.

Grey level discretization

The grey level discretization clusters similar intensity values into a specific interval (bin). This step has noise suppression properties and makes the feature computation tractable [42,43].

Two different methods are commonly used, which have their own strengths and limitations, but cannot be used interchangeably [42][43][44][45][46]. The first one consists of discretizing intensities according to a fixed bin number (FBN), while the second consists of discretizing intensities according to a fixed bin size (FBS). There is no consensus in the literature on the better method, and the description of this step is sometimes omitted in radiomics studies. In addition, the optimum bin number/bin width to employ in this discretization step also remains an unresolved issue [47]. When bins are too large (or too few), underlying biological characteristics may be lost by the process; when bins are too small (or too numerous), features remain disturbed by noise.

A balance is reached when discretization can filter out noise while retaining important biological components; therefore, optimal binning decision-making depends on the noise-inducing features (acquisition parameters) and the robustness of the features of interest to noise. This processing step was also the subject of the paper presented in Chapter 4. The FBN and FBS methods are described in details in the section 4.4.4. It may be noted that other discretization approaches exist [46], such as absolute resampling [44], or the use of a clustering algorithm (Max-Lloyd) [48], but they are not currently included in the IBSI guidelines.

Feature extraction

Radiomics features encompass two broad categories of features: handcrafted features through conventional radiomics and deep features obtained through deep learning.

Handcrafted features

Handcrafted features are calculated using predefined mathematical formulas proposed by image processing experts. Handcrafted features include semantic and agnostic features. Semantic features refer to existing tumor descriptors conventionally used in radiology and, agnostic features category is made up of purely computational features. Agnostic features can be divided into subgroups which describe shape, density, and texture of the ROI/VOI. Since several alternative methods and formulae for calculating such features exist, adherence to IBSI standards is strongly advised [34]. For this purpose many free software (stand-alone programs, modules and libraries) that comply with the IBSI standard such as CERR [49], S-IBEX [50],

LIFEx [51], MITK [52], RaCaT [53], and Pyradiomics [54] are today distributed.

Shape-based features

Shape, also known as morphology, describes the 2D or 3D geometry of the segmented structure. This category of features is independent of the grey level intensity distribution in an image and is extracted from the segmentation mask. Shape-based features allow in particular to study surface, volume, compactness, flatness, elongation, or sphericity of the VOI considered. The spiculated nature of the lesions is in favor of a tendency to spread and is often correlated with advanced stages [55]. In contrast, benign or less aggressive tumors most often have well-defined or circumscribed margins [56]. One study showed that the majority of radiomic shape features had strong direct or monotonic inverse correlations with tumor spiculatedness [57].

Intensity-based features

The intensity-based features, also known as first-order statistical features, describe the distribution of voxel intensities inside the ROI/VOI and relie on simple descriptors such as the maximum, minimum, mean, range, kurtosis, and skewness of the grey level intensities (Fig.

3.2A).

Texture-based features

The texture-based features, also known as second-order statistical features, provide information on the arrangement of grey level values in the ROI/VOI. Texture matrices-derived indices are excellent candidates for characterizing tumor heterogeneity. Commonly, texture feature classes are Grey Level Cooccurence Matrix (GLCM), Grey Level Run Length Matrix (GLRLM), Grey Level Size Zone Matrix (GLSZM), Neighbouring Grey Tone Difference Matrix (NGTDM), and Grey Level Dependence Matrix (GLDM). An example of the calculation of the different matrix classes is shown in Figure 3.2B. These different matrices thus provide complementary information:

• The Grey Level Co-occurrence Matrix represents the probability of observing a pair of values in voxels at a given distance, and in a given direction [58].

• The Grey Level Run-Length Matrix measures the number of consecutive voxels with the same value, aligned in a given direction for each intensity value [59][60][61][62].

• The Grey Level Size-Zone Matrix measures the number of neighboring voxels of the same intensity, for each intensity value [63].

• The Neighborhood Grey-Tone Difference Matrix measures the difference in intensity between neighboring voxels [64].

• The Grey Level Dependency Matrix quantifies the dependencies of the grey levels in an image, i.e., the number of neighboring voxels that are equal to the central voxel [65].

It should be noted that higher-order features can be extracted by applying various filters to preprocess the image to extract a higher number of radiomic features that might reflect more valuable information than the original ones. These filters include wavelets, Laplacian of Gaussian (LoG) filters and some simple filters such as square, square root, logarithmic, and exponential filters [54].

Deep features

Neural networks created to tackle classification or regression problems automatically discover deep learning features from medical images. Thus the features are self-designed for a defined problem and do not require prior knowledge to be extracted such as handcrafted features. The prominence of deep-learning in radiomics continues to rise due to its capacity to alleviate the need in radiologist knowledge and eliminate inter-expert variability. Recent studies have shown the advantage of deep learning algorithms in large datasets compared to handcrafted features [67][68][69]. Studies also show the complementary of integrating conventional and deep radiomic features [70,71].

Data handling

One of the general objectives of this thesis was to develop multi-variable models based on radiomics using machine learning. Despite their miscellaneous objectives (regression, classification, survival, etc.), the different ML approaches share common steps for the development of an algorithm: (i) constitution of the database with extraction, selection, and preprocessing of information, (ii) training and validation of the model on two different data sets where a first data set (training set) is used to train the model and the second one is used for its optimization (validation set), and (iii) final report of the model performance on an independent data set (test set). In this section, we will not go into details about the various machine learning models that were employed in this thesis and their underpin mathematical principles. A brief theoretical overview of machine learning concepts is however put forward.

1 1 1 3 0 2 1 3 3 0 3 3 1 0 1 0 1 1 1 3 0 2 1 3 3 0 3 3 1 0 1 0 1 1 1 3 0 2 1 3 3 0 3 3 1 0 1 0 1 1 1 3 0 2 1 3 3 0 3 3 1 0 1 0 1 1 1 3 0 2 1 3 3 0 3 3 1 0 1 0 0 1 2 3 0 0 1 1 1 1 2 2 0 2

Data preparation

The handling of radiomic data and its preprocessing prior to its use by a machine learning algorithm is a critical step. Indeed, this step can significantly impact the outcome. Suppose images are acquired with a wide range of scanning devices and manufacturers. The lack of standardized protocols results in significant variability in acquisition and reconstruction parameters. These parameters have been shown to affect the noise, contrast, and spatial resolution of medical images, impacting the subsequent measurement of shape, histogram, texture, and higher-order features extracted from the images [4]. Chapter 4 provides recommendations for MR image preprocessing in the radiomic context, including intensity standardization and grey level discretization. If actions have not been considered upstream of the extraction of radiomic features to decrease this variability using image preprocessing, a posteriori compensation method will have to be used. Recently, a data-driven post-processing method, called

ComBat [72], has shown its ability to harmonize the radiomic features efficiently. Initially proposed to correct for batch effects in genomics studies, it has demonstrated its effectiveness in PET [73], CT [74] and MRI [75] imaging, mainly in observational studies. As such, the method cannot be applied to ML pipelines straightforwardly. The ComBat method was thus the subject of the second paper (Chapter 5) to propose an adaptation that answers the need for ML compatibility. The ComBat method is detailed in section 5.4.2.

In ML, some steps are considered important to perform unbiased learning. These key steps are feature scaling, randomization, and class balancing (sampling artifacts).

The ranges of the different radiomic features vary widely. ML algorithms that use gradient descent as an optimization technique such as linear regression, logistic regression, neural network, etc., require data to be scaled. Indeed, feature value will affect the step size of the gradient descent. If a difference between ranges of features exists, then the step size is different for each feature. Data scaling ensures that the gradient descent proceeds smoothly towards the minima and that the gradient descent steps are being updated for all features at the same rate. In addition, the Euclidean distance is used in many classifiers (KNN, K-means, SVM, etc.) to compute the distance between two points. If one feature has a wide range of values, the distance will be overpowered by that particular feature. Therefore, the range of all features should be put on a common scale to ensure that each feature will contribute approximately equally to the final distance. The two most commonly used methods for continuous variables are normalization and standardization. The first specifically refers to a scaling between 0 and 1. In contrast, the second is generally used to refer to the application of a Z-Score, which consists of removing the mean and scaling values to unit variance.

Dataset randomization (shuffling) is a notable aspect in model development, which may significantly impact model performance. Data shuffling is necessary to eliminate biases/patterns in the split data sets before training the ML model. In the case of a class imbalance, the data may be also stratified to ensure that the same proportion of classes is maintained in the training and validation sets.

Class balance is a critical aspect for assessing ML classifiers' true performance. Important imbalance may lead to misleading findings. Over and undersampling methods may be employed to handle this issue. The synthetic minority oversampling technique (SMOTE), which generates new instances from the minority class that are not perfect copies of the real instances, is a widely used and approved approach for balancing classes [76]. A more recent version of SMOTE called ADASYN (adaptive synthetic sampling method for unbalanced learning) is optimized for learning from imbalanced data sets [77]. Additionally, generative adversarial networks (GAN)-based methods are increasingly being utilized to create synthetic compensations for these imbalances [78,79].

Feature qualification

Computational medical imaging currently classically incorporates about 50 to 5000 quantitative features per patient, and this number is still expected to increase [80]. This dauntingly large number of image features is often much higher than the number of patients included in a study. In addition, all of these data may not be useful for a particular task and lead to a high probability of false-positive results [81][82][83]. This phenomenon is referred to as the "curse of dimensionality," which designates various phenomena that arise when analyzing or organizing data that do not occur in low-dimensional settings [81]. Adjustments for multiple comparisons (Bonferroni correction [84]) and control of the false discovery rate (Benjamini-Hochberg [85])

are commonly used methods to account for this. In addition, the number of extracted features may be overkill, which reduces power and increases the likelihood of overfitting the data, reducing the ability to generalize to other datasets. Therefore, dimensionality reduction and task-specific feature selection are required in order to get the optimum performance. Despite the fact that there is a large variety of techniques available, they may be divided into two broad categories: feature selection and feature extraction. The first category retains only a subset of variables, classified according to a performance criteria, thus eliminating irrelevant and redundant variables. This selection of variables can be carried out using three approaches (Fig. 3.3):

• "Filter" methods: They are typically employed as a preprocessing step. The selection of features is machine learning algorithm-independent. Thus, features are chosen according to their correlation with the output variable of interest measured by different statistical tests such as Pearson's Correlation [86], analysis of variance (ANOVA) [87], linear discriminant analysis (LDA) [88], Chi-Square [89].

• "Wrapper" methods: These methods are based on greedy search algorithms that are considered computationally expensive. They evaluate all possible combinations of features and select the combination that produces the best result for a specific machine learning algorithm. Three distinct types of feature selection wrapper techniques exist: step forward feature selection, step backwards feature selection, and Bi-directional elimination (stepwise selection) [90,91].

• "Embedded" methods: This technique combines the feature selection and the machine learning algorithm into a single mathematical problem, e.g., linear regression with Ridge, Lasso, or ElasticNet regularization or random forest [92].

Selecting The second category corresponding to feature extraction uses all the information in the original set of features to compress it and produce new features projected into a space of smaller dimensions. This new reduced feature set should then be able to summarize most of the information contained in the original feature set. This category includes linear methods:

Principal Component Analysis (PCA) [93], Independent Component Analysis (ICA) [94], LDA [95], and non-linear methods: Locally Linear Embedded (LLE) [96], t-distributed Stochastic

Neighbor Embedding (t-SNE) [97], Isomap [98], Autoencoders [99].

Another method is to simply eliminate features based on reproducibility analyses to reduce the dimension by excluding features with relatively low reproducibility. The stability of radiomic features can be performed through analysis of test-retest data [100] and by evaluating the segmentation robustness to inter-observer variations [101]. The most common statistical tools for these analyses are the intra-class correlation coefficient (ICC) [102] and the concordance correlation coefficient (CCC) [103] which select features above a certain threshold defining the robustness criterion.

AI-based analysis Prerequisites to an AI initiative

Prior to embarking on an AI project, several prerequisites must be met: i) availability of consistent data; (ii) proper data curation; (iii) expert-driven data processing; and (iv) a genuine clinical issue to be addressed by the AI.

Additionally, the sample size is a critical factor to consider before doing an AI-based analysis.

Although it is common to come across AI or machine learning research with a relatively small sample size in the literature, the community should be aware that sample size is critical to avoid model fitting issues (Table 3.1) and improve generalizability to unseen data. Massive amounts of data are required for very sophisticated algorithms such as deep learning. However, in the event of limited or insufficient data, it is worth noting that certain well-known augmentation methods should also be explored, such as image modification or advanced techniques such as the use of GAN to create synthetic images [79]. For example, traditional image processing operations for data augmentation used in DL are as follows: random rotating, vertical and horizontal flipping, contrast modification, zooming, addition of noise, and many others. GAN algorithms can recognize patterns in input data and generate new samples similar to the input data.

Model development

Machine learning

Different types of models can be classified into two main categories: i) Supervised models where all data are labeled, and the algorithms learn to predict the outcome of the input data categorically for classification tasks, or continuously for regression tasks. ii) Unsupervised models where all data are unlabeled, and the algorithms learn the inherent structure from the input data, mainly by a clustering technique. Semi-supervised learning uses both labeled and unlabeled data in a training data set [104]. In machine learning, there are a plethora of techniques available, including k-nearest neighbors, naive Bayes, logistic regression, support vector machine, decision tree, random forest, boosting approaches, and many others [105].

Because no algorithm is inherently better than another, it is the job at hand, as well as the testing with a variety of algorithms, that allow one to identify the best suitable algorithm for a particular task. This phenomenon is referred to as the "No Free Lunch Theorem [106]."

Generally, one will look for a multivariate model composed of a minimum of highly predictive features. The idea is to favor the generability of the model to other patient cohorts than the one(s) from which it was trained. The Occam's razor principle suggests that the simplest model with the best predictive properties should be preferred, which is the crucial step in all machine learning approaches [107].

Table 3.1: Bias-Variance tradeoff. Overfitting and underfitting are often encountered issues that must be addressed to build more optimally fitted and generalizable models on new data. Underfitting refers to models that perform poorly on both training and test data. On the other side, overfitting refers to models that perform very well on training data but perform poorly on test data. In overfitting models, the algorithm learns both the important signal and the noise that causes the overfitting. In fact, all data sets include some amount of noise. However, when dealing with small data sets, the impact of noise may be considerably more apparent. In a general way, the simpler the model, the higher the bias, and the more complex the model, the higher the variance. RNNs are capable of processing sequence inputs and using the previous outputs as inputs by utilizing their internal memory. Long short-term memory (LSTM) [108] and gated recurrent units (GRU) [109] are two common kinds of RNN. On the other hand, generative models are unsupervised, which means they are trained without considering the class labels.

The aim of these models, in general, is to learn the data distribution in such a manner that they can create new data from a given distribution. In other words, generative models may extract natural and representative features from data and utilize them as inputs to classifiers. Thus, it is typical in radiomics to train a generative model and then utilize the acquired weights as the initial weights of a discriminative model. Several deep generative models have been used

in radiomics [69], including auto-encoder networks [110], deep belief networks [111], and deep

Boltzmann machines [112]. CNNs, in image analysis. In CNN, inside specific layers, such as convolutional ones, image inputs are directly scanned using small-sized filters or kernels, resulting in transformed images. Operations such as pooling and convolutions are crucial processes in CNN architectures, offering the most prominent characteristics of the images (e.g., edges). The activation functions, such as rectified linear unit (ReLU), sigmoid, and softmax, as well as regularization (such as dropout), are also essential components of deep learning systems. Currently, no method can be used to determine the appropriate number and type of layers for a particular task. A trialand-error approach is used to develop the best architecture. On the other hand, some established models and their variants are widely used for specific tasks, such as U-Net for segmentation.

Data splitting strategy

Today, radiomics is still considered a simple field of research. To be accepted in the clinical area, the development of a model and the report of the outcome must follow a particular methodology. One highly critical concept is the data division, which is linked to a particular purpose. The data should always be divided into three parts: training, validation, and testing sets.

• Training data: This dataset is used to fit the model. The same training data set may be attempted many times with various hyperparameter settings or entirely different machine learning methods. This enables evaluating the performance of various algorithm parameters. This procedure lays the groundwork for model selection, in which the optimal algorithm is chosen from among these many models. To prevent favoring overfitted models, the actual assessment for choosing the best model is done on a separate validation data set rather than on the training data.

• Validation data: This dataset is also called the development set. It is used for model selection and parameter tuning. In some ways, the validation data should be seen as a test data set for adjusting the algorithm's parameters. For instance, parameter selection may be fine-tuned by training the model several times on the training data and then estimating the accuracy of these various models using the validation set. A variety of parameter combinations is sampled and their correctness is evaluated against the validation set.

This accuracy is used to find the optimal parameter.

• Test data: This data set provides an unbiased evaluation of a final model fit. The test data must not be viewed throughout the parameter fitting and model selection processes to prevent overfitting. The test data is utilized just once throughout the procedure, at the very end. In fact, if the analyst makes any adjustments to the model based on the findings of the test data, the results will be influenced by the analyst's knowledge of the test data. The condition that a test data set can be looked only once is a very strict (and critical) criterion. Ideally, this dataset is independent, i.e., not used during training and has been collected prospectively in another institution [113].

In the literature on AI, there is often a terminological confusion between the validation and the test whose meaning is exchanged. The "test set" becomes the development set, and the "validation set" is the independent set used to evaluate the final performance of a model [114].

This is one of the glaring examples that taints artificial intelligence research giving a flawed use of a concept. The most frequently used validation methods are presented in Fig. 3.5 with a didactic approach. The validation method to be chosen is mainly determined by the problem, the requirements, and the computational capacity. Regarding the splitting ratio of the data, Table 3.2: Metrics derived from a confusion matrix. ACC is the accuracy, TPR is the true positive rate (also known as sensitivity or recall), TNR is the true negative rate (also known as specificity), PPV is the positive predictive value (also known as precision), BA is the balanced accuracy and F1 is the F1 score. The four basic cardinalities of the confusion matrix are true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN). or true positive rate, corresponds to the ability to detect the positive class. Specificity, also known as the true negative rate, corresponds to the ability to detect the negative class. Balanced accuracy is the arithmetic mean of sensitivity and specificity. Precision, also known as the positive predictive value, is the proportion of samples of the positive class correctly identified. F1-score is the harmonic mean of precision and recall. Table 3.2 summarizes metrics derived from the confusion matrix.

A survival study will mainly use the Harrell's Concordance Index (C-index) [115]. This metric is a generalization of the ROC-AUC that considers censored data. It corresponds to the probability that the model correctly predicts the order of death for a randomly selected pair of patients.

A segmentation task mainly uses the dice similarity coefficient (DSC) which is equivalent to the F1 score or the Hausdorff distance (HD). DSC is an overlap-based metric, while the HD is a distance-based metric which measures the maximal distance between the margin of two considered contours. HD is usually sensitive to noise and outliers in medical segmentations [116,117]. Therefore, a solution to remove outliers is to use a percentile approach such as the 95th percentile (HD95) [118].

Conventional statistical techniques may be used to compare the performance of AI algorithms [119]. A variety of statistical techniques are available for comparison, depending on the assumptions of the methods and the number of classifiers. For example, the student ttest, Wilcoxon signed-rank test, analysis of variance, Friedman test, and others are frequently employed. Multiplicity must be handled in numerous comparisons. According to the clinical application, the best-performing and most stable classifier is chosen.

Recommendations

Numerous biases may occur during radiomics researches, whether during image acquisition and preprocessing, segmentation, extraction, and selection of radiomic features, machine learning, or clinical validation [120,121]. Replication and external validation of radiomics studies are essential to provide adequate and compelling scientific data to support the translation of potential applications into clinical practice. The community must come to terms with a system of guidelines that everyone can rely on. According to a systematic review, only 17 % of radiomics researches addressed in-depth every methodological element relevant to image acquisition, preprocessing, or feature extraction [122]. Thus, more and more initiatives and recommendations have been issued to ensure the quality and methodological validity of radiomics studies. One example is the Radiomic Quality Score, which uses a 16 point checklist to assess the various technical and statistical aspects and the clinical validity of the radiomic analysis method [123].

A checklist for artificial intelligence in medical imaging (CLAIM) composed of 42 points was also proposed [124]. An independent international collaboration to standardize the extraction of image biomarkers from acquired imagery for high-throughput quantitative image analysis has also been initiated [34]. In this study, recommendations were provided regarding the optimal preprocessing and grey level discretization to be applied to brain MR images before radiomic features extraction, as no guidelines existed at that time. Intensity normalization and discretization methods were evaluated based on an overlapping cohort of patients who carried out a scan on two different machines (one with a magnetic field strength of 1.5T and the other at 3T) provided by the Sainte-Anne Hospital (GHU Paris) and in a machine learning task applied to tumor grade classification using a multicentric public dataset. The impact of these methods was evaluated separately on first-and second-order radiomics features.

Abstract

Radiomics .006), 0.79 (95% CI 0.76-0.82, P = .021) and 0.82 (95% CI 0.80-0.85, P = .005), respectively, using the Nyul, WhiteStripe and Z-Score normalization methods compared to no normalization. The relative discretization makes unnecessary the use of intensity normalization for the second-order radiomics features. Even if the bin number for the discretization had a small impact on classification performances, a good compromise was obtained using the 32 bins considering both T1w-gd and T2w-flair sequences. No significant improvements in classification performances were observed using feature selection. A standardized preprocessing pipeline is proposed for the use of radiomics in MRI of brain tumors. For models based on first-and second-order features, we recommend normalizing images with the Z-Score method and adopting an absolute discretization approach. For second-order feature-based signatures, relative discretization can be used without prior normalization. In both cases, 32 bins for discretization are recommended. This study may pave the way for the multicentric development and validation of MR-based radiomics biomarkers.

Introduction

Radiomics relies on the extraction of a wide variety of quantitative image-based features, including shape, histogram-based, textural and higher order statistics [1]. Along with machine learning techniques, radiomics is becoming an increasingly popular computer-aided diagnostic tool in the field of medical research [2,3]. Radiomics offers an almost unlimited supply of imaging biomarkers that can facilitate cancer detection, diagnosis, and prognosis assessment and the prediction of treatment response [1][2][3][4].

Magnetic resonance imaging (MRI) exhibits high soft tissue contrast and submillimetre spatial resolution. In the context of radiomics, a main issue is that MRI intensities are nonstandardized and are highly dependent on the manufacturer, sequence type and acquisition parameters [5]. Consequently, a large variability in image intensities among inter-patient and intra-patient acquisitions exists that could highly affect the extraction of the radiomic features, compromising the pooling and the reproducibility of published data using independent imaging sets [6,7].

To solve this problem, previous radiomics studies have focused on image preprocessing techniques. For example, it has been shown that bias field correction efficiently minimizes MR intensity inhomogeneity within a tissue region [8][9][10]. The variability generated by different voxel sizes can also be reduced by spatial resampling [9,11,12]. Moreover, brain extraction is mandatory to remove the skull regions that generate the most important variations in intensities and to define the region in which intensities should be considered before any image intensity normalization [13,14]. However, even though these three types of preprocessing of brain MRI are widely accepted by the community, there is no consensus within radiomics studies regarding the applied image normalization method (Table 4.1). In this study, we focused on three normalization methods that were selected for their representativeness within current radiomics studies (Nyul, WhiteStripe and Z-Score). These techniques include relatively simple (e.g., Z-Score) to more complex (e.g., WhiteStripe) formulations.

The technique developed by Nyúl et al. [15] and further extended by Shah et al. [16] is a piecewise linear histogram matching method. In particular, in this method, a standard histogram is learned from the training set and then used to linearly map the intensities of the image of interest. Shinohara et al. [17] described a statistical normalization method called WhiteStripe based on the intensity values of the normal-appearing white matter (NAWM).

The Z-Score method consists of subtracting the mean intensity of the entire image or a region of interest from each voxel value and dividing it by the corresponding standard deviation [18].

To calculate second-order features, also known as texture features, a grey-level discretization step clusters similar intensity levels into bins to minimize the noise impact and decrease calculation times [19]. This is an additional critical preprocessing step that does not express any consensus in the literature, and it is usually not truly clarified in radiomics studies (Table 4.1). Conventionally, the grey-level discretization can be defined as absolute if a fixed bin size (FBS) is used to cluster the intensities of the region of interest (ROI) or as relative when a fixed bin number (FBN), whose size depends on the minimum and maximum values within the same ROI, is preferred.

Even if several studies have shown variabilities in texture analysis depending on MRI acquisition parameters and the grey-level discretization step, none of them has assessed the combined impact of intensity normalization and grey-level discretization preprocessing methods on radiomic feature values in MRI [20][21][22][23][24].

The objective of this study was to assess the impact of three intensity normalization methods coupled with two methods for grey-level discretization on the challenging task of tumor grade classification in two independent cohorts. Finally, we propose recommendations to standardize the preprocessing techniques of brain MRI, which is crucial to achieve reliable radiomics-based machine learning models.

Material and methods

Data description

Two retrospective datasets were used for this study. DATASET1 included twenty consecutive patients with WHO grade II and III gliomas between January and June 2010 (Table 4.2). A previous article based on the same cohort analysed the robustness of conventional features (lesion volumes, ratios of cerebral blood volumes, contrast-to-noise ratios) depending on the magnetic field [25]. In this manuscript, the same cohort was considered to evaluate the

Image preprocessing

MR images from DATASET1 and DATASET2 were first corrected for the bias field effect using the N4ITK algorithm [44] as implemented in the Advanced Normalization Tools (ANTs)58 with default parameters. They were then spatially resampled on a 1 mm × 1 mm × 1 mm grid as suggested by Vallières et al. [45] using b-spline interpolation with ANTs. Images from DATASET1 were finally skull-stripped with the Brain Extraction Tool (BET) of the FSL software (FMRIB's Software Library) [46] and co-registered with a global linear registration including 12 degrees of freedom using ANTs to the T1w-gd sequence, considered as the reference.

As some differences occurred in the skull stripping between the 1.5 T and 3 T images for the same MR sequence, an intersection between the two masks was performed. For DATASET2, the method described by Bakas et al. was used for co-registration to recover the spatial domain in which the segmentations were performe [41,47]. Brain masks provided by Bakas et al. were applied for skull stripping. In both cases, MR images were finally normalized using 3 different methods (Nyul, WhiteStripe, Z-Score).

The Z-Score method normalizes image histograms by subtracting (µ brain ), corresponding to the mean intensity value of the considered ROI (here, the brain), from each voxel intensity I(x) and dividing the result by the standard deviation of the ROI (σ brain ):

I Z-Score (x) = I(x) -µ brain σ brain (4.1)
The WhiteStripe method normalizes image intensities by subtracting (µ ws ), which corresponds to the mean intensity value of the normal-appearing white matter (NAWM), from each voxel intensity I(x) and dividing the result by the standard deviation of the NAWM (σ ws ) [17].

As conventionally applied in the literature, the "white stripe" region was defined automatically in this work, using a threshold in intensities, corresponding to ± 5% of (µ ws ).

I WhiteStripe (x) = I(x) -µ ws σ ws (4.2)
Nyul's method corresponds to piecewise linear histogram matching [15]. The normaliza- Note that for the normalization process, no tumor exclusion from the brain mask was applied.

More details about intensity normalization methods can be found in the original papers [15][16][17]. The code used in this paper as well as details about the algorithm implementation [48] are available at https://github.com/jcreinhold/intensity-normalization.

Segmentation

A unique ROI including the tumor and peritumoral oedema was considered. These ROIs For DATASET2, the labelled regions supplied by Bakas et al. were merged.

Feature extraction and grey-level discretization

The open-source Pyradiomics package (version 2.1.2) was used to extract 18 first-order statistics and 73 textural features from the segmented tumor regions of both DATASETS [49]. The 5 texture feature classes were based on the grey-level co-occurrence matrix (GLCM, 22 features), grey-level run length matrix (GLRLM, 16 features), grey-level size zone matrix (GLSZM, 16 features), neighbourhood grey tone difference matrix (NGTDM, 5 features) and grey-level dependence matrix (GLDM, 14 features). Except for 4, all the features conformed to the definition provided by the Imaging Biomarker Normalization Initiative (IBSI) [50]. All the features used in this study are listed in Supplementary Data.

To assess the impact of the intensity discretization method on textural features, two approaches of grey-level discretization commonly used in the literature were implemented.

The FBS method assigns the same bin for every voxel intensity corresponding to the bin width w b . It is defined as follows:

X d,k = X gl,k w b - X gl,min w b + 1 (4.3)
where the minimum intensity in the ROI, X gl,min is subtracted from intensity X gl,k corresponding to the intensity of voxel k, and divided by the bin width w b .

X gl,min w b + 1 ensures that the grey-level rebinning starts at 1.

The FBN method discretizes every voxel intensity from an ROI to a fixed number of N g bins. It is defined as follows:

X d,k =    N g X gl,k -X gl, min X gl,max -X gl,min + 1, X gl,k < X gl,max N g , X gl,k = X gl,max (4.4)
where N g corresponds to the fixed number of bins between X gl,min and X gl,max , which are the minimum and maximum intensities of the ROI, respectively.

To correctly analyse the impact of grey-level discretization on pre-processed images on which the intensity ranges can be different, a scaling factor was computed for the FBS method, as shown in Eq. 4.5:

F BS = 1 F BN × mean Range (4.5)
where mean Range corresponds to the mean of the intensity intervals computed for all patient ROIs for one MR sequence. For the two datasets, 8 different bin numbers were applied: 8, 16, 32, 64, 128, 256, 512 and 1024.

Data analysis

R software (version 3.6.0) was used for the statistical analysis. Regarding DATASET1, JSD was used to compare each pair of intensity histograms before and after normalization [51].

A one-way analysis of variance (ANOVA) test was conducted to compare JSD values among the normalization methods. If the ANOVA test was statistically significant, a subsequent pairwise post hoc Tukey's multiple comparison test was performed. For both tests, a P value < 0.05 was considered significant. The CCCs and ICCs were computed to assess the stability of first-order and textural features across the two acquisitions before and after normalization (Supplementary Data). There are currently no conclusions on the optimal thresholds to be used for ICCs and CCCs. In the literature, the most commonly used values are 0.8 for the ICC and 0.85 to 0.9 for the CCC [52,53]. Lecler et al. [52] showed in 2019 that a CCC threshold of 0.9 overrides the value imposed by the ICC. Thus, it was concluded that a too-restrictive threshold could lead to loss of valuable information. In this work, radiomic features were defined as robust if the ICC and the CCC were > 0.8.

DATASET2 aimed to evaluate the usefulness of intensity normalization and to define the optimal grey-level discretization for a tumor grade classification task. Five widely used classifiers were implemented based on the scikit-learn library version 0.20.365. These included random forest, naïve Bayes, logistic regression, support vector machine and neural network multi-layer perception classifiers. Default parameters were chosen to prevent overfitting. Multiple classifiers were used to avoid limiting the conclusions to a single machine learning model.

Moreover, a five-fold stratified cross-validation was adopted. In all cases, feature values were normalized using the Z-Score method within the cross-validation. The average values of the balanced accuracies and the ROC-AUC and corresponding 95% CIs evaluated using the five left-out folds of the 5 machine learning models were reported. For the 95% CIs, bootstrapping including 1000 iterations was applied. Balanced accuracy is a performance metric that should be preferred to accuracy in the case of imbalanced datasets [54]. Model 1 included first-order features alone. Model 2 was based on textural features only. The added value of the combination of the two types of features was analysed in model 3. Model 4 included only features defined as robust, i.e., having both an ICC and a CCC > 0.8 in the DATASET1 experiment.

Results

Impact of the intensity normalization method on histograms and first-order features

Jensen-Shannon divergence (JSD) values showed significant differences (P < 0.001) related to the intensity normalization process for both T1w-gd and T2w-flair sequences (DATASET1).

On post hoc analysis, significantly higher JSD values were found when comparing images without normalization to Nyul (P < 0.001), WhiteStripe (P < 0.001) and Z-Score (P < 0.001) pre-processed images (Table 4.3). The numbers of first-order features defined as robust between the two acquisitions, depending on the normalization method, are summarized in

Impact of the intensity normalization method and grey-level discretization on textural features

Fixed bin number (FBN)

Fixed bin size (FBS)

When an absolute discretization was adopted, all normalization methods improved the number of robust features compared to no normalization, irrespective of the MR sequence (Fig. 4.3C,D -DATASET1). A higher number of bins was often associated with a higher number of robust features in both T1w-gd and T2w-flair MRI sequences. In particular, a maximum increase of 30 percentage points was reported for the T1w-gd sequence when the number of bins varied from 8 to 1024 (no normalization). For the T1w-gd sequence, considering a number of bins equal to 32, the Nyul, WhiteStripe and Z-Score methods achieved 33%, 21% and 49% robust features, respectively; regarding the T2w-flair sequence, these values were equal to 55%, 48% and 34%, respectively, for the same bin size.

The use of an MR intensity normalization method significantly improved the balanced ac-curacy in DATASET2 for the T1w-gd sequence (P < 0.001 -Fig. 4.4). At 32 bins, the mean balanced accuracy for tumor grade classification using only textural features from the T1w-gd sequence (model 2) was equal to 0.68 (95% CI 0.62-0.72) without normalization (Fig.

4.4C

). The same metric reached 0.76 (95% CI 0.74-0.79, P < 0.001), 0.76 (95% CI 0.73-0.79, P < 0.001), and 0.78 (95% CI 0.75-0.81, P < 0.001) when the Nyul, WhiteStripe and Z-Score methods were applied, respectively. Absolute improvement was poor for the T2w-flair sequence and did not exceed 0.04 for comparisons of no normalization and the Z-Score method (Fig.

4.4D

). No significant differences were observed between the different bin numbers for T1w-gd (P = 0.909) and T2w-flair (P = 0.597) sequences.

Performance comparison of different classification models

Discussion

Radiomics relies on the extraction of features from multimodal imaging, aiming to improve patient care. Although acquisition parameters strongly affect the content of MR images, only some recent studies have specifically focused on the impact of MRI preprocessing methods on radiomics features [10,24,55]. Here, we investigated the impact of three different intensity normalization approaches combined with two grey-level discretization methods on brain MRbased radiomics. In a majority of studies, FBS has, in fact, been presented as the default discretization method based on published PET/CT results [19,56]. This conclusion is relevant for quantitative or semi-quantitative modalities (e.g., HU in CT, SUV in PET) for which intensities have a physical meaning. In MRI, intensity values strongly depend on acquisition parameters, making the generalization of radiomics models even more challenging [22]. Recently, the IBSI has proposed recommendations for each imaging modality [50]. For MRI, a relative discretization is recommended to account for the variable intensity ranges.

First, we demonstrated that the use of an intensity normalization step improves the robustness of the first-order and FBS-based textural features using DATASET1 (Table 4.4 and Fig. In contrast, the Z-Score method is the simplest to implement, requires only a short computation time and is the most robust method because it considers all the voxels inside the brain mask. This latter produces very good results in terms of classification performances (Figs. 4.2, 4.3, S4.1 and S4.2) independent of the MR sequence and the grey-level discretization method, even though no statistical significance was achieved. Overall, normalization has a greater positive impact on the T1w-gd sequence than on the T2w-flair sequence. This is mainly because the intensity range of raw MR images is, on average, 5 times lower on T2w-flair images than on T1w-gd images. With the additional use of a grey-level discretization step for textural feature computation, intensity normalization is mandatory when absolute discretization is preferred for T1w-gd images (Fig. 4.4C). Classification performances obtained on DATASET2 highlight that intensity normalization is not needed when relative discretization is applied, making the preprocessing steps of skull stripping and intensity normalization unnecessary (Fig. 4.4A,B).

The evaluation of the impact of the number of bins for discretization is not trivial. Even if high numbers of bins increase feature robustness in the majority of the cases (Fig. 4. and intra-observer reproducibilities of textural features extracted from 6 MR sequences [23] based on manual and automatic segmentations. FBS was shown to be associated with a higher number of reproducible features based on a combination of ICCs and CCCs. In this study, the authors did not normalize the intensities before feature extraction, but they also did not limit the conclusions to a selected range of bin sizes or numbers. In our study, we found that the choice of the number of bins leads to small differences between 16 and 128 bins, with a maximum variation of 10% in the percentages of robust features (Fig. the most important step in reducing inter-machine effects. Compared to our study, they did not analyse the impact of different methods of normalization or discretization. Moreover, the comparison of their results to ours is difficult, as no interplay effect of the different preprocessing methods was analysed. In addition, there was no use of "skull stripping" prior to the application of intensity rescaling, which should have been a mandatory step [60]. Finally, this comparison is also challenging due to different cohorts and tasks applied.

Additional studies are awaited to confirm our results, which also need to be validated in other tasks. Of note, cross-validation was used to assess classification performances. Even if the use of an independent test set would have been preferable, the various train-test partitions combined with a bootstrapping strategy allowed us to draw conclusions efficiently. Regularization methods will have to be implemented in future studies to decrease the risk of overfitting.

In addition, only anatomical MR sequences have been considered. These images are, however, the conventional sequences for radiological assessment of cerebral lesions; the use of more quantitative functional imaging is still sparse in clinical practice. In this study, a unique ROI was delineated; thus, the choice of the ideal number of bins can be influenced by the sharpness of the intensities at the border of the lesion. As the number of voxels included in the tumor was negligible compared to the number of voxels in the whole brain (i.e., the volume of the tumor was equal to 7.5 ± 3.7% of the whole brain in DATASET2), no tumor exclusion was applied during the normalization process. This assumption could have biased, to a limited extent, the implementation of the normalization algorithms. In the second experiment, in which a classification task was studied, the results from DATASET1 regarding feature reproducibility were considered for feature selection in model 4. In DATASET1, a narrow set of acquisition and reconstruction parameters was investigated and compared to real-life dispar-ity, emphasizing the need for additional studies. Finally, some preprocessing step parameters, such as bias field correction and spatial resampling, could have affected comparisons. These two preprocessing methods have still been used in a large number of published studies that have demonstrated their importance for the robustness of features [44,45,61]. Recently, a compensation method to pool radiomic features from different centres has been suggested.

This data-driven post-processing method, called ComBat [62], seems to be able to harmonize radiomics data a posteriori. Initially proposed to correct batch effects in genomic studies, Com-Bat has demonstrated its effectiveness in PET [63] and CT [64]. The next step will consist of comparing ComBat with the preprocessing methods described in this article.

In conclusion, a standardized preprocessing pipeline is recommended for brain tumor radiomics analyses. For models based on first-and second-order features, the combination of Z-Score normalization and absolute discretization seems to be the best of the methods tested.

For works that consider only second-order features, the relative discretization without prior intensity normalization seems to be sufficient. Even if the bin number for the discretization has a small impact on classification performances, 32 bins appear to be a good compromise when T1w-gd and T2w-flair sequences are considered. The preprocessing methods used must be described in detail in the published papers to achieve reliable radiomics-based machine learning models. Such a pipeline will be pivotal for the implementation of large-scale multicentric studies and may pave the way for the development and validation of MR-based radiomics biomarkers.

Supplementary information

Data analysis

To estimate the robustness of features to MR change, the Concordance Correlation Coefficient (CCC) and the Intra-class Correlation Coefficient (ICC) were used [65]. ICC estimates the magnitude of the relationship between variables. ICC also considers rater bias. It ranges between 0 and 1, indicating null and perfect reproducibility respectively. In order to determine ICC for machine variability, which reflects feature variations for the same subject, a two-way mixed effect model, defined by McGraw and Wong [66], was used:

ICC = M S R -M S E M S R + (k -1)M S E (4.6)
Where M S R corresponds to the mean square for rows (subjects), M S E to the mean square for error and k to the number of measurements.

CCC is another measure of agreement which, unlike ICC, does not assume a common mean [12][13][14][15][16][17][18]. However, radiomic features are well recognized to be vulnerable to differences in MR imaging [19][20][21][22]. This weakness is hampering the integration of data from different centers in predictive analysis and/or machine learning (ML) algorithms and the construction of subsequent robust models. To voluntary ignore scanner-induced data heterogeneity, most neuroimaging studies have traditionally been limited to datasets from a single center [23]. In recent years, there has been an increasing trend towards the collection and sharing of neuroimaging data through the establishment of multi-institutional databases [24,25]. This effort to collect data covering a wide range of machine types and broad spectrum population (demographic) is essential for the development of diagnostic and prognostic biomarkers to enable robust translation of research into clinical practice. As a consequence, there is a strong and pressing need for standardization and/or harmonization [26,27]. Compensation for these effects can be seen at three different levels: (i) Image acquisition, (ii), Image processing and, (iii) Feature adjustment.

Figures and Tables

One of the solutions is to consider standardized procedures regarding imaging protocols to reduce the institutional effect and obtain more constant images [28]. However, this solution seems difficult to envisage on a large scale, since it requires convincing a large majority of centers to adopt the same protocol, which can be long and complicated to enforce. Moreover, the issue would remain for retrospective studies and would not negate the manufacturer/device effect either. The second solution is to consider a well-defined image processing pipeline that can harmonize images post-acquisition. A classical image processing process includes at least a bias field correction, an isotropic voxel resampling, a skull stripping and finally a standardization of the brain image intensities, which can be performed by the Nyùl et al., Hybrid White Stripe or Z-Score methods [29]. This approach is suitable for deep learning segmentation approaches to feed the network and shows promising results in radiomics studies [29,30]. Recently, deep learning techniques using fully convolutional neural networks for contrast harmonization have emerged, but require an overlap cohort of patients scanned with the respective protocols [31].

Besides, this may require patients to be reimaged, which may be impractical or impossible, or may limit the training data. Finally, the third solution consists in applying a correction directly to the derived radiomic features without any pre-correction on the image. The breakthrough approach in this category is ComBat, a batch effect correction tool originally used in genomics [32] and first adapted to harmonize diffusion MRI [33]. Recent studies applied to MRI showed that this method would lead to an efficient reduction of discrepancies in values 135 of radiomic features between centers and improve the accuracy of experiments with data from multiple scanners [34,35]. This method has, however, several drawbacks as the need for a representative statistical sample to estimate the batch effect parameters. Futhermore, this method does not meet two essential criteria for a machine learning applicability. First, the correct application of a feature scaling method requires applying the estimators learned on the training set to the test set. Second, if we want to see the applicability of a model and its translation into clinical practice, it has to be generalizable and make predictions on a single image from a site or scanner that was not part of the training set. In addition, it may not always be simple to define the notion of "batch effect", since this effect can be seen at different levels: i) site/center, ii) scanner, iii) variation in scanner parameters. Ideally, the determination of batch effect labels should correspond to the grouping of imaging data with similar image qualities, often associated with similar acquisition and reconstruction parameters.

We thus developed a method that should allow the applicability of feature adjustments in a highly multi-centric radiomics machine learning context. This method called AutoCombat allows a sample to be assigned to a specific batch by a constrained clustering method. In this method, the batch label can be defined by metadata summarizing the scanner and the associated acquisition characteristics (DICOM tags) or by image quality metrics measurements.

In this study, we aimed first to answer the question whether the harmonization strength of a classical preprocessing method is comparable to the ones of the ComBat and AutoCombat.

Second, we studied their respective performances on a classification task in machine learning.

Material and Methods

Dataset

Preoperative scans previously extracted from the The Cancer Imaging Archive (TCIA) [36] including both glioblastoma (TCGA-GBM, n=135) and low-grade-glioma (TCGA-LGG, n=108)

were considered [24,37,38]. All methods were performed in accordance with the relevant guidelines and regulations (Declaration of Helsinki). Selected DICOM files were pre-and post-contrast T1-weighted (T1w and T1w-gd), T2-weighted (T2w), and T2 Fluid-Attenuated Inversion Recovery (FLAIR) volumes (T2w-flair). These data presented high heterogeneity as they were collected from 11 different centers. Data from the three centers showing the lowest sample numbers were removed to ensure at least 5 samples in the training set after stratified data splitting [39,40] (see section Batch effect adjustement method and subsection Empirical Bayes method). The other criteria was the availability of sex and age information, which was not the case for one patient. At the end, 232 samples were kept corresponding to 125 GBM and 107 LGG from 8 different centers. Table 5.1 summarizes centers and associated numbers of patients included in our study. Notes.

TCGA: The Tumor Genome Atlas

Batch effect adjustement method

The ComBat harmonization method was originally designed for the field of genetics to overcome the "batch effect" observed in microarray analysis [32]. The term "batch effect" refers to nonbiological noise which affects samples to be analyzed. It can be due to diverse factors such as operator's methodology, sequencing technology, time of day of measurements, etc., and makes difficult direct comparisons. In radiomics studies, the different "batches" can be related to different imaging protocols or devices. One of the advantages is that ComBat can harmonize radiomic features by considering the batch as a covariate, while preserving the variance due to other known covariates such as gender or age for example.

Model-based location/scale adjustments

ComBat harmonisation is derived from the location (mean) and scale (variance) (L/S) method, in which the main idea is to transform the data of each batch so that they end up with the same mean and/or variance and thus eliminate the error introduced by the differences between the batches. For example, let Y ijf represents the value corresponding to feature f for sample j from batch i. The L/S adjustment method models the feature's value as:

Y ijf = α f + Xβ f + γ if + δ if ε ijf , (5.1)
where α f is the overall feature value, X is a matrix for the covariates of interest, and β f is the vector of regression coefficients corresponding to X. The error terms, ε ijf , can be assumed to follow a Normal distribution with expected value of mean zero and variance σ 2 f . The γ if and δ if respectively represent the additive and multiplicative batch effects corresponding to batch i for feature f. The estimation of these two terms allows to determine the value adjusted for the batch effect using the following equation:

Y * ijf = Y ijf -α f -X β f -γ if δ if + α f + X β f , ( 5.2)
where α f , β f , γ if and δ if are estimators of the parameters α f , β f , γ if and δ if .

Empirical Bayes method

ComBat method uses an empirical Bayes (EB) framework to improve the variance of the parameter estimates γ if and δ if . There exist both a parametric and non-parametric approach.

We give here a concise explanation about the parametric one, and additional details can be found in the original publication [32]. This method has the advantage to be robust even with small batch sizes. The minimum number of samples in each batch has been defined as 5 [39,40].

The first step in EB is to standardize the data by features to ensure they have a similar overall mean and variance. The standardized feature value Z ijf is given by:

Z ijf = Y ijf -α f -X β f σ f (5.3)
where Y ijf , α f and σ f are respectively the raw feature value, feature-wise mean and standard deviation estimates. X β f denotes the model's possible non-batch related covariates and coefficients. The standardized feature value Z ijf is assumed to be normally distributed accord-

ing to Z ijf ∼ N γ if , δ 2 if
, where the batch effect parameters are assumed with the following prior distributions

γ if ∼ N Y i , τ 2 i and δ 2 if ∼ Inverse Gamma (λ i , θ i ).
The moments method is used to estimate the hyperparameters γ i , τ 2 i , λ i , θ i empirically from standardized data. The EB estimates for the batch effect parameters, γ * if and δ 2 * if , given by the conditional posterior means, respectively, can be calculated from the distributional assumptions. Henceforth, the EB batch effect adjusted data γ * ijf can be calculated in a similar way to Eq. 5.2 as follows:

γ * ijf = σ f δ * if Z ijf -γ * if + α f + X β f (5.4)
The Combat method, as described in the original paper, centers the data on the overall, grand mean and pooled variance of all samples. This results in a harmonized location shifted data matrix that no longer corresponds to any initial batch which can lead to a loss of physical meaning. A modified version proposed that a reference batch label can be chosen to shift each sample to the mean and variance of this reference [41]. This is accomplished by simply changing the estimates of the standardization mean and variance, α f and σ f , to batch estimates, α if and σ if .

Thus, as part of the development of a machine learning model, ComBat's model parameters (e.g., α f , σ f , β f , γ * if and δ 2 * if ) learned from a training set should not involve any test set data, but should be stored for later transfer to unseen data.

AutoComBat approach

We proprose in this section, AutoComBat, based on the hypothesis that batch labels can be deduced from image metadata (DICOM tags) and/or image quality metrics.

DICOM tags and image quality metrics extraction

In the present work, two main classes of information were extracted from the DICOM files. Table 5.2 summarizes the DICOM tags of interest and the image quality metrics deduced from the data matrices themselves with their mathematical formulation.

• Metadata: Information extracted from the header of the DICOM file describing the MR device and acquisition parameters (i.e., Magnetic field, manufacturer, voxel sizes, ...). In total, 15 were considered (Table 5.2).

• Quality metrics: These metrics have recently been proposed to quantify the batch scanner effect in MRI as well as to detect artifacts [42]. This class includes statistical measures (e.g., range, variance, coefficient of variation) as well as second-order statistics and filterbased measures (e.g., contrast per pixel (CPP), entropic focus criterion (EFC), signalto-noise ratios corresponding to different regions). In total, 15 were considered (Table 5.2).

Determination of batch effect labels using clustering

Based on the extracted information, AutoComBat uses K-Means clustering with constraints [43] on the minimum cluster size to ensure the condition that ComBat uses a statistically representative sample from each identified batch. We set the minimum cluster size to 5 samples in this work as demonstrated to be statistically representative in ComBat [39,40], but this value can be changed in our approach. The features used to determine the batch effect were processed in two different ways, depending on whether they were discrete (Manufacturer, model name) or continuous (Voxel sizes, echo time, ...). The discrete variables were one-hot encoded, and a NaN category was added to account for the case where no missing value was encountered during training but could be experienced during the prediction phase. The continuous variables were treated by substracting the mean and scaling to unit variance. The K-Means constrained clustering was able to take into account missing values. For this, the missing values were initialized to the mean of their column, and an expectation-maximization (EM) algorithm was executed until convergence of stability in the label prediction. We set the threshold for the missing features to 25%, which means that for a given feature, 75% of the 139 5 Adaptation of the ComBat method for MRI radiomics PCV Percent coefficient of variation: coefficient of variation of the foreground for shadowing and inhomogeneity artifacts [48] σ F µ F CPP Contrast per pixel: mean of the foreground filtered by a 3×3 2D Laplacian kernel for shadowing artifacts [49] mean(conv2(F, f 1 )),

f 1 =    -1 -1 -1 -1 8 -1 -1 -1 -1    PSNR Peak
F i F max ln[ F i F max ], F max = i,j F 2 (i, j)
FBER Foreground-background energy ratio for ringing artifacts [54] median

(|F | 2 ) median(|B| 2 ) F is Foreground intensity voxels (F = n i=1 v fi n ) with v fi , i th foreground voxels. B is Background intensity voxels (B = n i=1 v bi n ) with v fi , i th background voxels.
F P is Foreground random patch voxels (n=5000, with a 5x5x5 patch-size). B P is Background random patch voxels (n=5000, with a 5x5x5 patch-size) data must be present for training. Furthermore, we added the possibility to embed a feature reduction before clustering, either with Principal Component Analysis (PCA) [44] or Uniform Manifold Approximation and Projection (UMAP) [45]. To determine the optimal number of clusters, the elbow method of the Yellowbrick library was used [46]. The elbow method runs the K-Means constrained clustering on the dataset for all possible values of K. Then, for each value of K, a metric is computed to evaluate quality of the clusters. By default, the scoring metric is the distortion, which calculates the sum of the squared distances from each point to its assigned cluster center. However, two other metrics can be used: the Silhouette score and the Calinski-Harabasz score. The Silhouette score calculates mean ratio of intra-cluster and nearest-cluster distance, while the Calinski Harabasz score calculates the ratio of dispersion between and within the clusters. The optimal value of K was determined automatically using the "knee point detection algorithm" which allows to determine the elbow, i.e., the point of inflection [47]. To use a reference batch in Combat, our approach estimated the most relevant cluster for this role as the one with the lowest within-cluster sum-of-squares (WCSS), defined as the sum of the squared distances between each member of the cluster and its centroid.

We implemented ComBat and AutoComBat in Python compatible with scikit-learn [55] to facilitate subsequent machine learning projects. Combat can use EB or more simpler L/S method. When EB is chosen, adjustments can be done in a parametric or non-parametric way. A reference batch can also be set in case the user prefers to use the modified version of ComBat. AutoCombat benefits from the ComBat inheritance. The code is available at the following address: https://github.com/Alxaline/ComScan.

To extract the image quality metrics and the metadata from the DICOM files, we have also developed a Python package available at the following address: https://github.com/ Alxaline/QAnT, mainly based on the image quality metric available in MRQy [42] (https:

//github.com/ccipd/MRQy). The main difference is that we extract the metrics directly per 3D patch and not by an average on 2D slices. Moreover, the metadata extraction is fully customizable, and the code has been accelerated by multiprocessing.

Image processing approach

Image preprocessing is an alternative approach to reduce the batch effect by applying various correction steps prior to the extraction of the radiomic features. The pipeline that we used on the DICOM files included 4 steps: bias field correction, coregistration (voxel size resampling), skull-stripping and z-score normalization [29]. First, the N4 bias field correction was applied for all the 4 MRI sequences considering the head area [4]. Then, for each patient, the T1w sequence was registered to the T1w SRI-24 atlas reoriented to the LPS (left-posterior-superior) coordinate system [56] using an affine transformation and a B-Spline interpolation. The resulting image, T1w reg , had a 1x1x1 mm 3 voxel size. The other MR sequences, i.e., T1w-gd, T2w and T2w-flair, were co-registered to T1w reg . The modalities were then skull-stripped to keep only the brain [57]. Finally, the z-score normalization was applied to the brain voxels by setting the mean to zero and the variance to one.

The package used for preprocessing is the cBrainMRIPrePro Python package available at the following address: https://github.com/Alxaline/cBrainMRIPrePro. This in-house package uses ANTsPy [58] and HD-bet [57] and enables the preprocessing of anatomical MR images in the form of a straightforward pipeline.

Radiomic feature extraction

The extraction of radiomic features was performed using the Python library Pyradiomics [59] v3.0. width. The intensity ranges from the whole patient dataset were used to calculate the optimal bin width leading to 32 bins, which was a reasonable balance [29].

Experiments and analysis

The experiments first sought to assess the strength of harmonization of each method on the radiomic features. Next, we evaluated the impact of these 3 methods on a problem of classifying brain tumors into two different categories: GBM and LGG.

For the two experiments, we separated the data into three sets: Training, Validation, and

Testing. This strategy allowed us to avoid overly optimistic results due to overfitting. Also, this strategy was prefered to k-fold cross validation to meet the requirements of the ComBat method, whose philosophy is to have at least 5 samples per center in the training set and due to the fact that the test cannot contain a sample of a batch label that has not been seen in the training phase. Also, a leave-one-out cross-validation strategy was not considered due to the computational cost. Thus, our validation set was used to maximize the optimization metric,

Experiment 1: Harmonization strength

White matter areas are distinguished by vast homogeneous regions with only minor variations in intensity between patients [60]. We exploited this consideration to hypothesize that the variation of radiomic feature values extracted from this area should be minimal between patients when the machine effect is reduced. To that end, a label map was created for every patient using Atropos which is a finite mixture modeling (FMM) segmentation approach [61].

Atropos made possible to extract three brain regions: the cerebral spinal fluid (CSF), the grey matter (GM) and the white matter (WM) automatically. The mask that defined the area to be labeled corresponded to the brain mask subtracted by the total tumor mask. The whole tumor corresponded to the union of the enhancing tumor (ET), necrotic tumor (NEC), non enhancing tumor (NET) and peritumoral edema (ED), as defined by Bakas et al. [24]. The Atropos label maps were all manually verified by an image scientist (A.C). Thirty randomly located 8x8x8 mm 3 patches were considered in the segmented white matter region as regions of interest (ROI). All ROI, i.e., the whole tumor and the white matter patches, were also remapped in each space of each raw MRI sequence. To consider only the batch effect and preserve biological associations in ComBat and AutoComBat, gender, age, and tumor type were kept as covariables. Age was treated as a categorical variable and two categories were considered: above and below 50 years of age, since they have previously been shown to generate differences in white matter MR signal [62]. This was necessary to meet the minimum sample size of 5 per category. For ComBat and AutoCombat, optimization was performed for each MRI sequence with a grid search to sift through each combination of hyperparameters. The number of combinations evaluated was 27 and 54 for ComBat and AutoComBat, respectively.

The parameter space used for the grid search is given in Table S5.1. The strength of the correction was assessed by the minimization of the objective function described in Eq 5.5 which corresponds to the average of the relative standard deviation (RSD) over whole set of features.

L = 1 n n f =1 RSD f = 1 n n f =1 σ f | µ f | × 100 (5.5)
Where n is the total number of features, σ f is the standard deviation and µ f is the mean of feature f . The 95% confidence interval (CI) of the RSD was computed for each feature using bootstrapping with 1000 rounds.

Experiment 2: Impact of harmonization on a classification task

We applied the different harmonization methods to a tumor grading task (LGG vs. GBM) and evaluated their respective performance using several ML algorithms. These algorithms were implemented to verify that performance was not related to the type of algorithm used.

The different algorithms that were selected were C-Support vector classification (SVC), k-nearest neighbors vote (KNN), logistic regression (LR), random forest (RF) and eXtreme Gradient Boosting (XGBoost). These classifiers are among the most used for supervised classification tasks and reflect the possible classification approaches with linear, non-linear, and ensemble classifiers. In the machine learning pipeline, min-max normalization of the radiomic features to the range 0-1 was included. Since the set of optimization spaces for the classifiers, ComBat and AutoCombat was huge, a Bayesian optimization with a Gaussian process was implemented. During Bayesian optimization, 120 parameter settings were sampled and Balanced Accuracy was considered as the optimization metric. The parameter space used for Bayesian optimization is given in Table S5.2. We reported results for 5 different metrics:

Area Under the Receiver Operating Characteristic Curve (ROC AUC), Balanced Accuracy, F1 score, Precision, and Recall.

The scikit-learn [55] v0.23.2 library and the scikit-optimize [63] v0.8.1 library were used for the ML and Bayesian optimization pipelines respectively. A method was considered as the best method (Top) if its RSD was the lowest, not included in the 95% CI of the raw data, and its 95% CI was not included in that of other methods. When multiple methods had overlapping CIs, they were all counted. In addition, the total number of significant features per method compared to the raw method is given. Comparing WM RSD of all methods to raw RSD, preprocess showed the highest harmonization capabilities for T1w-gd (82%), while it was ComBat for T2w-flair (79%), very similar to preprocess (78%). For T1w and T2w, ComBat was found to be superior (96% -T1w; 92% -T2w), followed closely by the preprocess method (93% -T1w; 85% -T2w). For AutoComBat, these values varied according to the characteristics used for the clustering (metadata and quality metrics, metadata only, quality metrics only) and the sequence type, but never outperformed Combat, except for T1w-gd. For example, using all available, metadata only, and quality metrics only features for clustering, the number of significant features showed an improvement of 54%, 62%, 75% compared to raw, respectively. Looking at top features, AutoCombat using QM only obtained the best results

Results

Illustration of the "batch effect"

Evaluation of harmonization strength based on WM

for T1w-gd with 75% of features obtaining the lowest RSD. The preprocess method showed the best performance for T2w-flair with 78% of features. For sequences not shown here (T1w and T2w in supplementary), ComBat was superior, with AutoComBat QM presenting very similar values. These findings on the test set agreed with the validation set, except for AutoCombat (all) in the T2w-flair sequence, which had shown the best performance in the validation set but did not generalize the same way in the test set. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q first order glcm

Features

Relative standard deviation (%) -Log scale Method q q q q q q AutoComBat (All)

Notes.

For AutoComBat, "All" means the use of Metadata and Quality Metrics. QM = Quality Metrics. (min: 0.66, max: 0.73), respectively. Here, AutoComBat (QM) performs worse with a median balanced accuracy of 0.61 (min: 0.52, max: 0.64). The value was equal to 0.61 (min: 0.64, max: 0.65) when preprocessing was applied. For the second-order features, AutoComBat (all)

Tumor grading performance

A

and AutoComBat (metadata) provided the best results with 0.78 (min: 0.67, max: 0.88) and 0.77 (min: 0.73, max: 0.82), respectively. Preprocessing obtained the worst results here with a value equal to 0.60 (min: 0.49, max: 0.66).

In supplementary, these results are also available for validation (Fig. S5.4). Results for additional metrics (F1 score, precision, recall and ROC auc) are also available in supplementary (Figs. S5.5-S5.12). 149 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q T1w T1w-gd T2w T2w-flair

First order

Second order 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.

Discussion

The aim of this study was to analyze the impact of the harmonization approach in an MR-based radiomics context, i.e., either upstream as image processing or downstream after the extraction of radiomic features. In addition, a clustering method that aims to automatically define the batch to which an image should be assigned, using information from the DICOM file metadata and/or quality metrics deduced from the raw images themselves, was proposed. In this work, a highly heterogeneous dataset including conventional MRI sequences (T1w, T1w-gd, T2w, T2wflair) from a large number of centers was voluntary considered to evaluate the generalizability of the proposed solution, and both classes of radiomic features (first and second-orders) were analyzed separately. Two types of experiments were conducted to quantify the impact of the harmonization strategy. In a first time, it was analysed on its ability to decrease RSD of radiomic features extracted from patches of the white matter over the whole patient cohort.

Second, a clinical grading task (HGG vs. LGG) was considered.

Batch assignment is not a trivial task when data are very heterogeneous as illustrated in Figure 5.2, as no consensus international guidelines exist regarding acquisition parameters in brain oncology. Indeed, spatial resolution, signal to noise ratio and contrast to noise ratio strongly depend on field gradients, B0 magnetic field, pulse sequence and its parameters in MR [64,65].

Clustering based on DICOM file metadata and/or quality metrics was proposed with the goal to minimize an objective function corresponding to the average of an RSD corresponding to 91 radiomic features extracted from WM. Image quality metrics have been introduced in ad-dition to conventionally used DICOM tags to facilitate batch assignment in case of lack of information in the DICOM header or when the number of patients considered for a certain type of acquisition is too low. AutoComBat reveals coherent batch allocations as illustrated in Figure 5.4, without a total scattering of the centers in the different clusters. Considering the four sequences (results only showed for T1w-gd), image size (rows and columns), voxel size, magnetic field and flip angle parameters were shown to have the highest weights in the clustering. Although weights were dependent on the sequence considered, the quality metrics that most often appeared with significant weights were CJV, PCV, EFC, SNR and variance.

Applied to a tumor grading task, our methodology showed different results depending on the sequence and class of features considered (first or second-orders). For the T1w-gd sequence, often considered as the most informative in neuro-oncology, image preprocessing yields the best results with a median balanced accuracy equal to 0.87 considering the first order features only while others methods range between 0.72-0.75. For second-order features, preprocessing also gives the best result with a median balanced accuracy equal to 0.87. This result is the best over all combinations of sequences and harmonization methods and is generalizable, i.e., with no discrepancies between the validation and the test. However, preprocessing underperforms the other strategies for the others sequences. AutoComBat (based either on Metadata or all features) has shown interesting properties, especially on the T2w-flair sequence with the best median value of balanced accuracy equal to 0.77. For this sequence, AutoComBat has demonstrated a good generalization compared to other methods, i.e., constant performance between the validation and the test sets (only 5% percentage difference). Conventional Combat was the best method for the T1 sequence with a median balanced accuracy of 0.81 and 0.73 for the first and second-order features, respectively.

In the literature, only a few works have been dedicated to the use of ComBat in MRI and more specifically applied to radiomics. The first work, which tested the ComBat approach for a radiomic application in the case of MRI, used a rescan on two separate machines, with the unique difference being the magnetic field (1.5T vs. 3T) [34]. They evaluated the method on with almost identical results: we have shown that 68% and 79% of the features for the T1wgd and T2w-flair, respectively, yielded a harmonization strength augmentation in the WM.

We think that the strength of ComBat is that it should learn some sources of variabilities, thus bypassing some preprocessing steps. Nevertheless, it remains interesting to include image resampling before features extraction so that the texture features remain rotation invariant and the correction of artifacts as bias field correction.

The interest in the ComBat approach was also evaluated in the recent study of Da-Ano et al. [35], where four versions of the nonparametric ComBat were compared in their ability to harmonize radiomic features in a multicenter context, including two clinical datasets. The first dataset was composed of 119 patients suffering from locally advanced cervical cancer and contained MR and PET images from three different centers. The second involved 98 patients with locally advanced laryngeal cancer from 5 centers who underwent contrast-enhanced computed tomography. Among the four versions, one version identified a reference center, in addition to the conventional version, on which radiomic features were transformed. The other two versions used conventional versions, but with the addition of Boostrap and Monte-Carlo strategies for improved robustness in the estimation. They showed that all four versions of ComBat showed a contribution in removing machine differences, improving the prediction performance of the given outcome. In addition, the version using a reference site gave the best results. For example, Modified ComBat resulted in a 6% improvement in balanced accuracy compared to untransformed data for the random forest algorithm in the prediction of local failure in locally advanced cervical cancer. When using ComBat in the 5 centers dataset, they were confronted with the fact that the machine parameters were very heterogeneous. Following this observation, they would have had to manually assign a batch to each image, leading to more than 15 labels, which they did not consider realistic due to the limited number of patients. We have shown from Figures 5.2 and S5.1 that there is no sense to affect to a same batch images coming from a single center but for which devices or acquisition parameters differ. This study, therefore, highlighted the urgent need to define an alternative for batch assignment, as already mentioned earlier. For this purpose, Da-Ano et al. [35] proposed an unsupervised hierarchical clustering technique applied directly to radiomic features. Using this technique, they were able to correctly cluster the patients in the dataset from the three centers with homogeneous acquisition parameters per center mentioned earlier into three different clusters. Only one patient was misclassified. Then, they applied clustering to the dataset with heterogeneous parameters to establish the ComBat "batch" labels. We believe that the direct use of radiomic features extracted from the tumor itself to define a "batch" could be biased by the clinical endpoint and lead to clusters correlated to the outcome. In their case, however, they tested the hypothesis by verifying that each resulting group had a similar percentage of non-responders. Using either information extracted from the DICOM headers and characterizing machine and parameters variability and/or using image quality metrics seems to be a better way to categorize images without any assumption. Another study used the ComBat approach with the goal to develop a model capable of capturing the relationship between image quality metrics and relative volume corrections for each region of the brain [66]. They demonstrated that the tool could reduce systemic scanner variations in new images from unknown scanners. This work supports the notion that identifying the "batch" with data that are irrelevant to the problem we are trying to solve and therefore unrelated to the clinical outcome of interest is promising.

In addition, to propose a generalizable alternative for batch allocation, the present study also gives tracks about the correct use of ComBat in a machine learning process applied to radiomics. We would like indeed to warn the community about the misuse of ComBat in several radiomics studies. This error, which consists in pooling all the data (train, val, test) and applying ComBat, leads to data leakage. In fact, as with any application of a normalization step in machine learning, it is indeed important to normalize data after their splitting to avoid introducing future information into the training explanatory variables (i.e., the mean and variance). Our code available at the following address https://github.com/ Alxaline/ComScan answers this problem by following the philosophy of scikit-learn with a fit and transform function. The hardest part in using Combat is that there are not always ground truths about the batch labels, in particular in the case of very heterogeneous data as it is in a multicentric context. The advantage of using clustering to determine the batch is that it becomes possible to know whether the imaging data not seen during the training stage lies outside the distribution of the training data. This does not solve the generalizability problem in a general way but gives an idea of the space in which the imager must be located for a developed radiomic signature to be applicable.

This study poses some limitations. First, the clustering method was limited to a constrained K-means, but other methods could be considered, such as Density-Based Spatial Clustering of Applications with Noise (DBSCAN). However, the proposed method has the advantage of not requiring to specify a priori the number of clusters, takes as argument the minimal sample, i.e., the smallest number of points needed to form a cluster, and is robust to noise. For AutoComBat, all the potential was not exploited because we were limited by the comparison with ComBat, which necessitates balancing patients between the sets depending on their origin center. For the same reason, we were limited to a simple data splitting strategy and were not able to use cross-validation, which would have limited overfitting. However, we have exploited the full potential of ComBat by exploring its complete hyperparameter space (reference site or not, parametric assumption or not, empirical Bayes strategy or not). We did not consider the discretization step as a variable parameter and have fixed it to a fixed bin width [29,67]. For the classification step, we did not try to establish the best model but put the emphasis on understanding the influence of each strategy on the radiomic features harmonization; that is why 153 we have created separate models with either first-order or second-order features. Furthermore, we did not evaluate the shape features, which can also be affected by the acquisition parameters. Finally, ComBat and AutoComBat should be further investigated for other datasets and other clinical tasks. * For AutoComBat, the number of components were fixed to 2 when a features reduction was applied.

Supplementary information

Table S5.2: Parameters space used for the Bayesian optimization of the different ML models.

ML models Parameters

Name

Notes.

ComBat and AutoComBat corresponds to the space defined in Table S5.1. An exception appears for AutoCombat, where the space of the number of components was in the range of 2 to 5 when a feature reduction technique was used.

Notes.

For AutoComBat, "All" means the use of Metadata and Quality Metrics. QM = Quality Metrics.

158 5.7 Supplementary information q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q first order glcm

Features

Relative standard deviation (%) -Log scale Method q q q q q q AutoComBat (All) 159 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q first order glcm gldm glrlm glszm ngtdm

T1w

T1w-gd 160 5.7 Supplementary information q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q T1w T1w-gd T2w T2w-flair

First order

Second order 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. 161 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q qq q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. .9: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T1w MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q F1 Precision Recall Roc Auc

First order

Second order 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0. emphasis was placed on favoring explainable classifiers to ease translation into clinic.

Materials and Methods

The steps of our study are summarized in Figure 6.1. The minimal size of 2 cm was chosen as GBM are usually > 2 cm at the diagnosis. We therefore wanted to exclude small BM from the analysis, to avoid a bias of size. For BM, we included patients with one or more brain lesions. However in cases of multiple lesions, only the largest was segmented for radiomic feature extraction.

Secondly, a test set was constituted after completion of the model development process in order to evaluate the final performance of the radiomic classifier on unseen lesions. As well, the test set included patients from both centers. Inclusion criteria for the test set were the same as for the training set. All patients included in the test set were required to have solitary lesions so that neuroradiologists were not influenced in their final diagnosis. Exclusion criteria of the study were therefore the same as those of the training set plus patients having multifocal or infra-tentorial lesions. All inclusion and exclusion criteria are summarized in the flow chart (Figure 6.2).

MRI Data

MR acquisitions were performed on the same 3 Tesla MR scanner, even if at two clinical sites.

MRI data included at least a post-contrast (gadoterate meglumine [Dotarem; Guerbet Laboratory]) three-dimensional T1-weighed Fast SPoiled Gradient Recalled (FSPGR) acquisition (post-contrast 3DT1), with the following parameters: repetition time: 10.2 ms; echo time: 3.4 ms; field of view: 22 cm; voxel size: 0.8 mm × 0.8 mm × 1.2 mm. Patients were excluded from this study if other imaging protocols were followed. Post-contrast 3DT1 MR images were only used as inputs of the radiomics classifier. To compare the performance between the classifier and neuroradiologists, clinical conditions were mimicked, and all available sequences of the imaging exam were thus analyzed by the neuroradiologists, as routinely conducted in a clinical setting.

Image analysis

Pre-processing. MR image preprocessing included bias field correction using the N4ITK algorithm [30] from the Advanced Normalization Tools (ANTs) library [31], skull-stripping with the Brain Extraction Tool (BET) of the FSL software (FMRIB's Software Library) [32] and Z-score normalization with a scaling factor of 100. No spatial resampling was performed due to data homogeneity. As well, no noise filtering was applied.

Tumor segmentation. Segmentation of the volume of interest, including the contrastenhanced and necrotic regions, was performed semi-automatically using Olea Sphere © (Olea Medical, La Ciotat, France). These two sub-regions corresponded to Labels 4 and 1 of the BraTS 2012-2016 challenge [33]. Within a region of interest defined by a trained radiologist (AdC, 5 years of experience), threshold-based grey level contouring and manual correction were used for the segmentation so that the volume of interest was carefully drawn along the tumor enhancement.

Feature extraction

One hundred radiomic features were extracted from the 3D MR images using the Python library PyRadiomics 2.1.2 [34] in which the feature definitions are consistent with the Image Biomarker Standardization Initiative (IBSI) [35]. The only exception is that PyRadiomics and IBSI use different definitions of the Kurtosis first-order feature, where Kurtosis is calculated using -3 and +3 in the IBSI and PyRadiomics referentials respectively. For first order features, an intensity shifting of 300 (equal to three standard deviations) was applied to ensure that the majority of the voxel intensities were positive before feature extraction. An absolute discretization with a fixed bin size equal to 37 was chosen [36,37]. This leads to a bin number of 32 considering the mean of the intensity intervals computed for all volumes of interest of patients of the training set (min intensity range: 575 -max intensity range: 2069). Six feature classes were considered: 18 first-order statistics, 14 shape-based features, 22 Grey Level Cooccurrence Matrix features (GLCM), 16 Grey Level Run Length Matrix features (GLRLM), 16 Grey Level Size Zone Matrix features (GLSZM), and 14 Grey Level Dependence Matrix features (GLDM).

Model building

The establishment of the classification model was based on the scikit-learn library version 0.23.2 [38] and included two steps applied to the training set: (1) selection of the ML classifier and feature scaling method and 2) optimization of the hyperparameters. In step 1), a nested crossvalidation was used given the moderately-sized dataset and 144 ML models combining 9 feature scaling methods (No Scaler, MaxAbsScaler, MinMaxScaler, Normalizer, PowerTransformeryeo-johnson, QuantileTransformer-normal, QuantileTransformer-uniform, RobustScaler, Stan-

Results

dardScaler) and 16 classifiers (AdaBoostClassifier, BaggingClassifier, BernoulliNB, Decision-TreeClassifier, ExtraTreeClassifier, ExtraTreesClassifier, GaussianNB, GradientBoostingClassifier, KNeighborsClassifier, LinearSVC, LogisticRegression, MLPClassifier, QuadraticDiscrim-inantAnalysis, RandomForestClassifier, RidgeClassifier, SGDClassifier) were compared. The nested cross-validation considered a stratified 5-fold cross-validation in the inner loop for hyperparameters tuning (grid search strategy) and a stratified 5-fold cross-validation in the outer loop for the evaluation of the performance of the model. In step 2), the model showing the lowest generalization error, as assessed by the balanced accuracy, was kept and a ten-repeated 5-fold cross-validation was performed. In this second step, a grid search method was implemented to optimize the final set of hyperparameters. Mean sensitivity, specificity, balanced accuracy, and area under the receiver operating characteristic curve (AUC) and their associated variances and 95% confidence intervals were calculated as performance metrics. Research spaces for hyperparameter tuning with grid search during nested cross-validation and crossvalidation are described in Table S6.1.

Evaluation on the test set and comparison to human performance

The final model was fitted using the entire training set and its performance evaluated on the test set including 37 patients (21 GBM and 16 BM). Images of the test set were then blindly analyzed by 5 neuroradiologists (R1, R2, R3, R4, R5). Two were neuroradiologists with more than 10 years of experience and 3 were radiology residents with about 6 months of training and practice in neuroradiology. The neuroradiologists had access to all MR sequences acquired in a routine MR imaging protocol, including 3D FLAIR, 2D T2, perfusion imaging, and pre and post-contrast 3DT1 sequences.

Statistics

Sensitivity, specificity, balanced accuracy and AUC were used to assess the diagnosis performance of the radiomic model. We applied a McNemar's test and evaluated its p-value to assess if the differences were significant between the diagnostic performance of the radiomic classifier and the diagnostic performance of the readers. The threshold was set at 0.05.

Results

Patients

267 GBM and 271 BM were pre-selected for the training set, and 71 GBM and 72 BM met the inclusion criteria respectively (Figure 6

Selected Machine Learning classifier

Table S6.2 summarizes the mean balanced accuracies and their associated standard deviations obtained for all tested combinations (scaling method + classifier). Combinations are ranked considering the lowest generalization error. The ML classifier providing the better performance using the nested cross-validation was the logistic regression combined to the power transform yeo-johnson scaling feature method which corresponds to a zero-mean, unit-variance normalization with a power transform applied feature wise to make distribution of each radiomic feature Gaussian-like. To limit overfitting, the classifier encompassed a ridge regression for regularization (l2 penalty assignment) with a C value equal to 0.7. The final logistic regression-based established signature was a combination of the 100 input radiomic features, in which the feature with the highest coefficient in the decision function was sphericity, with a coefficient of 1.48. All other features had absolute coefficient less than 0.96. The 20 predominant features

Results

had absolute coefficients superior to 0.38. Among these features, 5 were shape features, 2 were first-order metrics, and 13 were based on texture matrices, with 6 extracted from the GLCM matrix (Figure 6.3). Figure 6.3: Coefficient of each radiomic feature in the decision function for the proposed logistic regression model.

Diagnosis performance of the classifier with a ten-repeated 5-fold cross-validation

The model differentiated BM from GBM on the validation sets with a mean sensitivity of 85% (95% CI = [77%; 94%]), a specificity of 87% (95% CI = [78%; 97%]), a balanced accuracy of 86% (95% CI = [80%; 92%]), and an AUC of 92% (95% CI = [87%; 97%]) (Figure 6.4).

Diagnosis performance of the radiomic classifier on the test set

The classifier correctly identified 12/16 BM and 18/21 GBM. Corresponding sensitivity, specificity, balanced accuracy and AUC were respectively equal to 75%, 86%, 80%, and 85% (6ch/figures 6.4 and 6.5).

B A

Performance of the radiologists

The performances of the neuroradiologists are described in Table 6.2. Even though differences in diagnostic performance were not statistically significant, we can highlight the fact that two radiology residents (R3 and R4) had lower scores than the classifier (respective balanced accuracies of 72% and 72%) whereas the 2 neuroradiologists with 10 years of experience (R1 and R2) and one radiology resident (R5) had better scores than the classifier (respective balanced accuracies of 87%, 94% and 88% versus balanced accuracy of 80% for the classifier).

Discussion

We have developed a radiomic classifier to differentiate solitary BM and GBM based on postcontrast 3DT1 MR images with high diagnostic performances on the validation and test sets.

There was no statistically significant difference between classifier predictions and human reading by 5 trained neuroradiologists (2 neuroradiologists with 10 years of experience, and 3

radiology residents with about 6 months of training exclusively in neuroradiology in an expert center).

The radiomic classifier, a logistic regression combined to the power transform yeo-johnson scaling feature method, was chosen because of its high performance, simplicity, and because it allowed an interpretation of the underlying model. Indeed, the fact that the radiomic feature with the most important coefficient value in the classifier was a shape feature, i.e., sphericity, partly allows an explainability of our radiomic features-based classifier in contrast with the concept of the "black box" in some ML models, where even its designers cannot explain why the artificial intelligence reaches a decision [39]. It introduces the notion of analyzing a tumor with its representation in 3D to differentiate solitary BM and GBM, which is usually not available during conventional reading of sectional imaging. Indeed, sphericity is a 3D shape feature representing a measure of roundness of the tumor, with a value ranging from 0 to 1, where 1 indicates a perfect sphere. The classifier showed that GBM have lower sphericity than BM (Figure 6.6), which was expected given the morphological characteristics of BM and GBM on histopathological slides. The more spherical the lesion is, the more likely it is to be a BM.

Thus, the radiomic features-based classifier is consistent with current morphological characteristics between BM and GBM, also adding further information regarding tumor heterogeneity imperceptible to the human eye, as the radiomic classifier is also based on other texture and intensity features. This result is in line with a pioneering paper [40] that described in 2012 2D circularity as one of the best morphological features to differentiate BM from GBM on the basis of a cohort of 50 patients. In our study, we trained the ML classifier using a nested cross-validation and a ten-repeated 5-fold cross-validation on the training set in order to minimize overfitting. In addition to limit the extraction to 100 features (shape, first order and second order features) that we thought to be the most meaningful and interpretable, we selected a classifier model which could embed feature selection. For this model, L1 and L2 regularization methods were tested as hyperparameters. L2 method provided the best performance in the cross-validation process, validating the usefulness of the 100 features. The selected classifier was then applied on a test set of data, which demonstrates that the high performances obtained were not random but generalizable. In the test set, 12/16 BM were correctly classified leading to a sensitivity of 75%. Among the 4 BM incorrectly classified, 2 had leptomeningeal enhancement, one had ventriculitis adjacent to the lesion and the forth one had a multilocular lesion (Figure 6. The results of our study are consistent with the results of three previous studies which also used radiomic features-based classifiers on post-contrast 3D T1 MR images to differentiate BM from GBM. Among these studies, Chen et al. [41] achieved diagnostic performance slightly lower than our on 134 patients, however without applying image preprocessing [42][43][44] nor evaluation on a test set. Artzi et al. [45] built a radiomics-based classifier on 358 patients and evaluated its performance on a test set of 88 patients. Excellent performances were achieved on the test set. However, the radiomic analysis was carried out on 3 central slices only to simplify the segmentation process, which did not allow 3D shape features such as sphericity, to be taken into account. Moreover, there was no comparison to human performance. In 2019, Qian et al.

GBM

[46] used a cohort of 227 patients to train a ML classifier using cross-validation and evaluated it on an independent test set of 185 patients. Despite high diagnostic performances, there were biases in the study considering several radiomic features-based classifiers were evaluated on the test set. Finally, in 2020, Bae et al. [47] developed a Deep Neural Network classifier based on post-contrast 3D T1-weighted and T2-weighted MR images, which outperformed the best-performing traditional machine learning model. Results showed excellent performance on an independent test set (AUC of 0.956 on the test set) and outperformed scores of 2 trained neuroradiologists. However, comparing the literature is not a trivial task due to the use of different data sets, each with varying degrees of complexity, suggesting the need for publicly available data sets.

Our study had few limitations. First, we chose to build the radiomic features-based classifier on imaging data acquired on the same model of MR scanner with acquisitions performed with the same parameters in order to minimize inter-acquisition variability. This choice limited the number of patients included in the study. Several methods are available today to compensate for differences in image quality between scanners [36,48], which should allow the applicability of our signature in other centers. In addition, no spatial resampling was applied to the MR images prior to feature extraction. Although this step is mandatory to obtain rotationally invariant features, no bias was introduced in the machine learning pipeline, as the entire cohort had exactly the same imaging parameters. The developed signature can finally be generalized to new patients with MR images of different voxel sizes by integrating an additional resampling step (resampling at a voxel size of (0.8 mm x 0.8 mm x 1.2 mm)). Third, a semi-automatic method was used for tumor delineation and a single radiologist specialized in neurology performed the contouring of the lesions. Perturbation of the contours would have been an alternative to multiple segmentation to evaluate the robustness of the model developed to segmentation [49].

However, the semi-automatic contouring process has been shown to be reliable between raters for brain tumors [50]. An integrated diagnostic support system should include automatic segmentation of the volumes of interest to be considered for radiomics analysis. The automation of this step is now possible with high performance as demonstrated by the recent results of the BraTS challenge [51]. Then, the radiomic features-based classifier takes into account imaging data only. The addition of the patient's age, gender, and medical history elements would lead to holistic models enabling to analyze the correlations between radiomic/non-radiomic features, and to better assess the added value of such a signature compared to more readily available clinical features [49]. As well, only post-contrast 3DT1 MR images were considered.

A more complex classifier combining data from other sequences such as FLAIR, T2 [47] or perfusion MR sequences may improve classification performances. Finally, a larger cohort of

"uniform", "distance"] algorithm ["auto", "ball_tree", "kd_tree", "brute"] leaf_size [5,50]

"lbfgs" penalty solver "l2"

["lbfgs", "liblinear"] continued on next page Table S6.1 -continued from previous page

Model Parameters Space

MLP hidden_layer_sizes [ (1,5), (1,10), (1,15), (1,20), (1,25), (1,30), (1,35), (1,40), (1,45), (1,50), (1,55), (1,60), (1,65), (1,70), (1,75), (1,80), (1,85), (1,90), (1,95), (1,100), (1,105), (1,110), (

Notes.

No Scaler corresponds to the fact of not using a scaler in the machine learning pipeline.

Brain tumor segmentation with deep learning

Foreword

This Chapter presents a study published as the following paper: Théophraste In this study, we proposed an automatic solution for brain tumor segmentation following our participation in the international challenge of brain tumor segmentation 2020. This solution is based on multiple U-net like neural networks, mainly with deep supervision and stochastic weight averaging. Our ensemble ranked fifth out of 693 teams registered for the BraTS 2020 challenge segmentation task.

Abstract

Brain tumor segmentation is a critical task for patient's disease management. In order to automate and standardize this task, we trained multiple U-net like neural networks, mainly with Four distinct tumoral subregions can be defined from MRI: the "enhancing tumor" (ET) which corresponds to area of relative hyperintensity in the T1Gd with respect to the T1 sequence; the "non enhancing tumor" (NET) and the "necrotic tumor" (NCR) which are both hypo-intense in T1-Gd when compared to T1; and finally the "peritumoral edema" (ED) which is hyper-intense in FLAIR sequence. These almost homogeneous subregions can be clustered together to compose three "semantically" meaningful tumor subparts: ET is the first cluster, addition of ET, NET and NCR represents the "tumor core" (TC) region, and addition of ED to TC represents the "whole tumor" (WT). Example of each sequence and tumor subvolumes is provided in Figure 7.1 using 3D Slicer [1].

Multimodal Brain Tumor Segmentation challenge 2020

The Multimodal Brain Tumor Segmentation Challenge 2020 [4][5][6][7] was split in three different tasks: segmentation of the different tumor sub-regions, prediction of patient overall survival (OS) from pre-operative MRI scans, and evaluation of uncertainty measures in segmentation.

The Segmentation challenge consisted in accurately delineating the ET, TC and WT part of the tumor. The main evaluation metrics were an overlap measure and a distance metric. The commonly used Dice Similarity Coefficient (DSC) measures the overlap between two sets. In the context of ground truth comparison, it can be defined as follows:

DSC = 2T P 2T P + F P + F N (7.1)
with TP the true positives (number of correctly classified voxels), FP the false positives and FN the false negatives. It is interesting to note that this metric is insensitive to the extent of the background in the image. The Hausdorff distance [8] is complementary to the Dice metric, as it measures the maximal distance between the margin of the two contours. It greatly penalizes outliers: a prediction could exhibits almost voxel-perfect overlap, but if a single voxel is far away from the reference segmentation, the Hausdorff distance will be high. As such, this metric can seem noisier than the Dice index, but is very handy to evaluate the clinical relevance of a segmentation. As an example, if a tumor segmentation encompasses distant healthy brain tissue, it would require manual correction from the radiation oncologist to prevent disastrous consequences for the patient, even if the overall overlap as measured by the Dice metric is good enough.

Methods

Two independent training pipelines were designed, with a common neural network architecture based on the 3D U-Net with minor variations (described below). These two different training approaches were kept separate in order to promote network predictions' diversity. The specific details of each pipeline will be described below, and referred to as pipeline A and pipeline B.

Neural network architecture

After neural network architecture exploration, the chosen network used an encoder decoder architecture, heavily inspired by the 3D U-Net architecture from Çiçek et al [9]. The architecture used is displayed in Figure 7 In the following description, a stage is defined as an arbitrary number of convolutions that does not change the spatial dimensions of the feature maps. All convolutions were followed by a normalization layer and a nonlinear activation (ReLU layer [10]). Group normalization [11] (A) and Instance normalization [12] (B) were used as a replacement for Batch Normalization [13] due to a small batch size during training and good theoretical performance on non-medical datasets.

The encoder had four stages. Each stage consisted of two 3x3x3 convolutions. The first convolution increased the number of filters to the predefined value for the stage (48 for stage 1), while the second one kept the number of output channels unchanged. Between each stage, spatial downsampling was performed by a MaxPool layer with a kernel size of 2x2x2 with stride 2. After each spatial downsampling, the number of filters was doubled. After the last stage, two 3x3x3 dilated convolutions with a dilation rate of 2 were performed, and then concatenated with the last stage output.

The decoder part of the network was almost symmetrical to the encoder. Between each stage, spatial upsampling was performed using a trilinear interpolation. Shortcut connections between encoder and decoder stages that shared the same spatial sizes were performed by concatenation. The decoder stage performing at the lowest spatial resolution was made up of only one 3x3x3 convolution. Last convolutional layer used a 1x1x1 kernel with 3 output 205 channels and a sigmoid activation.

The previous winner of the BraTS challenge [14] limited their downsampling steps to 3. We hypothesized that further downsampling of the features maps, given the limited size of the input (128x128x128), would lead to irreversible loss of spatial information. As the last stage of the encoder takes much less GPU memory than the first, the dilation trick [15] was used to perform a pseudo fifth stage at the same spatial resolution as the fourth stage.

3D attention U-Nets were also trained, using the Convolutional Block Attention Module [16] added at the end of each encoder stage.

Loss Function

Inspired by the conciseness of the 2019 winning solution [14], the neural network was trained using only the Dice Loss [17] (A). The loss L is computed batch-wise and channel-wise, without weighting:

DSC = 1 - 1 N n S n * R n + ε S 2 n + R 2 n + ε (7.2)
with n the number of output channels, S the output of the neural network after sigmoid activation, R the ground truth label and a smoothing factor (set to 1 in our experiment). For diversity, the pipeline B used a slightly different formulation of the Dice Loss, without squaring the terms of the denominator. Similarly, optimization was made directly on the final tumor regions to predict (ET, TC and WT) and not on their components (ET, NET-NCR, ED). The neural network output was a 3-channel volume, each channel representing the probability map for each tumor region.

Deep supervision [18] was performed after the dilated convolutions, and after each stage of the decoder (except the last) as in [19]. Deep supervision was achieved by adding an extra 1x1x1 convolution with sigmoid activation and trilinear upsampling. Like the main output, each of this additional convolution resulted in a 3-channel volume, each channel representing the probability map for each tumor region (ET, TC and WT). The final loss was the unweighted sum of the main output loss, and the four auxiliary losses.

Image preprocessing

Since MRI intensities vary depending on manufacturers, acquisition parameters, and sequences, input 7-ch/figures needed to be standardized. Min-max scaling of each MRI sequence was performed separately, after clipping all intensity values to the 1 and 99 percentiles of the nonzero voxels distribution of the volume (A). Pipeline B performed a z-score normalization of the non-zero voxels of each IRM sequence independently.

7-ch/figures were then cropped to a variable size using the smallest bounding box containing the whole brain, and randomly re-cropped to a fixed patch size of 128x128x128. This allowed 206

Methods

to remove most of the useless background that was present in the original volume, and to learn from an almost complete view of each brain tumor.

Data augmentation techniques

To prevent overfitting, on-the-fly data augmentation techniques were applied in both pipelines, according to a predefined probability. The augmentations and their respective probability of application were:

• input channel rescaling: multiplying each voxel by a factor uniformly sampled between 0.9 and 1.1 (A: 80% probability, B: 20%).

• input channel intensity shift: Adding each voxel a constant uniformly sampled between -0.1 and 0.1 (A: not performed, B: 20% probability).

• additive gaussian noise, using a centered normal distribution with a standard deviation of 0.1.

• input channel dropping: all voxel values of one of the input channels were randomly set to zero (A: 16% probability, B: not performed).

• random flip along each spatial axis (A: 80% probability, B: 50%).

Training details

Models were produced by a five-fold cross-validation. The validation set was only used to monitor the network performance during training, and to benchmark its performance at the end of the training procedure.

Pipeline A:

For each fold, the neural network was trained for 200 epochs with an initial learning rate of 1e -4 , progressively reduced by a cosine decay after 100 epochs [20]. A batch size of 1 and the Ranger optimizer [21][22][23] were used. After 200 epochs, we performed a training scheme inspired from the fast stochastic weight averaging procedure [24]. The initial learning rate was restored to half of its initial value (5e -5 ), and training was done for another 30 epochs with cosine decay. Every 3 epochs, the model weights were saved. This procedure was repeated 5 times for a total of 150 additional epochs. At the end, the saved weights were averaged, effectively creating a new "self-ensembled" model. The Adam optimizer [25] was used without weight decay for the stochastic weight averaging procedure. 207

Second step

The first step gave two labelmaps per case. Based on the online validation dataset, the mean whole tumor dice metric of the pipeline B's ensemble was consistently higher than that of the pipeline A's ensemble. We hypothesized that models from pipeline B were better for predicting edema. To keep the score intact on ET and TC from models A, ET and NET/NCR predicted labels had to be left untouched. If A predicted background or edema and B predicted edema or background respectively, B predicted labels were kept. The merging procedure is shown in table 7.1

Ablation Study for Pipeline A

Experiments with and without dataset filtering and attention block were produced for pipeline A. Cross-validated results can be found in Table 7.2. There was no clear benefit of either strategy, hence we decided to keep the two best available models for each fold for this pipeline.

Results

Online Validation dataset

Testing dataset

Our final results on the testing dataset are displayed in table 2. These results ranked us among the top 10 teams for the segmentation challenge. A significant discrepancy between validation and testing datasets for the TC Hausdorff distance was visible, while all other metrics showed small but limited overfit.

Discussion

Our solution to the BraTS'20 challenge is based on standard approaches carefully crafted together: we used U-net 3D neural networks, trained with on-the-fly data augmentations using the Dice Loss and deep supervision, and inferred using test time augmentation and models predictions ensembling.

Many modern "bells and whistles" were tried: short additive residual connections [26], dense blocks [27], more recent neural networks backbone based on inverted residual bottleneck [28], newer decoder structure like biFPN layer [29], or semi-supervised setting using brain dataset from the Medical Decathlon [30]. None of these refinements led to significant improvement on the local validation set. We hypothesize that this was probably due to GPU memory constraints. Indeed, while these layers improve the model accuracy at a relatively small parameter cost, it increases significantly the size of the activation maps of the model, forcing us to use smaller networks (reduction of the number of output channels per convolutional layer). Reducing the crop size of the patch was not an option as this would have most probably reduced the network performance due to the lack of context. Moreover, all of these additions led to a significant increase of the training time, reducing the searchable space in the limited timeframe of the challenge.

Stochastic weight averaging at the end of the training was the most notable refinement we used. This training scheme was a remnant from the mean teacher semi-supervised training [31].

We did not benchmark its real potential but expect it to produce a more generalizable model, to prevent from overfitting on the training set and to remember the noisy labels. Indeed, it has been shown that a high learning rate could prevent such behavior, and we expect that our training benefits from the multiple learning rate restarts [32].

Notably, while our results were not state of the art for the BraTS 2020 challenge, the segmentation performance of our method is in the usual range of inter-rater agreement for lesion segmentation [33] and could already be valuable for clinical use. As an example, Figure

Conclusion

The task of brain tumor segmentation, while challenging, can be solved with good accuracy In this study, we aimed to create a large database of postoperative glioblastoma including multiparametric MRIs, associated subregions segmentation labels, and clinical and radiomic data. The objective is to make the dataset publicly available as well as all the code used for image preprocessing, segmentation, and radiomic features extraction in order to compensate for the lack of publicly obtainable data. This article describes the collected cohort of 247 patients and their clinical data, the complete methodology of image preprocessing, the methodology of assisted labeling using the model developed in Chapter 7, the final review by human experts, and the addition of labels specific to the postoperative situation. The radiomic features were also extracted following the methodology presented in chapter 6. It should be noted that the paper presented is being considered for submission to Nature Scientific Data, whose article construction guidelines may appear atypical and confuse the reader.

Abstract

Glioblastoma multiforme (GBM) is a grade IV tumor that is the most common primary brain tumor in adults. Despite aggressive treatment modalities (surgical resection, radiotherapy, and chemotherapy), the prognosis is poor, with a median overall survival (OS) estimated between 12 and 18 months. The lack of publicly available data is currently a barrier to multicenter studies, which are essential for clinical decision in routine clinical practice. To this end, we publish postoperative multiparametric (n=247) magnetic resonance imaging (MRI) and Computed Tomography imaging data from 247 patients of Gustave Roussy Cancer Campus collection and associated segmentation labels, radiomic features, and clinical data. MRI images were preprocessed using a state-of-the-art method, and glioblastoma subregion labels were generated by an automated Deep Learning framework and manually revised by trained brain imaging experts. The effort to make GBM neuroimaging data publicly available may provide new opportunities for changing clinical practice.

Background & Summary

Glioblastoma (GBM) is the most common primary brain neoplasm without any significant therapeutic advance for almost a decade. Currently, management of GBM is generally with a palliative intent to improve life expectancy and ensure the best possible quality of life for the patient [1]. The current standard of care associates maximum surgical resection followed by radiation therapy (RT) associated with a concomitant and adjuvant oral chemotherapy [2]. The quality of surgery is an established prognostic factor for survival [3]. Consequently, patients whose tumor localization does not permit tumor removal have restricted survival. However, even with adequate ablation, the invasive nature and rapid proliferation of GBM can not be managed by traditional treatment protocols.

Methods

GBM is characterized by a macroscopic tumor core and an infiltrative tumor component extending into adjacent tissues. The imaging distinction between edema, healthy tissue, and tumor cells is a critical issue especially for the planning of local treatment, such as radiotherapy [4]. GBMs are driving a specific strategy to deal with their heterogeneous properties because no single segmentation approach can be applied to all types of brain tumors [5]. To date, there are still major challenges to the applicability of artificial intelligence tools in clinical practice. One of the most important reason is the data collection required for model development, i.e. imaging data with related expert annotation, which are time consuming to retrieve and label [6]. Indeed, machine learning (ML) and deep learning (DL) models require large amounts of training data to generalize their performance into real-world applications. As well, there is a lack of large consolidated public data sets in the medical field, especially in imaging and glioblastoma. Over the past decade, international and multi-institutional efforts have been made to collect and share imaging and related medical data.

Methods

The Institutional Review Board of Gustave Roussy approved this Accountability Privacy Rulecompliant study in which all imaging data were collected. Informed consent was obtained from all alive participants. Permission to share the de-identified image data, segmentations, to reduce total computation time and the two best models with the lowest loss value on each validation set were kept. Model inference was done using ensemble of the two best performing models per fold (i.e., 10 models). The segmentations were produced in a one-pass fashion using the crop brain image. A post-processing step was performed to remove labels including less than 150 voxels. In this case, the label was replaced by the nearest value in the axial plane.

Radiomic features

The open-source Pyradiomics [22] library (version 3.0. Standardization Initiative (IBSI) [23], except the Kurtosis first-order feature, where Kurtosis is calculated using -3 and +3 in the IBSI and PyRadiomics referential, respectively. As in the automatic delineation step, voxel intensities of the cerebral area were standardized for each patient and each modality using the Z-Score method. Before feature extraction, a scaling factor of 100 was then applied to each intensity value. All intensity ranges of the full patient dataset were used to define the average minimum and maximum values for each sequence and each 8.5 Data Records label, from which the bin size was defined, considering 32 bins in the considered interval (fixed bin size discretization method) [24,25] (Table 8.6). The radiomic features are made available to the community to facilitate their use for any researcher who does not have sufficient computer skills and favor homogenization of the practices. They can be used to discover possible associations with clinical outcomes, treatment responses, or other outputs. Notes.

IR stands for intensity ranges.

Data Records

Dosimetric planning CT in treatment position and brain MRI were downloaded for each patient. The re-oriented, co-registered, and skull-stripped images, as well as the related computerassisted and manually reviewed segmentation labels in NIfTI format, were included in the dataset, which was made available through the UK Data Service. The file containers have been enriched to include a wide range of clinical records and radiomic features, which will facilitate survival analysis research.

Technical Validation

Data collection

Two radiation oncologists (G.K. and S.A., 6 and 8 years of expertise respectively) reviewed the clinical and radiological records of the 247 included patients.

Segmentation labels

The method developed to generate the segmentation labels was a slight variation of our model proposed in the BraTS 2020 challenge and ranked in the top 10 of the 78 participating teams submitting results in the final testing phase. To evaluate the performance of the final model presented in this paper, we used the 2020 online evaluation platform available on the University of Pennsylvania image processing portal (ipp.cbica.upenn.edu). The metrics used to benchmark the algorithm were as follows: Dice Similarity Coefficient (DSC), and 95% Hausdorff distance (HD). DSC is commonly used in the assessment of segmentation performance and measures the overlap between two sets. The 95% Hausdorff distance, as opposed to the standard HD distance, which measures the maximal distance between the margins of two contours, considers the 95th percentile and avoids outliers having too much weight. The regions evaluated using these metrics included the whole tumor, the tumor core, and the enhancing tumor. The tumor core considered the part of the tumor that is typically resected and therefore involved ET, NET, and NCR, and the whole tumor described the association of all tumor sub-regions (i.e., tumor core and peritumoral edema/infiltrated area). The median DSC values with their corresponding interquartile range (IQR) on the online validation set for the three evaluated regions, i.e., WT, TC, ET, were equal to 0.

Contour review and manual correction

Following the fully automatic segmentation of the 3 labels (NCR & NET, ED, ET) and their revisions, the two new labels, i.e., SC and POM, were manually segmented using the free open source software 3D Slicer [26][27][28]. POM labels showed hyper-intensity in T1 compared to normal whitte-matter. This label was usually quite easy to segment using masking with an adjusted range on using otsu thresholding (Fig. 8.4B). Surgical cavities appeared to be equivocal between experts. It was reminded to every expert that each of the MRI sequences (T1w, T1w-gd, T2-flair) had to show a low signal intensity to be considered as surgical cavity.

If these criteria were met although the patient had not undergone surgical resection in the clinical data record, this area was considered as part of the unenhanced label (NCR & NET), as in the case shown in Fig. 8.4D. Finally, if a hyposignal was found on T1w and T1w-gd, but not on T2-flair, it was categorized as unenhanced. During the labeling process, each case was iteratively reviewed to ensure satisfactory quality of the segmentations by an imaging specialist

Code availability

All code developed is packaged in the Python language. We used an in-house package cBrainMRIPrePro (https://github.com/Alxaline/cBrainMRIPrePro) to preprocess the data. This package wraps ANTsPy and HD-bet and enables conventional MRI preprocess-

Code availability

ing by provinding a pipeline that offers the ability to resample the images, correct MRI from bias field, co-register the MRI (to the SRI24 template or a reference sequence), perform skull stripping, and Z-Score standardization. We have also created a publicly available Github repository containing all steps for preprocessing, pre-trained model weights for segmentation, and scripts for radiomic feature extraction so that results can be reproduced

Abstract

Glioblastoma multiforme (GBM) is the most common and lethal form of primary malignant The different strategies implemented regarding stratification have performances comparable to the best results in the literature. This study paved the way for postoperative survival analysis in GBM patients using a Bayesian optimized XGboost model.

Introduction

Glioblastoma (GBM) is the most frequent adult primary brain tumor. The prognosis for patients with this type of tumor remains unfavorable, with an estimated average survival of 12 poorer prognosis and survival rates in older patients [7,8]. A functional Karnofsky score (KPS for Karnofsky Performance Status scale) higher than 70 or 80 is correlated with longer survival in GBMs [7,8]. The existence of a neurological deficit is also a pejorative factor in many studies [7,9]. Patients with a frontal location of their GBM survive longer than those with a temporal or parietal tumor (median survival of 11.4 months, 9.1 months, and 9.6 months respectively) [10]. The extension of the surgical resection is an important prognostic factor demonstrated in the literature. Most retrospective studies have shown a clear survival advantage for patients who have had total tumor resection [7,11,12]. Many biological markers have been shown as having predictive value for overall survival or response to complementary treatments. Overall survival is significantly longer in the case of MGMT methylation (21.7 months versus 12.7 months) for patients treated with Temozolomide [2]. The presence of mutations in the IDH gene is an important factor of good prognosis, the median survival of GBM with mutations 241 being 27 to 31 months, while that of GBM without mutations is 11.3 to 15 months [13][14][15].

Radiomics is the mathematical representation of tumor phenotype using high-dimensional data generated from segmented medical images. The quantitative data extracted from the images provides a wide variety of parameters, including shape, histogram and texture parameters [16,17]. Radiomics offers an almost unlimited number of imaging biomarkers that can help in cancer detection, diagnosis, prognosis assessment, and treatment response prediction [18]. Combined with machine learning techniques, radiomics has thus become an increasingly popular input for medical decision support tools.

To assess glioblastoma survival, studies usually focus on pre-operative data, as in the Multimodal Brain Tumor Segmentation Challenge (BraTS) [19][20][21]. In this challenge, one task is devoted to classifying subjects into three classes: long-survivors, short-survivors and midsurvivors [22][23][24][25][26]. Alternatively, some authors choose to consider only two classes, short-term and long-term survivors, leading the task easier but less informative [23,[27][28][29][30]. Finally, this problem can be directly addressed as a regression task [31][32][33][34][35]. Studies that have investigated glioblastoma survival in terms of classification have mainly used Random Forest (RF) or Support Vector Machine (SVM), whereas Cox regression model was preferred for survival analysis.

Among various machine learning models, one of the state-of-the-art ensemble approaches is eXtreme Gradient Boosting (XGBoost) introduced in 2016 [36]. XGBoost offers a very large panel of hyperparameters. Thanks to this diversity, it is possible to have total control over the implementation of Gradient Boosting. Because of these benefits, XGBoost consistently achieves the highest accuracy or Area Under the Receiver Operating Characteristic Curve (ROC AUC) in a variety of fields [37][38][39]. It has an impressive track record as the winning solution in many Kaggle machine learning competitions [40].

Due to the XGBoost's computational speed, generalization capabilities, and high predictive performance, we were interested in exploring its performance on the glioblastoma survival task.

We have combined this classification model with a Bayesian optimization in order to explore and get the most out of its hyperparameter space. We then developed models based on different sets of features, i.e., using only clinical features, radiomic features, or the combination of both.

Thus, the same prediction framework was used to answer the different survival stratification problems, i.e., classification with 2-classes (short and long) or 3-classes (short, medium, and long) and regression survival prediction.

Material and methods

Dataset

We used the MMI-PROB dataset (Chapter 8), which contains 247 sets of postoperative MRI sequences of adult GBM patients: T1-weighted axial MRI (T1w), T1-weighted axial MRI with gadolinium injection (T1w-gd), and T2-weighted axial FLAIR (Fluid Attenuated Inversion Recovery) (T2w-flair). In this dataset, MRI images were preprocessed using a state-of-the-art method, and glioblastoma subregion labels were generated by an automated deep learning framework and manually revised by experts in the field. Up to five regions per patient were provided as follows: "necrosis and non-enhancing" corresponding to non-enhancing tumor core on T1w-gd; "edema" corresponding to intense, peri-lesional, vasogenic edema, and tumor infiltration on T2w-flair; "enhancement" corresponding to the contrast-enhanced lesion on T1w-gd in hyposignal in T1w sequence; "surgical cavity" corresponding to surgical resection if present; and finally "postoperative modifications" corresponding to spontaneous or postbiopsy or post-surgical bleeding in hypersignal on T1w (Fig.

Algorithm eXtreme Gradient Boosting

XGboost follows the principle of the Gradient Boosting algorithm [36]. It corresponds to an ensemble learning method that aggregates "weak learners" which are decision trees, to form a strong learner. As a result, even if each tree has low predictive power, the decision-making rule based on the sum of the results of each tree is very reliable.

XGboost uses K additive trees to create the ensemble model. Then the final prediction ŷ is the score summation of all leaves and can be written as Eq. 9.1.

ŷi = φ(x i ) = K k=1 fk (x i ) (9.1)
where fk represents an independant tree structure q with leaf j having a weight w j . The main idea of XGBoost is to minimize the objective function which is the sum of 2 terms.

L(φ) = n i=1 l (ŷ i , y i ) + K k=1 Ω (f k ) (9.2)
where l is a differentiable loss convex function that measures the difference between the prediction ŷi and its ground truth y i . The second term Ω is the regularization term that penalizes the model complexity, to prevent overfitting.

XGBoost offers a very large panel of hyperparameters (Table 9.1). Thanks to this diversity of parameters, it is possible to have total control over the implementation of Gradient Boosting.

Hyperparameters and Bayesian Optimization

Hyperparameter optimization is expressed in the following form:

x = arg min x∈X L(x) (9.3)
Where L is the objective function to minimize defined at Eq. 9.2. x is the set of hyperparameters that yields the lowest value of the score, and x can take on any value in the domain X .

The Bayesian optimization process includes 4 steps:

1. Define the Gaussian process as surrogate model for modeling the objective function f and define its prior.

2. Utilize the Bayes rule to obtain the posterior given the set of observations (function evaluations).

3. Optimize an acquisition/utility function u, which is a function of the posterior for sampling the next point x t = arg min x u(x).

4. Add newly observations to the set of observations and repeat the process (2 -4) until the number of defined iterations is reached.

XGBoost includes three types of parameters: general parameters, booster parameters, and task parameters. General parameters, i.e., the base learner, relates to the selected booster and control the overall function. Booster parameters depend on the selected booster, and task parameters control the optimization. We used the Scikit-Optimize library, which is an open-source Python library that provides an implementation for Bayesian Optimization [41].

Explainability

To interpret the importance of features in XGBoost, we used the built-in strategy by examining the "gain". This denotes the relative contribution of each component to the model, determined by adding the contributions of each feature to each tree of the model. A higher value of this parameter relative to another feature indicates that the feature is more important for prediction.

Experimental design

Data were randomly divided between training (75%) and testing data (25%) considering the full dataset (N=247). To this, a stratified split using a three-class label was applied. The three categories were defined to lead to an equal distribution of patients according to the value of overall survival, with cut-off values defined using percentiles. With respect to overall survival • Experience 3: A regression problem corresponding to a survival analysis that considered overall survival as the time variable and death as the event.

For each experience, only clinical features, only radiomic features, and the combination of clinical and radiomic features were considered.

Data preprocessing

Data were not available for some features, either due to the absence of data, as in the case of some clinical features, or due to the absence of a segmentation label in a patient for radiomic features. The handling of these missing values was made either by imputing them to a constant value of -9999 or by not considering them a priori since XGBoost provides an integrated routine in which missing values can be treated. Categorical variables present in the clinical data such as "Sex", "Extent of Resection", "Resection > 90%", "MGMT Methylation", "IDH Mutation", "RT Technique", "Gliadel", "TMZ Concomitant", "Avastin Concomitant", "TMZ Adjuvant"

and "Avastin Adjuvant" were one-hot encoded. In the case of a categorical variable with two labels, we kept only one dummy variable. For the other clinical variables, as well as for the radiomic variables, since we use trees as base learners, data scaling was not mandatory.

Therefore, we left it to Bayesian optimization to use scaling of the data to zero mean and unit variance or not.

Searching domain of hyperparameters in XGBoost

As discussed in sections 9.4.2 and 9.4.2, XGBoost has many hyperparameters to optimize. For the choice of the booster, one can use either decision trees (gbtree and dart)

or linear models (gblinear). Unfortunately, there is limited literature on the comparison of the different base learners for boosting [42]. To the best of our knowledge, no systematic comparison is available for the particular case of XGBoost. We have chosen to define the base learner as the default in XGBoost, which is gbtree. The tree method was also fixed to "gpu_hist" to use the GPU implementation to achieve substantial speedup over multi-core CPUs [43]. The hyperparameters adopted in this study included: "learn-ing_rate", "gamma" ,"max_depth", "min_child_weight", "max_delta_step", "subsample", "colsample_bytree", "colsample_bylevel", "reg_lambda", "reg_alpha", "n_estimators", "scale_pos_weight". "Scale_pos_weight" was only used in experience 1 (only usable in the binary classification). For experiment 3, corresponding to an inferential analysis on censored survival data, we chose the accelerated failure time algorithm (AFT), which has already shown scriptives and tests were performed: (1) mean, standard deviation and t-test or ANOVA; (2) median, first, and third quartiles, and Kruskall-Wallis test and (3) absolute and relative frequencies and chi-squared or exact Fisher test when the expected frequencies were less than 5 in some cells. A Shapiro-Wilk test for normality was used with a cutpoint level at 0.05. A p-value < 0.05 was defined as significant in the bilateral analyses.

OS was defined as the time between baseline MRI and death. Patients were censored at the date of the last follow-up. OS was estimated using the Kaplan-Meier method. Comparisons of OS curves between two groups were tested by a log-rank test.

For the classification tasks of Experience 1 and Experience 2, we used Balanced Accuracy (Eq. 9.4) as performance metric, which is a standard classifier score that helps to make model comparisons in binary or multiclass classification tasks simple and straightforward [45]. We also computed ROC AUC that summarizes the probability of correct decision-making depending on the classification threshold. In addition, Recall (Eq. 9.7) reflecting the ability of the classifier to find all positive samples and Precision (Eq. 9.6) that is intuitively the ability of the classifier not to label as positive a sample that is negative was used. The F1 score (Eq. 9.5) can be interpreted as a weighted average of Precision and Recall. For experience 2, which was multiclass, we used One vs. One (OvO) ROC AUC, and we computed the average AUC of all possible pairwise combinations of classes, while we used a weighted version of Recall, Precision, and F1 score. For the survival analysis of experience 3, we used the Harrell's Concordance Index (C-index) [46,47]. The C-index is defined as the ratio of the number of concordant pairs to the number of total analyzable pairs of observations [48] (Eq. 9. Where T P is the number of true positives, TN is the number of true negatives, F P is the number of false positives and F N is the number of false negatives.

C-index =

i,j I Ti < Tj • I (η j > η i ) • ∆ j i,j I Ti < Tj • ∆ j (9.8)

Results

Patient population

Discussion

The aim of this analysis was to evaluate the efficiency of one of the state-of-the-art machine learning techniques, XGBoost, when combined with Bayesian optimization. This model has the advantage of being efficient in a classification task in two (short, long) or three (short, medium, long) classes and a regression task. We then used one of the largest and most homogeneous cohorts available to date, consisting of 247 patients who received concurrent chemotherapy (temozolomide 75 mg/m 2 ) and radiotherapy (60 Gy in 30 fractions) followed by adjuvant oral Temozolomide (150-200 mg/m 2 /day) for confirmed GBM (Chapter 8). This study considered postoperative MR images, from which radiomic features were extracted and combined with clinical variables, which, to our knowledge, has never been done before. Compared to a wellknown dataset such as BraTS [19][20][21], which presents only preoperative data and has only three segmentation labels, the dataset we used includes multiparametric MR imaging of postoperative glioblastomas with two additional labels: postoperative modifications and surgical cavity.

Table 9.5 compares the results of this analysis to other published work. For each of the different stratification modes considered (2 classes, 3 classes, regression), the results are consistent with the performances of the literature obtained from pre-operative images. For the 2 class stratification problem, none of the studies reported the performance in a dataset not used for model development, which can have led to overoptimistic reported results as Chen et al. [30] with an accuracy of 85% or Sanghani et al. [23] with an accuracy of 99%. For this problem, our results are close to Yang et al.'s [27] ones who used a proper methodology with Out-of-Bag (OOB) Score and reported an accuracy performance of 63%, where we have reported 67%. For the 3 class stratification problem, Shboul et al. [24] similarly adopted an XGBoost model. He used LOOCV for developing his model and reported an accuracy of 52% in a test data set, where we reported 53%. Macyszyn et al. [22] showed an accuracy of 80%. These excellent results can be partly explained by the use of a vast set of features derived from perfusion and diffusion sequences which were added to the features derived from structural MRI. Similarly, for the regression problem, Liu et al. [28] obtained slightly better results than us with a C-index of 0.74 versus 0.72, which can also be explained by the use of features extracted from advanced MR imaging. However, contrary to some studies, we were not able to show the contribution of radiomic variables in combination with clinical variables [30,32,34,35]. This can be explained by the higher number of clinical variables that we provided to the model, i.e., 20 compared to the studies mentioned in the determination of infiltration areas in the FLAIR hypersignal at the time of RT treatment preparation could be of major importance to provide the patient with a personalized treatment.

Indeed, assuming that a dose-response relationship exists for GBMs, it may be interesting to further target these high-risk areas of recurrence with increased doses using so-called dose painting in RT. Recently, a dose-painting multicenter phase III trial in newly diagnosed GBM (the SPECTRO-GLIO trial) has been conducted [13,14]. This 2-arm randomized trial sought to compare conventional RT with intensity modulated RT with dose escalation (72 Gy) guided by MRI spectroscopic imaging (areas of elevated Cho/NAA ratio). The trial showed that such a strategy did not significantly increase toxicity and did not significantly improve overall survival.

This study aimed to determine if it was thus possible to determine the areas of recurrence at the voxel scale from anatomical postoperative MRI imaging by deep-learning (DL).

Material and Methods

Data

The database was collected at Gustave Roussy Cancer Campus and was composed of 199 patients with 3 anatomical MRI imaging sequences (T1w, T1w-gd, and T2w-flair) with their respective annotations at two-time points, i.e., at pre-radiotherapy (postoperative) time and at recurrence time. All imaging data followed a standard preprocessing with the python package (https://github.com/Alxaline/cBrainMRIPrePro) including: bias field correction; registration with affine transformation of the T1w pre-RT to the anatomical template (SRI24) with isotropic voxel size of 1 × 1 × 1 mm 3 ; registration of the other sequences (T1w-gd pre-RT and T2w-flair pre-RT ) on the T1w pre-RT-registered ; registration of the relapse time sequences on the T1w pre-RT-registered , i.e., registration of the T1w recurrence , T1w-gd recurrence and T2w-flair recurrence on the T1w pre-RT-registered ; then skull-stripping. In this database, 5 subregions labels were provided corresponding to "necrosis and non-enhancing", i.e., non-enhancing tumor core on T1wgd; "edema" corresponding to intense, peri-lesional, vasogenic edema and tumor infiltration on T2w-flair; "enhancement" corresponding to the contrast-enhanced lesion on T1w-gd; "surgical cavity" corresponding to surgical resection if present; and finally "postoperative modifications" corresponding to spontaneous or post-biopsy or post-surgical bleeding on T1w. More details about the procedure of data preprocessing and labels delineation are provided in Chapter 8.

Recurrence map

Recurrence is defined by the apparition of enhancing tumor (ET) in a different location than it was spotted at the time of pre-radiotherapy [15]. Therefore, the recurrence labels consisted of the segmentation labels ET at the recurrence time, which were not apparent at postoperative time. The recurrence map was realized by boolean operation (Fig. 10.1C). Among the 199 patients, 140 had local recurrence (70%), defined as recurrence being 2 centimeters or closer to the primitive tumor location [15]. 175 (88%) had at least part of the recurring tumor developing from the edema. 11 patients had no recurrence label. A simple task learning strategy consisting in optimizing directly the weights of the network on the on the determination of the recurrence map was implemented first. For this purpose, a 3D U-Net model with minor modifications that has already shown outstanding performance in brain tumor segmentation was employed [16]. The model consisted of a four-level encoderdecoder with a pseudo-fifth level at the same spatial resolution as the fourth level. Concatenations interconnect each level. The initial number of filters was 48, and was doubled at each downsampling. The downsampling was performed by a MaxPool layer with a kernel size of 2 × 2 × 2 with stride 2, and the upsampling was performed using trilinear interpolation. The number of kernels in the decoder mirrored that of the encoder. ReLUs [17] were used as nonlinearities. The instance normalization [18] was used for feature map normalization. Auxiliary

Multi task.

The interest of multitask learning is the possibility to combine knowledge from two different tasks and make greater use of the predictive potential of large networks [19]. We experimented with two different architectures derived from the simple task network, either sequential or parallel, and compared them to results of the simple task network.

Sequential.

Using the 3 pre-RT MRI sequences, the first encoder-decoder aimed at segmenting the ET class of the the tumor. Then, using the 3 pre-RT MR sequences and the classes prediction from the first U-Net, the second networks predicted recurrence. The network is shown in figure 10.3.

Parallel.

The two U-Net encoder-decoder parallel model was composed of one shared encoder and two decoders specific to each task. One task was dedicated to the pre-RT ET segmentation and, the other was dedicated to recurrence map determination. The two U-Net encoder-decoder parallel model was trained simultaneously on both tasks. The network is shown in figure 10.4.

Optimization Strategy

To train the networks, we used a combination of multiple loss functions, each of them related to a specific task. Two prediction cases on the test set corresponding to the parallel 3D U-Net are presented in figures 10.5 and 10.6 and correspond respectively to the best prediction (DSC: 0.503) and a poor one where the network has predicted a recurrence area that does not exist (DSC: 0.000).

For the best cases of prediction, it seems that the network is based on the edema present on the T2-flair without specifically understanding any recurrence pattern.

We did not perform any post-processing procedure on the labels such as replacing the label under a cut-off value of voxels, as it can be done to increase the ET segmentation results in the BraTS challenge. Indeed, the purpose was not to artificially increase the results, i.e., with a DSC value of 1, for patients with no recurrence labels and a DSC value of 0 when the least voxel was predicted. Here, 5 cases were present in the test set. There is no recurrence area for this case. Recurrence prediction by the network is filled in green.

Conclusion

In this work, we proposed to determine the recurrence areas in glioblastoma patients from anatomical MRI by deep-learning. To our knowledge, this work was the first to propose this approach. We could see that this task seems complicated for the methods discussed with notably the exploitation of multitasking and loss change for model optimization. On the test set, the DSC values did not go beyond 0.2 whatever the approach, which is far too low to attempt any radiotherapy implementation at the voxel scale. Although conventional MRI is exceptional in providing a detailed structural assessment of the brain, here deep-learning has not been able to extract more information. It would then seem that advanced MRI techniques with the ability to interrogate pathophysiological properties of tumors such as perfusion-weighted imaging (PWI) or diffusion-weighted imaging (DWI), should be incorporated. We released the code on GitHub at https://github.com/Alxaline/recurrence_gbm, enabling other researchers to expand our results.

and the other at 3T) allowed us to assess the robustness of the radiomic features. Moreover, the methods were applied to a machine learning task concerning the classification of the grade of brain tumors on a strongly heterogeneous multicentric dataset. The optimization of these specific steps performed in the radiomic workflow improved the prediction and generalization properties and the robustness of the features extracted from MR images. This allowed us to define guidelines for image preprocessing and grey level discretization in the radiomic context of brain tumors, which is crucial for moving from academic research to a clinical applicability of developed radiomic signatures, in a quite multi-vendor user-dependent acquisition parameter context. For models based on first-and second-order features, the combination of Z-Score normalization and absolute discretization seems to be the best. Regarding the grey level discretization, 32 bins seem to be a good compromise when T1w-gd and T2w-flair sequences are considered. For the same purpose, we were interested in studying a so-called a posteriori method, i.e., whose harmonization acts directly on the extracted radiomic features (Chapter 5). This method, named Combat and derived from genomics, has been adapted for ML applicability in the radiomic context. This method being limited for use in radiomics due to the need for appropriate statistics and labels corresponding to the "batch effect", we sought to develop a generic method named AutoComBat. The ComBat method adapted for ML and AutoComBat were released as a Python Package. The Combat and AutoComBat methods were then compared to the "optimal" image preprocessing method that we put forward in Chapter 4. The different methods were compared based on their ability to decrease the relative standard deviation of radiomic features extracted from white matter and considering a brain tumor classification task using different machine learning models. ComBat, AutoCombat using image-derived quality metrics as inputs for batch assignment and preprocessing methods presented promising results on white matter harmonization, but with no clear consensus for all MR sequences. Preprocessing showed the best results on the T1w-gd sequence for the grading task. Our results are sequence, feature class and task dependent and require further investigations on other datasets. We also highlighted that although Combat is promising, its employment is confused by the community for ML tasks. Thus, for future work considering MR images, we will preferably focus on the use of image preprocessing for the development of generalizable signatures (Chapter 4). In addition to being quite easy to implement and not requiring any a priori knowledge, these steps allow an all-in-one solution, as they are necessary, for example, in an automatic segmentation task using DL.

A second step of the project was to develop a signature capable of differentiating glioblastoma from solitary metastasis in a project initiated by the Sainte-Anne Hospital (Paris GHU) (Chapter 6). For this purpose, a ML classifier based on radiomic features and trained on postcontrast T1-weighted three-dimensional MR imaging was developed. The methodology used for the preprocessing of MR images and radiomic features was the same as implemented in Chapter 4. We also took care to develop a ML model that was as explicable and generalizable as ). We decided to focus solely on optimizing a machine learning model, which is the state-of-the-art XGBoost model. We thus explored the model's capabilities using Bayesian optimization according to different modes of survival stratification (in 2 classes, 3 classes, or by regression). We were able to show that the different strategies implemented had performances similar to the best results in the literature. However, in all cases, the combination of radiomic and clinical variables did not show any superiority over clinical variables only. We could also see that a single clinical variable such as concomitant 281 corticosteroids (mg/kg) could differentiate in short and long survivors with a balanced accuracy of 67%. Finally, we investigated whether recurrence areas for glioblastoma were predictable from postoperative anatomical MRIs (Chapter 10). To this, deep-learning methods derived from Chapter 7 were used. We also explored the interest in multitasking. Unfortunately, the results were not conclusive, with the best model achieving a Dice score of 0.201 for recurrence prediction. It follows that predicting recurrence patterns from anatomical MRI seems to be an overly complicated task and would require the use of advanced imaging, such as PWI or DWI.

Perspectives

In this thesis, we first addressed the harmonization of MR images through preprocessing image methods and the use of a posteriori methods (ComBat and AutoComBat). While these methods improve the robustness of radiomic features and the generability and performance of machine learning models, there seems to be a new trend in using deep learning to perform this task. Thus, GAN approaches and their variants could be used for the harmonization of multicentric MR images without the need of paired data [8,9]. The advantage is that such a method could be considered as a whole in a single harmonization strategy integrating, for example, the correction of the inhomogeneity of the magnetic field, the intrinsic variation of the imagers, etc., without the need for a priori information. Moreover, due to the DL execution speed, MRI images would be corrected efficiently. Nevertheless, as we have shown, the implementation of a carefully crafted preprocessing of imaging data that would be integrated into a radiomic analysis tool workflow would allow for greater generability of radiomic signatures.

Thus, a consensus on the implementation of the preprocessing steps in a radiomic workflow seems essential before considering any clinical application.

Second, we developed an ML-based radiomic classifier for the distinction between glioblastoma and solitary brain metastasis. Several points could be considered to improve the results.

As a preliminary approach, to make the signature clinically applicable, the database would need to be augmented and multicentric. Thus, discussions are currently underway with the Assistance Publique -Hôpitaux de Paris for the sharing of data from multiple centers. In addition, clarification of the role of sphericity and its interest in differentiating these lesions, notably by a prospective study recorded in a trial database, could be performed. As a second approach, adding radiomic features extracted from T2w-flair [10], T2w [10,11], or PWI [12] sequences could improve the classification performance. Finally, the problem could be directly addressed by the use of deep learning.

Thirdly, we have developed an automatic segmentation tool for brain tumors. Progress in this task has been evolving every year for the past ten years, especially through the BraTS challenge and the development of new approaches in DL [13]. For the 2021 edition, an important international work of data collection and annotation has been done to reach 2000 cases against de ces dernières années, le domaine de l'analyse des images médicales a connu une croissance exponentielle, avec l'avènement de la "radiomique" dans les années 2010, et de l'intelligence artificielle telles que l'apprentissage automatique (ML) et l'apprentissage profond (DL). La Pour intégrer ces résultats à la clinique, les résultats précédents ont été utilisés pour construire un modèle ML basé sur les caractéristiques radiomiques extraites de l'image rehaussée T1 afin d'aider les cliniciens à différencier les GBMs des BMs. La participation au défi 2020 de segmentation des tumeurs cérébrales (BraTS) nous a permis de développer un réseau de segmentation DL. Afin de personnaliser les traitements de radiothérapie, ce réseau a aussi été utilisé pour constituer une base de données institutionnelle de 247 patients GBM postopératoires, à partir de laquelle une étude de survie a été menée. Enfin, nous avons collecté les images de récidive de 199 patients, et nous nous sommes intéressés à la prédiction des zones de rechute par DL.

Sur la base de nos résultats, nous avons démontré que la radiomique est un outil essentiel à considérer pour la mise en oeuvre de la médecine de précision. Cependant, les informations contenues dans les images anatomiques IRM sont insuffisantes pour certaines tâches (p. ex. la prédiction des zones de rechute) et bénéficieraient de l'utilisation d'une imagerie multimodale avancée et de données complémentaires. Plusieurs modèles radiomiques proposés dans cette thèse ont le potentiel d'être rapidement transférés en clinique, en particulier ceux concernant le diagnostic différentiel, la survie ou la segmentation automatique.

Title:

Radiomics and artificial intelligence applied to conventional MRI: towards personalized treatment of brain tumors

Keywords:

Brain tumors, Glioblastoma, Brain metastases, Radiomics, MRI, Automatic segmentation, Image standardization, Survival prediction, Recurrence prediction

Abstract:

Malignant brain tumors are classified into primary tumors, primarily gliomas, the most aggressive form of which is glioblastoma (GBM), and brain metastases (BMs). Brain magnetic resonance imaging (MRI) is the preferred non-invasive modality for disease assessment when a tumor is suspected. It plays a crucial role in the diagnosis, characterization, treatment planning, and follow-up of the disease. The reference for diagnosis is based on a biopsy, which is not without risk and can be a source of error. The interpretation of medical images is limited by the human eye, even when trained. Computational analyses such as radiomicsbased machine learning (ML) and deep learning (DL) hold significant potential in oncology. Radiomics is the process of extracting high-throughput quantitative features from medical imaging. It enables data extraction and the establishment of decision support systems with the goal of improving lesion identification and categorization, mutation characterisation, and the design of prognosis and prediction models. DL is distinguished by its ability to identify and decipher complex patterns in medical oncology images, enabling it to transform image analysis from qualitative and contextual to quantitative and reproducible. The thesis focused on the development of methodological aspects and different applications of radiomics-based ML and DL to brain tumor imaging.

Firstly, MRI intensities suffer from a strong dependence on acquisition and reconstruction parameters, making the use of radiomic indexes in a multicentre context vulnerable. Therefore, a first study was conducted to characterize the effect of image preprocessing, in particular image harmonization and grey level discretization, to propose recommendations regarding the optimal pipeline to apply. A second study focused on the correction method applied directly to the extracted features, i.e., the ComBat method derived from genomics, but whose application is today limited by the requirement of the knowledge of the label for each sample corresponding to the "batch effect" and a minimum number of samples per label. A new approach was proposed to overcome these two limitations and compared to the traditional ComBat method and a standard image processing method.

To integrate these results into the clinic, previous findings were used to build a ML model based on radiomic features extracted from T1-enhanced imaging to help clinicians differentiate GBMs from BMs. Participation in the 2020 Brain Tumor Segmentation (BraTS) Challenge allowed us to develop a DL segmentation network for brain tumors. To personalize radiation therapy treatments, this network was also used to build an institutional database of 247 postoperative GBM patients, from which a survival study was conducted. Finally, we collected recurrence images from 199 patients, and were interested in predicting recurrence areas by DL.

Based on our results, we have demonstrated that radiomic is an essential building block to consider for implementing precision medicine. However, the information contained in MRI anatomical images is insufficient for some tasks (e.g., prediction of recurrence areas) and would benefit from the use of advanced multimodal imaging and complementary data. Several radiomic models proposed in this thesis have the potential to be quickly moved to the clinic, in particular those with regard to differential diagnosis, survival, or automatic segmentation.

Université Paris-Saclay

Espace Technologique / Immeuble Discovery Route de l'Orme aux Merisiers RD 128 / 91190 Saint-Aubin, France

3 . 3

 33 Variable selection algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Simplistic deep learning representation . . . . . . . . . . . . . . . . . . . . . . 3.5 Simplistic representation of validation schemes . . . . . . . . . . . . . . . . . study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Balanced accuracies obtained for the tumor grade classification task using the 18 first-order features only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Percentages of the 73 textural features showing ICCs and CCCs values > 0.8 depending on the intensity normalization and the discretization method . . . 4.4 Balanced accuracies obtained for the tumor grade classification task using the 73 textural features only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S4.1 Balanced accuracies obtained for the tumor grade classification task using a combination of the 18 first-order and 73 textural features . . . . . . . . . . . S4.2 Balanced accuracies obtained for the tumor grade classification task using the features defined as robust based on the DATASET1 results . . . . . . . . . . S4.3 ROC-AUC obtained for the tumor grade classification task using the 18 firstorder features only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S4.4 ROC-AUC obtained for the tumor grade classification task using the 73 textural features only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S4.5 ROC-AUC obtained for the tumor grade classification task using a combination of the 18 first-order and 73 textural features . . . . . . . . . . . . . . . . S4.6 ROC-AUC obtained for the tumor grade classification task using the features defined as robust based on the DATASET1 results . . . . . . . . . . . . . . . 5 Adaptation of the ComBat method for MRI radiomics 5.1 Study design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Parallel coordinate plots per center of the information extracted from the dataset for the T1w-gd and T2w-flair MRI sequences . . . . . . . . . . . . . . 5.3 Harmonization strength evaluated on the WM features for the different MRI sequences (T1w-gd, and T2w-flair) on the test set . . . . . . . . . . . . . . . . 5.4 Clustering interpretation of AutoComBat for the T1w-gd sequence. (A, C) is feature importance and (B, D) is visual representation of the clustering based on the feature reduction method used in AutoComBat. (A, B) AutoComBat using all features. (C, D) AutoComBat using QM. . . . . . . . . . . . . . . . xii List of Figures 5.5 Balanced accuracy for the tumor grading task for the 5 machine learning models and the different MRI sequences on the test set for the first and second-order feature types depending on the harmonization method . . . . . . S5.1 Parallel coordinate plots per center of the information extracted from the dataset for the T1w and T2w MRI sequences . . . . . . . . . . . . . . . . . . S5.2 Harmonization strength evaluated on the WM features for the different MRI sequences on the test set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S5.3 Harmonization strength evaluated on the WM features for the different MRI sequences on the validation set . . . . . . . . . . . . . . . . . . . . . . . . . . S5.4 Balanced accuracy for the tumor grading task for the 5 machine learning models and the different MRI sequences on the validation set for the firstand second-order feature types depending on the harmonization method . . . S5.5 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T1w MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method S5.6 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T1w-gd MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method S5.7 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T2w MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method S5.8 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T2w-flair MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S5.9 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T1w MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S5.10 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T1w-gd MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S5.11 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T2w MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii S5.12 F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models on the T2w-flair MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Varlet, Johan Pallud, Myriam Edjlali, and Charlotte Robert. Development of a Machine Learning Classifier Based on Radiomic Features Extracted From Post-Contrast 3D T1-Weighted MR Images to Distinguish Glioblastoma From Solitary Brain Metastasis. Frontiers in Oncology, 11, 2021. The study was designed by Alix de Causans, Myriam Edjlali, Samy Ammari, Sylvain Reuzé, Charlotte Robert, and I. Arnault Tauziède-Espariat and Pascale Varlet reviewed histopathological data. Alexandre Roux, Edouard Dezamis, Johan Pallud, Arnault Tauziède-Espariat, Pascale Varlet, and Frederic Dhermain took care of the patients and retrieved the data. Alix de Causans did the lesion contouring. I performed: programming writing, image processing, radiomics extraction, machine learning, experiments management, and data analysis (Statistical part was shared with Alix de Causans, Myriam

5 .

 5 Alexandre Carré, Guillaume Klausner, Samir Achkar, Théo Estienne, Théophraste Henry, Angela Rouyar, Roger Sun, Grégoire Fournier, Frédéric Dhermain, Eric Deutsch, Charlotte Robert, Multimodal imaging for personalized radiation therapy of brain tumors (MMI-PROB) dataset with postoperative MRI glioblastoma including segmentation labels, clinical and radiomic features. Envisioned paper: Nature Scientific Data, [In Preparation]. The design of this study and the following ones was initiated as part of a project financed by PhysiCancer, focusing on the personalization of brain tumor treatments by radiotherapy. Charlotte Robert is the project coordinator. The collection of imaging data and its extraction from the Picture Archiving and Communication System (PACS) was made possible thanks to a code developed by Théo Estienne. Guillaume Klausner and Roger Sun collected the clinical data. I manually performed the sorting, cleaning, and collection of the missing data. I developed the pipeline methods, programming writing, and automatic segmentation labeling. I manually revised the automatic labels and proceeded to the labeling of the new labels. I wrote the paper and all co-authors reviewed it. 6. Alexandre Carré, Guillaume Klausner, Samir Achkar, Théo Estienne, Théophraste Henry, Angela Rouyar, Roger Sun, Grégoire Fournier, Frédéric Dhermain, Eric Deutsch, Charlotte Robert, Multimodal imaging for personalized radiation therapy of brain tumors (MMI-PROB) dataset with postoperative MRI glioblastoma including segmentation labels, clinical and radiomic features. Envisioned paper: Frontiers in Oncology, [In Preparation].The origin of this study comes from an internship subject proposed to Guillaume Klausner and Samir Achkar before the cohort was completed. The results being only moderate,

20

 20

Figure 1 . 1 :

 11 Figure 1.1: Yearly estimated number of new cases (in millions) from 2020 to 2040, both sexes, age [0-85+]. Data from International Agency for Research on Cancer (IARC) [8].

Figure 1 . 3 :

 13 Figure 1.3: The different patterns of tumor heterogeneity. (a) Intertumor heterogeneity: heterogeneity between tumors in different patients or within the same patient with several tumor deposits. (b) Intratumor heterogeneity: diversity within a tumor. (c) Intratumor genomic heterogeneity: cells with distinct genomic alterations within a tumor. (d) Intratumor epigenomic heterogeneity: cells with diverse states of differentiation based on different epigenomic states. (e) Intratumor microenvironment heterogeneity: differences in tumor stroma (extracellular matrix, vasculature and immune cells) within the same tumor. Reprinted by permission from [Copyright Clearance Center]: [John Wiley and Sons] [Journal of Internal Medicine] [19], [COPYRIGHT] (2021).

10 1. 2 Figure 1 . 4 :Figure 1 Figure 1 . 5 :

 10214115 Figure 1.4: Growth of publications on Radiomics and Deep-learning in the past 8 years by PubMed searching. The search keywords can be found at: (Radiomics, https: //bit.ly/2X5dlPl and (Deep-learning, https://bit.ly/3lx8X4U)

Objective 6 :• Objective 7 :

 67 To develop a combined survival model (radiomic and clinical features) in glioblastoma patients based on postoperative multiparametric MRI (Chapter 9). To determine areas of recurrence in patients with glioblastoma based on postoperative multiparametric MRI (Chapter 10).

1. 3 - 2 :- 3 :- 4 :

 3234 Thesis hypotheses, objectives and outline • Part I: Dive deeper in the subject of brain and radiomics Chapter Background information on brain tumors and medical imaging techniques used in neuro-oncology. Chapter Introduction to radiomic modeling, notably image preprocessing, feature calculation, and machine learning methods. • Part II: Radiomic features vulnerabilties: assessment and solutions Chapter This paper sought to define guidelines for the preprocessing of MR images, including intensity standardization and greyscale discretization in the radiomic context, which is crucial for the generalization of published image-based signatures.

- 7 :

 7 Part III: The distinction between glioblastomas and brain metastases -Chapter 6: This paper developed a machine learning model capable of discriminating between glioblastoma and a unique brain metastasis lesion, which put in light a major morphological criteria for the distinction. • Part IV: An automatic deep learning solution for tumor contouring Chapter This chapter is based on a paper realized in the context of the participation to BraTS 2020 challenge, which allowed us to develop an automatic brain tumor segmentation tool. • Part V: A postoperative glioblastoma database constitution: prognosis and recurrence analysis.

Figure 2 . 1 :

 21 Figure 2.1: Structure of the human brain. (A) The brain is divided into the cerebrum (cortex), brainstem, and cerebellum. (B) Lobes of the cerebral cortex. Modified from Servier Medical Art by Servier licensed under a Creative Commons Attribution 3.0 Unported License.

Figure 2 . 3 :

 23 Figure 2.3: Glial cells and their interactions with other cells in the brain environment. Modified from Servier Medical Art by Servier licensed under a Creative Commons Attribution 3.0 Unported License.

Figure 2 . 4 :

 24 Figure 2.4: Matter and cerebrospinal fluid in the human brain. (A) Cross-Section of the brain showing the two distincts regions -white matter and grey matter. (B) Sagittal plane of the brain showing circulation (red arrow) of the cerebrospinal fluid (dark blue). Modified from Servier Medical Art by Servier licensed under a Creative Commons Attribution 3.0 Unported License.

Figure 2 . 5 :

 25 Figure 2.5: Diagnostic approach for integrated histological and molecular classification of diffuse gliomas according to the 2016 WHO Classification. Reprinted by permission from [Copyright Clearance Center]: [Springer Nature] [Nature Reviews Clinical Oncology] [25], [COPY-RIGHT] (2021).

Figure 2 . 6 :

 26 Figure 2.6: Micro-and macroscopic view of a GBM tumor. (A) Histopathology of a GBM demonstrating characteristics of a high-grade astrocytoma: marked nuclear pleomorphism, multiple mitoses (one on white arrow), and multinucleated cells (one on black arrow), with cells arranged randomly in a pink fibrillar background on haematoxylin and eosin (H&E) stain. (B) Cross-section of the brain showing the brain tumor (black arrow). Subfigure a is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication. Subfigure b is made available under the Creative Commons Attribution-ShareAlike 4.0 International.

Figure 2 . 7 :

 27 Figure 2.7: Clinical pathway for IDH-wild-type glioblastomas, WHO grade IV. Reprinted from [45] under the Creative Commons CC0 1.0 Universal Public Domain Dedication.

Figure 2 . 8 :

 28 Figure 2.8: Dissemination stages of metastatic tumor cells to the brain. The tumor cells become detached from the primary tumor, perform intravasation, and then reach the cerebral microvascularization before beginning a process of adhesion to the vascular wall and then extravasation. The tumor cells then either enter apoptosis, remain latent for several months or years, or proliferate to give rise to a brain metastasis. Reprinted by permission from [Copyright Clearance Center]: [Springer Nature] [Nature Reviews Disease Primers] [49], [COPYRIGHT] (2021).

Figure 2 . 9 :

 29 Figure 2.9: Proton orientation. (A) Random spins orientation in the absence of an external magnetic field. (B) Spins alignment in the presence of an external magnetic field -→ B 0 leading to a macroscopic magnetization.

Figure 2 . 10 :

 210 Figure 2.10: Vectoriel components and excitation phase. (A) The net magnetization vector -→ M , decomposed into a longitudinal component -→ M z and a transversal component --→ M xy , (B) Excitation phase: the energy given by the RF pulse flips the net magnetization vector -→ M of an angle α (here α = 90 • ).

  TR and TE) > 2500 80 -120 PDw (long TR and short TE) > 2500 ∼ 15 T2w-flair (very long TR and TE) > 6000 80 -120

Figure 2 .

 2 Figure 2.12 highlight the four most common morphologic MR image types ordered for brain tumors on which the work of this thesis has been concentrated: the T1-weighted (T1w), T1-weighted post-gadolinium contrast agent (T1w-gd), T2-weighted (T2w), and T2 fluidattenuated inversion recovery (T2w-flair). Table2.6 provides the visualization information of the brain structures.

Figure 2 . 12 :

 212 Figure 2.12: Examples of the four most commonly used MR anatomical sequences: (A) T1weighted (T1w), (B) T1-weighted post-gadolinium contrast (T1w-gd), (C) T2weighted (T2w) and (D) T2-weighted fluid-attenuated inversion recovery (T2wflair). These images were taken from patient's TCGA-Q06-213 from the TCGA-GBM collection of the Cancer Imaging Archive (TCIA).

Figure 2 . 13 :

 213 Figure 2.13: Diffuse midline glioma H3K27M mut . (A) Conventional MRI: T2 (axial section), contrast-enhanced T1 (axial section), FLAIR (coronal section). (B) PWI: dynamic susceptibility contrast (DSC) curve (red and orange: tumor ROIs; green: normal-appearing white matter (NAWM)), DSC-cerebral blood volume (CBV) (red: tumor ROI), arterial spin labeling (ASL)-cerebral blood flow (CBF). DSC ROI evaluation revealed "aggressive" perfusion features (red ROI: CBV max , 4 mL/100 g; CBV max /CBV N AW M , 6.48); ASL-CBF showed highly perfused spots within the tumor tissue. (C) DWI: a clinically feasible single-slice ROI evaluation revealed low diffusion parameters (blue ROI: ADC mean , 0.87 mm 2 /s; ADC mean /ADC N AW M ratio, 1.19). (D) Q-ball tractography of corticospinal tracts, exhibiting a mild ventrolateral dislocation of the left tract due to the mass effect of the lesion. Reprinted from [95] under the Creative Commons Attribution 4.0 International.

Figure 2 .

 2 14 shows an example of typical short TE spectra from glioblastoma multiforme and intracerebral metastases.

Figure 2 . 14 :

 214 Figure 2.14: Example of MRS spectra for GBM and metastasis (typical short TE spectra). (A) Glioblastoma multiforme and (B) intracerebral metastases. Reprinted from

  dures. The images are exported in DICOM (Digital Image and Communication in Medicine) format to the institutional archiving system (PACS, Picture Archiving and Communication System). Traditionally for simplicity in research, these data are converted to the Neuroimaging Informatics Technology Initiative (NIfTI) format, an open file format [1] commonly used to store brain imaging data obtained by magnetic resonance imaging devices.

Figure 3 . 1 :

 31 Figure 3.1: Outline of the two kinds of radiomic workflow applied to MR images. After image acquisition, the images undergo different preprocessing steps such as bias field inhomogeneity correction, spatial resampling, skull-stripping, and standardization. The region of interest is then segmented either manually, semi-automatically, or automatically. Note that the segmentation step can take place before the preprocessing. For conventional radiomics, handcrafted features (histogram, shape, texture) are extracted from the region of interest after discretizing the grey levels. Machine learning algorithms are then used to determine the clinical outcome of interest. The deep-learning method allows to automatically extract features and determine the clinical outcome of interest, either directly from the previously defined segmentation or by including the segmentation step directly in the process.

BFigure 3 . 2 :

 32 Figure 3.2: Representation of radiomic feature types. (A) Histogram or first-order features reflect voxel intensity distribution only. (B) Textural or second-order features derived from texture matrices (eg, co-occurrence, run length, size-tone, difference, dependence) reflect the complex and unique spatial arrangement of voxels. Adapted by permission from [Copyright Clearance Center]: [Elsevier] [International Journal of Radiation Oncology • Biology • Physics] [66], [COPYRIGHT] (2021).

Figure 3 . 3 :

 33 Figure 3.3: Variable selection algorithms.

  From Afshine and Shervine Amidi Machine Learning cheatsheets covering the content of the Stanford CS 229 class.Deep learning networks that are discriminative and/or generative in design may be used to extract radiomic features. As the name implies, discriminative deep models attempt to extract features that differentiate classes (e.g., normal or malignant), and the objective is to minimize the prediction error. Therefore these models may directly categorize samples based on the retrieved features. Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are the most popular discriminative architectures in radiomics. CNNs are made up of three different types of layers, which are convolutional layers, pooling layers, and fullyconnected layers. When these layers are layered on top of one another, a CNN architecture is created.

Figure 3 . 4 :

 34 Figure 3.4: Simplistic deep learning representation. Neural networks are multiple-layered networks inspired by biological neurons. The analogy is as follows: dendrites are inputs, synapses are weights, the cell body is the transfer function, the axon is the element output. Many different deep learning architectures exist, includingCNNs, in image analysis. In CNN, inside specific layers, such as convolutional ones, image inputs are directly scanned using small-sized filters or kernels, resulting in transformed images. Operations such as pooling and convolutions are crucial processes in CNN architectures, offering the most prominent characteristics of the images (e.g., edges). The activation functions, such as rectified linear unit (ReLU), sigmoid, and softmax, as well as regularization (such as dropout), are also essential components of deep learning systems. Currently, no method can be used to determine the appropriate number and type of layers for a particular task. A trialand-error approach is used to develop the best architecture. On the other hand, some established models and their variants are widely used for specific tasks, such as U-Net for segmentation.

[ 1 ]

 1 Larobina, M. & Murino, L. Medical Image File Formats. Journal of Digital Imaging

  relies on the extraction of a wide variety of quantitative image-based features to provide decision support. Magnetic resonance imaging (MRI) contributes to the personalization of patient care but suffers from being highly dependent on acquisition and reconstruction parameters. Today, there are no guidelines regarding the optimal preprocessing of MR images in the context of radiomics, which is crucial for the generalization of published image-based signatures. This study aims to assess the impact of three different intensity normalization methods (Nyul, WhiteStripe, Z-Score) typically used in MRI together with two methods for intensity discretization (fixed bin size and fixed bin number). The impact of these methods was evaluated on first-and second-order radiomics features extracted from brain MRI, establishing a unified methodology for future radiomics studies. Two independent MRI datasets were used. The first one (DATASET1) included 20 institutional patients with WHO grade II and III gliomas who underwent post-contrast 3D axial T1-weighted (T1w-gd) and axial T2-weighted fluid attenuation inversion recovery (T2w-flair) sequences on two different MR devices (1.5 T and 3.0 T) with a 1-month delay. Jensen-Shannon divergence was used to compare pairs of intensity histograms before and after normalization. The stability of first-order and second-order features across the two acquisitions was analysed using the concordance correlation coefficient and the intra-class correlation coefficient. The second dataset (DATASET2) was extracted from the public TCIA database and included 108 patients with WHO grade II and III gliomas and 135 patients with WHO grade IV glioblastomas. The impact of normalization and discretization methods was evaluated based on a tumor grade classification task (balanced accuracy measurement) using five well-established machine learning algorithms. Intensity normalization highly improved the robustness of first-order features and the performances of subsequent classification models. For the T1w-gd sequence, the mean balanced accuracy for tumor grade classification was increased from 0.67 (95% CI 0.61-0.73) to 0.82 (95% CI 0.79-0.84, P =

  stability of first-order and second-order radiomics features across acquisitions. Each patient underwent two MR acquisitions on 1.5 T (Signa EchoSpeed, GE Healthcare, Milwaukee, Wisconsin, USA) and 3 T (Discovery MR750, GE Healthcare) scanners, with a mean interval of 7.4 (± 3.0) days. Inclusion criteria supposed that no clinical or morphological change related to the glioma occurred during this delay. This was certified by a blinded radiologist (SA, 10 years of experience, with 5 years of specialization in neuro-oncology). A post-contrast 3D axial T1-weighted (T1w-gd) sequence and an axial T2-weighted fluid attenuation inversion recovery (T2w-flair) sequence were acquired on each scanner.

  tion problem is addressed by learning a standard histogram from a set of images and linearly mapping the intensities of each image of interest to this standard histogram. The standard histogram is learned by averaging predefined landmarks deduced from histograms of the training set. The intensity landmark configuration C L = [1, 10, 20, 30, 40, 50, 60, 70, 80, 90, 99] (intensity percentiles) chosen in this study corresponds to the one defined by Shah et al. [16].

  were segmented for DATASET1 by an experienced radiation oncologist (GK, 4 years of experience) using the 3D Slicer open-source platform version 4.10.1 (https://www.slicer.org).

108 4. 5

 1085 ResultsA two-way ANOVA test was conducted to simultaneously evaluate the effect of normalization and discretization. If the ANOVA test was statistically significant, a subsequent pairwise post hoc Tukey's multiple comparison test was performed. For both tests, a P value < 0.05 was considered significant.The design of the study is detailed in Fig.4.1. -Enhancing tumor core (ET) -Non-enhancing part of the tumor core (NET) -Peritumoral edema (ED) order GRAY-LEVEL DISCRETIZATION: PYRADIOMICS SOFTWARE Machine Learning -Tumor grade classification Random Forest Naive Bayes Logistic Regression SVM Neural Networks Test & score: cross validation average accuracy Histogram Evaluation: Jensen-Shannon divergence First-order and second-order features evaluation: intra-class correlation coefficient (ICC) concordance correlation coefficient (CCC)

Figure 4 . 1 :

 41 Figure 4.1: Design of the study.

Figure 4 . 2 :

 42 Figure 4.2: Balanced accuracies obtained for the tumor grade classification task using the 18 first-order features only. Bar plots and associated error bars represent the average balanced accuracies and the 95% CIs obtained using all 5 test folds of the crossvalidation of the 5 machine learning models as a function of the normalization method, respectively. (A) T1w-gd MRI sequence only, (B) T2w-flair MRI sequence only.

Figure 4 .

 4 Figure 4.3 illustrates the percentage of the 73 textural features showing ICCs and CCCs higher than 0.8 depending on the intensity normalization and discretization method based on DATASET1. When a relative discretization was used (FBN), the WhiteStripe and Z-Score methods extracted the same feature values as the raw images, which explains the similar plots (Fig. 4.3A,B). Nyul's method provided the highest percentage of robust textural features compared to images without any normalization for the T1w-gd sequence, with a mean difference of 8 percentage points (Fig. 4.3A) for all discretization values. For the T2w-flair sequence, features extracted from original images were more robust than those obtained by Nyul's method (Fig. 4.3B). Between 16 and 128 bins, the percentages of robust features were quite stable, with a maximum variation of 10 percentage points regardless of the sequence and normalization method (Fig. 4.3A,B).

Figure 4 .Figure 4 . 3 :

 443 Figure 4.4 shows the mean balanced accuracies obtained from the five machine learning models trained on the tumor grade classification task (DATASET2) using the 73 textural features only (model 2) for different intensity normalization and discretization methods. No normalization and the WhiteStripe or Z-Score methods led to the same classification performances (Fig. 4.4A,B). Nyul's method resulted in 5% lower performances on average than no normal-

Figure 4 . 4 :

 44 Figure 4.4: Balanced accuracies obtained for the tumor grade classification task using the 73 textural features only. Bar plots and associated error bars represent the average balanced accuracies and the 95% CIs obtained using all 5 test folds of the crossvalidation of the 5 machine learning models as a function of the normalization method and number of bins, respectively. (A) FBN T1w-gd. (B) FBN T2w-flair. (C) FBS T1w-gd. (D) FBS T2w-flair. fixed bin number (relative discretization). FBS fixed bin size (absolute discretization).

4 .

 4 3C) and associated performances on the classification task based on T1w-gd images (Figs.4.2A,C -DATASET2). Nyul's harmonization method, based on a reference histogram, leads to the highest number of robust first-order features (Table4.4). However, it has already been shown that this piecewise linear transformation affects the texture of the images[17]. Additionally, piecewise mapping can be distorted when large tumors are considered. These observations are in accordance with our results showing that different texture feature values were obtained with the Nyul method compared to no normalization and the WhiteStripe and Z-Score methods with FBN discretization (Fig.4.3A,B). WhiteStripe intensity normalization performs a Z-Score normalization based on NAWM values. The WhiteStripe method is dependent on the accuracy of the white matter segmentation, which can affect the quality of the normalization.

  3), they tend to decrease performance in terms of classification accuracy when considering the T2-flair sequence (Fig.4.4B). Goya-Outi et al. investigated the impact of intensity binning combined with WhiteStripe normalization on 30 patients suffering from diffuse intrinsic pontine glioma[57]. They compared patient ranking based on radiomic features to visual assessment of the heterogeneity. The dataset was obtained using a single MR device and included 4 MR sequences (T1w, T1w-gd, T2w and T2w-flair). Three types of intensity binning were compared: (1) a constant bin size and relative bounds (FBS); (2) a constant number of bins and relative bounds (FBN); and (3) a constant number of bins and absolute bounds. For 20 out of the 240 indices, patient rankings obtained with binning (1) and (2) were highly correlated (|r| > 0.7). This number increased to 188 when comparing rankings obtained with binning (2) and (3) and was reduced to 9 when comparing (2) and (3). They subsequently adopted the absolute discretization (1), as it does not require the setting of absolute lower and upper bounds. Goya-Outi et al. have shown similar patient rankings for the large majority of 240 textural features when using different values of FBN(8, 16, 32, 64, 128) or FBS (0.75, 1, 2, 3, 4). More recently, Duron et al. evaluated the influence of grey-level discretization on inter-

  4.3 -DATASET1) regardless of the sequence and normalization method. Regarding the classification, increasing the number of bins above 128 significantly reduced the accuracy of the classification for the T2w-flair sequence for the FBN discretization. Based on our results (Figs. 4.4, S4.1 and S4.2), a number of bins equal to 32 seems to be a good compromise for brain MR analysis after Z-Score normalization, as it leads to the most informative radiomics signatures for both sequences, with acceptable calculation times.Preliminary feature selection based on robustness is widely used in radiomics[58, 59]. In the present study, no improvements in classification performances were observed using feature selection (FigureS4.2, Table4.5). These results suggest that, considering brain MR data for a grade classification task, a step of feature selection based on feature robustness could be optional.Most recently, 2 publications focused on the image preprocessing steps and their impact on radiomic features reproducibility in brain patients. Moradmand et al.[10] evaluated the impact of 5 combinations of image preprocessing on the reproducibility of 1461 radiomic features (i.e., spatial resampling, skull stripping, noise reduction, bias field correction and intensity normalization) extracted from different glioblastoma (GBM) subregions (i.e., oedema, necrosis, enhanced tumor). They showed that radiomic features extracted from necrotic regions were the most reproducible and recommended that, after the bias field correction step, noise filtering should be applied. In that work, no analysis of the optimal preprocessing based on a clinical classification or regression task was performed, making it difficult to compare their results to ours. In 2019, Um et al.[24] studied the impact of image preprocessing methods on 420 radiomic features extracted from MR images from two datasets: 50 patients from the TCGA-GBM dataset and 111 institutional patients. They evaluated five image preprocessing techniques: 8-bit global rescaling, 8-bit local rescaling, bias field correction, histogram normalization and isotropic resampling. Their goal was to evaluate the ability of a machine learning classifier to classify each patient according the cohort to which a patient belongs (covariate shift) depending on the preprocessing step performed. They also assessed the impact of each preprocessing step on an overall survival model. They showed that no single preprocessing step was sufficient to completely remove the machine effect. However, in their cohort, histogram normalization combined with a relative grey-level discretization (16, 32, 64 and 128 bins) was

Figure S4. 1 :

 1 Figure S4.1: Balanced accuracies obtained for the tumor grade classification task using a combination of the 18 first-order and 73 textural features. Bar plots and associated error bars represent the average balanced accuracies and the 95% CI obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a function of the normalization method and number of bins, respectively. (A) FBN T1w-gd. (B) FBN T2w-flair. (C) FBS T1w-gd. (D) FBS T2w-flair. fixed bin number (relative discretization). FBS fixed bin size (absolute discretization).

Figure S4. 2 :

 2 Figure S4.2: Balanced accuracies obtained for the tumor grade classification task using the features defined as robust based on the DATASET1 results. Bar plots and associated error bars represent the average balanced accuracies and the 95% CI obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a function of the normalization method and number of bins, respectively. (A) FBN T1w-gd. (B) FBN T2w-flair. (C) FBS T1w-gd. (D) FBS T2w-flair. fixed bin number (relative discretization). FBS fixed bin size (absolute discretization).

Figure S4. 3 :

 3 Figure S4.3: ROC-AUC obtained for the tumor grade classification task using the 18 firstorder features only. Bar plots and associated error bars represent the average ROC-AUC and the 95% CI obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a function of the normalization method, respectively. (A) T1w-gd MRI sequence only, (B) T2w-flair MRI sequence only.

Figure S4. 4 :

 4 Figure S4.4: ROC-AUC obtained for the tumor grade classification task using the 73 textural features only. Bar plots and associated error bars represent the average ROC-AUC and the 95% CI obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a function of the normalization method and number of bins, respectively. (A) FBN T1w-gd. (B) FBN T2w-flair. (C) FBS T1w-gd. (D) FBS T2w-flair. fixed bin number (relative discretization). FBS fixed bin size (absolute discretization).

Figure S4. 5 :

 5 Figure S4.5: ROC-AUC obtained for the tumor grade classification task using a combination of the 18 first-order and 73 textural features. Bar plots and associated error bars represent the average ROC-AUC and the 95% CI obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a function of the normalization method and number of bins, respectively. (A) FBN T1w-gd. (B) FBN T2w-flair. (C) FBS T1w-gd. (D) FBS T2w-flair. fixed bin number (relative discretization). FBS fixed bin size (absolute discretization).

Figure S4. 6 :

 6 Figure S4.6: ROC-AUC obtained for the tumor grade classification task using the features defined as robust based on the DATASET1 results. Bar plots and associated error bars represent the average ROC-AUC and the 95% CI obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a function of the normalization method and number of bins, respectively. (A) FBN T1w-gd. (B) FBN T2w-flair. (C) FBS T1w-gd. (D) FBS T2w-flair. fixed bin number (relative discretization). FBS fixed bin size (absolute discretization).

Figure 5 . 1 :

 51 Figure 5.1: Study design.

Figure 5 .5. 5 Figure 5 . 2 :

 5552 Figure 5.2 summarizes information extracted from the DICOM metadata and quality metrics derived from the raw DICOM images for T1w-gd and T2w-flair MRI sequences. A similar plot is available for T1w and T2w MRI sequences in Figure S5.1. This parallel coordinate plot facilitates the visualization of multivariate data and the observation of trends.Figure5.2A illustrates the difficulty of the task of assigning a batch label considering all acquisition

Fig. 5 .

 5 3 illustrates the impact of the different harmonization strategies on the features extracted from the WM for the test set for T1w-gd and T2w-flair MRI sequences, Fig. S5.2 for the test for T1w and T2w MRI sequences, and Fig. S5.3 for the validation set for all MRI sequences.

Figure 5 .and 1 .

 51 Figure 5.4 attempts to interpret the clusters by showing the variance within variables and between clusters. The variables have been previously scaled in the same range between 0 and 1. Only the 7 variables showing the highest variance of means between clusters were shown for sake of clarity. Fig. 5.4A,B considers all features (Metadata and QM), while Fig. 5.4C,D is focused on QM only. Only the T1w-gd sequence is considered here. The selected hyperparameters were similar for both with empirical_bayes = T rue, parametric = T rue, use_ref _batch = T rue, metric = distortion, except for the feature reduction method with UMAP for one (Fig. 5.4B) and PCA for the other (Fig. 5.4D). Based on Fig. 5.4A, we can see that 4 clusters were selected for RSD minimization considering all features. Cluster 2 contained only images with both a high number of rows and columns and low repetition

Figure 5 . 3 :

 53 Figure 5.3: Harmonization strength evaluated on the WM features (column) for the different MRI sequences (T1w-gd, first row, and; T2w-flair, second row) on the test set. Point represents the RSD value and, error bar is the 95% CI.

Figure 5 .

 5 Figure 5.5 summarizes the performance results of tumor grade classification in terms of balanced accuracy on the test set for the different MRI sequences (T1w, T1w-gd, T2w, T2w-flair) considering either first-or second-order feature classes only. For the T1w sequence and firstorder features, ComBat ranked first with a median performance for the 5 algorithms of 0.81 (min: 0.66, max: 0.84), while all other methods ranged from 0.70-0.74. Raw data yielded a performance of 0.77 (min: 0.76, max: 0.81). For second-order features, ComBat also ranked

Figure 5 . 4 :

 54 Figure 5.4: Clustering interpretation of AutoComBat for the T1w-gd sequence. (A, C) is feature importance and (B, D) is visual representation of the clustering based on the feature reduction method used in AutoComBat. (A, B) AutoComBat using all features. (C, D) AutoComBat using QM.

Figure 5 . 5 :

 55 Figure 5.5: Balanced accuracy for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) and the different MRI sequences (T1w, T1wgd, T2w, T2w-flair) on the test set for the first and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

  T2w-flair and T1w-gd sequences of 18 brain tumor patients with a limited set of 42 extracted radiomic features. The difference in harmonization realignment was quantified based on a Friedman test in two different regions (WM and tumor volume). Three types of images were considered to this: a raw image, an image normalized by the hybrid white stripe (hWS) method and resampled to a voxel size of 1x1x1 mm, and an image incorporating the previous steps but with the addition of ComBat. Using image preprocessing, an improvement of 19 percentage points for feature distribution realignment was found in WM regions and 38 percentage points in the tumor volume for the T2w-flair sequence compared to the raw image. By adding ComBat, they showed an improvement of 88 percentage points in WM regions of interest and 96 percentage points in tumor volume for the T2w-flair sequence. They concluded that image processing with the addition of ComBat completely eliminated the statistical differences between the radiomic features extracted from images acquired at 1.5T and 3T. Compared to Orlhac et al.[34] study, Combat was applied on raw images directly in the present work,

Figure S5. 2 :

 2 Figure S5.2: Harmonization strength evaluated on the WM features (column) for the different MRI sequences (T1w, first row, and; T2w, second row) on the test set. Point represents the RSD value and, error bar is the 95% CI.

Figure S5. 3 :

 3 Figure S5.3: Harmonization strength evaluated on the WM features (column) for the different MRI sequences (T1w, first row; T1w-gd, second row; T2w, third row, and; T2wflair, fourth row) on the validation set. Point represents the RSD value, and error bar is the 95% CI.

Figure S5. 4 :

 4 Figure S5.4: Balanced accuracy for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) and the different MRI sequences (T1w, T1w-gd, T2w, T2w-flair) on the validation set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 5 :

 5 Figure S5.5: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T1w MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 6 :

 6 Figure S5.6: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T1w-gd MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 7 :

 7 Figure S5.7: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T2w MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 8 :

 8 Figure S5.8: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T2w-flair MRI sequence on the test set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

  Figure S5.9: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T1w MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 10 :

 10 Figure S5.10: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T1w-gd MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 11 :

 11 Figure S5.11: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T2w MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

Figure S5. 12 :

 12 Figure S5.12: F1 score, Precision, Recall and Roc Auc for the tumor grading task for the 5 machine learning models (RF, SVC, XGBoost, KNN, LR) on the T2w-flair MRI sequence on the validation set for the first-and second-order feature types depending on the harmonization method. The vertical dashed line corresponds to the median value of the 5 ML algorithms.

1 .Figure 6 . 1 :

 161 Figure 6.1: Different steps of the study.

Figure 6 . 2 :

 62 Figure 6.2: Flow chart of patient inclusion.

Figure 6 . 4 :Figure 6 . 5 :

 6465 Figure 6.4: Areas under the receiver operating characteristics curve of the radiomic classifier after ten-repeated 5-fold cross-validation (A) and on the test set (B).

Figure 6 . 6 :

 66 Figure 6.6: Examples of 3D representation of a brain metastasis (A) for which the sphericity was equal to 0.76 and a glioblastoma (B) for which the sphericity was equal to 0.45. GBM: Glioblastoma; BM: Brain Metastasis.

Figure 6 . 7 :

 67 Figure 6.7: Four incorrectly classified BM of the test set. Two of them presented tumoral leptomeningitis (arrows, A, B), one a metastatic ventriculitis (C) and the forth one a multilocular lesion (D). Leptomeningitis and ventriculitis may have interfered with spatial delineation of tumor boundaries.

  1, 0.3, 0.5, 0.7, 0.

  "auto", "sqrt", "log2"If a parameter is not mentioned, it corresponds to the default value of version 0.23.2 of the scikit-learn library.

Chapter 7 Contents 7 . 1 Foreword 2 Abstract 3 Introduction 7 . 4 Methods 5 Results 7 . 6 Discussion 7 . 7 Conclusion 7 . 8 References

 77123745767778 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Clinical overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Multimodal Brain Tumor Segmentation challenge 2020 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Neural network architecture . . . . . . . . . . . . . . . . . . . . . 7.4.2 Loss Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.3 Image preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.4 Data augmentation techniques . . . . . . . . . . . . . . . . . . . . 7.4.5 Training details . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.6 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.7 Ablation Study for Pipeline A . . . . . . . . . . . . . . . . . . . . 7.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.1 Online Validation dataset . . . . . . . . . . . . . . . . . . . . . . 7.5.2 Testing dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7. 3 Introduction 7 . 3 . 1

 3731 deep supervision and stochastic weight averaging, on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 training dataset. Two independent ensembles of models from two different training pipelines were trained, and each produced a brain tumor segmentation map. These two labelmaps per patient were then merged, taking into account the performance of each ensemble for specific tumor subregions. Our performance on the online validation dataset with test time augmentation were as follows: Dice of 0.81, 0.91 and 0.85; Hausdorff (95%) of 20.6, 4,3, 5.7 mm for the enhancing tumor, whole tumor and tumor core, respectively. Similarly, our solution achieved a Dice of 0.79, 0.89 and 0.84, as well as Hausdorff (95%) of 20.4, 6.7 and 19.5mm on the final test dataset, ranking us among the top ten teams. More complicated training schemes and neural network architectures were investigated without significant performance gain at the cost of greatly increased training time. Overall, our approach yielded good and balanced performance for each tumor subregion. Our solution is open sourced at: https://github.com/lescientifik/open_brats2020 Clinical overview Gliomas are the most frequent primitive brain tumors in adult patients and exhibit various degrees of aggressiveness and prognosis. Magnetic Resonance Imaging (MRI) is required to fully assess tumor heterogeneity, and the following sequences are conventionally used: T1 weighted sequence (T1), T1-weighted contrast enhanced sequence using gadolinium contrast agents (T1Gd), T2 weighted sequence (T2), and fluid attenuated inversion recovery (FLAIR) sequence.

Figure 7 . 1 :

 71 Figure 7.1: Example of a brain tumor from the BraTS 2020 training dataset. Red: enhancing tumor (ET), Green: non enhancing tumor/ necrotic tumor (NET/NCR), Yellow: peritumoral edema (ED). Upper Left: T2 weighted sequence, Upper Right: T1 weighted sequence, Lower Left: T1-weighted contrast enhanced sequence, Lower Right: FLAIR sequence Middle: T1-weighted contrast enhanced sequence with labelmap overlay

Figure 7 . 2 :

 72 Figure 7.2: Neural Network Architecture: 3D U-Net [9] with minor modifications

  regions. Larger value of Hausdorff distance for ET compared to other tumor subregions is explained by the absence of the ET label for some cases. Consequently, predicting even one voxel of ET would lead to a major penalty for this metric. Example of segmented tumor from the online validation set is displayed in Figure7.3. It is hard to visually discriminate best from the average result, based on the mean dice score per patient (average across the three tumor sub-regions). However, our worst generated mask showed obvious error: contrast enhanced arteries were mislabeled as enhancing tumor.

Figure 7 . 3 :

 73 Figure 7.3: From left to right: ground truth example from the training set, and generated segmentations from our solution for three patients among the online validation set; respectively: best mean dice score (ET:0.95, WT:0.96, TC:0.98), average mean dice score (ET:0.73, WT:0.92, TC:0.93), and worst mean dice score (ET:0.23, WT:0.95, TC:0.13). Red: enhancing tumor (ET), Green: non enhancing tumor/ necrotic tumor (NET/NCR), Yellow: peritumoral edema (ED)

7. 4

 4 zooms in the tumor segmentation of the first two annotations of Figure7.3 (respectively manual ground truth annotations and best validation case).

Figure 7 . 4 :

 74 Figure 7.4: Zoomed version of the first two vignettes of Figure 7.3 Left: ground truth example from the training set. Right: generated segmentations from our solution for the best mean dice score patient on the validation set. Red: enhancing tumor (ET), Green: non enhancing tumor/ necrotic tumor (NET/NCR), Yellow: peritumoral edema (ED). It is interesting to note that both exhibit the same pattern: central non enhancing tumor core with surrounding enhancing ring and diffuse peritumoral edema.

Mean

  ) : 3.2 (0.6) min < med < max: 1.5 < 3 < 5.5 IQR (CV) : 0 (0.2)

3Figure 8 . 2 :

 82 Figure 8.2: Labels segmentations on each MRI modality (T1w, T1w-gd, T2w-flair) in two subjects for a single slice in the axial plane. The first case was a patient without a surgical cavity. The second case was a patient with a surgical cavity. (A) Original data. (B) Labels associated with the sequences used primarily for their delineation. (C) Final label assignment on each MRI sequence.

1 )

 1 was used to extract 105 radiomic features from the 5 labels of each MRI sequence (T1w, T1-gd, T2-flair), increasing the maximum possible number of radiomic indices per patient to 1575. Shape features (14 indices), first-order features (18 indices), and second-order features (73 indices) were computed. The latest also known as texture features were related to the spatial relationships between voxel intensities and computed from 4 matrices, including the Grey Level Co-occurrence Matrix (GLCM, 22 indices), the Grey Level Run Length Matrix (GLRLM, 16 indices), the Neighborhood Grey Tone Difference Matrix (NGTDM, 5 indices) and the Grey Level Dependence Matrix (GLDM, 14 indices). The features were consistent with the features described by the Imaging Biomarker

(

  Fig. 8.4.

(Figure 8 . 4 :

 84 Figure 8.4: Example of 5 typical cases encountered in delineation and associated review of labels. (A) Automatic contours with no corrections. (B) Addition of label 5 (POM). (C) Addition of label 4 (SC) and label 5 (POM). (D) Manual revision with correction of labels. (E) Equivocal case from SC (no resection for this patient).

  brain tumor in adult patients. Accurate risk stratification is critical for the development of a tailored approach to GBM management. The aim of this research is to use one of the stateof-the-art ensemble classifiers, namely XGBoost, in conjunction with Bayesian Optimization to develop a MRI-based prognostic classifier in GBM patients with postoperative imaging. In addition, we explore the use of different stratification modes (2-class, 3-class, and regression) using different subsets of features (clinical, radiomics, and a combination of both). This study included 247 adult GBM patients treated with the STUPP protocol at Gustave Roussy Cancer Campus between 2008 and 2015, which were retrospectively collected. Models were built using a training data set of 185 patients with cross-validation to ensure generalizability and evaluated using a test data set of 62 patients. A total of 1575 radiomics extracted from 5 tumor sub-volumes and 20 clinical features were used. The combination of radiomic features with clinical features did not show superior performance to the use of clinical features only. The increase of considered classes in the classification task implied the use of a higher number of features in the model, i.e., for 2 class and 3 class stratification, the clinical-features-based model used 1 and 17 features respectively, the radiomic-features-based model used 45 and 147 features respectively, and combined clinical & radiomic-features-based models used 5 and 196 respectively. The model using only clinical features for 2-class, 3-class, and regression stratification had a performance of 67% (balanced accuracy), 53% (balanced accuracy), and 0.72 (c-index), respectively on the test set. Missing data were handled by XGBoost which revealed interesting properties with predictive features such as MGMT even with a high missing rate (90%).

to 18 months [ 1 ]

 1 . The standard treatment recommended since 2005 (STUPP protocol) consists of maximum safe surgical resection followed by radiotherapy combined with concomitant and adjuvant chemotherapy[2, 3]. The radiotherapy volumes are delineated on a simulation scanner that is almost systematically fused with postoperative magnetic resonance imaging (MRI). A macroscopic tumor volume (GTV for Gross Tumor Volume), corresponding to the residual contrast and the surgical cavity on the postoperative MRI in T1 sequence with gadolinium injection, is initially segmented. The Clinical Target Volume (CTV), taking into account the microscopic extension of glial disease, is then constructed by adding a three-dimensional isotropic margin of 1.5 to 2 cm to the CTV, but adapted to respect the cerebral anatomical barriers. The radiotherapy dose is finally delivered to the Planning Target Volume (PTV), which consists of adding 3 to 5 mm around the CTV to take into account the patient's movement and residual positioning errors[4]. Despite the high compliance of PTV doses to prescription with new radiotherapy techniques based on intensity modulation, more than 80% of GBM recurrences occur within fields and irradiation volumes reflecting the radioresistant nature of this disease[5, 6]. Several clinical, radiological, and biological factors have been correlated with the prognosis of GBM. The main factors are age, neurological and cognitive status of the patient, tumor location, quality of surgical resection, adjuvant treatments received, loss of expression of O6 -methyl-guanine methyltransferase (MGMT), and the existence of isocitrate dehydrogenase (IDH) mutations. The incidence of GBM increases significantly with age, with

Figure 9 . 1 :

 91 Figure 9.1: Example of a brain tumor from the MMI-PROB dataset (Chapter 8). NCR & NET (label 1): necrotic and non-enhanced part of the tumor core in hyposignal on T1w, ED (label 2): edema corresponding to intense, peri-lesional, vasogenic edema / tumor infiltration on T2w-flair, ET (label 3): enhancing tumor exhibiting hypersignal on T1w-gd compared to T1w, SC (label 4): surgical cavity which had typically strong hyposignal on all MRI sequences, POM (label 5): postoperative modifications corresponding to post-biopsy bleeding or spontaneous bleeding in hypersignal on T1w.

( 42 < 24 <

 4224 OS), it corresponded in months to (3.42 < short ≤ 15.44), (15.44 < medium ≤ 27.24) and (27.24 < long ≤ 123.50) survival. Three main experiences were run: • Experience 1: A classification problem with 2 categories based on the 50th percentile (3.short ≤ 20.40 months) and (20.40 < long ≤ 123.50 months) of the overall survival of full data. Counts for each class in the training data were short = 95 and long = 90. For the testing data, these values were equal to short = 33 and long = 29. • Experience 2: A classification problem with 3 categories based on the 33th and 66th percentiles (3.42 < short ≤ 15.44 months), (15.44 < medium ≤ 27.24 months) and (27.long ≤ 123.50 months). The count values for each class in the training data were short = 62, medium = 62 and long = 61. For the testing data, these values were equal to short = 21, medium = 21 and long = 20.

  8). C-index values are ranging from 0.50 (i.e., a correct ordering in the survival of 1 out of 2 patients) to 1 (the best possible result for perfect classification) as an indicator of good predictors.

Figure 9 . 4 : 2 Figure 9 . 5 : 3 Figure 9 . 6 :

 94295396 Figure 9.4: Top 10 XGBoost feature importance (Gain) for Experiment 1. (A) Clinical features only, (B) Radiomic features only and, (C) Clinical & Radiomic features.

Figure 10 . 1 :

 101 Figure 10.1: Data considered in the determination of the recurrence including 3 MRI sequences (T1w, T2w-flair, T1w-gd). (A) Data at pre-RT time. At the pre-RT time, only the enhancing tumor (ET) is considered for multitasking. (B) Data at recurrence time. (C) A recurrence map is created with a boolean operation from the ET label at recurrence time and pre-RT time. (labels; NCR & NET: necrosis and non-enhancing tumor, ET: enhancing tumor, POM: post operative modifications)

Figure 10 . 2 :

 102 Figure 10.2: Network architecture considering the simple task strategy. 3D-Unet with minor modifications. For training, an input patch size of 128 × 128 × 128 was selected. The letter in each block is the number of channels (C = 48).

Figure 10 . 3 :

 103 Figure 10.3: Sequential multitask 3D U-Net architecture. For training an input patch size of 128 × 128 × 128 was selected. The letter in each block is the number of channels (C = 48). Loss 1 is dedicated to the optimization of the ET segmentation. Loss 2 is dedicated to the optimization of the recurrence segmentation.

Figure 10 . 4 :

 104 Figure 10.4: Parallel multitask 3D U-Net architecture. For training an input patch size of 128 × 128 × 128 was selected. The letter in each block is the number of channels (C = 48). Each loss was dedicated to one of the following tasks: ET segmentation or recurrence segmentation.

Figure 10 . 6 :

 106 Figure 10.6: Recurrence map prediction case with parallel 3D U-Net (DSC: 0.000). The first and second rows are, respectively, MRI sequences at pre-RT and recurrence time.There is no recurrence area for this case. Recurrence prediction by the network is filled in green.

11. 1

 1 General conclusionspossible by implementing an adequate methodology (nested CV for model selection, repeated CV for hyperparameterization, examination of feature importance in the model). Moreover, the developed model was also compared to the performance of a human expert. Performances of the radiomic classifier have been shown to that equal to that of the neuroradiologists with a balanced accuracy of 0.80. Furthermore, the radiomic feature with the largest coefficient value in the classifier, namely sphericity, is partly used to explain the developed model. This consideration has changed the current practice of the Sainte Anne radiologists who were not accustomed to a 3-D visualization of the tumor but rather only a cross-sectional analysis.The third step of the project was to develop a tool for automatic segmentation of brain tumors from conventional multiparametric MRI imaging (Chapter 7). For this purpose, we participated in the International Brain Tumor Segmentation Challenge 2020. Our solution was based on multiple U-net neural networks, mainly with deep supervision and stochastic weight averaging. The brain tumor segmentation task, although challenging, has been solved with good accuracy using a 3D U-net neural network architecture with a carefully designed preprocessing, training, and inference procedure. On the final test dataset, a Dice of 0.79, 0.89 and 0.84, as well as Hausdorff (95%) distances of 20.4, 6.7 and 19.5mm were obtained. Our ensemble ranked fifth among 693 teams registered for the segmentation task.The final step of the project involved several substeps (Chapters 8, 9 and 10). First, we collected a large database of postoperative glioblastomas from the Gustave Roussy Cancer Campus collection, including imaging and clinical data at two time points, i.e., pre-RT and recurrence. Using the automatic segmentation model established in Chapter 7, we were able to facilitate the obtention of the segmentation labels: ET, NCR & NET, and edema. Then, after a manual review of these labels obtained by deep-learning, we added two new cohortspecific labels: postoperative changes and surgical cavity. Chapter 8, therefore, presented the methodology used to preprocess and annotate the data set corresponding to the pre-RT time with the aim of making the cohort publicly available to facilitate research (segmentation task, survival analysis task, etc.). The data concerning the recurrence time was not made public because of its current use in ongoing studies. All the methodology implemented, from image preprocessing, automatic label segmentation, extraction of radiomic features, is presented, and the code has been made available in open source. Then, from these pre-RT data, we tried to determine the overall survival from the radiomic features extracted from conventional multiparametric MRI and clinical data (Chapter 9

  radiomique est comparée à une "biopsie virtuelle", permettant une évaluation dynamique non invasive et reproductible de l'ensemble de la tumeur et de son environnement. Elle désigne la traduction informatique d'images médicales en données quantitatives objectives de haute dimension afin de déterminer, par une analyse ultérieure, généralement à l'aide d'une technique ML, des biomarqueurs d'imagerie qui peuvent être utilisés pour appuyer les décisions cliniques. L'apprentissage profond, un sous-ensemble de l'apprentissage automatique, implique une forme particulière de réseau neuronal artificiel qui imite le système cognitif humain. Il présente l'avantage d'utiliser une approche axée sur les données et ne nécessite pas la définition préalable de caractéristiques par des experts humains. Les algorithmes d'apprentissage profond, tels que les réseaux neuronaux convolutifs (CNN), excellent dans la reconnaissance des formes et la découverte de formes complexes dans les données d'imagerie, ce qui permet d'obtenir de meilleures performances grâce à l'efficacité des caractéristiques générées. La thèse s'est concentrée sur le développement d'aspects méthodologiques et de différentes applications de ML et DL basés sur la radiomique à partir uniquement de l'imagerie par résonance magnétique et de séquences anatomiques des tumeurs cérébrales. Tout d'abord, les intensités IRM souffrent d'une forte dépendance aux paramètres d'acquisition et de reconstruction, rendant vulnérable l'utilisation d'indices radiomiques dans un contexte multicentrique. Par conséquent, une première étude a été menée pour caractériser l'effet du prétraitement des images, en particulier l'harmonisation des images et la discrétisation des niveaux de gris, afin de proposer des recommandations concernant le pipeline optimal à appliquer. Pour cela, un ensemble chevauchant d'images de patients ayant effectué une IRM sur deux machines différentes (l'une avec un champ magnétique de 1,5T et l'autre à 3T) nous a permis d'évaluer la robustesse des caractéristiques radiomiques. De plus, les méthodes ont été appliquées à une tâche d'apprentissage automatique concernant la classification du grade des tumeurs cérébrales sur un jeu de données multicentriques fortement hétérogènes. L'optimisation de ces étapes spécifiques réalisées dans le flux de travail radiomique a amélioré les propriétés de prédiction et de généralisation ainsi que la robustesse des caractéristiques extraites des images. Cela nous a permis de définir des recommandations pour le prétraitement des images et la discrétisation des niveaux de gris dans le contexte radiomique pour les tumeurs cérébrales, ce qui est crucial pour la généralisation des signatures basées sur les images publiées. Pour les modèles basés sur des caractéristiques de premier et de second ordre, la combinaison de la normalisation Z-Score et de la discrétisation absolue semble être la meilleure. En ce qui concerne la discrétisation du niveau de gris, 32 bins semblent être un bon compromis lorsque les séquences T1w-gd et T2w-flair sont considérées. Une deuxième étude s'est concentrée sur la méthode de correction appliquée directement aux caractéristiques extraites, c'est-à-dire la méthode ComBat dérivée de la génomique mais dont l'application est aujourd'hui limitée par l'exigence de la connaissance de l'étiquette de chaque échantillon correspondant à "l'effet de lot" et un nombre minimum d'échantillons par étiquette. Une nouvelle approche nommée Au-toComBat a été proposée pour surmonter ces deux limitations et a été comparée à la méthode traditionnelle ComBat et à une méthode standard de traitement d'images. Les différentes méthodes ont été comparées en fonction de leur capacité à diminuer l'écart-type relatif des caractéristiques radiomiques extraites de la matière blanche et de la tâche de classification des tumeurs cérébrales en utilisant différents modèles d'apprentissage automatique. ComBat, AutoCombat utilisant des métriques de qualité issues de l'image comme données d'entrée pour l'affectation des lots et les méthodes de prétraitement ont présenté des résultats prometteurs sur l'harmonisation de la substance blanche, mais sans consensus clair pour toutes les séquences IRM. Le prétraitement a donné les meilleurs résultats sur la séquence T1w-gd pour la tâche de gradation. Nos résultats dépendent de la séquence, de la classe de caractéristiques et de la tâche et nécessitent des investigations supplémentaires sur d'autres ensembles de données. Nous avons également constaté que bien que Combat soit prometteur, son emploi est confus par la communauté pour des tâches ML. Il s'ensuit qu'une étude portant sur une applicabilité clinique a été réalisée et a consisté à développer une signature capable de différencier les glioblastomes des métastases uniques à la suite d'un projet initié par l'hôpital Sainte-Anne (GHU de Paris). À cette fin, un classificateur ML basé sur des caractéristiques radiomiques extraites d'imagerie MR tridimensionnelle post-contraste pondérée en T1 a été développé. Nous avons également pris soin de développer un modèle ML qui soit aussi explicable et généralisable que possible par la méthodologie utilisée (validation croisée imbriquée pour la sélection du modèle, validation croisée répétée pour l'hyperparamétrage, examen de l'importance des caractéristiques dans le modèle). En outre, le modèle développé a également été comparé à la performance d'un expert humain. Il a pu être démontré que la performance du classificateur radiomique est égale à celle des neuroradiologues avec une précision équilibrée de 0,80. De plus, la caractéristique radiomique ayant la plus grande valeur de coefficient dans le classificateur, à savoir la sphéricité, est partiellement utilisée pour expliquer le modèle développé. Cette explication a changé la pratique actuelle du radiologue qui n'est pas habitué à une visualisation en 3-D de la tumeur mais plutôt à une vue en coupe. Une étape du projet a consisté à développer un outil de segmentation automatique des tumeurs cérébrales à partir de l'imagerie IRM multiparamétrique conventionnelle. À cette fin, nous avons participé à "l'International Brain Tumor Segmentation Challenge 2020" (BraTS). Notre solution était basée sur plusieurs réseaux de neurones U-net, principalement avec une supervision profonde. La tâche de segmentation des tumeurs cérébrales, bien que difficile, a été résolue avec une bonne précision en utilisant une architecture de réseau neuronal U-net 3D avec une procédure de prétraitement, d'entraînement et d'inférence soigneusement conçue. prédiction des zones de rechute) et bénéficieraient de l'utilisation d'une imagerie multimodale avancée et de données complémentaires. Plusieurs modèles radiomiques proposés dans cette thèse ont le potentiel d'être rapidement transférés en clinique, en particulier ceux concernant le diagnostic différentiel, la survie ou la segmentation automatique. 295 Titre : Radiomique et intelligence artificielle appliquées à l'imagerie IRM anatomique : vers un traitement personnalisé des tumeurs cérébrales Mots clés :

Dive deeper in the subject of brain and radiomics Background on brain and imaging Chapter 2 Contents 2.1 Brain tumors

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	2.1.1	Part I	33
	• Appendix B: Substantial French summary

33 2.1.2 Glial tumors . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36 2.1.3 Glioblastoma . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

40 2.1.4 Brain metastases . . . . . . . . . . . . . . . . . . . . . . . . . . .

42 2.1.5 Tumor response assessment criteria . . . . . . . . . . . . . . . . .

45 2.2 Medical imaging in neuro-oncology . . . . . . . . . . . . . . . . .

46 2.2.1 Magnetic resonance imaging . . . . . . . . . . . . . . . . . . . . .

47 2.2.2 Multimodal imaging . . . . . . . . . . . . . . . . . . . . . . . . .

57 2.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

Table 2 .

 2 1: WHO 2016 classification of glial tumors. The term "grade" refers to how distinct tumor cells seem from normal cells when examined under a microscope by a pathologist. The grading is from I to IV with the higher numbers indicating faster growth and greater aggressiveness.

	Grade Tumor entity
		Diffuse astrocytic and oligodendroglial tumors
	II	Diffuse astrocytoma, IDH-mutant Gemistocytic astrocytoma, IDH-mutant
	II	Diffuse astrocytoma, IDH-wildtype
	II	Diffuse astrocytoma, NOS
	III	Anaplastic astrocytoma, IDH-mutant
	III	Anaplastic astrocytoma, IDH-wildtype
	III	Anaplastic astrocytoma, NOS
		Glioblastoma, IDH-wildtype
	IV	Giant cell glioblastoma Gliosarcoma
		Epithelioid glioblastoma
	IV	Glioblastoma, IDH-mutant
	IV	Glioblastoma, NOS
	IV	Diffuse midline glioma, H3 K27M-mutant
	II	Oligodendroglioma, IDH-mutant and 1p/19q-codeleted
	II	Oligodendroglioma, NOS
	III	Anaplastic oligodendroglioma, IDH-mutant and 1p/19q-codeleted
	III	Anaplastic oligodendroglioma, NOS
	II	Oligoastrocytoma, NOS
	III	Anaplastic oligoastrocytoma, NOS
		Other astrocytic tumors
	I	Pilocytic astrocytoma Pilomyxoid astrocytoma
	I	Subependymal giant cell astrocytoma
	II	Pleomorphic xanthoastrocytoma
	III	Anaplastic pleomorphic xanthoastrocytoma
		Ependymal tumors
	I	Subependymoma
	I	Myxopapillary ependymoma
		Ependymoma
	I	Papillary ependymoma Clear cell ependymoma
		Tanycitic ependymoma
	II or III Ependymoma, RELA fusion-positive
	III	Anaplastic ependymoma
		Other gliomas
	II	Chordoid glioma of the third ventricle
	I	Angiocentric glioma
	-	Astroblastoma

Table 2 .

 2 2: Comparison of standard response criteria. Adapted from[80], with permission from Elsevier.

						RANO
		RECIST 1.0 [75] RECIST 1.1 [76]	Macdonald [77]	WHO [78]	(high-grade
						glioma) [79]
	Imaging modality	CT or MRI	CT or MRI	CT or MRI	Not specified	CT or MRI
						Contrast-
	Target lesion	Longest diameter ≥10 mm	Longest diameter ≥10 mm	Minimum size not specified	Minimum size not specified	enhancing lesions with two perpendicular
						diameters ≥10 mm
						At least two
	Maximum					lesions, and up to
	number of CNS	Five	Two	Not specified	All lesions	five lesions in
	target lesions					patients with
						multiple lesions*
	Measurement technique	Unidimensional	Unidimensional	Bidimensional	Bidimensional	Bidimensional
	Shrinkage
	required for	≥30%	≥30%	≥50%	≥50%	≥50%
	partial response
		Required in	Required in
	Confirmatory scans	non-randomised trials where response is the	non-randomised trials where response is the	Required at least 1 month apart	Required at least 4 weeks apart	Required at least 4 weeks apart
		primary endpoint	primary endpoint
						Stable or decreased
	Steroids	Not included	Not included	Stable or decreased	Not included	compared with time of baseline
						scan
	*For patients with multiple lesions, of which only one or two are increasing in size, the enlarging lesions should be considered
	the target lesions and other lesions will be considered as non-target lesions.

Table 2

 2

	.3: Role of imaging techniques in brain tumors. Reprinted from [84] licensed under a Creative
	Commons Attribution 3.0 Unported License.
		Role of Radiology Imaging Techniques
		Detection
			Localization
			Size
			Margins
			Extension
	Preoperatively	Characterization	Midline shift Compression Contrast enhancement
			Vascularity
			Supplying vessels
			Perifocal oedema
		Differentiation	Benign vs malignant
		Staging
		Tumor embolization
		Surgical planning
	Intraoperative	Surgical navigation
		Monitor the effect of treatment
	Postoperatively	Exclude recurrence
		Distinguish recurrent tumor from radia-
		tion necrosis
	2

.2.1 Magnetic resonance imaging

	Table 2.4: Imaging methods and the major utility in brain tumor imaging. Adapted from [85]
	licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License.
	MRI provides three-dimensional numerical images of the brain with sub-millimeter spatial res-
	olution. MRI was developed in the early 1970s and had its first medical application, thanks to
	the research of the American chemist Paul Lauterbur and the British physicist Peter Mansfield,
	both of whom won the Nobel Prize in Medicine in 2003, as well as the American Raymond
	Damadian [86, 87].
	MRI principles
	Nuclear magnetic resonance (NMR) phenomenon
	MRI is a medical imaging technique based on the NMR phenomenon. Magnetic resonance
	applies to atoms with an odd number of nucleons and an intrinsic magnetic moment. This

Imaging technique Major utility in brain tumor imaging

	CT	Mass effect, herniation, hydrocephalus, hemorrhage, calcifications
		Enhancement characteristics, necrosis, extent of the
	Pre and post-contrast T1	enhancing portion of the tumor, post operative modi-
		fications, bleeding
	T2/T2 FLAIR	Peri-tumoral edema (vasogenic and infiltrative), non-enhancing tumor
	T2* susceptibility sequence	Blood products, calcifications, radiation induced
	(SWI)	chronic micro-hemorrhages
	DWI/ADC	Reduced in highly cellular portions of tumor, postop-erative injury
	DTI	Tractography for surgical planning/navigation
	Perfusion (generally DSC)	Tumor/tissue vascularity
	MR spectroscopy	Metabolic profile
	fMRI	Pre-operative functional mapping, research into treat-ment effects
	PET/MR	Potential new radiotracers
	Notes.
	ADC, apparent diffusion coefficient; CT, computed tomography; DSC, dynamic suscepti-
	bility contrast-enhanced; DTI, diffusion tensor imaging; DWI, diffusion weighted imaging;
	FLAIR, fluid attenuated inversion recovery; fMRI, functional magnetic resonance imaging;
	PET, positron emission tomography; SWI, susceptibility weighted imaging.

Table 2 .

 2 6: Visual aspects of brain structures associated with each MRI sequence.

	Region	T1w	T2w	T2w-flair
	Bone	Dark black Dark black Dark black
	Air	Dark black Dark black Dark black
	Fat (within bone marrow)	Bright	Light grey Light grey
	CSF	Dark black	Bright	Dark black
	White matter	Light grey Dark grey Dark grey
	Grey matter	Dark grey Light grey Light grey
	Inflammation (infection, demyelination) Dark black	Bright	Bright

Chapter 3 Contents 3.1 Process of radiomic analysis

  . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

		69
	3.1.1 Imaging data acquisition . . . . . . . . . . . . . . . . . . . . . . .	69
	3.1.2 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	70
	3.1.3 Image preprocessing . . . . . . . . . . . . . . . . . . . . . . . . .	71
	3.1.4 Grey level discretization . . . . . . . . . . . . . . . . . . . . . . .	73
	3.1.5 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . .	73
	3.1.6 Data handling . . . . . . . . . . . . . . . . . . . . . . . . . . . .	75
	3.1.7 AI-based analysis . . . . . . . . . . . . . . . . . . . . . . . . . . .	80
	3.1.8 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . .	85
	3.2

86 3.1 Process of radiomic analysis

	The conventional radiomic analysis workflow involves five main steps, including imaging data
	acquisition with the segmentation of volumes of interest, image preprocessing, grey level dis-
	cretization, feature extraction, and finally, statistical analysis and development and validation
	of multivariate models for the intended clinical task by statistical or machine learning tech-
	niques. Deep-learning-based methods allow automatic feature extraction. The two kinds of
	radiomic workflow are depicted in Figure 3.1.

Deep learning Conventional radiomics

			Statistics	Feature extraction
						• Absolute
						• Relative
	Logistic	SVM	Histogram	Shape	Texture
	Regression
	Random				Preprocessing	Segmentation
	Forest	Acquisition
						• Manual
	Endpoints					• Semi-Automatic
						• Automatic
	Classification Regression			Bias field correction Resampling	Skull stripping	Standardization
	Survival
	analysis
						Segmentation
						may be
						integrated

Table 4 .

 4 1: Normalization methods and grey level discretization applied in recent radiomics studies dedicated to brain tumors.

	Ref.	Multi-center	Number of patients	MRI seq.	Normalization technique	GL discretization	Radiomics software	Features	Objective
									Investigate the feasibility of predicting H3
	Su et al.[26]	No	100	T2w-flair	-	-	Pyradiomics	18 first-order, 13 shape, 54 texture	K27M mutation status by applying an au-tomated machine learning approach to the MR radiomics features of patients with
									midline gliomas
									Develop and validate a model that can be
	Liu et al. [27]	Yes	130	T1w, T2w-flair	ComBat	-	Artificial Intelligence Kit (GE)	First-order, texture	used to predict the individualized treat-ment response in children with cerebral
									palsy
	Bologna et al. [12]	-	Phantom	T1w, T2w	Z-Score	32 FBN	Pyradiomics	18 first-order, 14 shape, 75 texture	Analysis of virtual phantom for prepro-cessing evaluation and detection of a ro-bust feature set for MRI-radiomics of the brain
	Elsheikh et al. [28]	Yes	135	T1w, T1w-gd, T2w, T2w-flair	-	-	-	First-order, texture	Analysis of multi-stage association of glioblastoma gene expressions with tex-ture and spatial patterns
	Tixier et al. [29]	Yes	90	T1w-gd, T2w-flair	-	128 FBN	CERR	72 features (first-order, texture, shape)	Study the impact of tumor segmentation variability on the robustness of MRI ra-diomics features
	Ortiz-			T1w,					Identify the presence of ischaemic stroke
	Ramón	No	200	T2w,	-	32 FBN	MATLAB	114 textures	lesions by means of texture analysis on
	et al. [30]			T2w-flair					brain MRI
				T1w,					Investigate the value of advanced multi-
	Vamvakas et al. [31]	No	40	T1w-gd, T2w,	-	-	MATLAB	11 first-order, 16 texture	parametric MRI biomarker analysis based on radiomics features and machine learn-
				T2w-flair					ing classification for glioma grading
	Tixier et al. [32]	Yes	159	T1w, T1w-gd, T2w-flair	-	128 FBN	CERR	286 features (first-order, shape, texture)	Evaluate the capacity of radiomics fea-tures to add complementary information to MGMT status, to improve the ability to predict prognosis
				T1w,				704 features	Identify the optimal radiomics-based ma-
	Wu et al. [33]	Yes	126	T1w-gd, T2w,	-	-	-	(first-order, shape,	chine learning method for isocitrate dehy-drogenase genotype prediction in diffuse
				T2w-flair				texture)	gliomas
	continued on next page
	103

Table 4 .

 4 1 -continued from previous page

	Ref.	Multi-center	Number of patients	MRI seq.	Normalization technique	GL discretization	Radiomics software	Features	Objective
	Artzi et al. [34]	No	439	T1w-gd	WhiteStripe	-	MATLAB	757 features (first-order, shape, texture)	Differentiate between glioblastoma and brain metastasis subtypes using radiomics analysis
									Investigate the feasibility of tumor type
	Kniep et al. [35]	No	189	T1w, T1w-gd, T2w-flair	WhiteStripe	-	Pyradiomics	18 first-order, 17 shape, 56 texture	prediction with MRI radiomics image fea-tures of different brain metastases in a multiclass machine learning approach for patients with unknown primary lesion at
									the time of diagnosis
				T1w,				2200 features	Predict overall survival in glioblastoma
	Sanghani et al. [36]	Yes	163	T1w-gd, T2w,	-	-	Pyradiomics	(first-order, shape,	multiforme patients from volumetric, shape and texture features using machine
				T2w-flair				texture)	learning
	Liu et al. [37]	Yes	84	T2w	Z-Score	-	MATLAB	131 features (first-order, shape, texture)	Develop a radiomics signature for predic-tion of progression-free survival (PFS) in lower-grade gliomas and investigate the genetic background behind the radiomics signature
								51 features	Distinguish true progression from ra-
	Peng et al. [38]	No	66	T1w-gd, T2w-flair	-	64 FBN	MATLAB	(first-order, shape,	dionecrosis after stereotactic radiation therapy for brain metastases with machine
								texture)	learning and radiomics
	Bae et al. [39]	No	217	T1w-gd, T2w-flair	WhiteStripe	-	Pyradiomics	796 features (first-order, shape, texture)	Investigate whether radiomics features based on MRI improve survival prediction in patients with glioblastoma multiforme (GBM) when they are integrated with clin-ical and genetic profiles
				T1w,				420 features
	Chen et al. [40]	Yes	220	T1w-gd, T2w,	Nyul	-	Pyradiomics	(first-order, shape,	Classify gliomas combining automatic seg-mentation and radiomics
				T2w-flair				texture)
	GL, Grey-Level; FBN, Fixed Bin Number

Table 4 .

 4 2: Datasets description including MR acquisition parameters. Some metadata information are missing (<10% of all patients). For the DATASET2, values representations are: mean [min -max]. The number of patients for each MR system is indicated in brackets. Additional information about DATASET2 are available in Bakas et al[41, 42].DATASET2 included pre-operative multi-institutional scans of The Cancer Genome Atlas (TCGA) Glioblastoma Multiforme (GBM) and Low-Grade Glioma (LGG) collections, publicly available in The Cancer Imaging Archive (TCIA). A total of 135 and 108 exams, including T1w-gd and T2w-flair sequences extracted from the TCGA-GBM and TCGA-LGG cohorts, respectively, were used (Table4.2)[41][42][43].

				DATASET1		DATASET2*
	Sequence	T1w-gd	T2w-flair	T1w-gd		T2w-flair
	Parameters
								Philips AchievaSiemens (17)
								GE Signa Genesis (52)
								GE Signa Excite (71)
								GE Signa HDx (3)
								GE Signa HDxt (8)
								Siemens Magnetom Vision (10)
	Manufacturer model	GE Signa HDxt	GE Dis-covery MR750	GE Signa HDxt	GE Dis-covery MR750		Hitachi Oasis (1) Philips Ingenia (6) Philips Intera (6) Philips Intera Achieva (1) Siemens Avanto (9)
								Siemens Skyra (1)
								Siemens Symphony (10)
								Siemens Trio (2)
								Siemens TrioTim (3)
								Siemens Verio (5)
								Undefined (38)
	Cohort						LGG	HGG	LGG		HGG
							1.16 (N=1),	0.5 (N=2), 1	1.16 (N=1),	0.5 (N=2), 1
	Magnetic Field Strength (T)	1.5	3.0	1.5	3.0	1.5 (N=51), 3.0 (N=47), undefined	(N=1), (N=82), (N=44) unde-1.5 3.0	1.5 (N=51), 3.0 (N=47), undefined	(N=1), (N=82), (N=44) unde-1.5 3.0
							(N=9)	fined (N=6)	(N=9)		fined (N=6)
	TR (ms)	11	10	9802	8000	1106 [6-5500]	890 [5-3286]	9686 11000]	[6000-	9581 11000]	[1000-
	TE (ms)	4	3	157	123	7 [3-17]	9 [2-105]	128 [94-158]	135 [74-355]
	Slice thickness (mm)	1.4	1.2	5.0	3.5	2.4 [1.0-5.0]	3.2 [1.0-6.0]	3.8 [2.0-5.0]	4.14 [1.2-6.0]
	Pixel (mm)	spacing	0.49 x 0.49	0.47 x 0.47	0.47 x 0.47	0.43 x 0.43	0.68 x 0.68 [0.39 x 0.39 -1.02 x 1.02]	0.77 x 0.77 [0.43 x 0.43 -1.02 x 1.02]	0.74 x 0.74 [0.39 x 0.39 -1.01 x 1.01]	0.77 x 0.77 [0.43 x 0.43 -1.01 x 1.01]
	Matrix Dimen-sions	288 x 288	320 x 288	256 x 192	352 x 192	303 x 2130 [224 x 134 -512 x 300]	283 x 204 [224 x 134 -512 x 300]	306 x 214 [256 x 112 -512 x 256]	283 x 194 [192 x 98 -512 x 320]
	FOV (mm)	250	240	240	220	244 [200-260]	235 [200-260]	237 [200-260]	228 [200-260]
	Pixel bandwidth (Hz/px)	65.12 65.12	122	195	166 [81-250]	162 [61-355]	153 [61-358]	170 [61-750]
	Flip angle (°)	17	15	90	90	53 [8-90]	70 [8-90]	100 [90-180]	102 [90-180]
	TR, repetition time; TE, echo time; FOV, field of view

*

Table 4 . 4 (

 44 DATASET1). Nyul's method provided the highest number of robust first-order features based on a threshold value of 0.80 for both intra-class correlation coefficients (ICCs) and concordance correlation coefficients (CCCs) for both T1w-gd and T2w-flair sequences with 16 and 8 features out of 18, respectively. Images without any normalization did not generate any robust feature for the T1w-gd and T2w-flair sequences.

Table 4 .

 4 3: Jensen-Shannon divergences on DATASET1 compared using a Turkey HSD test.

	Turkey HSD (mean difference)

* Significant (P < .05).

Table 4 .

 4 4: Number of first-order features with ICCs and CCCs > 0.80 on DATASET1.

		Number of first-order features
		with ICCs and CCCs > 0.80
	Method	T1w-gd	T2w-flair
	No normalization	0/18	0/18
	Nyul	16/18	8/18
	WhiteStripe	5/18	1/18
	Z-Score	9/18	1/18
			021) and 0.82 (95 %
	CI 0.80-0.85, P = 0.005) when applying the Nyul, WhiteStripe and Z-Score preprocessing
	methods, respectively (DATASET2) (Fig. 4.2A). For the T2w-flair sequence, this value was
	equal to 0.62 (95 % CI 0.59-0.64) when no normalization was applied and 0.56 (95 % CI
	0.52-0.59, P = 0.045), 0.57 (95 % CI 0.54-0.60, P = 0.164), 0.60 (95 % CI 0.57-0.63, P

For the T1w-gd sequence, the average balanced accuracy corresponding to the binary tumor grade classification task obtained from the 5 test folds and the five machine learning models using the 18 first-order features only (model 1) was equal to 0.67 (95 % confidence interval (CI) 0.61-0.73) when no normalization was applied. In comparison, this value was equal to 0.82 (95 % CI 0.79-0.84, P = 0.006), 0.79 (95 % CI 0.76-0.82, P = 0.= 0.770) when the Nyul, WhiteStripe and Z-Score methods were applied, respectively (Fig.

Table 4 .

 4 5 summarizes the mean balanced accuracy and the area under the receiver operating characteristic curve (ROC-AUC) obtained using 32 bins for the T1w-gd and T2w-flair sequences

	The results summarizing the average balanced accuracy and the corresponding 95% CI
	obtained using all 5 test folds of the cross-validation of the 5 machine learning models as a
	function of the normalization method and of the number of bins for models 3 and 4 are available
	in Figures S4.1 and S4.2.
	To illustrate the robustness of the observations independently of the performance metric,
	the results corresponding to the ROC-AUC metric for Figs. 4.2, 4.4, S4.1 and S4.2 are plotted
	in Figures S4.3, S4.4, S4.5 and S4.6, respectively.

based on first-order features only (model 1), second-order features only (model 2), first-and second-order features (model 3) and robust first-and second-order features only (model 4).

For model 4, the number of robust features included in the model, corresponding to features presenting ICCs and CCCs > 0.8 based on the DATASET1 experiment, are indicated in square brackets. In all configurations, model 3 reached a balanced accuracy similar to that of the best model previously obtained using first-order features only or second-order features only, i.e., model 1 for the T1w-gd sequence, except when a configuration including no normalization with FBN was considered, and model 2 for the T2w-flair sequence. Model 4 resulted in the same classification performances as model 3 in terms of balanced accuracy for the T1w-gd sequence, except when no normalization was coupled to FBS discretization. As an example, considering the T1w-gd sequence, Z-Score normalization, and FBN discretization, the mean classification accuracy was equal to 0.82 (95% CI 0.80-0.86) for model 3 and 0.81 (95% CI 0.78-0.84) for model 4. For the T2w-flair sequence, the accuracy decreased regardless of the considered configuration after applying feature selection. All trends were similar when the number of bins was modified (results not shown).

Table 4 .

 4 5: Summary of the average balanced accuracies and the corresponding 95% CI (DATASET2) obtained using all 5 test folds of the cross-validation of the 5 machine learning models (neural network, random forest, support vector machine, logistic regression, naïve Bayes) as a function of the normalization method. For both intensity discretization methods (FBN and FBS), 32 bins were used. For model 4, numbers of robust features as defined using DATASET1 are written in square brackets. BAC balanced accuracy, ROC-AUC area under the receiver operating characteristic curve.

		Model	1		2		3		4
			BAC	ROC-AUC	BAC	ROC-AUC	BAC	ROC-AUC	BAC	ROC-AUC
	T1w-gd
		Z-Score	0.82 (0.80-0.85) 0.91 (0.89-0.93) 0.80 (0.76-0.83) 0.86 (0.83-0.90) 0.82 (0.80-0.86) 0.90 (0.88-0.93) 0.81 (0.78-0.84) [32] 0.91 (0.89-0.94) [32]
	FBS	No normalization 0.67 (0.61-0.73) 0.74 (0.68-0.80) 0.68 (0.62-0.72) 0.75 (0.70-0.79) 0.69 (0.63-0.74) 0.75 (0.68-0.81) 0.58 (0.54-0.61) [9]	0.64 (0.59-0.69) [23]
		Nyul	0.82 (0.79-0.84) 0.90 0.87-0.92) 0.76 (0.74-0.79) 0.83 (0.80-0.86) 0.81 (0.78-0.84) 0.88 (0.85-0.91) 0.82 (0.79-0.85) [40] 0.89 (0.86-0.91) [43]
		WhiteStripe	0.79 (0.77-0.82) 0.88 0.86-0.90) 0.76 (0.72-0.79) 0.84 (0.81-0.87) 0.79 (0.76-0.82) 0.87 (0.84-0.90) 0.79 (0.76-0.82) [20] 0.88 (0.86-0.90) [28]
		Z-Score	0.82 (0.80-0.85) 0.91 (0.89-0.93) 0.78 (0.75-0.82) 0.86 (0.83-0.89) 0.80 (0.77-0.83) 0.90 (0.87-0.93) 0.83 (0.80-0.85) [45] 0.91 (0.88-0.93) [32]
	T2w-flair
	FBN	No normalization 0.62 (0.59-0.64) 0.64 (0.60-0.68) 0.65 (0.62-0.68) 0.70 (0.67-0.73) 0.63 (0.60-0.65) 0.70 (0.66-0.74) 0.60 (0.57-0.63) [23] 0.66 (0.63-0.70) [23]
		Nyul	0.56 (0.52-0.59) 0.61 (0.58-0.65) 0.67 (0.64-0.69) 0.72 (0.70-0.74) 0.66 (0.64-0.69) 0.71 (0.69-0.74) 0.62 (0.59-0.66) [27] 0.67 (0.63-0.70) [27]
		WhiteStripe	0.57 (0.54-0.60) 0.63 (0.60-0.67) 0.65 (0.62-0.68) 0.70 (0.67-0.73) 0.65 (0.62-0.67) 0.70 (0.67-0.73) 0.62 (0.59-0.65) [24] 0.67 (0.64-0.71) [24]
		Z-Score	0.60 (0.57-0.63) 0.65 (0.62-0.69) 0.65 (0.62-0.68) 0.70 (0.66-0.73) 0.67 (0.64-0.70) 0.72 (0.69-0.75) 0.63 (0.60-0.66) [24] 0.68 (0.65-0.72) [24]
	FBS	No normalization 0.62 (0.59-0.64) 0.64 (0.60-0.68) 0.60 (0.58-0.63) 0.64 (0.61-0.68) 0.59 (0.56-0.62) 0.64 (0.60-0.67) 0.56 (0.54-0.59) [7]	0.61 (0.58-0.65) [7]
		Nyul	0.56 (0.52-0.59) 0.61 (0.58-0.65) 0.64 (0.60-0.68) 0.71 (0.67-0.75) 0.62 (0.59-0.66) 0.70 (0.66-0.73) 0.59 (0.55-0.62) [50] 0.64 (0.61-0.68) [50]
		WhiteStripe	0.57 (0.54-0.60) 0.63 (0.60-0.67) 0.63 (0.60-0.66) 0.69 (0.67-0.73) 0.61 (0.58-0.64) 0.69 (0.65-0.72) 0.61 (0.58-0.64) [36] 0.68 (0.65-0.71) [36]
		Z-Score	0.60 (0.57-0.63) 0.65 (0.62-0.69) 0.64 (0.61-0.67) 0.70 (0.67-0.73) 0.64 (0.60-0.67) 0.71 (0.68-0.74) 0.61 (0.58-0.63) [36] 0.66 (0.62-0.69) [36]

FBN No normalization 0.67 (0.61-0.73) 0.74 (0.68-0.80) 0.80 (0.76-0.83) 0.86 (0.82-0.89) 0.76 (0.71-0.81) 0.83 (0.77-0.88) 0.73 (0.70-0.77) [23] 0.83 (0.80-0.86) [23] Nyul 0.82 (0.79-0.84) 0.90 (0.87-0.92) 0.76 (0.72-0.79) 0.83 (0.80-0.86) 0.81 (0.77-0.84) 0.88 (0.86-0.91) 0.81 (0.78-0.84) [43] 0.89 (0.86-0.92) [43] WhiteStripe 0.79 (0.77-0.82) 0.88 (0.86-0.90) 0.80 (0.76-0.83) 0.86 (0.83-0.90) 0.80 (0.77-0.84) 0.89 (0.86-0.92) 0.79 (0.76-0.83) [28] 0.89 (0.87-0.91) [28]

Table S4 .

 S4 1 -continued from previous page Order

of fea- tures Description Adaptation of the ComBat method for MRI radiomics Chapter 5 Contents 5.1 Foreword

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 5.2 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.4 Material and Methods . . . . . . . . . . . . . . . . . . . . . . . . 136 5.4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5.4.2 Batch effect adjustement method . . . . . . . . . . . . . . . . . . 137 5.4.3 AutoComBat approach . . . . . . . . . . . . . . . . . . . . . . . . 139 5.4.4 Image processing approach . . . . . . . . . . . . . . . . . . . . . . 141 5.4.5 Radiomic feature extraction . . . . . . . . . . . . . . . . . . . . . 142 5.4.6 Experiments and analysis . . . . . . . . . . . . . . . . . . . . . . 142 5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 5.5.1 Illustration of the "batch effect" . . . . . . . . . . . . . . . . . . . 144 5.5.2 Evaluation of harmonization strength based on WM . . . . . . . . . 146 5.5.3 Tumor grading performance . . . . . . . . . . . . . . . . . . . . . 148 5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

150 5.7 Supplementary information

  Radiomics is a rapidly evolving field based on computer vision techniques. It refers to the high-throughput extraction of numerous quantitative features (including shape, intensity, or texture) from images that can be used as potential biomarkers[9][10][11]. It has shown promise in brain cancer detection, diagnosis, molecular mutation characterization, prognosis, and outcome prediction in oncology

	described by quantitative scales such as Hounsfield units (HU) or semi-quantitative scales such
	as standardized uptake value (SUV).
	This Chapter presents a study submitted as the following paper: Alexandre Carré, Enzo Bat-
	tistella, Stephane Niyoteka, Eric Deutsch, and Charlotte Robert. AutoComBat: a generic
	method for harmonizing MRI-based radiomic features. Scientific Reports, [Submitted Septem-
	ber 2021].
	133

. . . . . . . . . . . . . . . . . . . . . 154 5.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 5.1 Foreword

Table 5 .

 5 1: Institutional information of patients of Bakas et al.[24] and patients selected in our study.

	Collection	Institutions	N N selected TCGA-ID
	TCGA-GBM Henry Ford Hospital, Detroit, MI	46	46	TCGA-06
		CWRU School of Medicine, Cleveland, OH	9	9	TCGA-19
		University of California, San Francisco, CA	22	22	TCGA-08
		Emory University, Atlanta, GA	6	0	TCGA-14
		MD Anderson Cancer Center, Houston, TX	25	25	TCGA-02
		Duke University School of Medicine, Durham, NC	10	9	TCGA-12
		Thomas Jefferson University, Philadelphia, PA	14	14	TCGA-76
		Fondazione IRCCSInstituto Neuroligico C. Besta, Milan, Italy	3	0	TCGA-27
	TCGA-LGG St Joseph Hospital/Medical Center, Phoenix, AZ	29	29	TCGA-HT
		Henry Ford Hospital, Detroit, MI	52	52	TCGA-DU
		Case Western Reserve University, Cleveland, OH	10	10	TCGA-FG
		Thomas Jefferson University, Philadelphia, PA	16	16	TCGA-CS
		University of North Carolina, Chapel Hill, NC	1	0	TCGA-EZ
	Total		243	232

Table 5 .

 5 2: Summary table of metadata and quality metrics extracted from the raw DICOM files

	Type	Name	Description
	Metadata

  1. A total of 91 features were extracted including 18 first-order and 73 second-order features compliant with the Image Biomarker Standardization Initiative (IBSI), except for the first-order feature Kurtosis, where Kurtosis is calculated using -3 and +3 in the IBSI

and PyRadiomics standards respectively

[26]

. The second-order features corresponded to 22 features from the Grey Level Co-occurrence Matrix (GLCM), 16 features from the Grey Level Run Length Matrix (GLRLM), 16 features from the Grey Level Size Zone Matrix (GLSZM), 5 features from the Neighborhood Grey Tone Difference Matrix (NGTDM) and 14 features from the Grey Level Dependence Matrix (GLDM). Prior to feature extraction, an intensity shifting of 300 was performed to guarantee that the majority of voxel intensities were positive. For each combination, i.e., MR sequence plus region of interest (white matter patches or whole tumor, see section Experiments and analysis), extraction was performed according to a specific bin

Table 5 .

 5 3 summarizes the number of features leading to a RSD lower than the one obtained considering the raw MR images by feature class and MRI sequence (T1w-gd and T2w-flair) on the test set when applying optimal Combat and AutoCombat methods. Data corresponding to the test set of T1w and T2w MRI sequences and validation sets for all MRI sequences are also available in TableS5.3 and S5.4, respectively.

Table 5 .

 5 3: Counts (%) of features for each harmonization method with a RSD (95% CI) lower than the one corresponding to the raw images for the T1w-gd and T2w-flair sequences on the test set. The main part of the table gives the number of features for which the considered method is evaluated as the best one, which is called "Top". Total vs. Raw gives the total number of features for each method that are significantly better compared to Raw.

	Method	first-order (n=18)	glcm (n=22)	Feature class gldm (n=14) glrlm (n=16)	glszm (n=16)	ngtdm (n=5)	(n=91)
	MRI Sequence							Total
	T1w-gd							Top	vs. Raw
	Preprocess	4	5	7	12	7	2	37 (41%) 75 (82%)
	ComBat	3	1	3	6	2	0	15 (16%)	62 (68%)
	AutoComBat All	2	2	0	1	5	1	11 (12%)	49 (54%)
	Metadata	3	3	1	2	5	1	15 (16%)	56 (62%)
	QM	16	20	8	10	10	4	68 (75%)	68 (75%)
	T2w-flair
	Preprocess	13	20	11	10	13	4	71 (78%)	71 (78%)
	ComBat	10	14	11	9	11	4	59 (65%)	72 (79%)
	AutoComBat All	6	5	8	7	8	2	36 (40%)	47 (52%)
	Metadata	12	12	6	4	10	1	45 (49%)	55 (60%)
	QM	2	0	2	0	1	0	5 (5%)	7 (8%)

Table S5 .

 S5 1: Parameters space used for the grid search for the ComBat and AutoComBat methods.

	Method		Parameters
		Name	Space
	ComBat	empirical_bayes	[True]	[False]
		parametric	[True, False]
		ref_batch	[0, 1, 2, 3, 4, 5, 6, 7, None]
	AutoComBat* empirical_bayes	[True]	[False]
		parametric	[True, False]
		use_ref_batch	[True, False]
		metric	[distortion, silhouette, calinski_harabasz]
		features_reduction [PCA, UMAP, None]

Table S5 .

 S5 3: Counts (%) of features for each harmonization method with a RSD (95% CI) lower than the one corresponding to the raw images for the T1w and T2w sequences on the test set. The main part of the table gives the number of features for which the considered method is evaluated as the best one, which is called "Top". Total vs. Raw gives the total number of features for each method that are significantly better compared to Raw.

	Method	first order (n=18)	glcm (n=22)	Feature class gldm (n=14) glrlm (n=16)	glszm (n=16)	ngtdm (n=5)	(n=91)
	MRI Sequence							Total
	T1w							Top	vs. Raw
	Preprocess	9	10	11	13	9	3	55 (60%)	85 (93%)
	ComBat	11	15	10	13	12	2	63 (69%) 87 (96%)
	AutoComBat All	8	4	3	3	3	0	21 (23%)	39 (43%)
	Metadata	1	4	0	1	2	0	8 (9%)	26 (29%)
	QM	16	15	8	8	10	3	60 (66%)	61 (67%)
	T2w
	Preprocess	14	9	8	8	8	3	50 (55%)	77 (85%)
	ComBat	15	12	11	12	13	3	66 (73%) 84 (92%)
	AutoComBat All	15	5	4	4	4	0	32 (35%)	52 (57%)
	Metadata	15	4	4	5	4	1	33 (36%)	48 (53%)
	QM	15	13	7	10	7	3	55 (60%)	59 (65%)
	Notes.

For AutoComBat, "All" means the use of Metadata and Quality Metrics. QM = Quality Metrics.

Table S5 .

 S5 4: Counts (%) of features for each harmonization method with a RSD (95% CI) lower than the one corresponding to the raw images for each MRI sequences on the validation set. The main part of the table gives the number of features for which the considered method is evaluated as the best one, which is called "Top". Total vs. Raw gives the total number of features for each method that are significantly better compared to Raw.

	Method	first order (n=18)	glcm (n=22)	Feature class gldm (n=14) glrlm (n=16)	glszm (n=16)	ngtdm (n=5)	(n=91)
	MRI Sequence							Total
	T1w							Top	vs. Raw
	Preprocess	13	14	9	14	9		62 (68%)	87 (96%)
	ComBat	15	16	11	15	12		71 (78%)	89 (98%)
	AutoComBat All	4	5	2	5	3		20 (22%)	31 (34%)
	Metadata	3	6	3	5	3		20 (22%)	35 (38%)
	QM	15	16	9	12	12		67 (74%)	72 (79%)
	T1w-gd
	Preprocess	4	6	8	12	8		40 (44%)	81 (89%)
	ComBat	2	0	2	4	1		9 (10%)	37 (41%)
	AutoComBat All	3	2	2	1	5		14 (15%)	51 (56%)
	Metadata	4	2	2	2	5		16 (18%)	54 (59%)
	QM	16	20	10	11	13	3	73 (80%)	74 (81%)
	T2w
	Preprocess	10	13	4	5	6		40 (44%)	56 (62%)
	ComBat	15	13	11	11	11		64 (70%) 73 (80%)
	AutoComBat All	13	11	4	4	3		36 (40%)	55 (60%)
	Metadata	14	7	5	4	5		36 (40%)	60 (66%)
	QM	15	13	11	11	11		55 (60%)	57 (63%)
	T2w-flair
	Preprocess	13	17	9	10	8		61 (67%)	62 (68%)
	ComBat	9	12	7	9	10		49 (54%)	56 (62%)
	AutoComBat All	11	14	9	11	10		58 (64%)	68 (75%)
	Metadata	10	10	5	4	9		39 (43%)	50 (55%)
	QM	6	3	2	3	3		18 (20%)	21 (23%)

Table 6 .

 6 .2). Median [minimum value -maximum value] 2D maximal diameter was equal to 53.39 mm [24.11mm -88.12 mm] for GBM and 41.40 mm [20.77 mm -77.92 mm] for BM. The test set included 37 patients (21 GBM and 16 BM). In this set, median 2D maximal diameter was equal to 54.93 mm [32.61 mm -102.53 mm] and 33.85 mm [22.41 mm -63.63 mm] for GBM and BM respectively. Patient characteristics and their repartition between Center 1 and Center 2 are summarized in Table 6.1. 1: Demographics and clinical characteristics at diagnosis of the patients included in the training set and in the test set.

	Training set	Test set

Table 6 .

 6 2: Sensitivities, specificities, balanced accuracies, positive predictive values, negative predictive values of the radiomic classifier and of the neuroradiologists (R1, R2, R3, R4, R5) on the test set. Se: Sensitivity; Sp: Specificity; PPV: Positive Predictive Value; PNV: Positive Negative Value; Se p-value : p-value (calculated with McNemar's test) of the difference between the sensibility of the radiomic classifier and the sensibility of the reader; Sp p-value: p-value (calculated with McNemar's test) of the difference between the specificity of the radiomic classifier and the specificity of the reader.

	Reader	Se* Sp* Balanced Accuracy PPV* PNV* Se p-value* Sp p-value*
	Radiomic classifier 0.75 0.86	0.8	0.8	0.82	-	-
	R1	0.88 0.86	0.87	0.82	0.9	0.41	1
	R2	0.94 0.95	0.94	0.94	0.95	0.08	0.16
	R3	0.69 0.76	0.72	0.76	0.69	0.65	0.41
	R4	0.63 0.81	0.72	0.71	0.74	0.48	0.65
	R5	0.81 0.95	0.88	0.93	0.87	0.65	0.16

*

Table S6 .

 S6 1 -continued from previous page

	Model	Parameters	Space
		min_samples_leaf [1, 2, 4]
		max_features	["auto", "sqrt", "log2"]
		class_weight	[None, "balanced"]
	ExtraTrees	n_estimators	200
		criterion	["gini", "entropy"]
		max_depth	[3, 4, 5, 6, 7, 8]
		min_samples_split [0.005, 0.01, 0.05, 0.10]
		min_samples_leaf [0.005, 0.01, 0.05, 0.10]
		max_features	["auto", "sqrt", "log2"]
		class_weight	[None, "balanced"]
	GaussianNB	var_smoothing	[1e-9, 1e-8, 1e-7, 1e-6, 1e-5]
	GradientBoosting	learning_rate	[0.15, 0.1, 0.05, 0.01, 0.005, 0.001]
		n_estimators	200
		subsample	[0.8, 0.9, 1]
		min_samples_split [0.005, 0.01, 0.05, 0.10]
		min_samples_leaf [0.005, 0.01, 0.05, 0.

Table S6 .

 S6 2: Ranked performance of all considered associations combining 9 feature scaling methods and 16 classifiers, obtained using a 5x5 nested cross-validation. Ranking was based on the best generalization score, i.e., the mean and standard deviation (Std) of the Balanced Accuracy corresponding to the outer loop.

	Table S6.2 -continued from previous page
				Mean	Std
	Rank Scaler	Model	outer	outer
				score	score
	89	Normalizer	ExtraTreeClassifier	Mean 0.610 0.060 Std
	Rank Scaler 90 Normalizer	Model QuadraticDiscriminantAnalysis 0.604 0.118 outer outer
	91	No Scaler	KNeighborsClassifier	score 0.584 0.073 score
	1 92	PowerTransformer-yeo-johnson LogisticRegression Normalizer LinearSVC	0.862 0.041 0.570 0.049
	2 93	StandardScaler Normalizer	LinearSVC GaussianNB	0.841 0.045 0.549 0.035
	3 94	RobustScaler No Scaler	LogisticRegression MLPClassifier	0.835 0.048 0.542 0.051
	4 95	PowerTransformer-yeo-johnson LinearSVC Normalizer SGDClassifier	0.834 0.041 0.513 0.017
	5 96	QuantileTransformer-uniform RidgeClassifier No Scaler SGDClassifier	0.834 0.054 0.507 0.014
	6 97	QuantileTransformer-normal MinMaxScaler	SGDClassifier BernoulliNB	0.834 0.073 0.501 0.041
	7 97	QuantileTransformer-normal QuantileTransformer-uniform BernoulliNB MLPClassifier	0.834 0.083 0.501 0.041
	8 98	QuantileTransformer-normal Normalizer	LogisticRegression LogisticRegression	0.828 0.076 0.499 0.021
	9	PowerTransformer-yeo-johnson RidgeClassifier	0.827 0.057
	10	StandardScaler	LogisticRegression	0.822 0.087
	11	QuantileTransformer-uniform MLPClassifier	0.814 0.081
	12	MaxAbsScaler	RidgeClassifier	0.813 0.071
	13	MinMaxScaler	RidgeClassifier	0.811 0.087
	14	MinMaxScaler	MLPClassifier	0.806 0.077
	15	StandardScaler	MLPClassifier	0.801 0.057
	16	PowerTransformer-yeo-johnson QuadraticDiscriminantAnalysis 0.800 0.037
	17	QuantileTransformer-normal	LinearSVC	0.800 0.051
	18	RobustScaler	LinearSVC	0.800 0.052
	19	QuantileTransformer-uniform LinearSVC	0.800 0.082
	20	PowerTransformer-yeo-johnson SGDClassifier	0.793 0.046
	21	No Scaler	LinearSVC	0.790 0.038
	22	PowerTransformer-yeo-johnson MLPClassifier	0.786 0.056
	23	MinMaxScaler	SGDClassifier	0.786 0.058
	24	StandardScaler	SGDClassifier	0.785 0.081
	25	RobustScaler	RidgeClassifier	0.783 0.055
	26	QuantileTransformer-uniform LogisticRegression	0.780 0.040
	27	MinMaxScaler	LogisticRegression	0.778 0.059
	28	MinMaxScaler	LinearSVC	0.778 0.070
	29	No Scaler	LogisticRegression	0.771 0.048
	30	MaxAbsScaler	MLPClassifier	0.770 0.067
	31	RobustScaler	SGDClassifier	0.766 0.058
	continued on next page

  Henry, Alexandre Carré, Marvin Lerousseau, Théo Estienne, Charlotte Robert, Nikos Paragios, and Eric 201 Deutsch. Brain Tumor Segmentation with Self-ensembled, Deeply-Supervised 3D U-Net Neural Networks: A BraTS 2020 Challenge Solution. In Alessandro Crimi and Spyridon Bakas, editors, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in ComputerScience, pages 327-339, Cham, 2021. Springer International Publishing.

Table 7 .

 7

	1: Merging procedure of the two labelmaps. 0: background, 1: necrotic and non-
	enhancing tumor core (NET), 2: peri-tumoral edema (ED), 4:enhancing tumor (ET)
			Model A
			0 1 2 4
			0 0 1 0 4
		model B	1 0 1 2 4 2 2 1 2 4
			4 0 1 2 4
	Table 7.2: Ablation study: results from cross-validation on the training set.
	Dice: mean(std)	ET	WT	TC
	U-Net like	0.8077 (0.011) 0.9070 (0.006) 0.8705 (0.013)
	+ Patients removal 0.8126 (0.019) 0.9043 (0.005) 0.8686 (0.012)
	+ Attention block	0.8144 (0.022) 0.9037 (0.008) 0.8701 (0.018)
	label, and similarly for the edema between the TC and the WT label (N onEnhanching =

T C -ET ; edema = W T -T C).

Table 7 .

 7 3 displays the results for the online validation data. Our models produced a Dice metric

	greater than 0.8. for each tumor region. Our two-pass merging strategy had no impact on the
	ET and TC segmentation performance of the pipeline A's ensemble, while greatly improving
	209

Table 7 .

 7 3: Performance on the complete BraTS'20 Online Validation Data for the merging strategy, unless otherwise specified.

	Metric (mean)	ET	WT	TC
	Dice (Pipeline A alone) 0.80585	0.89518 0.85415
	Dice (Pipeline B alone) 0.72738	0.91123 0.84921
	Dice	0.80585	0.91148 0.85416
	Sensitivity	0.81488	0.91938 0.84485
	Specificity	0.99970	0.99915 0.99963
	Hausdorff (95%)	20.55756 4.30103 5.69298
	WT segmentation. Single pass strategy already yielded good performance for all three tumor

Table 7 .

 7 4: Performance on the BraTS'20 Testing Data.

	Metric (mean)	ET	WT	TC
	Dice	0.78507	0.88595 0.84273
	Sensitivity	0.81308	0.91690 0.85934
	Specificity	0.99967	0.99905 0.99964
	Hausdorff (95%) 20.36071 6.66665 19.54915

  Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 8.4.1 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 8.4.2 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 8.4.3 Segmentation of Glioblastoma sub-regions . . . . . . . . . . . . . . 229 8.4.4 Computer-aided segmentation methodology . . . . . . . . . . . . . 231 8.4.5 Radiomic features . . . . . . . . . . . . . . . . . . . . . . . . . . 232 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 8.6.2 Segmentation labels . . . . . . . . . . . . . . . . . . . . . . . . . 233 8.6.3 Contour review and manual correction . . . . . . . . . . . . . . . . 234

		The MMI-PROB database Chapter	8
		constitution
	Contents
	8.1
	8.2
	8.3	Background & Summary . . . . . . . . . . . . . . . . . . . . . . . 218
	8.4
	8.5	Data Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
	8.6	Technical Validation . . . . . . . . . . . . . . . . . . . . . . . . . 233
	using 3D U-Net like neural network architecture, with a carefully crafted preprocessing, training and inference procedure. We open-sourced our training pipeline at https://github.com/ lescientifik/open_brats2020, allowing future researchers to build upon our findings, and improve our segmentation performance. 8.6.1 8.7

Table 8 . 1

 81 -continued from previous page

	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
	Event (death = 1)	1. 0	12 ( 4.9%)		0
	[factor]	2. 1	235 (95.1%)		(0.0%)
	Sex	1. F	93 (37.7%)		0
	[character]	2. M	154 (62.3%)		(0.0%)
	Diagnosis.Age	Mean (sd) : 57.8 (12.3)	240 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		19 < 60 < 80.9
		IQR (CV) : 14.5 (0.2)
	WHO.performance.status	1. 0	9 ( 3.6%)		0
	[factor]	2. 1	200 (81.0%)		(0.0%)
		3. 2	38 (15.4%)
	Karnofsky.Performance.Score	1. 60	16 ( 6.5%)		0
	[factor]	2. 70	22 ( 8.9%)		(0.0%)
		3. 80	72 (29.1%)
		4. 90	127 (51.4%)
		5. 100	10 ( 4.0%)
	Extent.of.Resection	1. GTR	92 (37.2%)		0
	[factor]	2. NO	66 (26.7%)		(0.0%)
		3. STR	89 (36.0%)
	Resection.>.90%	1. NO	155 (62.8%)		0
	[factor]	2. YES	92 (37.2%)		(0.0%)
	MGMT.Methylation	1. NO	8 (36.4%)		225
	[factor]	2. YES	14 (63.6%)		(91.1%)
	IDH.Mutation	1. NO	30 (83.3%)		211
	[factor]	2. YES	6 (16.7%)		(85.4%)
	RT.Technique	1. IMRT	9 ( 3.6%)		0
	[factor]	2. RT3D	238 (96.4%)		(0.0%)
	Gliadel	1. NO	89 (73.6%)		126
	[factor]	2. YES	32 (26.4%)		(51.0%)
	TMZ.Concomitant	1. NO	1 ( 0.4%)		0
	[factor]	2. YES	246 (99.6%)		(0.0%)
	TMZ.Concomitant.mg	Mean (sd) : 125.8 (21.1)	0 : 1 ( 1.0%)		143
	[numeric]	min < med < max:	70 : 1 ( 1.0%)		(57.9%)
		0 < 130 < 160	100 : 17 (16.3%)
		IQR (CV) : 20 (0.2)	120 : 31 (29.8%)
			130 : 11 (10.6%)
			135 : 1 ( 1.0%)
			140 : 28 (26.9%)
			150 : 13 (12.5%)
			160 : 1 ( 1.0%)
	Avastin.Concomitant	1. NO	243 (98.4%)		0
	[factor]	2. YES	4 ( 1.6%)		(0.0%)
	Corticosteroids.Concomitant.mg	Mean (sd) : 45.9 (38.1)	12 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		0 < 40 < 240
		IQR (CV) : 40 (0.8)
	Corticosteroids.Concomitant.mg/kg	Mean (sd) : 0.7 (0.6)	95 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		0 < 0.6 < 4.4
		IQR (CV) : 0.6 (0.9)
	continued on next page

Table 8 . 1

 81 -continued from previous page

	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
	TMZ.Adjuvant	1. NO	22 ( 8.9%)		1
	[factor]	2. YES	224 (91.1%)		(0.4%)
	TMZ.Adjuvant.cycles	Mean (sd) : 5 (3.1)	12 distinct values		4
	[numeric]	min < med < max:			(1.6%)
		0 < 6 < 12
		IQR (CV) : 3 (0.6)
	Avastin.Adjuvant	1. NO	237 (96.7%)		2
	[factor]	2. YES	8 ( 3.3%)		(0.8%)
	Corticosteroids.Adjuvant.mg	Mean (sd) : 39.9 (36.8)	12 distinct values		17
	[numeric]	min < med < max:			(6.9%)
		0 < 40 < 160
		IQR (CV) : 60 (0.9)
	Corticosteroids.Adjuvant.mg/kg	Mean (sd) : 0.6 (0.5)	83 distinct values		17
	[numeric]	min < med < max:			(6.9%)
		0 < 0.5 < 2.2
		IQR (CV) : 0.9 (0.9)

GTR, Gross Tumor Resection; STR, Subtotal Tumor Resection; MGMT, O[6]-methylguanine-DNA methyltransferase; IDH, isocitrate dehydrogenase; RT, Radiation Therapy; TMZ, Temozolomide; sd, Standard deviation; IQR, Interquartile range; CV, Coefficient of variation

Table 8 . 2 :

 82 T1w data specifications.

	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
	Rows	Mean (sd) : 503.2 (44.6)	256 : 7 ( 2.8%)		0
	[numeric]	min < med < max:	384 : 3 ( 1.2%)		(0.0%)
		256 < 512 < 512	512 : 237 (96.0%)
		IQR (CV) : 0 (0.1)
	Columns	Mean (sd) : 502.9 (45.5)	256 : 7 ( 2.8%)		0
	[numeric]	min < med < max:	348 : 2 ( 0.8%)		(0.0%)
		256 < 512 < 512	384 : 1 ( 0.4%)
		IQR (CV) : 0 (0.1)	512 : 237 (96.0%)
	Vox_X	Mean (sd) : 0.5 (0.1)	0.43!: 2 ( 0.8%)		0
	[numeric]	min < med < max:	0.45!: 5 ( 2.0%)		(0.0%)
		0.4 < 0.5 < 0.9	0.47!: 173 (70.0%)
		IQR (CV) : 0 (0.2)	0.49!: 11 ( 4.5%)
			0.51!: 40 (16.2%)
			0.55!: 5 ( 2.0%)
			0.57!: 1 ( 0.4%)
			0.59!: 1 ( 0.4%)
			0.62!: 2 ( 0.8%)
			0.90!: 7 ( 2.8%)
			! rounded
	Vox_Y	Mean (sd) : 0.5 (0.1)	0.43!: 2 ( 0.8%)		0
	[numeric]	min < med < max:	0.45!: 5 ( 2.0%)		(0.0%)
		0.4 < 0.5 < 0.9	0.47!: 173 (70.0%)
		IQR (CV) : 0 (0.2)	0.49!: 11 ( 4.5%)
			0.51!: 40 (16.2%)
			0.55!: 5 ( 2.0%)
			0.57!: 1 ( 0.4%)
			0.59!: 1 ( 0.4%)
			0.62!: 2 ( 0.8%)
			0.90!: 7 ( 2.8%)
			! rounded
	continued on next page

Table 8 . 2

 82 -continued from previous page

	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
	Vox_Z	Mean (sd) : 2.5 (1.2)	1.00 : 84 (34.0%)		0
	[numeric]	min < med < max:	1.20 : 1 ( 0.4%)		(0.0%)
		1 < 3 < 5.5	2.00 : 2 ( 0.8%)
		IQR (CV) : 2 (0.5)	3.00 : 132 (53.4%)
			3.50 : 2 ( 0.8%)
			4.00 : 10 ( 4.0%)
			5.00 : 15 ( 6.1%)
			5.50 : 1 ( 0.4%)
	PixelBandwidth	Mean (sd) : 99 (44.4)	61.05!: 134 (54.3%)		0
	[numeric]	min < med < max:	75.12!: 1 ( 0.4%)		(0.0%)
		61.1 < 61.1 < 288	122.07 : 11 ( 4.5%)
		IQR (CV) : 78.4 (0.4)	130.00 : 2 ( 0.8%)
			139.49!: 84 (34.0%)
			159.00 : 1 ( 0.4%)
			162.77!: 2 ( 0.8%)
			173.00 : 4 ( 1.6%)
			203.00 : 7 ( 2.8%)
			288.00 : 1 ( 0.4%)
			! rounded
	Manufacturer	1. GE Medical Systems	232 (93.9%)		0
	[character]	2. Philips Healthcare	1 ( 0.4%)		(0.0%)
		3. Philips Medical Systems	12 ( 4.9%)
		4. Siemens	2 ( 0.8%)
	ModelName	1. Achieva	12 ( 4.9%)		0
	[character]	2. Aera	2 ( 0.8%)		(0.0%)
		3. Discovery MR750	2 ( 0.8%)
		4. Discovery MR750w	86 (34.8%)
		5. Ingenia	1 ( 0.4%)
		6. Optima MR450w	1 ( 0.4%)
		7. Signa Excite	134 (54.3%)
		8. Signa HDxt	9 ( 3.6%)
	MagneticField	1. 1.5	1.50 : 146 (59.1%)		0
	[factor]	2. 3	3.00 : 101 (40.9%)		(0.0%)
	EchoNumbers	1 distinct value	1 : 247 (100.0%)		0
	[numeric]				(0.0%)
	EchoTime	Mean (sd) : 5.1 (3.4)	27 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		2.3 < 5.2 < 20
		IQR (CV) : 2.3 (0.7)
	EchoTrainLength	Mean (sd) : 1.6 (9.4)	1 : 244 (98.8%)		0
	[numeric]	min < med < max:	2 : 2 ( 0.8%)		(0.0%)
		1 < 1 < 149	149 : 1 ( 0.4%)
		IQR (CV) : 0 (5.9)
	InversionTime	1 distinct value	0 : 145 (100.0%)		102
	[numeric]				(41.3%)
	RepetitionTime	Mean (sd) : 76.3 (198.1)	67 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		6.5 < 12.4 < 1197
		IQR (CV) : 4.8 (2.6)
	FlipAngle	Mean (sd) : 32.5 (21.2)	10 : 1 ( 0.4%)		0
	[numeric]	min < med < max:	12 : 84 (34.0%)		(0.0%)
		10 < 35 < 111	35 : 134 (54.3%)
		IQR (CV) : 23 (0.7)	70 : 8 ( 3.2%)
			75 : 4 ( 1.6%)
			90 : 15 ( 6.1%)
			111 : 1 ( 0.4%)
	continued on next page

Table 8 . 3

 83 -continued from previous page

	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
	ModelName	1. Achieva	14 ( 5.7%)		0
	[character]	2. Aera	2 ( 0.8%)		(0.0%)
		3. Discovery MR750	3 ( 1.2%)
		4. Discovery MR750w	86 (35.0%)
		5. Ingenia	1 ( 0.4%)
		6. Optima MR450w	1 ( 0.4%)
		7. Signa Excite	130 (52.8%)
		8. Signa HDxt	9 ( 3.7%)
	MagneticField	1. 1.5	1.50 : 142 (57.7%)		0
	[factor]	2. 3	3.00 : 104 (42.3%)		(0.0%)
	EchoNumbers	1 distinct value	1 : 246 (100.0%)		0
	[numeric]				(0.0%)
	EchoTime	Mean (sd) : 4.2 (1.1)	39 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		2.8 < 5.2 < 5.2
		IQR (CV) : 2.3 (0.3)
	EchoTrainLength	Mean (sd) : 9.9 (36.7)	1 : 231 (93.9%)		0
	[numeric]	min < med < max:	110 : 8 ( 3.3%)		(0.0%)
		1 < 1 < 218	131 : 1 ( 0.4%)
		IQR (CV) : 0 (3.7)	149 : 1 ( 0.4%)
			173 : 1 ( 0.4%)
			215 : 2 ( 0.8%)
			218 : 2 ( 0.8%)
	InversionTime	1 distinct value	0 : 141 (100.0%)		105
	[numeric]				(42.7%)
	RepetitionTime	Mean (sd) : 19.9 (108)	75 distinct values		0
	[numeric]	min < med < max:			(0.0%)
		6 < 12.3 < 1210
		IQR (CV) : 4.8 (5.4)
	FlipAngle	Mean (sd) : 24.2 (11.6)	8 : 14 ( 5.7%)		0
	[numeric]	min < med < max:	10 : 1 ( 0.4%)		(0.0%)
		8 < 35 < 35	12 : 87 (35.4%)
		IQR (CV) : 23 (0.5)	15 : 5 ( 2.0%)
			17 : 9 ( 3.7%)
			35 : 130 (52.8%)
	ReconstructionDiameter	Mean (sd) : 245.9 (10.5)	220.00!: 1 ( 0.4%)		2
	[numeric]	min < med < max:	230.00 : 1 ( 0.4%)		(0.8%)
		220 < 240 < 300	240.00 : 164 (67.2%)
		IQR (CV) : 10 (0)	243.53!: 3 ( 1.2%)
			250.00 : 18 ( 7.4%)
			256.00 : 11 ( 4.5%)
			260.00 : 38 (15.6%)
			280.00 : 6 ( 2.5%)
			280.00!: 1 ( 0.4%)
			300.00 : 1 ( 0.4%)
			! rounded

Table 8 .

 8 4: T2w-flair data specifications.

	Variable	Stats / Values	Freqs (% of Valid)	Graph	Missing
	Rows	Mean (sd) : 501.7 (48.7)	256 : 8 ( 3.2%)		0
	[numeric]	min < med < max:	320 : 2 ( 0.8%)		(0.0%)
		256 < 512 < 512	400 : 1 ( 0.4%)
		IQR (CV) : 0 (0.1)	512 : 236 (95.5%)
	Columns	Mean (sd) : 501.7 (48.7)	256 : 8 ( 3.2%)		0
	[numeric]	min < med < max:	320 : 2 ( 0.8%)		(0.0%)
		256 < 512 < 512	400 : 1 ( 0.4%)
		IQR (CV) : 0 (0.1)	512 : 236 (95.5%)
	Vox_X
	[numeric]

Table 8 .

 8 6: Mean intensity ranges (IR) and corresponding selected bin width used for radiomic features extraction for each segmentation label and MRI sequences.

	Label	T1w mean IR bin width	T1w-gd mean IR bin width	T2-flair mean IR bin width
	1	495	15	467	14	753	23
	2	477	14	489	15	648	20
	3	473	14	849	26	747	23
	4	545	17	560	17	579	18
	5	770	24	770	24	858	26

  Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 An XGBoost model using Bayesian hyperparameter optimization to predict overall survival in adult glioblastomas from postoperative multiparametric MRI. Envisioned paper: Frontiers in Oncology, [In Preparation].In this study, we sought to explore the overall survival of the dataset presented in chapter 8 using radiomic and clinical features. We have decided to focus only on the optimization of a machine learning model, which is the XGBoost model, a state-of-the-art model. We therefore analyzed the ability of models designed with different feature classes (clinical, radiomics, and clinical and radiomics) to predict a survival task, evaluated according to different modes of stratification (in 2 classes, 3 classes, or by regression) using a Bayesian optimization. The particularity of this study, in comparison with those already published, is the use of postoperative MRI data. The performances obtained are consistent with previously published data and demonstrate the almost null contribution of imaging data to such a task.

	Robert.
	Determination of overall survival Chapter	9
	using radiomics and clinical
	MMI-PROB data
	Contents
	9.1
	9.7
	9.1 Foreword
	This Chapter presents a study in preparation for a publication as the following paper: Alexan-
	dre Carré, Guillaume Klausner, Samir Achkar, Théo Estienne, Théophraste Henry, Angela
	Rouyar, Roger Sun, Grégoire Fournier, Frédéric Dhermain, Eric Deutsch, and Charlotte

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 9.2 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 9.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 9.4 Material and methods . . . . . . . . . . . . . . . . . . . . . . . . 242 9.4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 9.4.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 9.4.3 Experimental design . . . . . . . . . . . . . . . . . . . . . . . . . 245 9.4.4 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . 246 9.4.5 Searching domain of hyperparameters in XGBoost . . . . . . . . . . 246 9.4.6 Statistical analysis and performance evaluation criteria . . . . . . . 247 9.4.7 Flowchart of the experiments . . . . . . . . . . . . . . . . . . . . 249 9.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 9.5.1 Patient population . . . . . . . . . . . . . . . . . . . . . . . . . . 250 9.5.2 Experiments results . . . . . . . . . . . . . . . . . . . . . . . . . 252 9.6

Table 9 .

 9 Statistical analyses (Tables 9.2 and 9.3) validated the correctness of the balance between the training and test sets, with no variable highlighting any significance. Table9.2 summarizes the available clinical characteristics, while Table9.3 considers the missing data. No statistical 2: Descriptive table summarising the clinical characteristics of patients by dataset.Missing data are not considered here.

	difference between the two data splits (p = 0.6) was identified for OS. Median OS was 20.8
	months (95% CI 18.3-23.3) in the train set and 19.4 months (95% CI 16.5-23.3) in the test set
	(Fig. 9.3).
		Train N=185	Test N=62	p.overall
	Sex:			0.244
	F	74 (40.0%)	19 (30.6%)
	M	111 (60.0%)	43 (69.4%)
	Diagnosis.Age	60.5 [51.9;66.5] 58.4 [51.5;64.5]	0.343
	WHO.performance.status:			0.384
	0	7 (3.78%)	2 (3.23%)
	1	153 (82.7%)	47 (75.8%)
	2	25 (13.5%)	13 (21.0%)
	Karnofsky.Performance.Score:			0.477
	60	11 (5.95%)	5 (8.06%)
	70	14 (7.57%)	8 (12.9%)
	80	58 (31.4%)	14 (22.6%)
	90	95 (51.4%)	32 (51.6%)
	100	7 (3.78%)	3 (4.84%)
	Extent.of.Resection:			0.873
	GTR	68 (36.8%)	24 (38.7%)
	NO	51 (27.6%)	15 (24.2%)
	STR	66 (35.7%)	23 (37.1%)
	Resection.>.90%:			0.902
	NO	117 (63.2%)	38 (61.3%)
	YES	68 (36.8%)	24 (38.7%)
	MGMT.Methylation:			0.602
	NO	6 (33.3%)	2 (50.0%)
	YES	12 (66.7%)	2 (50.0%)
	IDH.Mutation:			0.573
	NO	25 (86.2%)	5 (71.4%)
	YES	4 (13.8%)	2 (28.6%)
	RT.Technique:			1.000
	IMRT	7 (3.78%)	2 (3.23%)
	RT3D	178 (96.2%)	60 (96.8%)
	Gliadel:			0.963
	NO	69 (74.2%)	20 (71.4%)
	YES	24 (25.8%)	8 (28.6%)
	TMZ.Concomitant:			1.000
	NO	1 (0.54%)	0 (0.00%)
	continued on next page
	250

Table 9 .

 9 2 -continued from previous page

		Train N=185	Test N=62	p.overall
	YES	184 (99.5%)	62 (100%)
	TMZ.Concomitant.mg	120 [120;140]	135 [120;140]	0.383
	Avastin.Concomitant:			1.000
	NO	182 (98.4%)	61 (98.4%)
	YES	3 (1.62%)	1 (1.61%)
	Corticosteroids.Concomitant.mg	40.0 [20.0;60.0] 40.0 [22.5;60.0]	0.777
	Corticosteroids.Concomitant.mg/kg	0.68 (0.58)	0.63 (0.53)	0.546
	TMZ.Adjuvant:			0.982
	NO	17 (9.24%)	5 (8.06%)
	YES	167 (90.8%)	57 (91.9%)
	TMZ.Adjuvant.cycles	6.00 [2.25;6.00] 6.00 [3.00;6.00]	0.554
	Avastin.Adjuvant:			0.683
	NO	176 (96.2%)	61 (98.4%)
	YES	7 (3.83%)	1 (1.61%)
	Corticosteroids.Adjuvant.mg	40.0 [0.00;60.0] 30.0 [0.00;60.0]	0.782
	Corticosteroids.Adjuvant.mg/kg	0.56 [0.00;0.89] 0.37 [0.00;0.80]	0.464

Table 9 .

 9 3: Table of missing data by dataset. Table9.4). The clinical model used here 17 features (Fig.9.5A), the radiomic model used 147 features (Fig.9.5B) and the clinical and radiomic model used 496 features (Fig.9.5C).In Experiment 3, the models considering clinical and clinical & radiomic features have similar performance with a C-index of 0.72 in the test set. The AIC was 5, 66 and 390 for the clinical,

	Train N=185	Test N=62	p.overall

Table 9 .

 9 4: Experimental results for the different strategies (taking into account clinical, radiomic, clinical & radiomic characteristics) on the validation and test sets. The plus-minus sign (±) indicates the 95% confidence interval.

	Features used to design the model

Table 9

 9 One of the advantages of XGBoost is that it can handle missing values which is a major advantage when data are not present, which may be the case for segmentation labels in some patients (Surgical cavity, Enhancing tumor... ), or clinical data not available (IDH mutation, MGMT methylation...). For these missing clinical data, analyses were not routinely performed before 2016. In our case, the best model always selected the passthrough methodology, i.e., no imputation value was performed, letting the model handle missing values, regardless of the experience and the set of features used. We have shown that even with a very high number of missing data, some characteristics can be predictive, notably, the IDH status with more than 80% of missing data, which was in the top five most important features for experiments 2 and 3. Thus the model seems to determine the predictive dominance of these characteristics from a small number of patients with available information. It could also be seen that the clinical variables related to the treatment (even adjuvant treatments, which can be problematic in defining a baseline signature) were informative for the prognosis notably Corticosteroids Concomitant in mg/kg, and Corticosteroids Adjuvant in mg/kg. In addition, the T1w-gd and T2w-flair sequences stand out as the most informative sequences, which is in agreement with the medical expertise We achieved a reliable and reproducible estimate of patient survival using postoperative MRI by combining several sets of features. Notably, these survival estimates were obtained by repeated cross-validation on retrospective data and then confirmed in a separate set. The use of such cross-validation methods improves generalizability and assures that this model and its findings can be repeated in subsequent studies. One downside is that we used an only retrospective, single-center cohort, and it would have been beneficial to test the model on prospective, multicenter data to completely ensure generalizability. Another limitation is the stochastic nature of the algorithm due to the use of the GPU implementation, which led to the non-associativity of the floating-point summation. Moreover, we have selected only the

	Concomitant in mg/kg) and had a balanced accuracy of 67%. For the 3-class stratification,
	the model designed with clinical variables used only 17 variables and had a balanced accuracy
	of 53%. Similarly, considering the model designed with radiomic features, the 2-and 3-class
	stratification required the use of 45 and 147 features, and had a balanced accuracy of 62%
	and 48%, respectively. The regression model integrated clinical features, used 12 features, and
	yielded a C-index of 0.72. Thus, the regression model required fewer features than a 3-class
	model suggesting that the regression problem, which at first glance seems more complicated,
	could be explored foremost in prognostic studies.

.5

, where this number did not exceed 7 (sex, age, KPS (binary, score ≥ 80 or <80), tumor location, tumor volume, MGMT promoter status, and extent of surgery (biopsy, partial resection, or gross total resection)). In addition, no statistically significant difference was shown comparing the performance of the different designed models (clinical, radiomics, clinical & radiomics). However, it can be seen that the more complicated the stratification mode for classification, i.e., 2 classes to 3 classes, the highest the number of features used in the final model. For the 2-class stratification, the model designed with clinical variables used only one variable (Corticosteroids best model based on our validation, and it is possible that models using different feature sets lead to similar performance. Indeed, this was shown for different feature design models, as for example, in experiment 1, where the clinical & radiomic-features-based model showed only radiomic features among the most important features, with similar performance to the clinicalfeatures-based model. However, to remain consistent and not bias the exploration, experiences were run only once, thus the performance reported.

Finally, the workflow described in this study can serve as a model for future research on the development of XGBoost models and provide insights into the importance of clinical variables for survival in glioblastoma. It can also be used as a guide to thoroughly examine the effect of hyperparameter optimization on model efficiency.

Table 10 .

 10 1: Models performance on the training set with 5-fold cross-validation. Task 1 indicates the determination of the ET area. Task 2 indicates the determination of the recurrence area. The loss for each task is indicated (BCE: Binary Cross-Entropy, Dice BCE is the sum of Dice and BCE). DSC: Dice Similarity Coefficient, BAC: balanced accuracy, Sens: sensitivity, Spe: specificity.

	Network	Loss		Task 1			Task 2
			DSC	BAC	Sens	Spe	DSC	BAC	Sens	Spe
	Simple U-Net	BCE	-	-	-	-	0.036 ± 0.029	0.390 ± 0.098	0.034 ± 0.031	0.746 ± 0.207
		Dice + BCE	-	-	-	-	0.206 ± 0.028	0.655 ± 0.035	0.358 ± 0.048	0.952 ± 0.041
		Dice	-	-	-	-	0.218 ± 0.041	0.675 ± 0.055	0.399 ± 0.092	0.952 ± 0.043
	Sequential	BCE	0.710 ± 0.074	0.824 ± 0.046	0.687 ± 0.081	0.962 ± 0.036	0.037 ± 0.045	0.398 ± 0.086	0.034 ± 0.045	0.761 ± 0.161
		Dice + BCE	0.773 ± 0.033	0.885 ± 0.025	0.802 ± 0.044	0.968 ± 0.024	0.200 ± 0.037	0.646 ± 0.050	0.343 ± 0.065	0.950 ± 0.039
		Dice	0.780 ± 0.035	0.891 ± 0.034	0.814 ± 0.041	0.968 ± 0.031	0.023 ± 0.019	0.458 ± 0.028	0.020 ± 0.019	0.896 ± 0.040
		Dice	0.770 ± 0.026	0.864 ± 0.021	0.766 ± 0.039	0.962 ± 0.028	0.213 ± 0.028	0.679 ± 0.039 0.407 ± 0.089	0.951 ± 0.042
	Parallel	BCE	0.753 ± 0.025	0.863 ± 0.021	0.751 ± 0.031	0.974 ± 0.024	0.038 ± 0.045	0.426 ± 0.053	0.036 ± 0.045	0.817 ± 0.087
		Dice + BCE	0.785 ± 0.042	0.886 ± 0.019	0.798 ± 0.037	0.974 ± 0.024	0.214 ± 0.030	0.663 ± 0.054	0.365 ± 0.117	0.962 ± 0

.032 Dice 0.817 ± 0.029 0.901 ± 0.027 0.828 ± 0.039

					0.974 ± 0.024	0.030 ± 0.032	0.414 ± 0.069	0.028 ± 0.032	0.800 ± 0.119
	Dice	0.768 ± 0.052	0.877 ± 0.041	0.789 ± 0.058	0.965 ± 0.035	0.207 ± 0.026	0.674 ± 0.040	0.398 ± 0.075	0.950 ± 0.038

  Tumeurs cérébrales, Glioblastome, Métastases cérébrales, Radiomique, IRM, Segmentation automatique, Standardisation d'image, Prédiction de survie, Prédiction de rechute Les tumeurs cérébrales malignes sont classées en tumeurs primaires, principalement les gliomes, dont la forme la plus agressive est le glioblastome (GBM) et en métastases cérébrales (BMs). Lors de la suspicion d'une tumeur, l'imagerie par résonance magnétique (IRM) cérébrale est la modalité non invasive privilégiée pour l'évaluation de la maladie. Elle joue un rôle crucial dans le diagnostic, la caractérisation, la planification du traitement et le suivi de la maladie. La référence en matière de diagnostic repose sur une biopsie, non sans risque, et pouvant être source d'erreurs. L'interprétation des images médicales est quant à elle limitante par l'oeil humain, même entrainé. Les analyses informatiques telles que l'apprentissage automatique (ML) et l'apprentissage profond (DL) basées sur la radiomique présentent un potentiel important en oncologie. La radiomique est le processus d'extraction d'indices quantitatifs à haut débit à partir d'images médicales. Elle permet l'extraction de données et la mise en place de systèmes d'aide à la décision, visant à améliorer notamment la détection et la gradation des lésions, la caractérisation des mutations et la conception de modèles pronostics et prédictifs. DL se distingue par sa capacité à identifier et à déchiffrer des motifs complexes dans des images médicales, ce qui lui permet de transformer l'analyse qualitative et contextuelle à quantitative et reproductible. La thèse s'est concentrée sur le développement d'aspects méthodologiques et de différentes applications de ML et DL basés sur la radiomique à l'imagerie anatomique des tumeurs cérébrales. Tout d'abord, les intensités IRM souffrent d'une forte dépendance aux paramètres d'acquisition et de reconstruction, rendant vulnérable l'utilisation d'indices radiomiques dans un contexte multicentrique. Par conséquent, une première étude a été menée pour caractériser l'effet du prétraitement des images, en particulier l'harmonisation des images et la discrétisation des niveaux de gris, afin de proposer des recommandations sur le pipeline optimal à appliquer. Une deuxième étude a porté sur la méthode de correction appliquée directement aux caractéristiques extraites, à savoir la méthode ComBat issue de la génomique, mais dont l'application est aujourd'hui limitée par l'exigence de la connaissance du label de chaque échantillon correspondant à "l'effet de lot" et d'un nombre minimum d'échantillons par label. Une nouvelle approche a été proposée pour surmonter ces deux limitations et comparée à la méthode traditionnelle ComBat et à une méthode standard de traitement d'image.

	Résumé :

(50) -Primary breast cancer n (%) 13 (18.0) -3 (18) -Melanoma n

(%) 9 (12.5) -2 (12.5) -Primary colo-rectal cancer n (%) 5 (6.9) -0 (0) -Primary Clair cell carcinoma n (%) 4 (5.6) -1 (6.3) -Other primary cancer * n (%) 12 (16.7) -2 (12.5) -* Primary rare cancer: choriocarcinoma, sarcoma, salivary gland carcinoma, papillary carcinoma of the thyroid.

Acknowledgements

I would like to express my deep gratitude to Ms. Charlotte Robert, who mentored me during this work. This work is the result of three years of collaboration with her. I thank her for giving me the opportunity to do research in her team and for trusting me throughout my PhD journey. I would also like to thank Mr. Eric Deutsch, who welcomed me for three years in the research unit he heads Molecular Radiotherapy and Therapeutic Innovation (INSERM U1030 -RaMo-

Features name Grey level run length matrix (GLRLM) Short Run Emphasis (SRE) Long Run Emphasis (LRE) Grey Level Non-Uniformity (GLN) Grey Level Non-Uniformity Normalized (GLNN) Run Length Non-Uniformity (RLN) Run Length Non-Uniformity Normalized (RLNN) Run Percentage (RP) Grey Level Variance (GLV) Run Variance (RV) Run Entropy (RE) Low Grey Level Run Emphasis (LGLRE) High Grey Level Run Emphasis (HGLRE) Short Run Low Grey Level Emphasis (SRLGLE) Short Run High Grey Level Emphasis (SRHGLE) Long Run Low Grey Level Emphasis (LRLGLE) Long Run High Grey Level Emphasis (LRHGLE) Neighbouring grey tone difference matrix (NGTDM) Coarseness Contrast Busyness Complexity Strength Grey level dependence matrix (GLDM) Small Dependence Emphasis (SDE) Large Dependence Emphasis (LDE) Grey Level Non-Uniformity (GLN) Dependence Non-Uniformity (DN) Dependence Non-Uniformity Normalized (DNN) Grey Level Variance (GLV) Dependence Variance (DV) Dependence Entropy (DV) Low Grey Level Emphasis (LGLE) High Grey Level Emphasis (HGLE) Small Dependence Low Grey Level Emphasis (SDLGLE) Small Dependence High Grey Level Emphasis (SDHGLE) Large Dependence Low Grey Level Emphasis (LDLGLE) Large Dependence High Grey Level Emphasis (LDHGLE) continued on next page

Code availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 8.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 8.1 Foreword This Chapter presents a study in preparation for a publication as the following paper: Alexandre Carré, Guillaume Klausner, Samir Achkar, Théo Estienne, Théophraste Henry, Angela Rouyar, Roger Sun, Grégoire Fournier, Frédéric Dhermain, Eric Deutsch, and Charlotte Robert. Multimodal imaging for personalized radiation therapy of brain tumors (MMI-PROB) dataset with postoperative MRI glioblastoma including segmentation labels, clinical and radiomic features. Envisioned paper: Nature Scientific Data, [In Preparation].

there is no rule of thumb. A good starting point would be to split the data into 70% train, 15% validation, 15% test, or 80-10-10, 60-20-20. Figure 3.5: Simplistic representation of validation schemes. K-fold cross-validation splits all samples (n) into equal-sized groups known as folds (if K = n, this is identical to the Leave-One-Out method). K -1 folds are used to learn the prediction function, and the fold that is left out is utilized for validation. A nested cross-validation (CV) consists of an inner loop CV enclosed inside an outer CV. The inner loop controls model selection and hyperparameter tunning (similar to the validation set), while the outer loop is in charge of error estimates (similar to test set). Random subsampling randomly splits the data set into training and validation subsets. Unlike k-fold cross-validation, where the data set is divided into folds, the split here is random. A single split is formed using random sampling in the hold-out approach. Independent validation uses a new data set, preferably an external data set, for the validation part.

Performance evaluation

The evaluation metrics are related to the type of problem to be assessed. Typically, a classification problem's performance is evaluated using accuracy and the area under the receiver operating characteristic curve (ROC AUC). Accuracy is defined as the ratio of good predictions compared to all predictions. ROC AUC is used to determine the probability of correct decision making considering all threshold settings. It is worth noting that these measures may be insufficient evaluators of performance if the data set has an imbalance of classes. As a result, additional performance measures such as balanced accuracy, sensitivity, specificity, precision, and F1 measure should be included for further assessment. Sensitivity, also known as recall,

Part II

Radiomic features vulnerabilties: assessment and solutions

for machine difference at the outset. Lin [67] defines it as follows:

x + σ 2 y + (µ x -µ y ) 2 (4.7)

Where µ x and µ y are the means for the two variables (here the radiomic features), σ x and σ y are the corresponding variances and ρ is the correlation coefficient between the two variables. In this study, we implemented the ComBat method used for genomic harmonization and proposed modifications to address combat limitations for ML-radiomic applicability. These methods have been compared to the preprocessing recommendations seen in Chapter 4.

Abstract

The use of multicentric data is becoming essential for developing generalizable radiomic signatures. In particular, Magnetic Resonance Imaging (MRI) data used in brain oncology are often heterogeneous in terms of scanners and acquisition, which significantly impact quantitative radiomic features. Various methods have been proposed to decrease dependency, including methods acting directly on MR images, i.e., based on the application of several preprocessing steps before feature extraction or the ComBat method, which harmonizes radiomic features themselves. The ComBat method used for radiomics may be misleading and presents some limitations, such as the need to know the labels associated with the "batch effect." In addition, a statistically representative sample is required and the applicability of a signature whose batch label is not present in the train set is not possible. This work aimed to compare a priori and a posteriori radiomics harmonization methods and propose a code adaptation to be machine learning compatible. Furthermore, we have developed AutoComBat, which aims to automatically determine the batch labels. A heterogeneous dataset consisting of high and lowgrade gliomas coming from eight different centers was considered. The different methods were compared based on their ability to decrease relative standard deviation of radiomic features extracted from white matter and on their performance on a classification task using different machine learning models. ComBat and AutoCombat using image-derived Quality Metrics as inputs for batch assignment and preprocessing methods presented promising results on white matter harmonization, but with no clear consensus for all MR sequences. Preprocessing showed the best results on the T1w-gd sequence for the grading task. Our results are sequence, feature class and task dependent and require further investigations on other datasets.

Introduction

Either for clinical diagnosis, prognosis, and therapy assessment of brain pathologies or neuroscience research, magnetic resonance (MR) imaging is of prime importance. However, MR images are subject to wide quantitative variations inherent to this imaging modality, i.e., MR data acquired for the same patient but on different sites or scanners yield to different MR images [1][2][3]. Additionnal differences can also be attributed to artifacts such as bias field inhomogeneities, noise, motion, ghosting, or spike [4][5][6][7][8]. The major limitation of MRI compared to other imaging modalities is that the signal intensity described in grey values is arbitrary, unlike computerized tomography (CT) and positron emission tomography (PET), which are

Metadata DICOM

Part III

The distinction between glioblastomas and brain metastases

Abstract

Objectives: To differentiate Glioblastomas (GBM) and Brain Metastases (BM) using a a specificity of 87% [78; 97], a balanced accuracy of 86% [80; 92], and an AUC of 92% [87; 97] with cross-validation. Sensitivity, specificity, balanced accuracy and AUC were equal to 75%, 86%, 80% and 85% on the test set. Sphericity 3D radiomic index highlighted the highest coefficient in the logistic regression model. There were no statistical significant differences observed between the performance of the classifier and the experts' blinded examination.

Conclusions:

The proposed diagnostic support system based on radiomic features extracted from post-contrast 3DT1 MR images helps in differentiating solitary BM from GBM with high diagnosis performance and generalizability.

Introduction

Brain Metastases (BM) and Glioblastomas (GBM) are the two most frequent intra-cranial brain tumors in adults [1][2][3]. Currently, Magnetic Resonance Imaging (MRI) is the modality of choice for brain tumor characterization. Usually, BM present an encapsulated contrast enhancement, with regular and well-defined boundaries, whereas GBM have heterogeneous contrast enhancement with very irregular and fuzzy boundaries [4][5][6]. Nonetheless, their morphological characteristics remain very similar on MRI as both are lesions with annular contrast enhancement, necrotic center and a peritumoral zone in T2-weighted and Fluid-Attenuated Inversion Recovery (FLAIR) sequences. Advanced neuroimaging techniques such as perfusion MRI and Magnetic Resonance Spectroscopy (MRS) provide additional information to distinguish between the two tumor types, based on differences in the peritumoral area [7][8][9][10].

Although in the past decades various studies [11][12][13] have evaluated diagnostic performance of perfusion imaging and MRS, they have shown heterogeneous results in distinguishing these two tumor types, resulting in sensitivities and specificities ranging from 64 to 100% and 60 to 100% respectively. This high heterogeneity reflects the difficulty experienced in daily-practice to differentiate the two brain tumors, even using advanced neuroimaging techniques, particularly in the case of differentiating a GBM from a solitary BM revealing an unknown primary cancer (5 to 12% of BM [14,15]). Even though the final diagnostic will be given by histopathological examination and biomolecular analysis of the tumor tissue relying on the 2016 WHO classification [16], the presurgical distinction between these two types of tumors is crucial for adapting treatment strategies: for metastases less than 3-4 cm, a bloc resection or stereotactic radiosurgery will be planned depending on the lesion location [17], while GBM [18] should be treated with maximal safe resection, and concurrent chemoradiotherapy. Radiomics [19][20][21][22] is a recent area of research based on the simple observation that the human eyes have limitations, even those trained for medical image interpretation. Radiomics consists of extracting large numbers of predefined quantitative features from medical images with the ultimate goal of identifying subgroups of biomarkers able to guide patient's care and has shown promise in brain cancer detection, diagnosis, molecular mutation characterization, prognosis and outcome prediction [23][24][25][26][27][28][29]. In our study, we hypothesized that the morphological differences observed on post-contrast 3DT1 MR images would lead to differences in radiomic features between the two tumor types. The aim of this study was to therefore develop a radiomic features-based Machine Learning (ML) classifier, to evaluate its diagnostic performance on an unseen test set of patients, and to compare it to the diagnosis performance of neuroradiologists. A strong lesions studied would enable its generalizability.

In conclusion, we developed a radiomic features-based classifier based on post-contrast 3DT1 MR images that helps in differentiating GBM and solitary metastatic brain tumors with high diagnosis performance. The performance of the radiomic classifier equals that of neuroradiologists however needs to be improved in further studies including feature extraction applied on FLAIR and perfusion sequences. An interesting point is that the radiomic feature with the highest coefficient value in the classifier, namely sphericity, allows an explainability of the developed model. Future studies using this model on larger sets of patients may clarify its role and its benefit in differentiating these two lesions, particularly by a prospective study registered in a trial database.

Supplementary information

An automatic solution for tumor contouring Pipeline B:

The maximum number of training iterations was set to 400. The best model kept was the one with the lowest loss value on the validation set. A batch size of 3 and Adam optimizer with an initial learning of 1e -4 and no weight decay. Cosine annealing scheduler was used.

Common:

In order to train a bigger neural network, float 16 precision (FP16) was used, which reduced memory consumption, accelerated the training procedure, and may lead to extra performance [20].

The neural network was built and trained using Pytorch v1.6 (which has native FP16 training capability) on Python 3.7. The model could fit on one graphic card (GPU).

Inference

Inference was performed in a two-steps fashion. First, models available from each pipeline were ensembled separately, by simple predictions averaging. Consequently, two labelmaps per case, 1 Collection name will be adapted when the data will be online

Pre-processing

All MRI volumes were bias field corrected using the N4ITK algorithm [12] as implemented in ANTsPy which is a Python Library wrapping the C++ biomedical image processing library Advanced Normalization Tools (ANTs) [13]. A mask was supplied to weight the head area equally on each sequence. The mask was computed based on the Otsu Threshold method [14],

and was cleaned to ensure that the resulting head mask had no holes. We also paid attention to negative values in images since N4 involves a log transformation. To this, the intensities of each image were scaled (min-max normalization) to the range 100-1000 to contain only positive values. Then, the scaling was undone with the inverse transform after the bias field correction.

For each patient, the T1w sequence was registered to the T1w SRI-24 atlas [15] (1 mm × 1 mm × 1 mm), which was reoriented to the LPS (left-posterior-superior) image coordinate system.

Then, T1w-gd, T2w-flair, and CT were co-registered to the registered T1w. All registrations were performed with a 12-degrees-of-freedom global linear registration using ANTsPy. The volume corresponding to the T1w MRI modality was then skull-stripped using DL-based brain extraction from multisequence MRI (HD-BET) [16]. The resulting binary mask was filled with a structuring element corresponding to a square connectivity equal to one. Subsequent skullstripping was then performed applying this brain mask on CT and MRI sequences.

Segmentation of Glioblastoma sub-regions

As for the BraTS challenge, three labels were first considered: the necrotic and non-enhanced Step 3: Labels provided by automatic segmentation were reviewed by experts, and two new labels were manually added (SC and POM).

Step 4: Radiomic features were extracted from MRI preprocessed data (Step 1) and labels (Step 3). The normalization step was, however, necessary for steps 2 and 4.

Computer-aided segmentation methodology

To save substantial amount of time in label delineation, a computer-aided segmentation approach was implemented to delineate labels similar to the BraTS dataset, i.e., ET, NCR & NET and ED. Our solution for the BraTS 2020 challenge [17] was used to this, which is a deeply-supervised 3D Unet like neural network. Compared to the previous published study, only one training pipeline i.e., the version of the 3D-Unet with Instance Normalization, was used here for simplicity. In terms of preprocessing, a Z-Score standardization, which homogenizes the image histograms by subtracting (µ brain ), corresponding to the average intensity outstanding results in glioblastoma survival prognosis [44]. Compared to Cox proportional hazards regression, AFT allows the highest discriminatory performance for computing personalized survival curves. Table 9.1 summarizes the associated searching domain and hyperparameter definitions.

Table 9.1: Searching domain of hyperparameters in XGBoost. For Distribution: "uniform" points are sampled uniformly between the lower and upper bounds of the defined space; "log-uniform" points are sampled uniformly between 'log(lower, base)' and 'log(upper, base)' where log has base 10. ["normal", "logistic", "extreme"] -aft_loss_distribution_scale Scaling factor to be applied to the distribution.

Name

[0.1,3.0] uniform

Statistical analysis and performance evaluation criteria

All statistical analyses were performed using R Statistical Software (version 3.6.3; R Foundation for Statistical Computing, Vienna, Austria). The splitting group comparison for clinical data was assessed depending whether the row-variable was considered as continuous normaldistributed (1), continuous non-normal distributed (2), or categorical (3). The following de-where the indices i and j refer to pairs of observations in the sample. Factor ∆ j discards pairs of observations that are not analyzable due to the censoring of the data, i.e., ∆ j = 0. C-index estimates the concordance probability P (η j > η i | T i < T j ) which is calculated by comparing the ranks of two independent pairs of survival times T i , T j and risk scores η i , η j .

The concordance probability measures whether large values of η i , are associated with short survival times T i and inversely.

To compare the performance of Experiments 1 and 2 from multiple classifiers, i.e., using the different feature sets, we used Cochran's Q test [49,50]. If it was significant, i.e., with a p-value less than 0.05, we performed a post-hoc test, i.e., a pairwise McNemar's chi-square test between multiple groups [51]. The p-values were adjusted with the Bonferroni correction for multiple comparisons. For experience 3, the Akaike Information Criterion (AIC) was used [52,53].

Flowchart of the experiments

To ensure maximum reliability of performance evaluation, we used 5-fold cross-validation repeated 10 times so that 50 different hold-out sets were used to estimate model efficacy. The cross-validation was stratified for experience 1 and experience 2. For Bayesian optimization, 300 iterations were performed, corresponding to the number of parameter settings sampled.

Experiments results

Table 9.4 summarizes the performance of the best models obtained for the 3 experiments using the Bayesian Optimization for the 50 hold-out sets (average) and the test set.

In In this study, we investigated whether it was possible to determine the recurrence areas in patients with glioblastoma from conventional postoperative MRI imaging. The data in Chapter 8, which only considered the pre-radiotherapy time, was therefore supplemented by data corresponding to a second time point, i.e., the time of relapse. It should be noted that it was decided not to make public the two imaging times in Chapter 8 and only release the pre-RT time, as relapse data are still part of an ongoing study. Thus, only 199 patients from the previous cohort were eligible for this study, as patients had to present 3 anatomical MRI images (T1w, T1w-gd, T2w-flair) at the two time points of interest. Based on the development implemented in Chapter 7, deep learning methods were used, as well as the exploration of multitasking. It should be noted that the construction of the article is in the form of a short paper.

Abstract

Glioblastomas are the most common primary brain tumors in adult patients with a high lethality. This lethality rate is mostly due to their high proportion of local recurrence despite treatments. Today, radiotherapy is prescribed using a "one fits all" concept, i.e., the same dose is prescribed in the whole Planning Target Volume without any consideration of local tumor aggressiveness. We made the hypothesis that would benefit from voxel-scale radiotherapy management if we were able to determine at baseline future areas of recurrence. We thus investigated whether artificial intelligence using deep-learning models was capable of achieving this based on postoperative MR images only and explored the benefit of multitasking. To do so, we used a dataset of 199 patients with 3 anatomical MRIs who relapsed. Models exploiting multitasking did not show much better performance than a simple model. The best model obtained a DSC of 0.201 on the test set. Using deep-learning and anatomical MRI sequences, the determination of recurrence patterns seems to be a too complicated task. Thus, the use of advanced imaging sequences would surely be beneficial to this task.

Introduction

Glioblastomas (GBM) are the most common primary brain tumors in adult patients. The current treatment regimen includes maximal surgical resection followed by radiation therapy (RT), and temozolomide (TMZ) chemotherapy [1]. GBM prognosis is poor, with a median survival time of 15 months after initial diagnosis and death is caused mainly by tumor recurrence [2]. Indeed, even after the treatment, the tumor can continue to grow (progression) and almost always returns (recurrence). Recurrence is a foregone conclusion, occurring in approximately 90% of patients within the high dose radiation field (local-in-field) [3][4][5]. This propensity for recurrence is explained in part by the proliferation and radioresistance of GBM cells [6], and in particular their ability to infiltrate the edema [7,8]. To accommodate for the possibility of microscopic infiltration, the irradiation margins are very wide, resulting in extensive irradiation of the brain. Additionally, attempts to increase the radiation dose delivered to the target volume beyond the existing regimen have failed to improve patients' prognoses [9][10][11][12]. Thus,

Dice Loss.

The Dice Loss [20]N computed batch-wise and channel-wise, without weighting was written as:

where N is the number of voxels, p i and g i correspond to the predicted and ground truth labels per voxel respectively, and = 1e -5 is added to avoid zero division.

Binary Cross-Entropy Loss.

The Binary Cross-Entropy (BCE) is a measure of the difference between two probability distributions, typically the ground truth and the prediction.

Dice BCE Loss.

It consists of the sum of Dice and BCE loss:

Implementation details

The dataset was randomly split into training (159 patients) and testing (40 patients). Models were trained from scratch using a five-fold cross-validation procedure with a fixed split. In this case, the validation set was only utilized to monitor the network's performance throughout training and to assess the network's performance at the completion of the training scheme.

Models were trained on an Nvidia Tesla V100 (16 GB memory). The PyTorch version 1.9.0 [21] and MONAI version 0.6.0 [22] for utilities were used. Furthermore, FP16 training was used to reduce memory consumption, accelerated the training procedure, and gain a little extra performance [23]. For final testing, ensembling of the models of the five fold was used.

Pre-processing.

Images were cropped to a variable size using the smallest bounding box containing the whole brain to remove unnecessary background. Furthermore, to take into account MRI intensities variations depending on manufacturers, acquisition parameters, and sequences, all images were standardized to zero mean and unit variance only on valued voxels.

Training.

The three MRI sequences were randomly cropped to the fixed size of 128 × 128 × 128 and concatenated along the channel dimension as the model's input. For each fold, the neural network was trained for 135 epochs with an initial learning rate of 1e -4 . At 75% of the training, a cosine decay was performed. A batch size of 1 and the Ranger optimizer [24,25] were used with a weight decay of 1e -5 . Every 3 epochs, the model weights were saved, and the best model kept was the one with the lowest loss value during the on-fly validation. To avoid overfitting, on-the-fly data augmentation techniques were applied as random shift of intensities with an offset comprised between -0.1 and 0.1 (70% probability), random Gaussian noise using a centered normal distribution with a standard deviation of 0.1 (50% probability), random Gaussian smoothing (20% probability), random flip along each axis (70% probability)

and random rotation of 90°along the (x, z) plane (70% probability).

Inference.

In the inference phase, the three MRI sequences were cropped to a minimum bounding box containing the whole brain and zero-padded to be divisible by 8. Then MRI sequences were concatenated along the channel dimension as the input of the model.

Evaluation metrics

The Dice coefficient (DSC) [26] equivalent to F1 score was calculated for evaluating the performance of tumor segmentation as well as balanced accuracy, sensitivity, specificity:

Balanced accuracy = T P + T N T P + T N + F P + F N Sensitivity = T P T P + F N ,

where T P is the true positive, T N is the true negative, F P is the false positive, and F N is the false negative.

Results and Discussion

Table 10.1 shows the results on the 5 fold cross-validation (CV), while Table 10.2 shows the results on the test set for the different metrics. Task 1 shows the prediction of the ET area on the pre-RT images when the model is multi-task. Task 2 shows the prediction of the recurrence area. For each task, the loss used during the optimization is indicated. First, for

Conclusion and perspectives

General conclusions

Nowadays, personalized therapies, including RT, immunotherapy, and targeted therapies, have been developed to improve cancer management. The application of these methods needs a comprehensive understanding of the molecular and biological characteristics of each tumor.

Thus, methods allowing the characterization of tumor phenotypes based on the exploitation of high dimensional data from all types are of growing interest. The subsequent analysis of these data aims at supporting clinical decision-making. In particular, methods focused on medical imaging analysis are grouped under the term "radiomics", a term that emerged a decade ago.

Many studies have shown the contribution of radiomics in brain cancer detection and diagnosis, molecular mutation characterization, prognosis and outcome prediction [1][2][3][4][5][6][7]. However, as mentioned in section 1.2.3, several major obstacles have been reported in the literature such as the size of the patient cohorts analyzed in the studies, the lack of standardization practice, or the model building and interpretation pitfalls. Therefore, the results presented in this thesis have sought to address some of these issues with the global objective of utilizing AI to assist physicians in personalizing cancer treatment to each patient and moving toward precision medicine.

The project's first step was to develop methods capable of reducing the impact of acquisition and reconstruction parameters on the first and second-order radiomic indexes for different conventional MRI sequences (Chapters 4 and 5). A first study was conducted to evaluate the impact of image preprocessing, including image normalization and grey-level discretization steps used in radiomics (Chapter 4). For this, an overlapping set of MR images of patients who carried out a scan on two different machines (one with a magnetic field strength of 1.5T 660 the previous year [14]. This increase of data should thus favor the performance of DL tools.

However, the approach we have developed which have comparable performance to an expert is easily integrable in a radiotherapy workflow and can provide considerable time savings to the clinician in tumor delineation. The work of valorization towards the industry of this tool has been initiated with its integration into the GE tools as part of AI DReAM consortium led by GE Healthcare. Note that we also took part in the challenge of the 2021 year by developing a network based on a Bridge-Unet and improved with a concatenation of max and average poolings for downsampling, Squeeze-and-Excitation (SE) block, Atrous Spatial Pyramid Pooling (ASSP), and EvoNorm-S0 [15][16][17][18].

Finally, from the database of postoperative GBM that we have collected, prognostically relevant groups for treatment intensification could be realized. However, the contribution of radiomics to the clinical variables has not been proven to improve the performance of the prognostic model. Moreover, this task now seems complicated to optimize further. The BraTS challenge also goes in this direction and has chosen not to renew this task of predicting overall survival for GBM for the 2021 edition. However, the lack of clinical data such as IDH, MGMT, etc., is an obstacle to developing a more accurate model. In addition, we could also move towards the analysis of tumor growth between pre-and post-operative images, which could be indicative of patient survival.

The prediction of recurrence areas in GBM patients has proven unsuccessful based on the DL method and the input data used, i.e., MR anatomical images. To respond more successfully to this task, the integration of MRI advanced imaging data (PWI, DWI) or PET seems inevitable. Nevertheless, this research subject would be highly beneficial for the patient and

List of publications

Appendix

A

Journal (peer-reviewed) Sur la base de nos résultats, nous avons démontré que la radiomique est un outil essentiel à considérer pour la mise en oeuvre de la médecine de précision. Cependant, les informations contenues dans les images anatomiques IRM sont insuffisantes pour certaines tâches (p. ex. la