Berardino Barile

Multiple Sclerosis Functional Composite AD Axial Diffusitivity RD Radial Diffusitivity M D Mean Diffusitivity F A Fractional Diffusitivity CPD Canonical Polyadic Decomposition LL-1 Multilinear

Keywords: Multiview Kernel Based Multilinear SVD Approach Using Morphological Grey Matter Features 7, 1 Intuitive pipeline disability class, Brain Connections (idx edges), Interpretability: Percentage number of connections useful for predicting global EDSS score, 8, 7 Local Interpretability: Sensitivity analysis with multiple threshold values Ensemble of Boosting Models Using White Matter Fiber Bundles

Machine Learning (ML) is a subclass of artificial intelligence systems characterized by algorithmic procedures able to detect patterns, learn from them, and execute tasks autonomously. In recent years, the wide availability of powerful hardware and cloud computing has resulted in a broader adoption of ML algorithms in different areas of human lives, from recommendations on social media to its use for processing automation in factories, and its adoption will only grow further in the future. Usually, in the machine learning community, a distinction between two branches of learning algorithms is considered, such as deep learning and statistical learning models. The former is characterized by a brain-like logical structure reflecting the neuronal connections of the human brain network, where valuable patterns from large structured and unstructured data can be obtained. In the latter, a more statistically-based learning approach is considered. With the advent of computers, statistical problems gained popularity both in size and complexity. New ML algorithms development, increased data storage and computational power, led to the new field of data mining. In contrast, statistical and computational problems in biology and medicine created the field of bioinformatics. The application of ML systems started to spread in different contexts of the biomedical domain such as biomedical image analysis, which today represents one of the most active areas of research where new ML systems are developed each year. In particular, brain-related diseases represent a vast and complex area of research where the extraction of valuable insights from complex data structures may represent an important contribution to the evolution of yet incurable diseases such as Multiple Sclerosis (MS). The MS disease represents a chronic pathology of the central nervous system and leading cause of nontraumatic disability in young adults. MS is characterized by inflammation, demyelination and neurodegenerative pathological processes, which cause a wide range of symptoms, including cognitive deficits, physical impairments and irreversible disability. Concerning the diagnosis of the disease, the introduction of Magnetic Resonance Imaging (MRI) has constituted one of the most important revolutions in the last 30 years. Furthermore, advanced MRI techniques, such as brain volumetry, magnetization transfer imaging (MTI) and diffusion-tensor imaging (DTI) are nowadays the main tools for detecting alterations outside visible brain lesions and contributed to our understanding of the pathological mechanisms occurring in normal-appearing white matter. In particular, one of the most interesting approaches for the representation of MR images of the brain is the concept of connectome. The importance of brain connectivity as a valuable tool for the evaluation of intricate networks of anatomical regions of the human brain comes from the realization that the structural properties and functionality of the brain are intrinsically connected. These types of brain networks provide different information, reflecting complementary aspects of the brain circuit and organization. v In this doctoral thesis, the power of ML algorithms is combined with brain connectivity representation of the human brain network for the analysis of degenerated brain tissues of patients affected by MS. Specifically, the analysis of degenerated brain tissues in MS patients is performed considering two types of connectome data, such as white and grey matter graphs. First, a deep learning-based adversarial network model is proposed for the synthetic generation of realistic structural graphs indistinguishable from real data. The use of generative models in the context of brain-related diseases allows a better understanding of the underlying pathological mechanisms forming inside the brain as well as the evaluation of statistical test of hypothesis following a bayesian scheme. Second, grey matter tissue degeneration was analyzed by means of three of the most relevant morphological features, such as cortical thickness, curvature and area. A statistical and machine learning approach was considered, demonstrating that the analysis of grey matter tissue alterations represents a valuable strategy for the characterization of MS patients. Third, disability in MS patients was estimated based on an ensemble of boosting-based models organized following a hierarchical structure (i.e. stacking generalization approach). Such an analysis represents an important contribution toward a better understanding and treatment of the MS disability due to its impact on the quality of life in MS patients. For this reason, the analysis was performed considering both brain connectivity and fiber bundle data, reaching coherent conclusions in accordance with the clinical literature. Also, due to the importance of such an analysis and the impact of disability on the patient's daily activities, an interpretability model is also proposed in order to pin-down the most important line of reasoning followed by the ensemble model for disability estimation. The ability to provide interpretable explanations to the machine learning algorithms, usually considered as black boxes, represents one of the most important characteristics of automated systems. This is especially true in the context of brain-related diseases, such as MS, where the ultimate goal is to provide valuable insights to the neurologist for a better understanding and treatment of clinical symptoms, improving the quality of life of patients. In conclusion, the contribution provided by the present doctoral dissertation represents an effort toward a better understanding of neurodegenerative pathways forming inside the brain of MS patients. The use of automated systems based on machine learning algorithms represents nowadays the state-of-the-art approach to tackling complex problems and we hope one day they will provide important insights and contributions to the treatment of today's incurable diseases.

Introduction 0.1 Research Motivation

Multiple Sclerosis (MS) is an immune inflammatory demyelinating disease of the Central Nervous System (CNS). It represents the primary cause of non-traumatic handicap in young adults. Although the inflammatory phenomena as well as the demyelination of White Matter (WM) fibers are more and more understood, its etiological causes are still unknown. Generally speaking, the primary sign of illness is manifested as an isolated clinical symptom that may evolve in remitting types of clinical events. One patient out of four, however, starts directly with a progressive form. Patients with a remitting form can evolve to a secondary progressive stage after a period between 10 and 20 years. The high variability of the evolution of each patient and the duration of the natural history of the disease provide difficult challenges to the neurologist. Thus disease treatment remains a difficult task. Although symptoms can be treated and sometimes mitigated and the advances in medical literature promise to slow down and even stop the evolution of the disease, nowadays, no cure is yet available. Magnetic Resonance Imaging (MRI) is today the reference tool for the diagnosis of multiple brain diseases like MS as well as for the follow-up of patients. In particular, T1weighted (with and without contrast agent injection) and T2-weighted sequences allow the identification and characterization of WM lesions, the measurement of lesion load and brain tissue atrophy. Major scientific contributions are focused on the automatic detection and segmentation of WM lesions occurring in MS patients. This represents an important task since clinical measurements, such as lesion load and volume as well as shape and location of the lesions, can be automatically obtained using Machine Learning models, such as Deep Learning, which represents the state-of-the-art approach able to obtain outstanding results, reducing the neurologist's workload and ultimately helping him/her in classifying MS patients in their respecting clinical profiles. Notwithstanding, automatic classification of MS patients still remains an open problem. Multiple attempts have been proposed in the literature for MS profile classification based on these clinical measures.

However, only moderate correlation between these clinical measures and patient's clinical conditions have been observed, particularly when his or her disability status is concerned, leading to a clinical paradox. Furthermore, these techniques offer little information about the underlying pathophysiological mechanisms guiding the progression and characterization of each MS patient. For this reason, a better and more advanced approach for the analysis of the human brain network dynamics is in order.

As of today, little attempt has been made to tackle the problem of classification and prognosis of MS patients considering all clinical phenotypes. In fact, most of the works proposed in the literature concentrate on the discrimination between MS patients and healthy controls. Notwithstanding, from a clinical standpoint, comparing MS patients with different clinical conditions represents a more important and challenging task for the discovery of the underlying pathophysiological mechanisms and MS profile characterization. This existing gap might be due to the lack of sufficiently large datasets related to MS, making the use of deep learning models cumbersome. More traditional statistical learning methods might be more suitable for handling the problem of data scarcity. Notwithstanding, such models cannot be applied directly to images due to their high dimensionality and cannot handle multiple MRI modalities at the same time, which provide complementary information useful for MS profile characterization and prognosis. Finally, most of the works proposed in the literature focus on the analysis of WM tissue since this is where most of the lesions are located. Notwithstanding, in MS, the GM tissue is also heavily affected by the disease where demyelination of WM lesions has been directly linked with a reduction in thickness of cortical regions. For this reason, the analysis of GM tissue, primarily performed using the T1-weighted MR image modality, represents an interesting area of research where the scientific contributions based on ML methods are still scarce.

Overall, this dissertation aims at offering practical contributions to the analysis of degenerated patterns forming inside the brain network of MS patients. More precisely, the estimation and evaluation of the prognostic conditions characterizing MS are performed combining two important areas of research, such as graph theory and machine learning. The approaches proposed in this PhD allow to handle multiple MRI modalities in a joint fashion using more traditional statistical learning techniques, which demonstrated to better handle the problem of data scarcity. Notwithstanding, the power of deep learning can also be exploited indirectly even in cases where small and imbalanced datasets are provided, which represents a usual scenario in the biomedical domain. Moreover, besides methodological advances, the use of interpretable models also represents an important characteristic for better and more trustworthy use of ML-based methods, especially in the context of brain diseases like MS. Most of the best-performing models are considered as "black boxes" where the underlying line of reasoning followed by the algorithm is un-2 Berardino BARILE clear and little trust can thus be given to the final prediction. In this doctoral thesis, a new interpretability approach is proposed for the discovery of the underlying mechanism guiding the ML model, contributing toward more interpretable and understandable results coherent with our a-priori knowledge and intuition. Finally, the analysis of GM tissue degeneration was also proposed and based on statistical and ML approaches, confirming that GM tissue degeneration might be an important biomarker not only for the discrimination of MS patients and healthy subjects but even between different MS profiles.

The contribution offered in the present manuscript represents one of the preliminary attempts to exploit the power of connectome data analysis together with advanced machine learning algorithms for the analysis, classification and prediction of MS disability using information from the degenerated white and grey matter tissue.

Research Objectives

Nowadays, multiple attempts have been performed for the identification of meaningful patterns characterizing the MS disease. However, conclusions are not yet definitive regarding the impact and cause of brain network alterations, guiding the degenerative process characterizing the MS disease. With the evolution of advanced statistical methods and data-driven solutions, as well as the rise of ML applications in the context of biomedical, and in particular, brain image analysis, we are able to discover underlying hidden patterns and relations between multiple brain regions, which may shed light to the pathological processes of MS. Notwithstanding, such data-driven models require a large amount of data for an effective "learning". Thus, advanced methods are required to guide the algorithm toward a better process of relational discovery of patterns. To tackle these problems, three main objectives have been considered and listed below.

1. Generation of realistic brain connectomes data for the improvement of Multiple Sclerosis profiling in the context of data scarcity and class imbalance. This methodological contribution aim at solving the problem of limited and imbalanced data, usually characterizing the medical domain. In this context, an advanced artificial intelligent framework composed of two neural networks organized following an adversarial structure, known as Generative Adversarial Network (GAN), was used for the generation of realistic and biologically meaningful connectomes of MS patients. With this work, we demonstrated that structural graphs can be synthetically generated, preserving the real biological structural relationship seen in brains affected by the MS disease. The study and estimation of the underlying generation process describing the pathological conditions of MS patients represent an innovative and important approach for a better understanding of the physiopathological mechanisms guiding the MS disease. Additionally, the generation of meaningful structural connectome data improves the classification performance of traditional statistical learning techniques by augmenting an existing dataset, characterized by data insufficiency and strong imbalance between classes, combining real and synthetic data. In this work, we demonstrated that meaningful structural patterns between multiple grey matter regions are important for the discrimination of MS profiles, compared to the approach of simply and naively augmenting the dataset by means of classical methods (such as Random Oversampling and Synthetic Oversampling), which do not take into account the biological structural properties of the human brain network.

2. Automatic evaluation of the grey matter tissue degeneration for Multiple Sclerosis patients profiling The use of advanced imaging techniques such as Magnetic Resonance Spectroscopy (MRS) and Diffusion Tensor Images (DTI), offer the prospective for a better understanding of the underlying mechanisms characterizing the MS disease and may ultimately allow a better prediction of the patient's clinical course. Notwithstanding, the use of DTI images is less exploited in clinical applications due to the long acquisition time and cost-related constraints. For this reason, a novel approach to study the evolution of the MS disease is proposed and only based on classical anatomical MRI data, such as T1-weighted images. The analysis of the Grey Matter (GM) tissue degeneration has already demonstrated its contribution to the study of neurodegenerative effects in patients with MS, even at an early stage of the disease. Multiple morphological features of the GM tissue, such as thickness, curvature and area, were calculated and used for MS patients profiling. As first step, the regional thickness of each patient was analyzed using both statistical and predictive modeling and compared using two different parcellation strategies. Although lower performances were obtained using GM connectome data, compared to the results obtained from the WM connectome, this analysis represents an important advancement toward the exploitation of larger datasets, thanks to the fact that only the T1-weighted image is required, which represents one of the most important and used modalities in clinical practice. Notwithstanding, as a second analysis, the combination of multiple morphological features was considered by exploiting the important properties of factorization methods as multiview data fusion strategy. The use of multiple morphological biomarkers describing the degeneration of the GM tissue induced by the MS disease, provided a more reliable estimation of the prognostic conditions describing the early stages and progressive courses characterizing the MS pathology.

Automatic and interpretable evaluation of Multiple Sclerosis progression based

on cognitive and physical patients disability status. One of the most relevant aspects of a disease can be identified by its impact on the patient's daily activity. Multiple Sclerosis is a disease that occurs when the myelin sheaths surrounding the axons in the brain and spinal cord are damaged by attacks from the body's own immune system. Put simply, MS causes the immune system to destroy the coatings of nerve channels, shorting out nerve signals and limiting the capacity of the spinal cord and the brain to interact with each other. Inevitably, these alterations lead to cognitive and physical disability, hindering the quality of life of the patient. Additionally, MS is considered a disability by the Social Security Administration (SSA). Someone with MS can qualify for disability benefits if it is severe enough to prevent them from being able to work full time, causing tremendous consequences in terms of financial sustainability. Thus, a rapid, automatic and interpretable evaluation of the cognitive and physical conditions of a patient represents an important objective. To tackle the problem, multiple machine learning models were combined together following a hierarchical structure known as stacking generalization procedure. However, due to the relevance as well as the impact of such a topic on the patient's daily activities, the disability estimation analysis was performed also directly using the fiber bundles tracking. In fact, although very informative, structural connectivity does not take into account important fiber tracks such as the Corti-coSpinal Track and the Corpus Callosum which are known to be heavily involved in the process of tissue degeneration. Additionally, in order to offer more robust guidance to the neurologist's decision toward a better diagnosis and treatment of the patient's status, a new interpretability model was implemented with the aim to pin down the most important connections between pairs of GM regions, deemed to be relevant by the machine learning model for disability estimation. It should be also emphasized that, by providing interpretability to the predictive process, better trust can be given to the current prediction by comparing our a-priori expectation to the actual suggestions provided by the model.

Manuscript Overview

The present manuscript is divided into three main parts. In the first part, a thorough introduction of the main methods and concepts useful for the understanding of the methodological contributions provided in the present manuscript is presented. In the second part, Berardino BARILE we list the scientific contributions related to the implementation and development of machine learning algorithms for the classification of MS clinical profiles. In the third part, we describe the use of machine learning methods for the evaluation of the MS disease progression. In Figure 1, a schematic and intuitive overview of the structural organization of the different chapters is proposed. Chapter (Machine Learning and Pattern Recognition), the fundamental concepts of statistical learning and deep learning methods are introduced. They represent the main methodologies used in the present doctoral dissertation. First, the "learning" paradigm applied to machines is introduced, essential to understanding the underlying algorithms and statistical learning concepts at the core of any Machine Learning (ML) system. Additionally, the important statistical trade-off between bias and variance is introduced, which is profoundly linked with the way in which machines actually generalize to unseen scenarios. To this aim, the principal strategies of model validation, usually implemented in the context of machine learning applications, are highlighted. Then, the mathematical concepts describing the main algorithms implemented in the present doctoral work are explained starting from the most fundamental concepts and highlighting the main advan-tages and disadvantages of their use. In the second Chapter (MRI and Brain Connectivity), the principles of MRI are introduced, recalling the physical mechanisms at the origin of Nuclear Magnetic Resonance (NMR), allowing the generation of images. We also present here the clinical MRI systems as well as the main imaging modalities used in conventional MRI. Also, its relevance in the context of medical applications, especially for brain-related diseases and cognitive impairments, is highlighted, helping neurologists in providing appropriate diagnoses and treatments to patients. Specifically, an in-depth introduction to the MRI machinery is proposed, describing the main concepts and how different MRI modalities can be used for investigating important biomarkers in different brain tissues. For instance, non-conventional acquisition protocols, like Diffusion Tensor Image (DTI) data, allowed to obtain sensitive information, essential for a deep characterization of the White Matter (WM) tissue. On the other hand, conventional T1-weighted images allowed for an accurate analysis of GM tissue while FLAIR images represent one of the most used MRI modalities for WM lesions formation and detection. Also, recent development of MRI processing allowed the modeling of the human brain network with an interesting similar mathematical concept, the connectome. Graph analysis and brain connectivity represent nowadays one of the most hot topics in the biomedical domain and beyond, thanks to their great ability in preserving the structural relationship of different entities forming the brain network. A deep-dive into the principal concepts of graph theory is proposed, highlighting its importance for the characterization of specific properties, such as the "small world", demonstrated to be highly predictive of network degeneration in specific brain diseases like MS. Finally, in the third Chapter (Multiple Sclerosis), a brief introduction to the MS disease is proposed since it represents the main area of application of the present doctoral thesis. First, the epidemiology and physiopathology of the disease are introduced. Afterward, the four clinical profiles characterizing MS are described. In fact, the understanding of the disease dynamics of patients, categorized in different clinical profiles, represents an essential point for the development of automated algorithms able to provide helpful insights. Chapter four concludes Part I (State of the Art) of this PhD manuscripts, and details the dataset and softwares used throughout the remaining parts of the thesis.

In Part II (ML for Clinical MS Profiling), the main contributions provided by the present doctoral work are listed. As previously stated, they describe new possible solutions to the use of connectome data analysis and graph theory, mainly applied in the context of MS patient profiling. Specifically, in Chapter five (Connectome Data Augmentation using GAN), a new generative model architecture based on an adversarial structure is proposed for the generation of meaningful structural connectome data, preserving the Berardino BARILE structural network properties of the real-world brains affected by the MS pathology. Of the three objectives previously highlighted, this chapter tackles objective one by demonstrating the usefulness of generative models in a context where data scarcity and high imbalance between classes constitute an overwhelming problem for traditional statistical learning methods. In Chapter six (GM Thickness for MS profiling) and Chapter seven (Kernel-Based GM Analysis using MLSVD), we deal with objective two, where the analysis of the GM tissue was proposed and based on single and multiview morphological features such as thickness, curvature and area. More in detail, in the former, a statistical and data-driven investigation of the cortical thickness alteration was performed. Due to the novelty of such an approach, two different atlases were used and compared, such as FSAverage and Glasser2016, which provided a clear difference in the parcellation of GM regions with respectively 68 and 360 Regions of Interest (ROIs). Also, graph theory was employed and a statistical analysis of the six most important global graph metrics was implemented with the aim to highlight statistical differences between MS clinical profiles. Additionally, a machine learning analysis was also proposed, combining the predictive performances of four different machine learning models while using two parcellation strategies. For the latter, a kernelization-based approach is proposed. Each GM region pair were compared considering three different morphological measures, such as thickness, curvature and area. A factorization paradigm based on a MultiLinear Singular Value Decomposition (MLSVD) was used for the joint analysis of the three morphological features, improving the classification performance of MS patients.

Finally, in Part III (ML for evaluation of MS progression), the machine learning methodology was implemented with the aim of extracting insightful conclusions for the estimation and evaluation of the physical and cognitive conditions characterizing the disease progression of MS patients. Such an analysis was based on one of the most important clinical measures of disability, such as the Expected Disability Status Scale (EDSS) score. This analysis covers objective three and provides interpretable insights to the discovery of hidden mechanisms followed by the machine learning model for MS disability estimation. Specifically, in Chapter eight (Connectome-based Analysis), structural connectivity was used for the estimation of disability in patients affected by the MS disease. The white matter connections between pairs of GM regions of each patient were used as input to an ensemble of boosting models, organized together following a stacking generalization scheme in which linear regression was used as a higher-level model. Disability was measured by the EDSS score and estimated for each patient. Also, due to the importance of the topic, an interpretability model was proposed in order to pin-down the most important white matter fiber connections deemed to be relevant by the ensemble model for 8 Berardino BARILE 0.4. COLLABORATIONS disability estimation. In Chapter nine (MS Progression Based on Fiber Bundle Tracking), the work on disability is extended by applying the ensemble model directly to the white matter fiber bundles. The interest in using fiber bundles directly in place of a more compact representation of the brain network, by means of connectome data, is motivated by the fact that the CorticoSpinal Track and the Corpus Callosum are not taken into account during the process of graph generation. As previously stated, these fiber tracks have been demonstrated to be heavily involved in the process of tissue degeneration caused by the MS pathology. However, due to the high dimensionality of the data, an unsupervised feature extraction technique is required. For this reason, a Monte Carlo-based procedure was implemented comparing each fiber track of MS patients with those extracted from healthy subjects in such a way that a probabilistic interpretation of the pathological differences between the two groups can be highlighted and embedded into the learning process of the model. Additionally, the interpretability model used for the connectome data, was also applied in this context, highlighting results coherent with our a-priori expectation and with what is already known so far in the literature.

The manuscript ends with the main conclusions and remarks drawn from the contributions proposed in the present doctoral dissertation. In this chapter, we highlight the importance of Machine Learning systems for the analysis, diagnosis and prognosis of MS patients. The study of connectome data, combined with ML algorithms, are not yet fully tackled in the Multiple Sclerosis literature. With our contributions, we demonstrated that connectome data analysis may represent a powerful alternative to the standard medical approaches for classification and prognosis of MS patients and more generally for the analysis of brain-related diseases such as Multiple Sclerosis. Additionally, interpretability represents an important point for a trustworthy application of machine learning algorithms, especially for brain diseases. Notwithstanding, the concept of interpretable machine learning is relatively new, although much work has been done to uncover the underlying logic used by the black box model. Such effort has also been done in the multiple works proposed in this thesis, highlighting coherent results in line with our a-priori expectation and thus increasing the confidence and trust of the proposed approaches.

Collaborations

This PhD research represents a dual doctoral agreement based on a joint collaboration between the university Claude Bernard Lyon 1, within the MAGICS Team (MAGnetic resonance and optICS) within the CREATIS laboratory, and the BIOMED research group, within the STADIUS Center for Dynamical Systems, Signal Processing and Data Berardino BARILE Analytics in the Department of Electrical Engineering of the University KU Leuven. The present PhD research was supervised and guided by Prof. Dominique Sappey-Marinier and Prof. Sabine Van Huffel from the respective French and Belgian universities and it is part of the of the INSPiRE-MED project within the Marie Sklodowska-Curie Innovative Training Networks (ITN) program. It represents a research opportunity for collaboration between [START_REF] Brown | Machine learning on human connectome data from mri[END_REF] Early Stage Researchers (ESRs) in the field of biomedical imaging, particularly in the field of Magnetic Resonance Spectroscopy (MRS) and MR Spectroscopic Imaging (MRSI) combined with Positron Emission Tomography (PET) and enhanced by machine learning techniques. Multiple workshops were organized and the collaboration between ESRs allowed the development of innovative technological developments which are at the core of the present doctoral research program. These collaborations involved 12 academic university partners in 12 different EU countries, with an already established collaborative track record in R&D, and 9 industrial partners from the broad and competitive imaging sector. In particular, this PhD research exploited the knowledge and ideas developed during the collaboration with the "icometrix" industrial partner for the statistical analysis and advanced implementation of deep learning frameworks for the prognosis and segmentation of white matter lesions in patients with Multiple Sclerosis. Specifically, during the visits to the private company "Icometrix", senior researcher Diana Sima guided us toward a more in-depth understanding of the application of research studies in the business context, also providing guidance, insights and practical support and feedback for the development of original research ideas and solutions. It is also worth highlighting the collaboration with ESR Pooya Ashtari, with whom I participated to a "Multiple Sclerosis new lesion segmentation" challenge organized within the project "France Life Imaging" (FLI) and managed by three partners: the Empenn (means "Brain" in Breton language) ERL U1228 research team, the French cohort of MS patients "Observatoire Français de la Sclérose En Plaques" (OFSEP) and the transversal node of the national FLI infrastructure named "Information Analysis and Management" (IAM).

Introduction

The field of Statistics is constantly challenged by the problems that science and industry bring to its door. In the early days, these problems often came from agricultural and industrial experiments and were relatively small in scope. With the advent of computers, statistical problems gained popularity both in size and complexity. New algorithms development, increased data storage and computational power led to the new field of data mining while statistical and computational problems in biology and medicine have created the field of bioinformatics. Vast amounts of data are being generated in many fields, and the statistician's job is to make sense to all of them by finding patterns and extracting knowledge from data. This ability is generally referred to as "learning from data". Since artificial intelligence first achieved recognition as a discipline in the mid 1950's, machine learning has become a central research area. The ability to learn is a hallmark of intelligent behavior, so any attempt to understand intelligence as a phenomenon must include an understanding of learning. More concretely, learning provides a potential methodology for building high performance systems. Artificial Intelligence (AI) represents a broad branch in computer science that is focused on a machine's capability to produce rational behavior from external inputs. Despite the significant results obtained by the current state of the art, given the complexity and the huge volume of biological data, many traditional computer science techniques and algorithms fail to solve complex biological problems in the real world. Nevertheless, modern computational approaches based on machine learning can address these limitations. Machine Learning is an adaptive process that enables computers to learn from experience, learn by example, and learn by analogy. A schematic representation of the relationship between Artificial Intelligence, Machine Learning and Deep Learning is proposed in Figure 1.1.

Machine learning is the general term for when computers learn from data. It describes the intersection of computer science and statistics where algorithms are used to perform a specific task without being explicitly programmed; instead, they recognize patterns in the data and make predictions once new data arrives. A famous quote from T. Mitchell [START_REF] Mitchell | Machine learning[END_REF] says that, "a computer program is said to learn from experience E with respect to some class of tasks T and a performance measure P, if its performance at task T, as measured by P, improves with experience E". However, the easiest way to understand how a machine becomes intelligent is to compare it to how humans learn. For example, consider a child learning how to ride a bicycle. The child mounts the bicycle, grips the handlebars, and hopes to stay upright and in control. The child does not learn how to ride a bike by understanding the physics of biking but rather through trial and error. Over time, the child becomes instinctively adept in perceiving factors that can make him lose control of the

Learning Paradigm

Learning capabilities are essential for automatically improving the performance of a computational system over time on the basis of previous results. Such concept is particularly suitable in many contexts, including the biomedical domain. The learning problems that we consider can be roughly categorized as either supervised or unsupervised. In supervised learning, the goal is to predict the value of an outcome measure based on a number of input values; in unsupervised learning, there is no outcome measure, and the goal is to describe the associations and patterns among a set of input measures. More formally:

• Supervised Learning: is a machine learning paradigm for acquiring the inputoutput relationship information of a system based on a given set of paired inputoutput training samples [START_REF] Liu | Supervised learning[END_REF]. This kind of learning is used when the answer to the Berardino BARILE problem is known. In supervised learning, we are able to provide a set of samples which have a known label or result. A model faces a training process where it is required to make prediction and is corrected when those predictions are wrong. In other words, the goal of supervised learning is to build an artificial system that can learn the mapping between the input and the output, and can predict the output of the system given new inputs. In supervised learning, the two most common tasks are the following ones:

-Classification: in this case the label (y) is discrete, meaning that only natural values are allowed. This is the case of a yes/no answer (binary classification) or the choice of a particular category out of N possible categories (i.e., multi-class classification). The algorithm's objective is to find a function f that maps an input x to a category whose label is y, where the latter can be binary or multi-category.

-Regression: in this case the label (y) is continuous, meaning that all real values are allowed. In this case the main challenge is to obtain a function f which is able to produce an output ŷ such that d(y, ŷ) is minimal, where d represents any function of distance (i.e., squared difference, absolute difference, etc.)

• Unsupervised Learning: with this learning paradigm, we looks for previously undetected patterns in a data set with no pre-existing labels and with a minimum of human supervision. In contrast to supervised learning that makes use of humanlabeled data, unsupervised learning, also known as self-organization, allows for modeling of probability densities over inputs. Problems of this kind are the one for discovering groups of similar examples (clustering), or to give a new representation of the data in a high-dimensional space (i.e. manifold learning).

A notable phenomenon that characterizes all machine learning models is that a minimum training error does not necessarily indicate a good performance in general. Training is referred to as the learning process that estimates the parameters of the learner based on the ground truth, while testing refers to the evaluation of the model with parameters found during training by generating predictions for unseen data , i.e., the data used in testing have not been included in the training process. Therefore, even if a learner achieves a minimum error on the set of training data, it does not guarantee to perform well to the data unseen. The reason for this is mainly due to the possible overfitting to the training data, i.e., the model has too high order of complexity in learning the mapping. This issue is referred to as "generalizability". A good learning algorithm must have a good generalizability. To take into consideration the generalizability in designing the learner, the algorithm needs to balance the objective of minimizing the training error and the complexity of the learner. In the literature, this concept is referred to as "bias-variance tradeoff". [START_REF] Bruschi | Ultra-high-field 7-T MRI in multiple sclerosis and other demyelinating diseases: from pathology to clinical practice[END_REF] Berardino BARILE 1.3. THE BIAS-VARIANCE TRADEOFF

The Bias-Variance Tradeoff

The bias-variance tradeoff is a central problem in supervised learning. Ideally, one wants to choose a model that both accurately captures the regularities in the training data, but also generalizes well to unseen data. Unfortunately, it is typically impossible to do both simultaneously. High-variance learning methods may be able to represent their training set well but are at risk of overfitting to noisy or unrepresentative training data. In contrast, algorithms with high bias typically produce simpler models that may fail to capture important regularities in the data (i.e. underfitting). Thus, the bias-variance decomposition is a way of analyzing a learning algorithm's expected generalization error with respect to a particular problem. It can be decomposed as the sum of three terms, the bias, variance, and a quantity called the irreducible error, resulting from noise in the problem itself. Understanding how different sources of error lead to bias and variance, helps us improve the data fitting process resulting in more accurate models. More formally, let's consider with D = {(x 1 , y 1 ), ..., (x n , y n )} the training dataset and with f an unknown function such that y = f (x) + ǫ where ǫ represents a white noise irreducible error component. Additionally, let's define with f (x; D) a mapping function we would like to find, such that it approximates as closely as possible the true unknown function f (x). Without loss of generality, let's consider as loss function the mean squared error defined as (y -f (x; D)) 2 , where f (x; D) represents any supervised machine learning model. We can decompose its expected error on an unseen sample x as follows:

E D,ǫ (y -f (x; D)) 2 = Bias D f (x; D) 2 + V ar D f (x; D) + σ 2 (1.1)
where:

Bias D f (x; D) = E D f (x; D) -f (x)' (1.2)
and

V ar D f (x; D) = E D E D f (x; D) -f (x; D) 2 (1.
3)

The three terms on the left-hand-side of Eq. (1.1) can be conceived as follows:

• bias: is the first term in Eq. (1.1) and is taken as the difference between the expected (or average) prediction of our model and the correct value which we are trying to predict. To better understand this concept, imagine you could repeat the whole model building process more than once. Each time new data are collected and a new analysis is run, effectively creating a new model. Due to randomness in the data, the resulting models will have a range of predictions. Bias measures how far off, on average, these models' predictions are from the correct value.

• variance: is the second term in Eq. (1.1) and is taken as the variability of a model prediction for a given data point. Again, imagine you can repeat the entire model building process multiple times. The variance is how much the predictions for a given point vary between different realizations of the model.

• irreducible error: that third term is the noise term in the true relationship that cannot fundamentally be reduced by any model. Since all three terms are nonnegative, the irreducible error forms a lower bound on the expected error on unseen samples.

Given the true model and infinite data to calibrate it, we should be able to reduce both the bias and variance terms to 0. However, in a world with imperfect models and finite data, there is a tradeoff between minimizing the bias and minimizing the variance. An intuitive representation of the tradeoff between bias and variance can also be obtained from Eq. (1.1). In fact, given a limited set of data points and a defined trained model f (x; D) and a fixed error σ 2 , which is outside of our control, for each given (i.e. fixed) value of E D,ǫ (y -f (x; D)) 2 , if Bias D f (x; D) is decreased inevitably the V ar D f (x; D) should increase since all the terms in Eq. (1.1) are positive quantities. Trivially, the opposite relationship is also true. A schematic relationship between bias and variance is also depicted in Figure 1.2.

Model Validation

In machine learning, model validation is alluded to as the procedure where the prediction of the trained model is compared with our a priori-expectation (i.e. labels in the case of supervised learning). It represents an important step in order to understand what the model has exactly learned. In other words, a good validation strategy helps us in measuring the real generalization performance of the algorithm. Thus, model validation is the process that is carried out after model training, where the trained model is evaluated with a testing data set. The testing data may or may not be a chunk of the same data set from which the training set is procured. The importance of validating the model comes down to the fundamental question of why we train AI models. In fact, the ultimate goal for any machine learning model is to learn from examples, in such a manner that the model is capable of generalizing the learning to new instances never seen before. So, when we Figure 1.2: Relationship between bias and variance on the estimation of the total expected prediction error of a generic machine learning model. Image taken from https://scott.fortmann-roe.com/docs/BiasVariance.html approach a problem with a dataset in hand, it is very important that we find the right machine learning algorithm to create our model. Every model has its own strengths and weaknesses. For instance, some algorithms have a higher tolerance for small datasets, while others may be good with large amounts of data. For this reason, two different models using similar data can predict different results with different performances and hence model validation is required. Roughly speaking, two types of model validation techniques can be considered, namely:

• In-sample validation -testing data from the same dataset that is used to build the model.

• Out-of-sample validation -testing data from a new dataset that isn't used to build the model

The first of these two types of validation approaches is called "Resubstitution". Although it represents a type of validation, this procedure is not recommended since it allows the estimation of only the bias component of the error, while the variance component cannot be estimated. Thus, the generalization capacity of the model to unseen data cannot be tested. Notwithstanding, for linear regression models, there is a formula to approximate the generalization capacity of the model which does not require iterative refitting of the model to different data subsets. The Generalized Cross-Validation (GCV) [START_REF] Gene | Generalized cross-validation as a method for choosing a good ridge parameter[END_REF] is a statistical concept that takes into account the complexity of the model being used for training Berardino BARILE and it is defined as follows:

GCV = 1 n n i=1 y i -ŷi 1 -df /n (1.4)
where n represents the number of samples, y i is the i-th instance outcome in the training set, ŷi is the model prediction of that outcome and df is the degrees of freedom of the model. The degrees of freedom are an accounting of how many parameters are estimated by the model and, by extension, a measure of complexity for linear regression. Based on this equation, two models with the same sums of squared errors (the numerator) would have different GCV values if the complexities of the models were different. Notwithstanding, we already stated that the ultimate goal of a machine learning model is to learn useful patterns in the data to be exploited in order to make predictions on new unseen instances. The out-of-sample validation procedure allows us to evaluate exactly this, by measuring the error obtained from a generic machine learning model (not only linear regression) when applied to unseen data. Four types of out-of-sample validation techniques can be considered, between the most important, although multiple variants exist in the literature. In this thesis the following out-of-sample validation techniques will be discussed: Train-Test split, Monte Carlo Cross-Validation, k-Fold Cross-Validation, Leave-One-Out Cross-Validation and Nested Cross-Validation. This list does not represent an exhaustive picture of all possible validation methods although they are the most used ones in the machine learning community.

Train-Test split

It represents the easiest procedure for model evaluation and it consists of splitting the data into two different datasets: training and testing. The usual conventions for percentage splitting are 60/40 or 70/30 or 80/20 for training and testing size respectively. More precisely, in order to split the data in two separate datasets, first a percentage of instances are randomly selected from the entire population of available data and hold out from the rest of the data. The remaining data (i.e. not sampled) will represent the dataset we train the model on. This techniques is also called hold-out validation procedure for obvious reasons. By performing random splitting of the dataset we are able to estimate both the bias and variance component of the error obtained from the machine learning model. Notwithstanding, two main problems can still be detected. In fact, previous studies [START_REF] Molinaro | Prediction error estimation: A comparison of resampling methods[END_REF] demonstrated that validation using a single test set represents a poor strategy to evaluate the generalization performance of the model for two main reasons. First, the single splitting procedure may result in a testing dataset consisting of extreme cases, like for instance samples at the edge of the data distribution. Henceforth, the confidence to which we measure the generalizability of the machine learning model is reduced: does (or not) the model really generalize well or is the case that by sample chance alone a very easy (or very complicated) test set was selected? This problem can be mitigated with a large enough test sample. With a test dataset in the order of millions of instances this problem is negligible (i.e. the test dataset truly represents the statistical distribution of the population data). Second, for imbalanced data, the simple approach of random sampling instances may produce unreasonable splitting where, in the extreme case, only one class, for classification task, can be present in the test set. In order to avoid this problem, sampling should be performed by stratifying the data according to the labels associated to each instance such that the same group proportion observed in the original dataset is maintained in the test set.

Monte Carlo Cross-Validation

This validation procedure represents an extension of the Train-Test split approach previously discussed, where multiple evaluations of the model are performed by repeating the Berardino BARILE Train-Test split procedure multiple times (i.e. 100 resampling train-test splits). At each split, the model is trained on the training set instances and the performance evaluated on the remaining hold-out samples. The final evaluation is obtained by averaging over all the testing validation results. Although very appealing for evaluating the true generalization performance of the machine learning model, even in case of a small sample size, it comes at the cost of a higher computational effort. In fact, at each step, a train-test validation procedure is implemented, where the machine learning model is trained and tested at each iteration. The procedure is repeated until all the pre-defined number of resampling steps are performed. Finally, performance metric scores, obtained at each cycle, are collected and averaged out to obtain the final performance score.

k-Fold Cross-Validation

The essential idea behind this type of validation procedure is to divide the dataset in k-different folds where, at each iteration, k -1 folds are combined together forming a training set and the remaining hold-out fold is used as test set. The partitioning step performed to divide the dataset in different folds guarantees that each instance in the dataset is selected to be in the test set exactly once by "crossing" (i.e. selecting) the hold-out fold sequentially from left to right. In Figure 1.4, the k-fold cross validation procedure is illustrated, where at each iteration (i.e. experiment) one of the k-folds is selected to test the machine learning model (i.e. hold-out sample), while the remaining folds are combined forming a training set. This type of validation technique represents a nice trade-off between an optimal estimation of the error variance, and computational cost. Notwithstanding, it is worth mentioning that, compared to the Monte Carlo Cross-Validation approach, each instance in the dataset is used only once for training and once for testing. The choice of k is usually 5 or 10, but there is no formal rule. As k gets larger, the difference in size between the training set and the resampling subsets gets smaller. As this difference decreases, the bias of the technique becomes smaller. Additionally, in order to achieve good statistical properties (i.e. convergence toward the true and unknown error), for the obtained validation results, the original dataset needs to be shuffled prior to the k-fold splitting.

Leave-One-Out Cross-Validation

This type of validation strategy consists of splitting the dataset into a training set and a testing set such that all but one observation is used for training the machine learning model, while the remaining hold-out sample is used for testing. In other words, only one instance is selected from all the possible instances in the original dataset and hold-out from the training procedure. The evaluation of the model (i.e. prediction) is performed on only the left-out single instance. The procedure is repeated until all instances in the dataset are selected once to evaluate the predictive model performance. Figure 1.5 depicts the Leave-One-Out Cross-Validation procedure where each instance is selected one after the other and the model evaluated on only the left-out single sample at each iteration.

The obvious downside of this approach is the high computational cost. However, in particular cases, this represents the only possible strategy for model evaluation. For instance, in case in which the dataset is very small, a simple train-test split validation approach may hinder the performance of the machine learning model simply because the variability of the training set is reduced due to the splitting procedure. In fact, for an optimal training, the machine learning model needs to be exposed to as many different cases as possible in order to ensure that the training and test set are drawn from the same underlying distribution (i.e. same data generation process).

Nested Cross-Validation

This type of validation strategy arguably represents the most important approach for model validation. Notwithstanding, it represents a wider concept with respect to all other validation strategies previously discussed. In fact, as the name suggests, two nested types of validation steps are implemented, dividing the data in three sub-datasets (as opposed to the binary splitting discussed so far), and commonly referred to with the name of training set, validation set and test set. In order to understand the advantage of an additional splitting step (i.e. validation set), we need to introduce the concept of "hyperparameter tuning". This concept relates to almost all the machine learning models commonly used in real world applications for which an optimization of the hyperparameters is necessary in order to optimally guide the learning process and improve the generalizability of the model. In other words, the hyperparameter optimization has as objective that of finding a tuple of hyperparameters that yields an optimal model, which minimizes a predefined loss function on a given independent data. However, it is important to keep in mind that the independent dataset, used for optimally adjusting the hyperparameters of the model, does not allow us to estimate its true generalization capacity. To understand why, let's consider with Φ(γ) a generic machine learning model which takes as input a specific set of hyperparameters γ = {α 1 , α 2 , ..., α j }. Hyperparameter tuning consists in finding the best combination of α values such that the tuple γ, used by the machine learning model Φ, produces the best results (i.e. lower error) on an independent dataset not used during training. Multiple strategies have been described in the literature such as grid search, random search, bayesian optimization, gradient-based optimization and evolutionary optimization between the most used ones. For an in-depth description of these strategies we refer to [START_REF]Hyperparameter optimization in machine learning: Make your machine learning and deep learning models more efficient[END_REF]. Practically speaking, let's define with γ k a specific combination of hyperparameters used by the machine learning model for training. After a pre-specified number of itera-tions (i.e. steps), usually defined based on the hyperparameter optimization strategy chosen, a tuple of different machine learning models Υ = { Φ(γ 1 ), Φ(γ 2 ), ..., Φ(γ k )..., Φ(γ s )} is obtained. Thus, s defines the number of steps (i.e. model with specific settings) obtained during the searching process, while Φ(γ k ) represents a generic trained machine learning model with a specific combination of hyperparameters γ k . The optimization procedure is finalized by simply selecting the Φ(γ k )∀k ∈ {1, 2, ..., s} that maximizes the performance (i.e. minimize the loss) on an out-of-sample dataset (i.e. dataset not used for training the model). In the literature, this hold-out dataset is generally referred with the name of validation set. Notwithstanding, although the performance obtained from each model Φ(γ k )∀k ∈ {1, 2, .., k} represents a good estimation of the generalization capacity, by selecting the best model we are biased toward an optimistic representation of all the possible models in Υ. Statistically speaking, we can translate this concept by stating that, the best Φ(γ k ) model alone does not guarantee to measure the true generalization performance (i.e. true error distribution) that we could have obtained from a generic unseen dataset. For this reason, an additional independent dataset, called test set, is required for measuring the true generalization capacity of the selected model. The Nested Cross-Validation procedure thus represents the best approach for validating a machine learning model and it represents a flexible evaluation strategy in which different splitting procedures can be nested on one other. The improved generalization performance obtained by using this strategy comes however at the cost of an increased computational cost and at the need for a larger dataset, essential for the correct tuning and evaluation of the model.

Performance Metrics

The most important task in building any machine learning model is to evaluate its performance. In the previous Section an introduction to different validation strategies were discussed. Notwithstanding, a clear vision on what kind of metric to use for the correct evaluation of the machine learning model should be defined beforehand. Evaluation metrics are tied to machine learning tasks. There are different metrics for different tasks like classification and regression or segmentation just to name a few. Some metrics, like precision-recall, are useful for multiple tasks. Classification and regression are examples of supervised learning, which constitutes a majority of machine learning applications. Using different metrics for performance evaluation, we should be able to improve our model's overall predictive power before we roll it out for production on unseen data. Without doing a proper evaluation of the machine learning model by using different evaluation metrics, and only depending on accuracy, can lead to a problem when the respective

Classification Metrics

Classification is about predicting the class labels given input data. In binary classification tasks there are only two possible output classes (i.e. dichotomous variable). In multiclass classification, more than two possible classes can be present. In this dissertation only binary classification will be discussed. For a more in-depth understanding of the different classification metrics for multiclass classification we refer to [START_REF] Hastie | The elements of statistical learning[END_REF]. We will start by discussing a confusion matrix, also known as an error matrix, which represents in the field of machine learning, and specifically the problem of statistical classification, a specific table layout that allows visualization of the performance of an algorithm, typically a supervised learning one (in unsupervised learning it is usually called a matching matrix). Each row of the matrix represents the instances in an actual class while each column represents the instances in a predicted class, or vice versa. Both variants are found in the literature. In Figure 1.6 a representation of a generic binary confusion matrix is reported.

The name of such a table stems from the fact that it makes it easy to see whether the system is confusing two classes (i.e. commonly mislabeling one as the another). In such a table, the count of True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN) instances are reported. The first two (i.e. TP and TN) are reported in the main diagonal entry of the matrix and they represent the number of cases the positive and the negative class in the prediction and ground truth label match. Combined, they provide the total number of correctly predicted instances. Conversely, the last two elements (i.e. FP and FN), usually found in the out-of-diagonal positional entries of the matrix, represent the number of instances in which the model predicted the wrong class compared to the ground truth label. Together, they define the total number of instances in which the model mistakingly predicted the class label. It is worth noticing that False Positive can also be interpreted as type I error (similar to the statistical test of hypothesis) where the model predicted a positive class when in fact it was negative. Conversely, False Negative can be interpreted as type II error in which the model predicted a negative class when in fact it was positive. Depending on the specific application, we might be more interested to minimize one type of error with respect to the other. For instance, in the case of a clinical diagnosis, one might be more interested to limit the type II error since erroneously predicting the absence of an illness, like for instance a brain tumor, might be more risky then the reverse case (falsely predicting the present of a tumor), since in that latter case a more detailed check up might shed light on the error without any long term heath effect. By rearranging the term in Figure 1.6, different metrics can be obtained.

• Accuracy: It represents a simple measure of how often the classifier predicts correctly. Mathematically, it can be defined as follows:

Accuracy = T P + T N T P + T N + F P + F N (1.5)
In other words, it measures the percentage of instances correctly classified over the total number of instances. Although highly used in machine learning applications, this metric is not very informative in cases where class imbalance is present. For instance, if we consider the case in which a ratio 9:1 between positive and negative instances in the ground truth label is present, by simply predicting all the instances with a positive label it is possible to obtain a percentage accuracy of 90%. Notwithstanding, such a classifier is not very useful though as one can imagine. For this reason, additional classification metrics were developed in the literature.

• Precision: explains how many of the correctly predicted cases actually turned out to be positive. Mathematically, it can be defined as follows:

P recision = T P T P + F P (1.6)
Precision is useful in cases where False Positive is a higher concern with respect to False Negatives as explained before.

• Recall: also known as Sensitivity, it explains how many of the actual positive cases the machine learning model were able to predict correctly. It is a useful metric in cases where False Negative is of higher concern than False Positive. Mathematically, it can be defined as following:

Recall = T P T P + F N (1.7)
As explained before, this metric is particularly important in medical applications where raising a concern of a probable disease has less harmful effect with respect to the case in which the disease remains undetected.

• Specificity: It represents the fraction of samples correctly classified as negative.

Mathematically, it can be defined as follows:

Specif icity = T N T N + F P (1.8)
• F1-score: It gives a combined idea about Precision and Recall metrics and it reaches maximum value when precision and recall are equal. Mathematically, it can be defined as follows:

F 1 = 2 × P recision × Recall P recision + Recall (1.9)
Such a metric has the property of emphasizing more extreme values. In particular, it is appropriate to use this metric when either FP and FN are equally costly or the number of TN instances is high.

• Diagnostic Odds Ratio (DOR): is a measure of the effectiveness of a diagnostic test and it is formally defined as follows:

DOR = Specif icity × Sensitivity (1 -Specif icity) × (1 -Sensitivity) (1.10)
• Area Under the ROC Curve (ROC-AUC): It represents a more sophisticated classification metric since it allows a "threshold free" evaluation of the classification performance of a generic machine learning model. In fact, many classifiers give a posterior class probability, and not directly the class outcome, such as negative or positive score assignment. Thus, a decision threshold needs to be defined, which has an effect onto the overall model output and performance. On the left-handside of Figure 1.7, the horizontal axis is the posterior class probability of the model output. The vertical axis displays the percentage of samples with a given model output. Negative samples will have a lower probability of being assigned to the

Regression Metrics

Regression refers to predictive modeling problems that involve the estimation of a numerical value. Predictive modeling can be described as the mathematical problem of approximating a mapping function (f ) from input variables (X) to output variables (y). This is called the problem of function approximation. The problem of evaluating a regression model differs from the classification approach in the fact that we are not interested in the exact prediction of the output value y given a specific set of features, which represents a problem intractable for most machine learning applications. Instead, the interest, in this context, is to obtain an estimation of the continuous output value as close as possible to the true value. In this thesis, some of the most common regression metrics are described.

• Mean Squared Error (MSE): it is also known as mean squared deviation (MSD) of an estimator or procedure. It measures the average of the squares of the errors, that is, the average squared difference between the estimated values and the actual value. It is derived from the square of Euclidean distance and it is always a positive quantity that decreases as the error approaches zero. Mathematically, it can be defined as follows:

M SE = 1 n i (ŷ i -y) 2 (1.11)
The MSE is also known as the second moment about the origin and thus incorporates both the variance of the estimator (how widely spread the estimates are from one data sample to another) and its bias (how far off the average estimated value is from the true value).

• Root Mean Squared Error (RMSE): it is a modified version of the MSE in which the squared root is taken. In other words, it represents the square root of the second sample moment of the differences between predicted values and observed values or the quadratic mean of these differences. Mathematically it can be defined as following:

RM SE = √ M SE = 1 n i (ŷ i -y) 2 (1.12)
RMSE is always non-negative, and a value of 0 (almost never achieved in practice) would indicate a perfect fit to the data. RMSE is the square root of the average of squared errors. The effect of each error on RMSE is proportional to the size of the squared error; thus larger errors have a disproportionately large effect on RMSE. Consequently, RMSE is sensitive to outliers.

• Mean Absolute Deviation Error (MADE): is a measure of errors between paired observations expressing the same phenomenon. It is deeply related to the MSE but it differs from it in that instead to consider squared differences between true and estimated value (like in MSE), it uses the absolute difference. Mathematically it can be defined as following:

M ADE = 1 n i |ŷ i -y| (1.13)
The advantage of using MADE lies in the fact that it uses the same scale as the data being measured. This is known as a scale-dependent accuracy measure and therefore cannot be used to make comparisons between models prediction using different scales. Also, the MADE evaluation metric is not exactly equivalent to the root-mean squared error (RMSE), but it has an intuitive interpretation. In fact, it measures the simple average absolute vertical or horizontal distance between each point in a scatter plot in the Y = X line. In other words, MADE is the average absolute difference between X and Y , where X is the estimated quantity (i.e. ŷi ) while Y refers to the true value.

• Coefficient of Determination (R 2 ): It is an evaluation metric used in the context of statistical models. Its main purpose is either the prediction of future outcomes or the testing of hypotheses, on the basis of other related information. Notwithstanding, since it measure the proportion of the variation in the dependent variable that is predictable from the independent variable, it can be also considered as a metric of validation for regression tasks. Mathematically it can be defined as follows:

R 2 = 1 - SS res SS tot (1.14)
where SS res is known as "sum of squared residuals" and it is defined as follows:

SS res = i = (ŷ i -y i ) 2 = i e 2 i (1.15)
and where SS tot is known as "total sum of squares" and it is defined as follows:

SS tot = i = (y -y i ) 2 (1.16)
where y = 1 n i y i . It is important to realize that there are cases where the computational definition of R 2 can yield negative values. For instance, this situation can arise when the predictions that are being compared to the corresponding outcomes (i.e. true values) have not been derived from a model-fitting procedure such as linear regression or when such a model does not include an intercept. A more in-depth intuition and explanation of linear regression model will be discussed in the following paragraphs. In cases where negative values arise, the mean of the data provides a better fit to the outcomes compared to that obtained by the fitted model. Also, the coefficient of determination can be more informative than MADE, MSE, and RMSE in regression analysis evaluation as the former can be expressed as a percentage, whereas the remaining measures have arbitrary ranges.

Statistical Learning Models

In recent years, several methods based on computer vision and image processing techniques were developed for the automatic identification of pathological mechanisms. Such methods mostly exploit recent advances in artificial intelligence methods. In this thesis, we make distinction between two main areas of machine learning models, namely, Deep Learning and Statistical Learning models. In more detail, Deep Learning methods have recently achieved a breakthrough in a variety of computer vision as well as speech analysis problems on different benchmark datasets, and are attracting a very strong interest within the artificial intelligence community. Conversely, statistical learning algorithms mainly fell into the category of statistical models and data mining. This two different learning algorithms will be discussed in the following Sections.

There are different ways an algorithm can model a problem by interaction with a specific environment through the data that are fed to the model as input. In deep learning we saw that a set of nested functions are implemented in order to obtain a powerful tool for learning hidden patterns present in the data. This kind of machine learning approach demonstrated state of the art performances on unstructured data like images, text, audio and video related tasks. Notwithstanding, deep neural networks pose many challenges when applied to tabular data, such as lack of locality, data sparsity (missing values), mixed feature types (numerical, ordinal, and categorical), and lack of prior knowledge about the dataset structure (unlike with text or images) [START_REF] Kadra | Regularization is all you Need: Simple Neural Nets can Excel on Tabular Data[END_REF]. More traditional approaches based on conventional statistical models are also available and still represent state of the art approaches for modeling structured tabular datasets. However, any selection of tabular datasets cannot represent the full diversity of this type of data. The "no free lunch" principle states that no one model is always better or worse than any other model, demonstrating that deep learning is currently not all what is needed for tabular data, despite the recent significant progress [START_REF] Kadra | Regularization is all you Need: Simple Neural Nets can Excel on Tabular Data[END_REF]. As for today, the use of deep learning applications to modeling tabular data remains highly challenging [START_REF] Borisov | Deep neural networks and tabular data: A survey[END_REF], highlighting the importance of studying traditional statistical learning techniques. Kadra et al. [START_REF] Kadra | Regularization is all you Need: Simple Neural Nets can Excel on Tabular Data[END_REF] called the tabular datasets the last "unconquered castle" for deep neural network models. Nowadays, the number of statistical and machine learning models currently used in the literature is very high and many more are published each year. Notwithstanding, multiple studies have demonstrated that only few of them perform comparably with neural networks when tabular data are used. In particular, tree based models, combined with boosting and bagging techniques, represent nowadays the state of the art approaches to model tabular datasets [START_REF] Shwartz-Ziv | Tabular data: Deep learning is not all you need[END_REF][START_REF] Borisov | Deep neural networks and tabular data: A survey[END_REF]. For this reason, in this thesis, we will restrict the description of machine learning models to the following: Generalized Linear Regression Model (GLM), Support Vector Machines (SVM) and Decision Tree based models. Additionally, we will introduce the concept of bagging and boosting since these statistical approaches represent the real leap forward in performance. Finally, different types of blind source separation techniques based on tensor factorization methods are also described due to their high capability in extracting relevant information from high dimensional data.

1.6.1 Linear models for Regression 1.6.1.1 Generalized Linear Regression Model Linear regression is perhaps one of the most well known and well understood algorithms in statistics and machine learning. Machine learning, and more specifically the field of predictive modeling, is primarily concerned with minimizing the error of a model or making the most accurate predictions possible, at the expense of explainability. In applied machine learning we borrow and reuse algorithms from many different fields, including statistics, and use them towards these ends. For this reason linear statistical models can fell into the category of machine learning models as well as statistical learning models. It is important to understand that linear models can be categorized into a more general set of models usually referred as Generalized Linear Regression Model [START_REF] Mccullagh | Generalized linear models[END_REF]. In these models, the response variable y i is assumed to follow an exponential family distribution with mean µ i , assumed to be some (often nonlinear) function of x T i β. Some would call these "nonlinear" because µ i is often a nonlinear function of the covariates. However, McCullagh et. al [START_REF] Mccullagh | Generalized linear models[END_REF] consider them to be linear because the covariates affect the distribution of y i only through the linear combination x T i β. There are three components to any GLM:

• Random Component: specifies the probability distribution of the response variable. For instance, normal distribution for y corresponds to the classical regression model while a binomial distribution of y corresponds to the binary logistic regression model.

• Systematic Component: specifies the explanatory variables (x 1 , x 2 , ..., x k ) in the model, where k represents the number of regressors (features) used by the model. Specifically, these regressors are related considering a linear combination between model parameters β and features x from the dataset such that

β 0 + β 1 x 1 + β 2 x 2 + ... + β k x k
, where β 0 is called intercept and it estimates the baseline effect when all the corresponding features are set to zero.

• Link Function: specifies the link between the random and the systematic component. In other words, it indicates how the expected value of the response relates to the linear combination of explanatory variables. Usually, such a function is denoted with η and for a logistic regression model is defined as g(π) = η = log( π 1-π ), where π defines the probability of an event to occur.

Thus the GLM model can be expressed as follows:

g(π) = k i=0 β i x i + ǫ (1.17)
where g represents the link function and ǫ the random component.

Linear Regression Models and Least Squares

In a typical scenario, we have a set of training data D = {(x 1 , y 1 ), (x 2 , y 2 ), ..., (x n , y n )} composed of a set of features X and a response variable y from which we want to estimate the parameters β. The most popular estimation method is least squares in which the coefficients β = (β 0 , β 1 , ..., β k ) are selected such that the residual sum of squares (RSS) is minimized:

RSS = N i=1 (y i -f (x i )) 2 = N i=1 (y i -β 0 + k j=1 x ij β j ) 2 = (y -Xβ) T (y -Xβ) (1.18)
From a statistical point of view, this criterion is reasonable if the training observations D represent independent random draws from underlying population. Notwithstanding, even if the regressors X were not drawn randomly, the criterion is still valid if the response variable y is conditionally independent, given the inputs X. It is important to notice that Eq. (1.18) does not make any assumption regarding the validity of the model in Eq. (1.17) when g is the identity matrix. In other words, Least Squares Fitting is satisfied no matter how data arise since the criterion simply measures the average lack of fit. Thus, in order to obtain the best vector of coefficients β from Eq. (1.18), we follow the usual procedure and calculate the first derivative with respect to β as follows:

∂RSS ∂β = -2X T (y -β) (1.19)
The first derivative in Eq. (1. [START_REF] Calabrese | Grey matter lesions in MS: from histology to clinical implications[END_REF]) is equated to zero and solved for β obtaining:

β = (X T X) -1 X T y (1.20)
The above procedure assumes that there is no collinearity between the vector feature in the matrix X. Notwithstanding, methods have been implemented for dealing with such a problem. The most used solutions are two types of Shrinkage Methods. These types of methods are also very helpful when the number of features exceed the number of observations. In such a case, the Least Squares Fitting method is said to have not enough degrees of freedom to estimate parameters (i.e. not a unique solution to the minimization problem). It is also important to notice that, by retaining a subset of the predictors and discarding the rest (i.e. subset selection), we can obtain a model that is interpretable and has possibly lower prediction error than the full model. However, because it is a discrete process, which means that variables are either retained or discarded, it often exhibits high variance. The motivation behind shrinkage methods is that they are more continuous, and do not suffer as much from high variability.

Shrinkage Methods: Ridge Regression

Ridge regression shrinks the regression coefficients by imposing a penalty on their size. The ridge coefficients minimize the following penalized residual sum of squares:

β ridge = argmin β    N i=1   y i -β 0 - k j=0 x ij β j   2 + λ k j=1 β 2 j    (1.21)
where λ ≥ 0 represents a complexity parameter that controls the amount of shrinkage, which is to say that the larger the value of λ the greater the amount of shrinkage. The coefficients are shrunk toward zero. The idea of penalizing by the sum-of-squares of the parameters is also used in neural networks, where it is known as weight decay [START_REF] Hastie | The elements of statistical learning[END_REF]. It should be noticed that the ridge solutions are not immune from scaling the inputs before solving for Eq. (1.23). The RSS can thus be defined as:

RSS = (y -Xβ) T (y -Xβ) + λβ T β (1.22)
and the Ridge regression solution can be obtained through the following equation:

βridge = (X T X + λI) -1 X T y (1.23)
where I represents the identity matrix. It should be noticed that by choosing a quadratic penalty β T β, the final solution in Eq. (1.23) is represented as a linear function of y. The only modification with respect to Eq. (1.20) is the addition of a positive constant to the diagonal element of X T X before inversion. This aspect represents an important point since it makes the problem nonsingular even when X T X is not full rank (i.e. some of the columns in X are collinear). This aspect represents the main motivation for ridge regression when it was first introduced in statistics [START_REF] Hoerl | Ridge regression: Biased estimation for nonorthogonal problems[END_REF].

1.6.1.4 Shrinkage Methods: Lasso Regression Lasso regression is an additional shrinkage method like Ridge regression, with subtle but important modifications. In its Lagrangian form, it can be defined as follows:

β lasso = argmin β    N i=1   y i -β 0 - k j=0 x ij β j   2 + λ k j=1 |β j |    (1.24) Berardino BARILE
It should be noticed that the only difference with the Ridge regression is that in the Lasso regression the quadratic penalization term k j=1 β 2 j , also known as L 2 regularization, is replaced by the sum of absolute values of the parameters β, such that k j=1 |β j |, known as L 1 regularization term. This latter constraint makes the solutions nonlinear with respect to y and thus there is no closed form solution. Thus, in most cases, an optimization method based on quadratic programming is implemented. It should be noticed that, in Ridge regression, there is no such problem and the classical least squares solution can be implemented. Also, the use of a Lasso regression will cause some of the coefficients to be exactly zero as the parameter λ is increased. This implies that Lasso regression can also be used as a feature selection method. Finally, for both Ridge and Lasso regression, the optimal parameter λ is not obvious and a usual cross-validation procedure should be implemented.

Linear model for classification: Logistic Regression

Logistic Regression is a special type of linear regression model which seeks to find a linear relationship between the predictors X and a binary response Y, assuring at the same time that, the obtained probabilities, for each predicted instance of each possible event, remains in the range [0 -1] and their sum is equal to one. The usual mathematical formulation of Logistic regression, when the response variable is binary, can be written as follows:

g(π) = P r(G = k|X = x) = exp(β 0 + β T x) 1 + exp(β 0 + β T x) = β 0 + β T x (1.25)
Logistic regression models are usually fit using the conditional log-likelihood of the form:

l(θ) = N i=1 log p g i (x i , θ) (1.26)
where we simplified p g i = P r(G = k|X = x; θ). Let's define with p(x; θ) the probability of an event to occur while with 1p(x; θ) the probability of the event not occurring. In the binary case, we can express the log-likelihood in terms of the parameters β as follows:

l(θ) = N i=1 [y i log p(x i ; β) + (1 + y i ) log(1 -p(x i ; β))] = N i=1 y i β T x i -log(1 + e β T x i ) (1.27)
To maximize the log-likelihood, we set its derivatives to zero as usual, obtaining the following expression:

∂l(β) ∂β = N i=1 x i (y i -p(x i ; β)) = 0 (1.28)
To solve Eq. (1.28), the Newton-Raphson algorithm is usually used, which requires the second-derivative (i.e. Hessian matrix) defined as follows:

∂ 2 l(β) ∂β∂β T = - N i=1 x i x T i p(x i ; β)(1 -p(x i ; β)) = 0 (1.29)
Thus, let's denote with β old the parameter vector in a previous iteration, a single Newton update can be obtained as follows:

β new = β old - ∂ 2 l(β old ) ∂β old ∂β old T -1 ∂l(β old ) ∂β old
(1.30)

Support Vector Machines

Support vector machines (SVMs) is a supervised non-parametric statistical learning technique. Therefore there is no assumption made on the underlying data distribution. Its original formulation was proposed by Vapnik [START_REF] Vapnik | Estimation of dependences based on empirical data[END_REF]. The method is presented with a set of labeled data instances and the SVM training algorithm aims to find a hyperplane that separates the dataset into a discrete predefined number of classes consistent with the training examples. The term optimal hyperplane is used to refer to the decision boundary that minimizes misclassifications, obtained during the training phase. Learning, refers to the iterative process of finding a classifier with optimal decision boundary to separate the training patterns (in potentially high-dimensional space) and then to transfer the learned pattern to unseen data under the same configurations [START_REF] Zhu | Classification using aster data and svm algorithms: The case study of beer sheva, israel[END_REF]. In Figure 1.8 an intuitive representation of the optimal margin solution is proposed.

In its simplest form, SVMs are linear binary classifiers that assign a given test sample to a class from one of the two possible labels. In Figure 1.8, a simple scenario of a two-class separable classification problem, in a two-dimensional input space, is depicted. An important generalization aspect of the SVM model is that not all the available training examples are used for the identification of the hyperplane. The subset of points that lie on the margin, and called support vectors, are the only ones that define the hyperplane of maximum margin. The hyperplane (line when only two features are considered) with the maximal margin (i.e. the maximum distance between data points in both classes), represents the final decision boundary. Maximizing the margin distance provides a reinforcement role so that future data points can be classified with more confidence. Mathematically, let's define with y i ∈ {-1, 1} n the two class labels, where n defines the number Figure 1.8: Decision boundary for Support Vector Machine. Image taken from [START_REF] Mountrakis | Support vector machines in remote sensing: A review[END_REF] of instances given a set of weights w and a bias coefficient b. The "soft margin" solution can be identified by solving the following optimization problem:

min w,b 1 2 ||w|| 2 + λ n i=1 ǫ i subj. to y i (w T φ(x i ) + b) ≥ 1 -ǫ i , i = 1, ..., n ǫ i ≥ 0 i = 0, ..., n
where ǫ is a relaxation variable of the optimization problem. These elements are hyperparameters that need to be tuned in order to optimize the bias-variance trade-off. The φ(x i ) term represents a mapping function that assign feature vectors x i ∈ R m to a higher dimensional space. Thus, the Lagrangian duality formulation of this problem can be expressed as follows:

max α n i=1 α i - 1 2 n i,j=1 y i y j α i α j φ(x i )φ(x j ) subj. to 0 ≥ α i ≥ λ, i = 0, ..., n n i,j=1 α i y i = 0
where α i are Lagrange multipliers and • is the inner product operator and the corresponding K(x, y) = φ(x i )φ(x j ) is called kernel. Different kernel functions, mapping Figure 1.9: Decision Tree Model. Image taken from [START_REF] Jijo | Classification based on decision tree algorithm for machine learning[END_REF] input in higher dimensional space, are described in the literature, such as linear, polynomial, sigmoidal and radial basis function (RBF) kernel among the most used ones.

Decision Tree

The Decision Tree (DT) algorithm is part of the supervised learning algorithms family. Its main objective is to construct a training model that can be used to predict the class or values of the target variables through learning simple "if-else" decision roles inferred from the training dataset. In other words, this algorithm is characterized by a sequence of models that unites a series of basic tests efficiently and cohesively where a numeric feature is compared to a threshold value in each test [START_REF] Damanik | Decision tree optimization in c4. 5 algorithm using genetic algorithm[END_REF]. The DT algorithm can be used to solve both regression and classification problems. More precisely, DT is a technique in which any path beginning from the root node is described by a data separating sequence until a Boolean outcome at the leaf node is achieved. DT is one of the most powerful methods commonly used in various fields, including biomedical applications. An intuitive representation of the threshold-based decision roles implemented by the DT algorithm is proposed in Figure 1.9.

Thus in essence, DT makes decisions by splitting nodes into sub-nodes. This process is performed multiple times during the training process until only homogenous nodes are left. Therefore, node splitting is a key concept. It is the process of dividing a node into multiple sub-nodes to create relatively pure nodes and it can be broadly divided into two categories based on the type of target variable: Continues Target versus Categorical target. There are many techniques for constructing regression trees. One of the oldest and most utilized is the classification and regression tree (CART) [START_REF] Wei-Yin | Classification and regression trees[END_REF] model. Let's consider for instance a general regression problem. The model begins with the entire data set (S) and searches every distinct value of every predictor to find the predictor and split value that partitions the data into two groups (S 1 and S 2 ) such that the following overall sums of squared error is minimized:

SSE = i∈S 1 (y i -ŷ1 ) 2 + j∈S 2 (y j -ŷ2 ) 2 (1.31)
where ŷ1 and ŷ2 are the averages of the training set outcomes within groups S 1 and S 2 , respectively. Then within each of the groups S 1 and S 2 , this method searches for the predictor and split value that best reduces SSE. Because of the recursive splitting nature of regression trees, this method is also known as recursive partitioning. Also, the primary challenge in the decision tree implementation is to identify which attributes do we need to consider as the root node at each level. This is known as "attributes selection". We have different attributes selection measures to identify the attribute which can be considered as the root note at each level. The decision of making strategic splits heavily affects the model performance. When the outcome is numeric, the relevant improvement is the difference in the sum of squared errors between the node and its child nodes after the split. Mathematically, we can define the squared error as follows:

n i=1 (y i -c) 2 (1.32)
where n is the number of cases in the specific node, c is the average outcome of all cases at that node, and y i is the outcome value of the i-th case. If y i is close to c ∀i ∈ {1, 2, .., n}, then the resulting error is low. A good clean split will create two nodes which both have all elements close to the average outcome, thus reducing the overall error. Conversely, when the outcome is categorical, multiple types of measures can be considered. For instance, the Information Gain is used as such a measure and it is related to the concept of entropy. Mathematically, it can be defined as follows:

Inf ormationGain = 1 -Entropy (1.33)
Thus Entropy and Information Gain are two related concepts used to measure the impurity and randomness of the data. The value of Entropy always lies between 0 and 1. Its value is better when it is equal to 0 while it is worse when it is equal to 1. Mathematically, we 40 where p i represents the ratio of the samples with the i-th attribute value. Gini Impurity is another measure usually used in DT model for splitting nodes when the target variable is categorical. It is the most popular and the easiest way to split a decision tree model. The Gini Impurity is defined as follows: [START_REF] Eshaghi | Progression of regional grey matter atrophy in multiple sclerosis[END_REF] where:

GiniImpurity = 1 -Gini (1.
Gini = n i=1 p 2 i (1.36)
Additionally, Information Gain is a statistical property that measures how well a given attribute separates the training examples according to their target classification. It computes the difference between Entropy before the split and average Entropy after the splitting. Mathematically, it can be defined as follows:

Inf ormationGain = Entropy(bef ore) - K j=1
Entropy(j, af ter) (1.37) where "before" is the dataset before the split, K is the number of subsets generated by the split, and (j, after) is the subset j after the split. However, Information Gain is biased toward choosing attributes with a large number of values as root nodes. It means it prefers the attribute with a large number of distinct values. Gain Ratio overcomes the problem by taking into account the number of branches that would result before making the split. It corrects Information Gain by taking the intrinsic information of a split into account. Mathematically, we can define the Gain Ratio as follows:

GainRatio = Entropy(bef ore) -K j=1 Entropy(j, af ter) K j=1 w j log 2 w j (1.38)
The algorithm selection is also based on the type of target variables as well as the metric used for decision splitting. The most important algorithms used in the literature for DT model are: ID3, C4.5, CART, CHAID, MARS. For an exhaustive comparison between these algorithms we refer to [START_REF] Song | Decision tree methods: applications for classification and prediction[END_REF].

Problems and Solutions of Decision Tree Models

The common problem with DT is that, most often, the tree memorizes the training data set. This means that the model has low bias but high variance. A possible solution to this problem is to "prune" the tree eliminating terminal leaves based on a pre-defined logic, such as minimum number of observation in each leaf. In fact, if no limit is imposed to the number of splits made by the tree, we can ideally obtain 100% accuracy on the training data set by simply making one leaf for each observation. However, this does not represent an interesting behavior since the final aim of any machine learning model is that of generalizing to unseen data. Fully growing a complete tree affects the generalization performance of the model when predicting samples that are not part of the training set. This concept is called overfitting. For this reason, in almost all cases, the depth of the tree is limited (i.e. "pruning") in such a way that the generalization performance is preserved (i.e. limiting the depth of the tree). Notwithstanding, finding the "sweat spot" that allows to obtain good results on the training set (low bias) but, at the same time, able to generalize to unseen data (i.e. low variance) is generally very difficult. Hence why are tree-based models so popular? The concept of bias-variance trade-off in machine learning has been fully tackled and different techniques have been implemented demonstrating excellent performances for both cases of high bias or high variance problems. Decision tree models have demonstrated to be one of the best approaches to tackle this trade-off. For this reason we will now introduce two important statistical concepts: bagging and boosting.

Bagging

Bagging, also known as bootstrap aggregation, is a statistical technique which combines multiple machine learning models, trained in parallel, each considering a different subset of the data, producing different classifiers combined in an ensemble meta-learning algorithm with the aim to improve stability and accuracy of the model [START_REF] Breiman | Bagging predictors[END_REF]. In other words, bagging is a method for generating multiple versions of a machine learning model in a unique meta-model. The aggregation is obtained by averaging out the results (i.e. output) of each single DT model, when predicting a numerical outcome, or equivalently using majority voting when a classification task is considered. More formally, let's define with D = {(x n , y n ); n = 1, 2, ..., N } the underlying data distribution, where x n represents the set of features used as input to the machine learning model Φ(x|D). Also, let's consider with {D k }∀k = 1, 2, .., N the dataset consisting of N independent observations obtained from the same underlying distribution D. We aim to obtain a better performing model than a simple learning predictor trained on a single dataset D k . A reasonable solution would be to train multiple machine learning models, one for each single dataset D k , which we denote with {Φ(x|D k )}. We can obtain an ensemble of machine learning models by combining all the predictions of each model Φ(x|D k ). Notwithstanding, in real life applications, usually a single training dataset D T is provided without any access to the data generation process. A close enough solution to the aforementioned procedure is based on the repeated bootstrapped resample approach. Practically speaking, multiple samples of data are drawn with replacement from D T forming a set {D b } ∀b = {1, 2, .., B} where D b is obtained by sampling both rows with replacement, corresponding to data instances and columns, corresponding to features, from the original training dataset D T . The dataset D b is used as input to the machine learning model which we define as {Φ(x|D b )}. Finally, the models are combined in a meta-model by averaging or majority voting as previously discussed. This procedure is called bootstrapping aggregation or bagging. For a more formal mathematical definition and proof of the bagging technique we refer to [START_REF] Breiman | Bagging predictors[END_REF]. Intuitively, the idea of bagging follows naturally from the bootstrapping procedure as independent and homogeneous machine learning models are trained on different samples that can be considered as almost independent and identically distributed (iid) bootstrapped representations of the original dataset and their results aggregated through a weighted average. The reasoning behind training on these bootstrap samples is that if samples have low correlation, we get a better representation of the population. Notwithstanding, it is important to keep in mind that the bagging technique is used to reduce variance of a model while keeping bias constant. As a consequence, deep learning and decision tree models are the most used algorithms for the application of the bagging procedure since overfitting represents the main concern for these models. In Section 1.7 a more in-depth explanation of different deep learning architectures and related concepts is offered.

Random Forest

As previously stated, DT are sensitive to the specific data they are trained on. If the training data is changed (e.g. a tree is trained on a subset of the training data), the resulting DT can be quite different and in turn the predictions can be quite different (i.e. high variance). Random Forest (RF) is a DT-based model defined as an ensemble learning method where multiple DTs are combined applying the general technique of bootstrap aggregating (i.e. bagging). It can be used for classification, regression and other tasks, like anomaly detection. For classification tasks, the output of the RF is the class selected by most trees. For regression tasks, the mean or average prediction of the individual trees represents the final prediction. One of the advantages of RF is that it is invariant under scaling and various other transformations of feature values and it has been empirically demonstrated to be resilient to the inclusion of irrelevant features [START_REF] Hastie | The elements of statistical learning[END_REF]. This type of machine learning model exploits the high variance aspect of a basic DT model, meaning that trees that are grown very deep tend to learn highly irregular patterns, overfitting the training sets. After training, predictions for unseen samples can be made by combining the predictions from all individual trees:

Φ = 1 B B b=1 Φ(x|D b ) (1.39)
The algorithm implicitly performs feature selection by choosing an available node that produces the most homogeneous (i.e. purest) sub-branches using criteria such as Information Gain or Gini Index. Also, in RF, attributes are ranked according to their average rank scores across all trees. However, it is important to realize that, simply training many trees on a single training set would give strongly correlated trees (or even the same tree many times, if the training algorithm is deterministic); bootstrap sampling is a way of de-correlating the trees by showing them different training sets. Notwithstanding, another type of bagging schemes can also be considered. For instance, a modified tree learning algorithm can be considered such that it selects, at each candidate split in the learning process, a random subset of the features. This process is sometimes called "feature bagging". The reason behind this additional step is to limit as much as possible the correlation between model trees. In fact, if one or a few features are very strong predictors for the response variable (target output), these features will be selected in many of the B trees, causing them to be correlated. An interesting extension to the RF approach is to add one further step of randomization, yielding a new model known as Extremely Randomized Trees (ExtraTrees). While similar to RF, in the sense that it is also based on an ensemble of individual trees, there are two main differences. First, each tree is trained using the whole learning sample (rather than a bootstrap sample), and second, the top-down splitting in the tree learner is randomized. Practically speaking, instead of computing the locally optimal cut-point for each feature under consideration (based on, e.g., Information Gain or Gini Impurity), a random cut-point is selected. This additional randomization step is introduced following the same line of reasoning to which Random Forest is based on, reducing the correlation between decision tree models.

Boosting

Unlike many machine learning models, which focus on high quality predictions done by a single model, boosting represents an ensemble technique that seeks to improve the prediction power by training a sequence of weak models. The main rationale of boosting is to train machine learning models sequentially in such a way that each subsequent model performs especially well where previous ones failed to achieve a high predictive performance [START_REF] Freund | Experiments with a new boosting algorithm[END_REF]. More precisely, boosting is an ensemble learning technique that uses a set of machine learning algorithms to convert weak learner into strong learners with the aim of increasing the performance of the final meta-model on a specific task. The idea of combining multiple weak learners into a unique strong learner was first generated by the question posed by Kearns et. al [START_REF] Kearns | Crytographic limitations on learning boolean formulae and finite automata[END_REF]: "Can a set of weak learners create a single strong learner?". A weak learner is defined as a model that is only slightly correlated with the ground truth label, meaning that the model can produce predictions just slightly better than random guessing. In contrast, a strong learner is a model that is arbitrarily wellcorrelated with the true label. Also, boosting is a general algorithmic agnostic technique, which means that different machine learning models, other than DT, can be used. The procedure is implemented in such a way that each weak learner in the chain concentrates on minimizing the prediction error on a specific subset of instances wrongly predicted by the previous weak learner. In the work of Rojas [START_REF] Rojas | Adaboost and the super bowl of classifiers a tutorial introduction to adaptive boosting[END_REF], the author nicely demonstrated that as long as the performance of each single weak learner is slightly better than random guessing, the final model can be proven to converge to a strong learner. More precisely, after a weak learner is trained, a weight is assigned to each instance in the dataset (process known as "re-weighting") and the new subsequent learner is trained on the newly obtained weighted dataset. In other words, wrongly predicted input data gain a higher weight while examples that are predicted correctly lose weight. Thus, each subsequent weak learner focuses more on the examples that previous weak learners wrongly predicted. The boosting technique has been demonstrated to reduce the bias error component without increasing the variance component [START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF]. This means that although at each iteration (i.e. each new weak learner added to the chain) we increase the performance on the training set (lower bias), overfitting (i.e. high variance) does not occur until a very high number of iterations. This aspect has been thoroughly studied in the literature and it represents the basic building block of why boosting methods have became nowadays the state of the art approach for both regression and classification tasks when tabular datasets are used. Such datasets, as the name suggest, refers to data structured into a tabular form, it arranges data elements in vertical columns and horizontal rows where a cell is formed by the intersection of a column and row. More formally, let's define with f t (x) a weak classifier at iteration t, a strong classifier can be defined by summing all the weak classifiers such that:

F T (x) = T t=1 f t (x) (1.40)
where T represents the total number of boosting iterations and f t (x) output a -1 or +1 values for each instance in the training dataset. In the easiest case, F T (x) simply represents the sum of the sign obtained from each weak learner. Notwithstanding, in order to avoid overfitting the training dataset, almost always, the output of each weak learner is weighted based of a specific parameter α, which is called learning rate. One of the best approaches for rapidly converging toward the optimal solution is to tune the α parameter at each boosting iteration based on the prediction error obtained from the contribution of all weak learners until iteration T. This type of boosting method was the first popularized one and takes the name of Adaptive Boosting (AdaBoost). It can be defined as follows:

E t = i E [F T -1 (x i ) + α t h(x i )] (1.41)
where F T -1 (x i ) represents the boosting ensemble classifier that is obtained until iteration t -1, while f t (x) = α t h(x i ) is the weak learner that is added at iteration t. In [START_REF] Rojas | Adaboost and the super bowl of classifiers a tutorial introduction to adaptive boosting[END_REF] the optimal value for α t was derived, resulting in:

α t = 1 2 ln 1 -ǫ t ǫ t (1.42)
where ǫ t defines the residual of the ensemble boosting estimator up until iteration t. Eq. (1.61) essentially represents the negative logit function multiplied by 0.5. Nowadays, multiple variants of the Boosting algorithm are present in the literature. However, one of the most famous evolutions of such an approach is the Extreme Gradient Boosting (XGBoost) algorithm. A primary trait that distinguishes XGBoost from other boosting algorithms, such as the traditional AdaBoost, is its capability to use any differentiable loss function [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF]. This is achieved by using the gradient descent algorithm to minimize the loss function when training subsequent trees, hence the "gradient" term in its name. Moreover, the XGBoost algorithm easily scales to huge amounts of data, while using a minimal amount of resources. This is also the reason for the "extreme" term in its name -it pushes the computation limits of machines. Lastly, it often yields highly accurate results in practice, a reason for it to repeatedly be on top of competition-winning algorithms in data science challenges. For more specific information on ensemble learners and on gradient boosting (and boosting in general), we refer to the book "The elements of statistical learning" [START_REF] Hastie | The elements of statistical learning[END_REF]. For completeness, we also refer to Categorical Boosting (CatBoost) [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF] and Light Boosting [START_REF] Ke | Lightgbm: A highly efficient gradient boosting decision tree[END_REF] as additional well known boosting techniques demonstrated to reach state of the art performances in both regression and classification tasks. For a detailed mathematical description of these models we refer to Schapire et. al [START_REF] Schapire | Boosting: Foundations and algorithms[END_REF].

Deep Learning

In the last decade, Deep Learning models gained a lot of popularity, given their impressive results for the analysis and classification of images in a wide range of applications [START_REF] Goodfellow | Deep learning. Adaptive computation and machine learning[END_REF]. The study of DL has been inspired in part by the observation that biological learning systems are built out of very complex webs of interconnected neurons. If we observe the human brain we discover that it is formed by millions of little components called neurons. In rough analogy, DL models are built out of a densely interconnected set of simple units, where each unit takes a number of real-valued inputs (possibly the outputs of other units) and produces a single real-valued output (which may become the input to many other units) [START_REF] Mitchell | Machine learning[END_REF]. To better understand the mechanism under which artificial neural network model works, we can represent such a model as an oriented graph, where nodes are called neurons and neurons are connected to each other by weighted arcs. Each neuron fires or inhibits other neurons, depending on whether the value computed exceeds a certain threshold. A graphical comparison between the biological and artificial similarity of a single neuron is proposed in Figure 1.10. The first and most important type of artificial neuron is called perceptron and was first developed in between 1950s and 1960s by the scientist Frank Rosenblatt and inspired by earlier work by Warren McCulloch and Walter Pitts [START_REF] Mcculloch | A logical calculus of ideas immanent in nervous activity[END_REF]. More formally, given a vector x of values as input, it is possible to assign at each value of the vector x a specific weight (i.e. importance) such that the linear combination of all these value-to-weight products result with a desired value z:

z = x • w = n i=1 (x i w i ) + b (1.43)
where b is called the bias. Usually, a non-linear function is applied to the output z of the linear combination and it is called "activation function". In the next subsection a list of four of the most used activation functions are discussed.

Formally, let's define with y = f (x) a generic mapping function which takes a generic input vector "x" and produces and output "y". Following this paradigm, we can build a computational learning system by stacking multiple functions one after the other (i.e., nested) such that f * (x) = f (3) (f (2) (f (1) (x))), where the upper index denotes the order in which to apply each function (first f (3) then f (2) etc.). The function f (i) is usually referred to as i-th layer. More specifically, the first and last layer are called input and output layer Figure 1.10: Similarity between biological and artificial neuron. Image taken from https://pixabay.com/users/clker-free-vector-images while all the remaining layers in between are called hidden layers. The overall length of the chain gives the depth of the model and it is from this terminology that the name "deep learning" arises. The aim of the learning algorithm is to decide how to use the intermediate layers to best implement an approximation of f * (x), by iteratively updating the parameters θ.

Activation functions

In the context of DL, activation functions are the most informative ingredient of Deep Learning which is fundamentally used to determine the output of a DL model. Each layer of a neural network can employ arbitrary activation functions and the choice of the non-linearity strictly depends on the kind of problem we want to afford. A graphical representation of the four most used activation functions in DL are depicted in Figure 1.11. It is important to understand that, if a linear (i.e. no activation) function would be used, the output of the neuron would only be a linear combination of the inputs. Typical functional choices are:

• Sigmoid or Logistic:

σ(z) = 1 1 + e -z (1.44)
where z represents the input (i.e. linear combination of features) usually obtained from previous layers stacked together. The sigmoid non-linearity takes a real-valued number and "squashes" it into range [0,1]. It produces outputs similar to the Perceptron step-function with the main difference of being a continuous function. Historically it was frequently used. However, due to the drawback of saturation at either values of 0 or 1, it should not be used in middle/hidden layers since the gradient is small (near zero) for such extreme points, leading to computational issues. This is not a desirable behavior since during backpropagation it could cause the gradient to vanish, making learning difficult. In this context the initialization of the weights plays a fundamental role. For example too large weights could lead most of the neurons to saturate.

• Hyperbolic tangent or Tanh:

tanh(z) = 1 -e -2z 1 + e -2z = 2σ(2z) -1 (1.45)
The hyperbolic tangent is a continuous function that "squashes" a real-value number into range [-1,1]. Like the sigmoid, the tanh non-linearity suffers from the saturation problem, but it is centered at zero and this is a desirable property during training. For this reason, tanh is usually preferred to the sigmoid function for hidden layers activation.

• Rectified Linear Unit (ReLU):

ReLU(z) = max(0, z) (1.46)
ReLU non-linearity has become very popular in the recent years. Many works prove that ReLU units accelerate the learning convergence with respect to sigmoid and tanh non-linearities [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. The main drawback of the ReLU units is that they can vanish during training. Suppose that most inputs to a ReLU unit are negative. This cause the gradients flowing through the unit to be zero and then each neuron going into the unit will stop to respond to the variation in the error. This problem could cause the "death" of several parts of the network. A possible solution is obtained by simply modifying the part of the function at risk of null gradient, such as the negative feasible range. Leaky Rectified Linear Unit, or Leaky ReLU, is a type of activation function based on a ReLU, but it has a small slope for negative values instead of a flat slope. The slope coefficient is determined before training, i.e. it is not learnt during training, and thus it represents an hyperparameter. This type of activation function is popular in tasks where we may suffer from sparse gradients, for example when training generative adversarial networks.

Optimization

In the context of ANN, optimization refers to the process of adjusting the weights in order to obtain a desired output. One of the easiest ways to learn how weight optimization is actually performed is to start from random weights and iteratively adjusting these weights following a pre-specified role. The back-propagation algorithm [START_REF] Rumelhart | Learning representations by back-propagating errors[END_REF] is a highly efficient methodology that works with derivatives to find the optimal parameters. However, it is common that a solution to this equation is not a global solution, meaning that it is not possible to guarantee that the performance resulting from this set of parameters represents the best possible solution. To better understand this concept, let's define with w i the weight associated to the feature x i and with j the index of a sample, then one simple role for weight update is the following:

w(t + 1) = w i (t) + ∆w i (1.47)
where w(t + 1) represents the weight value at iteration t+1 while ∆w i represents the "direction" to follow in order to obtain the desired output (i.e. minimal error) and it is defined as follows:

∆w i = η(z j -y j ) (1.48)
In this formulation z j represents the predicted output from the network for example j while y j represent the desired output. Such a difference is called pseudo-residual and it measure the amount of error made by the model. Notwithstanding, it is both theoretically and empirically proven that aggressive adjustment of the weights based on the error with respect to a desired output leads to poor results. For this reason, the additional term η is usually added in order to mitigate the degree of which weights are adjusted and it is call learning rate. This process is repeated iteratively until a pre-defined stopping criterion is met. The strategy just described for weight updating is called gradient descent and, at present, many variants are proposed, which however based their main concept on the aforementioned procedure. It should be noticed that, in Eq. (1.48), a simple difference between prediction and target is considered. However, in practice, a differentiable function is always used which is called "loss function". Such a function can vary based on the optimization task and can be customized in order to meet the required pre-defined conditions. For instance, for a classical regression task, a least squared difference between predicted and target value can be considered, while for a classification task the cross-entropy loss Figure 1.12: Intuitive representation of the gradient descent optimization process. Image taken from https://www.analyticsvidhya.com/ is usually used. Intuitively, we can imagine the search space as a valley and we imagine a ball rolling down the slope of the valley. Our everyday experience tells us that the ball will roll to the bottom of the valley. Gradient descent uses this idea as a way to find a minimum for the function randomly choosing a starting point for an (imaginary) ball, and then simulate the motion of the ball. We could do this simulation simply by computing derivatives of the cost function, which effectively represent the direction in which we have to move in order to make the "ball" rolling downward. In Figure 1.12 an intuitive representation of the gradient descent optimization concept is depicted.

Backpropagation

Notwithstanding, multi-layer networks cannot be trained using the perceptron's learning rule previously described since it represents a simplification of the real learning process, in which we need to compute the error term between targets and predictions for the single unit. In most cases, in fact, a multi-layer network is used which does not respect the prerequisites previously explained due to the presence of multiple hidden-layers. The ground truth is available only for the last layer that outputs the predictions, so it is possible to evaluate the error only after the computation of the whole network. For this reason we need a new procedure able to carry the error information back to all the layers so that the network weights can be updated accordingly. Such an approach takes the name of "backpropagation" [START_REF] Rumelhart | Learning representations by back-propagating errors[END_REF] and it represents the essence of the learning role followed by modern neural networks. Practically speaking, backpropagation allows to compute the gradients for a general network, having arbitrary feed-forward topology and differentiable activation functions. The partial derivatives of the error with respect to a weight tells us the sensitivity of the error on its value. Then, by computing the partial derivatives with respect to the weights of the last layer, we can determine the amount by which each neuron is contributing to the error. From the basic concepts of calculus, we know that this calculation can be extended to all layers of the network by means of the "chain rule". In fact, it allows to further compute the partial derivatives of the error with respect to the weights of the previous layer until the contribution of each weight of the network to the error is determined. The backpropagation algorithm is then composed by three main steps:

• Forward pass: In this phase, information is propagated forward through the network to compute values from input to output for each neuron. Practically speaking, the input of each layer is fed to the f (i) function, where i defines the position of the hidden layer and output a set of values, which are used as input to the next function f (i+1) representing the next hidden layer. This process is repeated until the last layer is reached, which corresponds to the output of the network.

• Error computation: once the output of the last layer is obtained (output of the network), the error value is calculated by comparing the output of the network ŷ with the actual value y representing the ground truth. The type of function that is used in order to calculate the final error, guides the behavior of the network. For instance, with mean squared difference as loss function, instances with disproportionately high error value determine the behavior of the optimizer much more than other instances with lower error based on a quadratic rule. Conversely, by using the absolute deviation error, very large errors on few instances do not influence the overall behavior of the optimization process, thus providing a more conservative approach to the optimization task, although gradients may not be computed along the entire domain.

• Backward pass: The error information is propagated backward through the network starting from the last layer and recursively applying the chain rule until all the network parameters are updated. Intuitively, the optimal direction toward which we need to optimize the weights of each hidden layer is obtained thanks to the gradient information and the weights are updated considering only a small step toward the optimal direction (defined by the learning rate parameter).

Deep Learning Architectures

The way neurons and layers are connected determines the type of the architecture of the network. The choice about what type of neural network to use is strictly related to the problem we are facing. Indeed, different types of DL network are suited for solving different types of problems. For an in-depth overview of the main DL architectures we refer to Goodfellow [START_REF] Goodfellow | Deep learning. Adaptive computation and machine learning[END_REF]. In the following subsections, four different architectures are introduced, such as: Deep Artificial FeedForward Neural Network, Convolutional Neural Network, AutoEncoder Neural Network and Generative Adversarial Neural Network.

Deep Artificial FeedForward Neural Network

It is an artificial neural network wherein connections between nodes do not form a cycle. As such, it is different from its descendant: recurrent neural networks (see [START_REF] Goodfellow | Deep learning. Adaptive computation and machine learning[END_REF]). The feedforward neural network was the first and simplest type of artificial neural network where the information moves in only one direction (i.e. forward) from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network. Specifically, the output of a single neuron could serve as the input for a subsequent neuron. Thus, multiple neurons could be stacked on top of each other and their outputs can be combined into a single output node to form a multi-layer perceptron neural network. The stacked neurons are called the hidden layer and a neural network could contain multiple hidden layers to extract more complex information. A graphical intuition of this type of DL architecture is shown in Figure 1. [START_REF] Breiman | Random forests[END_REF] In this way, the number of parameters of a neural network easily exceeds those from other methods. These include the weights and biases, but usually the number of neurons in the hidden layer and the number of hidden layers represent additional hyperparameters. To train a multilayer Artificial FeedForward Neural Network, backpropagation is used. The calculation of the gradient starts at the weights of the final layer and its partial derivation is further used in the previous layers as explained in Section 1.7.2. As such, weights update information flow backward for efficient computation.

Convolutional Neural Network

The network becomes more powerful by increasing its size and complexity. Together with the rise of data availability and computer processing power, the field of deep learning has gained immense attention over the past decade. As a consequence, a multitude of network types have emerged, such as convolutional neural networks (CNN). It is one of the most commonly used networks in the context of brain image analysis and computer vision tasks, dominating most of the scientific contributions related to the MS disease.

54

Berardino BARILE Figure 1.13: Intuitive representation of a simple feedforward neural network for binary classification. Image taken from [START_REF] Dixon | Classification-based Financial Markets Prediction Using Deep Neural Networks[END_REF] Since their first definition by Yann LeCun et al. in 1999 [START_REF] Lecun | Object recognition with gradientbased learning[END_REF], the impressive results obtained in various tasks allowed an exponential growth of their popularity. The idea of convolution is inspired partly by computer science and partly by biology. The particular structure of these networks allows the property of translation invariance. This means that it learns the identification of each object in every possible position. Furthermore, CNNs are fast to train and particularly well-adapted to classify images. Convolutional neural networks use three basic ideas: local receptive fields, shared weights, and pooling:

• Local receptive fields: Differently from a fully connected network, where each neuron is connected with every neuron of the previous layer, in a classic CNN architecture each neuron is connected only with small, localized regions of the input image. Such local receptive field is slided over by a fixed number of pixels (stride) to connect to a second hidden neuron.

• Share weights and biases: Convolutional neural networks uses the same weights and biases for each hidden neuron so that the output of the i-th and j-th hidden neuron is given by:

σ(b + n-1 l=0 n-1 m=0 w l,m a j+l,k+m ) (1.49)
where n is the square root of the local receptive field's size, σ is the activation function, b is the shared value for the bias, w l,m is a n × n vector of shared weights and a x,j denotes the input values. Such mathematical formulation is known as convolution. Shared weights and biases give to convolutional neural networks an important characteristic. In fact, all the neurons in the first hidden layer detect exactly the same feature, like for instance edges in the images or other types of shapes, although at different locations in the image. Such property is called translation invariance principle. The map from the input layer to the hidden layer is known as feature map.

The network structure we described so far has just one feature map and thus it can detect just a single kind of localized feature/shape. However, to perform complex tasks, such as image recognition, we will need more than one feature map, and thus a complete convolutional layer, consisting of several different feature maps, is usually used.

• Pooling: In addition to the convolutional layers, convolutional neural networks usually contain pooling layers. They are used to simplify information coming from convolutional layers. Specifically, they work combining the output of a feature map in a single smaller map by considering a predefined statistic such as mean of max function of a subset of neurons.

In practice, when solving problems in the real world, these steps can be combined and stacked as many times as needed to address different problems. The more convolutionsl steps taken, the more complicated features the network will be able to learn and recognize.

AutoEncoder Neural Network

An autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning). The encoding is validated and refined by attempting to regenerate the input from the input encoded representation. More specifically, the autoencoder learns a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore insignificant information ("noise"). In other words, an autoencoder neural network has as objective that of copying its input to its output. Internally, it has a hidden layer that describes a set of feature code used to represent the input. The network may be viewed as consisting of two parts: an encoder function h = f (x) and a decoder function r = g(h), where the latter tries to reconstruct h from r, where r has usually a much lower dimensionality than h. A graphical representation of a classical FeedForward AutoEncoder architecture is proposed in Figure 1.14. The characteristics of reducing the input to a lower dimensional representation, essentially forces the network to consider only useful properties/patterns from the input data. Noteworthy is the fact that, the simplest way to perform the copying task perfectly would be to duplicate the signal. Instead, autoencoders are typically forced to reconstruct the input approximately, preserving only the most relevant aspects of the data. The first applications date back to the 1980s. Their most traditional application was dimensionality reduction or feature learning, but the concept became widely used for learning generative models of data.

Generative Adversarial Neural Network

A generative adversarial network, also known with the acronym of GAN, is a class of machine learning frameworks designed by Ian Goodfellow et al. [START_REF] Goodfellow | Generative adversarial nets[END_REF]. The basic idea consists of two neural networks playing a zero-sum game. Given a training set, this technique learns to generate new data with the same statistical properties as the training set. For example, a GAN trained on images can generate new instances of data that look realistic to human observers, with many realistic characteristics, becoming indistinguishable from real images. A visual representation of a GAN architecture for image data generation is depicted in Figure 1.15.

To understand GANs, it is important to first make clear the distinction between a discriminative and a generative model. The former represents a classical DL algorithm in which given the features of an instance of data they predict a label or category to which that data belongs. More formally, given a vector of feature x and a response variable y a discriminative model learn the function f (x) such that ŷ and y are as close as possible based on the "closeness" measure that it is considered (for instance based on squared difference). Thus, discriminative algorithms map features to labels and they are concerned Intuitive representation of a Generative Adversarial Architecture (GAN) in the context of brain image generation.

Image taken from https://wiki.pathmind.com/generative-adversarial-network-gan solely with that correlation such that the conditional probability P (y|x) is estimated. Conversely, generative algorithms are concerned with the opposite task. In fact, instead of predicting a label given certain features, they attempt to predict features given a certain label. Thus, the question a generative algorithm tries to answer is that given a response variable y which are the most likely features x generating such output?. In other words, generative models attempt to learn the opposite mapping function P (x|y) with respect to discriminative class of models. Practically speaking, in order to build a GAN, two neural networks are trained one against the other. One neural network, called generator, generates new data instances, while the other, called discriminator, evaluates them for authenticity. Intuitively, we can think of a GAN as the opposition of a counterfeiter and a cop in a game against crime, where the counterfeiter is learning to commit the crimes, and the cop is learning to detect them. Thus, both networks are trying to optimize a different and opposing objective function.

Unsupervised Manifold Learning

High-dimensional datasets can be very difficult to visualize. While data in two or three dimensions can be plotted to show the inherent structure of the data, equivalent high-dimensional plots are much less intuitive. To aid visualization of the structure of a dataset, the dimensionality must be reduced in some way. The simplest way to accomplish this task is by taking a random projection of data. Although this allows some degree of data structure visualization, the randomness of the choice discards too much information and the possible high variability inherent in the random process makes any intuition unreliable. To address this concern, a number of supervised and unsupervised 58 Berardino BARILE 1.8. UNSUPERVISED MANIFOLD LEARNING linear dimensionality reduction frameworks have been designed. However, in this dissertation, we will focus on three of the most used dimensionality reduction techniques such as Principal Component Analysis (PCA), Isometric Mapping (IsoMap) and t-distributed Stochastic Neighbor Embedding (t-SNE).

Principal Component Analysis

Principal Component Analysis (PCA) arguably represents one of the most used and studied unsupervised dimensionality reduction techniques used in the literature. PCA was first formulated in statistics by Pearson in 1901 who framed the analysis as "finding lines and planes of closest fit to systems of points in space" [START_REF] Pearson | On lines and planes of closest fit to systems of points in space[END_REF]. Nowadays, it represents the basic building block for multivariate data analysis. The central idea of PCA is to reduce the dimensionality of a dataset consisting of a large number of interrelated variables, while retaining, as much as possible, the variation present in the data. This is achieved by an orthogonal linear transformation (i.e. projection) that transforms the data to a new coordinate system such that the greatest variance by some scalar projection of the data comes to lie on the first coordinate, called Principal Components (PCs), the second greatest variance on the second coordinate, and so on. More formally, let's consider with X ∈ R (n,p) the matrix of data where n and p represent the number of instances and random variables, respectively. The transformation is defined by a set of vectors of weights or coefficients w k such that they map each row vector x (i) ∈ X to a new vector of principal component scores t k with i, k ∈ N as follows:

t k = x (i)k w k ∀i ∈ [1, 2...p] and k ∈ [1, 2, ..., l] (1.50)
where k represents the principal component and the individual variables t k considered over the dataset successively inherit the maximum possible variance from X.

In matrix notation, such formulation can be written as follows:

w (1) = argmax w =1 w T X T Xw = argmax w T X T Xw w T w (1.51)
The further k-th components can be intuitivelly found by subtracting the first k -1 principal components from X as follows:

Xk = X - k-1 s=1 Xw (s) w T (s) (1.52)
and then finding the weight vector which extracts the maximum variance from this new data matrix:

w (k) = argmax w T XT Xw w T w (1.53)
Notwithstanding, this procedure does not represent what it is actually performed by statistical softwares since it is numerically more convenient to optimize and make the weight vector orthogonal onto the PCs already found so far. Thus it is clear that, although PCA does not ignore covariances and correlations, it concentrates on variances. In fact, it can be shown that the principal components are eigenvectors of the covariance matrix. Thus, the principal components are often computed by eigen-decomposition of the data covariance matrix or singular value decomposition of the data matrix [START_REF] Jolliffe | Principal component analysis[END_REF]. Notwithstanding, one of the strong limitations of PCA is its linear assumption between correlated variables.

Isometric Mapping

Isometric Mapping (IsoMap) is a non-linear dimensionality reduction method based on the spectral theory which tries to preserve the geodesic distances in the lower dimensional space. It starts by creating a neighborhood network and then it uses graph distance to approximate geodesic distance between all pairs of points. The low dimensional embedding of the dataset is obtained through eigenvalue decomposition of the geodesic distance matrix. In other words, IsoMap is a technique that combines several different algorithms, enabling it to use a non-linear approximation to reduce dimensions while preserving local structures. The algorithm can be divided into four steps:

• Use a k-Nearest Neighbor (kNN) algorithm to find the k nearest neighbors of every data point, where k represents an arbitrary number of neighbors pre-specified by the user.

• Construct the neighborhood graph where points are connected to each other if they are each other's neighbors.

• Compute the shortest path between each pair of data points (i.e. nodes using graph terminology) using either the Floyd-Warshall or Dijkstra's algorithm.

• Given distances between each pair of points, project each point into a n-dimensional space, where n is an hyperparameter defined by the user, such that the original distance between points is preserved.

An intuitive representation of the IsoMap algorithm is proposed in Figure 1.16. It is possible to notice that when features are not linearly related, using Euclidean distance (like in the case of PCA) for dimensionality reduction provides unreasonable results. Conversely, the geodesic distance obtained from the IsoMap approach may provide more realistic representation of the data. In Figure 1.17 an intuitive representation of the two most important latent components (principal components), reported in the x-axis and y-axis respectively, Image taken from https://towardsdatascience.com/isomap-embedding-anawesome-approach-to-non-linear-dimensionality-reduction is depicted. In particular, the difference between the PCA and the IsoMap approach is highlighted with the latter showing a better manifold representation of the nonlinear relations between variables in high dimension.

t-distributed Stochastic Neighbor Embedding

The t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm is a statistical method for visualizing high-dimensional data based on the Stochastic Neighbor Embedding algorithm originally proposed by Sam Roweis and Geoffrey Hinton [START_REF] Hinton | Stochastic Neighbor Embedding[END_REF] and father extended by Laurens van der Maaten who proposed the t-distributed variant [START_REF] Van Der Maaten | Visualizing high-dimensional data using t-SNE[END_REF]. It represents a nonlinear dimensionality reduction technique which aims to model similar points by nearby observations and dissimilar points by distant observations with high probability. The t-SNE algorithm comprises two main stages. First, t-SNE constructs a probability distribution over pairs of high-dimensional objects in such a way that similar objects are assigned a higher probability while dissimilar points are assigned a lower probability. Second, t-SNE defines a similar probability distribution over the points in the low-dimensional map, and it minimizes the Kullback-Leibler divergence (KL divergence) between the two distributions with respect to the locations of the points in the map. Mathematically, given a set of data X ∈ R (n,p) , the t-SNE algorithm first computes the conditional probabilities as follows:

p j|i = exp ( x i -x j 2 /2σ 2 i ) k = exp (-x i -x k 2 /2σ 2 i ) (1.54)
where p i|i = 0 and j p j|i = 1 for all i. In other words, the similarity between x j and x i is the conditional probability, p j|i , that x j would pick x i as its neighbor if neighbors were picked in proportion to their probability density under a Gaussian centered at x i [START_REF] Van Der Maaten | Visualizing high-dimensional data using t-SNE[END_REF]. The parameter σ i represents the bandwidth of the Gaussian kernels and it is usually referred to as "perplexity" parameter. It is worth noticing that in the t-SNE procedures, the perplexity parameter dictates the shape of the mapping function. In the original paper, the author highlighted that as far as the perplexity parameter remains in the usual range [START_REF] Badeau | Multilinear singular value decomposition for structured tensors[END_REF][START_REF] Garyfallidis | Quickbundles, a method for tractography simplification[END_REF] the model is quite robust [START_REF] Van Der Maaten | Visualizing high-dimensional data using t-SNE[END_REF]. However, it is also important to notice that p j|i = p i|j . For this reason, the author proposes the following adjustment for the final joint probability:

p ij = p j|i + p i|j 2N (1.55)
so that p ij = p ji , p ii = 0 and i,j p ii = 1. Notwithstanding, the formulation of p ij is based on the Euclidean distance, which is heavily affected by the curse of dimensionality where in high dimensionality regime the distances loose the ability to discriminate, converging to a constant value. To adjust this behavior, a power transformation has been proposed

Figure 1.17: Comparison between the PCA and the IsoMap algorithm for data not linearly related. Image taken from https://towardsdatascience.com/isomap-embedding-anawesome-approach-to-non-linear-dimensionality-reduction such that the d-dimensional map of a random variable y ∈ R d reflects the similarities p ij as well as possible. To this end, a new measure q ij between two quantities in the map (i.e. manifold) is obtained as follows:

q ij = (1 + y i -y j 2 ) -1 k l =k (1 + y k -y l 2 ) -1 (1.56)
where q ii = 0. Eq. (1.56) represents an heavy-tailed Student t-distribution which is used to measure similarities between low-dimensional points in order to allow dissimilar objects to be modeled far apart in the map. In order to obtain the low dimensional representation of a generic point y i , the non-symmetric Kullback-Leibler (KL) divergence is considered and defined as follows:

KL (P Q) = i =j p ij log p ij q ij (1.57)
In order to obtain the mapping between p ij and q ij , Eq. (1.57) is minimized using gradient descent. The result of this optimization is a map that reflects the similarities between the high-dimensional inputs.

Tensor Factorization Models for Pattern Recognition

Blind Source Separation (BSS) represents a general method that aims to separate a set of source signals from a set of mixed signals, without the aid of information (or with very little information) about the source signals or the mixing process. It is most commonly applied in digital signal processing and involves the analysis of mixtures of signals where the objective is to recover the original component signals from a mixture signal. Notwithstanding, matrix and tensor factorization techniques also represent a particular application of BSS, in which the structural information hidden in the matrix (or tensor for data structure with more than two dimensions) is decomposed in k components, each representing a specific mode of the input. Thus, a factorization technique represents a general and widely used approach to discover hidden relations among different modes of high-dimensional data with applications in many research fields such as psychometrics, chemometrics, signal processing, numerical linear algebra, computer vision as well as neuroscience and graph analysis [START_REF] Stamile | Tensor based blind source separation in longitudinal magnetic resonance imaging analysis[END_REF]. The widespread use of multiview data have highlighted the limitations of standard "flatview" matrix models and the necessity to move toward more versatile data analysis tools. Higher-order tensors, also known as multiway arrays, enable such a fundamental paradigm shift toward models that are essentially polynomial, the uniqueness of which, unlike the matrix methods, is guaranteed under very mild and natural conditions [START_REF] Cichocki | Tensor decompositions for signal processing applications: From two-way to multiway component analysis[END_REF]. Benefiting Figure 1.18: Intuitive representation of a third-order tensor and its unfolding mode-1, mode-2 and mode-3. Image taken from [START_REF] Padhy | The power of tensor-based approaches in cardiac applications[END_REF] from the power of multilinear algebra as their mathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints which match data properties and extract more general latent components in the data than matrix-based methods. Notwithstanding, in this doctoral dissertation we limit our analysis to third-order tensors since such data structures are used in the list of contributions reported in Part II of this manuscript. Generally speaking, a third-order tensor can be decomposed in the product of three nonzero factors representing latent components (i.e., factor loadings) describing each mode of the tensor. A matrix X consists of row and column vectors. In a similar way, mode-n vectors or fibers are defined for tensors. A mode-n tensor fiber is the vector that is the result of fixing all indices except the n-th index. For example, the mode-1 fiber of a third-order tensor is analogue to a column vector and is denoted by x :jk . Similarly, the mode-2 and the mode-3 fibers are the row and the tube vectors (see Figure 1.18) and are denoted by x i:k and x ij: , respectively. A graphical illustration of a third order tensor and its mode-1, mode-2 and mode-3 matricizations (i.e. unfolding) of a third-order tensor is depicted in Figure 1.18.

Concept and Notation

If not clearly stated, we will use the notation described in [START_REF] Che | Randomized algorithms for the low multilinear rank approximations of tensors[END_REF]. For all contributions in this manuscript, we limit the analysis to third-order tensors of real positive values, which we denote with R + . We denote with lower case letters (e.g. x) scalar values, with uppercase letter (e.g. X) matrices and with Euler script letters (e.g. X ) tensors. In many applications, tensors are converted to matrices and vice versa. The transformation from a tensor to a matrix is referred to as tensor unfolding or flattening, a process where the elements of the tensor are reformatted in a lower-order structure [START_REF] Brett | Tensor decompositions and applications[END_REF]. In this dissertation, matrices with superscripts (e.g. U (n) ) define the mode-n factor matrix (loading) obtained from the factorization, while X (n) defines the unfolding of a tensor along its n-th mode. Practically speaking, for a tensor X ∈ R

(I 1 ×I 2 ×I 3 ) + of third-order, X (1) ∈ R + (I 1 ×I 2 * I 3 ) denotes the mode-1, X (2) ∈ R (I 2 ×I 1 * I 3 ) + denotes the mode-2 and X (3) ∈ R (I 3 ×I 1 * I 2 ) +
denotes the mode-3 unfolding. Also, we define with "⊗" the outer product operator which is usually defined as the tensor product of vectors. More precisely, the outer product of two coordinate vectors v ∈ R n and u ∈ R m can be mathematically defined as follows:

u ⊗ v = X where X ∈ R n×m (1.58)
Such a concept can be generalized considering two arbitrary matrices A ∈ R (m×n) and B ∈ R (p×q) where their outer product can be defined as follows:

A ⊗ B =           a 1,1 B a 1,2 B • • • a 1,n B a 2,1 a 2,2 B • • • a 2,n B . . . . . . . . . . . . a m,1 B a m,2 B • • • a m,n B           (1.59)
and it is known as the Kronecker product.

Finally, an important tensor-matrix multiplication is the mode-n product between a tensor X ∈ R (I 1 ×I 2 ×I 3 ) and a matrix A ∈ R (J×In) when I n should match one of the tensor dimensions [I 1 ,I 2 ,I 3 ] and it is defined with the symbol "• n " where "n" defines the unfolding mode. For example, a mode-1 tensor-matrix multiplication can be formally defined as follows:

(X • 1 A) i 2 ,i 3 = I 1 i 1 =i x i 1 ,i 2 ,i 3 a j,i 1 (1.60)
where i n ∈ I n defines the index position of the unfolded tensor X . Finally, the Frobenius-

1.9. TENSOR FACTORIZATION MODELS FOR PATTERN RECOGNITION norm of a generic third-order tensor X ∈ R

(I 1 ×I 2 ×I 3 ) +
is given by the following notation:

X F = X , X = I 1 i i =i I 2 i 2 =i
I 3 i 3 =i x 2 i 1 ,i 2 ,i 3 (1.61)
where X , X represents the inner product between two tensors. The rank of a tensor, defined as R = Rank(X ), can be thought of as the smallest number of rank-1 terms that can be used to approximate the original tensor as their sum. Additionally, for multilinear rank, we define the mode-n rank of a tensor X the dimension of the subspace spanned by its mode-n vectors, given by the number of right singular vectors of X (n) associated with its non-zero singular values [START_REF] Lathauwer | Decompositions of a higher-order tensor in block terms -part iii: Alternating least squares algorithms[END_REF]. More precisely, we refer to

multilinear rank L = [R 1 , R 2 , R 3 ] ∈ N 3 , where possibly R 1 = R 2 = R 3 ,

Canonical Polyadic Decomposition

A Canonical polyadic decomposition (CPD or CANDECOMP), also known as Parallel Factorization (PARAFAC), represents the most widely used and studied factorization approach for tensor-based analysis. The CPD approach approximates a tensor with a sum of R rank-1 tensors, where the term canonical implies that the number of rank-1 terms R is minimal. Mathematically, given a general tensor X ∈ R (z×q×p) , the CP decomposition can be written as follows:

X ≈ R r=1 u (1) r ⊗ u (2) r ⊗ u (3) r (1.62)
where u (1)

r ∈ R z , u (2) r ∈ R q and u (3) r ∈ R p ∀ 1 ≤ r ≤ R represent the component
vectors describing the three modes of the original tensor. An intuitive representation of the CPD model is depicted in Figure 1. [START_REF] Calabrese | Grey matter lesions in MS: from histology to clinical implications[END_REF]. A CPD factorization model of a third-order tensor is unique under mild conditions, which makes it one of the most used and well studied factorization models in the literature [START_REF] Padhy | The power of tensor-based approaches in cardiac applications[END_REF].

Considering a third-order tensor, the CPD factorization can be obtained by formulating the problem as a system of equations expressed in the following form:

Berardino BARILE X (1) = (U (1) • U (2) )U (3)T X (2) = (U (1) • U (3) )U (2)T X (3) = (U (2) • U (3) )U (1)T
Multiple types of algorithms have been proposed in order to minimize the following loss: minimize X X -X 2 F ≈ minimize U (1) ,U (2) ,U (3) X -U (1) , U (2) , U (3) 2

F
where . defines the factorization operation. One of the most popular and easy approach is known as Alternating Least Squares (ALS). It implies the recursive optimization of each of the components as defined in the following equations:

U (1) ← argmin U (1) ∈R + || X (1) -(U (1) • U (2) )U (3)T || U (2) ← argmin U (2) ∈R + || X (2) -(U (1) • U (3) )U (2)T || U (3) ← argmin U (3) ∈R + || X (3) -(U (2) • U (3) )U (1)T ||
In other words, ALS updates one factor matrix at a time, keeping the estimates of the other factor matrices fixed, and cycles over such conditional updates. This conditional update has the form of a classical linear Least Squares problem, i.e., the new factor matrices estimated are obtained by solving a set of linear equations in the Least Square sense. Notwithstanding, although very simple to implement practically, the ALS algorithm might take several steps to converge and it might also not converge to a global optimum since the final result strongly depends on the quality of the random initialization [START_REF] Rabanser | Introduction to tensor decompositions and their applications in machine learning[END_REF]. Numerically better algorithms have been presented in the literature [START_REF] Sorber | Optimization-based algorithms for tensor decompositions: Canonical polyadic decomposition, decomposition in rank-$(l_r,l_r,1)$ terms, and a new generalization[END_REF] and are included in the Tensorlab framework [START_REF] Hendrikx | Tensorlab. Online version[END_REF].

Notwithstanding, for most practical applications, factorization with rank-1 terms, performed by CPD, could be too restrictive for some applications as it does not model all variability in the data [START_REF] Preston | Electromyography and neuromuscular disorders clinical-electrophysiologic correlations. boston oxford johannesburg melbourne new delhi singapore; basic nerve conduction study[END_REF].

Multilinear Singular Value Decomposition

A more flexible approach to model the variability of the data is obtained by generalizing the Singular Value Decomposition (SVD) approach [START_REF] Lathauwer | Blind source separation by higher-order singular decomposition[END_REF], usually applied to data The general method is based on SVD obtained from a higher-order tensor and it is known in the literature as MLSVD or Higher-Order Singular Value Decomposition (HOSVD) [START_REF] Lathauwer | A multilinear singular value decomposition[END_REF]. Mathematically, we can define the MLSVD decomposition as follows:

X ≈ S • 1 U (1) • 2 U (2) • 3 U (3) (1.63)
where U (1) r ∈ R (z×i) , U (2) r ∈ R (q×j) and U (3) r ∈ R (p×s) respectively form orthonormal bases for the first, second and third mode of the tensor X . In other words, matrices U (1) , U (2) , U (3) respectively represent the left singular vectors of the SVD of the mode-1, mode-2 and mode-3 unfolding of the original tensor. The core tensor S ∈ R (i×j×s) explains the interaction between the different modes. The structure of the core tensor has all-orthogonality properties defined in all directions and their multilinear singular values, defined as the Frobenius norm of the consecutive slices, are ordered in decreasing order. This aforementioned property means that all slices of the core tensor S are mutually orthogonal to each other for each possible mode. Moreover, this structured representation ensures that variation in each mode is captured independently. For a more in-depth description of the model, we refer to Lathauwer et. al [START_REF] Lathauwer | A multilinear singular value decomposition[END_REF]. It is worth noticing that the MLSVD structure allows for a higher level of flexibility with respect to the CPD model. For CPD, in fact, factor loadings are represented as vectors u (n) and their outer products form a rank-1 tensor with multilinear rank (1, 1, 1). The number of all the rank-1 terms, used to approximate the original tensor X , represents the tensor rank R. For the MLSVD, instead, each mode-n unfolding matrix X (n) has a specific and possibly different low-rank R n . It is also worth emphasizing that when the mode ranks of the core tensor are smaller than those of the starting tensor, the resulting decomposition is known as truncated HOSVD.

Non-Negative Tucker Decomposition

A similar approach to the MLSVD model was proposed in [START_REF] Zhou | Efficient nonnegative tucker decompositions: Algorithms and uniqueness[END_REF] where the Tucker model [START_REF] Tucker | Some mathematical notes on three-mode factor analysis[END_REF] was used and non-negativity constraints on the core tensor and mode matrices were imposed, from which the name Non-Negative Tucker Decomposition (NTD) was derived. In other words, the same factorization structure proposed in Eq. (1.63) is employed. However, a major difference with respect to MLSVD factorization model is the fact that orthogonality constraints are not required in the general Tucker decomposition [START_REF] Badeau | Multilinear singular value decomposition for structured tensors[END_REF]. This implies that the core tensor does not have the all-orthogonality property and the low-rank matrices U (n) do not have orthonormality properties. In fact, the factor matrices and the core tensor are obtained by means of a multiplicative updating algorithm with non-negativity constraints imposed during optimization [START_REF] Boutsidis | Svd based initialization: A head start for nonnegative matrix factorization[END_REF]. Practically speaking, the multiplicative updating algorithms iteratively "matricize" tensor along each mode and then solve a Non-Negative Matrix factorization (NMF) problem. It should be noticed that the non-negativity condition, imposed on the factor matrices, represents a reasonable constraint when data pertains to R + , which means that the data can be described using only additive components. Additionally, compared with unconstrained Tucker decompositions, NTD is more likely to be unique and provides physically meaningful components [START_REF] Zhou | Efficient nonnegative tucker decompositions: Algorithms and uniqueness[END_REF].

Low Multilinear Rank Approximation

Among the general factorization model applications, the MLSVD decomposition can usually be thought of as a good starting point for constructing a tensor of the type shown in Figure 1.20. The LMLRA factorization model represents one of such cases. The difference with respect to the MLSVD approach lies in the fact that an additional optimization method is usually implemented. More precisely, for given three positive integers µ 1 , µ 2 , µ 3 , the low multilinear rank approximation of third-order tensor X ∈ R (I 1 ,I 2 ,I 3 ) can be obtained by rewriting the Eq. (1.63) as the optimization problem with respect to the Frobenius norm by considering the following formulation: 1) ,U (2) ,U (3) X

minimize Rank( X )≤R X -X 2 F = minimize S,U (
-S • 1 U (1) • 2 U (2) • 3 U (3) 2 F (1.64)
where S ∈ R (µ 1 ×µ 2 ×µ 3 ) and U (n) ∈ R In,µn with n ∈ [1, 2, 3] is orthonormal. If we denote with U (n) * the solution of the above optimization problem, then the tensor S • 1 P (1) • 2 P (2) • 3 P (3) where

P (n) = U (n) * U (n) * T
is the low multilinear rank approximation of X [START_REF] Maolin | Randomized algorithms for the low multilinear rank approximations of tensors[END_REF]. The MLSVD factorization approach does not provide, in general, the best approximation of the original tensor X in terms of Frobenius norm [START_REF] Lathauwer | A multilinear singular value decomposition[END_REF], justifying the use of additional optimization algorithms. Several algorithms have been developed in the literature for tensor optimization. Usually, nonlinear unconstrained optimization and Nonlinear Least Squares (NLS) optimization methods are used for the estimation of parameters in the core tensor and factor matrices due to its robust performance and high level of parallelization [START_REF] Lathauwer | Between linear and nonlinear: Numerical computation of tensor decompositions[END_REF].

Conclusion

Machine learning represents a broad mathematical field describing the ability of an algorithm to solve a specific task by "learning" from data without being explicitly programmed. In this chapter, we highlighted the learning paradigm at the core of every machine learning system, discussing its categorization in supervised and unsupervised setting based on the availability of the label guiding the learning process. Notwithstanding, a notable phenomenon that characterizes these algorithms is that a minimum training error does not necessarily guarantee good generalization performances, giving rise to the concept of bias-variance tradeoff. To mitigate the problem, multiple evaluation strategies were proposed in the literature depending on the particular condition in which the learner has to trade. Also, regularization penalties as well as advanced statistical concepts, like boosting and bagging, represent state of the art solutions for effectively dealing with the bias-variance problem, providing solutions to the generalization on unseen data.

Berardino BARILE

Magnetic Resonance Imaging

Nuclear Magnetic Resonance

Magnetic Resonance Imaging (MRI) derives from the Nuclear Magnetic Resonance (NMR) imaging. The basic idea of NMR is that certain nuclei will resonate and emit radio signal if placed in a strong magnetic field and pulsed with a certain radio-frequency energy. In order to clarify this concept, we will proceed with our explanation starting from the principal subject of NMR: the atomic nuclei. Indeed, by studying the global effects of all the atomic nuclei composing the matter, it is possible to have indirect information about the matter itself. The atomic nuclei that make up matter can be in two different spin states: they can have zero spin, which is the case for all atoms composed of an even number of protons and neutrons (i.e. 12 C, 16 O and others), or a non-zero spin (integer or half-integer).

Atomic nuclei with a non-zero spin have a kinetic moment s dependent from the angular momentum:

s = s(s + 1) * , ( 2.1)
where s is the bipolar angular momentum, s the spin, and the Plank normalization constant ( = h 2π ; h = 6, 626.10 -34 m 2 .kg.s -1 ). This kinetic moment leads to the appearance of a magnetic moment of spin µ :

µ = γ s (2.2)
where γ represents the gyromagnetic ratio of the element under consideration (γ = 42, 58 M Hz.T -1 for the hydrogen nucleus). Placed in an external magnetic field, these magnetic moments interact and align with the direction of the magnetic field. This alignment of the magnetic moments with the external field ( B 0 ) has the effect of creating a magnetization M such that:

M = Σ µ dV (2.3)
where µ represents the magnetic moment in the magnetic field B 0 . The frequency of precession is governed by the Larmor equation, defined as:

ω = -γ B 0 (2.4)
where γ is the magnetogyric ratio and every nucleus has its own specific value.

74 When stimulated by a radio frequency wave, at a frequency very close to that of Larmor, it is possible to disrupt the magnetization created by the field B 0 . This disturbance causes the nuclei to move from their fundamental energy state to a higher level, a phenomenon called resonance. Spontaneously, the nuclei return to their fundamental energy state by the emission of a radio frequency wave which will be the NMR signal. This phenomenon corresponds to the rotating magnetisation decays due to relaxation which can be subdivided into longitudinal or T1 recovery and transverse or T2 decay.

Berardino BARILE 2.1. MAGNETIC RESONANCE IMAGING M z (t)=M z (0) 1 -e -t T 1 M z (0) M z 0 t (a) Longitudinal
Longitudinal Recovery (T1) : (Fig. 2.1-a) describes the re-growth of the magnetization component in the z direction. It is a relaxation time constant which is an intrinsic property of each tissue. After 90 • pulse, when all the z-components are tipped into the transverse plane (M), T1 is the number of milliseconds it takes to grow to the 63% of the original orientation M z . The relationship is described by the following equation:

M z (t) = M z (0) 1 -e -t T 1 (2.5)
Transverse Decay (T2) : (Fig. 2.1-b) describes the decay of the signal in the xy plane. It occurs due to the interactions between spins as energy is released followed by an RF pulse. T2 decay is the number of milliseconds to reach the 37% of the magnetization in the xy (M x y) plane. It is described by the equation:

M xy = M xy (0)e -t T 2 (2.6)

Conventional MRI Sequences

The clinical interest of MRI exponentially increased since the first study [START_REF] Damadian | Tumor detection by nuclear magnetic resonance[END_REF] of in vivo T1 and T2, where the author demonstrated the differences between cancerous and Berardino BARILE normal tissue. Indeed, thanks to this noninvasive technique, in vivo investigation of human structures, difficult to analyze, was finally possible. All MRI system models have the same configuration. A coil with a direct current flowing through, represented by a magnet that creates the permanent magnetic field B 0 . This field must have the characteristics of being constant and homogeneous. The fact that the patient must be placed inside the coil imposes a minimum diameter which makes it difficult to homogenize the field. Additionally, the more intense the field, the greater the heating phenomena. The solution was therefore to exploit the superconductivity properties of certain materials such as NbTi at low temperatures. Thanks to the use of these superconducting materials, it is now possible to create fields of several Teslas without generating heat.

Nowadays, thanks to the increase in the permanent magnetic field, a gain in the intensity and homogeneity of the field was obtained. In order to refine this homogenization, the so-called "shims" are used; there are two types: passive shims, which are ferromagnetic wedges allowing "rough" homogenization according to the MRI environment, and active shims, which are coils that are usually also superconducting, allowing correction of these field inhomogeneities induced by the presence of the patient.

The basic MRI techniques to obtain brain images are called conventional MRI (cMRI) sequences. With this name, we usually refer to a well-defined set of standard MRI acquisition techniques that allow to obtain rather simple, yet informative, anatomical in-vivo images of the brain, or, in general, human body. Since the beginning of MRI, two main types of sequences were used: spin-echo and gradient-echo sequences. Historically, the first to appear were spin-echo sequences [START_REF] Carr | Effects of diffusion on free precession in nuclear magnetic resonance experiments[END_REF][START_REF] Hahn | Spin echoes[END_REF], characterized by the succession of two radiofrequency (RF) pulses, the first of 90 • creates the resonance phenomenon and then, at half the echo time (TE), a re-phasing pulse of 180 • compensates for the spin shifts due to field inhomogeneities at the end of each repetition time (TR). Figure 2.2 represents the timeline of a classical spin-echo sequence. By tuning specific acquisition parameters, it is possible to excite particular types of nuclei obtaining different types of information. In this sequence, two main parameters can be tuned: echo-time (TE) and the TR. In particular, the TR is the time between two excitations, the longer it is, the more time the longitudinal magnetization has had to grow back. The TE is the time between the excitation and the reading of the signal. Thus, the shorter the TE, the more contrast there is between tissues. The resulting image is called T1-weighted. Conversely, if the TR is longer, the longitudinal magnetizations of the different tissues will be repelled, eliminating the T 1 contrast. Interestingly, if the TE is long enough, the tissues with different T2 will have a higher contrast, which means that the image will be T2-weighted. Finally, with a long TR and a short TE, the image will be proton density (PD) weighted.

In summary, spin-echo sequences will be less sensitive to field inhomogeneities which implies a higher TE than gradient echo. As the 90 • pulse excitation reduces the longitudinal magnetization, the TR will become higher than gradient echo, resulting in a longer acquisition time. Gradient echo sequences are sensitive to field inhomogeneities and magnetic susceptibility effects, but requiring only a single selective pulse of less than 90 • will allow the use of much lower TRs than those used in spin-echo, and the acquisition time will be significantly reduced. The respective advantages and disadvantages of the two techniques have given rise to new techniques, derived either from spin-echo, gradient echo or a combination of the two techniques.

T1-weighted Imaging

In the T1-weighted (T1w) image, the TR value is chosen to be less than the T1 time (usually 500 ms) and the TE value is chosen to be less than T2 (usually 30 ms). Most lesions have a prolonged T1 time and they are dark in T1ws images; hence, tumors or infarctions could be missed [START_REF] Hendee | Magnetic resonance imaging part ii -clinical applications[END_REF]. An interesting property of T1w is related to its sensibility to obtain the best contrast for paramagnetic contrast agents (e.g. gadolinium-containing compounds). This property is extremely important especially in clinical setting where contrast agents are essential to perform a correct diagnosis, particularly in brain-related pathologies.

T2-weighted Imaging

Like the T1w imaging sequence, the T2-weighted (T2w) sequence is obtained tuning the values of the TR and TE acquisition parameters. Specifically, the TR value is chosen to be greater than T1 (usually 2000 ms) and the TE values are chosen to be less than T2 (usually 100 ms). In principle, the T2w images provide better contrast between pathological tissue and normal tissue, and the T1w images provide better anatomical details. In T2w imaging the dominant signals come from the WM tissue with high signal intensity and GM tissue with intermediate signal intensity.

FLAIR Imaging

The FLAIR sequence is similar to a T2-weighted image except that the TE and TR times are very long (greater than 9000 ms and 114 ms, respectively). By doing so, abnormalities remain bright but normal cerebrospinal fluid (CSF) is attenuated and made dark. This sequence makes the differentiation between CSF and an abnormality much easier and for this reason, T2-FLAIR images are useful to help the diagnosis of several neurodegenerative pathologies.

A visual comparison of the aforementioned three MRI modalities is proposed in

Diffusion MRI 2.2.1 Principles and Biological Interpretation

The principle in which diffusion MRI (dMRI) can be described is by relying on the concept of Brownian motion [START_REF] Brown | A brief account of microscopical observations made in the months of june[END_REF], in which the key idea relates to the concept that water is always moving. In a perfectly homogeneous environment, water moves with equal probability in all directions. However, the human body has complex diffusion properties and the relative proportion of the water distribution between intra and extracellular compartments is affected by the pathologic processes. At a fixed temperature, the rate of diffusion was described by Einstein, in 1905, by the following equation [START_REF] Einstein | Investigations on the theory of the brownian movement[END_REF]:

r 2 = 6Dt (2.7)
where r 2 is the mean square displacement of the molecules, t is the diffusion time and D is a constant value defined as follows:

D = k B T 6πηR (2.8)
where k B is the Boltzmann constant, T is the temperature of the medium, η is the dynamic viscosity of the medium, and R is the radius of the spherical particle. In our case, we are interested in studying in vivo brain structure in humans, since the primary aim of its use refers to prevention of possible worst case scenario. Diffusion weighted imaging (DWI) provides qualitative and quantitative information about the diffusion properties. It allows the mapping of the diffusion process of molecules in biological tissues, in vivo and non-invasively. A special kind of dMRI, such as diffusion tensor imaging (DTI), has been used extensively to map WM tractography in the brain. In Figure 2.4 the diffusion of water molecules in three different intracellular tissues is proposed. It is possible to observe that the diffusitivity of water molecules is directly dependent from the number and types of axons and is compromised by the number of myelin sheaths, number of myelinated axons, and length of myelin sheaths along axons. This implies that an increased anisotropy can be observed as myelination progresses. In other words, the movement of the water molecules in the tissue, is significantly modified by the structure of the tissue itself. Indeed, in structured tissue without myelin, the diffusion is not subject to strong physical constraints compared to the diffusion in tissue with partial or total myelin presence. This explanation represents a clear case in which brain lesions, like the one found in MS, can be indirectly visible when we study diffusion in human brain. Hence, analysis of diffusion in human brains is an important tool to extract useful information describing the structure and physiological processes of the brain tissue.

Acquisition Sequence

In order to obtain in vivo images of the brain showing the diffusivity in tissue, it is important to have MRI sequences capable to acquire that information. Modern diffusionweighted sequences all trace their origin to the Pulsed Gradient Spin-Echo (PGSE) technique developed by Edward Stejskal and John Tanner in the mid-1960's [START_REF] Mukherjee | Diffusion tensor MR imaging and fiber tractography: theoretic underpinnings[END_REF][START_REF] Thomsen | In vivo measurement of water self diffusion in the human brain by magnetic resonance imaging[END_REF]. This type of sequence is capable to acquire diffusion information from human tissue. The diffusion-weighted pulse sequence consists of the addition of a pair of diffusion gradients. Those gradients can be oriented in specific directions in order to measure the diffusivity. Specifically, the diffusion-weighted pulse sequence is composed of the additional diffusion-sensitizing gradients (DG's), applied on either side of the 180 • pulse. Afterwards, following the second DG, an image acquisition module is placed. This is typically an echo-planar sequence using rapidly oscillating phase and frequency gradients that generate multiple gradient echoes. Interestingly, these diffusion gradients can be oriented in specific directions in order to measure the diffusivity. Gradients are created by combining the directions in the 3-dimensional space (Figure 2.5). From the image, it is possible to observe that other parameters can be used in order to measure diffusion in a specific direction, which represents the duration of each gradient (δ) and the amplitude (G) of the gradient. These gradients should be intense and as short as possible. The intensity of these gradients is usually measured by the parameter b, which is expressed as:

b = γ 2 G 2 δ 2 ∆ - δ 3 (2.9) Berardino BARILE
where γ is the gyromagnetic ratio, δ is the duration of the scattering gradients and ∆ is the time between the two gradients. Its unity is seconds per square millimetres.

Typical values of b used in clinical applications range from 400 to 1500. The presence of these gradients alone allows the image to become diffusion weighted. Indeed, the first one dephases the water molecules, meaning the 180°pulse reverses the phase. Then, the second gradient, identical to the first, rephases the water molecules. The molecules that remain immobile in this time interval will have a zero phase after the second gradient and will therefore emit an unchanged signal, whereas the molecules that do modifies will result not to be completely rephased and will therefore present a weaker signal. This fact is known as signal attenuation. This attenuation will depend on the diffusion coefficient and the intensity of the gradients being used. More precisely, molecular motion results in loss of signal intensity due to incomplete rephasing of water proton spins, which change position between and during the applications of the 2 diffusion-sensitizing gradients [START_REF] Mukherjee | Diffusion tensor MR imaging and fiber tractography: theoretic underpinnings[END_REF]. This diffusion-weighted contrast can be fit to an exponential model:

S(b) = S 0 e -bD ln(S(b)) = ln(S 0 ) -bD (2.10)
where S 0 is the signal strength at the origin, S(b) the signal strength at the echo time and D the diffusion coefficient. This coefficient, also called Apparent Diffusion Coefficient (ADC), has remained for years the reference diffusion parameter. According to Eq. 2.11, it is possible to obtain the value for the ADC in each voxel. The equation can be expressed as follows:

ADC = log S S 0 b (2.11)
ADC value is a quantitative parameter used to study and measure diffusion changes due to the presence of various pathologies related to the brain [START_REF] Balashov | Acute demyelinating lesions with restricted diffusion in multiple sclerosis[END_REF]

Diffusion Tensor Imaging (DTI)

Diffusion data represent a powerful tool to obtain a large range of interesting information by simply studying the water diffusion in brain structures. Notwithstanding, in order to obtain qualitative results, diffusion MRI techniques usually require the acquisition of several images measuring diffusion in different directions in space, except for the measurement of ADC. These images are essential to correctly estimate the diffusion of water molecules in the 3-dimensional component of space. While the ADC allows an estimate of the mobility of water molecules in the tissue, which is crucial information for the characterization of cytotoxic edema occurring in ischaemic stroke, it does not capture information about the micro-architecture of the tissue. In particular, the anisotropy of water molecule diffusion in each voxel cannot be estimated using this technique. In order to extract information such as the anisotropy strength or the main directions of diffusion in each voxel, it is necessary to pre-process the images. To do this, several techniques have been proposed, although the most widely used is the diffusion tensor technique [START_REF] Basser | Diagonal and off-diagonal components of the self-diffusion tensor: their relation to and estimation from the nmr spin-echo signal[END_REF][START_REF] Basser | Estimation of the effective selfdiffusion tensor from the nmr spin echo[END_REF]. The model is rather simple, yet powerful and allows to obtain quantitative diffusion properties in the brain. Technically speaking, this technique is based on Bloch's equations [START_REF] Bloch | Nuclear induction[END_REF] and it calculates the covariance matrix of a three-dimensional Gaussian distribution that models the displacements of the molecules. By analogy with diffusion, Fick's first law can be rewritten in a tensorial manner to introduce the diffusion tensor:

J i = -D ij ∂C ∂ j , (2.12)
and the measurement of the NMR signal (S) can be related to the diffusion tensor :

ln S(b) S(b = 0) = - 3 i=1 3 j=1 b ij D ij , (2.13)
where S(b) and S(b = 0) are the diffusion-weighted and non-diffusion-weighted NMR signals, respectively, and b ij is defined as:

b i,j = γ 2 G i G j δ 2 ∆ - δ 3 ∀i, j = x, y, z (2.14)
which represents an extension of Eq. (2.14) applied to the 3-dimensional case, while the D ij is the diffusion tensor value. More precisely, let's consider with D ∈ R (3×3) the symmetric matrix representing the diffusion tensor can be represented as follows:

D ij =      D xx D xy D xz D yx D yy D yz D zx D zy D zz      ; b ij =      b xx b xy b xz b yx b yy b yz b zx b zy b zz      .
(2.15)

Due to the symmetry of the diffusion matrix, we can consider diagonalization in order to calculate the corresponding eigenvalues and the eigenvectors. The matrix D can be written as:

D ij =      D xx D xy D xz D xy D yy D yz D xz D yz D zz      =      λ 1 0 0 0 λ 2 0 0 0 λ 3           -→ ǫ 1 -→ ǫ 2 -→ ǫ 3      (2.16)
where λ 1 , λ 2 , λ 3 represent the eigenvalues and -→ ǫ 1 , -→ ǫ 2 , -→ ǫ 3 represent the corresponding eigenvectors. From these values, it is possible to easily obtain the information about the Berardino BARILE shape of the diffusion tensor. To clarify the concept, Figure 2.6 is shown and we can clearly observe the roles of eigenvalues and eigenvectors in the estimation of the ellipsoid describing the major directions of water diffusion. The relative importance of the eigenvalues allows to characterize the type of diffusion. Indeed, if one of them is much larger than the other two, the scattering will mainly take place in the direction of the eigenvector associated with it. If two eigenvalues are of the same order of magnitude and the third one is much smaller, than the scattering can take place in the plane defined by the two previous eigenvectors. Finally, if the three eigenvalues are of the same order, then no direction is favored and the diffusion is said to be isotropic. The obtained eigenvalues and eigenvectors are really important to understand diffusion properties of the diffusion tensor since they allow to calculate important metrics which summarize the characterization of the diffusitivity for a specific brain structure. For instance, one of the most famous and used metrics relates to the "axial diffusivity". It is the main eigenvalue (λ 1 ) and represents the part of the diffusion in a voxel who follows the principal diffusion direction. The "radial diffusivity" λ r = λ 2 +λ 3 2 who represents the average of the second and third eigenvalues, represents the part of the diffusion in the voxel that takes place in the normal plane associated with the vector -→ ǫ 1 and identified by the eigenvectors -→ ǫ 2 and -→ ǫ 3 . Another important metric is the "Mean Diffusitivity"

x y z λ 1 λ 2 λ 3 ε ! ε " ε #
M D = 3 1 λ i 3
which takes into account the three principal directions of the diffusion matrix. For a complete list of all derived metrics we refer to [START_REF] Kingsley | Introduction to diffusion tensor imaging mathematics: Part ii. anisotropy, diffusion-weighting factors, and gradient encoding schemes[END_REF].

Use of DTI

By estimating tissue micro-architecture, diffusion MRI is a technique that can be used to study many pathologies. This is particularly important to detect pathologies such as liver disease since it allows to determine the amount of fibrosis present in the parenchyma, to detect the presence of cancerous clusters and establish or not the diagnosis of hepatic cirrhosis [START_REF] Lee | Assessment of diffusion tensor mr imaging (dti) in liver fibrosis with minimal confounding effect of hepatic steatosis[END_REF]. Diffusion MRI has also shown its value in breast pathologies and in particular in breast cancer where it can detect malignant lesions [START_REF] Eyal | Parametric diffusion tensor imaging of the breast[END_REF]. Additionally, diffusion MRI has shown a high sensitivity to the pathological processes of demyelination, inflammation and edema formation. Indeed, in the brain, the water diffusion is constrained by axons, surrounded by a sheath of myelin. Diffusion will therefore mainly take place along the myelinated axons. An alteration of this myelin sheath leads to a reduction in axial diffusion and an increase in radial diffusion, and consequently, a decrease in the anisotropic fraction. Both inflammation and cytotoxic edema formation increase the water concentration and thus the diffusivity in the tissue.

Fiber Tracking

The word tractography refers to any method for estimating the trajectories of the fiber tracts in the WM [START_REF] Mori | Three-dimensional tracking of axonal projections in the brain by magnetic resonance imaging[END_REF]. Among the numerous methods proposed for tractography, streamline tractography is one of the most widely used. This method outputs discrete curves or trajectories (tracts) by successively stepping in the direction of the principal eigenvector (the direction of the fastest diffusion) [START_REF] O'donnell | An introduction to diffusion tensor image analysis[END_REF]. In Figure 2.7 an intuitive illustration of this concept is provided. In other words, the principal eigenvalue of the diffusion tensor model gives, in a particular voxel, the main direction of water diffusion. This direction will point to one of its contiguous voxels. The voxel reached by the diffusion direction can be used to find the direction pointing to the next contiguous voxel. This process can be repeated for a certain number of voxels since a termination criterion is met. The first set of voxels, also called "seeds", used to start this iteration chain, are usually selected in two different ways according to the type of tractography. For global brain tractography, the seed voxels are randomly selected from the whole WM. For the investigation of a specific WM tract, the seeds are selected by the user according to a specific atlas. According to this simple concept, it is then possible to connect contiguous voxels in order to obtain all the fibers representing the WM structure.

The aforementioned algorithm pertains to a particular family of algorithms called deterministic [START_REF] Hagmann | Mapping human whole-brain structural networks with diffusion mri[END_REF]. The main advantage of these kind of algorithms is the fact that they are very fast and allow to obtain quite good results in terms of accuracy in WM fiber reconstruction. Conversely, a limiting factor of these types of algorithms is the accuracy of Red lines are reconstructed trajectories. Image taken from [START_REF] Mukherjee | Diffusion Tensor MR Imaging and Fiber Tractography: Theoretic Underpinnings[END_REF] the path followed by the fibers. Indeed, these algorithms just follow one of the principal directions without taking into account other fibers that could give better information. In order to overcome this limitation, a new family of probabilistic algorithms was developed ( [START_REF] Behrens | Non-invasive mapping of connections between human thalamus and cortex using diffusion imaging[END_REF]). Probabilistic algorithms repeat the deterministic version many times by randomly perturbing the main fiber directions each time, producing a probabilistic connectivity matrix. Such maps indicate the probability that a given voxel is connected to a reference position [START_REF] Fillard | Quantitative evaluation of 10 tractography algorithms on a realistic diffusion mr phantom[END_REF], which allows a better representation of crossing fibers.

Brain Connectivity

We have known since the nineteenth century that the neuronal elements of the brain constitute a formidably complicated structural network. Modern non-invasive imaging techniques applied to the human brain, allow to map its intricate networks of anatomical regions and neural pathways at near-millimeter resolution. The resulting large-scale networks provide a comprehensive description of the brain structural connectivity, also called connectome [245]. Furthermore, its importance comes from the realization that the structural properties and functionality of the brain are intrinsically connected [START_REF] Sporns | Structure and function of complex brain networks[END_REF]. These types of brain networks provide different information, reflecting complementary aspects of the brain circuit and organization. DTI structural connectivity, in particular, describes anatomical connections, linking a set of neural elements. At the scale of the human brain, these connections generally refer to trajectories of White Matter (WM) pathways extracted from DTI. Conversely, functional connectivity (derived from functional MRI) is generally created from time series observations, and describes patterns of statistical dependence among neural elements. Nevertheless, brain connectivity can also be obtained from conventional MRI by measuring different morphological metrics of the GM on T1weighted images [START_REF] Raamana | graynet: single-subject morphometric networks for neuroscience connectivity applications[END_REF]. In such graphs, nodes represent GM areas obtained from the GM tissue parcellation, while edges represent degree of (dis-)similarities between nodes using morphological features extracted from GM tissue, such as thickness, curvature and area [START_REF] Macdonald | Automated 3-d extraction of inner and outer surfaces of cerebral cortex from mri[END_REF]. Within the general framework of graph theory, a brain connectivity is a network, comprising a set of nodes, corresponding to segmented cortical regions, also called Regions of Interests (ROIs), and edges, representing connections between each pair of ROIs. Structural brain connectivity can be processed into network form by means of a complex task, involving several steps and computational resources. Nodes are generally derived by parcellating cortical and sub-cortical GM regions, usually by defining a random parcellation into evenly spaced and sized voxel clusters. Once nodes are defined, their structural couplings can be estimated, and the full set of all pairwise couplings can then be aggregated into a connection matrix. The resulting network can be examined with tools and methods coming from graph theory. These offers a large set of experimental methodology for detecting, analyzing, and visualizing network architectures. A major promise of human connectomics is that it will lead to a deeper understanding of the biological substrates underlying brain and mental disorders [START_REF] Bassett | Human brain networks in health and disease[END_REF]. Notwithstanding, the primary aim is to map patterns of structural and morphological brain connectivity, uncovering emerging patterns of brain dynamics.

With this vision, the structure and functionality of the human brain are intrinsically related, and can be used to uncover potential brain and mental disorders, as well as brain injury and recovery [START_REF] Sporns | Structure and function of complex brain networks[END_REF]. To better understand the pipeline for graph generation, an intuitive representation of the main steps followed for structural and functional brain network generation is provided in Figure 2.8. For a more in-depth understanding of the types of brain connectivity that can be obtained from MRI data of different modalities, Figure 2.9 reports the combinations of three different types of connectomes generated from the analysis of structural, functional and morphological characteristics between nodes (i.e. brain regions). As previously anticipated, three techniques are commonly used to estimate the connections between nodes, allowing the study of these different types of connectivity. To study functional connectivity, links between nodes are estimated from the temporal correlations that exist between the BOLD signal of the GM regions (i.e. ROIs). In the case of anatomical connectivity, connections are obtained from correlations between the morphological features extracted for each node, reflecting the size, shape, arrangement and Berardino BARILE Figure 2.8: Structural and functional brain networks created through three main steps: (i) definition of network nodes (ii) estimation of a continuous measure of association between nodes (iii) generation of an association matrix by compiling pairwise associations between nodes. Image taken from [START_REF] Bullmore | Complex brain networks: graph theoretical analysis of structural and functional systems[END_REF] Figure 2.9: Three different types of connectomes generated from the analysis of structural, morphological and functional characteristics between nodes (i.e. brain regions). Morphological connectivity, is obtained connections are estimated from correlations between morphological characteristics of the grey matter tissue. Functional connectivity, is obtained from white matter connections, derived from temporal correlations of the BOLD signal. Finally, structural connectivity, are obtained from the connections estimated from the fibres generated by the tractography. density of cells in these regions. In the study of brain structural connectivity, the connections represent the number of fibres that connect them, estimated using the tractography technique.

These three types of brain connectivity provide different information, reflecting complementary aspects of brain network organization. It might also be interesting to study the relationship between these different connectivities. However, in this thesis we will focus on the pathological changes in WM (normal-appearing WM and lesion) as well as GM tissue that are caused by the MS disease. We will therefore focus our analyses on the structural and morphological connectivity of the brain, obtained using diffusion MRI and anatomical T1w imaging techniques. To do so, we will estimate metrics, reflecting the organizational properties of brain networks.

Analysis of Connectome through Graph Theory

Graph theory is a mathematical theory for the study of organized data in the form of networks. Graph theory was first proposed by Euler in 1735 to address the problem of Berardino BARILE the seven königsberg bridges. This work also introduced the notion of an Eulerian graph. An Eulerian graph is a graph in which there is a closed path (or cycle) passing through all edges only once. This property gave rise to Euler's theorem, which was only proved 130 years later by Carl Hierholzer. Afterwards, these methods were applied to numerous domains ranging from biological systems to computer networks. Their application to the study of social behavior has also made it possible to define a class of networks with common properties: the so-called "small-world" networks (see subsection 2.3.2). In such networks, short distances are observed between nodes that seem to be far apart in anatomical sense. It was also shown that this type of organization is observed in the human brain network. Notwithstanding, graphs can be represented as mathematical structures used to model pairwise relations between objects. To fully understand the concept of connectome data, a formal mathematical definition is in order. Let's define with a i,j where 1 ≤ i, j ≤ q an element of the connectivity matrix A ∈ R (q×q) , such that a i,j = Φ(i, j) represents the weights of the connections between node i and j, where Φ : N 2 → N. In this context, the connectome can be represented in the form of an adjacency matrix which is weighted and not-oriented: G(V, E, ω), where |V | = q and |E| = l represent the set of all nodes and edges with cardinality q and l respectively. Consequently, the set of all connections can be represented as follows:

E ⊆ {{i, j}|Φ(i, j) ≥ 0, ∀0 ≤ i, j ≤ q} (2.17)
However, two different topologies of graphs exist, such as directed and undirected graphs. In a directed graph, information can travel only in a specific direction (i.e. from node A to node B), which means that edges have orientations. Conversely, for an undirected graph, no direction is associated with edges and the information can travel in both directions (from node A to node B and vice-versa). However, in this thesis we mainly focus on weighted undirected graphs since practically speaking it is difficult to estimate the direction and orientation of the signal inside the brain. Also, we already highlighted that, in order to characterize the properties and topological organization of a graph, different metrics can be calculated. These metrics reflect the characteristics that can be associated with a graph: segregation, i.e. the "division" of the graph into communicating modules (i.e. group of similar nodes), integration, measuring the "speed of propagation" of information in the network, and centrality, measuring the importance of the nodes in the network. These measurements can be made at two scales, either at the node level, characterizing the properties of each node, which is referred to as a local metric, or at the network level, characterizing the graph as a whole, which is referred to as a global metric. An additional level of differentiation can be considered based on the association to a specific weight (i.e. cost) associated to each edge (i.e. link) inside the network. We can thus define as weighted graph networks in which at each edge a specific weight is associated (to traverse the graph from node A to node B a specific cost, or sum of costs is associated). Conversely, we refer to unweighted graph networks in which no weight is associated to any edge (or equivalently a weight of 1 is associated to all edges inside the network).

A "Small World"

Several graph architectures can be described according to the organization of the connections that exist in the network. This organization will confer particular properties to each type of graph. The simplest graph architecture will be the random graph. In such a graph, the distribution of the nodes does not follow any particular statistical law. These types of networks are characterized from the fact that no or low level of clusters (group of nodes highly connected between them but poorly connected with other clusters) can be identified. In such a case, all nodes are mostly connected to one another and the information can travel from a node A to a node B involving very few additional nodes. In contrast to the random graph, the regular graph exhibits the characteristic of being connected following the same pattern for each node, avoiding long connections between nodes far apart in the network. As a consequence, the information can travel less efficiently between nodes far apart inside the network since in order to "travel" from node A to node B many other nodes have to be involved. An intuitive representation of the concept of random and regular graph is proposed in Figure 2.10. For completeness we must say that other network architectures, such as scale invariant networks and hierarchical networks have also been defined in the literature, but are of little interest in the study of biological networks like the one of the human brain, which is the focus of this thesis and thus they will not be discussed here.

Notwithstanding, random and regular graphs are simple and intuitive models that allow some mathematical concepts to be established. However, it is easy to understand that this type of organization is only suitable for very few networks. In reality, it will most often be an intermediate model between a random network and a perfectly regular network. This intermediate condition between regular and random graph is what is known in the literature as "small-world" graphs. This term was coined in the 1950s through the study of social interactions. These studies made the surprising observation that in a group with a large number of individuals, all subjects are linked between them. This same property is found in many biological networks including the brain network [190] where both regular and random properties are present. The "small world" graph is reported on the left-hand side and it represents a structure which lies in between that of a regular graph and that of a random graph. Image taken from [START_REF] Watts | Collective dynamics of small-world networks[END_REF]

Graph Properties

The different metrics are all derived from fundamental properties of graphs and their nodes. The most basic of these properties is the degree of a node i, which represents the number of connections in which i will be involved. From this basic property, a more stable graph metric can be obtained. The density (D) of a graph is defined as the ratio between the number of connections existing in the graph over the number of possible connections. Mathematically, we can define density as follows:

D = l q 2 -q 2 = 2l q 2 -q (2.18)
where l represents the total number of connections inside the graph and q the number of nodes. An additional property of a graph is the number of triangles (composed of exactly three nodes and three connections), which can be defined as follows:

t i = 1 2 i,h∈N a i,j a i,h a j,h (2.19)
where t i represents the number of triangles in which node i is involved while j and h represent two additional nodes inside the graph and a i,j the corresponding edge weight. The third and last elementary property of a graph lies on the notion of distance between two nodes i and j. In a binary graph (where the weights of the existing connections are such that a i,j = 1, otherwise a i,j = 0), the distance between node i and node j is the 92 Berardino

BARILE

sum of the weights of connections between these two nodes. Intuitively, this property describes the number of minimal steps (walks) in the graph needed to reach node j starting from node i. As previously discussed, these basic concepts can be exploited in order to define three different and more complex measures describing the tendency of a graph to be segregated, forming clusters of nodes, or the characteristic of the graph in term of integration, meaning the tendency to each node to efficiently transfer information throughout the network.

Integration and Segregation Measures

Two measures of segregation are the most used in the literature: clustering coefficient (C) and modularity (Q). As already stated, these measures provide information reflecting the level of organization of the graph into modules by studying the relations of the neighboring nodes. In other words, the clustering coefficient measures the density of connections between the nodes in a node's neighborhood. If these connections are very dense, then this node and its neighborhood will form what is called a cluster or module. The definition of the clustering coefficient of a node is based on the notions of triangle (Eq. 2.19) and it can be defined as follows:

C i = 2t i k i (k i -1) (2.20)
where k i represents the degree of node i while t i describes the number of triangles in which node i is involved. The average of the clustering coefficients of all q nodes of a graph can also be calculated and it represents the global clustering coefficient (C) defined as follows:

C = 1 q i∈N C i = 1 q i∈N 2t i k i (k i -1) (2.21)
However, one of the disadvantages of the average is that it is strongly influenced by extreme values. Thus, if in the graph, some nodes have a low measure of degree, this measure could be biased. It is therefore more informative to calculate instead the Transitivity (T) metric, which is a normalized variant of the global clustering coefficient expressed as follows:

T = i∈V 2t i i∈V k i (k i -1) (2.22)
where i represents a generic vertex in the set of vertices V . Thus, by definition, graphs comprising nodes with a high clustering coefficient, and thus graphs with high transitivity, will be mostly composed of modules that are not very connected to each other and within Berardino BARILE which the nodes are strongly connected. The difference between these two densities of connections (intra-and extra-module) can be measured by means of the modularity metric which is expressed as follows:

Q = u∈M   e uu - v∈M e uv 2   (2.23)
where M is the number of non-overlapping modules in the network, and e uv is the proportion of links that connect nodes u and v in the module. As a final additional measure of segregation we can define the assortativity metric. It represents the coefficient of correlation (Pearson correlation coefficient) between the degrees of each node. It is mathematically defined as follows:

r = 1/m (i,j) k i k j -[1/m (i,j)∈E 1 2 (k i + k j )] 2 1/m (i,j) 1 2 (k 2 i + k 2 j ) -[1/m (i,j)∈E 1 2 (k i + k j )] 2 (2.24)
where k i and k j defines the degrees of the vertices at the ends of the i-th and j-th edge, m represents the maximum number of connections in the network while E represents the set of edges in the graph. For a thorough definition of such metric we refer to [START_REF] Newman | Assortative mixing in networks[END_REF]. Intuitively, positive values of this metric describe a network where nodes with high degree are preferentially connected with nodes of similar degree and vice-versa (positive correlation). Conversely, for negative assortativity values, the characteristic of the network is described by the tendency to associate nodes with low degree with other nodes with high degree (negative correlation). These segregation measures are excellent indicators of the modular organization of the network and the organization of connections in the network. They are therefore of great interest in the study of the brain network in which modules dedicated to particular functions have been observed. The second group of metrics are based on the concept of segregation. These metrics differ from the previous group which is almost always based on the concept of triangles. On the contrary, integration measures are mostly based on the concept of distance. Indeed, the most widely used metric in this field, the Characteristic Path Length (CPL), is simply the average of the distance matrix in the graph. Introduced by Watts and Strogats in 1998 [START_REF] Watts | Collective dynamics of small-world networks[END_REF], it is obtained as follows:

CP L = 1 q i∈V L i = 1 q i∈V j∈V,j¬i d i,j q -1 (2.25)
where L i is the average length of the shortest paths between node i and all other nodes in the graph and d i,j is the shortest distance between nodes i and j. If in the network, nodes are isolated, or even not connected, d i,j will be very large or infinite, increasing drastically the distance between the nodes and thus the value of the CPL. The fact that some nodes can have such an important influence on the measurement of a metric is not desirable. For this reason, a more reliable measure is the Efficiency of the graph, which is defined as the average of the inverse of the distance matrix. Specifically, for node i, the efficiency is obtained using the following formula:

E i = 1 q i∈V j∈V,j¬i d -1 i,j q -1 (2.26)
This concept can also be generalized to the whole graph by averaging the efficiency of each node. Mathematically, we can write it as follows:

E g = 1 q i∈V E i (2.27)
Integration metrics allow us to estimate the capacity of a network to efficiently transfer information between network nodes. Indeed, a network where distances between nodes are shorter will allow faster communication. Although both are based on the measurement of the distance matrix, the efficiency metric has the advantage of being less sensitive to the presence of disconnected nodes in the network and will therefore be a less biased marker of integration.

Centrality Measures

An intuitive concept relates to the fact that some nodes are more important than others inside the network. These type of nodes are generally called "hubs". Indeed, the removal of a central and highly connected node will have important consequances on the organization and flow of the information inside the network, especially toward peripheral and poorly connected nodes. For this reason, centrality measures are of extreme relevance to evaluate the importance of each single node with respect to all other nodes inside the network. The basic measure of importance of a node is the degree, which, as we have seen, measures the number of connections in which this node is directly involved. However, the degree of a node is not sufficient, on its own, to determine the node importance. Indeed, a node will only be deemed important if its degree is high compared to that of other nodes. Moreover, the degree does not measure the centrality of the node and other measures were introduced in the literature in order to overcome these limitations. An interesting measure of the centrality of a node is called "Betweenness Centrality" (BC), which is defined as the total number of shortest paths from node i to node j that pass through a specific node v of interest, which we define with ψ(i, j|q), over the total number of possible shortest paths, defined as ψ(i, j), averaged with respect to all the nodes in the network (q). Thus, Berardino BARILE 95 it can be formally defined as follows:

BC = 1 q v i,j∈V ψ(i, j|q) ψ(i, j) (2.28)
This measure provides a better estimate of the importance of each node in the network. Indeed, a node with a high BC will be involved in many direct connections. If this node were to be removed from the network, some of the connections between the other nodes would no longer be as direct, or even completely broken.

Importance of Graph Metrics

The aforementioned network measures are not intended to be an exhaustive list. However, they delineate important concepts for this thesis, and for the graph theoretical analysis in general, since they represent the most informative and used metrics, important for the analysis of brain regions in the human connectome. For this reason a summary of these concepts might be useful. The node degree is one of the most easily accessible graph measures and it is also highly informative relative to the distribution of nodes across the whole network. In many cases, the "degree" of a node is highly correlated with other more complex influence measures. Many of these measures capture the "centrality" of network elements, for example the "betweenness centrality". This measure is, in turn, related to communication processes. However, it is also often found to be highly correlated with the measure of "closeness centrality", quantifying the proximity of each node to the rest of the network. Another class of measures concerns the vulnerability aspects of a network. For example, the decrease/increase in "global efficiency" due to the deletion of single nodes or edges, or the variation of "assortativity", in conjunction with a decrease/increase of network density, should be used when attempting to identify crucial areas of the network [START_REF] Sporns | Structure and function of complex brain networks[END_REF]. Interestingly, a more in-depth analysis of the human brain network has shown that modules represent communities of densely interconnected neural elements that share common input and output projections and exhibit similar physiological responses [START_REF] Hilgetag | Clustered organization of cortical connectivity[END_REF]. To this end, hubs perform important integrative roles in structural networks. It should be noted that there is no unique way of detecting these hubs with graph theory tools. Bridging nodes indeed, are generally identified by their high degree, high centrality, and diverse connection profiles that straddle the boundaries between modules [START_REF] Sporns | Identification and classification of hubs in brain networks[END_REF]. A detailed analysis of the topology of human brain structural connectivity, revealed that regions including portions of the superior frontal cortex, superior parietal cortex, and the precuneus form clusters with highly interconnected hub. Also, several subcortical regions, including the thalamus, hippocampus, and part of the basal ganglia present modular characteristics and high number of hub nodes [START_REF] Van Den Heuvel | Rich-club organization of the human connectome[END_REF].

Conclusion

In this chapter an introduction to the MRI principles were provided, highlighting the main characteristics and concepts used for the identification of different modalities and their practical uses. Notwithstanding, the scientific contributions in this doctoral dissertation strongly relies on the analysis of connectivity data extracted from the combination of multiple MRI modalities. An introduction to graph theory and brain connectivity was proposed. The study of brain networks is a promising research filed. The development of new analytic tools and modeling approaches, together with methodological advances in the area of human neuroimaging, continue to allow ever more detailed analyses of the human structural and functional networks. As shown in Part II and III, graph methods represent a powerful approach for capturing the architectural organization of brain networks, which allows the analysis of pathological variations across individuals, time and different brain diseases.

Berardino BARILE

Introduction

As leading cause of non-traumatic disability in young people, Multiple Sclerosis (MS) represents an inflammatory autoimmune disease of the Central Nervous System (CNS). It is most frequently diagnosed in people aged between 20 and 40 years and today there are more than 1,2 million people living with MS in Europe. It appears between the ages of 20 and 40 in more than two out of three cases and affects mostly women, with a ratio of three women to two men. Several known physiopathological phenomena are at the origin of the manifestations of the disease, such as inflammation, demyelination and neurodegeneration [START_REF] Compston | Multiple sclerosis[END_REF]. However, its etiology remains unknown to this day. Usually, inflammation and demyelination are considered as the initial and prominent mechanisms, while neurodegeneration is more prominent in progressive phases of MS. The course of MS is highly varied and unpredictable. In most patients, the disease is characterized initially by episodes of reversible neurological deficits, which is often followed by progressive neurological deterioration over time.

Epidemiology

Epidemiology of MS is an interesting subject of study, indeed it allows to better understand how geographic factors can influence the development of MS. In recent years, knowledge of the geographical distribution of the disease, its survival data, and a better understanding of the natural history of the disease, have improved our understanding of the respective roles of endogenous and exogenous causes of MS. The study of geographic distribution of MS was first introduced by [START_REF] Kurtzke | Geographic distribution of multiple sclerosis: an update with special reference to europe and the mediterranean region[END_REF] who studied the MS distribution in Europe and the mediterranean regions and subsequently updated with the study [START_REF] Kurtzke | Multiple sclerosis in time and space-geographic clues to cause[END_REF] which highlighted the time and space-geographic causes of MS. In Figure 3.1 the worldwide geographic repartitioning of the MS disease is shown and it is easily seen that the geography distribution is not equally distributed.

From Figure 3.1 it is possible to identify zones with high and low risk associated with the disease. For instance Canada, Italy, Norway and Germany represent countries at high risk with more than 200 cases every 100,000 people. Conversely, countries like Cina, Brazil, Mexico and Malaysia are categorized as low risk with 0 to 25 cases every 100,000 people. The risk exposure to MS between populations migrating to different regions of risk has also been studied [START_REF] Kurtzke | Multiple sclerosis in time and space-geographic clues to cause[END_REF] demonstrating that adolescents migrating before the age of 15, present the same risk factor of their original region. Conversely, adults who migrate, acquire the risk factor typical of the new risk zone. These results show how the geographic partition of MS is related to environmental factors linked to a genetic susceptibility. Also

Physiopathology

We already discussed that MS represents a difficult disease to be studied due to the widely varying manifestation of clinical sympthoms. To better understand the pathological effect of the disease in the following we will discuss the physiopathology of MS. The central nervous system can be schematically considered as being composed of two main parts: the GM and the WM. The GM contains the majority of the cell bodies of the neurons, while the WM contains the axons of the neurons that connect the different areas of the GM. Neurons are the basic units of the nervous system. They have a diameter ranging from 5 to 150 µm and they are composed of a cell body, dendrites and axons. In Figure Berardino BARILE 3.2 a schematic representation of a multipolar neuron is depicted. The cell body is divided in nucleus and perinuclear cytoplasm. The cell body represents the most voluminous portion of the neuron, from the cell body dendrites and axon. Dendrites are the expansion of the cytoplasmic membrane, they principally receive stimuli from axons or other neurons. Axons are responsible for the nerve impulse transmission. The cell bodies and axons belonging to the same cell have the same plasma membrane. The difference in color between these two areas comes from the myelin sheath which surrounds all the axons present in the white matter. This myelin sheath is in fact a winding of the plasma membrane of the oligodendrocytes, forming an extremely stable lipid-rich multilayer. Along an axon, this myelin sheath is regularly interrupted by so-called nodes of Ranvier. This organization allows the transmission of nerve impulses in a saltatory manner, which allows a much higher speed of propagation than that observed in a non-myelinated fibre of the same diameter. Indeed, the increase in axon diameter allows an increase in the speed of propagation of the nerve impulse, as is the case in the peripheral nervous system [START_REF] Preston | Electromyography and neuromuscular disorders clinical-electrophysiologic correlations. boston oxford johannesburg melbourne new delhi singapore; basic nerve conduction study[END_REF].

To understand how the disease causes its symptoms, it is important to understand how nerve impulses are propagated in myelinated axons. In an amyelinated fibre, the impulse is generated by the opening of sodium channels, which causes membrane depolarization. In the case of myelinated fibres, the same phenomenon occurs, but because of the high lipid content and the number of layers in the myelin sheath, the latter is a good insulator, so the nerve impulse does not have to travel all the way down the fibre, but only through the nodes of Ranvier, which represents an important advantage as there are fewer sodium channels to open. Also, MS is characterized by an abnormal immune-mediated response who attacks the myelinated axons of neurons, inducing a progressive destruction of myelin. As myelin helps to speed up the nerve impulse transmission along the axon, a destruction of myelin decreases the capability of the axon to transmit the nerve impulse. The inflammatory processes seem to start after a cell-mediated response. In this part, macrophages recognize the myelin basic protein (MBP) and present this to the T lymphocytes. After their activation, they cross the Blood Brain Barrier (BBB) and trigger the immune response and the consequent inflammation. Recent studies also suggest that the beginning of an antibody-mediated response, with an abnormal production of antibody for myelin destruction, plays an important role in the progression of the inflammation with relative tissue damage [START_REF] Disanto | The evidence for a role of b cells in multiple sclerosis[END_REF]. Demyelination in a specific tissue area usually starts without axonal damage [START_REF] Noseworthy | Multiple sclerosis[END_REF], oligodendrocytes destruction and axonal damages are induced when repeated attacks appear in time. At the beginning of the pathology, a remyelination process is opposed to pathological demyelination. During this process, the oligodendrocyte progenitor cells differentiate in oligodendrocytes in order to repair the damaged tissue Berardino BARILE [START_REF] Goldschmidt | Remyelination capacity of the ms brain decreases with disease chronicity[END_REF]. Unfortunately, the capability of the oligodendrocyte progenitor cells to differentiate oligodendrocytes is reduced in MS. This limitation influences the capability to recover the damaged tissue. In the intermediate phase, the myelin contained in the tissue affected by the disease, is substituted with scarred tissue. In the late part, demyelination effects are not present and the tissue area does not contain inflammatory cells. The increased permeability of the BBB and the inflammatory attacks increase the clinical effect related to the neurodegeneration and atrophy. When all the reparation mechanisms of the tissue are exhausted, disability progressively increases.

Clinical Forms

Due to the dissemination of lesions throughout the central nervous system, the symptomatic manifestations associated with MS can be highly variable. They can be very discreet and go almost unnoticed, or on the contrary have major consequences. As a result, the initial diagnosis is often difficult to make and is often made a posteriori. In addition, the temporal evolution of the disease is not linear. In fact, it evolves by relapses followed or not by a remission which may be total or partial. Among the initial clinical manifestations, three are particularly common encountered [START_REF] Ouallet | Aspects cliniques, physiopathologiques, et thérapeutiques de la sclérose en plaques[END_REF]. The first is the retrobulbar optic neuritis, which manifests itself as painful sensations in response to eye movements and a rapid decline in visual acuity. The latter may disappear fairly quickly, fade away, or permanently. The second is the acute myelitis, which leads to sensory disorders in the most moderate cases, which can lead to paraplegia. The third is the brain stem damage, resulting in voluntary vision abnormalities. When faced with one of these symptoms, there is no indication that it is actually a case of MS. To confirm the diagnosis, further tests, including MRI, are required [START_REF] Polman | Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria[END_REF]. In Multiple Sclerosis, the course of the disease and the risk for developing permanent disability are very different from one patient to another and the prediction of long-term disability is still an open challenge. Generally speaking, four clinical profiles (also called phenotypes) are considered to characterize the MS disease: Clinically Isolated Syndrome (CIS), Relapsing Remitting MS (RRMS), Secondary Progressive MS (SPMS) and Primary Progressive MS (PPMS). These phenotypes will be discussed in the following subsections. In Figure 3

Clinically Isolated Syndrome (CIS)

Clinically Isolated Syndrome (CIS), is recognized as the first clinical presentation of the disease, showing characteristics of inflammatory demyelination. However, such symptoms are not definitive of an MS diagnosis which has yet to fulfill criteria of dissemination in time. This form of MS is the consequence of the series of one or two consecutive attacks from which the patients recovered completely, generally quite quickly, and without any progression or persistence of disability. Following this first attack, in 85% of the cases, the progression to a relapsing remitting form (RRMS) occurs.

Relapsing-Remitting MS (RRMS)

In this form of MS, we often observe unpredictable attacks. During the presence of those new attacks new clinical symptoms appear or old clinical symptoms evolve. This phase has a variable duration and could be followed by a partial or total remission. At this stage, the pathology can be inactive for months or years.

Secondary-Progressive MS (SPMS)

The Secondary-Progressive (SP) form is the evolution of the RR form, and is characterized by a permanent form of disability that gradually progresses in the time, indepen-dently of the presence of inflammatory attacks. This form involves 30% of MS patients.

Primary-Progressive MS (PPMS)

Primary progressive (PP) form is characterized by the absence of inflammatory attacks. The patients suffer from an accumulation of deficits and disabilities. All these effects can be stable for certain periods or can progressively degenerate in months or years.

During the course of the disease, and particularly in the progressive forms, a neurodegenerative component may be added to the pathological processes, leading to an accumulation of disability. These may include motor disorders (spasm, paraparesis, paraplegia), sensory disorders leading to pain, speech difficulties, dysphagia, visual disorders, and other disorders.

Diagnosis

The diagnostic criteria for MS have been continuously evolved since 1950's. The common aim for all the defined criteria until now, is to establish the dissemination in space and time of the clinical symptoms caused by the lesions in the Central Nervous System, and to rule out other diseases. Today, there are no symptoms, physical or biological findings that can be used to diagnose MS without a proper evaluation of the clinical condition from an expert neurologist The Revised McDonald Criteria, published in 2017 by the International Panel on the Diagnosis of Multiple Sclerosis, includes specific guidelines for using MRI and cerebrospinal fluid analysis to complete the diagnostic process. It is important to notice that, since MS is characterized by multifocal effects that can change during time, its diagnosis is not trivial. Diagnosis of MS is based on the multifocal and evolutive characteristics of the pathology. Specifically, differential diagnosis can be used in order to detect MS. In medicine, a differential diagnosis represents a scientific approach for the discrimination of a particular disease or condition from others that present similar clinical symptoms. More generally, a differential diagnostic procedure is a systematic diagnostic method used to identify the presence of a disease, excluding other possible alternatives. In MS, a systematic process for exclusion of alternative diagnoses is really hard to define, due to the large variety of MS symptoms. Moreover, since the symptoms of MS evolve and change over time, the use of the differential diagnosis is not sufficient to have a general overview of the pathology. For a more clear and detailed discussion of differential diagnosis in the context of MS we refer to [START_REF] Noseworthy | Multiple sclerosis[END_REF]. Notwithstanding, more com- plex and effective methods can be used, compared to the differential diagnosis. MRI is the diagnostic tool with the best sensitivity in the search for MS lesions. It allows to identify lesions, their location, and to evaluate their dissemination in space. In addition, a longitudinal follow-up of the patient makes it possible to identify the appearance of new lesions and thus to control whether there is dissemination over time. A positive MS diagnosis is based on four essential criteria: i) temporal lesions dissemination ii) spatial dissemination of MS lesions in particular regions of the central nervous system iii) presence of inflammatory processes in the central nervous system iv) absence of other progressive diseases.

In order to have a complete picture of the damage generated by MS in the brain three MRI sequences are needed to detect different types of lesions: T2w and FLAIR images, which allows to detect MS lesions as hyper-signal spots, T1w sequence where lesions are characterized by hypo-intensity signal spots and T1w sequence acquired after injection of a contrast agent (like Gadolinium) to detect regions where disruption of the hematoencephalic barrier is present. In Figure 3.4 lesions in the white matter tissue are visible based on 3D T1-Gd and 3D T2-FLAIR MRI modalities.

Disability Assessment and Medical Treatment

In MS, disability represents one of the most important aspect since it directly hinder the quality of life of patients. Much effort have been spent for assessing disability in MS Berardino BARILE and multiple rating scores have been proposed in the literature. However, the Kurtzke Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) scores represents the "gold standard" approaches for disability evaluation.

Expanded Disability Status Scale (EDSS)

The EDSS is a clinician-administered assessment scale evaluating the functional systems of the central nervous system and it represents the most popular and widely used instrument for disability evaluation in clinical practice. The EDSS is used to describe disease progression in patients with MS and to assess the effectiveness of therapeutic interventions in clinical trials [START_REF] Moock | Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis[END_REF]. It addresses disability in eight Functional Systems (FS): pyramidal, cerebellar, brain stem, sensory, bowel and bladder, visual, cerebral total, and cerebral mentation. The EDSS score is a composite ranging from 0 to 10 with half points increment and incorporates FS grades as well as the degree of functional disability and ambulation. For EDSS score between 0 to 4.5, the motor impairment is less pronounced and patients are usually characterized by normal ambulation. Conversely, for scores higher than 5 a progressive loss of ambulatory ability is expected [226]. These functional systems represent eight different areas of the central nervous system and each ranges from 0 (normal) to 5 or 6 (maximal impairment). These functional system grades, indications of mobility and restrictions in daily life and their combination are used to define the EDSS as an ordinal measurement. For EDSS scores lower than 3, the disability remains moderate, while it becomes significant, affecting daily activities, for EDSS values of 3 to 4.5. With values over 5, the walking impairment becomes very severe and permanent. The distribution of EDSS scores among MS patients is typically biphasic, accumulating around two to three points, and six to seven points, indicating that patients do not stay at each step of the scale for equally long periods. There are many criticisms of the EDSS, including the fact that it has high intra and inter-rater variability. In fact, different studies have demonstrated that for inter-rater reliability kappa values between 0.32 and 0.76 were observed while the intra-rater variability resulted to be somehow more stable [START_REF] Moock | Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis[END_REF]. Notwithstanding, degree of variability in the EDSS score between 1 to 3.5 were observed in both cases, especially for low disability scores. Additionally, it has poor assessment of upper-limb and cognitive function, and it lacks a linear relationship between score difference and clinical severity [226].

Multiple Sclerosis Functional Composite (MSFC)

Clinical outcome assessment in MS is challenging due to the diversity and fluctuating nature of MS symptoms. Clinical scales such as the EDSS have demonstrated some weaknesses in the assessment of key clinical dimensions of MS such as psychometric limitations. Based on analyses of pooled data from natural history studies and from placebo groups in clinical trials, in 1994 the National MS Society's Clinical Outcomes Assessment Task Force proposed a new multidimensional clinical outcome measure, the MS Functional Composite (MSFC) score. It represents an additional important instrument for the assessment of disability in patient affected by MS. The primary goal for creating the MSFC was to improve the standard measure of MS disability for clinical trials and to develop a multidimensional metric of overall MS clinical status [START_REF] Fischer | Functional composite measure (msfc): an integrated approach to ms clinical outcome. national ms society clinical outcomes assessment[END_REF]. The MSFC is a three-part performance scale for evaluating the degree of impairment in MS patients. It includes the assessment of leg function by moving a short walking distance ("Timed 25-Foot Walk", T25FT), the assessment of arm function using breadboard test ("9-Hole Peg Test", 9HPT) and an attention/concentration test to assess cognitive functions ("Paced Auditory Serial Addition test", PASAT). An integrated MSFC score is calculated using zscores. The raw scores for each test are transformed into z-scores to achieve a common metric in standard deviation unites. The z-score index represents the number of standard deviations a patient's test result is higher or lower than the average test result. This translates with a z value greater, lower or equal to zero respectively, compared to a reference population. The mean and standard deviation from the test results at the baseline visit for all patients in each study is used as the reference population values to create the z-scores for each component of the composite score. The z-score is calculated by subtracting the mean of the reference population from the test result and then dividing this by the standard deviation of the reference population. To ensure that all measures are in the same direction, a transformation is necessary. In creating the composite outcome measure, it was decided that a higher test result would indicate improvement from baseline. There is an ongoing debate about which dimensions to include in the MSFC (e.g. the inclusion of a vision testing) as well as how the reference population affect the standardized scoring (z-scores) of the MSFC.

Medical Treatment

There are different levels of treatment for the disease. The first one consists of a reeducation of the patient in order to allow him to recover a maximum of compromised functions or, if necessary, to learn to live with the disability caused by the disease. To this aim, symptoms can be treated to improve the quality of life of the patients without directly affecting the pathological process. Pain is usually treated with the prescription and administration of analgesics or antiepileptic. The second one is a treatment of the attacks, the goal of which is to reduce the number and the frequency of the attacks and then reduce the progression of the disability by increasing the remission. This type of treatment is mainly based on the administration of corticoids usually methylprednisolone. Finally, the treatment of the pathology tries to control its evolution. The main idea is to repair the demyelination by stimulation of the remyelination process. This treatment relies on the auto-immune nature of MS. Immunomodulators and Immunosuppressors drugs are often used to reduce the effect of MS.

Conclusions

MS is a complex disease difficult to predict. As a result, it has been the subject of much research, both in terms of its characterization and in the search for new therapeutic approaches. For the moment, we only know how to treat the symptoms of the disease or to try to delay it and lessen its effects, but we still do not know how to cure it while its origin has not yet been clarified. 110 Berardino

BARILE

Chapter 4

Dataset Description, Softwares and Conclusions

Dataset Description

For all the contributions listed in Part II and Part III of the present manuscript, the same dataset has been used including 90 MS patients. These patients are distributed in four clinical profiles (12 CIS, 30 RRMS, 28 SPMS, 20 PPMS) and were recruited at the MS clinic of Lyon Neurological Hospital. All patients underwent a MR examination on a 1.5T Siemens Sonata system (Siemens Medical Solution, Erlangen, Germany) using an 8-channel head-coil. Additionally, seventeen healthy volunteers with no history or signs of neurologic disorders served as control subjects in the study and were identified as healthy control (HC). All patients were examined at different time points, every six months during the first three years and then every year for the following four years, leading to a total of 660 MR scans. The gap between two consecutive scans is 6 or 12 months. Population statistics are reported in Tab. 4.1. Each patient underwent multiple brain MRI examinations over a different period, ranging from 2.5 to 6 years. The minimum number of scans per patient is 3, while the maximum is 6. Clinical history was collected and neurological examinations, including the EDSS and the MS functional composite tests, were performed by a board-qualified neurologist for all patients. Patients were diagnosed with MS according to the McDonald's criteria [START_REF] Mcdonald | Recommended diagnostic criteria for multiple sclerosis: guidelines from the international panel on the diagnosis of multiple sclerosis[END_REF]. They were then classified by the slice thickness = 3 mm, voxel size = 0.9 × 0.9 × 3 mm, acquisition time = 4.57 min. For all the studies, approval by the local Ethics Committee (CPP Sud-Est IV) and the French national agency for medicine and health products safety (ANSM) was obtained. Written informed consents were obtained from all patients prior to study initiation.

The DTI protocol was based on a 2D multi-slice spin-Echo-Planar Imaging (EPI) sequence (TR/TE=6900/86 ms, acquisition time=7 min). Fifty-one contiguous, 2.5mm thick, axial slices according to the anterior commissure-posterior commissure plane were acquired. Twenty-four diffusion-gradient directions (b = 1000 s mm 2 ) were applied. A nominal isotropic 2.5mm 2 resolution was obtained by using a matrix size of 96 × 96 over a field of view of 240 × 240mm. The b 0 image was acquired four times to increase signal to noise ratio while the other directions were acquired twice.

Software

If not otherwise stated the following softwares have been used for all the analysis in Part II and Part III of the present Manuscript. In particular, Freesurfer v6.0.0 image analysis suite [START_REF] Reuter | Within-subject template estimation for unbiased longitudinal image analysis[END_REF] was used for the preprocessing of the anatomical images, including cortical and sub-cortical GM segmentation. The MRView v3.0 was used for MRI data visualization and interpretation. Python v3.6 was used as main programming language for data manipulation, preprocessing and machine learning analysis. The Stata16 [START_REF] Statacorp | Stata statistical software: Release 16[END_REF] statistical software was used for the statistical analysis and hypothesis testing throughout the manuscript. The MATLAB R2021a statistical toolkit was used for the tensor factorization analysis along with the Tensorlab v.3.0 [START_REF] Vervliet | Tensorlab 3.0. Online version[END_REF] package.

Conclusions

An important issue in neuroscience is the characterization of human brain structure and function, and their alteration in brain diseases. Brain connectivity represents an hot topic in the neuroimage domain due to their ability to model functional, structural and morphological properties of the brain. In fact, connectomics arguably represents one of the most important topics for modelling the complexity of the human brain, where multiple entities, (i.e. neurons) are interconnected, forming an architecture of intricate net of relations, giving life to human cognitions. Notwithstanding, such a complex network can be subjected to alteration and pathological dysfunctions that may hinder the cognitive and physical ability of the patient. The use of connectomics has already been demonstrated effective for an accurate analysis of the human brain network [START_REF] Bullmore | Complex brain networks: graph theoretical analysis of structural and functional systems[END_REF][START_REF] Rubinov | Complex network measures of brain connectivity: Uses and interpretations[END_REF][START_REF] Hoerl | Ridge regression: Biased estimation for nonorthogonal problems[END_REF][START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF]245]. In Part I, an introduction to the main fundamental concepts needed for a better understanding of the contributions listed in Part II and Part III of the present manuscript was detailed. Specifically, in Chapter 1 an introduction to the main algorithms and statistical concepts at the core of any machine learning analysis were offered. In Chapter 2, an in-depth overview to the MRI machinery is proposed, describing the main concepts and how different MRI modalities can be used for investigating important biomarkers in different brain tissues. Also, a deep-dive into the principal concepts of graph theory was proposed, highlighting its importance for the characterization of specific properties. Finally, in Chapter 3, a brief introduction to the MS disease was also proposed since it represents the main area of application of the present doctoral thesis. The epidemiology and physiopathology of the disease were introduced along with the four clinical profiles characterizing MS. In Part II and Parti III of this doctoral dissertation, the power of connectome analysis is Berardino BARILE exploited in order to define MS disease phenotypes based on their underpinning mechanisms on both the structural and morphological aspects. To address this task, a data driven approach based on deep learning and statistical learning models is used in order to extract relevant insights able to facilitate the neurologist's challenge in the diagnosis and prognosis of patient with MS. The power of artificial intelligence, combined with the connectome representation of the human network organization, has not yet been fully tackled in the context of MS. In the list of contributions provided in Part II and Part III of this thesis, we aim toward this direction where an in-depth analysis of the complex mechanisms underlying the brain pathologies is proposed. To this end, the capability of deep learning models were explored with the aim of solving specific image processing tasks. These techniques have been proven capable of extracting highly meaningful statistical patterns in large-scale and high-dimensional datasets, showing great power and flexibility by learning to represent data as a nested hierarchy of concepts. However, other statistical learning methods, combined with connectome analysis, have also been demonstrated very effective for specific classification tasks, providing more stable and sometimes more interpretable results. Thus, a statistical and data-driven analysis is performed with a special focus on interpreting the underlying pattern of reasoning guiding the decision process of the machine learning model. This aspect represents an important point, especially in the biomedical domain where causal-effect relations are at its core. Additionally, MS is usually conceived as a white matter disease, thus neglecting the direct and indirect impact of other brain tissues such as grey matter. However, multiple epidemiological studies have demonstrated the involvement of grey matter to the overall tissue degeneration of the brain of MS patients. For this reason, in this thesis, we study the MS degeneration process in both the white and grey matter tissue. Although white matter tissue alteration represents a more informative aspect of the structural connections between different brain regions (i.e. ROIs), its accuracy is highly dependent on the estimation of white matter fibers by means of streamline tractography. Conversely, morphological features, such as thickness, curvature and area of the grey matter tissue, can be directly estimated from the anatomical T1-weighted image, which represents one of the most used modality in clinical applications as opposed to DTI. One important reason of using T1w image in clinics is due to the shorter and less expensive procedure needed for its acquisition compared to the diffusion tensor image, justifying the need of expanding MS analysis to secondary brain tissues.

II Machine Learning Models for

Classification of MS Profiles

Introduction

Artificial intelligence has revolutionized many areas of research, from economics and law to health-care. However, a large collection of data is essential for statistical evaluation and machine learning applications, particularly in the field of deep learning (DL). Indeed, DL frameworks have achieved remarkable results in many fields, such as pattern recognition, natural language processing, image processing, among others [START_REF] Milletari | V-net: Fully convolutional neural networks for volumetric medical image segmentation[END_REF][START_REF] Young | Recent trends in deep learning based natural language processing[END_REF]. The main advantage of using DL applications lies in their great ability to recognize hidden patterns in the data, thanks to the multiple nonlinear transformations produced by the sequential stacks of multiple layers [START_REF] Schwenker | Artificial Neural Networks in Pattern Recognition[END_REF][START_REF] Yun | Deep neural networks for pattern recognition[END_REF] However, huge amount of data are required for training this kind of models while in the context of biomedical domain, and particularly in medical imaging, extensive datasets are challenging to obtain due to systems availability, costs constraints, acquisition methodology, and pathology related variability [START_REF] Adibuzzaman | Big data in healthcare[END_REF][START_REF] Floca | Challenges of open data in medical research[END_REF], resulting in small and imbalanced datasets. Notwithstanding, when dealing with image data, different solutions have been proposed to overcome these limitations [START_REF] Perez | The effectiveness of data augmentation in image classification using deep learning[END_REF]. A general and widely accepted solution is to impose meaningless perturbations to the original data [START_REF] Goodfellow | Deep learning Book[END_REF] or to apply more advanced techniques, like rotation, reflection, scaling among others. These approaches offer straightforward alternatives for augmenting the training set, allowing DL models to reach better performance and/or more stable training [START_REF] Shorten | A survey on image data augmentation for deep learning[END_REF]. Recently, with the rise of DL, interesting alternatives have appeared and new generative DL-based models were proposed to obtain synthetic data with characteristics spanning the original data manifold [START_REF] Shorten | A survey on image data augmentation for deep learning[END_REF]. Therefore, in this study we refer to generative models as a subclass of DL frameworks able to generate complex data structure, including the recent modeling approach used to characterize brain networks by means of graph theory [START_REF] Baldassarre | Brain connectivity and neurological disorders after stroke[END_REF][START_REF] Rezazadeh | Brain connectivity in autism spectrum disorder[END_REF][START_REF] Guo | Deep neural networks on graph signals for brain imaging analysis[END_REF]. Given the great capability of graphs to represent complex relations among different areas of the brain, such relational data structure, started to be widely employed in many contexts, including social and behavioral studies. Additionally, advances in brain image acquisition and computer assisted methods have begun to provide meaningful results in support of clinicians, leading to a steadily growing use in the neuroscience community, particularly in brain imaging [START_REF] Rafael | How artificial intelligence is supporting neuroscience research: A discussion about foundations, methods and applications[END_REF]. Using magnetic resonance imaging (MRI), functional or structural brain connectivity can be obtained by analyzing temporal correlations of gray matter (GM) activity with resting-state functional MRI (fMRI) or reconstructing white matter (WM) fiber-bundles with diffusion tensor imaging (DTI), respectively. Such network-like structure of the human connectome consists of nodes, defined by parcellation of the brain grey matter (GM), and edges, corresponding to functional or structural links between the network nodes. These new approaches paved the way for a better characterization of brain networks, particularly in brain diseases such as Multiple Sclerosis (MS). The neurologist's challenge is to predict the disease evolution based on early clinical, biological and imaging markers available from disease onset. However, the complexity brought by connectome data is more cumbersome with respect to the grid-like pixel-bypixel representation found in images. In fact, due to the multiple interconnections between different nodes, connectome data represent a challenge for synthetic data generation for which simple operations, like edge swapping, would end up changing the entire structure of the graph network, jeopardizing the information they convey. [START_REF] Verma | Graphmix: Regularized training of graph neural networks for semi-supervised learning[END_REF]. In this study, forty-eight MS patients distributed across the two most frequent clinical courses, namely the RRMS course, which is followed, between 10 to 20 years later, by the SPMS course, were used. To demonstrate the advantage of our method, the dataset used in the present analysis was downsampled in the minority class based on the clinical characteristics highlighted in Table 4.1. A generative adversarial network framework, namely Generative Adversarial Neural Network AutoEncoder (AAE), is proposed for the generation of new synthetic structural brain connectivity data of MS patients. To achieve this, a prior is imposed to the latent space of the autoencoder network by means of an adversarial model. Moreover, a consistency loss is also introduced in order to increase the stability of the training process. New samples of brain connectivity data are generated by drawing from the parametrized latent space. An overfitting analysis over generated graphs, by exploiting graph properties, is proposed for model evaluation. The synthetic generated data can be used to augment the MS brain networks dataset to improve classification performances of classical machine learning methods like the Random Forest Classifier.

The paper is structured as follows. In Section 5.2, we illustrate the related literature, and in section 5.3, we provide a detailed description of our methodological approach. In Section 9.4, we describe our experimental results and finally, in Section 5.5, we draw our conclusions.

Related Work

Due to their ability to generate new data, generative models have gained a lot of interest in the computer vision and medical imaging research communities. The Generative Adversarial Network (GAN) framework has been previously used for generating realistic training images that synthetically augment datasets. Radford et al. [START_REF] Radford | Unsupervised representation learning with deep convolutional generative adversarial networks[END_REF] introduced a class of generative models called deep convolutional generative adversarial networks (DCGAN), to generate 2D brain MR images followed by an AutoEncoder (AE) neural network for image denoising. Makhzani et al. [START_REF] Makhzani | Adversarial autoencoders[END_REF] proposed a new method for regu-Berardino BARILE larizing AE by imposing an arbitrary prior on the latent representation. Calimeri et al.

[184] proposed a GAN for the automatic generation of artificial MR images of the human brain. They demonstrated that the power of adversarial training could be exploited for the generation of brain networks data, which are more complex than usual images. GAN frameworks have also shown to improve accuracy of image classification via generation of new synthetic training images. Frid-Adar et al. [START_REF] Frid-Adar | Gan-based synthetic medical image augmentation for increased cnn performance in liver lesion classification[END_REF], for instance, used synthetic medical image augmentation with GAN for the classification of liver lesions. Similarly, Salehinejad et al. [START_REF] Salehinejad | Generalization of deep neural networks for chest pathology classification in x-rays using generative adversarial networks[END_REF] used this framework to simulate pathology across five classes of chest X-rays in order to augment the original imbalanced dataset and improve the performance of a convolutional model in chest pathology classification. In the context of MS, Shui-Hua W. et al. [START_REF] Shui-Hua | Densenet-201-based deep neural network with composite learning factor and precomputation for multiple sclerosis classification[END_REF] proposed a new transfer-learning-based approach to identify MS patients with higher accuracy, comparing three different types of neural networks (DenseNet-121, DenseNet-169, and DenseNet-201), which make use of composite learning factors to different layers. Yu-Dong Z. et al. [START_REF] Yu-Dong | High performance multiple sclerosis classification by data augmentation and alexnet transfer learning model[END_REF] exploited the AlexNet model to classify MS patients and studied the best transfer-learning settings (i.e. number of layers transferred and replaced) obtaining high level of performance. Interestingly, other applications of adversarial variational training frameworks have been reported. For example, Zhang et al. [START_REF] Zhang | Adversarial variational embedding for robust semi-supervised learning[END_REF] proposed a semi-supervised learning Adversarial Variational Embedding for leveraging both the power of GAN as a high quality generative model and Variational AutoEncoder (VAE) as a posterior distribution learner. They demonstrated that the combination of VAE and GAN provided significant improvements of semisupervised classification. Imran et al. [START_REF] Imran | Multi-adversarial variational autoencoder networks[END_REF] used a network architecture that incorporates an ensemble of discriminators in a VAE-GAN network using datasets from the computer vision and medical imaging domains in order to generate new realistic images of medical data. They showed that the combination of this two generative models can lead to superior performances against state-of-the-art semi-supervised models both in image generation and classification tasks. However, the generation process become more cumbersome in the case of highly structured graph data. In order to address this challenge, many approaches have been reported. Chawla et al. [START_REF] Chawla | Smote: Synthetic minority oversampling technique[END_REF] proposed a GraphVAE method for generating small graphs using a Variational approach. This model is composed by a simple linearized decoder output, which produces a probabilistic fully-connected graph. Pan et al. [START_REF] Pan | Adversarially regularized graph autoencoder for graph embedding[END_REF] proposed a new architecture for which an adversarial training is combined with a graph autoencoder structure (ARAE). The framework encodes the topological structure and node content in a graph to a compact representation, on which a decoder is trained to reconstruct the graph structure. Freund et al. [START_REF] Feng | Graph adversarial training: Dynamically regularizing based on graph structure[END_REF] proposed an approach based on adversarial regularization of the latent space for generating graph structured data. They could demonstrate the ability of the model to embed graph-based data coherently, and at the same time, generate meaningful samples. Thus, Graph AE and VAE constitute today the best approach for embedding nodes and learn a low dimensional vector representation with applications to link prediction, node clustering and matrix completion. However, much less attention has been spent on generating the entire structure of graphs. Khoshgoftaar et al. [START_REF] Khoshgoftaar | An empirical study of learning from imbalanced data using random forest[END_REF] proposed a simple graph AE structure which does not use the graph convolutional network. They demonstrated that a straightforward linear model with adjacency matrices as inputs performed equally well in benchmark datasets like Cora, Citeseer and Pubmed citation networks.

Methods

Generative Adversarial Neural Network

GAN is a generative model approach based on differentiable neural networks where two actors are involved: a Generator [G q θ (z|x) (x)] and a Discriminator [D(v)] [START_REF] Goodfellow | Deep learning Book[END_REF]. The former is a neural network mapping the input x to the output z by training a network with structure q and parameters θ. In most applications, brand new data are generated by defining a prior on input noise variables. The latter is a network which takes as input v and outputs the probability that the input is coming from the true data distribution instead of being synthetically generated [184]. Formally, the adversarial game can be defined as a min-max problem following Eq. (5.1).

min G max D = E v∼p(v) [logD(v)] + E z∼p(z) [1 -logD(G q θ (z|x) (x))] (5.1)
Here, the first term represents the discriminator network's probability that true instances v from distribution p(v) are rightly classified. The second term in the summation, identifies the generator network's ability to fool the discriminator by producing data with probability distribution p(z) indistinguishable from that of the true data.

Generative Adversarial Neural Network Autoencoder

In this study, the adversarial training is used to train the proposed AAE model at generating synthetic structural brain networks. Fig. 5.1, illustrates the adversarial process.

The structure of the AAE model is defined by two adversarial neural networks: the generator and the discriminator. The former is an autoencoder composed by 13 layers for which fully connected and batch normalization layers alternate between one another except for the output layer. The input layer of the encoder is the number of upper triangular nodes Berardino BARILE Figure 5.1: Schematic representation of the proposed AAE model. Starting from the brain connectome data representation (adjacency matrix), conditional probability distributions were calculated, from which new batches of connectome data were sampled. From the vectorized representation of the sampled adjacency matrix, the encoder network compresses the input into a latent lower dimensional representation, while the decoder reconstructs the input from its compressed latent representation. The combination of the two networks defines the autoencoder generator of the adversarial framework. Conversely, the discriminator network takes as input the latent representation and a random noise vector and tries to discriminate between the two, effectively imposing a constraint on the latent distribution of the autoencoder. Finally, from the latent space, an additional classifier discriminates between RRMS and SPMS patients. in the graph (d = 3486). Subsequent fully connected layers have a number of neurons of 512, 256, 128, 100. This last encoder layer (q θ (z|x)) maps the input vector x ∈ R (1,d) to a lower dimensional space z ∈ R (1,c) with c = 100. The decoder p φ (x|z) is defined as a mirror representation of the encoder with the aim of reconstructing the original input. Furthermore, the encoder is provided with an additional branch, a fully connected layer with a single neuron, used as regularization with respect to the clinical form.

On the other hand, a second neural network is introduced, which takes two inputs. The first is a random standard gaussian vector v ∈ R (1,c) with c = 100 where:

v ∼ N (µ, σ 2 ) = 1 √ 2π e -z 2 2 (5.2)
with µ = 0 and σ = 1. The second is z ∈ R (1,c) obtained as the output of the encoder q θ (z|x). The second model (discriminator) produces a probability score, which defines the likelihood that the two input vectors are coming from the same underlying data distribution. Its architecture is composed by 6 layers in which fully connected and dropout layers alternate between one another. The LeakyReLU activation function with an alpha parameter of 0.2 is used for all of the middle layers in both the generator and the discriminator while for the output layers a sigmoid activation function is employed. Only for the generator, batch normalisation with momentum 0.8 is added after each feedforward layer except for the output layer. For the discriminator, dropout with parameter 0.2 is used between each layer excluding the output layer. Finally, the encoder model q θ (z|x) is connected to a second mirrored model p φ (x|z), the decoder, whose objective is to learn the inverse mapping function of the encoder. In order to maximize the reconstruction quality of the input data, an additional penalty is imposed to final loss function (5.3).

M SE = 1 kn n i=1 k j=1 x ij -p φ (x ij |z ij ) σ ij 2 (5.3)
where k defines the total number of possible connections in the upper triangular of the connectivity matrix, while n defines the size of the batch used for training the network. In other words, we penalized the loss function each time the reconstructed matrix is far from the original data matrix in terms of mean squared error (MSE). This constraint ensures that while the hidden space z is forced to follow a standard normal distribution, the output of the model will produce results that span the entire input space. Moreover, in order to improve the classification performance we imposed an additional form of regularization which bind the latent space to be "coherent" [START_REF] Jun-Yan | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF] between the encoder and the decoder network. Mathematically, this translates in Eq. (5.4):

M SE Coherence = 1 cn n i=1 c j=1 q θ (z|x) -Ψ θ ij IQR 2 (5.4) Berardino BARILE 123 with IQR = Q 3 -Q 1
where Q 1 and Q 3 are respectively the first and the third quartile of the distribution given by q θ (z|x) -Ψ θ where

Ψ θ ij = q θ (z ij |p φ (x ij |q θ (z|x ij )).
As long as the adversarial loss is concerned, let D(v) be the discriminator network, where v ∼ N (µ = 0, σ 2 = 1) = p(v), is a standard normal distribution. The related loss function will thus be defined as E v∼p(v) [logD(v)] for positive cases and E z∼p(z) [1 -logD(G q θ (z|x) (x))] for negative cases. In this last case, the generation of the latent space z is defined as G q θ (z|x) (x) with z ∼ q θ (z|x) = p(z). We would like that p(z) ≈ p(v), which implies that the latent space is distributed as a standard gaussian. On the contrary, we define with G(x) the final generator (composed of an encoder and a decoder) and its respective loss function as E z∼q θ (z|x) [log p φ (x|z)]. Henceforth, the adversarial loss will be defined as shown in Eq. (5.5).

L(θ, D(v), G q

θ (z|x) (x)) = E z∼q θ (z|x) [log p φ (x|z)]+ E v∼p(v) [logD(v)] + E z∼p(z) [1 -logD(G q θ (z|x) (x))] (5.5)
Roughly speaking, the Kullback-Leibler loss, usually employed in a VAE framework [START_REF] Doersch | Tutorial on variational autoencoders[END_REF], is now substituted with the adversarial loss. This model allows us to provide probabilistic descriptions of observations in latent space, which translates in the ability of the model to store latent attributes as probability distributions. In order to take into account the clinical form for each graph, an additional constraint has been imposed and defined as follows:

L CrossEntropy = - C j=1 y i,j log exp(a i ) j exp(a j ) (5.6)
where C is the number of the clinical forms.

The final loss function to optimize is thus obtained by summing up all the losses as defined in Eq. (5.7).

L f inal = L(θ, D(v), G q θ (z|x) (x)) + M SE M odel + M SE Coherence + L CrossEntropy (5.7)
As long as the parameters used for training the AAE are concerned, 500 iterations with a batch size of 64 are used for training the generator and the discriminator in an alternating fashion. The process terminates when the capability of the discriminator to distinguish synthetic samples from true samples remains stable approximately around 50%. The Adam optimizer is used for both models while the learning rate (lr), imposed for the discriminator, was chosen to be 10 times smaller than the generator (lr = 0.001). These settings, have been empirically observed to lead to a more stable training of the adversarial network, providing better results.

AAE Adversarial Training Pipeline

The whole pipeline for training the GAN framework and generating synthetic structural brain networks is summarized in Fig. 5.2. Generally speaking we can divide the entire workflow in three main phases: i) Training the AAE model ii) Using the AAE model for generating synthetic MS structural brain networks iii) Data augmentation for MS clinical form classification.

Training the AAE model

In order to properly train the proposed AAE model, a naïve data augmentation procedure is needed. The original dataset is split in training and test set by a leave-one-subjectout cross validation strategy (step 1). The training set was then exploited to calculate the conditional probability distribution defined in Eq. (5.8):

P (X = v|Y = y, Q = q) = n v p v iyq (1 -p iyq ) n-v (5.8)
Here, p iyq defines the probability of an edge i to be present in the vectorized representation of the upper triangular matrix x ∈ {0, 1} (1,d) with dimensionality d and i ∈ [0, d] given a class label y and a degree quantile q. The letter v defines the number of times the edge i is present and n the number of trials (number of subjects drawn).

It is worth noticing that outlier probabilities can be present. In fact, given that our training set is only the realization of a stochastic process, the fact that two regions are always (p iyq = 1) or never (p iyq = 0) connected might not be true in general. For example, it can be due to lack of data or biases in the collected dataset. To overcome this issue Eq. (5.9) is applied:

P (X|Y, Q) =      0.95, if p iyq > 0.95 0.05, if p iyq < 0.05 ∀ i, y, q (5.9)
It is important to notice that the described anomalies represents a tiny fraction of the total number of connections (≪ 1%) and do not mine the final result of the work. Yet, the operation is useful for a better generalisation capability and avoid overfitting.

The calculated probability density function (pdf) is then used as a "stamp" from which sampling new batches of data at each iteration (step 2 and 3). More in detail, given a class label y and a degree quantile q, a new vectorized representation of an upper triangular matrix can be obtained x ∈ {0, 1} (1,d) , where x i is assigned 1 with probability p iyq or 0 with probability 1p iyq . Here, x i denotes the presence or the absence of a connection Berardino BARILE in the i-th edge of the y-th MS class in the q-th degree quantile. Finally, a continuous transformation of the connectivity matrices is applied as following (Eq. (5.10)):

x i =      x i + U (0.01, 0.05), if x i = 0 x i * U (0.95, 0.99), if x i = 1 (5.10)
where U is the continuous uniform distribution. In other words, a noise component is introduced to the newly generated connectivity matrix. This strategy is usually implemented in the context of an adversarial model and it has been shown to provide more stable training [START_REF] Sønderby | Amortised MAP inference for image super-resolution[END_REF]. This naïve procedure for extracting new instances from the underlying likelihood distribution provides a double advantage: first, several samples can be generated, thus addressing the problem of limited training size. Second, the obtained instances resemble, at each iteration, the percentile distribution of the original dataset. Henceforth, this second advantage is important to overcome the problem of mode collapse, which constitute a major challenge when it comes to training an adversarial network [START_REF] Baldassarre | Brain connectivity and neurological disorders after stroke[END_REF]. It is worth noting that this data augmentation technique is only used to train the AAE model but cannot be implemented for actual MS connectivity data augmentation. Indeed, this naïve method is not able to produce enough qualitative results, due to the hypothesis of conditional independence imposed between pairs of nodes inside the graph. However, the resampling strategy guarantees that the dataset effectively used to train the adversarial framework is perfectly balanced with respect to a-priori information of the clinical profile and graph density that we are interested in generating.

Using the AAE model for generating synthetic MS structural brain networks

Once the model is trained, we are ready for the second phase in which the synthetic graphs are generated by sampling new instances from a standard gaussian distribution (Figure 5.2 step 4), with shape R (1,c) , and then used to feed the already trained decoder, obtaining new realistic connectome data.

MS clinical form classification

Finally, in the third phase, the synthetic dataset was used to augment the original training set (step 5) and fed to a classifier (step 6) in order to enhance the classification performance of MS clinical profiles (step 7). The predicted labels obtained from the classifier were compared with the left-out samples for performance evaluation. It is important to notice that neither the training set nor the test set were ever used by the adversarial 126 Berardino BARILE 5.4. RESULTS model to generate synthetic data. The proposed pre-processing approach thus reduces the overfitting tendency of the generative model.

Experimental Protocol

In this section, the results obtained by evaluating both the structural property of the connectivity matrices and their respective graph-derived metrics are reported. In order to perform the evaluation, a sample of data from a random normal distribution v ∼ N (µ = 0, σ 2 = 1) was drawn, where v ∈ R (1,c) and c = 100. N defines the number of samples to be generated and passed to the decoder p φ (x|z) to obtain a new sample of synthetic data. Finally, we demonstrate the usefulness of our approach by improving the classification performance of MS clinical forms, even in presence of strong imbalance between classes. Classical approaches, like ROS and the more efficient SMOTE [START_REF] Chawla | Smote: Synthetic minority oversampling technique[END_REF] are used for comparisons as well as the more recent adversarial model ARAE [START_REF] Pan | Adversarially regularized graph autoencoder for graph embedding[END_REF]. The performance metrics used for the evaluation are F 1 score , Precision and Accuracy which are defined in Eq. (5.11, 5.12, 5.13) respectively.

F 1 score = 2 • T P (T P + F P + F N )
(5.11)

P recision = T P T P + F P

(5.12)

Recall = T P T P + F N (5.13)
where the abbreviations TP, TN, FP, FN represent the True Positive, True Negative, False Positive and False Negative of instances respectively.

From now on, we refer to the connectivity graphs generated through the adversarial network as synthetic data, while the original dataset will be labelled as true data.

Results

Comparison of Data Augmentation Methods for MS Classification

The classification task was performed with a Random Forest Classifier (RF) with 100 trees, due to its robustness to overfitting and unbalanced dataset. Table 5.1 reports the average classification performances (with standard errors) between our method and the three oversampling techniques previously introduced. Fig. 5.3 shows the corresponding confusion matrices. Compared to the true unbalanced data used as reference, our method obtained higher performance, reaching an F 1 score of 81% instead of 65.7%. The ROS and SMOTE methods provided a score of 65.8% and 72.3% respectively, while the ARAE model reports a value of 70% showing a marginal improvement over the unbalanced baseline.

Evaluation of Synthetic Data Based on Graph Matrices

We want to evaluate both the coherence and the difference between true and synthetic data. Ideally, we aim at producing synthetic graphs that span the entire range distribution of the true data sample. In order to evaluate the properties of synthetic graphs, an equal number of data were generated with respect to the true data samples obtaining a perfectly balanced dataset. For both true and synthetic data, the global assortativity degree metric was calculated along with a percentile distribution of 1% width. In other words, the distributions of true and synthetic data were computed with the highest degree of precision following the idea that larger bandwidth will produce less precise comparisons by smoothing the distributions and providing a too optimistic result. In order to measure the distance between the two distributions, the mean absolute point-wise deviation was calculated (77.82 ± 46.07). The overlap proportion (OP) between the two distributions (true vs synthetic) is calculated using Eq. (5.14). [START_REF] Van Der Maaten | Visualizing high-dimensional data using t-SNE[END_REF][START_REF] Van Der Maaten | Learning a parametric embedding by preserving local structure[END_REF]. This algorithm applies a non-linear transformation to the original multidimensional data. It performs an embedding of data, mapping them in a lower dimensional space. Specifically, the algorithm ensures that, each high-dimensional element is mapped to a lower dimensional space in such a way that similar objects are modelled by nearby points and dissimilar objects are modelled by distant points with high probability [START_REF] Van Der Maaten | Visualizing non-metric similarities in multiple maps[END_REF][START_REF] Van Der Maaten | Accelerating t-sne using tree-based algorithms[END_REF]. Fig. 5.4 illustrates the embedded representation of the synthetic and true data. From the image, it can be observed that the two groups are fairly similar. It is worth to note that in the t-SNE procedures, the perplexity parameter dictates the shape of the mapping function. For this reason, multiple evaluations of this parameter has been performed using value from 10 to 60 at 10 units increment. In Fig. 5.4 the t-SNE results are illustrated for a perplexity parameter of 30. It should be noticed that the author pointed out that as far as the perplexity parameter remains in the usual range [START_REF] Badeau | Multilinear singular value decomposition for structured tensors[END_REF][START_REF] Garyfallidis | Quickbundles, a method for tractography simplification[END_REF] the model is quite robust [START_REF] Van Der Maaten | Visualizing high-dimensional data using t-SNE[END_REF]. In addition to the t-SNE representation, the embedding of the true and synthetic graphs was also performed using the Graph2Vec algorithm [START_REF] Narayanan | graph2vec: Learning distributed representations of graphs[END_REF], which is optimised for Berardino BARILE It is a transductive neural embedding framework used to learn from data-driven distribution representations of arbitrary sized graphs. This framework ensures that structurally similar graphs are represented close to one another, while dissimilar graph are depicted far apart. In other words, the model is able to preserve the first and second order proximity. The former is the local pairwise similarity between nodes linked by edges, while the latter indicates the similarity of the nodes neighbourhood structures. In order to numerically compare the true and synthetic data, the F 1 score metric was computed by linearizing the upper triangular part of the binary adjacency matrices (A t , A s ∈ R (1,d) respectively with d = 3486). Each true adjacency matrix is compared with every synthetic vector. A minimum F 1 score value of 63% and a maximum value of 82% was obtained with an average score of 76%.

OP =

Evaluation of Synthetic Data Based on Graph Features

Analyzing the metrics of synthetic graphs is important to capture meaningful features characterizing the structural brain connectome. For this purpose, six graph-based global [START_REF] Beauchene | Comparing brain connectivity metrics: a didactic tutorial with a toy model and experimental data[END_REF], and could provide a reliable measure of the quality of generated data.

In Fig. 5.6, the boxplot distribution between true and synthetic data is presented for each metric. Comparable values of median and interquartile range are observed for the two distributions.

As in the previous section, the t-SNE analysis has been repeated varying the perplexity parameter in the range between 10 and 60 at steps of 10. In Fig. 5.7, the t-SNE embedded representation shows similar distributions between true and synthetic data without obvious discrepancies.

In order to assess the similarity between the two groups, an additional evaluation based on the Kernel Density Estimation (KDE) function was performed. The distributions of true and synthetic datasets were estimated by means of the KDE function and their likelihoods were compared. This approach was originally introduced by Breuleux et al. [START_REF] Breuleux | Quickly generating representative samples from an RBM-derived process[END_REF] and applied in the context of generative adversarial networks in two reports [184,[START_REF] Goodfellow | Generative adversarial nets[END_REF]. The method estimates the probability of the synthetic data, by fitting a Gaussian Parzen window to the generated samples and reports the likelihood under this distribution. The bandwidth of the Gaussian window is obtained by cross-validating the training data. Afterword, the similarity between the two datasets has been computed by estimating the pdf of the KDE estimation, so that similar datasets could be represented by similar distributions. Value of 2773.66 and 2657.13 were obtained respectively by comparing the log-likelihoods for the true and synthetic data. This result suggests, once more, that the two groups of data are similar. Indeed, the two cumulative functions (true in blue and synthetic in orange) are rather close to one another (Fig. 5.8). Furthermore, they both follow approximately a straight monotonic-increasing path, which means that the probability mass is evenly distributed across all data samples in both groups.

As an additional test, the bandwidth value from 0.1 to 1 at steps of 0.1 has been increased in order to evaluate the robustness of our results. An increase in performances for every value greater than 0.1 (best cross-validation) has been observed. Finally, in order to offer a sample visualization of the true and synthetic connectivity matrices, Fig. 5.9 provides two visual examples from the RR and SP clinical forms. It is possible to notice that the true and synthetic data are very similar. results obtained when the coherence loss in Eq. (5.4) was added or excluded from the final objective function, respectively. Less spikes and noise are noticeable in the former case compared to the latter, implying a higher degree of stability of the adversarial training.

Evaluation of MSE Coherence

Discussion

In this work, an approach for generating new structural connectivity matrices of MS patients was presented. In a context of imbalanced data, the proposed framework was able to up-sample the minority class producing a much higher F 1 score (81%) with respect to the baseline unbalanced classification (66%). Furthermore, comparing to other classical oversampling techniques (ROS and SMOTE) and a graph-based adversarial network (ARAE), our method increases the classification performance by approximately +10%. The improvement can be related to the capability of our method to generate more biologically plausible connectomes that can better represent the different clinical forms. One of the possible explanations can be related to the additional classifier branch used as regularized factors. Indeed, it can help to preserve meaningful structural information which characterizes the different clinical forms.

Our method was evaluated by comparing true and synthetic data by means of visual and analytical techniques. In fact, the generated data should meet two requirements in order to be valid: first, they should preserve similar structural characteristics as the ones which can be observed in true MS brain networks and second, the new generated brain networks should not be simply copies of the original dataset (overfitting). In other words, while new synthetic graphs with enough diversity in terms of structure and properties need to be generated, they still have to be plausible and lie inside the manifold of the true data. In this work, we showed that both distributions (true and synthetic) were observed to be different but very similar to one another. Indeed, the average F 1 score obtained by comparing the binary adjacency matrix representation, is 76% (Range 63% to 82%). This means that a significant portion of the generated graphs is completely different with respect to the distribution of the true data samples. Notwithstanding, the boxplot comparison shown in Fig. 5.6 confirmed our visual observation of Fig. 5.7. The mean distributions of true and synthetic graph metrics are very close. However, one can notice a substantial variability in the synthetic group. From all this evidence the likelihood of overfitting the training set, by simply generating duplicates, is negligible. Instead, experiments highlighted the diversity of the synthetic samples, which lies inside the manifold of the true data, demonstrating that completely new instances have been generated. Moreover, the actual training set was never seen by the adversarial framework which was trained sampling from the conditional distribution proposed in Section 5.3.3, thus reducing the chance of overfitting even more. Once completed, the model will allow to increment the size of the available dataset and thus perform a much robust training of machine learning algorithms even with limited amount and unbalanced data, which constitutes a common scenario in the medical field. In addition, by learning the complete underline data distribution, it is possible to perform meaningful bayesian statistical testing of hypothesis as well as generating graphs with desired characteristics. Finally, the combination with other embedding methods, which learn a meaningful representation of single nodes, is of great interest. This study exploits the advantage of the AAE framework in the context of brain graphs data generation and can be easily expanded for the analysis of other brain diseases or other related domains. Some limitations should be also mentioned. First, our dataset represents structural connectivity matrices in which connections are binarized, discarding the valuable information conveyed by weighted graphs. Second, due to its simple architecture, our method will not be efficient for very large graphs. Moreover, in order to sample random batches from Eq. (5.8), in this work the clinical class and the degree percentile were used for conditioning. In fact, the limited amount of data did not allow to add other covariates like age and gender which are worth exploring if one has a large enough sample size. Finally, the computational time needed to perform the leave-one-subject-out cross-validation is not negligible since for each subject the AAE model has to be trained. However, in the context of brain network analysis, we rarely deal with much larger networks as the actual MRI data are limited to a maximum of a few hundred nodes, justifying the simplification proposed in this study. Additionally, our method performs well even with a limited in number and strongly imbalanced data, in agreement with a previous report [START_REF] Khoshgoftaar | An empirical study of learning from imbalanced data using random forest[END_REF].

Conclusion

In this study, a new data augmentation approach for connectome dataset was presented. Given the capability of graphs to represent complex brain networks, our approach provides a new tool for biomedical application, a domain in which the data availability is scarce and poorly behaving. Therefore, our connectome-based data augmentation approach represents a promising alternative to usual image-based techniques. Furthermore, the proposed data augmentation approach was capable to improve the MS classification performance even in cases of unbalanced data scenario.

Introduction

An important issue in neuroscience is the characterization of human brain structure and function, and their alteration in neurodegenerative diseases. In Multiple Sclerosis (MS), which is an immune-mediated inflammatory disease of the Central Nervous System (CNS), the axonal myelin as well as the axon are damaged in variable degrees. These two pathological processes lead to local White Matter (WM) and Gray Matter (GM) lesions [START_REF] Ghasemi | Multiple sclerosis: Pathogenesis, symptoms, diagnoses and cell-based therapy[END_REF] as well as to global GM atrophy that can evolve in a non-uniform pathway, meaning that some regions are more susceptible than others [START_REF] Preziosa | Progression of regional atrophy in the left hemisphere contributes to clinical and cognitive deterioration in multiple sclerosis: A 5-year study[END_REF][START_REF] Steenwijk | Cortical atrophy patterns in multiple sclerosis are non-random and clinically relevant[END_REF]. MRI detection of T2-weighted (T2w) lesions, mainly related to demyelination and inflammation, provides an accurate tool for diagnosis, through the assessment of their spatial and temporal distribution, and it represents the main imaging approach for monitoring brain damages in MS patients [START_REF] Craddock | Imaging human connectomes at the macroscale[END_REF][START_REF] Polman | Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria[END_REF]. More recently, deep learning methods demonstrated their good performances for lesion detection and segmentation [START_REF] Gabr | Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study[END_REF][START_REF] Jain | Two time point MS lesion segmentation in brain MRI: an Expectation-Maximization framework[END_REF][START_REF] Mckinley | Automatic detection of lesion load change in multiple sclerosis using convolutional neural networks with segmentation confidence[END_REF] and enabled the automatic classification of clinical profiles [START_REF] Tousignant | Prediction of disease progression in multiple sclerosis patients using deep learning analysis of MRI data[END_REF]. However, the moderate correlation of lesion load measurements with patients' disability remains an issue. More specific MRI acquisition methods such as magnetization transfer, spectroscopy, and Diffusion Tensor Imaging (DTI) have been developed to overcome this problem. For instance, DTI has demonstrated to be very sensitive for the detection of microscopic alterations occurring in normal-appearing WM [START_REF] Tae | Current Clinical Applications of Diffusion-Tensor Imaging in Neurological Disorders[END_REF] or in subcortical GM [START_REF] Hannoun | Correlation of diffusion and metabolic alterations in different clinical forms of multiple sclerosis[END_REF]. However, DTI acquisition remains limited in clinical routine. Although GM lesions have been known to occur in MS [START_REF] Calabrese | Grey matter lesions in MS: from histology to clinical implications[END_REF][START_REF] Geurts | Grey matter pathology in multiple sclerosis[END_REF], more recent immunohistochemical studies have provided additional compelling evidence that GM alteration in MS is extensive and clinically relevant [START_REF] Lassmann | Multiple Sclerosis Pathology[END_REF]. More recently, GM lesions have been evidenced by high field MRI [START_REF] Bruschi | Ultra-high-field 7-T MRI in multiple sclerosis and other demyelinating diseases: from pathology to clinical practice[END_REF][START_REF] Tallantyre | A comparison of 3T and 7T in the detection of small parenchymal veins within MS lesions[END_REF]. Also, GM atrophy is known to occur early in MS [START_REF] Fisher | Gray matter atrophy in multiple sclerosis: a longitudinal study[END_REF] and to be associated with MS motor deficits and cognitive impairment [START_REF] Klaver | Grey matter damage in multiple sclerosis: A pathology perspective[END_REF]. Graph theory represents a new important approach for characterizing the brain networks given its capability to represent complex relations among different areas of the brain [START_REF] Rubinov | Complex network measures of brain connectivity: Uses and interpretations[END_REF][START_REF] Guo | Deep neural networks on graph signals for brain imaging analysis[END_REF]. Multiple studies have already demonstrated the impact of graph network analysis for the investigation of brain diseases by using both local and global information of brain connectomes [START_REF] Brown | Machine learning on human connectome data from mri[END_REF][START_REF] Stam | The organization of physiological brain networks[END_REF]. Additionally, functional and structural brain connectivity can be obtained by means of advanced MRI techniques, such as resting-state functional MRI [START_REF] Bullmore | Complex brain networks: graph theoretical analysis of structural and functional systems[END_REF][START_REF] Jirsa | Book series: Understanding Complex Systems (UCS)[END_REF][START_REF] Rubinov | Complex network measures of brain connectivity: Uses and interpretations[END_REF][START_REF] Sporns | Organization, development and function of complex brain networks[END_REF] and DTI [START_REF] Stam | The organization of physiological brain networks[END_REF], respectively. More recently, Machine Learning (ML) methods were applied to connectome data and demonstrated good level of performance for the classification of MS courses when DTI based structural connectivity was used. Marzullo et al. [150] demonstrated that dynamic functional and structural connectome metrics outperformed results obtained from conventional MRI clinical data when distinguishing MS patients by impairment level. However, fMRI and DTI data are constrained by the long acquisition time and the need for complex processing techniques, limiting its applicability in clinical practice. Nevertheless, such network-like structure of the brain connectome can also be obtained based on conventional MRI, by measuring different morphological metrics of the GM on T1-weighted images [START_REF] Raamana | graynet: single-subject morphometric networks for neuroscience connectivity applications[END_REF]. In such graphs, nodes represent cortical areas obtained from the GM parcellation, while the edges represent a degree of dissimilarity between two GM regions. In particular, the morphological properties of the GM thickness have been investigated in the literature, demonstrating that the coordinated variations in human regional cortical thickness, as derived from structural MRI [START_REF] Macdonald | Automated 3-d extraction of inner and outer surfaces of cerebral cortex from mri[END_REF], follow a small-world topology at a macroscale level, characterized by a high degree of local clustering and short pathlengths linking individual network nodes [START_REF] He | Small-world anatomical networks in the human brain revealed by cortical thickness from MRI[END_REF]. Several studies have demonstrated the importance of GM changes and highlighted how the GM loss in one region can affect GM loss in other connected regions [START_REF] Mueller | The gray matter structural connectome and its relationship to alcohol relapse: Reconnecting for recovery[END_REF][START_REF] Naze | Robustness of connectome harmonics to local gray matter and long-range white matter connectivity changes[END_REF]. More recently, few studies have been using GM networks to better characterize degenerative illnesses, such as Alzheimer's Disease (AD). Dicks et al. [START_REF] Dicks | Single-subject gray matter networks predict future cortical atrophy in preclinical Alzheimer's disease[END_REF] showed that within individuals with preclinical AD, GM network measures predicted hippocampal atrophy rates, whereas other AD biomarkers did not. Hilger et al. [START_REF] Hilger | Predicting intelligence from brain gray matter volume[END_REF] exploited the GM connectome for predicting intelligence and demonstrated that the use of GM information yields significant predictions for the frontoparietal network and the cerebellum. In MS, GM atrophy is well known and the investigation of GM networks has been tackled [START_REF] Durand-Dubief | Reliability of longitudinal brain volume loss measurements between 2 sites in patients with multiple sclerosis: comparison of 7 quantification techniques[END_REF]. Kuceyeski et al. [START_REF] Kuceyeski | Baseline biomarkers of connectome disruption and atrophy predict future processing speed in early multiple sclerosis[END_REF] used a statistical modeling approach to show that GM atrophy is associated with the highest prediction accuracy of the patient's future processing speed in MS. Rovaris et al. [START_REF] Rovaris | Grey matter damage predicts the evolution of primary progressive multiple sclerosis at 5 years[END_REF] demonstrated that GM abnormalities provided an advantage over conventional MRI when studying disease-related accrual of tissue damage, representing one of the main factors associated with more severe disease evolution in PPMS patients. Muthuraman et al. [START_REF] Muthuraman | Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS[END_REF] analyzed GM and WM structural networks in order to classify CIS and RRMS patients using Support Vector Machine model, obtaining good level of accuracy.

In this study, the dataset described in Table 4.1 of Part I of the present Manuscript was used. We propose to first analyze the GM tissue alteration by constructing morphological connectomes from the thickness morphometric feature extracted from the T1w anatomical image. Second, graph characterization was performed based on six of the most important graph metrics describing the topological behaviour of the connectome. Third, statistical and ML analysis was performed in order to investigate the discrimination power of morphological graphs between MS clinical profiles. To our knowledge, this is the first attempt Berardino BARILE to characterize MS patients brain networks based on their GM atrophy and to perform automatic classification of all MS clinical profiles while taking into account the thresholding impact on the topological architecture of the GM network. A clear statistical analysis of the thresholding role for GM connectome binarization is lacking in the literature. In this study, a statistical procedure was implemented in order to fulfill this gap and better visualize global graph metrics variability at different thresholding values. Additionally, a complete study of the thresholding impact on the aforementioned graph metrics was performed in order to evaluate the change in the topological structure of the brain network. Finally, in order to evaluate the predictive power of the GM connectome, a ML analysis was also performed, and four ML classification models were trained to perform binary classification tasks of MS patients and subsequently combined in an ensemble model in order to enhance performances.

Methods

Morphological Connectivity Generation

Starting from 3D T1-weighted images, several preprocessing steps were applied to the anatomical image of each patient, including skull stripping, motion correction, registration to a standardized template, non-brain tissue removal, Talairach transformation, segmentation, intensity normalization, tessellation of the GM-WM boundary, topology correction, and surface deformation. Fig. 6.1 provides an intuitive representation of the pipeline used for constructing the GM connectome using thickness as morphometric feature.

The preprocessing of the anatomical images, including cortical and sub-cortical GM segmentation, was performed using the Freesurfer v6.0.0 image analysis suite [START_REF] Reuter | Within-subject template estimation for unbiased longitudinal image analysis[END_REF] along with the command "recon-all". The flag "qcache" was included in order to calculate GM features, such as GM thickness, using a smoothing parameter "fwhm" equal to 10. A detailed preprocessing pipeline can be found in Hanganu et al. [START_REF] Hanganu | Cortical thickness changes associated with photoparoxysmal response[END_REF]. The GM parcellation task was performed using two different atlases, the FSAverage [START_REF] Fischl | High-resolution intersubject averaging and a coordinate system for the cortical surface[END_REF][START_REF] Rosenke | A cross-validated cytoarchitectonic atlas of the human ventral visual stream[END_REF] and the Glasser2016 [START_REF] Glasser | A multi-modal parcellation of human cerebral cortex[END_REF], providing 68 and 360 brain regions (nodes), respectively. Each voxel was classified into four classes [WM, cortical GM, sub-cortical GM, cerebro-spinal fluid (CSF)] and the cortical thickness morphometric feature was calculated for each brain region. Vertex-wise correspondence was established between subjects by registering each anatomical image to the Freesurfer baseline template. The exact pipeline used to build the morphological connectome was explained in Raamana et al. [START_REF] Raamana | Does size matter? the relationship between predictive power of single-subject morphometric networks to spatial scale and edge weight[END_REF] with the code available at the following Github link: raamana.github.io/graynet/. The reliability of the pipeline has been validated using histological [START_REF] Rosas | Regional and progressive thinning of the cortical ribbon in huntington's disease[END_REF] and manual measurements [START_REF] Kuperberg | Regionally localized thinning of the cerebral cortex in schizophrenia[END_REF] demonstrat-144 Berardino BARILE Figure 6.1: Intuitive representation of the pipeline implemented for morphological connectome data generation. Starting from the magnetic resonance T1w image of a patient, the GM tissue was segmented in specific brain regions based on the parcellated areas obtained from a specific atlas used as template (FSAverage or Glasser2016) during the preprocessing phase. In red and blue the alternative path between the Glasser2016 or the FSAverage atlas is highlighted, leading to either 360 or 68 brain regions. Afterward, the thickness morphometric statistic was calculated for each GM region. Binary comparisons were performed for all possible combinations of GM regions (one to one comparison), employing the Manhattan distance formulation as dissimilarity metric. The results were organized in a squared symmetric adjacency matrix used for the analysis of graph characterization.

ing high performances across different scanner and field of strengths [START_REF] Reuter | Within-subject template estimation for unbiased longitudinal image analysis[END_REF]. Additionally, previous studies have performed similar analysis demonstrating the robustness of the implemented pipeline [START_REF] Hanganu | Cortical thickness changes associated with photoparoxysmal response[END_REF][START_REF] Muthuraman | Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS[END_REF]. The GM morphological connectome of each patient was obtained comparing the morphometric features (i.e. GM thickness) of each brain region, using the Manhattan distance formulation, obtaining a full squared symmetric matrix A ∈ R q×q , where q represents the number of nodes (brain regions). The corresponding graph representation of this adjacency matrix can be defined as G = (V, E, ω) where V defines the set of nodes representing brain regions (|V | = q), E represents the set of edges between these regions (|E| = m) and ω defines the strength of association [START_REF] Raamana | Impact of spatial scale and edge weight on predictive power of cortical thickness networks[END_REF][START_REF] Raamana | graynet: single-subject morphometric networks for neuroscience connectivity applications[END_REF].

Graph Binarization

In order to perform the analysis, the GM graphs has to be binarized to remove the weakest connections generated by the morphometric feature comparison. Henceforth, a percentile threshold 0 < τ < 1 was imposed and a corresponding binarized unweighted and undirected graph G = ( V , E) was obtained, where

| V | = q = |V | and | E| = m ≤ m.
It is worth noticing that the topology and density of the network are affected by the imposed threshold [START_REF] Bullmore | Complex brain networks: graph theoretical analysis of structural and functional systems[END_REF][START_REF] Simpson | Analyzing complex functional brain networks: fusing statistics and network science to understand the brain[END_REF]. In this work, a proportional thresholding approach was applied to determine the binarization value. Mathematically, this is equivalent to impose a function Φ : G → G. Let's define with vec(A i ) the vectorization of the matrix A for scan i of a specific patient and with vec(A : ) + + the concatenation of vec(A i ) ∀i ∈ K where K represents the set of all patients' scans. The total percentile distribution over the vectorized concatenation of all MRI scans can thus be defined as d vec(A) + + . Based on this formulation, a proportional thresholding strategy can be applied on d vec(A) + + by considering percentile values in the range 5% to 95% at steps of 5%, obtaining an absolute threshold value φ. For each scan of each patient, all connections for which ω < φ were selected. Formally speaking, for each percentile threshold τ , a function Φ can be defined as:

Φ =    0, if ω < φ 1, otherwise
For the statistical analysis, we followed the same approach proposed in Kocevar et al. [START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF]. The Coefficient of Variation (CV) was used as measure of variability and it is defined in Eq. (6.1):

CV = 100 x K (x -x) 2 K -1 (6.1) 146 Berardino BARILE 6.3.

STATISTICAL AND MACHINE LEARNING ANALYSIS

where x represents the specific graph metric.

Statistical and Machine Learning Analysis

Classification of MS Clinical Profiles

Four different ML models were considered in this study for the classification task of the four MS clinical profiles (CIS, RRMS, PPMS, SPMS). Specifically, Logistic Regression (LR), Random Forest (RF) [START_REF] Breiman | Random forests[END_REF], Support Vector Machine (SVM) [START_REF] Cortes | Support-vector networks[END_REF] and Adaptive Boosting (AdaBoost) [START_REF] Schapire | Explaining adaboost. Empirical Inference[END_REF] models. The predictive analysis was performed using Python 3.6 programming language along with sklearn package v0.24.2, while networkx v2.2 was used for global graph metrics calculation. The statistical analysis was performed using Stata16 software [START_REF] Statacorp | Stata statistical software: Release 16[END_REF] with the command xtmelogit for GLM model fitting. The four ML models were chosen as they are widely used and studied in the literature, are easy to train and tune with minimal number of hyperparameters to optimize during cross validation [START_REF] Witten | Data mining: practical machine learning tools and techniques[END_REF] and provide built-in feature importance for the prediction task, with the exception of SVM in case a non-linear kernel is used (i.e. Radial Basis Function Kernel). Additionally, simpler models are usually preferred with small datasets in order to avoid overfitting the training set [START_REF] Ghojogh | The theory behind overfitting, cross validation, regularization, bagging, and boosting: tutorial[END_REF]. Moreover, the different underlying assumptions used for each classifier may provide interesting insights into the classification performance. In fact, we observed that, combining these four models, by majority voting, leads to higher classification performances, effectively creating a final binary ensemble classifier. A simple description of these models is proposed in Subsection ??. Additionally, Fig. 6.2 was proposed in order to better understand the pipeline implemented for the classification task.

Statistical Analysis and Model Evaluation

The statistical analyses was performed using Generalized Linear Model (GLM) with logistic link [START_REF] Agresti | An introduction to categorical data analysis[END_REF]. The longitudinal aspect of the dataset was taken into account by considering both mixed and random effects. The evaluation of statistical differences between MS clinical profiles, for each global graph metric, was performed controlling for age and gender as confounding factors, which represent the fixed effects of the model, while the time component was modelled through the random effect. The tests were computed with a level of significance of 5%. Additionally, for the predictive performance analysis, in order to compare the classification results between the two atlases, the non-parametric Wilcoxon matched-pairs signed-rank test was used. For the classification task, the four ML models were trained stratifying patients by means of a nested 10 folds cross-validation strategy for which, at each iteration, 9 folds were subsequently split using the 80%-20% rule for the training and validation sets respectively, while the remaining observations (from the last holdout fold) were used as test set. It is important to notice that if a specific patient was assigned to the testing set, all the corresponding longitudinal scans were also selected in order to avoid biases in the evaluation strategy. Additionally, in order to optimize the models, a grid search strategy was employed. In particular, the following hyperparameters were optimized. For the LR model a regularization parameter in the interval between 1 and 100 was used, while for the RF model, the number of trees were optimized in the range between 10 and 500 and the max depth of each tree between 1 to 10. For the SVM model, a radial basis kernel (RBF) was used due to its good performance, as demonstrated in previous studies [START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF], along with a regularization parameter and kernel coefficient in the range between 1 to 100. Finally, for the AdaBoost model, the number of trees was tuned in the range between 10 and 500, with a learning rate between 0.1 and 3. Additionally, the thresholding value was also considered as an additional hyperparameter to be tuned due to the uncertainty in defining an optimal value in the range between 0.6 and 0.8 as explained in Section 6.2.2. In order to avoid data leakage, the search for the optimal threshold was only applied to the training set. The best parameters found from the grid search were chosen based on the predictive performances obtained on the validation set, 148 Berardino BARILE 6.4. RESULTS while the final results were obtained based on the performances on the hold-out test set and are reported in Section 8.6. The classification performances were evaluated based on the analysis of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) instances classified during the testing phase [START_REF] Gorunescu | Classification performance evaluation[END_REF]. Accuracy, precision, recall, and F 1 score were used as classification measures. Formally speaking, accuracy is defined as the ratio between the number of correct assessments and the number of all assessments ( T P +T N T P +T N +F P +F N ), precision defines the fraction of retrieved instances that are correctly classified ( T P T P +F P ) and recall identifies the portion of positive instances that are correctly identified ( T P T P +F N ). Finally, F 1 score is obtained combining precision and recall and is defined as 2 * P recision * Recall P recision+Recall . Additionally, the Area Under the Receiver Operating Characteristics (ROC-AUC) curve score was also calculated for completeness.

Results

Connectome characterization

Morphological GM connectivity was used with the aim to investigate the statistical differences between MS clinical profiles using six global graph metrics calculated using two different atlases for parcellation. In order to study the topological alteration of the GM network at different thresholding values, the binarization strategy described in Section 6.2.2 was applied and the morphological metrics calculated at each thresholding level. In particular, we are interested in observing the degree of variability (i.e. CV coefficient) along different percentile threshold values (Fig. 6.3). Similar behaviours were observed between the two atlases demonstrating that the variability of the network does not have a direct impact on the topological characteristics of the GM tissue. Considering BC and r as graph metrics, a reduced path was observed over the entire thresholding range, suggesting a minimum CV value τ equal to 0.7 and 0.8 respectively. These metrics play an important role in describing the information flow inside the network and the stability of centralized hubs. For this reason they should be taken in high consideration when the binarization approach is applied. Additionally, in Fig. 6.3 it is possible to observe that the Q metric variability resulted stable only in the range between 0.6 and 0.8 for both atlases, with the FSAverage atlas reporting an inconsistent behaviour in the percentile distribution between 0.1 to 0.55. In other words, the modular characterization of the brain network remained stable in the percentile range between 0.6 and 0.8 suggesting that intraclustering connections do not vary at small perturbations of the binarization value. The stability of this metric is crucial for maintaining the small-world property characterizing the human brain [START_REF] Sporns | Modular brain networks[END_REF]. For the remaining three global metrics, such as E g , T and D, an Berardino BARILE opposite trend was observed. For E g and T metrics an exponential increase in the CV coefficient was observed for a value of τ greater than 0.7. These results suggest that the information flow between nodes far apart inside the network, and possibly pertaining to different modular clusters, start to be compromised for higher thresholding values, due to the reduction of intra-cluster connections and lower number of triangular motifs (reduced local transitivity). Notwithstanding, stable value of CV in the range of τ between 0.4 and 0.7 can be noticed for the E g metric. Finally, the D metric increased steadily until a value of τ equal to 0.7 from which an increased rate was observed, particularly for the Glasser2016 atlas which represents a low parcellation approach.

From the analysis of the thresholding variability we can thus conclude that good topological characteristics can be observed for a value of τ in the range between 0.6 and 0.8. However, an optimal thresholding value is difficult to be justified in general. In our previous studies [START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF], considering WM structural connectomes, stable topological characteristics of the graphs were observed for a value of τ greater than 0.65. For the statistical analysis in Section 6.4.2, a binary thresholding value of 0.7 was applied, as best compromise, since it represents the central value of the suggested range, after which the topology of the GM network resulted unstable (high variability) due to the involvement of highly centralized hubs.

Statistical analysis of MS clinical profiles

Statistical analysis was performed on the six global metrics used for graph characterization. The statistical differences between patients in the four MS clinical profiles were assessed considering the two atlases, FSAverage and Glasser2016. As shown in Fig. 6.4 and Fig. 6.5, significant differences were detected between the four groups for most of the graph metrics.

A reduced variability between MS clinical profiles can also be observed from the FSAverage atlas compared to the Glasser2016 atlas. Specifically, considering the BC and Q metrics, a downward trend between the four MS clinical profiles can be observed with higher values (p<0.001) in preliminary stages of the disease, such us CIS and RRMS, and a decreased values for progressive MS courses (PPMS and SPMS). Conversely, for r, T , E g and D an increased trend among the four clinical profiles can be noticed, with CIS and RRMS patients exhibiting reduced global metric values in all cases compared to progressive courses (p<0.001). When comparing CIS with progressive courses, significant differences (p<0.05) were detected for almost all metrics, with the only exception observed for Q using FSAverage atlas. However, less evident differences in the global metrics distribution were observed between CIS and RRMS patients, especially if considering the high number of subjects outside the usual interquartile boxplot range. Addition- 6.1: Mean (st.dev) of the predictive performances of the ensemble model across the ten folds of the cross validation using FSAverage atlas Group F1 Precision Accuracy AUC CIS-RR 0.653 (0.19) 0.737 (0.17) 0.626 (0.21) 0.575 (0.24) CIS-PP 0.782 (0.19) 0.869 (0.14) 0.789 (0.18) 0.805 (0.16) CIS-SP 0.753 (0.15) 0.826 (0.13) 0.733 (0.16) 0.749 (0.18) RR-PP 0.725 (0.14) 0.733 (0.14) 0.743 (0.12) 0.695 (0.13) RR-SP 0.667 (0.07) 0.699 (0.07) 0.678 (0.06) 0.674 (0.07) PP-SP 0.663 (0.11) 0.708 (0.13) 0.665 (0.11) 0.673 (0.13) Table 6.2: Mean (st.dev) of the predictive performances of the ensemble model across the ten folds of the cross validation using Glasser2016 atlas Group F1 Precision Accuracy AUC CIS-RR 0.762 (0.11) 0.811 (0.09) 0.747 (0.12) 0.657 (0.15) CIS-PP 0.834 (0.12) 0.872 (0.09) 0.819 (0.13) 0.798 (0.14) CIS-SP 0.723 (0.07) 0.809 (0.07) 0.693 (0.07) 0.672 (0.09) RR-PP 0.858 (0.09) 0.883 (0.06) 0.868 (0.08) 0.836 (0.11) RR-SP 0.602 (0.10) 0.637 (0.11) 0.624 (0.08) 0.616 (0.10) PP-SP 0.719 (0.09) 0.735 (0.08) 0.716 (0.09) 0.724 (0.09) ally, significant differences (p<0.05) were observed between PPMS and SPMS, especially when the Glasser2016 atlas was used. From this results, it is possible to conclude that GM thickness resulted altered in progressive courses compared to earlier MS stages with brain networks exhibiting a reduced number of centralized nodes, and thus more randomized distribution of the information transfer, with higher tendency, for the remaining hubs, to be linked with high degree nodes.

MS clinical profiles classification

From the statistical analysis, significant differences were detected, suggesting good degree of separation between MS clinical profiles. Thus, in order to evaluate the predictive performance of the six global graph metrics, the four ML models were trained separately and subsequently combined in a unified binary ensemble model as described in Section 6.3. Although not statistically significant, higher classification performances were obtained by late integration using major voting, justifying the ensemble approach. The classification task was performed using the two atlases separately and the results compared in Table 6.1 and Table 6.2.

Interesting level of classification scores were observed for the eight binary-classification tasks (four ML models with two distinct atlases). Overall, comparable results were ob-served between the two atlases with the best thresholding value selected by the cross validation to be equal to 0.75 for both atlases. Moreover, high classification performance scores were obtained comparing patients in the primary stage of the disease, such as CIS patients, with progressive courses (PPMS and SPMS). In particular, for these comparisons, the obtained F 1 score resulted always above 0.7 for both atlases. Notwithstanding, when considering the AUC score, a lower classification performance was observed comparing CIS with SPMS group, especially when the Glasser2016 atlas was used, implying a mild imbalance effect. However, when the FSAverage atlas was used, the obtained AUC score resulted with a value of 0.75, suggesting that a more detailed GM pracellation may increase noice leading to less informative connectomes and a reduced classification performances. Interestingly, when comparing CIS with RRMS patients, higher performances were observed using the Glasser2016 atlas with an F 1 score of 0.76, compared with the FSAverage atlas for which an F 1 score of 0.65 was obtained (p<0.001). Again, the AUC score resulted lower in both atlases with values of 0.66 and 0.58 respectively, highlighting the lack of GM connectomes to discriminate between patients in the early stages of the disease, especially when a large (less refined) parcellation strategy was considered. Conversely, when comparing RRMS patients with PPMS, both atlases highlighted good level of separation between the two groups with an F 1 score of 0.86 and 0.72 respectively, coherently with the statistical analysis in Section 6.4.2. Surprisingly, when comparing RRMS patients with SPMS, poor classification performances were detected, especially when the Glasser2016 atlas was considered, probably due to the high variability between the two clinical profiles, as observed from the boxplot analysis in Fig. 6.4 and Fig. 6.5. For the comparison between progressive courses, a good level of classification performance was obtained using the Glasser2016 atlas, with an F 1 score of 0.72, compared to the result obtained using the more refined parcellation atlas, which resulted with an F 1 score of 0.66. As an additional ablation study, and in order to provide realistic comparisons with previous works [START_REF] Keerthi | Asymptotic behaviors of support vector machines with gaussian kernel[END_REF][START_REF] Muthuraman | Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS[END_REF], the results obtained using only the SVM model were also reported in Table 6.3 and Table 6.4, for the FSAverage and Glasser2016 atlas respectively.

Of particular relevance is the fact that the SVM model alone outperformed the ensemble model when the RRMS group was compared with progressive courses (PPMS and SPMS), using the FSAverage atlas. However, when Glasser2016 atlas was used, this result was not confirmed and the discrimination between RRMS and PPMS resulted equal to 0.58 and 0.59 respectively for the F 1 and AUC scores, again confirming the usefulness of the ensemble approach. A second ablation study was also performed in order to investigate the classification performances obtained by comparing early stages of MS patients (CIS and PPMS) with progressive MS patients (PPMS and SPMS). In particular, in Table 6. [START_REF] Badeau | Multilinear singular value decomposition for structured tensors[END_REF] Berardino BARILE Table 6.3: Ablation study: Mean (st.dev) of the predictive performances of the SVM model across the ten folds of the cross validation using FSAverage atlas Group F1 Precision Accuracy AUC CIS-RR 0.624 (0.15) 0.695 (0.13) 0.603 (0.17) 0.504 (0.20) CIS-PP 0.771 (0.15) 0.836 (0.14) 0.788 (0.12) 0.758 (0.17) CIS-SP 0.734 (0.15) 0.799 (0.14) 0.717 (0.16) 0.705 (0.21) RR-PP 0.793 (0.09) 0.808 (0.09) 0.795 (0.09) 0.787 (0.10) RR-SP 0.705 (0.07) 0.711 (0.07) 0.706 (0.07) 0.706 (0.07) PP-SP 0.646 (0.09) 0.702 (0.09) 0.618 (0.09) 0.618 (0.10) Table 6.4: Ablation study: Mean (st.dev) of the predictive performances of the SVM model across the ten folds of the cross validation using Glasser2016 atlas Group F1 Precision Accuracy AUC CIS-RR 0.752 (0.09) 0.785 (0.07) 0.737 (0.11) 0.603 (0.14) CIS-PP 0.831 (0.13) 0.872 (0.10) 0.814 (0.15) 0.795 (0.16) CIS-SP 0.724 (0.06) 0.812 (0.08) 0.693 (0.06) 0.671 (0.12) RR-PP 0.801 (0.08) 0.805 (0.07) 0.803 (0.07) 0.791 (0.09) RR-SP 0.582 (0.08) 0.586 (0.09) 0.591 (0.07) 0.585 (0.08) PP-SP 0.735 (0.08) 0.743 (0.08) 0.733 (0.08) 0.733 (0.09) and Table 6.6 all possible combinations considering more than two MS clinical profiles were considered using the FSAverage and Glasser2016 atlas respectively. However, the CIS group alone was not considered due to the excessive imbalance compared to progressive courses considered in this study. Notwithstanding, good level of classification performances were observed comparing CIS and RR groups with respectively PPMS and SPMS patients. In agreement with the binary comparisons, higher scores were obtained when using the Glasser2016 atlas compared to the FSAverage atlas although not significant (p>0.05). Conversely, when comparing RRMS patients with progressive courses, significant differences (p<0.05) were observed between the two at-Table 6.5: Ablation study: Mean (st.dev) of the predictive performances of the ensemble model across the ten folds of the cross validation using FSAverage atlas and multiclass binary comparisons

Group

F1 Precision Accuracy AUC CIS+RR-PP 0.654 (0.13) 0.682 (0.13) 0.655 (0.12) 0.666 (0.13) CIS+RR-SP 0.632 (0.12) 0.659 (0.13) 0.633 (0.12) 0.644 (0.12) CIS+RR-PP+SP 0.649 (0.11) 0.681 (0.12) 0.651 (0.11) 0.663 (0.11) RR-PP+SP 0.778 (0.09) 0.792 (0.10) 0.778 (0.10) 0.777 (0.10) lases with higher score (F 1 and AUC scores of 0.78) obtained using the more refined parcellation approach (FSAverage). Also, it is interesting to compare this result with what obtained considering CIS and RRMS together against progressive MS courses. In this case, an F 1 score of 0.65 was obtained for both atlases, suggesting that the inclusion of CIS patients happened to be armful to the classification performance, probably due to the high variability in the CIS group and the low number of patients.

Discussion

GM atrophy is known to occur in MS, even at the earliest stages of the disease [START_REF] Eshaghi | Progression of regional grey matter atrophy in multiple sclerosis[END_REF]. Brain morphological connectivity data offers new opportunities to identify potential biomarkers of GM atrophy by combining the power of graph representation with the anatomical GM thickness feature. To the best of our knowledge, the present study represents the first attempt to simultaneously characterize and classify MS patients using GM thickness correlation network data. Other studies have attempted similar tasks [START_REF] Muthuraman | Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS[END_REF] by characterizing GM network data considering only CIS and RRMS profiles, reaching good level of accuracy (97%). The present study extended this comparison to all MS clinical profiles and proposed a fully automated pipeline to generate GM connectivity graphs, from T1w images, using two different atlases. The two parcellation strategies were chosen in order to investigate whether a more refined parcellation can provide additional power for the discrimination between MS clinical phenotypes. Additionally, statistical analysis was performed in order to investigate if significant differences could be found when comparing six of the most important global metrics for graph characterization. However, as discussed in Section 6.2.2, the chosen thresholding strategy may influence the characteristics of the graphs and should be carefully chosen. Good properties were observed for thresholding values between 0.6 and 0.8. However, an optimal binarization score remains difficult to justify and should be investigated in future studies using larger datasets. It is important to notice that, the study of GM connectome thresholding is not yet tackled in Berardino BARILE the literature and usually no justifications are given when calculating graph metrics. In this study we try to provide some empirical justifications to the thresholding role applied for graph characterization based on the analysis performed in Section 6.4.1. However, the statistical results remained unchanged for different nearby thresholding values ensuring robustness around the thresholding neighbours. From the statistical analysis, the Glasser2016 atlas showed high degree of variation between clinical profiles, most likely due to the larger size of the connectome and thus more precise GM parcellation. Notwithstanding, more noice can be present in the morphological features extracted from each GM region since a reduced number of pixels were considered for creating each link inside the connectome. For this reason, a larger number of patients might be needed in order to confirm our results. However, similar results were observed in both cases. Specifically, when comparing CIS with progressive courses (PPMS and SPMS) good level of classification performances were obtained, in agreement with our a-priori expectation as well as a previous results [START_REF] Muthuraman | Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS[END_REF]. In fact, CIS represents the first stage of the disease usually characterized by tissue inflammation while PPMS and SPMS describe the progressive evolution of the disease where the demyelination process causes tissue degeneration. Notwithstanding, poor predictive performances were obtained comparing CIS and RRMS clinical profiles, particularly when the FSAverage atlas was used. These results were not coherent with what obtained in Muthuramen et al. [START_REF] Muthuraman | Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS[END_REF] for which high level of accuracy was obtained. However, the lack of discrimination between the two early stages of MS may be explained by the low number of patients in the CIS group and the high level of imbalance with respect to the RRMS profile, which constitutes the clinical form more common in MS, conform to clinical practice [START_REF] Zahoor | Demographic and clinical profile of multiple sclerosis in kashmir: A short report[END_REF]. Additionally, interesting results were obtained comparing multiple global metrics. In particular, patients in the early stages of the disease (CIS and RRMS) showed a more assortative brain structure compared with patients in progressive stages (PPMS and SPMS). The exact opposite was found in previous studies in which connectome data were obtained from WM streamline tractography [START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF]. This result is expected due to the opposite interpretation provided by GM and WM graphs. Indeed, while the former measures the degree of dissimilarity between two GM cortical regions (dissociative measure), the latter measures the interconnectivity between two brain regions by counting the number of streamline fibers obtained from the tractography (associative measure), thus providing opposite interpretations. Conversely, node centrality as well as modularity was reduced in patients with progressive courses, especially when high resolution parcellation was considered (Glasser2016). Again, this result is coherent with the WM analysis performed in Kocevar et al. [START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF]. These results may suggest that the small-word property of the human brain is altered in advanced stages of the disease conform to the literature [START_REF] Fleischer | Increased structural white and grey matter network connectivity compensates for functional decline in early multiple sclerosis[END_REF][START_REF] He | Impaired small-world efficiency in structural cortical networks in multiple sclerosis associated with white matter lesion load[END_REF]. These aspects were taken into account by (25.11) -0.51 (0.04) 0.59 (0.03) 0.28 (0.06) 0.32 (0.07) 0.13 (0.03) the ML models. In fact, for the FSAverage atlas, BC and r resulted the most important metric overall, suggesting strong predictive discrimination. Moreover, higher E g and D were observed in later stages of MS, consistent with the concept of progressive neural loss and consequently of structural hubs, increasing the randomness of the global network [START_REF] Rimkus | Gray matter networks and cognitive impairment in multiple sclerosis[END_REF]. Additionally, due to the high number of statistically significant differences between clinical profiles, observed for both atlases (Figure 6.4 and Figure 6.5), it is reasonable to question if single global graph metrics can naively classify MS patients in their respective clinical profiles. However, from the boxplot analysis and from the results reported in Table 6.7, we can conclude that a trivial binary thresholding cannot discriminate MS clinical profiles and more sophisticated ML models are required.

Nonetheless, this work has also some limitations. First, the reduced number of patients in our dataset, especially for the CIS group, may provide some degree of uncertainty in the generalization results. This has led to an imbalanced dataset when comparing the CIS group with other clinical profiles. In order to tackle the problem, weights were imposed to the cost functions of each ML model in order to regain balance during models optimization, reducing the likelihood to overfit the majority class. With a large enough sample size, a multi-class classification approach may be considered and under/over sampling techniques could be applied for the classification of MS patients across the four groups, which may offer a useful tool for clinical applications. However, the aim of this study is to investigate if GM tissue degeneration, combined with the connectome representation, can discriminate MS subgroups. Thus, in order to provide comparable results with previous works, in this study, binary comparisons between MS clinical profiles were considered. Second, the conditions followed in this study for binary thresholding represents our best attempt to empirically solve the binarization problem without inducing bias Berardino BARILE in the process. Thus, in this work we did not aim to validate the optimal thresholding role, which need to be thoroughly investigated in future studies. Rather, we aim to perform a statistical analysis independently from the predictive classification task that may follow, in which the thresholding value was considered as an additional hyper-parameter optimized during cross validation. Third, the classification results might be improved by including lesion filling during the GM segmentation, which has demonstrated to increase the accuracy of cortical thickness measurements in MS patients [START_REF] Magon | White matter lesion filling improves the accuracy of cortical thickness measurements in multiple sclerosis patients: a longitudinal study[END_REF]. However, this approach requires lesions to be segmented by experts, which represents a time consuming and expensive procedure. Automated lesion segmentation are today available, although imprecise segmentation may hinder the improvement provided by lesion filling, thus limiting its applicability in clinical practice.

Conclusion

Although MS is mainly considered as an inflammatory and demyelinating WM disease, it also exhibits extensive GM involvement and neuro-degenerative processes. An automated pipeline was proposed in this study to characterize GM graphs, extracted from the brain of MS patients, using morphological features such as GM thickness. The statistical analysis revealed that significant differences were present between multiple global metrics, highlighting the importance of GM connectome graphs. The analysis was performed using two different atlases, such as FSAverage and Glasser2016, in order to investigate the impact of GM parcellation on MS profiles discrimination. Overall coherent results were observed between the two parcellation strategies with higher classification performances obtained with the Glasser2016 atlas when comparing PPMS with RRMS and SPMS patients. This result was confirmed by the statistical analysis which highlighted high differences between MS profiles when a more refined GM parcellation was considered. Additionally, the thresholding value for graph characterization was also investigated. However, no clear results were observed from the statistical analysis, although good characterization properties and higher stability of graph metrics were observed for percentile values between 0.6 and 0.8. Notwithstanding, the predictive analysis suggested a binarization threshold of 0.75 obtained through cross validation. The results obtained in this work are of great interest considering that only the anatomical T1w image was used for MS groups classification, which represents the MRI modality most used in clinical applications. In future studies, we will embed the WM and GM connectomes in a unified graph representation with the aim of improving the classification performance of MS patients and exploit the complementary information provided by the different brain tissue types. Additionally, the study of longitudinal changes in MRI biomarkers across the four

Introduction

In this work, the entire dataset described in Table 4.1 of Part I of the present Manuscript was used. A factorization approach based on MultiLinear Singular Value Decomposition (MLSVD) [START_REF] Lathauwer | A multilinear singular value decomposition[END_REF], was developed for the discrimination of MS patients between early stages of the disease, such as CIS and RRMS, and progressive courses (PPMS and SPMS). The multi-view information, obtained from the cortical Grey Matter (GM) surface reconstruction output of FreeSurfer [START_REF] Reuter | Within-subject template estimation for unbiased longitudinal image analysis[END_REF], was exploited to enhance classification performance based on three morphological features, namely thickness, curvature and area. To our knowledge, this is the first attempt to perform classification of MS patients using a factorization method applied to kernel multi-view data. Moreover, only the anatomical MR image modality (T1-weighted) was used for the task, avoiding the burden of acquiring Diffusion Tensor Imaging (DTI) and/or functional Magnetic Resonance Imaging (MRI) data, which require long and expensive procedures. Furthermore, in order to demonstrate the generalization capacity of the proposed approach, the properties of MLSVD were exploited, generating a projection matrix. Such a matrix was used to project new incoming patients in the same embedding space obtained from our labeled training dataset, and subsequently classified using Random Forest (RF) [START_REF] Breiman | Random forests[END_REF].

Data Acquisition and Pre-processing

In this study, the preprocessing of the anatomical images, was performed using Freesurfer v6.0.0 image analysis suite and three GM features, namely thickness, curvature and area were calculated. In Figure 7.1, an intuitive pipeline for the kernel data generation is proposed. The GM parcellation task was performed using the Desikan atlas [START_REF] Rosenke | A cross-validated cytoarchitectonic atlas of the human ventral visual stream[END_REF], yielding 68 different brain regions. For each region, a list of values v ∈ R (d×1) + , with d representing its total number (usually in the order of thousands), were calculated for each specific GM region and are called vertices. These vertex values represent small triangles on the flattened surface, following the approach proposed by [START_REF] Schwartz | Computer-aided neuroanatomy: Differential geometry of cortical surfaces and an optimal flattening algorithm[END_REF], from which morphological features were calculated by means of the Freesurfer toolkit. For more information we refer to Dale et al. [START_REF] Dale | Cortical surface-based analysis. segmentation and surface reconstruction[END_REF] and Fischl et al. [START_REF] Fischl | Cortical surface-based analysis: Ii: Inflation, flattening, and a surface-based coordinate system[END_REF]. It is important to notice that, for each scan, more than 180.000 features were generated, which represents a problem for standard Machine Learning (ML) models due to the curse of dimensionality [START_REF] Bellman | Dynamic programming[END_REF].

Kernelization, Tensorization and Normalization

In order to reduce the dimensionality of the problem, a three-steps approach was implemented. First, we perform kernelization of the high-dimensional dataset by means of Minkowski distance formulation. The kernelization step maps the high-dimensional data in a possibly higher-dimensional one in which relationships are assumed to be linear. Second, the obtained kernel dataset was restructured into a third order tensor which is assumed to be low-rank. Third the obtained tensor was decomposed using a tensor-based higher-order SVD (where the learnt components are not necessarily "sources" but correspond to bases) for the extraction of feature embedding used as input to the RF classifier.

Kernelization

Let's define with K f (x i , x j ) the kernel function for the morphological feature f . K f (x i , x j ) represents a scalar value defining (dis-)similarity between region i and region j where i = j ∀i, j ∈ {1, 2, ..., q} such that:

K f (x i , x j ) =   q i,j;∀i =j |x f i -x f j | γ   1 γ (7.1)
where x f i and x f j represent the set of vertex values for morphological feature f pertaining to brain region i and j respectively, and γ = 1 defines the norm. This distance measure is also called Manhattan or taxicab distance. Thus the kernel matrix X f ∈ R (q×q) + can be obtained by arranging each pairwise comparison K f (x i , x j ) in a squared matrix, where i and j represent rows and columns respectively. The process was performed independently for all morphological features f , finally obtaining three real positive kernel matrices X t , X c , X a ∈ R (q×q) + (i.e., thickness, curvature and area).

Tensorization and Normalization

The three morphometric kernels were stacked along the first dimension such that X tca ∈ R (z×q) + and z = 3q. Thus, X tca represents the vertically stacked adjacency matrices of a patient scan. A third-order tensor can be obtained by concatenating all the stacked kernel matrices along the third mode, obtaining a tensor X tca ∈ R

(z×q×p) +

, where p defines the number of patient scans. Finally, normalization was performed along the third mode employing a min-max operation such that each value (i.e. dissimilarity) of the matrix is standardized in the range [0 -1].

Berardino BARILE

Tensor-based multi-view factorization

In this work, a factorization approach, based on MLSVD, was employed for the extraction of feature embedding from a third-order tensor, and used for MS patient profiling. Such a tensor can be decomposed in the product of three nonzero factors, representing latent components (i.e., factor loadings), each of which describing a specific mode of the tensor. In this work, we refer to multilinear rank L = [r 1 , r 2 , r 3 ] ∈ N 3 , where possibly r 1 = r 2 = r 3 , as a triplet of values such that r n describes the mode-n rank of the mode-n unfolding of the original third-order tensor X , which we denote as X (n) .

Multilinear Singular Value Decomposition

The general method is based on the Singular Value Decomposition (SVD) obtained from a higher-order tensor. Mathematically, we can define the MLSVD decomposition as follows:

X ≈ S • 1 U (1) • 2 U (2) • 3 U (3) (7.2)
where U (1) ∈ R (z×r 1 ) , U (2) ∈ R (q×r 2 ) and U (3) ∈ R (p×r 3 ) respectively form orthonormal bases for the first, second and third mode of the tensor X . In other words, the matrices U (1) , U (2) , U (3) represent respectively the first r 1 ,r 2 and r 3 left singular vectors of the SVD of the mode-1, mode-2 and mode-3 unfolding of X . The operator "• n " represents a mode-n tensor-to-matrix product. The core tensor S ∈ R (r 1 ×r 2 ×r 3 ) explains the interaction 166 Berardino BARILE 7.5. RESULTS AND DISCUSSION between the different modes and has all-orthogonality properties meaning that all slices of S are mutually orthogonal. Moreover, this structured representation ensures that variation in each mode is captured independently. For a more in-depth description of the model, we refer to [START_REF] Lathauwer | A multilinear singular value decomposition[END_REF].

Feature Extraction and Classification

In this work, we exploited the properties of the MLSVD factorization model to generate feature embedding able to classify MS patients. Practically speaking, from Eq. (7.2) it is possible to rearrange the terms on the right-hand-side such that:

x tca ≈ u (3) S (3) U (1) ⊗ U (2) T (7.3)
where

S (3) ∈ R (r 3 ×r 1 r 2 ) +
represents the mode-3 unfolding of the core tensor S, ⊗ defines kronecker product and u (3) ∈ R (1×r 3 ) + represents the row vector of the mode-3 matrix U (3) associated to the multi-view matrix X tca of a specific patient scan. Also, x tca ∈ R (1×zq) + represents the vectorization of the matrix X tca . Consequently, X tca ∈ R (p×zq) + represents the vectorization matrix describing all patients. The embedding space can be obtained as follows:

X embed = X tca U (1) ⊗ U (2) T S (3) (7.4) where X embed ∈ R (p×r 3 ) represents the embedded matrix used for classification, while S T (3) represents the transposed matrix of the mode-3 core tensor unfolding. RF was used as classifier and trained on the embedding representation of the multi-view data. The number of trees was set to 300 for computational reasons.

Results and Discussion

The experiment was performed employing a stratified 10-fold Cross Validation (CV) strategy. Each patient (including all consecutive scans to guarantee independency of the test set) was randomly assigned to only one of the 10 folds, thereby preserving the percentage of samples of each class. Also, at each fold iteration, factorization was applied only on the training data (9 folds) while the test data (left-out fold) were hold out. The projection matrix U (1) ⊗ U (2) S T

(3) was used to project each left-out multi-view matrix (i.e. test set) in the desired embedding space. Table 7.1 reports the classification results obtained when comparing early MS stages (CIS and RRMS) with progressive MS courses. All possible mode-3 multilinear rank combinations between 50 and 300 at steps of 50 were considered. Mode-1 and mode-2 multilinear ranks were constrained to their maximal value equal to q = 68, representing the size of the kernel data, following the approach proposed in Boussé et al. [START_REF] Bousse | Irregular heartbeat classification using kronecker product equations[END_REF]. Notice that the results remain stable at different mode-3 multilinear rank values, which defines the dimensionality of the embedding space to which we project new incoming patients. Specifically, for almost all mode-3 multilinear ranks, the Area Under the ROC Curve (AUC) scored higher than 0.7, which represents a good result considering the low amount of data and that only the T1-weighted image was used. For completeness, the F1 score as well as the Precision and Recall scores were also reported. In this case, lower values were obtained due to the thresholding binarization used to discretize probabilities, imposed at a standard value of 0.5. Conversely, the AUC score represents a threshold free classification metric and it might represent a more accurate measure of classification performance.

Mode

Conclusion

In this work, a new pipeline was implemented for the classification of MS patients. Interesting results were obtained considering that only the T1-weighted image modality was used. To our knowledge, this is the first attempt to apply tensor factorization for MS patients profiling. We demonstrated that the proposed approach provides stable results for different mode-3 multilinear rank values, which represents the size of the embedding space used for classification. In future works, we aim to improve the results by further optimizing the factorization approach, used for dimensionality reduction, including relevant constraints and automated rank-selection. Also, our method will be applied on a much larger dataset (Observatoire Français de la Sclérose en Plaques -OFSEP), demonstrating the generalization performances of the proposed pipeline.

III Machine Learning Models for

Estimation of MS Disability

Introduction

In the context of brain imaging, new modeling approaches have been proposed to characterize the brain networks by means of graph theory [START_REF] Yong | Graph theoretical modelling of brain connectivity[END_REF][START_REF] Guo | Deep neural networks on graph signals for brain imaging analysis[END_REF], given the capability of graphs to represent complex relations among different areas of the brain. Graph representation is not novel in the scientific field, and it is widely applied in many contexts, including social behavioral studies. Notwithstanding, their use is steadily growing in the neuroscience community, particularly in brain imaging. Using magnetic resonance imaging (MRI), both functional and structural brain connectivity can be obtained by analyzing temporal activity correlations between brain grey matter (GM) regions using resting-state functional MRI and by modelling white matter (WM) fiber-bundles using diffusion tensor imaging (DTI), respectively. Such network-like structures of the human connectome consist of nodes, based on the parcellation of brain GM regions, and edges, determined by the underlying functional activity or structural links between the network nodes. These new approaches laid the foundation for a better characterization of pathological processes and micro-architectural damages in different brain diseases, such as MS. Moreover, the use of connectome data in the field of machine learning (ML) has grown exponentially in the last decade. However, compared to a single ML model, the ensemble-based approaches, where two or more models are combined together in order to enhance the final prediction, is widely used in a range of different tasks [START_REF] Vlahavas | Random k-labelsets: An ensemble method for multilabel classification[END_REF][START_REF] Liu | A combinational feature selection and ensemble neural network method for classification of gene expression data[END_REF]. The popularity of ensemble-based methods is perhaps attributable to their relatively superior performance in comparison to the other single machine learning algorithms.

Conversely, the interest in interpreting and understanding ML models has become popular only recently [START_REF] Gilpin | Explaining Explanations: An Overview of Interpretability of Machine Learning[END_REF]. The importance to understand the line of reasoning followed by the ML model is that it conveys more trust to the obtained results and can be used as an additional check in the ML validation process. Generally speaking, interpretability can be achieved either by restricting the complexity of the ML model (intrinsic methods) or by applying methods that analyze the model after training (post-hoc methods). Moreover, a further distinction can be obtained between local and global interpretable models based on whether the interpretation method explains an individual prediction or the entire model behavior. In this study, we first show that our framework is able to estimate disability with a low degree of error. For this purpose, an ensemble of four boosting models are combined in a meta-model, by means of linear regression, in order to minimize the estimation error. The task of estimating EDSS directly from connectome data may represent an important improvement over traditional MRI methods since DTI can detect pathological mechanisms outside visible lesions and thus better characterize the relationship with disability. Second, a global and local interpretability model is proposed in order to pin-down the brain connections most likely responsible for explaining different degrees of disability. According to our knowledge, this is the first attempt to apply interpretable ML techniques to correlate brain connectivity with MS disability using DTI structural graphs.

The remainder of the paper is structured as follows. In Section 8.2, relevant work in the field is reviewed. Section 8.3 provides a detailed description of the preprocessing steps used for generating connectome-based data. Moreover, the entire workflow followed in this study is also described. Section 8.4 describes the proposed ensemble model. A particular focus to the interpretability of our model is offered in section 8.5. Section 8.6 reports the parameter setting used in this work and the results obtained. Finally, in Section 8.7 and 8.8, our findings are discussed and conclusions are drawn.

Related Work

In the context of MS, advanced MRI methods have already provided high degree of accuracy in tasks related to lesion detection and segmentation [START_REF] Gabr | Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study[END_REF][START_REF] Jain | Two time point MS lesion segmentation in brain MRI: an Expectation-Maximization framework[END_REF][START_REF] Mckinley | Automatic detection of lesion load change in multiple sclerosis using convolutional neural networks with segmentation confidence[END_REF], as well as classification of clinical forms [START_REF] Akkus | Deep learning for brain MRI segmentation: state of the art and future directions[END_REF][START_REF] Tousignant | Prediction of disease progression in multiple sclerosis patients using deep learning analysis of MRI data[END_REF].

From a clinical perspective, MS disability prediction constitutes one of the most relevant aspects due to its impact on the therapy decision tree [START_REF] Kuvalekar | Quality of life among persons with physical disability in udupi taluk: A cross sectional study[END_REF]. Few ML methods have already shown their prognostic value based on conventional MRI [START_REF] Hagmann | Mapping human whole-brain structural networks with diffusion mri[END_REF][START_REF] Law | Machine learning in secondary progressive multiple sclerosis: an improved predictive model for short-term disability progression[END_REF][START_REF] Zhao | Exploration of machine learning techniques in predicting multiple sclerosis disease course[END_REF]. For example, Tousignant et al. [START_REF] Tousignant | Prediction of disease progression in multiple sclerosis patients using deep learning analysis of MRI data[END_REF] proposed an automatic end-to-end deep learning framework, based on parallel convolutional pathways, using multi-center MRI data images obtained in RRMS patients. Washimkar et al. [156] used also conventional MRI data images to identify lesions responsible for cognitive decline and physical disability. However, the poor correlation between the lesion load measurements with patients' disability remains an issue [START_REF] Declan | Resolving the clinico-radiological paradox in multiple sclerosis[END_REF]. For this reason, numerous advanced MRI techniques, such as DTI, were developed to early detect micro-alterations of WM. Recent studies have demonstrated the importance of DTI for the analysis of the human brain [START_REF] Kolasa | Diffusion tensor imaging and disability progression in multiple sclerosis: A 4-year follow-up study[END_REF][START_REF] Stamile | Multiparametric non-negative matrix factorization for longitudinal variations detection in white-matter fiber bundles[END_REF][START_REF] Zurita | Characterization of relapsingremitting multiple sclerosis patients using support vector machine classifications of functional and diffusion MRI data[END_REF][START_REF] Marzullo | Classification of multiple sclerosis clinical profiles via graph convolutional neural networks[END_REF] and paved the way for the development of DTI-based brain structural connectivity features for a finer analysis of brain networks and a better characterization of MS patients. In this scenario, Marzullo et al. [START_REF] Marzullo | Prediction of multiple sclerosis patient disability from structural connectivity using convolutional neural networks[END_REF] developed a new Deep Learning (DL) approach to estimate the EDSS based on structural connectivity data. Following this initial study, we propose a new ensemble of ML methods to better estimate the disability status of MS patients based on brain structural connectivity. Additionally, a fully interpretable counterfactual technique is proposed with the aim of pinning down brain connections that are mostly responsible for disability prediction.

Methods

Brain Structural Connectivity Generation

In this study, five-hundred-thirteen MS scans from the dataset described in Table 4.1 of Part I of the present manuscript were used due to the lack of evaluation in the disability score. Starting from 3D T1-weighted images, cortical and sub-cortical GM segmentation was performed [START_REF] Reuter | Within-subject template estimation for unbiased longitudinal image analysis[END_REF]. From the segmentation, 84 regions (nodes) are identified based on the atlas Desikan [START_REF] Desikan | An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest[END_REF]. At the same time, each voxel was classified into four classes [WM, cortical GM, sub-cortical GM, cerebro-spinal fluid (CSF)] in order to be used for the subsequent step of tractography. Eddy current correction and non-brain voxels stripping were performed using the FMRIB Software Library [70] as pre-processing steps for diffusion. In each voxel, the diffusion orientation distribution function (dODF) is computed [START_REF] Tournier | MRtrix: diffusion tractography in crossing fiber regions[END_REF] in order to estimate the main diffusion directions. The maximum spherical harmonics of order k was calculated by (k+1)(k+2) 2 < d where d represents the number of acquired diffusion directions while k was set to 4 to match with the acquisition protocol. Based on the four tissue-class segmentation, anatomically constrained probabilistic streamline tractography (ACT) was performed from dODF [START_REF] Smith | Anatomically-constrained tractography: improved diffusion MRI streamlines tractography through effective use of anatomical information[END_REF][START_REF] Tournier | MRtrix: diffusion tractography in crossing fiber regions[END_REF]. From this, an undirected weighted graph G = (V, E, ω) can be defined, where |V | = q represents the set of brain regions, |E| = m represents the set of connections between these regions and ω defines the number of fibers connecting two nodes. More precisely, the structural brain connectivity representation of G was generated by combining brain GM parcellation, obtained after segmentation of T1-weighted MR images, with the WM fiber-tracking, reconstructed from DTI acquisitions. In particular, a general connectivity matrix A ∈ N q×q was obtained by summing the number of streamlines connecting each pair of nodes for a specific subject. More formally, let a i,j be a general element of the matrix A where 1 < i, j < q and q = 84, then the element a i,j of A is defined by Σ : N 2 → N and represents the number of fibers connecting the node i with the node j.

Proposed Model

In this study, a stacking ensemble model, based on multiple gradient boosting algorithms, is developed in order to estimate the disability status of MS patients starting from the corresponding brain connectome. Besides, the proposed approach focus on interpretability of the model, by providing a counterfactual explanation for the obtained results. For this reason, an additional statistical model, based on conditional logistic regression, is combined with the predicting ensemble model. Figure 8.1 depicts the complete workflow of this study. First, the ensemble model is trained using real connectome data and referred in Figure 8.1 as "original data" in order to distinguish it from the perturbed (counterfactual) data. This dataset constitutes the input of the ensemble model, or otherwise the brain connectivity of MS patients in their adjacency matrix form. The green box encloses the training process for which an estimation of the EDSS score is obtained. Once the training process is completed, the final ensemble model is ready to be interpreted by the statistical model. The red box encloses the interpretability model. For each hold-out sample, graph perturbation based on conditional probability distribution is performed in order to obtain valid counterfactual samples as described in Section 8.5.2. The perturbed (counterfactual) dataset is used as input for the trained ensemble model in order to estimate the counterfactual EDSS score (ŷ perturbed ). The obtained counterfactual dataset, combined with the EDSS estimation, is used by the statistical model to retrieve the causal effect of the WM fiber connections on the EDSS disability score. Finally, the average of the β coefficients, conditioning on the disability class, is considered in order to obtain the final score of importance.

A thorough explanation of each phase of the workflow proposed in this study is offered in the following sections.

Proposed Model for EDSS estimation

Ensemble modelling is an advanced ML approach where multiple models are assembled to predict an outcome. The ensemble model aggregates the prediction of each single model to produce better performance on unseen data. As long as the base-models are diverse and independent, the variance of the prediction error decreases when the ensemble approach is used [START_REF] Kotu | Predictive analytics and data mining[END_REF]. In order to combine all these base-models, a stacked generalization technique is applied. Basically, a meta-learning algorithm is built to learn how to best combine the predictions from multiple base ML algorithms. The benefit of the stacking approach is that it can harness the capabilities of a range of well-performing models on a classification or regression task and make predictions that have better performance than any single model of the ensemble [START_REF] Witten | Data mining: practical machine learning tools and techniques[END_REF].

In this study, all the base-models used to build the meta-learning algorithm are treebased boosting models. In ML, boosting is a method for creating an ensemble of weak learners stacked sequentially. It starts by fitting an initial model to the data and keeps adding new learners that focus on accurately predicting the cases where the previous model performs poorly. In order to establish a connection with the statistical framework, a gradient-descent based formulation of boosting methods was used [START_REF] Friedman | Greedy boosting approximation: a gradient boosting machine[END_REF][START_REF] Friedman | Additive logistic regression: a statistical view of boosting[END_REF][START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF].

In the following subsection, a brief explanation of the gradient boosting algorithm is provided, as well as the specific details on how each single base-model is combined in a higher-level model.

Gradient Boosting Algorithm

Gradient boosting is a type of machine learning boosting technique that uses subsequent sets of weak learners added together such that the residual error obtained from the previous learner is reduced. It relies on the intuition that the best possible next model, when combined with previous models, minimizes the overall prediction error. More precisely, Gradient boosting exploits differentiable loss functions by using gradient descent approximation to minimize the objective function when adding subsequent learners. Moreover, the function that estimate f is parametrized in an additive functional form:

f (x) = f K (x) = K k=0 fk (x) (8.1)
where K represents the number of iterations, f0 (x) represents the starting model and { fk (x)} K k=1 defines the set of subsequent learners. The final model f (x), obtained by sequentially training multiple learners on the residuals of the previous one, is the overall ensemble function. The optimization function is thus defined as:

(ρ t , θ t ) = argmin ρ,θ N i=1 [-g t (x i ) + ρ h(x i , θ)] 2 (8.2)
where h(x, θ) represents the subsequent base-learner, ρ defines the learning rate and:

g t (x) = E y = ∂Ψ(y, f (x)) ∂f (x) x f (x)= f t-1 (8.3)
while the loss function is defined to be the squared-error L 2 loss:

Ψ(y, f (x)) L 2 = 1 2 (y -f (x)) 2 (8.4)
Most often, a regularization term is added to the final loss function in order to avoid overfitting. In this context, the minimization objective becomes:

(ρ t , θ t ) = argmin ρ,θ N i=1 [-g t (x i ) + ρ h(x i , θ)] 2 + k Ω(f k ) (8.5)
where Ω(f ) represents the regularization term used for penalizing the complexity of the model and is defined in Eq. ((8.6)).

Ω(f ) = γT + 1 2 λ||ω|| (8.6) Berardino BARILE
where T defines the number of leaves in the tree and γ represents the corresponding weights while the second term in the summation represents the L1 regularization. The additional term added to the final loss helps in smoothing the final learned weights in order to select a model which is the simplest between those that perform the best on the test data.

Stacking of Ensemble Models

Stacking Generalization involves the process of combining the predictions from multiple ML models. Given multiple ML models that are skillful on a problem, a higher-level model is defined in order to best combine the results obtained from each single model. The models are fit on the same dataset and a meta-model is used to learn how to best combine the predictions from the contributing base-models [START_REF] Wolpert | Stacked generalization[END_REF]. In this study, four different boosting-based models are used: Gradient Boosting Machine (GBM) [START_REF] Friedman | Greedy function approximation: A gradient boosting machine[END_REF], Extreme Gradient Boosting (XGBoost) [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF], Categorical Gradient Boosting (CatBoost) [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF], Light Gradient Boosting (LightBoost) [START_REF] Ke | Lightgbm: A highly efficient gradient boosting decision tree[END_REF]. These models are thus combined together in a meta-model in which linear regression is used as higher-level model. Although all models are Gradient Boosting algorithms, each of them exhibits a specific behavior for building trees and dealing with overfitting. In particular, while GBM subsample rows before creating each tree, XGBoost subsample both rows (i.e. instances) and columns (i.e. features) for training each individual base learner and for the splitting strategy. LightBoost uses a novel technique of Gradient-based One-Side Sampling (GOSS) to filter out the data instances for finding the optimal split value for building week-learner-trees while XGBoost implements pre-sorted-histogram-based algorithm as splitting strategy. Finally, CatBoost deals with the problem of prediction shift for which given a prediction model F the distribution of F (x k ) | x k leads to a shift from the distribution of F (x) | x [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF]. In this case, ordered boosting implementation, a modification of standard gradient boosting algorithm, which avoids target leakage, is used.

Methods for Interpretability

After explaining the proposed model used for disability estimation, in this section a new implementation of the counterfactual model is proposed to deal with regression tasks and uncover the causal link between brain connections and disability. A counterfactual explanation describes a causal situation in which, if a specific event X had not occurred, a consequent event Y would not occur. In practice, counterfactual thinking requires to imagine a hypothetical situation that has in fact never happened. In other words, it pro-not-null feature values are selected. Finally, in order to define a unique set of features the union of all base-learner sets is considered.

Local Interpretability Model

A local counterfactual explanation refers to the technique with which a ML blackbox model can be probed in order to understand why it made a certain prediction in a pre-confined area of the output space. In order to define a valid counterfactual, three requirements need to be met:

1. prediction obtained from counterfactual should be as close as possible to the objective output;

2. counterfactual should be as similar as possible to the original instance, which translates in changing as few features as possible;

3. counterfactual instance should have feature values that are likely

In order to offer a feasible interpretation of the model behavior, the output space (EDSS score) has been partitioned in three levels of disability (Low [1 -3], Medium (3 -4.5] and High (4.5 -8]) defined by our neurologist (FDD) coherently with the literature [START_REF] Preston | Electromyography and neuromuscular disorders clinical-electrophysiologic correlations. boston oxford johannesburg melbourne new delhi singapore; basic nerve conduction study[END_REF]. For each disability levels, a truncated normal distribution is assumed as described in Eq. (8.8).

f (x; µ, σ, x 1 , x 2 ) = 1 σ φ( x-µ σ ) Φ( x 2 -µ σ ) -Φ( x 1 -µ σ ) (8.8)
if x 1 < x < x 2 and 0 otherwise where x 1 and x 2 defines the upper and lower bounds of each distribution respectively. The parameter µ defines the center of the class and σ defines its dispersion, while φ represents the standard normal distribution and Φ its cumulative distribution. The applied strategy for constructing counterfactuals is based on input perturbations of patients data (features perturbation). In Figure 8.2, a graphical representation of the perturbation process is depicted. It is important to notice that the conditional distribution in Eq.(8.8) is only used to retrieve the final score to be optimized, expressed in terms of probability. In other words, given a patient in the center-mass (tail) of the distribution (red area) of a specific disability class (either low, medium or high), the algorithm aims at "moving" the subject to the tail (center-mass) of the distribution (purple area) without switching to a new class. This is equivalent to say that, the probability of a patient to pertain to a specific class should decrease (increase), in absolute value, under (over) the threshold obtained substituting the corresponding µ ± σ into Eq. (8.8). In order to satisfy the requirements for good counterfactuals, conditional probability distributions are considered for applying perturbations. The mathematical formulation is offered in Eq. (8.9)

S x i,j,z = {U dx 1 ,dx 2 | d x i,j,z ∀ i ∈ 1, ..., | d x j,z } (8.9)
where d x j,z defines the percentile distribution over the j-th edge and the z-th EDSS disability class, while U dx 1 ,dx 2 represents the uniform distribution between the percentile x 1 and x 2 :

U dx 1 ,dx 2 =      1 x 2 -x 1 , ∀x i ∈ [x 1 , x 2 ] 0, ∀ x 1 ≤ x i | x i ≥ x 2
which implies that the number of connections between the x 1 -th and the x 2 -th percentile is equally likely. The goal is to perturb the lowest number of edges in order to increase (or decrease) the likelihood of a subject to make him/her more (or less) likely to pertain a certain disability class of interest (move a subject from the tail to the center-mass or viceversa). This process ensures the first two requirements for a good counterfactual. In order to satisfy the third requirement, the conditional probability distribution is used to perturb the original input. This corroborates the idea of a highly likely counterfactual since it lies inside the manifold of the true data. Following this line of reasoning, an iterative process is applied where, for each subject, one thousand counterfactuals are generated conditioning on the disability class. The causal effect of each edge onto the probability of pertaining to a pre-defined class of disability is obtained by conditional logistic regression (Eq. (8.10)).

P (Y | X, β, π) = e β 0 +Xβ 1 + e β 0 +Xβ π class (8.10) where π defines the disability class of interest. The process is repeated for each patient and the average of the β parameters is considered.

Results

In this section, the results obtained from both the model performance and the interpretability methods are presented. For each base model used in the stacking, only a limited number of parameters have been tuned. For all the boosting models 300 trees were used except for the XGBoost model for which 500 trees were used. The learning rate was imposed to 0.2 while the maximum depth parameter was imposed at 3 for all the models but XGBoost for which 5 was used. All other parameters were left unchanged at their default values. In order to evaluate results, 10-fold cross-validation analysis was performed.

Concerning the counterfactual model, the parameter µ is imposed to be the center disability class value (low: 2 -medium: 3.75 -high: 5.5), except for the high disability class for which a lower value is considered to compensate for its skewness. The σ parameter was imposed to 0.5 for all the classes except for the middle disability class for which 0.25 was used to keep proportion unaltered.

Results were evaluated by means of Root Mean Squared Error (RMSE):

RM SE = 1 n n i=1 fi -µ i σ i 2 (8.11)
which measures the weighted average root squared difference between the estimated values and the actual value. Moreover, in order to avoid possible misinterpretations of the true predicting power due to outliers errors, Mean Absolute Deviation Error (MADE) was also considered:

M ADE = 1 n n i=1 fi -µ i σ i (8.12)
In both cases, the parameter σ is imposed to be a unitary vector while f defines the predicted value from the model and µ the expected value.

Predictive Model Performance

First, with the aim of having a baseline result for comparison, conventional MRI information, obtained combining T1w image with FLAIR image, were used for training. The average RMSE over all 10 folds was computed for each base-model as well as for the ensemble model. Results are illustrated in Tab. 8.1. Then, structural connectivity was used for training. Again, single base-model average RMSE was computed along with the average RMSE of the ensemble model and reported in Tab. 8.1. A superior performance of the stacking ensemble can be observed (RMSE 0.92 ± 0.28 whilst the MADE score is 0.8 ± 0.7). The discrepancy between RMSE and MADE suggested that for some subjects the estimated EDSS score is biased, leading to a skewed distribution of the error prediction with a longer tail on the right-hand side. Additionally, the non-parametric Wilcoxon Mann-Whitney test of hypothesis is used for comparing the performance obtained from connectome data with the one obtained using conventional MRI measures. It is possible to conclude that the former provide a significant advantage (p-value < 0.05), with the only exception of linear regression (LR).

A more detailed analysis of the proposed ensemble model was conducted, when using structural connectivity as input. Figure 8.3 depicts the error distribution of the proposed ensemble by disability levels. A negative correlation between disability and prediction error can be noticed. Less disabled patients (EDSS of [0, 4.5] ) show the highest error. Conversely, higher level of disability correlates with a lower estimation error. The ANOVA test of hypothesis was performed in order to asses the statistical difference between the groups (F-score: 19.54).

Interpretability of the Model

The predictive capability of the ML model is combined with global and local explanation in order to extract insights of the model behavior. In this study, the interpretability of ML model is used to respond to two main questions:

• which are the common characteristics that discriminate subjects with low disability status from those with a medium or high disability score?

• which are the physiopathological insights that can be derived from these results?

To better understand the global contribution of each brain connection in estimating disability, Figure 8.5 depicts the relative edge importance, obtained through Eq. (8.7), sorted in ascending order. Only the first 200 most relevant edges are reported for visual purposes. The red line delimits the point from which the cumulative relative importance starts to increase exponentially to a value higher than 10%. From this, it is possible to infer that if we are able to tolerate an increase in error prediction of approximately 10%, only the first 56 most important edges can be considered (those on the right-hand side of the red line) which represents the 2% of the number of total connections. With the aim of investigating the global interpretability of the proposed model, Figure 8.6 reports the most important brain regions useful for estimating the EDSS score. For each brain region, the average importance of all the connections contained in a specific region of interest is calculated based on Eq. (8.7). Frontal, Temporal and Parietal are the areas that stand out followed by Sub-Cortical GM regions. In particular, the highest number of connections used by the model are observed in Frontal-Frontal, Frontal-Parietal, Temporal-Temporal and Temporal-Parietal links. These regions correlate with damages usually observed in MS patients [START_REF] Andica | Gray matter alterations in early and late relapsing-remitting multiple sclerosis evaluated with synthetic quantitative magnetic resonance imaging[END_REF][START_REF] Bonavita | Morphostructural MRI abnormalities related to neuropsychiatric disorders associated to multiple sclerosis[END_REF][START_REF]Correlation between white matter damage and gray matter lesions in multiple sclerosis patients[END_REF].

Single connections between specific brain areas are also investigated. For visual purposes, in Table 8.2 only the edges with importance higher than the 95-th percentile threshold are reported. The importance score is obtained by normalizing the logistic regression coefficients calculated after perturbing the input as explained in Section 8.5.2. The Left Banks Superior Temporal Sulcus represents the most important area for all the three classes of disability. In fact, the reference score of importance (with a value of 1.0) involves the aforementioned area in all three cases. Considering a threshold of importance of 0.5 (the median of the importance range) in two out of three levels (low, medium and high), all the values are higher than the threshold (Left Banks Superior Temporal Sulcus -Left Posterior Cingulate and Left Banks Superior Temporal Sulcus -Right Precuneus), confirming the importance of these areas for disability level discrimination. To better understand the strength of the brain WM fiber connections and the effects of the demyelination in patients with MS, Table 8.3 reports the average number of WM fiber connections between the brain regions selected by our model for the three disability classes under investigation. In most cases, the number of fiber links more than halve its dimension. With the objective to understand if the fiber thickness reduction is statistically significant, an ANOVA statistical test of hypothesis is performed. Thirteen out of nineteen connections resulted statistically significant at a 5% confidence level meaning that a significant reduction in fiber bundles thickness is present due to the worsening of the disease between the selected links. However, one might expect that such significant strength reduction is expected as the pathology progress. For this reason the ANOVA statistical test of hypothesis has been repeated considering all connections for a baseline comparison. Notice that only 10% and 7% of the total WM fiber links resulted statistically significant at a 5% and 1% confidence level respectively, confirming that the model focus on the fibers with a significant reduction in thickness in order to estimate disability. Finally, by looking at Table 8.2 and Table 8.3, two additional aspects need to be highlighted. On the one hand, the connections in the left hemisphere are the most relevant for predicting disability. In fact, by considering the 95-th percentile of the most important brain connections, 68% of the areas are from the left hemisphere coherently with what shown in the literature [START_REF] Preziosa | Progression of regional atrophy in the left hemisphere contributes to clinical and cognitive deterioration in multiple sclerosis: A 5-year study[END_REF]. On the other hand, interhemispheric connectivity represents another aspect of interest. The rate between intra and inter-hemispheric connections resulted to be 1: 0.8 while the percentage of inter-hemispheric connections over the total connectivity density is 45%. For comparison, the overall percentage of inter-hemispheric connections rate over all patients is 14%.

In order to exclude a causal link between the aforementioned results and the 95-th percentile threshold, a sensitivity analysis was performed considering all possible threshold values in the range [0 -0.9]. In particular, Figure 8.7 depicts the percentage of left-hemispheric connections (blue) and inter-hemispheric connections (red) for different threshold levels. The obtained representation was derived by applying a line search over all possible threshold values for which the number of left and inter-hemispheric connections is calculated. To obtain a reliable representation of the curve dynamic, onedimensional gaussian filter is applied at multiple alpha levels in the range [0.1 -2) at steps of 0.1. A negative and positive correlation is observed for the left and inter-hemispheric connections respectively. On the one hand, by increasing the absolute marginal effect threshold, from which a connection is deemed to be relevant, a clear increase in number of inter-hemispheric connections can be observed. On the other hand, an increase in threshold value lead to a decrease in number of left-hemispheric connections.

Discussion

MS is a neurological condition for which symptoms and severity can vary widely as the disease can affect both physical and cognitive functioning. With the aim to understand the underline dynamics between disability and brain connectivity, an ensemble of boosting methods was proposed and the ability of the model to estimate disability, measured by EDSS, was assessed. The capability of our method to discover latent relationships correlated with disability progression opens new and interesting perspectives for further investigations. Notwithstanding, we demonstrated that our method outperformed the results obtained by Marzullo et al. [START_REF] Marzullo | Prediction of multiple sclerosis patient disability from structural connectivity using convolutional neural networks[END_REF] for which a convolutional neural network (CNN) was considered (RMSE 1.08 ± 0.09). However, their results remain coherent with what is reported in this study. In particular, by observing the error distribution, a clear difference between low and high disability status is highlighted. This is expected since higher levels of disability correlates (F-score: 19.54) with more degenerated WM fiber bundles, which are easily detectable by the model through the connectome based representation. On the contrary, defining a consistent evaluation of disability in the lower range of the EDSS scale is cumbersome as already confirmed by the literature [START_REF] Ebers | Disease evolution in multiple sclerosis[END_REF]. In the early stage of MS, clinical symptoms are highly variable, and the resulting EDSS scoring is more difficult to obtain and thus less accurate. Furthermore, DTI-based structural connectivity data represents an interesting alternative to the conventional clinical measures, like WM lesion volumes, for EDSS estimation. In fact, while it may seem obvious that patients who develop new lesions are worse off than those without new lesions, there is weak direct correlation with the clinical picture [START_REF] Barkhof | The clinico-radiological paradox in multiple sclerosis revisited[END_REF], defining the well known clinical paradox. In this study, DTI was used to characterize WM alterations in MS patients, avoiding the drawbacks brought by the simple lesion load analysis and achieving good level of performance. In fact, we demonstrated that the use of clinical variables lead to a worsening of the estimation performance compared to DTI data. In contrast, DTI acquisition is more cumbersome to obtain in clinical routine. Notwithstanding, our approach provides a global and local interpretability tool to better understand the underlying damaged connections leading the predictive model performance. Global analysis highlights the importance of Frontal, Temporal and Parietal brain regions. The involvement of these regions were also discovered in previous studies for which major depression and obsessive-compulsive disorder was present in patients with MS [START_REF] Bonavita | Morphostructural MRI abnormalities related to neuropsychiatric disorders associated to multiple sclerosis[END_REF]. Cognitive impairment and behavioural changes are well known to reflect brain damages in the Fronto-Temporal parts of the brain. Different studies demonstrated the link between these areas and the autonomy in performing activities of daily living [START_REF] Curral | Dementia in multiple sclerosis: report of a case with cortical gray matter involvement and frontotemporal-like clinical features[END_REF].

Aside from global brain region, local WM link analysis showed the importance of Left Berardino BARILE Banks Superior Temporal Sulcus and Left Caudal Anterior Cingulate areas. The former is the sulcus separating the superior temporal gyrus from the middle temporal gyrus in the temporal lobe of the brain. Interestingly, its thickness has been found to correlate with cognitive performance and motor skills in patients with MS [START_REF] Achiron | Superior temporal gyrus thickness correlates with cognitive performance in multiple sclerosis[END_REF]. The latter represents the frontal part of the cingulate cortex surrounding the frontal part of the Corpus Callosum. The reduction in fiber thickness is associated with loss in verbal and figural fluency as well as with hypo energetic cognitive functioning in MS patients [START_REF] Geisseler | Cortical thinning in the anterior cingulate cortex predicts multiple sclerosis patients' fluency performance in a lateralised manner[END_REF][START_REF] Parisi | Cognitive rehabilitation correlates with the functional connectivity of the anterior cingulate cortex in patients with multiple sclerosis[END_REF]. Finally, the sensitivity analysis confirmed the intuition of the lateralized localization of cognitive dysfunction and the importance of intra-hemispheric connections. On the one hand, cortical thickness is usually reduced in the left hemisphere [START_REF] Preziosa | Progression of regional atrophy in the left hemisphere contributes to clinical and cognitive deterioration in multiple sclerosis: A 5-year study[END_REF]. In particular, the reduction of gray matter thickness is found to be correlated with cognitive dysfunction in MS patients, especially in the early stage of the disease [START_REF] Achiron | Superior temporal gyrus thickness correlates with cognitive performance in multiple sclerosis[END_REF][START_REF] Pan | Cognitive dysfunction lateralizes with NAA in multiple sclerosis[END_REF]. A different vulnerability of the two brain hemispheres to irreversible structural damage may be among the factors contributing to clinical and cognitive worsening of the disease. Moreover, a growing number of studies have suggested that brain hemispheres have different susceptibility to damage accumulation [START_REF] Preziosa | Progression of regional atrophy in the left hemisphere contributes to clinical and cognitive deterioration in multiple sclerosis: A 5-year study[END_REF][START_REF] Thompson | Dynamics of gray matter loss in alzheimer's disease[END_REF]. These reports advocate that left hemisphere, which is dominant for both handedness and language functions, might be more vulnerable to the accumulation of damage as a consequence of its overuse. The different vulnerability of the two cerebral hemispheres has been already observed in neurodegenerative diseases such as Alzheimer [START_REF] Filippi | Does hemispheric dominance influence brain lesion distribution in multiple sclerosis?[END_REF]. On the contrary, neuroimmune endocrine circuits, which are relevant in MS pathogenesis, are known to have a different hemispheric distribution and might be responsible for inter-hemispheric differences in lesion formation. The importance of inter-hemispheric connections for disability prediction confirms the previous knowledge already reported in the literature, that the corpus callosum plays a significant role in explaining MS evolution. The corpus callosum is the largest fiber tract in the brain and it is responsible for the formation of the major connections between the two cerebral hemispheres and, in most cases, MS patients report significant WM fiber bundles abnormality in this tract [START_REF] Evangelou | Regional axonal loss in the corpus callosum correlates with cerebral white matter lesion volume and distribution in multiple sclerosis[END_REF][START_REF] Ozturk | MRI of the Corpus Callosum in multiple sclerosis: association with disability[END_REF]. Notwithstanding, this study has few limitations. First, the connectome analysis is performed employing a Desikan atlas [START_REF] Desikan | An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest[END_REF], gross parcellation of the brain, and an exclusion of the spinal cord where lesions have a significant role in determining disability. Second, the skewness of the EDSS distribution heavily affects the final results. In fact, only very few subjects with an EDSS over 6 were initially included to prioritize patients with less damage. Such abnormal distribution along the EDSS scale have certainly lead to higher uncertainty in this range.

Conclusion

In this study, an implementation of stacking boosting generalization is proposed with the aim to estimate disability in MS patients. We demonstrated the capability of our model to reach good level of performance with an average RMSE value of 0.92 (± 0.28). In order to better understand the model operation and identify the drawbacks in the estimation of the EDSS score, a detailed analysis based on three predefined classes of disability and four clinical forms are provided. In this regard, patients in the lower disability scale report a higher error compared with more disabled patients. Furthermore, with the aim of explaining the complex decision roles followed by the ML algorithms, a new counterfactual model evaluation is presented. For each "true" patient, multiple counterfactuals have been generated and a statistical analysis was performed in order to understand the line of reasoning followed by the ML model. Interestingly, four main outcomes are worth emphasizing. First, from the global interpretability point of view, frontal, temporal and parietal brain regions are detected as the most ones. Second, concerning local interpretability, Left Banks Superior Temporal Sulcus and Left Caudal Anterior Cingulate are identified as the most relevant areas. Third, a clear left hemispheric lateralization is detected, which implies a more likely WM fiber damage in areas located on the left part of the brain. Finally, intra-hemispheric connections are also deemed to be relevant since they involve the Corpus Callosum fiber tract often significantly affected by the illness. Previous work has demonstrated that pathways connecting the supplementary motor areas, which traverse the central portion of the Corpus Callosum, may play a role in explaining disability. All these findings are in agreement with the literature.

Chapter 9 Ensemble of Boosting Models Using White Matter Fiber Bundles

This work represents an extension of the work described in Chapter 8. The results reported in this Chapter describes a preliminary attempt to link disability directly to fiber bundle tracks, overcoming the problem provided by connectome data. Also, it represents an additional validation of the methodology proposed in Chapter 8 with results coherent with our a priori expectation and with the current literature.

Introduction

In previous studies, the analysis of Grey Matter (GM) structures has demonstrated to be a powerful tool to evaluate the dynamics of disease progression [START_REF] Mesaros | Thalamic damage predicts the evolution of primary-progressive multiple sclerosis at 5 years[END_REF]. In WM, a relationship between damaging and repairing mechanisms that occur in the lesions formation is revealed [START_REF] Rovira | Magnetic resonance monitoring of lesion evolution in multiple sclerosis[END_REF]. Few Machine Learning (ML) approaches have already shown their prognostic value based on conventional MRI measures, which may include the volume of new and/or enlarging T2w-lesions and/or gadolinium-enhanced active T1w-lesions of WM, as markers of inflammation, and the atrophy of global brain and/or GM and the volume of chronic black-holes T1w-lesions of WM, as markers of degeneration. Combined all together, these conventional MRI measures can provide surrogate markers of pathological mechanisms and disease evolution if longitudinal studies are performed. However, weak correlation between Magnetic Resonance Imaging (MRI) lesion load and clinical disability (even when lesion size is considered) constitutes an important challenge and it is usually referred to as clinical-radiological paradox [START_REF] Barkhof | The clinico-radiological paradox in multiple sclerosis revisited[END_REF]. Several studies have attempted to exploit the power of DTI [START_REF] Stamile | Multiparametric non-negative matrix factorization for longitudinal variations detection in white-matter fiber bundles[END_REF][START_REF] Kocevar | Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses[END_REF] due to its great sensitivity to detect microarchitectural alterations of normal appearing WM tissue [START_REF] Jütten | Diffusion tensor imaging reveals microstructural heterogeneity of normal-appearing white matter and related cognitive dysfunction in glioma patients[END_REF]. For instance, Marzullo et al. [START_REF] Marzullo | Classification of multiple sclerosis clinical profiles via graph convolutional neural networks[END_REF] exploited DTI brain graph to estimate the EDSS by means of Convolutional Neural Network (CNN). However, the connectome representation of the human brain does not allow an easy identification of fiber degeneration. Moreover, multiple apriori important fiber bundles are not identified, like the Cortical-Spinal Track (CST) and the Corpus Callosum (CC), and they can only be inferred. Thus, to better characterize the relationship between degenerated fibers and disability, a lower level analysis is in order. By merging the spatial information of fiber tracking with the diffusion metrics derived from the diffusion tensor, it is possible to characterize the presence of "pathological events" (anomalies) that may occur along the WM fiber pathways. Moreover, DTI gives information about water diffusion. In particular, Fractional Anisotropy (FA) measures the degree of anisotropy (diffusion in any direction) of water molecules within tissues (i.e. axons). Its measures can reveal microstructural alterations in normal-appearing WM fibers before being visible in conventional MRI [START_REF] Commowick | Detection of dti white matter abnormalities in multiple sclerosis patients[END_REF].

In this work, we propose a fully automated method to estimate MS disability (measured by the EDSS score) based on Fractional Anisotropy (FA) values extracted from fiber bundle tracks of MS brain patients. Our approach is able to exploit the huge amount of data provided by DTI acquisition reaching competitive level of performance. Moreover, by using single fiber tracks for each bundle in the brain we are able to exactly locate the most influential bundle responsible for the prediction of the EDSS score.

Methods

Data Pre-Processing

In the following, a detailed description of the background techniques and methods used for this study is provided. The same dataset described in Chapter 8 of the present manuscript was used for the analysis. The entire data processing is composed of three steps: i) registration and pre-processing of DTI data; ii) tractography and fiber bundle extraction; iii) comparison between the estimated fiber bundle bootstrapped signal obtained from healthy controls and that of MS patients. Diffusion images were processed using the FRMIB software Library (FSL) [70]. First, eddy current correction filter was applied to the 24 diffusion volumes using the b 0 image as reference. The FA values were computed using the FDT module of FSL. Finally, all diffusion maps of each subject were co-registered (non-rigid) on the Illinois Institute of Technology Atlas (IIT3) [START_REF] Varentsova | Development of a high angular resolution diffusion imaging human brain template[END_REF]. In order to extract the fiber bundles, all the 20 regions of interests (ROIs) contained in the JHU fiber bundle atlas [START_REF] Hua | Tract probability maps in stereotaxic spaces: analyses of white matter anatomy and tract-specific quantification[END_REF] were used as seeds and masks for tractography. To this end, a probabilistic streamline approach was applied to the diffusion data of IIT3 atlas using the MRTrix probabilistic tractography algorithm [START_REF] Tournier | MRtrix: diffusion tractography in crossing fiber regions[END_REF]. This process was repeated for each fiber-bundle in the atlas. The suppression of the false-positive fibers obtained during the tractography was performed by post-processing tractography outputs. More in detail for each fiber bundle the Quick Bundle cluster algorithm [START_REF] Garyfallidis | Quickbundles, a method for tractography simplification[END_REF] was applied and only the cluster with the highest mean fibers length was selected. The last process consisted of the automatic extraction of the diffusion metrics from the fiber bundle. Based on the resampled fibers, each fiber point (x i , y i , z i ) was associated with the FA diffusion metric value of its corresponding voxel, allowing the characterization of the diffusion properties of the entire bundle. The corresponding table of the FA diffusion metric was generated in which each entry of the feature vector contains the diffusion value of one of the n voxels belonging to the fiber bundle. Note that due to the different lengths and width of the fiber bundles, the size of the feature vectors is not the same among them. Notwithstanding, while the width of each fiber bundle is composed of a different number of streamline fibers proportional to the fiber bundle thickness, the length of each fiber is standardized to a length of one-hundred values as proposed in [START_REF] Zhang | Evaluation of standardized and study-specific diffusion tensor imaging templates of the adult human brain: Template characteristics, spatial normalization accuracy, and detection of small inter-group fa differences[END_REF].

Unsupervised Feature Extraction

In ML, feature selection is one of the most important steps during data preparation. Its role is to highlight relevant features that can be easily exploited by the ML model in order to achieve better performances and/or higher stability during training. In this work, an Berardino BARILE iterative Monte-Carlo-based approach is proposed to detect anomalous variation of water diffusion (measure by FA) along the fiber bundle of an MS patient (possible fiber degeneration). More precisely, two main steps are applied: i) creation of a standard fiber bundle atlas of FA values from 24 healthy controls; ii) probabilistic comparison between the healthy FA atlas with that of an MS patient (per bundle comparison). Firstly, one-hundred random streamline fibers (corresponding to roughly 10% of the total streamline fibers inside a bundle) were extracted from a bundle of a healthy controls. Then, the average of all sampled FA signal streamlines was calculated obtaining an unbiased approximation of the bundle signal profile. The process was repeated for each bundle of a subject and for each subject. The overall (standard) representation of the signal profile for a bundle was calculated applying a bootstrap resampling strategy on all the 24 healthy controls and repeated independently for each bundle. Henceforth, the corresponding bootstrapped means and standard deviations were obtained. In the second step, the same procedure was applied for each MS patient individually (without the bootstrap resampling step). One-hundred random streamlines fibers were extracted from each bundle and the average signal profile is retrieved. The mean signal profile is thus compared with the corresponding mean bundle signal calculated from the healthy controls and a binary vector is generated (Eq. (9.1)).

I ± (µ(x β,κ ), µ(x β,κ )) ∀κ∈K;β∈B =              vec(1) + , if x β,κ > 3σ(µ(x β,κ )) vec(1) -, if x β,κ < 3σ(µ(x β,κ )) 0 otherwise (9.1)
The subscript K defines the set of all possible FA values along a fiber bundle (length of a bundle), B represents the set of all bundles (20 from JHU atlas) while µ and µ define the mean FA values for the MS patient and the corresponding bootstrapped mean FA signal obtained from the healthy controls, respectively. Finally, vec(1) + and vec(1) -define the vector of anomalies, above and below the average standard confidence distribution respectively. In other words, for an MS patient, the calculated mean FA value is deemed to be anomalous (values of 1) if it differs more than 3 standard deviations (3σ) from the corresponding mean bootstrapped value obtained from healthy controls. In order to obtain a statistically robust identification of the anomalies, the entire process is repeated onehundred times for each MS patient. At each iteration, the binary vector I ± (µ, µ) from Eq. (9.1) is calculated. The final anomaly frequency is computed as the accumulated sum of all binary vectors. Mathematically, this translates as:

1 G G g=1 I ± g (µ(x β,s ), µ(x β,s
)) where G defines the number of iterations. Notice that due to the independent sampling procedure, the resulting frequency vector can be interpreted probabilistically.

Proposed Model

In this work, an ensemble of four tree-based ML models (ExtraTreesRegressor [START_REF] Geurts | Extremely randomized trees[END_REF], XGBoost [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF], LightBoost [START_REF] Ke | Lightgbm: A highly efficient gradient boosting decision tree[END_REF], CatBoost [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF]) are combined in a meta-model in order to estimate EDSS score from MS patients. The motivation for using ensemble models is to reduce the generalization error of the prediction. As long as the base-models are diverse and independent, the prediction error of the meta-model decreases when the ensemble approach is used [START_REF] Kotu | Predictive analytics and data mining[END_REF]. The motivation behind the selection of these four models lies in their well-recognized performances with high dimensional data [START_REF] Buhlmann | Boosting methods: Why they can be useful for highdimensional data[END_REF][START_REF] Xu | Classifying very high-dimensional data with random forests built from small subspaces[END_REF]. Moreover, each of the base-model selected in this work tackles the regression problem differently, providing a useful contribution to the ensemble. In order to combine the single prediction obtained from each base-model, the average predicted score is calculated.

Model Interpretability

In order to identify the most influential bundles used by the model to predict EDSS score, a perturbation-based algorithm is derived. Let's define with α ∼ U nif orm(x lower , x upper ) the amount of perturbation to be imposed, with γ ∼ Bernouilli(p) the sign of the perturbation, with β lef t and β right (β major and β minor for Corpus Callosum (CC)) the right and left bundle to be perturbed and with χ β the cross-sectional FA values for the specific bundle. The following equation is applied:

χ β = χ β lef t × α 1 × γ 1 ++χ β lef t × α 2 × γ 2 (9.2)
where + + refers to concatenation. In other words, the coupled left-right bundle (majorminor for CC) is perturbed by increasing (decreasing) the corresponding FA values of a bundle along its entire cross-section by a percentage sampled from a uniform distribution in the range [0-0.5]. The sign of the perturbation (+: increase FA, -: decrease FA) is modeled by a Bernoulli distribution with p=0.5. It is important to notice that while the perturbation is applied uniformly to the entire cross-section of a bundle, preserving the original form (shifting the entire signal profile), the percentage and sign of the perturbation and its magnitude for the left (major for CC) and right (minor for CC) bundle is independently sampled. Moreover, in order to avoid unrealistic perturbations, both left and right (major and minor) bundles are considered simultaneously and their relationship evaluated by means of joint multivariate likelihood. The mathematical formulation is provided in Eq. where κ defines the cross-sectional dimension of the bundle (100 points), λ the estimated parameter of a Poisson distribution, β the included bundle and ζ the upper bound of likelihood discrepancy between the perturbed sample with respect to the original data. In other words, the perturbed dataset is selected if and only if its likelihood has a radius distance less than ζ from that of the original data.

Parameter Tuning and Performance Evaluation

Ensemble model is a well-known method used for reducing the variance of the prediction error, achieving better level of performance compared with each single base-method alone [START_REF] Kotu | Predictive analytics and data mining[END_REF]. However, the drawback of this method lies in the higher computational cost and increased complexity for parameter tuning. In order to tackle these problems, two steps are applied. Firstly, in order to limit the computational cost of the entire process, the dimensionality of the dataset (≥ 10000 signal values for each bundle) is efficiently reduced by the pre-processing step discussed in Section 9.2.2. Secondly, in order to perform an effective parameter tuning, bayesian optimization method [START_REF] Snoek | Practical bayesian optimization of machine learning algorithms[END_REF] was employed using cross-validation. Among all parameters, four resulted to be the most effective. In particular, the maximum depth of an internal tree was imposed to 6 for all base-models, while the minimum number of samples required to split an internal node was 4 and the learning rate was imposed to 0.01 (only for boosting methods). The number of estimators (shallow trees) was imposed to 30 along with the DART boosting method [START_REF] Rashmi | Dart: Dropouts meet multiple additive regression trees[END_REF] for the LightBoost model while for all others 100 trees were used. The max number of features to sample during tree fitting is imposed to 30% for all estimators while the subsampling of instances was imposed to 50%. In order to evaluate the model, a 10-fold cross validation analysis is performed. Three metrics were used to evaluate the prediction performance: i) Root Mean Squared Error (RMSE) defined as (||x ix i || 2

2 ) 1/2 ii) Mean Absolute Deviation Error (MADE) defined as ||x ix i || 2 and iii) Coefficient of Determination (R 2 ) defined as 1 -SSres SStot where SS res = i (x ixi ) and SS tot = i (x ixi ).

Results

The results obtained from the 10-fold cross-validation strategy show the capability of the ensemble model to reach good level of performance. The average RMSE score obtained is 1.08 (±0.12) and the MADE score resulted to be 0.82 (±0.10). Notwithstanding, it is interesting to measure how much of the total EDSS variance the ensemble 200 Berardino BARILE model is able to explain, on average, from the hold-out samples. In this regard, the R 2 score is also proposed, resulting in a value of 0.58 (±0.07). With the aim of investigating the impact of the standardized healthy subject to the final prediction performance of the ensemble model, the experiment has been repeated using the standard FA atlas provided by Neuro-Imaging Tools and Resources Collaboratory (NITRC) [START_REF] Zhang | Evaluation of standardized and study-specific diffusion tensor imaging templates of the adult human brain: Template characteristics, spatial normalization accuracy, and detection of small inter-group fa differences[END_REF]. The average RMSE score obtained using the NITRC fiber bundle atlas is 1.12 (±0.10) while the MADE score resulted to be 0.87 (±0.09) and the R 2 score was 0.51 (±0.09). Due to the high skewness of the residual error distribution (RMSE>MADE implying a long right tail), the Mann-Whitney U test of hypothesis was used. The statistical test did not reject the null hypothesis of equal performances and any statistical evidence in terms of difference in median can be inferred between the two approaches (p-value=0.58). In order to dive deeper into the model performance, Fig. 9.1 depicts the density estimation obtained from Gaussian kernel of the error prediction conditioning on three classes of disability (Low [1 -3], Medium (3 -4.5], and High (4.5 -8]), defined by our neurologist (FDD) and coherent with the literature [START_REF] Kurtzke | Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS)[END_REF]. From the image, it is possible to notice that, the conditional distributions statistically differ between the three disability classes (ANOVA F-score: 919.35). The low disability class reports the highest prediction error and the highest dispersion (MSE = 2.75 ± 0.52). Conversely, the medium disability class reports the lowest error and the lowest dispersion with an MSE score of 0.29 (± 0.07) while the high disability class reported an MSE of 1.35 (±0.32). As long as the interpretability model is concerned, Figure 9.2 depicts the degree of importance for each major bundle coupled. The highest importance was obtained for the Cortico-Spinal Tract (CST) bundle followed by the Corpus Callosum (CC) as can be noticed in the image on the left-handside (panel A). The remaining bundles report a significantly lower score of importance. Notwithstanding, it is interesting to obtain a deeper understanding of the importance of the bundles when the CST and CC are not taken into account. For this reason, Fig. 9.2-B, reports the importance distribution for each bundle when the CST and CC are omitted. The highest level of importance is obtained for the Hippocampus (HP) and the Anterior Thalamic Radiation (ATR).

Discussion and Conclusion

In this work, an ensemble of tree-based ML models is developed in order to estimate EDSS score from MS patients. Additionally, a new interpretability model, based on fiber bundle perturbation is developed in order to evaluate the contribution of each bundle to the prediction of the disability score. However, due to the high dimensionality of the dataset (≥ 200000 considering features for the whole brain), a pre-processing step is in Berardino BARILE

Conclusion

Artificial Intelligence represents an invaluable tool for the analysis of complex data structures and the automation of complex decisions without human intervention. In the biomedical domain, an ever increasing number of applications using automated systems based on Machine Learning (ML) algorithms were developed. Nowadays, the use of MRI data is crucial to better understand the pathological mechanisms of complex brain diseases such as MS. Notwithstanding, in MS, the high variability of the evolution of each patient and the duration of the natural history of the disease provide difficult challenges to the neurologist. Thus, disease treatment remains a difficult task. However, a new interesting approach for the analysis of brain network alterations relates to the concept of graph theory, which represents a powerful approach for the analysis of degenerative brain diseases. In this doctoral thesis we aimed to address three main objectives:

1. Generation of realistic brain connectomes data for the improvement of Multiple Sclerosis profiling in the context of data scarcity and class imbalance 2. Automatic evaluation of grey matter tissue degeneration for Multiple Sclerosis patients profiling

Automatic and interpretable evaluation of Multiple Sclerosis progression based on cognitive and physical patients disability status

Multiple pipelines and Machine Learning approaches were proposed for achieving these three goals. In the following sections, the research contributions regarding these objectives are summarized.

Synthetic data generation in the context of data scarcity and imbalance between MS classes

In the context of biomedical image analysis, one of the most relevant issues relates to the availability of sufficiently large datasets, needed to appropriately train machine learning models. The problem of data scarcity is pervasive in the biomedical image domain, particularly when disabling diseases, such as MS, are considered. Also, in most cases, there is an imbalance in the distribution of different clinical profiles or disease types which naturally describes the probability of particular symptoms to occur. Notwithstanding, such problems may constitute a challenge for modern machine learning frameworks, hindering the effectiveness of the learning process and ultimately reducing the generalization performance on unseen data. Thus, a large amount of data with perfectly balanced 208 Berardino BARILE 10.1. CONCLUSION classes is often an important requirement for reaching good generalization capacity. Multiple attempts have been made for mitigating these problems. A few existing models have been proven effective for traditional tabular datasets, where rows represent instances and columns represent features. However, for more complex data structures, like the one provided by connectomes, these approaches are less effective, especially when the dimensionality of the data is large. In Chapter 5, a new deep-learning-based model was proposed and based on structural connectivity data. The goal of such a model was that of generating meaningful graph representations with biological meaning, where the real structure of the brain, affected by MS, is preserved. In this work, we demonstrated that new biologically meaningful structural connectomes can be generated even in the context of data scarcity and imbalance between classes. To evaluate the quality of the generated connectomes, graph theory was employed along with six of the most important global graph metrics, used to compare true and synthetic data. This work represents an important contribution to the analysis of degenerated MS brain networks. In fact, such generative models can learn the underlying data distribution from which real data are obtained, thus providing meaningful insights that may be exploited for future statistical and bayesian analysis. The study and estimation of the underlying generation process, represents an innovative and important approach for a better understanding of the physiopathological mechanisms guiding the MS disease. Additionally, the proposed framework was able to up-sample the minority class producing a much higher F 1 score (81%) with respect to the baseline unbalanced classification (66%). Furthermore, comparing to other classical oversampling techniques (ROS and SMOTE) and a graph-based adversarial network (ARAE), our method increases the classification performance by approximately +10%. This represents an important results since data insufficiency and strong imbalance between classes are common hallmarks of most of the datasets in the biomedical field. In this work, we demonstrated that meaningful structural patterns between multiple grey matter regions are important for the discrimination of MS profiles, compared to the approach of simply and naively augmenting the dataset by means of classical methods (such as Random Oversampling and Synthetic Oversampling), which do not take into account the biological structural properties of the human brain network.

Evaluation of grey matter tissue degeneration for MS profiling

The problem of MS classification represents a challenging task where patients need to be partitioned into four MS profiles based on cognitive and physical symptoms. From a neurologist's perspective, such a tedious task represents a difficult and time-consuming procedure. It needs to be performed by an experienced neurologist and it represents an important step for a proper diagnosis and treatment of patients. For this reason, the au-tomation of such a task may represent an important milestone which helps to better study the different pathological evolutions of the MS disease. Nowadays, the problem of automatic classification of MS patients is not yet fully tackled in the literature since the lack of available benchmark datasets limits the development of advanced machine learning-based approaches. Conversely, the availability of data related to healthy subjects is abundant and benchmarks dataset are readily available, which represents an incentive to perform automatic discrimination between healthy and pathological brains. For this reason, most of the works proposed in the literature focus on the discrimination between MS patients and healthy controls. Notwithstanding, from a clinical perspective, comparing MS patients with different clinical conditions, represents a more important and challenging task for the discovery of the underlying pathophysiological mechanisms and MS profile characterization. In this doctoral dissertation, a statistical and machine learning approach was proposed for the analysis of degenerative patterns extracted from the grey matter tissue of MS patients. In Chapter 6, a statistical and machine learning analysis was performed for the discrimination between MS clinical profiles based on cortical thickness extracted from each grey matter region. The statistical analysis revealed a high number of significant differences (p<0.05) between the four MS groups, confirming that cortical thickness degeneration can represent an important hallmark for the study of MS disease characterization. In this analysis, we noticed that the discrimination between early stages and progressive courses reported the highest difference in terms of statistical significance. Additionally, for a more robust evaluation of the obtained results, two different atlases were used for grey matter regional parcellation, such as FSAverage and Glasse2016, with respectively 68 and 360 brain regions (i.e. ROIs), and compared repeating all the experiments independently for both cases. All the conclusions remained consistent between the two parcellation strategies with a slight improvement obtained using a smaller regional atlas. It should be noticed that the analysis of grey matter tissue alteration represents an innovative approach to the task of MS profiling. For this reason, a machine learning analysis was also performed and based on traditional statistical learning methods. Such an approach might result more convenient for handling the problem of data scarcity, which characterize the biomedical field, as opposed to the more complex deep learning-based architectures. Four statistical learning methods were trained and combined together in an ensemble model using a late integration scheme based on majority voting. Additionally, for a better understanding of the type of parcellation to be used to obtain the best discrimination performances, the predictive analysis was performed on both atlases. One again, coherent results were obtained between the two parcellation approaches with slightly better classification performances obtained with the Glasser2016 atlas, although not consistent among all binary comparisons. From this analysis we can conclude that only incon-sistent and mild improvement can be obtained using a more refined parcellation strategy while increase the dimensionality of the data. Also, although lower performances were obtained compared with results proposed in the literature and primarily based on white matter tissue analysis, to our best knowledge, our approach represents one of the first attempts to use exclusively the standard anatomical T1w image modality for automatic MS profiling. This approach avoids the burden to acquire the Diffusion Weighted Image (DTI), which represents an expensive and less used modality in clinical applications due to its long acquisition time. In Chapter 7, an evolution of the work described in Chapter 6 is provided. A kernelbased factorization approach was developed for the discrimination between early stages and progressive MS courses by combining the information of three morphological features extracted from the gray matter tissue, such as cortical thickness, curvature and area. With this work, we highlighted the benefit of combining multiple morphological characteristics describing the grey matter tissue degeneration in patients with MS. The multiview approach resulted in superior and faster training with an almost 10% increase in classification performance compared to the approach proposed in Chapter 6 and only based on the analysis of the cortical thickness degeneration. To our knowledge, this work represents the first attempt to perform data fusion using grey matter morphological features for the classification of MS clinical profiles.

Automatic and Interpretable Evaluation of MS disease progression

Magnetic Resonance Imaging (MRI) is today the reference tool for the diagnosis of the MS disease as well as for the follow-up of patients during the disease course. In this thesis, we showed that the power of connectomics, obtained from Diffusion Tensor Image (DTI), as well as from more classical MRI data, such as T1-weighted images, can be exploited for the analysis of both white and grey matter tissue alteration, highlighting the importance of such data structure for human brain network representation. Also, the power of connectome data was combined with cutting-edge ML approaches for the classification and prognosis of MS patients. The use of advanced imaging techniques such as Magnetic Resonance Spectroscopy (MRS) and Diffusion Tensor Images (DTI), offer the prospective for a better understanding of the underlying physiopathological mechanisms characterizing the MS disease and may ultimately allow a better prediction of the patient's clinical courses. For this reason, one of the most important objectives of this doctoral thesis is to better predict MS disability progression measured by the Expected Disability Status Scale (EDSS) score, which represents the most widely used disability Berardino BARILE measure in clinical practice. This objective was handled in Chapter 8 and Chapter 9 listed in Part III of the present Manuscript. In the former, the disability condition of each MS patient was estimated using an ensemble of four boosting-based models combined together following a stacking generalization scheme in which linear regression was used as a high-level model. Other attempts have been proposed in the literature for the evaluation of disability conditions using structural connectivity, like the one proposed by Marzullo et. al [START_REF] Marzullo | Prediction of multiple sclerosis patient disability from structural connectivity using convolutional neural networks[END_REF] and based on a Deep Learning Convolutional architecture. The work proposed in Chapter 8 extends this approach by considering an ensemble of statistical learning methods which, in the context of data scarcity, represents a better choice compared to the data hungry approach provided by deep learning models. Additionally, due to the importance of the analysis of MS disability, an interpretability model was also developed in order to pin-down the most important fiber connections deemed to be relevant by the machine learning models for disability estimation. Such an analysis represents the main goal for an effective treatment of the MS disease and for the improvement of the life quality of patients. The field of interpretable machine learning represents a new and growing area of research which attracted already a lot of attention in the past few years due to its capability to highlight the most important line of reasoning guiding the decision process followed by the machine learning algorithms, which are usually interpreted as "black boxes". The interpretability model proposed in Chapter 8, represents one of the first attempts to provide such an explanation in the context of MS disability estimation. Additionally, the results provided by the interpretable model demonstrated to be coherent with our a-priori expectation and with the existing knowledge already described in the literature, such as the involvement of the Corpus Callosum (CC), which is affected by the MS degeneration process. Notwithstanding, the use of connectome data in the context of MS disability has also some limitations since it does not take into account the Corti-coSpinal Tract (CST), which represents an important part of the central nervous system deeply involved in the process of motor impairment. For this reason, the disability analysis performed using connectivity data was also extended by using fiber bundle signals based on Fractional Anisotropy (FA) calculated from the tractography analysis. Notwithstanding, the use of more refined features extracted from the white matter tissue comes at the cost of a more complicated analysis due to the high dimensionality of the data. For this reason, a Monte-Carlo-based bootstrapping procedure was implemented for the extraction of relevant anomalous regions, obtained by comparing MS patients with healthy subjects. The resulted dataset was used by the ensemble model for the estimation of the EDSS score. Additionally, the same interpretability model proposed in Chapter 8 was used in this context for the underlying pattern discovery followed by the machine learning model. Coherent results were obtained compared to the structural connectivity approach while the CST and the CC were deemed to be the most relevant fiber bundles used by the ensemble model for disability estimation, increasing the confidence in the proposed method.

In conclusion, the combination of connectome data analysis with modern predictive ML approaches, represents a useful tool for the analysis of complex data structures such as that found in the human brain network. Notwithstanding, the works proposed in this doctoral thesis represent only a fraction of the multiple ways of analysis offered by graph theory. For instance, it would be of great interest to investigate the impact of grey matter degeneration on the estimation of physical (EDSS) and functional (MSFC) scores and compare them to the results obtained using the white matter connectome. Additionally, multiple studies have also highlighted the importance of grey matter tissue degeneration as an important biomarker for the prediction of the worsening of clinical conditions. This also represents an interesting evolution of the series of works in which this PhD research can proceed. A more detailed description of the future directions in which the results obtained by this PhD research should evolve is proposed in the next Section.

Future Directions

The findings and contributions described in this present PhD manuscript represent our best attempts to tackle two of the most important problems of machine learning applications for Multiple Sclerosis disease detection and diagnosis. Notwithstanding, they represent only a fraction of the possible evolution of applications in the context of MS disease. In the following subsections, three of the most important ideas are described as natural extensions of the multiple works proposed in this thesis.

Disability estimation based on morphological features from the grey matter tissue

Recent imaging and pathology studies have demonstrated that MS affects both cerebral GM and WM tissue. Characteristic findings in GM include focal regions of demyelination, activated microglia, apoptotic neurons, and atrophy of cortical and deep GM structures. Focal GM lesions are difficult to detect using conventional imaging because of low contrast and small lesion size. However, GM atrophy can be reliably measured from standard MR images, like T1w image. This is exactly what was performed in Chapter 6 and Chapter 7 where three morphological features obtained from the GM tissue, such as cortical thickness, curvature and area, were exploited for the discrimination of MS patients in their respective clinical phenotypes. Notwithstanding, one evident analysis is

Berardino BARILE yet missing from the puzzle, such as the exploitation of the multiview morphological features for MS disability estimation. In fact, in this doctoral thesis, only the WM structural connectome data were used for the estimation of the EDSS score, which, as previously explained, represents one of the most important metrics used in clinical practice for measuring the disease progression. Thus, a natural extension to the list of works proposed in this PhD thesis would be that of considering the information extracted from the degenerated GM tissue to estimate disability of MS patients. Multiple studies have already demonstrated its strong implication in the worsening conditions of patients affected by the MS disease. This analysis was missing from the list of contributions proposed in this doctoral dissertation due to a lack of available time. Notwithstanding, the use of more traditional anatomical images, such as T1w, represents one of the most acquired modalities in clinical applications, justifying the extension of the MS disability estimation analysis to the GM tissue. We already had access to one of the biggest dataset nowadays available in France on the MS disease called "Observatoire Français de la Sclérose en Plaques" (OFSEP) from which more than 2000 MS patients were already preprocessed for the extraction of GM morphological features following the procedure highlighted in Chapter 6 and Chapter 7. To our knowledge, this work will represent one of the first attempts to use the T1w image modality for the estimation of the disability condition characterizing the MS population without the time-consuming approach based on deep learning architectures. Notwithstanding, we also plan to perform additional analysis which may lead to a more in-depth understanding of the real contribution provided by the GM tissue degeneration. More precisely, we first plan to compare the performance of a deep learning model with the approach suggested in Chapter 7 and based on a multilinear SVD factorization model. With this work, the aim will be to show whether the deep learning approach really provides an advantage compared to our method. An ablation study will also be performed in order to understand the minimal number of MS patients required by the deep learning model to improve the performance over our approach. This analysis represents an important contribution since in the biomedical setting the development of effective ML pipelines are hindered by data scarcity. Thus detecting the minimal training size will be of great help for future investigations. Additionally, the interpretability analysis proposed in Chapter 8 and Chapter 9, which has been already demonstrated valuable for discovering the most important lines of reasoning followed by any ML method, will also be applied in this context, providing explainability to the ML procedures and ultimately contributing to a better understanding of the GM tissue degeneration in MS patients.

Predicting longitudinal Grey Matter tissue degeneration as reliable measure for MS disease progression

As stated in the previous section, the EDSS score represents a widely used and wellvalidated scale for measuring MS disability. Since disability progresses only slowly, the EDSS is fairly robust for measuring outcome over the long term. For example, the median time for an MS patient to progress to an EDSS score of 6 from disease onset is 15 years [START_REF] Ebers | Disease evolution in multiple sclerosis[END_REF]. However, for measuring short-term outcome, compatible with the two to three-year time frame, EDSS represents a less reliable score for assessing future disease progression. For this reason, new and more sensible approaches are needed for evaluating possible pathological variations detected in the brain of MS patients over a short period of time, allowing the clinician to provide customizable and fast treatments. We already stated that GM tissue degeneration has been shown to be relevant for the analysis of the worsening conditions of a patient. In the previous section, we highlighted that a new large cohort of MS subjects was acquired from the OFSEP dataset and preprocessed for the calculation of morphological features such as cortical thickness, curvature and area. Thus, one possible extension of the series of works proposed in this PhD manuscript would be that of predicting future GM tissue degeneration in a time frame of two to three years. In fact, GM atrophy was linked with the worsening of cognitive and physical conditions, which deeply affect the quality of life of MS patients. Notwithstanding, the study of longitudinal GM atrophy was already tackled in the literature. For instance in the study of Eshaghi et. al [START_REF] Eshaghi | Progression of regional grey matter atrophy in multiple sclerosis[END_REF], a large cohort of MS subjects was recruited and the association between EDSS score and GM atrophy was performed. Notwithstanding, our work proposal differs from what is already present in the literature on two main points. First, a multiview approach based on all morphological features (i.e., thickness, curvature and area) extracted from the GM tissue will be used, together with an ML-based predictive analysis, as opposed to the more simple statistical association approach. Our aim is to provide to the neurologist, pathways of GM tissue degeneration useful for early intervention and treatment of the MS disease. Notwithstanding, such a task can be performed even in relation to other degenerative brain diseases such as Alzheimer's and Parkinson's. Second, instead of focussing on the EDSS score, which has been shown to be unreliable for low degrees of disability [START_REF] Ebers | Disease evolution in multiple sclerosis[END_REF], a new and more trustworthy score for measuring brain tissue degeneration might be obtained considering the degree of GM tissue atrophy by means of a combination of the three GM morphological characteristics previously introduced. In fact, in the early stages of the disease, which is where treatments need to be shown to be effective in order to preserve patients' autonomy and quality of life, it is more difficult to assign an EDSS score unambiguously. For example, an EDSS of 2 rather than 3 may Berardino BARILE vary between neurologists. For these reasons, potential treatment benefit in clinical trials has principally been determined using softer outcome measures such as relapse rates. The approach herein proposed represents an additional and more reliable data-driven solution for assessing and predicting MS progression, in particular in its early stages.

Improving the automatic diagnosis of MS patients by combining GM and WM tissue information

In MS, both WM and GM tissues are affected by the disease. Notwithstanding, the implication and causal linkage between these two brain tissues are still unclear. In this PhD we already demonstrated the usefulness of ML methods for the discovery of hidden patterns able to provide good performances for MS profiles discrimination and disability estimation. Notwithstanding, the multiple analyses performed in this manuscript consider the white and grey matter features separately in the sense that the main source of information is derived from just one of these two possible sources. For example, in Chapter 6 and Chapter 7, multiple GM morphological features were extracted and used with the aim of discriminating between MS clinical phenotypes. Conversely, in Chapter 5 and 8, the information extracted from the WM fibers were organized in a graph by counting the number of connections between GM regions and represented in an adjacency matrix. One possible evolution of the present PhD work might be that of combining these sources of information by fusing together fiber bundle data, extracted from tractography and including non-brain fibers like the CorticoSpinal Tract and the Corpus Callosum, with the multiview morphological features obtained from the GM tissue. Both of these information sources were demonstrated to be useful for the task of MS profiling and disability estimation and their combination may shed more light on the different contributions that these features may jointly provide. As first step, it might be useful to simply concatenate together these two sources of information and observe which of the features (or combination of them) will be selected by the ML model. This analysis would already provide useful insights into the tissue degeneration occurring at different stages of the MS disease. Second, by segmenting lesions in the WM tissue, it would be possible to estimate a causal link between fiber demyelination and GM tissue atrophy by matching the fibers passing through the lesions with the morphological characteristics of the GM regions to which these fibers are connected. To the best of our knowledge, this analysis has not yet been performed, despite the importance that it may provide for a better understanding of the MS disease.

Technological advances in automated computer methods represent an ever growing area in both research and applied science. The list of works proposed in this doctoral thesis aim at providing a consistent and tangible contribution toward a better use of powerful intelligence systems, which we hope one day, will shed light on the understanding and cure of important life threatening diseases such as Multiple Sclerosis.

Berardino BARILE

Figure 1 :

 1 Figure 1: Overview of the chapters in this PhD manuscript

Figure 1 . 1 :

 11 Figure 1.1: Relationship between Artificial Intelligence, Machine Learning and Deep Learning. Image taken from https://levity.ai/blog/difference-machine-learning-deeplearning

Figure 1 .Figure 1 . 3 :

 113 Figure 1.3: The Train-Test split validation procedure. The original dataset is split in two smaller datasets with usual percentages ratio of 60/40 or 70/30 or 80/20 for training and testing size respectively.

Figure 1 . 4 :

 14 Figure 1.4: The k-fold Cross-Validation procedure. The original dataset is first divided in k different folds. Subsequently, an iterative approach is performed such that one fold is hold-out and used as test set, while the remaining k-1 folds are combined forming the training dataset. The procedure is repeated until all the folds are used exactly once for model validation.

Figure 1 . 5 :

 15 Figure 1.5: The Leave-One-Out Cross-Validation procedure. At each iteration, one instance is selected from the original dataset to evaluate the performance of the machine learning model. The final evaluation is obtained combining the evaluation results of all instances, used exactly once for model evaluation.

Figure 1 . 6 :

 16 Figure 1.6: Confusion Matrix

Figure 1 . 7 :

 17 Figure 1.7: Illustration of the threshold free AUC-ROC value for a binary classification task. Image taken from https://medium.com/the-owl/evaluation-metrics-part-3-47c315e07222

  The overall variance error component is reduced combining multiple deep decision trees, trained on different parts of the same training set. In particular, given a training dataset D T = {(x n , y n ); n = 1, 2, ..., N }, bagging repeatedly (B times) selects a Berardino BARILE random sample with replacement from the training set and fits tree-based models with the following procedure: • Sample, with replacement, N training examples from D T , obtaining a subsample of data D b • Train a classification or regression tree Φ(x|D b )

Figure 1 . 11 :

 111 Figure 1.11: Comparison of four of the most used activation functions in Deep Learning

Figure 1 . 14 :

 114 Figure 1.14: Intuitive representation of an AutoEncoder architecture. Image taken from https://www.alanzucconi.com/2018/03/14/an-introduction-to-autoencoders/

  Figure 1.15:Intuitive representation of a Generative Adversarial Architecture (GAN) in the context of brain image generation.Image taken from https://wiki.pathmind.com/generative-adversarial-network-gan

Figure 1 . 16 :

 116 Figure 1.16: Intuitive representation of the IsoMap algorithm for data in three dimensions. Image taken from https://towardsdatascience.com/isomap-embedding-anawesome-approach-to-non-linear-dimensionality-reduction

  as a triplet of values defining the respective mode-rank, such that R n describes the mode-n rank of the unfolding matrix X (n) . In the following subsections, four of the most used factorization models are introduced, such as: Canonical Polyadic Decomposition (CPD) , Multilinear Singular Value Decomposition (MLSVD), Non-Negative Tucker Decomposition (NTD) and Low Multilinear Rank Approximation (LMLRA).

Figure 1 . 20 :

 120 Figure 1.20: Schematic representation of factorization method based on MLSVD

M 2 (

 2 xy (t)=M xy (0)e -t T

Figure 2 . 1 :

 21 Figure 2.1: Time evolution of the Longitudinal (a) and Transverse (b) magnetization.

Figure 2 . 2 :

 22 Figure 2.2: Description of the classical spin-echo sequence. Image taken from[START_REF] Sinnaeve | The stejskal-tanner equation generalized for any gradient shape. an overview of most pulse sequences measuring free diffusion[END_REF]

  Figure 2.3.

Figure 2 . 3 :

 23 Figure 2.3: Visual comparison between T1-w, T2-w and FLAIR modalities. Image taken from https://case.edu/med/neurology/NR/MRI%20Basics.htm

Figure 2 . 4 :

 24 Figure 2.4: Description of the unmyelinated, partly myelinated, and myelinated tissue.Image taken from[START_REF] Vorisek | Evolution of anisotropic diffusion in the developing rat corpus callosum[END_REF]

Figure 2 . 5 :

 25 Figure 2.5: δ represents the duration of each gradient, ∆ is the interval between the onset of the diffusion gradient before the refocusing pulse and that after the refocusing pulse, G is the amplitude of the diffusion gradient and RF indicates radiofrequency pulses.

Figure 2 . 6 :

 26 Figure 2.6: Representation of the ellipsoid estimated by eigenvalues and eigenvectors of the diffusion matrix.

Figure 2 . 7 :

 27 Figure 2.7: Representation of the fiber tracking algorithm based on the direction of the principal eigenvector component. Arrows represent primary eigenvectors in each voxel.Red lines are reconstructed trajectories. Image taken from[START_REF] Mukherjee | Diffusion Tensor MR Imaging and Fiber Tractography: Theoretic Underpinnings[END_REF]

Figure 2 . 10 :

 210 Figure 2.10: Intuitive representation of different types of graphs. On the left-hand-side the regular graph is depicted while in the middle the random graph structure is represented.The "small world" graph is reported on the left-hand side and it represents a structure which lies in between that of a regular graph and that of a random graph. Image taken from[START_REF] Watts | Collective dynamics of small-world networks[END_REF]

Figure 3 . 1 :

 31 Figure 3.1: Geographic distribution of Multiple Sclerosis across the world. Image taken from https://www.atlasofms.org/map/global/epidemiology/number-of-people-with-ms

Figure 3 . 2 :

 32 Figure 3.2: Schematic representation of a multipolar neuron. Image taken from https://www.alamy.com/stock-photo-motor-neuron-labeled-49222111.html

Figure 3 . 3 :

 33 Figure 3.3: Classification of multiple sclerosis clinical profiles according to the patients disability progression. Image taken from www.clevelandclinicmeded.com

Figure 3 . 4 :

 34 Figure 3.4: Image 3D T2-FLAIR (a) and 3D T1-Gd (b) for an RRMS patient showing chronic lesions in the white matter tissue.

Chapter 5 Generative

 5 Adversarial Network for Connectome Data Augmentation This contribution represents an extended version of a published work with the following reference: Barile B, Marzullo A, Stamile C, Durand-Dubief F, Sappey-Marinier D. Data augmentation using generative adversarial neural networks on brain structural connectivity in multiple sclerosis. Comput Methods Programs Biomed. 2021 Jul;206:106113

Figure 5 . 2 :

 52 Figure 5.2: Schematic Representation of the entire workflow. (1) The original dataset is split in training and test set by means of a leave-one-subject-out cross validation strategy. Considering only the training set, conditional probabilities were calculated and mini-batch random samples were drawn (2), at each cycle, for training the AAE model (3). Once the training process was completed, a batch of random instances were sampled from a standard gaussian distribution (4) and fed to the trained decoder to produce synthetic graphs, used to augment the original training dataset[START_REF] Badeau | Multilinear singular value decomposition for structured tensors[END_REF]. The resulting augmented dataset was used to train a classifier[START_REF] Baldassarre | Brain connectivity and neurological disorders after stroke[END_REF] to predict MS clinical profiles[START_REF] Barkhof | The clinico-radiological paradox in multiple sclerosis revisited[END_REF]. The entire process was repeated for each patient and the predicted MS class was compared with the actual class from the left-out subjects by means of F1-score.

Figure 5 . 3 :Figure 5 . 4 :

 5354 Figure 5.3: Confusion matrices for the classification of MS clinical profiles

Figure 5 . 5 :

 55 Figure 5.5: Graph2Vec embedding comparison of structural graphs: True vs Synthetic data

Figure 5 . 6 :

 56 Figure 5.6: Boxplot distributions of graphs metrics: True vs Synthetic data

Figure 5 . 7 :

 57 Figure 5.7: Embedded t-SNE representation of graph metrics: True and Synthetic data

Figure 5 . 8 :

 58 Figure 5.8: KDE cumulative density function estimation of structural graphs: True and Synthetic data

Figure 5 . 9 :

 59 Figure 5.9: Structural Graph Comparison: True and Synthetic data

Figure 5 . 10 :

 510 Figure 5.10: Training loss comparison with (A) and without (B) coherence loss

Chapter 6 Ensemble

 6 Model for Grey Matter Connectome Data Analysis This contribution represents a revised version of the work submitted and currently under revision to the journal: Frontiers in Robotics and AI: Barile B, Ashtari P, Durand-Dubief F, Stamile C, Marzullo A, Maes F, Van Huffel S, Sappey-Marinier D. Classification of Multiple Sclerosis Clinical Profiles using Machine Learning and Grey Matter Connectome, Frontiers in Robotics and Al

  [START_REF] Marzullo | Classification of multiple sclerosis clinical profiles via graph convolutional neural networks[END_REF] demonstrated the capability of a deep learning model to reach better levels of accuracy for the classification of MS patients clinical profiles. Tuzlu et

Figure 6 . 2 :

 62 Figure 6.2: Schematic representation of the Machine Learning pipeline implemented for the classification of MS clinical profiles. Starting from the adjacency matrix representation of the brain connectome for a patient, six global graph metrics (Betweenness Centrality (BC), Assortativity (r), Transitivity (T ), Efficiency (E g ), Modularity (Q) and Density (D)) were calculated and used as input to the four Machine Learning models (Logistic Regression, Random Forest, Support Vector Machine and AdaBoost) independently trained. The four models were then combined in a final ensemble model by averaging their predictive results.

Figure 6 . 3 :

 63 Figure 6.3: Variability study over the entire percentile threshold range, using two different atlases (FSAverage ( ) and Glasser2016 ( )). Each block defines the Coefficient of Variation (% CV), expressed in percentage values (y-axis), at each specific percentile threshold in the range [0 -1] (x-axis) for a defined global graph feature. For visual purposes, the upper and lower threshold values were cut off at a convenient value in order to avoid image flattening.

Figure 6 . 4 :Figure 6 . 5 :

 6465 Figure 6.4: Boxplot comparison between four MS clinical profiles over six global graph metrics using FSAverage atlas. Differences between clinical profiles were determined employing a generalized mixed effect model with age and sex as controlling factors (*p < 0.05; **p < 0.01; ***p < 0.001 )

Chapter 7 Multiview

 7 Kernel Based Multilinear SVD Approach Using Morphological Grey Matter Features This contribution represents an extended version of the work submitted and accepted for the 30th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning: Barile B, Ashtari P, Maes F, Durand-Dubief F, Van Huffel S, Sappey-Marinier D. A Kernel Based Blind Source Separation Approach for Classification of Multiple Sclerosis Clinical Profiles. European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, BE, October 5-7, 2022

Figure 7 . 1 :

 71 Figure 7.1: Intuitive pipeline of the GM connectome data generation.

Figure 8 . 1 :

 81 Figure 8.1: Complete model pipeline. The green box ( ) represents the training process of the stacked ensemble model while the red box ( ) represents the explainability model. (1) Starting from original data (WM brain graphs) the stacked ensemble model is trained to estimate the EDSS score. Notice that the original dataset is only used for training and validating the model and no other training is performed in any of the following steps. (2) For each hold-out set during the cross validation, the original dataset is perturbed in order to generate valid counterfactual WM brain graph samples. (3) The perturbed (counterfactual) sample are used to probe the trained ensemble model and a corresponding counterfactual EDSS score is obtained (ŷ P erturbed ). (4) A statistical analysis based on conditional logistic regression model is performed in order estimate the β coefficients describing the importance of each WM connection in the brain graph representation of the generated counterfactual sample. (6) Conditioning on the disability class (low -medium -high), the β coefficients are averaged in order to obtain the final score of importance.

Figure 8 . 2 :

 82 Figure 8.2: Truncated Normal distribution conditioned on disability class. Red centermass of the distribution -Purple tail of the distribution. The µ parameter defines the center of the class while σ the corresponding dispersion. Counterfactual data are obtained moving patients from the center-mass (or tail) to the tail (or center-mass) of the distribution. The double arrows illustrate the switching process. Notice that the corresponding counterfactual instances always pertain to the same disability class as the original patient.

Figure 8 .

 8 Figure 8.4 depicts the average EDSS for the true and estimated disability score. The two lines are obtained by, first, sorting the measured and estimated EDSS scores with respect to the hold-out set for each of the ten folds of the cross-validation. Then, for each

Figure 8 . 3 :

 83 Figure 8.3: MADE prediction: Prediction error by class of disability

Figure 8 . 4 :Figure 8 . 5 :

 8485 Figure 8.4: Comparison between measured (red ) and estimated (blue ) EDSS score. The two lines are obtained by, first, sorting the measured and predicted EDSS scores with respect to the test set for each of the ten folds of the cross-validation. Then, for each sample, we computed the average measured and predicted scores, obtaining an average measured and predicted distribution.

Figure 8 . 6 :

 86 Figure 8.6: Global Interpretability: Percentage number of connections useful for predicting global EDSS score

Figure 8 . 7 :

 87 Figure 8.7: Local Interpretability: Sensitivity analysis with multiple threshold values for Left Hemispheric Dominance (blue ) and Inter Hemispheric connections (red )

  (9.3)L(λ, x 1 , x 2 , ..., x n ) = β κ exp(-λ) 1 x β,κ ! λ x β,κ (9.3) Berardino BARILE subj. to |L origin -L perturb | ≤ ζ

Figure 9 . 1 :Figure 9 . 2 :

 9192 Figure 9.1: Gaussian Kernel Density Estimation (KDE) based on the MSE prediction error for three classes of disability

Table 4 .

 4 1: Summary information of the dataset partition by MS clinical profiles (CIS, RR, SP, PP) and healthy subjects. Average values for Age and Disease Duration (DD) with standard deviation in parenthesis is reported. Median values of EDSS is provided along with range of variation in parenthesis.

		N Patients N Scans	Age	DD	Sex (%)	EDSS
	HC	24	24	35.7 (10.1)	-	60	-
	CIS	12	64	30.7 (6.2)	3.0 (1.9)	47	1 (0-3)
	PP	20	140	35.4 (5.4)	7.5 (2.9)	66	4 (2.5-6.5)
	RR	30	233	27.5 (7.7)	8.3 (4.9)	82	2.5 (0-4.5)
	SP	28	215	27.9 (7.6) 14.9 (6.0)	40	5 (3-7)

neurologist in one of the four clinical profiles characterizing MS, based on their clinical history and status. The average age of MS patients is 31.7 (± 7.6) while disease duration at first scan is 6.8 years (± 3.7). The average EDSS score is 3.8 (± 2.1) while the median is 4 with a range of [0 to 7]. Also, both MS patients and HC subjects underwent a MR examination on a 1.5T Siemens Sonata system (Siemens Medical Solution, Erlangen, Germany) using an 8-channel head-coil. The conventional MR protocol of a 3 dimensional T1-weighted (magnetization prepared rapid gradient echo-MPRAGE) sequence with repetition time/echo time/time for inversion (TR/TE/TI) = 1,970/3.93/1,100 ms, flip angle = 15 • , matrix size = 2.5 × 2.5 × 2.5 mm 3 field of view (FOV) = 256 × 256 mm, slice thickness = 1 mm, voxel size = 1 × 1 × 1 , acquisition time = 4.62 min, and a fluid attenuated inversion recovery (FLAIR) sequence with TR/TE/TI = 8,000/105/2,200 ms, flip angle = 150 • , matrix size = 192 × 256, field of view (FOV) = 240 × 240 mm,

Table 5 .

 5 1: Performances of MS Clinical Forms Classification using different data aug-± 11.58 81.1 ± 12.14 64.8 ± 11.83 AAE (ours) 81.0 ± 10.37 86.25 ± 10.36 79.65 ± 10.51 Average classification performance (with standard errors) based on a leave-one-subject-out cross validation strategy on the original dataset (True Data) and after data augmentation of the training set using the Random Over Sampling (ROS) technique, Synthetic Minority Oversampling Technique (SMOTE), Adversarially Regularized Graph Autoencoder (ARAE) and our approach (AAE)

	mentation methods
	Strategy	F 1 score	Precision	Recall
	True Data	65.65 ± 12.34 77.49 ± 12.8 61.68 ± 12.63
	ROS	65.84 ± 11.97 77.49 ± 12.76 61.76 ± 12.08
	SMOTE	72.32 ± 11.18 83.04 ± 11.28 68.45 ± 11.39
	ARAE	70.0

Table 6 .

 6 6: Ablation study: Mean (st.dev) of the predictive performances of the ensemble model across the ten folds of the cross validation using Glasser2016 atlas and multiclass binary comparisons

	Group	F1	Precision	Accuracy	AUC
	CIS+RR-PP	0.671 (0.12) 0.745 (0.06) 0.681 (0.11) 0.704 (0.08)
	CIS+RR-SP	0.658 (0.12) 0.731 (0.06) 0.668 (0.10) 0.691 (0.08)
	CIS+RR-PP+SP 0.648 (0.11) 0.721 (0.06) 0.658 (0.10) 0.681 (0.08)
	RR-PP+SP	0.704 (0.09) 0.761 (0.08) 0.704 (0.08) 0.732 (0.08)

Table 6 .

 6 7: Mean (st.dev) values of global metrics [Betweenness Centrality (BC), Assortativity (r), Transitivity (T), Efficiency (E g ), Modularity (Q) and Density (D)] were calculated on binarized graphs for τ =0.7 using FSAverage and Glasser2016 atlas

				FSAverage Atlas
		BC	r r r	E g E g E g	T	D	Q
	CIS 48.83 (3.83) -0.62 (0.04) 0.58 (0.02) 0.44 (0.08) 0.27 (0.05) 0.13 (0.03)
	RR	47.81 (3.94) -0.60 (0.04) 0.59 (0.02) 0.41 (0.08) 0.29 (0.05) 0.13 (0.03)
	PP	44.28 (3.14) -0.57 (0.05) 0.60 (0.02) 0.42 (0.08) 0.33 (0.05) 0.12 (0.02)
	SP	46.20 (4.94) -0.57 (0.05) 0.59 (0.03) 0.39 (0.09) 0.31 (0.06) 0.13 (0.03)
				Glasser2016 Atlas
	CIS 263.76 (23.97) -0.55 (0.04) 0.57 (0.02) 0.26 (0.06) 0.25 (0.06) 0.15 (0.03)
	RR 254.73 (19.19) -0.53 (0.03) 0.58 (0.02) 0.26 (0.04) 0.28 (0.05) 0.14 (0.02)
	PP 239.77 (16.56) -0.50 (0.03) 0.60 (0.02) 0.30 (0.04) 0.33 (0.05) 0.11 (0.02)
	SP 242.86

Table 7 .

 7 1: Classification performance between inflammatory (CIS+RRMS) and progressive MS courses (PPMS+SPMS). Standard deviations were reported in parentheses.

	-3 Rank	F1	Precision	Recall	AUC
	50	0.680 (0.118) 0.693 (0.117) 0.687 (0.118) 0.699 (0.132)
	100	0.679 (0.124) 0.688 (0.122) 0.688 (0.124) 0.721 (0.154)
	150	0.681 (0.128) 0.696 (0.127) 0.689 (0.128) 0.713 (0.162)
	200	0.713 (0.131) 0.728 (0.127) 0.718 (0.129) 0.741 (0.118)
	250	0.672 (0.116) 0.686 (0.115) 0.679 (0.114) 0.751 (0.131)
	300	0.677 (0.127) 0.693 (0.121) 0.684 (0.124) 0.748 (0.151)

Table 8 .

 8 1: Ablation study: Performance comparison between single base-models and stacking ensemble Performance comparison between single base-models and the stacked ensemble model. The values in the table report the average RMSE score (standard deviation in parenthesis) calculated over the same 10 folds of the cross validation. Linear Regression (LR) is also considered for comparison. The first line of the table (Graphs) reports the performance comparison based on the connectome data. In the second line of the table (Clinical Measures) seven clinical measures are used for estimating EDSS (lesions volume, blackholes volumes, Cortical GM volume, deep GM volume, lateral ventricles, patient age, sex). Last line reports the p-value comparison employing the non parametric Wilcoxon Mann-Whitney test.

		CatBoost	GBM	XGBoost	LightBoost	LR	Stack
	Graphs	1.065 (0.264) 0.998 (0.269) 1.026 (0.312) 1.026 (0.287) 1.301 (0.185) 0.92 (0.278)
	Clinical Measures	1.575 (0.456) 1.606 (0.437) 1.608 (0.422) 1.575 (0.384) 1.41 (0.376) 1.562 (0.405)
	p-value	0.0013	0.0005	0.0008	0.0008	0.1967	0.0003

Table 8 .

 8 2: Brain Area Connection Importance by class of disability

	Brain Connections		Importance
	Area 1	Area 2	Low Medium High
	Left Caudal Middle Frontal	Left Superior Temporal	0.528	0.619	0.095
	Left Caudal Anterior Cingulate	Right Fusiform	0.322	0.317	0.208
	Left Caudal Middle Frontal	Left Parstriangularis	0.364	0.205	0.282
	Left Banks Superior Temporal Sulcus Right Transverse Temporal	0.394	0.255	0.417
	Left Caudal Anterior Cingulate	Left Inferior Parietal	0.206	0.323	0.09
	Left Caudal Anterior Cingulate	Right Parsopercularis	0.161	0.422	0.23
	Left Banks Superior Temporal Sulcus Left Posterior Cingulate	0.555	0.71	1.00
	Left Banks Superior Temporal Sulcus Right Cuneus	0.294	0.317	0.279
	Left Caudal Middle Frontal	Left Precuneus	0.462	0.129	0.13
	Left Banks Superior Temporal Sulcus Right Pericalcarine	1.00	0.402	0.108
	Left Banks Superior Temporal Sulcus Left Caudal Anterior Cingulate 0.095	0.111	0.237
	Left Banks Superior Temporal Sulcus Right Precuneus	0.693	1.00	0.635
	Left Caudal Anterior Cingulate	Right Superior Frontal	0.216	0.434	0.638
	Left Caudal Anterior Cingulate	Right-Hippocampus	0.465	0.296	0.117
	Left Caudal Anterior Cingulate	Left Lateral Occipital	0.229	0.346	0.238
	Left Banks Superior Temporal Sulcus Left Parstriangularis	0.844	0.865	0.306
	Left Caudal Anterior Cingulate	Left Inferior Temporal	0.427	0.223	0.465
	Left Caudal Anterior Cingulate	Left Paracentral	0.304	0.361	0.106
	Most important brain connections and relative importance for three classes of EDSS disability
	score: low ([1 -3]), medium ((3 -4.5]) and high ((4.5 -7]).
	188		Berardino BARILE

Table 8 .

 8 3: Brain Area Number of Connection by class of disability

	Brain Connections		N Connections
	Area 1	Area 2	Low Medium High	Anova
	Left Caudal Middle Frontal	Left Superior Temporal	562	295	230	***
	Left Caudal Anterior Cingulate Right Fusiform	534	493	362	***
	Left Caudal Middle Frontal	Left Parstriangularis	315	166	95	**
	Left Superior Temporal Sulcus Right Transverse Temporal	289	187	152	*
	Left Caudal Anterior Cingulate Left Inferior Parietal	286	219	222	***
	Left Caudal Anterior Cingulate Right Parsopercularis	285	207	127
	Left Superior Temporal Sulcus Left Posterior Cingulate	285	161	73	***
	Left Superior Temporal Sulcus Right Cuneus	272	222	214
	Left Caudal Middle Frontal	Left Precuneus	268	297	351	**
	Left Superior Temporal Sulcus Right Pericalcarine	197	116	88	**
	Left Superior Temporal Sulcus Left Caudal Anterior Cingulate 167	147	64	***
	Left Superior Temporal Sulcus Right Precuneus	152	81	41
	Left Caudal Anterior Cingulate Right Superior Frontal	127	77	51	**
	Left Caudal Anterior Cingulate Right-Hippocampus	104	114	73	***
	Left Caudal Anterior Cingulate Left Lateral Occipital	91	54	62	**
	Left Superior Temporal Sulcus Left Parstriangularis	61	26	20
	Left Caudal Anterior Cingulate Left Inferior Temporal	57	27	7	**
	Left Caudal Anterior Cingulate Left Paracentral	52	49	52	**

Mean number of WM fibers connection for the 95-th percentile most important edges divided in three classes of EDSS disability score: low ([1 -3]), medium ((3 -4.5]) and high ((4.5 -7]). One-way Anova test of hypothesis was performed with (*: 10% -**: 5% -***: 1% confidence level)

Berardino BARILE

Berardino BARILE1.6. STATISTICAL LEARNING MODELS

Berardino BARILE 1.7. DEEP LEARNING

Berardino BARILE1.9. TENSOR FACTORIZATION MODELS FOR PATTERN RECOGNITION

Berardino BARILE 1.10. CONCLUSION

Berardino BARILE3.6. DISABILITY ASSESSMENT AND MEDICAL TREATMENT

Berardino BARILE 6.6. CONCLUSION groups represents and interesting evolution of the present work that we aim to investigate.Berardino BARILE

Berardino BARILE 10.2. FUTURE DIRECTIONS

Acknowledgements

U (1) U vides a what-if explanation for model output.

In this work, a counterfactual-based model is proposed with the aim of explaining the global and local behaviour of a general trained black-box model. Roughly speaking, we are interested in investigating which features (graph connections) are more relevant to determine the EDSS score with respect to (i) the general disability score, or (ii) to different levels of disability. On the one hand, the global behaviour of a regression problem is defined as the smallest set of features that can be considered while maintaining the same level of accuracy. Moreover, a local explanation is obtained by looking for features that explain a confined part of the output space. Similarly, to the global analysis, a local counterfactual explanation of a prediction can be interpreted as the smallest change to the feature values (hereafter feature perturbation) needed to obtain the output of interest.

Global Interpretability Model

Based on our definition of global interpretability, we seek the smallest number of important features that best explain the output prediction obtained by the model. Intuitively, if we are able to select the smallest number of features, which ensure a good level of accuracy, the anchor of the feature space has been found. In order to do this, we exploit the characteristic of tree-based boosting methods where for each specific feature the importance is calculated at a single tree-level and averaged along all the trees of the ensemble [START_REF] Leray | Epidemiology of multiple sclerosis[END_REF] as specified in Eq. (8.7).

where:

In this formulation J -1 represents the number of internal nodes of the k-th tree while v(j) defines the l-th input feature and I represents the corresponding identity function. In other words, the importance for each feature is calculated averaging over all the maximal estimated improvement î2 j , defined in terms of reduction in entropy, for the l-th selected feature. In Layman's terms, a weighted frequency for a specific feature for each basemodel is calculated. The higher the number of times a specific feature has been used, on average, by all the trees of the ensemble to make a splitting decision, the higher its importance. In order to obtain a final common score, a normalization process is applied. This process is repeated for each boosting-based-lerner in the meta-model. For each baselearner, only the features with a score of importance greater than the 5-th percentile of Berardino BARILE

Publications and Communications

Journal Publications