ASM Active Shape Model

Being the second leading cause of cancer-related death worldwide, liver cancer is considered as a major public health challenge. Current ways to characterize the liver cancer are either invasive, through the extraction of tissues samples or non-invasive, typically through interpretation of medical images.

Radiomics is a new field developed in the early 2010s to extract the encoded information within medical images in order to provide a more objective and image-guided assessment of the disease.

Conventional radiomics however suffers from limitations related to the hand-crafted design of the extracted features or the computation of characteristics only in a manually defined region or volume of interest. Deep learning changed the way to apprehend several medical imaging related problematics thanks to its ability to encode morphological properties in images only by using the pixel intensities as input. The radiomics field was impacted by this new set of algorithms, and a new branch of radiomics emerged, where the delineation of the region of interest and/or the extraction of features are performed using deep learning approaches.

Contributions

In this work we present strategies to automatically provide characterization of the liver tumors. We present a review of current liver-related hand-crafted radiomics studies while assessing their experimental design through the Radiomics Quality Score, and propose different ways to improve their reproducibility.

Often performed manually, the delineations of the tumors can suffer from various bias, such as the inter and intra-observer variability, or its undefined borders leading to errors when diagnosing, classifying or staging the liver cancer. Therefore, we present a robust automatic pipeline to perform semantic segmentation of liver tumors that take advantage of deep cascaded networks architectures and the incorporation of dynamic contrast-enhanced information. A main prerequisite for the liver tumors characterization is the study of dynamic multiphase images, however, the major challenge that we faced in our work is the difficulty to obtain (either internally or externally) databases containing liver tumors multiphase images. To characterize the tumors, we combined several datasets to tackle the Introduction automatic histological grade prediction and we propose a novel deep radiomics pipeline which uses relevant semantic imaging features to assess the grade in a slice-wise fashion.

Outline of the thesis

The first chapter aims at presenting the different types of liver cancers, and particularly the hepatocellular carcinoma (HCC), which will be the main topic of our work. We describe the different steps of hepatocarcinogenesis, which transforms healthy hepatic tissues into cancerous ones, in terms of the evolution of the cells and the associated pathological changes. We then detail the current ways to establish the diagnosis either through extraction and inspection of tissue samples (biopsy) or noninvasively using medical imaging techniques. We outline the different modalities usually exploited to establish the diagnosis, and focus on X-ray computed tomography (CT) for the rest of this research work.

The analysis of medical images is commonly performed by the naked eye, but technological advances allow the creation of computer assisted diagnosis tools to investigate the link between imaging features and biological characteristics. We present in the second chapter the radiomics technique allowing the extraction of quantitative features from images. First, we introduce the conventional pipeline based on engineered features, generally referred as HCR (Hand-Crafted Radiomics). To emphasize the limitations of this approach and to question its reproducibility across clinical studies, we review HCR-liver related paper and assess the experimental design according to the Radiomics Quality Score (RQS). Then, we introduce the alternative paradigm brought by Deep-Learning Radiomics and review its application in the context of HCC studies.

We then focus on a key step of the radiomics pipeline, being the automatic segmentation of liver and its tumors. The first methods developed to perform this task were based on anatomical prior-knowledge or manual interactions, and often failed due to the limited amount of training samples, or when facing pathological cases. Automatic deep learning segmentation can address most of these issues by looking for relevant features directly from the images only. We review in chapter 3 both DL-based and non DL-based liver tumor segmentation methods.

In chapter 4, we present our multiphase cascaded architecture to automatically segment both the liver and tumors and to partition the tumor into necrotic and active tissues.

In the last chapter we present our preliminary work to automatically predict the histological grade of HCCs. Finally, conclusion and perspectives are given.

Chapter 1

Liver cancer: biological and clinical aspects

Introduction

This section aims at emphasizing the potential value brought by our work regarding the management of liver cancer. After a brief introduction about the different types of liver cancers, we will focus on the most common primary one: the Hepatocellular Carcinoma. We will review its risk factors, and the way it progresses in the liver, before exposing the different invasive or non-invasive ways to establish the diagnosis. Finally, we present the different staging systems and treatments available to provide the better chance of survival to the diseased patients. A precise description of the different stages of the disease and the subsequent pathological changes is necessary in order to understand how our work can further be incorporated in the clinical practice.

Liver cancers: a major public health challenge

The liver is a key organ in the human body, responsible for the synthesis of several proteins, and playing a major role during the digestion, particularly with the production of bile that is further stored in the gallbladder. It is also essential for the breakdown of numerous hormones, and it has a central position in the human blood flow system with its unique dual blood supply, being one of the three only portal systems of the human body and the only venous one.

The liver can suffer from various pathologies, which include liver cancer, that is now considered as a major public health challenge due to its high incidence and mortality rates. In their latest statistical report in 2019, the World Health Organization (WHO) ranked it as the fifth cancer type in terms of incidence with about 841,000 new cases annually, and as the fourth cause of cancer-related deaths worldwide, with about 782,000 annual deaths. Mortality and incidence rates are between 2 and 3 times higher among men than women in most regions of the world, making it the second type of cancer in terms of deaths for males [START_REF] Bray | Global Cancer Statistics 2018: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries[END_REF]. More details about incidence and mortality rates are illustrated in both the figure 1

.1 & 1.2.
The liver cancer can either be referred to as primary, meaning that it is originally growing in the liver itself, or as secondary, in case of extrahepatic cancers that metastases in the liver. In case of metastases, cancerous cells often originate from the lung, the breast, and some parts of the digestive system, such as the colon, with the colorectal cancer being the main source of extrahepatic metastases [START_REF] Høyer | Radiotherapy for liver metastases: A review of evidence[END_REF][START_REF] Sahani | Current status of imaging and emerging techniques to evaluate liver metastases from colorectal carcinoma[END_REF]. Even though secondary liver cancers are more frequent than primary ones, we have decided to focus on the latter since it is easier to understand the entire process from the appearance of the malignant cells to the physiological changes. Apprehending secondary liver cancers might require to analyze the primary site, something that is beyond the scope of our research work. Primary liver 1.3. HCC RISK FACTORS cancers comprise hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (iCCA), mixed hepatocellular cholangiocarcinoma (HCC-CCA) and other rare tumors, notably fibrolamellar HCC (FLC) and pediatric neoplasm hepatoblastoma [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF][START_REF] Lozano | Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the Global Burden of Disease Study 2010[END_REF][START_REF] Bosman | WHO classification of tumours of the digestive system[END_REF]. HCC accounts for nearly 90% of primary liver cancers, with the highest incidence in Asia and in the Sub-Saharan countries, mainly due to the high prevalence of hepatitis B virus (HBV) [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF][START_REF] Torre | Global cancer statistics, 2012[END_REF]. The second most common one is iCCA, with a noticeable high incidence in Southeast Asian countries, and mainly developed in patients with primary sclerosing cholangitis (PSC), biliary duct cysts, hepatolithiasis, or parasitic biliary infestation with flukes [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF][START_REF] Bridgewater | Guidelines for the diagnosis and management of intrahepatic cholangiocarcinoma[END_REF].

The high incidence of HCC among the primary liver cancers and future collaboration with HCC expert Pr. Baumert leads us to focus our research work on this specific cancer type. We will now detail its risk factors, development, diagnosis methods and treatment techniques.

HCC risk factors

The main reason leading to HCC is the cirrhotic status of the liver, but its development is often related to the presence of other chronic liver diseases. Its incidence is heterogeneous worldwide because of the variable prevalence of risk factors: most cases occurring in sub-Saharan Africa and eastern Asia are associated with HBV and aflatoxin B1 1 exposure. In the USA, Europe and Japan, Hepatitis C (HCV) and the excessive alcohol consumption are the main risk factors [START_REF] Forner | Hepatocellular carcinoma[END_REF]. Other risk factors include the presence of NAFLD (Non Alcoholic-Fatty Liver Disease), obesity and diabetes [START_REF] Marengo | Liver Cancer: Connections with Obesity, Fatty Liver, and Cirrhosis[END_REF]. NAFLD was defined about 20 years ago, and regroups a spectrum of progressive liver diseases that encompasses simple steatosis, nonalcoholic steatohepatitis (NASH) and ultimately cirrhosis [START_REF] Marengo | Liver Cancer: Connections with Obesity, Fatty Liver, and Cirrhosis[END_REF]. It has been identified as the underlying cause of patients presenting with HCC unrelated to virus and alcohol [START_REF] Marrero | NAFLD may be a common underlying liver disease in patients with hepatocellular carcinoma in the United States[END_REF]. Several other studies confirmed that NAFLD is a risk factor for patients with either noncirrhotic liver [START_REF] Paradis | Hepatocellular carcinomas in patients with metabolic syndrome often develop without significant liverfibrosis: A pathological analysis[END_REF][START_REF] Dyson | Hepatocellular cancer: The impact of obesity, type 2 diabetes and a multidisciplinary team[END_REF] or cirrhotic liver [START_REF] Wong | Nonalcoholic steatohepatitis is the most rapidly growing indication for liver transplantation in patients with hepatocellular carcinoma in the U.S[END_REF][START_REF] Ascha | The incidence and risk factors of hepatocellular carcinoma in patients with nonalcoholic steatohepatitis[END_REF][START_REF]Temporal trends of nonalcoholic fatty liver disease-related hepatocellular carcinoma in the veteran affairs population[END_REF]. Obesity has been established as a risk factor for several types of cancer, including liver cancer, and a study on 900,000 American adults reported a mortality rate five times higher in patients with a body mass index of 35 kg/m compared to the group with a normal BMI (Body Mass Index) [START_REF] Calle | Overweight, Obesity, and Mortality from Cancer in a Prospectively Studied Cohort of U.S. Adults[END_REF]. Several other studies were conducted to show the relation between HCC and obesity in the UK, Korea, Sweden, Taiwan and also a multicentric European study [START_REF] Chen | Metabolic Factors and Risk of Hepatocellular Carcinoma by Chronic Hepatitis B/C Infection: A Follow-up Study in Taiwan[END_REF][START_REF] Schlesinger | Abdominal obesity , weight gain during adulthood and risk of liver and biliary tract cancer in a European Cohort[END_REF][START_REF] Samanic | Relation of body mass index to cancer risk in 362,552 Swedish men[END_REF][START_REF] Oh | Effects of excess weight on cancer incidences depending on cancer sites and histologic findings among men: Korea National Health Insurance Corporation study[END_REF][START_REF] Batty | Obesity and overweight in relation to organ-specific cancer mortality in London (UK): Findings from the original Whitehall study[END_REF].

Diabetes has also been identified as an independent risk factor for HCC [START_REF] Forner | Hepatocellular carcinoma[END_REF], especially types 2 diabetes [START_REF] Noto | Substantially increased risk of cancer in patients with diabetes mellitus: A systematic review and meta-analysis of epidemiologic evidence in Japan[END_REF][START_REF] Wideroff | Cancer incidence in a population-based cohort of patients hospitalized with diabetes mellitus in denmark[END_REF][START_REF] El-Serag | Diabetes Increases the Risk of Chronic Liver Disease and Hepatocellular Carcinoma[END_REF][START_REF] Davila | Diabetes increases the risk of hepatocellular carcinoma in the United States: A population based case control study[END_REF][START_REF] Inoue | Diabetes Mellitus and the Risk of Cancer[END_REF].

16

CHAPTER 1. LIVER CANCER: BIOLOGICAL AND CLINICAL ASPECTS

HCC pathophysiology

Liver cancer cells of origin

The basic hepatic cells are divided into parenchymal (hepatocytes, which constitute between 60 and 80% of the total liver mass, and cholangiocytes as depicted in the figure 1.3) and non-parenchymal cells (fibroblasts, stellate cells, Kupffer cells, and endothelial cells) [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF].

Figure 1.3: Structure of the liver lobule and location of candidate cell of origin of liver cancer. Different types of cancer (ie, HCC, iCCA, mixed HCC-CCA) can originate in the liver, depending on the transformation event and the cell type undergoing neoplastic transformation. ©Sia et al. [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF] The two main primary liver cancers types, HCC and iCCA have been considered to be distinct tumors that originate from specific cell populations. Nonetheless, they have recently been recognized as subtypes of a continuous spectrum of diseases. Therefore, several hypotheses exist about the cells of origin of the different primary liver cancer. One of them suggests that liver tumors can be generated by hepatic progenitor cells, since during liver development, both hepatocytes and cholangiocytes both arise from a common progenitor as depicted in the figure 1.4. However, both tumor subtypes can also arise distinctly from mature parenchymal cells [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF]. This latest hypothesis is the one that we will describe in the following, and we will focus specifically on the development of HCCs.

Hepatocarcinogenesis

The main process behind the evolution of HCC is called hepatocarcinogenesis, which is defined as the progressive transformation of nonmalignant liver cells into HCC [START_REF] Choi | CT and MR imaging diagnosis and staging of hepatocellular carcinoma: Part I. Development, growth, and spread: Key pathologic and imaging aspects[END_REF]. This transformation from nonmalignant liver cells into HCC is not fully understood, but the chronic inflammation present in the liver results in cycles of cell injury-death-regeneration, that stimulate epidemic changes and accumulation of genetic damages [START_REF] Thorgeirsson | Molecular pathogenesis of human hepatocellular carcinoma[END_REF][START_REF] Aravalli | Cellular and molecular mechanisms of hepatocellular carcinoma: An update[END_REF][START_REF] Brody | An inflammatory proposal for hepatocarcinogenesis[END_REF][START_REF] Trevisani | Recent advances in the natural history of hepatocellular carcinoma[END_REF]. The high inter and intra-patient heterogeneity that exists for HCCs is explained by the fact that several molecular variants of HCC may be produced, among the patients population, and even within different regions of the same tumor [START_REF] Trevisani | Recent advances in the natural history of hepatocellular carcinoma[END_REF][START_REF] Frenette | Hepatocellular Carcinoma: Molecular and Genomic Guideline for the Clinician[END_REF]. The hepatocarcinogenesis 1.4. HCC PATHOPHYSIOLOGY Figure 1.4: Multiple cells of origin of primary liver cancers. HCC and iCCA can develop via transformation of mature hepatocytes and cholangiocytes, respectively. There is evidence that hepatic progenitor cells (HPCs), their intermediate states, or dedifferentiated hepatocytes can form liver tumors with progenitor-like features, including mixed HCC-CCA, such as CLC. Mature hepatocytes can also be reprogrammed into cells that closely resemble biliary epithelial cells and contribute to development of iCCA. ©Sia et al. [START_REF] Sia | Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis[END_REF] is outlined by the gradual de-differentiation of abnormal nodular lesions, as described in the figure 1.5 [START_REF] Park | Update on precursor and early lesions of hepatocellular carcinomas[END_REF][START_REF] Frenette | Hepatocellular Carcinoma: Molecular and Genomic Guideline for the Clinician[END_REF]. Over time, the more differentiated surrounding tissues are replaced by the growing less differentiated ones. Figure 1.5: Schematic drawing illustrates typical changes in intranodular hemodynamics and OATP (Organic anionic transporting polypeptides) expression during multistep hepatocarcinogenesis. As shown, multistep hepatocarcinogenesis is characterized by successive selection and expansion of lessdifferentiated subnodules within more well differentiated parent nodules. The subnodules grow and eventually replace (blue arrows) the parent nodules. Progressed HCCs show expansile growth (red arrows) and characteristically are encapsulated with fibrous septa. Earlier nodules lack these structures and show replacing growth. During hepatocarcinogenesis, the density of portal triads diminishes while the density of unpaired arteries increases. ©Choi et al. [START_REF] Choi | CT and MR imaging diagnosis and staging of hepatocellular carcinoma: Part I. Development, growth, and spread: Key pathologic and imaging aspects[END_REF] It is possible that HCCs may arise from malignant cells without following this process, and without transitioning through histologically definable intermediate steps the process is then referred to as "denovo hepatocarcinogenesis" [START_REF] Taguchi | Morphologic approach to hepatocellular carcinoma development in man: De novo or the so-called 'dysplastic nodule-carcinoma' sequence?[END_REF].

The different stages of the hepatocarcinogenesis

As depicted in the figure 1.6, hepatocarcinogenesis is composed of several steps, where each stage presents specific characteristics: CHAPTER 1. LIVER CANCER: BIOLOGICAL AND CLINICAL ASPECTS Figure 1.6: HCC can develop inside MRNs (MacroRegenerative Nodules). This type of carcinogenesis is called nodule in nodule. However, most HCCs develop outside MRNs, and precancerous lesions can be identified in areas of large and small cell dysplasia or irregular regeneration (de novo carcinogenesis). ©Trevisani et al. [START_REF] Trevisani | Recent advances in the natural history of hepatocellular carcinoma[END_REF] • Cirrhotic nodules are well-defined rounded regions of cirrhotic parenchyma surrounded by scar tissue and measuring less than 15 mm in diameter [START_REF] Zimmermann | Terminology of nodular hepatocellular lesions[END_REF]. Even though they are usually considered as non-malignant, hepatocytes that they contain may develop dysplastic features, thus transforming the cirrhotic nodules into a dysplastic foci or nodules.

• Dysplastic foci are not identified via in vivo imaging, but may be recognized histologically. They are not well understood and may develop into dysplastic nodules.

• Dysplastic nodules correspond to nodular lesions that differ macro and microscopically from the surrounding parenchyma. They are observed in 25% of cirrhotic livers, and can be classified into low-grade (that appear like cirrhotic nodules) or high-grade (more similar to well-differentiated HCCs)

• Early HCCs grow gradually by replacing the parenchyma, unlike apparent progressed HCCs that displace or destroy the liver parenchyma. During their evolution, they tend to surround structures like portal tracts or central veins without destroying them.

They are microscopically indistinguishable from high-grade dysplastic nodules, since they tend to be vaguely nodular without having capsules nor distinct margins.

Even if they are considered as ancestors of progressed HCCs [START_REF] Park | Update on precursor and early lesions of hepatocellular carcinomas[END_REF], their progression rate is not clearly defined [START_REF] Khalili | Indeterminate 1-2-cm nodules found on hepatocellular carcinoma surveillance: Biopsy for all, some, or none?[END_REF].

• Progressed HCCs are generally separated between those measuring less than 2cm and those larger -Those smaller than 2cm tend to be nodular with a well-defined margin. They are different from early HCCs in the way that they grow by expanding into and compressing the adjacent parenchyma. They tend to be histologically moderately differentiated in the vast majority of the cases, and are associated with vascular invasion and intrahepatic metastasis [START_REF] Park | Update on precursor and early lesions of hepatocellular carcinomas[END_REF].

-The large progressed HCCs tend to have a more aggressive biological behavior, and are associated with a higher histological grade, with a higher presence of vascular invasion and metastasis. They are histologically composed of poorly differentiated or undifferentiated cancer cells that spread into the surrounding sinusoids, thus often characterized by an ill-defined boundary [START_REF] Kudo | Real practice of hepatocellular carcinoma in Japan: Conclusions of the Japan society of hepatology 2009 kobe congress[END_REF][START_REF] Beasley | HEPATOCELLULAR CARCINOMA AND HEPATITIS B VIRUS[END_REF][START_REF] El-Serag | Hepatocellular Carcinoma[END_REF][START_REF] Baffy | Hepatocellular carcinoma in non-alcoholic fatty liver disease: An emerging menace[END_REF][START_REF] Mcglynn | The Global Epidemiology of Hepatocellular Carcinoma: Present and Future[END_REF][START_REF] Tyson | Risk factors for cholangiocarcinoma[END_REF][START_REF] Theise | Hepatocellular carcinoma[END_REF][START_REF] Trevisani | Recent advances in the natural history of hepatocellular carcinoma[END_REF].

• Multifocal HCC can be the result of a synchronous development of several independent liver tumors originating from a primary tumor [START_REF] Trevisani | Recent advances in the natural history of hepatocellular carcinoma[END_REF], knowing that patients with HCC are at higher risk of developing new tumors.

Physiological changes brought by the hepatocarcinogenesis

These several steps defining hepatocarcinogenesis are most of the time accompanied by some pathophysiological alterations:

• Angiogenesis: that is histologically characterized by the development of unpaired (or nontriadal) arteries as depicted in the figure 1.5 and the transformation of hepatic sinusoids into continuous capillaries, a.k.a "sinusoidal capillarization" [START_REF] Palmer | Are common factors involved in the pathogenesis of primary liver cancers? A meta-analysis of risk factors for intrahepatic cholangiocarcinoma[END_REF][START_REF] Coleman | Mechanisms of Human Hepatocarcinogenesis[END_REF].

Besides these changes, the portal tracts, containing both the non-tumoral hepatic arteries and the portal veins, progressively decrease [START_REF] Kojiro | Early hepatocellular carcinoma and dysplastic nodules[END_REF].

Whereas the portal inflow to the nodule diminishes, the formation of unpaired arteries causes an increase in arterial flow [START_REF] Palmer | Are common factors involved in the pathogenesis of primary liver cancers? A meta-analysis of risk factors for intrahepatic cholangiocarcinoma[END_REF][START_REF] Coleman | Mechanisms of Human Hepatocarcinogenesis[END_REF]. This difference in blood inflow is such that we observe a total inversion of tendency during the hepatocarcinogenesis, with a decrease of arterial flow accompanied by a preservation of portal venous flow in the early phases, and a decrease of portal blood flow with an increase of arterial flow in the later phases, as depicted in the figure 1.7.

• Venous drainage: During hepatocarcinogenesis, the way the blood is drained by the lesions is subject to modifications as depicted in the figure 1.8.

First, the blood is evacuated via hepatic veins, then sinusoids are used and replace the hepatic veins that start to be blocked. Later during the process, the sinusoids start to collapse and the blood cannot be delivered through this way, therefore, the only remaining way is through the portal veins [START_REF] Ueda | Hypervascular hepatocellular carcinoma: Evaluation of hemodynamics with dynamic CT during hepatic arteriography[END_REF][START_REF] Kitao | Hepatocarcinogenesis : Multistep Changes of Drainage Vessels at CT during Arterial Portography and Purpose : Methods : Results : Conclusion[END_REF].

This change along the disease progression may explain the phenomenon of corona enhancement, which appears mainly on progressed hypervascular HCC, and which referred to an enhancement of the peritumoral parenchyma starting a few seconds after the enhancement of the tumor. Worth noting that early HCCs are not concerned by this phenomenon [START_REF] Matsui | Hepatocelluar nodules in liver cirrhosis: Hemodynamic evaluation (angiographyassisted CT) with special reference to multi-step hepatocarcinogenesis[END_REF].

• Tumor Capsule and Fibrous Septa: are observed in about 70% of progressed nodular HCCs [START_REF] Kojiro | Early hepatocellular carcinoma and dysplastic nodules[END_REF], with a capsule consisting of two distinct layers. Some studies have shown that tumors with an intact capsule were associated with a lower recurrence rate than those without (tumors of similar size), suggesting that the capsule may retard the tumor dissemination, especially the tight inner layer that might act as a physical 1.4. HCC PATHOPHYSIOLOGY 21 barrier [START_REF] Ng | Tumor encapsulation in hepatocellular carcinoma. A pathologic study of 189 cases[END_REF].

Even though advanced HCCs with an intact capsule have a more favorable prognosis than HCCs of the same size without any capsule, they have a worse prognosis than early HCCs that are unencapsulated [START_REF] Iguchi | Both fibrous capsule formation and extracapsular penetration are powerful predictors of poor survival in human hepatocellular carcinoma: A histological assessment of 365 patients in Japan[END_REF].

• Fat content: It has been demonstrated that hepatocytes may accumulate fat during the early phases of hepatocarcinogenesis, causing the tumors to be more steatotic. This steatosis is highly frequent for early HCCs of about 1.5 cm in diameter, but decreases when the size and the grade of the tumor increases. This might probably be because unpaired arteries become more developed once the tumor progresses, resolving ischemic conditions, and provoking the regression of steatosis [START_REF] Kojiro | Early hepatocellular carcinoma and dysplastic nodules[END_REF][START_REF] Kutami | Pathomorphologic study on the mechanism of fatty change in small hepatocellular carcinoma of humans[END_REF].

• Iron content may accumulate early in the process of hepatocarcinogenesis, noticeable essentially in low or high grade dysplastic nodules [START_REF] Kojiro | Pathologic diagnosis of early hepatocellular carcinoma: A report of the international consensus group for hepatocellular neoplasia[END_REF]. In the later stages, hepatocytes become resistant to iron accumulation due mainly to the utilization of iron by neoplastic cells and a higher cellular proliferation, thus early and progressed HCCs are often iron free [START_REF] Gurusamy | Trace element concentration in primary liver cancers -A systematic review[END_REF].

• Diminution in the expression of OATP Transporters. Some of these transporters belonging to the OATP (Organic anionic transporting polypeptides) group are thought to be responsible for the uptake of two hepatobiliary-specific gadolinium-based contrast agents, namely the gadoxetate disodium and gadobenate dimeglumine.

Their expression level tends to be inversely proportional to the advancement of the disease [START_REF] Kitao | The uptake transporter OATP8 expression decreases during multistep hepatocarcinogenesis: Correlation with gadoxetic acid enhanced MR imaging[END_REF][START_REF] Tsuboyama | Hepatocellular carcinoma: Hepatocyte-selective enhancement at gadoxetic acid-enhanced MR imaging -Correlation with expression of sinusoidal and canalicular transporters and bile accumulation[END_REF].

The different alterations are often the results of early or more advanced stages of hepatocarcinogenesis.

They can appear individually or can be combined within the same tumor.

However, the HCC can often later spread either intra or extra-hepatically:

• Intrahepatic metastasis: corresponds to one of the most important spread mechanisms of progressed HCCs, and occurs mainly when malignant cells enter portal venules that drain the primary tumor, before spreading into the surrounding parenchyma.

These metastases around the primary tumor in the form of small satellites, and are located within its venous drainage area [START_REF] Nakashima | Portal vein invasion and intrahepatic micrometastasis in small hepatocellular carcinoma by gross type[END_REF]. Even though malignant cells can easily invade vessels, extrahepatic metastases occurring in organs such as the lungs, the lymph nodes, the bones or the adrenal glands, are late manifestations of the disease [START_REF] Theise | Hepatocellular carcinoma[END_REF][START_REF] Trevisani | Recent advances in the natural history of hepatocellular carcinoma[END_REF].

• Vascular invasion: is a late manifestation of hepatocarcinogenesis, affecting mainly the progressed HCCs [START_REF] Ha | Primary carcinoma of the liver: a study of 100 cases among 48,900 necropsies[END_REF], and allows a distinction between primary and secondary liver cancers which uncom-CHAPTER 1. LIVER CANCER: BIOLOGICAL AND CLINICAL ASPECTS monly present an invasion of intrahepatic vessels [START_REF] Okuda | Hepatocellular carcinoma: Clinicopathological aspects[END_REF].

These invasions are either classified as micro or macroscopic, and usually are accompanied by a poor prognosis, as they provide a way for cancerous cells to propagate in the liver or systemically.

Moreover, patients suffering from cancers with vascular invasion tend to have a higher recurrence rate after ablation, resection and transplantation, therefore, vascular invasion is often considered as a contraindication for these specific treatments [START_REF] Llovet | The Barcelona approach: Diagnosis, staging, and treatment of hepatocellular carcinoma[END_REF].

Furthermore, another alteration, called "Tumor Capsule invasion" may increase the risk of vascular invasion, as HCC cells may infiltrate through the tumor capsule into the surrounding parenchyma.

All the changes occurring during hepatocarcinogenesis, leading to a more or less aggressive tumor, or even to multiple tumors, need to be assessed as early as possible to provide the best treatment available to the patient in order to increase its chances of survival.

HCC diagnosis

Biopsy

One of the standard ways to determine or confirm the cause and the stage of liver disease, as well as to inform treatment decisions and establish prognosis is to perform a biopsy. This surgical procedure consists of the sampling of a specimen measuring typically between 1 and 3 cm in length, with a diameter comprised between 1.2 and 2 mm [START_REF] Bravo | Liver Biopsy[END_REF].

The sampled tissues are then reviewed by pathologists, who report the degree of inflammation, steatosis, fibrosis and some other features such as cellular inclusions. The histological assessment is supported by clinicians providing the clinical context to complete the histologic interpretation, and some external blood markers such as the AFP (Alpha-FetoProtein) level can be incorporated to attest the presence of a malignant mass [START_REF] Bai | The prognostic correlation of AFP level at diagnosis with pathological grade, progression, and survival of patients with hepatocellular carcinoma[END_REF][START_REF] Heimbach | AASLD guidelines for the treatment of hepatocellular carcinoma[END_REF]. Histological slice grading system was introduced by Edmonson-Steiner and corresponds to the level of cellular differentiation. 4 different groups were initially created and illustrated in the figure 1.9. This grading systems however suffers from several limitations, at such an extent that new grading systems were introduced.

Histological HCC grading systems

Several types of histological grading systems exist, with the most frequent ones being ES 1954 (Edmonson-Steiner) [START_REF] Ha | Primary carcinoma of the liver: a study of 100 cases among 48,900 necropsies[END_REF] and WHO 2010 (World Health Organization) [START_REF] Bosman | WHO classification of tumours of the digestive system[END_REF]. Differences between these two grading systems are given in the figure 1.10. Even if ES 1954 is the most widely used grading system, a lot of divergences in the assessment of the grade can be found in the different studies reviewed by Martins et al. [START_REF] Martins-Filho | Histological grading of hepatocellular carcinoma-a systematic review of literature[END_REF].

The organization of the patients in different numbers of tiers (3 or 4) G1, G2, G3 and potentially G4, and the possible classification into low or high differentiation level reveals a high variability in the assessment of the histological grade, as seen in the figure 1.11. As an example, some studies suggested that G2 is closer to G1 than to G3 [START_REF] Han | Prognostic significance of the worst grade in hepatocellular carcinoma with heterogeneous histologic grades of differentiation[END_REF][START_REF] Zucman-Rossi | Genetic Landscape and Biomarkers of Hepatocellular Carcinoma[END_REF], therefore, a classification that separates both G1 and G2 in one group and G3 (with potentially G4 if available) in the other can be understood. This difference in the classification of the patients was also noticed by Han el al. [START_REF] Han | Prognostic significance of the worst grade in hepatocellular carcinoma with heterogeneous histologic grades of differentiation[END_REF] and detailed in the figure 1.12. They reviewed different articles that used the histological grade to predict the Figure 1.12: Classification groups used by the studies reviewed by ©Han et al. [START_REF] Han | Prognostic significance of the worst grade in hepatocellular carcinoma with heterogeneous histologic grades of differentiation[END_REF] The two possible solutions were to consider the worst grade (as recommended by the ES grading system), or to consider the most present one (recommended by the WHO grading system), however clear explanations of the decision taken in such cases are often undisclosed in the different studies.

The question regarding which histological HCCs grading system to use remains open. Moreover, various teams proposed to introduce a more complete grading system where the different features are

Limitations

The biopsy suffers from several limitations. Sampling errors are common, as reported by a study where laparoscopic biopsy samples, taken simultaneously from the right and the left lobes of several patients suffering from HCV, were interpreted as cirrhotic in one lobe and fibrotic (F3) in the other in almost 15% of the cases [START_REF] Regev | Sampling error and intraobserver variation in liver biopsy in patients with chronic HCV infection[END_REF]. It has also been proven that the accuracy of both the diagnosis and the staging of the disease highly depend on the size of the specimen, with misinterpretations when too small samples are collected [START_REF] Colloredo | Impact of liver biopsy size on histological evaluation of chronic viral hepatitis: The smaller the sample, the milder the disease[END_REF].

The interpretation of the samples is very subjective as it has been proven in some studies, with the 2 Important to distinguish from the pathological stage which mainly corresponds to the TNM.

HCC DIAGNOSIS

27

estimation of both inflammatory activity and fat burden that suffered from a high inter and intraobserver variability [START_REF] Bedossa | Intraobserver and Interobserver Variations in Liver Biopsy Interpretation in Patients with Chronic Hepatitis C[END_REF]. Moreover, being a surgical gesture, the biopsy can suffer from adverse events going from simple pain to death in some rare cases [START_REF] Rockey | Liver biopsy[END_REF][START_REF] Castera | Patient-administered nitrous oxide/oxygen inhalation provides safe and effective analgesia for percutaneous liver biopsy: A randomized placebo-controlled trial[END_REF][START_REF] Seeff | Complication Rate of Percutaneous Liver Biopsies Among Persons With Advanced Chronic Liver Disease in the HALT-C Trial[END_REF][START_REF] Piccinino | Complications following percutaneous liver biopsy[END_REF]. Finally, they happen to be costly since they involve an expert gastroenterologist or interventional radiologist, a pathologist, and need to be performed in facilities with adequate equipment.

Even though biopsy is still used to evaluate the degree of fibrosis or cirrhosis, primary liver cancers are now most often diagnosed on the basis of imaging studies alone in clinical practice. Recent advancements in medical imaging enable the radiologist to discern the advancement of focal liver lesions on the basis of vascularity and physiological features, especially through multiphasic, contrast-enhanced, cross-sectional imaging (either computed tomography CT, or magnetic resonance MR).

CT and MR imaging

As explained previously, hepatocarcinogenesis is a long process that can render the tumor very aggressive, and the more the disease evolves, the worse the prognosis will be for the patient, thus an early detection of HCC is critical to improve the survival of affected patients.

The majority of the current guidelines recommend ultrasonography (US) as the primary imaging test for surveillance [START_REF] Choi | CT and MR imaging diagnosis and staging of hepatocellular carcinoma: Part I. Development, growth, and spread: Key pathologic and imaging aspects[END_REF]. CT and MRI are generally not chosen for surveillance but recommended by some guidelines when the US is limited due to obesity or when the risk factors are very high [START_REF] Omata | Asian pacific association for the study of the liver consensus recommendations on hepatocellular carcinoma[END_REF][START_REF] Llovet | EASL-EORTC Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF].

If the surveillance is positive, the main guidelines advocate in favor of multiphasic CT and MR with extracellular agents for the non-invasive diagnosis and staging [START_REF] Kudo | Real practice of hepatocellular carcinoma in Japan: Conclusions of the Japan society of hepatology 2009 kobe congress[END_REF][START_REF] Llovet | EASL-EORTC Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF][START_REF] Omata | Asian pacific association for the study of the liver consensus recommendations on hepatocellular carcinoma[END_REF][START_REF] Bruix | Management of hepatocellular carcinoma: An update[END_REF].

Multiphasic CT and MRI can be categorized by the type of contrast agents used:

• Imaging with extracellular contrast agents permits a diagnosis of HCC based on the physiological changes in the blood flow produced by hepatocarcinogenesis. Images are typically acquired before (precontrast) and dynamically after the injection of a contrast agent.

The contrast agent is generally administered at a rate of 4-6 mL/sec for CT and around 2 mL/sec for MRI [START_REF] Schima | Multidetektor-CT (MDCT) der leber[END_REF][START_REF] Frydrychowicz | Hepatobiliary MR imaging with gadolinium-based contrast agents[END_REF]. The dose is usually adjusted to the body weight, where 1.5-2 mL/kg to achieve an iodine dose of 525 mg/kg of iodine are administered in the case of CT.

Concerning the dynamic contrast enhanced images, three phases are typically acquired, namely the late hepatic arterial, the portal venous, and the delayed phase as illustrated in the figure 1.16:

-The late hepatic arterial phase is characterized by the full enhancement of the hepatic artery with all its branches and an enhancement of the portal vein [START_REF] Choi | CT and MR imaging diagnosis and staging of hepatocellular carcinoma: Part I. Development, growth, and spread: Key pathologic and imaging aspects[END_REF]. The portal vein is however not supposed to be enhanced by the antegrade flow [START_REF] Miraglia | Predictive factors of tumor response to trans-catheter treatment in cirrhotic patients with hepatocellular carcinoma:A multivariate analysis of pre-treatment findings[END_REF]. This phase is critical for the characterization of hypervascular HCCs, since it catches the CHAPTER 1. LIVER CANCER: BIOLOGICAL AND CLINICAL ASPECTS peak arterial enhancement of tumors [START_REF] Kim | Optimal scan window for detection of hypervascular hepatocellular carcinomas during MDCT examination[END_REF]. The early hepatic arterial phase is often omitted by the centers as only hepatic arteries are enhanced but portal veins are not, which render the detection of hypervascular tumors very difficult. One of the main problems associated with this acquisition is the difficulty to detect the peak arterial perfusion, since fixed delay is often not reliable. Some techniques such as contrast agent bolus tracking, or the selection of the best image after multiple continuous acquisitions may be recommended [START_REF] Earls | Hepatic arterial-phase dynamic gadolinium-enhanced MR imaging: Optimization with a test examination and a power injector[END_REF].

-The portal venous phase is characterized by an enhancement of both hepatic artery (arterial) and portal veins. It is however worth noting that the contrast agent still circulates in the body and will still be present in hepatic arteries as well (but with a lower concentration than for the arterial phases).

The portal venous phase is generally acquired at around 60-80 seconds after the start of the injection, but no clear guidelines are given concerning a precise way to determine the best moment for the acquisition of this given phase.

-The delayed phase is acquired 3-5 minutes after the injection and helps to understand how the liver restores the contrast agent back to the rest of the body, especially in case of inhomogeneous washout. Both the portal venous and the delayed phases are critical for the characterization of washout and/or capsule appearance, and are crucial for the differentiation of small HCCs from small ICCs [START_REF] Iannaccone | Hepatocellular carcinoma: Mole of unenhanced and delayed phase multidetector mew helical CT in patients with cirrhosis[END_REF][START_REF] Rimola | Cholangiocarcinoma in cirrhosis: Absence of contrast washout in delayed phases by magnetic resonance imaging avoids misdiagnosis of hepatocellular carcinoma[END_REF]. Some centers decide to skip the delayed phase, but it can lead to a loss of information.

Precontrast images are useful in case of detection of enhancement and evaluation of its degree by subtracting the post-contrast image with the preconstrast one. They are also useful for patients with iron-rich nodules to detect hyper attenuation before injection of contrast agent, thus avoiding In addition to these classical vascular phases, examinations performed with MR imaging usually include other sequences such as the T1-weighted in-phase, the T2-weighted fast-spin-echo and some diffusion-weighted sequences.

• In order to fully understand the behavior of HCC cells, one can use extracellular agents as explained above, or use intracellular agents. In the latter case, hepatobiliary agents allow a better diagnosis through understanding the hepatocellular function in addition to the vascularity information. These agents first enhance the extracellular space before entering the hepatocytes through the OATP specific receptors. To acquire this hepatobiliary phase, which can exclusively be obtained on MRI, different agents exist, and they mainly differ in their hepatocellular uptake, thus reaching an enhancement at different moments. In the exception to the delayed phase for some specific agents, the normal phases can be acquired, along with some new phases such as the "transitional phase", which represent a transition from extracellular-dominant to intracellular-dominant enhancement [START_REF] Nakamura | Clinical significance of the transitional phase at gadoxetate disodiumenhanced hepatic MRI for the diagnosis of hepatocellular carcinoma: Preliminary results[END_REF]. Examples of images obtained with after a multiphasic MR examination are illustrated in the figure 1.17

Unlike extracellular agents, the hepatobiliary contrast agents show promise for differentiating early HCCs and premalignant nodules from lower-risk nodules [START_REF] Takayama | Hypovascular hepatic nodule showing hypointensity in the hepatobiliary phase of gadoxetic acid-enhanced MRI in patients with chronic liver disease: Prediction of malignant transformation[END_REF][START_REF] Kumada | Evolution of hypointense hepatocellular nodules observed only in the hepatobiliary phase of gadoxetate disodium-enhanced MRI[END_REF][START_REF] Kobayashi | Relationship between signal intensity on hepatobiliary phase of gadolinium ethoxybenzyl diethylenetriaminepentaacetic acid (Gd-EOB-DTPA)-enhanced MR imaging and prognosis of borderline lesions of hepatocellular carcinoma[END_REF][START_REF] Kim | Hypovascular hypointense nodules on hepatobiliary phase gadoxetic acidenhanced MR images in patients with cirrhosis: Potential of DW imaging in predicting progression to hypervascular HCC[END_REF][START_REF] Hyodo | Hypovascular nodules in patients with chronic liver disease: Risk factors for development of hypervascular hepatocellular carcinoma[END_REF][START_REF] Park | Small hepatocellular carcinomas: Improved sensitivity by combining gadoxetic acid-enhanced and diffusion-weighted MR imaging patterns[END_REF][START_REF] Kim | Gadoxetic Acid-Enhanced MRI Versus Triple-Phase MDCT for the Preoperative Detection of Hepatocellular Carcinoma[END_REF]. Main advantages of choosing CT over MRI examination is that it is widely available, rapid, robust CHAPTER 1. LIVER CANCER: BIOLOGICAL AND CLINICAL ASPECTS and requires less expertise than MRI when interpreting images. It however exposes patients to radiation doses, and provides relatively lower soft-tissue contrast than when MRI is used. MRI permits the assessment of a great number of tissue characteristics, but is time-consuming, less robust, and prone to artifacts. Even though MR imaging examination may be preferred at some academic centers, there is no consensus for recommending this modality over CT in community or less-specialized centers [START_REF] Choi | CT and MR imaging diagnosis and staging of hepatocellular carcinoma. Part II. Extracellular agents, hepatobiliary agents, and ancillary imaging features[END_REF][START_REF] Heimbach | AASLD guidelines for the treatment of hepatocellular carcinoma[END_REF].

Therefore, it has been decided to mainly focus our research work on CT images. These recent innovations in the medical imaging fields were accompanied by the creation of several clinical criteria that helped the radiologists to take decisions concerning the treatment choice, depending mainly on the characteristics of the tumor.

HCC staging & treatment

One of the most widely used criteria for the assessment of HCCs is the BCLC (Barcelona Clinic Liver Cancer) staging system, recently refined by [START_REF] Llovet | Design and endpoints of clinical trials in hepatocellular carcinoma[END_REF], and illustrated in the figure 1.18 [START_REF] Llovet | Design and endpoints of clinical trials in hepatocellular carcinoma[END_REF].

As described in the figure below, the main factors used to build this staging system rely on the tumor status, namely, the number and the size of the nodules, the presence or the absence of vascular invasion and extrahepatic spread. Other characteristics such as the liver function (as defined by the Child-Pugh classification [START_REF] Pugh | Transection of the oesophagus for bleeding oesophageal varices[END_REF] , the potential presence of portal hypertension and both serum bilirubin and albumin levels), and an indicator called the general performance status describing the overall level of functioning of the patient [START_REF] Oken | Toxicology and response criteria of the Eastern Cooperative Oncology Group[END_REF]. One of the main advantages brought by the BCLC staging system is to provide strong guidelines concerning the treatments to follow depending on the disease progression. Hereafter, we detail the different treatment techniques available for patients suffering from HCCs when seeking for curation:

• Surgical resection, where the best candidates are patients with solitary tumor and preserved liver functions [START_REF] Forner | Hepatocellular carcinoma[END_REF].

A resection is defined as curative if all macroscopic evidence of primary and second tumor are detected in the remnant liver or in any other organs by dynamic CT or MR within 4 weeks after surgery [START_REF] Yang | Surgical resection for advanced hepatocellular carcinoma according to Barcelona Clinic Liver Cancer (BCLC) staging[END_REF].

As mentioned by previous studies, resection remains the treatment of choice in patients without cirrhosis suffering from HCC. Patients with cirrhosis have to be carefully selected to reduce the risk of postoperative liver failure [START_REF] Bruix | Management of hepatocellular carcinoma: An update[END_REF][START_REF] Llovet | EASL-EORTC Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF][START_REF] Verslype | Hepatocellular carcinoma: ESMO-ESDO clinical practice guidelines for diagnosis, treatment and follow-up[END_REF].

• Liver transplantation, which mainly benefits patients who are not good candidates for surgical resection, and fits better those within the Milan criteria (solitary tumor up to 5cm or less than 3 nodules measuring each one less than 3cm) [START_REF] Llovet | EASL-EORTC Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF][START_REF] Mazzaferro | Milan criteria in liver transplantation for hepatocellular carcinoma: An evidence-based analysis of 15 years of experience[END_REF]. In theory the transplantation may cure the tumor and the underlying cirrhosis at the same time, however, this procedure suffers from a scarcity in terms of donors, with an increase in the waiting time that led up to 20% of the potential receivers to drop out of the list before finding a donor, even with the existence of bridge therapies that can preserve the patient health during the waiting time [START_REF] Llovet | Intention-to-treat analysis of surgical treatment for early hepatocellular carcinoma: Resection versus transplantation[END_REF]. Some modifications have been done to the initial inclusion criteria for the transplant candidates after poor recurrence results were obtained [START_REF] Llovet | EASL-EORTC Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF].

CHAPTER 1. LIVER CANCER: BIOLOGICAL AND CLINICAL ASPECTS

• Image-guided ablation (IGA) is the most frequently used therapeutic strategy but its efficacy is limited by the size of the tumor, its location and the respiratory motion of the patient. This technique belongs, just like resections, to primary line treatments for patients with early stage HCC (BCLC stage 0-A) for whom surgical management is contraindicated [START_REF] Bruix | Management of hepatocellular carcinoma: An update[END_REF][START_REF] Heimbach | AASLD guidelines for the treatment of hepatocellular carcinoma[END_REF]. It has also been noticed that patients with very small HCCs (less than 2cm in diameter) can benefit from IGA to avoid any surgical procedure [START_REF] Forner | Hepatocellular carcinoma[END_REF].

Several methods have been developed to perform the destruction of the tumor, and radiofrequency ablation (RFA) remains the most popular technique. Other techniques (thermal or non-thermal) have been adopted to overcome limitations of RFA.

The risk of complications is higher when the procedure is performed on tumors located along the surface of the liver. Indeed, puncture can cause bleeding, and heating can cause complications by provoking injury on adjacent organs. The intervention should be performed by an oncologist with sufficient experience, in order to assess the risk of causing damages to the gastrointestinal tract [START_REF] Breen | Image-guided ablation of primary liver and renal tumours[END_REF].

However, RFA techniques have recently been improved by the assistance of 3D navigation systems, allowing better planning with multiple overlapping ablation zones, a more accurate placement of the probes and an assessment of the results intraoperatively, thanks to image fusion. This new generation of techniques is called "stereotactic RFA" [START_REF] Perrodin | Percutaneous stereotactic image-guided microwave ablation for malignant liver lesions[END_REF][START_REF] Bale | Stereotactic Radiofrequency Ablation of Hepatocellular Carcinoma: a Histopathological Study in Explanted Livers[END_REF][START_REF] Laimer | Minimal ablative margin (MAM) assessment with image fusion: an independent predictor for local tumor progression in hepatocellular carcinoma after stereotactic radiofrequency ablation[END_REF].

Therefore, the question whether to choose resection over RFA is still open [START_REF] Heimbach | AASLD guidelines for the treatment of hepatocellular carcinoma[END_REF][START_REF] Bale | Stereotactic Radiofrequency Ablation of Hepatocellular Carcinoma: a Histopathological Study in Explanted Livers[END_REF].

• Transarterial Chemoembolization (TACE), which has survival benefits in asymptomatic patients with multifocal disease without vascular invasion or extrahepatic spread can also be considered [START_REF] Forner | Hepatocellular carcinoma[END_REF]. TACE can also be applied to patients in earlier stages, who are not suitable for the other therapeutic available treatments. This clinical situation is known as "treatment stage migration" and reflects a certain flexibility in the BCLC clinical interpretation [START_REF] Burrel | Survival of patients with hepatocellular carcinoma treated by transarterial chemoembolisation (TACE) using Drug Eluting Beads. Implications for clinical practice and trial design[END_REF].

The same standards are used for the patient follow-up, and can be supported by other metrics such as the RECIST (standing for "Response evaluation criteria in solid tumours") which assess the tumor response, but presents several limitations and has already been modified in the past to incorporate changes [START_REF] Eisenhauer | New response evaluation criteria in solid tumours : Revised RECIST guideline ( version 1 . 1 )[END_REF].

As explained here, general staging systems rely mainly on the macro changes caused by the disease, ignoring many histological measurements, such as the histological grade [START_REF] Ha | Primary carcinoma of the liver: a study of 100 cases among 48,900 necropsies[END_REF].

Current methods to evaluate the tumor and to propose the best available treatment still suffer from a lot of limitations, especially because they rely on subjective interpretation of images, or clinical data, 1.6. HCC STAGING & TREATMENT 33 and also because of the discrete staging systems that are used in a clinical routine.

The main hypothesis of our research work is that medical images may be analyzed and interpreted to provide an objective feedback to the clinicians, and help them with non-invasive tools to give to patients suffering from HCCs the best chances of survival.

The radiomics concept has been introduced to address these issues, by providing an objective evaluation of the medical images at a finer scale than the one used by the medical experts during their visual inspection. It was designed to non-invasively obtain predictive or prognostic models, in order for example to assess the tumor response to a treatment, the recurrence after surgery or predict the type of tumors. Radiomics studies comes with several prerequisites such as the need to respect ethical considerations regarding patient data privacy, or the standardization of the databases [START_REF] Tang | Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology[END_REF]. Since its creation, the radiomics field underwent several modifications in its conception to the extent that two different branches are now distinguishable: the hand-crafted radiomics (HCR) and the deep learning radiomics (DLR). The key concepts of the radiomics pipeline and differences between the two existing branches will be discussed in the next chapter.

Chapter 2

Radiomics: state-of-the-art

Introduction

As explained previously, there are mainly two ways for establishing the diagnosis and the staging of liver cancers: the biopsy and the diagnostic imaging. The biopsy, as detailed earlier, suffers from a lot of drawbacks. It remains an invasive procedure, with a high cost in terms of resources, and does not consider the tumor heterogeneity. The diagnostic imaging on the other hand, is not invasive, provides information about the tumor shape, the growth over the time, and is less prone to bias due to tissue heterogeneity. The biopsy remains the only available ground truth since diagnosis imaging still suffer from the medical expert subjectivity. The biopsy and the medical imaging analysis are currently complementary considering that they provide different information. In our work, we are trying to prove that medical images can be used to improve the visual inspection of the medical expert and at long-term replace invasive procedure such as the biopsy.

The improvements in the medical imaging field allow acquisition of data enabling a better characterization of the patients phenotype. In the case of brain imaging, the augmentation of contrast, on MR images, thanks to the injection of contrast agents such as gadolinium-based agents (method mentioned before) is an important technique for the evaluation of brain and liver tumors [START_REF] Drevelegas | Imaging of brain tumors with histological correlations[END_REF][START_REF] Zhou | Radiologically Defined Ecological Dynamics and Clinical Outcomes in Glioblastoma Multiforme: Preliminary Results[END_REF][START_REF] Thian | Liver-specific agents for contrast-enhanced MRI: Role in oncological imaging[END_REF]. This tool allows a delineation of large tumors and an early detection of small metastatic lesions. The different MRI sequences (e.g. the T1 weighted sequences) also allow an internal separation of the tissue within the same tumor (active vs necrotic part of the tumor) [START_REF] Drevelegas | Imaging of brain tumors with histological correlations[END_REF]. Support brought by the innovations in the medical imaging field have been demonstrated on other organs such as the liver [START_REF] Davnall | Assessment of tumor heterogeneity: An emerging imaging tool for clinical practice?[END_REF], the breast [START_REF] Koolen | 18F-FDG PET / CT as a staging procedure in primary stage II and III breast cancer : comparison with conventional imaging techniques[END_REF] or the colon [START_REF] Sahani | Current status of imaging and emerging techniques to evaluate liver metastases from colorectal carcinoma[END_REF] with a consequent benefit in terms of diagnosis. However, even though the advancements in the medical imaging fields have allowed these performances, the interpretation of the medical images remains subjective and not quantitative. In order to improve the quality of the diagnostic, it is important to decode the information present in the image, often difficult or impossible to assess with the naked eye (such as textural-based information).

Introduced in the 80s, CAD (Computer Assisted Diagnosis) tools were the first to establish a link between the imaging features and the biological characteristics of the patients [START_REF] Doi | Computer-aided diagnosis in medical imaging: Historical review, current status and future potential[END_REF]. In order to go 36 CHAPTER 2. RADIOMICS: STATE-OF-THE-ART along with those new systems, standards were introduced such as the one created by the WHO (World Health Organization) or the RECIST (Response Evaluation Criteria in Solid Tumors) [START_REF] Jaffe | Measures of response: RECIST, WHO, and new alternatives[END_REF], where the objective was to assess the evolution of the disease following the progression of the tumor size, but here again, those criteria suffer from a too high dependence with the observers.

The term radiomics was introduced in the early 2010s, allowing the computation of more features than the traditional CADs (more than a thousand vs only a dozen previously) and bringing a more complete diagnosis, since CADs were often limited to distinguish benign vs malignant lesions [START_REF] Afshar | From Hand-Crafted to Deep Learning-based Cancer Radiomics: Challenges and Opportunities[END_REF]. This new technique allows some breakthroughs in various applications such as the cancer diagnosis, the detection of the tumors (with the identification of malignant lesions), their classification, the estimation of the patient survival, the prediction of the aggressivity of the tumors, their recurrence, or the advancement of the disease. In the clinical practice, this new method also allows an improvement in the way biopsies are performed, with the identification of the areas where the extraction should be performed [START_REF] Gillies | Radiomics: Images Are More than Pictures, They Are Data[END_REF] or even by predicting when a biopsy is helpful or not [START_REF] Liu | Radiomic Features Are Associated With EGFR Mutation Status in Lung Adenocarcinomas[END_REF].

Compared to the above-mentioned criteria based on a naked-eye examination, we are now able to rely on a computer to analyse the gray-levels at a finer scale. Therefore, two approaches exist, the HCR (Hand-Crafted Radiomics) based on mathematical engineered features, relying on the textural, intensity and shape based properties of the volume of interest, and the DLR (Deep-Learning radiomics), where the extracted features are estimated from the input data without any prior knowledge.

Handcrafted Radiomics

In this section we will describe the HCR pipeline, by first exposing the different steps of the classical workflow, before analyzing the different studies that used HCR on patients suffering from HCCs. We will conclude with the different improvements that should be brought to enhance the predictive value of radiomics.

A conventional radiomics workflow (based on HCR features) starts with the acquisition and the reconstruction of medical images, followed by the segmentation of those images, which is a critical step since HCR features are extracted from organs or lesions having indistinct borders [START_REF] Gillies | Radiomics: Images Are More than Pictures, They Are Data[END_REF]. Once the different areas were segmented, the features are extracted and quantified, and a statistical analysis is performed to select only the most relevant one. The final step consists in building a model that use the selected features to perform the wanted task, which is often either the tumor characterization or its prognosis. The pipeline is illustrated in the figure below 2.1.

We will now describe these different steps, before analyzing recent state-of-the-art HCR studies, before establishing a list of measures needed to be taken in order to improve the quality and the Figure 2.1: Conventional radiomics workflow as depicted by ©Scrivener et al. [START_REF] Scrivener | Radiomics applied to lung cancer: a review[END_REF] reproducibility of future radiomics works.

HCR workflow

Image acquisition and reconstruction

As explained previously (see chapter 1), ultrasonography (US) is the recommended modality as primary imaging test for surveillance. If the surveillance is positive, CT or MR examinations are performed for the diagnosis and the staging of the disease. For the reasons exposed previously, namely the availability, and its robustness when compared to MRI, we will focus on HCR studies based on CT imaging data.

Without entering into the details of how a CT scan works, we can assume that performances of the CT imaging depend mainly on some settings such as the slice thickness, the capability for projecting the density variations into image intensities and the reconstruction algorithm which aims at converting tomographic measurements into cross-sectional images. It has been demonstrated that radiomics features can differ between different scanners with the same settings [START_REF] Berenguer | Radiomics of CT features may be nonreproducible and redundant: Influence of CT acquisition parameters[END_REF]. It is also common to differentiate CT images into two categories, the screening where low dose images are used and the diagnosis with higher quality of contrast obtained with higher doses [START_REF] Thawani | Radiomics and radiogenomics in lung cancer: A review for the clinician[END_REF]. The investigated studies aim for the diagnosis of the disease, therefore higher doses were used during the imaging acquisition.

Images are typically combined with other clinical sources when computing the radiomics features.

Among them, gene expressions, clinical data such as the age, the gender or the past medical history, blood biomarkers or other prognostic markers such as the tumor size, the stage or the recurrence are the main non-imaging sources of data that are used in the radiomics workflow. However, they can be difficult to acquire, normalize and integrate in a radiomics pipeline, therefore, features are most commonly extracted from images only.

CHAPTER 2. RADIOMICS: STATE-OF-THE-ART

Segmentation

Historically, the segmentation was performed manually, hence, prone to the inter-and intra-observer variability, as depicted below in the figure 2.2. Figure 2.2: Inter-observer variability between 4 different observers on multiphasic images ((a) corresponding to arterial phase images and (b) to portal venous phase images), with a dice similarity coefficient from 0.19 to 0.93, ©Bakr et al. [START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF] To reduce this bias, automatic segmentation techniques were developed. These techniques based on the intensity of the pixels suffer from the fact that the intensity of the pixels, in the case of abdominal organs, is often close from one organ to another. On the other hand, models based on statistical representations often require the computation of an energy function, that can involve a large number of parameters, thus being difficult to compute and optimize (more details concerning the liver semantic segmentation techniques are given in the chapter 3).

Features

Once the volume of interest is delineated, different features can be extracted. On one hand, features can be chosen a priori for their ability to translate the physiological behavior expected by the experts.

For example, studies performed on the lungs showed a correlation between the textural homogeneity and the survival of the patients [START_REF] Ganeshan | Tumour heterogeneity in non-small cell lung carcinoma assessed by CT texture analysis: A potential marker of survival[END_REF][START_REF] Balagurunathan | Reproducibility and Prognosis of Quantitative Features Extracted from CT Images[END_REF], or the grade [START_REF] Al-Kadi | Texture analysis of aggressive and nonaggressive lung tumor CE CT images[END_REF]. This knowledge can also be used in the case of brain tumors to assess the response to a treatment, by observing for example the vascular or cellular density [START_REF] Cao | Clinical investigation survival prediction in high-grade gliomas by MRI perfusion before and during early stage of RT[END_REF][START_REF] Zhou | Radiologically Defined Ecological Dynamics and Clinical Outcomes in Glioblastoma Multiforme: Preliminary Results[END_REF]. However, a prior knowledge is not always available for the wanted task, therefore, the alternative is to extract a huge quantity of features and to determine the most relevant one by using for example some machine learning algorithms.

Features can be regrouped into different categories depending on their statistical order (first, second or higher order). For the features belonging to the first statistical order, the intensities of the volume of interest are summarized by an histogram, and different values are computed such as the uniformity or the entropy. Even though these features are often sensitive to the acquisition settings and by the way the histogram is computed, they already permitted the prediction of the malignancy of breast lesions [START_REF] Parekh | Radiomics: a new application from established techniques[END_REF].

Second order statistical features are meant to extract the textural properties of the volume, by considering the neighboring relationship between pixels. This will play an important role especially for the characterization of the heterogeneity of the tissues. This relation is captured by several descriptive matrices (GLCM, GLRLM, . . . ) [START_REF] Parekh | Radiomics: a new application from established techniques[END_REF].

Shape features are often extracted directly from the VOI in order to analyze its geometrical properties (such as the overall volume occupied by the tumor, its sphericity, its roughness or its fractal dimension).

Those features also permitted the prediction of the response to treatment in previous studies [START_REF] Thawani | Radiomics and radiogenomics in lung cancer: A review for the clinician[END_REF].

Finally, higher order features allow the extraction of imaging features in various frequency domains (the Wavelet features are the most commonly used higher order features [START_REF] Thawani | Radiomics and radiogenomics in lung cancer: A review for the clinician[END_REF]).

Features Selection

As listed here, a large quantity of features can be extracted and they tend sometimes to be highly correlated. The important number of extracted features can often cause overfitting during the creation of a predictive model. A features reduction phase is often required, either in a supervised manner CHAPTER 2. RADIOMICS: STATE-OF-THE-ART (features are selected for their discriminant power in regards with the wanted task) or in non-supervised manner (the main objective is to suppress the redundant features without considering the different labels) [START_REF] Parekh | Radiomics: a new application from established techniques[END_REF].

Among the supervised methods, one can distinguish the univariate ones, where the features are tested one by one depending on their contribution to the final decision (Wilcoxon or Fisher test), from the multivariate where the features are regrouped into subsets before being tested against the output class (such as the Wrapper or the Embedded methods).

Unsupervised methods (such as the PCA: principal component analysis) do not consider the label of the data and try to reduce the dimensionality of the features space, however the new principal component are difficult to interpret.

Once the number of features is reduced, the next step is to construct a predictive model, by using either clustering methods (patients are regrouped based on a metric depending on the retained features, but their label can not be inferred from the clustering) or classification ones (where models such as RF: random forests or SVM: support vector machines are trained from the selected features in order to predict the wanted clinical criteria). Concerning the prediction of the survival, it is common to implement slightly different models such as the Kaplan-Meier or the Cox Proportional Hazard [START_REF] Oikonomou | Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy[END_REF].

In the radiomics studies, one of the main goals is the stability of the features against the preprocessing steps described above. In order to reach this objective, it is possible for the patients to undergo the medical imaging examinations several times (test-retest), and the segmentations can be performed by several experts or even by the same expert several times [START_REF] Van Griethuysen | Computational Radiomics System to Decode the Radiographic Phenotype[END_REF].

In summary, when dealing with classical radiomics pipelines, reaching the best results will often depend on the best combinations between the extraction of the features, the technique used to reduce the number of features and the method implemented to create the model. Every modification on the cited steps can have a huge impact on the predictable performances of the created model.

In the next section, we will analyze the different HCR studies performed on patients suffering from HCC and who underwent CT examination. We will first describe the different choices made in each step of the classical pipeline, before presenting ways to improve the quality of future HCR work.

HCR applied to the HCC

In order to analyze the different methods implemented in the HCR field, we reviewed 15 studies performed on patients suffering from HCC and who underwent CT scan examination. Initially, 23 primary liver cancer-related studies have been scanned in our review [START_REF] Wakabayashi | Radiomics in hepatocellular carcinoma: a quantitative review[END_REF], we then selected the 15 HCC-related ones.

We will first describe the different targets of the studies and the details of the cohorts through the number of patients and the clinical criteria that preceded their selection. We then compare the different imaging acquisition protocols, and the way the regions/volumes of interest are delineated. Finally, we will analyze the different features that appeared to be relevant in the studies, before proposing some tracks to improve the reproducibility and the performances of future radiomics work. Details concerning the experimental settings of the studies, the endpoints and the different retained features can be found in the table 2.1.

Experimental setup

The vast majority of the studies were designed to predict the survival of the patients after surgery, or any other type of treatment [START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF][START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF][START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF][START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF]. In clinical trials, the traditional way to evaluate the survival is through the OS (Overall Survival), which corresponds to the duration from either the date of diagnosis of the disease or the start of its treatment, to either the end of the trial or the death of the patient. Being often assimilated to the survival rate, new finer metrics tend to be preferred such as the DFS (Disease Free Survival), which corresponds to the duration from the beginning of the treatment to the date of the recurrence of the disease.

Other ways to evaluate the response to a given treatment were also evaluated in some of the reviewed studies, such as the presence or absence of recurrence [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF], the local control which assess the end of the growth of a tumor [START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF], and other ways to compute the sensitivity to a treatment [START_REF] Kuo | Radiogenomic Analysis to Identify Imaging Phenotypes Associated with Drug Response Gene Expression Programs in Hepatocellular Carcinoma[END_REF][START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF].

Another important aspect that is often assessed by the reviewed studies is the physiological changes brought by the disease, such as the aggressive profile of the tumor, usually translated by the presence of MVI (MicroVascular Invasion), and its association to genes expression [START_REF] Kuo | Radiogenomic Analysis to Identify Imaging Phenotypes Associated with Drug Response Gene Expression Programs in Hepatocellular Carcinoma[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF][START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF][START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF].

The entire 15 studies had a retrospective design, and the number of selected patients varied from 28 [START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF] to 319 [START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF], with a median of 125 patients per study, as illustrated in the figure 2.3. They were all built with data from patients who underwent CT examination, except one, that decided to mix data obtained from both CT and MR examinations [START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF].

Regarding the CT scan protocol, only one study decided to use images acquired before the injection of contrast agent [START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF], whereas two other studies used images acquired at only one phase (early arterial phase for [START_REF] Raman | Preliminary Data Using Computed Tomography Texture Analysis for the Classification of Hypervascular Liver Lesions[END_REF], and portal venous phase for [START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF]), and the remaining studies analyzed multiphase images. Among them, four studies utilized images acquired at both early arterial and portal venous phases [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Kuo | Radiogenomic Analysis to Identify Imaging Phenotypes Associated with Drug Response Gene Expression Programs in Hepatocellular Carcinoma[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF], while the rest were based on traditional triphasic images [START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF][START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF][START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF][START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF]. Concerning the acquisition protocols, early arterial phase images are most of the time acquired around 30s after the injection of contrast agent (between 22 and 35s), and some studies used bolus tracking method to estimate the best acquisition moment instead of using the same timing for all the patients. Portal venous phase images are acquired between 45 and 70s after the injection, and the delayed images obtained during an even larger interval (between 90 and 300s after the injection).

Knowing that images are the key elements in the computation process of the radiomics features, this high variation within the acquisition protocols is the first reason why the standard HCR pipeline should be standardized.

Once the images acquired, the following step consists in selecting the region of interest to compute the features, or evaluate the physiological properties of the lesions to the naked eye.

ROI selection and features extraction

The selection of the region/volume of interest and/or the assessment of the physiological characteristics of the tumor is often performed by one or multiple experienced radiologists. This step of the pipeline was performed by only a single expert in rare cases in the reviewed studies [START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF][START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF], whereas it was usually performed by two experts [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF][START_REF] Raman | Preliminary Data Using Computed Tomography Texture Analysis for the Classification of Hypervascular Liver Lesions[END_REF][START_REF] Kuo | Radiogenomic Analysis to Identify Imaging Phenotypes Associated with Drug Response Gene Expression Programs in Hepatocellular Carcinoma[END_REF][START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF]. When more than one expert is involved, the authors decided to implement ways for quantifying the inter and intra-observer variability, such as the ICC (Intraclass Correlations Coefficient) [START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF][START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF], or the Cohen-k statistics [START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF]. Regarding now the selection of the ROI, we can separate the reviewed studies by the type of features being extracted. On one hand, when using imaging traits corresponding to visual inspection of the tumor characteristics, it is common to use the whole tumor to perform the evaluation. For example when evaluating the vascular invasion, some features like the peritumoral enhancement need to be estimated globally [START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Kuo | Radiogenomic Analysis to Identify Imaging Phenotypes Associated with Drug Response Gene Expression Programs in Hepatocellular Carcinoma[END_REF]. On the other hand, when using computational features, the analysis can be performed on the whole tumor or on a single slice. Some studies decided to compute the features on the entire 3D ROI, as an example Cozzi et al. [START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF] used the entire tumor independently on the tissues present within it , Xia et al. [START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF] Note that some studies decided to compute the features using both the entire volume and the slice-wise approach (e.g. Taouli et al. [START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] evaluated imaging traits globally and computed the ratio using a slice-wise fashion, Peng et al. [START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF] did the same in their study by computing features using an ROI placed at the largest-cross sectional area, and evaluating imaging traits globally).

Worth mentioning that before the computation of features, it is common to filter the images using different kernel sizes, in order to enhance different elements of the volumes such as the blood vessels for example. All reviewed studies that computed quantitative features filtered their images with a Laplacian of Gaussian algorithm to extract features of specific sizes, as depicted in the figure 2.5.

Features selection

Once the ROI delineated and pre-processed, the following step consists in extracting the features, and as explained previously, the choice on which features to extract depends on the type of features the study is going to rely on, quantitative or semantic.

In the case of quantitative features, the reviewed studies often decided to focus on a single category of features (first-order, textural features, higher-order. . . ), thus obtaining a relative small number of features (27 for Li et al. [START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF], 32 for Raman et al. [START_REF] Raman | Preliminary Data Using Computed Tomography Texture Analysis for the Classification of Hypervascular Liver Lesions[END_REF]). However, despite choosing a specific category The images were displayed using a red or blue scale showing negative or positive pixels, respectively. ©Akai et al. [START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF] of features, this number can increase, when combining the native features with the spatial filter, and the different contrast enhanced phases, as in Akai et al. where a total of 96 features are extracted from the initial 6 histogram-based features [START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF].

Some other studies decided to extract the maximum number of possible features by combining the previously mentioned group of features, and thus obtained 300+ features [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF]. The problem in this case, often called "curse of dimensionality", corresponds to a high number of features relative to the number of individuals, and that can cause some troubles when further training the predictable model.

When imaging traits are preferred to classical radiomics features, the number of extracted characteristics is generally below 10, with a high predominance of changes brought by the hepatocarcinogenesis, such as the presence of internal arteries, or the wash-in wash-out effect (example of imaging traits can be observed in the figure 2.6 & 2.7). Even though the number of extracted features is often small, they often correspond to the absence or presence of physiological properties that are sometimes difficult to quantify and that will highly depend on the observer experience.

After the different features are obtained, the next step in the pipeline consists in the selection of the features and the building of the predictive model.

The vast majority of the reviewed studies decided to implement a logistical regression model in order to assess the correlation between features and the study endpoint. Among the existing methods, time-related approaches such as the Cox regression model [START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF][START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF][START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF] and the Kaplan-Meier survival analysis [START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF][START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF] are the most commonly used.

Different statistical approaches were used by the studies that tried to determine the link between selected features and study endpoints such as the LASSO (Least Absolute Shrinkage and Selection

Operator) algorithm [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF]

First Order Statistics

• Shape [START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF] • Skewness [START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF][START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF][START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF] • Kurtosis [START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF] • Mean [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Raman | Preliminary Data Using Computed Tomography Texture Analysis for the Classification of Hypervascular Liver Lesions[END_REF][START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF] • Energy [START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF] • Entropy [START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF] • Peak [START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF] • Standard deviation [START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF] • Enhancement ratio [START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] • Tumor-Liver difference [START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] Second Order Statistics

• Gray Level matrices [START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF][START_REF] Cozzi | Radiomics based analysis to predict local control and survival in hepatocellular carcinoma patients treated with volumetric modulated arc therapy[END_REF] • Cluster prominence [START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF] Higher Order Statistics

• Wavelets [START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF][START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF] • Gabor [START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Bakr | Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study[END_REF] Morphological features

• Tumor margin volume [START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF] • Tumor size 1 [START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] 1 Not as an exact value, but classified (for example the categories were smaller than 2 cm, between 2 and 5 cm, and larger than 5 cm in [START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF])

Two Traits Predictor of Venous Invasion

• Internal arteries [START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Kuo | Radiogenomic Analysis to Identify Imaging Phenotypes Associated with Drug Response Gene Expression Programs in Hepatocellular Carcinoma[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF][START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] • Hypoattenuating halos [START_REF] Renzulli | Can Current Preoperative Imaging Be Used to Detect Microvascular Invasion of Hepatocellular Carcinoma?[END_REF][START_REF] Peng | A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis b virus-related hepatocellular carcinoma[END_REF][START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF][START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF] Intensity-related features

• Peritumoral enhancement [144]
• Presence of a Tumor-Liver difference [START_REF] Banerjee | A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma[END_REF] Textural-related features

• Non-smooth Tumor Margins [144, 142, 147]
• Infiltrative patterns [START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] • Mosaic appearance [START_REF] Taouli | Imaging-based surrogate markers of transcriptome subclasses and signatures in hepatocellular carcinoma: preliminary results[END_REF] 2.2. HANDCRAFTED RADIOMICS

53

Concerning the studies based on quantitative features we can notice that first-order statistical features is the most common discriminant type, which is normal because this group of histogram-based characteristics is often implemented in the existing radiomics tools, whereas higher order statistical features require more advanced knowledge to be implemented, and often lack of interpretability.

Even though Li et al. [START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF] decided to extract only Wavelet features because they consider that the current way of computing textural features is too dependent on the imaging acquisition settings, first and second-order statistical features remain a good indicator of the textural heterogeneity which is often correlated with the physiological advances of the disease.

Worth also noting that imaging traits, often analyzed either alone, or in combination with quantitative features, remain discriminant enough in a lot of studies, because they are directly linked to the physiological changes produced by the disease. Despite needing expertise to be extracted, such as in Segal et al. where 32 different imaging traits were analyzed, their predictable power drives us to consider them in future radiomics studies and focus on a way to quantify them [START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF].

Study reproducibility

Although the majority of the reviewed studies obtained good predictable results in regards with the wanted prediction task, their stability to the experimental settings and their reproducibility remain questionable.

In 2017, Lambin et al. [START_REF] Lambin | Radiomics: The bridge between medical imaging and personalized medicine[END_REF] who remains one of the founders of the radiomics fields [START_REF] Lambin | Radiomics: Extracting more information from medical images using advanced feature analysis[END_REF] published a study proposing a way to rethink the HCR pipeline and assess the reproducibility of future radiomics studies . This assessment is performed thanks to the RQS (Radiomics Quality Score), which evaluates a total of 16 components with various weights. The evaluation of the different components allows the computation of a score ranging from 0 to 36 points where highest weights are given to criteria allowing a better reproducibility of the study, such as the prospective aspect of the study (7 points over 36) or the presence of a validation step in the proposed workflow (5 points over 36, with a penalty of 5 points when no validation at all is present).

Our reviewed studies were evaluated in regards with the RQS, in consensus with a medical research fellow (Wakabayashi Taïga) [START_REF] Wakabayashi | Radiomics in hepatocellular carcinoma: a quantitative review[END_REF]. The details of the different scores per study are reported in the table 2.4. Image Protocol quality The mean obtained RQS by the reviewed studies was 8.73 ± 5.57 points, which corresponds to less than 23% of the maximal possible score, and only one study obtained a more than 50% of the maximum points, which translates the lack of robustness of the current HCR state-of-the-art studies applied to the HCC.

TER 2. RADIOMICS: STATE-OF-THE-ART

1 2 2 2 2 1 2 2 2 2 1 2 2 2 Multiple segmentations 1 0 0 0 1 1 1 0 0 1 0 1 1 1 Phantom Studies 0 0 0 0 0 0 0 0 0 0 0 0 1 0 Multiple time points 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Features reduction -3 3 3 3 3 3 -3 3 3 3 3 3 3 3 Mutlivariate analysis 1 1 1 1 1 0 0 0 1 1 1 1 1 1 Biological correlates 0 1 1 1 0 1 0 0 0 0 0 1 0 0 Cut-off analysis 0 0 0 0 1 1 1 0 1 1 1 0 1 1 Discrimination statistics 2 1 0 0 1 1 1 1 2 1 1 1 1 1 Calibration Statistics 0 0 1 2 0 1 0 0 1 1 1 1 0 2 Prospective study 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Validation -5 -5 -5 -5 -5 3 -5 -5 -5 -5 -5 5 -5 2 Gold standard comparison 2 2 2 2 0 0 2 0 2 0 2 2 2 2 Clinical utility 2 2 2 2 2 2 2 2 2 2 0 2 2 2 Cost-effectiveness analysis 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Open science Data 0 0 0 0 0 1 0 0 0 0 0 0 0 0
The figure 2.8 allows us to see the most respected criteria in regards with the RQS guidelines. The criteria the less respected by the different reviewed studies were all those related with the reproducibility and the robustness. Among them, the analysis of images acquired at multiple times, the implementation of phantom studies to detect inter-scanner differences and features sensitive to those settings, or even the prospective design of the studies that will ensure inclusion of patients undergoing the same protocols were almost always neglected. Even though the different reviewed studies obtained successful results in regards with the wanted task, their robustness relative to the experimental settings and their reproducibility can be questioned, especially when analyzing their results obtained on the radiomics quality scoring system [START_REF] Lambin | Radiomics: The bridge between medical imaging and personalized medicine[END_REF].

One way to allow a better reproducibility for the future radiomics studies is to fulfill the maximum possible criteria introduced by the RQS standard, and to use standard ways to compute the quantitative features, thanks to open-source libraries such as pyradiomics [START_REF] Van Griethuysen | Computational radiomics system to decode the radiographic phenotype[END_REF]. Another way is to reduce as much as possible the impact of human-based interpretation in the HCR workflow. This can be done by replacing the hand-crafted annotations, and the engineered process of features computation and selection by a standardized, automated and data-driven pipeline, similar to what is performed in more recent DLR studies.

Deep Learning Radiomics

In this section we will first present the value brought by the deep learning in the medical imaging field, before focusing on the differences between the HCR and the DLR strategies. We will then describe in detail the DLR concept before presenting the different reviewed studies tackling liver-related problems using a DLR approach. We will outline the different steps of their pipelines such as the use of multiphasic images, the way they incorporated experts' annotations and their choice regarding the deep network architectures.

DL applied to the medical imaging

The main breakthrough brought by DL was its ability to detect morphological properties in images only by using the pixel intensities as input, whereas traditional machine learning methods often required sophisticated hand-crafted features to achieve descent results [START_REF] Litjens | A survey on deep learning in medical image analysis[END_REF][START_REF] Suzuki | Overview of deep learning in medical imaging[END_REF]. Deep learning networks achieved state-of-the-art results in many medical-related applications among which classification, localization, detection, registration and segmentation [START_REF] Ker | Deep Learning Applications in Medical Image Analysis[END_REF]. Even with a small number of training cases, those performances were realized thanks to architecture choice, data augmentation or transfer learning [START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF][START_REF] Hu | Deep learning for image-based cancer detection and diagnosis -A survey[END_REF]. As exposed previously, the automatic segmentation of the tumors brings a volumetric information, that is more powerful than the diameter only, and that allows to compute the tumor burden, which has an importance when estimating the efficiency of a given treatment [START_REF] Gobbi | The clinical value of tumor burden at diagnosis in Hodgkin lymphoma[END_REF][START_REF] Bornemann | OncoTREAT: A software assistant for cancer therapy monitoring[END_REF][START_REF] Heußel | Quantitative CT-verlaufskontrolle von lebermalignomen nach RECIST und WHO im vergleich zur volumetrie[END_REF][START_REF] Kuhnigk | Morphological segmentation and partial volume analysis for volumetry of solid pulmonary lesions in thoracic CT scans[END_REF][START_REF] Puesken | Prediction of lymph node manifestations in malignant lymphoma: Significant role of volumetric compared with established metric lymph node analysis in multislice computed tomography[END_REF][START_REF] Bauknecht | Intra-and interobserver variability of linear and volumetric measurements of brain metastases using contrast-enhanced magnetic resonance imaging[END_REF].

DLR to tackle HCR limitations

Being a recent field, radiomics starts to mature when applied to several organs such as the lungs or the breast, but it still struggles when applied to the liver, especially because of the scarcity of available data and the complexity of its anatomy. As exposed previously, the first studies targeting liver cancer almost always rely on hand crafted features. The main limitation observed is the lack of reproducibility originating from engineered features, that sometimes result from complex processes which can be difficult to imitate, and often fail to work on different databases. HCR often rely on manual or semi-automatic expert annotations, which often require a complex and time-consuming process, that also provide several constraints, such as the poor inter-expert reproducibility. The extracted features are not necessarily relevant to encode the observed structure, it is needed to increase their number, hence requiring a complex dimensionality reduction step. Another limitation is the difficulty to find the perfect association between features extraction, selection and the statistical analysis used to reach

DLR workflow

In this branch of radiomics, the features are extracted through a deep-learning process, avoiding a manual extraction. A neural network can be trained to generate the most relevant features. These features can either be kept in the network for the final pathological target prediction, or used as input in a different model (such as a SVM or a RF). Compared to the HCR, less prior knowledge is required, and the features can be extracted in an end-to-end manner, using only the raw images as input, and without necessarily providing any segmentation. It has also been demonstrated that performances of those networks increase with the size of the training dataset [START_REF] Cheng | Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans[END_REF]. Eliminating the segmentation phase when evaluating the diagnosis, allows to reduce the workload of experts, and provides a solution to the observer-dependency. When training DLR networks, the original image can be combined with the segmentation or any other pre-processed step result such as the gradient image for example [START_REF] Sun | Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis[END_REF] to improve the relevance of the extracted features.

Generally, DLR studies can be classified depending on the type of input used, the training strategy or the type of architecture chosen to extract the features.

As input, deep radiomics networks can consider 2D slices independently, however this technique does not bring sufficient information since the decision mainly depends on the entire volume of interest.

The different outputs obtained in a slice-wise fashion can be fused to get a volume-wise decision. The volume by itself can also directly be used as input, however it can raise several issues such as the size of the voxels or the slice thickness. Finally, the classification can be performed by considering a series of volumes corresponding to the entire examination of the patient [START_REF] Shen | Learning from Experts: Developing Transferable Deep Features for Patient-Level Lung Cancer Prediction[END_REF], but here again the question regarding the normalization of the input can be raised.

Once the type of input is chosen, the studies differ depending on the type of network and the training strategy utilized. The networks can be trained from scratch using only the available data or a pre-trained architecture can be fine-tuned. In the first case, the obtained network will be specific to the wanted task, but this specialization can also lead to troubles such as overfitting or the sensitivity to class imbalance. The impact of those problematics can be limited with the help of data augmentation (use of existing data to generate new artificial samples) [START_REF] Kumar | Discovery Radiomics for Pathologically-Proven Computed Tomography Lung Cancer Prediction[END_REF], multi-task training (where the number of parameters is limited by the training of several task using the same network) or the incorporation of the proportion of each class present in the data when building the cost function [START_REF] Jamaludin | SpineNet: Automatically pinpointing classification evidence in spinal MRIs[END_REF]. The other strategy CHAPTER 2. RADIOMICS: STATE-OF-THE-ART consists in using an architecture pre-trained most of the time on natural images, and then re-train only a specific part using the desired dataset [START_REF] Echaniz | Ongoing Work on Deep Learning for Lung Cancer Prediction[END_REF][START_REF] Huynh | Digital mammographic tumor classification using transfer learning from deep convolutional neural networks[END_REF][START_REF] Paul | Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma[END_REF]. It is worth noting that this type of training constrains the pipeline to be slice-wise since existing architectures are often pre-trained on 2D images.

Finally, the features can be extracted using either a supervised or an unsupervised approach. In the supervised case, the most commonly used networks are based on convolutional layers (CNN), followed by one or multiple dense layers to predict the output class. While the network is trained to perform the classification, the features are extracted either after a fully connected layer [START_REF] Paul | Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma[END_REF], or after one of the convolutional layers [START_REF] Li | Deep Learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma[END_REF].

Other variants that are also built with convolutional layers as key components can also be implemented (RNN: Recurrent Neural Networks, LSTM: Long Short Term Memory or Capsule Net). Their goal is to get rid of the limitations caused by the input format that need to be fixed, and by the difficulty to consider an entire 3D volume during the training [START_REF] Azizi | Deep recurrent neural networks for prostate cancer detection: Analysis of temporal enhanced ultrasound[END_REF]. In the unsupervised case, the objective is to learn the data distribution, so that new data from the same distribution can be generated. The most commonly used architecture in this case is the auto-encoder, made up with a part that contracts the information (encoder), in order that the most useful one is conserved to regenerate the original data (decoder). Auto-encoder can be built on top of convolutional layers [START_REF] Echaniz | Ongoing Work on Deep Learning for Lung Cancer Prediction[END_REF], or trained with the aim of being insensitive to the noise added to the input data (denoising auto-encoder) [START_REF] Sun | Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis[END_REF][START_REF] Kim | Deep feature learning for pulmonary nodule classification in a lung CT[END_REF]. Following the same principles which are to reconstruct the original input data using only the most relevant features, some studies implemented other type of generative models such as the DBN (Deep Belief Networks) [START_REF] Sun | Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis[END_REF] or Deep Boltzmann Machines [START_REF] Suk | Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis[END_REF].

Some studies are referred to as hybrid, when features are combined with other sources of data (combination of different modalities [START_REF] Oikonomou | Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy[END_REF] or association with clinical data such as genomic data [START_REF] Emaminejad | Fusion of Quantitative Image and Genomic Biomarkers to Improve Prognosis Assessment of Early Stage Lung Cancer Patients[END_REF]), or when only a part of the pipeline implements deep learning methods, either for the extraction of the features [START_REF] Paul | Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma[END_REF] or when the decision is taken with a fusion between HCR and DLR features [START_REF] Huynh | Digital mammographic tumor classification using transfer learning from deep convolutional neural networks[END_REF].

DLR applied to the liver

Even though the number of studies targeting the liver is increasing, the vast majority of them can be categorized as HCR, and only a few are currently based on deep learning. The main reason behind that is the late emergence of deep learning and the recent outbreak of new architectures and concepts that are often first developed and evaluated in other fields than the medical imaging one (e.g. Residual Network, DenseNet, Capsule Net).

However, we have decided to review the first DLR-liver related studies, in order to understand how DL architectures can successfully be incorporated in a radiomics pipeline. We describe the study endpoints, the data preprocessing, the implementation and training strategies, before analyzing their performances.

DEEP LEARNING RADIOMICS

59

A more detailed analysis of the reviewed studies is available in the following link5 .

Study Endpoint

Within the reviewed DLR studies, the majority of them are targeting a characterization of the tumor, either the classification of FLLs (Focal Liver Lesions) [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF][START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF][START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF][START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF], the estimation of the fibrosis stage [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF], or the prediction of the histological grade [START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF] when two of them focused on the response to treatments, either for recurrence after resection [START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF] or the response after TACE (TransArterial ChemoEmbolization) [START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF].

We selected the studies using CT images to perform their analysis, and realized that the vast majority of them used multiphasic images to perform their research, knowing that the evolution of contrast medium is often correlated with the pathological features of the liver as mentioned in the chapter 1.

The only one that used single phase images, was the one targeting an estimation of the fibrosis stage, and their method was based on portal phase images only [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF]. Regarding the multiphasic studies, there is no consensus about the delay between the injection of the contrast agent and the acquisition of the different phases. They tend to prefer triphasic acquisition, with images acquired before the injection of contrast agent, at early arterial phase and a third phase, either portal venous [START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF][START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF][START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF] or a delayed one [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF][START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF]. Peng et al. decided to get rid of the NECT (Non-Enhanced CT ) phase, but still chose a triphasic acquisition (AR, PV, DELAY). The only retained study performing its analysis on MR images [START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF], also used multiphase images, were 5 phases were processed (precontrast, late arterial, portal venous, equilibrium and delayed phases).

Image processing pipeline

Concerning the image processing pipeline, the data used to train the deep networks are most commonly selected via placement of a bounding box by the experts around the hepatic lesion6 , before a registration between the different phases in case of multiphasic acquisition to compensate the effects of body motion and/or breathing.

The manual delineation of the ROIs is usually done by one or more experts on the raw image [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF][START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF][START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF][START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF][START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF], and only one study decided to perform an automatic segmentation of both the parenchyma and the lesion with the application of a random-walker algorithm, before being checked by experts [START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF]. When the method is based on a 2D approach, selected images are often those presenting the maximal cross-sectional proportion of the lesions, except one study targeting the estimation of the fibrosis stage, that centered the ROI so it displayed the ventral aspect of the liver [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF]. Only one study incorporated 3D information in their pipeline, but they also started the placement of the VOI with the slice presenting the maximal proportion of tumor, and extended it to adjacent slices [START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF].

After placing a bounding box, the images are registered either manually [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF][START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF][START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF] or via the application of a non rigid registration with anatomical constraints [START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF]. No real registration was mentioned for two studies [START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF][START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF] but Yamada et al. evaluated the effects of the registration in the prediction performances of their networks (as depicted in the figure 2.9), and after training several networks with misregistered data (shifted, rotated, skewed) they concluded that in the vast majority of the cases, no statistical significance can be found between the performances of the networks trained with manually registered data and those of the networks trained with misregistered data [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF]. It is worth noting that some studies performed their analysis on cropped or resized JPEG images [START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF][START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF]. This choice might cause loss of data whereas the raw images are used elsewhere with the native HU (Hounsfield Unit) intensities.

Finally, in the reviewed studies, only two of them combined images with clinical data [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF], whereas the other only used image data, which is comprehensible because clinical data are often difficult to retrieve, and can also be difficult to integrate in a deep network, as illustrated in the figure 2.10.

Architecture choice

The reviewed studies differ mainly in the way they built their deep architecture.

In most of the cases, convolutional layers are used for the extraction of the most discriminant features. The main question being whether to use a pre-trained network or to train a network from scratch. In the case of fine-tuning, the most commonly used architectures are the AlexNet and the ResNet [START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF][START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF][START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF][START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF], but it is also usual to compare the results obtain by different pre-trained architectures [START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF][START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF]. The general method is to recycle an architecture trained on a huge dataset such as ImageNet, freeze the weights of the early layers (responsible for the high levels features), adjust and train the last layers on the current database to be more specific. An illustration of this process can be found in the figure 2.11.

Figure 2.11: Illustration of the pre-training strategy adopted by ©Wang et al. [START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF] The rest of the reviewed studies created a custom architecture and trained it from scratch [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF][START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF][START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF]. Two of them used classical convolutional layers followed by max pooling layers, early in the network to extract relevant features, as depicted in the figure 2.12.

Since multiphasic studies often stacked the different phases as channels to feed the deep network, one study decided to extract the temporal information through a different paradigm by using LSTM layers [START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF]. As depicted in the figure 2.13, their architecture first extracted the features using what they called a "ResGLBlock" per phase, with two scaled data as input (a large one with the entire lesion, and a smaller one with finer details), and conserved the temporal information via bidirectional LSTM layers.

Yang et al also created a custom architecture to incorporate the dynamic information [START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF]. The When using a pre-trained architecture, the input size is often dictated by the native architecture (224x224x3 for example for the ResNet architecture [START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF][START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF]), and the different studies need to resample their inputs to fulfill those requirements, which can sometimes affect the performances of the network, whereas custom architectures allow a usage of custom sizes [START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF][START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF][START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF]. However, the size of the lesions or other extracted ROI is often different from one patient to the other, therefore, this

problem is still open.

One way to render the deep networks robust to those changes is through multiscale training. Yasaka et al. for example trained the network with patches cropped at different resolutions from the initial lesion ROI after application of standard geometrical transformations such as rotation or shift [START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF]. The same process of extracting different patches from an initial ROI is performed by two other studies, where the goal is also to balance the different classes [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF].

The networks are then usually trained in a cross validation fashion [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF][START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF][START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF], or validated on external datasets [START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF] to be less affected by the effect of randomness, and to be less prone to overfitting.

Performances

Regarding their performances on their testing sets, the different studies concluded first that fine tuning allows an improvement in the accuracy of the DLR network, when compared to training from scratch (e.g. Wang et al. reported an improvement from 83.7 to 91.2% regarding the FLL type classification accuracy of their model when using a pre-trained network) [START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF][START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF]. Several studies also demonstrated that multiphase images increase the performances or the DLR networks, when compared to single phase input only [START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF]. Instead of training the DLR networks only with images, it is possible to combine them with clinical data which can be difficult to collect, and challenging to integrate in a deep architecture, but are proven to improve the accuracy of the networks in some cases [START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF].

Reported results showed moderate to good accuracy for the wanted tasks. For example, the reported mean accuracy is above 0.90 for the studies targeting a classification between Focal Nodular Hyperplasias, Cysts, HCC and Hemangioma (0.91 in both [START_REF] Liang | Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images[END_REF][START_REF] Wang | Classification of Focal Liver Lesions Using Deep Learning with Fine-Tuning[END_REF]), and it slightly drops to 0.84 when more complex categories are integrated (iCC, combined HCC and difference between HCC and early HCCs) [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF]. Another study performed the classification between HCCs and non HCCs, by additionally incorporating the differentiation stages for the HCC group, but still had comparable performances than experienced radiologists in the diagnostic performances [START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF].

The study targeting the estimation of the fibrosis stage reported results less accurate than those obtained using elastography data (MRE: Magnetic resonance elastography or TE: Transient elastography), but they were the first to perform this analysis on CT images, and their results could be improved with the inclusion of volumetric information, and other sources of data [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF]. The only study targeting the prediction of the histological grade using MR images as input was able to correctly classify the HCCs into the 3 differentiation groups with a mean accuracy of 74%.

CHAPTER 2. RADIOMICS: STATE-OF-THE-ART

Their study however suffers from a lot of limitations such as the reduced size of the cohort, the imbalanced data and the fact that the analysis was only performed in a manually drawn VOI [START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF].

Finally, the studies predicting a response to a treatment reported a high accuracy with an AUC of 0.82 when predicting the recurrence after TACE [START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF], and accuracy above 0.83 in the two external validation sets when estimating the response of TACE in HCC [START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF].

Those results still can be improved, especially by increasing the size of the cohort, or by replacing the manual placement of the bounding boxes with an automatic segmentation method in order to reduce the dependency to single or multiple experts [START_REF] Yasaka | Deep learning for staging liver fibrosis on CT: a pilot study[END_REF][START_REF] Peng | Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging[END_REF].

As a conclusion, the different reviewed studies tend to agree on the fact that a multiphase analysis is necessary to precisely describe and encode the pathological features of the disease. For the rest of the pipeline, no real consensus exists but several strategies are implemented especially to compensate for the small size of the databases. Worth also noting that the reviewed studies correspond to the first DLR liver-related applications, therefore, they tend to tackle the less complex problems such as the FLLs classification. We believe that currently, when only a small amount of data is available, and when the differentiation can be performed using simple criteria, then an HCR-based pipeline will be superior to a DLR one. With future improvements regarding DL applied to the medical imaging field, and with more publicly available data, the next DLR liver-related studies will be ready to tackle more complex challenges.

In our research work, we conducted a study aiming for the histological grade prediction, consequently, we designed a pipeline following the aforementioned conclusions. The key element in our work was an automatic cascaded liver and tumor segmentation performed on registered multiphase CT images. It is worth noting that we are the first to perform prediction of the histological grade from multiphase CT images using a fully automatic pipeline.

Chapter 3

Segmentation of Liver Tissue in computed tomography: state-of-the-art

Introduction

The automatic segmentation of the tumors allows to obtain a volumetric information which corresponds to a better predictor when compared to the diameter only, according to the Response Evaluation Criteria in Solid Tumor (RECIST) standard [START_REF] Eisenhauer | New response evaluation criteria in solid tumours : Revised RECIST guideline ( version 1 . 1 )[END_REF][START_REF] Ye | Radiological biomarkers for assessing response to locoregional therapies in hepatocellular carcinoma: From morphological to functional imaging (Review)[END_REF]. A precise volumetric segmentation of both the liver and its tumors is a prerequisite in case of treatment planning, (TARE, TACE, percutaneous thermal ablation, radiotherapy surgical resection,. . . ) especially to adjust the treatment procedures to the volume of the lesions [START_REF] Al-Nahhas | Selective Internal Radiation therapy[END_REF][START_REF] Yamada | Hepatic artery embolization in 120 patients with unresectable hepatoma[END_REF][START_REF] Albain | Radiotherapy plus chemotherapy with or without surgical resection for stage III non-small-cell lung cancer: a phase III randomised controlled trial[END_REF][START_REF] Rossi | Percutaneous RF interstitial thermal ablation in the treatment of hepatic cancer[END_REF]. Associating liver and tumors segmentation allows the computation of the tumor burden 1 , which has an importance in case of surgical resection, or when estimating the efficiency of a given treatment [START_REF] Nordlinger | Surgical resection of colorectal carcinoma metastases to the liver: A prognostic scoring system to improve case selection, based on 1568 patients[END_REF][START_REF] Jagannath | Tumor burden assessment and its implication for a prognostic model in advanced diffuse large-cell lymphoma[END_REF][START_REF] Gobbi | The clinical value of tumor burden at diagnosis in Hodgkin lymphoma[END_REF][START_REF] Bauknecht | Intra-and interobserver variability of linear and volumetric measurements of brain metastases using contrast-enhanced magnetic resonance imaging[END_REF][START_REF] Bornemann | OncoTREAT: A software assistant for cancer therapy monitoring[END_REF][START_REF] Heußel | Quantitative CT-verlaufskontrolle von lebermalignomen nach RECIST und WHO im vergleich zur volumetrie[END_REF][START_REF] Kuhnigk | Morphological segmentation and partial volume analysis for volumetry of solid pulmonary lesions in thoracic CT scans[END_REF][START_REF] Puesken | Prediction of lymph node manifestations in malignant lymphoma: Significant role of volumetric compared with established metric lymph node analysis in multislice computed tomography[END_REF].

Relying on human expert delineations currently corresponds to the most common way to assess segmentation methods [START_REF] Bilic | The Liver Tumor Segmentation Benchmark (LiTS)[END_REF], although it does not correspond to a true gold standard [START_REF] Heimann | Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets[END_REF] and often requires several experts to segment the different structures for reducing the operator-dependent bias [START_REF] Echegaray | Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma[END_REF][START_REF] Moltz | Advanced segmentation techniques for lung nodules, liver metastases, and enlarged lymph nodes in CT scans[END_REF].

In order to compare digitally obtained segmentations with the expert outputs, different metrics have been implemented over time. However, as explained above, manually segmenting the liver and its different structures is a tedious, time-consuming (up to 90 minutes per patient [START_REF] Gotra | Liver segmentation: indications, techniques and future directions[END_REF]) and operatordependent task, as seen in the figure 3.1. Furthermore, the constitution of an open database needs to respect several ethical aspects, and the patient data privacy. This is the main reason why only a small number of samples have been publicly released over time.

Being a soft organ, the liver presents a large variation in shape among the population (inter-patient variation), and can also be subject to deformations due to respiratory and heart motions in case of multiphasic acquisition after injection of contrast agents (intra-patient variation).

On computed tomography images, the intensities of the liver and those of nearby organs (heart, spleen, stomach) are very similar. The images are often acquired after the injection of contrast agents to offer key information (e.g. wash-in/wash-out). Nonetheless, the acquisition protocol may differ among the institutions (type of contrast agent, moments of injection, injection rate, . . . ) and the obtained contrast-enhanced images often differ with the patient characteristics (e.g. blood flow). Moveover, the liver can easily be affected by diseases such as cirrhosis, and is often the site of either primary or secondary cancers with varying contrast levels (hyper-/hypo-intense tumors, peritumoral enhancement) and shapes (non-smooth margins, hypoattenuating halos, . . . ). All these reasons cause the automatic or semi-automatic segmentation of the liver and its structures to be challenging.

In this chapter, we are first covering the different publicly available datasets as long as the different challenges launched to raise the interest of the scientific community in liver semantic segmentation. We are then investigating the first methods developed to perform the semantic segmentation of the liver and/or liver tumors, before presenting most recent advances brought by deep learning (DL).

Publicly available datasets & open challenges

Hand-crafted feature based semantic segmentation methods

A total of 61 studies performing either the segmentation of the liver and/or its tumors were reviewed to introduce the different hand-crafted feature based segmentation methods. Even though the majority of the studies combined several concepts to perform the segmentation, they were classified based on the algorithm using the most domain-specific knowledge [START_REF] Moghbel | Review of liver segmentation and computer assisted detection/diagnosis methods in computed tomography[END_REF]. The reviewed studies included in this section were initially dedicated to the liver segmentation 3 since this task received initially more interest from the researchers than the liver tumors segmentation task. However, the majority of the detailed methods can also address the semantic segmentation of liver tumors or the segmentation of other abdominal organs 4 .

The first methods that have been used for the semantic segmentation of liver tissue relied mainly on computer vision and machine learning methods. In most of the cases, prior anatomical knowledge about the intensity, the shape or the position of the liver is incorporated in the process. The different methods are supported by several pre-and post-processing steps, and may require one or multiple interactions with experts.

Among the most utilized methods, we will first present those based on the intensity of the voxels, before mentioning the ones requiring manual interactions such as region growing or graph-theory based methods. We will then introduce the geometric deformable models that can be exploited to add local constraints, before introducing methods requiring prior shape knowledge such as probabilistic atlases (PA) or statistical shape models (SSM). Finally, we will mention the different studies that combined the aforementioned methods with the addition of machine learning based steps.

Image Intensity based methods

Some studies decided to first rely on the gray level distribution of the liver in the training images to get an initial segmentation during the inference, before applying a refinement step consisting on active contours, region growing, or morphological techniques with anatomical constraints.

Lim et al. [START_REF] Lim | Automatic segmentation of the liver in CT images using the watershed algorithm based on morphological filtering[END_REF][START_REF] Lim | Segmentation of the Liver Using the Deformable Contour Method on CT Images[END_REF][START_REF] Lim | Automatic liver segmentation for volume measurement in CT Images[END_REF], combined prior-knowledge such as the location of the liver and its intensity distribution with active contours to segment the liver. The rough estimation is determined via thresholding and morphological operations in a multiscale fashion. K-means clustering is applied to remove unwanted tissues from the candidate area. Active contour, as described later, is then implemented to refine the liver boundary by first detecting the boundary between liver and adjacent organs. The segmentation pipeline is depicted in the figure 3.2.

Lee et al. [START_REF] Lee | Identifying Multiple Abdominal Organs from CT Image Series Using a Multimodule Contextual Neural Network and Spatial Fuzzy Rules[END_REF], proposed a multimodal contextual neural network using pixel intensity and neighborhood dynamic to get a rough liver segmentation. The refinement step is performed using fuzzy rules based on prior knowledge such as location or textural properties. Liu et al. [START_REF] Liu | Liver segmentation for CT images using GVF snake[END_REF], developed a method to segment the liver by first detecting the potential contour implementing a Canny Edge

Region Growing

This set of methods is based on the placement of an initial seed point/region and evolves based on the image properties. As an example, Beck and Aurich [START_REF] Beck | Hepatux -a semiautomatic liver segmentation system[END_REF], proposed to manually place seed points iteratively until the entire liver is segmented by 3-D filling using non-linear interpolation rules. Qi et al. [START_REF] Qi | Semi-automatic segmentation of liver tumors from CT scans using Bayesian rule-based 3D region growing[END_REF], tackled the lesion segmentation problem by using a region growing strategy with manually placed seeds. The lesion is modeled as a mixture of Gaussians. A new voxel is added to the region by comparing its intensity and neighborhood to each of the lesions Gaussian through the Bhattacharyya distance. Since manual placement of seeds still suffers from problems such as inter and intra-operator variability, some studies suggested to automatically define seed points to start the algorithm with.

Rusko et al. [START_REF] Rusko | Fully automatic liver segmentation for contrast-enhanced CT images[END_REF][START_REF] Ruskó | Automatic segmentation of the liver from multi-and single-phase contrast-enhanced CT images[END_REF] proposed to define the original seed region by considering a set of rules such as the intensity range of the liver or its anatomical position relative to the heart. Susomboon et al. [START_REF] Susomboon | A hybrid approach for liver segmentation[END_REF] CHAPTER 3. SEGMENTATION OF LIVER TISSUE IN COMPUTED TOMOGRAPHY: STATE-OF-THE-ART decided to partition the abdominal region via intensity-based EM algorithm to find the distribution of soft tissues in the whole volume. The liver region used as seed was further isolated using quad-tree decomposition and textural features. Kumar et al. [START_REF] Kumar | Automatic liver and lesion segmentation: A primary step in diagnosis of liver diseases[END_REF], proposed a region growing method to segment the liver where the initial seed point corresponds to the centroid of a rough liver region obtained based on several anatomical and intensity constraints. To ensure that the centroid does not lie on lesions or dark objects, its intensity must be in the liver likelihood range. A post-processing step was proposed to delineate potential lesions within the obtained liver area. The process is depicted in the figure 3.5. iteratively test the homogeneity of the region after adding candidate points [START_REF] Pohle | Segmentation of medical images using adaptive region growing[END_REF] . In Beck and Aurich, the filling step is controlled by a nonlinear coupling criterion, which computes the weighted intensity difference between a seed point and its neighborhood. Weights are controlled by a non-normalized Gaussian function applied on both the distance to the seed point and on intensity difference [START_REF] Beck | Hepatux -a semiautomatic liver segmentation system[END_REF].

However region growing methods have the disadvantage to poorly handle pathological livers, and often fail due to leakage in presence of weak boundaries.

Geometric deformable models

Both geometric deformable models (aka active contours) and level set based methods rely on boundary tracking to perform the segmentation.

The basic idea of active contours (introduced in 1988) is to come from an initial contour and deform it until it reaches the contour of the object. The deformation function reaches its minimum when the boundary of the object is found. The function used to deform the contour is an energy function that tries to control the smoothness of the curve and that is attracted by the object boundaries.

One of the main drawbacks of this method is that the final curve will always have the same topology as the initial one, meaning for example that multiple objects can not be detected. To overcome those limitations, new models, based on the theory of curve evolution and geometric flows, were designed.

Caselles et al. [START_REF] Caselles | Geodesic Active Contours[END_REF] proposed a formulation where finding the contour will be equivalent to finding a geodesic curve of minimal weighted length, thus the formulation of geodesic active contours .

In the active contours formulation, a parametric characterization of the boundary is utilized, whereas in the level set paradigm, they are embedded as a time-dependent partial differential equation (PDE).

Osher and Fedkiw [222] proposed a definition, where the boundary of an object can be regarded as a zero-level set of a time-dependent function which is evolving based on the speed at which the contour evolves along its normal direction. The negative values correspond to voxels belonging to the liver regions, and positive values to the outside. This paradigm was first implemented by Pan and Dawant [START_REF] Pan | Automatic 3D segmentation of the liver from abdominal CT images: a level-set approach[END_REF], who used the level set technique for the automatic segmentation of liver in CT scans. They realized that the classical formulation can not accurately handle cases where weak boundaries are present in noisy or non-uniform images. Therefore, they decided to use an accumulative curvature-dependent speed function that would depend on the front past history. In order to be compliant with abdominal CT scans segmentation, a priori anatomical knowledge was added to the process by considering the distance from the liver to the skin and the ribs. Initial detection of the skin and the ribs is illustrated in the figure 3 Garamendi et al [START_REF] Garamendi | Automatic segmentation of the liver in CT using level sets without edges[END_REF], proposed to segment the liver using a geometric level set method. After manually placing a circle in the liver region to segment, different candidates ROI were iteratively obtained by Lee et al. [START_REF] Lee | Efficient Liver Segmentation exploiting Level-Set Speed Images with 2 .5D Shape Propagation[END_REF] computed a speed image through a fast marching algorithm, by looking at gradient magnitude in the original image. A rough segmentation is retrieved by performing a 2.5D propagation after placing seed points on the top and the bottom of the liver. A level set method is implemented to refine the obtained area. The evolution of the level-set based contour is controlled by the initial speed image values and by the curvature. Vessels near the liver boundary can be added to the final map using a rolling ball algorithm. Wimmer et al. [START_REF] Wimmer | Two-stage semi-automatic organ segmentation framework using radial basis functions and level sets[END_REF] semi-automatically segmented the liver by extending 2D cross-sectional annotations, in 3D using RBF (Radial Basis Function) and applied a level set technique to refine the obtained volume. The initial area is obtained by manually performing 2 cross-sectional segmentations per axis, and extending them in 3D via RBF after sampling some points along the given contours, as depicted in the figure 3.7. The level set reconstruction surface was then used to refine the obtained shape. The function used takes into account the points initially present after manual segmentation. This method however suffers from errors when an initial surface includes high-contrast boundaries not belonging to the liver, because they will tend to attract the surface during the deformation process.

Graph-theory based methods

The random walker algorithm, which was introduced by Grady et al. [START_REF] Grady | Random walks for image segmentation[END_REF] and the Graph Cut (GC) method correspond to semi-automatic5 methods where the user provides seeds for both the background and the region to segment. They both interpret pixels of the image as nodes on a graph where edges represent adjacency between pixels. The weights on the edges correspond to the similarity between adjacent pixels 6 .

The GC will then try to find the minimum cost function between all possible cuts on the graph to separate the object from its background.

GC often suffers from the "small cut" problem where only the seeds are separated from the rest of the image, whereas the random walker method does not suffer from this problem since it is not aiming for the smallest boundary, as represented in the figure 3.9.

The random walker algorithm has been implemented by Maier et al. [START_REF] Maier | Automatic Liver Segmentation using the Random Walker Algorithm[END_REF] to proceed the segmentation of the liver. Seed points were automatically detected in several regions in a slice-wise fashion. Based on the skin surface and some anatomical assumptions, the rib cage and liver seed points are generated.

Background seeds are obtained by detecting the air, the fat and the bones. Dong et al. [START_REF] Dong | Simultaneous segmentation of multiple organs using random walks[END_REF] proposed a The key part of graph cut methods is to define a relevant energy function, which can sometimes be difficult.

Probabilistic atlases (PA) and Statistical Shape Models (SSM) both utilize prior information on the liver. The main goal of those techniques is to improve the segmentation by not relying only on the gray-level distribution since it was proven not to be sufficient, especially knowing the intensity similarity that the liver presents with close organs [START_REF] Zhou | Constructing a probabilistic model for automated liver region segmentation using non-contrast X-ray torso CT images[END_REF][START_REF] Park | Construction of an abdominal probabilistic atlas and its application in segmentation[END_REF][START_REF] Slagmolen | Atlas based liver segmentation using nonrigid registration with a B-spline transformation model[END_REF][START_REF] Rikxoort | Automatic segmentation of the liver in computed tomography scans with voxel classification and atlas matching[END_REF].

Probabilistic Atlases

To construct a probabilistic atlas, images from the training set are registered to a reference image, whereas manual delineations are warped onto the template image and averaged to get a probability of belonging to the liver for each voxel of the space. The generated atlas is then incorporated in the segmentation process. The studies differ one another on the way they construct the atlas, and on how it is incorporated in the segmentation process. Park et al. [START_REF] Park | Construction of an abdominal probabilistic atlas and its application in segmentation[END_REF] used the MIAMI method (Mutual Information for Automatic Multimodality Image Fusion) which combines a TPS-based (Thin Plate Splines) registration method with MI (Mutual Information) as a similarity metric to segment multiple abdominal organs. The TPS is a registration method that uses control points as constraints to the interpolating function. They used 36 control points that were placed in different organs by experts, and they performed the registration separately for each organ to not be biased by the presence of more control points on the liver surface. The different points can be seen in the figure 3.11.

Zhou et al. [START_REF] Zhou | Constructing a probabilistic model for automated liver region segmentation using non-contrast X-ray torso CT images[END_REF] extracted the so-called "anatomical structure" of each patient, which is composed of the bone structure (determined using gray-level thresholding) and the diaphragm (based on the shape of the air within the lungs) to obtain the liver segmentation. A matrix transformation of the TPS is calculated between each one of the training cases and a predefined standard anatomical structure [START_REF] Zhou | Construction of a probabilistic atlas for automated liver segmentation in noncontrast torso CT images[END_REF]. The atlas is then computed by combining the obtained positions using a voting strategy. When performing the segmentation, the same strategy is performed: the current patient's anatomical structure is estimated, and deformed to the standard one. The gray-level distribution is also used to compute the final probability as well as the spatial location as depicted in the figure 3.12.

Slagmolen et al. [START_REF] Slagmolen | Atlas based liver segmentation using nonrigid registration with a B-spline transformation model[END_REF] resampled all training images to a selected one using affine registration with In those different studies, the segmentation method applied is often related to the way the atlas is created. Therefore Park et al. decided to complete the segmentation step by incorporating the atlas information in a bayesian framework which combines MAP and EM, with gray level distribution as basis [START_REF] Park | Construction of an abdominal probabilistic atlas and its application in segmentation[END_REF]. The segmentation is then refined using a MRF (Markov Random Field) regularization step.

In the other studies, non-rigid or elastic registration is performed from the atlas to the target case [START_REF] Slagmolen | Atlas based liver segmentation using nonrigid registration with a B-spline transformation model[END_REF][START_REF] Linguraru | Automated liver segmentation using a normalized probabilistic atlas[END_REF][START_REF] Li | Fully automated liver segmentation for low-and high-contrast ct volumes based on probabilistic atlases[END_REF], after constraining the input, using either manually defined ROI around the liver [START_REF] Slagmolen | Atlas based liver segmentation using nonrigid registration with a B-spline transformation model[END_REF], or analyzing the gaussian intensity distribution to remove irrelevant parts [START_REF] Li | Fully automated liver segmentation for low-and high-contrast ct volumes based on probabilistic atlases[END_REF].

Statistical Shape Models

Statistical shape models on the other hand try to extract features of the model instead of building an average map. Heimann et al. [START_REF] Heimann | A shape-guided deformable model with evolutionary algorithm initialization for 3D soft tissue segmentation[END_REF] used an evolutionary algorithm to create a population of possible liver shape configurations and evaluated all the population members using a local appearance model based on a kNN-classifier utilizing the profiles. Saddi et al. [START_REF] Saddi | Global-to-Local Shape Matching for Liver Segmentation in CT Imaging[END_REF] constrained the boundary of the liver to fit the global-shape learned from the training samples, and refined it locally using a template-matching algorithm. For both problems, they used the intensity distribution inside and outside the liver to find the best region to segment. Zhang et al. [START_REF] Zhang | Automatic liver segmentation using a statistical shape model with optimal surface detection[END_REF] performed the detection of the liver using a 3-D generalized Hough transform so that each vertex on the surface of the average shape model is stored in a table, by considering its coordinates relative to the liver centroid. The most probable liver centroid is computed for a given test image. A search is then performed to determine the points belonging to the boundary knowing the intensity distribution inside the liver and the properties of the gradient along the boundary. An optimal surface detection algorithm is finally applied to refine the obtained boundary by using a graph-search strategy. The different steps of the algorithm can be visualized in the figure 3.14.

Ling et al. [START_REF] Ling | Hierarchical, learning-based automatic liver segmentation[END_REF] went further by proposing a hierarchical segmentation of the liver in a coarse-to-fine

Methods integrating machine learning based steps

Some studies decided to combine the aforementioned techniques with the introduction of machine learning methods such as shallow neural networks or SVM (Support Vector Machine) to segment the liver and potentially its tumor.

Tsai and Tanashi [START_REF] Tsai | Neural-network-based boundary detection of liver structure in CT images for 3-D visualization[END_REF] used a neural network based on regional histogram to get a rough liver segmentation. Several post-processing steps were added, such as Laplacian filtering to obtain the boundary followed by a smoothing using B-spline functions. Gao et al. [START_REF] Gao | Automatic liver segmentation technique for three-dimensional visualization of CT data[END_REF] first located the liver by extracting peaks on gray-level histograms, and applied domain-based knowledge to remove irrelevant tissues. The liver region is refined based on adjacent slices properties, before a boundary refinement is performed via a deformable contour technique. Schimdt et al. [START_REF] Schmidt | Cognition network technology for a fully automated 3d segmentation of liver[END_REF] incorporated a set of intensity distributions, neighboring relationship between organs and geometrical constraints to segment different parts of the abdomen. Their study was based on several anatomical assumptions, but fails when the liver presents a slightly non-standardized aspect, especially when large tumors are present. Freiman et al.

[266] applied a multi-resolution, multi-class smoothed Bayesian classification followed by morphological adjustments and an active contour refinement for the segmentation of the liver. The classification was performed by iteratively looking at liver and potential tumor gray-level distributions. MAP rule is applied for the two classes. The liver region is adjusted using morphological operations, before an active contour refinement step based on gray-level intensity gives the final segmentation. Freiman et al. [START_REF] Freiman | Liver tumors segmentation from CTA images using voxels classification and affinity constraint propagation[END_REF] further proposed a method for the semi-automatic segmentation of liver tumors, where a SVM classification is employed to separate healthy and tumorous tissues. A 3D energy function is then applied using affinity constraints to get the final VOI. Florin et al. [START_REF] Florin | Liver segmentation using sparse 3D prior models with optimal data support[END_REF] proposed a shape model to segment the liver, that will describe the shape variations only using a small number of key slices. This model, called Sparse Information Model, consists of key slices that are selected to be sufficient so when combined with an interpolation function, the 3D volume can be reconstructed. Those key slices are further used for the segmentation but the result is highly sensitive to initializations.

CHAPTER 3. SEGMENTATION OF LIVER TISSUE IN COMPUTED TOMOGRAPHY: STATE-OF-THE-ART

To segment the liver, Schenk et al. [START_REF] Schenk | Efficient semiautomatic segmentation of 3D objects in medical images[END_REF] combined the "livewire" algorithm, which is a semi-automatic 2D segmentation method, with a shape-base model to approximate the contour between the manually defined slices. The interpolation used to combine the user-defined segmentations is an object-based interpolation considering the distance transforms. Maklad et al. [START_REF] Maklad | Blood vessel-based liver segmentation through the portal phase of a CT dataset[END_REF] proposed to segment the liver by analyzing the blood vessels structure. Abdominal blood vessels are first enhanced via bias field correction as depicted in the figure 3 Chartrand et al. [START_REF] Chartrand | SEMI-automated liver CT segmentation using Laplacian meshes[END_REF] started from a set of user-defined contours, and obtained an initial shape using variational interpolation. Boundary points were then extracted using a template matching method. Intensity profiles were computed in a narrow band around the previously established boundary, before a non-rigid registration scheme based on Laplacian mesh optimization deformed it to the real liver boundaries. Goryawala et al. [START_REF] Goryawala | A low-interaction automatic 3D liver segmentation method using computed tomography for selective internal radiation therapy[END_REF] first extracted 5 different regions on the volumes using a k-means algorithm (the liver, surrounding tissues, peripheral muscles, rib/spinal cord and the air)

where clusters are initialized by user-defined seed points on the central liver slice. An intensity based region growing algorithm is then applied using two more seed points on the top and bottom of the liver, and refined using a 3-axis growing strategy. The slice presenting the largest cross-sectional liver part is automatically identified before being proposed to the user to get a precise boundary which will then be copied in all slices containing liver voxels. This boundary is finally refined using a localized region growing method. One advantage of the presented method is that it seems to be weakly influenced by the user-defined initializations. Li et al. [START_REF] Li | A multistep liver segmentation strategy by combining level set based method with texture analysis for CT images[END_REF] computed the total variation and the L1 norm to initialize the liver shape, before applying a level set method guided by local and global energies, and refining the obtained region using gray-level co-occurrence matrices.

Mostafa et al. [START_REF] Mostafa | CT liver segmentation using artificial bee colony optimisation[END_REF] proposed an artificial bee colony clustering technique to segment the liver, which is then refined using morphological operations . The final step consists of a region growing method to enhance the segmentation obtained previously. Shi et al. started with a blood vessel shape initialization, and deformed the liver shape using a region-specific deformable framework [START_REF] Shi | A hierarchical local region-based sparse shape composition for liver segmentation in CT scans[END_REF]. The blood-vessel shape allows the algorithm to get a more accurate initialization based on the patient specificities. Al-Shaikhli et al. [START_REF] Al-Shaikhli | Automatic 3D Liver Segmentation Using Sparse Representation of Global and Local Image Information via Level Set Formulation[END_REF] Conze et al. [START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF] semi-interactively performed the segmentation of the parenchyma, and the separation between the active and the necrotic part of the lesions on multiphase CT images. Images were registered between the different phases using PV (Portal Venous) phase images as reference, and supervoxels 7

were computed at different scales using a SLIC (simple linear iterative clustering ) algorithm on PV phase images. A random forest was trained on a set of retained supervoxels, and used to get a label prediction. They proved the value of using dynamic CECT and implementing hierarchical multi-scale supervoxels to better segment the liver tissues.

It has been proven that methods relying only on intensity are not robust enough to produce acceptable results especially when dealing with some primary tumors that present high textural heterogeneity.

Prior knowledge and engineered features were added over time to increase the performances, but remained sensitive to the size of the dataset used in the training process, and the amount of interaction to incorporate in the workflow. Historically, semi-automatic methods were used for liver tissue segmentation, while they now tend to be replaced by automatic techniques, offering reproducibility and fewer interactions with experts. However, newer methods still suffered when dealing with pathological livers, often presenting irregular shape or intensity patterns. Recently, deep learning has changed the way of comprehending different computer vision related problems, especially in the medical imaging field, where ,for example, state-of-the-art semantic segmentation methods rely now on these techniques.

Deep Learning based semantic segmentation methods

The different DL studies introduced for the liver tissue semantic segmentation purpose share some common properties that will be exposed hereafter. We will fist detail the different types of DL architectures that have been developed to perform this task, before introducing the sequential segmentation concept shared by almost all of the reviewed studies. The details regarding the networks input type, the training strategies, and the inference schemes were then discussed. Details regarding the reviewed studies can be found in the appendix A.

FCN-based architectures

The use of fully convolutional networks (FCN) was quickly democratized, and popular imaging-related architectures such as VGG-16 or AlexNet were transformed to FCNs by replacing dense layers with convolution layers [START_REF] Ben-Cohen | Deep Learning and Data Labeling for Medical Applications[END_REF][START_REF] Bellver | Detection-aided liver lesion segmentation using deep learning[END_REF]. An illustration of the resulting architecture is given in the figure 3.20.

The FCN allows an architecture to predict as output an object having the same size as the input.

Those architectures were perfectly suitable for the required segmentation tasks.

The most recent studies showed a predominance for U-Net like networks [START_REF] Vorontsov | Liver lesion segmentation informed by joint liver segmentation[END_REF][START_REF] Yuan | Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation[END_REF]. The U-Net was

Cascaded architectures

To segment the liver and the lesions it might contain, the vast majority of the studies used a cascaded architecture where a coarse liver segmentation is performed, before either being refined [START_REF] Yuan | Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation[END_REF] or directly used to segment the lesions [START_REF] Han | Automatic Liver Lesion Segmentation Using A Deep Convolutional Neural Network Method[END_REF][START_REF] Li | H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes[END_REF][START_REF] Kaluva | 2D-Densely Connected Convolution Neural Networks for automatic Liver and Tumor Segmentation[END_REF][START_REF] Ben-Cohen | Deep Learning and Data Labeling for Medical Applications[END_REF][START_REF] Christ | Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks[END_REF]. An example of cascaded architectures is given in the figure below.

Interestingly, some studies decided to extract features before combining them to get a final segmentation map. For example, Bi et al. proceed first to an extraction of both liver and lesion features, before combining them with the original image to perform a pixel-wise categorical classification, as illustrated in the figure 3.25 [START_REF] Bi | Automatic Liver Lesion Detection using Cascaded Deep Residual Networks[END_REF].

Whereas Vorontsov et al. [START_REF] Vorontsov | Liver lesion segmentation informed by joint liver segmentation[END_REF] separated the extraction of liver features and lesion features, as detailed in the figure 3.26. They added a classifier at each step which used features concatenated from

Input type (2D, 2.5D, 3D, patches, multiphase)

The first deep-learning related study that performed the semantic segmentation of the liver or its lesions, used patches as input. Li et al. [START_REF] Li | Automatic Segmentation of Liver Tumor in CT Images with Deep Convolutional Neural Networks[END_REF] trained a classification model where patches were considered as positive if at least 50% of its pixels are tumoral. Details of their network are given in the figure 3.27. Later, Frid-Adar et al. [START_REF] Frid-Adar | Patch-Based Techniques in Medical Imaging[END_REF] also implemented a patch-based segmentation, but by separating non-lesion patches on whether they are located in the liver interior or in the liver boundary. The classification was performed through a multi-scale approach, followed by a CNN-based FP reduction step.

However, the majority of the studies implemented a slice-by-slice segmentation [START_REF] Chlebus | Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing[END_REF][START_REF] Kaluva | 2D-Densely Connected Convolution Neural Networks for automatic Liver and Tumor Segmentation[END_REF][START_REF] Bi | Automatic Liver Lesion Detection using Cascaded Deep Residual Networks[END_REF], whereas some others chose to introduce volumetric consistency by using 3 or sometimes 5 adjacent slices as input in a 2.5D-manner [START_REF] Han | Automatic Liver Lesion Segmentation Using A Deep Convolutional Neural Network Method[END_REF][START_REF] Yuan | Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation[END_REF][START_REF] Bellver | Detection-aided liver lesion segmentation using deep learning[END_REF].

Worth noting that only a few studies incorporated 3D layers in their architecture, such as Concerning the incorporation of temporal information, Sun et al. were the first to utilize multiphase information during the DL-driven automatic segmentation of liver lesions [START_REF] Sun | Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs[END_REF]. One FCN network per channel was trained to extract features before they were merged through a features fusion layer.

Training strategies (pre-processing, pre-trained networks. . . )

Regarding the training strategies, no real consensus can be established.

Concerning the pre-processing steps, the majority of the studies decided to perform the liver coarse segmentation on a down-sampled representation of the images to limit the computational cost, before and the lesions, whereas none of them were present in the other half [START_REF] Bi | Automatic Liver Lesion Detection using Cascaded Deep Residual Networks[END_REF]. The type of slices chosen is directly linked to the loss function used to train the network, where some studies implemented a weighted version of the cross entropy (categorical or binary) [START_REF] Han | Automatic Liver Lesion Segmentation Using A Deep Convolutional Neural Network Method[END_REF][START_REF] Bellver | Detection-aided liver lesion segmentation using deep learning[END_REF][START_REF] Ben-Cohen | Deep Learning and Data Labeling for Medical Applications[END_REF][START_REF] Christ | Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks[END_REF], and some others decided to directly associate the target metric with the loss function through the Dice or Jaccard distance [START_REF] Yuan | Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation[END_REF][START_REF] Chlebus | Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing[END_REF][START_REF] Vorontsov | Liver lesion segmentation informed by joint liver segmentation[END_REF]

Inference schemes (post-processing, ensemble learning)

Finally, the inference schemes present also some differences among the different proposed methods.

The vast majority of the studies applied a post-processing step to refine the obtained liver segmentation by extracting the largest connected component [START_REF] Li | H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes[END_REF][START_REF] Han | Automatic Liver Lesion Segmentation Using A Deep Convolutional Neural Network Method[END_REF][START_REF] Yuan | Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation[END_REF][START_REF] Bellver | Detection-aided liver lesion segmentation using deep learning[END_REF][START_REF] Kaluva | 2D-Densely Connected Convolution Neural Networks for automatic Liver and Tumor Segmentation[END_REF]. They also often took advantage of the cascade paradigm where lesions activations outside of the predicted liver can be removed [START_REF] Li | H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes[END_REF][START_REF] Yuan | Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation[END_REF][START_REF] Vorontsov | Liver lesion segmentation informed by joint liver segmentation[END_REF]. In other studies, the reduction of the FP (False Positives) was done through an additional classifier filtering the different lesions candidates. Chelbus et al. [START_REF] Chlebus | Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing[END_REF] for example implemented an Some other studies decided to add a more sophisticated step by implementing Conditional Random Fields (CRF) [START_REF] Christ | Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks[END_REF][START_REF] Rafiei | Liver Segmentation in CT Images Using Three Dimensional to Two Dimensional Fully Convolutional Network[END_REF][START_REF] Dou | 3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes[END_REF], however, these models tend to be difficult to train, and may also increase the inference time.

The combination of different networks in a so-called ensemble-learning way was also commonly realized by the different studies.

Conclusion

As exposed previously, no real consensus can be found to design a generic DL-related liver tissue semantic segmentation method. However, the state-of-the-art studies share some common properties such as their cascaded architecture, the use of both pre-and post-processing or the use of FCN-based networks as basis. The different teams that participated in either the MICCAI17 or ISBI17 LITS challenge were reviewed by the organizers, who came to the same conclusions [START_REF] Bilic | The Liver Tumor Segmentation Benchmark (LiTS)[END_REF]. They also realized that the best tumor-segmentation results were obtained for large lesions, and within a specific lesion-liver HU difference interval (a difference between -10 and -60). They also noticed that top-ranked methods used some 3D approaches in their architecture, showing perspectives to capture the whole volume context in the future. They are planning to organize again the same type of competition in the future, by particularly providing multiple ground truths and potentially splitting the lesion segmentation task into large and small lesions, since current methods still struggle in segmenting the small lesions.

More recent DL-related studies have been using publicly available databases such as LITS as a way to compare themselves to state-of-the-art methods. As a matter of facts, the key common elements that have been proposed previously served as a basis for new studies.

In the following section, we will present our experiments that were conducted to automatically perform the semantic segmentation of the liver and its potential tumors. Inspired by the above mentioned work, a sequential architecture consisting of several specialized networks was trained by considering the dynamic of the liver thanks to contrast-enhanced CT images. The relevant features computed by our deep cascaded multiphase architecture will subsequently be extracted to predict the histological grade.

Chapter 4

Proposed method for the semantic segmentation of HCC

Introduction

As explained previously, state-of-the-art deep learning techniques allow to perform automatic segmentation of both the liver and its structures with a precision close to the one obtained by the experts. In this section, we will first describe the implemented networks and the different used datasets, before presenting the experiments. After selecting an optimal set of hyperparameters, we evaluated the advantages brought by a cascaded architecture, and the way to implement it, before describing the way to incorporate the multiphase information in our architecture. The results and the conclusions of this work were published as a journal paper [START_REF] Ouhmich | Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks[END_REF].

Motivations

We implemented several architectures to validate the hypothesis that current deep networks can perform automatic delineation of both the liver and its tumors. The wash-in wash-out being an important characteristic of the tumor (see chapter 1), we decided to design an architecture that extract features encoding this specific information. To objectify that incorporating the dynamic information through the use of contrast-enhanced images can improve the performance of the network, we compared results obtained using single and multi-phase networks.

The comparison of several architectures leads to the conclusion that the cascaded approach corresponds to the best paradigm when dealing with small multiphase datasets.

It it worth noting no existing studies proposed to automatically delineate HCC on multiphase CT images using a DL architecture, and that features computed by the our architecture will further be used in a radiomics purpose to predict the histological grade.

Cascaded architecture

Even though no consensus was made regarding existing methods, we noticed that several studies implemented a sequential pipeline that can be modeled as cascaded architecture. The cascade consists in several networks trained to perform a specific simpler task, and that are sequentially connected, to provide a final annotation map. In our case, the goal is the segmentation of the liver and its internal structures with the differentiation between parenchyma, and both the active and the necrotic parts of the hepatic tumor. Our cascade will consequently be composed of 3 networks, as depicted in the figure 4.1. The first one will be specialized to segment the liver in abdominal axial slices, the second will delineate the contours of the tumor in the predicted liver ROI, whereas the final one will differentiate between active and necrotic parts within the obtained tumor ROI [START_REF] Ouhmich | Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks[END_REF]. Each one of the given networks will share the same U-Net architecture.

U-Net network

As a basis architecture, we have decided to implement U-Net like networks because it has been previously used for the semantic segmentation task, and has proven to give good results even with a small number of training samples.

The original U-Net architecture was developed by Ronneberger et al. [START_REF] Ronneberger | U-Net: Convolutional Networks for Biomedical Image Segmentation[END_REF] and initially designed for the delineation of cells in microscopic images. As detailed previously, the network can be divided into two subparts, a contraction one where the information contained within the images is compressed, through In comparison to the original architecture, we have implemented zero-padding convolutions to preserve the image size and obtain a segmentation map with the same spatial dimension as the input image. We conserved the same settings as in the original architecture concerning the number of filters to use at each stage, starting with a pair of convolutions of 64 filters each, and reaching a 32 × 32 × 1024 features map in the bottleneck part of the network.

The same naming-system as in our study will be used here [START_REF] Ouhmich | Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks[END_REF]. Single-phase elementary networks will be referred to by both the input phase and the segmentation target, as an example, PV-Lesion will refer to the network responsible for the segmentation of the lesion, with PV (Portal Venous) phase images as input. The complete U-Net architecture for this specific elementary network is depicted in the figure 4.2.

In order to evaluate the improvements brought by the cascaded architecture, we also trained the original U-Net architecture to perform simultaneously the whole internal tissues segmentation task.

The same naming system as previously will be used where "Full" corresponds to the simultaneous segmentation task, thus, AR-Full will refer to the network dedicated to the segmentation of both the parenchyma, the active and the necrotic part of the lesions simultaneously, with AR images as input.

An illustration of the network is given in the figure 4.3.

Multiphase information

Only the AR and PV phases were considered in the multiphase networks because NECT phase images do not provide enough inter-tissue contrast.

In order to incorporate the multiphase information in our pipeline, we investigated 2 different strategies.

The first one, referred to as DMP (Dimensional MultiPhase), consists in concatenating both the AR and PV images as input to the network (see figure 4.4). The second one referred to as MPF (MultiPhase Fusion), consists in performing both the encoding and the decoding separately for each phase, before merging the output maps (simple addition on the obtained features maps), as depicted in the figure 4.5.

Datasets

The different datasets used in our research work are detailed in the table 4.1. As we can see in the table, and except for LITS-dB and 3DIrcad-dB (3DIrcad-dB being a subset of the LITS-dB [START_REF] Bilic | The Liver Tumor Segmentation Benchmark (LiTS)[END_REF]), they are all different in their construction. Differences can be found on the annotated areas (chosen sparse slices or entire 3D volumes) and on the annotated tissues that can be found in the dataset (some of them only contain ground truth annotations for the liver and the tumors it might contain such as LITS-dB, whereas some others contain only ground truth annotation for the tumor such as TCIA-dB). Another crucial difference concerns the contrast enhanced phases available in each dataset (LITS-dB contains only single phase images whereas TCIA-dB, TheraHCC-dB and G-dB present multiphasic images).

TER 4. PROPOSED METHOD FOR THE SEMANTIC SEGMENTATION OF HCC

To prove the ability of the deep learning to perform the automatic segmentation of both the liver and its internal tissues, such as the parenchyma and both the active and the necrotic part of the tumor, we first performed our experiments on TheraHCC-dB since this database was previously used for the same task [START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF], because it presents multiphase images and finally because this is the only available dataset with complete ground truth for both liver parenchyma, active and necrotic part of the tumors.

Experiments

Material

TheraHCC-dB is composed of images from seven patients, all suffering from HCC and who underwent CECT (Contrast-Enhanced Computed Tomography) examinations, resulting in a total of 13 CT sequences.

More details about the standard CECT examination protocol can be found in the chapter 1.5.2. In our case, images were acquired at 4 different moments: one before the injection of the contrast medium (NECT: Non Enhanced CT) , and the 2 others after the injection to reflect both the arterial (AR) (~20-25s after injection) and the portal venous (PV) phases (~60-70s after injection).

Eight regularly sampled slices across each one the 13 sequences were segmented by 4 experts, resulting in 104 labeled slices.

The segmentation maps obtained from each one the 4 experts were fused using the STAPLE algorithm to reach a consensus map [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation[END_REF].

Data pre-processing

The first task to implement in this study was the inter-phase registration so that environmental effects, such as respiratory motions, will not affect the performances of the networks, and to ensure that a given voxel is at the exact same position for the different CECT volumes of a patient. The registration was performed using a diffeomorphic deformable registration algorithm, where the PV images were used as reference, since they contained the original expert annotations [START_REF] Avants | Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain[END_REF][START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF][START_REF] Ben-Cohen | Deep Learning and Data Labeling for Medical Applications[END_REF][START_REF] Christ | Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks[END_REF].

Another bias that can affect the deep semantic segmentation networks training is the heterogeneous image sizes and voxel resolutions present within the training images. To avoid this bias, it has been decided to scale them so that they all have a 512 × 512 axial size3 and an isotropic voxel resolution of 0.97mm 2 .

The data normalization is another aspect that needs to be considered before feeding the images in the

Training

In order to validate our hypotheses, we have decided to first run experiments on the 3DIrcad-dB to set the most crucial hyperparameters such as the learning rate, the decay, the depth of the network or the type and the amount of data augmentation. The selection process is detailed in the appendix B, and here is the list of the chosen hyperparameters:

• Lr: 1e-4

• Decay: 1e-4

• Number of epochs: 20

• Optimizer: Adam

•
L = - 1 n N i=1 ω class i Pi log P i + (1 -Pi ) log(1 -P i ) (4.1)
We implemented the entire training and inference pipeline with Python 3.6, using the library Keras (version 2.2.4) with a Tensorflow backend (v 1.8 to be compatible with older networks). The different networks were trained on a single NVIDIA GTX 1070 with 8GB VRAM. The obtained learning curves (as exposed in the figure 4.6) allow us to see that even though a small number of images were used to train our networks, we were able to correctly address the wanted segmentation tasks by limiting the overfitting effect, despite the high number of parameters of the networks.

Results

In this section, mean DSCs were computed in a slice-wise fashion for each target class and the different methods were statistically compared using the Wilcoxon signed paired rank tests, since slice-wise DSCs did not follow a normal distribution. We used the DSC as a metric to compare the models since it is one of the most commonly used metric for the semantic segmentation task (other metrics can be found in the appendix D). We first compared {NECT, AR, PV, DMP, MPF}-Liver networks to evaluate which phase allows better liver segmentation. We then trained {NECT, AR, PV, DMP, MPF}-Lesion and {NECT, AR, PV, DMP, MPF}-Necrosis networks separately on the TheraHCC-dB by masking all values outside the liver using ground truth annotations in order to assess whether multiphase information is really useful for the segmentation. The results are given in table 4.2 (the mean DSCs are expressed as a percentage). The results highlighted that the cascaded version performed significantly better than {•}-Full networks for segmenting the active part of the lesion (Cascaded DMP vs PV-Full, P = 0.001).

The resulting segmentation maps allow us to estimate the necrosis rate (which will be the ratio between the necrotic and the tumor areas). This metric is commonly used for diagnosis and prognosis of the treatment outcome. In this configuration 5 , our workflow provided estimates of this valuable biomarker with a mean error rate of 13.0 %, which is accurate enough for clinical application.

However, our method presents some limitations especially because false-positives and false-negatives occurring at a given stage of the cascaded architecture are leading to other problems later in the pipeline.

As an example, when considering the liver segmentation task, missing parts of the tumors are not going to be considered in the masked version used as input for the tumor segmentation task, whereas extra non-hepatic tissues considered as liver by the liver segmentation step, can lead to mis-segmentation later for the tumor and/or the necrosis segmentation task. When segmenting the liver using an axial CT slice as input, our networks usually mis-segment other abdominal organs sharing the same intensity as the liver, such as the spleen, the heart or the stomach, as depicted in the figure 4.7. In some other cases, some parts of the liver or even some portions of its tumors can be missed, as depicted in the Regarding the tumor segmentation task, it has been noticed that active parts of the tumor with intensity close to parenchyma are often prone to mis-segmentation, and are sometimes considered as belonging to the parenchyma, as depicted in the figure 4.9. Some other parts of the liver such as the fat portions close to main vessels are sometimes considered as being tumors, since we are facing a binary classification task and they will often be closer to the tumor class than they are to the parenchyma class, as illustrated in the figure 4.9.

Finally, for the necrosis segmentation step, when the inter-tissue difference is strongly marked, the segmentation of the necrosis is accurate, but when it is less recognizable, the segmentation is more prone to errors as depicted in the figure 4.10. The results will also depend on the experts' annotation method, because they sometimes consider more tissues than the central necrosis when differentiating between active and necrotic parts, as illustrated in the figure 4.10.

When compared with another study that have been using a different dataset composed of MR images, we achieved slightly better segmentation results [START_REF] Zhang | Liver Tissue Classification Using an Auto-context-based Deep Neural Network with a Multi-phase Training Framework[END_REF]. We evaluated our method on the same database used in [START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF], where a manual expert interaction was required for the segmentation phase, which is not the case in the present deep learning approach. To allow fair comparison, the evaluation was conducted on the areas where all the experts reached an agreement as in [START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF]. The mean patient-wise segmentation DSCs are depicted in table 4.4. Our method enabled a better segmentation of the lesions and both necrotic and active parts. Therefore, we were able to predict the patient-wise necrosis rate with a slightly better precision.

Conclusions and discussion

When comparing results obtained by the specialized networks, we proved that the addition of the multiphase information provided better segmentation results than when only single phase images are used. Statistically significant improvement was obtained for the segmentation of both the liver and the With the same experimental conditions, our solution provided a better accuracy for the segmentation of the lesions, and for both their active and the necrotic parts, than the one obtained using a semi-Figure 4.12: From top to bottom : Raw images, ground truth and results of the fully automatic segmentation of liver tissue automatic technique with expert interaction [START_REF] Ouhmich | Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks[END_REF][START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF]. We also obtained equivalent results than those from a similar study considering MR images as input [START_REF] Zhang | Liver tissue classification using an auto-context-based deep neural network with a multi-phase training framework[END_REF].

We noticed some mis-segmentations occurring for all the given tasks (liver, tumor, and necrosis segmentation separately) mainly because of mis-interpretation with some tissues sharing the same properties than some of the targeted classes (spleen, heart or stomach being considered as liver during the liver segmentation task, or active part of the tumor being considered as belonging to the parenchyma when segmenting the tumor on the masked liver ROI). These errors represent one of the main drawbacks of the cascaded architecture since some retained (or missed) tissues will be considered (or forgotten) for the remaining steps in the cascaded pipeline. These effects can be minimized by some post-processing steps such as the consideration of a single connected component after the liver segmentation task, or the filtering of the retained ROIs after the tumor segmentation task, but in our study we did not implemented any of the mentioned post-processing steps because we were only considering the actual performances of the networks, and no the performances of the entire segmentation pipeline.

We concluded that the combination of multiphase registered images used in a cascaded architecture allows us to automatically perform the semantic segmentation of liver tissues.

Chapter 5

Histological grade prediction

Introduction

In this chapter, we present a deep-radiomics based histological grade prediction pipeline. To train this model, we had to use several datasets since none of the available ones contained both precise experts' delineation for the volumes of interest (for liver and liver tumor areas in a multiphase fashion), and the histological grade. We first updated our cascaded semantic segmentation architecture (by modifying the models and training it on larger datasets) and proved the ability of our pipeline to automatically segment unlabeled datasets (tested on a dataset different than the training dataset). We then used the features produced by the tumor semantic segmentation networks to build an automatic deep-radiomics histological grade prediction network. We compared our preliminary results with those obtained using a classical HCR paradigm.

To our knowledge, only one study tackled the problem of estimating the histological grade of HCCs using a DL architecture, but with MR images as input [START_REF] Yang | Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study[END_REF]. More details regarding this study can be found in the section 2.3.4. We have decided to tackle the same issue, but we implemented a fully automatic pipeline where both the segmentation and the grade prediction steps were performed by DL networks.

We start by describing the datasets used to train both the liver and the liver tumor segmentation networks, and the ones used to train and test the histological grade prediction pipelines (TCIA-dB). We then expose the different steps of both the HCR and the DLR pipelines to perform the histological grade prediction. We conclude by exhibiting our preliminary results and propose different areas of improvement.

Material

To build an automatic histological grade prediction pipeline, we decided to extract relevant imaging features from our multiphase liver tumor semantic segmentation network. These features will then be processed by another neural network for the histological grade prediction. To build such an architecture we used 3 different datasets since none of the available datasets was consistent enough to be used alone

Liver segmentation inter-dataset comparison (LITS-dB vs TCIA-dB)

Our objective is to automatically segment the liver on both the AR and the PV phased volumes of the TCIA-dB. In the available datasets (as presented in the table 4.1), only TheraHCC-dB and LITS-dB 5.2. MATERIAL 117 contained expert liver delineation. TheraHCC-dB contains annotations only on sparse slices across the liver, whereas LITS-dB contains full 3D pixel-wise liver annotation but it only contains single-phase images, without any information regarding the acquisition phase (AR, PV and potentially DELAY volumes are mixed in the dataset). Regarding its design and the high number of segmented cases it contained, we decided to train our liver segmentation network using the LITS-dB volumes and test it on the TCIA-dB patients.

We inspected both datasets to assess if a liver segmentation network trained using the LITS-dB volumes could perform the wanted task. Therefore, a medical expert was asked to determine the differences between the two datasets with a visual examination of the CT volumes (inspection of the types of tumors present in both datasets, as well as the different artifacts that can affect the training, such as the presence of benign hepatic lesions, of fat accumulation or metallic artifacts) and through a quantitative analysis of the HU intensities after placement of ROIs (especially to prove the presence of cirrhotic patients in both datasets and for the comparison between AR and PV labeled volumes in both datasets).

For the visual examination, the medical expert observed that both datasets contained liver volumes affected by large solitary tumors (HCC-like lesions), but some other smaller solitary tumors have also been found in both datasets, as depicted in the figure 5.2. It is worth noting that only the LITS-dB contained volumes with multiple lesions suspected to be metastases, as illustrated in the figure 5.3.

Both datasets presented livers with benign lesions, track of fat deposit, or other metallic artifacts, as illustrated in the figure 5.4. Diseased livers were also present in both datasets, where severe signs of cirrhosis were noticed, as exposed in the figure 5.5.

The quantitative analysis aimed to prove the existence of cirrhotic patients in both datasets, and allows an inter and intra-phase evaluation, essentially to prove that a liver segmentation network trained using the LITS-dB volumes can perform the liver segmentation on both AR and PV volumes.

The quantitative analysis has been performed on randomly chosen patients and consisted on the placement of 5 ROIs by the medical expert in the liver parenchyma, the air, the spleen, the bone and the aorta. The aorta was chosen to analyze the changes in terms of contrast agent concentration1 .

The liver parenchyma and the spleen were also affected by the contrast agent diffusion, but they were mainly used to diagnose diseased patients since a difference of more than 18.5 HU2 between both areas at portal venous phase could be a sign of cirrhosis [START_REF] Wells | Computed Tomography Measurement of Hepatic Steatosis: Prevalence of Hepatic Steatosis in a Canadian Population[END_REF]. It is worth noting that the ROIs in the liver parenchyma were placed in the peripheral parts of the liver to avoid vessels. The bone and the air ROIs served as control since the air is supposed to have always the same intensity (-1000 HU independently A large percentage of patients with diagnosed from HCC are also suffering from cirrhosis. The cirrhosis affects the liver parenchyma, which results in the presence of both textural and blood flow alterations.

Performing an automatic segmentation on images from a patient suffering from cirrhosis will be challenging, especially if the training dataset is only composed of patient with normal parenchyma, therefore, we sought to prove the presence of cirrhotic patients in both the training and testing datasets.

The quantitative analysis proved the probable presence of cirrhotic patients in both datasets, as depicted in the figure 5.7.

Finally, regarding the available contrast-enhanced phases per dataset, it has been proved that LITS-dB contains volumes that can be labeled as arterial and some other that can be labeled as portal venous (see figure 5.8). Since all the patients of the TCIA-dB were supposed to be affected by HCC, it would have been ideal to train our liver segmentation network with a dataset only composed of patients suffering from HCC (or by a large solitary tumor), but only 20 patients of the 131 from the LITS-dB were affected by a single lesion larger than 2cm 3 . Since we considered that the type of lesions will only slightly affect the liver segmentation, we decided to keep all the patients of the LITS-dB (independently of the number of lesions present in the liver) to train our liver segmentation network. Moreover, it would have been optimal to train one network per injected phase, or a single network with multiphase images as input. However LITS-dB being composed of both AR and PV volumes (with in-between phase volumes as depicted in the figure 5.8), we used the entire dataset to train a single network, allowing us to segment the liver on both AR and PV volumes independently.

Liver tumor segmentation inter-dataset comparison (G-dB vs TCIA-dB)

We also compared both the training (G-dB) and the testing (TCIA-dB) datasets used for the liver tumor segmentation task using the same analysis as the one performed previously (see section 5.2.1).

Therefore, a medical expert was asked to evaluate both visually and quantitatively the differences between the two datasets. To perform the visual examination, he was asked to evaluate the type of We can see a clear separation between arterial (green dots) and portal venous (red crosses) volumes among the TCIA-dB patients. Portal venous volumes present a higher variance for their liver parenchyma intensities, which can be explained by the differences in terms of duration between the injection of contrast medium and the acquisition of the images. It seems clear that the majority of volumes from the LITS-dB were acquired during the portal venous phase whereas some of them were acquired during the arterial phase. The rest of the volumes can be classified as belonging to the late arterial or the early portal venous phase.

characteristics present in both datasets, not only in the liver but also by comparing the tumors using imaging features. As exposed previously, TCIA-dB contains both large and small tumors. The same panel of tumor sizes was present in the G-dB dataset, as illustrated in the figure 5.9.

Some artifacts were also found in the G-dB dataset, such as the presence of benign lesions, fat accumulation or other metallic artifacts in the hepatic region (see figure 5.10). As it was the case in the TCIA-dB dataset, diseased livers were also present among the G-dB patients, as illustrated in the figure 5.11. Regarding the tumor specific area, several imaging features were found in both datasets:

• the presence of internal arteries

• a peritumoral enhancement

• the presence of either smooth or non-smooth margins

• an hypoattenuating halo surrounding the tumor

• a high textural heterogeneity

• a wash-in/wash-out pattern Images illustrating each of the above mentioned features are given in figure 5.12.

The same quantitative analysis as previously was performed (see section 5.2.1), with the placement of 5 ROIs in randomly chosen patients of both the training (G-dB) and the testing (TCIA-dB) datasets. ROIs were placed in the liver parenchyma, the air, the spleen, the bone and the aorta. These areas

were chosen mainly to detect potential cirrhotic patients in both datasets, as illustrated in the figure 5.15, and to assess the homogeneity for the given phases among the different datasets, as depicted in the figure 5.16.

Histological grade dataset

Regarding the histological grade prediction, both the training and the testing steps have been performed

on the TCIA-dB dataset in a cross-validation manner. This dataset contains images from 18 patients, where 9 were diagnosed with a grade 3 (G3), 7 with a grade 2 (G2) and 2 with a grade 1 (G1). In order to obtain a balanced dataset, it has been decided to split them into two groups, the first containing G1 and G2 patients, and the second containing G3 patients, as it has been done previously in the literature since G2 was considered as being closer to G1 than to G3 [START_REF] Han | Prognostic significance of the worst grade in hepatocellular carcinoma with heterogeneous histologic grades of differentiation[END_REF][START_REF] Zucman-Rossi | Genetic Landscape and Biomarkers of Hepatocellular Carcinoma[END_REF]. Patients from the first group (G1 and G2) were considered as having a low grade (LG), whereas those from the second group have a high grade (HG). Left: G-dB images, right: TCIA-dB images. Wash-in is defined as an enhancement of the lesion of the arterial phase higher than the one of the liver parenchyma, whereas the wash-out is defined as a lesion hypodense/hypointense compared to the liver parenchyma on the portal venous and/or delayed phases. We can see a clear separation between arterial (red) and portal venous (green) volumes for both the G-dB and the TCIA-dB patients. We can however notice a higher heterogeneity among the mean parenchyma intensity in the TCIA-dB dataset, which could be a sign of standardized acquisition settings within the G-dB.

METHODS

127

Methods

In this section, we present our cascaded automatic liver tumor segmentation architecture, inspired from the one implemented in the section 4, especially the so-called CECT-Liver network, responsible for the automatic segmentation of liver on both AR and PV volumes. We then present both the HCR-based and the DLR-based histological grade prediction networks.

Automatic liver tumor segmentation

Our cascaded liver tumor segmentation architecture is composed of one CECT-Liver network, presented hereafter, followed by a multiphase tumor segmentation network, already presented in the section 4.5

(both the MPF-Tumor and the DMP-Tumor networks were trained and their accuracy compared).

We trained the CECT-Liver network on the 131 volumes of the LITS-dB using the same hyperparameters as the ones detailed previously. The network shares the same settings as the previously defined PV-Liver and AR-Liver networks (see figure 5.17). We used here the architecture with 1024 filters at bottleneck, corresponding to about 32M parameters. Once the segmentation is performed on each of the axial slices of a given patient, we implemented 2 post-processing steps by first extracting the biggest connected component in the entire 3D volume before applying a binary opening operation to the retained mask.

Before training our multiphase tumor segmentation network, we have to ensure that both the liver and its internal structures such as the potential tumors are located at the same spatial position between the different CECT volumes (both AR and PV volumes in our case). We therefore decided to register AR and PV volumes of the G-dB (training dataset) and the TCIA-dB (testing dataset) datasets. We implemented the registration pipeline using ANTs [START_REF] Avants | Advanced normalization tools (ANTS)[END_REF], since it has already been used for liver CT scans registration [START_REF] Zhao | Recursive Cascaded Networks for Unsupervised Medical Image Registration[END_REF][START_REF] Zhao | Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network[END_REF]. During the registration process, the liver segmentation was used as a registration mask, and we decided to set the PV volume as target (fixed) volume since it usually presents the finest voxel resolution when compared to AR (or DELAY) volume. The registration pipeline described in figure 5.18 was implemented as follows: we initially resampled the AR volume so that it has the same resolution as the corresponding PV volume. We then performed the liver segmentation using CECT-Liver on both the PV and the resampled AR volumes in a slice-wise manner with a classical post-processing consisting in applying a binary opening operation to the mask and conserving the biggest connected component (see green arrows in the figure 5.18). We obtained a liver mask for both the PV and the resampled AR volumes. We finally applied the registration using a dilated version of the two masks obtained as registration masks (as depicted by the dashed blue arrows in figure 5.18; we dilated the liver mask with a SSE of 5cm when setting the registration mask in order to counter any error in the segmentation process, and to always have both the liver and its border included in the

Histological grade prediction

To perform an automatic histological grade prediction, we compared a traditional HCR-based algorithm where features were extracted from either the automatically predicted or the expert-defined tumor ROI, and our DLR-based algorithm where features were extracted from the second network of the cascaded tumor segmentation architecture before being processed by another neural network.

The TCIA-dB dataset with its associated annotations (liver and liver tumor segmentations) obtained thanks to the procedure described in section 5.3.1) will be used to compared both the HCR and the DLR-based histological grade prediction pipelines.

HCR-based histological grade prediction pipeline

We first performed the prediction of the histological grade using the HCR features by extracting singlephase features (from either the AR or the PV volume). In order to investigate the value brought by the dynamic information, we have decided either to perform the analysis on stacked single-phase features or to extract the features from a so-called perfusion volume, where each voxel intensity corresponds to the difference between the PV and the registered AR volume. An illustration of the process to obtain the perfusion volume is given in the figure 5.19. The features extraction step was performed using the PyRadiomics python module3 . This package allows the extraction of features on various representations of the raw image. In our case, we decided to extract features from both the original image and those obtained after application of LoG4 or Wavelet filtering methods. When using the original images, a total of 107 features are extracted, while only 93 features are extracted from the LoG filtered images (since shape-based features are removed). The wavelet filtering method produced 8 different images, hence a total of 744 features to extract (93 for each version). When combining all produced features for a single image, we reach a total of 944 features (when considering only one filter size for the LoG filtering step). Once the features were extracted, a logistical model was built using the least absolute shrinkage and selection operator algorithm (LASSO).

The objective is to minimize the negative log-likelihood contribution L for each observation i of our training dataset, as described in the equation 5.1.

L = - N i=1 y i (β 0 + β 1 x i ) -ln 1 + e (β0+β1xi) + λ p j=1 |β j | (5.1)
Features were first normalized before the optimal λ regularization parameter was selected. Features with non-zero coefficients β are retained for the construction of the predictive model.

The model was trained in a CV-manner and the histological grade was predicted patient-wise. The hyperparameters such as the λ regularization term were optimized at each fold and the accuracy of the model was computed by counting the number of correctly classified patients in the testing set throughout the entire cross-validation evaluation.

DLR-based histological grade prediction architecture

The main choices when implementing the HCR-based pipeline were the method for the extraction of the dynamic features, the ROI to consider when extracting these features and finally the choice of features to extract from the images, that are often hand-crafted and may not encode the entire relevant information contained in the images. To overcome these limitations, we created an automatic DLR-based architecture for the prediction of the histological grade. The architecture was trained and tested on the 18 volumes of the TCIA-dB dataset. The first step consisted in the automatic segmentation

METHODS

131

of the tumor on multiphase images, as described in the section 5.3.1. To predict the histological grade we decided to focus on what we called the relevant imaging semantic features.

Before applying the extraction of the features, we normalized the dimension of the different volumes of the dataset, so that each voxel measures 0.68 × 0.68mm in the axial plane (because it corresponds to the resolution of the images used to train our semantic segmentation network), and that the volumes have a 2.5mm z-spacing (corresponding to the spacing of the majority of the PV volumes in TCIA-dB).

When predicting the histological grade, we focused only on the centrally located tumor slices, since the histological grade corresponds to a measurement of the evolution of the disease, which tends to have more physiological effects at the center of the tumor. Centrally located slices will therefore exhibit the highest grade for a given patient which in this case corresponds to the observed patient-wise grade (ES1954 histological grading system [START_REF] Ha | Primary carcinoma of the liver: a study of 100 cases among 48,900 necropsies[END_REF]). It is worth noting that a slice-wise architecture was built, first because the features were computed in a slice-wise manner and second because the histological grade tends to be heterogeneous in the lesion, meaning that a slice-wise approach allows us to give a finer prediction to find potential areas with a more advanced disease.

The network responsible for the multiphase segmentation of liver tumors is composed of 2 classical U-Net networks (as illustrated in the figure 4.5), where each one takes either the AR or the PV image as input. We believe that the compressed information present in the bottleneck part of the networks may be sufficient to encode the useful information present in the image (the U-Net will work as an auto-encoder for the semantic information). Therefore, we extracted for each patient of the TCIA-dB, this encoded information in a slice-wise manner, represented by two 32 × 32 × 512 features cubes (one per phase in the MPF-Tumor architecture) per slice as depicted in the figure 5.20.

The architecture depicted in figure 5.21 works as a dimensionality reduction algorithm, where the first 1D convolutional layers are dedicated to reduce the number of features initially present (32 × 32 × 512). The dimensionality reduction step is performed for each phase separately, before the remaining features are combined (simple addition in the features space). A final dense layer takes the remaining features as input and computes the probability of belonging to each class thanks to a softmax activation function (LG vs HG).

Knowing the composition of TCIA-dB (9 high grade patients vs 9 low grade ones), we performed a 9-fold CV training, so that each patient is at least present once in the testing set, and so that the

Experiments & Results

Liver tumor segmentation

To obtain a quantitative evaluation of the prediction accuracy of the CECT-Liver network, we tested it on the TheraHCC-dB. We obtained a mean slice-wise DSC of 90.4 ± 17.5 on the PV images and 86.9 ± 19.1 on the AR images. These results, close to those obtained in our previous work using a CV approach [START_REF] Ouhmich | Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks[END_REF], proved that CECT-Liver can perform liver segmentation on both AR and PV unseen images. The CECT-Liver network was also able to segment unseen volumes of the TCIA-dB in both AR and PV phases even when the liver presents a big lesion, as we can see in the figure 5.22. Even though the overall visual predictions seemed very accurate, some mis-segmentation cases have been noticed. Liver segmentation errors often appear on top/bottom axial slices and are often due to our 2D approach, where the 3D context is lost, and where it is often very difficult to see the extremity of the liver in a single slice, as depicted in the figure 5.23. As exposed previously, the main drawback of our approach corresponds to cases where the missed liver segment is occupied by a liver tumor (see 5. Regarding the liver tumor segmentation, we evaluated both DMP and MPF architectures since no statistical differences were available when comparing results obtained for the tumor segmentation in Second row depicts a bottom liver slice where a big portion of the liver occupied by a tumor has been missed by the liver segmentation. Bottom row also depicts a bottom liver slices where a small portion is missed by the network, but the hepatic lesion is missed as well (green arrow).

Figure 5.24: Example of liver mis-segmentation where a big portion of the lesion has been missed by our liver segmentation network. In this specific case, the portion of the tumor that has been missed corresponded to the necrotic area which was remarkably darker than usual (less than 10HU).

our previous work on TheraHCC-dB [START_REF] Ouhmich | Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks[END_REF].

After training both architectures with the same parameters, we obtained a mean patient-wise DSC of 73.2 ± 20.6 with MPF architecture versus 64.9 ± 27.2 when using the DMP when evaluating the The results highlighted that radiomics features extracted from the automatically predicted tumor ROI seemed to be as relevant as the ones extracted from the experts' defined tumor ROI. This indicates that our semantic segmentation network tends to retain relevant features (at least the features necessary for the wanted task). The incorporation of LoG-based features seems to improve the predictive capacities of the model, and we also noticed that the accuracy was higher when filtering images with small filters (in our case 1mm filter gave the best results). This suggests that small details can be more important than larger ones for the determination of the histological grade, hence the necessity to create a model that aims for local prediction of histological grade to consider the heterogeneity within the tumor. We also realized that areas with low intensity within the tumor had their importance in the determination of the histological grade, especially the presence of necrotic areas, since most relevant features were glm-based features sensitive to small and large areas with low gray level intensity (SALGLE, SDLGLE and LRLGLE features) and intensity-based features sensitive to objects with low gray level intensity (first order 10 th percentile). When comparing the type of input, we realized that the best results were

EXPERIMENTS & RESULTS

137

obtained when extracting features from the AR volumes, potentially meaning that the information relative to the histological grade is encoded through the difference between hypervascular structures of the tumor (more expressed in the AR-phase volumes) and the necrotic areas.

We finally compared the HCR-based model with our DLR-based newly created architecture. After testing several combinations for the hyperparameters, we fixed the number of retained features to 8 as depicted in the figure 5.21 (meaning that after the features dimensionality reduction, we obtained a 32 × 32 × 8 cube per phase), and we considered a 2cm volume (corresponding to 8 centrally located slices with a 2.5mm spacing) when training/testing our architecture.

With our CV-training, we were able to correctly predict the patient-wise histological grade of 15 patients over the 18 of the dataset, as detailed in the table 5.2 (a patient is considered as being correctly predicted when at least half of the retained slices were annotated with the correct GT class).

When considering a slice-wise prediction, we were able to correctly predict ~74% of the slices.

It has been observed that the classification confidence was often correlated with the distance to the center of the tumor, where central tumor slices were correctly classified with a higher probability than distant slices, which were either classified with a lower confidence, or even misclassified, as we can see in the figure 5.27. The quality of the semantic segmentation also often guided the accuracy of the prediction, as depicted in the figure 5.28. These results provided a more detailed prediction than the one consisting of a single patient-wise classification. Being able to compute the histological grade locally (here in a slice-wise fashion) allows us to visually focus on the heterogeneous regions that are crucial when needing to establish a diagnosis.

Our pipeline can also provide a map of the best biopsy sites that will further be necessary in the clinical practice to either evaluate the progression of the disease or its prognosis.

Conclusion

The preliminary results presented here on a very reduced database tend to prove that multiphase images incorporated in a cascaded architecture corresponds to the best combination when performing semantic segmentation of liver and its tumors. Moreover, our preliminary results proved that imaging features extracted from our cascaded multiphase segmentation architecture were relevant enough to be incorporated in a deep radiomics pipeline for the histological grade prediction. Compared to the HCR features-based pipeline, our DLR pipeline allows us to propose a way to predict the histological grade in a fully automatic fashion and at both a patient-wise and a slice-wise scale. It would have also been possible to compute the HCR features in a slice-wise fashion but some of them would have lost their interpretability such as the shape-based features supposed to encode a volumetric information. Even though the current DLR features-based pipeline performed slightly better than the HCR-based pipeline (15/18 patients correctly classified using the DLR-based pipeline vs 14/18 at best for the HCR-based pipeline), the lack of the data does not permit to conclude to the superiority of one or another of the pipelines. Even though our results need to be confirmed by external studies, they tend to show that features extracted in an automatic fashion from the raw images can be more relevant than classical hand-crafted features. Our histological grade prediction results were on par with the ones reported by the only study performing histological grade prediction but using a different dataset composed of MR images and using manually drawn regions of interest.

In the following section we present the final conclusion as well as different axes of improvement for the present work.

Conclusion and Perspectives

Contributions

The purpose of the thesis was to propose solutions to better use medical images in order to improve the characterization of the liver tumors. We successfully presented results proving the ability of deep learning to accurately delineate the liver and its tumors on dynamic CECT images. Moreover, we presented preliminary results proving the ability of automatically extracted deep multiphase imaging features to predict the histological grade of HCCs.

After describing the potential causes that can lead to the different types of liver cancer, we motivated our choice to mainly focus on hepatocellular carcinoma (HCC). To better characterize the liver tumors, and especially the HCCs, we have decided to rely on medical images solely, through a quantitative and objective analysis. Regarding the modality choice, we retained computed tomography because it currently corresponds, with MRI, to the most widely used modality for a non-invasive assessment of the disease. With our medical studies review, we showed that the use of dynamic temporal images is a prerequisite to conduct an imaging based liver cancer study.

In the clinical practice, images are usually analyzed by the experts with the naked eye, but the technological advancements allowed the creation of computer assisted diagnosis tools (CADs), where a few number of imaging features were initially used to differentiate benign and malignant lesions. A technology called radiomics has been developed to compute a higher number of features but it took a long time before this technology was applied to the liver, especially because of the scarcity of publicly available datasets. We provided a detailed description of the radiomics pipeline, based on a manual segmentation of the ROI, followed by the extraction of a high number of engineered quantitative features, thus being called HCR (Hand-Crafted Radiomics).

We presented our review, where a total of 15 HCR studies applied on HCC patients have been analyzed.

We evaluated them against the radiomics quality score (RQS) which has been developed to assess the robustness and the reproducibility of radiomics studies. We pointed out the lack of reproducibility of the studies, with a mean RQS of 8.73 ± 5.57 points out of a possible maximum value of 36 points. Several important criteria were found as being ignored by the majority of the studies, such as a prospective design, the use of open-sourced data, the evaluation of the prediction on a validation dataset, or the extraction of features at multiple timepoints.

The emergence of deep learning has changed the way a lot of imaging related problems are predicted necrosis rate when evaluating the treatment outcomes.

With a robust registration algorithm and an updated version of our cascaded architecture (where specialized networks are trained on a sufficient amount of data using both the LITS_dB and the G_dB datasets), we were able to perform the semantic segmentation of liver and its tumors on unseen cases.

We validated this assumption by performing the segmentation of TCIA tumors, and obtained a mean patient-wise DSC of 73.2 ± 20.6.

We implemented both a HCR and a DLR based pipeline to predict the histological grade of the TCIA patients. Both were trained in a CV manner so that each patient is present once in the testing set. The HCR pipeline consisted in a LASSO logistic regression model responsible for the selection of the relevant features and performing the prediction of the grade of the test patients. The best results

were obtained after computation of radiomics features on the original and the filtered images (using LoG). We found that the most relevant features were those affected by the presence of zones with low intensity such as the necrotic areas. Combining multiphase volumes is somehow difficult when using an HCR based pipeline, indeed, features computed from the "perfusion" volume and stacked single-phase features did not allow a better prediction accuracy. No significance differences could be observed between results obtained when computing features from the expert defined tumor ROI and those obtained when extracting features from the automatically segmented tumor ROI. This suggested that our deep semantic segmentation cascaded architecture retains the relevant imaging features to perform the wanted task.

We then implemented a DLR based architecture to predict the histological grade automatically and provide a prediction at a finer scale. We proved that features learned by our multiphase semantic segmentation network are relevant to perform this task. We predicted the grade for central tumor slices, and after a CV training, we correctly predicted 74% of them. When considering the patient histological grade as being the most frequent one in the central tumor slices, we were able to correctly classify 15 patients over the 18 of the dataset. Regarding the small number of patients involved in the study, it is impossible to determine which one of the implemented paradigms (HCR vs DLR) has the best predictive abilities. It is also worth noting that the results that we presented are only preliminary and further studies are required to confirm them.

As a conclusion, our research work is the first to propose a fully automatic DLR pipeline with multiphase images as input. Our strategy is dedicated to small databases where the entire annotations are often not available but the same workflow can be applied for larger datasets. Our DLR architecture was dedicated, in this research work, to the prediction of the histological grade of HCC patients, but our strategy can be extended to other characteristics of HCC or other types of liver cancers.

Regarding the lack of publicly available datasets, we encourage the creation of an open 3D CECT dataset containing liver volumes of both healthy and diseased patients with precise and complete pathological information, and expert delineations of liver, hepatic vessels and lesions for each phase.

Perspectives

Several axes of improvement can be considered regarding our research work. We will discuss potential improvements to bring to the semantic segmentation of liver tumors, before raising issues regarding the current standards for dynamic CECT acquisition and their impact to future multiphasic CT-based radiomics studies. Finally we present ways to incorporate our work in other DLR-liver related studies, by encouraging the creation of open and precisely annotated CECT datasets.

Semantic Segmentation

As discussed before, the HCR paradigm often requires manual experts segmentations, and several studies already discussed the inter-and intra-observer variability present among the obtained delineations.

This variability is translated in the quality of the retained features, which tend not to be robust enough when evaluating the predictive strength of the model.

The DLR paradigm allows one or multiple steps of the radiomics pipeline to be implemented with deep learning approaches. We decided to focus on the segmentation step and performed it automatically thanks to deep neural networks, thus reducing the number of potential biases. The automatic segmentation, as performed in our research work, however requires a high number of training cases to offer a realistic delineation of the targeted tissues.

In the case of liver tumors, a cascaded architecture stacking a first network responsible for the segmentation of the liver and a second dedicated only to detect the tumor within the obtained liver mask is currently the method allowing to obtain the most accurate results. State-of-the-art liver segmentation methods currently allow us to reach annotations similar to those obtained by the experts with mean DSC often above 0.95-0.96. We believe that there is no real need to improve the liver segmentation in the future, and that more interest needs to be shown in tumor and other liver tissues segmentation.

Regarding the segmentation of liver tumors, current state-of-the-art results were obtained thanks to post-processing steps such as false positives (FP) filtering method to reduce the number of objects misclassified as lesions. It might be that current semantic segmentation architectures still lack the ability to discriminate between real lesions and other areas of the liver sharing the same textural properties such as regions close to the vessels as illustrated in the figure 5.29.

This low recall might be explained by the poor quality of the currently available segmentation datasets.

Liver annotations often contain non-hepatic areas (such as air most of the time) or imperfections close to the organ borders. When performing the automatic liver segmentation, the deep neural networks will most of the time be able to avoid these regions. However, in case they are integrated in the liver mask, they will often be misclassified as tumors by the second network in the cascade (responsible for the tumor segmentation).

To overcome this issue, publicly available datasets such as the LIST-dB need to be properly resegmented. They usually only contain annotations for the parenchyma and/or the tumors, but a third class (or more) could be incorporated, in order to consider tissues belonging to none of these groups such as the cysts or the blood vessels.

As explained in the section 3.4, current state-of-the-art tumor segmentation results are often obtained with 2D or 2.5D networks, but the post-processings steps usually help to improve the obtained accuracy (via 3D CRF or even simple morphological operations). Several studies implemented a U-Net like architecture, where the information is compressed before being decoded to obtain the final segmentation map. One of the key concepts in these architectures is the propagation of features learned in the earlier layers later in the network, either thanks to skip-connections, residual units or even densely connected layers. Another way to perform the segmentation could be by implementing a multi-scale pyramidal architecture where tumor-related features can be learned directly from the raw images at multiple scales instead of being learned from previously computed features, which is often the case in the aforementioned architectures.

It might also be interesting to perform the segmentation with a full 3D architecture. Some studies already investigated this solution [START_REF] Dou | 3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes[END_REF], but they could not yet outperform state-of-the-art results.

Some other concepts could be incorporated in future architectures. Jin et al. [START_REF] Jin | RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans[END_REF] for example proposed a network that integrated both U-Net and attention residual mechanism to carry out the segmentation In the case of multiphase images, the expert performing the segmentation will usually try to obtain only one segmentation volume shared by all the registered volumes. This representation might be coherent for the liver mask, but we believe that it needs to be modified for internal liver tissues. The lesions, for example, will have a different behavior from one phase to the other (hypo-or hyper-dense tumors for example), consequently, one segmentation per phase should be performed. This might potentially help the deep neural networks to better understand the dynamic of the lesions, and also address the problem of small mis-registration when only one ground truth segmentation map is shared by all the available phases images.

One long-term objective could be to create, with the help of radiological experts, a dataset containing registered multiphase CT volumes with expert annotations performed following the aforementioned protocol.

Deep Radiomics

Regarding our DLR study, we believe that the prediction of the histological grade can be improved.

As an example we advocate for a grade prediction at a finer scale, but it will require to know the exact position of the extracted sample, or to obtain a map of the heterogeneous regions after surgical removal of liver tissues through liver resection or liver transplant for example.

In our research work we investigated a patch-wise prediction of the histological grade but the results were not as promising as those obtained through our slice-wise approach, especially because our patch-wise semantic segmentation network needs to be improved.

We are considering the relevant imaging features as being the ones extracted from the bottleneck part of our U-Net network but other types of auto-encoders could be investigated to better extract the relevant information from the raw images, or from the retained features. We could have also incorporated clinical data (AFP (Alpha-FetoProtein) levels, age, . . . ) in our pipeline, however such data are often difficult to retrieve, and challenging to combine with imaging features, consequently, we decided to focus only on images in the current work.

It is worth also noting that the current registration steps still suffers from limitation essentially because they were performed using the predicted liver masks as registrations masks. Since, the automatic liver delineation is not perfect, this can result in slightly misregistered volumes. Therefore, we advocate an incorporation of an automatic deep learning based registration step in our DLR pipeline, or a workflow requiring no registration at all (for example by considering features learned in each phase separately and combining them for the grade prediction).

If more interest is shown in the future towards the prediction of histological grade, we believe that the existing grading systems have to be standardized by considering more criteria than just the worst or the most frequent grade present in the histological slices, which is currently the case.

Our DLR architecture obtained promising preliminary results for the prediction of the histological grade, but we believe that the same architecture can be adapted to focus on other tasks, such as the prediction of recurrence after treatment using longitudinal studies.

They obtained a Dice of 0.93, increased at 0.94 after applying the 3D CRF, for the liver segmentation, and a Dice of 0.56 for the lesions. They applied the same cross-validation learning on an internal database of 100 CT scans (Train: 60, Val: 20, Test: 20) and they reached a mean dice of 0.91 for the liver and 0.61 for the lesions.

Dou et al. (2016)

In 2016, the first fully 3D neural network dedicated to the segmentation of the liver was developed by Dou et al. Their 3D CNN network was built with large kernels, and had intermediate supervised layers to combat the gradient vanishing problem. In order to implement deeply-supervised mechanism, 3D Deconvolutional layers were built, to bridge coarse feature volumes, to dense probability ones.

The SLIVER07 database containing 30 patients was evaluated, where 20 patients were used to train the networks, and 10 to test it. The global loss of the network incorporated an intermediate loss at each of those stages, and it was noticed that this mechanism allowed a better convergence than a classical 3D-CNN. A fully connected CRF model was also applied in the transverse plane to refine the obtained volume. The results obtained in a cross-validation manner confirm the gain brought by the deeply-supervised mechanism. On the test set, the results were on par with state-of-the-art "classical" methods (level-set, shape-based model. . . ), but with a faster processing time.

Rafiei et al. (2018)

3D convolutional layers were also used by Rafiei et al (2018) to perform the liver segmentation task. Their architecture incorporated an encoding part where 3D layers only were used, whereas the decoding part consisted only of 2D layers. At each iteration, a block of 512x512x38 slices was fed to the network, and the decoding part was mainly intended to capture the 3D shape of the liver. The connection between the encoding part and the decoding part is done by custom skip connections, which first select the center slice in the 3D volume, and then crop it to meet the required size for the concatenation. To combat the classical problem of segmentation errors at the edges of the liver, they used a spatial-weighted-cross-entropy loss function, where pixels close to the liver border will be assigned a higher weight than those in the center. To help the generalization abilities of the network, a dropout layer was added just after the encoding part. Once the segmentation performed, they applied a fully connected CRF model only on the border pixels especially to reduce computation time.

Training and evaluation was completed on the MICCAI 2015 database containing 30 patients. Data augmentation was utilized to combat the lack of data, and 7 images were generated from one original image by rotation in the range [-30, 30]°.

They slightly improved the results obtained by a previous study which used a multi-altas patch-based from scratch, and a fine-tuning after training the network on the Cifar-10 dataset. The network was tested on a database containing 132 CT scans, with a total of 498 metastases. Images were first resampled to get a 0.71-fixed spacing, which corresponded to the finest available spacing in the dataset.

They confirmed the improvement brought by the separation of the non-lesion patches into two different categories (TPR of 85.9 vs 80 when using a strict binary classification). They also found that fine-tuning from Cifar-10 was not as successful as training from scratch (82.8 vs 85.9 on the TPR) .

A.2 LITS papers

In 2017, the Liver Tumor Segmentation (LITS) benchmark was created, where the goal was to have a standard database with a sufficient amount of humanly annotated data to assess the quality of a given model to perform one or multiple tasks, among which, the liver segmentation, the tumor segmentation and detection, and the tumor burden estimation.

The dataset contains a total of 201 CT scans, acquired in 7 different hospitals, and reviewed by 3 independent radiologists. The visible (131 volumes) and the hidden part (70 volumes) were separated so that each institution has the same proportion on both sides.

A high heterogeneity is present in the database, where from 0 up to 75 tumors are present per volume, and an average tumor-liver HU difference of 31.94 HU.

Two challenges have been launched by ISBI and MICCAI, aiming to segment this database. Some of the top-ranked studies that participated have been reviewed since they shared some common specifications. The 64-channels feature maps encoding both the 2D and the 3D information were merged so that the last layer of their architecture could extract the so-called hybrid features, before a pixel-wise probability is generated.

For the coarse segmentation step, they decided to resample the images so they have a fixed spacing of 0.69x0.69x1.0 mm , but for lesion segmentation, the network was trained with images at their original resolution.

that it maximizes directly the performance that is computed from the output of the network. The architecture is composed of a so-called convolutional path where the information is compressed, and a deconvolutional path, where the full image resolution is recovered. Dropout layers were also used at the end of the convolution path, and the other just before the last deconvolution layer.

During the first step of their cascade, images were down-scaled to 128x128 and the volumes were resampled with a slice thickness of 3mm. The training process of this specific step was made only from liver slices plus 5 slices above and below it. For the inference, the segmentation was performed on each slice, and pixels were considered as being part of the liver after applying a thresholding of 0.5 on the probability map, and after extracting the largest connected component.

To train the second step, which consisted of the refinement of the previously obtained liver area, the original volume was firstly resampled to get a fixed slice-thickness of 2 mm. The liver 3D bounding box was extracted from the annotated liver, with 10 more voxels in the 3 directions, and the axial dimension was adjusted to 256x256 pixels.

The inference was achieved by extracting the slices from the predicted liver area, and applying the prediction with the second network. The same post-processing step was implemented to conserve the 3D consistency.

In order to train the last network, the liver VOI was extracted as previously, but by keeping the original resolution to avoid losing small details. The training was completed using only lesion slices, and an additional image was added beside the original image, where intensity difference between lesion and liver voxels was enhanced via 3D regional histogram equalization.

During the inference, all activations outside the predicted liver were removed.

Their method achieved average DSCs of 0.96 for liver segmentation (Dice global of 0.967), 0.66 for tumor segmentation (Dice global of 0.82), and a root mean square error (RMSE) of 0.02 for tumor burden estimation.

Bellver et al. (2017)

The cascade paradigm was also the key element in the study of was implemented in order to deal with class balancing. When training the network responsible for the segmentation of the lesion, they decided to suppress the loss outside the liver area, and considered only pixels inside the liver region when computing the weights of their loss function. Each one of the networks were trained with 2.5D input to exploit the 3D information. After executing both the liver and lesions segmentation, they developed a lesion detector to reduce the number of FP. A patch-based approach was used for the training of the detector, with patches of 80x80 pixels sampled through the liver (having at least 25% overlap with the liver), and classified as positive if at least 50 pixels of the region belong to a lesion. The detector was trained using a ResNet-50-like architecture, pre-trained again on ImageNet, and that was customized by replacing the last classification layer by a binary-classification layer (healthy/non-healthy).

A post-processing invoking a fully-connected 3D CRF model was further called to refine the obtained regions.

Their results proved that the detector helps the network to reduce the number of FP, and also confirmed that learning to segment the lesions only from the liver area helps the network.

With their method, they obtained a Dice per case of 0.59 on the test set of LITS.

Chlebus et al. (2018)

Chlebus et al. (2018) submitted their results to both ISBI and MICCAI challenges while targeting only the lesion segmentation. In their study they not only described the methods used to generate their segmentation, but they challenged the LITS dataset by comparing it with manual annotations from another expert.

3 different U-Net-like architectures (axial, sagittal, coronal) on 4 resolution levels with additional short skip connections dedicated to the segmentation of tumors only were trained on the LITS database.

The loss function considered only the voxels belonging to the liver. In order to reduce the number of FP, an additional step was added where hand-crafted features were computed on the 3D objects obtained after the segmentation step. In order to assess the performance of the neural network, a human (MTRA: medical-technical radiology assistant) was asked to segment the LITS dataset. The dice per case between MTRA and LITS was 0.70. The neural network reached a DSC of 0.51 on the LITS (4.6 FP/case). After using the classifier, the Dice was improved to 0.58, and the number of FP was reduced to 0.7/case. They finally reached a dice per case of 0.68 for the lesion and 0.96 for the liver.

Among the several hand-crafted features, the most relevant one for detecting the FP were shape-based features as long the distance from the object to the boundary. Their first network was trained thanks to a spatial-weighted-cross-entropy as loss function with a higher loss at the edges of the liver, relative to the interior. The same loss function was used for the second network, with a higher weight given to the tumor region, and was combined with the dice loss.

As post-processing, they simply decided to mask all tumor activations that were located outside the predicted liver.

On the 70 volumes of the hidden set (LIST) the average Dice for the liver was 0.91 and 0.72 for the lesions.

Bi et al. (2017)

Bi et al. (2017) also developed a cascaded architecture where a first ResNet segment both the liver and the lesion, and the second one combining the original image with the two obtained probability maps to produce the final segmentation map.

Images were preprocessed by first clipping the intensities to the range [-160, 240], before applying a min-max normalization. To train their networks, they used a total of more than 8,000 randomized slices, where half of them presented both liver and lesions, and the other half where the liver was not present at all.

The original network was pre-trained on the ImageNet dataset for 100 epochs, before being fine-tuned with augmented data from LITS (random scaling, crops and flips).

During inference, a multiscale-strategy was used by resizing the original image to different scales from 512x512 to 640x640 with a step of 32 at each stage, and averaging the obtained outputs.

The use of the multiscale-strategy slightly improves the results obtained using only their cascaded ResNet. A slight improvement was also notable between their cascaded version and the classical ResNet.

original images, but some methods used 3D convolutions for the lesions segmentation task, with smaller input resolution images. Some other techniques used 2.5 input.

Images were pre-processed by clipping the HU values, but without a clear consensus regarding the range to apply. The images are then often normalized and resampled to a specific resolution. The size of the dataset is often increased after the application of data augmentation, which corresponds most of the time to standard geometrical transformations.

The most commonly used loss functions were the weighted cross entropy loss function, having the advantage of solving the class imbalance problem. The Dice loss and the Jaccard loss were also used and are useful because they allow the network updates its weights to reach the best scores regarding the final used metric.

After performing the segmentations, the vast majority of the teams used post-processing techniques to refine their results. Among these techniques, they reported the extraction of connected components, the removal of lesions activation outside of the liver, the fill of erroneous holes in the lesions shape via morphological operations, the application of connected CRF, or the use of a classifier (such as Random Forest) to detect the FP.

Ensemble learning was also applied by the top-ranked methods, either via axis-related networks, the combination of different rescaled input data, or the use of different networks trained in a CV-manner.

Regarding the results, they realized that the best tumor-segmentation results were obtained for large lesions, and within a specific lesion-liver HU difference interval (a difference between -10 and -60).

The top-ranked methods used some 3D approaches in their architecture (Chlebus et al, Li et al),

showing perspectives to capture the whole volume context in the future.

The introduction of such a standard database led to a rise in the interest towards automatic liver and lesions segmentation. The organizers are planning to relaunch the same type of competition in the future, by particularly providing multiple ground truth (instead of only 1 for LITS) and potentially splitting the lesion segmentation task into large and small lesions, since current methods still struggle in segmenting the small lesions. The residual attention mechanism has been introduced in 2017 by Wang et al. [START_REF] Wang | Residual attention network for image classification[END_REF] to perform image classification, with the idea that attention mechanism can help the network focusing on specific parts

A.3 New hybrid techniques

B.1 Learning Rate & Decay

We first evaluated the best pairs of learning rate and decay when the other parameters such as the number of epochs or the depth of the network are fixed.

Here are the results obtained with the following fixed parameters:

• Optimizer: Adam We can see that using a too small initial learning rate (1e-5) will not be enough, for the given number of iterations, to approach the minimum during the gradient descent. On the contrary, when the learning rate is set at a very high value such as 1e-3, with a small decay (1e-5 or lower in our case), the network continuously overshoots the minima, ending with a very large loss. We can see that a too large network depth can cause problems, probably because the size of the training dataset is too small (potential overfitting).

For the next experiments, we have decided to fix the learning rate to 1e-4 and the decay to 1e-4, with a number of filters at the bottleneck stage of the network of 1024, similar to the original architecture [START_REF] Ronneberger | U-Net: Convolutional Networks for Biomedical Image Segmentation[END_REF].

Once those hyperparameters were set, we investigated the amount and the type of data augmentation to apply during the training.

B.3 Data augmentation

We investigate the impact of geometrical data augmentation by first fixing the amount of data augmentation to 4, meaning that each image in the training set will be used to create 4 new images.

The other hyperparameters are fixed as follow:

• Lr: 1e-4

• Decay: 1e-4

• Optimizer: Adam

Translation

We first investigated the value brought by translations during the data augmentation process, by changing the shift from large to small deltas.

Here are the results for each of the chosen sets (mean slice-wise % Dsc, expect for the weighted combined metric):

Shift delta Parenchyma Tumor Combined

No Augm.

Flip

We also evaluated the value brought by flipped images during the data augmentation process. Flip can only be performed horizontally or vertically, thus a maximum of 2 images can be generated for a given initial image. We can see that increasing the number of generated images improves the segmentation performances of the network.

For this network, we haven't seen any value when adding a dropout with a rate of 0.2, however this might be discussed because with a higher number of epochs, the addition of dropout might be beneficial. We finally increased the number of training epochs, and changed the augmentation factor, to realize that the best accuracy was obtained with an augmentation factor of 20 and a network trained with 30 epochs.

Epochs

Augmenting the data with a factor higher than 20 (e.g 24 in the table below) tends not to improve the performances of the network. The same phenomenon is observed when the number of training epochs is increased at more than 30, probably because the network will start to overfit, due to a too large number of training iterations, or due to the presence of images that are too similar.

Where X is the mean of X

Skewness:

skewness = 1 NP NP N i=1 X (i) -X 3 1 NP NP i=1 X (i) -X 2 3
where X is the mean of X.

Skewness measures the asymmetry of the distribution of values about their mean value.

Kurtosis:

kurtosis = 1 NP NP i=1 X (i) -X 4 1 NP NP i=1 X (i) -X 2 2
Where X is the mean of X.

Kurtosis is a measure of the "peakedness" of the distribution of values in the image ROI. A higher kurtosis implies that the mass of the distribution is concentrated towards the tail(s) rather than towards the mean. A lower kurtosis implies the reverse: that the mass of the distribution is concentrated

• HY = - Ng i=1 p y (i) log 2 [p y (i) + ǫ] be the entropy of p y , • HXY = - Ng i=1
Ng j=1 P (i, j) log 2 [P (i, j) + ǫ] be the entropy of P (i, j),

• HXY 1 = -

Ng i=1

Ng j=1 P (i, j) log 2 (p x (i) p y (j) + ǫ),

• HXY 2 = - Ng i=1 Ng j=1 p x (i) p y (j) log 2 (p x (i) p y (j) + ǫ)
We can define the following features:

(i + j -µ x -µ y ) 2 p (i, j) 6. Contrast: contrast = Ng i=1 Ng j=1 (i -j) 2 p (i, j)
Contrast is a measure of the local intensity variation, favoring values away from the diagonal (i = j).

A larger value correlates with a greater disparity in intensity values among neighboring voxels. Here, ǫ is an arbitrarily small positive number (≈2.2×10 -16 )

Low Gray Level Zone Emphasis (LGLZE):

LGLZE =

C.5 Gray Level Dependence Matrix (GLDM) Features

A Gray Level Dependence Matrix (GLDM) quantifies gray level dependencies in an image. A gray level dependency is defined as the number of connected voxels within distance δ that are dependent on the center voxel. A neighboring voxel with gray level j is considered dependent on center voxel with gray level i if |i -j| ≤ α. In a gray level dependence matrix P (i, j) the (i, j) th element describes the number of times a voxel with gray level i with j dependent voxels in its neighborhood appears in image.

Let:

• N g be the number of discreet intensity values in the image,

• N d be the number of discreet dependency sizes in the image,

• N z be the number of dependency zones in the image, which is equal to Ng i=1 N d j=1 P (i, j)

• P (i, j) be the dependence matrix,

• p (i, j) be the normalized dependence matrix, defined as p (i, j) = P (i,j) Nz .

We can define the following GLDM features:

1. Small Dependence Emphasis (SDE):

SDE = Ng i=1 N d j=1 P (i,j) i 2 N z
A measure of the distribution of small dependencies, with a greater value indicative of smaller dependence and less homogeneous textures.

Large Dependence Emphasis (LDE):

LDE = Ng i=1 N d j=1 P (i, j) j 2 N z

Gray Level

Low Gray Level Emphasis (LGLE):

LGLE =

SDLGLE = Ng i=1 N d j=1 P (i,j) i 2 j 2 N z

Small Dependence High Gray Level Emphasis (SDHGLE):

Measures the joint distribution of small dependence with higher gray-level values.

Large Dependence Low Gray Level Emphasis (LDLGLE):

LDLGLE = Ng i=1 N d j=1 P (i,j)i 2 j 2 N z
14. Large Dependence High Gray Level Emphasis (LDHGLE):

LDHGLE = Ng i=1 N d j=1 P (i, j) i 2 j 2 N z

C.6 Neighbouring Gray Tone Difference Matrix (NGTDM) Features

A NGTDM quantifies the difference between a gray value and the average gray value of its neighbors within distance δ. The sum of absolute differences for gray level i is stored in the matrix. Let X gl be a set of segmented voxels and x gl (j x , j y , j z ) ∈ X gl be the gray level of a voxel at position (j x , j y , j z ),

then the average gray level of the neighborhood is:

A i = A (j x , j y , j z ) (C.1) = 1 W δ kx=-δ δ ky=-δ δ kz=-δ x gl (j x + k x , j y + k y , j z + k z ) (C.2) (C.3) where (k x , k y , k z ) = (0, 0, 0) andx gl (j x + k x , j y + k y , j z + k z ) ∈ X gl
Here, W is the number of voxels in the neighborhood that are also in X gl .

We can define the following NGTDM features:

1. Coarseness:

Coarseness = 1 Ng i=1 p i s i 2. Contrast: Contrast =   1 N g,p (N g,p -1) Ng i=1 Ng j=1 p i p j (i -j) 2     1 N v,p

E.2 Historique analyse d'images médicales

Comme évoqué auparavant, l'imagerie médicale apporte une alternative sérieuse à des méthodes invasives telles que les biopsies. Les récentes améliorations dans le domaine de l'imagerie médicale permettent d'acquérir des données de plus en plus pertinentes pour permettre une meilleure estimation des caractéristiques phénotypiques du patient. Dans le cas par exemple du cerveau, l'augmentation de contraste, sur des images IRM, rendue possible via l'injection d'un produit de contraste comme le gadolinium est une technique importante pour l'évaluation des tumeurs cérébrales [START_REF] Zhou | Radiologically Defined Ecological Dynamics and Clinical Outcomes in Glioblastoma Multiforme: Preliminary Results[END_REF][START_REF] Drevelegas | Imaging of brain tumors with histological correlations[END_REF]. Cet outil permet une délinéation des grosses tumeurs et une détection précoce des petites lésions métastatiques.

Les différentes séquences IRM (comme la carte T1 pondérée) permettent également une séparation des différents tissus au sein d'une même tumeur (tumeur active/nécrose) [START_REF] Drevelegas | Imaging of brain tumors with histological correlations[END_REF]. Le bénéfice quant à l'utilisation des nouvelles techniques d'imagerie a également été démontré sur d'autres organes comme le foie [START_REF] Davnall | Assessment of tumor heterogeneity: An emerging imaging tool for clinical practice?[END_REF], le sein [START_REF] Koolen | 18F-FDG PET / CT as a staging procedure in primary stage II and III breast cancer : comparison with conventional imaging techniques[END_REF] ou encore le colon [START_REF] Sahani | Current status of imaging and emerging techniques to evaluate liver metastases from colorectal carcinoma[END_REF] avec un apport conséquent en termes de diagnostic.

Cependant, même si l'amélioration des techniques d'imagerie a permis de telles avancées, l'interprétation des images reste le plus souvent subjective et non quantitative. Une telle quantification est possible via l'extraction et l'utilisation de caractéristiques précédemment difficiles voire impossibles à distinguer à l'oeil nu. Les outils d'aide au diagnostic médical (CAD) ont été les premiers, dès les années 80, à tenter d'établir un lien entre les descripteurs "images" et les caractéristiques biologiques du patient [START_REF] Doi | Computer-aided diagnosis in medical imaging: Historical review, current status and future potential[END_REF]. Afin d'accompagner ces nouveaux systèmes, des standards ont vu le jour comme le "WHO" ou le "RECIST" [START_REF] Jaffe | Measures of response: RECIST, WHO, and new alternatives[END_REF] dont l'objectif est d'évaluer l'évolution de la maladie en suivant la progression de la taille de la tumeur, mais encore une fois, ces critères souffrent d'une dépendance trop importante par rapport à l'observateur. La radiomique à proprement parler a fait son apparition en 2010, permettant de calculer plus de descripteurs que les CADs (plus de 1000 contre seulement 8 à 20 précédemment) et apportant un diagnostic plus complet, les CADs étant la plupart du temps limités à distinguer les masses malignes des masses bénignes [START_REF] Afshar | From Hand-Crafted to Deep Learning-based Cancer Radiomics: Challenges and Opportunities[END_REF]. Cette nouvelle technique a permis une avancée dans de nombreuses applications que sont le diagnostic du cancer, la détection des tumeurs (avec l'identification des tumeurs malignes), leur classification, l'estimation de la survie du patient, la prédiction de l'agressivité des tumeurs, de la récurrence ou encore du grade de la maladie. Sur un plan pratique, cette nouvelle technique a également permis d'améliorer l'appréhension des biopsies, en identifiant des zones à risque sur lesquelles effectuer E.2. HISTORIQUE ANALYSE D'IMAGES MÉDICALES 193 le prélèvement [START_REF] Gillies | Radiomics: Images Are More than Pictures, They Are Data[END_REF], ou bien encore en préconisant de réeffectuer une biopsie [START_REF] Liu | Radiomic Features Are Associated With EGFR Mutation Status in Lung Adenocarcinomas[END_REF]. Suite à l'essor du machine et du deep-learning, la radiomique a connu de nombreuses modifications, si bien que deux approches existent à ce jour. On distingue le paradigme classique (hand-crafted radiomics) qui se base sur des descripteurs dits manuels, de celui où une ou plusieurs étapes font appel à l'apprentissage profond (deep-learning radiomics).

Radiomique classique

L'approche classique nécessite d'effectuer une segmentation manuelle, et de ce fait, sensible à la variabilité inter-observateur [START_REF] Kumar | Discovery Radiomics for Pathologically-Proven Computed Tomography Lung Cancer Prediction[END_REF]. Pour échapper à cette contrainte, des techniques de segmentation semi-automatique ont été mises en place. Certaines, basées sur l'intensité des pixels afin de déterminer leur classe d'appartenance, souffrent du fait que l'intensité des pixels de la tumeur est souvent proche de celle de certains organes avoisinant, d'autres, font appel à des modèles statistiques, nécessitant parfois de calculer une fonction d'énergie trop coûteuse en termes de nombre de paramètres. Une fois le volume d'intérêt délimité, les différents descripteurs peuvent être extraits. Là encore, deux procédés diffèrent l'un de l'autre. Dans un premier temps, les descripteurs peuvent être choisis a priori pour leur capacité à illustrer le comportement attendu par les experts. C'est le cas par exemple sur le poumon où différentes études ont prouvé une corrélation entre l'homogénéité au niveau textural et le taux de survie [START_REF] Cao | Clinical investigation survival prediction in high-grade gliomas by MRI perfusion before and during early stage of RT[END_REF][START_REF] Parekh | Radiomics: a new application from established techniques[END_REF], ou le grade [START_REF] Al-Kadi | Texture analysis of aggressive and nonaggressive lung tumor CE CT images[END_REF]. Ces connaissances a priori peuvent également être utilisées dans le cas de tumeurs cérébrales pour observer la réaction provoquée par un traitement donné notamment en observant la densité vasculaire et cellulaire [START_REF] Oikonomou | Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy[END_REF][START_REF] Zhou | Radiologically Defined Ecological Dynamics and Clinical Outcomes in Glioblastoma Multiforme: Preliminary Results[END_REF]. Les connaissances a priori ne sont pas toujours suffisantes pour répondre à un problème donné, c'est pourquoi une autre possibilité est d'extraire une grande quantité de descripteurs et de déterminer les plus pertinents en utilisant par exemple des algorithmes d'apprentissage automatique. Les descripteurs extraits peuvent être regroupés en différentes catégories : les descripteurs statistiques du premier ordre, ceux du second ordre, et ceux d'un ordre supérieur. Pour les descripteurs de premier ordre, le volume d'intérêt est transformé en histogramme à partir duquel sont calculées différentes valeurs comme l'uniformité ou l'entropie. Ces valeurs étant le plus souvent sensibles aux paramètres d'acquisition de l'image comme l'épaisseur des coupes ou encore les modalités de construction de l'histogramme, ont pourtant permis de déterminer le caractère malin de tumeurs du sein [START_REF] Parekh | Radiomics: a new application from established techniques[END_REF]. Des descripteurs de forme peuvent également être directement extraits du volume d'intérêt, afin d'analyser la structure géométrique de ce dernier (comme la surface totale occupée par la tumeur ou encore sa sphéricité). Ces descripteurs ont déjà permis de prédire la réponse provoquée par un traitement [START_REF] Thawani | Radiomics and radiogenomics in lung cancer: A review for the clinician[END_REF]. Les descripteurs du second ordre ont comme vocation d'extraire les caractéristiques texturales du volume, en considérant la relation de voisinage entre les pixels, qui joue un rôle important notamment pour la caractérisation de l'hétérogénéité des tissus.

Cette relation est capturée par différentes matrices descriptives (GLCM, GLRLM, . . .) [START_REF] Parekh | Radiomics: a new application from established techniques[END_REF]. Enfin, les descripteurs d'ordre supérieur permettent de capturer des caractéristiques images dans de nombreux domaines de fréquence (la transformée en ondelette étant la plus fréquemment utilisée [START_REF] Thawani | Radiomics and radiogenomics in lung cancer: A review for the clinician[END_REF]). Un grand nombre de descripteurs peuvent alors être extraits, cependant la plupart d'entre eux sont fortement corrélés, ce qui peut engendrer un sur-apprentissage lors de la création du modèle prédictif. Une phase de réduction du nombre de descripteurs doit alors être effectuée, de manière supervisée (ce qui signifie qu'ils sont sélectionnés en fonction de leur caractère prédictif au regard de la tâche souhaitée), ou non supervisée (le principal objectif étant de supprimer les descripteurs redondants en ne considérant pas les différentes étiquettes) [START_REF] Parekh | Radiomics: a new application from established techniques[END_REF]. Parmi les méthodes supervisées, on distingue les méthodes univariées, où les descripteurs sont testés les uns après les autres en fonction de leur contribution à la classe cible (test de Wilcoxon, ou de Fisher), des méthodes multivariées (où les descripteurs sont regroupés dans des sous-ensembles avant de tester leur corrélation avec la classe d'arrivée). Les méthodes non supervisées (comme l'ACP par exemple) sont quant à elles moins sujettes au sur-apprentissage car elles ne considèrent pas l'étiquette des données, mais n'ont pour seul but que de réduire la dimensionnalité de l'espace des descripteurs. Une fois le nombre de descripteurs réduit, il reste à construire un modèle prédictif, en utilisant des méthodes de clustering (les patients sont regroupés en fonction d'une métrique dépendante des descripteurs retenus) ou de classification (où des modèles comme les forêts d'arbres aléatoires ou encore les machines à vecteur de support sont entraînés à partir des descripteurs retenus dans le but de prédire le caractère clinique souhaité). Pour la prédiction de la survie du patient, il est commun de faire appel à des modèles un peu différents (Kaplan-meier ou encore Cox Proportional

Hazards [START_REF] Oikonomou | Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy[END_REF]). Un des objectifs recherché est la stabilité des descripteurs vis-à-vis des étapes de pré-traitement. Pour cela, le patient peut subir l'examen d'imagerie à plusieurs reprises (test-retest) et les segmentations peuvent être effectuées par plusieurs experts [START_REF] Van Griethuysen | Computational radiomics system to decode the radiographic phenotype[END_REF]. Une étude radiomique classique repose donc sur la meilleure combinaison entre l'extraction des descripteurs, la technique de réduction ainsi que la méthode de création du modèle. La moindre modification sur chacune de ces étapes peut avoir un impact conséquent sur le caractère prédictif du modèle résultant, c'est pourquoi une nouvelle branche de la radiomique, se basant en partie ou entièrement sur l'apprentissage profond, a fait son apparition.

Radiomique sur base d'apprentissage profond

Dans cette branche de la radiomique, les descripteurs ne sont plus manuellement extraits, mais ils le sont par le biais de l'apprentissage profond. Un réseau neuronal profond peut être choisi pour discriminer les descripteurs les plus pertinents, ils peuvent ensuite être conservés dans le réseau afin d'être entraînés sur la tâche souhaitée ou bien être utilisés comme entrée dans un modèle différent (SVM ou arbre de Les résultats obtenus coupe à coupe peuvent alors être fusionnés pour obtenir une décision à l'échelle du volume. Ce dernier peut aussi directement être utilisé comme entrée, cependant le problème se pose sur les différences en termes de représentation des volumes (taille des voxels, épaisseur des coupes). Enfin, la classification peut se faire en considérant une série d'études pour le même patient [START_REF] Shen | Learning from Experts: Developing Transferable Deep Features for Patient-Level Lung Cancer Prediction[END_REF], mais là encore un problème de normalisation se pose, tant le nombre de coupes par patient peut varier. Une fois le type d'entrée choisi, les études peuvent varier en fonction de leur stratégie d'entraînement. Les réseaux peuvent être entraînés entièrement à partir des données à disposition, ou des architectures pré-existantes peuvent être ré-utilisées. Dans le premier cas, le réseau obtenu sera complètement ajusté à la tâche souhaitée, mais cette spécialisation peut être à l'origine de problèmes comme le sur-apprentissage ou la sensibilité à un déséquilibre de répartition des classes. L'impact de ces deux problématiques peut être limité à l'aide de l'augmentation de données (utilisation des données d'entraînement existantes pour en générer aléatoirement des nouvelles) [START_REF] Kumar | Discovery Radiomics for Pathologically-Proven Computed Tomography Lung Cancer Prediction[END_REF], l'entraînement multi-tâche (où le nombre de paramètres libres du réseau est limité suite à un entraînement simultané de la même architecture sur des tâches différentes) [START_REF] Jamaludin | SpineNet: Automatically pinpointing classification evidence in spinal MRIs[END_REF], ou encore l'adaptation de la fonction de coût à la proportion des données de chaque classe [START_REF] Jamaludin | SpineNet: Automatically pinpointing classification evidence in spinal MRIs[END_REF].

L'autre stratégie consiste à utiliser une architecture pré-entraînée sur des images naturelles la plupart du temps, puis d'en ré-entraîner uniquement une partie sur la tâche souhaitée [START_REF] Li | Deep Learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma[END_REF][START_REF] Azizi | Deep recurrent neural networks for prostate cancer detection: Analysis of temporal enhanced ultrasound[END_REF]. Enfin, les différents descripteurs sont extraits, là encore en suivant une approche supervisée ou non-supervisée.

Dans le premier cas, les architectures les plus utilisées sont les réseaux à base de couches de convolution l'information a priori de la forme du foie telles que les atlas probabilistes (Probabilistic Atlases) ou les

Radiomique pour la prédiction du grade

Nous avons ensuite effectuer la prédiction du grade histologique en comparant 3 stratégies:

• Une approche radiomique traditionnelle en effectuant l'extraction des descripteurs directement à partir de la ROI définie par l'expert

• Une approche dite "hybride" avec extraction de descripteurs "traditionnels" à partir de la ROI segmentée automatiquement à l'aide de nos réseaux de neurones

HCR

  Hand-Crafted Radiomics HCV Hepatitis C Virus iCCA Intrahepatic Cholangiocarcinoma IGA Image-guided Ablation kNN k-Nearest Neighbours MAP Maximum A Posteriori MRI Magnetic Resonance Imaging MVI MicroVascular Invasion NAFLD Non Alcoholic-Fatty Liver Disease NASH Nonalcoholic Steatohepatitis NECT Non-Enhanced Computed Tomography PA Probabilistic Atlas PCA Principal Components Analysis PDM Point Distribution Model PSC Primary Sclerosing Cholangitis PV Portal Venous RBF Radial Basis Function RECIST Response evaluation criteria in solid tumours RFA Radiofrequency Ablation ROI Region Of Interest RQS Radiomics Quality Score SSC Sparse Shape Composition SSM Statistical Shape Model SVM Support Vector Machine TACE Transarterial Chemoembolization Introduction Context This research work is part of the RADIAL (RADiomics Image Analysis of the Liver) project, carried out by the institute for minimally invasive surgery from Strasbourg (IHU), where one of the objectives is to improve the treatment and the prevention of liver cancer.

14 CHAPTER 1 .

 141 Figure 1.1: Incidence per cancer type in 2018, worldwide, both sexes, all ages, as described by the WHO [2]

Figure 1 . 7 :Figure 1 . 8 :

 1718 Figure 1.7: The net effect is that intranodular arterial supply diminishes initially and then increases; progressed HCCs typically show arterial hypervascularity compared with background liver ©Choi et al. [29]

Figure 1 . 9 :

 19 Figure 1.9: Histological grades as illustrated by ©Turdean et al. [66], G1: Well differentiated, G2: Moderately differentiated, G3: Poorly differentiated, G4: Undifferentiated

CHAPTER 1 .

 1 Figure 1.11: Distribution of the studies according to the grading reference, number of tiers, and data analysis, as detailed by Martins et al. [67] ©2017 Martins-Filho, Paiva, Azevedo and Alves

  classified separately as suggested by Martins et al. and depicted in the figure 1.14.

25 Figure 1 . 13 :

 25113 Figure 1.13: Examples of heterogeneous histological slices, as depicted by Pawlik et al. [70] ©2007 Lippincott Williams & Wilkins, Inc

Figure 1 . 15 :

 115 Figure 1.15: Concordance of pathological and clinical tumor grade estimation, as detailed by Pawlik et al. © 2007 Lippincott Williams & Wilkins, Inc. [70]

Figure 1 . 16 :

 116 Figure 1.16: Example of contrast-enhanced computed tomography images (from left to right: late arterial, portal venous and 3-minute delayed phase images) of patients suffering from infiltrative HCC with macrovascular invasion. Black arrows depict hyperenhancing tumor arteries, while asterisks depict areas of arterial hyperenhancement with delayed phase partial washout. White arrowheads depict enhancement of the right portal vein and its vein. ©Choi et al. [86]

Figure 1 . 17 :

 117 Figure 1.17: Example of multiphasic MR images of a patient suffering from HCC, Top-row: from left to right: Precontrast image, late hepatic arterial phase with hyperenhancement and portal venous phase image. Bottom-row: left: transitional phase image with apparent washout of contrast material from tumor, right: hepatobiliary phase image obtained 20 minutes after injection showing hypointense tumor relative to surrounding parenchyma. ©Choi et al. [86]

Figure 1 .

 1 Figure 1.18: BCLC staging classification, as illustrated by Llovet et al. ©Llovet et al. [97]

Figure 1 .

 1 Figure 1.19: TNM classification as described by the ©AJCC Cancer Staging [101]

Figure 2 . 3 :

 23 Figure 2.3: Number of patients included in the reviewed HCR studies

Figure 2 . 4 :

 24 Figure 2.4: Illustration of intratumor partition for two representative patients (TCGA-DD-A1EB and TCGA-BC-A69H). The first column shows the tumor outlines on the original CT image. The second column shows the heatmaps of calculated local entropy on tumor images. The third column shows the three subregions marked with different colors after intratumor partitioning. ©Xia et al.[START_REF] Xia | Radiogenomics of hepatocellular carcinoma: multiregion analysis-based identification of prognostic imaging biomarkers by integrating gene data-a preliminary study[END_REF]

Figure 2 . 5 :

 25 Figure 2.5: Screenshot of the CT texture analysis software. A polygonal ROI was drawn on the tumor (a). Processed images using Laplacian of Gaussian filters with SSF of 2 mm (b), 4 mm (c), and 6 mm (d) were automatically generated. The images were displayed using a red or blue scale showing negative or positive pixels, respectively. ©Akai et al.[START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF]

CHAPTER 2 .

 2 Figure 2.6: From top to bottom: Internal arteries, Hypodense halo, Textural heterogeneity, as illustrated by ©Segal et al.[START_REF] Segal | Decoding global gene expression programs in liver cancer by noninvasive imaging[END_REF]

Figure 2 . 8 :

 28 Figure 2.8: Average score percentage per criteria in the RQS for the reviewed studies

2. 3 .

 3 DEEP LEARNING RADIOMICS 57 the wanted target. Those limitations, and the rise of new computational resources associated with the emergence of deep learning techniques allowed the development of a new branch in the radiomics field, called DLR (Deep-Learning Radiomics) [121].

Figure 2 . 9 :

 29 Figure2.9: Illustration of the manually registered images and the effect of transformations (shift, rotation, skew) in the three phases, by ©Yamada et el.[START_REF] Yamada | Dynamic contrast-enhanced computed tomography diagnosis of primary liver cancers using transfer learning of pretrained convolutional neural networks: Is registration of multiphasic images necessary?[END_REF]

Figure 2 . 10 :

 210 Figure 2.10: Illustration given by ©Wang et al. of the different tested deep networks where raw images are combined with the clinical data[START_REF] Wang | Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data[END_REF]

Figure 2 . 12 :Figure 2 . 13 :

 212213 Figure 2.12: Architecture used by ©Yasaka et al.where the features are first extracted through convolutional layers before the prediction is performed using fully connected layers.[START_REF] Yasaka | Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT[END_REF]

Figure 2 . 14 :

 214 Figure 2.14: MCF-3DCNN architecture as detailed by ©Yang et al. [181]

66 CHAPTER 3 .Figure 3 . 1 :

 66331 Figure 3.1: Interobserver variability: For each metastasis, the whole lesion volume and the largest axial cross-section were segmented by two readers. a Purple line (reader 1) versus yellow line (reader 2) contouring. The largest two-dimensional (2D) region of interest (ROI) of the main lesion was confronted with two circular ROIs, one inside the metastasis and one outside it. b Purple line (reader 1) 2D versus yellow line (smallest circular ROI inclusive of the whole lesion) versus azure line (largest circular ROI completely inside the lesion). ©Rizzetto et al.[START_REF] Rizzetto | Impact of inter-reader contouring variability on textural radiomics of colorectal liver metastases[END_REF]

  detector algorithm, modified by the template liver position obtained via thresholding. A Gradient Vector Flow (GVF) field computation is used by the snake algorithm to obtain an accurate liver boundary detection. The final segmentation is obtained semi-automatically by asking the user to select the slices where the segmentation gave accurate results, and propagate this segmentation iteratively to adjacent slices. An example is depicted in the figure3.3.

Figure 3 .Figure 3 . 3 :

 333 Figure 3.2: Intensity based liver segmentation workflow used by ©Lim et al. [207]

Figure 3 . 4 :

 34 Figure 3.4: Estimation of the liver ROI based on the anatomical rib position as described by ©Foruzan et al. [213]

Figure 3 . 5 :

 35 Figure 3.5: Liver segmentation pipeline after the placement of a seed at the estimated liver centroid position as detailed by ©Kumar et al. [219]: (a) Original image, (b) simplified image, (c) eroded image, (d) cross indicates the centroid (seed point) on the eroded image, (e) region grown liver, (f) segmented liver after postprocessing, (g) AFCM clustered lesion, (h) segmented lesion, (i) segmented liver and lesion outlined in the original image

. 6 .

 6 Wang et al.[START_REF] Wang | Shape-intensity prior level set combining probabilistic atlas and probability map constrains for automatic liver segmentation from abdominal CT images[END_REF] combined probabilistic atlas (PA) and level set to perform the segmentation of the liver. Training volumes were pre-processed before the liver region was segmented, and stored with the denoised CT volumes. During the inference phase, the different training denoised CT volumes are used to compute the weighted PA for the current test patient. The liver ROI is then extracted using the weighted PA, and improved by estimating 5 different regions (heart, liver, right kidney, spleen and bone). Finally, a combined shape-intensity model is generated from the training samples, and used to refine the mask by level-set based segmentation.

Figure 3 . 6 :

 36 Figure 3.6: Skin surface localization (a-c) followed by the extraction of the ribs (d-f) as described by ©Pan et al. [223]

Figure 3 . 7 :

 37 Figure 3.7: Semi automatic segmentation pipeline as proposed by ©Wimmer et al. [229]: (a) Manual delineation of a liver cross-section; spline control points are depicted in white (b) Six cross-sectional contours are defined, two for each orthogonal view orientation. (c) Visualization of calculated 3-D normals (d) Rendering of the interpolated surface.

Figure 3 . 8 :

 38 Figure 3.8: The different steps of the segmentation process developed by ©Suzuki et al. [234]: (a) Original CT image. (b) Anisotropic diffusion noise reduction. (c) Scale-specific gradient magnitude calculation. (d) Nonlinear grayscale conversion. (e) Geodesic active contour segmentation.

Figure 3 . 9 :

 39 Figure 3.9: Comparison of random walker algorithm to graph cuts for a weak boundary with small seeds. Note that a 4-connected graph was used in these experiments. (a) Original (synthetic) image created with a diagonal black line with a section completely erased. (b) Graph cuts solution Since surface area of seeds is smaller than the weak boundary, the smallest cut minimally surrounds the seeds. (c) Random walker solution. (d) Probabilities associated with the random walker algorithm offer a notion of segmentation confidence at each pixel. ©Grady et al. [238]

Figure 3 . 10 :

 310 Figure 3.10: Segmentation Pipeline detailed by ©Dong et al. [240]: (a) The segmented liver (red) and spleen (green) of the previous slice; (b) The current slice; (c) Candidate pixel by applying a threshold to the GC; (d) The rough objects (red and green) and background (blue) seed points; (e) The fine seed points using the NBT method; (f) The segmentation results

Figure 3 . 11 :Figure 3 . 12 :

 311312 Figure 3.11: Example of control points distribution used in the MIAMI method as described by ©Park et al. [245]

  They were first described by Cootes et al.[START_REF] Cootes | Active shape models -their training and application[END_REF], who introduced the concept of Active Shape Models (ASM) which are built from a set of segmented training images and consist of two parts, a geometrical model, and a local appearance model. The geometrical model describes the shape and is represented by a PDM (Point Distribution Model), a dense collection of landmark points on the surface of the object. Given the location of the landmarks for each training case, a principal component model can be built in order to approximate all valid shapes. The local appearance model describing the boundary, is used as an additional one to detect the modeled shape in the new image. It is based on the local gray value appearance around the boundary. A PCA or a kNN model can be used to determine the possible profiles at each landmark. The model is then guided by internal forces that should keep the shape of the deformable model similar to the one of the underlying SSM, and external forces that drive the deformable surface towards the best fit to the data.To perform the segmentation of the liver, Montagnat and Delingette[START_REF] Montagnat | Volumetric medical images segmentation using shape constrained deformable models[END_REF] followed this scheme by deforming the model with global constraints (shape constraints and transformation constraints) to reduce the degrees of freedom, and registered the obtained model on the volumetric data including an additional external force based on local gradient computation. Lamecker et al.[START_REF] Lamecker | Segmentation of the liver using a 3D statistical shape model[END_REF] proposed a model that combined global constraints that minimized the geometric distortion for shape correspondence, and local constraints by considering the intensity profiles for the boundary search. The evolution of the boundary search for one example can be seen in the figure3.13.

Figure 3 . 13 :

 313 Figure 3.13: Evolution of boundary search in one example as described in ©Lamecker et al. [253]: At each of the six steps the model at its current position and shape is shown as a 3D surface. Below each surface one axial slice with the intersection line of the surface is displayed. Step 1: initial positioning of the mean model in the CT data such that the upper borders of the bounding boxes match. Step 2: result of the initial optimization of the position parameters. Step 3-5: results of the combined position and shape optimization with increasing number of shape modes (3, 9, 21). Step 6: final result (42 shape modes)

Figure 3 . 14 :

 314 Figure 3.14: State of the surface mesh in a coronal slice from back after each segmentation step. The top row shows a relatively easy case while the bottom row shows a relatively difficult case. (a) and (d) After localization by 3-D GHT. (b) and (e) After model subspace initialization. (c) and (f) Final result after optimal surface detection. ©Zhang et al. [256]

Figure 3 . 15 :

 315 Figure 3.15: Example of how the incorporation of the curvature improved the quality of the segmentation as reported by ©Erdt et al. [260]

Figure 3 . 16 :

 316 Figure 3.16: Hierarchical division of the liver shape for multi-level statistical shape model as described by ©Okada et al.[START_REF] Okada | Automated Segmentation of the Liver from 3D CT Images Using Probabilistic Atlas and Multilevel Statistical Shape Model[END_REF]

Figure 3 . 17 :

 317 Figure 3.17: Example of errors obtained during the segmentation process as exposed by ©Saddi et al. [255]

  .18, and then extracted through thresholding and region growing techniques. Those vessels are further classified into hepatic and non-hepatic ones by applying distance transforms. The boundary between the liver and its surrounding organs is then constructed equidistantly from the hepatic vessels and the non-hepatic ones. A post-processing step is finally applied to refine the obtained area, by filling holes and classifying boundary tumors as belonging or not to the liver based on their relation to hepatic blood vessels.

Figure 3 . 18 :

 318 Figure 3.18: Results of the blood vessels enhancement: (a) before enhancement (b) after enhancement, ©Maklad et al. [270]

  that is then used to construct a rough polygonal mesh representation of the liver. The mesh is finally refined via homotopy-based optimization using the SSC as reference. Huang et al.[START_REF] Huang | Automatic liver segmentation using multiple prior knowledge models and free-form deformation[END_REF] first localized the liver using a trained AdaBoost classifier where image features (intensity based and contextual information) were treated as input. The liver was then registered using the SSM incorporating all training shapes in combination with a kNN classification. A Free-form deformation is finally applied to refine the segmentation using the SSM to compute forces applied to the mesh. The entire pipeline is depicted in the figure3.19.

Figure 3 . 19 :

 319 Figure 3.19: The four steps of the liver segmentation framework: (a) liver model location; (b) registration with liver distance map; (c) shape fitting under appearance guidance; (d) free-form deformation. ©Huang et al. [279]

Figure 3 . 20 :

 320 Figure 3.20: Example of a FCN architecture implemented to perform the liver tumor segmentation task, as described by ©Bellver et al. [283]

Figure 3 .

 3 Figure 3.21: U-Net architecture as originally implemented and described by ©Ronneberger et al.[286]

Figure 3 . 22 :

 322 Figure 3.22: Residual connection as described by ©He et al. [292]

Figure 3 .

 3 Figure 3.23: A deep DenseNet with three dense blocks. The layers between two adjacent blocks are referred to as transition layers and change feature-map sizes via convolution and pooling. As described by ©Huang et al. [293]

Figure 3 . 24 :Figure 3 . 25 :Figure 3 . 26 :

 324325326 Figure 3.24: Cascaded liver and lesion segmentation architecture as initially implemented by ©Christ et al. [294]

Figure 3 . 27 :

 327 Figure 3.27: Patch-wise architecture as detailed and described by ©Li et al. [295]

  Li et al. who computed 3D inter-slice features in 12-slices blocks all along the different CT scans [291]. Rafiei et al. [297] decided to incorporate 3D layers only in the encoding part of their 3D-2D-FCN model. The connection between the encoding part and the decoding part was done by custom skip connections to concatenate the middle slice of the 3D volume with its corresponding 2D features map in the decoding part. Dou et al. were the first to build an entire 3D network with large kernels, and had intermediate supervised layers to combat the gradient vanishing problem [298]. Details of their network are given in the figure 3.28.

Figure 3 . 28 :

 328 Figure 3.28: Full 3D architecture as detailed and implemented by ©Dou et al. [298]

  object based random forest classifier, Bellver et al. proposed a patch-based lesion detector as depicted in the figure 3.29 [283], whereas Frid-Adar et al. integrated an additional CNN designed to detect FP [296].

Figure 3 . 29 :

 329 Figure 3.29: Results of the lesion detection network. Blue and red lines indicate the liver and lesion ground truth, respectively.Yellow and green lines are the segmentation results for liver and lesion. The light blue bounding boxes are the windows detected as having a lesion. All positive pixels at the output of the segmentation network will be removed if they disagree with the results of lesion detection, as reported by ©Bellver et al.[START_REF] Bellver | Detection-aided liver lesion segmentation using deep learning[END_REF]

98 CHAPTER 4 .

 984 PROPOSED METHOD FOR THE SEMANTIC SEGMENTATION OF HCC

Figure 4 . 1 :

 41 Figure 4.1: Cascaded network: The first network takes as input a CT image and segments the liver. The resulting segmentation map is used to remove non-liver pixels in the input data of the second network which performs the segmentation of lesions. The last network segments the necrosis within the lesions. The three binary masks are combined in the final segmentation map.

4. 4 .

 4 U-NET NETWORK 99 the extraction of high to low level features, and a decoding part where the compressed information is used to reconstruct a high resolution segmentation. The architecture was initially composed of 19 convolutional layers, with a rectified linear unit function as activation. The input image had an initial size of 512 × 512 pixels, and at each stage of the encoding part, 2 convolutional layers are stacked with an increased number of filters. The initial pair of convolution layers used 64 filters each, whereas the final stage of the encoding stage used 1024 filters. After each pair of convolutional layers, a max pooling layer is applied to halve the spatial dimension of the features maps. As a result, a 30 × 30 × 1024 features map is produced at the bottleneck of the network. This representation is then reformatted in the decoding part to obtain a segmentation map with a size equivalent to the one of the input image. In order to increase the spatial dimension of the features maps, Up-Convolutional layers are implemented. It is worth noting that the number of filters used in each of the pair of convolutional layers is decreased from the bottleneck to the final layer of the network. The last convolutional layers will map the obtained features to the final number of dimensions of the segmentation maps, which will correspond to the number of classes to predict. The final layer implements a softmax function, to simulate a prediction of appartenance to each one of the output class.

Figure 4 . 2 :Figure 4 . 3 :

 4243 Figure 4.2: PV-Lesion network used to segment lesions within the liver with a PV image as input

4. 6 . DATASETS 101 Figure 4 . 4 :

 610144 Figure 4.4: DMP-Full network that combines the AR and the PV images as an input to segment the parenchyma and both the active and the necrotic parts of the lesions. Here, the two channels are considered as features for the first layer

Figure 4 . 5 :

 45 Figure 4.5: MPF-Full network: initially, AR and PV images are processed separately. The resulting maps are merged (by simple addition) at the end

  deep network. In order to reduce the effect of extreme values from regions present in the tomographic images (such as the bones or the air), and to enhance the intensity of the liver voxels, we first clipped the HU values to be in the range [-100, 400] , corresponding to the most commonly observed liver intensities range. The retained intensities were finally mapped to the interval [0, 1] 4 .

•--

  Number of filters at bottleneck: 1024 Input image size: 512 • Data augmentation -Rotations in the interval [0, 40] Translations with shift in the interval [-0.1, 0.1] Horizontal and vertical flips • Augmentation factor: 20 When transferring those settings to the TheraHCC-dB, we implemented both translation and the addition of gaussian noise in the training since it slightly improved the performances. In order to remove any bias, the same set of hyperparameters has been used for both the {.}-Liver, {.}-Lesion, {.}-Necrosis and {.}-Full networks when training the TheraHCC-dB, regardless of the type of input (single phase or multiphase).The original U-Net architecture implemented by Ronneberger et al. had a total of 32M parameters, and the results obtained when setting the hyperparameters tend also to prove that the best results were obtained using an architecture with the same number of parameters (with 1024 filters at bottleneck as mentioned above). When performing the segmentation of both the necrotic and the active parts of the tumor from a masked liver image, our cascaded version consists of 2 concatenated U-Net networks (one to segment the tumor and the second to differentiate between the active and the necrotic part) therefore, we decided to reduce the number of parameters to 8M parameters for each network of the cascaded (512 filters used at bottleneck instead of 1024) in comparison with the versatile networks (which have a total of 32M parameters) to ensure that no bias existed in favor of the cascaded networks. Only the MPF-{.} had a total of 16M parameters since it already consists of 2 combined U-Net networks as illustrated in the figure4.5. To cope with the high imbalanced data present in our datasets (e.g. number of voxels belonging to the tumor class vs those belonging to the parenchyma class when segmenting the lesion in the liver), we considered the weighted cross-entropy function as loss function when training our network.

Figure 4 . 6 :

 46 Figure 4.6: Two examples of obtained learning curves for the tumor segmentation task for different folds in the cross validation process. The first example depicts a standard learning curve where the training loss decreases gradually, followed by the validation loss, while both the training and the validation accuracy increase gradually. A higher variance can be seen in the second case, probably because of the small number of training images regarding the size of the network.

figure 4 .8. 108 CHAPTER 4 .Figure 4 . 7 :

 4108447 figure 4.8.

Figure 4 . 8 :

 48 Figure 4.8: Examples of liver mis-segmentations where some parts of the actual liver are missed by the network. Left: raw images, middle: expert ground truth, right: prediction. First row: an example where a portion of the liver parenchyma is missing in the predicted liver segmentation mask, second row: an example where a portion of the liver tumor necrosis is not included in the predicted liver segmentation.

Figure 4 . 9 :

 49 Figure 4.9: Examples of tumor mis-segmentations. Left: raw images, middle: expert ground truth, right: prediction. First row: example of prediction where a portion of the active part of the tumor is mis-considered as belonging to the liver parenchyma, second row: example where some air captured by the liver envelope has been annotated as belonging to the liver parenchyma by the expert, but it was considered as tumoral tissues by the segmentation network.

Figure 4 . 10 :

 410 Figure 4.10: Examples of necrosis mis-segmentations. Left: raw images, middle: expert ground truth, right: prediction. First row: example where the distinction between the necrotic and the active part of the tumor is not strongly marked, which leads to errors in the segmentation with a larger proportion of the tumor considered as active. Second row: example where the expert extended the annotated necrotic area to its simple central region, whereas the network only considered the central part of the tumor as being necrotic.

Figure 4 . 11 :

 411 Figure 4.11: From top to bottom : Raw images with HU values inside the liver, Ground truth, DMP-Full segmentation, Cascaded DMP segmentation

116 CHAPTER 5 .Figure 5 . 1 :

 116551 Figure 5.1: Description of the entire workflow to build our DLR-based histological grade prediction. LITS-dB and G-dB are used to train respectively the liver (green arrows) and the liver tumor (red arrows) semantic segmentation network, whereas TCIA-dB is used to train the histological grade prediction using semantic imaging features extracted from the liver tumor segmentation network (pink arrow).

Figure 5 .

 5 Figure 5.2: Example of small and large tumors from both the training and the testing datasets. Left: LITS-dB images, right: TCIA-dB images, top: small tumors, bottom: large tumors. Tumors are circled by a green ellipse.

Figure 5 . 3 :

 53 Figure 5.3: Example of patient from the LITS-dB with a high number of tumors susceptible of being metastases. Right : raw image, left: expert segmentation where red represents liver parenchyma and green represents tumors.

Figure 5 . 4 :

 54 Figure 5.4: Example of artifacts present in both training and test datasets. Left: LITS-dB images, right: TCIA-dB images. First row presents patients with benign hepatic lesions (yellow arrows), the second row presents liver with tracks of fat accumulation (orange arrows), whereas the last row gives examples of images with presence of metallic artifacts (red arrows).

Figure 5 . 5 :

 55 Figure 5.5: Example of cirrhotic patients present in both datasets. Left: LITS-dB images, right: TCIA-dB images. We can see the irregular shape (cyan arrows) of the liver with presence of hypodense masses (represented by pink arrows) in both cases.

Figure 5 . 6 :

 56 Figure 5.6: Example of ROI placement for the quantitative evaluation of differences between the LITS-dB and the TCIA-dB. Red: liver parenchyma, green: air, blue: spleen, yellow: bone, cyan: aorta.

Figure 5 . 7 :

 57 Figure 5.7: Histogram representing the distribution of the difference between parenchyma and spleen intensities, where a difference higher than 18.5 HU (black vertical line) can be a sign of cirrhosis (Bins=5). Red and blue curves represent the associated kernel density estimation for the difference distribution.

5.2. MATERIAL 121 Figure 5 . 8 :

 12158 Figure 5.8: Mean aorta intensity vs mean liver parenchyma intensity. Each point represents one volume.We can see a clear separation between arterial (green dots) and portal venous (red crosses) volumes among the TCIA-dB patients. Portal venous volumes present a higher variance for their liver parenchyma intensities, which can be explained by the differences in terms of duration between the injection of contrast medium and the acquisition of the images. It seems clear that the majority of volumes from the LITS-dB were acquired during the portal venous phase whereas some of them were acquired during the arterial phase. The rest of the volumes can be classified as belonging to the late arterial or the early portal venous phase.

Figure 5 . 9 :

 59 Figure 5.9: Example of small and large tumors from the G-dB images, top: small tumors, bottom: large tumors, left: raw image, left: expert segmentation.

Figure 5 . 10 :

 510 Figure 5.10: Example of artifacts present in the G-dB dataset. First row depicts a patient with benign hepatic lesion (yellow arrow), the second row presents liver with tracks of fat accumulation (orange arrow), whereas the last row gives examples of images with presence of metallic artifacts (red arrow).

5.2. MATERIAL 123 Figure 5 . 11 :

 123511 Figure 5.11: Example of cirrhotic patients present in the LITS-dB dataset. In both images, we can see the irregular shape (cyan arrows) of the liver.

Figure 5 . 12 : 125 Figure 5 . 13 :

 512125513 Figure 5.12: Examples of tumor characteristic imaging traits relative to the tumor border present in both training and test datasets. Left: G-dB images, right: TCIA-dB images.First row: peritumoral enhancement, which can be defined by the presence of a detectable region of enhancement adjacent to the tumor border in the arterial phase. Second and third rows: illustration of the tumor margin, that can be categorized as a smooth margin (second row) when facing nodular tumors with smooth contours or as non-smooth margin (third row) when facing non-nodular tumors with an irregular margin with budding portions at the periphery in both AR and PV images. Bottom row: hypoattenuating halo, defined as a rim of hypoattenuation partially or completely circumscribing the tumor on peritumoral PV images.

Figure 5 . 14 :

 514 Figure 5.14: Example of specific wash-in/wash-out traits present in both training and test datasets.Left: G-dB images, right: TCIA-dB images. Wash-in is defined as an enhancement of the lesion of the arterial phase higher than the one of the liver parenchyma, whereas the wash-out is defined as a lesion hypodense/hypointense compared to the liver parenchyma on the portal venous and/or delayed phases.

Figure 5 . 15 :

 515 Figure 5.15: Histogram representing the distribution of the difference between parenchyma and spleen intensities for the patients of G-dB dataset (green) and the TCIA-dB dataset (blue). A difference higher than 18.5 HU (black vertical line) can be a sign of cirrhosis.

Figure 5 . 16 :

 516 Figure 5.16: Mean aorta intensity vs mean liver parenchyma intensity. Each point represents one volume.We can see a clear separation between arterial (red) and portal venous (green) volumes for both the G-dB and the TCIA-dB patients. We can however notice a higher heterogeneity among the mean parenchyma intensity in the TCIA-dB dataset, which could be a sign of standardized acquisition settings within the G-dB.

Figure 5 . 17 :

 517 Figure 5.17: We trained our network using each one of the 131 volumes of the LITS-dB, and tested the obtained architecture on both AR and PV TCIA-dB volumes.

5. 3 . METHODS 129 Figure 5 . 18 :

 3129518 Figure 5.18: Illustration of the registration pipeline applied to the images of TCIA-dB. The first green arrows correspond to the liver segmentation using a network trained on the LITS-dB. Dashed blue arrows correspond to the ANTs registration pipeline, where a dilated version of the predicted liver annotations maps are used as registration masks. The ANTs algorithm implements 3 transformations: a rigid, an affine, and a diffeomorphic Syn transformation that computes a deformation grid [302]. The 3 steps of the ANTs algorithm allows us to obtain registered multiphase images.

Figure 5 . 19 :

 519 Figure 5.19: Left: Portal venous image, middle: registered AR image, right: perfusion image where each pixel's intensity is obtained by computing the difference between its PV and its AR intensity.

  training and the test sets contains both the same number of patients per class (7 patients from each class in the training set and 1 patient from each class in the testing set).

Figure 5 . 20 :

 520 Figure 5.20: Red and blue areas correspond to the bottleneck part of the U-Net network where the features extraction is performed. Each image is then represented as a 32 × 32 × 512 features cube.

Figure 5 . 21 :

 521 Figure 5.21: Slice-wise histological grade prediction using both AR and PV retained semantic imaging features

  23 and 5.24). As explained earlier, an other problem often occurring is the mis-segmentation of other abdominal organs, that are sometimes considered as being part of the liver. However, our post-processing steps usually correct these mistakes by removing the undesired tissues from the final segmentation map (as illustrated in the figure5.25).

Figure 5 . 22 :

 522 Figure 5.22: Example of liver segmentation using the CECT-Liver network on TCIA-dB patients (Top row AR images, bottom row: PV images, left: Raw images, right : liver segmentation as overlay).

Figure 5 . 23 :

 523 Figure 5.23: Examples of liver mis-segmentation cases with left being the raw images and right being the liver segmentation prediction obtained by our CECT-Liver network (green area corresponds to the retained liver segmentation, whereas the purple area corresponds to voxels removed by the postprocessing steps). Top row depicts an axial top liver slice, where nearly half of the liver has been missed.Second row depicts a bottom liver slice where a big portion of the liver occupied by a tumor has been missed by the liver segmentation. Bottom row also depicts a bottom liver slices where a small portion is missed by the network, but the hepatic lesion is missed as well (green arrow).

Figure 5 . 25 :

 525 Figure 5.25: Examples of liver mis-segmentation cases with left being the raw images and right being the liver segmentation prediction obtained by our CECT-Liver network (green area corresponds to the retained liver segmentation, whereas the purple area corresponds to voxels removed by the postprocessing steps). In the given examples we have the stomach (top row), the spleen (second row) and the colon (bottom row) that have been mis-interpreted by the network as being part of the liver, however in each case the post-processing steps allowed us to remove the unwanted tissue from the remaining segmentation map.

Figure 5 . 26 :

 526 Figure 5.26: Example of an image from the TCIA-dB, with the obtained predicted tumor segmentation using the MPF-Tumor segmentation network (left: raw, middle: expert annotation, right: obtained segmentation)

Figure 5 . 27 :Figure 5 . 28 :

 527528 Figure 5.27: Example of slice-wise histological grade prediction of one patient from the TCIA-dB.Here we observed that the confidence was correlated with the relative position to the center. Even though the semantic segmentation of the tumor was correctly performed, distant slices tend to be either misclassified or classified with a lower confidence than central slices. Details of tumor semantic segmentation are given for 3 slices (raw image where pixels outside the predicted liver are masked in the left side and where the tumor GT delineation is given in red, and tumor segmentation heatmap in the right side)

Figure 5 . 29 :

 529 Figure 5.29: Tumor candidates marked with a dashed white line were classified as false positives, as described by Chelbus et al. [288].

147 One

 147 solution could be to implement an histogram specification algorithm to "normalize" images from a given phase. The first step will consist in the computation of a mean histogram per phase (which requires to have a sufficient amount of CT images correctly labeled), before transforming the volumes of a given patient to match the obtained mean histogram, as depicted in the figure 5.30.

Figure 5 . 30 :

 530 Figure 5.30: Histogram specification experiment conducted to normalize images from the same phase, top row (from left to right): slice before histogram specification, mapping function for the chosen range [-100, 400]; bottom row (from left to right): representation of the mean phase histogram, the current slice histogram and the transformed one after application of histogram specification; slice after histogram specification.

Li et al. ( 2017 )

 2017 Li et al. (2017)combined both 2D and 3D information to get the final classification. They first trained a Residual Network to get a coarse liver segmentation. Then, a 2.5D DenseUNet (densely connected path with UNet-like connections) architecture was developed to get a 64-channels features map corresponding to the 2D features. A pixel-wise classification step was added to get a pixel-wise 2D classification. The binary maps were stacked to a 12-slices 3D block, and combined as input with the original 3D volume to obtain a 64-channels features map encoding the 3D inter-slice information.

  Bellver et al. (2017).The main architecture they used was the DRIU (deep retinal image understanding), consisting of VGG16 backbone with last Dense Layers replaced by a convolutional layer. The network was pre-trained on the ImageNet dataset, before fine-tuned with CT images having intensity clipped to the range[-150, 250] and further normalized. To train the network, the weighted binary cross entropy (BCE)

  Kaluva et al. (2018)Kaluva et al. (2018)proposed to build their model with the DenseNet architecture as a basis. The DenseNet is constructed such that a given layer l is connected to each of its predecessors l-1, l-2, . . . 0 in a concatenated manner. The number of feature maps at a given stage is known as growth rate. The number of feature maps in the classical DenseNet grow linearly with the depth. After each dense block, a Transition Down (TD) layer is introduced to reduce the spatial dimensionality of the feature maps.Their cascaded model consisted of 2 steps, one specified on the liver segmentation, and the second one on the lesions segmentation. To train the liver segmentation network, the original CT images were downsampled to 256x256 pixels before normalizing HU intensities clamped to the range[-100, 300]. No downsampling was performed when training the lesions segmentation network, and comparing to the first network, 3 differents HU windows ([0, 100],[-100, 200] and [-100, 400]) were applied on the original image and combined to get a 3-channels input. To deal with class imbalancing, the first network was trained only on liver slices (with 10 slices above and below it), whereas the second one was trained only on lesion slices (plus 5 slices above and below). They also decided to connect the input and the output of each Dense block in the downsampling part, in a ResNet-like fashion.

Jin et al. ( 2018 )

 2018 Jin et al. (2018)proposed a network that integrated both U-Net and attention residual mechanism to proceed the segmentation of both the liver and the lesions.

B. 3 .

 3 DATA AUGMENTATION 165

2

  Here, c is an optional value which shifts the intensities to prevent negative values in X. This ensures that voxels with the lowest gray values contribute the least to RMS, instead of voxels with gray level intensity closest to 0.Standard Deviation measures the amount of variation or dispersion from the Mean Value. By definition, standard deviation = √ variance.

  j -µ x -µ y ) 4 p (i, j) j -µ x -µ y ) 3 p (i, j)

p 1 :M= j 21 .

 121 (i, j) -µ x µ y σ x (i) σ y (j) y (k) log 2 (p x-y (k) + ǫ) (i, j) log 2 (p (i, j) + ǫ)13. Informal Measure of Correlation (IMC) IM C1 = HXY -HXY 1 max {HX, HY } 14. Informal Measure of Correlation (IMC) 2:IM C2 = 1 -e -2(HXY 2-HXY ) CC = second largest eigen value of Q Q (i, j) = Ng k=0 p (i, k) p (j, k) p x (i) p y (k)17. Inverse Difference Moment Normalized (IDMN): Maximum Probability:maximum probability = max (p (i, j)) i, j ∨ θ) 2 N z (θ) 2RLNN measures the similarity of run lengths throughout the image, with a lower value indicating more homogeneity among run lengths in the image. This is the normalized version of the RLN formula.

ppp

  (i, j ∨ θ) (i -µ)2 180 APPENDIX C. RADIOMICS FEATURES GLN measures the variability of gray-level intensity values in the image, with a lower value indicating more homogeneity in intensity values. variability of gray-level intensity values in the image, with a lower value indicating a greater similarity in intensity values. This is the normalized version of the GLN formula. (i, j) (j -µ) 2 Here,µ = Ng i=1 Ns j=1 p (i, j) j (i, j) log 2 [p (i, j) + ǫ]

pp

  (i, j) (j -µ) 2 , where µ = (i, j) log 2 [p (i, j) + ǫ]

45,

  , where p i = 0, p j = 03. Busyness:C.6. NEIGHBOURING GRAY TONE DIFFERENCE MATRIX (NGTDM) FEATURES i -jp j | , where p i = 0, p j = 0 p i s i + p j s j p i + p j , where p i = 0, p j = 0 i + p j ) (i -j) where p i = 0, p j = 0 190 APPENDIX E. RÉSUMÉ LONG (FRANÇAIS) structurales. Actuellement, l'analyse visuelle est un processus très coûteux en temps humain et l'exactitude du diagnostic repose sur la formation et l'expérience personnelle de l'anatomo-pathologiste. Certains standards sont définis internationalement, tels que la classification des tumeurs. Cela permet d'assurer une certaine cohérence entre les observateurs. Cependant, bien que la formation des anatomopathologistes et le contrôle de qualité aient abouti à une concordance suffisante dans de nombreux domaines de l'anatomo-pathologie, il existe encore d'importantes variations dans l'interprétation des phénomènes biologiques observés et la précision varie considérablement pour certains systèmes de notation ou biomarqueurs. Concernant les HCCs, l'analyse anatomo-pathologique donne lieu à une évaluation des tissus prélevés où différentes caractéristiques sont décrites, notamment l'état des cellules hépatiques via un score appelé "grade histologique". Deux standards existent pour cela, la classification d'Edmonson-Steiner de 1954 et la classification de l'OMS datant de 2010, dont les caractéristiques des différents groupes sont détaillés sur la figure E.1.

Figure E. 1 : 191 Figure E. 2 :

 11912 Figure E.1: Différences entre le système de classification Edmonson Steiner et le système mis en place par l'OMS.

Figure E. 3 :

 3 Figure E.3: Workflow d'un projet radiomique : 1) Acquisition d'image, 2) Segmentation par un expert, 3) Extraction automatique des descripteurs, 4) Comparaison à la base d'apprentissage

E. 2 .

 2 HISTORIQUE ANALYSE D'IMAGESMÉDICALES 195 décision par exemple). Comparé à la radiomique classique, aucune connaissance a priori n'est nécessaire, et les descripteurs peuvent être extraits de manière automatique. Les réseaux profonds peuvent être entraînés de bout en bout, en utilisant uniquement les images originales en entrées, sans nécessairement devoir fournir de segmentation. Il a également été démontré que les performances de ces réseaux allaient en s'améliorant avec un nombre grandissant de données d'entraînement[START_REF] Cheng | Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans[END_REF]. L'élimination de l'étape de segmentation, lors de la phase d'évaluation du diagnostic, permet de réduire la charge de travail des experts et répond au problème de la dépendance à l'observateur. Les segmentations automatiques n'étaient à ce jour, pas encore suffisamment précises pour pouvoir être utilisées lors d'applications dites "sensibles". Lors de l'entraînement, les annotations des différents zones d'intérêt peuvent néanmoins être combinées à l'image originale (de même que d'autres représentations, comme le gradient par exemple[START_REF] Sun | Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis[END_REF]) afin d'améliorer la qualité des descripteurs extraits. D'une manière générale, les études radiomiques faisant appel à l'apprentissage profond peuvent être différenciées selon plusieurs aspects que sont le type d'entrée utilisée, la stratégie d'entraînement et enfin le choix de l'architecture privilégiée pour extraire les descripteurs. En entrée des réseaux profonds, on peut considérer les coupes 2D indépendamment les unes des autres, cependant cette technique n'apporte pas suffisamment d'information tant la décision dépend grandement de l'intégralité du volume d'intérêt.

  Figure E.5: À gauche l'image originale, au milieu la vérité terrain et à droite la prédiction obtenue après utilisation du réseau en cascade

Score Author Modality & slice thickness Mean tumor size Treatment #Patients & Inclusion Criteria Segmentation Computed features Retained features Retained features category Study endpoints Results %RQS 1 (total points)

																	1
																	(total
																	points)
	Cozzi et Raman et	NECT Contrast	-Adenoma	Radiotherapy -	138 Patients 80 patients	Segmentation done us-ROIs were selected from	35 32	extracted features	Compacity SD and Mean of	Quantitative Quantitative	OS & local Diagnosis	AUC of the model is First-order statisti-	(5) (1)
	al. [136] al. [141]	3mm CT (25s)	7 ± 3cm	(volumetric	BCLC stages 17 FNH	ing the CTV (clinical multiple axial slices	features (mean, SD, en-	Energy histogram			control	of	0.80 cal features able to
			3mm		FNH	modulated	from A to C, 19 Adenomas	target volume) which (from 5 to 10 slices)	6 geometry and tropy, skewness,	GLNU			the tumor	Survival could be differentiate 3 types
					6 ± 3cm	arc therapy)	Child-Pugh 25 HCCs	is manually contoured 2 experts	histogram kurtosis)				predicted with a ra-of hypervascular le-
					HCC		stages A-B 19 normal liv-	(whole tumor analysis)	29 GLM				diomics signature sions with a 15% of
					8 ± 3cm		ers										error rate)
	Zhou et	Contrast	-	Hepatectomy	215 Patients	Largest cross-sectional	300	features	Histogram fea-	Quantitative	Recurrence	First-order	sta-	(9)
	al. [137] Kuo et al.	CT Contrast	-	-	who 30 Patients under-	area of the tumor, man-No segmentation	(Mean, 6 imaging traits SD,	tures (skewness, Tumor margin	Semantic	MVI status	tistical The tumor margin features	(7)
	[142]		(30 CT	and (30-			went partial no patients	ual delineation Images analyzed visually	Kurtosis, Skew-(internal ar-	energy, means, Internal arteries		combined showed strong cor-with
			60s) 35	and			hepatectomy received	Exclusion of necrosis by 2 experts.	ness, GLM) teries, textural	. . . )			clinical relation with MVI, factors
			1.25mm 60-70s)			chemo before	2 experts		heterogeneity,				can predict early TNM.
			2.5mm			resection					wash-in-wash-				recurrence Internal	arteries
												out,	necrosis,				showed correlation
	Akai	et	Contrast	3.7cm	Hepatectomy	127 patients	Manually setting the	96 tumor margin) features	Entropy, skew-	Quantitative	OS & DFS	First-order with MVI	statis-	(9)
	al. [138]	CT		(2.4-				ROI to include the	(mean,	sd,	ness and kurto-		tical features were
	Banerjee	(27-28, 40 Contrast	7.0cm) 2.8cm	Hepatectomy	157 patients	tumor within the slice Only imaging features	positive 3 imaging traits calue	sis The 3 imaging	Semantic	OS and RFS	sufficient to pre-Combination of the	(19)
	et	al.	and 90s) CT (30-	(1.8-	or LT 1	72 resection	at its max diameter. were evaluated by 5 ra-	pixels, (internal arter-en-	traits were re-		dict three different imag-postoperative
	[143]		5mm 35, 60-70,	4.5cm)		85 LT 2		Single radiologist diologists	tropy, kurtosis, ies, hypodense	tained			survival ing traits was corre-
			180-300s)			MVI	diag-				skewness) halo, tumor-				lated with MVI
			2.5-3mm			nosed in 45				liver difference)
	Chen et	Contrast	-	Hepatectomy	61 patients patients	ROI	was	delineated	84 features	Textural	fea-	Quantitative	OS & DFS	Tumor	prognosis	(6)
	al. [139]	CT	(25			with	only	around the tumor out-	12 Gabor	tures,	Gabor		could be predicted
	Renzulli	and 70s) Contrast	3.3cm	Hepatectomy	one 125 patients lesion	line at the longest Only imaging features	9 Wavelet 5 imaging traits	and Wavelet as All except the	Semantic	MVI status	using Gabor and The 4 retained traits	(3)
	et	al.	1.25mm CT (25-	(1.8-		and survival where hepatic	dimension were evaluated by 2 ra-	7 GLCM Dimensions	key features lesion dimen-		Wavelet responses were correlated with
	[144]		30, 45-60,	5.2cm)		above resection was 3	2 experts diologists		Lesions number	sions			the presence of MVI
			180-300s)			months indicated				Non-smooth				in HCC
			2.5mm								margins
	Li et al.	Contrast	8.0cm	Hepatectomy	130 patients	Irregular ROI manu-	27 TTPVI 3 features	2 Wavelet fea-	Quantitative	OS	and	Wavelet	features	(7)
	[140]		CT (70s)	(5.1-	or TACE	86 treated by	ally drawn around the	(Wavelet) Peritumoral	tures correlated		Treatment	correlated	with
			1.25mm	18.7cm)		LR and 22 by	largest-cross	sectional	enhancement	with survival		sensitivity	survival suggesting
							TACE		tumor outline							a suitable treatment
									2 radiologists							choice

1 RQS: Radiomics Quality

Author Modality & slice thickness Mean tumor size Treatment #Patients & Inclusion Criteria Segmentation Computed features Retained features Retained features category Study endpoints Results %RQS 1 (total points)

	Segal et	Contrast	-	Hepatectomy	Train: 30		No segmentation, visual	32	imaging	Internal arteries	Semantic	OS & MVI	Internal	arteries	42 (15)
	al. [145]	CT (three-			Test: 32		examination	traits (Capsule,	and hypodense	combined with hy-
			phasic)						2 radiologists.	Wash-in-Wash-	halo			podense halo can
											out,	Tumor-				predict OS, MVI
											Liver difference,				Both are correlated
											. . . )						with	HCC	gene
																	expression
	Zheng et	Contrast	-	Hepatectomy	Train: 212		ROI delineated around	110 GLM fea-	6 GLCM fea-	Quantitative	Recurrence	Textural	features	47 (17)
	al. [146]	CT	(22			Test: 107		the tumor outline of	tures			tures			& OS	sufficient to pre-
			and 60s)			patients		the largest cross-sectinal							dict	postoperative
			5mm				without		area.							recurrence	and
							anticancer		2 radiologists							survival
							therapy
	Peng et	Contrast	4.9-	-	Train: 184		ROI semi-automatically	5 imaging traits	Nonsmooth		Semantic &	MVI status	Radiological	fea-	47 (17)
	al. [147]	CT	(30,	6.4cm		Test: 120		segmented in the largest	(tumor margin,	tumor margins,	Quantitative	tures and a ra-
			60	and			Partial hep-	cross-sectional area	peritumoral		hypoattenu-		diomics	signature
			120s)				atectomy		2 experts	enhancement,		ating	halos	computed with first-
			5mm				with patho-		hypoattenu-		and	internal	order	statistical
							logically			ating	halo,	arteries + 8	features	showed
							confirmed			internal arter-	radiomics	fea-	correlation	with
							HCC				ies, tumor-liver	tures (Entropy,	MVI
											difference)	&	shape, GLRLM,
											980	quantita-	GLCM)
											tive features
	Bakr	et	Contrast	7.4cm	-	28	patients	3 ROIs were placed on	464	features	Textural		fea-	Quantitative	MVI status	Textural	features
	al. [127]	CT	(AR			with surgical	different cross sections of	(intensity, tex-	tures			computed	using
			with bolus			resection	of	the tumors (one central,	ture, shape)					single-or combined-
			tracking,			a previously	one superior and one in-							phased images were
			PV, delay)			untreated		ferior)							correlated with MVI
			Thickness			HCC			4 radiologists
			≤ 3mm

Author Modality & slice thickness Mean tumor size Treatment #Patients & Inclusion Criteria Segmentation Computed features Retained features Retained features category Study endpoints Results %RQS 1 (total points)

	Taouli et	Contrast	5.7±3.2cm -	38 patients	Global inspection of	11	imaging	Infiltrative pat-	Semantic &	Signature	Correlation	was	19 (7)
	al. [148]	CT	(AR			26 CT/ 12	the imaging traits and	traits	(wash-	tern,	mosaic	Quantitative	of MVI &	found	between
		with bolus			MRI	"slice-wise" evaluation	in-washout,	appearance,			aggressive	some imaging traits
		tracking,			36 Liver re-	for the enhancement	hypovascular-	presence		of		phenotype	and the aggressive
		PV at 70s,			section	ratio and the wash-out	ity)		MVI, large size			profile of the tumors
		Delay at			2 LT 4	ratio	3	Quantita-
		180s)					2 radiologists	tive	features
								(contrast ratios)
	Xia et al.	Contrast	12 tu-	Hepatectomy	38 patients	Tumor was firstly delin-	37 features (1st	Volume		of	Quantitative	OS	The	volume	of	22 (8)
	[149]	CT	(30,	mors	or LT		eated then divided into	order, geometry,	transition	re-			transition between
		55	70,	smaller			3 spatially distinct sub-	textural) And 4	gion & cluster			tumor and liver, and
		300s)		than			regions (using a multi-	features for the	prominence				the	heterogeneity
		2.5-5mm	5cm, 26			parametric clustering)	whole tumor						of the lesion were
				larger			1 radiologist								correlated	with
															survival.

Table 2

 2

	.1: HCR reviewed studies details

  decided to separate areas based on their textural properties as depicted in the figure 2.4, but the analysis remains global. Other studies placed several ROIs all throughout the tumor (3 ROIs for Bakr et al. and from 5 to 10 for Raman et al.). However, CHAPTER 2. RADIOMICS: STATE-OF-THE-ARTthe majority of the studies decided to place the ROI at the largest cross-sectional area[START_REF] Li | Computed tomography texture analysis to facilitate therapeutic decision making in hepatocellular carcinoma[END_REF][START_REF] Chen | Texture analysis of baseline multiphasic hepatic computed tomography images for the prognosis of single hepatocellular carcinoma after hepatectomy: A retrospective pilot study[END_REF][START_REF] Zhou | CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma[END_REF][START_REF] Akai | Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest[END_REF][START_REF] Zheng | Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients[END_REF].

Table 2 .

 2 3: List of semantic features used in the reviewed studies

Table 2 .

 2 4: RQS score details per criteria for the reviewed studies

	Criteria	Bakr	Kuo et	Taouli	Xia et	Chen	Segal	Raman	Renzulli
		et	al.[142]	et	al.[149]	et	et	et	et
		al.[127]		al.[148]		al.[139]	al.[145]	al.[141]	al.[

144] Akai et al.[138] Li et al.[140] Cozzi et al.[136] Banerjee et al.[143] Zhou et al.[137] Peng et al.[147] Zheng et al.[146]

Table 3 .

 3 Up to today, and compared to other organs (brain, breast, lung)[197], only a small number of computed tomography images datasets have been publicly available for liver tissue segmentation purposes. A table presenting details of the publicly available datasets can be found in the table 3.1.Some of the available datasets have been launched by medical-related institutions. IRCAD (ResearchInstitute Against Digestive Cancer)for example, provided both 3DIrcadb sets[198], with the first containing 20 abdominal CT scans, where 15 of the patients suffered from liver tumor, and the second containing only 2 CT scans with patients suffering from FNH (Focal Nodular Hyperplasia).Thanks to the different segmentation challenges launched throughout the time, some other sets have been provided and offered real benchmarks. The Segmentation of the Liver (Sliver'07) was organized in conjunction with MICCAI 2007 and offered 40 CT volumes[START_REF] Van Ginneken | 3D segmentation in the clinic: A grand challeng[END_REF], where 30 volumes were available during the challenge, and 10 hidden cases were used to evaluate the proposed methods. In 2015 the

	Name	#Cases	Liver	Tumor GT	Tumor	Other GT	#Experts	Volume size	Voxel size	Challenge	Hidden	Phases
			GT		Type			(pixels)	(mm)		Cases
	3Dircad-01		true	true	Undefined	Blood vessels	1	axial: 512x512	axial:0.56 -0.87	-	-	Undefined
								z : 74 -260	z: 1 -4
	3Dircad-02		true	true	FNH	-	1	axial: 512x512	axial: 0.96	-	-	1 AR
								z: 167 -219	z: 1.8 -2.4			1 PV
	Sliver07		true	true	Undefined	-	1	-	axial: 0.55 -0.8	MICCAI07	10	PV
									z: 1 -3
	MIDAS		false	true	Undefined	-	5	-	-	-	-	Undefined
	LITS		true	true	Undefined	No	3	axial: 512x512	axial:0.56 -1	MICCAI17	70	Undefined
								z: 42 -1026	z: 0.45 -6	ISBI 2017
	TCIA	2	false	false	HCC	No	-	axial:512x512	axial: 0.58 -0.98	-	-	mixed
								z: 26 -192	z: 0.8 -5
	CHAOS		true	false	Healthy	No	3	axial: 512x512	axial: 0.7 -0.8	ISBI 2019	-	PV
					Livers			z: 77 -105	z: 3 -3.2
	ImageClef Liver		true	true	-	No	-	x: 190-308	axial: 0.68 -1.01	ImageClef 2015	10	Undefined
				(Bounding				y: 213-238	z: 0.40 -2.5
				Box)				z: 41 -588

1: Publicly available datasets 2 CT and/or MR images volumes 68 CHAPTER 3. SEGMENTATION OF LIVER TISSUE IN COMPUTED TOMOGRAPHY: STATE-OF-THE-ART Image Clef challenge offered 50 CT scans with expert annotations [202], whereas 201 volumes have been provided by the organizers of both ISBI and MICCAI in 2017 for the Liver Tumor Segmentation (LITS)

benchmark

[START_REF] Bilic | The Liver Tumor Segmentation Benchmark (LiTS)[END_REF]

. In 2019 the organizer of ISBI further granted access to 40 cases for the Combined (CT-MR) Healthy Abdominal Organ Segmentation (CHAOS)

[203]

.

In these different datasets, the volumes were annotated by one or multiple experts, with predominantly pixel-wise delineations of the areas of interest (the ImageClef dataset is the only one with bounding box annotations for the lesions). Only one annotation volume per case is given which is designed to translate the consensus among the experts, however, the inter-expert variations can not be correctly caught by this format. Moreover, most of the available databases present heterogeneous data in terms of geometry (number of slices per patient, volume size, resolution), pathology (the number of lesions per volume in the LITS dataset goes from 1 to more than 70) and regarding the acquisition protocols (e.g. the acquisition phase is undefined for the LITS data where arterial and portal venous volumes are mixed up).

  proposed a level set formulation guided by a combined region-based and voxel-wise cost function. For the global image term, textural features (GLCM) as long as intensity-based and volume properties were computed. Initialization is performed based on prior knowledge about liver geometry and intensity distribution. The local information is represented by the shape prior obtained from a hierarchical patch-based division of the liver. The newly constructed level-set formulation is then

iteratively calculated to get the liver boundary. Xu et al.

[START_REF] Xu | Efficient multi-atlas abdominal segmentation on clinically acquired CT with SIMPLE context learning[END_REF]

performed a multi-organ segmentation via context learning followed by SIMPLE atlas selection (Selective and Iterative Method for Performance Level Estimation). Context-Learning was done via intensity distribution analysis with a GC. The different organs were modelised by several atlases using the SIMPLE method. During inference the different organs segmentations were fused using joint level fusion before a final abdominal segmentation can be obtained.

Wang et al.

[START_REF] Wang | A homotopy-based sparse representation for fast and accurate shape prior modeling in liver surgical planning[END_REF]

learned the liver shape model from a set of training samples, by implementing a Sparse Shape Composition (SSC). When segmenting a new case, they first initialized the liver boundary

Table 4 .

 4

	Db Name	Db	retained	Axial	Slice	Available	contrast-	Contains	Tumor	Liver	Tumor	Necrosis	#Experts
		number	of	voxel size	Thickness	enhanced phases	Tumor	type	Ground	Ground	Ground
		cases			(mm)	(mm)					truth	truth	truth
	TheraHCC-dB	104 2D slices		0.66-0.97	0.7-1.25	NECT, AR & PV	Yes	HCC	true	true	true	4
	LITS-dB	131 3D vol.		0.55-1	0.45-6	Single phase images but	Yes	Mixed	true	true	false	3
							mixed (AR & PV)
	3DIrcad-dB 1	15 3D vol.		0.57-0.87	1.25-4	Single phase images but	Yes	Mixed	true	true	false	-
							mixed (AR & PV)
	TCIA-dB 2	18 3D vol.		0.62-0.90	2.5-7	AR&PV		Yes	HCC	false	true	true	1
	G-dB	79 3D vol.		0.58-0.98	0.8-5	AR&PV		Yes	HCC	false	true	false	1

1: Datasets used in our experiments (NECT: Non-Enhanced Contrast CT, AR: Arterial, PV: Portal Venous) 1 Subset of the 3DIrcad-01 presented in the table 3.1 where only the 15 volumes with liver tumors were retained 2 Subset of the one presented in the table 3.1 where only the 18 exploitable volumes with both AR and PV phases were retained

Table 4 .

 4 before, and compared it to both {NECT, AR, PV, DMP, MPF}-Full networks. We evaluated them in terms of liver tissue classification performance on images where the values outside the ground truth liver area were masked. The mean DSCs are reported in table 4.3. Examples of segmentation results are given in figure 4.11. 2: Segmentation results using single-phase vs multiphase methods on TheraHCC-dB. True lesion mask Active Tumor 65.6 ± 20.4 63.9 ± 22.6 71.6 ± 20.7 75.5 ± 17.4 73.2 ± 18.6

	Input	Target	Network
			NECT	AR	PV	DMP	MPF
	Raw CT True liver mask True liver mask True lesion mask Necrosis Liver Parenchyma Lesion	81.1 ± 27.7 89.5 ± 13.2 88.7 ± 11.4 89.9 ± 15.6 86.5 ± 13.7 82.5 ± 18.6 88.7 ± 15.4 90.5 ± 13.2 77.4 ± 24.1 77.4 ± 20.2 87.8 ± 9.7 88.5 ± 11.7 67.5 ± 15.5 69.7 ± 16.3 77.8 ± 12.4 78.5 ± 13.3	88.2 ± 16.0 86.9 ± 17.8 86.6 ± 10.3 78.8 ± 11.7

Since both DMP-Lesion and DMP-Necrosis led to the best results, we combined them in a cascade 4.7. EXPERIMENTS 107 as explained

Table 4 .

 4 3: Segmentation results using {•}-Full vs cascaded architectures on TheraHCC-dB. Active Tumor 42.2 ± 24.0 43.2 ± 26.1 53.5 ± 24.2 51.3 ± 25.6 52.0 ± 23.3 59.6 ± 22.5

	Target	Network
		NECT-Full	AR-Full	PV-Full	DMP-Full	MPF-Full	Cascaded DMP
	Parenchyma	85.

3 ± 14.9 84.1 ± 17.9 87.0 ± 19.0 82.9 ± 19.7 87.9 ± 15.9 90.5 ± 13.2 Necrosis 62.6 ± 18.6 63.5 ± 21.5 75.7 ± 14.4 73.7 ± 14.1 75.6 ± 13.4 75.8 ± 15.1

Table 4 .

 4 We ± 24.6 for the segmentation of the active part, and 68.1 ± 23.2 for the necrotic part of the lesions. We also provided a necrosis rate per patient with a mean error of 15.9% when compared to expert necrosis rate estimation6 . Examples of fully automatic segmentation results are given in figure4.12.

		Semi-interactive method [281]	ours
	Parenchyma Lesion Necrosis Active Tumor Necrosis rate error	93.7 ± 3.4 90.7 ± 6 83.0 ± 12.9 75.2 ± 10.9 7.84 ± 4.4	92.2 ± 4.7 91.8 ± 4 83.6 ± 11.7 82.0 ± 6.4 7.10 ± 1.4

4: Average patient-wise segmentation DSCs (on full agreement expert area) with the semiinteractive approach of

[START_REF] Conze | Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans[END_REF]

and the cascaded DMP method finally combined the DMP-Liver, the DMP-Lesion and DMP-Necrosis networks in a cascaded fashion, as described in the figure

4

.1. This allowed us to perform a fully automatic segmentation of the raw (unmasked) CT images. We reached average slice-wise DSCs of 78.3 ± 22.1 for the segmentation of the parenchyma, 50.6

Table 5 .

 5 2: Confusion matrix regarding the DLR-based patient-wise histological grade prediction

			True grade
			LG	HG	Total
	Predicted grade	LG	7	1	8
		HG	2	8	10
		Total	9	9	18

  Since those two metrics are highly related to the number of voxels of each class, we decided in order to combine them to incorporate the number of voxels of each one of the two classes to create a new weighted metric that will help us select the best networks. Table B.3: Segmentation results obtained with the combined metric

	• Number of epochs: 10
	• Number of filters at bottleneck: 512
	• Input image size: 256
	• No data augmentation
	Table B.1: Parenchyma segmentation results (mean slice-wise % Dsc)
					Decay
			10 -1	10 -2	10 -3	10 -4	10 -5
		10 -3	82.9	91.9	90	80.3	9.6
	Lr	10 -4	81.3	90.8	90.8	91.9	93.7
		10 -5	32.4	76.7	88	89.1	89.7
	Table B.2: Tumor segmentation results (mean slice-wise % Dsc)
					Decay
			10 -1	10 -2	10 -3	10 -4	10 -5
		10 -3	7.9	23.5	24.5	16.1	1.5
	Lr	10 -4	6	15.6	15.6	23.6	21.7
		10 -5	5.4	5.4	14	17	15.2
					Decay
			10 -1	10 -2	10 -3	10 -4	10 -5
		10 -3	11.6	26.9	27.7	19.3	1.9
	Lr	10 -4	9.7	19.3	19.3	27.0	25.2
		10 -5	6.7	8.9	17.6	20.5	18.9

  Table B.5: Tumor segmentation results (mean slice-wise % Dsc)

			# Bottleneck filters
	Lr	Decay	128	256	512	1024
	10 -3	10 -2	76.1	92.1	91.9	7.2
	10 -3	10 -3	76.5	88.7	87	0
	10 -4	10 -4	87.1	92.8	93.9	94.2
	10 -4	10 -5	89.7	92.6	89.9	93.6
			# Bottleneck filters
	Lr	Decay	128	256	512	1024
	10 -3	10 -2	18.5	17.3	23.5	2.8
	10 -3	10 -3	23.5	22.1	24.5	0
	10 -4	10 -4	14.6	22	25.9	27.6
	10 -4	10 -5	12.2	19	15.2	26
	Table B.6: Segmentation results obtained with the combined metric
			# Bottleneck filters
	Lr	Decay	128	256	512	1024
	10 -3	10 -2	21.3	21.0	26.9	3.0
	10 -3	10 -3	26.1	25.4	27.6	0.0
	10 -4	10 -4	18.2	25.5	29.2	30.9
	10 -4	10 -5	16.0	22.6	18.9	29.3

Augm. Factor Dr Parenchyma Tumor Combined

	10	20	0	94.8	40.5	43.2
	10	20	0.2	94.7	38.4	41.2
	Using the original image size (512x512) seems also to improve the accuracy of the network, but in
	these experiments, we trained our networks with a reduced resolution (256x256) to reduce the training
	time.

Epochs Augm. Factor ImgSize Parenchyma Tumor Combined

	10	20	256	94.8	40.5	43.2
	10	20	512	95.1	42.7	45.3

10. High Gray Level Emphasis (HGLE):

		Ng	N d	P (i,j)
		i=1	j=1	i 2
			N z
	HGLE =	Ng i=1	N d j=1 P (i, j) i 2 N z
	11. Small Dependence Low Gray Level Emphasis (SDLGLE):

•

  Une approche deep radiomics avec extraction des descripteurs directement à partir du réseau de neurones responsable de la segmentation sémantique de la tumeur Pour cette tâche, la seule base de données avec une vérité terrain histologique disponible était la base TCIA dont 18 cas étaient exploitables. Les différents cas ont été séparés en deux catégories (9 patients avec grades faibles : Edmonson G1 et G2, vs 9 patients avec grades élevés : Edmonson G3). Les résultats préliminaires obtenus en utilisant l'approche radiomique traditionelle et l'approche hybride sont présents dans le tableau E.1.Table E.1: Nombre de patients correctement classifiés sur 18.

	Settings			Number of correctly classified patients (over 18)
	Image type	Tumor Segmentation	input = AR	input = PV	input = Perfusion	input = AR & PV
	Original	Expert GT	12	10	7	11
	Original + LoG	Expert GT	13	10	13	13
	Original + LoG + Wavelet	Expert GT	12	11	11	11
	Original	predicted	11	11	12	12
	Original + LoG	predicted	14	12	13	14
	Original + LoG + Wavelet	predicted	9	10	10	11

Liver transplantation

Two-Trait Predictor of Venous Invasion: Internal arteries and Hypoattenuating halo

Liver Transplantation

http://bit.ly/dlr_studies_summary

Although some parts of the pipeline are being processed by DL networks, it is worth noting that among the analyzed DLR-studies, the definition of the region of interests and the registration is still often manually performed, hence, highlighting current limitations of the first DLR-based studies.

proportion of the liver affected by cancer

6 studies among the 61 retained ones performed both the segmentation of the liver and its tumors, whereas 7 others segmented both the liver and other abdominal organs

can be fully automatic in some cases

The random walker will compute, for each unlabeled pixel, the highest probability that a random walker reaches the first labeled pixel whereas the GC will try to find the optimal cut for the entire image considering all the labeled seed, thus being more likely to fail in case of weak boundaries[START_REF] Grady | Random walks for image segmentation[END_REF]

Either cropping or padding was performed so that the images have the same size

DL studies usually tend to perform data standardization but here HU values are already normalized. The used range helps us to avoid potential bias caused by images acquired at a delayed phase with a different contrast

In the next chapter, we will update our cascaded semantic segmentation architecture to perform the segmentation of liver and liver tissues on larger datasets, and ultimately extract relevant semantic imaging features to perform an automatic prediction of the histological grade.

Assuming that the aorta is the best area to measure the concentration of contrast agent

In cirrhotic livers, the portal system is the most affected, to such an extent that the arterial inflow increases, hence the contrast inflow, whereas the spleen is less affected.

https://pyradiomics.readthedocs.io/en/latest/

Laplacian Of Gaussian with various σ filter sizes

+-,.+.* & 1 toxin that can readily on foods and that has been classified as carcinogen by the International Agency for Research

CONTENTS

patient-wise DSC). We confirmed the benefit of the cascaded architecture since these results were obtained using an architecture where the first network was trained on LITS-dB and the second on G-dB.

Such an architecture allows consequently to obtain good results even though the number of training cases is small, and the different available datasets homogeneous in the type of annotated areas. The cascaded architecture also permits to use a single-phase network for the first stage and a multiphasic network for the second.

Histological grade prediction

When evaluating the HCR-based histological grade prediction model, we presented the accuracy of the model in terms of how many patients in the testing set were correctly classified throughout the 9-fold cross-validation training. Results are given in table 5.1.

142

CONCLUSION AND PERSPECTIVES comprehended. The radiomics field has been also impacted by this novel paradigm, and a new approach called DLR (Deep-Learning Radiomics) has been initiated, where one or several steps of the radiomics pipeline are performed by deep learning algorithms. We reviewed 8 studies implementing a DLR pipeline to characterize liver tumors, and realized that a key part of the radiomics pipeline lies in the segmentation of the ROI. This step has been found to suffer from a high inter-and intra-observer variability, thus having consequences for the accuracy of the final radiomics-based prediction.

We consequently reviewed the different studies using deep learning architectures to perform automatic segmentation of the liver and its tumors. We extracted the common key settings shared by the majority of them, such as the cascaded architecture or the use of fully convolutional networks. However, we realized the lack of studies presenting results obtained from multiphase images, which has been a key element in our work.

We performed the automatic segmentation of an internal dataset composed of 104 sparse biphasic liver slices obtained from patients suffering from HCC (images available before the injection of contrast medium: Non-Enhanced CT, and at both arterial and portal venous phases). Considering how challenging the segmentation of liver tissues is, the amount of data, and the success of such an architecture, we decided to train several specialized networks in a cascaded way.

When evaluating the performances of each specialized network, we validated the hypothesis that the use of multiphase information allows a better accuracy than single phased based networks for each task, with significant difference obtained for the segmentation of the liver (mean DSC of 89.9 ± 15.6 for the multiphase network vs 89.5 ± 13.2 for the best single phase network, p = 0.005) and the active part of the lesions (mean DSC of 75.5 ± 17.4 vs 71.6 ± 20.7, p = 0.001).

Regarding single phase networks, the PV phase was the one allowing the most accurate segmentation, with significant difference vs AR and NECT for the segmentation of the parenchyma (mean DSC of 88.7 ± 15.4) , the lesion (mean DSC of 87.8 ± 9.7), and both the necrotic (mean DSC 77.8 ± 12.4) and the active (mean DSC of 71.6 ± 20.7) parts of the lesion.

We validated the hypothesis that several specialized networks combined in a cascaded architecture perform better than a single network addressing all the tasks simultaneously (obtained mean slice-wise DSC of 90.5 ± 13.2 for the parenchyma, 75.8 ± 15.1 for the necrosis and 59.6 ± 22.5 for the active part of the tumor when using the cascaded architecture with the liver GT mask as input).

In a fully automatic manner (without using the liver GT mask), we were able to reach promising results

(regarding the size of the dataset), with a mean slice-wise DSC of 78.3 ± 22.1 for the parenchyma, 50.6 ± 24.6 for the active tumor and 68.1 ± 23.2 for the necrotic part of the tumor.

We were also able to automatically compute the necrosis rate of the tumors, with a mean error of 15.9% when compared with the experts obtained rates. This prediction is accurate enough to consider the

CONCLUSION AND PERSPECTIVES

of both the liver and the lesions. The residual attention mechanism has been introduced in 2017 by Wang et al. [START_REF] Wang | Residual attention network for image classification[END_REF] to perform image classification, with the idea that the attention mechanism can help the network focusing on specific parts of the image. The study from Jin et al. was the first to use the attention mechanism for semantic segmentation purposes. On the hidden test set of LITS, they outperformed a lot of 2D-based methods, but were still far from the top-ranked teams.

However, new paradigms such as the self-attention mechanism, in combination with state-of-the-art 2D and 3D architectures are certainly an avenue for the improvement of the automatic liver and tumors segmentation tasks [START_REF] Chen | A DUAL-ATTENTION DILATED RESIDUAL NETWORK FOR LIVER LESION CLASSIFICATION AND LOCALIZATION ON[END_REF].

Dynamic contrast-enhanced images

In our research work, the key element is the extraction of features from dynamic contrast-enhanced images. However, it has been very difficult to collect dynamic CECT based images, since only a few publicly available datasets contain this type of volume.

There is a huge room for improvement regarding this specific type of dataset. It has been mentioned

in chapter 1, that the use of images obtained after injection of contrast medium largely improves the quality of the diagnostic. Nonetheless, we realized that multiphasic images could be better used in either the semantic segmentation and/or the radiomics field.

The first issue is related to the acquisition protocol required to obtain this type of images. Most of the time, the different retained phases, namely arterial, portal venous or delayed phases are acquired following either visual inspection of the radiologists 5 , or after a specific duration following the injection moment that can differ from one study to the other. Another criteria to consider is the physiological characteristics of the patient such as its weight that will determine how the contrast medium is diffused through the liver.

When dealing with multiphase databases, we believe that these key elements need to be considered, at least at the moment of injection and/or the duration between the injection and the acquisition.

Currently, only the DICOM format allows the addition of such metadata (it is only possible to record the exact acquisition moment but not the injection moment) but they are barely reported and are often left to default values.

Obtaining such data seems really challenging, hence, some techniques need to be implemented to deal with current multiphase images. For example, we believe that a "phase correction system", responsible for the normalization of images belonging to the same phase, could be implemented . The automatic detection of the amount of contrast medium still in the liver could also help to predict the exact duration between the injection of the contrast medium and the acquisition. The aim of the study was to perform tumor segmentation using a classification model with patches as input and convolutional layers introduced early in the network to recognize low-level features. A patch was considered as positive if at least 50% of its pixels are tumoral. Five different patch sizes were tested to train a CNN dedicated to perform a binary classification between tumoral and non-tumoral labels.

Using cross-validation on a DB containing 26 CECT volumes, it was noticed that 17x17 patches were the one that reached the best accuracy. The CNN achieved slightly better performance than classical ML techniques built on hand-crafted features (AdaBoost, RF and SVM).

Ben-Cohen et al. (2016)

Ben-Cohen et al. (2016), used the famous VGG-16 architecture to construct his own network. The dense layers were converted into convolutional layers, especially the output classification layer that was removed and replaced by a 1x1 convolutional layer with 2 channels for a pixel-wise classification.

The remaining layers were distributed in a FCN-8s-like architecture allowing to combine low-layer information with high-layer information. They also decided to provide not only one slice as input, but 3 successive slices (namely 2.5D input) to help the network retrieve the 3D information. The output of the network corresponding to the segmentation of the middle slice. In order for the 3D information to be consistent from one patient to another, they interpolated the slices in the z-axis to have a fixed 1mm-spacing.

Data augmentation was used to generate natural-like images in order to artificially increase the size of the database. In their case, 4 different synthetic images were generated from one raw image by applying a scaling in the range [0.8, 1.2].

Two different databases were used to train their model, the SLIVER07 dataset was used to train the APPENDIX A. DETAILED ANALYSIS OF THE DL SEMANTIC SEGMENTATION REVIEWED ARTICLES liver segmentation part, and an internal database of 20 patients, with only 68 liver-segmented slices, and 43 lesions-segmented slices, was used to train the second part of the architecture.

To evaluate their model, it was decided to consider the entire segmentation for the first stage by computing the Dice, whereas an object-based detection metric was implemented for the second stage, by analyzing the false positive rate per case (FPC).

The liver segmentation performance was evaluated on the 68 liver-annotated slices of their internal dataset, and they obtained a dice of 0.89 (note that only a slight improvement was obtained thanks to the 2.5D input: 0.88 vs 0.89).

The accuracy of the lesion detection model was compared to both a patch-based approach where 17x17 patches were sampled within the liver, and a sparsity-based model. The FCN achieved better results than the other two methods reaching a FPC of 0.74.

When using the entire cascaded architecture, they realized that the automatic liver segmentation often doesn't include darker borders, and that those pixels could further be classified as FP, so the fully automatic segmentation leads to an improvement in FPC.

They also conducted some synthetic experiments and realized that the network mostly relies on the gray level differences.

Christ et al. (2016)

Christ et al, provided a cascaded architecture for the segmentation of both the liver and the potential lesions it contains. Their motivations were similar since they believed that the network in the first step will learn filters specific to the detection and the segmentation of the liver in an abdominal axial CT image, while considering that the liver ROI will help reduce the number of FP for the lesion segmentation task. The U-Net architecture introduced by Ronneberger et al which was used as a basis, was chosen for its ability to combine low and high-level features through so-called skip-connections.

In order to prove their hypothesis, they worked with the 3DIRCADb dataset containing 20 patients, where only the 15 patients presenting hepatic tumors were retained.

The images were pre-processed by first clipping the HU values in the range [-100, 400], followed by a histogram equalisation. Data augmentation was also used to increase the size of the database via rotations, flips and addition of gaussian noise, so they were able to train their models with more than 20k 2-D slices. In order not to train their networks from scratch, they used the pre-trained UNet They tried to perform the segmentation of the lesions within the liver, but the network failed to converge, certainly due to the lack of sufficient training data, even when using a large number of iterations. To solve this issue, they first pre-trained the network to segment the liver in the image before fine-tuning it for the lesion segmentation.

They finally proved by the obtained results, an improvement of the segmentation accuracy when using the multiphase information. They also decided to use two different scales to capture fine (20x20) and global spatial information (50x50), before concatenating the obtained features from both sides at the end of the network. Patches were randomly sampled to respect class-balancing, and data augmentation was applied via random flips and rotations [5°, 130°, 300°]. Intensity was shifted so they all share the same mean intensity.

Frid

APPENDIX A. DETAILED ANALYSIS OF THE DL SEMANTIC SEGMENTATION REVIEWED ARTICLES

From their experiments, they realized that the DenseUNet performed better than simple DenseNet, confirming the importance of long skip-connections. Interestingly, the 2D version of this DenseUNet allowed a better learning behaviour, when compared to the 3D version, and this hypothesis was confirmed by the prediction results (improvement of 0.89 points for the Dice per case, and 0.3 points for the Dice global). They also proved that the hybrid features help the network improve the obtained results.

Using the final architecture, they got a mean dice per case of 0.72 and a dice global of 0.82.

Han et al. (2017)

Han et al. (2017) decided to combine the residual network and the UNet, by building a resnet-like network with UNet skip connections between encoding and decoding parts.

They also provided 5 stacked slices as input in a 2.5D fashion, so the network can rely on the information provided by adjacent slices.

The first step of their model consisted of getting a coarse liver segmentation, before being refined in the second step which also segmented the lesions it may contain.

A weighted cross-entropy loss function with empirically chosen weights was implemented to train the network. The entire CT slices resampled to a fixed resolution of 1x1x2.5 mm were used to train the liver segmentation, whereas only liver slices with their original resolution were utilized for the lesions segmentation step.

During inference, the input volume is resampled to 1x1x2.5 mm , then the liver segmentation is performed before getting the largest 3D connected component. The original resolution is then used to perform the lesion segmentation in order to get finer details from the liver and its lesions. A post-processing step was employed so that connected components considered as lesions will be removed if the maximal probability value is less than a given threshold.

The entire process allowed them to reach a mean dice per case of 0.67.

Yuan et al. (2017)

Yuan et al. (2017) proposed a hierarchical convolution-deconvolution neural network to carry out the segmentation of both the liver and the lesions.

The first step of their architecture consisted of getting a coarse liver segmentation on the entire 3D volume, before the second step refined the probability map, and the final one executed the lesions segmentation.

The same architecture was used for the three different steps, where 2.5D input was chosen, and training was performed with the Jaccard distance as loss function. This function takes advantage from the fact

A.2. LITS PAPERS ARTICLES

They finally obtained a Dice of 0.64 for the segmentation of the lesions.

Vorontsov et al. (2018)

Voronstov et al. (2018) decided also to use a cascaded architecture where two U-Net-like networks were combined, but they decided to build a model allowing an end-to-end training of the entire cascade.

The first network took as input an axial slice and predicted the probability of each pixel to belong to the liver, and the second one took as input both the original slice and the output of the first network.

The latter implemented short skip connections in a resnet-fashion to propagate the liver localisation information to all the layers.

Images were pre-processed by dividing the input intensities to 255, and clipping the remaining values to the range [-2, 2], and only liver slices were used to train the network with the dice loss as loss function.

Data augmentation was applied via flips, rotations, zoom and elastic deformations. The network was pre-trained on the down-sampled images (256x256) then fine-tuned on the full resolution slices.

To improve the segmentation, a classifier was added at the end of each network, aiming to introduce a so-called cross-slice context. The segmentation was performed on 3 adjacent slices, and the output features were combined by a classifier to predict the middle-slice segmentation. During inference, 3 different networks were associated via a voting strategy, where each one provided a prediction corresponding to the average of 4 oriented images after flipping the original one. The final liver segmentation corresponded to the largest connected component, and lesions were kept when belonging to a dilated version of the predicted liver. This prevents missing lesions when the liver is slightly under-segmented.

They finally obtained a Dice per case of 0.66 for the lesions and of 0.95 for the liver.

Bilic et al. (2019) LITS Review

The majority of the papers that were submitted for these 2 challenges share some common properties, Their model contained 3 stages, the first one to get a coarse liver segmentation from 2D images, the second one to refine the obtained volume of interest using a 3D architecture, and the last one to segment tumors from the precise liver VOI.

Each network was built according to the U-Net shape but using residual blocks as key components.

The combination of both residual blocks and attention mechanism allows the network to learn on deep architectures, but with a focus on localizations that are relevant for the requested task. The images were pre-processed by first clipping the HU intensities to the range [-100 , 200], before applying a zero-mean and a min-max normalization. The network was then trained using the dice loss as loss function. In the first stage, a set of 2/3 liver images and 1/3 randomly picked slices were down-sampled to 256x256 pixels to train the network. The coarse liver segmentation was obtained through the extraction of the largest connected component.

Large 3D patches of 224x224x32 slices were extracted from both the lesions and non-lesions area and were provided to train the second network.

Finally, for the lesions segmentation task, they tested 3 different patch-sizes (32x32x32, 64x64x32 and 128x128x32) before realizing that the larger the patch was, the richer the extracted information will be.

The patches were randomly sampled within the liver.

Since the model was trained on 5 CV-folds, they implemented a voting strategy from the five resulting networks to get the final segmentation.

They obtained a Dice per case of 0.59 for the lesion segmentation task on the hidden set of LITS, outperforming a lot of 2D-based methods, but still far from the top-ranked teams.

Appendix B

Hyperparameters selection

In this chapter, we reported the experiments that we run on the 3DIrcad-dB to find the best sets of hyperparameters for the tumor segmentation networks. Results were reported here to allow better readability in of our complete semantic segmentation work (see chapter 4.7.3)

Here is the list of hyperparameters to set in our case:

• Learning rate techniques that better suits small architectures and less complex problems, so we have decided to follow an heuristic approach to find the best set of parameters.

We first tried to set the best pairs of learning rate and decay, by freezing the rest of the parameters.

For the chosen optimizer, the learning rate and the decay will determine how the gradient descent is performed. A too large initial learning rate will cause the network to continuously overshoot the minimum, whereas a too small one will require a large amount of iterations to reach the minimum, as depicted in the figure B.1 hereafter.

1 Evaluating one set of hyperparameters can take up to 48h using a single our networks implemented using Keras with a Tensorflow backend and trained on a single NVIDIA GTX 1070 graphics card with 8 GB VRAM. Once the best candidates were found, we tried to modify the depth of the network. In the case of the U-Net architecture, one easy way to change the depth of the network is to add or remove one stage in both the encoding and the decoding parts. This can be directly modeled by the number of filters used in the bottleneck part of the network (originally 1024). In our case, using 1024 filters in the bottleneck part of the network, will correspond to an architecture with almost 32M parameters. Setting these first 3 parameters (learning rate, the decay and the depth of the network) gave us a baseline segmentation accuracy that we tried to improve with the data augmentation. As exposed previously, the standard way to apply data augmentation is to perform geometrical transformations to the original images so that the network will further be invariant to those modifications.

The classical transformations applied are:

• Translation: Each translation is performed in one randomly chosen direction, where the delta value corresponds to the amplitude of the shift in terms of number of pixels (e.g shift of 0.1 for an initial size of 256 corresponds to a shift of 26 pixels in one direction).

• Flip : Horizontal or Vertical flip of the original image.

• Rotation : Apply a rotation of a given angle to the original image.

We decided in our case to only investigate the data augmentation that will prevent the aspect of the liver and its internal tissues. In the case of elastic deformation, we can not testify that the retained features will still be discriminant in our deep radiomics approach for the grade classification if the aspect of the internal liver tissues is changed. We have performed for each of the following experiments a 5-Fold Cross-Validation training on the 3DIrcad-dB (see table 4.1 for more details), and the reported results correspond to the mean slice-wise dice for the given region (parenchyma or lesion).

B.1. LEARNING RATE & DECAY

We selected the pairs providing the best accuracy and investigated the behavior of the gradient descent when slightly changing the depth of the network.

B.2 Network's depth

Once the pairs of learning rate and decay selected, we have changed the depth of the network. This setting is controlled in our pipeline by a parameter called "bottleneck filters".

In the above mentioned experiments, this number was set to 512, and we tried to change it with the following values: {128, 256, 512, 1024}.

Here are the reported results for both the segmentation of the parenchyma and the tumor when changing the depth of the network: We can see here that flip as data augmentation brings a clear value to the segmentation performances of the network.

Rotation

When applying rotation, we have to carefully choose the angles used during the rotation since too small angles can cause overfitting and too large angles can be useless because representing unrealistic data.

Here are the results obtained for the chosen rotation sets (mean slice-wise % Dsc, expect for the weighted combined metric)::

Rotation angle Parenchyma Tumor Combined

No Augm. The final set of hyperparameters for the segmentation of both the parenchyma and the tumor is the following:

• Lr: 1e-4

• Decay: 1e-4

• Number of epochs: 30

• Optimizer: Adam

Radiomics features

Here we detail the different formulas to compute the first-order and textural radiomics features, as described by PyRadiomics (https://pyradiomics.readthedocs.io/en/latest/).

C.1 First-order statistics

Let:

• X be a set of N P voxels included in the ROI,

• P (i) be the first order histogram with N g discrete intensity levels, where N g is the number of non-zero bins,

• p (i) be the normalized first order histogram and equal to P (i) NP .

We can define the following features:

1. Energy:

Here, ǫ is an arbitrarily small positive number (≈2.2×10 -16 )

Minimum:

minimum = min (X)

10 th Percentile

The 10 th percentile of X, a robust alternative to the minimum gray-value.

90 th Percentile

The 90 th percentile of X, a robust alternative to the maximum gray-value.

Maximum:

The maximum gray level intensity within the ROI. maximum = max (X)

Mean:

The average gray level intensity within the ROI.

Median:

The median gray level intensity within the ROI.

Interquartile Range:

interquartile range = P 75 -P 25

Here P 25 and P 75 are the 25 th and 75 th percentile of the image array, respectively.

Range:

The range of gray values in the ROI. range = max (X) -min (X)

Mean Absolute Deviation (MAD):

MAD is the mean distance of all intensity values from the Mean Value of the image array.

towards a spike near the Mean value.

Variance:

. Uniformity:

C.2 Gray Level Co-occurrence Matrix GLCM Features

A normalized GLCM is defined as P (i, j; δ, α), a matrix with size N g × N g describing the second-order joint probability function of an image, where the (i, j) th element represents the number of times the combination of intensity levels i and j occur in two pixels in the image, that are separated by a distance of δ pixels in direction α. The distance δ from the center voxel is defined as the distance according to the infinity norm. For δ=1, this results in 2 neighbors for each of 13 angles in 3D (26-connectivity).

Let:

• ǫ be an arbitrarily small positive number (≈2.2×10 -16 )

• P (i, j) be the co-occurence matrix for an arbitraryδ and α, ,

• p (i, j) be the normalized co-occurence matrix and equal to P (i,j) P (i,j) ,

• N g be the number of discrete intensity levels in the image,

• p x (i) = Ng j=1 P (i, j) be the marginal row probabilities,

• p y (i) = Ng i=1 P (i, j) be the marginal column probabilities,

• µ x be the mean gray level intensity of p x ,

• µ y be the mean gray level intensity of p y ,

• σ x be the standard deviation of p x ,

• σ y be the standard deviation of p y ,

• p x+y (k) = • HX = - (i -µ x ) 2 p (i, j)

C.3 Gray Level Run Length Matrix (GLRLM) Features

A GLRLM quantifies gray level runs, which are defined as the length in number of pixels, of consecutive pixels that have the same gray level value. In a gray level run length matrix P (i, j ∨ θ), the (i, j) th element describes the number of runs with gray level i and length j occur in the image along angle θ.

Let:

• N g be the number of discreet intensity values in the image,

• N r be the number of discreet run lengths in the image,

• N p be the number of voxels in the image,

• N z (θ) be the number of runs in the image along angle θ, which is equal to

Ng i=1

Nr j=1 P (i, j ∨ θ), and 1 ≤ N z (θ) ≤ N p ,

• P (i, j ∨ θ) be the run length matrix for an arbitrary directionθ,

• p (i, j ∨ θ) be the normalized run length matrix, defined asp (i, j|θ) = P (i,j∨θ) Nz(θ) ,

We can define the following features:

Short Run Emphasis (SRE):

Nr j=1

Here, ǫ is an arbitrarily small positive number (≈2.2×10 -16 )

Low Gray Level Run Emphasis (LGLRE):

LGLRE =

Ng i=1

Nr j=1

High Gray Level Run Emphasis (HGLRE):

Nr j=1 i 2 P (i, j ∨ θ) N z (θ)

Short Run Low Gray Level Emphasis (SRLGLE):

Nr j=1

Short Run High Gray Level Emphasis (SRHGLE):

Nr j=1

Long Run Low Gray Level Emphasis (LRLGLE):

Nr j=1

C.4 Gray Level Size Zone Matrix (GLSZM) Features

A GLSZM describes the amount of homogeneous connected areas within the volume, of a certain size and intensity, thereby describing tumor heterogeneity at a regional scale. A voxel is considered connected if the distance is 1 according to the infinity norm (26-connected region in 3D). In a GLSZM P (i, j), the (i, j) th element equals the number of zones with gray level i and size j appear in image.

Contrary to GLCM and GLRLM, the GLSZM is rotation independent, with only one matrix calculated for all directions in the ROI. The mathematical formulas that define the GLSZM features correspond to the definitions of features extracted from the GLRLM.

Let:

• N g be the number of discreet intensity values in the image,

• N s be the number of discreet zone sizes in the image,

• N p be the number of voxels in the image,

• N z be the number of zones in the ROI, which is equal to

Ng i=1

Ns j=1 P (i, j), and 1 ≤ N z N z ≤ N p

• P (i, j) be the size zone matrix,

• p (i, j) be the normalized size zone matrix, defined as p (i, j) = P (i,j) Nz .

We can define the following GLSZM features:

1. Small Area Emphasis (SAE):

Ns j=1

Large Area Emphasis (LAE):

Ns j=1 P (i, j) j 2 N z

Gray Level Non-Uniformity (GLN):

Evaluation metrics

Here we detail the most commonly used evaluation metrics.

D.1 Overlap measures

Dice and Jaccard are defined as follow:

Relative volume difference (RVD), corresponding to an asymmetric measure and defined as follow:

D.2 Surface Distance Metrics

This type of metric evaluate the quality of the segmentation by measuring the distance between the surfaces of reference and the predicted areas.

Let S(A) denote the set of surface voxels of A, the shortest distance between an arbitrary voxel v and S(A) will be defined as :

||v -s A || , where ||.|| corresponds to the Euclidean distance.

We can compute the following metrics :

APPENDIX D. EVALUATION METRICS

The Average Symmetric Surface Distance (ASD):

The Maximum symmetric surface distance(MSD):

Résumé long (Français) E.1 Contexte Scientifique

Radial

Cette thèse s'inscrit dans le projet Radial mené par l'institut de recherche de chirurgie guidée par l'image à Strasbourg dont l'un des buts est l'amélioration du traitement et de la prévention du cancer du foie.

Cancer

Le cancer est la seconde cause de mortalité au monde et le cancer du foie est le second type de cancer le plus répandu avec près de 800.000 morts dénombrés en 2019 selon l'OMS. Les cancers du foie peuvent être caractérisés comme "primaires" avec une apparition et une croissance qui se fait entièrement dans le foie, ou "secondaires", dans le cas d'une migration de cellules tumorales à partir d'un organe distant, jusqu'au foie pour donner lieu à une tumeur hépatique. Bien que moins fréquentes que les tumeurs secondaires, notre travail de recherche portera principalement sur les tumeurs primaires, avec une attention particulière sur les hépatocarcinomes cellulaires (HCC). Cette décision fut prise dans l'optique de préparer une future collaboration avec le professeur Patrick Baumert, expert dans l'étude des HCCs. De manière générale, la méthode standard pour évaluer la nature d'une tumeur consiste soit à effectuer une biopsie suivie d'une analyse anatomo-pathologique sur les tissus prélevés, soit à acquérir puis analyser des images tomographiques des organes atteints par la maladie.

Biopsie & Anatomopathologie

La biopsie, geste invasif, peut être compliquée, voire impossible à réaliser en raison des difficultés d'accès à la tumeur. L'anatomo-pathologie est la spécialité médicale dédiée à l'étude morphologique des anomalies macroscopiques et microscopiques des tissus biologiques et des cellules pathologiques prélevés sur un être vivant ou décédé. Elle est cruciale dans la caractérisation des tumeurs par analyse de tissus biopsiés. Classiquement cela implique une analyse visuelle par un médecin anatomo-pathologiste d'un échantillon de tissu prélevé sur le patient, à l'aide d'un microscope, pour identifier leurs propriétés (CNN) avec une ou plusieurs couches denses dont l'objectif est de prédire la classe de sortie. Tandis que le réseau est entraîné à effectuer la classification, les descripteurs sont extraits soit en queue de réseau (après l'une des dernières couches denses) [START_REF] Paul | Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma[END_REF], soit après l'une des couches de convolution [START_REF] Li | Deep Learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma[END_REF]. D'autres variantes, toujours à base de couches de convolution, ont vu le jour, comme le RNN (recurrent neural network), le LSTM (long-short-term memory) ou le Capsule Net. Leur objectif est de s'abstraire de la limitation provoquée par la taille des images en entrée qui est nécessairement fixe, et par la difficulté à considérer l'intégralité d'un volume 3D lors de l'entraînement [START_REF] Azizi | Deep recurrent neural networks for prostate cancer detection: Analysis of temporal enhanced ultrasound[END_REF]. Dans le cas non supervisé, l'objectif est de conserver les descripteurs responsables de la distribution des données afin de pouvoir créer de nouveaux échantillons à partir de cette distribution. L'architecture la plus utilisée dans ce cas est l'auto-encodeur, constituée d'une partie qui contracte l'information (encodeur) afin de ne conserver que celle essentielle à sa reconstruction (décodeur). Les auto-encodeurs peuvent être construits à partir de couches de convolution [START_REF] Echaniz | Ongoing Work on Deep Learning for Lung Cancer Prediction[END_REF], ou bien encore entraînés de manière à être insensibles au bruit ajouté aux données d'entrée [START_REF] Echaniz | Ongoing Work on Deep Learning for Lung Cancer Prediction[END_REF][START_REF] Emaminejad | Fusion of Quantitative Image and Genomic Biomarkers to Improve Prognosis Assessment of Early Stage Lung Cancer Patients[END_REF]. Dans le même objectif, qui est de pouvoir reconstruire l'image d'entrée en ne conservant que les descripteurs primordiaux, certaines études ont utilisé des réseaux dits de croyances profondes (DBN) [START_REF] Sun | Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis[END_REF] ou encore des machines de Boltzmann profondes [START_REF] Suk | Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis[END_REF]. La radiomique peut également faire appel à des méthodes dites hybrides où les descripteurs peuvent être soit combinés avec d'autres sources de données (ils peuvent être calculés à partir d'images acquises suivant différentes modalités [START_REF] Oikonomou | Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy[END_REF], ou bien associés à d'autres sources de données comme la génomique [START_REF] Emaminejad | Fusion of Quantitative Image and Genomic Biomarkers to Improve Prognosis Assessment of Early Stage Lung Cancer Patients[END_REF]), soit être issus de modèles avec uniquement une partie du processus faisant appel à l'apprentissage profond, que ce soit au niveau de l'extraction des caractéristiques [START_REF] Paul | Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma[END_REF], ou au niveau de la décision en fusionnant l'approche classique et celle basée sur l'apprentissage profond [START_REF] Huynh | Digital mammographic tumor classification using transfer learning from deep convolutional neural networks[END_REF]. Même si la radiomique semble se diriger de plus en plus vers une automatisation complète avec l'utilisation grandissante de l'apprentissage profond, il existe toujours un manque de transparence sur les choix effectués par les différents réseaux, c'est pourquoi certaines études ont décidé de combiner la prédiction du modèle automatique avec l'avis d'un expert [START_REF] Oakden-Rayner | Precision Radiology: Predicting longevity using feature engineering and deep learning methods in a radiomics framework[END_REF].

État de l'art radiomique

Dans cette thèse, un important travail de recherche a été effectué, avec pour objectif de recenser les différentes méthodes utilisant la radiomique pour l'aide au diagnostic sur le cancer du foie. Cette analyse détaillée a permis de soulever les différentes limitations que présente le paradigme dit "classique", notamment à l'aide d'une métrique (Radiomics Quality Score) pénalisant les études dont la reproductibilité n'est pas garantie [START_REF] Lambin | Radiomics: The bridge between medical imaging and personalized medicine[END_REF]. Cette partie de la thèse a donné lieu à une publication [START_REF] Wakabayashi | Radiomics in hepatocellular carcinoma: a quantitative review[END_REF] dans laquelle nous avons démontré que des critères importants étaient manquants comme une analyse basée sur des images obtenues à différents moments de la maladie, le caractère prospectif des études, E.3. SEGMENTATION SÉMANTIQUE 197 l'utilisation de données en libre accès ou encore l'utilisation d'un jeu de validation pour confirmer les résultats obtenus. Nous avons également effectué une analyse des études utilisant la radiomique sur base d'apprentissage profond afin d'analyser les HCCs. Il n'existe pour lors que peu d'études de la sorte, notamment car les modèles neuronaux utilisés sont souvent très récents (U-Net, ResNet, DenseNet, . . . ). La plupart d'entre elles se focalisant sur une caractérisation de la tumeur (avec notamment une classification du type de lésions focales, ou sur une estimation du taux de fibrose), tandis que les autres tentent de prédire la réponse à un traitement donné. Les différentes études ont souvent considéré la dynamique de la tumeur dans leur prise de décision, et ont construit des réseaux neuronaux le plus souvent à base de couche de convolution sachant que l'analyse s'est faite quasiment intégralement sur des images tomographiques utilisées seules. Parmi toutes les études examinées, seule une effectua une analyse de la tumeur en considérant une zone automatique segmentées par une algorithme de forêts d'arbres aléatoires, tandis que les autres se sont basées sur une zone manuellement segmentée par un ou plusieurs experts, le plus souvent en ne considérant pas l'information volumétrique (analyse uniquement sur la coupe présentant le plus grande proportion axiale de la tumeur). Cette étude nous a permis de voir les principaux travaux effectués récemment et nous avons décidé de créer une architecture capable d'une part de segmenter automatiquement les zones tumorales, et d'autre part d'effectuer une analyse clinique de l'avancement de la tumeur en prédisant le grade histologique. Avant de présenter notre contribution, nous rappelons les principes et les avancées effectuées en matière de segmentation sémantique appliquée au foie et aux tumeurs hépatiques.

E.3 Segmentation sémantique

Une analyse des méthodes de segmentation sémantique du foie ainsi que de ses structures internes a été entreprise, avec comme objectif d'effectuer une délinéation totalement automatisée du foie, ainsi que des différentes structures qu'il présente (tumeurs et nécrose) sur des images tomographiques obtenues à partir de patients souffrant de HCC. Le but étant de créer un modèle radiomique dont les descripteurs discriminants seront extraits à l'aide de l'apprentissage profond et ceci afin de permettre une prédiction ne dépendant uniquement que des données d'entrée.

Segmentation sémantique état de l'art

Sur les images tomographiques, les intensités présentes dans le foie et dans les organes avoisinants (coeur, rate, estomac) sont souvent très similaires. Les images sont souvent acquises après injection d'un produit de contraste, afin de bénéficier de l'utilisation de l'information temporelle par le biais du wash-in/wash-out. Néanmoins, le processus d'acquisition peut varier d'un institut à l'autre (type de APPENDIX E. RÉSUMÉ LONG (FRANÇAIS) produit de contraste, temps d'acquisition de chacune des phases, vitesse d'injection du produit), et les résultats peuvent différer en fonction des caractéristiques du patients (poids, flux sanguin, . . . ). De plus, l'aspect du foie peut varier en fonction des maladies qu'il subit comme la cirrhose, ou les tumeurs primaires ou secondaires. Toutes ces raisons augmentent la difficulté des tâches de segmentation automatiques ou semi-automatiques du foie et de ses structures internes.

Manque de données publiques annotées

Qu'elle soit semi-automatique ou entièrement automatique, l'implémentation d'un algorithme de segmentation sémantique nécessite une base de données d'apprentissage avec un nombre de cas suffisant pour permettre une reproductibilité élevée sur des données de test. Contrairement à d'autres organes comme le cerveau, le sein ou les poumons, il n'existe que peu de données d'images tomographiques de foie qui soient annotées par des experts. Les premières bases de données publiques dédiées à cet exercice, ont été mises à disposition par des instituts médicaux comme l'Ircad ou la NLM (National Library of Medicine). N'offrant que quelques dizaines d'images de patients souffrant de différentes pathologies hépatiques, des initiatives dans la communauté scientifique ont permis la mise en place de compétitions de segmentation sémantique (challenges) qui ont donné lieu à la création de nouvelles bases contenant plusieurs dizaines de cas chacune (comme LITS). Ces bases sont généralement annotées finement (voxel par voxel) par un ou plusieurs experts en consensus. Elles présentent une forte hétérogénéité, que ce soit sur la géométrie des volumes (nombre de coupes, résolutions), ou bien sur la pathologie dont souffre les patients qui la constitue (nombre de tumeurs, type de tumeurs). L'accès limité aux images tomographiques de foie ainsi que la forte hétérogénéité présente dans les bases de données rendent la tâche de segmentation d'autant plus difficile.

Méthodes classiques de segmentation sémantique

Les premières méthodes développées pour la segmentation sémantique des structures du foie reposaient sur des algorithmes d'apprentissage machine et une forte dépendance aux connaissances anatomiques a priori. Dans la majorité des cas, une information anatomique a priori relative à l'intensité, la forme ou encore la position du foie est intégrée dans le processus. Les différentes méthodes sont accompagnées de différentes étapes de pré et de post-traitement, qui la plupart du temps nécessitent une interaction avec l'utilisateur. Parmi les méthodes les plus utilisées, on distingue celles basées sur l'intensité des voxels comme les méthodes de seuillage ou bien celles effectuant une détection de contour avant d'implémenter une ou plusieurs étapes de raffinement. D'autres techniques reposent sur des interactions manuelles comme celles nécessitant le placement de graines initiales avant de faire grandir une région d'intérêt (Region Growing) ou encore celles basées sur la théorie des graphes. Les méthodes reposant sur E.3. SEGMENTATION SÉMANTIQUE cas, des étapes de pré-et de post-traitement afin d'améliorer la qualité des résultats comme c'est le cas avec l'extraction de la composante connexe la plus grande lors de l'étape de segmentation du foie.

Cependant dans les différentes études analysées, parmi celles utilisant des images CT, aucune n'a utilisé la dynamique de la tumeur pour effectuer une délinéation précise de la zone tumorale, notamment à cause de la difficulté pour obtenir des images CT dynamiques de patients malades qui soient également segmentées par un ou plusieurs experts.

Expériences pour la segmentation sémantique des structures du foie

Étant donné les différents choix effectués par la plupart des études de segmentation sémantique avec apprentissage profond, nous avons décidé de construire une architecture de réseaux de neurones en cascade afin de délimiter les contours du foie ainsi que des parties actives et nécrotiques des différentes tumeurs qui s'y trouvent. L'opportunité s'est présentée d'avoir accès à une base de données ayant fait l'objet d'une étude dans le même cadre [START_REF] Sun | Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs[END_REF]. La base a été créée à partir de 7 patients atteints se situera près du centre de la tumeur car c'est généralement à partir de là que la maladie évolue, avec une apparition de nécrose après un certain temps. Le réseau de neurones choisi pour effectuer la prédiction du grade s'apparente à un algorithme de réduction de dimensionnalité où un cube de 32x32x512 descripteurs est réduit en un cube de 32x32x8 éléments avant d'effectuer la prédiction via un softmax. Les descripteurs utilisés en entrée seront ceux retenus par le modèle de segmentation sémantique multiphase des tumeurs. Après un entraînement en validation croisée pour permettre une meilleure reproductibilité, l'architecture composée de couches de convolutions nous a permis de prédire, dans le meilleur des cas, le bon grade pour 15 patients sur les 18, avec 74% des coupes correctement prédites. La matrice de confusion obtenue est reportée dans le tableau E.2, avec une précision de 0.

Résumé en anglais

To evaluate the status of a liver tumor, we usually perform a biopsy followed by an anatomo-pathological evaluation of the extracted sample.

However, the biopsy, due to the small sampling size, does not testify the intra and inter-tumor heterogeneity, thus struggling in assessing precisely the phenotypical characteristics of the patients. Recent progress in medical imaging and data science fields enabled the emergence of a new technique called radiomics, that is partially answering these challenges. In this thesis, we have been focusing on hepatocellular carcinoma, and we built new imaging methods to characterize this widespread pathology. By incorporating temporal information through multiphase images, specialized UNet-like networks have been stacked in a cascaded architecture to provide a semantic segmentation of both the liver and its internal tissue (parenchyma, active & necrotic part of the tumor). To characterize the strong heterogeneity that resides in the tumor, we predict the histological grade on a fine scale (slice-wise), by re-using the features learned from the semantic segmentation network. Our preliminary results enable the production of a fine-detailed map of the tumor that separates well differentiated areas from poorly ones. Even though these results need to be confirmed with a larger cohort, we believe that medical images combined with deep modeling techniques may soon be introduced in a clinical workflow to help diagnose and evaluate the phenotypical characteristics of pathologies such as liver cancer.

Keywords: Liver cancer, Hepatocellular Carcinoma, Histological grade, Contrast Enhanced CT, Deep Learning, Semantic Segmentation, Radiomics