Titre : Utilisation de l'intelligence artificielle pour les procédures de curiethérapie prostatique guidée par l'image Mots clés : Curiethérapie, apprentissage profond, dosimétrie, apprentissage automatique, segmentation d'images médicales, cancer de la prostate.

Résumé: Les procédures de radiothérapie visent à exposer les cellules cancéreuses aux rayonnements ionisants. L'implantation permanente de sources radioactives à proximité des cellules cancéreuses est une technique classique pour guérir le cancer de la prostate à un stade précoce. Le processus implique l'acquisition d'images du patient, la délimitation des volumes cibles et des organes à risque à l'aide de l'imagerie, la planification du traitement, l'implantation de grains radioactifs guidées par l'image et l'évaluation postimplantatoire. L'analyse d'images médicales basée sur l'intelligence artificielle peut être bénéfique pour des procédures de radiothérapie. Elle peut aider à faciliter et à améliorer l'efficacité des procédures en segmentant automatiquement les organes cibles dans les images et en extrapolant des informations cliniquement pertinentes. Cependant, la délimitation manuelle des volumes cibles est toujours la routine standard pour la plupart des centres cliniques, ce qui prend du temps et n'est pas à l'abri de variations intra et inter-observateurs. Dans cette thèse, nous visons à développer des solutions de traitement d'images médicales pour automatiser divers étapes des procédures actuelles de curiethérapie de la prostate guidée par l'image, notamment l'identification des grains radioactifs à partir d'images de scanner X et la segmentation du volume cible clinique à partir d'images médicales.

Dans la première application, nous avons développé et évalué une nouvelle technique de détection et d'identification des grains radioactifs implantés sur des scanner X postimplantatoire en rapport avec la curiethérapie prostatique. Cela permet aux experts d'évaluer la qualité du positionnement de grains radioactifs guidées par l'image en calculant les paramètres dosimétriques. En particulier, il permet le calcul de dosimétrie post-implantatoire de la curiethérapie de rattrapage de la prostate réalisée des années après la curiethérapie initiale dans le cadre de récidive de cancer de la prostate. La deuxième application impliquait le développement de méthodes d'apprentissage profond pour délimiter automatiquement les volumes cibles cliniques. Nous avons évalué les méthodes proposées sur une base de données cliniques d'échographie transrectale peropératoire et des images scanner X post-implantatoire de la curiethérapie prostatique guidée par l'image. L'évaluation de notre méthode a été ensuite étendue à d'autres applications d'analyse d'images médicales. Nos méthodes ont donné des résultats prometteurs menant à une perspective essentielle pour des tâches d'analyse d'images médicales efficaces et précises. Elles peuvent être rebuvant être appliquées pour automatiser la gestion des procédures de curiethérapie prostatique guidée par l'image.

Title: Artificial intelligence for image-guided prostate brachytherapy procedures Keywords: Brachytherapy, deep learning, dosimetry, machine learning, medical image segmentation, prostate cancer.

Abstract:

Radiotherapy procedures aim at exposing cancer cells to ionizing radiation. Permanently implanting radioactive sources near to the cancer cells is a typical technique to cure early-stage prostate cancer. It involves image acquisition of the patient, delineating the target volumes and organs at risk on different medical images, treatment planning, image-guided radioactive seed delivery, and postimplant evaluation. Artificial intelligence-based medical image analysis can benefit radiotherapy procedures. It can help to facilitate and improve the efficiency of the procedures by automatically segmenting target organs and extrapolating clinically relevant information. However, manual delineation of target volumes is still the standard routine for most clinical centers, which is time-consuming, challenging, and not immune to intra-and inter-observer variations. In this thesis, we aim to develop medical image processing solutions to automate various components of the current image-guided prostate brachytherapy procedures, including radioactive seeds identification from CT images and clinical target volumes segmentation from different medical images. In the first application, we developed and evaluated a new technique for detecting and identifying implanted radioactive seeds on postimplant CT scans of prostate brachytherapy. This allows experts to evaluate the quality of the image-guided radioactive seed delivery by computing the delivered dosimetric parameters. Specifically, it allows computing the postimplant dosimetry of salvage prostate brachytherapy performed years after primary brachytherapy in the treatment of relapsed prostate cancer. The second application involved the development of deep learning methods to delineate clinical target volumes automatically. We evaluated the proposed methods on a clinical database of intraoperative transrectal ultrasound and postimplant CT images of image-guided prostate brachytherapy. The evaluation is then extended to other medical image analysis applications. Our methods yielded promising results and opening important perspectives towards efficient and accurate medical image analysis tasks. They can be applied to automate the management of image-guided prostate brachytherapy procedures.

Contents

Glossary v

List of Figures

Glossary

3D Three Dimensional.

ABS American Brachytherapy Society.

AI Artificial Intelligence.

ANN Artificial Neural Network.

ANNs Artificial Neural Networks.

CNNs Convolutional Neural Networks.

CT Computed Tomography.

CTV Clinical Target Volume. PET Positron Emission Tomography.

DL

PSA Prostate-Specific Antigen.

PTV Planning Target Volume.

TRUS Transrectal Ultrasound.

US Ultrasound. viii

Chapter 1

Introduction

In this chapter, we first present the primary motives for the works developed in this thesis. We then introduce the overview of the proposed solutions that include the objectives and our contributions.

Next, we present the dissertation outline, including a brief overview of our works presented in the following chapters.

Motivation

Prostate cancer (PCa) is one of the most frequently diagnosed types of cancer. It is also one of the leading causes of cancer death worldwide. Thanks to advances in diagnosis and treatment methods, patients diagnosed with PCa have shown high survival rates [START_REF] Tuppin | Prostate cancer outcomes in france: treatments, adverse effects and two-year mortality[END_REF]. For example, low dose rate brachytherapy (LDR-BR) is a typical minimally invasive and effective treatment option for cases with localized low and intermediate-risk PCa. It is associated with good oncological outcomes, together with good functional results [START_REF] Viktorin-Baier | Long-term oncological and functional follow-up in low-doserate brachytherapy for prostate cancer: results from the prospective nationwide swiss registry[END_REF].

In LDR-BR, tiny radioactive seeds are permanently implanted through the perineum into the prostate gland using a grid template under transrectal ultrasound (TRUS) image guidance. However, the implantation procedure is often associated with significant uncertainties. For example, uncertainties in the dispensing of the radioactive seeds to the desired position of the gland and in delineating the contour of the prostate gland accurately in TRUS images. Due to such inevitable uncertain scenarios, it is recommended to perform a postimplant analysis for every patient who has received LDR-BR.

Computed tomography (CT) imaging is the choice of imaging modality for the postimplant analysis [START_REF] Nag | The american brachytherapy society recommendations for permanent prostate brachytherapy postimplant dosimetric analysis[END_REF].

The CT imaging is required to analyze the dose distribution to the clinical target volumes (CTVs) (e.g., prostate) and surrounding healthy organs (e.g., rectum and urethra). It is expected to optimize the dose to the cancerous region (or prostate volume) and avoid unnecessary dosage or toxicity to the healthy organs. Postimplant LDR-BR analysis requires precise delineation of the CTVs and detection of the implanted seed's position from CT images. This helps to assess the implantation quality and to analysis patient outcomes.

Moreover, cancer can locally relapse after the patient has been treated with primary LDR-BR [START_REF] Tetreault-Laflamme | Long-term prostate specific antigen stability and predictive factors of failure after permanent seed prostate brachytherapy[END_REF].

Hereafter, when we say primary LDR-BR, we refer to the situation when a patient receives LDR-BR treatment for the first time. A new LDR-BR (called salvage LDR-BR) is a good option to treat locally relapsed tumors [START_REF] Créhange | Salvage reirradiation for locoregional failure after radiation therapy for prostate cancer: Who, when, where and how?[END_REF]. In salvage LDR-BR, the implantation procedures and postimplant dosimetry analysis are almost similar to the primary LDR-BR procedures. However, the implantation is more focused on the locally relapsed cancer position, unlike in the primary LDR-BR, during which the entire prostate volume is often considered homogeneously.

The challenges and uncertainties associated with the primary LDR-BR implantation and postimplant analysis techniques are also prevalent in salvage LDR-BR. Besides, postimplant dose analysis of salvage LDR-BR requires identifying the newly implanted seeds from the seeds implanted from the primary LDR-BR. Indeed, the radiation lifetime of the seeds implanted in primary LDR-BR generally expires a long time before the salvage LDR-BR. Therefore, in the postimplant CT-exams of salvage LDR-BR, the dose distribution needs to be computed by identifying the newly implanted seeds. However, CT-exams of the postimplant salvage LDR-BR contains cumulated seeds from the first and the second (salvage) LDR-BR, making it difficult to compute the dosimetry from the newly implanted seeds. Regarding medical image analysis, machine learning (ML)-based techniques have shown promising results [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF]. ML methods make predictions on unseen data after being trained on a finite training dataset.

LDR-BR for

Artificial intelligence (AI) system, mainly based on deep learning (DL) methods, seems a promising approach that can help in automating medical image processing tasks. It is useful to extrapolate clinically relevant information and speed-up contouring of target organs from the medical images, and thereby it can benefit both the clinical worflow and case health. Moreover, DL methods have successfully been leveraged to learn discriminative image features automatically using artificial convolutional neural networks (CNNs). However, current DL methods have two main drawbacks. First, DL methods are prone to produce unrealistic image segmentation results. Segmentation results with atypical errors that could not made manually. Secondly, DL methods lack generalization property that leads a major limitation to adapt a developed DL method into different clinical centers or new acquisition setups.

In supervised DL methods, the algorithm is subjected to training inputs knowing their corresponding desired outputs as a reference. Thus, it learns a rule on how to map the inputs to outputs from 3

Objectives

the training data. The learned rule can then be applied on the unseen testing data. The main aspects that derived DL methods are the availability of big data, processing power (dedicated hardware), and advances in DL algorithms. However, despite the promising results from AI systems based on DL methods, they seems relatively far from clinical utilization. Moreover, in medical imaging domain, supervised DL methods are often constrained with the limited datasets, in particular there is often scarcity of labeled references (ground truths). The datasets in LDR-BR procedures are not exception.

In this work, we will focus on investigating and developing image processing methods that can be performed semi-automatically and automatically in the image-guided LDR-BR procedures for PCa.

Specifically, we will focus on investigating DL methods for the prostate CTV contouring in LDR-BR procedures, and to develop a technique for postimplant dosimetry evaluation of salvage LDR-BR performed years after primary LDR-BR for relapsed PCa. The clinical objective is to facilitate the clinical workflows in LDR-BR procedures for PCa and thereby to help experts to evaluate and improve the clinical procedures, enhance clinical decisions, and thereby improve patient outcomes.

Objectives

The main goal of our work is to develop automated image processing tools that can help experts to facilitate the image-guided LDR-BR procedures for PCa. Mainly, the objectives can be categorized into two. The first objective is to detect and identify the radioactive seeds from postimplant CT images of salvage LDR-BR that can enable us to assess the salvage LDR-BR procedures. To this end, we investigated image processing methods to detect implanted seeds in postimplant CT-exams of salvage LDR-BR. We also investigated image registration methods to identify the new radioactive seeds implanted in salvage LDR-BR by registering it to the postimplant CT-exams of the primary LDR-BR.

These procedures then enabled us to evaluate the postimplant dosimetry of salvage LDR-BR.

The second objective is to reduce the need for user interaction in LDR-BR procedures, mainly in the contouring of the target organs. In this regard, we investigated automation approaches using DL methods. To this end, both interactive and fully automatic DL approaches were considered. While the interactive approach is aimed to developed a fast image annotation method in the case of scarce labeled datasets and accurate segmentation in cases of ambiguous images, the fully automatic approaches is for cases when we have sufficiently large labeled datasets for training. Both approaches are tested for prostate CTV segmentation in LDR-BR procedures and the evaluation is also extended to other medical image processing applications.

In the course of developing automated image processing tools for image-guided LDR-BR procedures, the following contributions were made:

• Proposing a postimplant dosimetry analysis approach from CT images of salvage LDR-BR performed years after primary LDR-BR for relapsed PCa.

• Developing a fast and interactive DL method for medical image annotation and segmentation with primary application to the image-guided LDR-BR procedures.

• Developing an image segmentation method for fast and reliable estimation of the prostate CTV from TRUS images using automatic DL method.

• Proposing and extensively evaluating a novel DL framework for medical image segmentation.

• Proposing a generative adversarial network (GAN) framework for post-processing of medical image segmentation methods.

Dissertation outline

The rest of this thesis is subdivided into six chapters.

In chapter 2, we present the background of the thesis covering concepts that are essential in this work. In particular, technical topics, including prostate cancer, treatment techniques, LDR-BR procedures, and deep learning methods in medical imaging, are introduced.

In chapter 3, a detailed literature review of state of the art in the treatment of prostate cancer using LDR-BR, including the clinical work-flows, medical imaging techniques, computerized medical image processing approaches, and challenges are presented. It is mainly a literature review of the contributions of medical imaging and image processing methods for LDR-BR procedures for PCa. In summary, the chapter covers the following points:

• Difficulties and limitations in LDR-BR for PCa.

• Advantages of the different imaging techniques (CT, MRI, TRUS, and fluoroscopy) and associated challenges in LDR-BR for PCa.

• Technical details of computer vision methods developed across LDR-BR, including classification, segmentation, registration, and detection approaches.

• Compared the developed image processing algorithms for radioactive seed detection from one and multi-modal medical images.

• Our perspectives concerning how to enhance the dose distribution computation techniques in LDR-BR procedures.

Afterward, our proposed solutions related to dosimetry computation in postimplant salvage LDR-BR from CT images are presented in chapter 4. To automatically detect and identify seeds in postimplant CT-exams of salvage LDR-BR for dosimetry computation, we introduce:

• An automatic radioactive seed detection and orientation estimation from postimplant CT-exams of LDR-BR using classical machine learning approaches.

• A landmark-based registration of detected seeds from primary LDR-BR CT-exam and salvage LDR-BR CT-exam.

• A dosimetry computation of the newly implanted radioactive seeds in postimplant CT-exams of salvage LDR-BR after detecting, registering, and removing the primary seeds from the primary LDR-BR stage.

Dissertation outline

The last two chapters (chapters 5 and 6) relate to our proposed DL-based solutions for medical image analysis with primary application to the image-guided prostate brachytherapy procedures. In chapter 5, we present interactive and automatic DL methods for prostate CTV segmentation from postimplant CT-exams and intraoperative TRUS-exams of the LDR-BR for PCa. The interactive approach is designed to address the difficulties in developing supervised DL methods in cases of scarce labeled datasets and the need for accurate segmentation results with minimal expert interaction. The extended fully-automatic method is then applied for prostate CTV segmentation from TRUS images.

Moreover, to evaluate the interactive approach for multi-structure image segmentation, we considered the echocardiographic image segmentation task.

In learning, and fine-tuning are also briefly introduced. Finally, we introduce the databases as well as the software frameworks used in this thesis.

Brachytherapy

Human cells grow and divide at normal health conditions to form new cells that are useful for the body.

Damaged or old cells die and are replaced by new cells. However, when the development of the cells is abnormal and can infiltrate and destroy healthy surrounding body tissues, it is known as cancer.

Then, the cells become more and more abnormal, damaged, or old cells survive when they should die, and new cells form when they are not needed by breaking the orderly process of forming healthy cells.

Collected mass of abnormal cells is referred to as tumors. It can be malignant when abnormal cells can spread into nearby healthy tissues. It can also be benign, high proliferation of normal cells thus, normal tissues or organs in the nearby can be squeezed or discharged. However, these cells do not spread into nearby tissues. Generally, the name of a particular cancer is given for an organ or tissue where cancer appears. In particular, prostate cancer (PCa), refers to cancer that develops in the prostate gland.

Prostate cancer

Prostate

The prostate gland is a small organ that is part of the male reproductive and urinary system. The prostate gland produces a whitish alkaline fluid that makes up part of the semen to protect and nourish the sperm. Anatomically, the anterior part of the prostate is surrounding the beginning of the urethra, and situated at the base of the bladder, while the posterior part is in front of the rectum, as shown in Zone division is used in clinical practices to describe the functional and pathological processes within the prostate gland. The four-zone divisions are the transition zone, the anterior fibro-muscular (also known as stroma) zone, the peripheral zone, and the central zone, as shown in Figure 2.2.

Each zone accounts for different volume size of the prostate and distinctive functions such as:

• Peripheral zone (PZ) comprises the posterior part of the gland surrounding the distal urethra and accounts for 60% of the entire prostate volume.

• Transition zone (TZ) surrounds the proximal urethra. It consists of 5-10% of the volume of the prostate gland. It is part of the prostate which grows throughout life.

• Central zone (CZ) encloses the ejaculatory ducts and represents about 25% of a normal healthy prostate gland in a young adult.

• Anterior fibromuscular zone or stroma (AFT) extends from the bladder neck to the external urethra sphincter. It makes up approximately 30% of the gland.

Prostate cancer causes

PCa is the second leading cause of death in the male patient from cancer following lung cancer, but it is the most common cancer in the male patient. It is also one of the most frequently diagnosed types of cancer worldwide. Cancer begins within the prostate tissues when the cells divide and grow abnormally. The abnormal cells then can continue, multiply, and possibly spread beyond the prostate gland.

Moreover, around 70-80% of prostatic cancer appears in the peripheral zone. A term stage is usually used to determine the rate of the spread of cancer (from stage 1 to stage 4); Stage I or II are localized while stage III or IV are locally advanced or metastatic, meaning that cancer cells have spread outside the prostate and can disseminate through different pathways. Common symptoms of PCa include pain in the pelvis, difficulties in urinating, blood in the urine, and erectile dysfunction depending on the stage. Among the common causes of cancers, including prostate cancer, are population aging, obesity, and adoption of cancer-causing lifestyles such as smoking and physical inactivity [7] [8]. Moreover, the history of a family of cancers may suggest that some prostate cancer patients could have a genetic predisposition.

Diagnosis and treatment options for PCa

Thanks to the prostate cancer screening and treatment technologies, patients diagnosed with PCa have shown high survival rates. PCa is often screened using the prostate-specific antigen (PSA) test with or without digital rectal examination (DRE) [START_REF] Richenberg | Esur prostate mr guidelines 2012[END_REF]. However, the most well-founded diagnosis for prostate cancer staging is based on collecting sample tissues using biopsy needles under the guidance of transrectal ultrasound (TRUS) images. These collected sample tissues are then analyzed to evaluate the stage of cancer. Mostly, the Gleason grading system is used to evaluate the aggressiveness of cancer cells [START_REF] Epstein | An update of the gleason grading system[END_REF]. Another approach is using magnetic resonance imaging (MRI) [START_REF] Hansen | Multicentre evaluation of magnetic resonance imaging supported transperineal prostate biopsy in biopsy-naïve men with suspicion of prostate cancer[END_REF], which is the choice of modality to visualize the tumor. Once PCa is diagnosed and classified into different stages (e.g., stage 1, 2, 3, and 4), a treatment method is selected.

Low dose rate brachytherapy procedures

LDR-BR is as effective as radical prostatectomy but is less detrimental to the quality of life (i.e., urinary and sexual functions), and hence usually used to cure localized PCa with low, intermediate, and high-risk cancer whatever the risk group either as monotherapy or as a boost or as salvage [START_REF] Kishan | Radical prostatectomy, external beam radiotherapy, or external beam radiotherapy with brachytherapy boost and disease progression and mortality in patients with gleason score 9-10 prostate cancer[END_REF].

Indeed, it is characterized by shorter treatment delivery time compared to the external beam therapy that requires multiple consecutive sessions. It is also characterized by minimally invasive procedures, short hospital stay, lower cost, and less exposure to ionizing radiation than external beam radiation therapy.

The entire LDR-BR procedure might involve several steps, mainly treatment planning, implantation, and postimplant analysis. In the treatment planning, the boundary of the clinical target volumes (CTVs), such as the prostate gland, is determined by a radiation oncologist. It is often done on 2D images obtained from ultrasound and a transrectal ultrasound probe. The radiation oncologist contours the boundary of the prostate on the 2D images to provide the CTV. After the CTV is determined, the In this stage, some margins are often included in the CTV. It is done to consider uncertainties while delivering the planned dose to the CTV. Usually, the CTV with the margins is referred to as Planning Target Volume (PTV).

During the implantation day, several steps are followed. First, the patient is anesthetized and positioned in the same way as he was in the treatment planning day. Secondly, tiny radioactive seeds are loaded into needles. Thirdly, the loaded seeds are permanently implanted into the prostate gland through the template grid (see Figure 2.4 a). The real-time visualization of the instruments (needles) and anatomical structures are performed using TRUS images [START_REF] Girum | A deep learning method for real-time intraoperative us image segmentation in prostate brachytherapy[END_REF] (see Figure 2.4 b). The radiation oncologist uses the treatment planning information such as the CTV to deliver the prescribed doses.

Nowadays, these procedures might be different among clinical centers. For example, some centers might perform planning during the implantation day, others in real-time during the implantation procedures.

As the intraoperative stage is not free of uncertainties and challenges, such as biomechanical factors (motion of gland due to the needle forces, inevitable patient motion, and soft tissue deformation), postimplant (postoperative) evaluation is essential as patient follow up. It is used to evaluate the entire procedures by computing the dose distributions delivered to the prostate CTV and the organs at risk (OARs) (e.g., rectum and urethra). For example, one month following the implantation day, the percentage of the prescribed dose received by 90% of the prostate (D90) and percentage of the prostate volume receiving 100% of the prescribed dose (V100) are evaluated. It is mainly applied for I-125 type of seeds, while day zero dosimetry analysis is often typical for Cs-131 and Pd-103 types of implants. Generally, the postimplant dosimetry is computed on CT images. To do so, two main steps are essential. The first step is the contouring of the prostate CTV and the rectum. The second step is

Imaging in low dose rate brachytherapy

determining the position of the implanted seeds.

Imaging in low dose rate brachytherapy

In LDR-BR, different imaging techniques and devices are used for the planning, implantation, and dosimetry analysis. In this section, we introduce the most commonly used imaging techniques, such as ultrasound and computed tomography, and their application in LDR-BR procedures.

Ultrasound imaging

Ultrasound imaging uses high-frequency sound waves to view inside the CTV. As the ultrasound images are captured in real-time, they are used to show the movement of targets. Indeed, ultrasound imaging is a noninvasive, easily portable, and relatively inexpensive device. The ultrasound operating frequencies are often between 1 and 10 MHz. Ultrasound imaging produces images through the reflection (backscattering) of the sound wave from interfaces between tissues and small structures within tissue. Thus, the strength of the sound signal and the time it takes for the wave to travel through the body provide the information necessary to produce the image. Ultrasound imaging has a high spatial resolution, particularly at high frequencies, and involves no ionizing radiation. There are some drawbacks associated with an ultrasound imaging technique. The first drawback is that ultrasound images are noisy. The second drawback is that some organs cannot easily be imaged because gas and bone impede the passage of ultrasound waves [START_REF] Ammari | An introduction to mathematics of emerging biomedical imaging[END_REF].

In LDR-BR, ultrasound imaging is a standard-of-care in the intraoperative stage with a transrectal ultrasound (TRUS) probe (transducer) to acquire a sparse volumetric representation of the gland.

The TRUS probe is often placed on a stepper that allows us to easily move the probe to see the entire prostate volume in three dimensions (3D) (i.e., axial, coronal, and sagittal orientations) (see an example of the axial view in Figure 2.4). This is done by emitting sound waves into the prostate gland that echoes back to the probe to create a contiguous acquisition and then the video of the organ [START_REF] Smith | High-speed ultrasound volumetric imaging system. i. transducer design and beam steering[END_REF].

Thus, TRUS is useful to visualize the instruments (needles) and the anatomical structures, including the deformation of the gland in real-time. Moreover, the radiation oncologist can use real-time TRUS images to determine the volume of the gland. It helps to perform real-time dosimetry computation that could allow experts to make adjustments while implanting the seeds.

Computed tomography

Computed tomography (CT) uses special X-ray equipment to create detailed scans of areas inside the body. A CT scan combines multiple X-ray attenuation projections from different angles to produce detailed cross-sectional images of areas inside the body. To do so, it uses a moving X-ray tube and a stationary detector ring. Detectors collect the attenuated photon energy struck on them and convert

Dosimetry analysis in low dose rate brachytherapy

it to an electrical signal. Using the recorded intensity profiles from different angles, a back-projection algorithm is performed to create the studied volume's 3D matrix. From this 3D matrix, different reconstruction algorithms are used to create images. The CT images allow doctors to get 3D views of certain parts of the body, such as soft tissues, the pelvis, lungs, and bones. However, CT imaging techniques are characterized by the low contrast imaging between soft tissues.

In LDR-BR, CT images are necessary for the dosimetry analysis. This is due to their excellent imaging characteristics of the implanted radioactive seeds that produce high signals that lead to their easy detection (see Figure 2.6). Thus, after the intraoperative LDR-BR, obtaining a CT scan of a patient is necessary to assess how the seeds were implanted. Moreover, based on the seed's position, the dose delivered to the gland, to the OARs, and the healthy surrounding organs are calculated, as described in the following section.

Dosimetry analysis in low dose rate brachytherapy

Dosimetry determines and relates the radioactivity absorbed by the tumors and organs from ionizing radiation. The radiation dose is sometimes known as absorbed dose, which is the amount of radiation energy deposited in tissue divided by the tissue's mass. Thus, the response of the tumor is correlated with the absorbed dose. Ideally, the ionizing radiation dose is expected to deliver the prescribed dose to the tumor areas while minimizing it, if possible avoiding it, to the surrounding healthy tissues.

Specifically, when the absorbed dose to the tumor is high, the tumor cells will be killed by radiation, and the higher the likelihood of a cure. Contrarily, when the absorbed dose to the healthy surrounding tissues is high, the more likely and severe radiation's undesirable toxic side-effects.

Dosimetry is important in correlating the dose distributions with the clinical outcomes, and even in treatment planning to avoid unnecessary excess toxicity or under-dosing the tumors. Therefore, calculating the dosimetry is profoundly important to estimate radiation protection, risk assessment, diagnostic dose estimates, and treatment planning. However, generally, radiation dose calculations are lengthy, tedious, and error-prone when performed manually. Thus, it is common to utilize software when submitting dosimetry estimates of patients [START_REF] Stabin | Radar commentary: evolution and current status of dosimetry in nuclear medicine[END_REF].

In LDR-BR, dosimetry can be computed in the treatment planning, intra-operative, and postoperative steps. However, in this thesis, we will focus on dosimetry computation in the postoperative step.

At the postoperative LDR-BR step, the success of the entire brachytherapy procedure is assessed by computing the delivered dose distributions such as D90 (percentage of the prescribed dose received by 90% of the prostate) and V100 (percentage of the prostate volume receiving 100% of the prescribed dose). The dose delivered to the OARs should also be evaluated. The dose distribution is calculated as a function of distance from the seeds' location to the prostate target or OARs. It is generally computed on CT images by precisely determining the volume of the prostate and the rectum and the implanted seeds' position. In practice, LDR-BR dose calculations are often performed using TG43 formalism, where the dose is calculated to a homogeneous water medium [START_REF] Miksys | Large-scale retrospective monte carlo dosimetric study for permanent implant prostate brachytherapy[END_REF]. In LDR-BR, experienced physicians can use the dosimetry information to refine and perfect the procedure during the intraoperative step. Moreover, the data that is provided by postimplant dosimetry can be used in patient outcome analysis.

Uncertainties in low dose rate brachytherapy

The LDR-BR technique has demonstrated its effectiveness in curing low-risk and intermediate-risk PCa as well as high-risk as a boost. However, as it is a lengthy process and many expertise involved at different steps, the outcome and morbidity rate is evaluated by patient follow up records and recurrence state. The accuracy of the entire treatment is, in fact, as a result of the accuracies in every step, such as accuracies of the procedures in the PCa diagnosis and staging, treatment planning, intra-operative, and postoperative steps. Therefore, it might need careful interrelationship among these stages for the efficient treatment of PCa.

For example, due to the inevitable error-prone stage of intra-operative implantation, the American Brachytherapy Society (ABS) recommends that every patient (who has received LDR-BR-based PCa treatment) has to receive a postimplant assessment [START_REF] Davis | American brachytherapy society consensus guidelines for transrectal ultrasound-guided permanent prostate brachytherapy[END_REF]. In this regard, as mentioned before, CT images are the choice of modality to detect seed positions to compute the delivered dose distribution. It can be done the implant day or the radionuclide-specific day optimized for decay and edema resolution after implantation. In particular, day zero dosimetry analysis is often typical for Cs-131 and Pd-103

implants. Dosimetry analysis one month after the implantation is conventional for I-125 implants.

Unfortunately, cancer can locally relapse, even if it is rare (about 2-10%). Though the recurrence of cancer may or may not be associated with the challenges in LDR-BR procedures, one can not omit the contributions of the uncertainties in LDR-BR procedures. The most common uncertainties emerge either in the intra-operative stage, such as in the seed delivering process and contouring of the target volumes or in the postimplant dose distribution analysis, such as identifying the implanted radioactive seeds and contouring of target volumes.

Ideally, for example, the seeds are inserted in parallel by aligning them in the needle insertion direction either in stranded or loose (seen in Figure 2.4). In practice, however, the seed placement depends on biomechanical factors such as the gland's motion due to the needle forces, inevitable patient motion, and soft tissue deformation, including edema and possible intra-operative changes. Consequently, the seeds might be placed outside of the targeted position despite the special care when placing the needles

Salvage low dose rate brachytherapy

and delivering the seeds [START_REF] Davis | American brachytherapy society consensus guidelines for transrectal ultrasound-guided permanent prostate brachytherapy[END_REF]. These scenarios might result in the dropping off the seeds out of the body during urination and migration of seeds to other organs. Moreover, closely spaced seeds appear as joined seeds in the postimplant CT-exams, making the seed detection challenging.

Another source of uncertainty, if not a lengthy process, is in contouring of the prostate CTV on TRUS and CT images. Accurate prostate volume contouring would help deliver sufficient radiation dose to the prostate volume while minimizing it to the urethra and the surrounding tissues, such as the bladder and rectum. However, failing to delineate the prostate boundary accurately might lead to unnecessary morbidity. For example, overestimating the volume of the prostate can result in rectal wall toxicity. On the other hand, underestimating the prostate volume can lead to under-treatment.

Moreover, CTV contouring is performed at different steps of the LDR-BR using different medical images. The standards are TRUS images for the treatment planning and the intra-operative steps, and CT for the postoperative dosimetry evaluation. However, CTV contouring is cumbersome, challenging, prone to subjective errors, and a time-consuming task. This is worst in the case of TRUS images because these images are affected by speckles, shadowing, and even the prostate's boundary is with low contrast , particularity at the base and the apex of the gland. Prostate delineation on CT images is also challenging as there is low contrast between the prostate's boundary and the surrounding soft tissues.

As mentioned before, the CT and TRUS images are often associated with low contrast imaging of the soft tissues, particularly the prostate zones. Thus, it is not easy to accurately delineate the CTV on these images, but can be solved using MRI. However, as of current clinical practices, MRI modalities

are not yet implemented in the daily routine of prostate brachytherapy procedures.

Therefore, the LDR-BR workflow and the treatment outcomes can highly benefit from automating the procedures, such as automatic contouring of the prostate and automatic detecting of the implanted radioactive seeds.

Salvage low dose rate brachytherapy

In a conventional clinical workflow, a patient follow-up is often necessary after the first LDR-BR procedures to study the status of PCa and thereby analyze the treatment outcomes. When the PCa locally recurs, a second LDR-BR (often named as salvage LDR-BR) can be the preferred PCa treatment technique [19] [20]. The procedures of salvage LDR-BR are similar to the primary LDR-BR procedures, except in the postimplant analysis. Indeed, in the salvage LDR-BR the implantation is done close to the locally relapsed cancer, unlike in the primary LDR-BR that considers the entire prostate volume.

Postimplant CT-exam of salvage LDR-BR is composed of radioactive seeds from both the primary and the salvage LDR-BR. However, radiation oncologists should make dosimetric analysis only from the radioactive seeds implanted during salvage LDR-BR. Indeed, the radiation lifetime of the primarily implanted seeds mostly expires by the time of salvage LDR-BR. Thus, radiologists should not consider the dose from the primary seeds while performing postimplant dose analysis.

Besides the challenges mentioned in the dosimetry analysis of primary LDR-BR (section 2.4), performing a postimplant dosimetry assessment on salvage LDR-BR poses another difficulty. It is required to differentiate the cumulated seeds based on their implantation time, i.e., from the primary or the salvage LDR-BR. Among the main factors contributing to the challenges in detecting and identifying salvage seeds in postimplant CT-exams include:

• Prostate volume changes over time. Crehange et al. [START_REF] Créhange | Cold spot mapping inferred from mri at time of failure predicts biopsy-proven local failure after permanent seed brachytherapy in prostate cancer patients: Implications for focal salvage brachytherapy[END_REF] found that the postimplant prostate volumetric change over time and concluded that prostate volume decreases through time. Since the radioactive seeds are placed within the prostate, their position and orientation depend on the shrinking momentum of prostate tissue. Consequently, the position and orientation obtained using CT-examination from the first LDR-BR may not be the same as in the salvage LDR-BR CT-examination.

• Implanted radioactive seeds have almost the same size and shape, regardless of the moment of implantation. Visually differentiating the seeds on CT exams is almost impossible.

• Implanted radioactive seeds drift over time. Pinkawa et al. [START_REF] Pinkawa | Evaluation of source displacement and dose-volume changes after permanent prostate brachytherapy with stranded seeds[END_REF] shown that the position of radioactive seed changes through time. Hence the independent displacement of each seed makes identification more complicated.

• Implanted radioactive seeds can appear overlapped. Practically, in postimplant CT-exams, the seeds appear overlapped with each other in random orientations, making the detection and

Deep learning in medical imaging

recognition of each seed more challenging.

• Seeds migrate to other tissues or drop out through urination [START_REF] Tapen | Reduction of radioactive seed embolization to the lung following prostate brachytherapy[END_REF]. The number of implanted and detected seeds might not be the same, leading to the uncertainties in detecting the seeds from the postimplant CT images.

• Radiation oncologists could use the CT images obtained at the postimplant of the first primary LDR-BR and the salvage LDR-BR to identify the salvage seeds. However, these CT images are often acquired at different intervals of time and potentially different imaging setup. Specifically, their image thickness and pixel spacing position could be different. Therefore, nonrigid registration must be used to align these CT images.

The above-listed problems can be grouped into subproblems related to segmentation algorithms, subproblems related to classification, and finally, subproblems related to registration and recognition.

Thus we will consider these three subproblems to perform accurate dosimetry analysis to improve optimal patient care. We now explore the primary concepts, detailed formalisms, and notions of deep learning methods in the context of medical image analysis. These concepts would provide an introduction to our works regarding the development of automatic methods for medical image analysis using DL methods.

Deep learning in medical imaging

This section introduces the basic concepts and notions commonly used in DL methods. Please refer to the work of Bishop [START_REF] Bishop | Pattern recognition and machine learning[END_REF] for more detailed concepts of DL methods. Moreover, we discussed the trends and applications of DL methods in radiology [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF].

Recently, artificial intelligence (AI) has become again a topic of interest in industries and academic applications. Generally, AI is defined as the science and engineering of making intelligent machines, especially intelligent computer programs that exhibit characteristics associated with intelligence in human behavior, including among other faculties of reasoning, learning, goal seeking, problem-solving, and adaptability as defined in the work of Monostori [START_REF] Monostori | Artificial Intelligence[END_REF]. Moreover, a computer system that performs data-driven tasks, which in turn requires human intelligence such a visual perception, speech recognition, decision-making, and translation between languages, can be referred to AI. In various tasks AI system has surpassed humans [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF]. In fact, AI, based on (DL) methods, has become an integral part of different automated systems such as in search engines to find images on the web or talk to digital assistants on smartphones and home entertainment systems [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF].

Machine learning (ML) is a subfield of AI. The DL is the subfield of ML [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF]. The availability of big data, advances in DL algorithms, and processing power (dedicated devices) are the driving factors that revolutionized the DL applications. DL is composed of artificial neural networks (ANNs), a rough artificial model of the brain neurons (seen in Figure 2.7).

Artificial neural networks

Here we introduce the main concepts and fundamental blocks in ANNs. An artificial neural network A given ANN can be considered as a two-class model (defined here as perceptron). First, the input vector x is transformed using linear combinations of fixed nonlinear basis functions φ(x) [START_REF] Bishop | Pattern recognition and machine learning[END_REF]: The superscript of w denotes the corresponding parameters (neural network weights) are of the given l layer. Moreover, the subscripts ji denotes the flow of information from node i to node j. Note that each hidden and output unit has an associated bias parameter, but we omitted for clarity in this figure .   allows for any fixed offset in the data. Equation 2.1 can be rewritten as:

y(x, w) = f N i=0 wiφi(x) (2
y(x, w) = f w T φ(x) (2.
= M i=1 uif (w T i x + w b,i ) (2.3)
where ui is the combining weight of the neurons. Here, the neural network's parameters θ = (u1, w b,1 , ..., uM , w b,M , wM ) T are trainable. To do so, the true output y and the networks approximated output ŷ are compared. Therefore, neural networks are a general class of parameter nonlinear functions form a vector x of input variables to a vector y of output variables. An example of combined ANN layers is shown in Figure 2.8. The term layer refers to the collection of ANN nodes operating together at a specific depth within a neural network. For example, figure 2.8 has 4 layers. Layer 1 is the input layer, and layer 4 is the output layer, and the other layers being between the input and output are known as hidden layers, i.e., layers 2 and 3.

Neural network training

Here we discuss how the network parameters θ are generally calculated. To do so, first, a function known as an objective function (or cost function) is defined. It mainly focuses on how to compare the network's predicted output and the target output. The target output can be labeled (sometimes called ground truth or reference) or unlabeled data. Labeled data is when expertise provides a reference for a given task, while unlabeled data is when there is no explicit reference of the target. If a DL model is developed using a labeled dataset, it is known as supervised DL method. On the other hand, if it is developed using unlabeled data, it is called the unsupervised DL method. Another approach is called the weakly supervised DL method, in which only partially labeled datasets are used in the training of the DL model.

The ANN learns from a given dataset by computing and optimizing the objective function (also known as loss function). The objective function is used to evaluate how well the DL method models the given data and thereby predict the results to be the same with the ground truths. However, how are the network's parameters, θ, determined?

The network's parameters θ are computed using the concept of gradient descent [START_REF] Bottou | Large-scale machine learning with stochastic gradient descent[END_REF]. A backpropagation (or sometimes called error propagation) algorithm is used to compute the gradient descent while training [START_REF] Bishop | Pattern recognition and machine learning[END_REF], during which information alternately forwards and backward through the neural networks (see Figure 2.8).

Lets consider the outputs yi are linear combinations of the input variables xi as in eq. 2.3 and ignoring the weighting factor ui and the bias w b,i for simplicity, the equation becomes:

ŷk = M i=1 w ki xi (2.4)

Deep learning in medical imaging

If we consider the objective function is an error between the predicted values and target output values, it is defined as: with respect to a weight wij is given by

Errorm = 1 2 M k (ŷ mk -y mk ) 2 (2.
∂Errorm ∂wij = (ŷmj -ymj)xmi (2.6)
Thus, in the feed-forward network (means when the input xi propagates from the input layer to the output layer), each node computes a weighted sum of its inputs as [START_REF] Bishop | Pattern recognition and machine learning[END_REF]:

aj = i wjizi (2.7)
where zi is the activation of a node, which sends an input to other node j, and wji is the weight associated with that connection (see Figure 2.8). For example, given a feed-forward (equation 2.7) signal from node i to node j, the new node j transforms the input signal by a nonlinear activation functions f (.) as in equation 2.2 as:

zj = f (aj) (2.8)
The process of feeding all inputs to a given node and the given node into the next node, when this process is applied from the input to the output, is known as forward-propagation. After computing the error at the last layer's output, the error needs to propagate back, known as error propagation (propagation in short). The output error Errorm depends on the weight wji only through the summed input aj to node j. Applying chain rule for partial derivative in equation 2.6, it becomes:

∂Errorm ∂wji = ∂Errorm ∂aj ∂aj ∂wji (2.9)
from equation 2.7, ∂a j ∂w ji = zi, and hence the partial derivative becomes:

∂Errorm ∂wji = ∂Errorm ∂aj zi = δjzi (2.10)
where ∂Errorm ∂a j = δ. From this equation, one can see that the derivative of the error can be obtained by multiplying δ for the node at the output end of the weight by the value of z for the node at the input end of the weight. The nodes at the output layer have ŷk as output and y k as reference, hence, we can calculate δ k as:

δ k = ŷk -y k (2.11)
For the inner (hidden) nodes, we can compute them using the chain rule for partial derivatives,

δj = ∂Errorm ∂aj = k ∂Errorm ∂a k ∂a k ∂aj (2.12)
where the sum runs over all units (layers) k to which unit j sends connections. This arrangement is illustrated in Figure 2.8. We can observe that a variation in aj yields a variations in the error functions only through variations in the variables a k . Then, given the feed-forward equation 2.7, and the nonlinear activation function 2.8 and δj = ∂Errorm ∂a j

, the backpropagation formula can be written as:

δj = ∂Errorm ∂aj = k ∂Errorm ∂a k ∂a k ∂aj = f (aj) k w kj δ k (2.13)
For the last layer's output nodes, we know the values of δ's. Therefore, applying recursively backward direction from the output layer through the hidden layer until the input layer in a feed-forward network enables the network to backpropagate the error. Thus, the term backpropagation comes from the fact that we start computing the error at the output layer and go back for each layer.

Given a training set (i.e knowing the input vector xm and the output vector y m ), the backpropagation algorithm can be summarized as [START_REF] Bishop | Pattern recognition and machine learning[END_REF]:

1. Perform forward propagation to compute the activations of all the hidden and output nodes using feed-forward equation 2.7 and the nonlinear activation function 2.8.

2. Using equation 2.11, compute δ k for all the output units.

3. Compute all δ's using equation 2.13 to obtain δj for every hidden unit in the network.

4. Compute the required derivatives using equation 2.10.

For m batch methods, the total error derivative is the linear sum of individual errors for each pattern in the training set that results in:

∂Error ∂wji = m ∂Errorm ∂wji (2.14)

Convolutional neural networks

One approach of extracting image features invariant to image translation and scaling is using convolutional neural networks (CNNs). CNNs are multilayer perceptrons, where each neuron in one layer is connected to all neurons in the next layer. Note that a neuron's input area is known as the receptive field and is fully connected layers, such as CNNs, the receptive field is the entire previous layer [START_REF] Bishop | Pattern recognition and machine learning[END_REF].

In the convolutional layer, the nodes are arranged into planes, named the feature maps. Another distinguishing feature of CNNs is that a given layer's neurons can share the same neural network weights and biases. This enables CNNs to be memory efficient by sharing a single vector of weight and bias, instead of having an individual bias and vector weighting for each receptive field. For example, let us consider a feature map consists of 100 units organized in a 10 × 10 grid, assuming each node taking Another essential layer in CNNs is the subsampling layer (sometimes called pooling layers). From each convolutional layer's feature map, corresponding sub-sampling layers are followed. Thus, each subsampling layer takes inputs from a small receptive field in the convolutional layer's corresponding feature map. When the subsampling is performed on small receptive fields of convolutional layers, it is called local subsampling or pooling. On the other hand, when the receptive field is formed, all the convolutional layer neurons are called global subsampling or pooling. Most used pooling operations are maximum or average, selecting the maximum value or average value from each subregion or the entire convolutional layers, respectively, for local and global subsampling. Figure 2.10 illustrates both max and average pooling operations with a 2 x 2 filter size and stride of 2. Stride is the number of pixels by which we move the filter matrix over the input feature map. For example, a stride of 1 is moving the filter matrix one pixel at a time, and stride of 2 is then moving the filter 2 pixels at a time. Hence, smaller stride yields large feature maps, while larger stride yields smaller feature maps. Thus, using local receptive fields reduces the network's number of weights compared to the fully connected networks. Furthermore, the pooling operation reduces the dimension of the feature map yielding an improved computational efficiency. Therefore, the three specific features of CNNs are using local receptive fields, weight sharing, and sub-sampling mechanisms.

Deep learning in medical imaging

Training, validation, and testing

In supervised DL methods, the given labeled datasets are often divided into training, validation, and testing data, as shown in Figure 2.11. However, this data division technique into three independent datasets is an ideal case where we have a large number of annotated datasets. The division often to the medical image dataset. In the fine-tuning approach, the ANN's weights are retrained from the new annotated imaging domain [START_REF] Gu | Ce-net: Context encoder network for 2d medical image segmentation[END_REF], while the pre-trained weights are used as initialization weights.

The conventional techniques used to expand limited medical image datasets in developing DL methods include data augmentation, data annotation via mining text reports, data annotation via active learning, domain adaptation, and data synthesis via generative adversarial networks [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF].

In the last few years, DL methods have successfully been utilized in various radiotherapy tasks.

For example, several DL methods have been studied for radiotherapy treatment step, computer-aid diagnosis, image registration, treatment planning, and outcome prediction [START_REF] Meyer | Survey on deep learning for radiotherapy[END_REF]. The patient workflow in radiotherapy is often complicated, involving several steps. In this regard, deep learning-based automatic methods can help to facilitate and improve the clinical workflow. However, despite the excellent progress of DL methods in medical image analysis, it is rarely used in clinical routines [START_REF] Sahiner | Deep learning in medical imaging and radiation therapy[END_REF][START_REF] Meyer | Survey on deep learning for radiotherapy[END_REF][START_REF] Boldrini | Deep learning: a review for the radiation oncologist[END_REF].

Therefore, in our work, we have developed both semiautomatic and automatic DL methods that can be used to facilitate the image-guided LDR-BR procedures, thereby helping experts to extract clinically useful information and analyze the clinical outcomes as well as speed up the contouring of target organs.

Databases and software frameworks

The clinical materials used in our work were collected from CGFL, Dijon, and Institut Curie, Paris,

France. Moreover, in some of our proposed medical image processing methods, we have also used publicly available datasets to demonstrate and compare our method's performance with other states of the art image processing methods. Here, we briefly introduce the primary datasets used in this thesis.

A detailed description of each dataset will be provided in the corresponding chapters.

Computed tomography images

These datasets were collected from CGFL, Dijon, and Institut Curie, Paris, France. The in-plane resolution varies from 0.39 × 0.39 mm 2 to 0.58 × 0.58 mm 2 with a slice thickness of 2.5 mm helical mode, 120 kVp, 172 mm FOV, and 440 mAs/slice. These CT-exams were obtained from male cases who underwent at least primary LDR prostate brachytherapy treatment with iodine-125. A radiation oncologist then used a VaeeriSeed planning software (Varian Medical Systems Inc, Palo Alto, CA) to delineate corresponding clinical target contours, which is then used to analyze the postimplant dose distribution of the treatment. Hence, these contours and dosimetry analyses are used as references in this thesis. Moreover, CT images of both postimplant primary and salvage LDR-BR are used to detect the implanted radioactive seeds and then identify implanted radioactive seeds in the salvage LDR-BR.

Databases and software frameworks

Transrectal ultrasound images

These datasets were collected at the anticancer center of Dijon (CGFL), France. The TRUS images were acquired during the intra-operative procedures of male patients who underwent LDR-BR prostate cancer treatment under TRUS guidance. In each case, a series of transverse 2D TRUS images were acquired at 1 mm intervals. The TRUS probe was placed on a stepper, which enabled us to move along the transverse axis to acquire parallel transverse slices, allowing the entire prostate volume to be covered in 3D. The pixel size of each transverse slice was 0.1038 × 0.1038 mm 2 . In these images, corresponding prostate clinical target volume (CTV) was delineated by radiation oncologist using a

VariSeed planning software (Varian Medical Systems Inc, Palo Alto, CA). These contours are then used as references to compare our proposed deep learning methods.

Other datasets

Other used publicly available datasets that only require online registration to access them are the Automated Cardiac Diagnosis (ACDC) MICCAI challenge in 2017 [START_REF] Bernard | Deep learning techniques for automatic mri cardiac multistructures segmentation and diagnosis: is the problem solved?[END_REF], the Multi-structure Ultrasound Segmentation (CAMUS) [START_REF] Leclerc | Deep learning for segmentation using an open large-scale dataset in 2d echocardiography[END_REF], and the Hear-EU cochlear dataset [START_REF] Gerber | A multiscale imaging and modelling dataset of the human inner ear[END_REF]. These datasets were used to evaluate and verify our method's performance for different medical image segmentation applications, other than the single prostate structure segmentation.

Software frameworks

Different software environments have been used to develop the methods proposed in this work. The deep learning methods were implemented on an i7 computer with 32-GB RAM and a dedicated GPU (NVIDIA TITAN X, 12 GB, and NVIDIA TITAN V, 12 GB). Implementation and development of the proposed methods were done in Python, Keras API, with Tensorflow backend.

Moreover, the radioactive seed detection and identification methods were developed in the MATLAB environment.

Chapter 3

Medical imaging and computer vision trends in LDR-BR for PCa

As introduced in the previous chapter, LDR-BR has been widely accepted for the treatment of earlystage PCa. It involves successive sessions, different medical imaging modalities, and experts from different specializations. Although each session might be different among clinical centers, summarizing the developed image processing algorithms that aim to aid radiologists in the image-guided LDR-BR procedures in a single review article would help to have a global overview of the different approaches.

Moreover, reviewing all algorithms used across the treatment procedures and studying the trade-off in considering the different medical imaging and medical image processing methods might be a first step towards developing a consensual LDR-BR process. With these premises, in this work, we conducted a review of the available relevant articles focusing on the contributions of the medical imaging and image processing methods for LDR-BR procedures for PCa. The review will present the current techniques, highlight their drawbacks, and suggest future directions that might improve the radioactive seed management systems. This review paper is under submission, as in the following paper: Automatic seed identification for dosimetry analysis on CT images

In this chapter, we propose a new method to evaluate the dose distribution (dosimetry) of salvage LDR-BR performed years after primary LDR-BR for the treatment of relapsed PCa. To this end, we will present an image processing methods to detect the implanted seeds on postimplant CT-images.

A landmark-based registration approach will also be presented to identify the seeds implanted during salvage LDR-BR from the seeds implanted during the primary LDR-BR on postimplant CT images.

Finally, the dosimetry parameters of the radioactive seeds implanted in salvage LDR-BR will be reported. This approach allows us to evaluate the salvage LDR-BR implantation quality. We test the performance on six patients who underwent salvage LDR-BR for the treatment of relapsed PCa using iodine-125 seeds.

The proposed method has been published in the following paper [START_REF] Girum | Inferring postimplant dose distribution of salvage permanent prostate implant (ppi) after primary ppi on ct images[END_REF]: Chapter 5: Interactive and automatic deep learning methods for image-guided LDR-BR procedures 34

K.B.

Interactive medical image annotation

In this section, we present an interactive DL method for medical image segmentation. The method's central idea is motivated by the techniques experts use while manually delineating a given target organ on an image. In fact, experts delineate a given target organ on an image considering both their prior knowledge of the target, particularly the shape prior, and the intensity distribution of the given medical image. Similarly, our method employs prior shape prediction from pseudo-landmarks indicated by the user and the intensity distribution information from the given input image. Our approach has been evaluated on intra-operative and post-operative, respectively, TRUS and CT images of imageguided prostate LDR-BR. The evaluation has also been extended to multi-structure segmentation from echocardiographic images. The proposed method has been published in the following paper [START_REF] Girum | Fast interactive medical image segmentation with weakly supervised deep learning method[END_REF]:

Girum KB, Créhange G, Hussain R, Lalande A.

Fast interactive medical image segmentation with weakly supervised deep learning method.

International Journal of Computer Assisted Radiology and Surgery.

Vo. 15, no. 9, pp. 1437-144, 2020.

DOI: https://doi.org/10.1007/s11548-020-02223-x

Interactive medical image annotation

Discussion

The proposed interactive DL method showed promising results for different medical applications. First, it can be used to delineate accurately medical images with minimum manual inputs. For example, precise prostate delineation on CT and TRUS images can be achieved using our interactive segmentation approach. Secondly, the proposed method can be used for domain adaptation applications. In particular, the proposed method can be trained from available annotated datasets and then applied it to other applications with a small annotated dataset but for the same target of either the same imaging modality with different acquisition settings or a different modality. This application is validated using prostate CTV segmentation from TRUS and CT images and vice-versa by considering one imaging modality as available annotated data and the other modality as scarce data. Although the prostate shape is not precisely the same between the two modalities, it could be considered deformed shapes of each other's modalities. Other areas of application can be to easily adapt a given developed DL method to inter clinics and new image acquisition settings. It is well known that the current DL methods suffer from the generalization problem. This produces a scenario in which a developed DL model performs poorly when the testing data are out of the training dataset distribution. Thus the proposed method can be used to annotate new images quickly. The annotated new images from the same or different centers can then use to retrain the developed DL method. By doing so, the proposed method continuously learns from new imaging domains or new image acquisition settings. Moreover, we believe that intermediate experts might select the pseudo-landmarks to annotate larger datasets.

In this work, we presented a single network for the prostate segmentation from TRUS and CT images as well as echocardiographic image segmentation applications. As per the multi-structure segmentation, we used a pseudo-landmarks from a single structure (using only the four contour landmarks at the myocardium) to generate multiple structures. Although we presented the segmentation results of all imaging modalities from a single network for simplicity, the segmentation accuracy for the multistructure could be improved by increasing the depth of the proposed method's segmentation network.

The main limitation of the proposed method lays in introducing the landmarks manually. The landmarks could be modeled using variational auto-encoders or by first applying a segmentation method to get the target's approximated mask. From the automatically segmented mask, extract the pseudolandmarks and automatically generate the target's prior shape knowledge. Moreover, despite the method's promising results in its 2D version, it has computational overhead from manual interaction.

Therefore, enhancing the method to the 3D model could allow reducing the required number of landmarks a user has to pinpoint. For example, in the case of prostate segmentation, considering four pseudo-boundary landmarks at the left, right, top, and bottom points of the prostate gland on a given 3D image could be used to generate the shape prior. Next, comparing a fully automatic supervised DL method trained from manually annotated datasets and interactively annotated datasets using our method would be necessary.

As will be presented in the next section, based on the premises of this interactive approach, we developed a fully automatic DL method. We applied it to TRUS image segmentation in the intraoperative prostate LDR-BR.

Chapter 5: Interactive and automatic deep learning methods for image-guided LDR-BR procedures 36

5.2 Automatic ultrasound image segmentation in LDR-

BR

This section details the proposed fully automatic DL method for the prostate CTV segmentation from intra-operative TRUS images. The central hypothesis of our method is that developing a learning-based shape prior to modeling with learning-based pixel intensity distribution classification in parallel can be an appropriate approach for low contrast image segmentation, such as TRUS images.

The proposed method employs an anatomical structure reconstruction from two aspects. Firstly, to be less dependent on the low contrast and artifacts signal across the full organ's boundary, we employ a learning-based shape prior reconstruction from automatically sampled boundary coordinates (defined as pseudo-landmarks) of the prostate gland using a deep neural network. Secondly, hierarchical features learned directly from the input images are employed for pixel classification using U-net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]. The initial developed method and results from applying it in prostate segmentation from CT and MR images were presented in [START_REF] Girum | Deep generative modeldriven multimodal prostate segmentation in radiotherapy[END_REF]. Please refer to the appendix B for the paper. We focused on using the MR images to learn the shape prior reconstruction from the automatically sampled boundary coordinates.

The learned shape prior reconstruction model is then applied to CT image segmentation. To this end, it enabled us to transfer the shape prior reconstruction process between the two modalities. In continuation of this method, we applied it to the intraoperative TRUS image segmentation. However, we used only the TRUS images to train the shape prior reconstruction model. As future work, we will consider a trained shape prior reconstruction model from MR images. Indeed, segmentation of the prostate on MR images is more reliable than the segmentation on TRUS and CT images. As discussed in chapter 3, prostate segmentation from MRI is also less prone to inter and intra-observer variations.

The developed method has been evaluated in a clinical databases of 145 cases who underwent LDR-BR under TRUS guidance. It has also been compared with other states of the art DL methods. Please refer to the appendix C for details of the used metrics, loss functions, and network architectures. The proposed method has been published in the following paper [START_REF] Girum | A deep learning method for real-time intraoperative us image segmentation in prostate brachytherapy[END_REF]:

Girum KB, Lalande A, Hussain R, Créhange G.

A deep learning method for real-time intraoperative US image segmentation in prostate brachytherapy.

International Journal of Computer Assisted Radiology and Surgery.

Automatic ultrasound image segmentation in LDR-BR

Discussion

The proposed shape learning-based segmentation method has achieved promising results for prostate CTV segmentation from intraoperative TRUS images. Moreover, the channel wise feature calibration strategy (i.e., squeeze-and-excitation network [START_REF] Hu | Squeeze-and-excitation networks[END_REF]) has appeared to increase segmentation accuracy.

Another advantage of our proposed DL method is the real-time feature to segment the prostate in intraoperative TRUS images. Indeed the real-time aspect is mandatory in this kind of procedure.

However, to model the spatial-temporal information in the real-time implantation procedures, further developments should be investigated using synthetic prostate images. The spatial-temporal information can also be learned using a long short-term memory approach [START_REF] Hines | The neuron simulation environment[END_REF]. Furthermore, the proposed method needs to be validated from MR images corresponding to the TRUS images. As the prostate segmentation in TRUS images is often prone to inter and intra-observer variations, the developed ground truth database could be biased.

Using the works presented in this chapter as input and addressing the limitations of CNNs-based segmentation in producing incomplete and unrealistic segmentation results, we developed a new DL method that can be applied for accurate and robust medical image segmentation with strong anatomical plausibility. The method is then evaluated on different medical image segmentation applications, including prostate segmentation from CT images, cardiac segmentation from short-axis cine-MRI and long-axis echocardiographic images, and inner ear segmentation form µCT images. Note that the cardiac segmentation from MRI and ultrasound images and inner ear segmentation from µCT images were included in this thesis as an extension of the initial objectives. They are used to validate our proposed method's efficiency on different organs and imaging modalities. Thus, analogously in the future, the proposed methods could be further applied to segment male pelvic structures such as the rectum, bladder, and prostate. Moreover, they might also be used to segment the multi-structure prostate zones.

The proposed method is then presented in the following chapter. However, the current DL methods never get a second chance to see the segmentation results [START_REF] Bishop | Pattern recognition and machine learning[END_REF].

To address these limitations, in this chapter, we present new DL methods by modeling the image segmentation problem as a two networks task. To this end, in the first part of this chapter, we present a DL method that involves two networks that function iteratively. While the first network is designed to segment a given input image from a defined pixel-wise objective function, the second network aims to provide the first network a second chance to look back to the predicted results. Therefore, the second network regularizes the first network's training process by feeding back its results, thereby improving segmentation quality over time. In the second part of the chapter, we present a general-purpose post-processing method. It is developed for the post-processing of any given DL network for medical image segmentation. It employs a 3D generative adversarial neural network (GAN) [START_REF] Goodfellow | Generative adversarial nets[END_REF]. Therefore, the proposed post-processing method aimed to be plugged after any DL methods. It would help to refine the results by using the predicted probabilities outputs from a given network.

Learning with context feedback loop for robust medical image segmentation

The recent DL methods learn discriminative image features automatically using CNNs. However, they are prone to produce unrealistic image segmentation results. Mainly, they tend to produce results with single or multiple unexpected holes, over segmented structures, under segmented structures, and yield anatomically incorrect results. Most of these errors are not prevalent in manual segmentation. It is because DL methods based on CNNs emphasize on learning texture features from a pixel-wise objective function [START_REF] Geirhos | Imagenettrained cnns are biased towards texture; increasing shape bias improves accuracy and robustness[END_REF].

To alleviate this problem, studies have focused on either incorporating shape prior that constrains the segmentation processes [START_REF] Oktay | Anatomically constrained neural networks (acnns): application to cardiac image enhancement and segmentation[END_REF] or using post-processing methods, for example, using auto-encoder networks [START_REF] Larrazabal | Post-dae: Anatomically plausible segmentation via post-processing with denoising autoencoders[END_REF]. However, these methods require separate modeling of the shape of the target and might not work well for irregularly shaped structures. Moreover, post-processing approaches require additional work for the modeling of the shape and are greatly influenced by the results from the first network. Most important, most post-processing methods tend to produce plausible results. However, it should be noted that plausible results are not always necessarily accurate.

In this chapter, we introduce a fully automatic DL method for accurate and robust medical image segmentation by formulating the segmentation problem as a two systems task. It employs two interconnected networks via feedback loop-based image segmentation approach. It has been evaluated on multiple datasets, in particular on prostate segmentation from CT images, inner ear segmentation from µCT images, cardiac segmentation from echocardiography and cine-MR images. Our method yielded both accurate and plausible results, and the plausibility was achieved without the need to incorporate shape prior or apply post-processing method.

The proposed method has been submitted as in the following paper, and please refer to the appendix D for more experimental results. The proposed method showed consistent accuracy improvement over state of the art DL methods.

The performance was measured using both volumetric and distance metrics. It performed well for single and multi-structure segmentation. Several experiments have also been performed to inspect the method's performance, including the ablation study in the merging strategy, the training scheme, and the network configurations. The experimental results demonstrate segmentation with the introduced feedback loop approach increase accuracy on all target structures, but significantly on the complex structures.

A critical aspect of our method is that it produces segmentation results that are similar throughout the testing datasets. The measured standard deviation of both the Dice index and Hausdorff distance error values among the testing sets was very small compared to the other state of the art DL methods.

Therefore, based on these results, we can conclude that the proposed method is robust to segment low contrast medical images. In control theory, the feedback loop is a critical part of a closed system where one uses the output signal to make adjustments in the input signal. Thereby, it improves the robustness of the system to outliers and produces the desired output. Analogously, our method tries to mimic the feedback looping strategy used in the control theory.

In the future, we will apply the proposed method for the prostate CTV segmentation from intraoperative TRUS images and compare it with the proposed shape-prior model-based segmentation in chapter 5. Moreover, we will explore conditions for maximizing the integration strategy of the two systems through their latent space. In cases of available large annotated datasets, a 3D version of the proposed method can indeed be used to have accurate segmentation results.

Another solution to correct the often incomplete and anatomically invalid segmentation results produced by a given automatic DL-based medical image segmentation is to apply post-processing methods. In this regard, in the following section, we proposed and evaluated an adversarial neural network-based post-processing approach for anatomically regularized volume segmentation, which can then be used to refine the probabilistic output of a given 2D or 3D neural network.

3D Adversarial neural networks for anatomically regularized image segmentation

Motivated by our previous works, in particular, our method presented in section 6.1, we proposed a general-purpose DL method for post-processing application of any given segmentation network. It aims to alleviate the current challenges of DL method-based automatic medical image segmentation techniques, which are often prone to producing incomplete and anatomically invalid segmentation results.

Meanwhile, different studies have shown improved segmentation results by applying different postprocessing methods. Convolutional auto-encoders [START_REF] Masci | Stacked convolutional auto-encoders for hierarchical feature extraction[END_REF] based on either denoising auto-encoder [START_REF] Vincent | Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion[END_REF] or variational auto-encoder [51] [START_REF] Painchaud | Cardiac segmentation with strong anatomical guarantees[END_REF] have been proposed as post-processing approaches. All these approaches consider the segmentation results of any given segmentation network. Artificially degrading the ground truth has also been proposed to model the error that could come from a given network.

These approaches generally showed promising results [START_REF] Larrazabal | Post-dae: Anatomically plausible segmentation via post-processing with denoising autoencoders[END_REF].

However, modeling the errors that could come from a given network is complicated and sometimes impossible. Moreover, another limitation of these approaches is that they do not consider the segmentation network's probabilistic output. Because a given network produces a segmentation result with some confidence or probabilistic level, using the probabilistic output would enhance the performance of the post-processing methods.

The other perspective of our approach is that, using the probabilistic intermediate values of a network while training would be beneficial to have more datasets. It can be assumed as data augmentation technique.

Therefore, in this chapter, we presented a new deep learning method for post-processing a given DL method by using its predicted probabilistic output. It employs a 3D adversarial neural network to regularize the segmentation process, which can refine the probabilistic output of any 2D or 3D neural network for any given application. It has been evaluated on prostate segmentation from TRUS images and cine-MR image segmentation applications. We have also performed extensive ablation studies as well as comparisons with the state of the art method.

The proposed method has been submitted as in the following paper:

Girum KB, Créhange G, Lalande A.

3D Adversarial neural networks for anatomically regularized image segmentation.

IEEE journal of biomedical and health informatics. 2020 6.2 3D Adversarial neural networks for anatomically regularized image segmentation

Discussion

The proposed post-processing method uses a generative adversarial neural network. It is based on the hypothesis that a given deep learning-based segmentation network outputs a probabilistic pixel's values. Thus, using the initial segmentation network's probabilistic output could capture potential uncertainties produced on a given input image.

We demonstrated the usefulness and advantages of our approach on TRUS datasets. As an extension, we have also experimented on cardiac cine-MRI datasets. These datasets have different shapes with single and multi-label structures. The experimental results, on both TRUS and cardiac cine-MRI datasets, reveal that the proposed method outperforms the state of the art denoising algorithm for both the datasets demonstrating the advantages of incorporating a 3D adversarial neural network as a post-processing step.

The proposed method can bring erroneous segmentation results produced by a given deep learning method to the corrected and plausible results. It introduces a negligible computational cost. 3D-based based post-processing is preferred that can enable to capture of the full geometry of anatomical target structures. However, depending on the available datasets, additional pre-processing steps might require.

For example, in the cardiac cine-MRI segmentation, the provided dataset has a different number of slices. It thus required an additional interpolation technique to resize into the same dimension. Indeed deep learning methods require to have the same input image's dimensions. In such scenarios, the 2D-version of the proposed method is preferable.

Post-processing approaches generally rely on the output of a given segmentation method. They do not get a chance to look at the input image, making them highly dependent on the segmentation method's accuracy. For example, in our work, to evaluate the post-processing method's effect and compare it with another state of the art method [START_REF] Larrazabal | Post-dae: Anatomically plausible segmentation via post-processing with denoising autoencoders[END_REF], we considered an arbitrary segmentation method (i.e., U-net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]). We did not consider to optimize the given segmentation network. Thus, in the future, integrating the proposed adversarial neural network-based post-processing with the segmentation network through a feedback loop, similar to the method presented in the previous section, would improve the performance.

annotated data in the medical imaging domain is limited. Moreover, current deep learning methods are prone to produce unrealistic image segmentation results. To solve these problems, we developed two deep learning approaches. The first one is a fast and interactive deep learning method used to segment medical images accurately. Moreover, it can also be used to generate an annotated dataset with minimal manual interaction in medical image segmentation applications. The method was tested in intraoperative TRUS and postoperative CT images of prostate LDR-BR, and 2D echocardiographic image segmentation applications. It showed promising results in producing accurate results to solve the bottleneck of deep learning methods in adapting to inter clinical variations and speeding-up the annotation process. Secondly, we developed a fully automatic deep learning method for accurate and robust medical image segmentation by formulating the segmentation problem as a two systems task.

The proposed approach is then extensively evaluated on private as well as on public datasets. In particular, we used short-axis cine-MRI, long-axis echocardiographic images, CT of the prostate, and CT of the inner ear segmentation applications. The proposed method yielded promising results leading to an essential perspective for efficient and accurate medical image analysis tasks. It can also be applied to automate the management of image-guided prostate brachytherapy procedures.

In summary, in this work, novel methods have been presented. First, a new technique has been proposed to perform dose distribution analysis of a salvage LDR-BR performed years after primary LDR-BR using machine learning techniques for image processing, as introduced in chapter 4. In continuation of our work, the following interesting areas of research can be suggested:

• The proposed radioactive seed identification method essentially uses a pipeline of image processing methods such as segmentation, registration, interpolation, and clustering algorithms. The registration method involves a semi-automatic based landmark selection. To render the landmark selection automatic, a large dataset is required with a manually identified radioactive seeds for validation. Machine learning methods could then be used to learn the patterns of the seeds.

• Commercial software (Varian Medical Systems Inc, Palo Alto, CA) has been used to analyses the dosimetry by manually delineating the target structures, including the prostate and the rectum on CT images. Target organ delineation is challenging in the context of CT images because of the poor visibility of the soft tissues, such as prostate contour. Manual delineation of the targets is thus prone to errors or variations between experts. To alleviate such a problem, the delineation should be done on MR images. By its imaging characteristics, there is a better contrast between soft tissues on MRI (specifically of the prostate boundary), particularly on T2-weighted images.

• The principal application of this work was to analyze the dosimetry of cases who have received a salvage LDR-BR years after they had a primary LDR-BR in the treatment of relapsed PCa.

Thus, using the proposed method to identify the radioactive seeds and superimposing them with the relapsed tumor areas from multi-modal imaging approaches (such as merging of information from CT and PET or MRI) would help better analyze the tumor outcomes. Indeed, it could help to determine where PCa relapses occur and investigate solutions for such conditions.

In the second contribution, regarding automating some components of the current LDR-BR procedures for PCa, we proposed new deep learning methods. We have also extended the validation of these methods to other medical image analysis applications and compared with the current state of the art deep learning methods. In continuation of the proposed methods, the following research areas can be suggested as follows:

• The proposed deep learning method for prostate CTV segmentation from CT images, presented in chapter 6, can be used to propose a region of interest for the seed identification method presented in chapter 4. It could improve the seed detection approach by providing a region where the seeds are located, i.e., the prostate gland. This approach would avoid the scenario in which calcifications could be detected as seeds besides the lower processing time for seed identification on a given small region of interest.

• The proposed interactive deep learning framework for medical image annotation (presented in section 5.1) uses a concept of modeling the prior knowledge of a target from pseudo-landmark inputs by the user and the intensity information from the input image. The framework is efficient, fast, reliable, and can be used for domain adaptation. However, considering the following points can improve the proposed method's application and accuracy. First, in our experiment, we automatically extracted the user inputs from the available ground truth, assuming they are pinpointed from the users. However, this could lead the method to over-fit the created databases.

Therefore, cross-validating the proposed method from using pseudo-landmarks inputted from different experts at a different level of experience would be valuable. Secondly, the number of landmarks can be reduced by using the 3D version of the proposed method. Because considering the 3D shape of organs might allow us to use only a few mesh points to reconstruct the shape, it could not necessarily need to pinpoint pseudo-landmarks per image. For example, considering only the superior, inferior, anterior, posterior points of the 3D prostate gland could be enough to generate the shape prior.

• The fully automatic method for ultrasound image segmentation in prostate brachytherapy (presented in section 5.2) showed promising results. It involves simultaneously learning the shape prior of the prostate and the distinctive intensity distribution of the input image. However, addressing the following points would improve the accuracy of the proposed method. Firstly, cross-validating the technique from aligned MRI as well as from different clinical centers and experts is necessary. In this case, it requires modeling the shape prior using MR images, while the image feature extraction network is trained from the TRUS images. Secondly, the proposed proof of concept method was tested on TRUS images obtained from the intraoperative LDR-BR step. However, to be used in clinical practices, further studies are required to incorporate the spatial-temporal information that could be valuable for real-time applications. Moreover, more experiments on phantom data could be crucial to validate the method.

• The introduced new deep learning framework for medical image segmentation (presented in chapter 6) by interconnecting two networks via feedback loop method showed promising results. It was evaluated on four different clinical databases and outperformed the state of the art deep learning methods. Segmentation with the introduced feedback loop strategy leads to accurate and plausible results. Moreover, the anatomical plausibility was achieved without explicit mod-

2. 4 A 2 . 8

 428 ) A typical image-guided radioactive seed implantation technique. B) Realtime transrectal ultrasound prostate image. The prostate delineation in red and the planned position of seed insertion (needle tips) in the small white circle for dosimetry analysis. Image modified from Cancer Research UK. Used with permission. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Examples of prostate gland images obtained from: (A) TRUS, (B) MRI, and (C) CT. A brachytherapy grid was superimposed over the original ultrasound images that appeared as white dots on the TRUS images. . . . . . . . . . . . . . . . . . 2.6 Postimplant CT-exams at the same position of a patient from: (a) primary LDR-BR with only primary implanted seeds, (b) salvage LDR-BR with cumulated primary and salvage implanted seeds. . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Biological and computational neurons. Source for the biological neuron: https: //www.nichd.nih.gov/health/topics/neuro/conditioninfo/parts. . . . . . iii An example of four-layer artificial neural networks. The input x i , hidden layers, and output ŷi variables are represented by nodes. The weight parameters w l ji are represented by the links between the nodes with the arrows indicating the direction of information flow through the networks. The superscript of w denotes the corresponding parameters (neural network weights) are of the given l layer. Moreover, the subscripts ji denotes the flow of information from node i to node j. Note that each hidden and output unit has an associated bias parameter, but we omitted for clarity in this figure. . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 An example of 5 × 5 neural network weights. . . . . . . . . . . . . . . . . . . . . 2.10 An example of max and average pooling operations over an input feature map of size 4 x 4 and depth 1. A 2 x 2 filter size and stride of 2 is applied. . . . . . . 2.11 Data preparation and training scheme of a supervised deep learning method. . . iv List of Tables v vi

  PCa uses several medical imaging devices for tasks such as image-guided diagnosis, treatment planning, image-guided intervention, and treatment evaluation. Medical image computing and computer-assisted interventions (i.e., computerized medical image analysis) can highly benefit the LDR-BR procedures to extrapolate clinically useful information, speed up the contouring of target organs, evaluate the treatment procedures, and thereby improve the patient outcomes. Specifically, the LDR-BR for PCa involves several redundant and time-consuming tasks. The prostate CTV segmentation in the implantation and postimplant analysis, respectively, from TRUS and CT images is a good example. Another example is in detecting implanted radioactive seeds from postimplant LDR-BR CT images for dose distribution analysis. Thus, developing an accurate and robust medical image analysis methods for the image-guided prostate brachytherapy procedures can be beneficial to facilitate the clinical workflow and improve patient health.

7 2. 1

 71 Brachytherapy

Figure 2 . 1 .

 21 The prostate size of normal adults reaches 20 cc (3 cm length, 4 cm in width, and 2 cm in depth) and a weight of 20 gr. Indeed, the prostate volume grows throughout a lifetime. Every decade after 40 years old, the prostate volume commonly rises concomitantly with benign prostatic hypertrophia (benign tumor).

Figure 2 . 1 :

 21 Figure 2.1: Anatomical localization of the prostate in the male reproductive system in the sagittal view and the neighborhood organs in the right image. Credit: Prostate anatomy image c American Cancer Society 2017. Used with permission.

1 BrachytherapyFigure 2 . 2 :

 122 Figure 2.2: Zone of the prostate gland in axial view. Source available at https://www. slideshare.net/amolgulhane58/amol-april-prostate-cancer-imaging, 2020.

Figure 2 . 3 :

 23 Figure 2.3: Examples of prostate cancer treatment techniques. IMRT stands for intensitymodulated radiotherapy.

Figure 2 . 4 :

 24 Figure 2.4: A) A typical image-guided radioactive seed implantation technique. B) Real-time transrectal ultrasound prostate image. The prostate delineation in red and the planned position of seed insertion (needle tips) in the small white circle for dosimetry analysis. Image modified from Cancer Research UK. Used with permission.

Figure 2 . 5 :

 25 Figure 2.5: Examples of prostate gland images obtained from: (A) TRUS, (B) MRI, and (C) CT. A brachytherapy grid was superimposed over the original ultrasound images that appeared as white dots on the TRUS images.

  The necessary steps in performing a CT-based postimplant dose distribution analysis are: (I) outlining the prostate volume on each CT image, (II) determining the number of seeds needed and their accurate position, (III) calculating the dose to each point in the matrix of grid points in a selected volume including the prostate, (IV) generation of isodose curves which can be superposed on each CT image, (V) generation of a dose-volume histogram (DVH) for the prostate and dosimetric information for the critical structures.

Figure 2 . 6 :

 26 Figure 2.6: Postimplant CT-exams at the same position of a patient from: (a) primary LDR-BR with only primary implanted seeds, (b) salvage LDR-BR with cumulated primary and salvage implanted seeds.

Figure 2 .

 2 6 illustrates an example of CT examinations of one patient taken at different time intervals. In the cumulated seeds of Fig. 2.6 (b), it is ambiguous to identify the seeds that are implanted during the salvage LDR-BR, which are the only source of radiation at this stage.

(

  ANN) simulates the functioning of a biological neuron, and ANNs are thus composed of a large number of highly interconnected neurons working in unison to solve a given problem. Lets consider a single neuron, which is the fundamental unit of ANNs (Figure 2.7).

Figure 2 . 7 :

 27 Figure 2.7: Biological and computational neurons. Source for the biological neuron: https: //www.nichd.nih.gov/health/topics/neuro/conditioninfo/parts.

. 1 )Figure 2 . 8 :

 128 Figure 2.8: An example of four-layer artificial neural networks. The input x i , hidden layers, and output ŷi variables are represented by nodes. The weight parameters w lji are represented by the links between the nodes with the arrows indicating the direction of information flow through the networks. The superscript of w denotes the corresponding parameters (neural network weights) are of the given l layer. Moreover, the subscripts ji denotes the flow of information from node i to node j. Note that each hidden and output unit has an associated bias parameter, but we omitted for clarity in this figure.

2 ) 1 Chapter 2 :

 212 where w = (w0, ..., wN ) T and φ = (φ0, ..., φN ) T . The nonlinear basis functions φ allows the function y(x, w) to be a non-linear function of the input vector x. It is because considering only the linearity of the parameters wi simplify the analysis of the class of the models. The activation function takes an input x multiplied by a weight of w. Bias, w b , allows us to shift the activation function by adding a constant value to the input. For instance, in the cases of a perceptron and binary classification problem the nonlinear activation function f (.) is given by a step function: f (x) = +1 for a ≥ 0, and f (x) = -Background of the study 20 for a ≤ 0. Other common activation functions are linear, sigmoid, Tanh, rectified linear unit (ReLU), leaky ReLU, parameterized ReLU, exponential linear unit(ELU), and Softmax[START_REF] Sibi | Analysis of different activation functions using back propagation neural networks[END_REF] [START_REF] Bishop | Pattern recognition and machine learning[END_REF]. Note that the main purpose of the non-linearity function is to help the network learn the often non-linear real-world data. Moreover, a single neuron allows us only to model linear decisions and might fail to solve a complex problem. A linear combination of M individual neurons are thus often used as ŷ

5 )

 5 where ŷmk = y k (xm, w) is the transformed non-linear function of the input vectors xm, m = 1, ..., M , together with a corresponding target vectors ym. The gradient of the error function in equation 2.5

Figure 2 . 9 :

 29 Figure 2.9: An example of 5 × 5 neural network weights.

Figure 2 . 10 :Figure 2 . 11 :

 210211 Figure 2.10: An example of max and average pooling operations over an input feature map of size 4 x 4 and depth 1. A 2 x 2 filter size and stride of 2 is applied.

Girum KB,

  Créhange G, Bessiéres I, Walker PM, Butler WM, Lalande A.Medical imaging and computer vision trends in low dose rate brachytherapy for prostate

Chapter 6 Deep

 6 learning methods for medical image processing Recently, deep learning (DL) driven medical image processing has shown unprecedented success leading to automatic and accurate medical image analysis. DL methods use mainly convolutional neural networks (CNNs), pooling layers, and up-convolutional neural networks to extract discriminative image features automatically. The extraction process involves training from a defined pixel-wise objective function through a backpropagation algorithm[START_REF] Bishop | Pattern recognition and machine learning[END_REF]. However, as the current CNNs are strongly biased towards recognizing textures rather than shapes, training from pixel-wise objective function alone is not free of limitation, such as producing unrealistic and incomplete segmentation results. Despite the success of the current DL methods in achieving good quantitative results, they largely lack clinical interpretability. Indeed, it tends to produce sometimes unexpected and atypical results. These results produce an error that could not be made by manual segmentation. Consequently, segmentation results with such errors do not render trustworthy the clinical information extracted from these segmented structures.Moreover, in the current state of the art DL methods, including those that use shaper prior, the training is through the backpropagation technique. The gradient of the error between the predicted values and the reference values is backpropagated to train and update the neural network weights.

  Deep Learning.

	Glossary	Glossary
	LDR-BR Low Dose Rate Brachytherapy.
	ML Machine Learning.
	mpMRI Multi-parametric Magnetic Resonance Imaging.
	MR Magnetic Resonance.
	MRI Magnetic Resonance Imaging.
	DRE Digital Rectal Examination.
	GAN Generative Adversarial network.
	HDR High Dose Rate.
	LDR Low Dose Rate.
	vii

OARs Organs at Risk. PCa Prostate Cancer.

Vol. 15, no. 9, pp. 1467-1476. DOI: https://doi.org/10.1007/s11548-020-02231-x

Acknowledgments ix 1 Introduction

Acknowledgments

First and foremost, I would like to express my sincerest gratitude to my advisors, Prof. Gilles Créhange and Dr. Alain Lalande, for the opportunity to pursue, first as a master thesis intern and later as a Ph.D. student. Thank you for your trust, patience, guidance, and mentorship that introduced me to the research path. I have learned an incredible amount from your generous support and sincere advice through the Ph.D. years, which helped me to accomplish this milestone. Thank you very much. I am also profoundly grateful to Paul-Michael Walker, Igor Bessières, Wayne M. Butler, Magali Quivrin, and Antoine Merlet, who have contributed significantly scientific knowledge to the development of the project and the production of the scientific papers. I am deeply grateful to thank my thesis committee members Prof. Irène Buvat, Prof. Jocelyne Troccaz, and Dr. Olivier Bernard for accepting to evaluate and provide feedback on my thesis. I would also like to thank the Ph.D. students in the ImViA laboratory, Raabid Hussain, Youssef Skandrani, Muhammad Arslan, Roberto Marroquín, Adrian Taboada, and Nerea Payan. Thank you, everyone, at the University of Burgundy, Dijon campus, for your kind supports. I would also like to thank many members of the CGFL, anticancer center, community. Special thanks go to my Ethiopian and Eritrean friends in Dijon: Hiliwi, Solomon, Wegahta, Lelise, Debritu, Tesfit, Tsega, and Tewodros. Thank you all for your support, and of course, for letting me

Discussion

In this work, we presented the main choices of medical imaging modalities and contributions of image processing algorithms across the image-guided LDR-BR procedures for PCa. We discussed the developed approaches for prostate segmentation, PCa diagnosis, and radioactive seed detection.

Studies have shown that automatic prostate segmentation is more reliable on MR than on TRUS and CT images. However, recently, several groups have proposed DL approaches for prostate CTV segmentation from TRUS and CT images [START_REF] Karimi | Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images[END_REF] [START_REF] Girum | Deep generative modeldriven multimodal prostate segmentation in radiotherapy[END_REF]. Despite the promising results using the advanced DL methods, segmentation methods developed on these imaging modalities still require cross-validation from corresponding MR images [START_REF] Karimi | Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images[END_REF], because the annotation process on TRUS and CT images is often prone to inter and intra-observer variations. Thus, the developed methods might be biased by the created ground truths other than the segmentation problem.

To diagnosis a PCa, mpMRIs can be used. DL methods have achieved promising results by making use of the different sequences of MR images. For example, T2-weighted MRI for prostate segmentation and diffusion-weighted MRI for PCa detection have been studied [START_REF] Song | Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric mri[END_REF], which in turn requires registration between them. In almost all proposed DL approaches for prostate segmentation, incorporating the shape prior of the prostate is prevalent.

Different image processing algorithms have been used for the intraoperative and postoperative radioactive seed detection. The purpose of these algorithms was to enhance the dose distribution computation. Classical ML methods such as clustering, thresholding, principal component analysis, connected component analysis have been extensively used to detect, separate overlapped seeds, and estimate seeds' orientation from different medical imaging modalities. Meanwhile, a few studies have proposed to use DL methods to detect the seeds in CT [START_REF] Yuan | A deep regression model for seed localization in prostate brachytherapy[END_REF], ultrasound [START_REF] Golshan | Automatic detection of brachytherapy seeds in 3d ultrasound images using a convolutional neural network[END_REF], and MR [START_REF] Sanders | Development and clinical implementation of seednet: A sliding-window convolutional neural network for radioactive seed identification in mri-assisted radiosurgery (mars)[END_REF] images.

Moreover, several groups have shown an interest in developing MRI only or co-registering MRI with CT or TRUS for the LDR-BR procedures. It is because MR images are prevalent for prostate delineation. Furthermore, selecting an imaging modality that minimizes ionization to the patient and automating the lengthy procedures would maximize patient care.

Computerized image processing methods can revolutionize the LDR-BR for PCa in parallel with the optimized image acquisition settings. They can be a great help to facilitate the often lengthy sessions in LDR-BR. Moreover, it could be used to analyze the patient outcomes from large data, which then can be used to analyze and improve the treatment procedures. In this regard, the DL approach seems promising to automate the time-consuming tasks. However, similar to the other medical image analysis domains, there is an annotated data scarcity in LDR-BR, making the development and validation of algorithms difficult.

In summary, this review might benefit the brachytherapy communities by providing the current and trending imaging and image processing methods used to facilitate the LDR-BR procedures. There is no previous review paper in this area. This work is then used as an input to meet our objectives by identifying the current method's drawbacks and using some of the proposed promising approaches, as will be discussed in the following chapters.

Chapter 3: Medical imaging and computer vision trends in LDR-BR for PCa

Discussion

In this work, we proposed a new technique for dose distribution (dosimetry) analysis of postimplant salvage LDR-BR for PCa using CT images. It involved three consecutive stages. First, we developed a method to detect the implanted seeds on CT images of postimplant prostate LDR-BR. Secondly, we employed a landmark-based registration of the detected seeds from postimplant CT-exams of the primary LDR-BR and the salvage LDR-BR. The detected seeds from the primary LDR-BR are then removed from the CT-exams of the salvage LDR-BR, leaving only the seeds from the salvage LDR-BR procedures on the place. Thirdly, the dose distribution from the radioactive seeds implanted in the salvage prostate brachytherapy is evaluated. We used VariSeed planning software (Varian Medical Systems Inc, Palo Alto, CA) to calculate the dose distributions. The dosimetry parameters such as V100 and D90 are reported, which are routinely used to assess the implantation quality.

In this work, we showed that dosimetry analysis of salvage LDR-BR procedures performed years after primary LDR-BR is possible through the proposed technique. Although our study was limited by the number of cases (only six patients), it might open an essential perspective into the image-based analysis of the relapsed PCa. It can be used to correlate the primary LDR-BR implantation quality and the relapsed PCa locations. This could help to determine where the PCa relapses and to investigate solutions for such conditions.

In order to improve the dosimetry analysis in salvage LDR-BR, the following limitations should be addressed. First, the proposed method should be validated in a larger database of CT-exams.

Secondly, further studies are required to avoid the manual selection of the landmarks used to register the two CT-exams. If a large database can be collected, patterns can be learned using ML algorithms for automatically identifying the salvage seeds from the primary seeds. Thirdly, while removing the primary seeds, a robust pixel interpolation technique should be developed. Indeed, while performing the dosimetry analysis, the primary seeds should not be considered or should not appear as artifacts.

The dosimetry is computed by considering the prostate as a water medium. However, it is easy for radiation oncologists to assess the radiomics outputs when the image is of good quality. For example, radiation oncologists would trust the radiomics output when the image is free of either metallic artifacts or artifacts introduced while removing the primary seeds. Thus, to address this scenario, parallel work has been done in our group by Merlet et al. [START_REF] Merlet | Specified metal artifact reduction (mar) on ct-scan for dosimetry accuracy in i-125 prostate brachytherapy[END_REF].

Finally, automatic prostate segmentation on CT images is necessary to facilitate the dosimetry evaluation. It is currently performed manually, prone to inter and intra-observer variations, timeconsuming, balky, and sometimes difficult. To address this challenging task, in the next chapter, we will investigate automatic prostate segmentation approaches in the image-guided LDR-BR procedures using DL methods.

Chapter 5

Interactive and automatic deep learning methods for image-guided LDR-BR procedures As presented in chapter 3, the clinical work-flow of LDR-BR for PCa meets several medical imaging techniques. In LDR-BR procedures, the prostate CTV is often delineated manually at several steps on different medical imaging modalities. Despite the need for reliable prostate segmentation in LDR-BR procedures, it is often cumbersome, time-consuming, and prone to inter and intra-observer variations. Thus, in this chapter, we propose DL-based solutions to facilitate the delineation of the prostate CTV in the LDR-BR work-flows. To this end, this chapter has two main sections.

In the first section, we present a new automated DL approach that allows experts to interact with the process to improve the segmentation accuracy, in order to be acceptable in the clinical practice.

By extension, it is also designed to annotate medical images with minimal manual interaction in the case of having small annotated datasets or incorporating new imaging settings into a given annotated database. The new annotated data can then be used to develop a fully automatic supervised DL method. In the second section of this chapter, we present a fully-automatic DL method for prostate CTV segmentation from TRUS images acquired in the intraoperative LDR-BR stage.

Chapter 7

Perspectives & conclusions

Radiotherapy procedures, including image-guided intervention and treatment evaluation, require several medical imaging systems. Moreover, patient work-flow in radiotherapy is complex. There are many steps involved that use different imaging devices. Several redundant and time-consuming tasks are also prevalent. The prostate CTV segmentation in the intraoperative implantation and postimplant analysis steps of prostate low dose rate brachytherapy, respectively, from TRUS and CT images, is a good example. Medical image computing and computer-assisted interventions (i.e., computerized medical image analysis) can highly benefit the radiotherapy procedures to extrapolate clinically useful information, speed up contouring of target organs, evaluate treatment procedures, and thereby improve patient outcomes.

Internal radiotherapy, known as low dose rate brachytherapy (LDR-BR), has proven effective for the treatment of organ-confined early-stage prostate cancer. It involves inserting radioactive elements permanently inside the prostate gland under transrectal ultrasound (TRUS) guidance, aiming to maximize radiation dose to the prostate gland and minimize it to the organs at risk (OARs). However, cancer can locally relapse. It might then require a second LDR-BR. In both LDR-BR stages following the day-30 or day-0 implantation, the dosimetry is evaluated from a CT-exam. However, CT-exams of the second postimplant LDR-BR contains cumulated seeds from the first and the second LDR-BR procedures, making it difficult to compute the dosimetry from only the newly implanted seeds. To address this problem, we developed an automatic image processing system to detect and identify the implanted seeds from the postimplant CT-exams. Thus, it enabled us to evaluate the dosimetry of the newly implanted seeds in the second LDR-BR known as salvage prostate brachytherapy.

Among others, artificial intelligence based on machine learning and deep learning methods can help to automate the image processing steps in the radiotherapy procedures. Indeed, it is useful to extrapolate clinically relevant information and speed-up contouring of target organs from the medical images, and thereby it can benefit the clinical workflow. Moreover, deep learning methods based on convolutional neural networks (CNNs) have successfully been leveraged to learn distinctive image features. These techniques require sufficiently large and diverse annotated data from experts. However,

Appendix A Publications

This thesis manuscript is primarily based on six journal papers and one conference paper. These papers are indicated by GIRUM KB as the first author in the following list of publications. In this appendix, we will present a new deep learning method for multimodal prostate segmentation in radiotherapy. It aims to model and transfer the shaper prior from a good contrast imaging modality into other low contrast imaging modality but considering the same target. To this end, we proposed a learning-based shape prior modeling and registration approach that learns the anatomical structure of an organ, which is invariant across the different medical imaging modalities. For example, in our experiment, the prostate shape prior was learned from MR images and applied it into prostate segmentation from CT images. It was evaluated on clinical databases of MR and CT images.

Journal publications

The initial results of the proposed method were orally presented at the Artificial intelligence in radiation therapy (AIRT) conference, conducted in conjunction with MICCIA 2019 and has been published as follows [START_REF] Girum | Deep generative modeldriven multimodal prostate segmentation in radiotherapy[END_REF]: In this appendix, we provide the supplementary materials of the paper presented in section 5.2 [START_REF] Girum | A deep learning method for real-time intraoperative us image segmentation in prostate brachytherapy[END_REF], and it includes details about the cost function, the evaluation metrics, the model details, and the network architectures. In particular, the Dice similarly coefficient loss and cross-entropy loss adapted in the training of our model will be presented.

Moreover, the used evaluation metrics and their formulas, such as the Dice similarity coefficient, Hausdorff distance, specificity, sensitivity, and accuracy, are presented. The detailed architecture of the shape prior modeling network and building blocks of other architectures used for comparison are discussed.

Appendix C: A deep learning method for real-time intraoperative US image segmentation in prostate brachytherapy: supplementary material 54

It has been published as supplementary material and can be accessed online (https://static-content.