Prof Fabrice Meriaudeau

Dr Arnau Oliver

Dr Xavier Lladó

Dr Robert Martí

Joan C Vilanova

Mrs Nathalie Choffay

Ms Clémence Fontaine

Ms Joseta Roca

Mrs Aina Roldan

Keywords: evaluation : prostate segmentation in TRUS images, comparison, 3, 2 Qualitative comparison of prostate segmentation, 4, 1 Prostate segmentation quantitative results

Freixenet for their constant support and guidance towards the completion of this thesis. Their expertise in the field of medical image analysis and computer vision have significantly improved

Resum executiu

La segmentació de la pròstata en imatges d'ecografia transrectal (TRUS) i en imatges de ressonància magnètica (RM) facilita l'estimació del volum d'aquesta glàndula, el registre d'imatges entre ambdues modalitats, així com la planificació quirrgica de biòpsies guiades per imatge. L'objectiu d'aquesta tesi, doncs, és el desenvolupament d'eines automàtiques per a una segmentació de la pròstata de manera precisa, robusta i computacionalment eficient en ambdues modalitats d'imatges.

La contribució principal d'aquest tesi és la segmentació de les imatges ecogràfiques de la pròstata. El mètode proposat es basa en dos passos ben diferenciats. Primer, a través d'un aprenentatge probabilístic inicial, s'aconsegueix una primera localització aproximada de la pròstata i que serveix per, en un segon pas, inicialitzar i permetre evolucionar de manera automàtica dos models deformables independents, guiats a partir de la informació de forma i regió de la pròstata estimada en el primer pas. El primer model deformable s'obté explícitament a partir de l'anàlisi de components principals (PCA) d'un conjunt de punts del contorn, que permet modelar la forma de la pròstata, i de l'anàlisi PCA de la distribució de probabilitat dins de la regió pròstatica, que permet modelar la textura d'aquesta. Un tercer anàlisi PCA permet correlacionar ambdues distribucions. D'altra banda, un segon model deformable es deriva implícitament de l'anàlisi PCA de la funció distància obtinguda amb el conjunt de dades d'entrenament etiquetades. La consegüent evolució d'aquesta corba s'obté mitjanant la minimització del funcional Mumford-Shah, el qual es basa en un conjunt d'estadístics regionals obtinguts a partir de l'estimació de les probabilitats a posteriori de les regions internes i externes de la pròstata.

La segona contribució d'aquesta tesi és la segmentació automàtica de la pròstata en imatges 3D de RM. De manera similar a les imatges ecogràfiques, el sistema v vi combina les probabilitats d'un aprenentatge supervisat amb una segmentació inicial, en aquest cas, obtinguda a partir d'un atles probabilístic creat amb els volums d'entrenament. La segmentació final s'obté a través d'una minimització basada en grafs.

El resultat final és, doncs, el desenvolupament d'eines que permeten una segmentació acurada i robusta de la pròstata tant en imatges ecogràfiques com de ressonància magnètica, millorant de forma substancial i significant la precisió dels mètodes desenvolupats fins a l'actualitat.

Résumé

Le cancer de la prostate est considéré comme un problème majeur de santé publique dans le monde occidental et il est le deuxième cancer le plus fréquent chez les hommes après le cancer du poumon. Les statistiques de Cancer Research UK montrent que plus de 338,000 personnes sont annuellement diagnostiquées avec le cancer de la prostate en Europe et 913,000 dans le monde entier [START_REF]Prostate Cancer Statistics -Key Facts[END_REF]. Les taux de diagnostic les plus élevés sont observés aux Etats-Unis, en Australie, en Nouvelle-Zélande et en Europe, tandis que les taux les plus bas sont observés en Asie du sud et en Asie centrale [START_REF]Prostate Cancer Statistics -Key Facts[END_REF]. Le cancer de la prostate représente environ 7,1% de tous les cancers diagnostiqués et est responsable de 3,4% des décès liés à un cancer. Environ 10.000 décès chaque année sont associés au cancer de la prostate et près d'un homme sur 6 est susceptible d'être diagnostiqué avec le cancer de la prostate au cours de sa vie.

Actuellement, les outils de dépistage tels l'antigène spécifique de la prostate (PSA en anglais) détectent la maladie dans 25-30% des cas [START_REF] Schroder | Screening and prostate-cancer mortality in a randomized european study[END_REF]. La procédure de biopsie, généralement réalisée à l'aide d'une sonde d'échographie transrectale (ETR), présente des erreurs d'échantillonnage dues au fait que la localisation du cancer est inconnue au moment de la biopsie d'une part, et, d'autre part, part le fait que les tissus cancéreux ne sont souvent pas visibles dans les images obtenues par échographie transrectale (images ETR). Lors d'une biopsie avec un guidage par l'image limité, un échantillonnage systématique permet jusqu'à 20 échantillons de tissus de la prostate. Néanmoins, les cancers agressifs qui souvent nécessitent un traitement immédiat peuvent ne pas être détectés, ce qui conduit à des biopsies répétées. En outre, l'agressivité ou non du cancer ne peut pas être déterminé avec précision, ce qui provoque l'anxiété du patient, un sur-traitement et une augmentation des coûts. L'utilisation d'images ETR pour la biopsie est maintenant une norme suivie par les urologues pour le dépistage du cancer de la prostate. Toutefois, l'imagerie par résonance magnétique (IRM) offre un meilleur contraste des tissus mous par rapport aux images ETR. Ainsi, certaines tumeurs malignes visibles par l'IRM ne le sont pas avec les images ETR comme illustré par l'image de la figure 1.

En fusionnant les deux modalités IRM et échographie transrectale, il est possible de développer des outils performants de diagnostic. C'est dans ce contexte que s'inscrit le projet PROSCAN qui est une collaboration entre le centre de recherche VICOROB (Computer Vision and Robotics Group) de l'université de Gérone et le Girona Magnetic Resonance Center du CHU de Gérone. L'objectif principal du projet PROSCAN est le développement d'un système de recalage d'images multimodales pour faciliter une meilleure visualisation des tumeurs malignes dans les biopsies de la prostate guidées par images ETR. Le cadre ix L'objectif principal du projet est le recalage d'images IRM de la prostate acquises avant la biopsie avec des images ETR en temps-réel pendant l'échographie transrectale afin d'offrir une meilleure visualisation des tissus malins.

Un recalage basé sur les contours de la prostate réduit de manière significative le temps de calcul et améliore précision. Par conséquent, la segmentation de la prostate dans les images IRM et ETR est une étape préalable nécessaire à la fusion des deux modalités. L'objectif principal de cette thèse est de développer des méthodes de segmentation précises et rapides de la prostate dans les images IRM ET ETR afin de faciliter la fusion d'images multimodales dans le cadre du projet PROSCAN.

x Nous avons divisé l'objectif principal de cette thèse en un ensemble de sousobjectifs en fonction des critères d'exigence dans chaque modalité comme indiqué par le projet PROSCAN. Ainsi, pour les images ETR une méthode de segmentation rapide, automatique, précise et robuste d'images 2D est nécessaire pour segmenter la prostate dans des séquences vidéo. De même, pour l'IRM une méthode précise et robuste doit être développée pour segmenter la prostate dans un volume 3D.

Nous avons commencé notre travail par une étude approfondie des méthodes de segmentation dans les deux modalités échographie transrectale et IRM. Les principales similitudes et les différences entre les diverses méthodes, leurs forces et faiblesses ont été analysées. Les méthodes de segmentation de la prostate peuvent être regroupées dans quatre catégories différentes, selon les informations utilisées pour guider la segmentation :

• Méthodes basées sur le contour et la forme : Ces méthodes utilisent l'information donnée par les contours e de la prostate pour la segmentation. Néanmoins, l'information de contour étant peu fiable dans les images ETR ainsi que dans la base et l'apex de la prostate dans les images IRM, il est nécessaire d'utiliser l'information a priori sur la forme de la prostate pour obtenir de meilleurs résultats.

• Méthodes basées sur les régions : Ces méthodes utilisent l'intensité locale ou des statistiques de l'intensité (moyenne et écart-type) pour atteindre la segmentation. Les méthodes de cette catégorie se distinguent principalement par la méthode d'optimisation utilisée. Par exemple, les méthodes basées sur un atlas utilisent un modèle de la prostate obtenu à partir d'images d'apprentissage manuellement segmentées et tentent de minimiser la différence d'intensité entre le modèle et une nouvelle à segmenter. En revanche, dans les méthodes de type level-sets les statistiques (moyenne et écart-type) de l'intensité de la prostate dans les images d'apprentissage sont utilisées pour maximiser la distance entre la prostate et les régions de fond de l'image. Un modèle déformable implicite est ensuite propager de manière à minimiser son énergie dans la zone de convergence des deux régions (prostate et fond).

xi

• Méthodes de classification supervisée et nonsupervisée : Ces méthodes utilisent différentes caractéristiques (intensité ou réponses de filtres) pour diviser et/ou classifier l'image en deux régions : la prostate et le fond. L'objectif de ces méthodes est de regrouper des objets semblables sur la base d'un vecteur de caractéristiques. Contrairement aux méthodes basées sur les régions et la minimisation d'une fonction d'énergie, un seuillage appliqué à une certaine mesure de proximité ou de similarité est utilisé pour regrouper les objets semblables (ici des pixels ou voxels).

• Les méthodes hybrides : Ces méthodes combinent les informations de contour, forme, région et quelque fois le résultat d'une étape de classification supervisée ou nonsupervisée pour segmenter la prostate. Elles sont plus robustes aux artefacts d'imagerie et au bruit.

Segmentation de la prostate dans les images ETR L'analyse des méthodes de segmentation montre que les approches qui combinent les informations de forme et de contour donnent les meilleurs résultats. Aussi, nous proposons d'utiliser le modèle AAM (Actice Appearance Model) qui a prouvé son efficacité pour la segmentation de la prostate dans les image d'échographie transrectale [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF][START_REF] Cootes | Active Appearance Models[END_REF]. Le modèle AAM permet de combiner les informations de forme et d'apparence en une unique fonction de coût à optimiser. De plus, l'étape d'optimisation par descente de gradient faite hors-ligne réduit considérablement les temps de calcul.

Les images obtenues par échographie transrectale possèdent généralement une faible qualité ainsi qu'un faible contraste. Pour améliorer la robustesse de notre méthode de segmentation, nous introduisons des caractéristiques de texture extraits avec les ondelettes de Haar et des filtres en quadrature. Les résultats obtenus montrent que cette information de texture accroit la précision de la segmentation. Par ailleurs, l'augmentation du temps de calcul due à l'utilisation des filtres est compensé par l'augmentation de la précision.

Pour une initialisation automatique, nous avons développé un modèle probabiliste basé sur une classification supervisée. Un classifieur est construit à partir d'un ensemble d'images d'apprentissage manuellement segmentées. Ce classifieur est utilisé pour obtenir une pré-segmentation de la prostate dans l'image ETR dans laquelle on attribue à chaque pixel une probabilité d'appartenance à la prostate. Un nouveau modèle AAM est ensuite construit dans lequel les intensités sont remplacées par les probabilités obtenues à l'étape précédente. Les résultats obtenus montrent que cette approche permet une initialisation automatique tout en améliorant la précision de la segmentation.

Enfin, pour obtenir un modèle plus robuste nous avons utilisé la fonctionnelle de Mumford-Shah qui permet de définir une fonction de coût à optimiser comprenant à la fois les informations d'apparence, de forme et de topologie locale de la prostate. Les nombreux résultats qualitatifs et quantitatifs présentés dans la suite de ce manuscrit montrent que notre méthode donne de meilleurs résultats comparé à diverses autres approches. La figure 3 montre des exemples de résultats obtenus avec le modèle AAM traditionnel [START_REF] Cootes | Active Appearance Models[END_REF] et notre méthode.

Segmentation de la prostate dans les images IRM

Le but du projet PROSCAN étant la fusion d'images multimodales de la prostate, il faut pouvoir segmenter correctement la prostate dans les images IRM. La segmentation 3D de la prostate dans les images IRM est rendu difficile par les variations xiii inter-patients de la forme, la taille et des déformations de la prostate. Cependant, il a été montré dans différents travaux que les méthodes de segmentation basée sur un atlas obtiennent de bons résultats lorsqu'ils sont validés avec un grand nombre de données [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF][START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF]. Plus récemment, [START_REF] Li | Learning image context for segmentation of prostate in ct-guided radiotherapy[END_REF] a utilisé une approche par classification supervisée pour la segmentation de la prostate dans les images CT (imagerie par tomographie axiale calculée par ordinateur). Motivés par ces travaux, nous proposons de combiner les deux approches (atlas et classification supervisée) pour la segmentation 3D de la prostate dans des volumes IRM. Plus précisément, les probabilités obtenues avec une approche de classification supervisée sont combinées avec celles obtenues avec un atlas probabiliste pour fournir une pré-segmentation de la prostate. Cette pré-segmentation est ensuite affiner par une optimisation avec la méthode Graph-Cuts [START_REF] Boykov | Graph cuts and efficient n-d image segmentation[END_REF].

Un diagramme décrivant la méthode est présenté dans la figure 4. Les résultats obtenus montrent que cette approche est robuste par rapport aux variations de forme et de taille de la prostate.

La figure 5 montre des exemples de résultats obtenus avec notre méthode de segmentation.

Liste de publications

Publications liées à la segmentation de la prostate en échographie transrectale:

• [IJCARS 2012] S. Ghose, A. Oliver, R. Martí X. Lladó J. Freixenet, J. Mitra

Chapter 1 Introduction

The field of medical image analysis could be broadly categorized into medical image segmentation, registration, motion tracking of an organ of interest and anatomical and physiological parameter estimation. The motivation of this thesis is in developing an accurate, robust, computationally efficient prior knowledge based prostate segmentation algorithm in trans rectal ultrasound (TRUS) and magnetic resonance imaging (MRI). In this thesis we detail the development of a powerful schema of using posterior probabilities for building shape and region priors deformable models. In this introductory chapter we highlight the motivation behind development of prostate segmentation algorithm, provide the organizational layout of this thesis and highlight the core contribution of this thesis.

Prostate cancer

Prostate cancer is considered a major health problem in the western world and it is the second most common cancer among the male pouplation after lung cancer. phy, in cryotherapy and in transurethral microwave therapy. In addition, prostate gland segmentation also facilitates multimodal image fusion for tumor localization in biopsy, minimally invasive ablative and radiation therapy. However, manual segmentation of the prostate is a tedious task, prone to inter and intra observer variability. Therefore, automatic or semi-automatic computer aided prostate segmentation methods have been developed in the last decade. The choice of a prostate segmentation method depends on imaging modality, computational time requirement and degree of segmentation accuracy that is necessary to achieve. Low contrast, speckle, and imaging artifacts like the shadow region significantly challenges the development of computer aided prostate segmentation methods. Similarly, magnetic bias and deformation of the prostate due to insertion of the endorectal probe adversely affect accuracies of prostate segmentation methods in MRI. In the next three sections we explore the three modalities in more detail.

TRUS

TRUS is primarily used in determining prostate volume and in prostate biopsy due to the fact that it is inexpensive, portable and real-time in nature [START_REF] Shao | Prostate Boundary Detection From Ultrasonographic Images[END_REF]. Note from Figure 1.1 that the prostate gland can be often observed as a hypoechoic mass surrounded by a hyperechoic halo [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF] and TRUS images of the prostate are characterized by speckle, shadow artifacts and low contrast [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF] as observed in Figure 1.2.

Added to these challenges the prostate shape and size may vary significantly. In Figure 1.3 we observe how prostate shape and size may vary across different datasets. Moreover depending on machine manufacturer and acquisition parameters image contrast may vary. For instance, Figure 1.4 we observe how contrast varies with different machine manufacturer. while Figure 1.5 shows the change in contrast depending on acquisition parameters for the same machine.

MRI

MRI is primarily used in diagnostic and treatment planning for prostate diseases [START_REF] Makni | Combining a Deformable Model and a Probabilistic Framework for an Automatic 3D Segmentation of Prostate on MRI[END_REF][START_REF] Martin | Atlas-based Prostate Segmentation Using an Hybrid Registration[END_REF], since it provides good soft tissue contrast and enables a better lesion detection and staging for prostate cancer. Figure 1.6 shows an axial view of the middle slice of a prostate in MRI. In addition, dynamic contrast enhanced MRI (DCE-MRI) aids in identifying malignant tissues from the diffusion rate of the contrast agent [START_REF] Liu | Unsupervised Segmentation of the Prostate Using MR Image Based on Level Set with a Shape Prior[END_REF][START_REF] Ozer | Supervised and Unsupervised Methods for Prostate Cancer Segmentation with Multispectral MRI[END_REF][START_REF] Puech | Computer-Assisted Diagnosis of Prostate Cancer Using DCE-MRI Data: Design, Implementation and Preliminary Results[END_REF] and magnetic resonance spectroscopy aids in identifying malignant tissues from the relative concentration of different metabolites (like citrate, choline and creatine) [START_REF] Halpern | Imaging of the prostate[END_REF].

Multi-parametric MRI of the suspicious cases can reduce the number of unnecessary biopsies by differentiating between normal and cancer cases [START_REF] Vilanova | Peripheral zone prostate cancer in patients with elevated psa levels and low free-to-total psa ratio: detection with mr imaging and mr spectroscopy[END_REF]. Improvement in detection rate from 20 to 60% is observed with patient specific biopsy planing with the aid of multi-parametric MRI information. However prostate segmentation in MRI is necessary to aid personalized treatment planing which itself is a challenging task due to significant inter-patient difference in prostate shape, size and volume, as illustrated in Figure 1.7. Prostate shapes may vary depending on presence of endorectal coil. For example in Fig. 1.7(a) and 1.7(b) no endorectal coil is present. However in Fig. 1.7(c) an endorectal coil deforms the prostate. Furthermore endorectal coil introduced to enhance contrast around the prostate also introduces intensity heterogeneities of the prostate tissue as illustrated in Figure 1.8.

Soft tissue contrast varies within a prostate. In the central region the soft tissue contrast is better and it is relatively easy to segment the prostate. However the soft tissue contrast falls in the base and the apex region of the prostate making these regions difficult to segment, as illustrated in Figure 1.9. Automatic detection of prostate cancer from multi-parametric MRI with high specificity is difficult. Often radiologists batch read multi-parametric prostate MRIs to detect suspicious lesions. However it is difficult to achieve a high specificity and sensitivity in such batch processing framework. Furthermore, the success to treat aggressive cancers depends on early detection and treatment. Hence it is necessary to detect the smaller tumors of size 2-5 mm [START_REF] Hoeks | Prostate cancer: multiparametric mr imaging for detection, localization, and staging[END_REF] which could be better detected with automatic processing of multi-parametric MRI. The success of such a system depends on automatic organ detection, segmentation and classification of suspicious tissues.

CT

CT is generally used in prostate brachytherapy to determine the placement of the radioactive seeds and also to confirm the seed location post-procedure [START_REF] Halpern | Imaging of the prostate[END_REF]. The high attenuation of the radioactive seed produces high intensity in CT images as could be visualized in Figure 1.10. Note, that distinguishing external and internal anatomy of prostate from CT images is difficult due to poor soft-tissue contrast.

Prostate segmentation in CT is a challenging task due to poor soft tissue resolution as observed in Figure 1.10. Moreover the presence of bowel gas may deform prostate in un-natural manner.

Objectives of the thesis

Currently prostate specific antigen (PSA) screening facilitates prostate cancer detection in 25-30% cases [START_REF] Schroder | Screening and prostate-cancer mortality in a randomized european study[END_REF] with low specificity. This results in false detection in 60-70% cases resulting in higher percentage of undesirable prostate biopsies. Furthermore TRUS guided prostate biopsies suffer from sampling error as they are performed without the knowledge of cancer location in the prostate. Approximately 30% of these biopsies miss prostate cancer and often targeted re-biopsies results in detection of cancer in 40% cases [START_REF] Jemal | Cancer statistics[END_REF].

In recent years MR guided biopsies have improved the detection rate from 20 to 60% after negative TRUS guided biopsy. However MR guided biopsy is expensive and time consuming. Fusion of the TRUS and MR images during TRUS guided biopsies could improve positive detection rate in TRUS guided biopsies [START_REF] Baumann | Prosper: image and robotguided prostate brachytherapy[END_REF]. However such tools to fuse the two imaging modalities suffers from computational inefficiency and inacuracy [START_REF] Van De Ven | Required accuracy of mr-us registration for prostate biopsies[END_REF].

The Computer Vision and Robotics group (VICOROB) of the University of

The PROSCAN project

TRUS guided prostate biopsies is now a standard used by urologists. However, MRI provides better soft tissue contrast compared to TRUS images. Hence often a malignant tissue that is visible in MRI would not be visibly distinguished in TRUS images, as illustrated in Fig. 1.11.

The primary objective of this project was to develop a multimodal image registration system to facilitate better visualization and localization of malignant tissues in TRUS guided prostate biopsies. The framework of the PROSCAN project is illustrated in Fig. 1.12. It could be observed, that the objective of the project is multimodal registration of a pre-acquired prostate MRI with TRUS images in near real time during TRUS guided prostate biopsy to facilitate better visualization of the malignant tissues. Currently, very few solutions to achieve the fusion between US and MRI exist. Current commercial solutions, which may be described as still in a prototype phase of development, include the Eigen "Artemis" device and the Medcom "BiopSee" system. All existing solutions require high degree of user intervention and are not computationally efficient.

Relative concentration of different metabolites like citrate, choline and creatine are important parameters in determining malignant tissues [START_REF] Halpern | Imaging of the prostate[END_REF]. The secondary objective of the PROSCAN project was to determine suspicious tissues from magnetic resonance spectroscopy data and project the information into TRUS videos during TRUS guided prostate biopsies to improve on true positive rate for malignant tissue sampling.

Multimodal registration performed on prostate contours significantly reduces computational time and improves registration accuracies [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. Hence, prostate segmentation in MRI and TRUS images is a necessary prerequisite step for the fusion of the two modalities.

The primary objective of this thesis is to develop accurate and computational efficient prostate segmentation methods in TRUS and MRI to facilitate multimodal image fusion in PROSCAN project.

The multimodal image fusion algorithm developed for the PROSCAN project depends on minimizing the algebraic error between the binary masks of the fixed TRUS and the moving MR images when both are acted upon by a set of nonlinear polynomial functions. The non-linear diffeomorphic transformation of the MR images are based on thin-plate splines. The point correspondences required for the thin-plate splines are established by a statistical measure. In addition to the set of non-linear equations formed by thin-plate splines, the regularized bending energy and the correspondence localization error are included in the system of equations to obtain meaningful realistic transformations of anatomical targets. The details of the proposed method can be further studied in [START_REF] Mitra | A spline-based non-linear diffeomorphism for multimodal prostate registration[END_REF]. Since the method relies completely on segmented prostate images starting from establishment of point correspondences around the prostate boundaries and the minimization of algebraic error of the mask images, automatic segmentation of prostate is an important step in the PROSCAN project.

We split the primary goal of this thesis into a set of sub-objectives depending upon the segmentation requirement criteria in each modality as outlined by the PROSCAN project. Hence for the TRUS images a fast, automatic, accurate and robust 2D prostate segmentation method was necessary to segment the prostate in video sequences. Similarly for MRI an accurate and robust 3D prostate segmentation method had to be developed.

Thesis summary and organization

In this Section we summarize the content of each chapter of the thesis.

Chapter 2, A Review of Prostate Segmentation Methods

In this chapter we present an extensive survey of the prostate segmentation methods in TRUS and MRI found in the literature. The objective of this chapter is to study the key similarities and differences among the different methods, highlighting their strengths and weaknesses. We define a new taxonomy for prostate segmentation strategies that allows first to group the algorithms and then to point out the main advantages and drawbacks of each strategy.

Chapter 3, Shape and Appearance Prior Models for Prostate Segmentation in TRUS Statistical shape and appearance model of Cootes et al. [START_REF] Cootes | Active Appearance Models[END_REF] efficiently incorporates shape and intensity priors in deformable model. In this chapter we explore a novel approach of the use of image features in traditional statistical shape and appearance model of Cootes et al. [START_REF] Cootes | Active Appearance Models[END_REF]. We detail the extension that we have made to [START_REF] Cootes | Active Appearance Models[END_REF] to facilitate automatic initialization and evolution of the model. Moreover, an implicit model of shape and appearance priors is adopted for accurate prostate segmentation in TRUS images.

Chapter 4, Graph Cut Optimization in a Stochastic Framework for Prostate Segmentation in MRI

In this chapter we explore the graph cut energy minimization framework in a stochastic domain to segment prostate. Briefly, a supervised learning framework is devel-Figure 1.10: In CT imaging, the soft tissue contrast is poor. However the modality is useful in detecting radioactive seeds as observed in this figure .   oped to achieve a soft classification of the prostate. Graph cut energy minimization in the probabilistic domain provides segmentation of the prostate in MRI.

Chapter 5, Contributions and Suggestion of Future Research

The thesis is concluded by summarizing the major contributions of the thesis and by suggesting possible future research directions associated with this work. Moreover, in this last chapter, a list of the publications related to this thesis is included. This chapter reviews the methods developed for prostate gland segmentation in TRUS and MR images. The objective of this chapter is to study the key similarities and differences among the different methods, highlighting their strengths and weaknesses. We define a new taxonomy for prostate segmentation strategies that allows first to group the algorithms and then to point out the main advantages and drawbacks of each strategy. A discussion on choosing the most appropriate segmentation strategy for a given imaging modality is provided and a quantitative comparison of the results as reported in literature is also presented.

Introduction

Computer aided prostate segmentation in TRUS and MRI is a challenging task. Each of the modalities have a different challenge associated with them. For instance low contrast, speckle and imaging artifacts like the shadow region hinder accurate prostate segmentation in TRUS images. Similarly, magnetic bias around the endorectal coil and deformation of the prostate gland due to insertion of the endorectal coil adversely affect prostate segmentation accuracies in MRI as observed This chapter presents an up-to-date summary of the techniques developed for prostate segmentation in TRUS and MRI. We classify and review the different approaches found in the literature in order to show similarities and differences and further to extract advantages and drawbacks from the reviewed algorithms. To have an overall qualitative estimation of the performance of the different methods, we have grouped the methods according to their theoretical approach and have presented their evaluation metrics and degree of validation. Note that a quantitative comparison of different prostate segmentation methodologies is difficult in absence of public data sets, publicly available software, and standardized evaluation metrics.

The outline of this chapter is as follows. The state-of-the-art computer-aided prostate segmentation procedures are classified and presented in Section 2.2. In Section 2.3, validation and quantitative evaluation of the prostate segmentation in TRUS, and MR images are provided. Discussion on selection of an efficient prostate segmentation technique based on imaging modality is presented in Section 2.4.

Prostate segmentation methods

In this work, we classify the prostate segmentation methods according to the theoretical computational approach taken to solve the problem. We believe that such a classification successfully points out the key algorithmic similarities and dissimilarities, highlighting their strengths and weaknesses at the same time. We glob-ally classify the methods into different strategies: contour and shape based, region based, supervised and un-supervised classification methods based, and hybrid methods. We further refine these groups to produce a more local classification schema. For instance, contour and shape based methods are further classified into edge, probabilistic filters and deformable models. The proposed taxonomy is shown in Figure 2.1. Note that level sets methods appear under contour and shape based and region based methods. This is due to the fact that level sets can be guided by either boundary or region information.

We have grouped the prostate segmentation methods in four different groups, according to the information used to guide the segmentation. Broadly,

• Contour and shape based methods: These methods use prostate boundary/edge information to segment the prostate. Since often edge information is unreliable in TRUS and in the base and the apex region of the MR images, prior shape information is incorporated to provide better results.

• Region based methods: These methods use local intensity or statistics like mean and standard deviation in an energy minimization framework to achieve segmentation. The methods in this category primarily varies depending on the energy minimization framework. For example in atlas based methods a model of the prostate is created from manually segmented training images and intensity difference between the model and a new un-segmented image is minimized. In contrast, in region based level sets prior mean and standard deviation information of the prostate region are used to maximize the distance between prostate and background regions. An implicitly defined deformable model propagates depending on region based statistical moments whose energy is minimized at the zone of convergence of the two regions.

• Supervised and un-supervised classification methods: These methods use features like intensity or higher dimensional features like filter responses to cluster and/or classify the image into prostate and background regions. The objective of such methods is to group similar objects together based on the feature vector. A thresholding scheme is used based on some proximity or distance measure to group similar objects together. • Hybrid methods: The objective of the hybrid methods is to combine information from contour, shape, region and/or supervised or un-supervised classification information to segment the prostate.

We have outlined the advantages and disadvantages of the reviewed prostate segmentation approaches in Table 2.1.

In the following subsections, the reviewed methods are described according to the presented taxonomy. Moreover, for each category, the approaches are grouped and described according to the imaging modalities: TRUS and MRI.

Contour and shape based segmentation

Contour and shape based methods exploit contour features and shape information to segment the prostate. These methods can be categorized into edge based methods, probabilistic filters and deformable model segmentation techniques. Deformable model based techniques are further classified into active contour models, deformable meshes, active shape models, level sets and curve based segmentation. The following subsections discuss individually each of these categories.

Approaches

References Advantages Disadvantages

Contour and Shape Edge [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] Easy to extract Edge information is unreliable and often broken Probabilistic filters [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF], [START_REF] Sahba | Segmentation of Prostate Boundaries Using Regional Contrast Enhancement[END_REF] Robust against noise along boundary Difficult to initialize and to extend to 3D Deformable Models ACM [START_REF] Knoll | Outlining of the Prostate Using Snakes with Shape Restrictions Based on the Wavelet Transform (doctoral thesis: Dissertation)[END_REF], [START_REF] Ladak | Prostate Segmentation from 2D Ultrasound Images[END_REF], [START_REF] Ding | Slice-Based Prostate Segmentation in 3D US Images Using Continuity Constraint[END_REF], [START_REF] Zaim | Automatic Segmentation of the Prostate from Ultrasound Data Using Feature-Based Self Organizing Map[END_REF] Easy to implement, produces smooth contours Depends on reliable edge information, good initialization required, largescale deformations produce spurious corners Mesh [START_REF] Ghanei | A Three-Dimensional Deformable Model for Segmentation of Human Prostate from Ultrasound Images[END_REF] Shape information is preserved Reliable edge information is often necessary, rigid shape representation, slow in speed ASM [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF], [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF], [START_REF] Hodge | Prostate Boundary Segmentation from Ultrasound Images Using 2D Active Shape Models: Optimisation and Extension to 3D[END_REF] Shape representation and variation in Gaussian space is defined Inaccurate in large-scale shape variations, extension to 3D is difficult, need of training Contour levelset [START_REF] Kachouie | An Elliptical Level Set Method for Automatic TRUS Prostate Image Segmentation[END_REF] Contour implicitly defined, easy extension to 3D Depends on reliable edge information, slow in speed Curve fitting [START_REF] Hu | Prostate Surface Segmentation from 3D Ultrasound Images[END_REF], [START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF], [START_REF] Badiei | Prostate Segmentation in 2D Ultrasound Images Using Image Warping and Ellipse Fitting[END_REF], [START_REF] Mahdavi | Semi-Automatic Segmentation for Prostate Interventions[END_REF] Easy to implement, fast Rigid shape structure, reliable edge information is necessary Region Atlas [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF], [START_REF] Dowling | Fast automatic multi-atlas segmentation of the prostate from 3d mr images[END_REF] Automatic, robust to contrast differences, incorporate prior shape and intensity information Building atlas is not trivial and prone to registration errors, slow in speed of segmentation Graph partitioning [START_REF] Zouqi | Prostate Segmentation from 2D Ultrasound Images Using Graph Cuts and Domain Knowledge[END_REF] Efficient optimization, region based information could be incorporated Extracting edges in an image using gradient filters like Prewitt, Robert, Sobel, Shen and Castan and Canny is a popular practice in image processing. However, in the presence of noise gradient filters often detect false edges and also the detected edges are often broken. Although computationally expensive edge linking algorithms have to be designed to produce connected edges, in most cases it is necessary to combine edge based algorithms with intensity based and texture based information for accurate segmentation [START_REF] Pham | Current Methods in Medical Image Segmentation[END_REF].

TRUS

Prostate segmentation based on edge information seems to be particularly difficult in TRUS images. Traditional edge detection filters fail to obtain accurate edges due to the low contrast, speckle and other imaging artifacts like shadow regions. To overcome these problems, Liu et al. [START_REF] Liu | Computerised Prostate Boundary Estimation of Ultrasound Images Using Radial Bas-Relief Method[END_REF] propose to use a radial bass relief representation of the prostate, which consists in superimposing the original image with a zoomed negative of the same. Kwoh et al. [START_REF] Kwoh | Outlining the Prostate Boundary Using the Harmonics Method[END_REF] used harmonics from the Fourier transform to reduce spurious edges of this representation. Other approaches aim to reduce the speckle from the original image. For instance, Aarnink et al. [START_REF] Aarnink | Edge Detection in Prostatic Ultrasound Images Using Integrated Edge Map[END_REF] used local standard deviation to identify homogeneous and heterogeneous regions in the image in a multi-resolution framework, and this information was considered for detecting the prostate boundary with more reliability. In contrast, Pathak et al. [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] reduced speckle by applying a stick filter based on the non-zero correlation value of speckle over large distances. The intensity value of the central pixel was replaced by the average of the intensity values in the horizontal, vertical and diagonal directions of a given size. The resulting image was further smoothed using an anisotropic diffusion filter. Some basic prior knowledge of the prostate, such as shape and echo pattern, is used to detect the most probable edges describing the prostate. Finally, patientspecific anatomic information is integrated during manual linking of the detected edges to segment the prostate.

MRI

The use of typical edge detector operators in MR images can produce many false edges due to the high soft tissue contrast. Hence, Zwiggelaar et al. [START_REF] Zwiggelaar | Semi-Automatic Segmentation of the Prostate[END_REF] used first and second order Lindeberg directional derivatives [START_REF] Lindeberg | Edge Detection and Ridge Detection With Automatic Scale Selection[END_REF], in a polar coordinate system to identify the edges. An inverse transform of the longest curve selected after non maximal suppression of disconnected curves in the vertical direction was used to obtain the prostate boundary. On the other hand, Samiee et al. [START_REF] Samiee | Semi-Automatic Prostate Segmentation of MR Images Based on Flow Orientation[END_REF] used prior information of the prostate shape to refine the prostate boundary. Average gradient values obtained from a moving mask (guided by prior shape information) were used to trace out the prostate boundary. In a similar way, Flores-Tapia et al. [START_REF] Flores-Tapia | Semi Automatic MRI Prostate Segmentation Based on Wavelet Multiscale Products[END_REF] used a priori shape information of the prostate to trace out the boundary by the movement of a small mask on a feature space constructed from the product of the detail coefficients of the Haar wavelets in a multi-resolution framework.

Probabilistic filtering

Probabilistic filters like the Kalman filter [START_REF] Welch | An introduction to kalman filter[END_REF], the probabilistic data association filter (PDAF) [START_REF] Rasmussen | Probabilistic Data Association Methods for Tracking Complex Visual Objects[END_REF] and particle filters [START_REF] Djuric | Particle Filtering[END_REF] have been successfully used to segment images. These methods model the boundary of an organ as a probabilistic trajectory of a moving object where the motion is governed by a dynamic model subject to a particular uncertainty. Segmentation algorithms based on probabilistic filters are fast as no optimization framework is necessary [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF]. However, these methods may be sensitive to the initialization and the extension to 3D segmentation is complicated. Hence, to the best of our knowledge no method has been developed for 3D segmentation of the prostate in MRI.

TRUS

Abolmaesumi et al. [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF] used PDAF to segment the prostate in TRUS images. The stick filter [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] was used to reduce speckle and enhance the contrast. The authors argued that the boundary of the prostate was given by a trajectory of an object whose motion was governed by a model from a finite set of known models at any given radius. The models differed in uncertainty levels and structures, and switched between the models depending on the Markov transitional probability [START_REF] Li | Markov Random Field Models in Computer Vision[END_REF]. The authors assumed that the acceleration could be modeled by Gaussian noise and the model produced a noisy version of the actual position of the particle. Each trajectory was associated with a Kalman filter and the output was combined with an interactive multiple model and PDAF to estimate the boundary location. On the other hand, as the prostate in TRUS images is characterized by a hypoechoic mass surrounded by hyperechoic perimeter [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF], Sahba et al. [START_REF] Sahba | Segmentation of Prostate Boundaries Using Regional Contrast Enhancement[END_REF] used median filtering followed by top hat and bottom hat transforms to effectively separate bright areas from dark regions trapping the characteristic feature. Binary thresholding followed by morphological filtering produced a smooth contour of the boundary. Subsequently, a Kalman filtering followed by a fuzzy inference produced the final prostate contour.

Deformable model based segmentation

Deformable model segmentation techniques are influenced by theories from geometry, physics and mathematical optimization. Geometry imposes constraints on the model shape, physical theories guide the evolution of the shape in space, and optimization theory guides the model to fit the available data [START_REF] Bankman | Handbook of Medical Image Processing and Analysis[END_REF]. Deformable models are often associated with internal and external energies. External energies propagate the deformable model towards the object boundary and internal energies preserve smoothness of the contours during deformation. Internal and external energies associated with a deformable model are combined and included in an energy minimization framework to segment anatomical structures by warping to the edges with minimum deformation away from their mean shape. The methods proposed in a deformable model framework may be broadly classified into active contour models, deformable mesh, active shape models, level sets and curve fitting.

Active contour models

The active contour model (ACM) or snake was initially developed by Kass et al. [START_REF] Kass | Snakes: Active Contour Models[END_REF]. On initialization close to an edge, the active contour model evolves following the direction of the gradient in progressive deformation and stops at the edge. However, different external energies like balloon force [START_REF] Cohen | Finite Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images[END_REF], distance potential force [START_REF] Acton | Biomedical Image Analysis: Tracking[END_REF] and gradient vector flow [START_REF] Xu | Snakes, Shapes, and Gradient Vector Flow[END_REF] have been proposed to improve the capture range of active contour.

TRUS

Considering low contrast in TRUS images, localization of true prostate edge to produce external energy is a real challenge. Knoll et al. [START_REF] Knoll | Outlining of the Prostate Using Snakes with Shape Restrictions Based on the Wavelet Transform (doctoral thesis: Dissertation)[END_REF] used maxima of a multi-scale dyadic wavelet to determine prostate edges. Balloon force was used as the external force to deform a snake towards the maxima of the dyadic wavelet transform to segment the prostate in a multi-resolution framework. The form restricted contour deformation and its initialization by template matching are performed in a coarse to fine segmentation process based on a multiscale image edge representation containing the important edges of the image at various scales. To improve on the internal force of the ACM, Ladak et al. [START_REF] Ladak | Prostate Segmentation from 2D Ultrasound Images[END_REF] used cubic interpolation between four points selected by the user to produce a discrete dynamic contour (DDC) [START_REF] Lobregt | A Discrete Dvnamic Contour Model[END_REF]. Ding et al. [START_REF] Ding | Prostate Segmentation in 3D US Images Using the Cardinal-Spline Based Discrete Dynamic Contour[END_REF] used a cardinal spline to construct the initial contour of the prostate from three or more manually selected points located in the prostate boundary. The final contour produced in one slice was used to initialize the neighboring slices. To improve on the capture range of the gradient force, Jendoubi et al. [START_REF] Jendoubi | Segmentation of Prostate Ultrasound Images Using an Improved Snakes Model[END_REF] used gradient vector flow [START_REF] Xu | Snakes, Shapes, and Gradient Vector Flow[END_REF] computed from the gradient map obtained using Sobel and Laplacian of Gaussian as external force to drive active contour towards the boundary of the prostate. Zaim et al. [START_REF] Zaim | An Energy-Based Segmentation of Prostate from Ultrasound Images Using Dot-Pattern Select Cells[END_REF] used difference of Gaussian followed by non maximal suppression to detect dot patterns that were coherent with prostate tissue texture. An active contour constructed from manual delineations of prostate with dot pattern and gradient as external energy was used to segment the prostate.

Deformable mesh

Broadly, deformable meshes could be categorized into shape constrained deformable mesh or parametric deformable mesh. The methods included in the first category usually start dividing an initial manual segmentation in triangular and tetrahedral facets. Subsequently, similar to an ACM framework, the mesh deforms under the influence of internal and external forces to produce the desired segmentation. The objective of internal forces is to maintain a smooth surface while an external force drives the model towards the boundary of the organ. Often, the principal curvature of the surface is used as internal energy and the gradient of the image is one of the most popular choices for external energy. However, gradient is usually combined with texture to improve the segmentation results. On the other hand, in the parametric deformable model, the deformable mesh is constructed on the basis of a three dimensional geometrical figure like a sphere, ellipsoid or a cube that has a close resemblance with the organ. Geometrical parameters are used for internal energy computation. Either gradient or texture or both are used as external forces to deform the mesh.

TRUS

To maintain the prostate shape, Ghanei et al. [START_REF] Ghanei | A Three-Dimensional Deformable Model for Segmentation of Human Prostate from Ultrasound Images[END_REF] used a shape constrained de-formable mesh in a multi-resolution framework to achieve three dimensional segmentation of the prostate. Principal curvature of a surface from Todd and McLeod's method [START_REF] Todd | Numerical Estimation of the Curvature of the Surfaces[END_REF] was used as the internal force. The external force was computed from the expansion and the restoration model proposed by Rao and Ben-Arie [START_REF] Rao | Optimal Edge Detection Using Expansion Matching and Restoration[END_REF]. A Gaussian noise model was assumed, while an edge was considered as a step function.

An impulse response function was generated and applied to the volumetric data to generate the gradient. The gradient obtained in the process was used as the external force for mesh propagation for segmenting the prostate.

Active shape model

In absence of prior shape information, the final segmentation output of deformable models often vary widely from the shape of the anatomical structure. Cootes et al. [START_REF] Cootes | The Use of Active Shape Model for Locating Structures in Medical Images[END_REF] proposed the active shape model (ASM) that worked in the deformable model framework maintaining the principal modes of shape variations of the anatomical structures under study. Principal modes of shape variations are identified by principal component analysis (PCA) of the point distribution models (PDM) [START_REF] Cootes | The Use of Active Shape Model for Locating Structures in Medical Images[END_REF] aligned to a common reference frame with generalized Procrustes analysis. Shape space is assumed to be Gaussian and is represented with a mean shape added to weighted principal modes of variations identified from PCA.

With the initialization of the shape model, each landmark is searched within local vicinity to reach a better position with respect to the edges with a minimum displacement constraint that maintained the shape. Once all landmarks were displaced, scaling, rotation and translation parameters are chosen that minimizes the distance between the deformed contour and the shape model. Prior shape information incorporated in an active model makes it robust to noise and artifacts and produces improved segmentation results. In order to consistently set the corresponding landmarks automatically, the minimum descriptor length and Hill's algorithm [START_REF] Hill | A Framework for Automatic Landmark Identification Using A New Method of Nonrigid Correspondence[END_REF] are proposed. The different methods primarily differed in the optimization framework and the feature space used for modeling the deformation.

TRUS

Shen et al. [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF] used rotational invariant Gabor features computed with respect to the TRUS probe to characterize the prostate boundaries in multiple scales and multiple orientations. The Gabor features are further reconstructed to be invariant to the rotation of the ultrasound probe and incorporated in the prostate model as image attributes for guiding the deformable segmentation. The real and imaginary parts of Gabor features were used for smoothing and edge detection, respectively. A hierarchical deformation strategy is then employed, in which the model adaptively focuses on the similarity of different Gabor features at different deformation stages using a multiresolution technique from coarse to finer features, to achieve segmentation. Similarly, Betrouni et al. [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF] enhanced the prostate edge and reduced noise using a priori knowledge of the noise in TRUS images. An ASM was then used to produce the segmentation of the prostate. Hodge et al. [START_REF] Hodge | Prostate Boundary Segmentation from Ultrasound Images Using 2D Active Shape Models: Optimisation and Extension to 3D[END_REF] used the mean of manual segmentation from three experts to produce the ground truth value for prostate in TRUS images. An ASM was constructed from manually delineated contours after the reduction of noise using a median filter. The authors proposed to modify the PDM of Cootes to generate all plausible shapes by dividing the prostate mid gland images into three regions and creating three plausible prostate shapes for each.

MRI

Cootes et al. [START_REF] Cootes | The Use of Active Shape Model for Locating Structures in Medical Images[END_REF] proposed prostate segmentation as one of the applications of their generic ASM model in 2D. Zhu et al. [START_REF] Zhu | A Hybrid ASM Approach for Sparse Volumetric Data Segmentation[END_REF] proposed a hybrid of two and three dimensional ASM to segment the prostate in MR data sets. A three dimensional ASM was built that represented the shape variance of the prostate. In each iteration, the three dimensional ASM was updated by the final search result of two dimensional segmentation. The authors claimed that, their hybrid ASM had a superior performance in sparse three dimensional data sets as compared to 3D ASM, since 3D ASM built from sparse data was inefficient in detecting all possible modes of shape variations.

Edge based level sets

The level sets framework introduced by Osher et al. [START_REF] Osher | Fronts Propagating With Curvature-Dependent Speed: Algorithms based on Hamilton-Jacobi Formulation[END_REF] is a popular, powerful and efficient tool for medical image segmentation. This framework was developed to study curve propagation in higher dimensions.

The level set is allowed to expand starting from a seed point in a direction normal to the curve surface that produces the segmented contour, with a speed inversely proportional to the intensity gradient. The evolution finally stops where the intensity difference is highest in a local neighborhood. Hence, the propagation of the curve in a level set framework may help in finding an object boundary, and allows efficient curve splitting and merging based on topological changes.

TRUS

Considering intensity heterogeneity of the prostate gland, it is difficult to segment prostate with traditional level set initialized on gray-scale images. Hence, Kachouie et al. [START_REF] Kachouie | An Elliptical Level Set Method for Automatic TRUS Prostate Image Segmentation[END_REF] used Gaussian filtering followed by morphological filtering to classify the mid gland image into prostate and non prostate regions. An elliptical level set automatically initialized inside the prostate region was used to segment the prostate using first and second order moments of a Gaussian probability density function. The authors then used modified local binary patterns (LBP) to extract texture features of the prostate gland in TRUS images [START_REF] Kachouie | A Medical Texture Local Binary Pattern For TRUS Prostate Segmentation[END_REF]. Gradient magnitude information of the modified LBP map was used as the external force to drive the elliptical level set to convergence, thereby segmenting the prostate.

Curve fitting

Parametric curves like splines, ellipses and Bézier curves are often used to segment the prostate due to a close resemblance between the central gland of the prostate and an elliptical curve. Curve parameters are used as internal force and gradient as external force to deform the curve towards the prostate boundary.

TRUS

Hu et al. [START_REF] Hu | Prostate Surface Segmentation from 3D Ultrasound Images[END_REF] used an ellipsoid, initialized from manual delineations of the limits of the axes, to produce 3D prostate segmentation. Ellipsoid warping using thin plate splines transformation was used to map the user selected six control points to the end of the semi major axis of the ellipsoid to ensure a better fitting. The deformation of the ellipsoid was influenced by the internal and external forces to produce the segmentation. In a similar way, Ding et al. [START_REF] Ding | Slice-Based Prostate Segmentation in 3D US Images Using Continuity Constraint[END_REF] used a deformable super ellipse to just obtain an initial estimate of the prostate contour. Subsequently, the initial parameters of the super ellipse and gradient information of the image were jointly optimized to produce the final segmentation. To reduce propagation errors, a continuity constraint based on an autoregressive model was imposed on the initialization of the contour in new slices. Badiei et al. [START_REF] Badiei | Prostate Segmentation in 2D Ultrasound Images Using Image Warping and Ellipse Fitting[END_REF] also used an elliptical curve to segment the prostate. The ellipse was fitted through six user defined points.

The deformation of the prostate was modeled with a sine function in the angular direction and with a Gaussian function in the radial direction. The warping function was built using these two functions to create an elliptical shape for the prostate. Finally, segmentation of the prostate was achieved by ellipse fitting to the prostate boundary obtained by interacting multiple modes PDAF [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF] and reverse warping. In contrast to these works, Saroul et al. [START_REF] Saroul | Prostate Segmentation in Echographic Images: A Variational Approach Using Deformable Super-Ellipse and Rayleigh Distribution[END_REF] used a tapered super ellipse to segment the prostate. The prostate gland was divided into eight octants and the intensities of each octant were modeled using a Rayleigh distribution. The tapered super ellipse was combined with the probability density functions of the intensities of the prostate and non prostate region in an energy optimization framework to segment the prostate region. Mahdavi et al. [START_REF] Mahdavi | Semi-Automatic Segmentation for Prostate Interventions[END_REF] used a similar tapered ellipsoid to segment the prostate. The authors used untapering and warping of the image to make the shape of the prostate elliptical. Probe center as well as the bottom, center, middle right, and bottom right of the prostate gland were selected by the user. The image was then transformed to polar coordinates with the center of the probe as the coordinate center. This aided in untapering and warping of the image. After initial fitting, a deformation model was used to get the final fitting of the prostate boundary traced by interacting multiple modes PDAF [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF]. The obtained ellipse was used to initialize other slices of the ellipsoid. The process continued for all the slices to obtain a segmented prostate in 3D.

Region based segmentation

Predominant intensity distributions of the prostate region in different imaging modalities have been exploited by researchers to develop region based segmentation algorithms. Region based segmentation methods are further categorized into atlas, graph partitioning and level set methods.

Atlas

An atlas is created from a set of manual segmentations of an anatomical structure registered to a common coordinate frame. The atlas is then used as a reference to segment images of a new patient. Therefore, in atlas based segmentation, the segmentation problem is treated as a registration problem, since the segmentation is based on finding an one-to-one transformation mapping a pre-segmented atlas image to the new target image. Atlas guided segmentation is well suited for segmentation of structures that are stable over a large population, like the human brain [START_REF] Cabezas | A review of atlas-based segmentation for magnetic resonance brain images[END_REF].

MRI

Klein et al. [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF] followed a multi-atlas approach to segment the prostate. Affine registration and subsequently a non rigid registration using cubic B-splines [START_REF] Rueckert | Diffeomorphic Registration Using B-Splines[END_REF] in a multi-resolution framework was used to register the training volumes to the test volumes. Corresponding transformation was applied to the label images of the training dataset. In the next step the most similar atlas scans were selected based on the measure of similarity computed from normalized mutual information. To combine these atlas scans to a single segmentation, majority voting and the STAPLE algorithm are used to produce the final segmentation. Recently, Dowling et al. [START_REF] Dowling | Fast automatic multi-atlas segmentation of the prostate from 3d mr images[END_REF] improved on the results obtained by [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF] by introducing a pre-processing step of bias field correction, histogram equalization and anisotrpic diffusion smoothing. Dowling et al. then used rigid, affine and diffeomorphic demons registration to generate multiple labels of the test image. The most similar labels were identified and fused to generate the final segmentation. Langerak et al. [START_REF] Langerak | Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (simple)[END_REF] proposed a new schema for fusion of the labels in a multi atlas segmentation framework. They proposed to combine segmentation result of all the labels to produce the target label. Each of the labeled images of each of the atlas was compared to the target label. Labels below a certain threshold were discarded and the target label was re-estimated with the selected labels. The process continues in an iterative manner to provide the final estimated segmentation label.

Graph partition

In graph based segmentation methods pixels or group of pixels or voxels are considered as nodes while edges (gradients) between pixels are often considered as costs.

The graph is then partitioned by minimizing a cost function and closely related pixels are grouped together. Different graph partitioning algorithms like minimum spanning tree, minimum cut, and normalized cuts may be used for such purpose [START_REF] Boykov | Graph Cuts in Vision and Graphics: Theories and Applications[END_REF].

TRUS

Zouqi et al. [START_REF] Zouqi | Prostate Segmentation from 2D Ultrasound Images Using Graph Cuts and Domain Knowledge[END_REF] built a graph partition scheme to segment the prostate. The graph was built with nodes and edges. Pixels were the nodes while horizontal edges that connected these nodes represented edge discontinuity penalties. User defined pixels from the object and the background were used to build two special nodes: the source and the sink terminal. The max flow algorithm [START_REF] Boykov | An Experimental Comparison of Mincut/Max-flow Algorithms for Energy Minimization in Vision[END_REF] gradually increased the flow sent from the source to the sink along the edges in the graph given their costs. Upon termination, the maximum flow saturated the graph. The saturated edges corresponded to the minimum cost cut giving an optimal segmentation. The initial contour obtained after graph cut segmentation was further refined in a fuzzy inference framework that determined the membership of a pixel based on the region based statistics.

Region based level sets

In contrast with the traditional boundary based level sets, Chan and Vese [START_REF] Chan | Active Contours Without Edges[END_REF] used region based statistics in their energy minimization criteria to propagate the level set and segment the image. The method obtained superior results in the absence of strong edges and in presence of white noise since the stopping criteria was dependent on region based statistics.

TRUS

To produce a uniform region for the prostate, Fan et al. [START_REF] Fan | 3D Prostate Surface Detection from Ultrasound Images Based on Level Set Method[END_REF] set the value of a cubical voxel to 0 if the difference between the minimum and the maximum intensity values in the voxel was below 2. The value was set to 1 if the difference was greater than 2 but less than a threshold. This fast discriminative approach was used to extract the prostate region and used in a region based level set framework to segment the prostate in three dimensions.

Supervised and un-supervised classification based algorithms

In pattern recognition (PR) a feature could be defined as a measurable quantity that could be used to distinguish two or more regions. More than one feature could be used to differentiate regions and an array of these features is known as a feature vector. The vector space associated with feature vectors is known as Classifiers use a set of training data with labeled objects as priori information to build a predictor to assign a label to future un-labeled observations. In contrast, in clustering methods a set of feature vectors are given and the goal is to identify groups or clusters of similar objects on the basis of the feature vector associated with each. Proximity measures are used to group data into clusters of similar types.

Classifier based segmentation

In classifiers based segmentation the prostate is seen as a prediction or learning problem. Each object in a training set is associated with a response variable (class label) and a feature vector. The training set is used to build a predictor that can assign class label to a object on the basis of the observed feature vector.

TRUS

Intensity heterogeneity, unreliable texture features and imaging artifacts pose challenges in the feature space to partition. Zaim [START_REF] Zaim | Automatic Segmentation of the Prostate from Ultrasound Data Using Feature-Based Self Organizing Map[END_REF] used texture features, spatial information and gray-level values in a self organizing map neural network to segment the prostate. In a more recent work [START_REF] Zaim | Feature Based Classification of Prostate US Images Using Multiwavelet and Kernel SVM[END_REF] the authors used entropy and energy of symmetric, orthonormal, and second order wavelet coefficients [START_REF] Geronimo | Fractal Functions and Wavelet Expansions Based on Several Scaling Functions[END_REF] of overlapping windows in a support vector machine (SVM) classifier. Mohammed et al. [START_REF] Mohamed | Prostate Tissue Characterization Using TRUS Image Spectral Features[END_REF] used spatial and frequency domain information from multi-resolution Gabor filters and prior knowledge of prostate location in TRUS images to identify the prostate. Parametric and non parametric estimation of power spectrum density of the Fourier transform along with ring and wedge filter [START_REF] Randen | Filtering for Texture Classification: A Comparative Study[END_REF] of the region of interest (ROI) were used as feature vectors to classify TRUS images into prostate and non prostate region using non linear SVM.

Clustering based segmentation

The goal of clustering based methods is to determine intrinsic grouping in a set of un-labeled data based on some distance measures. Each data is associated with a feature vector and the task is to identify groups or clusters of similar objects on the basis of the set of feature vectors. The number of groups is assumed to be known and implicitly one must select the relevant feature, distance measure and the algorithm to be used.

TRUS

Richard et al. [START_REF] Richard | Automated Texture Based Segmentation of Ultrasound Images of the Prostate[END_REF] used the mean shift algorithm [START_REF] Comaniciu | Mean Shift: A Robust Approach Toward Feature Space Analysis[END_REF] in texture space to determine the mean and covariance matrix for each cluster. A probabilistic label was assigned to each pixel determining the membership of a pixel with respect to every cluster. Finally, a compatibility coefficient and pixel spatial information was used for probabilistic relaxation and refinement of the prostate region.

Hybrid segmentation

Combining a priori boundary, shape, region and feature information of the prostate gland may improve segmentation accuracy. This section discusses the methods that have combined two or more of the methods presented in previous sections.

TRUS

As discussed a mid gland image of the prostate in axial slices in TRUS images is often characterized by a hypoechoic mass surrounded by a hyperechoic halo. In order to capture this feature, Liu et al. [START_REF] Liu | Automatic Segmentation of Prostate Boundaries in Transrectal Ultrasound (TRUS) Imaging[END_REF] proposed to use radial search from the center of the prostate to determine the edge points of the prostate. The key boundary point was identified from the largest variation in gray value in each line. An average shape model constructed from manually segmented contours was used to refine the key points. A similar schema was adopted by Yan et al [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. In this case, contrast variations in normal vector profiles perpendicular to the PDM were used to automatically determine salient points and produce prostate boundaries. Salient points were determined by discarding points that fall in shadow regions. Prior shape information of the prostate shape aided determining the missing points in shadow regions in TRUS images. Optimal search performed through vector profiles perpendicular to the salient points was used to determine prostate boundary with a discrete deformable model in a multi-resolution, energy minimization framework.

Modeling shape and texture features and using them to segment a new image has been used by many researchers. The schema primarily varied in the approach adopted for the creation of the shape and the texture model. For instance, Zhan et al. [START_REF] Zhan | Automated Segmentation of 3D US Prostate Images Using Statistical Texture-Based Matching Method[END_REF] proposed to model the texture space by classifying into prostate and non prostate regions the texture features captured by rotational invariant Gabor filter by means of a SVM. This classified feature space was subsequently used as an external force in a deformable model framework to segment the prostate. In their consequent work [START_REF] Zhan | An Efficient Method for Deformable Segmentation of 3D US Prostate Images[END_REF], the authors proposed to speed-up the process by using Zernike moments [START_REF] Ghosal | Orthogonal Moment Operators for Subpixel Edge Detection[END_REF] to detect edges in low and middle resolutions and maintaining the texture classification using Gabor features and SVM. In a different way [START_REF] Zhan | Increasing Efficiency of SVM by Adaptively Penalizing Outliers[END_REF], the authors also proposed to reduce the number of support vectors by introducing a penalty term in the objective function of the SVM, which penalizes and rejects the outliers. Finally, Zhan et al. [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF] proposed to combine texture and edge information to improve the segmentation accuracy. Multi-resolution rotational invariant Gabor features of the prostate and non-prostate regions were used to train a Gaussian kernel SVM system to classify textures of prostate regions. In the deformable segmentation procedure, SVM were used to label voxels around the surface of deformable model as prostate or non prostate tissues. Subsequently, the surface of the deformable model is driven to the boundary by the deformation force of labeled prostate tissues. The step of tissue labeling and the step of label-based surface deformation were dependent on each other, the process was carried out iteratively until convergence.

A similar schema was adopted by Diaz and Castaneda [START_REF] Diaz | Semi-automated Segmentation of the Prostate Gland Boundary in Ultrasound Images Using a Machine Learning Approach[END_REF]. Asymmetric stick and anisotropic filters were firstly applied to reduce speckle in TRUS images. A DDC was produced using cubic interpolation of four points initialized by the user. The DDC deformed under the influence of internal force, gradient magnitude and damping forces to produce the contour of the prostate. Features such as intensity mean, variance, output of back projection filter, and stick filter were used to construct the feature vectors. The pixels were classified into prostate and non prostate regions using SVM. Subsequently, DDC was automatically initialized from the prostate boundary and used to obtain the final contour of the prostate. Cosío et al. [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF] used position and gray level value of a prostate in TRUS image in a three Gaussian mixture model to cluster prostate, non prostate tissues and to identify halo around the prostate in TRUS images. A Bayes classifier was used to identify prostate region. After pixel classification the ASM is initialized with the binary image using a global optimization method. The optimization problem consists of finding the optimum combination of four pose and two shape parameters, which correspond to an approximate prostate boundary in the binary image. A multi population genetic algorithm with four pose and ten shape parameters was used to optimize an ASM in a multi-resolution framework to segment the prostate.

Another common hybrid approach is to use both shape and intensity distribution to segment the prostate. Medina et al. [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF] used an AAM framework [START_REF] Cootes | Active Appearance Models[END_REF] to model the shape and the texture space of the prostate. In this framework the Gaussian model of the shape and intensity created from PCA analysis is combined to produce a combined mean model. The prostate was segmented exploiting the prior knowledge of the nature of the optimization space in minimizing the difference between the target image and the mean model. Gong et al. [START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF] proposed to use a deformable super ellipse to produce a shape model of the prostate. Using the deformable super ellipse as the prior shape model for the prostate, the end goal was to find the optimal parameter vector that best describes the prostate in a given unsegmented image. The initial parameters were used in maximum a posteriori (MAP) framework to obtain the optimized parameters for the ellipse.

Later, Tutar et al. [START_REF] Tutar | Semiautomatic 3D Prostate Segmentation from TRUS Images Using Spherical Harmonics[END_REF] used the average of three manually delineated prostate contours to construct a three dimensional mesh with spherical harmonics to represent the average model of the prostate. With 8 harmonics, a feature vector of 192 elements was reduced to 20 using PCA. Users initialize the algorithm by outlining the prostate boundaries in mid gland axial and sagittal images. Therefore, the problem of finding the shape parameter vector that would segment the prostate in the spatial domain was reduced to find the optimal shape parameters in parametric domain that maximized the posterior probability density of a cost function, which measures the degree of agreement between the model and the prostate edge in the image. Yang et al. [START_REF] Wang | Segmentation of Prostate from 3D Ultrasound Volumes Using Shape and Intensity Priors in Level Set Framework[END_REF] proposed to use min/max flow [START_REF] Malladi | Image Processing via Level set Curvature Flow[END_REF] to smooth the contours of the 3D model of the prostate created from 2D manual delineation. The primary modes of shape variations were identified with PCA and morphological filters were used to extract region based information of the prostate gland. The shape model and region based information were subsequently combined in a Bayesian framework to produce an energy function, which was minimized in a level set framework.

Garnier et al. [START_REF] Garnier | Prostate segmentation in HIFU therapy[END_REF] used 8 user defined points to initialize a 3D mesh of the prostate. Two algorithms were used to determine the final segmentation of the prostate. First, DDC with edge as external force and the 6 central gland user de-fined points as landmarks was used to deform the mesh to segment the prostate. Next, the initial mesh was used to create the graph and in second stage image features like gradients were introduced to build the cost function. Finally, graph-cut was used to determine the prostate volume. The graph cut results were refined with DDC to improve the results.

MRI

Prior shape and size information of the prostate were exploited by Vikal et al. [START_REF] Vikal | Prostate Contouring in MRI Guided Biopsy[END_REF] to build an average shape model from manually delineated contours. The authors used the Canny filter to determine edges after pre-processing the images with a stick filter to suppress noise and enhance the contrast. The average shape model was used to discard pixels that did not follow similar orientation as the model. The obtained contour was further refined by the removal of gaps using polynomial interpolation. The segmented contours obtained in the middle slices were used to initialize slices lying above and below the central slice.

The use of a Bayesian framework to model the texture of the prostate is common in MR images. For instance, Allen et al. [START_REF] Allen | Differential Segmentation of the Prostate in MR Images Using Combined 3D Shape Modelling and Voxel Classification[END_REF] proposed to segment the prostate in an expectation maximization (EM) framework treating the three distinctive peaks in intensity distribution as mixture of three Gaussians (background, central region and periphery of the prostate). A shape restricted deformable model with the clustered pixels as a deformation force was then used to segment the prostate. Similarly, in Makni et al. [START_REF] Makni | Combining a Deformable Model and a Probabilistic Framework for an Automatic 3D Segmentation of Prostate on MRI[END_REF], the intensities of the prostate region were modeled as a mixture of Gaussians. They proposed a Bayesian approach where the prior probability labeling of the voxels was achieved by using a shape restricted deformable model and Markov field modeling. The conditional probability was associated with the modeled intensity values, and the segmentation was achieved by estimation of an optimum label for prostate boundary pixels in a MAP decision framework.

Although atlas based registration and segmentation of the prostate has become popular in recent time, the obtained segmentation results had to be refined with a deformable model to improve the accuracy. Martin et al. [START_REF] Martin | Atlas-based Prostate Segmentation Using an Hybrid Registration[END_REF] used a hybrid registration minimizing intensity and geometry energies for registering the atlas. The minimization of the intensity based energy aimed at matching the template image with the reference image while the minimization of the geometric energy matched the model points of the template image to the scene points belonging to the reference image. Finally, a shape constrained deformable model was used to refine the results. More recently, Martin et al. [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF] used a probabilistic atlas to impose further spatial constraints and segment the prostate in three dimensions.

Shape and texture modeling of the prostate were merged in the work of Tsai et al. [START_REF] Tsai | A Shape-Based Approach to the Segmentation of Medical Imagery Using Level Sets[END_REF], who used a shape and region based level set framework to segment prostate in MR images. One of the contours was fixed and used as the reference system where all the other contours were affine transformed to minimize their difference in a multi resolution approach. PCA of the shape variability captured the primary modes of variations and was also incorporated in the level set function, along with region based information such as area, sum of intensities, average intensity and variance information. The minimization of the level set objective function produced the segmented prostate. The authors also suggested a coupled level set model of the prostate, the rectum, and the internal obturator muscles from MR images to segment these structures simultaneously [START_REF] Tsai | Coupled Multi-shape Model and Mutual Information for Medical Image Segmentation[END_REF]. The algorithm was made robust by allowing the shapes to overlap with each other, and the final segmentation was achieved by maximizing the mutual information of the three regions. Similarly, Liu et al. [START_REF] Liu | Unsupervised Segmentation of the Prostate Using MR Image Based on Level Set with a Shape Prior[END_REF] used a deformable ellipse to segment prostate boundary after Otsu thresholding [START_REF] Otsu | A Threshold Selection Method from Gray-Level Histograms Systems[END_REF] of the image in prostate and non prostate region. A shape constrained level set initialized from the elliptical fitting of the prostate was used to further refine the results. Finally, post processing of the gradient map of the prostate and the rectum produced the final segmentation. Firjani et al. [START_REF] Firjani | Novel stochastic framework for accurate segmentation of prostate in dynamic contrast enhanced mri[END_REF] modeled the background and the foreground pixels with Gaussian mixture Markov random field and used the information of probability of a pixel being prostate in building the shape model. The shape and the intensity were jointly optimized with a graph cut based algorithm. The authors extended their work for 3D segmentation of the prostate [START_REF] Firjani | A new 3d automatic segmentation framework for accurate segmentation of prostate from dce-mri[END_REF]. Zhang et al. [START_REF] Zhang | Boundary delineation in prostate imaging using active contour segmentation method with interactively defined object regions[END_REF] proposed an interactive environment for prostate segmentation. Region and edge based level sets were used to segment the prostate from the background depending on foreground and background region based information provided by the user.

Gao et al. [START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF] represented the shapes of a training set as point clouds. Particle filters were used to register clouds of points created from prostate volumes to a common reference to minimize the difference in pose. Shape priors and lo-cal image statistics were incorporated in an energy function that was minimized to achieve prostate segmentation in a level set framework. More recently, Toth et al. [START_REF] Toth | Accurate prostate volume estimation using multifeature active shape models on t2-weighted mr[END_REF] used a series of 50 Gaussian kernels of variable size to extract prostate texture features. ASM constructed from manually delineated contours of training images was automatically initialized depending on the most probable location of the prostate boundary to achieve segmentation. Later, Toth et al. [START_REF] Toth | A magnetic resonance spectroscopy driven initial-ization scheme for active shape model based prostate segmentation[END_REF] in addition to intensity values, used mean, standard deviation, range, skewness, and kurtosis of intensity values in a local neighborhood to propagate ASM automatically initialized from magnetic resonance spectroscopy (MRS) information. MRS information was clustered using replicated k-means clustering to identify prostate in mid slice to initialize multi feature ASM. More recently Toth et al. [START_REF] Toth | Multi-feature landmark-free active appearance models: Application to prostate mri segmentation[END_REF] developed a multi feature landmark free active appearance model in which shape model was derived in a levelset based framework that was propagated by multiple image derivative attributes derived from mean, standard deviation, and Sobel features of a region. Chowdhury et al. [START_REF] Chowdhury | Concurrent segmentation of the prostate on mri and ct via linked statistical shape models for radiotherapy planning[END_REF] used probabilistic atlas and random forest based classification to segment prostate. Region based levelset were used to generate a 3D volume and ASM was used to refine the segmentation results. Khurd et al. [START_REF] Khurd | Facilitating 3d spectroscopic imaging through automatic prostate localization in mr images using random walker segmentation initialized via boosted classifiers[END_REF] localized the center of the prostate gland with Gaussian mixture model and expectation maximization based clustering after reducing magnetic bias in the images. Thresholding on the probabilistic map of the prostate obtained with random walker based segmentation algorithm [START_REF] Grady | Random walks for image segmentation[END_REF] to segment the prostate.

Validation and qualitative performance evaluation

The performance of prostate segmentation algorithms is usually evaluated comparing the output of the method with a ground truth (gold standard) obtained from manual delineations of the prostate done by experienced radiologists. Hodge et al. [START_REF] Hodge | Prostate Boundary Segmentation from Ultrasound Images Using 2D Active Shape Models: Optimisation and Extension to 3D[END_REF] advised to use the mean of the manual segmentations of different radiologists and/or of the same radiologist at different times to reduce inter and intra observer variations in preparation of the ground truth value.

Analyzing the literature we have seen that the evaluation metrics could be cate-gorized into qualitative and quantitative based metrics. In a qualitative evaluation, the obtained contour is visually compared with the ground truth value. In contrast, for quantitative evaluation, an error between the obtained contour and the ground truth is numerically computed. Typically, these error metrics could be classified into contour based, area based and volume based methods. Contour based metrics rely on computing how close the ground truth and the obtained contours are. Typical metrics used are the Hausdorff distance (HD) [START_REF] Belogay | Calculating the hausdorff distance between curves[END_REF], the mean absolute distance (MAD) [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF], mean distance (MD) [START_REF] Sahba | Segmentation of Prostate Boundaries Using Regional Contrast Enhancement[END_REF], maximum distance (MaxD) [START_REF] Liu | Automatic Segmentation of Prostate Boundaries in Transrectal Ultrasound (TRUS) Imaging[END_REF], and root mean square error (RMSQ) [START_REF] Zhu | A Hybrid ASM Approach for Sparse Volumetric Data Segmentation[END_REF]. Area based errors are based on computing how much the ground truth and the obtained areas overlap. It can be measured by the Dice similarity coefficient (DSC) [START_REF]MICCAI. 2009 prostate segmentation challenge MICCAI[END_REF], area accuracy [START_REF] Badiei | Prostate Segmentation in 2D Ultrasound Images Using Image Warping and Ellipse Fitting[END_REF], area sensitivity [START_REF] Badiei | Prostate Segmentation in 2D Ultrasound Images Using Image Warping and Ellipse Fitting[END_REF], area specificity [START_REF] Diaz | Semi-automated Segmentation of the Prostate Gland Boundary in Ultrasound Images Using a Machine Learning Approach[END_REF], area overlap [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF], area overlap error [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF], and area error [START_REF] Sahba | Segmentation of Prostate Boundaries Using Regional Contrast Enhancement[END_REF] metrics. Finally, volume overlap error and difference, average difference [START_REF] Knoll | Outlining of the Prostate Using Snakes with Shape Restrictions Based on the Wavelet Transform (doctoral thesis: Dissertation)[END_REF], overlap [START_REF] Tutar | Semiautomatic 3D Prostate Segmentation from TRUS Images Using Spherical Harmonics[END_REF], detection, false detection, centroid distance [START_REF] Freedman | Model-Based Segmentation of Medical Imagery by Matching Distributions[END_REF], and similarity [START_REF] Chen | Segmenting the prostate and rectum in ct imagery using anatomical constraints[END_REF] are used for computing a 3D overlapping error. However, DSC, specificity, sensitivity, accuracy and HD of voxels are also used in terms of voxels to determine volumetric overlap [START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF]. The evaluation metrics for prostate segmentation are enlisted in Table 2.2, 2.3 and 2.4 where each table shows the metrics for contour, region and volume respectively.

Ideally a comparison of different state-of-the-art prostate segmentation methodologies on a public dataset should have been done to evaluate the performance of the state-of-the-art methods. However, a quantitative comparison of different methodologies is difficult in absence of public software, data sets and standardized evaluation metrics. In addition, the methods are developed using wide variety of algorithms with specific application requirements. Hence, such a quantitative comparison of different prostate segmentation methods on the same dataset with some standardized metrics is extremely difficult as could be observed in some recently published works [START_REF] Martin | Atlas-based Prostate Segmentation Using an Hybrid Registration[END_REF][START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF][START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF][START_REF] Mahdavi | Semi-Automatic Segmentation for Prostate Interventions[END_REF][START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF][START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF][START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF][START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF]. Nevertheless, to have an overall quantitative estimate of the functioning of some of the state-of-the-art works in the literature we present the reported results in Tables 2.5, 2.5, 2.5, and 2.6 for TRUS and MRI respectively.

The index of the Tables is expanded below.

• The name of the first author has been used as a reference of the paper.

M D = 1 N N j=1 d j
[120], [START_REF] Knoll | Outlining of the Prostate Using Snakes with Shape Restrictions Based on the Wavelet Transform (doctoral thesis: Dissertation)[END_REF], [START_REF] Ladak | Prostate Segmentation from 2D Ultrasound Images[END_REF], [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF], [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF], [START_REF] Hodge | 3D Prostate Boundary Segmentation From Ultrasound Images Using 2D Active Shape Models[END_REF], [START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF], [START_REF] Hu | Prostate Surface Segmentation from 3D Ultrasound Images[END_REF], [START_REF] Liu | Automatic Segmentation of Prostate Boundaries in Transrectal Ultrasound (TRUS) Imaging[END_REF], [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF], [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF], [START_REF] Tang | Geometric Model Based Segmentation of the Prostate and Surrounding Structures for Image Guided Radiotherapy[END_REF] Mean absolute distance (MAD) [START_REF] Ladak | Prostate Segmentation from 2D Ultrasound Images[END_REF], [START_REF] Ding | Slice-Based Prostate Segmentation in 3D US Images Using Continuity Constraint[END_REF], [START_REF] Hodge | 3D Prostate Boundary Segmentation From Ultrasound Images Using 2D Active Shape Models[END_REF], [START_REF] Hu | Prostate Surface Segmentation from 3D Ultrasound Images[END_REF], [START_REF] Liu | Automatic Segmentation of Prostate Boundaries in Transrectal Ultrasound (TRUS) Imaging[END_REF], [START_REF] Gong | Prostate Ultrasound Image Segmentation Using Level set-Based Region Flow With Shape Guidance[END_REF], [START_REF] Tutar | Semiautomatic 3D Prostate Segmentation from TRUS Images Using Spherical Harmonics[END_REF], [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF], [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF], [START_REF] Vikal | Prostate Contouring in MRI Guided Biopsy[END_REF], [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF] [89] [134] Maximum distance (MaxD)

M AD = 1 N N j=1 |d j | [106],
M axD = max |d j |
[80], [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF], [START_REF] Hodge | 3D Prostate Boundary Segmentation From Ultrasound Images Using 2D Active Shape Models[END_REF], [START_REF] Hu | Prostate Surface Segmentation from 3D Ultrasound Images[END_REF], [START_REF] Liu | Automatic Segmentation of Prostate Boundaries in Transrectal Ultrasound (TRUS) Imaging[END_REF], [START_REF] Tutar | Semiautomatic 3D Prostate Segmentation from TRUS Images Using Spherical Harmonics[END_REF], [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF], [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF] Table 2.3: Evaluation metrics for area accuracy [121],

[48], [START_REF] Vikal | Prostate Contouring in MRI Guided Biopsy[END_REF], [START_REF] Liu | Unsupervised Segmentation of the Prostate Using MR Image Based on Level Set with a Shape Prior[END_REF] Sensitivity (SN) SN = T P T P +F N

[80], [START_REF] Badiei | Prostate Segmentation in 2D Ultrasound Images Using Image Warping and Ellipse Fitting[END_REF], [START_REF] Mohamed | Prostate Tissue Characterization Using TRUS Image Spectral Features[END_REF], [START_REF] Costa | Automatic Segmentation of Bladder and Prostate Using Coupled 3D Deformable Models[END_REF] Specificity (SP)

SP = T N T N +F P [11] Accuracy (AC) AC = T P +T N T P +T N +F P +F N [80],
[11], [START_REF] Mohamed | Prostate Tissue Characterization Using TRUS Image Spectral Features[END_REF] Overlap (OV) OV = T P F P +F N

[5], [START_REF] Zaim | An Energy-Based Segmentation of Prostate from Ultrasound Images Using Dot-Pattern Select Cells[END_REF], [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF], [START_REF] Zaim | Automatic Segmentation of the Prostate from Ultrasound Data Using Feature-Based Self Organizing Map[END_REF], [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF] Overlap Error (OE)

OE = 1 -Ov [125], [120] Surface distance (SD)
Given unsigned distance d s between between the algorithmic segmented surface and ground truth.

SD = 1 N s=1 N d s [40], [ 45]
, [START_REF] Li | Learning image context for segmentation of prostate in ct-guided radiotherapy[END_REF], [START_REF] Rousson | Constrained Surface Evolutions for Prostate and Bladder Segmentation in CT Images[END_REF], [START_REF] Freedman | Model-Based Segmentation of Medical Imagery by Matching Distributions[END_REF], [START_REF] Song | Optimal Graph Search Segmentation Using Arc-Weighted Graph for Simultaneous Surface Detection of Bladder and Prostate[END_REF] Table 2 [76],

.4: Evaluation metrics for volume accuracy

[91], [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF],

[50], [START_REF] Langerak | Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (simple)[END_REF],

[40], [START_REF] Acosta | Atlas based segmentation and mapping of organs at risk from planning ct for the development of voxel-wise predictive models of toxicity in prostate radiotherapy[END_REF], [START_REF] Feng | Segmenting ct prostate images using population and patient-specific statistics for radiotherapy[END_REF], [START_REF] Chen | Level Set Segmentation with Both Shape and Intensity Priors[END_REF], [START_REF] Li | Learning image context for segmentation of prostate in ct-guided radiotherapy[END_REF] Sensitivity (SN) 2×M SD [START_REF] Knoll | Outlining of the Prostate Using Snakes with Shape Restrictions Based on the Wavelet Transform (doctoral thesis: Dissertation)[END_REF],

SN = T P T P +F N [35], [ 94
[66], [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF],

[8], [START_REF] Freedman | Model-Based Segmentation of Medical Imagery by Matching Distributions[END_REF] Average difference (AVD)

AV D = M SV -ASV M SV [77],
[65], [START_REF] Hu | Prostate Surface Segmentation from 3D Ultrasound Images[END_REF] Overlap (VO)

V O = M SV ∩ASV M SV ∪ASV [137], [51], [134] Overlap error (VOE) V OE = 1 -V O [154] Error (VE) V E = M SV +ASV -2(M SV ASV ) M SV +ASV [90] Centroid dis- tance (VCD)
Given ground truth centroid c m and c a algorithmic segmented volume centroid. • The segmentation dimension (Dim) gives the output of a given segmentation methodology. The output can be in two (2D) or three (3D) dimensions.

V CD = |c m -c a | [117], [49]
• B/A indicates whether base and apex slices were considered for 2D segmentation.

• Pre-Proc indicates the type of pre-processing used in the method.

• In. indicates the use of endo-rectal coil in acquisition of MR images.

• • The automation (Auto) column specifies the degree of manual interaction that was necessary. The process is considered automatic if the degree of manual interaction was restricted to training.

• The measure column refers to the measures used by the authors to present their obtained results.

• The last column (Validation) gives the number of images or data sets (volumes) that were used to validate the developed algorithm.

The discussion on the evaluation procedures is given in Section 2.3.1. Afterwards, a discussion about choosing an appropriate method for a given imaging modality is carried out in Section 2.4.

Open problems

We have explained in previous sections the validation procedures followed by the researchers. From the reported results a set of of open problems are revealed. 1. Manual delineation of the prostate contours is considered to be the gold standard to which the result of a segmentation methodology is compared. Only few authors considered the mean of delineated contours by different experts and of the same expert at different time to reduce inter and intra observer variability of the process. Rasch et al. [START_REF] Rasch | Definition of the prostate in ct and mri: a multi-observer study[END_REF] quantified inter observer variabilities in MRI. He found that, the average ratio between the volume derived by one observer for a particular scan and patient and the average volume was 0.95, 0.97, and 1.08 for the three observers. Under such inter observer variabilities an interesting option could be the use of prostate phantom to validate volume information obtained using computer aided segmentation. Another interesting option could be validation of the annotated data by biopsy, histology or extracted prostate information.

2. The unavailability of public prostate databases makes quantitative comparison of the segmentation algorithms difficult. Moreover, the quality of results depend on both the scans and quality of contouring. Lately, MICCAI prostate challenge datasets for MRI are being used for comparison [START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF]. A public datasets of prostate images [START_REF]Prostate MRI database[END_REF] could also be used for validation. The quality of the images vary with MR and TRUS machines, as advanced machines produce images of superior quality. Thus, it becomes almost impossible to compare the performance of two algorithms separated by a span of significant number of years. 4. Very few fully automatic methods have been developed, and often manual initialization and sometimes manual editing is encouraged. This may be suitable for off-line procedures like the estimation of prostate volume, but unsuitable for on-line procedures like real time fusion of multi-modal images [START_REF] Xu | Real-time MRI/TRUS Fusion for Guidance of Targeted Prostate Biopsies[END_REF].

Choosing an appropriate segmentation method

Choice of a proper segmentation methodology is dependent on contexts like imaging modality and the final target application of the process. Hence, we have provided recommendation of selection of a particular segmentation technique based on these two basis. We have divided the section into TRUS and MRI subsections and have provided recommendations based on applications for both the modalities.

TRUS

TRUS image of a prostate has low contrast and the signal is often corrupted by speckle, shadow artifacts and micro-calcifications [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. There are two different ways to deal with speckle. One option is to minimize its effect in the image using, for example, stick filters [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF], that allow reducing speckle while enhancing the contrast of the image. The second option is to take benefit of this information, which can be done modeling speckle as a Rayleigh's distribution [START_REF] Saroul | Prostate Segmentation in Echographic Images: A Variational Approach Using Deformable Super-Ellipse and Rayleigh Distribution[END_REF]. Any of these options could be employed for pre-processing of the image and prepare it for further analysis.

Prostate volume determined from segmented TRUS images serves as an important parameter in determining presence of benign or malignant tumor during diagnosis of prostate diseases. Three commonly used prostate volume measurement techniques in TRUS are planimetry calculation, prolate ellipse volume calculation, and an ellipsoid volume measurement technique. Segmentation of prostate in 2D in the axial slices in the mid gland region to determine maximum area and height is useful in determining volume in all these techniques. Note in Table 2.5 that no one tried a pure region based approach to segment the prostate. This is due to the fact that these algorithms fragment the prostate into a large number of small regions due to the heterogeneity inside the prostate gland. Related with edge based approaches, we noted that pure contour based methods like edge detection [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] are being replaced (or expanded) with methods that combine prostate shape and region based information [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF][START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF][START_REF] Diaz | Semi-automated Segmentation of the Prostate Gland Boundary in Ultrasound Images Using a Machine Learning Approach[END_REF], providing a more robust approach in presence of speckle and low contrast. In contrast, Abolmaesumi et al. [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF] and Sahba et al. [START_REF] Sahba | Segmentation of Prostate Boundaries Using Regional Contrast Enhancement[END_REF] proposed an interesting option of modeling the prostate contour as a Gaussian distribution. Such assumption provides more robustness to contour based methods in low contrast. ASM is another edge based approach frequently used for prostate segmentation in 2D images. However, such models are dependent on reliable edge information and hence may be adversely affected in presence of shadow artifacts [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. Moreover, the automatic initialization and extension to 3D is difficult [START_REF] Zhu | A Hybrid ASM Approach for Sparse Volumetric Data Segmentation[END_REF]. However, shape constrained deformable models have been successfully employed by different authors [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF][START_REF] Tutar | Semiautomatic 3D Prostate Segmentation from TRUS Images Using Spherical Harmonics[END_REF][START_REF] Wang | Segmentation of Prostate from 3D Ultrasound Volumes Using Shape and Intensity Priors in Level Set Framework[END_REF] as observed in Table 2.5. Automatic delineation of the prostate in mid-gland images further reduces inter observer variabilities.

In prostate brachytherapies, oncologists should prepare a set of parallel TRUS ultrasound images and manually segment each 2D slice to obtain the prostate volume which is then used to plan the location of the seeds. Hence, fast semi-automatic or automatic prostate segmentation in 2D slices or 3D volume could be useful in such procedures. Mahdavi et al. [START_REF] Mahdavi | Semi-Automatic Segmentation for Prostate Interventions[END_REF] method of fitting an ellipsoid to prostate edges is a very useful method for such a scenario as the method has shown good volumetric overlap accuracy.

Automatic, and fast prostate segmentation from 2D US images is often necessary in image guided prostate biopsy or robot assisted surgery [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. DDC and super quadrics are computationally efficient procedures to segment the prostate in 2D. However, the fastest segmentation of the prostate contour had been reported using partial ASM [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF] and probabilistic filtering [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF]. Considering semi-automatic approach adopted by [START_REF] Abolmaesumi | Segmentation of Prostate Contours from Ultrasound Images[END_REF], the method developed by Yan et al. [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF] is well suited for real time segmentation of the prostate in two dimensions. Note that the speed of a given segmentation method could be improved if the method could be parallelized and implemented in graphical processing unit as well as if an off-line learning of the optimization space could be adopted. Moreover, segmentation of prostate in TRUS videos could be modeled as a tracking boundary problem to achieve near real time segmentation.

Supervised and un-supervised classification based methods have the advantage of being fully automatic [START_REF] Zaim | Automatic Segmentation of the Prostate from Ultrasound Data Using Feature-Based Self Organizing Map[END_REF][START_REF] Mohamed | Prostate Tissue Characterization Using TRUS Image Spectral Features[END_REF], although a training is necessary in the ones using a classifier. However, the intensity heterogeneity and unreliable texture of the prostate gland challenge again the development of a pure clustering or classification schema for the prostate. Therefore, an interesting and common option is the use of a clustering or/and classification schema for initial segmentation and subsequently use the obtained information for the initialization and/or propagation of a deformable model to produce the final segmentation [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF]. Another common hybrid approach is the integration of shape and intensity information jointly optimized in an AAM framework [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF]. Off-line learning of the optimization space aids in fast prostate segmentation. However, such methods are affected by large-scale contrast variances and use of texture information in place of raw intensity improve segmentation accuracies.

MRI

MR images of the prostate have better soft tissue contrast compared to TRUS images. However, insertion of the endorectal coil to enhance contrast in the prostate region introduces intensity inhomogeneities that may adversely affect the segmentation accuracy of algorithms dependent on pixel intensities. Viswanath et al. [START_REF] Viswanath | Empirical evaluation of bias field correction algorithms for computer aided detection of prostate cancer on t2w mri[END_REF] performed an extended review of three techniques [START_REF] Sled | A nonparametric method for automatic correction of intensity nonuniformity in mri data[END_REF], [START_REF] Styner | Parametric estimate of intensity inhomogeneities applied to mri[END_REF] and [START_REF] Cohen | Rapid and effective correction of rf inhomogeneity for high field magnetic resonance imaging[END_REF] applied to magnetic field bias field correction. The authors concluded that bias field correction algorithm should be application specific. For example it was observed that [START_REF] Sled | A nonparametric method for automatic correction of intensity nonuniformity in mri data[END_REF] performed best with the goal of identifying cancer and on the other hand bias field corrections can adversely affect clustering and classification based techniques of segmentation. It is to be noted that MR images with endorectal coil are relatively simpler to segment due to higher contrast of images around the prostate and well deformed shape of the rectum.

Prostate segmentation from MR images is frequently used for volume determination, surgical planing and multi-modal image registration. In all these applications prostate segmentation could be done automatically or semi-automatically. However, minimum human interaction is desired to minimize human induced variations and errors. In the last decade, deterministic and probabilistic atlases are frequently used for 3D segmentation of the prostate. Such methods are automatic, robust to intensity variability and to noise [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF]. Martin et al. used both deterministic [START_REF] Martin | Atlas-based Prostate Segmentation Using an Hybrid Registration[END_REF] and probabilistic atlas [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF] to segment the prostate, although the obtained segmentations were refined with a deformable model. Pair-wise atlas selection schema of Dowling et al. [START_REF] Dowling | Fast automatic multi-atlas segmentation of the prostate from 3d mr images[END_REF] has shown greater accuracy compared to [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF][START_REF] Dowling | Automatic atlas-based segmentation of the prostate[END_REF]. Hence, for atlases pairwise registration is better compared to average atlas based segmentation. In Table 2.6 we observe that Klein et al. [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF] with atlas based segmentation achieved an impressive overlap accuracy of 0.85 DSC value when validated with 50 data sets. However, Martin et al. [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF] with probabilistic atlas and deformable model based segmentation achieved similar overlap accuracy and a good contour accuracy values when validated with 36 data sets. Atlas based methods [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF] and probabilistic modeling of the prostate region [START_REF] Martin | Atlas-based Prostate Segmentation Using an Hybrid Registration[END_REF] provide a more robust approach in presence of these inhomogeneities.

Deformable models are frequently used for prostate segmentation. Makni et al. [START_REF] Makni | Combining a Deformable Model and a Probabilistic Framework for an Automatic 3D Segmentation of Prostate on MRI[END_REF] used information coming from an initial classification scheme to initialize a deformable model. Note also that automatic methods are primarily developed using classifiers, atlas and deformable models. It has to be noted that anatomical structures around the prostate may affect the prostate deformation. Modeling the anatomical structures like bladder and rectum along with prostate will provide additional flexibility to the segmentation algorithm [START_REF] Tsai | Coupled Multi-shape Model and Mutual Information for Medical Image Segmentation[END_REF]. A hybrid segmentation method that incorporates shape and intensity priors achieves good segmentation accuracy [START_REF] Tsai | A Shape-Based Approach to the Segmentation of Medical Imagery Using Level Sets[END_REF][START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF]. Accuracies of segmentation of prostate in MRI using 3D ASM depends on initialization. Cosio et al. [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF] provided an efficient initialization scheme in their work using Bayesian classification. In recent years Toth et al. [START_REF] Toth | A magnetic resonance spectroscopy driven initial-ization scheme for active shape model based prostate segmentation[END_REF] have used clustering of spectral data obtained in DCE MRI to initialize 3D ASM. Segmentation accuracies were improved using feature driven ASM.

Discussions

Diagnostic imaging has become an indispensable procedure in medical science. Methods of imaging the patient anatomical structures have improved the diagnosis of pathologies, creating new avenues of research in the process. Automatic segmentation of anatomical structures from different imaging modalities like US, and MRI has become an essential step to reduce inter and intra-observer variability, improving contouring time thereafter. This chapter reviewed the methods involved with prostate segmentation. Strength and limitations of the segmentation methodologies have been discussed along with the presentation of validation and performance evaluation of the same. Finally, a discussion on choosing an appropriate segmentation methodology for a given imaging modality and medical purpose has been carried out. It has been highlighted that prostate segmentation techniques should utilize geometric, and spatial information, intensity, texture, and imaging physics priors

Chapter 3 Shape and Appearance Prior Models for Prostate Segmentation in 2D TRUS images

Automatic or semi-automatic prostate segmentation in TRUS images is a challenging task. Low contrast images, speckle and imaging artifacts like the shadow region and micro-calcifications adversely affect segmentation accuracies. Heterogeneous intensity distribution of the prostate region, deformation of the prostate due to insertion of the transrectal probe and inter-patient shape and size variations inhibits the design of a global descriptor of the prostate. To deal with the prostate segmentation challenges, in this chapter, we propose a novel parametric model derived from principal component analysis (PCA) of shape and image features to segment the prostate.

Introduction

Prostate segmentation from TRUS images is a challenging task in the presence of speckle, shadow artifacts, intensity heterogeneities inside the prostate gland and low contrast. As we have observed in chapter 2, to address the challenges of prostate segmentation in TRUS images Shen et al. [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF] and Betrouni et al. [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF] proposed to use prior prostate shape information in their models. The use of prior shape 53 54 Chapter 3. Shape and Appearance Prior Models for Prostate Segmentation in 2D TRUS images information improved segmentation accuracy in presence of imaging artifacts and low SNR. Yan et al. [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF] proposed to incorporate prior shape information of the prostate in a partial active shape model framework. The model selectively discarded salient points for building the shape model to improve segmentation accuracy in the presence of shadow artifacts in TRUS images. Gong et al. [START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF] proposed to use deformable super ellipse to segment the prostate. Shape constraints of the model were effective in achieving impressive segmentation results. Zhan et al. [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF] demonstrated the effectiveness of incorporating both prior shape and texture information in their prostate segmentation framework. Medina et al. [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF] used Active Appearance Model (AAM) [START_REF] Cootes | Active Appearance Models[END_REF], an efficient framework of combining shape and intensity priors, to segment the prostate in 2D TRUS images achieving an overlap ratio of 96%.

Motivated by the approaches of [START_REF] Cootes | Active Appearance Models[END_REF] and [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF] we propose to use AAM in our segmentation framework. However to deal with the challenges of prostate segmentation in TRUS images we propose two different appearance models for the prostate. The first appearance model is derived from PCA of texture features extracted with quadrature filters and in the second model PCA of the posterior probabilities obtained in a supervised learning schema is used. Statistical shape model is often used in prostate segmentation [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF], [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF], [START_REF] Hodge | 3D Prostate Boundary Segmentation From Ultrasound Images Using 2D Active Shape Models[END_REF]. However, few methods incorporate shape and texture priors to segment prostate in TRUS images. In these models often the texture priors are introduced with the classification of texture features ( [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF], [START_REF] Diaz | Semi-automated Segmentation of the Prostate Gland Boundary in Ultrasound Images Using a Machine Learning Approach[END_REF], [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF]) or with region based statistics in a levelsets framework ( [START_REF] Gong | Prostate Ultrasound Image Segmentation Using Level set-Based Region Flow With Shape Guidance[END_REF]).

In contrast, our model significantly differs in the optimization space. In our model, the shape and the appearance model are built from PCA of the shape and the appearance space and the two are tightly coupled in the combined model of shape and appearance. Computational complexities involved with texture prior models in [START_REF] Zhan | Deformable Segmentation of 3D Ultrasound Prostate Images Using Statistical Texture Matching Method[END_REF], [START_REF] Diaz | Semi-automated Segmentation of the Prostate Gland Boundary in Ultrasound Images Using a Machine Learning Approach[END_REF], and [START_REF] Cosío | Automatic Initialization of an Active Shape Model of the Prostate[END_REF] is high due to the extraction and classification of the texture space. Therefore, in our first model we adopt an off-line learning of the optimization space framework to develop a fast prostate segmentation method. The schema of the optimization space is very close to the model used by [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF]. However, it is to be noted that [START_REF] Medina | A 2D Active Appearance Model For Prostate Segmentation in Ultrasound Images[END_REF] used traditional AAM [START_REF] Cootes | Active Appearance Models[END_REF] to segment the prostate and we have improved on time and accuracy over [START_REF] Cootes | Active Appearance Models[END_REF] using a contrast invariant texture descriptor with the quadrature filters. We consider that our approach is well designed for fast prostate segmentation often needed for computer guided prostate brachytherapy and prostate biopsies [START_REF] Yan | Optimal Search Guided by Partial Active Shape Model for Prostate Segmentation in TRUS Images[END_REF].

To facilitate automatic initialization of our model we adopt a supervised learning framework to achieve probabilistic classification of the prostate in our second model. Use of posterior probabilities obtained in the classification schema in building the appearance model further improve segmentation accuracies. We adopt two different optimization schema with two different objectives. A fast off-line optimization schema similar to Cootes et al. [START_REF] Cootes | Active Appearance Models[END_REF] is adopted to facilitate fast prostate segmentation necessary for near real time segmentation of the prostate. Further, we use multiple mean models of shape and appearance priors derived from spectral clustering of combined shape and appearance parameters to improve on segmentation accuracies. We also explore the use of Mumford-Shah functional based energy minimization to improve on segmentation accuracies for off-line segmentation of the prostate.

The rest of the chapter is organized as follows. Next section explains statistical shape and appearance model including their limitations. Speckle reduction with Haar wavelets, quadrature filters to enhance texture and statistical model of shape and texture are formulated in Section 3.3. In Section 3.4 we explore the use of posterior probabilities for building the appearance model and automatic initialization. Multiple mean models of shape and probability priors derived from spectral clustering is proposed in Section 3.5. A Mumford Shah energy minimization framework is proposed in Section 3.6. Quantitative and qualitative evaluations of our method are presented in Section 3.7. We draw conclusions in Section 3.8.

Statistical shape and appearance model

The process of building a statistical model of shape and appearance could be subdivided into three major components: building the shape model, building the appearance model and building the combined shape and appearance model.

Point Distribution Model (PDM) [START_REF] Cootes | The Use of Active Shape Model for Locating Structures in Medical Images[END_REF] built from manually segmented contours are aligned to a common reference frame with Generalized Procrustes Analysis (GPA) [START_REF] Gower | Generalized Procrustes Analysis[END_REF]. Principal Component Analysis (PCA) of the aligned PDMs identify TRUS images the principal modes of shape variations. In our shape model we build the PDM automatically from equal angle sampling of the extracted contours of the ground truth images. The PDMs of the extracted contours are aligned to a common reference frame by minimizing the difference in translation, scale, and orientation by GPA. Fig. 3.1 illustrates the process of aligning the contours after automatic landmark detection.

Finally, PCA of the aligned contours are used to identify the primary modes of shape variation. The shape model may be formalized in the following manner:

s = s + Φ s θ s (3.1)
where s represents the shape model, s denotes the mean shape, Φ s contains the first p eigenvectors of the estimated joint dispersion matrix of shape and θ s represents the associated weight. The process of building the shape model is illustrated in Fig. Intensity distribution is sampled from shape free reference for each of the pieces to build intensity profile of the training images. PCA of the intensity distribution is used to identify the principal components of intensity variations and suppress noise to build the appearance model. The model may be formalized in the following manner:

For building the appearance model Delaunay's triangulation of the mean model

t = t + Φ t θ t (3.2)
where t represents the texture model built from the training images and t denote the mean intensity. Primary modes of intensity variations are represented by Φ t with θ t being the corresponding weight. The process of building the appearance model is illustrated in Fig. 3.3.

The model of shape and intensity variations are combined in a linear framework as, where W denotes a weight factor coupling the shape and the texture space. Finally, a third PCA of the combined model ensures the reduction in redundancy of the same and is given as,

b = W θ s θ t = W Φ T s (s -s) Φ T t (t -t) (3.3) TRUS images
b = V c (3.4)
where V is the matrix of eigenvectors and c the appearance parameters.

Optimization and segmentation of a new instance

In our model, we incorporate AAM optimization proposed by Cootes et al. [START_REF] Cootes | Active Appearance Models[END_REF].

The prior knowledge of the optimization space is acquired by perturbing the combined model with known model parameters and perturbing the pose (translation, scale and rotation) parameters. A linear relationship between the perturbation of the combined model (δc) and the residual texture values (δt), and between the perturbation of the pose parameters (δp) and the residual texture values are acquired in multivariate regression frameworks as,

δc = R c δt, δp = R p δt (3.5)
where R c and R p refer to correlation coefficients. Given a new instance, on initialization of the model, the difference of the appearance value with the mean model is used to determine the residual value δt. The combined model (δc) and the pose parameters (δp) are updated using Eq. 3.5 to generate new shape and combined model and hence new texture. The process continues in an iterative manner until the difference with the target image remains unchanged.

Limitations of statistical shape and appearance model applied to prostate segmentation in TRUS

In Section 1.1.1 we have observed that TRUS images of the prostate are characterized by speckle, shadow artifacts, intensity heterogeneities inside the prostate gland. Furthermore low contrast of the images and contrast variabilities depending on machine manufacturer and acquisition parameters introduces large scale variabilities in intensities. Traditional statistical shape and appearance model considers the appearance space to be Gaussian. Large scale variations of the intensities due to changes in contrast and imaging artifacts produces an inefficient representation of the appearance space with a single Gaussian model. Considering the appearance model and it's relation to the optimization framework of statistical shape and appearance model, large scale variations in intensities introduces segmentation inaccuracies.

Similarly as discussed in Section 1.1.1 inter-patient prostate shape and size may vary and a single Gaussian model representation of the shape space will introduce segmentation inaccuracies. Furthermore the offline optimization framework of statistical shape and appearance model assumes a linear relationship between the change in appearance space and change in the pose parameters as observed in Eq. 3.5. However, as demonstrated by Cootes et al. [START_REF] Cootes | Active Appearance Models[END_REF] this linear relationship holds within a small window of deviation of the pose parameters. Hence large scale deviation of either the pose parameters or the intensities away from the Gaussian space will produce erroneous results. TRUS images To reduce contrast variabilities we adopt two approaches. In the first model we extract contrast invariant texture features with quadrature filters to build our appearance model. The model is presented in Section 3.3. However a manual initialization is necessary for the model. In our second model we adopt a supervised learning framework of random forest to produce a soft classification of the prostate. The posterior probabilities obtained in the learning framework aid in building our appearance model. Such an approach enables automatic classification and reduces intensity variabilities as intensities are substituted with posterior probabilities to build our appearance model. The model is formulated in Section 3.4.

To better approximate the Gaussian space of shape and appearance we derive multiple mean Gaussian models of shape and texture. The approach is motivated by central limit theorem [START_REF] Kallenberg | Foundations of Modern Probability[END_REF] which states that a non Gaussian distribution can be better approximated with multiple Gaussian distribution. We adopt spectral clustering of the shape and appearance parameters to group similar prostates together to build multiple mean models. Given a new instance all the mean models are applied to segment the test image simultaneously. The mean model producing the least fitting error is selected as the final segmentation. The schema has the advantage of better approximating the Gaussian space of shape and appearance. Furthermore multiple mean model aids in improving the range of pose parameters that could be handled by the optimization framework of statistical shape and appearance model. Incidentally that aids in maintaining the linear relationship between the change in pose and the appearance parameters as proposed in Eq. 3.5. The model is formulated in Section 3.5.

Texture in appearance model

In Section 3.2.2 we have discussed the difficulties in building an accurate appearance model in the presence of large scale contrast variabilities. To facilitate better representation of the appearance model in this section we propose to build our appearance model from contrast invariant texture descriptors. In this model the intensities of the image are substituted with approximation coefficients of Haar wavelet transform to obtain a better representation of the underlying texture. Subsequently we propose a novel method to represent prostate texture with local phase information of the quadrature filters is adopted to provide a contrast invariant representation of the texture space.

In this section, we propose a model that transforms the image with Haar wavelets, discard the Haar wavelets detail coefficients to reduce speckle, and then use quadrature filters [START_REF] Boukerroui | On the Choice of Band-Pass Quadrature Filters[END_REF] to enhance texture information to propagate a statistical model of shape and texture in a multi-resolution framework. The key contributions of this work are:

• Enhancement of texture information of the prostate region from phase responses of quadrature filter pairs and

• Use of the texture information in training and propagation of a statistical model of shape and appearance that improves on computational time and segmentation accuracy when compared to the traditional active appearance model [START_REF] Cootes | Active Appearance Models[END_REF] as discussed in Section 3.3.2.

The proposed method is based on three major components, 1. Reduction of speckle with Haar wavelets as discussed in Section 3.3.1.

2. Use of band pass quadrature filters to enhance texture information in the prostate region.

3. Incorporating local phase information as texture in building the statistical model of appearance.

The primary components of the proposed method are explained in details in the following subsections.

Speckle reduction with haar wavelets

Speckle is an interference pattern in TRUS images that often corrupts underlying texture [START_REF] Noble | Ultrasound Image Segmentation: A Survey[END_REF]. Since Gaussian blurring for removal of speckle produces spurious edges, Pathak et al. [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] developed 'stick' filters to reduce speckle without corrupting underlying prostate texture. Jendoubi et al. [START_REF] Jendoubi | Segmentation of Prostate Ultrasound Images Using an Improved Snakes Model[END_REF] and Zaim et al. [START_REF] Zaim | An Energy-Based Segmentation of Prostate from Ultrasound Images Using Dot-Pattern Select Cells[END_REF] used median filtering to reduce speckle and improve on segmentation accuracy. Like The wavelet coefficients framework could be formalized in the following manner. First, let a n-level wavelet transform be denoted by

w = a T u T 1 . . . u T n T (3.6)
where, a and u represent the approximation and the detail coefficients respectively, and w is the wavelet transformed image. The detail coefficients are suppressed to where, C corresponds to a modified identity matrix with the rows corresponding to the detail coefficients removed. In Fig. 3.5(b), 3.5(c), and 3.5(d) we observe that the detail coefficients in horizontal, vertical and diagonal directions primarily represent the speckle and micro-calcifications (labeled A) of Fig. 3.5(a). Hence, discarding the detail coefficients visually reduces speckle and micro-calcifications as observed in Fig. 3.5(e). Suppressing the high frequency components reduces texture information. However, according to Petrou and Sevilla [START_REF] Petrou | Image Processing: Dealing With Texture[END_REF] significant texture information of an image is preserved in the approximation coefficients at each scale and they suggest analyzing the approximation coefficients in multi-resolution to extract texture. Therefore, in our model to ensure the uniformity of texture inside the prostate we suppress the detail coefficients to reduce speckle and the effect of micro calcifications, producing a better representation of the underlying prostate texture.

Quadrature filters to enhance prostate texture

Band-pass quadrature filters tuned at different orientations and spatial frequencies are often used as a basis for obtaining local estimates of phase, energy, orientation and frequency [START_REF] Aach | On Texture Local Energy Transform Versus Quadrature Filters[END_REF]. These estimates are known to provide good discrimination of different textures [START_REF] Knutsson | Texture Analysis Using Two-Dimensional Quadrature Filters[END_REF]. However, the estimates from these filters are noisy and depend on the choice of the quadrature filters [START_REF] Boukerroui | On the Choice of Band-Pass Quadrature Filters[END_REF]. An optimal quadrature filter should demonstrate the following properties,

• The filter must be designed to form quadrature pairs to allow for separate detection of local phase and amplitude.

• It should be well localized in both the space and frequency domains.

• It should be polar separable in frequency domain for accurate and easy estimation of the local orientations.

• It should satisfy a zero response for a constant signal (zero DC) to be invariant to gray level shift.

Large bandwidth zero DC filters constructed from log-Gabor function is a well known choice to build quadrature filters applied to various computer vision applications [START_REF] Felsberg | The Multidimensional Isotropic Generalisation of Quadrature Filters in Geometric Algebra[END_REF], [START_REF] Kovesi | Image Feature from Phase Congruency[END_REF], [START_REF] Mulet-Parada | 2d+t Acoustic Boundary Detection in Echocardiography[END_REF]. Log-Gabor filter pairs may be formalized in the following manner. The one-dimensional log-Gabor filter in the frequency domain is a Gaussian function on a logarithmic scale given by the following equation,

G 1 (ω) = n c exp - ln 2 (ω/ω 0 ) 2 ln 2 (κ β ) (3.8)
where ω 0 is the peak tuning frequency and 0 < κ β < 1 is dependent on the bandwidth β of the filter by the following relation

β = -2 √ 2 √ ln 2 ln κ β (3.9)
The normalization constant n c is given by

n c = exp - 1 8 ln (κ β ) 2 -2 √ π ω 0 ln (κ β ) (3.10)
The spatial domain counterparts of the quadrature filters are used in a convolution scheme to estimate local phase and amplitude information of an image. The local phase and amplitude of a 1D signal f (x) are defined using analytical signal f A (x),

f A (x) = f (x) -if H (x) (3. 11
)
where i = √ -1 and f H (x) is the Hilbert transform of f (x) defined by:

f H (x) = 1 π ∞ -∞ f (τ ) τ -x dτ (3.12) ⇔ F H (ω) = F (ω).isign(ω), (3.13)
where F (ω) is the Fourier transform of f (x) and

sign(ω) = -1 ω < 0 +1 ω ≥ 0 (3.14)
Therefore, the analytical signal in Fourier domain is obtained from (3.11) and (3.13) as

F A (ω) = F (ω). [1 + sign(ω)] . (3.15)
The local amplitude of the signal is obtained as,

A(x) = f A (x) = f 2 (x) + f 2 H (x) (3.16)
and the local phase by,

φ(x) = arctan (f (x)/f H (x)) (3.17)
However, the Hilbert transform of the analytical signal is defined over the entire signal and localization in small spatial and frequency span is necessary to extract local phase and amplitude information. Zero response to a constant signal (gray level invariance) and symmetric (even) filters are essential for extraction of local phase and amplitude information. Considering the even symmetric filter to be f e TRUS images we have,

fA (x) = f e (x) * f (x) -iH(f e (x) * f (x)) (3.18) = (f e (x) -iH(f e (x)) * f (x) = (f e (x) -if o (x)) * f (x).
where H(.) is the Hilbert transform and '*' is the 1-D convolution operator and f o (x) is the Hilbert transform of f e (x). In practice, an approximation of the local amplitude or energy ( Â(x)) and phase ( φ(x)) is obtained by using band-pass quadrature even and odd filter pair f e (x) and f o (x) respectively where,

Â(x) = [f e (x) * f (x)] 2 + [f o (x) * f (x)] 2 (3.19) φ(x) = arctan {f e (x) * f (x)/f o (x) * f (x)} . (3.20)
For multi-dimensional signals the analysis is performed at a set of filter orientations and the results are combined to provide local information [START_REF] Boukerroui | On the Choice of Band-Pass Quadrature Filters[END_REF]. The log-Gabor filters are well suited for local frequency estimations and its zero DC component makes it more invariant to gray level intensity changes making it robust to large scale contrast variance [START_REF] Granlund | Signal Processing for Computer Vision[END_REF]. In our model, we use log-Gabor quadrature filters to estimate local phase of the prostate gland providing a better representation of the underlying prostate texture uncorrupted from large scale gray level differences due to presence of shadow artifacts and speckle. The log-Gabor filter is optimized for TRUS prostate images using quadrature filter optimization toolbox of [START_REF] Andersson | Adaptive Filtering[END_REF]. Bandwidth of 2 octaves and peak tuning frequency of π/3 ensures optimized localization in spatial and frequency domains to estimate local phase information in prostate images. The log-Gabor quadrature filter in the frequency domain and the corresponding even and odd pairs in spatial domain and phase response of quadrature filter pairs for prostate image are shown in Fig. 3.6.

The schema from building the model to the fitting of a new instance is illustrated in Fig. 3.7. The figure shows that from a set of training images a shape space and a appearance space are built and combined in a AAM framework to produce a mean model. Given a new test image, the mean model with the knowledge of the optimization space segments the prostate from test image. Fitting of a new instance is computationally expensive. However, fitting time improves with the reduction of difference between the target and the mean model. Contrast invariant representation of our appearance model reduces the texture difference between the mean model and the target images. This facilitates improvement of fitting time of the mean model to the target image. The work flow requires additional time to transform the image into a new representation. The computational complexity involved with traditional AAM is [START_REF] Cootes | Active Appearance Models[END_REF] is O(n). Inside a single loop the differences between the mean model and the target image are computed and corresponding pose parameters are determined using Eq. 3.5. Introduction of quadrature filter introduces a nested loop that increases computational complexity to O(n 2 ). However, the benefit of acquiring a contrast invariant representation of the prostate texture outweigh this additional time requirement. This will be shown in the experimental evaluation.

The multi-resolution functioning of the model is illustrated in Fig. 3 and segmentation is performed from coarse to fine resolution as observed in Fig. (b), (c), (d) and(e). The mean model is initialized by clicking in any position close to the center of the prostate decided on visual inspection. The mean model initialization and subsequent multi-resolution segmentations are produced based on the local phase estimation from quadrature filters. The semi-automatic model is accurate and robust, however manual intervention is necessary in initialization of the model. Considering the objective of the PROSCAN project as discussed in Chapter 1 an automatic and accurate 2D segmentation method for the prostate is necessary. In the next section we propose a soft classification of the prostate in a supervised learning framework that facilitates automatic initialization and propagation of the

Probability theoretic approach to appearance model

In Section 3.2.2 we have discussed the difficulties in building an accurate appearance model in the presence of imaging artifacts and contrast variations. In Section 3.3 we proposed a novel appearance model built from contrast invariant local phase information of quadrature filter. However the model is semi-automatic as manual initialization is necessary. In this section we discuss a novel prostate segmentation method in which appearance and spatial context based information from the training images are used in a supervised learning schema to achieve a probabilistic classification of the prostate. Subsequently intensities are substituted with posterior probabilities to reduce inter dataset contrast variations and the appearance model is TRUS images built from PCA of the posterior probabilities of the prostate region. Soft classification of the prostate facilitates automatic initialization and propagation of the model in a multiresolution framework. We present our supervised learning framework of random forest in the next section.

Random forest based probabilistic classification

Decision trees are discriminative classifiers which are known to suffer from overfitting. However, a random decision forest or random forest achieves better generalization by growing an ensemble of many independent decision trees on a random subset of the training data and by randomizing the features made available at each node during training [START_REF] Geremia | Spatial decision forests for ms lesion segmentation in multi-channel mr images[END_REF]. During training, to minimize the pose and intensity variations, our datasets are rigidly aligned and the inter-patient intensity variations are normalized. The data consists of a collection of V = (X, F ), each centered at 3 × 3 neighborhood of pixels, where, X = (x, y) denotes the pixel position and the feature vector F constitutes of the mean and standard deviation of the 3 × 3 pixel neighborhood. Each tree τ i in random forest receives the full set V , along with the label and the root node and selects a test to split V into two subsets to maximize the information gain. A test constitutes of a feature and a feature response threshold. The left and the right child nodes receive their respective subsets of V and the process is repeated at each child node to grow the next level of the tree. Growth is terminated when either the information gain is minimum or the tree has grown to a maximum depth specified. Each decision tree in the forest is unique as each tree node selects a random subset of features and threshold. During testing, the test image is rigidly aligned to the same frame of the training datasets and its intensities are normalized. The pixels are routed to one leaf in each tree by applying the test (selected during training). Each pixel of the test dataset is propagated through all the trees by successive application of the relevant binary test to determine the probability of belonging to class c. When reaching a leaf node l τ , where τ ∈ [1..., Γ] (where Γ represents the last tree) the posterior probabilities (P τ (c|V )) are gathered in order to compute the final posterior probability of the pixel defined by P (c|V ) = 1 Γ Γ τ =1 P τ (c|V ). Computation of class posterior probabilities in random forest is illustrated in Fig. 3.9. In our automatic prostate segmentation model, the statistical shape and appearance models are derived from PCA of prostate shape and posterior probability values of the prostate region. The shape model is built using Eq. 3.1, the appearance model is built using Eq. 3.2 and the combined model is built using Eq. 3.3. We adopt a similar off-line optimization framework as discussed in Section 3.2. However in contrast to traditional shape and appearance model of [START_REF] Cootes | Active Appearance Models[END_REF] and our shape and texture model as discussed in Section 3.3 we use posterior probability of the prostate region determined from random forest classification to build, initialize and propagate our model. The proposed schema of our method is illustrated in Fig. 3.10.

The automatic model for prostate segmentation produces good segmentation accuracies. However with larger variabilities in shape and appearance parameters segmentation accuracies of the model decreases. As discussed in Section 3.2.2, with larger datasets inter-patient prostate shape and size varies significantly and a single Gaussian model representation of the shape space introduces segmentation inac-TRUS images curacies. Furthermore the offline optimization framework of statistical shape and appearance model assumes a linear relationship between the change in appearance space and change in the pose parameters as observed in Eq. 3.5. However, the linear relationship holds within a small window of deviation of the pose parameters. Hence large scale deviation of the pose parameters away from the Gaussian space produce erroneous results. In order to improve on segmentation accuracies we propose to use multiple mean models of shape and appearance parameters to segment the prostate. The framework of multiple mean models of shape and appearance parameters is discussed in the next section.

Multiple mean models 73

Multiple mean models

Traditionally a statistical shape and appearance model assumes the shape and the appearance spaces to be Gaussian. However in real cases, inter-patient prostate intensity distribution, prostate shapes and sizes may vary significantly. In such circumstances, a single mean model is inefficient to capture the variations of shape and appearance spaces and it would be inappropriate to approximate them with a single Gaussian distribution. To better approximate the Gaussian space of shape and appearance we propose to use multiple mean Gaussian models of shape and texture. The approach is motivated by central limit theorem which states that a non Gaussian distribution can be better approximated with multiple Gaussian distribution [START_REF] Kallenberg | Foundations of Modern Probability[END_REF].

We adopt manual and spectral clustering of the shape and appearance vectors to group similar prostates together to build multiple mean models to improve on segmentation accuracies. We present the schema of manual clustering next followed by automatic spectral clustering to group similar prostates together.

Manual clustering

The process of manual clustering and building multiple mean models is as follows; we have 23 datasets that we want to cluster depending on similar shape and appearance parameters. Initially the 1st dataset is chosen as the reference to register datasets 3 to 23 to produce a mean model of shape and texture. This mean model is used to test dataset 2. The sum of squared difference of the posterior probabilities between the mean model and dataset 2 is recorded as fitting error after the final segmentation. Likewise, with the fixed reference (dataset 1), we build the second mean model registering datasets 2 and 4-23 to test on dataset 3 and record the fitting error. The process is repeated for all datasets from 4-23. This provides 22 model fitting errors for the test datasets with dataset 1 as reference (Fig. 3.11). Consequently, the reference dataset is changed from 2 through 23 and the entire process is repeated for all the datasets (23 in total). The entire procedure yields 23 graphs of model fitting errors (one for each dataset). We have analyzed these 23 model fitting error graphs and have observed that with less fitting error (< 2000 units, where units signifies the sum of squared differences of the probability values of TRUS images the prostate region between the mean model and the target image) we have higher accuracy in segmentation (in terms of Dice similarity coefficient, mean absolute distance etc.). This is not surprising considering the fact that the objective function of our optimization framework tries to minimize the fitting error between the mean model and target image with respect to the pose parameters. Hence, an increase in fitting error indicates a reduction in segmentation accuracies. An empirical error value is determined from these graphs, above which, the segmentation accuracy is reduced (in our case the threshold value is 1700 units). The reference dataset that has a fitting error less than the empirical value for maximum number of test datasets is identified (dataset 1 in our case). The datasets below this fitting error are grouped together (datasets 1, 6, 8, 10, 15 and 21(Fig. 3.11)) and are removed from further grouping. The process is repeated until all the datasets are grouped. These groups of datasets provide individual mean models. However, increasing the number of mean models (decreasing the fitting error threshold) improves segmentation accuracy with additional computational time in a sequential execution framework. Hence, the choice of optimum number of mean models depends on the segmentation accuracy and computational time requirement of the process.

Manual clustering of similar prostates together improves segmentation accuracies as observed in Section 3.7. However the process of building multiple mean models is time consuming and demands a high degree of manual intervention and fine tuning of the fitting error parameter. In order to facilitate automatic clustering of similar prostate together we propose to use spectral clustering of combined shape and appearance vector as explained in next section.

Spectral clustering

The objective of clustering is to employ a grouping scheme that considers the prostates with similar shape and appearance parameters to build a set of mean models and produce an accurate optimization space. Furthermore, the number of mean models should change dynamically depending on combined shape and appearance parameters of the training datasets. To address both the issues, we propose to use spectral clustering in the combined space of shape and appearance parameters.

Spectral clustering relies on the eigen structure of a similarity matrix to partition points into disjoint clusters with high intra-cluster similarity and also high inter-cluster dissimilarity. Moreover, the number of clusters could be determined dynamically from the principal components in the eigen space. During training, the combined shape and appearance eigenvectors are obtained by Eq. (3.2) and Eq. (3.3). Cosine similarities of the combined vectors of P eigenvectors are used to construct a P × P similarity or affinity matrix W that measures the similarity between the P points. W i,j is large when the points indexed by i and j are likely to be in the same cluster. The problem may be defined in terms of a complete graph with vertices υ = 1, ..., P and an affinity matrix with weights W i,j , for i, j ∈ υ. The objective is to determine K disjoint clusters A = (A k ) k∈1,....,K , where k A k = υ, that optimizes the cost function of K-way normalized cut defined as,

C(A, W) = K k=1   i∈A k ,j∈υ\A k W i,j   / i∈A k ,j∈υ W i,j (3.21)
The K-way normalized cut C(A, W) may be simplified as D -1/2 WD -1/2 where D = diag(W) [START_REF] Bach | Learning spectral clustering[END_REF]. The algorithm may be summarized as,

1. Input:Similarity matrix W ∈ R P×P 2. Compute first k eigenvectors of D -1/2 WD -1/2
where D = diag(W) to build the matrix U . The number of clusters k is determined by the number of the largest k-eigen vectors (obtained from 98% of total variations) of the normalized Laplacian C(A, W) of the affinity matrix. The k-means clustering therefore groups the similar prostates to form k mean models. The process of spectral clustering is illustrated in Fig. 3.12.

During validation, the test dataset is removed and multiple mean models are built with the aid of spectral clustering. Given the test dataset all mean models are applied to segment the images. Pixel wise posterior probability differences between the mean models and the test image is recorded as fitting errors. The mean model producing the least fitting error after deformation is considered as the achieved segmentation. The proposed schema of our method is illustrated in Fig. 3.13. As observed in Fig. 3.13 we build a similarity matrix of combined shape and appearance vector W and use k-mean clustering of the similarity space to group similar prostates together.

The problem of non-Gaussian shape space has been discussed by Cootes et al. in their work in [START_REF] Cootes | A mixture model for representing shape variation[END_REF]. The authors state ASM cannot accommodate non-linear variations of the mean shape and often non-linear PCA is adopted to solve the problem. However non-linear PCA assumes that a plausible shape may be generated by varying weight vector associated with eigen vectors in a non linear space which is not true always. Cootes et al. adopted a Gaussian mixture modeling for the shape space and used expectation maximization based clustering to generate different Gaussian shapes. Like ASM a mean shape was placed on the image and the landmarks were displaced to a better position depending on the edge information. However the model shape and the pose parameters were updated depending on the probability threshold value of the landmarks determined for the different Gaussian. Zhu et al. [START_REF] Zhu | Improving asm search using mixture models for grey-level profiles[END_REF] built a Gaussian mixture model for the landmark profiles to find the optimal placement of the landmark. The optimal position was determined from the maximum probability value determined in a Gaussian mixture model framework.

Our approach significantly differs from both the works [START_REF] Cootes | A mixture model for representing shape variation[END_REF][START_REF] Zhu | Improving asm search using mixture models for grey-level profiles[END_REF]. One major problem of using an expectation maximization framework to approximate the shape and the appearance space is that the number of Gaussian distributions required to approximate the two is difficult to determine automatically. Cootes et al. proposed to use small number of Gaussian distribution to approximate the shape space. However by fixing the number of Gaussian distribution we tune our method according to the available training datasets that reduces the generalization capability of the model. In our model the number of Gaussian components is determined automatically with spectral clustering. The number of Gaussian components varies depending on the variability of the dataset. This makes the model more generic and new train-TRUS images ing dataset could be easily incorporated in our framework to build multiple mean models. Furthermore, we perform spectral clustering on both shape and appearance parameters to build multiple Gaussian combined models which is very different from the approach of Cootes et al. where multiple Gaussians are developed from shape probability distribution and Zhu et al. where landmark profile intensity distribution is used to determine the optimal landmark position in a Gaussian mixture model framework. Furthermore, unlike [START_REF] Cootes | A mixture model for representing shape variation[END_REF][START_REF] Zhu | Improving asm search using mixture models for grey-level profiles[END_REF] we use similarity space of our combined shape and appearance vector to build multiple models.

In all our previous models we have explored the use of explicit active contour models to segment the prostate. However, the use of explicit active contour models reduces contour accuracies as depending on deformation of such particle based active contours corners may appear or the particles may come close and finally collapse. In order to improve on contour accuracies we propose an implicit active contour model based on variational model and Mumford-Shah energy minimization framework in the next section.

Mumford-Shah energy minimization

In this Section we propose a variational model driven by Mumford-Shah (MS) functional [START_REF] Mumford | Optimal approximations of piece-wise smooth functions and associated variational problems[END_REF] for segmenting the prostate in TRUS images. According to MS functional an image is modeled as piece wise smooth function and energy is minimized by penalizing the distance between the image and the model, lack of smoothness within a region and the length of boundaries of a region.

In our case a parametric representation of the implicit curve is derived from principal component analysis (PCA) of the signed distance functions (SDFs) of the labeled training data to impose shape prior. Posterior probability of the prostate region determined from random forest classification facilitates initialization and propagation of our model in a MS energy minimization framework. The parameters of the evolving curve are determined from minimization of region and contour based energy as proposed in [START_REF] Bresson | A variational model for object segmentation using boundary information and shape prior driven by the mumford-shah functional[END_REF]. The proposed method is developed on two major components: A) supervised learning framework of random forest to determine posterior probability of a pixel being prostate, and B) adapting implicit shape, boundary and intensity prior model of [START_REF] Bresson | A variational model for object segmentation using boundary information and shape prior driven by the mumford-shah functional[END_REF] to incorporate the posterior probabilities of the prostate region for initialization and evolution of the implicit curve.

In TRUS images prostate region have a heterogeneous intensity distribution and depending on the acquisition parameters the region based statics (mean and standard deviation) of the prostate may not significantly vary from the background. Moreover, shadow artifacts and speckle may adversely affect the region based statistics (determined from intensities) of the prostate and the background. Significant separation of the intensities of the prostate and the background is essential for MS energy minimization framework. Moreover, inaccurate region based statistics of the prostate and the background adversely affect levelsets propagation and hence segmentation accuracies. Therefore, to reduce intensity variations inside the prostate region, and significantly separate the intensities of the prostate and the background we propose to determine the posterior probability of the image pixels being prostate in a supervised learning framework of random forest and substitute intensities with probabilities to achieve a better representation of the prostate and the background as discussed in Section 3.4.1.

The problem of segmenting the prostate using a shape prior, and global and local image information could be resolved by minimizing,

F = F shape + F region + F boundary (3.22)
The process of building the shape model of the prostate starts with the alignment of n segmented prostate images of the training set with intensity based affine registration to minimize pose differences. The boundaries of each of the n aligned prostates are embedded as the zero levelset of n separate SDFs Ψ with negative distances assigned to the inside and positive distance assigned to the outside of the prostate boundary. The mean levelset function of the prostate is computed from the average of these n SDFs, Φ = 1/n n i=1 Ψ i . Contour alignment and the computation of the mean levelset is illustrated in Fig. 3.14. To extract the shape variations of the prostates Φ is subtracted from each of the n SDFs to create n mean-offset functions Ψ. Each 2D mean-offset Ψi is reshaped into a column vector. Then the shape variability matrix of n prostates is given by S = ψ1 , ψ2 , ...., ψn . PCA of S yields the sorted matrix of principal components W k (k is 98% of the total shape variations) and a vector of eigen coefficients x pca . Finally the shape model is given TRUS images The energy associated with the shape term may be given as,

F shape = 1 0 φ2 (x pca , h x T (C (q))) C ′ (q) dq, (3.23)
where φ2 (x pca , h x T (C (q))) = φ2 (x pca , C (q)) ≈ Ĉxpca -C (q) 2 C is the active contour at point q, x pca is the vector of eigen coefficients and h x T is an element of a group of geometric transformation parameterized by x T the geometric transformation matrix. This essentially evaluates the shape difference between the contour C and the zero levelset Ĉ of the shape function φ as shown in Fig. 3. [START_REF] Bookstein | Principal warps: Thin-plate splines and the decomposition of deformations[END_REF].

By minimizing this energy we restrict the levelset evolution to follow prostate shape prior.

As discussed in Section 3.4.1, intensity of the image is substituted with posterior probabilities obtained with random forest. According to Chan and Vese [START_REF] Chan | Active Contours Without Edges[END_REF] MS functional model the curve parameters were determined from minimization of region based energy given by,

E cv = R u (I -κ) 2 dA + R v (I -γ) 2 dA (3.24)
Evolution of the curve ensured segmentation of the image into two region u and v with mean intensities κ and γ without any specific shape. In our model, the MS functional of [START_REF] Chan | Active Contours Without Edges[END_REF] is modified to incorporate shape prior. The region based energy term as a function of the shape φ is given as,

F region = Ω Θ in H φ (x pca , x T ) dΩ (3.25) + Ω Θ out H -φ (x pca , x T ) dΩ
where H (.) is the Heaviside function and Θ r = |I -µ r | 2 + µ |▽µ r | 2 and µ is the mean r = in or out of the prostate shape prior. Gradient descent minimization of the energy term aids in determining the shape x pca and the pose parameters x T of the evolving curve to drive the shape model towards a homogeneous intensity region with the shape of interest. However, the model cannot handle local deformation like irregular boundaries of the prostate. Hence a new energy term is introduced as F boundary that aids in capturing the local edge variations around the global shape Figure 3.16: Illustration of shape function φ(x pca , C(q)). The green contour gives the shape model and the red contour shows the evolving contour. The objective is to minimize the distance between the evolving contour and the shape model.

variations. Local edge information is captured by the energy term given as,

F boundary = 1 0 g (|▽I (C (q))|) C ′ (q) dq (3.26)
where g (•) is a Gaussian kernel applied on the image gradient (∇I). The energy function F of Eq. (3.22) is minimized using the gradient descent optimization. In Fig. 3.17 we illustrate the working principle of the model.

Experimental results

Datasets

The TRUS images were acquired using a 6.5 MHz side-ring probe with SIEMENS Allegra and TOSHIBA Xario machines. We have validated the accuracy and robustness of our method with 46 axial mid gland TRUS images of the prostate with a resolution of 348×237 pixels from 23 prostate datasets in a leave-one-patient-out evaluation strategy. As reported by Pathak et al. [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] and Gong et al. [START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF] the mid gland mean segmentation inter observer variability may range from 3.71±1.81 mm [START_REF] Pathak | Edge-Guided Boundary Delineation in Prostate Ultrasound Images[END_REF] to 1.82±1.44 mm [START_REF] Gong | Parametric Shape Modeling Using Deformable Superellipses for Prostate Segmentation[END_REF]. Therefore, for the evaluation of our experiment the ground truth was developed in a manner very similar to that of the MICCAI prostate challenge 2009 [START_REF]MICCAI. 2009 prostate segmentation challenge MICCAI[END_REF], where manual segmentations performed by an expert radiologist were validated by an experienced urologist. Both doctors have over 15 years of experience in dealing with prostate anatomy, prostate segmentation, and ultrasound guided biopsies.

Results

As discussed in Chapter 2 no single evaluation measure has become standard in evaluating prostate segmentation algorithms. Therefore, to validate our 2D prostate segmentation methods we have used most of the popular prostate segmentation evaluation metrics like Dice Similarity Coefficient (DSC) [START_REF] Dice | Measures of the amount of ecologic association between species[END_REF], 95% Hausdorff Distance (HD) [START_REF] Belogay | Calculating the hausdorff distance between curves[END_REF], Mean Absolute Distance (MAD) [START_REF] Yan | Optimal Search Guided by Partial Active Shape Model for Prostate Segmentation in TRUS Images[END_REF], specificity [START_REF] Diaz | Semi-automated Segmentation of the Prostate Gland Boundary in Ultrasound Images Using a Machine Learning Approach[END_REF], and sensitivity [START_REF] Badiei | Prostate Segmentation in 2D Ultrasound Images Using Image Warping and Ellipse Fitting[END_REF].

We have compared our results achieved with statistical shape and texture model of quadrature phase information (SSTM), statistical shape and probability prior model (RF-AAM), multiple mean models with manual clustering (RF-AAM-Mult1) and with spectral clustering (RF-AAM-Mult2), implicit shape and probability prior model (RF-Impl), with the traditional AAM proposed by Cootes [START_REF] Cootes | Active Appearance Models[END_REF]. For the random forest based classification, we have fixed the number of trees to 100, the tree depth to 30 and the lower bound of information gain to 10 -7 . These parameters For building multiple mean models, during validation, the test dataset is removed and multiple mean models are built with the aid of manual and spectral clustering. The number of mean models varied between 5 to 7 depending on the training datasets. Given an image of the test dataset, the image is classified with random forest and all the mean models are initialized at the center of gravity of the classified region (as prostate) to segment the prostate in parallel. Fitting or registration errors between the mean models after final deformation and the test image are computed by normalizing the pixel-wise intensity differences. Finally, the segmentation by the mean model that provided the least fitting error is selected as optimum segmentation. Table 3.1 shows the comparison of different methods. In general, our models performs better than the traditional AAM. This could be attributed to the fact that incorporating a better representation of the underlying prostate texture compared to raw intensities in AAM improves segmentation accuracy. Incorporating local phase information from log-Gabor quadrature filters improves the overlap and contour ac-curacy of our SSTM model compared to traditional AAM. However use of posterior probability facilitates automatic initialization and improvement of contour and overlap accuracy as could be observed in RF-AAM. The model is further improved with multiple mean models as could be observed from the results of RF-AAM-Mult1 and RF-AAM-Mult2. However considering generic and automatic nature of multiple mean models with spectral clustering, the automatic model is preferred over manual clustering model of RF-AAM-Mult1 even though manual clustering provides better segmentation accuracies. Finally, the use of implicit shape and appearance prior model improves on segmentation accuracy of RF-AAM-Mult.

Overlap accuracy computed from DSC has comparable values between traditional AAM, and SSTM. Two-tailed paired t-tests [START_REF] Gmez-Villegas | Reconciling Bayesian and Frequentist Evidence in the Point Null Testing Problem[END_REF] were performed to identify whether there were statistically significant differences between the methods in terms of accuracies. SSTM model has a statistically significant improvement in t-test for MAD with p = 0.0002 compared to traditional AAM. Comparing the Hausdorff distance, SSTM model achieves statistically significant p = 0.0001 compared to the traditional AAM. It is to be noted that a high DSC value and contour accuracy metrics like HD and MAD are all equally important in determining the segmentation accuracy of an algorithm. In this context we may claim that the segmentation accuracy of SSTM is superior to AAM.

Comparing AAM and SSTM with RF-AAM model we find that the use of posterior probabilities improves contour and overlap accuracy compared to AAM however we achieve similar contour and overlap accuracy value compared to SSTM. Comparing RF-AAM-Mult1 and RF-AAM-Mult2 with AAM, SSTM, and RF-AAM we find that the use of both posterior probabilities and multiple mean models significantly improves overlap and contour accuracies. We achieved a statistically significant improvement in t-test p-value<0.0001 for DSC, HD and MAD compared to traditional AAM, SSTM and RF-AAM.

In the two models RF-AAM-Mult1 and RF-AAM-Mult2 we adopt clustering approach to build multiple Gaussian models of shape and appearance. However the approach adopted is very different as explained in Section 3.5.1 and 3.5.2. In manual clustering we use registration error or fitting error threshold of a mean model to a test image to group prostates. In spectral clustering we use similarity measure of shape and appearance vectors to cluster the prostates. The resulting cluster of the two TRUS images methods were different. This is due to the fact that in RF-AAM-Mult1 we consider the Gaussian space within which the model varies to find the registration error. In RF-AAM-Mult2 we assume that there are multiple Gaussians of combined shape and appearance vectors and try to build a model for each. To have a better idea of the manual clustering we remove dataset 23 as a test dataset and cluster remaining 22 datasets (1-22 datasets) using the method described in Section 3.5.1. The graphs for each of the five groups are displayed in Fig. 3.18. Hence according to the manual clustering schema the five groups are; Group 1 -1,6,8,10,15,and 21, Group 2 -2,14, and 17, Group 3 -4,7,13, and 19, Group 4 -3,5,20, and 22 and Group 5 -9,11,12,16, and 18. Performing spectral clustering with the same 22 datasets we observe the group changes. The grouping with spectral clustering is given as Group 1 -2,5,12,14, and 17, Group 2 -13,15,16,18, and 19, Group 3 -3,6,8,9,20,22, Group 4 -1 and 7, Group 5 -10 and 21 and Group 6 -4 and 11. This explains why we have different results in Table 3.1 for the two models RF-AAM-Mult1 and RF-AAM-Mult2.

Comparing our RF-Impl with AAM, SSTM, RF-AAM and RF-AAM-Mult2 we observe that use of implicit shape and appearance prior model significantly improve overlap and contour accuracies and a statistically significant improvement in t-test p-value<0.0001 for DSC, HD and MAD is achieved compared to traditional AAM, SSTM, RF-AAM, RF-AAM-Mult1 and RF-AAM-Mult2.

All the methods compared in Table 3.1 are implemented in Matlab 7 on an Intel Core i5, 2.8 GHz processor and 8 GB RAM. The mean segmentation time of SSTM is 1.30±0.03 seconds, compared to 1.51±0.07 seconds of traditional AAM. The fitting of a new instance is computationally expensive in AAM. However, fitting time improves with the reduction in difference between the target and the mean model and this is observed when we compare the computational time of AAM and SSTM. In SSTM contrast invariant texture descriptor of the prostate tissue, the texture difference between the mean model and the target images will be considerably less. This facilitates improvement of fitting time of the model to the target image. The work flow requires additional time to transform the image into a new representation. However, the benefit of acquiring a contrast-invariant representation of the prostate texture outweighs this additional time requirement. However the best segmentation time is achieved with random forest based statistical shape and appearance model (RF-AAM, RF-AAM-Mult1 and RF-AAM-Mult2). With Matlab we achieve a mean segmentation time of 0.67±0.02 seconds and we believe this could be improved further with C++ coding to make segmentation possible in near real time. The overlap and contour accuracies of RF-Impl model comes at a price of increased computational time. The mean segmentation time of the RF-Impl model is 103.66±2.55 seconds which makes the model unsuitable for near real time procedures. However the improved segmentation accuracies imply that the model is well suited for off-line automatic and accurate prostate segmentation in TRUS images.

The robustness of the SSTM against low SNR, intensity heterogeneities, speckle noise and micro-calcification is illustrated in Fig. 3.19 where our proposed model successfully avoids the artifact and segments the prostate with an accuracy of 97%.

In Fig. 3.20 we illustrate the necessity of a deformable model based segmentation after random forest classification. We observe that random forest classification of Fig. 3.20(a) and Fig. 3.20(d) produces mis-classified regions in Fig. 3.20(b) and 3.20(e). Our statistical shape and probability prior models working on initial segmentation improve segmentation accuracies as observed in Fig. 3.20(c) and 3.20(f). Qualitative improvement in segmentation accuracy of RF-AAM-Mult compared to RF-AAM is illustrated in Fig. 3.21 for two datasets. In Fig. 3.21, we observe that segmentation accuracy of our model with spectral clustering (in Fig. 3.21(b) and Fig. 3.21(d)) is better compared to RF-AAM (in Fig. 3.21(a) and Fig. 3.21(c)). In Fig. 3.22 the first column shows the results achieved with AAM [START_REF] Cootes | Active Appearance Models[END_REF] and the second column shows the results achieved with RF-AAM-Mult method. In Fig. 3.23 we compare the performance of different levelsets qualitatively. From Fig. 3.23 we deduce that use of posterior probabilities in MS energy minimization framework is necessry to improve on prostate segmentation accuracies in TRUS images. To provide qualitative results of our method we present a subset of results in Fig. 3.24. The first row shows the results achieved with AAM [START_REF] Cootes | Active Appearance Models[END_REF] and the second row shows the results achieved with RF-Impl model. The method RF-AAM-Mult2 is accurate however accuracy depends on random forest classification. Mis-classification around the prostate contour may reduce segmentation accuracies as could be observed in Fig. 3.25 Ideally a comparison of different state-of-the-art prostate segmentation methodologies on our dataset should have been done to evaluate the performance of our method to that of the state-of-the-art. However a quantitative comparison of different prostate segmentation methodologies is difficult in absence of a public dataset and standardized evaluation metrics. Nevertheless, to have an overall qualitative estimate of the functioning of our method we have compared with some of the works in the literature in Table 3.2. Analyzing the results we observe that our mean DSC value is better compared to area overlap accuracy values of Betrouni et al. [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF] and Ladak et al. [START_REF] Ladak | Prostate Segmentation from 2D Ultrasound Images[END_REF] and very close to the area overlap error value of Shen et al. [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF]. However, it is to be noted that we have used more images compared to [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF]. Our MAD value is comparable to [START_REF] Betrouni | Segmentation of Abdominal Ultrasound Images of the Prostate Using A priori Information and an Adapted Noise Filter[END_REF], [START_REF] Shen | Segmentation of Prostate Boundaries from Ultrasound Images Using Statistical Shape Model[END_REF], and to [START_REF] Ladak | Prostate Segmentation from 2D Ultrasound Images[END_REF]. From these observations we may conclude that qualitatively our method performs well in overlap and contour accuracy measures.

Discussions

In this chapter we have explored the use of statistical shape and appearance prior model for prostate segmentation in TRUS images. We have demonstrated that the use of contrast invariant texture features improves segmentation accuracies and reduces segmentation time requirement of the procedure. To facilitate automatic initialization we adopted a probabilistic modeling of the TRUS images. Image intensities were substituted with posterior probabilities obtained in a supervised learning schema of random forest for automatic initialization and propagation of shape and appearance prior model. Finally we demonstrated that the use of an implicit shape and appearance prior model improves segmentation accuracies compared to the explicit shape and appearance prior model. However considering fast prostate segmentation requirement of the PROSCAN project multiple mean models of shape and posterior probabilities provides an computationally efficient and accurate method suitable for the purpose. As discussed in Chapter 2, it is observed inter-patient prostate shape, size, deformation and intensity variations along with imaging artifacts challenge 3D automatic segmentation of the prostate. However, atlas based prostate segmentation methods have achieved good segmentation accuracies when validated with large number of MRI datasets [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF][START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF]. In recent years Li et al. [START_REF] Li | Learning image context for segmentation of prostate in ct-guided radiotherapy[END_REF] have adopted a supervised learning approach for prostate segmentation in CT images. Motivated by these approaches we propose to combine the two approaches to achieve a probabilistic classification of the prostate voxels. The probabilistic classification is achieved by the fusion of the posterior probabilities determined with a probabilistic atlas and a supervised learning framework of random forest. Finally, graph cut based energy minimization [START_REF] Boykov | Graph cuts and efficient n-d image segmentation[END_REF] of the posterior probabilities produces the hard 3D segmentation of the prostate. The proposed method is robust to inter-patient shape, size and intensity variabilities. The key contributions of this work are:

• Fusion of the posteriors from random forest and probabilistic atlas to achieve probabilistic classification of the prostate.

• Use of graph cut in the stochastic domain to achieve segmentation of the prostate.

The performance of our method is compared with some of the works in the literature [START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF][START_REF] Gubern-Merida | Atlas Based Segmentation of the Prostate in MR Images[END_REF][START_REF] Dowling | Automatic atlas-based segmentation of the prostate[END_REF] that have used the same datasets. The remaining chapter is organized in the following manner. Section 4.2 provides a description of the datasets used, in Section 4.3 we discuss our proposed segmentation framework, followed by the results and discussions in Section 4.4. Finally, the chapter concludes in Section 4.5.

Datasets

We have validated the accuracy and robustness of our approach with the 15 MRI public dataset of MICCAI prostate challenge [START_REF]MICCAI. 2009 prostate segmentation challenge MICCAI[END_REF] in a leave-one-patient-out validation strategy. The datasets are axial T2 volumes acquired with fast relaxation fast spin echo MR imaging. The slice thickness is 4 mm, repetition time of 5100 ms

Probabilistic atlas

Recently Martin et al. [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF] and Dowling et al. [START_REF] Dowling | Automatic atlas-based segmentation of the prostate[END_REF] have used demon registration to build atlases that have achieved promising results for prostate segmentation. Following a similar approach, we use the probabilistic atlas-based segmentation of the prostate using demons registration [START_REF] Thirion | Image matching as a diffusion process: an analogy with maxwell's demons[END_REF]. Demons registration computes the voxel velocities or transformation field between the moving and reference volumes.

The displacement field is computed on a regular grid with one displacement vector per voxel. The demons energy is computed from the difference of voxel intensities between the moving and reference volumes. The minimization of the energy gradient provides the corresponding update (U ) of a given transformation field (S). Edge forces of both the moving and reference volumes improve the registration convergence and stability. If M and F represent the moving and reference volumes respectively, then the voxel velocity u at voxel p with m and f as the respective voxel intensities is given by Eq. (4.1) and the demons energy E(u) is given by Eq. (

. u = (m -f )∇f |∇f | 2 + α(m -f ) 2 + (m -f )∇m |∇m| 2 + α(m -f ) 2 (4.1) E(u) = F -M • (S + U ) 2 + σ 2 i σ 2 x U 2 . (4.2) 4.2)
where ∇f and ∇m are the respective intensity gradients and α is a normalization factor that adjusts the force strength, σ 2 i and σ 2 x are the constants for intensity and transformation uncertainties, respectively.

The process of atlas construction begins with alignment of N manually segmented training datasets to a common reference. One among N training datasets is manually selected by an expert to reduce bias and N -1 datasets are registered to the reference dataset. The registration is done in two stages, intensity based affine registration of N -1 datasets to the reference dataset is followed by the non-rigid demons registration. The mean volume is computed by averaging all patient volumes aligned to the reference volume. The probability map is obtained by averaging the deformed patient volume labels. Given a new patient dataset, the atlas is first registered to the dataset using affine and demons based registration. Once registered, the transformation of the atlas probability map determines the probabilistic segmentation of the new patient dataset given by P at . Following [START_REF] Klein | Automatic Segmentation of the Prostate in 3D MR Images by Atlas Matching Using Localized Mutual Information[END_REF][START_REF] Gubern-Merida | Atlas Based Segmentation of the Prostate in MR Images[END_REF] we man-ually select the volume-of-interest encompassing the prostate, the bladder and the rectum to reduce the computational time.

Random forest based classification in MRI

MRI intensities of the prostate and the background regions are difficult to differentiate. Also the inter-patient intensities inside the prostate region may vary significantly depending on the acquisition parameters and imaging artifacts. Such intensity variations may affect graph cut based energy minimization framework. Therefore, to reduce the intensity variations and significantly differentiate between the prostate and the background regions we propose to substitute intensities with posterior probabilities of a voxel being prostate. Our probabilistic classification problem is addressed in a supervised learning schema of random decision forest [START_REF] Geremia | Spatial decision forests for ms lesion segmentation in multi-channel mr images[END_REF] as proposed, explained and illustrated in Section 3.4.

The training phase begins with the normalization of intensities of the training volumes of interest and with their rigid alignment to minimize the pose and intensity variations. The inter-patient intensity variations are linearly normalized between 0 and 1 and we use Evangelidis et al. [START_REF] Evangelidis | Parametric image alignment using enhanced correlation coefficient maximization[END_REF] method for rigid alignment. The data for training consists of a collection of 3 × 3 × 3 neighborhood of voxels, centered at V = (X, F ), with X = (x, y, z) denoting the position of the voxel associated with a feature vector F . The feature vector F consists of the mean and standard deviation of the 3×3×3 voxel neighborhood. Each tree t in a decision forest of T trees receives the full data set V along with the label and selects a test to split V into two subsets to maximize information gain where, a test is a feature response threshold. The left and the right child nodes receive their respective subsets of V and the process is repeated at each child node to grow the tree. The growth is terminated if either the information gain is minimum or the tree has grown to a maximum specified depth. Each decision tree in the forest is unique as each tree node selects a random subset of features and threshold.

During testing, the manually selected volume of interest of the test dataset encompassing the prostate, the bladder and the rectum with normalized intensities is rigid aligned to the pre-registered training data. Each voxel of the test dataset is propagated through all the trees by successive application of the relevant bi-Segmentation in MRI nary test to determine the probability of belonging to class c. When reaching a leaf node l t in all tree with t ∈ [1..., T ], posterior probabilities (P t (c|V )) are gathered in order to compute the final posterior probability of the voxel defined by P (c|V ) = 1 T T t=1 P t (c|V ). Geremia et al. [START_REF] Geremia | Spatial decision forests for ms lesion segmentation in multi-channel mr images[END_REF] imposed spatial restriction on the classified voxels by incorporating spatial information of the voxels obtained from the atlas. Similarly, to impose probabilistic implicit shape and spatial prior to the decision forest classification, we obtain a probabilistic shape and spatial prior model P sp of the prostate by averaging the intensity-based affine registration of the ground truth obtained from the training datasets. P sp is aligned with the center of the volume obtained from decision forest classification and the shape and spatial priors are imposed on the random forest classification by obtaining the likelihood value of a voxel being prostate as P lk = P (c|V ) × P sp .

Probabilistic segmentation of the prostate obtained using a probabilistic atlas (P at ) is fused with the likelihood values P lk to achieve the final probabilistic classification of the prostate by P f n = log (P at ) + log (P lk ). Log likelihood minimizes the effect of error incorporated either from the demon registration or from the random forest classification.

Graph cut based energy minimization

The segmentation problem may be formulated as maximum a posteriori estimation of a Markov random field, and could be solved in a graph cut energy minimization framework [START_REF] Boykov | Graph cuts and efficient n-d image segmentation[END_REF]. The graph G = V x, ǫ is defined as a set of voxels V x and a set of edges ǫ connecting neighboring voxels where the objective is to compute the best cut that minimizes the sum of the costs of the edges. Close neighboring voxels have higher edge costs. Two specially designated terminal nodes Sr (source) and T a (sink) that represent the prostate and the background have to be manually selected by the user. However, we use soft classification of the prostate to automatically determine Sr and T a. Typically, the neighboring voxels are interconnected by edges in a regular grid like structure. The objective of graph cut based energy minimization is to completely separate the terminals Sr and T a, thereby segmenting the prostate from the background. In our model, we build the graph with soft classification of the voxels and use graph cut over the soft classification to achieve the final 3D Our model could be formalized as; let a be a voxel and B be the set of all voxels and x a be 0 or 1 depending on a belonging to the background or the prostate and x c is the neighbor of x a . Let E a be the individual voxel matching cost for a; E a,c vary inversely with the difference of intensities of voxels a and c. Then the cost function is given as,

E = a∈B E a (x a ) + (a,c)∈ǫ E a,c (x a , x c ) (4.3)
where ǫ is the set edges of neighboring voxels. The first term represents the cost information related to data, while the second term represents a smoothness related cost. Energy E is minimized by max-flow/min-cut based graph cut [START_REF] Boykov | Graph cuts and efficient n-d image segmentation[END_REF]. Graph cut based energy minimization is illustrated in Fig. 4.2.

Experimental Results

Results

During validation, probabilistic atlas and decision forest are build with 14 training datasets as discussed in sections 4.3.1 and 4.3.2. The number of trees were fixed to 100, tree depth to 30 and the lower bound of information gain to 10 -7 in decision forest as these parameters produced promising results with test images. The features of random forest were limited to mean and standard deviation of voxels. During testing, the probabilistic atlas is registered to the test dataset and the probabilistic labels are transformed to achieve a probabilistic segmentation of the prostate. Next, We have used the popular prostate segmentation evaluation metrics like Dice similarity coefficient (DSC), and 95% Hausdorff distance (HD) to evaluate our method.

To have an overall quantitative estimate of our performance we have compared our method with the results published in the MICCAI prostate challenge 2009 [START_REF] Gubern-Merida | Atlas Based Segmentation of the Prostate in MR Images[END_REF][START_REF] Dowling | Automatic atlas-based segmentation of the prostate[END_REF] and with the work of Gao et al. [START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF] in Table 4.1. Please note that [START_REF] Dowling | Automatic atlas-based segmentation of the prostate[END_REF] used a probabilistic atlas for their segmentation achiving a DSC value of 0.73; however, our stochastic framework which combines the probabilities from decision forest and probabilistic atlas produces better results (DSC 0.91). In fact, statistically significant improvement in DSC and HD of student P test t-value < 0.0001 has been achieved compared to [START_REF] Dowling | Automatic atlas-based segmentation of the prostate[END_REF][START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF]. Moreover [START_REF] Gao | A Coupled Global Registration and Segmentation Framework with Application to Magnetic Resonance Prostate Imagery[END_REF] used shape and local region based statistics of mean and standard deviation of the voxels to propagate their levelsets to achieve a deterministic segmentation of the prostate. We use similar features but employ a stochastic approach and use a MAP-MRF approach to compensate mis-classifications and achieve better results.

The combined framework of probabilistic atlas and random forest works as the two approaches produces better results in different zones of the prostate. The probabilistic atlas produces good results in the central region of the prostate while random forest produces better results in the base and the apex regions. Quantitatively, atlas based registration produces a DSC of value of 0.74±0.18, random forest produces a DSC value of 0.78±0.25, with average model refinement of random forest classification a DSC of 0.81±0.15 is achieved and combining the two probabilities in log scale produced DSC of 0.87±0.09 is obtained. Graph cut base segmentation further improves the result to 0.91±0.04 as observed in Table 4.1.

In Fig. 4.3 we illustrate the results obtained with random forest and probabilistic atlas for the apex, central and the base slices. From Fig. 4.3 we observe that spatially constrained random forest produces better results in the base and apex regions of the prostate compared to probabilistic atlas. However, probabilistic atlas produces better results in the central region.

In Fig. 4.4 we observe that random forest produces mis-classifications outside the volume of interest in one of the base slice. However the product of random forest probability and average model suppress such mis-classifications to improve classification accuracy.

Qualitative results of our method are presented in Fig. 4.5. In Fig. 4.5 we observe that our combined framework of probabilistic atlas, random forest and graph cut optimization approximates the ground truth closely. Prostate segmentation in the base and the apex regions are fairly difficult due to low contrast images and produces segmentation errors with atlas based segmentation [START_REF] Martin | Automated Segmentation of the Prostate in 3D MR Images Using a Probabilistic Atlas and a Spatially Constrained Deformable Model[END_REF]. However our model that combines probabilistic atlas with a supervised learning framework of random forest produces good results as could be observed from the segmentation of the base and the apex slices.

Ground truth and segmented volume of some more datasets are given in Fig. 4.6. It could be observed from Fig. 4.6 that our model closely approximates the ground truth volume constructed from the manual segmentations. The inter patient differences in prostate volumes are once again observed from these datasets.

Discussions

A novel schema of graph cut based energy minimization in a stochastic domain obtained with atlas based segmentation and shape constrained decision forest with the goal of segmenting the prostate in MRI has been proposed. Our method is robust to significant shape, size and contrast variations in MRI compared to some existing work in the literature. The proposed method has shown promising results however the algorithm should be validated with more datasets and feature selection may improve the probabilistic classification of the random forest. Chapter 5

Conclusions

In this last chapter a summary of the thesis is presented. We present our conclusions and provide insight into future course of research. Besides, a list of the publications directly related to this thesis is given.

Summary of the thesis

The primary goal of this thesis was to provide accurate, robust and computationally efficient algorithms for prostate segmentation in TRUS and MRI since registration performed on prostate contours significantly reduces computational time and improves registration accuracies [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. The registration algorithm developed for the PROSCAN project [START_REF] Mitra | A spline-based non-linear diffeomorphism for multimodal prostate registration[END_REF] is more accurate compared to some of the methods available in literature like B-splines [START_REF] Rueckert | Nonrigid registration using free-form deformations: Application to breast MR images[END_REF] and traditional thin-plate splines [START_REF] Bookstein | Principal warps: Thin-plate splines and the decomposition of deformations[END_REF][START_REF] Rohr | Landmark-based elastic registration using approximating thin-plate splines[END_REF]. The method primarily works on binary masks obtained from segmentation of prostate in TRUS and MRI and hence segmentation of prostate is absolutely necessary for the registration method. In recent times we have also developed a computationally efficient version of the registration method that is computationally more efficient compared to [START_REF] Rueckert | Nonrigid registration using free-form deformations: Application to breast MR images[END_REF][START_REF] Bookstein | Principal warps: Thin-plate splines and the decomposition of deformations[END_REF][START_REF] Rohr | Landmark-based elastic registration using approximating thin-plate splines[END_REF]. In this method the deformation parameters of the thin-plate splines are learnt offline using a training set of TRUS-MR images and the resulting deformation models formed by spectral clustering are applied in their Gaussian spaces to register a new pair of TRUS-MR images. A detailed study of this fast registration method can be found in [START_REF] Mitra | Spectral clustering to model deformations for fast multimodal prostate registration[END_REF].

Chapter 5. Conclusions

In the thesis a thorough review of prostate segmentation methods in TRUS and MRI was carried out. Careful analysis of the methods revealed that shape and appearance priors in segmentation algorithms improves segmentation accuracies. Active appearance model (AAM) of Cootes et al. [START_REF] Cootes | Active Appearance Models[END_REF] was selected as it combines shape and appearance prior in a single cost function in the optimization framework. Further, off-line gradient decent optimization significantly reduces computational time requirement of the process.

In TRUS images the image quality is adversely affected by speckle and imaging artifacts. To deal with the challenges we introduced texture features extracted with Haar wavelets and quadrature filters in AAM to improve on segmentation accuracies. We demonstrated that the increase in segmentation time of the process due to the introduction of the filter could be compensated by accurate texture information from the filters. The new model improved on segmentation accuracies without significantly increasing the segmentation time requirement of the process.

To facilitate automatic initialization of our model we adopted a probability theoretic approach to image modeling. Supervised learning provided a probabilistic model of the prostate and a new statistical shape and appearance model was developed in which image intensities in the appearance model was substituted with the probabilities. Such an approach aided in automatic initialization and improved segmentation accuracies.

Probabilities obtained from the supervised learning framework was introduced in Mumford-Shah energy minimization framework of a variational model to improve on segmentation accuracies further. Parametric representation of the implicit curve was derived from principal component analysis (PCA) of the signed distance representation of the labeled training data to impose shape prior. Posterior probability of the prostate region facilitated initialization and propagation of our model. The cost function of the new model developed from implicit shape priors, region based energy and local topology was minimized in a Mumford-Shah energy minimization framework to improve segmentation accuracies.

Later for prostate segmentation in MRI, the same supervised learning framework was adopted for producing a 3D probabilistic representation of the prostate. Posterior probabilities obtained from the learning framework was combined with probabilities obtained from atlas based segmentation to produce a initial 3D soft

• A new graph cut energy minimization in stochastic domain for prostate segmentation MRI is proposed. Leave-one-patient-out validation with MICCAI prostate challenge database produced significant improvement in segmentation accuracies compared to some of the methods in literature.

Further work

Accurate prostate segmentation in TRUS and MRI is a challenging task and the challenges varies from one imaging modality to another. In this thesis we have discussed various solutions that have been adopted and described them in detail. However, an important number of ideas remain underdeveloped and needs to be analyzed and investigated in detail. Future directions for research are organized in three different blocks: further work to improve segmentation accuracies and improve on segmentation time requirement, future research directions departing from this thesis, and technological directions.

Segmentation accuracies and time

In this thesis we have demonstrated that incorporating texture information in statistical shape and appearance model improves segmentation accuracies. However texture information of the prostate region is unreliable and unpredictable. An interesting direction for research could be the identification of the discriminitative features that identifies the prostate in TRUS and MRI and use of such features in a learning framework to improve on segmentation accuracies. We have adopted the supervised learning framework of random forest for a soft classification of the prostate. However, the use of integral window and incorporating Haar like features in GPU based parallelization could significantly reduce segmentation time requirement of the process. Considering that discriminitative features could be identified and used in parallel random forest based classification the requirement of a deformable model may be entirely removed from the process further reducing computational time requirement of the prostate segmentation. Treating prostate segmentation as a tracking problem of prostate contours in consecutive frame using particle filters would reduce computational cost of the process. Finally, translation of the entire 5.2 Further work 115 codes from Matlab to C++ will improve computational time of the entire process.

In this thesis we have explored statistical shape and appearance model for prostate segmentation in TRUS. However, in these models the propagating contour is derived from linear PCA of shape and appearance parameters. However, use of non-linear PCA may improve shape and appearance model by better approximating the primary components of the shape and appearance. Another interesting direction for research would be the use of independent component analysis (ICA) to de-correlate the shape and appearance spaces.

Future research lines departing from this thesis

The second group of further work focuses on the development of new research directions stemming from this thesis. We think that primary focus for such research direction should be the optimization framework adopted for segmentation.

We have focused on three different kind of energy minimization framework for this thesis. We adopted an off-line gradient decent optimization for prostate segmentation in TRUS to improve on computational time requirement of the process. We have also explored a Mumford-Shah energy minimization in the variational framework that produces more accurate results for the TRUS images compared to out previous method but requires additional computational time optimization. Finally, we have explored graph cut based energy minimization framework for prostate segmentation in MRI to adopt a MAP-MRF approach to reduce mis-classification by imposing spatial constraints. However, each of these optimization framework could be improved further for instance adopting a non-linear approach to shape and appearance modeling in off-line gradient decent optimization may improve shape and the appearance model further. Further, using a sparse representation in variational model may improve computational time without sacrificing segmentation accuracies. Segmentation accuracies with graph cut energy minimization could be improved further by introducing a shape prior in the framework.

Technological further work

The last further direction is the implementation of the proposed algorithms into a clinical practice to improve the rate of positive biopsies in the PROSCAN project.

After the implementation of the proposals, we will have larger database to clinically test the algorithms. In fact, we expect that using a larger training database will significantly improve segmentation accuracies considering prior learning of shape and appearance parameters adopted in the algorithm.

Prostate segmentation is still an open problem and with advancement of technology for diagnosis, treatment and follow up of prostate diseases new requirements have to be met. Multimodal image fusion of at least two imaging modalities provides valuable information. For example, the fusion of MRI and PET imaging should aid in identifying malignant tissues more accurately. However, for such a method to work in a real scenario, automatic, accurate and real time fusion of the two imaging modalities is necessary. Under such circumstances automatic real time segmentation of the prostate and registration on prostate contours would improve accuracy and efficiency. Automatic and accurate real time segmentation of the prostate may be achieved with efficient algorithms designed for graphical processing units. An increase in 3D prostate segmentation methods will be the trend in coming years due to the increasing use of 3D imaging modalities, where efficient and accurate algorithms are necessary. In that sense, information from dynamic contrast enhanced MRI, and MR spectroscopy, will be increasingly used as additional features for automatic segmentation.

Related publications

A list of publications of the author for the PhD candidacy is given below, ordered according to their topic.

Publications related to the prostate segmentation in TRUS:

• [IJCARS 2012] S. Ghose, A. Oliver, R. Martí X. Lladó J. Freixenet, J. Mitra

Figure 1 :

 1 Figure 1: Contraste des tissus de la prostate en échographie transrectale et l'IRM. Un meilleur contraste des tissus mous dans l'image IRM permet la visualisation d'une tumeur qui n'est pas visible dans l'image ETR.

Figure 2 :

 2 Figure 2: Illustration de l'objectif du projet PROSCAN

  xii

Figure 3 :

 3 Figure 3: Exemple de segmentation de la prostate dans une image ETR. De haut en bas: résultat avec le modèle AAM tradiotionel et résultat avec notre méthode. Le vrai contour de la prostate est représenté par la courbe verte.

Figure 4 :

 4 Figure 4: Segmentation 3D de la prostate dans les images IRM. Le contour/volume vert est obtenu à partir de la vérité terrain tandis que le contour/volume rouge est obtenue par la méthode proposée.

  xvii

Figure 5 : 1 Contents 3

 513 Figure 5: Exemple de segmentation 3D de la prostate dans des images IRM. Le contour/volume vert est obtenu à partir de la vérité terrain tandis que le contour/volume rouge est obtenue par la méthode proposée.

Figure 1 . 1 :

 11 Figure 1.1: In TRUS imaging, the axial view of the prostate is shown as a hypoechoic mass surrounded by a hyperechoic halo in (a). For a better understanding of the prostate region, the contour is outlined in green in (b).

Figure 1 . 2 :

 12 Figure 1.2: Prostate segmentation challenges , A=Low Contrast, B=Micro Calcification, C=Intensity heterogeneity inside prostate, D=Speckle.

Figure 1 . 3 :

 13 Figure 1.3: Inter patient variation in prostate shape and size of six patients.

Figure 1 . 4 :Figure 1 . 5 :

 1415 Figure 1.4: Contrast variation depending on machine manufacturer.

Figure 1 . 6 :

 16 Figure 1.6: In MRI imaging, the better soft tissue contrast enables lesion detection. Image (a) shows a prostate in MRI. For a better understanding of the prostate region, the contour is outlined in green in (b).

Figure 1 . 7 :

 17 Figure 1.7: Inter patient variation in prostate shape, size and volume. (a), (b), and (c) shows inter-patient variation in the central region of the prostate. (d),(e), and (f) show inter-patient volume variation. The volumes are created from manual segmentations.

Figure 1 . 8 :

 18 Figure 1.8: Contrast and shape variation around prostate depending on placement of the magnetic coil.

Figure 1 . 9 :

 19 Figure 1.9: In MRI, the soft tissue contrast varies depending on the region of the prostate. Image in the center shows prostate in different region and image in the right shows corresponding segmentations.

1. 3 13 TumorFigure 1 . 11 :Figure 1 . 12 : 2 A

 3131111122 Figure 1.11: Prostate in TRUS and MRI. Better soft tissue contrast in MRI aids in visualization of a tumor that is not visible in TRUS images.

Chapter 2 .Figure 2 . 1 :

 221 Figure 2.1: Our proposed taxonomy of prostate segmentation.

Chapter 2 .

 2 A Survey of Prostate Segmentation Methods in TRUS and MRI feature space. Supervised and un-supervised classification based techniques aim at obtaining a partition of the feature space into a set of labels for different regions. Primarily classifier and/or clustering based techniques are used for the purpose.

  of finite points A = {a 1 , a 2 , ....a p } and B = {b 1 , b 2 , ...., b q } HD (A, B) = max (h (A, b) , h (B, A)) where h (A, B) = max a∈A (min b∈B ab )

  +T P )+(T P +F N )

  of finite voxels A = {a 1 , a 2 , ....a p } and B = {b 1 , b 2 , ...., b q } HD (A, B) = max (h (A, b) , h (B, A)) where h (A, B) = max a∈A (min b∈B ab ) +T P )+(T P +F N )

  (M SV ASV )-(M SV ASV )

  The segmentation criteria shows in what category the algorithm is classified. Hybrid segmentation methodologies are specified with the type of algorithms that are combined to produce the final segmentation (the acronyms of this row are: DM = Deformable model, ASM = Active shape model, AAM = Active appearance model, GA = Genetic algorithm, EM = Expectation maximization, DDC = Discrete dynamic contour, ACM = Active contour model, SVM = Support vector machine, ANN = Artificial neural network, S-R Level set = Shape and region based level set).

2. 5

 5 Discussions 51 to improve accuracy, as demonstrated by hybrid methods. Hybrid methods that use prior information from different sources are more robust to noise and produce superior results

Figure 3 . 1 :

 31 Figure 3.1: Automatic landmark detection and alignment (a) Prostate image in TRUS (b) Ground truth from manual segmentation (c) Contour extracted from ground truth (d) Automatic landmark detection with equal angle sampling (e) Contours overlap before alignment (b) Contour overlap after alignment with generalized Procrustes analysis.

Figure 3 . 2 :

 32 Figure 3.2: Illustration of the shape prior model. Primary modes of shape variations are identified with PCA of the aligned contours. The mean shape prior model is given by x, P are shape eigenvectors and λ corresponding eigenvalues. Considering a Gaussian shape representation new shapes could be generated.

Figure 3 . 3 :

 33 Figure 3.3: Illustration of the appearance model. Primary modes of appearance variations are identified with PCA of the sampled image.The mean appearance prior model is given by g, P g are shape eigenvectors and λ corresponding eigenvalues. Considering a Gaussian shape representation new shapes could be generated.

Figure 3 . 4 :

 34 Figure 3.4: Second level Haar wavelet decomposition of the prostate. The top-right, bottom-left and bottom-right boxes show the horizontal, vertical and diagonal detail coefficients respectively after the 1st level wavelet decomposition. The top-left box (the 1st level approximation coefficient) further consists of 4 smaller images where again, the top-right, bottom-left and bottom-right images are the detail coefficients of the 2nd level wavelet decomposition and the top-left comprises of the 2nd level approximation coefficients.

Figure 3 . 5 :T

 35 Figure 3.5: First level wavelet decomposition of a prostate image, A = Microcalcification, (a) 2D TRUS image of the prostate, (b) discarded horizontal detail coefficients, (c) discarded vertical detail coefficients, (d) discarded diagonal detail coefficients, (e) processed image of the prostate.

Figure 3 . 6 :

 36 Figure 3.6: Log-Gabor filter in the frequency and spatial domain. (a) The filter in frequency domain, (b) even part of the filter in spatial domain, (c) odd part of the filter in spatial domain, (d)prostate Image, (e) and its phase response of quadrature filter pairs.

Figure 3 . 7 :

 37 Figure 3.7: Schematic representation of our approach. The final segmentation is given in red contour and ground truth in green. Abbreviations used AAM = Active Appearance Model, PDM = Point Distribution Model, GPA = Generalized Procrustes Analysis.

Figure 3 . 8 :

 38 Figure 3.8: Multi-resolution functioning of the model (a) Initialization of the mean model (black contour) (b) level 4 segmentation result, (c) level 3 segmentation result, (d) level 2 segmentation result, (e) final segmentation result. Ground truth shown with white contour while the black contour shows the segmentation achieved.

Figure 3 . 9 :

 39 Figure 3.9: Random forest classification framework (a) Random forest training (b) Random forest classification with a test image.

Figure 3 . 10 :

 310 Figure 3.10: Schematic representation of our approach. The final segmentation is given in red contour and ground truth in green. Abbreviations used AAM = Active Appearance Model, PDM = Point Distribution Model, GPA = Generalized Procrustes Analysis.

Figure 3 . 11 :

 311 Figure 3.11: Mean models fitting errors for with dataset 1 as reference.

3 .Figure 3 . 12 :

 3312 Figure 3.12: Illustration of spectral clustering.

Figure 3 . 13 :

 313 Figure 3.13: Schematic representation of our approach. The final segmentation is given in black contour and ground truth in white. Abbreviations used AAM = Active Appearance Model, PDM = Point Distribution Model, GPA = Generalized Procrustes Analysis.

Figure 3 .

 3 Figure 3.14: (a) The aligned contours of the training prostate images. (b) SDFs of the aligned training dataset with black contour showing the mean shape.

1 Figure 3 .

 13 Figure 3.15: (a), (b), (c), and (d) show the first four primary eigenmodes of variations of the prostate. (f) Shows the mean shape (black contour) Φ and (e) and (g) represent the variance in mean shape (black contour) with Φ ± 2λ 1 given by the magenta contour.

Figure 3 . 17 :

 317 Figure 3.17: Illustration of working of our model. (a) is the image to be segmented, (b) random forest classification, (c) contour energy, (d) region based energy, (e) shows the initial levelsets (red contour) with SDFs and (f) segmentation with our model.

Figure 3 . 18 :

 318 Figure 3.18: Manual clustering fitting error graph for the 5 groups. The green datapoint denotes the group members and the red datapoints denotes the datasets that was not considered as they were already grouped. For example in (b) dataset 1,6,8,10,15, and 21 are denoted with red datapoints as they were grouped in group 1 as shown in (a). The datasets that are grouped for group 2 are 2,14, and 17 denoted with a green datapoints.

Figure 3 .

 3 Figure 3.19: (a) Artifacts in TRUS image of the prostate, A=Low SNR, B=Micro Calcification, C=Intensity heterogeneity inside prostate, D=Speckle Noise. (b) Manual initialization of the mean model, (c) Final segmentation result. White contour ground truth and black contour is the segmentation achieved.

Figure 3 . 20 :

 320 Figure 3.20: Illustration of the requirement of shape and probability prior model. (a) and (d) are the images to be segmented, (b) and (e) show the random forest classification with mis-classified regions and (c) and (f) show the final segmentation achieved.

Figure 3 . 21 :

 321 Figure 3.21: Improvement in segmentation accuracies with spectral clustering and multiple mean models. (b) and (d) show segmentation with, and (a) and (c) show segmentation without multiple mean models.

Figure 3 . 22 :

 322 Figure 3.22: The green contour gives the ground truth and the red contour gives the obtained result. Column 1 shows the results achieved with AAM and column 2 with RF-AAM-Mult model.

Figure 3 . 23 :

 323 Figure 3.23: Illustration of working of our model and qualitative results of different levelsets. (a) is the image to be segmented, (b) shows the initial levelsets (red contour) with SDFs, (c) segmentation with our model, (d) segmentation with levelsets [23], (e) segmentation with levelset [23] on posteriors, (f) segmentation with levelsets [21] on intensity. In (c), (d), (a), and (f) green contour is the ground truth and red contour is the obtained segmentation.

Figure 3 . 24 :

 324 Figure 3.24: The green contour gives the ground truth and the red contour gives the obtained result. Row 1 shows the results achieved with AAM and row 2 shows results of our model for the corresponding prostates.

Figure 3 . 25 :

 325 Figure 3.25: The green contour gives the ground truth and the red contour gives the obtained result.

Figure 4 . 2 :

 42 Figure 4.2: Sr(S) and T a(T ) terminals are identified automatically from posterior probabilities. Graph cut based energy minimization in 3D provides the segmentation.

Figure 4 . 3 :

 43 Figure 4.3: Spatially constrained random forest produces better results compared to probabilistic atlas in the base and the apex region of the prostate. However probabilistic atlas produces better results in the central region of the prostate. Form top to down rows apex, central and base images of prostate. First column original image, second column atlas based segmentation and third column spatially constrained random forest based segmentation.

Figure 4 . 4 :

 44 Figure 4.4: (a) is one of the base slice. (b) is a random forest classification of the slice with mis-classifications outside the volume of interest. (c) shows the average model for the base slice and (d) shows the probabilities obtained by the fusion of (b) and (c) that reduces mis-classifications.

Figure 4 . 5 :

 45 Figure 4.5: Subset of segmentation results of 3 datasets. The green contour/volume is created from the ground truth and red contour/volume is created from obtained segmentation.

Figure 4 . 6 :

 46 Figure 4.6: Subset of segmentation results of 6 more datasets. The green volumes are created from the ground truth and red volumes are created from obtained segmentation of the corresponding volumes.

  , J.C. Vilanova, J. Comet, and F. Meriaudeau. Statistical shape and texture model of quadrature phase information for prostate segmentation. International Journal of Computer Assisted Radiology and Surgery, Volume 7, Issue 1, pp 43-55, Heidelberg, Germany, Springer-Verlag, 2012. • [IAPR ICPR 2012] S. Ghose, J. Mitra, A. Oliver, R. Martí, X. Lladó, J. Freixenet, J.C. Vilanova, J. Comet, D. Sidibé and F. Meriaudeau. A Mumford-Shah Functional based Variational Model with Contour, Shape, and Probability Prior information for Prostate Segmentation. IAPR International Conference on Pattern Recognition, (accepted), Tsukuba, Japan. August 2012.

	xiv
		Random Forest Training
	Feature
	Vector (V)
	Training Images and
	Ground Truth	Implicit Shape Prior
	Intensity Based
		Affine
	Registration
						å
	Probabilistic Atlas Creation
		Probabilistic
	Rigid and Non	Atlas
	Rigid Demon				Graph Cut
	Registration				T
		Rigid and Non Rigid
		Demon Registration			S
	Test Images	Random Forest Testing
		v	v	split nodes leaf nodes	P(c|v)	3D Optimized Segmentation
		tree t1		tree tT
	V	……
		category c
		category c

• [IEEE EMBC 2012] S. Ghose, J. Mitra, A. Oliver, R. Martí, X. Lladó, J.

  3.2 Statistical shape and appearance model . . . . . . . . . . . . . . . . . 3.2.1 Optimization and segmentation of a new instance . . . . . . . 3.2.2 Limitations of statistical shape and appearance model applied to prostate segmentation in TRUS . . . . . . . . . . . . . . . 3.3 Texture in appearance model . . . . . . . . . . . . . . . . . . . . . . Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Further work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Segmentation accuracies and time . . . . . . . . . . . . . . . . 5.2.2 Future research lines departing from this thesis . . . . . . . . Our proposed taxonomy of prostate segmentation. . . . . . . . . . . . 3.1 Automatic landmark detection and alignment . . . . . . . . . . . . . 3.2 Illustration of the shape prior model . . . . . . . . . . . . . . . . . . 3.3 Illustration of the appearance model . . . . . . . . . . . . . . . . . . Schematic representation of our approach . . . . . . . . . . . . . . . . 4.2 Graph cut based prostate segmentation . . . . . . . . . . . . . . . . . Prostate segmentation in MRI . . . . . . . . . . . . . . . . . . . . . . 4.6 Segmented prostate volumes in MRI . . . . . . . . . . . . . . . . . .

	iv	Contents List of Figures	Contents	iii vii
		4.5 5 Conclusions 4.1 4.3 Spatially constrained random forest classification of the prostate . . . 4.4 Improvement in probabilistic classification with spatial priors. . . . . 5.1 Bibliography List of Figures 4.5

3.3.1 Speckle reduction with haar wavelets . . . . . . . . . . . . . . 3.3.2 Quadrature filters to enhance prostate texture . . . . . . . . . 3.4 Probability theoretic approach to appearance model . . . . . . . . . . 3.4.1 Random forest based probabilistic classification . . . . . . . . 3.5 Multiple mean models . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Manual clustering . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Spectral clustering . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Mumford-Shah energy minimization . . . . . . . . . . . . . . . . . . . 3.7 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Graph Cut Optimization in a Stochastic Framework for Prostate Segmentation in MRI 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Proposed segmentation framework . . . . . . . . . . . . . . . . . . . . 4.3.1 Probabilistic atlas . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Random forest based classification in MRI . . . . . . . . . . . 4.3.3 Graph cut based energy minimization . . . . . . . . . . . . . . 4.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Technological further work . . . . . . . . . . . . . . . . . . . . 5.3 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Contraste des tissus de la prostate en échographie transrectale et l'IRM. Un meilleur contraste des tissus mous dans l'image IRM permet la visualisation d'une tumeur qui n'est pas visible dans l'image ETR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii 2 PROSCAN Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix 3 Exemple de segmentation de la prostate dans une image ETR. De haut en bas: résultat avec le modèle AAM tradiotionel et résultat avec notre méthode. Le vrai contour de la prostate est représenté par la courbe verte. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii 4 Segmentation 3D de la prostate dans les images IRM. Le contour/volume vert est obtenu à partir de la vérité terrain tandis que le contour/volume rouge est obtenue par la méthode proposée. . . . . . . . . . . . . . . xiv 5 Exemple de segmentation 3D de la prostate dans des images IRM. Le contour/volume vert est obtenu à partir de la vérité terrain tandis que le contour/volume rouge est obtenue par la méthode proposée. . . xvii 1.1 Prostate in TRUS images. . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 Prostate segmentation challenges . . . . . . . . . . . . . . . . . . . . 5 1.3 Inter patient variation in prostate shape and size. . . . . . . . . . . . 6 1.4 Contrast variation depending on machine manufacturer. . . . . . . . . 7 1.5 Contrast varies depending on acquisition parameters for the same machine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.6 Prostate in MRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.7 Inter patient variation in prostate shape, size and volume. . . . . . . 9 1.8 Contrast and shape variations. . . . . . . . . . . . . . . . . . . . . . . 10 1.9 Apex, central and base of prostate in MRI . . . . . . . . . . . . . . . 11 v vi List of Figures 1.10 Prostate in CT images . . . . . . . . . . . . . . . . . . . . . . . . . . 1.11 Tissue contrast of prostate in TRUS and MRI . . . . . . . . . . . . . 1.12 PROSCAN Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 3.4 Second level Haar wavelet decomposition of the prostate . . . . . . . 3.5 First level wavelet decomposition of a prostate image . . . . . . . . . 3.6 Log-Gabor filter in the frequency and spatial domain . . . . . . . . . 3.7 Schematic representation of texture based appearance model . . . . . 3.8 Multi-resolution functioning of texture based appearance model . . . 3.9 Random forest classification framework . . . . . . . . . . . . . . . . . 3.10 Schematic representation of probability prior appearance model. . . . 3.11 Manual clustering of similar prostates . . . . . . . . . . . . . . . . . . 3.12 Illustration of spectral clustering. . . . . . . . . . . . . . . . . . . . . 3.13 Schematic repesentation of multiple mean model. . . . . . . . . . . . 3.14 Levelset based mean model . . . . . . . . . . . . . . . . . . . . . . . . 3.15 Levelset based shape model . . . . . . . . . . . . . . . . . . . . . . . 3.16 Illustration of shape function . . . . . . . . . . . . . . . . . . . . . . . 3.17 Illustration of levelset based prostate segmentation . . . . . . . . . . 3.18 Fitting error graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.19 Multiresolution functioning of the model . . . . . . . . . . . . . . . . 3.20 Illustration of the requirement of shape and probability prior model . 3.21 Improvement in segmentation accuracies with spectral clustering and multiple mean models . . . . . . . . . . . . . . . . . . . . . . . . . . 3.22 Qualitative results with multiple mean models . . . . . . . . . . . . . 3.23 Illustration of qualitative results of different levelsets . . . . . . . . . 3.24 Qualitative results with levelsets . . . . . . . . . . . . . . . . . . . . . 3.25 Inaccurate segmentation . . . . . . . . . . . . . . . . . . . . . . . . .

Table 1 .

 1 1: Advantages and disadvantages of the prostate imaging techniques. Primarily TRUS, MRI and computed tomography (CT) imaging are used in the diagnosis, treatment, and follow-up of prostate cancer. The use of a particular modality depends on the clinical aim. The main features associated with the different imaging modalities are summarized in Table 1.1. Prostate segmentation from TRUS and MRI plays a key role in different stages of the clinical decision making process. For instance, prostate volume, that can be directly determined from prostate gland segmentation, aids in diagnosis of benign prostate hyperplasia. The prostate boundary is utilized in different treatments of prostate diseases, like prostate brachytherapy, high intensity focused ultrasonogra-

	Statistics from Cancer Research UK show that more than 338, 000 people are diag-
	nosed with prostate cancer every year in Europe and 913, 000 worldwide [1]. The
	highest rate of prostate cancer cases are diagnosed in USA, Australia, New Zealand,
	Western and Northern Europe, while the lowest rates are observed in South and
	Central Asia [1]. Prostate cancer accounts for approximately 7.1% of all cancers
	diagnosed and 3.4% of all cancers deaths. Approximately 10,000 deaths every year

are associated with prostate cancer and about 1 in every 6 men is expected to be diagnosed with prostate cancer during their lifetime.

Table 2 .

 2 1: Advantages and disadvantages of the reviewed prostate segmentation approaches.

Table 2 .

 2 2: Evaluation metrics for contour accuracy

Table 2 .

 2 5: Quantitative evaluation : prostate segmentation in TRUS images

	Reference	Year Dim Segmentation	Auto	Performance	Validation
		Criteria
			Measure	Value

Table 2 .

 2 5: Quantitative evaluation : prostate segmentation in TRUS images continued...

		Reference	Year Dim Segmentation	Auto	Performance	Validation
			Criteria
					Measure	Value
			2002 3D DM -Curve	No	Contour MD	(-)0.2±0.28	5 data sets
		Hu [68]	Fitting		Contour MAD	mm 1.19±0.14
						mm
					Contour MAXD	7.01±1.04
						mm
					Volume VD	7.2±3.4%
		Gong [58]	Fitting 2004 2D DM -Curve	No	Contour MD	mm 1.36±0.58	125 images
	Contour		2006 2D DM -Curve	No	Contour HD Area SN	3.42±1.52 mm 97.4±1%	17 images
		Badiei [11]	Fitting		Area AC	93.5	+/-
						1.9%
					Contour MAD	0.67±0.18
						mm
					Contour MaxD	2.25±0.56
						mm
		Mahdavi[90]	2011 3D DM -Curve	No	Volume VE	6.63±0.9%	21 datasets
			Fitting
		Zaim [148]	2005 2D Classifier -	Yes Area OV	91%	10 images
	PR	Mohammed [99]	ANN SVM 2006 2D Classifier -	Yes Area SN	83.30%	18 regions
					Area AC	93.75%

Table 2 .

 2 5: Quantitative evaluations : prostate segmentation in TRUS images continued...

		Reference	Year Dim Segmentation	Auto	Performance	Validation
			Criteria
						Measure	Value
			2002 2D Edge	and	Yes Contour MD	0.4±1.3 mm	282 images
		Liu [85]	Average
			shape model		Contour MAD	0.9±0.9 mm
	Hybrid Methods	Gong [57] Medina [95]	2005 2D Level and Curve set Fitting 2005 2D AAM	No No	Contour MAXD Contour MAD Area OV Contour MD	3.8 mm 0.64, 0.52 1.16 mm 1.13, and 96% 3.58±1.49 pixels	4 images 95 images
			2006 3D Mesh	and	No	Contour MAD	1.26±0.41	30 data sets
		Tutar [137]	Average				mm
			shape model		Contour MaxD	4.06±1.25
							mm
						Volume VO	83.5±4.2%
			2006 3D SVM,	DM	Yes Contour MD	1.07(≈ 0.33	6 data sets
		Zhan [154]	and Mesh			mm)±0.1
							voxels
						Volume VOE	4.31±0.4%
						Volume VD	2.39±1.29%
		Yang [142]	2006 3D Shape model	Yes Correct segmenta-	82%	11 data sets
			and Level set		tion rate
		Cosío [33]	ASM 2008 2D EM	and	Yes Contour MAD	mm 1.65±0.67	22 images
						Contour MaxD	3.93±1.9
							mm
			2008 3D ACM	and	No	Volume SN	80%	7 data sets
		Diaz [35]	SVM
						Volume AC	> 90%
						Volume SP	> 90%
		Yan [147]	2010 2D ACM	and	Yes Contour MAD	2.01±1.02	10 data sets
			ASM				mm
		Garnier [51]	cut 2011 3D Mesh, graph and	No	Volume VO	86.36±3.78% 28 data sets
			DDC
						Volume HD	4.79±1.62
							mm

Table 2 .

 2 6: Quantitative evaluation : prostate segmentation in MR images

		Reference	Year Dim Segmentation	Auto	Performance	Validation
			Criteria
						Measure		Value
	Bound.	Samiee [121] Tapia [48] Zhu [158]	2006 2D Edge Based 2008 2D Edge Based 2007 3D DM -ASM	No No No	Area DSC Area DSC RMSD		0.905±0.001 0.93±0.005 5.4±2.9 mm	datasets images datasets
		Klein [76]	2008 3D Atlas		Yes Volume	median	0.85	datasets
						DSC
			2010 3D Atlas		Yes Volume DSC error	0.05	datasets
						Volume SN/SP er-	0.05
	Reg.	Langerak [81] Dowling [40]	2011 3D Atlas		ror Yes Volume DSC Area SD		0.86 2.0±1.3 mm	datasets
		Allen [8]	2006 3D EM and DM No	Contour MAD Volume VD	2.8±0.8 mm 6.5±5.4%	datasets
		Martin [93]	2008 3D Atlas	and	No	Mean error		3.3±1.9 mm	datasets
			DM
	Hybrid Methods	Makni [91] Vikal [139] Liu [86]	2009 3D DM Bayes Classifier 2009 3D Edge Shape guidance Level set 2009 2D DM	and and and	Yes Volume HD Volume DSC No Contour MAD Area DSC Yes Area DSC	9.62 mm 0.90 2.0±0.6 mm 0.93±0.3 0.91±0.03	datasets datasets datasets
		Firjani [46]	2010 2D Intensity and	Yes Area OE		5.2±1.2%	images
			shape
		Firjani [47]	2011 3D Intensity and	Yes Contour MD		0.8±0.9 mm	images
			shape
			2010 3D Atlas		Yes Area SD		2.41 mm	datasets
			and DM			DSC		0.84
		Martin [94]				RMSD		1.97
			2010 3D Shape and	No	Volume DSC		0.82±0.03	datasets
		Gao [50]	Edge guidance for		Volume HD		10.22±4.03	data sets
			level sets					mm
		Toth [132]	2011 3D Shape and	Yes Volumetric ratio	1.05±0.21	datasets
			Edge Based
			2011 3D Shape and	Yes Volume VO		0.7	datasets
			Edge Based
		Toth [134]				Contour HD		7mm
			2012 3D Shape and	Yes Volume DSC		0.88±0.04	datasets
			Edge Based
		Toth [133]				MAD		1.51±0.78
								mm
			2012 3D ASM, Atlas	Yes Volume DSC		0.82±0.52	datasets
			and PR
		Chowdhury [26]				MAD		1.88±0.56
								mm

  3. Lack of standardized metrics in evaluation of segmentation result makes the comparison of developed methodologies difficult, as shown in Table 2.2 and Table 2.3. Mean average distance, maximum distance, average distance, area of overlap, area difference, volume overlap and volumetric error are just a few of the commonly used metrics. However, since MICCAI prostate challenge 2009 [96], Hausdorff distance and DSC are being increasingly used.

Table 3 .

 3 1: Prostate segmentation quantitative comparison (HD, and MAD in mm, Spec., and Sens. are for Specificity and Sensitivity respectively. Time is given in seconds.) Statistically significant values are italicized

	Method DSC	HD	MAD	Spec.	Sens.	Time
	AAM	0.94±0.03 4.92±0.96 2.15±0.94 0.89±0.03 0.99±0.01 1.51±0.07
	[30]
	SSTM	0.95±0.02 3.82±0.88 1.26±0.51 0.94±0.03 0.978±0.02 1.30±0.03
	RF-	0.95±0.06 3.76±2.03 1.40±0.91 0.93±0.41 0.97±0.02 0.67±0.02
	AAM
	RF-	0.97±0.01 1.78±0.73 0.49±0.20 0.95±0.01 0.99±0.00 0.67±0.02
	AAM-
	Mult1
	RF-	0.96±0.01 2.51±0.93 0.84±0.31 0.94±0.02 0.99±0.01 0.67±0.02
	AAM-
	Mult2
	RF-	0.97±0.01 1.73±0.24 0.42±0.09 0.95±0.02 0.99±0.00 103.66±2.55
	Impl
	were chosen empirically as they produced promising results with the test images.

Table 4 .

 4

	1: Prostate segmentation quantitative results
	Method	DSC	HD
	Gubern-Merida	0.79	7.11 mm
	[62]
	Dowling [41]	0.73±0.11	-
	Gao [50]	0.82±0.05	10.22±4.03
	Our Method	0.91±0.04	4.69±2.62
	a probabilistic classification of the voxels is achieved with shape restricted decision
	forest and atlas-based segmentation probabilities.

  , J.C. Vilanova, J. Comet, and F. Meriaudeau. Statistical shape and texture model of quadrature phase information for prostate segmentation. International Journal of Computer Assisted Radiology and Surgery, Volume 7, Issue 1, pp 43-55, Heidelberg, Germany, Springer-Verlag, 2012. Meriaudeau, and J.Mitra. Quadrature phase-based statistical shape and appearance for prostate segmentation. Proceedings of Computer Assisted Radiology and Surgery, Springer, Volume 6, pp. S12-S16, Berlin, Germany. June 2011. • [SPIE Medical Imaging 2011] S.Ghose, A.Oliver, R.Martí, X.Lladó, J.Freixenet, J.C.Vilanova, and F.Meriaudeau. Prostate segmentation with local binary patterns guided active appearance model. SPIE Conference on Medical Imaging : Image Processing. Proceedings of the SPIE, Volume 7962, pp. 796218-796218-8. Lake Buena Vista, Orlando, Florida. February 2011. • [MICCAI PCI 2010] S.Ghose, A.Oliver, R.Martí, X.Lladó, J.Freixenet, J.C.Vilanova, and F.Meriaudeau. Texture guided Active Appearance Model propagation for prostate segmentation. MICCAI Workshop on Prostate Cancer Imaging: Computer Aided Diagnosis, Prognosis, and Intervention, LNCS 6367, pp 111-120. Beijing, China. September 2010. Publications related to the prostate segmentation in MRI: • [IAPR ICPR 2012] S. Ghose, J. Mitra, A. Oliver, R. Martí, X. Lladó, J. Freixenet, J.C. Vilanova, J. Comet, D. Sidibé and F. Meriaudeau. Graph Cut Energy Minimization in a Probabilistic Learning Framework for 3D Prostate Segmentation in MRI. IAPR International Conference on Pattern Recognition, (accepted), Tsukuba, Japan. August 2012. • [IEEE ICIP 2012] S. Ghose, J. Mitra, A. Oliver, R. Martí, X. Lladó, J. Freixenet, J. C. Vilanova, D. Sidibé, F. Meriaudeau. A Coupled Schema of Probabilistic Atlas and Statistical Shape and Appearance Model for 3D Prostate Segmentation in MR Images. International Conference on Image Processing (ICIP), to appear, San Diego, USA, Sep-Oct, 2012. • [SPIE Medical Imaging 2012] S. Ghose, A. Oliver, R. Martí, X. Lladó, J. Freixenet, J.C. Vilanova, and F. Meriaudeau. A hybrid framework of multiple active appearance models and global registration for 3D prostate segmentation in MRI . SPIE Conference on Medical Imaging : Image Processing. Proceedings of the SPIE, Volume 8314, pp. 8314S1-8314S9. San Diego, California. February 2012. • [IEEE SITIS 2010] S. Ghose, A. Oliver, R. Martí, X. Lladó, J. Freixenet, J.C. Vilanova, and F. Meriaudeau. Prostate segmentation with texture enhanced Active Appearance Model. IEEE International Conference on Signal-Image Technology and Internet-Based Systems, pp. 18-22. Kuala Lumpur, Malaysia. December 2010. Publications related to the survey of prostate segmentation algorithms: • [CMPB]S. Ghose, A. Oliver, R. Martí, X. Lladó, J. Freixenet, J. Mitra, J. C. Vilanova, D. Sidibé, F. Meriaudeau. A Survey of Prostate Segmentation Methodologies in Ultrasound, Magnetic Resonance and Computed Tomography Images. Computer Methods and Programs in Biomedicine (accepted).Other publications related to PROSCAN project:• [MIA] J.Mitra, Z. Kato, R. Martí, A. Oliver, X. Lladó, D. Sidibé, S. Ghose, J.C. Vilanova, J. Comet, and F. Meriaudeau. A spline-based non-linear diffeomorphism for multimodal prostate registration. Medical Image Analysis (accepted). • [IJCARS 2012] J. Mitra, R. Martí, A. Oliver, X. Lladó, S. Ghose, J.C. Vilanova, and F. Meriaudeau. Prostate multimodality image registration based on b-splines and quadrature local energy. International Journal of Computer Assisted Radiology and Surgery, Volume 7, Issue 3, pp 445-454, Heidelberg, Germany, Springer-Verlag, 2012. • [IAPR ICPR 2012] J. Mitra, Z. Kato, S. Ghose, D. Sidibé, R. Martí, X. Lladó, A. Oliver, J.C. Vilanova, and F. Meriaudeau. Spectral clustering to model deformations for fast multimodal prostate registration. IAPR International Conference on Pattern Recognition, (accepted), Tsukuba, Japan. August 2012. • [IEEE EMBC 2012] J. Mitra, S. Ghose, D. Sidibé, R. Martí, A. Oliver, X. Lladó, J.C. Vilanova, J. Comet, and F. Meriaudeau. Joint Probability of Shape and Image Similarities to Retrieve 2D TRUS-MR Slice Correspondence for Prostate Biopsy. IEEE Conference of the Engineering in Medicine and Biology Society, (accepted), San Diego, California. August 2012.

• [IAPR ICPR 2012] S. Ghose, J. Mitra, A. Oliver, R. Martí, X. Lladó, J.

Freixenet, J.C. Vilanova, J. Comet, D. Sidibé and F. Meriaudeau. A Mumford-F.

(a) (b) (c) (d) (e)

(a) (b) (c) (d)

(a) (b) (c) (d) (e)(f)

Acknowledgments • [IEEE ICIP 2012] J. Mitra, S. Ghose, D. Sidibé, A. Oliver, R. Martí, X. Lladó, J. C. Vilanova, J. Comet, F. Meriaudeau. Weighted Likelihood Function of Multiple Statistical Parameters to Retrieve 2D TRUS-MR Slice Correspondence for Prostate Biopsy. International Conference on Image Processing (ICIP), to appear, San Diego, USA, Sep-Oct, 2012. • [SPIE Medical Imaging 2012] J. Mitra, A. Srikantha, D. Sidibé, R. Martí, A. Oliver, X. Lladó, S. Ghose, J.C. Vilanova, J. Comet, and F. Meriaudeau. A shape-based statistical method to retrieve 2D TRUS-MR slice correspondence for prostate biopsy. SPIE Conference on Medical Imaging . SPIE Conference on Medical Imaging : Image Processing. Proceedings of the SPIE, Volume 8314, pp. 8314M1-8314M9. San Diego, California. February 2012. • [IEEE DICTA 2011] J. Mitra, Z. Kato, R. Martí, A. Oliver, X. Lladó, S. Ghose, J.C. Vilanova, and F. Meriaudeau. A non-linear diffeomorphic framework for prostate multimodal registration. IEEE International Conference on Digital Image Computing: Techniques and Applications, pp 31-36. Noosa, Australia. December 2011.

Chapter 4

Graph Cut Optimization in a Stochastic Framework for Prostate Segmentation in MRI Variations in inter-patient prostate shape, and size and imaging artifacts in magnetic resonance images (MRI) hinders automatic accurate prostate segmentation. In this chapter we propose a graph cut based energy minimization of the posterior probabilities for automatic 3D segmentation of the prostate in MRI. A probabilistic classification of the prostate voxels is achieved with a probabilistic atlas and a random forest based learning framework. The posterior probabilities are combined to obtain the likelihood of a voxel being prostate. Finally, 3D graph cut in the stochastic space provides segmentation of the prostate.

Introduction

Prostate segmentation in MRI facilitates volume estimation, multi-modal image registration, surgical planing and image guided prostate biopsies. Accuracies and speed of multimodal image registration could be significantly improved by performing the image registration on segmented prostates [START_REF] Yan | Discrete Deformable Model Guided by Partial Active Shape Model for TRUS Image Segmentation[END_REF]. In our PROSCAN project, as discussed in Chapter and echo time of 107.7 ms were acquired with 1.5 Tesla machines. The prostate images used for the experiment have an average size of 256×256 pixels with a pixel dimension being 0.2734 mm. Manual segmentations performed by an expert were validated by another expert to prepare the ground truth.

Proposed segmentation framework

The proposed method is developed on three major components: 1) Probabilistic atlas based segmentation, 2) Random forest based probabilistic classification of the voxels being prostate, and 3) Graph cut based energy minimization of the combined probabilities. The schema of our proposed method is illustrated in Fig. 4.1.

segmentation of the prostate. The initial soft segmentation was minimized in a graph-cut based energy minimization framework to achieve prostate segmentation. The new stochastic approach to prostate segmentation improved on segmentation accuracies compared to some of the work in literature.

Contributions

To summarize the major contributions of this thesis are:

• An extensive survey of prostate segmentation methods in TRUS and MRI is carried out. A new taxonomy for prostate segmentation strategies depending on theoretic approach to segmentation is proposed and advantages and drawbacks of each strategy is pointed out. A discussion on choosing the most appropriate segmentation strategy for a given imaging modality is provided along with the quantitative results as reported in literature. We conclude from our literature review that shape and appearance priors in a prostate segmentation algorithm improves segmentation accuracies.

• A new statistical shape and texture enhanced appearance model is proposed for prostate segmentation in TRUS images. Use of texture features extracted with Haar wavelets and quadrature filters improves segmentation accuracies compared to traditional shape and appearance model in leave-one-patient-out validation framework.

• A new probability theoretic approach to model prostate images in TRUS is proposed. Posterior probabilities determined in a supervised learning framework facilitates automatic initialization and evolution of the curve. Multiple mean models automatically determined from spectral clustering of shape and probability priors significantly improves prostate segmentation accuracies in TRUS images in leave-one-patient-out validation framework.

• A new shape an appearance prior variational model is proposed. Mumford-Shah energy minimization in stochastic domain improves on prostate segmentation accuracies in TRUS images in leave-one-patient-out validation.