J Alison Noble

Senior Member, IEEE Djamal Boukerroui

Ultrasound Image Segmentation: A Survey

Keywords: Segmentation, ultrasound, B-scan, review

This article reviews ultrasound segmentation methods, in a broad sense, focusing on techniques developed for medical B-mode ultrasound images. First, we present a review of articles by clinical application to highlight the approaches that have been investigated and degree of validation that has been done in different clinical domains. Then, we present a classification of methodology in terms of use of prior information. We conclude by selecting 10 papers which have presented original ideas that have demonstrated particular clinical usefulness or potential specific to the ultrasound segmentation problem.

I. INTRODUCTION

Ultrasound image segmentation is strongly influenced by the quality of data. There are characteristic artefacts which make the segmentation task complicated such as attenuation, speckle, shadows and signal dropout; due to the orientation dependence of acquisition that can result in missing boundaries. Further complications arise as the contrast between areas of interest is often low. However, there have been recent advances in transducer design, spatial/temporal resolution, digital systems, portability etc that mean that the quality of information from an ultrasound (US) device has significantly improved [START_REF] Bridal | Milestones on the road to higher resolution, quantitative, and functional ultrasonic imaging[END_REF]. This has led to increased use of ultrasound in not only its traditional area of application, diagnosis (and computer-aided detection (CAD)), but also emerging areas such as image-guided interventions, and therapy. Thus, there is currently a re-emergence of interest in understanding how to do one of the oldest image processing tasks, image segmentation, applied to ultrasound data. As this review attempts to demonstrate, while in other areas of medical imaging (notably X-ray Computed Tomography (CT) and Magnetic Resonance Imaging (MRI)), application of general image processing methods can suffice, in the case of ultrasound data, customized methods, which attempt to model the imaging physics in some way have proven to be more successful. This article focuses on techniques developed for native (conventional) B-mode ultrasound images and not segmentation of radio-frequency (RF) signal or Doppler images. Both 2D and 3D segmentation are considered, although the literature on 2D methodology is far more extensive, reflecting the still relatively low (but increasing) use of 3D image acquisition technology in clinical practice today.

In practice, one finds that most new segmentation methodology is reported by placing it in context of related literature Manuscript received August 22, 2005 for a chosen clinical application and the few US segmentation reviews have been reported from this clinical domain perspective e.g. [START_REF] Hammoude | Endocardial border identification in two-dimensional echocardiographic images: review of methods[END_REF]- [START_REF] Zimmer | Image segmentation in obstetrics and gynecology[END_REF]. To our knowledge this is the first comprehensive review of ultrasound segmentation methodology in a broad-sense. This points to some common methodology issues across different clinical applications in terms of prior modeling. There are a larger number of papers which describe general segmentation methods which show application of a method to one or two ultrasound images or sequences without specific reference to ultrasound image formation or context. We have not included these as they are much more difficult to identify. We have also not considered the segmentation of ultrasound parametric images such as elasticity images, and functional ultrasound contrast image sequences. Finally, we recognize that in the future more work is likely to be done in segmentation based on the RF signal but access is not widely available so we have not considered this here. See [START_REF] Dydenko | Towards ultrasound cardiac image segmentation based on the radiofrequency signal[END_REF], [START_REF] Husby | Estimating blood vessel areas in ultrasound images using a deformable template model[END_REF] for recent segmentation papers based on this.

The outline of the paper is as follows. In Section II we review principal works on ultrasound image segmentation classified by clinical application as most researchers are interested in a segmentation solution for a given clinical problem. This section focuses on application areas in which the majority of effort has been focussed, namely segmentation of echocardiography, breast ultrasound, transrectal ultrasound (TRUS), intravascular ultrasound (IVUS) images, and ultrasound images acquired in obstetrics and gynaecology. Examples from representative methods of the first four are respectively shown in figures 1-4. In Section III we summarize performance assessment and clinical validation that has been done in these areas to-date. Section IV will be of particular interest to researchers working on new segmentation algorithm development. Here we focus on reviewing collectively first papers which have been motivated by and incorporate knowledge of ultrasound physics (particularly speckle models) and then articles that have developed segmentation solutions that used other prior information (intensity, shape and temporal models) We conclude in Section V by listing 10 papers which have made contributions specific to ultrasound segmentation in terms of novel ideas and/or validation and may be useful as an introduction to those new to the field.

II. CLASSIFICATION BY CLINICAL APPLICATION

In this section we have included papers which have undergone a reasonable clinical validation as well as papers which may have been less-well validated but have particular features which make them well-suited to the specific clinical application domain.

A. Cardiology

Echocardiography (ultrasound imaging of the heart) has been one of the driving application areas of medical ultrasound and the literature on methods for automatically segmenting and tracking the left ventricle (the normal chamber assessed for heart disease) is extensive. Most attention has been given to tracking the motion of the endocardium (blood pool/tissue border) to allow for estimation of left ventricular areas/volumes and derived measures such as the ejection fraction 1 , and for regional wall motion assessment. In particular, these measures are used in diagnosis and assessment of ischaemic heart disease. Most analysis is based on 2D acquisitions in which it is implicitly assumed that the principal component of motion is in the plane of the acquisition slice. The standard 2D diagnostic views used for this are called the parasternal short-axis (SAX), and apical four-chamber (4C), two-chamber (2C) and three-chamber (3C) views. The latter 3 are also sometimes referred to as (apical) long-axis (LAX) views. The quality of data, and hence challenges for segmentation, vary depending on the view due to the anisotropy of ultrasound image acquisition, artefacts such as shadowing from the lungs, and attenuation which can be strong. Segmentation methods should also have strategies for avoiding the papillary muscles. Reliably finding the outer wall (often called epicardial border detection) is much more challenging, particularly from apical views. There are many papers on left ventricle tracking which deal with the tracking/deformation model independent of feature extraction or the imaging modality. We have not discussed these papers. See the review by Frangi et al. [START_REF] Frangi | Three-dimensional modeling for functional analysis of cardiac images: A review[END_REF] for a discussion of this.

3D echocardiography is a relatively new imaging modality and image analysis is not yet well-developed. Early work used either freehand ultrasound (with 2D image acquisition synchronized with recording the location of the slice with a position sensor), rotational 3D probes which acquired a sparse set of 2D image sequences, or real-time 3D echocardiography based on the Volumetrics system [START_REF] Von Ramm | Real time volumetric ultrasound imaging system[END_REF], [START_REF] Binder | Three-dimensional echocardiography -principles and promises[END_REF]. However, a second generation of real-time 3D echocardiography systems have recently been produced by Philips Medical Systems. This data is of higher quality than the rotational 3D probe and Volumetrics systems [START_REF] Franke | Second-generation real-time 3D echocardiography: A revolutionary new technology[END_REF]. One therefore has to take particular care in interpreting the results from early 3D work as being representative of what can be achieved today.

This review focuses on endocardial border detection unless otherwise stated. Much of the early work focused on still (single) frame segmentation -simulating what a cardiologist often does which is look at end-diastole (maximum expansion) and end-systole (maximum contraction) frames to compute measures such as the ejection fraction 1 . However, to fully assess heart function, analysis needs to be done over the whole cardiac cycle. Cardiologists also use a movie of a heart in decision-making as the speckle pattern associated with deforming tissue can be observed in a movie whereas in a still frame the speckle pattern is not always useful. As a result, and in contrast to some other clinical application areas (see later sections), in echocardiography it is perhaps more logical to view segmentation as a spatio-temporal problem. Examples of echocardiographic image segmentation: Short axis images from [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF] [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF] (a) [START_REF] Mikic | Segmentation and tracking in echocardiographic sequences: Active contours guided by optical flow estimates[END_REF] [START_REF] Mikic | Segmentation and tracking in echocardiographic sequences: Active contours guided by optical flow estimates[END_REF] (b); long axis images from [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF] [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF] (c).

More recent successful approaches have taken this viewpoint. An early review of endocardial border detection is presented in [START_REF] Hammoude | Endocardial border identification in two-dimensional echocardiographic images: review of methods[END_REF]. However, in the intervening 7 years or so there has been extensive research and progress in this area. Examples of segmentation results are illustrated in Fig. 1.

1) 2D endocardial border detection: The most popular approach has been to treat echocardiographic endocardial segmentation as a contour finding approach. This is not straightforward as the contrast around the border of the left ventricle chamber varies depending on the relative orientation of the border to the transducer direction, and attenuation. Thus conventional intensity gradient-based methods have had limited success on typical clinical images.

Mishra et al. [START_REF] Mishra | A GA based approach for boundary detection of left ventricle with echocardiographic image sequences[END_REF] proposed an active contour solution where the optimization was performed using a genetic algorithm. In a first image, low pass filtering and morphological operations were used to define an initial estimate of the contour. A nonlinear mapping of the intensity gradient was used in the energy functional which is minimized. The final contour was used to initialize contour finding in the next time frame. The convergence of the method was compared to solving the active contour using the conventional constrained quasi-Newton method. Manual delineations were done on 20 frames by 2 experts and the average compared to the automated algorithm to show that the inter-variability between experts was similar to the difference between the manual and automated methods. The area correlation was found to be 0.92. This was a preliminary evaluation from which strong conclusions cannot be drawn.

Mignotte and Meunier [START_REF] Mignotte | A multiscale optimization approach for the dynamic contour-based boundary detection issue[END_REF] chose to use a statistical external energy in a discrete active contour for the segmentation of short axis parasternal images, arguing that this was well-suited in ultrasound images with significant noise and missing boundaries. To this end, a shifted Rayleigh distribution was used to model gray levels statistics. The multiscale optimization strategy of Heitz et al. [START_REF] Heitz | Multiscale minimization of global energy functions in some visual recovery problems[END_REF] was adapted to perform the energy minimization. The same optimization strategy was used for a maximum-likelihood (ML) region segmentation to extract a crude endocardial initial contour for the snake algorithm. Several segmentation results were shown and qualitatively good results were obtained.

Mignotte et al. [START_REF] Mignotte | Endocardial boundary estimation and tracking in echocardiographic images using deformable template and markov random fields[END_REF] proposed a boundary estimation algorithm, posed in a Bayesian framework where the prior information was modeled using deformable templates. This was a fully automatic unsupervised (i.e no learning) approach. The estimation problem was formulated as a maximum a posteriori (MAP) optimization solved by means of a Genetic Algorithm. As in their previous work, [START_REF] Mignotte | A multiscale optimization approach for the dynamic contour-based boundary detection issue[END_REF], the likelihood term relied on a shifted Rayleigh statistical model of the gray levels distribution. The template was defined by a circle along with a set of admissible affine global, non-affine global and local transformations. Illustrative results on simulated data and on a series of short axis images (50 frames) were shown. A comparison to manual delineations by two experts was also presented. The region-based segmentation method of Boukerroui et al. [START_REF] Boukerroui | Segmentation of ultrasound images -multiresolution 2D and 3D algorithm based on global and local statistics[END_REF] follows a similar (Bayesian) methodological approach.

Level sets are often considered as an alternative to active contours and this approach has also been considered for echocardiographic image segmentation. Yan et al. [START_REF] Yan | Applying improved fast marching method to endocardial boundary detection in echocardiographic images[END_REF] considered applying the level set method to echocardiographic images using an adaptation of the fast marching method. To reduce errors attributed to using local feature (intensity gradient) measurements, they used an average intensity gradientbased measure in the speed term. The method was applied to a parasternal short axis and an apical 4-chamber view sequence but the results were only discussed qualitatively. The latter example highlighted problems in using a purely edge-based level set method. Lin et al. [START_REF] Lin | Combinative multi-scale level set framework for echocardiographic image segmentation[END_REF] presented an interesting variant of the level set segmentation idea which combines edge and region information in a level set approach across spatial scales (a pyramid). This method assumes that a boundary is a closed curve. It was applied to rotational 3D echocardiography data although the segmentation method was applied to individual 2D slices. On 24 apical images, 3 experts manually delineated the borders of the left ventricle. Using the mean average difference between contours as the metric they showed that the automatic method was within the variability of the manual delineations. Their method was said to work well when the quality of data was reasonably good but the method depends on being able to extract a contour at a high pyramid level.

Extending their previous work on using prior shapes in a variational framework [START_REF] Chen | Using prior shapes in geometric active contours in a variational framework[END_REF], Chen et al. [START_REF] Chen | A coupled minimization problem for medical image segmentation with priors[END_REF], [START_REF]Using prior shape and intensity profile in medical image segmentation[END_REF] used prior shape and intensity profiles to constrain the evolution of a geometric active contour. The problem was formulated as a coupled optimization problem where the prior intensity profiles were used to assist the estimation of the balance between the image information and the shape priors. The intensity profiles were model free as they were based on the maximization of a mutual information based criterion. Illustrative results were presented on long axis echocardiographic images.

Klinger et al. presented a fairly classic approach to segmenting short-axis echocardiographic images using techniques based on mathematical morphology [START_REF] Klingler | Segmentation of echocardiographic images using mathematical morphology[END_REF]. Data was time averaged (N = 10) to give a time-averaged cycle with reduced noise and then a series of morphological operators applied to identify the left ventricle cavity from which the border of the cavity could be found. This was tested on 7 canine datasets where ischaemia had been induced in the animals to change heart motion. The area and shape correlation between the algorithm and an observer had r > 0.93. This is a relatively early paper and highlights issues at this time in terms of obtaining data (even on animals) of good enough quality to perform automated analysis.

Segmentation of the left ventricle can also be considered a region-based or pixel-classification problem. In this case the goal is to label pixels as either belonging to the blood pool or myocardium.

Boukerroui et al. [START_REF] Boukerroui | Segmentation of ultrasound images -multiresolution 2D and 3D algorithm based on global and local statistics[END_REF], working in a Bayesian framework, presented a robust adaptive region segmentation. The adaptive property considers local class mean with a slow spatial variation, to compensate for the non-uniformity of ultrasound echo signals within the same tissue. The authors also used a multiple resolution implementation, as suggested by Ashton and Parker [START_REF] Ashton | Multiple resolution bayesian segmentation of ultrasound images[END_REF], to justify the Gaussian probability density function (pdf) approximation at the lower resolution. The method was evaluated against manual delineation by an observer on a long-axis sequence and compared with another region-based segmentation method due to Xiao et al. [START_REF] Xiao | Segmentation of ultrasound B-mode images with intensity inhomogeneity correction[END_REF]. Qualitative results were also shown on breast ultrasound data.

Artificial neural network (ANN) based methods have been used for region-based segmentation [START_REF] Rekeczky | CNN-based spatiotemporal nonlinear filtering and endocardial boundary detection in echocardiography[END_REF], [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF]. For instance, Binder et al. [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF] employed a 2-layer backpropagation network that was trained by manually identifying 369 sample regions (7 × 7 pixels) from 8 training images to segment the image as tissue (n = 279) or blood pool (n = 90). Parameters (gray level mean and variance, contrast, entropy and homogeneity from the co-occurrence matrices) were used in the ANN. Endocardial borders were found by first performing a radial search for candidate border points in single frames and then using a spatio-temporal contour linking step. The method was applied to end-diastolic and end-systolic parasternal short axis (SAX) images from 38 patients with the data being of variable quality (12 good, 13 moderate and 13 poor). This is one of the few studies that has explicitly looked at data of varying quality. Segmentation was successful in 34 of the 38 datasets. The automated method was compared with manual tracing by 2 experts. The ANN showed good correlation (r = 0.99) but from Bland-Altman analysis tended to overestimate the cavity areas. Technetium radionuclide ventriculography (Tc-RNV) was performed on 12 patients to enable the ejection fraction to be compared. The automated system showed a correlation of r = 0.93 with Bland-Altman analysis showing that the ANN overestimated the ejection fraction. Over all the analyses, the best correlations were with the best data.

2) Spatio-temporal methods (2D + Time): It is natural to view echocardiographic segmentation as a spatio-temporal problem, since the object of interest moves (in a non-rigid way) and speckle decorrelates over the sequence. Spatiotemporal analysis potentially leads to better localization of a border, and gives a local estimate of image velocity as a side product of the segmentation. For instance, Friedland et al. [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF] modeled the temporal continuity by means of a Markov Random Field (MRF). This is, to our knowledge, pioneering work that defined boundary detection as a minimization of an energy function with a Markovian spatio-temporal regulariza-tion. Polar coordinates were used for the contour definition and the Simulated Annealing (SA) algorithm was used for the optimization. Herlin and Ayache [START_REF] Herlin | Features extraction and analysis-methods for sequences of ultrasound images[END_REF] used a spatiotemporal version of the Deriche edge detector in a method for tracking cardiac boundaries but their approach was not fully developed or validated in clinical practice. Mulet-Parada and Noble [START_REF] Mulet-Parada | 2D+T acoustic boundary detection in echocardiography[END_REF] proposed a local-phase-based approach to spatiotemporal endocardial border detection. They argue that local phase is a better basis for ultrasound-based feature detection and segmentation because local phase is theoretically invariant to intensity magnitude. They proposed using a derived measure from phase, called feature asymmetry for detecting the endocardium in a spatio-temporal space (2D+T). They used the output of this feature detector (location, velocity and confidence in measurement) as the feature vector in a motion tracking algorithm that was validated against manual delineations by a cardiologist. Local-phase has since been used in preference to intensity gradient for feature detection in a number of echocardiography studies [START_REF] Mulet-Parada | Intensity independent feature extraction and tracking in echocardiographic sequences[END_REF]- [START_REF] Sanchez-Ortiz | Automated 3D echocardiography analysis compared with manual delineations and SPECT MUGA[END_REF].

A number of authors have used variations of the active contour concept to segment and track left ventricle boundaries.

Chalana et al. developed a multiple active contour method to detect both the endocardium and epicardium in shortaxis views [START_REF] Chalana | A multiple active contour model for cardiac boundary detection on echocardiographic sequences[END_REF]. They defined the left ventricle by an active surface model, which they reparameterized to show that the surface could be represented by two planar curves representing the endocardial and epicardial borders. They used the image intensity gradient as the attracting force and invoked temporal continuity via an external energy term that constrained the motion between consecutive frames. Their method was validated on 44 clinical datasets against manual delineations by 4 experts. The area correlation coefficient between their method and the average manual outline was 0.95 for the epicardium and 0.91 for the endocardium. They reported that the computer algorithm gives a lower boundary difference error for the epicardium than the endocardium. This may be due to the fact that their algorithm was initialized with a manual delineation of the epicardium.

Kucera et al. [START_REF] Kucera | Segmentation of sequences of echocardiographic images using a simplified 3D active contour model with region-based external forces[END_REF] presented an example of applying 3D active contours to 2D+T echocardiographic data using a regionbased external force. This is a relatively early work that used region information in a contour based segmentation approach. It can be compared to more recent level set based segmentation methods such as [START_REF] Lin | Combinative multi-scale level set framework for echocardiographic image segmentation[END_REF], [START_REF] Sarti | Maximum likelihood segmentation of ultrasound images with rayleigh distribution[END_REF].

Mikic et al. proposed an active contour approach where propagation of a fitted contour from one frame to another was guided by optical flow estimates [START_REF] Mikic | Segmentation and tracking in echocardiographic sequences: Active contours guided by optical flow estimates[END_REF] provided by the method of Singh and Allen [START_REF] Singh | Image-flow computation: An estimationtheoretic framework and a unified perspective[END_REF] (See also [START_REF] Boukerroui | Velocity estimation in ultrasound images: a block-matching approach[END_REF]). This can be viewed as providing knowledge to guide the initialization of an active contour optimization. The image force in the active contour was defined in terms of a Gaussian smoothed intensity gradient. Parameters in the active contour were determined empirically. The method was evaluated on 8 image sequences (3 short axis, 3 long-axis and two aortic root) and compared with 3-5 manual delineations. For endocardial border segmentation, the area correlation coefficient between their method and the average area was 0.99, 0.95 and 0.99 for three short axis sequences and the distance between the active contour and manual outlines was 1.15, 1.29 and 2.07 mm respectively (see Fig. 1(a) for an example result taken from their paper).

Setarehdan et al. developed a fuzzy multi-scale edge detection (FMED) method for endocardial and epicardial border detection [START_REF] Setarehdan | Automatic cardiac LV boundary detection and tracking using hybrid fuzzy temporal and fuzzy multiscale edge detection[END_REF] that uses a wavelet transform to define the various levels of resolution of image content. In this work an edge was defined as a point with maximal membership of the edge fuzzy set. Temporal information was included in edge detection by defining a moving edge fuzzy membership function as well as an edge fuzzy membership function. The combined edge and motion membership function was then maximized to define edge points. This method was applied to SAX images where a radial-search-based strategy was used and the edge detection applied in 1D along rays from an automatically found center with some pre-processing restricting the search for the endocardial and endocardial borders. The method was evaluated on simulated data and 14 datasets from healthy subjects compared to delineations by a single expert. The correlation between the areas defined by the algorithm and the expert for the endocardial border was 0.96 and for the epicardial border was 0.95. The rms radial (sampled) distance between the algorithm and expert defined boundaries was found to be 1.84 mm (epicardium) and 3.5 mm (endocardium).

Dias and Leitão [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF], proposed an algorithm for the joint endocardial and epicardial border estimation following the earlier work of Friedland and Adam [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF] and Figueiredo and Leitão [START_REF] Figueiredo | Bayesian estimation of ventricular contours in angiographic images[END_REF]. As in [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF], [START_REF] Figueiredo | Bayesian estimation of ventricular contours in angiographic images[END_REF] polar coordinates were used for the contour representation. However to our knowledge, this is the first approach which takes into account speckle statistics in boundary optimization. The estimation problem was modeled in a Bayesian framework as a MAP estimation problem with Rayleigh statistics for the data term and a spatiotemporal MRF for the regularization term. An iterative multigrid dynamic programming algorithm was used to solve the optimization problem. Results are given on simulated data and on one short axis sequence (half of a cardiac cycle). See Fig. 1(a) for an example result taken from their paper. Recently Figueiredo et al. [START_REF] Figueiredo | Unsupervised contour representation and estimation using B-splines and a minimum description length criterion[END_REF], within the same framework, presented an unsupervised parametric contour segmentation based on B-spline representations. A Minimum Description Length (MDL) criterion was used to estimate the number of B-spline knots. In contrast to [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF], where careful parameter tuning of the prior terms was required, this leads to an approach less dependent on parameter settings as the tradeoff between smoothness/robustness is optimal in the sense of MDL. Illustrative segmentation examples on a number of medical applications, including cardiac short-axis and intravascular ultrasound images were shown.

Jacob et al. [START_REF] Jacob | Evaluating a robust contour tracker on echocardiographic sequences[END_REF]- [START_REF] Jacob | A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography[END_REF] developed a Kalman-filter based endocardial and epicardial border tracking method based on the dynamic contour tracking approach of Blake and Isard [START_REF] Blake | Active Contours[END_REF]. In this approach a spatio-temporal contour model is constructed which has two parts; a shape-model constructed from manual delineations of the LV borders of a training set; and a motion model. An image measurement model is also defined (for instance in [START_REF] Jacob | Evaluating a robust contour tracker on echocardiographic sequences[END_REF] a 1D local-phase based feature detector is used for endocardial border detection, and in [START_REF] Jacob | A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography[END_REF] a wavelet-based method is used for epicardial border detection). A Kalman filter strategy is then used to combine the contour model with the image measurements. The approach was evaluated in a small case study on stress echocardiography data (4 datasets: one 4C and three 2C) and the performance of the tracking and interpretation compared with manual border delineations using 6 quantitative geometric measures [START_REF] Jacob | A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography[END_REF]. The area correlation coefficient was found to be 0.98 (n = 27). Bland-Altman analysis on endocardial/epicardial areas showed that endocardial border estimation was more accurate than epicardial border detection although the authors comment that strong conclusions can not be drawn from this as manual delineation is difficult to do. In [START_REF] Jacob | Quantitative regional analysis of myocardial wall motion[END_REF] Jacob et al. attempt to perform an evaluation against clinical scoring of border motion and myocardial thickening by cardiologists (n = 4 stress datasets (as used in [START_REF] Jacob | A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography[END_REF], 2 clinical observers)). This is one of the few papers that has considered stress-echocardiography data, but as noted by the authors, only a limited number of datasets were analyzed, and the data quality, though typical of that seen in clinical practice was quite low.

Bosch et al. [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF] used an adaption of the Active Appearance Model (AAM) approach that they called the active appearance motion model (AAMM) to represent the shape and appearance of the endocardium as well as its motion (see Fig. 1(c) for an example result taken from their paper). An important step in their method was nonlinear intensity normalization. Their method was tested on 129 4-chamber datasets (n = 65 for training, n = 64 for testing). An expert traced the borders of the images. Manual delineations were compared with the algorithm contours with an average distance error of 3.3 mm comparable with inter/intra observer variability for 97% of the data.

Mitchell et al. [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF] developed a 3-D AAM for segmenting 2D+T 4-chamber echocardiography data (2D sequences with the third dimension time). The approach is spatio-temporal i.e. adjusts itself in space and time. The model was learned from manually segmented examples in a training stage. The method was evaluated on the same data as used in [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF]. The method was deemed successful in 57 of the 64 datasets. The endocardial average distance error was 3.9 mm which is slightly worse than the method of Bosch et al. [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF] which they explain as being due in part to the extra degree of freedom but also to the fact that the 3-D method did not employ intensity normalization.

3) 3D and 4D endocardial border detection: In relatively early work on 3D endocardial border detection, Coppini et al. [START_REF] Coppini | Recovery of the 3D shape of the left ventricle from echocardiographic images[END_REF] considered segmentation and reconstruction of the left ventricle from freehand ultrasound data where data was acquired at 4 fixed angles with respect to the 4-chamber view. Boundaries were found in each 2D slice as the zero-crossings of a Laplacian of a Gaussian edge operator. Edge points were then linked to form contours. A neural network was then used to classify edge segments as part of a boundary or not (training was done on data from 4 subjects, with testing on 3 others). An elastic surface model was then fit to the edges. Preliminary assessment of the different processing stages was given.

Song et al. treated the segmentation problem as a surface fitting problem posed in a Bayesian framework where the goal was to find a 3D surface which has the greatest pos-terior probability given the images [START_REF] Song | Integrated surface model optimization for freehand three-dimensional echocardiography[END_REF]. They argued that by omitting an explicit feature detection step their method is less sensitive to low-quality images. They tested their method using 45 data sets from normal and diseased hearts; 20 of the datasets were used to train the model (determine pixel appearance and pixel class prediction models). The average epicardial and endocardial surface projection distance errors for end-diastolic frames were found to be 3.2 mm and 2.6 mm respectively. This is interesting but preliminary work that to our knowledge has not be pursued further.

Wolf et al. [START_REF] Wolf | ROPES: A semiautomated segmentation method for accelerated analysis of threedimensional echocardiographic data[END_REF] described a semi-automatic method of segmentation called the restricted optimal path exploring segmentation (ROPES) approach which was applied to 2D slices of 3D transesophaegal echocardiography (TEE) data. It is based on the idea of first finding candidate edge points which satisfy a multiscale criterion and minimizing a cost function which forms line segments and then closed contours. Comparison is made with manual delineations where a mean distance in-plane error of 3.44 mm was reported (c.f. 2.92 mm interobserver distance).

Angelini et al. [START_REF] Angelini | LV volume quantification via spatiotemporal analysis of real-time 3-D echocardiography[END_REF] considered segmentation of real-time 3D echocardiography data from a Volumetrics system. First, the 3D+T data was pre-processed with a wavelet analysis (using brushlets) to reduce speckle. Then a 2D balloon deformable model, using an intensity gradient based force, was fitted to each slice and the volume derived from this. The method was tested on 6 clinical datasets with comparison made with MR manual segmentation on end-diastole and end-systole frames. They referenced a paper by Takuma et al. [START_REF] Takuma | Assessment of left ventricular function by real-time 3-D echocardiography compared with conventional nonivasive methods[END_REF] which claims an inter-observer variability of 8.3% and intra-observer variability of 3.7% for volume estimation with disk summation on real-time 3D (RT3D) echocardiography data. Their method shows an 8.9% error compared to manual delineation on MR data whereas manual delineation on RT3D versus manual delineation on MR showed a much higher average value. This paper, as with our own experience, shows that one has to take care in using manual delineation on ultrasound data as a reference.

Montagnat et al. [START_REF] Montagnat | Anisotropic filtering for model-based segmentation of 4D cylindrical echocardiographic images[END_REF] developed a 3D+T model-based segmentation method for rotational 3D echocardiography. This work is most interesting for its use of a cylindrical geometry for analysis, and using a combined edge and regionbased approach. 4D anisotropic diffusion filtering was first applied to the data to reduce speckle (no distinction was made between temporal and spatial dimensions). This method assumes boundaries are located at high intensity gradients and the parameter in the diffusion process is a function of the intensity gradient magnitude. Thus this work does not explicitly account for imaging physics. Then a deformable model (discrete simplex mesh) is fit to the filtered data where the external forces are provided by a 3D gradient operator defined in cylindrical co-ordinates (the 3D equivalent of the work of Herlin and Ayache [START_REF] Herlin | Features extraction and analysis-methods for sequences of ultrasound images[END_REF]). Specifically the intensity gradient magnitude and orientation are used, together with a region-based force based on a local analysis of intensity values. Here it is perhaps surprising that the anisotropic diffusion was also not done in the natural frame of reference of acquisition. Qualitative results are presented.

Although strictly speaking filtering methods, the next 2 papers are theoretically elegant approaches which have shown promising early results as ultrasound segmentation methods. The solutions are posed in a variational framework and have properties that make them well-suited to 3D echocardiography segmentation. Sarti et al. [START_REF] Sarti | Nonlinear multiscale analysis of 3D echocardiographic sequences[END_REF] developed a model for multiple scale analysis of space-time echocardiographic sequences based on solving a nonlinear partial differential equation combining ideas of the Perona-Malik anisotropic diffusion model with the Galilean invariant movie multiscale analysis of Guichard [START_REF] Guichard | A morphological, affine, and galilean invariant scalespace for movies[END_REF], [START_REF]Axiomatisation des analyses multi-échelles d'images et de films[END_REF]. Coherent spatio-temporal structures are preserved by the filtering operation. Their method was applied to a single example of rotational 3D echocardiographic data. In an extension [START_REF] Mikula | On anisotropic geometric diffusion in 3D image processing and image sequence analysis[END_REF], the Perona-Malik spatial diffusion was replaced with a curvature driven level-sets non-linear anisotropic smoothing [START_REF] Preußer | A level set method for anisotropic geometric diffusion in 3D image processing[END_REF]. That model was invariant under gray value transformations and, unlike previous curvature evolution models, it well-preserved edges and corners on level-sets while still allowing tangential smoothing along edges. In later work, Mikula et al. presented a truly coupled spatio-temporal anisotropic diffusion [START_REF] Mikula | Morphological image sequence processing[END_REF]. In that work time smoothing was curvature driven, and the authors argue, resulted in better smoothing with velocity discontinuity preservation in contrast to the previous acceleration based smoothing. Note that, in contrast to the 4D diffusion model used in [START_REF] Montagnat | Anisotropic filtering for model-based segmentation of 4D cylindrical echocardiographic images[END_REF], the models of [START_REF] Sarti | Nonlinear multiscale analysis of 3D echocardiographic sequences[END_REF], [START_REF] Mikula | Morphological image sequence processing[END_REF], treat the temporal dimension explicitly as time rather than treating space and time as equivalent dimensions.

Corsi et al. [START_REF] Corsi | Left ventricular volume estimation for real-time three-dimensional echocardiography[END_REF] presented a level set based segmentation approach which was applied to real-time 3D echocardiography from a Volumetrics system. The classical inflationary term in the speed function was removed in order the avoid leakage. Hence, close initialization of the level set function was required. Three parameters also had to be chosen. The level set was initialized using manual delineation on a few (short axis) frames. The method was tested in vitro (on balloons of various volumes n = 18) with a correlation coefficient of r = 0.99 and an underestimation of the true volume. On clinical data (n = 20) a correlation of r = 0.97 was found when a comparison was made with MR volumes treated as the reference. A -15.58 ml bias was observed in this case (underestimation by level set method).

Sarti et al. presented a level set segmentation algorithm -the subjective surfaces [START_REF] Sarti | Subjective surfaces: A method for completing missing boundaries[END_REF], [START_REF]Subjective surfaces: A geometric model for boundary completion[END_REF] -for images, such as ultrasound images, which can have missing boundaries. Their approach differs fundamentally from previous level-set segmentation algorithms which concern only the zero level set. In this work, all the level sets, hence the surface, evolve under a suitably chosen speed function. The flow sharpens the surface around the edges and connects segmented boundaries across the missing information. This work was further extended by Mikula et al. [START_REF] Mikula | Handbook of Medical Image Analysis: Segmentation and Registration Models[END_REF] and Corsaro et al. [START_REF] Corsaro | Semi-implicit covolume method in 3D image segmentation[END_REF]. An efficient and robust semi-implicit complementary volume numerical scheme was proposed in order to solve the Riemannian mean curvature flow equation in 2D [START_REF] Mikula | Handbook of Medical Image Analysis: Segmentation and Registration Models[END_REF] and 3D [START_REF] Corsaro | Semi-implicit covolume method in 3D image segmentation[END_REF]. Segmentation examples of the classical Kanizsa triangle were shown to demonstrate that the method can accommodate missing boundaries. Examples on 3D cardiac [START_REF] Mikula | Handbook of Medical Image Analysis: Segmentation and Registration Models[END_REF] and fetal ultrasound [START_REF] Sarti | Subjective surfaces: A method for completing missing boundaries[END_REF] images were also shown. Examples breast ultrasound from Madabhushi and Metaxas (2003) [START_REF] Madabhushi | Combining low-, high-level and empirical domain knowledge for automated segmentation of ultrasonic breast lesions[END_REF]. From left to right: Original image, contrast-enhanced version, detected boundary points and the output of deformable model.

4) Segmentation of the myocardium and epicardium:

Finally there is a very limited literature explicitly looking at segmentation and analysis of the myocardium [START_REF] Kerut | Echocardiographic texture analysis using the wavelet transform: differentiation of early heart muscle disease[END_REF] and epicardial border detection [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF], [START_REF] Feng | Epicardial boundary detection using fuzzy reasoning[END_REF], [START_REF] Malassiotis | Tracking the left ventricle in echocardiographic images by learning heart dynamics[END_REF]. These are both challenging to do from native B-mode images. There is also very little literature specifically focussed on getting methods to work on general clinical data rather than data from subjects with a good acoustic window i.e. which give good images. We have highlighted some of the few works that have done so above [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF], [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF], [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF]. One further exception is the work of Boukerroui et al. [START_REF] Boukerroui | Enhancement of contrast regions in suboptimal ultrasound images with application to echocardiography[END_REF] which considers how to enhance Bmode images to reduce the affect of attenuation and enhance features. Although that method was shown quantitatively to reduce attenuation, enhance features and not introduce artefacts after enhancement, that approach has not to-date been fully tested in clinical practice.

B. Breast cancer

Ultrasound imaging of the breast is typically done as an adjunct to physical examination and (X-ray) mammography when breast cancer is suspected [START_REF] Fornage | Ultrasound of the breast[END_REF]. Conventional B-mode ultrasound does not detect microcalcifications as well as Xray mammography (a key indicator of cancer) but is used to help distinguish benign masses (cysts and fiboradenomas) from malignant cancerous masses. There have been a number of clinical studies that have looked at assessing how well masses can be characterized from a single image by human observers in terms of visually perceived texture and geometric properties [START_REF] Stavos | Solid breast modules: use of sonography to distinguish between benign and malignant lesions[END_REF]- [START_REF] Arger | Interreader variability and predictive value of is descriptions of solid masses: pilot study[END_REF]. This has motivated researchers to look at automating the detection and the characterization of imagederived diagnostic indicators of breast cancer (computer-aided diagnosis or CAD) either from a static image frame or image sequences. Thus the principal segmentation challenges pertain to characterizing the textured appearance and geometry of a cancer relative to normal tissue, and accommodating artefacts such as the possibly strong attenuation across an image and shadowing as well as the 'fuzziness' of cancerous mass boundaries which makes border delineation difficult. Importantly, as we see below, the studies of Stavos et al. [START_REF] Stavos | Solid breast modules: use of sonography to distinguish between benign and malignant lesions[END_REF]- [START_REF] Arger | Interreader variability and predictive value of is descriptions of solid masses: pilot study[END_REF] have greatly influenced the design of algorithms for breast mass detection. Interestingly, no significant work has looked at the screening case, i.e. most work has assumed the presence of a, typically single, suspicious mass.

Horsch et al. [START_REF] Horsch | Automatic segmentation of breast lesions on ultrasound[END_REF] presented a method involving thresholding a pre-processed image that has enhanced mass structures. Comparison is made of a partially automatic and fully au-tomatic version of the method with manual delineation on 400 cases/757 images (124 "complex" cysts, 182 benign masses, 94 malignant masses). They compute four imagebased features (shape, echogeneity, margin and posterior acoustic behavior) defined respectively in terms of the depthto-width ratio, autocorrelation, "normalized radial gradient", and comparison of gray levels, to test their effectiveness at distinguishing malignant and benign masses. This method was further evaluated in [START_REF]Computerized diagnosis of breast lesions on ultrasound[END_REF], [START_REF] Horsch | Performance of computer-aided diagnosis in the interpretation of lesions on breast sonography[END_REF] to assess the advantages of different features using linear discriminant analysis where the best 2 features were found to be the depth-to-width ratio (shape) and normalized radial gradient (margin). In later work aimed at further automating the method Drukker et al. [START_REF] Drukker | Computerized lesion detection on breast ultrasound[END_REF] extended this work to include mass detection by proposing to first filter the images with a radial gradient index filtering technique. The method was tested on the same database as in [START_REF] Horsch | Automatic segmentation of breast lesions on ultrasound[END_REF], [START_REF]Computerized diagnosis of breast lesions on ultrasound[END_REF]. They showed that 75% of lesions were correctly identified. See also Drukker et al. for further validation on a second database [START_REF] Drukker | Computerized detection and classification of cancer on breast ultrasound[END_REF].

Neural Network (NN) based methods have proved to be popular in this area. These aim to make a classification decision based on a set of input features. For instance, Chen et al. [START_REF] Chen | Diagnosis of breast tumors with sonographic texture analysis using wavelet transform and neural networks[END_REF] presented a NN approach where input features were variance contrast, autocorrelation contrast, and the distribution distortion in the (Daubechies) wavelet coefficients and an multilayered perceptron neural (MLP) network with one hidden layer was trained by error backpropagation. The method was applied to a database of 242 cases (161 benign, 81 carcinoma) giving a sensitivity of 98.77% and specificity of 81.77%. They strongly argued that image texture was an important component that made their method successful.

Huang and Chen [START_REF] Huang | Watershed segmentation for breast tumor in 2-D sonography[END_REF] proposed an approach that integrates the advantages of NN classification and a watershedsegmentation methods to extract contours of a breast tumor from US images. The main novelty of this work is in the preprocessing step which helps effectively the watershed algorithm by means of a reasonably good selection of markers. The authors proposed to use a self-organizing map (SOM) texturebased NN in order to select adaptively (ie locally) from a set of nine predefined filters the appropriate preprocessing filer to use. Their method was tested on a database of 60 images (21 benign, 39 carinomas), 40 used for training, 20 for testing. Measures of contour difference and area difference between the method and manual delineation were evaluated although strong conclusions cannot be drawn from this evaluation.

The above methods treat the segmentation approach as a general image processing problem i.e. do not take into consideration that the image is an ultrasound image, and do not explicitly consider speckle or imaging artefacts. There are only a few examples of efforts that have attempted to use domain knowledge in breast ultrasound image segmentation.

Xiao et al. [START_REF] Xiao | Segmentation of ultrasound B-mode images with intensity inhomogeneity correction[END_REF] presented an Expectation-Maximization method that simultaneously estimates the attenuation field at the same time as classification of regions into different (intensity-based) regions. The number of regions (classes) needs to be specified, which in the intended application is not a strong limitation. That method was tested on experimental data with different Time Gain Compensation (TGC) settings to show that their approach gave consistent segmentations under different TGC settings but has not undergone a large clinical assessment. This method is compared to that of Boukerroui in [START_REF] Boukerroui | Segmentation of ultrasound images -multiresolution 2D and 3D algorithm based on global and local statistics[END_REF].

Madabhushi and Metaxas [START_REF] Madabhushi | Combining low-, high-level and empirical domain knowledge for automated segmentation of ultrasonic breast lesions[END_REF] combined intensity, texture information and empirical domain knowledge used by radiologists with a deformable shape model in an attempt to limit the effects of shadowing and false positives. (see Fig. 2 for an example result taken from their paper). Their method requires training but in the small database (N = 42). Using manual delineation of the mass by a radiologist as a reference, and the Hausdorff distance and average distance as boundary error metrics, they showed that their method is independent of the number of training samples, shows good reproducibility w.r.t. parameters, and gives a True Positive area of 74.7%. They also argued that it has automation advantages over the work of Horsch et al. [START_REF] Horsch | Automatic segmentation of breast lesions on ultrasound[END_REF].

Although some of the previous methods can be applied in 3D, the literature on 3D is less extensive. For instance, Chen et al. [START_REF] Chen | 3-D breast ultrasound segmentation using active contour model[END_REF], Chang et al. [START_REF] Chang | Segmentation of breast tumor in three-dimensional ultrasound images using three-dimensional discrete active contour model[END_REF]- [START_REF] Chang | Support Vector Machines for diagnosis of Breast tumors on US images[END_REF] and Sahiner et al. [START_REF] Sahiner | Computerized characterization of breast masses on threedimensional ultrasound volumes[END_REF] take a deformable active contour approach. Chang et al. [START_REF] Chang | Segmentation of breast tumor in three-dimensional ultrasound images using three-dimensional discrete active contour model[END_REF] applied a 3D version of the "sticks algorithm" [START_REF] Czerwinski | Detection of lines and boundaries in speckle images -application to medical ultrasound[END_REF] to reduce speckle noise, followed by 3D morphological filtering (opening and closing, but details not given) as pre-processing steps. Then they applied an active contour which uses intensity and intensity variance information. The method was tested on 8 tumors (4 benign, 4 malignant) with volume estimates compared with estimates by manual delineations. Using the match rate as a performance metric, the average match rate was about 95%.

Sahiner et al. [START_REF] Sahiner | Computerized characterization of breast masses on threedimensional ultrasound volumes[END_REF] compared 2D and 3D intensity-gradient active contour segmentation based methods, the active contour initialized by hand, and with algorithm parameters determined empirically. Having found the segmentation solution, depthto-width ratio, a posterior shadowing feature measure, and 72 texture features based on co-occurrence matrix analysis were computed around the boundary for each 2D slice and linear discriminant analysis used to classify volumes. Four radiologists graded the volumes in terms of perceived malignancy on a scale 1 -10. They showed that the radiologist and computer-based methods were not statistically different in classification (A z = 0.92 versus A z = 0.87 average for radiologists). However they did not look at the accuracy of segmentation in depth and recognized that this was an area of possible improvement.

B-mode ultrasound imaging can also be used to guide, in real-time, a needle to a mass in biopsies and there has been a few recent papers describing work aimed at automatically tracking the needle during this procedure to localize the position of the mass. The task here is one of contour detection. Ding and Fenster, for instance, assume that the needle is straight and develop an algorithm for needle detection based on the Hough transform [START_REF] Ding | A real-time biopsy needle segmentation technique using Hough Transform[END_REF].

C. Prostate

Prostate cancer is the most commonly diagnosed cancer in adult and ageing men. Early detection and early intervention of progressive prostate cancer may help to reduce death rate. Transrectal ultrasound (TRUS) imaging has long been an essential tool for prostate cancer diagnosis. In fact, the combination of prostate specific antigen (PSA) testing with TRUS and digital rectal examination (DRE) has been responsible for diagnosing most prostate cancers in the United States each year. Prostate volumes and boundaries play an important role in diagnosis, treatment and follow-up of prostate cancer (see, for example [START_REF] Shao | Prostate boundary detection from ultrasonographic images[END_REF]). In general, prostate boundaries are routinely outlined in transverse parallel 2D slices along the length of the prostate. This has led to the development of a number of methods for (semi-) automatic detection of prostate boundaries 2 . An example of prostate segmentation from Gong et al. ( 2004) [START_REF] Gong | Parametric shape modeling using deformable superellipses for prostate segmentation[END_REF] is shown in Fig. 3.

Classical techniques have been proposed for prostate segmentation; derivative edge detection was used in [START_REF] Chen | Segmentation and reconstruction of prostate from trasrectal ultrasound images[END_REF], and, non-linear filtering (minimum/maximum filter) was proposed in [START_REF] Aarnink | A practical clinical method for contour determination in ultrasonographic prostate images[END_REF]- [START_REF] Aarnink | Edge detection in prostatic ultrasound images using integrated edge maps[END_REF]. Both supervised classification methods such as methods based on Neural Networks [START_REF] Prater | Segmenting ultrasound images of the prostate using neural networks[END_REF] and unsupervised pixel classifiers, for instance using Laws texture features in a probabilistic framework [START_REF] Richard | Automated texture-based segmentation of ultrasound images of the prostate[END_REF] have also been proposed. These methods do not use shape modeling and knowledge of ultrasound physics and have undergone limited validation.

Recent work has focused on incorporating prior information about shape and speckle models. For instance, Knoll et al. [START_REF] Knoll | Outlining of the prostate using snakes with shape restrictions based on the wavelet transform[END_REF] proposed employing a parametrization of a snake based on a 1D dyadic wavelet transform as a multiscale boundary curve analysis tool. The initialization of the snake used template matching between contour models of a training set and significant image edges. The authors used a 1D wavelet based method in order to constrain the shape of the snake to evolve towards predefined models. The contour deformation method was integrated in a coarse-to-fine segmentation framework based on multiscale image edges represented by the modulus maxima of the 2D dyadic wavelet transform. The fully automatic method was tested on 77 images from 11 patients against two experts. The analysis showed that shape information slightly improves results. However, no statistical analysis (of significance nor variance) was shown for the error comparison against the inter-observer variability.

Ghanei et al. [START_REF] Ghanei | A threedimensional deformable model for segmentation of human prostate from ultrasound images[END_REF] designed a 3D discrete deformable model to semi-automatically outline the prostate boundaries. Good segmentation results were obtained, in comparison to manual contours obtained slice-by-slice, subject to close initialization of the surface to the true boundaries.

A 2D semi-automatic discrete dynamic contour (DDC) model for prostate boundary segmentation was proposed by Ladak et al. [START_REF] Ladak | Prostate boundary segmentation from 2D ultrasound images[END_REF] based on the Lobregt and Viergever model [START_REF] Lobregt | A discrete dynamic contour model[END_REF]. The same team extended their work to 3D images in Hu et al. [START_REF] Hu | Prostate boundary segmentation from 3D ultrasound images[END_REF]. The underlying model is similar to that in Ghanei et al. [START_REF] Ghanei | A threedimensional deformable model for segmentation of human prostate from ultrasound images[END_REF]. The former used 4 (for 2D, 6 for 3D) manually selected points on specific locations of the prostate boundary and then applied an appropriate interpolation to complete the initial model. The latter used a few initial contours that were drawn on different slices. The specified points were generally clamped during the deformation of the DDC which uses some directional information (based on the fact that prostate tissue appears darker than the non prostate tissues in ultrasound). An editing tool was also provided to semiautomatically correct the segmentation results by specifying more boundary points and then running the DDC algorithm again. The accuracy of the 2D and 3D algorithm were assessed using global and local measures, however against only one expert radiologist. 117 images of 19 patients were used for the 2D algorithm and 6 patients with an average of 24 slices, acquired using a rotational probe to form a 3D data set, were used for the 3D algorithm. The analysis showed that the semiautomatic segmentation was in agreement with the manual one except where the prostate boundaries were missing. Some other interesting conclusions can be drawn from this work. For example 13% of cases were difficult to assess and the 3D algorithm performed better that the 2D one especially at the lateral margins of the prostate. Recently, Chiu et al. [START_REF] Chiu | Prostate segmentation algorithm using dyadic wavelet transform and discrete dynamic contour[END_REF], presented a variant of Ladak et al.'s [START_REF] Ladak | Prostate boundary segmentation from 2D ultrasound images[END_REF] algorithm where the intensity gradient, used in the external energy force of the DDC model, was estimated using the dyadic spline wavelet transform.

Pathak et al. [START_REF] Pathak | Edge-guided boundary delineation in prostate ultrasound images[END_REF] used the Sticks algorithm [START_REF] Czerwinski | Detection of lines and boundaries in speckle images -application to medical ultrasound[END_REF] followed by anisotropic edge-preserving smoothing based on the weakmembrane algorithm for semi-automatic boundary detection. A Canny edge detector was then used to construct an edge map rather than edges detected by the weak-membrane algorithm. This enabled prior knowledge-based filtering of the edge map to be used to detect only the relevant edges. The method was semi-automatic as user interaction was employed in the edge-linking stage. The experiments on 125 images from 16 patients with the help of five expert observers showed an increase of consistency in prostate delineation when automatically detected edges were used as a visual guide during the manual outlining process. In order to restrict user interaction, a deformable superellipse was employed as a parametric shape model for the prostate in a later work [START_REF] Gong | Parametric shape modeling using deformable superellipses for prostate segmentation[END_REF]. Superellipses are, on the one hand, attractive as only 2 pose parameters and 4 shape parameters need to be estimated, and on the other, as acknowledged by the authors, very restrictive as they can only model symmetric shapes. Apparently this restriction was not a draw back for prostate segmentation as the fitting error was less than the inter-observer variability. The authors chose a Bayesian framework to solve the fitting problem and used their previous work [START_REF] Pathak | Edge-guided boundary delineation in prostate ultrasound images[END_REF] to estimate the edge likelihood and a training data set to estimate the prior distributions parameters of the superellipes shape parameters. The segmentation algorithm was semi-automatic as the user had to specify more than two points to give an approximate scale of the prostate. The algorithm was tested and analyzed under the same conditions as in [START_REF] Pathak | Edge-guided boundary delineation in prostate ultrasound images[END_REF] using the measures defined in [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF]. The mean distance between the computer-generated boundaries and the average of the five manual outlines was significantly smaller than the average of the inter-observer variability. (see Fig. 3 for an example result).

Shen et al. [START_REF] Shen | Segmentation of prostate boundaries from ultrasound images using statistical shape model[END_REF] proposed a statistical shape model for automatic 2D prostate boundaries detection. Their model takes into account the nature of TRUS images by considering the probe position in the shape normalization step and by making the image features descriptors invariant to probe rotation. Gabor filters at different orientations and scales were used for the data fidelity term and affine invariant features, which capture the prostate shape in a hierarchical fashion, were employed for the model term. The work presents several original contributions mainly in modeling the prior geometry of the prostate in the TRUS images. Modeling of the prostate boundaries by means of Gabor filters might be improved by taking into account the quadrature property of Gabor filters (see eg. [START_REF] Mulet-Parada | 2D+T acoustic boundary detection in echocardiography[END_REF] and references therein). Their method was evaluated on 8 images. This is a preliminary evaluation from which strong conclusions cannot be drawn 3 . The authors extended their work to segment 3D data in [START_REF] Zhan | Automated segmentation of 3D US prostate images using statistical texture-based matching method[END_REF], and interestingly, they used a kernel support vector machine (KSVM) to estimate a probability density function of prostate tissue voxels based on the Gabor features. Unfortunately, the modification did not decrease the processing time. Recently, the authors proposed an efficient variant of their previous work by using Zernike Moment based edge detection at the coarse resolutions and kept the SVM modeling only for the finest resolution. They also decreased the number of support vectors without a loss of accuracy [START_REF]An efficient method for deformable segmentation of 3D US prostate images[END_REF]. The new algorithm ran 10 times faster. Shen et al. [START_REF] Shen | Optimized prostate biopsy via a statistical atlas of cancer spatial distribution[END_REF] also worked on a 3D statistical atlas of the spatial distribution of cancer for optimized prostate biopsy. One hundred prostate histological samples were used in the atlas construction. A 99% biopsy success rate was achieved using seven needles.

Abolmaesumi and Sirouspour [START_REF] Abolmaesumi | An interacting multiple model probabilistic data association filter for cavity boundary extraction from ultrasound images[END_REF] developed a 2D boundary extraction method based on a probabilistic data association filter. They posed the 2D segmentation problem as an estimation of a moving object along the cavity boundary, where the motion is governed by a finite set of dynamical models subject to uncertainty. Hence, the motion model along the contour models the prior smoothness of the contour. Angular discretisation of the contour from a manually selected seed point inside the cavity was used to estimate the radial distance of the boundary points to the seed point. In order to increase robustness to noise and accuracy they combined multiple trajectory models, with different data noise levels and trajectory smoothness/dynamics. They used a probabilistic data association filter (PDAF) for boundary extraction and the interacting multiple model estimator (IMM) for model combination 4 . The boundary extraction was performed recursively using a modified Kalman filter with an approximately constant angular velocity model and where the candidate edge points were estimated using a 1D derivative operator along each radial direction. A mathematical analysis of the PDAF filter with respect to the process and noise parameters was provided to help model selection based on the noise level in the image and the shape of the object boundary. Segmentation examples on several ultrasound applications (heart, breast, prostate) demonstrated the potential of this approach. Some limitations and open issues were highlighted mainly due to the angular discretisation and implicitly, the performance sensitivity to the choice of the seed point. The algorithm was fast and could be implemented in real time. However, the 3D extension of the approach is not obvious. The model combination is particularly interesting. It can be viewed as an adaptive regularization of the boundaries because the models have different noise/smoothness tradeoffs. The IMM selects the dominant model where there is a disagreement and takes some kind of an average solution where the models agree. However, the selection of the edge candidates is based on intensity gradient and hence ignores the nature of the data. This was improved in [START_REF] Abolmaesumi | Segmentation of prostate contours from ultrasound images[END_REF] by using the Sticks algorithm [START_REF] Czerwinski | Detection of lines and boundaries in speckle images -application to medical ultrasound[END_REF] for edge enhancement as a pre-processing step before applying the PDAF/IMM algorithm.

Yu et al. [START_REF] Yu | Segmentation of the prostate from suprapubic ultrasound images[END_REF] presented a two-step semi-automatic activecontour based segmentation method. First, an initial elliptical approximation of the contour was obtained using two manually selected points. A rough binary segmentation was then obtained by optimizing an area-weighted mean-difference criterion in a level sets framework. Finally, a finer segmentation was accomplished via a parametric active contour model in a polar coordinate system (see also [START_REF] Kovalski | Three-dimensional automatic quantitative analysis of intravascular ultrasound images[END_REF] for a polar coordinate active contour). In the second stage, a Speckle Reducing Anisotropic Diffusion, (SRAD [START_REF] Yu | Speckle reducing anisotropic diffusion[END_REF]), was applied to enhance the images and the instantaneous coefficient of variation (ICOV) was utilized as an edge cue in the derivation of the external energy of the active contour model. Note that log-compressed B-scan images have to be decompressed before applying the SRAD algorithm. It is reported in [START_REF] Yu | Speckle reducing anisotropic diffusion[END_REF], [START_REF]Edge detection in ultrasound imagery using instantaneous coefficient of variation[END_REF] that the ICOV allows for balanced and well-localized edge strength measurements in bright as well as in dark regions of speckle images. In summary, the first stage of the segmentation algorithm uses global image statistics and the second one uses local image features to refine the segmentation. It is unclear why the authors used 2 steps since both energy functions could be minimized in the level sets framework (see eg. [START_REF] Kimmel | Geometric Level Set Methods in Imaging, Vision and Graphics[END_REF]). A preliminary evaluation of the approach was presented on 27 selected images acquired from 6 patients. Manual delineations were performed by 3 sonographers. Only images with a 'reasonable' interobserver variability were used. The computer generated boundaries agreed with the average manual contours within 2 mm. 76% of the results fell within one standard deviation of the manual contours.

D. Other Cancer Types

The previous two subsections have considered the two main areas of application of ultrasound in diagnosis of cancer. As ultrasound is increasingly used in the planning and treatment of interventions and therapy it is worth mentioning that there are a growing number of publications related to (non-breast and prostate) cancer diagnosis and therapy. Recent papers include those looking at ultrasound segmentation of the kidney and gallbladder [START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF], [START_REF] Levienaise-Obadia | Adaptive segmentation of ultrasound images[END_REF]- [START_REF] Dokur | Segmentation of ultrasound images by using a hybrid neural network[END_REF] and the liver [START_REF] Kotropoulos | Segmentation of ultrasonic images using Support Vector Machines[END_REF]- [START_REF] Lee | Unsupervised segmentation of ultrasonic liver images by multiresolution fractal feature vector[END_REF].

E. IVUS (vascular disease)

IntraVascular UltraSound (IVUS) is increasingly used as a complementary tool to contrast X-ray angiography. It is a real-time, high resolution and non-invasive imaging technique, which provides valuable anatomical information about the coronary arterial wall and plaque. Recent developments, include ECG-gated acquisition to overcome cyclic motion artifacts and IVUS elastography which provides biomechanical property estimates of vessel wall and artherosclerotic plaques, that are helpful in diagnosis and in guiding interventional procedures [START_REF] Bom | New developments in intravascular ultrasound imaging[END_REF], [START_REF] Radeva | Angiography and plaque imaging: Advanced segmentation techniques[END_REF]. Accurate segmentation of the lumen, the plaque and the wall borders is a prerequisite for quantitative analysis. Most, if not all, of the IVUS segmentation literature, uses contour modeling and could be classified into one of 3 categories: edge driven [START_REF] Kovalski | Three-dimensional automatic quantitative analysis of intravascular ultrasound images[END_REF], [START_REF] Sonka | Segmentation of intravascular ultrasound images: A knowledge-based approach[END_REF]- [START_REF] Klingensmith | Validation of an automated system for luminal and medial-adventitial border detection in three-dimensional intravascular ultrasound[END_REF], statistical [START_REF] Cardinal | Intravascular ultrasound image segmentation: A fastmarching method[END_REF]- [START_REF] Hansen | Bayesian contour detection in a time series of ultrasound images through dynamic deformable template models[END_REF] and high-level knowledge driven [START_REF] Bovenkamp | Multi-agent segmentation of IVUS images[END_REF], [START_REF] Olszewski | Segmentation of intravascular ultrasound images: a machine learning approach mimicking human vision[END_REF]. We review some of the important papers in this section.

Sonka et al. [START_REF] Sonka | Segmentation of intravascular ultrasound images: A knowledge-based approach[END_REF] presented a semi-automatic knowledgebased segmentation approach that identifies the internal and external lumina and the plaque-lumen boundaries. A novelty of the approach was the attempt to incorporate knowledge to mimic constraints used by an experienced ultrasonographer. The method used dynamic programming to solve a cost function based on edge strength and incorporated a priori knowledge about cross-sectional arterial anatomy (such as object shape, edge direction, double echo pattern, wall thickness). However, the method has several modeling limitations namely: the edge strength did not include speckle statistics and some prior information was included in a binary fashion with hard thresholds. A similar model was used by Takagi et al. [START_REF] Takagi | Automated contour detection for highfrequency intravascular ultrasound imaging: a technique with blood noise reduction for edge enhancement[END_REF], and reported a similar performance. Their work differs from the work of Sonka et al. mainly in the preprocessing step. An adaptive spatio-temporal filtering of speckle was performed based on pre-segmentation of blood and tissue areas.

Subtraction of two consecutive IVUS images in time acquired at the same position can increase the signal-to-noise ratio of the lumen area, i.e. obtain good contrast between the lumen and other parts in the image. Bouma et al. [START_REF] Bouma | Automated lumen definition from 30 MHz intravascular ultrasound images[END_REF] compared several combinations of filtering techniques (Gaussian, median, anisotropic diffusion) followed by lumen detection (thresholding, region-growing, discrete dynamic contour (DDC) [START_REF] Lobregt | A discrete dynamic contour model[END_REF]). Experiments were carried out on 15 IVUS images, obtained using an average of 20 subtracted images all acquired at the same location. The evaluation against 4 experts showed that all the methods performed well on good quality data and less well on difficult cases. The automatic DDC, preceded with a small scale median filter was found to be the best alternative method to manual segmentation.

An extension of the DDC model has been proposed for 3D semi-automatic segmentation of the lumen and adventitial borders in serial IVUS images [START_REF] Shekhar | Three-dimensional segmentation of luminal and adventitial borders in serial intravascular ultrasound images[END_REF]. The same group presented a faster method, the fast active surface (FAS) [START_REF] Klingensmith | Evaluation of threedimensional segmentation algorithms for the identification of luminal and medial-adventitial borders in intravascular ultrasound images[END_REF] based on Williams and Shah's fast active contours (see Fig. 4(a) for an example result). The former method is a force-acceleration technique and the latter is a neighborhood-search method, and both snakes used only intensity gradient information. An editing tool was provided for user correction. In experiments, both techniques performed well although the FAS was 60 times faster. The FAS method was further validated on 529 ECG-gated IVUS images against 4 observers [START_REF] Klingensmith | Validation of an automated system for luminal and medial-adventitial border detection in three-dimensional intravascular ultrasound[END_REF] using the measures defined in [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF]. Kovalski et al. [START_REF] Kovalski | Three-dimensional automatic quantitative analysis of intravascular ultrasound images[END_REF] used a simplified 3D balloon model where the control points can move only along the radial direction, and used polar coordinates to represent the contours which removes one degree of freedom. The performance of the method was assessed on 88 images against two observers. The reported inter-observer area variability was in the same range as in [START_REF] Klingensmith | Validation of an automated system for luminal and medial-adventitial border detection in three-dimensional intravascular ultrasound[END_REF]. However, the computer to manual area variability was slightly higher than the inter-observer one, for both the lumen and adventitial borders. In contrast to the method in [START_REF] Klingensmith | Evaluation of threedimensional segmentation algorithms for the identification of luminal and medial-adventitial borders in intravascular ultrasound images[END_REF], the method of Kovalski et al. [START_REF] Kovalski | Three-dimensional automatic quantitative analysis of intravascular ultrasound images[END_REF] was fully automatic.

The above methods use simple derivative operators to estimate intensity gradients. Recently Pardo et al. [START_REF] Pardo | Discriminant snakes for 3D reconstruction of anatomical organs[END_REF], proposed a statistical deformable model that was applied to IVUS images. They used a bank of Gaussian derivative filters, at different orientations and scales, to locally describe edge/nonedge cues, in an adaptive fashion (i.e locally). The feature space was reduced by linear discriminant analysis, and a parametric classifier was employed to guide the model deformation by minimizing the dissimilarity between measured image features and the learned ones. The statistical learning approach made the algorithm more robust in comparison to using intensity gradient information, which supposes a prior model of the edge 5 .

A Bayesian level set approach was proposed for a semiautomatic segmentation of the lumen, intima and media borders using the shifted Rayleigh law as a model for the graylevel statistics [START_REF] Cardinal | Intravascular ultrasound image segmentation: A fastmarching method[END_REF]. This work is based on the previously published Bayesian level set method of Sifakis et al. [START_REF] Sifakis | Bayesian level sets for image segmentation[END_REF]. Robustness to initialization was assessed, and a comparison made to an intensity gradient level-set method, intensity gradient based snakes [START_REF] Klingensmith | Evaluation of threedimensional segmentation algorithms for the identification of luminal and medial-adventitial borders in intravascular ultrasound images[END_REF] and PDF-snakes using the same model as in [START_REF] Mignotte | A multiscale optimization approach for the dynamic contour-based boundary detection issue[END_REF]. Fifteen IVUS images were used, acquired at 20Mhz without ECG-gating, and with 3 different initializations. The PDFsnakes were more stable on average. However the Fast-Marching method had the smallest Hausdorff distance. As expected the gradient-snake had the largest variability to initialization as it does not model well the data. In a more recent work [START_REF] Cardinal | Automatic 3D segmentation of intravascular ultrasound images using region and contour information[END_REF], the same team proposed an automatic variant of their previous work (automatic estimation of the PDF region statistics parameters and the initial level set contours). In the new algorithm, the speed function was based on region statistics and gray level gradient information rather than region statistics alone. The algorithm was tested on 600 IVUS frames acquired at 20MHz from a diseased superficial femoral artery and on a sequence of 86 simulated IVUS images. The Hausdorff distances were below 0.066 mm for the simulated data and below 0.344 mm for the in-vivo data in comparison to one observer manual border delineation.

A posteriori probability based segmentation using a Rayleigh distribution to model the data constraint and a 1D Markov process to model the contour prior, was also used in [START_REF] Brusseau | Fully automatic luminal contour segmentation in intracoronary ultrasound imaging -A statistical approach[END_REF], [START_REF] Haas | Segmentation of 3D intravascular ultrasonic images based on a random field model[END_REF]. Both methods use polar coordinates, are fully automatic and are related to previous work by Friedland and Adam [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF] and Dias and Leitão [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF]. Haas et al. [START_REF] Haas | Segmentation of 3D intravascular ultrasonic images based on a random field model[END_REF] considered multiple contours and modeled the sequence as a first-order Markov process to account for the temporal continuity of the vessel walls. Brusseau et al. [START_REF] Brusseau | Fully automatic luminal contour segmentation in intracoronary ultrasound imaging -A statistical approach[END_REF] considered only the segmentation of the luminal border (see Fig. 1(b) for 2 results taken from their paper). However as the authors showed, the luminal border does not always correspond to the global maximum of the posterior pdf, nor to the first maximum, along radial direction. The algorithm first detected the 'reliable' points that are the first or the global maximum along the radial direction and then choose either the global or the first maximum depending on the contour prior energy. Both methods were evaluated on several clinical cases against two experts, and good correlation was obtained. Hansen et al. [START_REF] Hansen | Bayesian contour detection in a time series of ultrasound images through dynamic deformable template models[END_REF] presented a fully Bayesian analysis of space time process of IVUS image based on deformable template models and Markov Chain Monte Carlo (MCMC) simulation. Unfortunately they used a Gaussian model for both tissues and only applied their method to one sequence. Guérault et al. [START_REF] Guérault | Modelization and segmentation of intravascular ultrasound images[END_REF] presented a constrained maximum likelihood approach. The originality of their work was not in the data term, which uses a Rayleigh model, but in the modeling of the prior information about the geometric deformations inherent in IVUS images. Based on the hypothesis of a circular vessel, they constrained the search space using a parametric model of the contour which takes into account the catheter position and its orientation. A stochastic minimization procedure was utilized for the maximisation of the criterion.

Recently, two approaches have been published which aim to incorporate high level knowledge in IVUS image segmentation. A machine learning approach mimicking the human visual system was proposed for automatic detection of the luminal and the medial-adventitial borders in [START_REF] Olszewski | Segmentation of intravascular ultrasound images: a machine learning approach mimicking human vision[END_REF]. The accuracy of the method was assessed on a reasonable data set. Bovenkamp et al. [START_REF] Bovenkamp | Multi-agent segmentation of IVUS images[END_REF], considered solving the segmentation problem using a multi-agent knowledge approach. Its elaborate high-level knowledge (450 rules) takes control over simple low-level segmentation algorithm.

A major drawback of IVUS is its inability to consider the vessel curvature and the orientation of the imaging catheter. Hence, quantifications performed on these data are inevitably distorted, since the vessel curvature remains unconsidered. Fusion between intravascular ultrasound and biplane angiography provides a solution for correct 3D reconstruction of the IVUS data (see eg. [START_REF] Radeva | Angiography and plaque imaging: Advanced segmentation techniques[END_REF], [START_REF] Wahle | Accurate volumetric quantification of coronary lesions by fusion between intravascular ultrasound and biplane angiography[END_REF]). This is out of the scope of this review. Nevertheless, we comment that the research in this field uses angiography information only in the reconstruction step of the 3D vessels. This is a valuable source of information that could be used profitably in the 3D segmentation step of the IVUS images.

Finally, we note that some attention has also been given to developing methods related to the diagnosis and treatment of vascular disease using a standard B-mode ultrasound transducer (ie. not IVUS), including work on segmentation of carotid vessels [START_REF] Gill | Accuracy and variability assessment of a semiautomatic technique for segmentation of the carotid arteries from three-dimensional ultrasound images[END_REF]- [START_REF] Baillard | Robust 3D segmentation of anatomical structures with level sets[END_REF] and abdominal aortic aneurysms [START_REF] Ravhon | Validation of ultrasonic image boundary recognition in abdominal aortic aneurysm[END_REF].

F. Obstetrics and gynaecology

The non invasive nature of ultrasound is a strong argument for its use in obstetrics and gynaecology. In obstetrics, segmentation provides valuable measurements in order to assess the growth of the foetus and in diagnosis of fetal malformation. Most analysis is based on 2D scans. Standard measurements include the biparietal diameter, head circumference, length of the fetal femur, the abdominal circumference and the amniotic fluid volume (see eg. [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF], [START_REF] Pathak | Interactive automatic fetal head measurements from ultrasound images using multimedia computer technology[END_REF]- [START_REF] Chalana | Automatic fetal head measurements from sonographic images[END_REF]). There is often a sharp contrast between the face of a foetus and the surrounding amniotic fluid, allowing automatic boundary detection. Hence, obstetrics is a potential field of application for volume rendering and visualization (see eg. [START_REF] Lee | Three-dimensional ultrasonographic presentation of micrognathia[END_REF], [START_REF] Gee | Engineering a freehand 3D ultrasound system[END_REF]). In gynaecology, most of reported segmentation algorithms address ovarian follicle measurement and cyst detection and measurement [START_REF] Zimmer | Image segmentation in obstetrics and gynecology[END_REF].

Muzzolini et al. [START_REF] Muzzolini | Multiresolution texture segmentation with application to diagnostic ultrasound images[END_REF] used a split-and merge segmentation approach to segment 2D US images of ovarian follicles. The split and merge operations were controlled by means of a simulated annealing algorithm (Metropolis). The authors used a texture-based measure for block comparison in the splitting and merging process which takes into account the block size. In a later work, the same team proposed a robust texture feature selection method based on outlier rejection to be used in the segmentation algorithm [START_REF]Texture characterization using robust statistics[END_REF].

A semi-automatic method for ovarian follicles segmentation was reported by Sarty et al. [START_REF] Sarty | Semiautomated segmentation of ovarian follicular ultrasound images using a knowledge-based algorithm[END_REF]. The technique is a knowledge based approach and is similar to earlier work of the team [START_REF] Sonka | Segmentation of intravascular ultrasound images: A knowledge-based approach[END_REF]. It simultaneously detects the inner and the outer border of the follicle of interest, which was interactively selected by defining an angular region of interest. Both border detections were based on the minimization of a cost function using heuristic graph searching techniques. Polar coordinates were used and edges strength and direction were considered in the cost function definition which incorporates some prior knowledge. The outer wall was automatically detected after the detection of the inner wall, for which frequent manual correction were required. Measurements based upon the algorithm's outer border correlated well with those based on a human expert manual boundary tracing (31 US images were used). A similar work was reported by Krivanek and Sonka for follicle segmentation [START_REF] Krivanek | Ovarian ultrasound image analysis: Follicle segmentation[END_REF]. An automatic approximation of the inner follicles wall using watershed segmentation on a smoothed image was first performed. Then the knowledge graph searching method was applied to detect both walls of the follicle of interest.

Potočnik and Zazula proposed a 3 step algorithm for automatic segmentation of follicles in 2D US images [START_REF] Potočnik | Automated analysis of a sequence of ovarian ultrasound images. Part I: segmentation of single 2D images[END_REF]. The first step automatically detected seed points based on a combination of a watershed segmentation and several thresholding operations on prefiltered images. In the second step, accurate boundary detection of the follicles was performed using a region growing segmentation starting from the previously detected seed points. The pixel aggregation was controlled using a criterion based on the mean gray level values and a weighted gradient. The final step was a recognition step which used prior knowledge about the follicles in order to eliminate nonfollicle detected regions (shape, size, and localization within an image). The authors extended this work to the segmentation of a sequence of images [160]. The new algorithm used the detected objects in the previous frame in order to predict the object in the next frame by means of a Kalman filter. The algorithm was tested on 50 cases with a recognition rate around 78% and a boundary accuracy around 1.1 mm (mean error in comparison to manual boundary tracing). The results are encouraging probably due mostly to the appropriate prior knowledge used in the algorithm 6 .

Zimmer et al. [START_REF] Zimmer | The distribution of the local entropy in ultrasound images[END_REF] made an interesting observation, showing that the local gray level entropy is a normal variable, under the assumption that the local gray level pdf follows a Gaussian, Rayleigh, Weibull or a Nakagami distribution. This suggests that the local entropy may be successfully used for region segmentation when its histogram shows multiple distinct peaks. Indeed, the same team presented a bivariate extension of an entropy-based thresholding method, the 'minimum cross entropy thresholding', in which the segmented variable is replaced by a linear combination of the gray level and the local entropy [START_REF] Zimmer | A two-dimensional extension of minimum cross entropy thresholding for the segmentation of ultrasound images[END_REF]. The binary segmentation algorithm was demonstrated on US images with ovarian cysts in [START_REF] Zimmer | A two-dimensional extension of minimum cross entropy thresholding for the segmentation of ultrasound images[END_REF] and was used in the segmentation step in a recent work for quantitative analysis and malignancy detection of ovarian masses [START_REF] Zimmer | An automatic approach for morphological analysis and malignancy evaluation of ovarian masses using B-scans[END_REF]. The demonstration of the normality of the local entropy under more complete models, like the generalized Kdistribution, is still to be done (see Sec. IV-B).

All of the above methods operate in 2D which mimics clinical practice. However, one is really looking at a threedimensional object, and the interest is to measure volume rather than a linear dimension (Euclidean distance). Romeny et al. [START_REF] Romeny | Computer assisted human follicle analysis for fertility prospects with 3D ultrasound[END_REF] described a method for segmenting follicles from 3D ultrasound data which was fully automatic and consisted of two steps (follicles centers detection followed by 1D edge detection along radial rays generated from the follicle's center). The centers detection method was based on the socalled winding number of the intensity singularity 7 . The edge points along the 1D profiles were detected using a robust multiscale zero-crossing of the second derivative. The lifetime of an edge pattern over scale was used as a measure of its significance.

Gooding et al. [START_REF] Gooding | Volume reconstruction from sparse 3D ultrasonography[END_REF] presented a level set based method for segmenting sparse freehand ultrasound data of the ovary where the speed function is a weighted sum of three terms; a surface reconstruction term, a regularization term and an image term. The image term was region-based and defined in terms of non-parametric probabilities of labeled regions. The method required a good initialization. The method is more extensively discussed in [START_REF] Gooding | 3D ultrasound image analysis in assisted reproduction[END_REF]

ESV % error 8.9 ml 6( 12) OB(1), CR(MRI) EDV % error 5.9 ml EF % error 4.1 %

Montagnat [54] 2003 ROT3D 3D+T S/R (+) (-) Initialization QL - - 1 -
• In general, to validate medical image segmentation methods quantitatively one can use simulation, phantom studies, animal model studies, and clinical studies using manual delineation, another modality as a "gold standard" or reference, or some correlation with clinical outcome. The literature on ultrasound segmentation indicates that some simulation and phantom studies have been performed but predominantly clinical data is used in validation. This partly reflects the difficulty in defining realistic simulations and phantoms. An effective ultrasound simulation package, 'Field II' is available [START_REF] Jensen | Field: A program for simulating ultrasound systems[END_REF], [START_REF] Jensen | Fast simulation of ultrasound images[END_REF] 8 . The package provides an excellent framework to simulate ultrasound transducer fields and ultrasound imaging using linear acoustics. The program has been used in number of application to provide ground truth data (see eg. [START_REF] Jardim | Segmentation of fetal ultrasound images[END_REF], [START_REF] Boukerroui | A multiparametric and multiresolution segmentation algorithm of 3-D ultrasonic data[END_REF]- [START_REF] Hassenpflug | Speckle classification for sensorless freehand 3-D ultrasound[END_REF]).

• Manual delineation on clinical data is by far the most popular means of performance assessment although depending on the clinical area, there can be significant interexpert and intra-expert variability (quite often, simply because manual segmentation is not a task that they would normally do) and delineation can be difficult to do. Therefore one needs to take care in interpreting results when manual delineation is used as the reference. • There is also a general lack of standardization of performance measures, which makes it difficult to directly compare methods. The one exception here is in cardiology, where the method of Chalana and Kim [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF] has been used by a number of groups (however, see [START_REF] Alberola-Lopez | Comments on: A methodology for evaluation of boundary detection algorithms on medical images[END_REF] for a correction of the Percent Statistic test). This has the advantage that multiple experts are used to define the reference.

The method of Chalana and Kim assumes that segmentation boundaries can be well-defined. This is not the case, for example, for malignant breast masses where a region-based metric is more applicable. Performance metrics used in breast ultrasound segmentation performance assessment have employed quite simple area-based measures with comparison with a single expert or multiple experts treated separately. This area might benefit from moving towards using performance methods that use multiple expert segmentations such as the STAPLE (simultaneous truth and performance level estimation) algorithm [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation[END_REF].

• From the Tables we note the low number of papers reported with validation on databases with more than 50 cases. • There are also no standard databases on which different groups can compare methods, so comparisons have only been made between methods developed by individual groups. This is problematic for the field as a whole, as clinical ultrasound image quality varies a lot, more so than in other areas of medical imaging such as CT and MRI.

IV. METHODOLOGY A. What makes a good ultrasound segmentation method?

Due to the relatively low quality of clinical ultrasound images, a good ultrasound image segmentation method needs to make use of all task-specific constraints or priors. This is an explicit or implicit assumption in all successful methods in the literature. Methods can broadly be defined in terms of those that make use of imaging physics constraints, those that make use of anatomical shape constraints, or temporal constraints or a combination thereof (one could also include functional constraints, such as a perfusion model, but segmentation of contrast agent images is not considered in this review). In the following subsection we consider first speckle, the characteristic feature of ultrasound images, then intensity-based priors and finally how geometric and temporal priors have been employed.

B. Speckle -noise or a feature?

Speckle gives ultrasound images their characteristic granular appearance. It inherently exists in coherent imaging, including ultrasound imaging. The analysis of this signal-dependent effect has been a major subject of investigation in the medical ultrasound imaging community as well as the optical (laser) and radar imaging communities. The texture appearance of the observed speckle does not correspond to underlying tissue structure. However, the local brightness of the speckle pattern does reflect the local echogeneity of the underlying scatterers. Speckle can be undesirable and hence is seen as noise to be reduced, or, as signal carrying some information about the observed tissues. Thus from a segmentation perspective you may chose to remove it or utilize it for the information it contains.

The received echo signals are obtained by a coherent summation of echo signals from many ultrasound scatterers. Speckle has a random and deterministic nature as it is formed from backscattered echoes of randomly or coherently distributed scatterers in the tissue [START_REF] Wagner | Statistics of speckle in ultrasound B-scans[END_REF], [START_REF] Burckhardt | Speckle in ultrasound B-mode scans[END_REF]. It has been shown that the statistical properties of the received signal, and thus of the echo envelope, depend on the density and the spatial distribution of the scatters [START_REF] Wagner | Statistics of speckle in ultrasound B-scans[END_REF]. Several distribution families have been proposed in the literature. For the special case of a large number of randomly located scatterers, the statistics of the envelope signal shows a Rayleigh distribution (see eg. [START_REF] Wagner | Statistics of speckle in ultrasound B-scans[END_REF], [START_REF] Burckhardt | Speckle in ultrasound B-mode scans[END_REF]). In this condition the speckle is called fully developed. Deviations from such special scattering conditions have been previously modeled via:

• The Rice distribution to account for a coherent component due to the presence of a regular structure of scatterers within the tissue [START_REF] Wagner | Statistics of speckle in ultrasound B-scans[END_REF], [START_REF] Insana | Analysis of ultrasound image texture via generalized Rician statistics[END_REF].

• The K-distribution to account for low effective scatter density (partially developed speckle) (see eg. [START_REF] Shankar | Use of non-Rayleigh statistics for the identification of tumors in ultrasonic B-scans of the breast[END_REF], [START_REF] Jakeman | K-distributed noise[END_REF]).

Unfortunately, the Rician family fails to account for reduced scatterer densities, and the K-distribution model does not take into account the presence of a coherent component. General models have been proposed in the literature namely the generalized K-distribution and the homodyned K-distribution (see eg. [START_REF] Jakeman | K-distributed noise[END_REF], [START_REF] Dutt | Ultrasound echo envelope analysis using a homodyned K-distribution signal model[END_REF]) and most recently the Rician Inverse of Gaussian distribution [START_REF] Eltoft | Speckle: Modeling and filtering[END_REF]. The three models are general and can account for the different scattering conditions. However, the analytical complexity with these families is significant. Some effort has been put into efficiently estimating the parameters [181]- [START_REF] Sijbers | Maximum-likelihood estimation of Rician distribution parameters[END_REF] and alternative simple models have also been proposed such as the Nakagami family (see eg. [START_REF] Shankar | A general statistical model for untrasonic backscattering from tissues[END_REF]).

Note that all these models are for the envelope of the received echo signal. In clinical ultrasound imaging systems, a log-compression is performed to control the dynamic range of the image. An approximate algorithm for the estimation of the log-compression parameter from pixel values of a displayed B-scan images was recently proposed [START_REF] Prager | Decompression and speckle detection for ultrasound images using the homodyned K-distribution[END_REF]. Statistics of the log-compressed echo envelope have also been proposed [START_REF] Dutt | Statistics of the log-compressed echo envelope[END_REF], [START_REF] Clifford | On the statistical characteristics of log-compressed Rayleigh signals: Theoretical formulation and experimental results[END_REF].

A number of tissue characterization algorithms [START_REF] Insana | Analysis of ultrasound image texture via generalized Rician statistics[END_REF], [START_REF] Shankar | Use of non-Rayleigh statistics for the identification of tumors in ultrasonic B-scans of the breast[END_REF], [START_REF] Shankar | A general statistical model for untrasonic backscattering from tissues[END_REF], [START_REF] Prager | Decompression and speckle detection for ultrasound images using the homodyned K-distribution[END_REF], [START_REF]Non-Rayleigh first-order statistics of ultrasonic backscatter from normal myocardium[END_REF]- [START_REF] Thijssen | Ultrasonic speckle formation, analysis and processing applied to tissue characterization[END_REF], denoising algorithms [START_REF] Eltoft | Speckle: Modeling and filtering[END_REF], [START_REF] Dutt | Adaptive speckle reduction filter for logcompressed B-scan images[END_REF], local entropy statistics [START_REF] Zimmer | The distribution of the local entropy in ultrasound images[END_REF] and an MRF model [START_REF] Bouhlel | New Markov random field model based on K-distribution for textured ultrasound image[END_REF] based on the above models have been proposed in the literature. There is also an extensive literature on speckle reduction which has been proposed as a pre-segmentation step; recent works include wavelets based methods [START_REF] Achim | Novel Bayesian multiscale method for speckle removal in medical ultrasound images[END_REF]- [START_REF] Xiao | A diffusion stick method for speckle suppression in ultrasonic images[END_REF], anisotropic diffusion methods [START_REF] Mikula | Morphological image sequence processing[END_REF], [START_REF] Yu | Speckle reducing anisotropic diffusion[END_REF], [START_REF] Abd-Elmoniem | Realtime speckle reduction and coherence enhancement in ultrasound imaging via nonlinear anisotropic diffusion[END_REF]- [START_REF] Tauber | A robust speckle reducing anisotropic diffusion[END_REF] and others [START_REF] Czerwinski | Detection of lines and boundaries in speckle images -application to medical ultrasound[END_REF], [START_REF] Eltoft | Speckle: Modeling and filtering[END_REF], [START_REF] Evans | Biased motion-adaptive temporal filtering for speckle reduction in echocardiography[END_REF]- [START_REF] Ito | Morphological operations by locally variable structuring elements and their applications to region extraction in ultrasound images[END_REF].

C. Priors 1) Image features:

• Gray level distribution: As noted in the previous section, the Rayleigh model of speckle has proved a popular choice. A Rayleigh distribution was used in the anisotropic diffusion edge detection method of [START_REF] Steen | Scale-space and boundary detection in ultrasound imaging, using signal-adaptive anisotropic diffusion[END_REF] and in statistical segmentation methods in [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF], [START_REF] Figueiredo | Unsupervised contour representation and estimation using B-splines and a minimum description length criterion[END_REF], [START_REF] Brusseau | Fully automatic luminal contour segmentation in intracoronary ultrasound imaging -A statistical approach[END_REF], [START_REF] Haas | Segmentation of 3D intravascular ultrasonic images based on a random field model[END_REF], [START_REF] Guérault | Modelization and segmentation of intravascular ultrasound images[END_REF], [START_REF] Jardim | Segmentation of fetal ultrasound images[END_REF]. Most recently the Rayleigh distribution was incorporated into the level set method of Sarti et al. [START_REF] Sarti | Maximum likelihood segmentation of ultrasound images with rayleigh distribution[END_REF] and of Cardinal et al. [START_REF] Ledesma-Carbayo | Spatio-temporal non-rigid registration for ultrasound cardiac motion estimation[END_REF]. A shifted Rayleigh distribution was used in the active contour approach of [START_REF] Mignotte | A multiscale optimization approach for the dynamic contour-based boundary detection issue[END_REF], [START_REF] Mignotte | Endocardial boundary estimation and tracking in echocardiographic images using deformable template and markov random fields[END_REF] and in level set segmentation algorithm in [START_REF] Cardinal | Intravascular ultrasound image segmentation: A fastmarching method[END_REF], [START_REF] Baillard | Robust 3D segmentation of anatomical structures with level sets[END_REF]. Other gray level distribution models have also been used in the ultrasound segmentation literature: for instance, the Gaussian [START_REF] Boukerroui | Segmentation of ultrasound images -multiresolution 2D and 3D algorithm based on global and local statistics[END_REF], [START_REF] Ashton | Multiple resolution bayesian segmentation of ultrasound images[END_REF], [START_REF] Hansen | Bayesian contour detection in a time series of ultrasound images through dynamic deformable template models[END_REF], [START_REF] Baillard | Robust 3D segmentation of anatomical structures with level sets[END_REF], [START_REF] Boukerroui | A multiparametric and multiresolution segmentation algorithm of 3-D ultrasonic data[END_REF], [START_REF] Paragios | Active shape models & segmentation of the left ventricle in echocardiography[END_REF]- [START_REF] Shao | Automatic 3D prostate surface detection from TRUS with level sets[END_REF], exponential [START_REF] Paragios | Active shape models & segmentation of the left ventricle in echocardiography[END_REF], Gamma [START_REF] Tao | Tunnelling descent: A new algorithm for active contour segmentation of ultrasound images[END_REF] and Beta [START_REF] Martin-Fernandez | An approach for contour detection of human kidneys from ultrasound images using Markov random fields and active contours[END_REF] distributions. • Intensity gradient (and higher derivatives): The motivation for using intensity gradient as a feature comes from the computer vision literature where based on a photometric model, high intensity gradients or equivalently intensity step changes/discontinuities in intensity are frequently associated with edges of objects. In ultrasound segmentation it is therefore appropriate to use intensity gradient as a segmentation constraint if the goal is to find acoustic (impedance) discontinuities. Strictly speaking one is also assuming that there is an approximately constant intensity on either side of the boundary (ie speckle has low amplitude). It has proved a popular constraint with or without prior speckle reduction, for example, in [START_REF] Herlin | Features extraction and analysis-methods for sequences of ultrasound images[END_REF], [START_REF] Chalana | A multiple active contour model for cardiac boundary detection on echocardiographic sequences[END_REF], [START_REF] Angelini | LV volume quantification via spatiotemporal analysis of real-time 3-D echocardiography[END_REF], [START_REF] Montagnat | Anisotropic filtering for model-based segmentation of 4D cylindrical echocardiographic images[END_REF], [START_REF] Sahiner | Computerized characterization of breast masses on threedimensional ultrasound volumes[END_REF] [49], [START_REF] Gong | Parametric shape modeling using deformable superellipses for prostate segmentation[END_REF], [START_REF] Knoll | Outlining of the prostate using snakes with shape restrictions based on the wavelet transform[END_REF], [START_REF] Ladak | Prostate boundary segmentation from 2D ultrasound images[END_REF], [START_REF] Hu | Prostate boundary segmentation from 3D ultrasound images[END_REF]- [START_REF] Pathak | Edge-guided boundary delineation in prostate ultrasound images[END_REF], [START_REF] Martin-Fernandez | An approach for contour detection of human kidneys from ultrasound images using Markov random fields and active contours[END_REF], [START_REF] Chen | A dual-snake model of high penetrability for ultrasound image boundary extraction[END_REF], [START_REF] Chen | Cell-based dual snake model: A new approach to extracting highly winding boundaries in the ultrasound images[END_REF], [START_REF] Sonka | Segmentation of intravascular ultrasound images: A knowledge-based approach[END_REF], [START_REF] Takagi | Automated contour detection for highfrequency intravascular ultrasound imaging: a technique with blood noise reduction for edge enhancement[END_REF], [START_REF] Shekhar | Three-dimensional segmentation of luminal and adventitial borders in serial intravascular ultrasound images[END_REF]- [START_REF] Klingensmith | Validation of an automated system for luminal and medial-adventitial border detection in three-dimensional intravascular ultrasound[END_REF], [START_REF] Sarty | Semiautomated segmentation of ovarian follicular ultrasound images using a knowledge-based algorithm[END_REF], [START_REF] Krivanek | Ovarian ultrasound image analysis: Follicle segmentation[END_REF]. Radial gradient estimation was also used in [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF], [START_REF] Abolmaesumi | An interacting multiple model probabilistic data association filter for cavity boundary extraction from ultrasound images[END_REF], [START_REF] Abolmaesumi | Segmentation of prostate contours from ultrasound images[END_REF], [START_REF] Ravhon | Validation of ultrasonic image boundary recognition in abdominal aortic aneurysm[END_REF], [START_REF] Romeny | Computer assisted human follicle analysis for fertility prospects with 3D ultrasound[END_REF]. To reduce the effect of speckle before gradient estimation, median filtering [START_REF] Potočnik | Automated analysis of a sequence of ovarian ultrasound images. Part I: segmentation of single 2D images[END_REF],

morphological smoothing [START_REF] Mishra | A GA based approach for boundary detection of left ventricle with echocardiographic image sequences[END_REF], speckle reduction [START_REF] Gong | Parametric shape modeling using deformable superellipses for prostate segmentation[END_REF], [START_REF] Pathak | Edge-guided boundary delineation in prostate ultrasound images[END_REF], [START_REF] Yu | Segmentation of the prostate from suprapubic ultrasound images[END_REF], [START_REF]Edge detection in ultrasound imagery using instantaneous coefficient of variation[END_REF], coarse-to-fine optimization [START_REF] Boukerroui | Segmentation of ultrasound images -multiresolution 2D and 3D algorithm based on global and local statistics[END_REF], [START_REF] Lin | Combinative multi-scale level set framework for echocardiographic image segmentation[END_REF], [START_REF] Ashton | Multiple resolution bayesian segmentation of ultrasound images[END_REF], [START_REF] Boukerroui | A multiparametric and multiresolution segmentation algorithm of 3-D ultrasonic data[END_REF], thresholding based adaptive smoothing [START_REF] Krivanek | Ovarian ultrasound image analysis: Follicle segmentation[END_REF], [START_REF] Potočnik | Automated analysis of a sequence of ovarian ultrasound images. Part I: segmentation of single 2D images[END_REF] and other prefiltering techniques [START_REF] Sahiner | Computerized characterization of breast masses on threedimensional ultrasound volumes[END_REF], [START_REF] Aarnink | A practical clinical method for contour determination in ultrasonographic prostate images[END_REF], [START_REF] Takagi | Automated contour detection for highfrequency intravascular ultrasound imaging: a technique with blood noise reduction for edge enhancement[END_REF], [START_REF] Bouma | Automated lumen definition from 30 MHz intravascular ultrasound images[END_REF] have been used. Gradient based level set algorithms have also been the most popular [START_REF] Mishra | A GA based approach for boundary detection of left ventricle with echocardiographic image sequences[END_REF], [START_REF] Yan | Applying improved fast marching method to endocardial boundary detection in echocardiographic images[END_REF], [START_REF] Chen | Using prior shapes in geometric active contours in a variational framework[END_REF], [START_REF] Corsi | Left ventricular volume estimation for real-time three-dimensional echocardiography[END_REF], [START_REF] Sarti | Subjective surfaces: A method for completing missing boundaries[END_REF], [START_REF] Mikula | Handbook of Medical Image Analysis: Segmentation and Registration Models[END_REF], [START_REF] Corsaro | Semi-implicit covolume method in 3D image segmentation[END_REF] and only few have used region information [START_REF] Sarti | Maximum likelihood segmentation of ultrasound images with rayleigh distribution[END_REF], [START_REF] Yu | Segmentation of the prostate from suprapubic ultrasound images[END_REF], [START_REF] Cardinal | Intravascular ultrasound image segmentation: A fastmarching method[END_REF], [START_REF] Baillard | Robust 3D segmentation of anatomical structures with level sets[END_REF], [START_REF] Shao | Automatic 3D prostate surface detection from TRUS with level sets[END_REF] or a combination of both [START_REF] Lin | Combinative multi-scale level set framework for echocardiographic image segmentation[END_REF], [START_REF] Chen | A coupled minimization problem for medical image segmentation with priors[END_REF], [START_REF]Using prior shape and intensity profile in medical image segmentation[END_REF], [START_REF] Ledesma-Carbayo | Spatio-temporal non-rigid registration for ultrasound cardiac motion estimation[END_REF].

The limitations of the choice relate to the anisotropy of image acquisition, as the strength of a boundary response/edge is a function of the relative orientation of the transducer to the boundary, signal attenuation with depth reduces the strength of boundaries for deep objects, and speckle gives a strong intensity gradient response. Thus intensity gradient information has only really proved a useful constraint for analyzing good quality clinical images and used in conjunction with other features as mentioned above as well as in [START_REF] Chang | Segmentation of breast tumor in three-dimensional ultrasound images using three-dimensional discrete active contour model[END_REF], [START_REF] Yu | Segmentation of the prostate from suprapubic ultrasound images[END_REF], [START_REF] Levienaise-Obadia | Adaptive segmentation of ultrasound images[END_REF], [START_REF] Martin-Fernandez | An approach for contour detection of human kidneys from ultrasound images using Markov random fields and active contours[END_REF], [START_REF] Pardo | Discriminant snakes for 3D reconstruction of anatomical organs[END_REF], [START_REF] Potočnik | Automated analysis of a sequence of ovarian ultrasound images. Part I: segmentation of single 2D images[END_REF]. • Phase: The local phase provides an alternative way to characterize structure in an image which has been used for example in [START_REF] Mulet-Parada | 2D+T acoustic boundary detection in echocardiography[END_REF]- [START_REF] Sanchez-Ortiz | Automated 3D echocardiography analysis compared with manual delineations and SPECT MUGA[END_REF]. Measuring local phase, or rather phase congruency over spatial scales, provides a way to characterize different intensity features in terms of shape of the intensity profile rather than the intensity derivative magnitude; for example a step edge has a phase value of 0 or π, an intensity ridge has a phase of π/2. Thus, phase has been suggested as a more robust feature for acoustic boundary detection. Speckle also has a phase signature so appropriate spatial scales have to be selected. Generally, phase is estimated using quadrature filter banks. Thus there is a link between phase-based methods and other wavelet methods. • Similarity measures: Within the current discussion of Bmode ultrasound segmentation, similarity measures have been used to provide a displacement estimate as a constraint in spatio-temporal analysis rather than a standalone segmentation feature (c.f. use in image registration). The literature on ultrasound-ultrasound similarity measures is very small. The sum-of-squares difference (SSD) is used, for example in [START_REF] Mikic | Segmentation and tracking in echocardiographic sequences: Active contours guided by optical flow estimates[END_REF]. Cohen and Dinstein developed an ultrasound-specific similarity measure based on the assumption that both image blocks contained speckle that could be modeled as a Rayleigh distributed multiplicative corruption [START_REF] Cohen | New maximum likelihood motion estimation schemes for noisy ultrasound images[END_REF]. This method was shown to be better than the SSD or normalized cross-correlation in [START_REF] Boukerroui | Velocity estimation in ultrasound images: a block-matching approach[END_REF].

A variant of Mutual information that uses neighborhood information, the Mutual Information of Image Geometry (MIIG) was proposed in [START_REF]Using prior shape and intensity profile in medical image segmentation[END_REF]. The MIIG similarity was used for the estimation of an average intensity profile from a training set. This intensity prior was taken into account in the segmentation by means of the same similarity measure. • Texture measures: Although, ultrasound texture patterns are intrinsically dependent on the imaging system, they characterize the microstructure of the tissues being im-aged. The distribution (coherent and/or random) of scatterers and their relative sizes to the wavelength of the incident ultrasound pulse, produces different texture patterns. As noted in the section on Speckle, ultrasound speckle statistics/measures have been successfully utilized to discriminate breast masses ( [START_REF] Shankar | Use of the Kdistribution for classification of breast masses[END_REF]- [START_REF] Shankar | Classification of ultrasonic b-mode images of breast masses using Nakagami distribution[END_REF], [START_REF] Shankar | The use of the compound probability density function in ultrasonic tissue characterization[END_REF], [START_REF] Shankar | Classification of breast masses in ultrasonic B scans using Nakagami and K-distributions[END_REF] and references therein) and Bouhlel et al. [START_REF] Bouhlel | New Markov random field model based on K-distribution for textured ultrasound image[END_REF] have recently proposed a MRF for textured ultrasound images based on the K-distribution and on a multiplicative degradation model. However, general statistical texture analysis methods have been used with some success in this field. One potential advantage of texture analysis is that most texture measures are based on statistical patterns of the intensities and thus independent of the physics of the imaging system (see for example [START_REF] Tuceryan | The Handbook of Pattern Recognition and Computer Vision[END_REF], [START_REF] Rolland | Analysis of texture in medical imaging. Review of the literature[END_REF]). Thus, in segmentation, where the aim is often to characterize the imaged object rather than necessarily characterize its true physical properties, texture analysis methods have proved successful. Texture characterization has been used. for instance, in classification and discrimination studies of liver tissues [START_REF] Nicholas | Tissue characterization from ultrasound B-scan data[END_REF]- [START_REF] Lee | Ultrasonic liver tissues classification by fractal feature vector based on M-band wavelet transform[END_REF] breast masses [START_REF] Horsch | Automatic segmentation of breast lesions on ultrasound[END_REF]- [START_REF] Chen | Diagnosis of breast tumors with sonographic texture analysis using wavelet transform and neural networks[END_REF], [START_REF] Chang | Support Vector Machines for diagnosis of Breast tumors on US images[END_REF], [START_REF] Sahiner | Computerized characterization of breast masses on threedimensional ultrasound volumes[END_REF], [START_REF] Shankar | Classification of breast masses in ultrasonic B scans using Nakagami and K-distributions[END_REF], [START_REF] Garra | Improving the distinction between benign and malignant breast lesions: The value of sonographic texture analysis[END_REF]- [START_REF] Chen | 3-D ultrasound texture classification using run difference matrix[END_REF], carotid plaques [START_REF] Wilhjelm | Quantitative analysis of ultrasound B-mode images of carotid atherosclerotic plaque: correlation with visual classification and histological examination[END_REF], [START_REF] Christodoulou | Texture-based classification of atherosclerotic carotid plaques[END_REF], coronary plaques [START_REF] Zhang | Tissue characterization in intravascular ultrasound images[END_REF], [START_REF] Vince | Comparison of texture analysis methods for the characterization of coronary plaques in intravascular ultrasound images[END_REF], myocardial tissues [START_REF] Kerut | Echocardiographic texture analysis using the wavelet transform: differentiation of early heart muscle disease[END_REF], [START_REF] Mojsilovic | Wavelet image extension for analysis and classification of infarcted myocardial tissue[END_REF], the thyroid gland [START_REF] Mailloux | Texture analysis of ultrasound B-mode images by segmentation[END_REF], [START_REF] Smutek | Image texture analysis of sonograms in chronic inflammations of thyroid gland[END_REF], ovarian masses [START_REF] Zimmer | An automatic approach for morphological analysis and malignancy evaluation of ovarian masses using B-scans[END_REF] and prostate tissues [START_REF] Basset | Texture analysis of ultrasonic images of the prostate by means of co-occurrence matrices[END_REF], [START_REF] Scheipers | Ultrasonic multifeature tissue characterization for prostate diagnostics[END_REF]. In general, good classification rates have been obtained when large regions-of-interest are investigated (64 × 64 or 32 × 32 pixels). They are not as good with smaller sample sizes, i.e., when the pathological area is very small. Texturebased segmentation algorithms have also been proposed in the ultrasound literature and used in various clinical application domains such as: ovaries [START_REF] Muzzolini | Multiresolution texture segmentation with application to diagnostic ultrasound images[END_REF], IVUS [START_REF] Mojsilovic | Automatic segmentation of intravascular ultrasound images: A texturebased approach[END_REF], prostate [START_REF] Richard | Automated texture-based segmentation of ultrasound images of the prostate[END_REF], [START_REF] Shen | Segmentation of prostate boundaries from ultrasound images using statistical shape model[END_REF], [START_REF] Zhan | Automated segmentation of 3D US prostate images using statistical texture-based matching method[END_REF], breast [START_REF] Madabhushi | Combining low-, high-level and empirical domain knowledge for automated segmentation of ultrasonic breast lesions[END_REF], [START_REF] Huang | Watershed segmentation for breast tumor in 2-D sonography[END_REF], [START_REF] Boukerroui | Multiresolution texture based adaptive clustering algorithm for breast lesion segmentation[END_REF], liver [START_REF] Chen | A textural approach based on Gabor functions for texture edge detection in ultrasound images[END_REF], [START_REF] Lee | Unsupervised segmentation of ultrasonic liver images by multiresolution fractal feature vector[END_REF], echocardiography [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF] and kidney [START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF], [START_REF] Dokur | Segmentation of ultrasound images by using a hybrid neural network[END_REF]. Literally hundreds of texture measures have been used in the literature (statistical, structural, model based, transform based/spectral). Texture features derived from Haralick's co-occurrence matrices, although computationally expensive to compute, have performed well in a number of applications and have been used in combination with other measures or for comparison in a number of studies [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF], [START_REF] Chen | Diagnosis of breast tumors with sonographic texture analysis using wavelet transform and neural networks[END_REF], [START_REF] Sahiner | Computerized characterization of breast masses on threedimensional ultrasound volumes[END_REF], [START_REF] Boukerroui | A multiparametric and multiresolution segmentation algorithm of 3-D ultrasonic data[END_REF], [START_REF] Nicholas | Tissue characterization from ultrasound B-scan data[END_REF]- [START_REF] Yeh | Liver fibrosis grade cation with B-mode ultrasound[END_REF], [START_REF] Gaitini | Feasibility study of ultrasonic fatty liver biopsy: Texture vs. attenuation and backscatter[END_REF], [START_REF] Garra | Improving the distinction between benign and malignant breast lesions: The value of sonographic texture analysis[END_REF]- [START_REF] Kuo | Computer-aided diagnosis of breast tumors with different US systems[END_REF], [START_REF] Chen | Use of the bootstrap technique with small training sets for computer-aided diagnosis in breast ultrasound[END_REF], [START_REF] Wilhjelm | Quantitative analysis of ultrasound B-mode images of carotid atherosclerotic plaque: correlation with visual classification and histological examination[END_REF], [START_REF] Zhang | Tissue characterization in intravascular ultrasound images[END_REF], [START_REF] Vince | Comparison of texture analysis methods for the characterization of coronary plaques in intravascular ultrasound images[END_REF], [START_REF] Smutek | Image texture analysis of sonograms in chronic inflammations of thyroid gland[END_REF], [START_REF] Basset | Texture analysis of ultrasonic images of the prostate by means of co-occurrence matrices[END_REF], [START_REF] Boukerroui | Multiresolution texture based adaptive clustering algorithm for breast lesion segmentation[END_REF]- [START_REF] Thijssen | Correlations between acoustic and texture parameters from RF and B-mode liver echograms[END_REF].

Several authors have also emphasized the importance of a multiple resolution characterization (see eg. [START_REF] Muzzolini | Multiresolution texture segmentation with application to diagnostic ultrasound images[END_REF], [START_REF] Wu | Texture features for classification of ultrasonic liver images[END_REF]) or the use of an intelligent parameters adjustment to the ultrasound resolution [START_REF] Kuo | Computer-aided diagnosis of breast tumors with different US systems[END_REF].

2) Shape: Edge cues and region information are often not sufficient for a reliable and accurate segmentation. In this case shape constraints are often found to effectively improve results. Because of attenuation, shadowing artefact and speckle, shape constraints have proved popular and successful recently in B-mode ultrasound image segmentation [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF], [START_REF] Mignotte | Endocardial boundary estimation and tracking in echocardiographic images using deformable template and markov random fields[END_REF], [START_REF] Chen | Using prior shapes in geometric active contours in a variational framework[END_REF]- [START_REF]Using prior shape and intensity profile in medical image segmentation[END_REF], [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF], [START_REF] Gong | Parametric shape modeling using deformable superellipses for prostate segmentation[END_REF], [START_REF] Shen | Segmentation of prostate boundaries from ultrasound images using statistical shape model[END_REF], [START_REF] Shen | Optimized prostate biopsy via a statistical atlas of cancer spatial distribution[END_REF], [START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF], [START_REF] Paragios | Active shape models & segmentation of the left ventricle in echocardiography[END_REF], [START_REF] Fan | An adaptive template-matching method and its application to the boundary detection of brachial artery ultrasound scans[END_REF].

Shape information is embedded in segmentation methods in several ways. Probably the most classical shape constraint involves boundary regularization, say as in the choice of the internal terms in an active contour. A second way to impose a shape constraint is by using a parametric shape and a preferred shape can be imposed, for example in a probabilistic framework, using the learned distributions of the shape parameter over a set of training examples. An example of such a shape constraint used in ultrasound segmentation is in the work of Gong et al. [START_REF] Gong | Parametric shape modeling using deformable superellipses for prostate segmentation[END_REF]. A close variant is to impose a shape constraint by restricting the space of possible shapes to a predefined model shape with a set of transformations (global and/or local). This representation does not impose a parametrization of the shape and does not require training. In this case the constraint is imposed on the transformation set, and global and local transformations may be penalized differently. See [START_REF] Mignotte | Endocardial boundary estimation and tracking in echocardiographic images using deformable template and markov random fields[END_REF] for a good example of this. A widely and acknowledged alternative way to employ a shape constraint is via the use of a Point Distribution Model (PDM) [START_REF] Cootes | Active shape models-their training and application[END_REF]. A PDM describes the average shape and the most characteristic shape variations of a set of training samples obtained using Principal Component Analysis (PCA). The use of a PDM in segmentation is known as an Active Shape Model (ASM). As in the previous model, no parametrization of the shape is needed. However, the set of possible shapes is defined by the principal modes of shape variation in the training set. In general, prior shape knowledge is modeled in a probabilistic framework. For instance, a Gaussian distribution [START_REF] Cootes | Active shape models-their training and application[END_REF] or a mixture model [START_REF] Cootes | A mixture model for representing shape variation[END_REF]. Although ASMs are suitable to describe free-form deformations, a point-to-point correspondence of the shape is necessary for the estimation of the average shape and its mode of variations. An extension to include intensity prior information was proposed, called the Active Appearance Model (AAM) in [START_REF] Cootes | Active appearance models[END_REF]. Another extension is to include motion information as in [START_REF] Hamarneh | Deformable spatio-temporal shape models: extending active shape models to 2D+time[END_REF] and the Active Appearance Motion Model (AAMM) in [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF]. An extension of the AAM to 3D was proposed by Bosch et al. [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF]. An advantage of the AAM approach is that it attempts to learn both the geometric shape and the boundary model of an object. A limitation is that it assumes clinical images from different subjects have similar appearance. This can be difficult to achieve on routine clinical images due to natural subject-to-subject tissue property variations and operator-to-operator variation in acquisition. This led Bosch et al. to propose a nonlinear normalization step to reduce these effects.

To overcome the limitation of point-to-point correspondence in the learning stage, Leventon et al. proposed to use a level set representation of the shape [START_REF] Leventon | Statistical shape influence in geodesic active contours[END_REF]. An example of the use of such a shape model on US images is provided by Xie et al. [START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF]. In order to allow more flexibility for the shape variation, (i.e avoid the Gaussian assumption mentioned above), a level set variant of the ASM, the Implicit ASM was proposed in [START_REF] Rousson | Implicit active shape models for 3D segmentation in MR imaging[END_REF] and applied to echocardiography in [START_REF] Paragios | Active shape models & segmentation of the left ventricle in echocardiography[END_REF]. In this model, the evolving surface is not restricted within the modeled space, but only attracted to it using a distance function. To allow more flexibility, Chen et al. proposed, again in a level set framework, to constrain the solution using a distance function to the learned average shape (i.e PCA is not used to learn the principal mode of variation) [START_REF] Chen | Using prior shapes in geometric active contours in a variational framework[END_REF]. Prior intensity information was also taken into account in a similar way as in their recent work [START_REF] Chen | A coupled minimization problem for medical image segmentation with priors[END_REF], [START_REF]Using prior shape and intensity profile in medical image segmentation[END_REF].

Finally, we would like to emphasise the fact that a shape model is only as good as the training samples from which it was built and the chosen shape-space model framework (eg. linear, non linear). An important issue, still open to our knowledge, concerns the aptitude of shape constraints to handle disease cases.

3) Time: Ultrasound is often employed because it is realtime and thus the data available for segmentation is a temporal image sequence rather than a static frame. Therefore it is sometimes useful to employ temporal priors in segmentation i.e. consider segmentation as a spatio-temporal process. The most obvious example application is cardiac segmentation where the object moves in a periodic pattern. Respiration can also introduce (unwanted) motion. One way to handle motion is to register the data prior to segmentation. The other approach is to handle motion (correction or estimation) as part of the segmentation process directly via a temporal prior.

Temporal priors or models can take a number of forms. The choice depends on how well the motion can be parameterized and whether computation speed is a determining factor in designing the segmentation method. We illustrate this with some example methods below.

First temporal priors may be "weak" in the sense of insisting on global [START_REF] Chalana | A multiple active contour model for cardiac boundary detection on echocardiographic sequences[END_REF] or local [START_REF] Dias | Wall position and thickness estimation from sequences of echocardiograms images[END_REF], [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF], [START_REF] Sarti | Nonlinear multiscale analysis of 3D echocardiographic sequences[END_REF] temporal coherence or smoothness. Another simple way to invoke a weak notion of temporal coherence is to propagate a segmentation result from one frame to initialize the segmentation in the next frame [START_REF] Mishra | A GA based approach for boundary detection of left ventricle with echocardiographic image sequences[END_REF]; a variant of this is to adjust the position of the initializing segmentation by using image velocity estimates [START_REF] Mikic | Segmentation and tracking in echocardiographic sequences: Active contours guided by optical flow estimates[END_REF].

A constant velocity model or other simple parametric model might be assumed to allow propagation of a result from one frame to initialize segmentation in another. A variant of this is to use a Kalman filter to blend a spatio-temporal model with feature measurements as in [START_REF] Jacob | Quantitative regional analysis of myocardial wall motion[END_REF], [START_REF] Jacob | A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography[END_REF]. If the form of the parametric model is not obvious or motion patientspecific then the temporal model may be estimated from data. For example some authors have explicitly incorporated local estimates of image velocity [START_REF] Mikic | Segmentation and tracking in echocardiographic sequences: Active contours guided by optical flow estimates[END_REF], [START_REF] Herlin | Features extraction and analysis-methods for sequences of ultrasound images[END_REF], [START_REF] Mulet-Parada | 2D+T acoustic boundary detection in echocardiography[END_REF], [START_REF] Setarehdan | Automatic cardiac LV boundary detection and tracking using hybrid fuzzy temporal and fuzzy multiscale edge detection[END_REF]. Malassiotis et al use a temporal learning-filtering procedure to refine the output of active contour tracking [START_REF] Malassiotis | Tracking the left ventricle in echocardiographic images by learning heart dynamics[END_REF]. In the work of Mitchell et al [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF] the temporal model is learned from manually traced segmentations.

Other authors have explicitly added time as an extra dimension to an N-dimensional segmentation problem [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF], [START_REF] Boukerroui | Segmentation of ultrasound images -multiresolution 2D and 3D algorithm based on global and local statistics[END_REF], [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF], [START_REF] Montagnat | Anisotropic filtering for model-based segmentation of 4D cylindrical echocardiographic images[END_REF], [START_REF] Boukerroui | A multiparametric and multiresolution segmentation algorithm of 3-D ultrasonic data[END_REF]. Only a few authors have considered solving the spatio-temporal segmentation problem where the space parameters are treated differently from the time parameter [START_REF] Sarti | Nonlinear multiscale analysis of 3D echocardiographic sequences[END_REF], [START_REF] Mikula | Morphological image sequence processing[END_REF].

V. CONCLUSION To conclude this review we have selected 10 influential papers in the ultrasound segmentation literature. These are not claimed to be the 'best' but have been selected based on criteria of the ultrasound-specific model they have employed and/or whether evaluation has been performed on a reasonable number of clinical datasets. In no particular order of importance:

• [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF] [START_REF] Binder | Artificial neural networks and spatial temporal contour linking for automated endocardial contour detection on echocardiograms: A novel approach to determine left ventricular contractile function[END_REF]: although not a recent paper, this is a good illustration of the application of neural networks and learning to (cardiac) ultrasound image segmentation, which is also one of the few ultrasound segmentation methods that has been explicitly tested on a reasonable number of clinical images of varying quality. • Dutt and Greenleaf (1996) [START_REF] Dutt | Statistics of the log-compressed echo envelope[END_REF]: statistics of logcompressed echo envelope using an underlying Kdistribution model before compression. Most ultrasound data comes from machines where log-compression has been done. Hence it is important that the model takes into account this non-linear transformation (speckle prior). • Mulet-Parada (2000) [START_REF] Mulet-Parada | 2D+T acoustic boundary detection in echocardiography[END_REF]: proposes an intensity-invariant image feature (local phase) for acoustic boundary and displacement estimation as an alternative to intensity derivatives (image prior). • [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF] [START_REF] Friedland | Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated anneal[END_REF]: an early contribution viewing ultrasound segmentation as a spatio-temporal problem;

(cardiac) boundary detection is solved as a minimization of an energy function with a Markovian spatio-temporal regularization.

• Mikula et al. (2004) [START_REF] Mikula | Handbook of Medical Image Analysis: Segmentation and Registration Models[END_REF]: and references therein. Presents a level set method for segmenting objects with missing boundaries (as frequently observed in ultrasound images). • [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF] and [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF] [START_REF] Bosch | Automatic segmentation of echocardiographic sequences by active appearance motion models[END_REF], [START_REF] Mitchell | 3-D active appearance models: Segmentation of cardiac MR and ultrasound images[END_REF]:

concerns the application and clinical validation of active appearance and active appearance motion modelling (intensity, shape and motion priors). • [START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF] [START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF]: presents an ultrasound segmentation method that combines texture and shape prior information in a level set framework; (texture and shape prior). • Mignotte et al. (2001) [START_REF] Mignotte | Endocardial boundary estimation and tracking in echocardiographic images using deformable template and markov random fields[END_REF]: a good illustration of employing an imaging physics prior (a shifted Rayleigh distribution to model gray level statistics), and a shape prior (a deformable template) to solve boundary estimation via a multiscale minimization. (gray level distribution and shape prior, probabilistic framework). • [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF] [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF]: concerns the evaluation of boundary segmentation methods w.r.t. multiple expert delineations. In ultrasound segmentation this is particularly important as there can be uncertainty in manual delineation. This is a quite widely employed method in echocardiographic image segmentation and used elsewhere in the medical image segmentation evaluation literature. • Abolmaesumi and Sirouspour (2004) [START_REF] Abolmaesumi | An interacting multiple model probabilistic data association filter for cavity boundary extraction from ultrasound images[END_REF]: presents a 2D contour segmentation approach where two contour models are combined in order to achieve smooth results and allow rapid changes in the boundary. This could be seen as an adaptive regularization of the boundaries because the models have different noise/smoothness tradeoff.

Some final comments relating to open issues in this area. In this review we have presented a snapshot view of the stateof-the-art in B-mode ultrasound image segmentation as seen in the literature in the year 2005. First, (section III) we have seen the need for more effort on segmentation validation to better understand the strengths and limitations of methods both through comparison with other methods in the literature and on larger (ideally standard) databases of data. This is important to encourage the adoption of methods in clinical practice. Even though ultrasound imaging is so widely used in clinical diagnosis and image-guided interventions, the field is far behind other areas of clinical image analysis, such as X-ray mammography or (rigid) image registration in this respect. Second, in Section IV we have argued that successful ultrasound segmentation methods should utilize geometric, temporal, intensity and imaging physics priors. The choice of which constraints to use is in part application specific, but many constraints hold for generic ultrasound image segmentation. Encouragingly, evidence in the recent literature points towards doing this in a more disciplined way, a trend which we hope this review will encourage other researchers to follow.

  Fig. 1. Examples of echocardiographic image segmentation: Short axis images from Dias and Leitão (1996) [11] (a) Mikic et al. (1998) [12] (b); long axis images from Bosch et al. (2002) [13] (c).

Fig. 2 .

 2 Fig. 2.Examples breast ultrasound from Madabhushi and Metaxas (2003)[START_REF] Madabhushi | Combining low-, high-level and empirical domain knowledge for automated segmentation of ultrasonic breast lesions[END_REF]. From left to right: Original image, contrast-enhanced version, detected boundary points and the output of deformable model.

Fig. 3 .

 3 Fig. 3. Segmentation example of transverse US images of the prostate from Gong et al. (2004) [88]; The solid contours are the ground truth established by averaging five experts manual outlinings and the dotted contour are the computer generated boundaries.

Fig. 4 .

 4 Fig. 4. Segmentation example of IVUS images (vascular disease); (a) example taken form Klingensmith et al. (2000) [132] showing the initial contour and the final borders; (b) two segmentation examples of the automatic algorithm by Brusseau et al. (2004) [136].

  ; revised April 5, 2006. Asterisk indicates corresponding author.

* J. Alison Noble is with the Department of Engineering Science University of Oxford, Oxford OX1 3PJ, UK. D. Boukerroui is with HEUDIASYC, UMR CNRS 6599 Université de Technologie de Compiègne, BP 20529 -60205 Compiègne Cedex, France

TABLE I VALIDATION

 I OF CARDIAC ULTRASOUND SEGMENTATION METHODS. THE METHODS ARE FIRST SORTED BY IMAGE DIMENSION (2D, 2D+T,3D,3D+T), THEN BY DEGREE OF AUTOMATION (NON-AUTOMATIC, AUTOMATIC) AND FINALLY BY YEAR OF APPEARANCE (ASCENDING ORDER).

	Reference	Year	Modality	Seg.	Ad hoc	Auto-	Interaction type	Evaluation	Performance			Validation/Illustration
				criteria	parameters -mation			Measure	Value	Number	Type
	Friedland [29]	1989	LAX 2D	C	(+/-)	(-)		QL	-	-	1(11)	-
	Binder [28]	1999	SAX	R	(+)	(-)	ROI, Manual training	QN(m)	Area correlation Bland-Altman Area (mm 2 ) Ejection Fraction (EF)	0.99 -0.69 ± 1.7 0.93	38(76)	OB(2), CR(NMR)
									Bland-Altman EF (%)	-7.01 ± 6.11
	Lin [20]	2003	SAX 2D, ROT3D C/R	(+)	(-)	Seed point	QN(m)	Contour AD(end)	1.643 ± 0.503 pxl	4(24)	OB(3)
	Mishra [14]	2003	SAX	C	(+)	(-)	ROI	QN(m)	Area correlation Contour MAD	0.92 Plotted frame-by-frame	1(20)	OB(2)
	Yan [19]	2003	SAX, 4C	C	(+)	(-)	Seed point	QL	-	-	-	-
	Chen [22]	2004	LAX 2C	C	(+/-)	(-)	Ellipse initial Manual Training	QN(m)	Contour AD (end)	1.97 pxl	3(3)	OB(1)
	Mignotte [15]	2001	SAX	C/R	(+)	(+)	-	-	-	-	-	CL
	Mignotte [17]	2001	SAX	C/R	(+)	(+)		QN(m)	Rate correct. pxl classification	79%	1(50)	OB(2)
									Contour AD (end)	3.61 ± 1.68 mm
	Chalana [36]	1996	SAX	2D+T C	(+/-)	(-)	ROI, Manual initial. of epi. QN(m)	Area correlation (end)	0.91	44(88)	OB(4)
									Contour AD (epi)	2.80 ± 1.28 mm
									Area correlation (epi)	0.95
	Kucera [37]	1997	2D+T SAX 3D TEE	2D+T C	(+/-)	(-)	Manual Initial. (circle) Correction	QN(m)	Contour MAD (end)	0.4 mm	19(38), 1(9)	OB(5)
			SAX1						Contour AD Area correlation	1.15 ± 0.2 mm 0.99		OB(3)
	Mikic [12]	1998	SAX2	2D+T C	(+)	(-)	ROI, Manual initial	QN(m)	Contour AD Area correlation	1.29 ± 0.35 mm 0.95	1	OB(4)
			SAX3						Contour AD Area correlation	2.07 ± 0.40 mm 0.99		OB(5)
	Setarehdan [41]	1999	SAX	2D+T C	(+)	(+)		QN(m)	RMS radial based distance (end) Area correlation (end)	3.5 mm 0.96	1(25)	OB(1)
									RMS radial based distance (epi) Area correlation (epi)	1.84 mm 0.95	1(25)	OB(1)
	Dias [11]	1996	SAX	2D+T C	(+)	(+)		QN(m)	Ejection Fraction	0.68/0.7	1	OB(1)
	Mulet-Parada [31]	2000	SAX, 2C	2D+T C	(+)	(+)		QL	-	-	-	-
	Jacob [45]	2001	LAX 2C	2D+T C	(+)	(-)	Initialization	QN(c)			4 (rest and peak stress) OB(2)
	Bosch [13]	2002	4C	2D+T C	(-)	(+)	Manual Training (AAM)	QN(m)	corresp. point MAbD (end) Area correlation coeff.	3.35 ± 1.22 mm 0.84	64(1024)	OB(1)
	Mitchell [48]	2002	4C	2D+T C	(+)	(+)	Manual Training (AAM)	QN(m)	corresp. point MAbD (end) Area correlation coeff.	3.90 ± 1.38 mm 0.79	64(1024)	OB(1), CL Bosch[13]
	Boukerroui [18]	2003	LAX	2D+T R	(+/-)	(+)	-	QN(m)	Area error (pxl 2 ) Contour AD	-7.59 ± 6.71 % 5.96 ± 2.20 pxl	1(106)	OB(1), CL Xiao [26]
	Coppini [49]	1995	3D freehand	C	(+)	(+)		QN(m)	volume correlation coeff	> 0.98	3	OB(1)
									(underestimate manual volume)
	Corsi [61]	2002	RT3D(1)	S	(+)	(-)	Initialization	QN(c) QN(c)	Ejection fraction Bland-Altman volume	y=0.90x+4.16 r=0.87 -15.58 ± 20.55 ml	20(40)	CR=MRI
	Song [50]	2002	3D freehand	S	(+)	(-)	Initialization	QN(m)	Surface projection dist. Err (epi) Surface projection dist. Err (end)	3.2 ± 0.85 mm 2.6 ± 0.78 mm	25(25)	OB(1)
	Wolf [51]	2002	3D TEE	C	(-)	(-)	Seed point	QN(m)	Contour MAvD (end)	3.44 ± 1.18 mm	20

TABLE II VALIDATION

 II OF BREAST ULTRASOUND MASS DETECTION AND CLASSIFICATION METHODS: CR(B/A)=BIOPSY/ASPIRATION; THE METHODS ARE FIRST SORTED BY IMAGE DIMENSION (2D,3D), THEN BY DEGREE OF AUTOMATION (NON-AUTOMATIC, AUTOMATIC) AND FINALLY BY YEAR OF APPEARANCE (ASCENDING ORDER).

	Reference	Year	Modality	Seg.	Ad hoc	Auto-	Interaction type	Evaluation	Performance		Validation/Illustration
				criteria parameters -mation			Measure	Value	Number	Type
	Chen [79]	2002	2D B-mode	R	(+)	(-)	ROI specified	QN(c)	Sensitivity Specificity	98.77 81.37	242(242)	CR(B)
	Huang [80]	2004	2D B-mode	R/C	(-)	(-)	ROI specified	QN(m)	Match rate Precision ratio	94.66 % 81.70 %	20(20)	OB(1)
	Horsch [74], [75] 2001, 2002	2D B-mode	R	(+)	(+/-)	Lesion center	QN(m), QN(c)	Area overlap t = 0.4, detection correct	0.94 (auto) 0.97 (partial auto)	400(757)	OB(3), CR(B/A)
									ROC Az	0.87 (auto) 0.89 (partial auto)
	Drukker [77]	2002	2D B-mode	R/C	(+)	(+)	-	QN(m), QN(c)	Area overlap t = 0.4, detection correct ROC Az	0.75 0.84 %	400(757)	OB(3), CR(B/A)
	Xiao [26]	2002	2D B-mode	R	(-)	(+)	-	QL	-	-	-	-
	Madabhushi [73]	2003	2D B-mode	R/C	(+/-)	(+)	-	QN(m)	Mean boundary error Normalized true positive overlap	6.6 pxl 75.10 %	42(42)	OB(1)
	Drukker [78]	2004	2D B-mode	R/C	(+)	(+)	-	QN(c)	ROC (Actual lesions and FP detection) Az ROC Cancers from other detections Az	0.91 0.81	458(1740) CR(B)
	Chen [81]	2003	3D mechanical scan C	(+)	(-)	ROI on 3 2D frames	QN(m)	Average volume match rate	0.88	8(8)	OB(1)
	Chang [82]	2003	3D mechanical scan C	(+)	(-)	ROI specified	QN(m)	Average volume match rate	0.95	8(8)	OB(1)
	Sahiner [85]	2004	3D mechanical scan R/C	(+)	(-)	3D ellipsoid initialization	QN(m)	ROC Az	0.92	102	OB

TABLE III VALIDATION

 III OF PROSTATE ULTRASOUND SEGMENTATION METHODS. MAD=MEAN ABSOLUTE DIFFERENCE; MAXD = MAXIMUM DISTANCE. AD=AVERAGE DISTANCE; HD = HAUSSDORFF DISTANCE (SEE [102]). SEE LADAK ET AL. [97] FOR AREA SENSITIVITY AND ACCURACY. SEE TABLE II FOR THE SORTING RULES.

	Reference	Year	Modality	Seg.	Ad hoc	Auto-	Interaction type	Evaluation	Performance	Validation/Illustration
				criteria parameters -mation			Measure	Value	Number	Type
									Contour MAD	04.70 pxl (2.61 mm)
	Knoll [95]	1999 2D	C	(+)	(-)	-	QN(m)	Area difference	08.48 %	11(77)	OB(2)
									Area accuracy	78.06 %
									Contour MAD	04.4 ± 1.8 pxl
	Ladak [97]	2000 2D	C	(+)	(-)	Manual Initial. (4 points) Manual correction	QN(m)	Contour MaxD Area accuracy Area sensitivity	19.5 ± 7.8 pxl 90.1 ± 3.2 % 94.5 ± 2.7 %	19(117)	OB(1)
									Area correlation	0.993
									Contour AD	0.7 ± 0.4 mm
	Pathak [101]	2000 2D	C	(+/-)	(-)	Manual Initial. (1 points)	QN(m)	Contour HD	1.8 ± 1.0 mm	16(125)	OB(5)
									Boxplots are also shown
	Abolmaesumi [109], [110] 2004 2D	C	(+/-)	(-)	Seed point	QN(m)	Area overlap	98 %	6	OB(2)
	Chiu [100]	2004 2D	C	(+)	(-)	Manual Initial. (4 points) Manual correction	QN(m)	Contour MAD Contour MaxD Area difference	02.89 ± 0.88 pxl 10.73 ± 4.56 pxl 03.50 ± 2.40 %	6(114)	OB(1)
	Gong [88]	2004 2D	C	(+)	(-)	Manual Initial. (4 points) Manual Training	QN(m)	Contour AD Contour HD Boxplots are also shown	1.82 ± 1.44 mm 4.54 ± 2.93 mm	16(125)	OB(5)
									Contour RMS	01.16 ± 0.4 mm
	Yu [111]	2004 2D	C/R	(+/-)	(-)	ROI, Manual initial	QN(m)	Area accuracy	86.1 ± 5.6 %	6(27)	OB(3)
									Area sensitivity	92.3 ± 4.0 %
									Contour AD	3.20 ± 0.87 pxl
	Shen [103]	2003 2D	C/R	(+)	(+)	Manual training (ASM)	QN(m)	Area overlap error	3.98 ± 0.97 %	8	OB(1)
									Area difference	1.66 ± 1.68 %
	Hu [99]	2003 ROT3D	S	(+)	(-)	Manual Initial. (6 points) Manual correction	QN(m)	Contour MAD Contour MaxD Volume difference	1.19 ± 0.14 mm 7.01 ± 1.04 mm 7.16 ± 3.45 %	6(143)	OB(1) CL (Ladak [97])
									Contour AD	1.10 ± 0.16 vxl
	Zhan [105]	2004 3D	S/R	(+)	(+)	Manual Training	QN(m)	Volume overlap error	4.13 ± 0.55 %	6	OB(1)
									Volume difference	2.23 ± 1.19 %

TABLE IV VALIDATION

 IV OF IVUS ULTRASOUND SEGMENTATION METHODS. MAD = MEAN ABSOLUTE DIFFERENCE. WI= WILLIAM'S INDEX; CI= CONFIDENCE INTERVAL. AD=AVERAGE DISTANCE; HD = HAUSSDORFF DISTANCE (SEE [102]). RMS= ROOTS MEAN SQUARED ERROR; MAXD= MAXIMUM DISTANCE (PIXEL-TO-PIXEL). SEE TABLE II FOR THE SORTING RULES.

	Reference	Year	Modality	Seg.	Ad hoc	Auto-	Interaction type	Evaluation	Performance	Validation/Illustration
				criteria parameters -mation				Measure	Value	Number	Type
									Media	Contour MaxD Contour RMS	0.33 ± 0.07 mm 0.07 ± 0.02 mm
	Sonka [128]	1995	2D 30Mhz	C	(+)	(-)	ROI	QN(m)	Intima	Contour MaxD Contour RMS	0.33 ± 0.09 mm 0.08 ± 0.02 mm	38	OB(1)
									Lumina	Contour MaxD Contour RMS	0.33 ± 0.10 mm 0.09 ± 0.03 mm
	Takagi [129]	2000	2D 40Mhz	C	(+)	(-)	ROI	QN(m)	Media	Area difference Area correlation	-0.18 ± 1.36 mm 2 0.950	78	OB(1)
									Lumina	Area difference Area correlation	-0.15 ± 0.84 mm 2 0.945	193
	Brusseau [136]	2004	2D 20Mhz	C/R	(+)	(+)	-	QN(m)	Lumina	Contour MAD Area MAD	0.099 ± 0.032 mm 8.2 ± 5.4 %	15(not given)	OB(2)
									Lumina	Area correlation Area WI (95% CI)	0.99 0.60 (0.53, 0.68)	8(78)	OB(6) (In vivo)
										Area WI (95% CI)	0.66 (0.47, 0.84)
	Shekhar [131] 1999	3D	S	(+)	(-)	Manual initial	QN(m)	Media	HD WI (95% CI)	0.63 (0.54, 0.73)	1(28)
										AD WI (95% CI)	0.58 (0.50, 0.67)		OB(6)
										Area WI (95% CI)	0.46 (0.37, 0.54)		(In vitro)
									Lumina	HD WI (95% CI)	0.59 (0.55, 0.63)	1(55)
										AD WI (95% CI)	0.64 (0.56, 0.71)
										Area WI (95% CI)	0.88 (0.78, 0.97)
									Media	HD WI (95% CI)	0.80 (0.74, 0.86)
	Klingensmith [132]	2000	3D 40Mhz ECG gated	S	(+)	(-)	Manual initial Editing correction	QN(m)		AD WI (95% CI) Area WI (95% CI)	0.79 (0.75, 0.83) 0.62 (0.37, 0.93)	3(185)	OB(4)
									Lumina	HD WI (95% CI)	0.70 (0.65, 0.75)
										AD WI (95% CI)	0.68 (0.57, 0.81)
								QN(s)		Volume correlation	0.99	1(15)	phantom
	Kovalski [112]	2000	3D ECG gated	C	(+)	(-)	Initialization	QN(m)	Media Lumina	Area absolute difference Area absolute difference	06.5 ± 7.6 % 15.2 ± 17.4 %	5(44)	OB(2)
										Area WI (95% CI)	0.99 (0.95, 1.04)
									Media	Area difference	-0.07 ± 0.67 mm 2
	Klingensmith [133]( [132])	2003	3D 30Mhz ECG gated	S	(+)	(-)	Manual initial Editing correction	QN(m)		Area correlation Area WI (95% CI)	0.92 0.98 (0.89, 1.06)	9(529)	OB(4)
									Lumina	Area difference	-0.07 ± 0.67 mm 2
										Area correlation	0.97
	Pardo [141]	2003	3D	C/R	(+/-)	(-)	Manual Initial. Manual Training	QN(m)	Lumina	Contour distance	2.30 ± 3.46 pxl	1(400)	OB(5)
	Haas [137]	2000	3D 20Mhz	C/R	(+)	(+)	-	QN(m)	Media Lumina Contour radial difference Contour radial difference	-0.064 ± 0.156 mm -0.025 ± 0.163 mm	29(not given)	OB(2)
										Area correlation	0.89
									Media	Area difference AD	-0.470 ± 1.140 mm 2 0.105 ± 0.094 mm
	Cardinal [135])	2005	3D 20Mhz	S/R	(-)	(+)	-	QN(m)		HD Area correlation	0.300 ± 0.104 mm 0.974	1(60)	OB(1)
									Lumina	Area difference AD	-0.240 ± 1.070 mm 2 0.083 ± 0.111 mm
										HD	0.344 ± 0.237 mm

ejection fraction = end-diastolic volume -end-systolic volume end-diastolic volume × 100.

Shao et al. [3] presented a review on prostate boundary detection for articles before 2002. Our review overlaps with this on but also discusses more recent contributions.

The prostate shape model was constructed using 10 manually segmented images. To our knowledge, this small sample is unlikely to representatively characterize the shape variability of the prostate. Indeed, an equivalent approach was used by Wu et al.[START_REF] Wu | Automatic prostate boundary recognition in sonographic images using feature model and genetic algorithm[END_REF]; in which the modeling on 27 sample boundaries produced 9 principal variations of the boundary (see[START_REF] Shao | Prostate boundary detection from ultrasonographic images[END_REF] for a review of[START_REF] Wu | Automatic prostate boundary recognition in sonographic images using feature model and genetic algorithm[END_REF] and[START_REF] Xie | Segmentation of kidney from ultrasound images based on texture and shape priors[END_REF] for an alternative shape model).

As acknowledged by the authors, the combination is not original in itself as it is used previously for radar management and tracking. It is the framework in which it is used (segmentation) that is original.

The idea of statistical edge detection was originally proposed by Yuille; See[START_REF] Konishi | Statistical edge detection: Learning and evaluating edge cues[END_REF] and the references therein, for a recent paper on this.

However, the assumptions made by the authors to justify some parts are in contradiction with each others (once they claimed a Gaussian model and in other parts a Rayleigh model)!

The winding number is defined as the number of times the image gradient vector rotates over 2π when a closed curve is traversed about a fixed point i.e. the path integral of the angular increment of the direction of the gradient vector over a closed neighborhood around a point.

Free software for a number of operating systems is available at: http://www.es.oersted.dtu.dk/staff/jaj/field/.

ACKNOWLEDGMENTS

The authors would like to thank the anonymous reviewers for their thoughtful suggestions on how best to present this review, and the authors (too many to name in person) who kindly provided copies of their papers which are discussed in the review or permission to use figures from their papers.