Vincent Bismuth
email: vincent.bismuth@med.ge.com

Régis Vaillant

Niels Guillard

Laurent Najman

A comprehensive study of stent visualization enhancement in X-ray images by image processing means

Keywords: X-ray imaging, image enhancement, image quality, registration, segmentation, guide-wire, stent, angioplasty, cardiology

In this work we propose a comprehensive study of Digital Stent Enhancement (DSE), from the analysis of requirements to the validation of the proposed solution. First, we derive the stent visualization requirements in the context of the clinical application and workflow. Then, we propose a DSE algorithm combining automatic detection, tracking, registration and contrast enhancement. The most original parts of our solution: landmark segmentation and non-linear image registration are detailed. Finally, we validate the algorithm on a large number of synthetic and clinical cases. Performance is characterized in terms of image quality, automation, and execution time. This work is, to the best of our knowledge, the first comprehensive article on DSE, covering problem statement, proposed solution, and validation strategies.

Coronary artery disease is the most common cause of sudden death, and the most common reason for death of men and women over the age of 20 [START_REF] Rosamond | Heart Disease and Stroke Statistics-2007 Update: A Report From the American Heart Association Statistics Committee and Stroke Statistics Subcommittee[END_REF]. This disease tends to narrow the lumen of coronary arteries by the accumulation of atheromatous plaques within their walls. The narrowing of the artery lumen due to plaque progression, or plaque rupture can be cured by the expansion of a fine metallic mesh called a stent that is implanted in an artery wall acting as a scaffolding to open the lumen thereby restoring blood flow. Stent placement is performed as part of a percutaneous coronary interventional procedure. The procedure is typically performed under the guidance of X-ray fluoroscopy delivering real time video of the clinical tools and devices in the patient's anatomy, (Fig. 1 (a)), typically at 15 frames per second. The clinician first introduces a metallic guide-wire inside the artery that serves as support for sliding an angioplasty balloon equipped with a stent. The angioplasty balloon is used to expand the stent opening the lumen and simultaneously expands the stent and embeds it into the vessel wall. In order to visually assess the location of the balloon/stent on the guidewire, the guidewire is equipped with two highly radio-opaque marker-balls delimiting the position and extent of the devices. Fig. 1 (b) depicts a part of an X-ray image, with a guide-wire, the deflated balloon (invisible), its two marker-balls and the deployed stent. Proper positioning of the stent and apposition onto the vessel wall are key to the success of the procedure and patient safety [START_REF] Garg | Coronary stents: Current status[END_REF][START_REF] Alfonso | Intravascular Ultrasound Findings During Episodes of Drug-Eluting Stent Thrombosis[END_REF][START_REF] Fujii | Stent underexpansion and residual reference segment stenosis are related to stent thrombosis after sirolimus-eluting stent implantation: An in-travascular ultrasound study[END_REF][START_REF] Bertrand | Incidence of stent under-deployment as a cause of in-stent restenosis in long stents[END_REF]. However visualizing stents with conventional X-ray images is extremely challenging. Indeed coronary stents are characterized by their low radio-opacity and their fast motion. The recommended solution to assess stent deployment and apposition is to perform Intra-Vascular Ultra-Sound imaging (IVUS). This modality, considered as the gold standard, provides cross sectional images of the vessel, depicting the stent, the vessel wall and enables quantitative length and area measurements.

Unfortunately systematic use of IVUS is impractical since it would add significant time and cost to the procedure. It is only used in 4.5% of the cases in Europe, 14% in the USA and 60% in Japan where it is reimbursed. In this paper, we consider techniques enabling stent visualization that are consistent with the normal workflow of an angioplasty procedure. Although they cannot bring direct information on the apposition of the stent onto the vessel wall, they can provide relevant images for assessing deployment irregularities, lesion treatment and potentially measuring stent expansion.

Since 2000, an image processing technique called digital stent enhancement (DSE) has emerged to produce an enhanced image of a stent from an X-ray image sequence [START_REF] Close | Improved image guidance of coronary stent deployment[END_REF][START_REF] Florent | Medical viewing system and method for enhancing structures in noisy images[END_REF][START_REF] Bismuth | Elastic registration for stent enhancement in x-ray image sequences[END_REF]. It has gained interest from clinicians over the past years, leading imaging system manufacturers to collaborate with them to study this topic [START_REF] Koolen | Stentboost image enhancement: first clinical experience[END_REF][START_REF] Ross | Registration and integration for fluoroscopy device enhancement[END_REF][START_REF] Conway | Measurement of coronary stent expansion using stentboost image enhancement software: a comparison with intravascular ultrasound[END_REF][START_REF] Mishell | Determination of adequate coronary stent expansion using stentboost, a novel fluoroscopic image processing technique[END_REF][START_REF] Ohanessian | Abstract 4909: Usefulness of stentboost enhancement for stent expansion during coronary angioplasty in daily practice[END_REF][START_REF] Bismuth | Elastic registration for stent enhancement in x-ray image sequences[END_REF][START_REF] Córdova | Digital enhancement of stent images in primary and secondary percutaneous coronary revascularisation[END_REF][START_REF] Funck | Quantitative evaluation of an elastic stent enhancement technique in x-ray image sequences[END_REF]. To perform DSE, the clinician first shoots an X-ray sequence of the deployed stent while keeping the delivery balloon in place. It is typically a short X-ray sequence showing the stent at 15 frames per second where the stent motion is induced by the heartbeat and the breathing of the patient. As in any regular X-ray sequence, the visibility of the stent is limited. The motion of the stent is inferred by the DSE algorithm based on the motion of balloon marker-balls that serve as landmarks. Motion compensated image integration is the corner stone of DSE and enables a significant reduction of the noise while preserving the details of the motion compensated components of the image (Fig. 1 (c) and (d)). Studies show that stent visibility is significantly improved by this technique [START_REF] Koolen | Stentboost image enhancement: first clinical experience[END_REF][START_REF] Ohanessian | Abstract 4909: Usefulness of stentboost enhancement for stent expansion during coronary angioplasty in daily practice[END_REF][START_REF] Funck | Quantitative evaluation of an elastic stent enhancement technique in x-ray image sequences[END_REF]. Moreover this image quality improvement improves the ability of an observer to detect stent under deployment [START_REF] Ross | Registration and integration for fluoroscopy device enhancement[END_REF]. DSE techniques have emerged independently from two different viewpoints. [START_REF] Close | Improved image guidance of coronary stent deployment[END_REF] were interested in the representation of an X-ray image as a superimposition of transparent layers and proposed a general framework to estimate the layers present in a sequence of images. They considered that if the motion of a given layer was known, its intensities could be estimated by averaging the value of each pixel along its trajectory in the image sequence. They relied on the assumption that different layers have independent motions. They proposed an application of their method to stent enhancement, inferring the motion of the layer of the stent by the motion of the balloon marker-balls. In this case, the layer decomposition method boiled down to detecting the successive positions of the marker-balls in the sequence and integrating the images along the resulting trajectory. Independently and simultaneously, [START_REF] Florent | Medical viewing system and method for enhancing structures in noisy images[END_REF] were interested in motion compensated noise reduction and filed a patent describing a method to enhance the visibility of stents. Similarly they proposed to detect and track the balloon marker-balls, to use them to infer the motion of the stent and to average the values of the pixels along their trajectories. They were the first ones to describe an algorithm aimed at detecting the marker-balls with limited user interaction.

Both Close and Florent used only the marker-balls to estimate the motion of the stent, neglecting the non-linear deformations of the vessel that occur along the cardiac cycle. In our previous work [START_REF] Bismuth | Elastic registration for stent enhancement in x-ray image sequences[END_REF] we have demonstrated that significant improvement in sharpness can be achieved by employing the guide-wire joining the markers in a stent deformation model to compute a non-linear motion field.

In this paper, we report on new contributions from both application and technical perspectives. Within the DSE application domain, we note that the following ideas are presented for the first time (i) a complete analysis of DSE from clinical problem statement to performance assessment, (ii) the description of a fully automated algorithm, (iii) an exhaustive validation of this DSE algorithm. From an image processing perspective, the first novelty stands in the guide-wire segmentation techniques that are thoroughly described in the paper. It consists in spanning a set of parametric curves satisfying the physical constraints of our application and integrating profiles along the curve. This integration step is what makes the technique very robust to noise (as demonstrated in the validation section 2.2.2). Image noise, a result of minimizing patient and clinician dose, is a significant obstacle for guidewire detection. This technique has several nice properties that are demonstrated in the paper : the segmented guide-wire is smooth, the execution time is reasonable, and it is robust to a large amount of noise. The second novelty in medical image processing is the registration technique. Our analysis of the problem of DSE led us to propose a non-linear registration in a domain where linear registration has previously been considered sufficient. The superiority of our registration is demonstrated on a large number of clinical sequences. This technique fulfills the requirements that make it suitable to register not only stents but also any medical device inserted inside a vessel, or vessels themselves and can thus have various applications.

The paper is organized as follows: In section 1 we define user requirements and flow them down to quantifiable metrics. We show how the main characteristics of a successful DSE algorithm can be deduced by the analysis of the problem. Then we describe a fully automated algorithm to perform DSE that segments the marker-balls and the guide-wire and registers images in a non-linear manner to take into account vessel and stent deformations. In section 2, we validate the fulfillment of the user needs on a large number of cases, addressing the perfor-mance of the segmentation of the landmarks, the image quality improvements and comparing non-linear to linear registration. In section 3, we discuss the results and expose the challenges that are still not met by DSE. Conclusions are drawn in the last section.

Method

Clinical needs

From the clinician's standpoint, the value of a DSE technique depends upon three factors: improvement in image quality, limited user interaction and reasonable execution time. For each factor, we defined a metric and set goals that capture clinical needs. The first factor, image quality, is the most subjective and therefore the most difficult to measure. For our application we defined a 5-grade image quality scale quantifying the visibility of stents:

1. the stent is hardly visible 2. the stent border can be guessed 3. the stent border is clearly visible 4. the stent border is clearly visible and some struts are visible 5. the stent is perfectly visible In order to quantify improvement in image quality, two clinicians rated images before and after the application of the DSE technique. They considered that a statistically meaningful improvement of one grade is necessary for the method to be adopted. Regarding user interaction, clinical feedback suggested that a DSE technique that is successful in 80% of the cases in a fully automatic manner, and in 90% when the user selects a region of interest (ROI), was sufficient to be regularly employed in practice. In terms of execution time, one has to keep in mind that an angioplasty procedure typically lasts between 20 minutes and 2 hours. During an interventional cardiac procedure, clinicians do not need to have DSE images in realtime, but in a "reasonable" time after the deployment of the stent. The delay induced by the computation of the DSE image should not disturb the regular work-flow of the procedure. Clinical feedback suggests that a DSE technique should not exceed 30s execution time.

From problem statement to algorithm design

The main characteristics of a DSE algorithm that will fulfill the clinicians' requirements can be deduced from a detailed analysis of the DSE problem. In this section we study how image quality can be improved. In the case of stents, it can be split into four factors : noise, contrast, zoom and sharpness.

Noise

The major source of noise in X-ray fluoroscopy is quantum noise. It follows a Poisson distribution and it is spatially correlated but temporally white [START_REF] Spekowius | Simulation of the imaging performance of x-ray image intensifier/TV camera chains[END_REF][START_REF] Aach | Digital image acquisition and processing in medical x-ray imaging[END_REF]. There are two sources of information redundancy in the image sequences that can be used to reduce noise: the spatial content of each image, and the temporal content of the Figure 2: Illustration of a deployed stent in an x-ray image processed with standard contrast enhancement techniques, which may be considered a form of DSE. The resulting stent visualization is substantially inferior to that produced in a dedicated technique, as shwon in Fig. 1 (d).

image sequence. The temporal redundancy is advantageously utilized by integrating the values of each pixel of the stent along its trajectory. Therefore it requires the estimation of the trajectory of each pixel of the stent. Noise being uncorrelated from one image to another, integrating n nimages decreases the noise by √ n. DSE sequences typically contain 30 frames, that corresponds to a short acquisition of 1 to 2 seconds. The potential noise reduction factor is thus approximately 5. It is usually enough to make a major difference in image quality. The spatial content of a stent image being very complex, intra image denoising could result in stent detail degradation. Therefore a DSE algorithm will preferentially restrict denoising to the temporal domain.

Contrast

DES seeks to emphasize the subtle contrast variations induced by the presence of the stent. However, traditional processing and display strategies are generally optimized for the visualization of large structures like coronary vessels while concurrently suppressing noise, and not for the fine stent details. Consequently, contrast and windowing of traditional processing approaches are suboptimal for stent visualization(Fig. 2). Dedicated image processing, like unsharp masking, must be set in place to enhance the fine details of the images.

Zoom

The stent only represents a small portion of the original images. Typically a 100 2 area in a 1024 2 image. DSE can directly improve stent visualization by zooming on the region of interest and masking un-necessary image parts. The comparison of Fig. 1 (a) and (c) illustrates this.

Sharpness

The sharpness of the enhanced image intrinsically depends upon the precision of the registration. Any misregistration errors result in blurring of the stent. Registration is crucial, considering the stent geometry relative to the optical magnification and detector pixel pitch. Stent struts are typically 0.16 mm wide, less than 0.2 mm pitch [START_REF] Ross | Registration and integration for fluoroscopy device enhancement[END_REF]. Taking into account a typical magnification factor of 1.4 for cardiac interventions, the projection of the stent struts onto the detector are on the order of one pixel. Finally the stent structure is blurred by the system Mean Transfer Function (MTF) that tends to spread the stent strut over the neighbor pixels. Determining the required precision of the registration and its implications for the design of a stent enhancement algorithm was the subject of dedicated studies. [START_REF] Ross | Registration and integration for fluoroscopy device enhancement[END_REF] determined that DSE is robust to some degree of error in the registration process. They centered their approach on a task oriented framework, evaluating the ability of an observer to detect stent under deployment. They concluded that it was improved by DSE up to a registration error of 2 pixels but that the benefit exhibited a significant fall off beyond 1.25 pixels of error. On our side, we attempted to quantify the effect of the registration error directly on the image quality regardless of a specific task. We imaged several still stents with an X-ray angiographic system. We averaged these images with various simulated registration error (Fig. 3). It turned out that stent struts were visible up to an error below 1 pixel. An error between 1 and 2 pixels would preserve the but make delineation of stent struts very difficult. Any error above 2 pixels would impair the visibility of the stents. These two experiments confirmed that given stent geometry and acquisition resolution, registration errors less than a pixel are tolerable, whereas registration error in excess of 2 pixels severely degrades performance rendering the algorithms benefit to be questionable. In order to minimize this error we analyzed its origins. We identified two primary root causes for misregistration: landmark segmentation error; and the error in inferring stent motion relative to that of the landmarks. Stent and landmark motion is complex due to projection of 3D motion onto a 2D plane, and stent motion that is independent of the landmarks(see section 3.4). We concluded that, in order to maximize stent visualization, we need to minimize landmark segmentation error (ideally sub-pixel). Additionally, to minimize the stent motion estimation error, we have added the guidewire supporting the markers balls to the set of landmarks. We show that by considering guide-wire and marker ball motion, we have improved the accuracy and robustness of stent motion estimation (Fig. 4).

To conclude this section, our analysis of the DSE problem resulted in the determination that a successful DSE algorithm must automatically process the images with a high success rate (> 80%) and enable a computation time (< 30s) to fit within the interventional work-flow. Moreover, in order to attain optimal image quality, it shall perform noise reduction preferentially by temporal techniques, include a background removal and adaptive zoom for visualization. Last but not least it shall detect the marker-balls, the guide-wire and register the stents with a high precision, ideally sub-pixel.

DSE algorithm

In this section, we describe a DSE algorithm derived from the objectives and constraints previously listed. We first give a general overview of the method, then we detail the parts that (0)

(1.5)

(0.5) (2)
(1) (2.5) are specific to our approach. Finally we explain our strategy to find a relevant setting of the many parameters that are involved.

Algorithm overview

The DSE algorithm that we propose can be divided into four main steps (Fig. 5) : landmark detection and tracking, image registration, image combination, and display processing. A bottoms-up approach is pursued in landmark detection and tracking. First, in each image, we detect points representing potential marker-balls, then we form pairs of points and build tracks of pairs. The detection of the potential marker-balls is close to the problem of detecting micro calcifications in mammograms [START_REF] Cihan | An application of topological median on detection and clustering of microcalcification in digital mammograms[END_REF][START_REF] Grimaud | La géodésie numérique en morphologie mathématique. application à la détection automatique des microcalcifications -2eme partie[END_REF]. Similarly, we pre-process images with a dark top hat [START_REF] Soille | Morphological Image Analysis[END_REF]. It enables the removal of the background variations while retaining dark objects of a given scale. In practice with a relevant setting of the structuring element, the marker-balls, the guidewire and the stent are preserved (Fig. 6). Moreover efficient implementations exist [START_REF] Van Herk | A fast algorithm for local minimum and maximum filters on rectangular and octagonal kernels[END_REF]. In the resulting image the potential marker-balls can be characterized as being local minima in a given range of intensity. Forming pairs of potential markers and buildings tracks of pairs along the sequence (Fig. 7) is constrained by a priori knowledge of device (stent/marker ball/guide wire) geometry, and the characteristics of cardiac motion (see section 1.4). We assign a figure of merit to each track based on its regularity and attenuation properties of its candidate markers. Finally, we identify the most promising track and we segment the guide-wire supporting each pair of markers. Image registration of the stent relies on a model to infer stent motion from the position of the landmarks. Each image is registered to a reference position in a non-linear fashion. Image combination consists of averaging all the registered frames together to produce a denoised image. Then the contrast is set to enhance stent details. It benefits from the dark top hat preprocessing that removes background variations. Finally zooming is performed centering on the landmarks. The most challenging problems addressed by our algorithm are the segmentation of the guide-wire and the non-linear registration. We first published them in the patent application [START_REF] Vaillant | System and method to enhance visualization of an object in a vascular vessel[END_REF]. We detail them in the following sections.

guide-wire segmentation

Segmenting the guide-wire in an X-ray image is known to be a difficult problem. Indeed, guide-wires are fine elongated structures, typically 3 to 5 pixels wide, of low Contrast to Noise Ratio (CNR). A typical CNR is around 3, but may fall below 1 in the most challenging cases. Previous research on guidewire segmentation falls into two categories. One category is based upon building a map of the probability that a guide-wire is present [START_REF] Bismuth | A comparison of line enhancement techniques: applications to guide-wire detection and respiratory motion tracking[END_REF], and the second category explicitly segments the guide-wire. The second category is of interest for our application. On the one hand, [START_REF] Barbu | Hierarchical learning of curves application to guidewire localization in fluoroscopy[END_REF] presented a sophisticated approach to guide-wire segmentation based upon a machine learning formalism. They attempted to solve a problem that is more general than ours since they segment the whole guide-wire, whereas we are only interested in a local segmentation at the close vicinity of the marker-balls. On the other hand, [START_REF] Florent | Medical viewing system and method for enhancing structures in noisy images[END_REF] addresses exactly our problem with a minimal cost path approach but does not guarantee the continuity of the first derivative of the guide-wire curve that is required for our application, detailed in section 1.3.4. Therefore we developed a new guide-wire detection procedure designed at producing a smooth curve, while being robust to high level of noise and enabling fast computation. Our observation of clinical sequences suggests that the guide-wire between the markers may be modeled as a simple curve. Most of the time it may approximated by a parabola. This observation is shared by [START_REF] Florent | Medical viewing system and method for enhancing structures in noisy images[END_REF]. In a subset of cases, the guidewire possesses an inflexion point and may be accurately modeled by a set of two parabolas or a third degree polynomial expression.

In the general case, a simple set of smooth parametric curves will describe all the possible guide-wire configurations between the markers, as well as in a short distance beyond the markers.

Based on these observations, we decided to specify a discrete set of curves spanning a family of parametric curves (Fig. 8). A figure of merit is assigned to each curve in the set quantifying how well it represents the image content. We retain the parametric curve yielding the highest figure of merit. This approach shares some similarities with one of [START_REF] Donoho | Beamlets and multiscale image analysis[END_REF] who demonstrated that it is possible to detect and segment curves in high noise images if they belong to a parametric set of curves that can be efficiently spanned. The challenge lies in the design of the figure of merit which is robust to the low CNRs of imaged guide-wires. In order to do so we averaged intensity profiles line segments which are uniformly spaced between the marker balls, perpendicular to the guidewire, and centered on the guidewire (Fig. 8). The resulting average profile enables robust contrast estimation due to the reduction in noise. Analysis of the averaged profile is performed in two steps. First, a test is performed on the shape of the profile to determine if it has a significant minimum at its center (Fig. 8 (d)). Then contrast is measured by computing the difference between the value at the minimum and the values at the extremities. Evaluations on clinical data have demonstrated that this method enables guidewire segmentation in extremely noise-corrupted images (Section 2.2.2 and Fig. 13 (c) and (d)).

Constrained registration derived from a stent deformation model

Once the landmarks (marker-balls and guide-wire) have been segmented, we must then estimate the motion field. Secondly, we must define how to extend this motion field beyond the landmarks encompassing the image area occupied by the stent. Finally, we utilize the motion field for each image to temporally register the image sequence. We denote (I t ) 1≤t≤n as the input We see that the noise is high and there is no particular pattern in this curve. (d) Averaged profile. We see that the curve follows the expected pattern: a smooth curve with a significant minimum at its center.

image sequence of n images and we define I t 0 as the reference frame over which all frames are temporally registered. The motion fields are computed with respect to this reference position.

The definition of a motion field on the landmarks is straightforward. We force the marker centers to match from one image to another and the guide-wire to match according to the curvilinear abscissa. We define a reference point, O t , in each image that is the point of the guide-wire between the markers that separates it into two curves of equal length (Fig. 9.a). It is equidistant to the two marker-balls on the guide-wire. We compute the signed curvilinear abscissa from this point. We cope with global foreshortening by normalizing the curvilinear abscissa to the total length of the guide-wire between the markers. The motion field registers points relative to their normalized curvilinear abscissa (Fig. 9.b).

In order to extend the motion field from the landmarks to the stent we incorporated design constraints reflecting the physical properties of stents. First, stents must be very rigid on their radial axis in order to maintain the desired artery diameter under the pressure of the artery wall. Consequently, the radius of the stent at a given point must be constant over the image sequence.

Second, stents must be flexible on their longitudinal axis in order to bend with the vessels to follow the heart contractions and expansions (Fig. 10). Therefore stents follow the deformations of the major axis of the vessel. Finally, we assume that stents do not shrink along the axis of the vessels. These motion as-sumptions were confirmed through observation and analysis of clinical image sequences.

Although the major axis of the vessel is unknown, the guidewire can be used as a surrogate estimate. The first two constraints can be translated into constraints with respect to the position of the guide-wire. Imposing that the distance of any point of the stent to the guide-wire is invariant along the sequence is enough to fulfill the two first constraints. In order to satisfy the stent length invariance constraint, we consider the orthogonal projection of each point of the stent onto the guide-wire (F p on Fig. 9.a) and impose that the curvilinear abscissa of this point be invariant from one image to another. The constrained registration proposed has been demonstrated to perform well in practice (section 2.4), although alternate approaches are possible. In the following section, we detail the image registration based on the aforementioned considerations. The design of this image registration technique and its usage are new and particularly tailored to the application of DSE. Moreover, one can observe that the constraints we listed and the solution that we propose in the next section may be extended beyond this context to register other medical tools traveling inside vessels and potentially vessels themselves as summarized in our previous work [START_REF] Bismuth | Elastic registration for stent enhancement in x-ray image sequences[END_REF].

Non-linear Registration

We build a system of coordinates (Fig. 9.a) in each image I t such that for any point P, d 1 is its distance to the curve defining the guide-wire and F P the point of the guide-wire that actually minimizes this distance. F P is unique outside of the cut locus of the curve. Let us call d 2 the curvilinear abscissa from O t to F P along the guide-wire curve divided by the total length of the curve between the markers. We propose to use d 1 and d 2 as coordinate system. In order to create a coordinate system without ambiguities, let us give signs to d 1 and d 2 . Let us call M 1 and M 2 the centers of the two marker-balls in this frame. We define the curvilinear abscissa d 2 in the direction from O t to M 1 to be positive and the direction from O t to M 2 to be negative. Noting the curve defining the guide-wire is smooth, we can define a tangent to the curve t(F P ) at any point F P of the curve, pointing in the direction of positive curvilinear abscissa. Finally, we also define a normal vector n(F P ) such that ( t(F P ), n(F P )) is direct and we assign d 1 the sign of the scalar product t(F P ). F P P. Outside of the cut locus of the curve defining the guide-wire, T t , that maps a point P of I t to (d 1 , d 2 ), is a continuous one to one mapping but is not defined on the cut locus. On a database of 241 clinical cases, we observed that the stent is never overlapping the cut locus. This can be explained by the rather high radii of curvature of the guide-wires in the clinical images. Therefore there is no artifact introduced by this transformation on clinical images in the area of interest.

In order to perform registration, we need to define the inverse transforms T -1 t that maps (d 1 , d 2 ) to a point P. The definition of T -1 t is straightforward. From the point O t , we define the point F P as the only point on the guide-wire having the signed curvilinear abscissa d 2 . Then we define the point P = T -1 t (d 1 , d 2 ) by moving away from F P perpendicularly to t(F P ) by (signed distance) d 1 . The registration of a point P of the image I t 0 on an image I t is the point P ′ t computed according to:

P ′ t = T -1 t (T t 0 (P)) (1)
Conversely any point P ′ t of an image I t is registered to the point P of I t 0 according to:

P = T -1 t 0 (T t (P ′ t )) (2)
Let Ĩt be the result of applying equation ( 2) to every point in I t .

In any image Ĩt , the marker-balls and the guide-wire are exactly at the same location as in I t 0 . This also defines the stent motion provided its motion is subject to the constraints of 1.3.3. The image combination simply consists in averaging all the images Ĩt to produce the enhanced image I enhanced :

I enhanced = 1 n n t=1 Ĩt (3) O t M 1 M 2 d 2 d 1 P F p (

Parameter setting and algorithm validation

Detecting the marker-balls, forming pairs, tracking them, and segmenting the guide-wire required setting a large number of parameters. In order to determine optimal parameter settings, we collected a large database of 241 clinical sequences, representing approximately 7000 markers pairs. An operator manually determined the position of each marker ball in each image, and marked some guide-wire points in a subset of the images. Over this database we have been able to estimate typical values and variability of marker characteristics, marker motion, and guide-wire curvature. Moreover it enabled testing the landmark detection performances for a given set of algorithm parameters by comparing the results given by the algorithm to the ground truth. This process enabled parameter tuning for the detailed DSE algorithm. We found that the optimal setting of some key parameters is as follows: the size of the structuring element for the subtraction must be set to 9 pixels, the maximum height of the parabola describing the guide-wire must be set to 20% of the distance between the markers, and the distance between two markers does not vary more than 15% from one frame to another. The results presented in the next section have been obtained with the best parameter set that we tested.

Results

According to the clinical criteria defined in section 1.1, we elaborate on the validation framework for our algorithm. We have quantified its ability to automatically segment the landmarks. Results regarding the precision and robustness of their segmentation is also reported. Clinical interest of the technique has been assessed by evaluating the improvement it brings in image quality. Finally, in order to assess the performance of this DSE algorithm to other existing methods, we compare the performance of the linear marker-ball based registration to our non-linear registration approach.

Landmark detection performance

We have separated the problem of estimating the landmark detection/tracking performance into two sub-problems. The first category is to quantify, on clinical sequences, how well the automatic marker-ball segmentation is able to detect and track the marker-balls. The main challenges are robustness to complex image content (anatomical structures and medical tools) in the detection algorithm, and to cardiac and respiratory motion with respect to tracking. The second category addresses quantitative accuracy of the segmentation, given the marker balls are detected by the algorithm. We noted in the section 1.2.4 that we would like to reach a sub-pixel precision to insure the sharpness of the DSE image. Quantitative performance measurement of the registration accuracy requires precise knowledge of (subpixel) ground truth. This precision cannot be achieved by a human operator on clinical sequences. Therefore we set up a specific simulation to assess it and is the subject of the next section 2.2. In the present section we deal with landmark detection and tracking performance. To this end, we had an operator mark the position of the markers on clinical image sequences. The position marked by the operator lies inside of the markerball but is not precisely at its center since this exact position is unknown. We compared the position of the detected markers to this ground truth on every image, calculating the number of false negatives, true positives and false positives (there are no true negatives since the marker-balls are present in every frame of the sequence). Marker ball being ellipses of large axis between 7 to 10 pixels, the tolerance was set to ±5 pixels. We computed the number of cases where the markers that our DSE algorithm output were closer than the tolerance to the ground truth. On a database of 241 clinical sequences, illustrating 7230 pairs of markers, our algorithm exhibited a true positive rate of 91.5%, a false positive rate of 1.5% and a false negative rate of 7.5%. We studied the detection performance in more detail to tell if some frames are regularly failed in every image sequence, or if some images sequences completely failed whereas others were completely successful. The histogram of the true positives, false positives and false negatives per image sequence are reported in Fig. 11. We can observe that some frames are regularly missed in each sequence as false negatives (2 to 3 frames). This does not have a strong impact on the final image quality. They often coincide with the small number of frames were the markers are very blurry due to the heart acceleration at the end of the rest phase. These images contain only very low information for stent enhancement, since the stent is also blurred. Regarding false positives, we can notice that in 98% of the sequence there are less than 5% of them. The overall performance over the database is very satisfying, and far beyond the 80% acceptance criterion set in 1.1.

Landmark segmentation precision and robustness

As explained in section 1.1 the precision of the detection of the landmarks is crucial to the sharpness of the DSE image. Since marking the position of the landmarks on clinical images with a sub-pixelic precision is extremely challenging, we used synthetic images to estimate the precision and the robustness of our landmark segmentation techniques.

Marker ball segmentation precision

For this task, we simulated a marker ball on a guide-wire using a model of a sphere and of a cylinder corrupted by typical noise and system point spread function (see Fig. 12). We set the CNR of the synthetic marker-balls to values ranging from 5 to 10, since it is the range of values observed in clinical sequences. We applied our technique to the synthetic images and estimated for each input CNR the average distance between the detected marker location and the real marker center. A marker was considered detected if its center lied within the synthetic marker ball. The marker detection rate was 98%. This excellent result can be explained by the simplicity of our synthetic sequences. A more meaningful success rate is the one of 91.5% reported previously reported in 2.1 computed on the clinical database. The synthetic sequences are however particularly interesting to estimate the precision of the marker detection. We found that the average error is less than 0.5 pixels both along the guide-wire and perpendicularly to it. As one can expect from the structure of the test images, the tests demonstrated that our marker localization process is more precise perpendicular to the guide-wire than parallel to it.

Figure 12: An image of a synthetic marker-ball and guide-wire used to quantify the precision of the marker-ball segmentation.

guide-wire segmentation precision and robustness

Two main factors impact the success of guide-wire segmentation. On the one hand, the precise shape of the guide-wire, since we assume that it can be modeled by a set of parametric curves. On the other hand, the contrast to noise ratio of the guide-wire. In order to address the topic of the shape of the guide-wire, we have selected 10 clinical sequences out of 241 on the criteria of very diversified guide-wire shapes and orientations, and rather favorable image quality. We ran our segmentation algorithm on each of these sequences and visually assessed the quality of the result. All segmentations were visually perfect (Fig. 13 (a) and(b)). This experiment validated that the set of selected parametric curves spans the natural variability of guide-wire configurations. We then considered these segmentations to be the ground truth for these images. Given the guide-wire width of 3 to 5 pixels and the difficulty to delineate precisely its centerline, we considered that this segmentation was precise within a range of plus or minus 1.5 pixel. Recognizing the challenge in the segmentation of the guide-wire being attributed to the low CNRs often encountered in clinical conditions, we degraded the 10 clinical images with noise to evaluate the robustness of the segmentation (Fig. 13 (c) and (d)). We ran our algorithm on each degraded image and computed the Hausdorff distance of the segmentation to the ground truth (Fig. 13 (e) and (f)). We generated 1400 degraded images adding noise of various standard deviations, up to producing images with 3 times the original noise level, which is considered an extreme situation. The CNR of the guide-wires ranged from 0.6 to 5.6 with an average of 1.7 ± 0.9. Over all the degraded images the segmentation error was 0.8 ± 1.2 and was below 1.5 pixels in 93% of the cases. This error increased with an increase in noise. For the cases where the noise was less than twice the original noise its repartition was 0.5 ± 1.0, below 1.5 in 97% of the cases. When the noise was two to three times the original noise it was 0.9 ± 1.3, again below 1.5 in 91% of the cases. An analysis of the histogram of the Hausdorff distances demonstrates that the overall 6% of failed segmentations fall equally into two categories (Fig. 14). Half of them are the tail of a distribution centered on the true segmentation. The other half are very large errors (superior to 7 pixels) where the segmentation departed from the guide-wire. In these cases, the algorithm waqs influenced by the image content, for example the border of a stent. The overall results are very satisfying with the segmentation being accurate in 97% of the cases portraying realistic conditions, and in 91% in extremely difficult cases which are less commonly encountered.

Image quality improvement

The image quality has been rated independently by two clinicians, before and after the application of the stent enhancement software according to the 5-grade image quality scale presented in section 1.1. This comparison on 80 clinical sequences was the subject of a previous publication [START_REF] Funck | Quantitative evaluation of an elastic stent enhancement technique in x-ray image sequences[END_REF]. A paired T-test demonstrated that an improvement of 1.0 point out of 5 is statistically significant (p ≤ 0.001). A larger study including 100 patients and 196 sequences confirmed that our DSE technique improves image quality by more than one point on the same image quality scale on a large variety of imaging conditions and stents (p ≤ 0.001). The histograms of the scores before and after application of DSE on the 196 images sequences is presented in Fig. 15. We assessed inter-observer variability by computing the linearly weighted kappa [START_REF] Viera | Understanding interobserver agreement: the kappa statistic[END_REF]. Since the unprocessed and processed images are different in nature, we computed it separately on both. The kappa measuring agreement between the two observers at rating unprocessed images is 0.34 demonstrating a "fair agreement". On processed images, a higher kappa of 0.49 was observed, suggesting "moderate agreement".

Non-linear versus linear registration

In this section we evaluate the impact of the method of the registration on the enhanced image (see [START_REF] Bismuth | Elastic registration for stent enhancement in x-ray image sequences[END_REF]). We visually inspected the enhanced images with both registration techniques on a database of 144 sequences. We reported whether there was a significant difference in the stent and if the stent was better visualized with one of the registration techniques. The observed differences fell into two categories. Category A groups the cases where the same details are visible and one technique creates more blurring on the stent. Category B groups the cases when one technique substantially outperforms the other such that more stent more are observed with it. Table 1 summarizes the results which suggest that the proposed non-linear curve-based technique performs significantly better than the linear marker-ball based registration. Indeed for 27.9% of the cases, the non-linear registration produces a better image than the linear one, whereas the opposite only happens in 3.4% of the cases. Moreover in 11.6% of the cases the non-linear registration is superior and differs from the linear registration with a difference of category B. Conversely, the occurrance of the linear registration being superior was less than 1%. Fig. 16 depicts examples where the non-linear registration is better suited than the linear one. Around the two markers, the enhanced images are very similar, but on the center of the stent the linear registration produces a strong blurring and details were lost. On the contrary, these details were enhanced with the curve-based registration.

Execution time

Our DSE technique implemented on a PC with a 3.2 GHz single-core processor performs on average in 16.6s over a database of 40 cases with a standard deviation of 3.0s. The maximum execution time was 30s and the minimum was 13s. The algorithm is fast enough to fit seamlessly in the interventional work flow. With a dedicated implementation, on a multicore Xeon X3450 processor with a 2.66 GHz clock speed, the average time drops down to approximately 3s.

Discussion

The role of DSE in clinical practice

As a consequence of the growing interest of clinicians for DSE, several studies have been conducted [START_REF] Mishell | Determination of adequate coronary stent expansion using stentboost, a novel fluoroscopic image processing technique[END_REF] [START_REF] Koolen | Stentboost image enhancement: first clinical experience[END_REF][START_REF] Conway | Measurement of coronary stent expansion using stentboost image enhancement software: a comparison with intravascular ultrasound[END_REF][START_REF] Kalpesh | Tct 2005 abstract: Determination of adequate stent expansion: A comparison of quantitative coronary angiography, intravascular ultrasound, and novel stentboost x-ray imaging[END_REF][START_REF] Ohanessian | Abstract 4909: Usefulness of stentboost enhancement for stent expansion during coronary angioplasty in daily practice[END_REF][START_REF] Córdova | Digital enhancement of stent images in primary and secondary percutaneous coronary revascularisation[END_REF][START_REF] Choi | Tct-375: Impact of stentboost enhancement guided percutaneous coronary intervention on midterm angiographic and clinical outcomes[END_REF][START_REF] Funck | Quantitative evaluation of an elastic stent enhancement technique in x-ray image sequences[END_REF][START_REF] Milouchi | Le rehaussement de contour de stent : un nouvel outil dans l optimisation du résultat de l rtangioplastie coronaire. 16eme assises du Collége National des Cardiologues des Hopitaux Généraux[END_REF] that demonstrated that DSE is bringing a significant improvement in image quality and that it is deemed useful in clinical practice in a large proportion of cases (≈ 70%). DSE can help evaluate the deployment of the stent and detect some under-deployments or non optimally treated lesions. In such cases, clinicians can post-dilate the stents to improve their expansion. The rate of stent postdilatation using DSE ranges from 30% to 70% depending on the study. Further studies are needed to better understand the role and the impact of DSE on clinical practice, for instance comparing post-dilatation rates with and without DSE. Only one study [START_REF] Choi | Tct-375: Impact of stentboost enhancement guided percutaneous coronary intervention on midterm angiographic and clinical outcomes[END_REF] investigated the impact of DSE on the main clinical endpoints such as the occurence of major adverse cardiac events and target lesion re-vascularisation. Based on a two arm set up involving nearly 900 patients, they demonstrated significant improvement of the clinical endpoints at mid-term follow-up. Moreover, various teams [START_REF] Koolen | Stentboost image enhancement: first clinical experience[END_REF][START_REF] Conway | Measurement of coronary stent expansion using stentboost image enhancement software: a comparison with intravascular ultrasound[END_REF][START_REF] Kalpesh | Tct 2005 abstract: Determination of adequate stent expansion: A comparison of quantitative coronary angiography, intravascular ultrasound, and novel stentboost x-ray imaging[END_REF][START_REF] Mishell | Determination of adequate coronary stent expansion using stentboost, a novel fluoroscopic image processing technique[END_REF][START_REF] Córdova | Digital enhancement of stent images in primary and secondary percutaneous coronary revascularisation[END_REF] studied the relationship between the stent expansion quantification performed with this technique with respect to IVUS as gold standard. The five studies led to the interesting result that the relationship is linear with a rather good correlation (r ≈ 0.7). It is a promising step towards quantitative assessment of stent expansion using DSE. However DSE alone cannot provide any information regarding the apposition of the stent onto the vessel wall. The only recognized technique to perform this remains intra-vascular imaging.

Non-linear versus linear registration

Although section 2.4 demonstrated that the non-linear registration outperformed performed linear registration, we must keep in mind that it is more computationally intensive. Indeed, the non-linear registration requires precise segmentation of the guide-wire to compute the motion fields using curvilinear abscissa and projections of points onto curves. Nevertheless, it is noteworthy that the overall execution time of the non-linear registration DSE is compliant with the clinicians needs. However some users may prefer an alternate trade-off between image quality and latency within the application. An additional performance comparison between marker-ball based and guidewire based registration may be made with respect to the accuracy of the stent diameter. Since clinicians can perform measurements on the enhanced stent to assess if it is properly deployed [START_REF] Koolen | Stentboost image enhancement: first clinical experience[END_REF][START_REF] Córdova | Digital enhancement of stent images in primary and secondary percutaneous coronary revascularisation[END_REF], a registration that preserves the actual diameter of the stent is preferred. As far as the stent follows the motion model described in 1.3.3, the guidewire based registration preserves the stent diameter. On the contrary, there is no such guarantee with the marker ball based registration. Indeed, the linear transform includes a scaling to cope with the change of distance between the markers from frame to frame. The change in distance between the marker-balls can be the result of a change of curvature of the guide-wire due to the heart beat. In such a case, scaling the whole image is not adequate because it will modify the diameter of the stent. Guide-wire based registration enables better image quality and probably more accurate stent diameters, whereas marker ball based registration results in faster processing.

Comparison with related work

The technique described here is the skeleton of the software feature StentViz (Morris, 2009) commercialized on the Innova systems (GE Healthcare). There exists in the literature two teams that described similar techniques we can compare to. The first one, [START_REF] Close | Improved image guidance of coronary stent deployment[END_REF][START_REF] Close | Improved Localization of Coronary Stents Using Layer Decomposition[END_REF]; [START_REF] Close | Image guidance for coronary stent deployments -us patent 6532380[END_REF], deals with transparent layer decomposition. These pioneer articles do not aim at presenting an automatic DSE algorithm. They demonstrated that motion compensated temporal denoising based on the motion of the balloon can improve the visibility of stents and help a typical user to position more accurately its edges. The second team published a patent [START_REF] Florent | Medical viewing system and method for enhancing structures in noisy images[END_REF] describing a method to perform motion compensated noise reduction based on the detection of one or several markers. They belong to the firm, Philips, that commercializes the DSE software called StentBoost. They cite the application to a pair of marker-balls for stent enhancement and describe a method. Their method is semi automatic (it requires the user to input a region of interest). It is similar to the one described here from a structural standpoint: marker candidates are detected, then pairs are formed in each image. The main difference of their approach versus ours is that they choose the best pair of markers independently on each frame without forcing the pairs to have a smooth motion within the image sequence. We notice that they pay particular attention to designing a marker ball detector that will not produce responses along uniform ridges and that they segment the guide-wire between the markers. Additionally, although they segment the guide-wire, they do not use it for registration. Because the patent only provides a description and no result or evaluation of the technique, comparison is limited. They also published on the use of maker-ball detection and tracking for 3D stent reconstruction and compared it to IVUS [START_REF] Schoonenberg | Projection-based motion compensation and reconstruction of coronary segments and cardiac implantable devices using rotational x-ray angiography[END_REF]. In this publication they briefly mention a marker-ball detection and tracking algorithm that relies on the temporal smoothness of the marker motion (as in the technique that we exposed in 1.3.1), but their main focus is the validation of the technique. Finally, two other DSE products exist, one from Siemens called IC stent and one from Paieon called StentOp. Comparisons were not possible as the technical details are not publicly available.

Observed limitations of the technique

In 7% of the cases, stents do not appear clearly after stent enhancement (IQ scores equal to 1). There are various explanations that we can propose for this. The first one is that although the registration was perfect, the input CNR was so small that noise reduction was not efficient enough to make stent struts visible. Our set of test sequences includes on purpose approximately half of the sequences in challenging imaging conditions. Although this is much more challenging than daily clinical practice, we believe that it is not a good explanation for every case. Some of the low scores have been obtained in good imaging conditions. We do not have a final explanation, however we suspect that in a significant proportion of the cases there was registration error (Fig. 18 (a)). Since the landmarks are almost always very accurately detected by our method, we think that the registration errors is due to the fact that the stent does not follow the expected motion. We identified two cases. First, there can be motion between the stent and the landmarks since they are not rigidly linked. A visual inspection of the database indicates that in some critical cases this motion can exceed 10 pixels. Our method does not behave well in these cases. Secondly, the motion of the stent occurs in 3D and may not be compensated by 2D image deformations. Fig. 17 shows a stent under two view points differing by a rotation of a few degrees around the long axis of the stent. We can observe that the appearance of the struts is completely different and therefore cannot be compensated by a 2D registration. A potential solution to theses cases where stent registration is not perfect, could be to select a subset of images in the sequences in which the motion of the stent is less significant. Ross [START_REF] Ross | Registration and integration for fluoroscopy device enhancement[END_REF] noticed on one clinical case that using a subset of frames restricted to a specific interval of the heartbeat yielded a sharper stent image. This research direction needs further studies to be confirmed.

The registration model we used is also challenged in the case of multiple stents at a vessel bifurcation. In such a case there is typically one stent in the main vessel and another one in a side branch. The presence of the landmarks in one of the vessels is not enough to predict the motion of the other vessel. Therefore the second stent is usually blurred as a consequence of the incorrect registration. This clinical situation would require extending the method to detect and register landmarks in both vessels. Fig. 18 (b) shows an example of a result of stent enhancement at a bifurcation.

(a) (b)

Figure 18: Illustration of imperfect registration in some cases (see text for details). (a) Although the landmarks are well registered and sharp, the stent struts do not appear and the upper part of the stent is blurry. (b) Case of two stents at a bifurcation. Only the marker-balls and the guide-wire supporting them are used as registration landmarks. Therefore, the stent in the side branch is not registered as accurately as the one in the main branch and is blurrier.

Another limitation to stent visualization is the presence of structures are superimposed to the stent and hide it. The landmarks for instance, and especially the guide-wire, impair the visibility of some stent struts. Moreover, in some clinical situations, highly radio-opaque sternal wires that do not follow the motion of the stent can severely degrade its visualization. Specific algorithms [START_REF] Funck | Guide-wire subtraction in enhanced stent images[END_REF] have been designed to address these limitations, but this is outside of the scope of this article.

Conclusion

We have presented a comprehensive approach to DSE. We studied the problem from a clinical standpoint to determine a successful algorithmic solution. The most original parts of our DSE technique, the guide-wire segmentation and non-linear registration, have been described and tested thoroughly. Validation on a large number of synthetic and clinical images demonstrated that DSE improves significantly image quality (by more than 1 point out of 5), works automatically (in 91% of the cases) and performs fast enough (16.6±3s) to be integrated in a typical angioplasty workflow with success. Moreover our experiments established that the use of the guide-wire as an input landmark to non-linear registration of images were beneficial to DSE in 28% of the cases. Further work may include addressing motion between the landmarks and the stent.

Figure 1 :

 1 Figure 1: (a) An image from a clinical image sequence with the clinician's tools positioned in the patient's vasculature; (b) the region in (a) delimited by the gray square with annotation of tools/devices; (c) the region in (b) by the gray square with a deployed stent; (d) demonstration of the improved stent visualization through application of DSE to (a).

Figure 3 :

 3 Figure 3: Impact of the misregistration error. Resulting DSE image subject to varying misregistration error. The images were produced by registering thirty frames with a registration error following a uniform distribution and averaging the frames. The misregistration error is indicated in pixel units below each image.

Figure 4 :Figure 5 :

 45 Figure4: Illustration of the limitations of the linear, marker-ball based registration. The two top images depict a stent at different deformations during the cardiac cycle. The bottom is the result of marker-ball based registration. Although the two marker-balls are accurately registered, they do not capture the non-linear deformation of the stent. The guide-wire that supports the markerballs undergoes a non-linear deformation similar to the stent. We use it as additional landmark in our non-linear registration.

Figure 7 :

 7 Figure 6: (a) Original image, (b) top hat transform applied to (a).

Figure 8 :

 8 Figure 8: (a) A set of parametric curves used to direct gudie-wire segmentation candidate between two markers. (b) In this clinical image, the dark line along the guide-wire depicts a parametric curve to be evaluated. The set of dark short line segments represent the perpendicular profiles that are computed for this curve. (c) Illustration of the values along one of the profiles of image (b).We see that the noise is high and there is no particular pattern in this curve. (d) Averaged profile. We see that the curve follows the expected pattern: a smooth curve with a significant minimum at its center.

Figure 9 :

 9 Figure 9: (a) Notations for the curve based registration. (b) Illustration of the motion field on the landmarks

Figure 11 :

 11 Figure 11: In dark, histogram of the percentage of the true positive per image sequence. In gray, histogram of false positive per image sequence. In white, histogram of false negative per image sequence.

Figure 13 :

 13 Figure 13: Figures illustrating the characterization of the guide-wire segmentation. (a) Input clinical image, (b) guide-wire segmented on image (a), (c) Image (a) degraded up to three times the original noise, (d) guide-wire segmented on image (c), (e) Another input clinical image, (f) Ground truth (in black) and actual segmentation (in white) of the guide-wire in image (e) degraded. The Hausdorff distance, of 12 pixels in this case, is illustrated by the dashed white line.

Figure 14 :

 14 Figure 14: Histogram of the Hausdorff distances in pixels between the ground truth guide-wires and the ones segmented on degraded images.

Figure 15 :

 15 Figure 15: Image quality scores histograms. In gray the values before DSE and in black after.

Figure 16 :

 16 Figure 16: Comparison of the linear and the non-linear registration. (a) Sequence A processed with the linear registration, (b) Sequence A processed with the non linear registration, (c) Sequence B processed with the linear registration, (d) Sequence B processed with the non linear registration.

Figure 17 :

 17 Figure 17: Illustration of the variability of the appearance due to strut superimpositions patterns. (a) and (b) depict a same stent under two view points differing from few degrees.

Table 1 :

 1 Influence of the transform

	curve-based better	=	similarity better
	B	A		A	B
	16.3%	11.6%	68.0% 2.7%	0.7%