Pierre Chatelain

Alexandre Krupa

Nassir Navab

3D ultrasound-guided robotic steering of a flexible needle via visual servoing

We present a method for the three-dimensional (3D) steering of a flexible needle under 3D ultrasound guidance. The proposed solution is based on a duty-cycling visual servoing strategy we designed in a previous work, and on a new needle tracking algorithm for 3D ultrasound. The flexible needle modeled as a polynomial curve is tracked during automatic insertion using particle filtering. This new tracking algorithm enables real-time closed-loop needle control with 3D ultrasound feedback. Experimental results of a targeting task demonstrate the robustness of the proposed tracking algorithm and the feasibility of 3D ultrasound-guided needle steering.

I. INTRODUCTION

Medical procedures such as biopsy or localized tumor ablation require the insertion of a needle towards a precise anatomical target. The accuracy of the insertion has a critical impact on the outcome of the intervention, and depends highly on the skills of the operator. Therefore robotic needle guidance has the potential to improve diagnostic and therapeutic accuracy, as it was recalled in a recent study on robotic needle guidance for prostate cancer management [START_REF] Kaye | Robotic ultrasound and needle guidance for prostate cancer management: review of the contemporary literature[END_REF]. The most commonly used imaging modality for needle guidance is ultrasonography, which presents the advantages to be noninvasive and low cost compared to other imaging modalities. The high frame rate provided by the ultrasound devices also enables real-time feedback on the needle's position.

Ultrasound-guided robotic needle insertion has become a strong area of interest in the scientific community during the past ten years. Hong et al. [START_REF] Hong | An ultrasound-driven needle-insertion robot for percutaneous cholecystostomy[END_REF] designed a robotic system to drive a rigid needle towards a moving target tracked using 2D ultrasound. This system required the needle to be aligned with the ultrasound image plane. Wei et al. [START_REF] Wei | 3D TRUS guided robot assisted prostate brachytherapy[END_REF] proposed a 3D transrectal ultrasound (TRUS) guided robotic assisted system for prostate brachytherapy, which allows oblique needle insertion trajectories. However, even rigid needles are subject to deflection due to contact forces with surrounding tissues, and their trajectory is therefore not necessarily straight. This effect is particularly strong with flexible needles, which can undergo large deformations during insertion. The tip of such needles is generally beveled, so that contact forces with the tissues are asymmetrical, leading to a natural bending. The concept of needle steering, first proposed in [START_REF] Dimaio | Needle steering and motion planning in soft tissues[END_REF], consists in taking advantage of the needle flexibility to guide it with a non-straight trajectory. The main advantage of this technique is that the needle can be guided precisely towards the target without having to perform multiple retractions and re-insertions, thus reducing the duration of the intervention. In addition, flexible needle steering allows the system to reach otherwise unattainable targets by avoiding obstacles such as vessels or bones [START_REF] Dimaio | Needle steering and motion planning in soft tissues[END_REF], [START_REF] Alterovitz | Planning for steerable bevel-tip needle insertion through 2D soft tissue with obstacles[END_REF].

The kinematic model generally adopted for bevel-tip needle steering is that of a nonholonomic bicycle model [START_REF] Webster | Nonholonomic modeling of needle steering[END_REF]. Various path planning methods have been proposed to predict the needle trajectory based on this model. Most approaches use inverse kinematics to plan the needle path preoperatively [START_REF] Duindam | Threedimensional motion planning algorithms for steerable needles using inverse kinematics[END_REF]. The desired trajectory can be achieved thanks to a dutycycling control strategy [START_REF] Minhas | Modeling of needle steering via duty-cycled spinning[END_REF], which enables precise control of the needle curvature. It has also been proposed to adjust the needle path intraoperatively. Wood et al. [START_REF] Wood | Needle steering system using duty-cycled rotation for percutaneous kidney access[END_REF] use a succession of planar arc-trajectories to steer the needle in a 2D plane. The current tip position is estimated from camera images thanks to an extended Kalman filter. Obstacle avoidance can be achieved using probabilistic methods, such as the rapidly-exploring random tree (RRT) algorithm [START_REF] Lavalle | Randomized kinodynamic planning[END_REF]. Xu et al. [START_REF] Xu | Motion planning for steerable needles in 3D environments with obstacles using rapidlyexploring random trees and backchaining[END_REF] use the RRT algorithm and back-chaining to steer the needle in a 3D environment with obstacles. Bernardes et al. [START_REF] Bernardes | Robot-assisted automatic insertion of steerable needles with closed-loop imaging feedback and intraoperative trajectory replanning[END_REF] combine the RRT-based path planning with the duty-cycling control strategy to steer a needle in a 2D plane with closed-loop visual feedback from a camera. Then they adapted this method to steer the needle along a 3D trajectory composed of a succession of 2D planar arcs [START_REF] Bernardes | 3D robust online motion planning for steerable needles in dynamic workspaces using duty-cycled rotation[END_REF]. This method was tested in a simulation environment. We recently proposed a new duty-cycling approach which permits nonplanar 3D trajectories [START_REF] Krupa | 3D steering of a flexible needle by visual servoing[END_REF]. Based on the visual servoing framework [START_REF] Chaumette | Visual servo control, part I: Basic approaches[END_REF], this approach does not require any path planning. Visual feedback was provided by an optical camera observing the needle in a translucent phantom.

The main limitation of ultrasound-guided needle steering is currently the lack of a robust needle detection technique in ultrasound images. Indeed, most of the recent control strategies have been tested with visual feedback provided by a camera observing the needle in a translucent phantom. A few studies have just been published which use ultrasound imaging to track the needle during robotic insertion. Adebar et al. [START_REF] Adebar | 3D ultrasound-guided robotic needle steering in biological tissue[END_REF] use 3D Doppler ultrasound to detect a bent-tip needle which is vibrated, thus highlighting the needle shape in the Doppler image. The needle is steered into ex vivo bovine tissues towards a fixed target using a duty-cycling controller with intraoperative re-planning based on the detected needle pose. Vrooijink et al. [START_REF] Vrooijink | Needle path planning and steering in a three-dimensional non-static environment using two-dimensional ultrasound images[END_REF] place a 2D ultrasound transducer perpendicular to the direction of insertion and automatically re-position the transducer during insertion so that the needle tip stays in the image plane. In this previous work, the needle is steered in 3D with moving obstacle avoidance using the RRT algorithm and a duty-cycle controller. In this configuration the 2D ultrasound image does not display the needle shaft, so that the orientation of the needle tip has to be estimated by integrating the displacements of the detected tip in the ultrasound image plane and the out-of-plane motion of the 2D ultrasound probe. A segmentation of the full needle shaft in 3D would enable a more accurate estimation of the needle's tip orientation.

Different algorithms have been proposed to detect the needle shaft in ultrasound images. In [START_REF] Hong | An ultrasound-driven needle-insertion robot for percutaneous cholecystostomy[END_REF] the Hough transform (HT) is used to detect a straight needle in 2D ultrasound images. The generalized HT allows the detection of more complex needle shapes, e.g. modeled by a polynomial curve [START_REF] Neshat | Real-time parametric curved needle segmentation in 3D ultrasound images[END_REF]. However the computation of the generalized HT on large 3D volumes remains computationally expensive, even with an implementation on a graphics processing unit (GPU). Alternatively, the random sample consensus (RANSAC) algorithm has been used to detect polynomial curves in 3D ultrasound volumes [START_REF] Uherčík | Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images[END_REF]. Zhao et al. [START_REF] Zhao | Tracking micro tool in a dynamic 3D ultrasound situation using Kalman filter and RANSAC algorithm[END_REF] improve the stability of this detection using a Kalman filter. In [START_REF] Chatelain | Real-time needle detection and tracking using a visually servoed 3D ultrasound probe[END_REF], we use the predictions of the Kalman filter to further reduce the RANSAC search space, and control the position of the ultrasound probe attached to a robotic arm in order to maintain the needle visibility. While the RANSAC algorithm is robust to outliers, its precision is limited, and therefore an additional optimization has to be performed in order to obtain the desired accuracy. Also, RANSAC-based techniques rely on a thresholding of the ultrasound images, and the choice of the threshold has a significant impact on the quality of the results. Learning-based segmentation techniques have also recently been proposed, using a multi-layer perceptron network [START_REF] Rocha | Flexible needles detection in ultrasound images using a multi-layer perceptron network[END_REF] or a boosted classifier [START_REF] Hatt | Enhanced needle localization in ultrasound using beam steering and learning-based segmentation[END_REF].

In this paper, we propose a new algorithm for tracking a flexible needle in 3D ultrasound, based on particle filtering [START_REF] Gordon | Novel approach to nonlinear/non-Gaussian Bayesian state estimation[END_REF]. This algorithm provides an accurate real-time detection of a robot-guided flexible needle, and does not require any prior learning. In contrast with the RANSAC-Kalman approach, the area of interest is defined intrinsically by the model, and the critical thresholding step is avoided. In addition, we use this new needle tracking algorithm as a closed-loop feedback for our recent duty-cycling control strategy [START_REF] Krupa | 3D steering of a flexible needle by visual servoing[END_REF], leading to a 3D ultrasound-guided needle steering technique.

We present our methods in section II. We start by describing the model used to represent the needle (II-A), then we detail our new needle tracking algorithm (II-B) and the 3D needle steering technique (II-C). In section III we present the results of experiments performed on a gelatin phantom.

II. METHODS

A. Needle model

Flexible needles can bend during insertion due to contact forces with the surrounding tissues. Thus, the shape of the needle is usually modeled as a polynomial curve [START_REF] Uherčík | Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images[END_REF], [START_REF] Chatelain | Real-time needle detection and tracking using a visually servoed 3D ultrasound probe[END_REF], as represented in Fig. 1. With this model the set of points forming the needle at a given time t can be defined as:

z n x n C (1) t tip C (3) t C (4) t entry point F n C (2) t ω z v z r z us x us F us
C (a, H t ) = H t [ 1 a . . . a n-1 ] (1)
where H t ∈ R 3×n is the matrix of parameters, n -1 is the order of the polynomial, and a ∈ [0, 1] is the curvilinear coordinate. By convention, we will consider that a = 1 corresponds to the tip of the needle, and a = 0 to the entry point, i.e. the intersection between the needle and the border of the field of view. The order n -1 of the polynomial determines the degree of flexibility of the shape. Possible values are, for example, n = 2 for a straight line, n = 3 for a parabola, and n = 4 for a cubic curve. Higher-order polynomials might lead to instability due to over-fitting.

The polynomial curve can be equivalently defined by a set of n control points C (i) t ∈ R 3 , i = 1, . . . , n. In our model we define these control points as equally spaced between the entry point C

(1) t and the tip C

(n) t : C (i) t = C i -1 n -1 , H t (2)
We present in section II-B a new tracking algorithm to infer the shape of the needle from 3D ultrasound images. In addition to the shape of the needle, we need to know the orientation of the tip bevel. To this end, we parameterize the rotation of the needle around its axis with an angle θ t , defined with respect to a reference orientation. We assume that there is no torsion of the shaft; hence the set of parameters (H t , θ t ) fully defines the needle pose at the time t.

Following the nonholonomic needle motion model described in [START_REF] Webster | Nonholonomic modeling of needle steering[END_REF], the needle when inserted without rotating the shaft follows a planar arc trajectory of radius r (see Fig. 1) in a plane defined by the bevel orientation. The natural curvature K = 1 r depends on the mechanical properties of the needle and surrounding tissues. Controlling both the insertion velocity v z and the rotational velocity ω z allows the adjustment of the effective curvature of the trajectory. In addition, by adjusting the shaft rotation angle θ t corresponding to the bevel orientation, we can change the plane in which the needle evolves, leading to 3D needle steering. We detail this control scheme in section II-C.

B. Needle tracking

In the Bayesian tracking framework, the target is defined as a state vector x t following dynamics of the form x t = f t (x t-1 , ν t-1 ), where f : R N × R N -→ R N is the dynamics function modeling the evolution of the target, and ν t-1 is an independent and identically distributed (i.i.d.) state noise. The purpose of tracking is to estimate the state vector x t based on some measurement z t = h t (x t , µ t ), where µ t-1 is an i.i.d. measurement noise.

In our case, the state vector corresponds to the needle's position, which we model as the concatenation of all control points defining the needle:

x t = C (1) t . . . C (n) t (3)
The dynamics of the system is supposed to be known through an estimation of the external needle insertion velocity. For instance, such estimation can be made by the odometry of a robot inserting the needle or via an external tracking device. We assume that the needle does not buckle during insertion, so that the scalar insertion velocity at the needle tip corresponds to the insertion velocity component of the insertion device. Given an insertion velocity v t ∈ R over a period δt, the state update equation for the needle tip can be written as:

C (n) t = C (n) t-1 + v t δtu t-1 (1) + ν (n) t-1 (4)
where ν

(n) t-1 ∈ R 3 is the tip's state noise and u t-1 (1) is the unitary direction vector of the curve at the needle tip, which can be computed analytically as:

u t-1 (a) = H t-1 [ 0 1 2a . . . (n -1)a (n-2) ] ||H t-1 [ 0 1 2a . . . (n -1)a (n-2) ] || 2 (5)
Considering that the ultrasound probe does not move with respect to the subject, the entry point is defined as a fixed point in the volume frame. Therefore its update equation is simply

C (1) t = C (1) t-1 + ν (1) t-1 , where ν (1)
t-1 ∈ R 3 is the entry point's state noise. The noise model for the entry point allows the system to correct its position if the needle drifts due to tissue deformation.

As all other control points are defined to be equally spaced between the entry point and the needle tip, their state update equation can be approximated with:

C (i) t = C (i) t-1 + i -1 n -1 v t δtu t-1 i -1 n -1 + ν (i) t-1 (6)
where ν

(i)
t-1 is the state noise for the control point C (i) , which allows the detection of needle shaft drifting. The motion of the ultrasound probe could also easily be incorporated in the system model by modifying ( 4) and ( 6) in a similar way as in [START_REF] Chatelain | Real-time needle detection and tracking using a visually servoed 3D ultrasound probe[END_REF].

The key idea behind particle filtering is to approximate the posterior distribution of the tracked object with a large number M of random samples x m t , m = 1, . . . , M (the particles) with an associated weight w m t . In our case each particle represents the shape of a needle modeled as a polynomial curve, and the weights are updated according to the likelihood of the particle given the current ultrasound image. That is:

w m t ∝ p(z t |x m t ) (7)
with the constraint ∑ M m=1 w m t = 1. The term z t corresponds to the ultrasound image observed at time t.

The current state is approximated from this discrete posterior distribution as:

x t = M ∑ m=1 w m t x m t ( 8
)
The computation of the weights requires some model of the needle's appearance in the image in order to approximate the likelihood p(z t |x m t ). For example, one could apply statistical learning techniques to estimate this likelihood. In this paper we consider a simple model which presents the advantage to be very cost-effective, and that does not require any prior learning. We propose to use directly the intensity of the voxels along the needle's shaft. More specifically, we consider the mean intensity V m t along the curve defined by the particle x m t :

V m t = 1 L 1 a=0 V (C (a, H m t ))da (9)
where L is the length of the curve and V (P) is the intensity at the position P. Indeed, the intensity of the echos reflected by the needle is in general higher than that of soft tissues. Even if the needle's appearance can be discontinuous (the shaft does not always reflect very well, and the corresponding intensities can be in the same range as surrounding speckle), integrating the intensities along the curve allows discrimination between the different particles. In addition, the needle's tip typically appears as a very bright spot due to reflections on the bevel. We take this observation into account in our model, and define the weight w m t as a trade-off between the mean shaft intensity and the tip intensity:

w m t ∝ V m t + αV (C (1, H m t )) ( 10
)
where α is a positive constant. Giving more weight (higher α) to the tip intensity ensures a precise detection of the needle tip. An advantage of this measure in terms of computation time is that we only need to access the intensities of the volume along the curves defined by the particles, and not in the whole volume.

Then we use a sequential importance re-sampling filter (SIR) to update the particles during tracking. The initial state is determined manually, and the weights are initialized equally as w m 0 = 1 M . At each iteration the SIR performs the following operations:

1) Update the particles according to the model's dynamics. 2) Re-compute the weights w m t according to [START_REF] Lavalle | Randomized kinodynamic planning[END_REF]. 3) Estimate the current state according to [START_REF] Minhas | Modeling of needle steering via duty-cycled spinning[END_REF]. 4) Re-sample. The re-sampling step is designed to avoid the degeneracy phenomenon, where the weight of one particle tends to 1 while the others are negligible. The degeneracy of the system can be detected by computing the effective number of particles:

N e f f = 1
∑ M m=1 (w m t ) 2 (11)
In order to avoid degeneracy, the particles are re-sampled when the effective number of particles falls below a threshold N 0 . In this case, the particles are re-sampled M times with replacement with probability proportional to their weight, and the weights are then reset to 1 M . The objective of resampling is to eliminate particles with small weight while generating more particles from those with higher weights.

The proposed SIR-based needle tracking algorithm is designed to track a curved needle in ultrasound images using an estimation of the insertion velocity v t applied at the needle base, provided by the robot odometry or an external tracking device. The particle filter scheme allows the system to correct for imprecisions in the velocity estimation and to detect the bending of the needle, based on ultrasound measurements. We will now detail the second part of our contribution, that is using the ultrasound-based needle tracker as a closed-loop feedback for needle steering.

C. 3D ultrasound-based needle steering

The duty-cycling technique relies on applying a succession of two types of needle motion:

• pure insertion, where the needle follows an arc of natural curvature K, • insertion and fast rotation, where the needle follows a straight line. The duty-cycling technique enables control of the effective curvature K e f f of the needle by alternating the two phases with a specific ratio between the combined rotation/insertion period T rot and the pure insertion period T trans :

K e f f = K(1 -DC) (12)
where DC = T rot (T rot + T trans ) -1 is the duty-cycle ratio. We denote T = T rot + T trans the total duty-cycle period.

We recently proposed a 3D duty-cycling control strategy for robotic needle steering [START_REF] Krupa | 3D steering of a flexible needle by visual servoing[END_REF]. In this method we control continuously (at the rate of the duty-cycle period) the two lateral angular velocities (n) ω x , (n) ω y and the insertion velocity (n) v z of the needle in the needle tip frame F n . These velocities are linked together by the equations:

(n) ω x = cos(θ )K e f f (n) v z ( 13
) (n) ω y = sin(θ )K e f f (n) v z ( 14
)
where θ is an accumulation angle applied to the needle around its axis during the current duty-cycle period, such that the absolute angle θ t obtained at the end of the cycle becomes θ t = θ t-T + 2π + θ . Therefore controlling the effective curvature K e f f acts on the amplitude of (n) ω x and (n) ω y , while controlling θ acts on the ratio between (n) ω x and (n) ω y . detected by the SIRbased needle tracker described in section II-B, as well as the unitary direction vector u t (1) defined in [START_REF] Alterovitz | Planning for steerable bevel-tip needle insertion through 2D soft tissue with obstacles[END_REF] to compute the visual features s = [ l xz l yz ] . These features correspond to the distance l xz (resp. l yz ) between the target point O and its projection onto the needle direction line projected on the x-z (resp. y-z) plane of F us (see Fig. 2(a)):

✓ y ✓ x O O N N needle! needle! target! target! projection in plane x-z! projection in plane y-z! d xz d yz l yz l xz F us F us ! x us ! z us ! z us ! y us (a) (b)
l xz = d xz sin(θ y ) (15)
l yz = -d yz sin(θ x ) (16)
Notice that the visual features can be computed directly in the ultrasound volume frame, so that no calibration is required between the robot frame and the volume frame. Finally, we use the control law we proposed in [START_REF] Krupa | 3D steering of a flexible needle by visual servoing[END_REF]:

(us) ω x (us) ω y = -λ J -1 ω xy (s -s * ) (17)
where s * = [ 0 0 ] is the desired value of the visual feature vector, λ > 0 is the control gain, and J ω xy is the Jacobian that relates the variations of the visual features to the needle's lateral velocities us ω x and us ω y expressed in the ultrasound volume frame F us . This Jacobian only depends on measures extracted from the ultrasound volume by the needle detector:

J ω xy = 0 d xz cos(θ y ) -d yz cos(θ x ) 0 ( 18
)
where d xz and d yz denote the length of the projection of the vector -→ NO on the x-z and y-z plane respectively. In practice, we then express the lateral control velocities (n) ω x and (n) ω y of the needle's tip in the frame F n , thanks to a velocity transformation that depends on the direction vector u t (1) defined in [START_REF] Alterovitz | Planning for steerable bevel-tip needle insertion through 2D soft tissue with obstacles[END_REF] and the angle θ t , provided respectively by the tracking algorithm and the odometry of the robot. This provides the effective curvature K e f f and the accumulation angle θ to use in the duty-cycling control:

K e f f = 1 (n) v z (n) ω 2 x + (n) ω 2 y (19) θ = atan (n) ω y (n) ω x (20)
As the proposed features are invariant to the insertion velocity (n) v z , we can fix this velocity to a constant, or leave it to the appreciation of the clinician in the context of semiautonomous needle insertion. The insertion velocity v t = (n) v z is used to update the tracker state model in [START_REF] Webster | Nonholonomic modeling of needle steering[END_REF].

III. RESULTS

In this section we present experimental results of our 3D ultrasound-guided needle steering technique. We detail our experimental setup in section III-A, and present the results of a targeting experiment in section III-B).

A. Experimental setup

For the experiments a 22 gauge bevel-tip flexible needle (Angiotech Chiba MCN2208) is rigidly attached to the endeffector of a 6-DOF Viper s650 robot (Adept Technology Inc., USA). The needle is inserted into a home-made gelatin phantom simulating soft tissues. The scene is observed with a 4DC7-3/40 3D motorized ultrasound transducer (Ultrasonix Medical Corporation, Canada) which is maintained still. The experimental setup is presented in Fig. 2(b). The ultrasound volumes are reconstructed from raw data using the 3D scan conversion method described in [START_REF] Lee | Intensity-based visual servoing for non-rigid motion compensation of soft tissue structures due to physiological motion using 4D ultrasound[END_REF]. In the prescan volumes, the linear resolution of the scan-lines is 0.308 mm, the angular resolution of the frames (angle between two consecutive scan-lines) is 0.609 • , and the angle between two consecutive frames is 1.46 • . The dimensions of the reconstructed volumes are 255 × 379 × 171 voxels, and the side of a voxel is 0.65 mm. The scan conversion and image processing are performed on a workstation (Intel Core i7, NVIDIA Quadro K2000) which receives the ultrasound data from the SonixTouch ultrasound system and enables communication with the robot. In order to provide visual feedback to the user, we extract and display two orthogonal slices where the current needle position is highlighted.

B. Needle steering

For our 3D needle steering experiments the needle is initially inserted into the phantom until the tip is visible in the ultrasound volume. The initial position of the needle is detected manually by clicking in the x-z and y-z planes. Given this initial detection, the target is defined 8 cm away from the needle tip and with an offset of 7 mm with respect to the line defined by the direction vector of the needle (5 mm in the x direction and 5 mm in the y direction). For our SIR-based needle tracker we use n = 4 control points (cubic curve), M = 1000 particles, and an isotropic Gaussian noise model with a standard deviation σ (n) = 1.0 for the tip, σ (2) = σ (3) = 0.25 for the inner control points, and σ (1) = 0.0 for the entry point. Setting the noise variance to 0 for the entry point was satisfactory in our setup where the probe was not moving and no drift of the entry point was observed. In a first experiment we inserted a needle into the phantom without duty-cycling control, with a constant insertion velocity v * z = 1 mm/s and no angular velocity. The observed trajectory of the needle allowed a rough estimation of its natural curvature to K = 2 m -1 .

We now describe the experiment performed to test our needle steering technique, using the control law [START_REF] Vrooijink | Needle path planning and steering in a three-dimensional non-static environment using two-dimensional ultrasound images[END_REF]. The needle is inserted with a constant insertion velocity v * z = 0.5 mm/s, and the control gain is set to λ = 0.01. The position of the needle in the ultrasound volume at different instants is presented in Fig. 3. One can see that the target is successfully reached by the needle tip, with a final positioning error of 1.08 mm. Fig. 4(a) and (b) show the trajectory of the needle projected on the x-z and y-z planes. The feature errors l xz and l yz are reported in Fig. 4(c) and (d). Both features converge, with a final feature error of 0.453 mm for l xz and 1.07 mm for l yz . These errors are close to the resolution of the reconstructed volume (0.65 mm). The difference of precision between the two features can be explained by the lower resolution of the prescan ultrasound volume in the direction corresponding to the probe sweep, which is less than half the angular resolution of the frames.

IV. CONCLUSION

We proposed a new algorithm for tracking a flexible needle in 3D ultrasound, and demonstrated its utility in the context of ultrasound-guided 3D needle steering. Our method was validated through a targeting experiment in a gelatin phantom, where the target was defined as a fixed point in the ultrasound volume. In the case of organic tissues, it is of interest to define the target as an anatomical target, which might be displaced due to tissue motion. In future work we will consider the use of target tracking in conjunction with our framework in order to perform an insertion in biological tissues in a dynamic environment.

Fig. 1 .

 1 Fig. 1. Representation of the needle as a polynomial curve with 4 control points. Controlling the translational velocity v z and the rotational velocity ω z allows the adjustment of the radius of curvature r of the trajectory. F n represents the frame attached to the needle tip, and F us the frame attached to the ultrasound volume.

Fig. 2 .

 2 Fig. 2. (a) Projections of the needle and target in the image 3D frame. (b) Experimental setup.

  Visual error l yz (cm).

Fig. 4 .

 4 Fig. 4. Results of the targeting task by visual servoing: (a-b) needle trajectory projected on the x-z and y-z planes, (c-d) visual feature errors.

Fig. 3 .

 3 Fig. 3. Position of the needle in the ultrasound volume during automatic insertion, presented as a projection in the x-z (a-d) and y-z (e-h) plane. The needle detected by the tracker is represented as a red line, and the target as a green cross. The yellow arrow corresponds to the direction vector of the needle at the tip, and the blue points represent the position of the needle tip for all particles.

Pierre Chatelain is with Université de Rennes 1, IRISA, France and Computer Aided Medical Procedures, Technische Universität München, Germany pierre.chatelain@irisa.fr

Alexandre Krupa is with Inria Rennes -Bretagne Atlantique, IRISA, France alexandre.krupa@inria.fr

Nassir Navab is with Computer Aided Medical Procedures, Technische Universität München, Germany and the Johns Hopkins University, MD, USA nassir.navab@in.tum.de

ACKNOWLEDGMENT

This work is partially supported by the BFS funding for project RoBildOR.