Loranne Vernisse
email: loranne.vernisse@univ-poitiers.fr

Olivier Guillermet

André Gourdon

Roland Coratger

Interaction between perylene-derivated molecules observed by low temperature scanning tunneling microscopy

Keywords: STM, NC-AFM, Low Temperature, Perylene derivatives, Single Molecule, Molecular orbitals PACS: 68.37.Ef, 81.07.Nb

Derivative perylene molecules deposited on Ag(111) and on NaCl(001) ultrathin layers have been investigated using low temperature STM and NC-AFM.

When the metallic substrate is held at ambient temperature during evaporation, the molecules form characteristic trimers on the Ag(111) surface and interact through their polar groups. Close to the steps, the molecules form linear structures and seems to stand side by side. On the other hand, after deposition on a substrate cooled at liquid helium temperature, single molecules are observed both on metal and on NaCl. On the ultrathin insulator layers, the STM images present characteristic contrasts related to the molecular orbitals which favours the localization of aldehyde groups. In this case, the lateral molecular interactions may induce the formation of small assemblies in which the electronic levels are slightly shifted. A possible interpretation of this phenomenon is to take into account polar interactions and charge transfer between neighboring molecules.

Introduction

In order to create electronic devices based on molecules, one of the principal challenges is to be able to observe and control molecular assemblies. Thereafter, these assemblies will be used to study charge transfer for example between a single molecule and a polymeric structure [1] or the choosen molecules will be the precursors of larger systems able to facilitate charge transfer [2,3]. The molecular interactions within these assemblies is often driven by competitive forces either between the molecules in the assembly [4] or between the molecules and the substrate on which they are deposited [5] leading to different molecular lattices. Then, very different behaviours are expected between networks dominated by covalent bonding or by hydrogen interactions. In covalent assemblies, the intermolecular forces entirely determine the network structure since the chemical bonds are directive and very strong [6,7]. Then, the molecular shape is used to produce stable networks of the desired geometry and reactivity depending of the used tecton [8]. On the other hand, hydrogen interactions are weaker and sometimes results in metastable structures in which the number of H bonds determines the final self assembly [9]. However, these large molecular domains assembled by non covalent forces are technologically appealing since they can make possible the fabrication of nanostructures with controlled functionality, size and shape [10]. Most of the studies reported so far concern large domains in which an important number of molecules are self-assembled and participate to the observed network [11]. However, when the domains are very small, it appears interesting to study the interactions between the first molecules implied in the structure [12,13,14]. This may concern only a few molecules and in this case, their precise position in the native network can reflect the nature of the interactions involved in the first growth steps. Thus, the formation of these first structures can be used to predict the nature of the intermolecular interactions, an important point that cannot be always addressed in a complete monolayer. Among the large number of possible candidates, perylene derivative molecules are objects of peculiar interest for various applications. Indeed, they are used in photonics and optoelectronics, where they can be used as pigments or colorant [START_REF] Aftergut | Red perylene dichroic dye containing liquid crystal formulations[END_REF], in light-emitting diodes [16], transistors [17], photovoltaic cells [18,19].

Optoelectronic characteristic of perylene derivative compounds are due to the π-conjugated system of its polyaromatic structure. On the other hand, it is possible to change the characteristics of these molecules using appropriate peripheral groups, particularly if these groups are polar [20,21].

In our investigations, we propose to use perylene derivative molecules as twodimensional models in the localization and interaction of local charges. Bearing this mind, we studied 3-perylene carbaldehyde molecules (PC) by STM and NC-AFM on an Ag(111) surface and on ultrathin films of NaCl. The PC molecule is a precursor of numerous perylene derivatives. Its chemical structure is presented Fig. 1. It is composed by a perylene group (in black) and an aldehyde group (in red). Although the molecule is neutral in gaz phase, its charge repartition is expected to be non uniform because of the peripheral group. Then, this molecule is going to be used as a reference system in the exploration of molecular models with various local charges and their possible interactions. The synthesis of PC has been carried out according to [22]. The molecules were evaporated from a W filament 0.15 mm in diameter. This evaporator was first pumped by a turbo molecular pump during at least 12 h. Then, before evaporation, the filament was submitted to high temperature outgassing cycles.

Evaporations on the substrate held at room temperature were performed in the preparation chamber The filament was heated at 0.34 A during 30 sec.

Afterwards, the substrate was immediately transfered in the STM chamber to be cooled down at liquid helium temperature. The time between deposition and cooling process is estimated to be about 5 minutes, which is relatively large as compared to the typical diffusion time of molecules on metallic surfaces. One can therefore estimate that these conditions are sufficient to reach equilibrium in the formation of structures involving several molecules. Evaporations have also been conducted with the substrate held at liquid helium temperature, i.e. directly on the STM head through a small hole 1 mm in diameter drilled in one of the windows of the He cryostat. The filament was heated at 0.34 A during 2 min.

The small solid angle supports very small evaporation rates with typical values in the 1/1000 monolayer range. These conditions were used for single molecule experiments in which thermal diffusion was inhibited. A particularity of PC molecules were their strong interaction with the different UHV chambers and also with the electronic connections in vacuum. This involved tip oscillations after several days of experiments and led to systematic baking at a temperature of 150 °C for 24h. Therefore, to prevent irremediable contamination, only low coverage experiments were performed. STM tips made of tungsten wires 0.2 mm in diameter were prepared by electrochemical etching and cleaned in UHV using direct current heating. Bias voltage was always applied to the sample.

The STM is equipped with a tuning fork of the qPlus sensor type allowing noncontact atomic force microscopy. The resonance frequency of the tuning forks used in this study was typically 25 kHz and their quality factor was in the 10000 to 20000 range. NC-AFM/STM tuning fork tips were made of platinumiridium wires 0.05 mm in diameter cut by focused ion beam once glued to the cantilever.

Results and discussion

Evaporation at ambient temperature. The first molecular assemblies of PC have been observed after deposition of molecules with a submonolayer coverage on the Ag(111) surface held at room temperature. The surface was then cooled down around 4.5 K for STM imaging. The molecules appears as small rectangles (1.1 nm wide and 1.35 nm length) with an apparent height of about 0.15 nm at -1 V. This geometry is close to the shape of the molecular structure. After deposition and diffusion on the flat terraces, small assemblies are produced. They are mostly composed of three molecules in a characteristic triangular organization (Fig. 2a and2b), where each monomer is oriented with an angle of 60°with respect to the others in a clockwise or anticlockwise orientation, giving chiral assemblies (Fig. 2c). The characteristic and reproducible geometry of these trimers suggests an interaction due to the polar groups resulting from a specific organization of the H bonds including the aldehyde moieties and the hydrogen atoms of the three molecules. Because of the molecular asymmetry due to the COH part, chiral structures are favored and each molecule arranges to favor the maximum of interaction with its neighbors. Few assemblies involving more than three molecules were observed. For example, a couple of five molecule assemblies were observed on terraces (black arrow Fig. 2a). In this case, the behavior is always the same: extra molecules only come in addition to the triangular feature. However, such structures were mostly found next step edges (red arrow Fig. 2a), and around defects such as sulfur atoms always present on the Ag(111) surface (black arrow Fig. 2b).

Molecules can be adsorbed in a head to tail configuration (Fig. 2b). This only occurs along step edges under the influence of the well-known Smoluchowski effect and the induced local dipolar moment [23]. To confirm the role of the COH moiety, the same type of perylene but with another peripheral group (perylene ethanoic acid: PEA) was also deposited on Ag(111) using the same experimental conditions (see supplementary materials). In this case, the structures were very different with a complete panel of assemblies including monomers, dimers, 115 trimers,... In addition, no regular alignment along step edges were observed for PEA. This suggests that the PC trimer structures clearly result from the presence of the COH group while the regularity of the linear chain along the step implies a head to tail organization.

The PC trimers were also investigated by NC-AFM. Before switching into the constant-height mode to obtain the ∆f images, the tip was positioned above the metal surface with tunneling parameters identical to those previously used for STM imaging (1.2 pA and -400 mV). Thereafter, this height was kept constant and was not modified during ∆f imaging. The bias voltage of -400 mV was determined using ∆f (V) curves and was chosen so that the contribution due to the electrostatic force was minimized. This value compensates the average contact potential difference (CPD) between tip and sample.

The NC-AFM image of the frequency shift at constant height is shown Fig. 3a. For clarity, the molecular structure has been superimposed in Fig. 3b. The dark circle surrounding the trimer is mostly a consequence of attractive Van der Walls interactions and geometric effects of the structure. The carbon cycles are clearly distinguishable inside each monomer, especially inside monomer 1.

On this image, only perylene groups are resolved and the bond localization of the aldehyde group remains unclear. A first hypothesis consists to consider that the HC=O bond is in a different horizontal plane, which could hide it.

A second hypothesis, more realistic, is based on the close dependence between the tip height and the probed interactions [24,25,26]. Indeed, C=O bonds were already observed by Zhang et al. in 8-hydroxyquinoline (8hq) molecules adsorbed on Cu(111) with a CO functionalized tip [27]. They highlighted the fact that observation of the C=O bond of this molecule requires to get closer to the surface to 30 pm. As a small change of the tip height during the image is therefore enough to change the type of interaction probed, it is possible that the HC=O bond is not probed. Future experiments are going to be performed to study C=O bonds as a function of the tip height. However, this image perfectly shows the chiral structure of the trimer and is also in good agreement with the structural model presented in Fig. 2c.

Evaporation at low temperature. The first experiments suggest that single molecules cannot be observed when the deposition is performed at ambient temperature.

As diffusion induces the production of small clusters (stabilized by intermolec- For this and to preserve the electronic properties of the molecules, PC molecules were evaporated on a Ag(111) surface partly covered by NaCl islands. These islands generally have a thickness of two monolayers, denoted as NaCl(2ML)/Ag(111). Thereafter, deposition of molecules with a submonolayer coverage at a sample temperature held at 4.5 K led to individual molecules adsorbed both on the metallic surface and on the thin insulator.

The molecules present the same rectangular shape as previously described.

However, the low temperature hinders diffusion and makes it possible to study single objects before the formation of trimers. Figure 4 shows PC molecules adsorbed on Ag(111) (A) and NaCl (B), and imaged at a positive bias voltage of 1.5 V. If the molecules present the rectangular shape described above on Ag(111) (molecules 1), the same molecules observed at the same bias on NaCl exhibit a more complex geometry (molecules 2).

Once adsorbed on the ultra thin NaCl layer, the molecules present an interesting contrast that depends on the sign and value of the used bias voltage. with different sizes and shapes. These lobes are always the same whatever the position of the molecules with respect to the NaCl surface. This is the case in Fig. 4 in which the single molecules appear identical although molecule labeled 2 is close to a small defect and another one close to a step edge. This means that a few angstroms away from these defects, the effects of these latter on the NaCl electronic structure such as the well-known Cl vacancy state reported by Repp et al. [28] do not affect the observed structures. However, we note in specific cases, a small shift of the threshold voltage for LUMO imaging when molecules are very close from a island edge. This shift is about 100 meV, does not appear systematically and depends on the molecule orientation with respect to the step direction.

These features results from the electronic structure of the molecule and the well-known decoupling properties of the ultra-thin NaCl layer. I(V) spectra were also acquired on PC molecules adsorbed on NaCl and We now focus our attention on the images produced above the LUMO onset.

A small scale STM topography of this geometry and its computed counterpart are represented on Figs. 6a and 6b. The calculated image has been computed using the coupling between the atomic orbitals of each atom inside the molecule and a single s orbital of the tip (in this case a Cu atom is used for simplicity).

Details of these calculations are given in supplementary materials.

The good agreement between the experimental images and the calculated ones suggests that this simple model is sufficient in this case to interpret the STM results. The observed and calculated structure consists of two main lobes, The comparison shows that it is perfectly possible to localize the aldehyde group as the lobe 3 on the STM image.

Using this imaging method above the LUMO threshold, the small depressions previously observed in Fig. 5b can be located with respect to the molecular structure (lobe 3) and interpreted as the aldehyde moiety. Depressions on topographic STM images have previously been observed around molecules and can involve local charge transfer with oxygen atoms [29,30]. To confirm that the observed dark contrast below the Fermi level could reflect a charge transfer, A first interpretation could be to invoke the adsorption location on the NaCl(001) surface [31]. Using Fig. 7a, an intermolecular distance of 1.34 nm has been measured. On the one hand, this distance is not consistent with the different adsorption sites on the NaCl surface (squared lattice with a parameter of 0.4 nm). On the other hand, this energy shift has never been observed on single PC molecules except if these latter are adsorbed exactly on a defect for example straddling a step. Moreover, the positions of the PC molecules are randomly distributed on the NaCl surface after evaporation on the cold substrate since diffusion is blocked and molecules adsorb as it is. Then, it appears highly unlikely that these observations are linked to the adsorption site except if one admits that the trimer formation induces three unstable molecule positions that cannot be reached by a single molecule. This result can also be due to different adsorption heights leading to different hybridizations with the surface. In this case, it should be supposed that this mechanism is perfectly progressive over the three objects, leading to a quasi-linear evolution of the molecule-surface distance from molecule 1 to molecule 3. A more likely interpretation of this phenomenon is to take into account the intermolecular interactions due to the CHO peripheral groups. Such a phenomenon has already been reported for linear oligomers composed of trimers and tetramers connected by hydrogen bonds [32]. In this work, the calculations performed in the gas phase show that the electronic characteristics of the connected monomers and particularly the frontier orbitals along the small assemblies are significantly modified by charge transfer.

Due to this phenomenon, the HOMO and LUMO states are each located at one end of the chain. For PC molecules, it has been previously demonstrated that charge transfer has been decisive in the trimer formation on the Ag(111) substrate at ambient temperature (Fig. 2c) and in the adsorption along the steps.

Because of the proximity of the three molecules in the row and the decoupling role of the NaCl layer, it is possible that the electrostatic interactions between polar groups play a significant role and induce a small charge transfer.

To support these assumptions, calculations have also been performed. Three PC molecules in gas phase have been positioned in the same plane at an interatomic distance corresponding to the experimental results (i.e., 1. center to center) and with the same orientation (see Fig. 8). The position of the frontier orbitals has been investigated using a semi-empirical approach and the MOPAC code with the PM6 method [START_REF] Young | Computational Chemistry: A Practical Guide for Applying 405 Techniques to Real World Problems[END_REF]. The results are reported in Table 1. These calculations clearly show a shift of the energy levels of molecule 1 and

Figure 1 :

 1 Figure 1: Chemical structures of 3-perylene carbaldehyde (PC) molecule

Figure 2 :

 2 Figure 2: STM images of PC molecules (a) 100 nm x 100 nm (b) 50 nm x 50 nm (V bias =2 V, It=2 pA, T=4.5 K) (c) 12 nm x 12 nm STM image (V bias =-1.5 V, It=2 pA, T=4.5 K) of two molecular assemblies with their schematic representation respectively above and below the STM image.

Figure 3 :

 3 Figure 3: (a) Image of the frequency shift ∆f on a PC trimer (z=cst, V bias =-400 mV, I=1.2 pA) (b) Superposition of the chemical structure of perylene and the ∆f image

Figs. 5a and

  Figs. 5a and 5b show three PC imaged respectively at positive and negative bias voltages. Note that the molecule located on top has been slightly moved by the tip between the two images. When a negative voltage is applied, the molecule

Figure 4 :

 4 Figure 4: STM image after adsorption of PC molecules on a Ag(111) surface (A areas) partially covered by a bilayer of NaCl (B area) (V bias =1.5 V, I=1 pA, T=4.5 K). The black square shows an assembly of 3 molecules.

Figure 5 :

 5 Figure 5: Observation of three PC molecules at: a) positive bias voltage (1.5 V). b) Negative bias voltage (-1.5 V). (I=1 pA, T=4.5 K)). c) dI/dV spectrum performed on a PC molecule adsorbed on NaCl. d) Profile of a PC observed at negative bias voltage showing a small depression (also indicated by black arrows in (b).

labeled 1 and 2 ,

 2 slightly asymmetric, and two antennas, the first composed by one lobe (3) and the second by two lobes (4 and 5). Figs. 6c and 6d present the ball and stick representation of the molecule and the calculated contribution of the LUMO states. Both are oriented in the same direction as Figs. 6a and 6b.

Figure 6 :Figure 7 :

 67 Figure 6: a. STM images corresponding to the LUMO of a PC molecule adsorbed on a NaCl bilayer (V bias =1.2 V, I=1 pA, T=4.5 K) b. Computing image of the energy of the overlap of the HOMO and the 4s orbital of a copper atom (tip) c. Ball-and-stick representation of a PC molecule adsorbed on a NaCl bilayer. d. Calculated positive and negative contributions of the LUMO. The molecule orientation is the same in the four images.

Figure 8 :

 8 Figure 8: Schema representing the assembly of three PC molecules on NaCl(2ML)/Ag(111)

3

  as compared to the central molecule or a single molecule. This shift (about -0.23 eV for 1 and +0.2 for 3) is of the same order than the value experimentally obtained (0.1 eV) and with an evolution from molecule 1 to 3 consistent with the STM observations. Note also that the characteristics of the central PC are very close to those of a free molecule. Thus, the left molecule receives an excess of charge -δq. As a consequence, the energy difference between the Fermi level and the LUMO state decreases and this state is observed at a lower voltage. On the contrary, the right molecule, which transfers this charge to the central one, looses a charge -δq. In this case, the E F -LUMO increases and the unoccupied state is observed at a higher voltage. In this process, the central molecule remains uncharged and behaves as expected from the single molecule experiments. This kind of phenomenon is obviously impossible on a metal and is only allowed by the decoupling properties of the thin NaCl layer.

  In this paper, PC molecules deposited on a Ag(111) substrate and on NaCl ultra-thin layers have been investigated by STM and NC-AFM at low temper-ature. The peculiar structure of the molecule peripheral groups induces the formation of characteristic trimers at ambient temperature while the observation of single objects is possible when the sample is cooled at 4.5 K. On ultrathin NaCl layers, it is well known that the molecule shape is related to its molecular orbitals. Using simple STM image calculations, we demonstrate that the localization of the aldehyde group is then possible. The formation of small clusters suggests that these peripheral groups favor the electrostatic interactions and this could slightly change the relative position of the frontier orbitals with respect to the Fermi level. From this point of view, the experiments conducted on the NaCl layer bring new insigths in the interactions between derivative perylene molecules. At ultra-low coverage, these investigations can help to explain the geometry of the molecular assemblies that take place on a metallic susbtrate or at least illustrate what are the dominant intermolecular forces within these domains.

Table 1 :

 1 Results of the PM6 calculations in the gas phase for a single PC and for three PC molecules each separated by 1.3 nm and with the orientation presented in Fig.8.

	3 nm from

PC adsorbed on NaCl.


Paul Mignon

Philippe Poignet

Jocelyne Troccaz
email: jocelyne.troccaz@univ-grenoble-alpes.fr

Title (max 116car): Automatic Robotic Steering of Flexible Needles from 3D Ultrasound Images in Phantoms and Ex-vivo Biological Tissue Abbr. title (max 55car): Experimental evaluation of robotic needle steering

Keywords: needle steering, needle detection, 3D ultrasound, echogenic needle, ex-vivo tissue, robotics

Robotic control of needle bending aims at increasing the precision of percutaneous procedures. Ultrasound feedback is preferable for its clinical ease of use, cost and compactness but raises needle detection issues. In this paper, we propose a complete system dedicated to robotized guidance of a flexible needle under 3D ultrasound imaging. This system includes a medical robot dedicated to transperineal needle positioning and insertion, a rapid path planning for needle steering using bevel-tip needle natural curvature in tissue, and an ultrasound-based automatic needle detection algorithm. Since ultrasound-based automatic needle steering is often made difficult by the needle localization in biological tissue, we quantify the benefit of using flexible echogenic needles for robotized guidance under 3D ultrasound. The ``echogenic'' term refers to the etching of microstructures on the needle shaft. We prove that these structures improve needle visibility and detection robustness in ultrasound images. We finally present promising results when reaching targets using needle steering. The experiments were conducted with various needles in different media (synthetic phantoms and ex-vivo biological tissue). For instance, with nitinol needles the mean accuracy is 1.2mm (respectively 3.8mm) in phantoms (resp. biological tissue).

INTRODUCTION

Needle-based procedures are among the most common diagnostic or therapeutic gestures, from "simple" drug injection to cancer biopsy. Most often, the practical conditions or the difficulty to visualize the surgical zone and the anatomical structures, reduce the overall accuracy of the percutaneous action. In some cases, this accuracy may critically influence the success of the clinical procedure. In clinical practice, the needles must be rigid, and clinicians try to avoid needle bending during insertion, for instance by rotating the needle. Rarely, in specific cases, the clinician may apply forces on the needle base to bend it in order to reach areas usually inaccessible because of the presence of bones or critical structures. The development of medical robotics has led to the introduction of a novel approach based on automatic flexible needle steering in order to generate curved trajectories towards a target in presence of obstacles. Such trajectories can be produced by the application of forces on the needle base to bend it. This has a significant drawback since the required intensity increases with the insertion depth. The resulting stress on the tissue might create traumas. Another way of bending the needle is obtained by biological tissue manipulation and deformation but several clinical applications are not compatible with such an approach. A third steering approach -the one we selected -makes use of asymmetric bevel-tip needles. Forces applied by biological tissue reaction on the tip during insertion deflect the needle in the direction of the sharpened side of the bevel. Finally, alternative approaches consist in developing specific needles [START_REF] Ko | Closed-loop planar motion control of a steerable probe with a "programmable bevel" inspired by nature[END_REF] or other devices such as concentric tubes. [START_REF] Dupont | Design and control of concentric-tube robots[END_REF] More extensive citations can be found in review papers. [START_REF] Rossa | Issues in closed-loop needle steering[END_REF][START_REF] Van De Berg | Design choices in needle steering-A review[END_REF] In all cases, robotic needle steering requires real-time abilities to detect a needle and a potentially moving target from images, to plan needle trajectory and to determine robot's inputs to reach that target.

Trajectory Planning

Earlier research projects used the application of forces and torques on the needle base in order to bend it. This required accurate modeling of the needle insertion and bending in living tissue: finite element models [START_REF] Dimaio | Needle steering and model-based trajectory planning[END_REF] or virtual spring methods [START_REF] Glozman | Image-guided robotic flexible needle steering[END_REF] have been proposed. The latter reached real-time performances. More recently, the same ``base manipulation'' approach using 2D ultrasound (US) as feedback was proposed. [START_REF] Neubach | Ultrasound-guided robot for flexible needle steering[END_REF] In Ref. [START_REF] Rossa | A Hand-Held Assistant for Semiautomated Percutaneous Needle Steering[END_REF] , a beveled needle is modeled as a cantilever compliant beam which oscillation modes are estimated. Most often, the beveled needle deformation has been approximated using a nonholonomic model called unicycle model. [START_REF] Webster | Nonholonomic modeling of needle steering[END_REF] This model is used for planning by several groups. [START_REF] Alterovitz | The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty[END_REF][START_REF] Park | Diffusion-based motion planning for a nonholonomic flexible needle model[END_REF] To make it computationally efficient, a 3D algorithm based on a rapidly-exploring random tree (RRT) was introduced. [START_REF] Xu | Motion planning for steerable needles in 3D environments with obstacles using rapidly-exploring random trees and backchaining[END_REF] RRT is well-adapted for fast re-planning from real-time information (electromagnetic feedback [START_REF] Patil | Needle steering in 3-D via rapid replanning[END_REF] or 2D image feedback [START_REF] Bernardes | Robot-assisted automatic insertion of steerable needles with closed-loop imaging feedback and intraoperative trajectory replanning[END_REF] ). These last two works generate the robot control inputs using the duty-cycling approach for 2D [START_REF] Wood | Needle steering system using dutycycled rotation for percutaneous kidney access[END_REF] and 3D 37 automatic needle steering. Duty cycling allows controlling the curvature of the needle during insertion by alternating high-speed rotation and no-rotation periods with a specific duty-cycle. During rotation periods, the needle follows a straight line while it bends during insertion with no rotation. The resulting trajectory can be modeled by a circular path with a specific curvature. This method simplifies path planning algorithms by using arcs with different radii of curvature.

Needle Tracking

Needle localization in soft tissue is a challenging issue. It can be partly addressed using localizers which are commonly used during computer assisted medical intervention. However, because they cannot give any information about needle deformation during insertion, Ref. [START_REF] Abayazid | 3d flexible needle steering in soft-tissue phantoms using fiber bragg grating sensors[END_REF] proposed to use mechanical strain sensors integrated on the needle shaft. Although promising, this requires complex calibration and/or the integration of sensors inside medical devices and it is not yet available for clinical routine.

Medical imaging modalities remain the most common way to detect a needle and a mobile target. X-ray imaging offers low visibility of soft tissues. It is also associated with risks of irradiation for patient and clinical staff. Magnetic Resonance Imaging (MRI) has good performance in visualizing soft tissues [START_REF] Patel | Closedloop asymmetric-tip needle steering under continuous intraoperative MRI guidance[END_REF] but it suffers from incompatibility with metallic objects, reduced working space and availability issues for interventions. Ultrasound is a safe, convenient and low-cost imaging modality. Recently, two groups 2,3 developed needle steering systems using RRT re-planning and US feedback in ex-vivo tissue. However, in both systems, the needle detection algorithms constrain the needle orientation relative to the ultrasound image plane.

US noise and artifacts make needle detection difficult. This has stimulated extensive research for the past years. Needle segmentation in 2D US images has been addressed using multiple Hough transforms. [START_REF] Okazawa | Methods for segmenting curved needles in ultrasound images[END_REF] More recently, a new needle segmentation algorithm based on ultrasonic spectral analysis was proposed. [START_REF] Beigi | Needle trajectory and tip localization in real-time 3-D ultrasound using a moving stylus[END_REF] The method benefits from the clinician's hand tremor when holding the needle. Unfortunately, it does not work in real-time. Regarding 3D US images, the first challenge is to quickly process large and noisy volumes. 2D detections by Hough transform in planar projections of the volume may be combined for 3D localization. [START_REF] Uhercik | Multi-resolution parallel integral projection for fast localization of a straight electrode in 3D ultrasound images[END_REF] In Ref. [START_REF] Abayazid | Ultrasound-guided three-dimensional needle steering in biological tissue with curved surfaces[END_REF] , a motor translates the 2D US probe to detect and follow the needle tip during insertion. Others authors [START_REF] Adebar | Recursive estimation of needle pose for control of 3Dultrasound-guided robotic needle steering[END_REF] proposed to reconstruct the entire needle shaft using a frame-by-frame algorithm associated with an unscented Kalman filter and color-Doppler imaging. These last two algorithms both constrain the needle orientation with respect to the image acquisition plane, which may limit their clinical applicability. Finally, the Random Sample Consensus algorithm (RANSAC) was also introduced for direct 3D flexible needle detection with short computation time [START_REF] Uhercik | Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images[END_REF] and applied to needle steering. [START_REF] Chatelain | 3D ultrasound-guided robotic steering of a flexible needle via visual servoing[END_REF] In our previous work [START_REF] Mignon | Beveled-tip needle-steering using 3D ultrasound, mechanical-based Kalman filter and curvilinear ROI prediction[END_REF] , we have associated a 3D RANSAC algorithm with a mechanical-based model to predict a region of interest and to filter the segmentation result. This method makes the needle detection more robust. This robust feedback is then used for needle steering using a brachytherapy robot prototype and rapid-replanning RRT approach. Preliminary testing has been conducted using paraffin gel phantoms and PVC phantoms.

The step from phantoms to ex-vivo tissue is very challenging. First, ex-vivo tissue is generally highly heterogeneous, leading to large differences between planned and actual paths. Moreover, needle segmentation is very difficult. This is due, among other reasons, to: miscellaneous tissue structures visible in ultrasound, ultrasound signal attenuation with depth, artifacts or needle orientation relative to the US propagation direction. Some existing methods allow enhancing needle visibility. For instance US waves can be oriented in a direction perpendicular to the needle to maximize the reflected signal and to increase needle visibility. [START_REF] Cheung | Enhancement of needle visibility in ultrasound-guided percutaneous procedures[END_REF] This method is already embedded in commercial products of Ultrasonix, Sonosite or General Electric. However, this requires prior knowledge about needle orientation and it only works with straight rigid needles. Echogenic needle design has also been explored. Microstructures located on the surface of the needle have been used to diffuse the incoming US waves and to increase signal reflection back to the transducers. The microstructures are generally grooved along the needle shaft but could also be embedded in a polymer coating. [START_REF] Bergin | Echogenic polymer coating: does it improve needle visualization in sonographically guided biopsy?[END_REF] Refs. [START_REF] Kamada | A quantitative comparative study of a new echogenic needle for nerve bocks[END_REF][START_REF] Park | Phantom study of a new laser-etched needle for improving visibility during ultrasonography-guided lumbar medial branch access with novices[END_REF] describe the use of echogenic needles during ultrasound hand-guided nerve block interventions. To the best of our knowledge, none of the robotized needle steering systems used echogenic needles to achieve controlled insertion in biological tissue, and the impact of these modified needles on automatic needle detection has never been quantified.

Contributions

In this paper, we introduce a needle steering system based on a medical robotic prototype dedicated to transperineal needle insertion. [START_REF] Hungr | A 3-D ultrasound robotic prostate brachytherapy system with prostate motion tracking[END_REF] We describe how this medical robot is controlled using a RRT-based rapid re-planning approach and a unicycle model. The aim is to obtain an advanced integrated system for flexible needle insertion. The approach is explained in section 4. The feedback used for needle guidance is based on a needle tracking algorithm using a RANSAC algorithm combined with the computation of a dynamic region of interest (ROI) using a mechanical-based prediction model. This novelty allows determining the future position of the ROI from one US volume to another. We also introduce the use of echogenic needles for improving detection performances. Finally, we evaluate in section 5 the full system on a significant number of experiments on phantoms and ex-vivo biological tissue.

MATERIALS AND METHODS

General approach

The control of needle deformation during its insertion in a synthetic or biological tissue is based on rapid re-planning: the needle path is continuously updated from information about the needle and target positions obtained by processing 3D US images. This rapid re-planning control can compensate for disturbances, such as tissue deformation or patient motion as well as potential modeling errors or simplifications. It can provide high reliability despite imperfect modeling of biological tissue, needles and their interactions.

As shown in Fig. 1, the controller is divided in two modules. The high-level control is composed of needle segmentation, filtering, path planning and duty-cycling control (DCC). The low-level control is based on the PROSPER robot 14 itself.

Trajectory planning and duty-cycling

Trajectory planning is based on combining the unicycle model and a 3D RRT algorithm, inspired by previous work. [START_REF] Abayazid | Ultrasound-guided three-dimensional needle steering in biological tissue with curved surfaces[END_REF][START_REF] Bernardes | Robot-assisted automatic insertion of steerable needles with closed-loop imaging feedback and intraoperative trajectory replanning[END_REF][START_REF] Patil | Needle steering in 3-D via rapid replanning[END_REF] The developed method generates paths with at most two arcs. This is justified by the low curvature achievable in biological tissue and the narrow field of view offered by the US probe. Searching for simpler feasible paths reduces computation time without loss of generality.

Each time a US volume has been acquired, reconstructed and processed, the algorithm selects the best path among all the possible paths generated from the current position to the target. The optimal path has a minimum length and a maximum distance to the obstacles. The method returns three parameters (L, , ) representing the selected circular path A (see Fig. 2) to be executed by the robot for the next motion. L is the insertion length, between 0 and L max , the length of the arc that reaches the target.  is the arc curvature and is bounded by  max , the maximum curvature of the beveled-tip needle.  in [0,360] is the angle defining the orientation of the plane in which the arc is included. These three parameters are then transformed in robot commands by the DCC algorithm. The duty-cycle DC corresponds to the rotation time of the needle divided by the insertion time. A simple equation allows computing DC from the curvature of the planned arc: DC = 1-/ max . DC is then converted into insertion and rotation time for PROSPER taking into account velocity limits when specified.

Needle detection and tracking

As mentioned, RANSAC can rapidly detect a curved needle in 3D without constraining the needle orientation. Its input is a set of candidate voxels resulting from 3D image thresholding. The threshold was chosen in order to select a given percentage of brighter voxels in the region of interest, based on the needle size. For our experiments, 5% was the optimal choice. The candidate voxels contain not only the needle but also noise and artifacts. To determine the needle position, RANSAC randomly selects a set of voxels and fits a polynomial curve to them; then it tests all the voxels of the initial set in order to classify them as inliers (on the needle) or outliers, depending on their distance to the curve. The maximum acceptable distance for inliers is half the needle diameter as visible in an US image. In our set-up, the needle diameter in the image is about 3 mm. This process is repeated n times and RANSAC selects among the n curves, the one having the largest number of inliers. This algorithm has proved to be efficient when the image is good enough but it may fail when the signal coming from the needle is low. Thus, we proposed to increase the segmentation robustness in US images by defining a curvilinear region of interest (ROI) -see Fig. 3 -around the needle prediction using Kalman filtering. Two simple models were tested for predicting next needle position: one kinematic model based on needle tip position and insertion velocity and one mechanical model describing needle-tissue interaction by virtual springs. Parameters of this model were selected from force measurements during insertions of needles in phantoms. These models and the parametrization, as well as first performance evaluation are described in ref. [START_REF] Mignon | Beveled-tip needle-steering using 3D ultrasound, mechanical-based Kalman filter and curvilinear ROI prediction[END_REF] . Because needle visibility in biological tissue could results in much lower needle detection performance we also investigated the use of echogenic needles (see Section 4.5).

Target tracking

The target corresponds to a desired location computed by a planning system: for instance based on the dose planning for brachytherapy. In our needle steering experiments, the user manually defined the desired final location of the needle tip in the initial US volume. Because biological tissues are deformable and may move during the needle insertion, the target has to be tracked. The block-matching algorithm [START_REF] Strintzis | Maximum likelihood motion estimation in ultrasound image sequences[END_REF] that we used consists in exhaustively searching for the target neighborhood of the previous image in a largest region of interest of the current image. This algorithm has been validated using ex-vivo tissue (see Section 4.5).

Hardware and software environments

US-guided brachytherapy robot and computing environment

The experimental setup is composed of the PROSPER robot, an Ultrasonix RP ultrasound device and a 4DEC9-5/10 endorectal 3D US probe. The US voxel size is 0.4x0.4x0.4mm. The US machine, running Windows, integrates an IntelCore 2(1.8GHz) processor with 2Go RAM. Each frame of the 3D US volume is acquired using the Ultrasonix library and is sent to an external computer with a TCP/IP connection. This computer, which runs the application, is a Windows IntelCore i7 (3.4GHz) with 16Go RAM and equipped with a NVidia Quadra 600 graphical card with 1Go memory. The code written in C++ was developed in the open source framework CamiTK (Computer-assisted medical intervention ToolKit -cf. http://camitk.imag.fr/). The external computer is responsible for 3D volume reconstruction, needle segmentation, path planning and robot control. 3D US acquisition and reconstruction lasts about 1 s. The segmentation takes between 300 and 700 ms (depending on the size of the ROI) and the planning time is around 150 ms. Thus, the acquisition time is longer than the segmentation and planning time and runs in parallel to them. The feedback frequency is therefore limited by the time required to acquire and compute each US volume. In the described experiments, the maximum needle insertion speed was set to 0.5mm/s.

The PROSPER robot has seven motors organized in two independent modules. Five of them are dedicated to the needle positioning out of the patient. The two other motors control the insertion and the rotation of the needle inside the body (see Fig 4). Previous work [START_REF] Hungr | A 3-D ultrasound robotic prostate brachytherapy system with prostate motion tracking[END_REF] demonstrated the robot ability to position the needle tip on predefined physical targets (beads) in prostate deformable phantoms. [START_REF] Hungr | A realistic deformable prostate phantom for multimodal imaging and needle-insertion procedures[END_REF] Before entering the phantom, the needle was aligned with the physical target. The mean distance between the needle tip and the target was 2.98 mm, measured on more than 100 insertions. With straight needles, the robot could only compensate prostate motion and deformation in the insertion direction. Along this direction, the accuracy was in average 1.61 mm.

Echogenic steerable needles

We used two different types of 30° beveled needles for steering experiments: nitinol needles (Ø 0.5 mm) and stainless steel biopsy needles (Ø 0.71 mm). Nitinol, a super-elastic Nickel/Titanium alloy, allows the needle to make highly curved paths. Biopsy needles are less deformable but correspond to clinically used needles. We have not quantified the achieved curvature since 3D steering with double bending makes it quite difficult. Nitinol needles with 0.5mm diameter typically have a radius of curvature of about 500/600mm -it depends on the elasticity of the medium in which the needle is inserted. Biopsy needles have a larger radius of curvature. As proposed in previous work, [START_REF] Wedlick | Characterization of pre-curved needles for steering in tissue[END_REF] all needles used in this study were pre-bent 3 mm distally from the tip to artificially increase bending. This resulted in smaller radii of curvature: in some of the experiments we have observed locally a radius of curvature of 30 to 50mm for nitinol and around 100mm for steel needles.

The signal reflected from a needle to the US sensor is maximum when the main axis of the needle is perpendicular to the US wave propagation direction. To make needles more visible, modifying their surface quality provides scattering effects that may compensate for a reflected signal loss. CO2 laser etching makes it possible to groove the needle in order to obtain regularly spaced diffusing structures (see Fig. 5). The diffusion process requires a spatial frequency of the structure (d) smaller than the ultrasound wavelength .  = /f where f and  are the transducer frequency and the sound velocity in the medium. In our case, f is 7 MHz and a common estimate of the sound velocity in soft tissue is 1540 m/s. Thus, the spacing of the scattering structure must be less than 220. We chose 150 , a value used for the EchoBright™ needle sold by Halyard.

Experiments

Detection and tracking:

The ground truth is given by manually fitting the needle model -a Bezier curve -to the images acquired during and after insertion. Visibility improvement of echogenic needles was quantified in 2D and 3D images with three different environments: water, paraffin phantom and ex-vivo tissue (pork tenderloin). For each medium, a standard needle and an echogenic one with the same characteristics (diameter, material, etc.) were inserted. The insertion angle was the same for the two conditions (standard versus etched) and did not influence needle visibility. The image threshold and RANSAC parameters (number of iterations, criterion to select inliers, etc.) were also similar. The RANSAC algorithm was applied 100 times for each needle. Since the choice of the ultrasound plane could influence the needle visibility in 2D, we applied the same process to 3D volume with both types of needles. In 2D and 3D cases, the result of each detection was classified as successful or unsuccessful depending on its distance to the ground truth. An error at the needle tip greater than 2 mm in position and 10° in orientation was declared unsuccessful.

Regarding needle tracking, in previous works [START_REF] Mignon | Beveled-tip needle-steering using 3D ultrasound, mechanical-based Kalman filter and curvilinear ROI prediction[END_REF] , we compared in a single experience the kinematic and mechanical models for ROI prediction with Kalman filtering when this paper concerns 20 different image sequences of needle insertions (7 in ex-vivo tissue, 13 in paraffin phantoms).

Target tracking:

The block matching was implemented on a GPU. It reached a run time of 150 ms, using a block size of 5x5x5 mm and a searching area of 10x10x10 mm. The method was validated on an ex-vivo tissue (pork tenderloin) covered by paraffin gel. A target motion was simulated by moving the US probe with respect to the phantom. The US probe was attached to the PROSPER robot endeffector and a controlled displacement was performed describing the 15mm edges of a planar square. During this movement, the probe head was always in contact with the paraffin gel. A visible target was selected in the image of the ex-vivo tissue at the beginning of the experiment. The detected motion was compared to the controlled displacement.

Needle steering:

Three different environments were used to evaluate the ability to steer a flexible needle: homogeneous phantom, heterogeneous phantom and ex-vivo tissue (pork tenderloin).

The heterogeneous phantom was composed of paraffin gel with Vybar™ polymer (from Baker Hughes). The polymer modifies the phantom elasticity. Several layers were created with different Vybar™ concentrations to mimic the brachytherapy environment including the skin, the perineal tissue and the prostate capsule. Fig. 6 shows a schematic of the layers and US shear-wave elastography measurements with the Aixplorer™ from Supersonic Imagine.

For these three types of environment, echogenic needles were inserted under several conditions regarding the presence of obstacles and the motion of the target.

For all experiments, a target position was defined manually by the user in the initial image. In order to show the potential of needle steering in compensating for unexpected inaccuracies and perturbations, the needle initial trajectory (before insertion) was given an artificial misalignment to the target. The definition and value of this initial misalignment are given in Table 2. The initial position of the needle was reached using the "positioning module" of the robot while insertion experiments were conducted using the two degrees of freedom of its "insertion module"

In the phantoms, block matching could not be used for target tracking because of a lack of background structures (due to phantom transparency to US). Thus, we applied a virtual displacement to this target, during the insertion, to simulate an additional perturbation as it could be detected with a real target. In the ex-vivo tissue, both static and moving targets were tested. In the latter case, the tracking algorithm updated the target positions.

In both homogeneous and heterogeneous phantoms, the needle steering experiments were conducted with and without virtual obstacles. In contrast, needle steering was tested without obstacles in ex-vivo tissue. Indeed, the observed natural radius of curvature of needles (around 150mm) and the maximum insertion length allowed by PROSPER (60mm) make obstacle avoidance very difficult in this case.

Obstacles, tested for phantoms, were spheres whose diameter ranged from 8 to 20mm. Most of them were positioned in the first half of the space separating the initial tip position from the target.

Needle steering was also tested using echogenic biopsy needles in ex-vivo tissue. Five insertions towards mobile targets were conducted without considering obstacles. The target was tracked using the block-matching method. Since the biopsy needle is more rigid than the nitinol one and has a larger radius of curvature, the introduced misalignments were smaller than for the nitinol needle experiments. These experiments are presented separately since the initial conditions are not similar to experiments with nitinol needles.

RESULTS

Detection and tracking

Fig. 7 shows the US images corresponding to each medium with both echogenic and standard needles. The luminance profiles displayed for each needle clearly show that the luminance is higher for echogenic needles in each image. In addition, standard needles appear to create more artifacts resulting in a worse overall appearance.

Returned results, in terms of successful detections, for water, paraffin phantom and pork tenderloins are 2%, 1% and 6% respectively with standard needles against 100%, 100% and 90% for echogenic ones. The detection rate is definitely better for the echogenic needle in each environment both in 2D and in 3D. This confirms the benefit of using echogenic needles.

Regarding the tracking by optimizing the ROI, using the kinematic model only reached 65% of success whereas the tracking of the needle using the mechanical model reached 100% of success. This shows that the mechanical model significantly increases the detection robustness in the 20 sequences.

Concerning target tracking, the error between the measured motion of the target in the image and the square trajectory of the probe was 0.11 +/-0.9 mm.

Nitinol needle steering experiments

The reported positioning results correspond to the final distance separating the needle tip from the target. The results of all the experiments are summarized in Table 3. For each case, five insertions were performed. The mean, standard deviation and extreme values are given and discussed in the next sections.

In the homogeneous phantom, the final positioning error between the needle tip and the target is 1.2+/-1.1 mm. With one obstacle, the final error increases up to 2.4+/-1.4 mm due to the high curvature of the path and the difficulty of reaching the target after obstacle avoidance. The results are very promising considering the US volume quality and its spatial resolution.

In the heterogeneous phantom, the perturbation generated by the heterogeneity of the material increases the positioning error up to 2.1 +/-1.1 mm without obstacle, which can also be considered as a good result taking into account ultrasound quality. With the presence of obstacles, this error increases up to 2.4 +/-1.8 mm, which is approximately the same as in homogeneous phantom. The results could be explained by the large perturbation caused by the obstacle that obviates the influence of phantom heterogeneity.

In the ex-vivo biological tissue, the results are promising with an average error of 3.8 +/-1.7 mm with a static target and 4.4 +/-1.0 mm with a mobile target tracked using the block-matching algorithm. The measured motion of the target was 1.1 +/-0.5 [0.4; 1.6] mm. A typical experiment is illustrated in Fig. 8.

In ex-vivo experiments, the high heterogeneity of biological tissue greatly disturbs the needle steering and it results in increasing the positioning error. Moreover, an important phenomenon has been observed during ex-vivo needle insertions: in some cases, unexpected changes of needle direction occurred, particularly when the needle rotates. These unexpected movements could be explained by the winding of nerves or muscle fibers around the needle that results in force application on the needle shaft. These disturbances greatly influence the final error in some cases.

Biopsy needle steering experiments

Finally, similar experiments on ex-vivo tissue with a tracked target and no obstacle were performed using biopsy needles. The measured motion of the target was 1.0 +/-0.9 [0.1; 2.4] mm. The tip position error for five steering experiments are 2.2 +/-0.6 [1.5; 2.8] mm.

All these experiments are demonstrated in the associated video1 .

DISCUSSION

Steering experiments on ex-vivo tissue are more challenging than phantom experiments due mainly to needle visibility limitation and to the complex structure of the environment. Using echogenic needles is a way of dealing with this visibility issue. In this first study, the echogenic needles had a very simple design and were grooved only on one side. The experiments have nevertheless shown that such echogenic needles offer improved visibility in several insertion environments. This even allowed guiding needle in ex-vivo tissue with high robustness. IFuture works include the test of our needle steering system on living tissue for full validation.

However, it should be noted that, for approximately 15% of the ex-vivo needle steering experiments, the grooved needles broke due to a large bending. In this case, the experiments had to be repeated with new needles. This phenomenon never occurred with standard needles used in previously published work nor during our experiments in phantoms with modified needles. We assume that our process of needle etching weakened the needle by creating cracks in the material. The winding of tissue around the needle, adding to this weakness, probably results in this amount of needle braking. In the future, mechanical tests should be performed on echogenic needles to evaluate changes in their mechanical properties. An optimized etching process should also be investigated in order to make the needle stronger while keeping flexibility and visual enhancement. It is also necessary to ensure via histopathology analysis that the grooves do not increase the damage to the tissue compared to a conventional needle.

We also observed that the rotation of the needle in ex-vivo tissue sometimes induced unpredictable needle bending. Our hypothesis is that this bending is due to the winding of muscular or nervous fibers around the needle during rotation. This creates unpredictable external forces applied to the needle. In previous work, 3 similar unexpected deviations were observed when the needle collided vessels in a beef liver. Ref. [START_REF] Tsumura | Histological evaluation of tissue damage caused by rotational needle insertion[END_REF] also mentions this issue and describes how its occurrence is detected using a force sensor located on the needle base. This observation, coupled with the potential increase of tissue damage caused by needle rotation, provides another argument against the use of continuous rotation to guide needle insertion into biological tissue. In the near future, our control must be optimized [START_REF] Rucker | Sliding mode control of steerable needles[END_REF] to reduce the needle rotation, for instance by replacing it with alternating rotations (clockwise and counterclockwise). We believe that this improvement could greatly increase the steering precision of our system by avoiding unpredictable deformations. We also believe that it will reduce the potential damage done to the tissue due to needle rotation. Torsional models [START_REF] Reed | Modeling and control of needles with torsional friction[END_REF] could also be investigated for increased accuracy.

The direct comparison to previous similar experiments (US, ex-vivo) is made difficult due to the different experimental conditions and way of reporting. Ref. [START_REF] Abayazid | Ultrasound-guided three-dimensional needle steering in biological tissue with curved surfaces[END_REF] used targets in ex-vivo chicken tissue embedded in a gelatin phantom. Experiments include obstacle avoidance. The insertion distance is in average much longer that ours since it ranges between 86 and 103mm; let us remind that we are limited by the size of the 3D US volume. Reported accuracy is 1.82+/-0.58mm. Since a very significant part of the trajectory is in the gelatin phantom, this accuracy has probably to be compared to our heterogeneous phantom experiments. In Ref. [START_REF] Adebar | 3-D ultrasound-guided robotic needle steering in biological tissue[END_REF] , experiments conducted in beef liver result in an average error of 1.57mm for six insertions. As reported by the authors, many other trials resulted in unexpected deviations of the needle generating much larger errors. These "failures" are not included in the statistics whilst our reported statistics include all experiments except when the needle broke. It should also be underlined that, in the two previously cited papers, the reported experiments do not include any initial misalignments where we introduce significant initial errors. Finally, we can compare our performance to linear insertion of a biopsy needle initially aligned with the target using the Prosper robot. [START_REF] Hungr | A 3-D ultrasound robotic prostate brachytherapy system with prostate motion tracking[END_REF] In the heterogeneous phantom, the average accuracy was 2.98 mm compared to 2.2 mm, measured here, when steering the biopsy needle in ex-vivo tissue (cf. 5.3) with a significant initial misalignment (see table 2). Therefore, the new method demonstrates its ability to reach a target by steering a needle despite large perturbations simulated by the initial target misalignment.

TABLES, FIGURE LEGENDS

Figure 1 :

 1 Figure 1: Schematic of the needle steering approach based on rapid re-planning.

Figure 2 :

 2 Figure 2: Parametric description of generated arcs: insertion length L (a), curvature  (b) and path angle  (c).

Figure 3 :

 3 Figure 3: Schematic of the curvilinear 3D ROI selection using a side margin of 2 pixels and a tip margin of 1 pixel.

Figure 4 :

 4 Figure 4: Experimental environment for needle steering on ex-vivo tissue. (a) Prosper general architecture, (b) set-up

Figure 5 :

 5 Figure 5: Schematic (a) and picture (b) of the Nitinol needle after fiber laser etching.

Figure 6 :

 6 Figure 6: Experimental setup ready for needle steering on ex-vivo tissue and elastography measurements.

Figure 7 :

 7 Figure 7: Ultrasound image of echogenic and standard needles and luminance profiles in water (a), paraffin phantom (b) and pork tenderloin (c). The blue and red dotted lines in the US images correspond respectively to the echogenic and standard needle detections. The other bright line in the images corresponds to the interface of the phantom with the plexiglas support.

Figure 8 :

 8 Figure 8: Result of a needle guidance in ex-vivo tissue. (a) XY projection of the final pose of the needle, (b) XZ projection of the final pose of the needle and (c) needle tip trajectory during insertion.

Table 1 :

 1 Main existing flexible needle-steering systems and innovations. Legend: Needle loc: CMOS=camera, EM=electromagnetic localizer, 2.5D US= 2D US probe moved externally,OL=optical localize ; Planning: IK=inverse kinematics, S.Anneal=simulated annealing, V.Servo=visual servoing; Model: FE=FiniteElements, VS=virtual springs, VB=vibrating beam;U=unicycle; Validation: Simu=simulation, Ex=ex-vivo tissue, Ph=phantom.

	Ref	Base/	Guidance Needle	Planning	Model Obstacle	Constraint	Valid
		bevel-		localization			avoid.	on needle
		tip						orientation
	[11]	Base	2D	none	IK	FE	x		Simu
	[12]	Base	2D	X-ray	IK	VS	x		Ex
	[17]	Base	2D	2D US	IK	VS	x		Ph
	[31]	Bevel	2D	CMOS	IK	U	x		Ph
	[6]	Bevel	2D	CMOS	Rapid-RRT U	x		Ph
	[32]	Bevel	3D	CMOS+EM RRT	U	x		Ph
	[3]	Bevel	3D	2.5D US	RRT	U	x	x	Ex
				Doppler
	[22]	Bevel	3D	EM	Rapid-RRT U	x		Ex
	[2]	Bevel	3D	2.5D US	Rapid-RRT U	x	x	Ph+Ex
	[21]	Bevel	3D	MRI	IK	U			Ph
	[24]	Bevel	2D	2.5D	S.Anneal.	VB		x	Ph+Ex
				US+OL
	[9]	Bevel	3D	3D US	V.Servo	U			Ph
	Our	Bevel	3D	3D US	Rapid-RRT U	x		Ph+Ex
	system

Table 2 :

 2 Initial misalignments (distance between the target and the initial needle direction -mean +/-standard deviation [min; max], in mm).

	Medium	Homogeneous	Heterogeneous	Ex-vivo biological
		phantom	phantom	tissue
	Needle
	Nitinol	9.81 +/-6.26	10.39 +/-6.16	7.95 +/-3.63
	needle	[0.3; 24.47]	[3.67; 21.51]	[0.67; 11.63]
	Biopsy			5.74+/-2.25
	needle			[3.38; 9.32]

Table 3 :

 3 Accuracy of the nitinol needle tip position in different media with different conditions (five trials per medium and per condition -mean +/-standard deviation [min;max], in mm).

	Conditions	Static target	Moving target
		No obstacle	With obstacle	No obstacle
	Homogeneous	1.2 +/-1.1	2.4 +/-1.4
	phantom	[0.5; 2.4]	[0.7; 4.8]
	Heterogeneous	2.1 +/-1.1	2.4 +/-1.8
	phantom	[0.6; 3.0]	[0.4; 5.2]
	Ex-vivo biological	3.8 +/-1.7		4.4 +/-1.0
	tissue	[0.5; 5.6]		[3.2; 6.0]

See https://youtu.be/0mDiwXyhjD8

We developed a needle steering system based on 3D ultrasound images. The use of echogenic needle greatly improves the detection rate of our needle detection algorithm. This improvement allows guiding a needle in biological tissue in closed-loop control using a RRT based rapid replanning. This development, integrated with a prostate brachytherapy robot prototype, is a significant step toward clinical application of robotic needle steering. Even under the presence of unpredicted disturbances, probably due to tissue winding around the needle, our statistics on needle tip positioning error are promising. This work shows that needle steering can be an asset for many needle-involved robot-assisted surgical interventions. However, significant work certainly still remains to prove the non-invasiveness of the approach in living biological tissue.

ACKNOWLEDGEMENTS:

This work was partly supported by the French ANR within the ``Investissements d'Avenir'' program (Labex CAMI) under reference ANR-11-LABX-0004.


Laurent Guillo
email: laurent.guillo@irisa.fr

Xiaoran Jiang
email: xiaoran.jiang@inria.fr

Gauthier Lafruit
email: gauthier.lafruit@ulb.ac.be

Christine Guillemot
email: christine.guillemot@inria.fr

In response to the call for Visual Test Material for immersive applications [N17462, N17471] video content acquired with a R8 Raytrix camera and converted into 5x5 Multiview format is proposed. Each view is provided with a disparity map.

Introduction

In response to the call for Visual Test Material for immersive applications [N17462, N17471] video content acquired with a R8 Raytrix camera [1] and converted into 5x5 Multiview format is proposed with a disparity map associated to each view. Three different scenes have been captured by the camera. The video content, capturing conditions and how the views were extracted are detailed below. Then available data for download are listed.

Description of the video content

Three different scenes have been captured with a R8 Raytrix video camera equipped with a 35mm lens. These videos are:

• ChessPieces,

• Boxer-IrishMan-Gladiator,

• ChessPieces-MovingCamera.

These videos were recorded at 30 fps with the Raytrix software RxLive 3.1 [2]. Each frame has a 1920x1080 resolution. The first frame of each video is displayed in Figure 1 : The Raytrix R8 camera equipped with a 35mm lens can precisely capture a scene included in a 10cm x 10cm x 10cm volume. The proposed captured scenes are included in such a volume.

Chess Pieces

The video sequence "ChessPieces" contains four chess pieces, two knights and tow pawns turning on a turn table and with a calendar as background. The R8 Raytrix camera was located at 35cm from the scene. It is composed of 300 frames.

Boxer-IrishMap-Gladiator

The video sequence "Boxer-IrishMan-Gladiador" contains three Lego characters turning on a turn table and with a calendar as background. It is composed of 300 frames.

ChessPieces-MovingCamera

In the video "ChessPieces-MovingCamera" the scene, composed of chess pieces, is static but the camera is moving. The latter is located at 60 cm from the scene. The video contains 300 frames.

Extraction of views

The Raytrix software RxLive [2] stores data received from the R8 camera as a Raytrix sequence file. We used the Raytrix API [3] to access each Raytrix frame in the Raytrix sequence file in order to extract twenty-five views as if we had twenty-five virtual cameras (5 rows x 5 columns). Each camera is supposed to be 4 mm away from its neighbour. To do so, we take advantage of the function Rx::ApiLF::RxSetPar . Called with the parameter "Rx::ApiLF::EPar::VirtCamPinholeStd_ViewOffsetMM_g", it allows to specify an offset of a virtual camera in the plane perpendicular to the z-axis of main lens. The offset has a horizontal value of 4 mm and a vertical value of 4 mm and is measured from the center of main lens. The

Disparity maps

For each view of each frame, a disparity map was computed. As we have extracted 25 views per frame, 25 disparity maps were computed per frame: one disparity map for each extracted view. To do so, we used a simplified version of works presented in [START_REF] Jiang | Depth estimation with occlusion handling from a sparse set of light field views[END_REF] and apply it to the extracted views.

Below are, for instance, the disparity maps of the central view (i.e. coordinates (3,3)) that we obtained for the first frame of each video: For each video, a minimal and maximal disparity (in pixels) between two adjacent views is given in the following table: These values can be used to get the disparity values from the disparity map provided as PNG. A linear conversion can be applied where 0 is the minimal disparity and 255 the maximum disparity.

Data available for download

Several kinds of data can be downloaded from the web site describing and hosting the data [5] for each frame:

• a raw picture, i.e. the raw picture captured by the sensor of the Raytrix camera,

• 5x5, i.e. 25, extracted views • 5x5, i.e. 25, disparity maps, each of them being associated to a specific extracted view.

All this data is provided for the 300 frames of the videos, but the number of the first frame is different according to the video:

• ChessPieces : 1,

• Boxer-IrishMan-Gladiator: 400,

• ChessPieces-MovingCamera: 210.

Moreover, for each video an XML file containing the calibration parameters of the Raytrix camera during the capture is provided (cf. 8.2).

Video name

Minimal disparity (pixels) Maximum disparity (pixels) Chess-Pieces -1.52 0.45 Boxer-IrishMan-Gladiator -1.50 0.39 ChessPieces-MovingCamera -1.67 0.11

Figure 1 :

 1 Figure1: First frame of the three videos "ChessPieces", "Boxer-IrishMan-Gladiator" and "ChessPieces-MovingCamera"

Figure 2 :

 2 Figure 2: 25 virtual cameras per frame one for each extracted view

Figure 3 :

 3 Figure 3: Disparity map of the central view of the first frame for "ChessPieces", "Boxer-IrishMan-Gladiator" and "ChessPieces-MovingCamera"

Table 1 :

 1 minimal and maximal values for disparity maps

INTERNATIONALE DE NORMALISATION ISO/IEC JTC1/SC29/WG1 & WG11 ISO/IEC JTC1/SC29/WG11 MPEG2018/m42468 ISO

Annex

Sensor and MLA description

The Raytrix API (cf. https://www.raytrix.de/Rx.ApiLF.3.1/namespace_e_par.html) provides information about the sensor of the R8 camera and its Micro Lens Array (MLA). The following

Calibration parameters

The calibration parameters used for the dataset can be downloaded from the web site [5] for each video as an XML file. Here is an example (associated to "ChessPieces"): <RayCalibData version="1.0"> <offset units="pix"> <x>13.251828948215</x> <y>-7.587459995462</y> </offset> <diameter units="pix">34.976657867432</diameter> <rotation units="rad">0.000000000000</rotation> <lens_border units="pix">0.999999915334</lens_border> <tcp units="virtual_depth">2.133058547974</tcp> <lens_base_x units="lens"> <x>1.000000000000</x> <y>0.000000000000</y> </lens_base_x> <lens_base_y units="lens"> <x>0.500000000000</x> <y>-0.866025388241</y> </lens_base_y> <sub_grid_base units="lens"> <x>3.000000238419</x> <y>-1.732050776482</y> </sub_grid_base> <distortion type="radial"> <function>constant</function> <parameter count="0">0</parameter> <offset units="pix"> <x>0.000000000000</x> <y>0.000000000000</y> </offset> </distortion> <lens_type id="0"> <offset units="lens"> <x>0.000000000000</x> <y>0.000000000000</y> </offset> <depth_range units="virtual_depth"> <min>1.000000000000</min> <max>3.200000047684</max> </depth_range> </lens_type> <lens_type id="1"> <offset units="lens"> <x>1.000000000000</x> <y>0.000000000000</y> </offset> <depth_range units="virtual_depth"> <min>2.599999904633</min> <max>4.199999809265</max> </depth_range> </lens_type> <lens_type id="2"> <offset units="lens"> <x>-1.000000000000</x> <y>0.000000000000</y> </offset> <depth_range units="virtual_depth"> <min>3.500000000000</min> <max>100.000000000000</max> </depth_range> </lens_type> </RayCalibData>