Nicolas Andreff
email: nicolas.andreff@femto-st.fr

Sounkalo Dembélé

Brahim Tamadazte

Zill E Hussnain

Syed Zill-E Hussnain

Epipolar geometry for vision-guided laser surgery

Keywords: Laser Surgery, Visual Servoing, Epipolar geometry

HAL is

INTRODUCTION

The µRALP project involves the development of a system for endoluminal laser phonosurgery, i.e. surgery of the vocal chords using a laser emitted from inside the larynx. Indeed, in current laryngeal laser surgical procedures, a beam of incision laser is projected on the target position of the soft tissue from the working distance of 400mm by means of a rigid laryngoscope (Fig. 1). This yields safety concerns for the patient and staff, as well as limitations to accuracy. More, this so-called laryngeal suspension position of the patient requires an extreme extension of the neck, which makes it painful several days after the operation. One such laryngeal laser surgical system is the AcuBlade TM . It claims to be a robotics system because it features fully autonomous pattern following (line for resection, circle for ablation) by the laser spot, once the surgeons has defined the dimension and position of the pattern intra-operatively based on direct visualization (through the microscope) of the op- erating site. The laser spot is thus automatically controlled, which means that the objective of µRALP to automate the control of the laser phonosurgery is totally in line with the industrial and medical practices in the field.

Yet, in the AcuBlade system, the laser spot is controlled in an open-loop mode from the surgeon's viewpoint. Indeed, once the surgeon has chosen the pattern parameters, the robot follows it blindly, relying on its internal calibration and sensors. Any deviation from this plan is forbidden (to the exception of the foot pedal switch of the power laser). However, during the ablation and even more during the resection, the tissues move and change. Therefore, the planned pattern must be very small and can not guarantee high accuracy. Moreover, in case of an endoscopic laser steering system (Fig. 2), one can not guarantee any time stability of the micro-robot calibration.

The only way to increase the accuracy and the extension of the patterns is to close the control loop over an exteroceptive sensor, namely the imaging system. Closing the loop over a camera is known as visual servoing and as being a technique robust to calibration errors and to changes in the environment of the robot.

Visual servoing has been used in minimally invasive surgery to perform two types of applications. The first type is relative to tissue motion tracking and compensation. By opposition to industrial objects human tissues are alive and than submitted to heart and breathing motions. These physiological motions complicate the task of the surgeon, and their compensation improve the precision of surgical interventions. The second type of applications deals with tracking and guidance of instruments. In this case, the objective is more classifcal for visual servoing: positionning of a target by using visual feedback.

Motion compensation

In motion compensation using image-based control, the motion of tissues is estimated from images of an imaging system and compensated by means of a robotized instrument.

Gangloff and his colleagues at Strasbourg have developed an active heart beating stabilizer by using a predictive control scheme [START_REF] Ginhoux | Active filtering of physiological motion in robotized surgery using predictive control[END_REF] and a robust control scheme [START_REF] Bachta | Active stabilization for robotized beating heart surgery[END_REF]. They obtained interesting results with simulation experiments as well as in vivo experiments with pigs. In both cases the acquisition rate reached 500 Hz. In [START_REF] Krupa | Realtime motion stabilization with b-mode ultrasound using image speckle information and visual servoing[END_REF] authors have used visual servoing and speckle information in ultrasound images at 12 Hz to stabilize a probe in a region of interest. Simulation experiment have been used to validate the developments. In [START_REF] Kesner | Ultrasound servoing of catheters for beating heart valve repair[END_REF] authors have integrated heart motion compensation in the 3D positioning of a catheter using 3D ultrasound images. A Kalman filtering has allowed to take into account the 50-100 ms delay of acquisition and a tracking error of 0.77 mm was obtained with in vivo tests with pigs. In [START_REF] Chen | An unconstrained virtual bone clamper for knee surgical robot using visual servoing technique[END_REF] a virtual bone clamper is achieved by means of visual servoing with a stereo camera running at 10 Hz and a Kalman filtering.

The core of these applications is motion tracking and for that solutions already developed in machine vision can be used: correlation [START_REF] Ortmaier | Motion estimation in beating heart surgery[END_REF], Lucas-Kanade tracker or SURF [START_REF] Elhawary | Robust feature tracking on the beating heart for a robotic-guided endo-scope[END_REF].

Instrument guidance

In instrument guidance by means of visual servoing many developments have been reported in the litterature.

In 2003, [START_REF] Krupa | Autonomous 3-d positioning of surgical instruments in robotized laparoscopic surgery using visual servoing[END_REF] proposed the use of image-based visual servoing to perform 3D positioning of a surgical instrument. But only simulation experiments were performed with a camera running at 50 Hz and a phantom instrument. Krupa and his colleagues at Rennes have proposed the 3D positioning of a probe using visual servoing in ultrasound images: image-based visual servoing on a simulation setup in [START_REF] Krupa | Guidance of an ultrasound probe by visual servoing[END_REF], visual servoing using moments on simulation setup running at 25 Hz in [START_REF] Mebarki | 2d ultrasound probe complete guidance by visual servoing using image moments[END_REF].

In [START_REF] Becker | Semiautomated intraocular laser surgery using handheld instruments[END_REF] the authors have developed a semiautomated laser intraocular surgery. Their setup included a laser probe attached to a micromanipulator, a stereo camera working at 30 kHz, a 3D sensor running at 2 kHz and a control system running at 1 kHz. All the aspects of the application were presented: preoperative procedure for the selection of the sites to burn, calibration of stereo camera using the data of the 3D sensor, 3D surface reconstruction and registration of current images with preoperative images, PID control of the micro-manipulator for the positioning of the laser on the site to burn. This is a very interesting paper, which clearly can be useful for laser phonomicrosurgery.

In [START_REF] Mattos | Safe teleoperation based on flexible intraoperative planning for robotassisted laser microsurgery[END_REF] and [START_REF] Dagnino | New software tools for enhanced precision in robotassisted laser phonomicrosurgery[END_REF] the team of Mattos at Genoa describe their experiments in laser phonomicrosurgery. They have developed a complete equipment including laser sources, micro-robots to control the position of the laser beam, an optical microscope viewing and recording images and a haptic device. They also realized algorithmic developments to follow the laser spot a predefined path. For the moment, they do treat the problem in a static point of view: they do not include the dynamics of the system (three-dimensional tissue, tissue movement, kinematic model of microrobot, ...).

In [START_REF] Reilink | Imagebased flexible endoscope steering[END_REF] the authors used directly images delivered by an endoscope to guide it inside the body. Simulation experiment show an improvement of 68% with respect to manual steering.

Related developments

In addition to above publications dealing with visual servoing directly, it is also necessary to cite the publications [START_REF] Ota | A highly articulated robotic surgical system for minimally invasive surgery[END_REF][START_REF] Rivera-Serrano | A transoral highly flexible robot: Novel technology and application[END_REF] which describe the development of a flexible robot for It is endowed with a 15 K fiber bundle connected to a 640×480 pixels camera, an illumination lamp, two 4.2 mm tool ports. It has been possible to introduce it in cadavers without laryngeal suspension, to observe vocal cord and to perform a retraction and cauterization on the base of tongue. However, the system is only teleoperated which might limit its accuracy with respect to the delicate phonosurgery requirements. Such a system should thus be enhanced by adding visual servoing and, probably also, a laser for replacing the currently used mechanical scalpel.

Contribution of the paper

The contribution of this paper is to discuss the control of a laser (namely, the invisible incision laser for incision with co-axial visible HeNe laser pointer) over the vocal fold, and by extension, the control of a laser over any surface. It shows that making call to geometry can simplify the control: no matrix inversion, no explicit knowledge or reconstruction of the 3D scene.

VISUALLY-GUIDED LASER SURGERY

In this section, we analyse the control of the laser spot with the micro-robot using an optic fiber bundle to bring the image of the scene onto a high-speed camera(Fig. 2 and3).

This control can be done in two ways: using the standard visual servoing equations or using the above grounding analogy.

A word on standard control

Let us note z the direction of the laser beam reflecting from the steering mirror towards the vocal fold, P the position of the laser spot on the tissue surface, and p the position of the laser spot in the image.

Then, it is trivial to write, in the reference frame R 0 attached to the zero-reference of the steering mirror:

0 P = d 0 z (1)
where d is the distance traveled by the laser from the mirror to the tissue. This distance can not be measure, to the contrary of 0 z which can be obtained from the microrobot encoders. However, it can be modeled if one approximates the tissue surface in P by a plane of equation:

0 n T 0 P -d 0 = 0 (2)
where 0 n is the orientation of the surface normal in R 0 and d 0 is the distance of the plane to the origin of R 0 .

Using this model, one finds:

d = d 0 0 n T 0 z (3)
On the other hand, the perspective projection equation yields:

p = K c P Z ( 4
)
where K is the matrix containing the intrinsic parameters, P is now expressed in the camera frame R c , Z is the unmeasured depth along the line of sight passing through p as well as the third coordinate of c P and p represents the homogeneous coordinates of p.

To apply, the usual visual servoing approach, one needs to differentiate the latter with time:

ṗ = 1 Z K I 2 p c Ṗ (5)
One has another expression for c Ṗ by differentiating (1):

0 Ṗ = ḋ 0 z + d 0 ż (6)
and expressing the latter in R c :

c Ṗ = c R 0 ḋ 0 z + d 0 ż (7)
Now, from 3, one gets (under the simplifying assumtion that the surface plane does not change):

ḋ = - d 0 0 n T ( 0 n T 0 z) 2 0 ż (8)
Putting ( 7) and ( 8) into ( 5) gives therefore ṗ under the form:

ṗ = L(d, Z, 0 n, d 0 , c R 0 , p, 0 z) 0 ż (9)
where L is of dimension 3 × 3. Inverting it allows to convert the image velocity of the laser spot into the velocity of the laser beam, which, in turn, shall be converted into microrobot velocity through the differential inverse kinematic model. Therefore, this controller needs the estimation of d and Z, which can be obtained by triangulation between the laser beam (known from the joint values q), the line of sight outgoing from the camera through the laser spot projection ( p) and the robot-camera calibration.

We have not implemented this controller, nor pushed further the details of the calculation (so far), because there is a more elegant way of treating the problem, which does not require any explicit triangulation nor any matrix inversion.

Laser visual servoing using epipolar geometry

On the opposite to the above method, which is totally generic, the method we propose here is totally handmade and tailored to the specific case of a laser beam being observed by a camera. Indeed, the set-up in Fig. 3 is analogous to a degenerate case of epipolar geometry in Fig. 4. Thereby, points p and p ′ are the images of the same spatial point P and are hence linked by the epipolar constraint:

p′⊤ F p = 0 ( 10
)
where F is the fundamental matrix of the 2-view system [START_REF] Hartley | Multiple view geometry in computer vision[END_REF]. Actually, this epipolar constraint is defined up to a scale factor, and thereby, p′ can be replaced by 0 z, the unit vector describing, in the microrobot baseframe R 0 , the direction of the laser beam from the mirror to the vocal fold:

0 z ⊤ F p = 0 (11)
This equation expresses the fact that the origin of the camera, the pivot point of the micro-robot, the laser beam, the line of sight and the laser spot on the tissue are coplanar. It can be also interpreted in three ways:

0 z ⊥ F p (12) p ⊥ F ⊤ 0 z (13)
and both F p and F ⊤ 0 z represent the (non-unit) normal vector to the epipolar plane, respectively in the micro-robot and the camera frame.

The time derivative of the epipolar constraint is:

(F ⊤ 0 z) ⊤ ṗ + (F p) ⊤ 0 ż = 0 (14)
Now, we can decompose 0 ż into a component orthogonal to the epipolar plane and a component inside the latter:

0 ż = α 0 h + β 0 z × 0 h ( 15
)
where 0 h = F p F p . Replacing this expression into the epipolar constraint and reordering the terms, we get:

α = - (F ⊤ 0 z) ⊤ F p ṗ (16)
Actually, α only depends on the projection of ṗ onto the normal to the epipolar plane, but expressed in the camera frame, i.e. along c h = F ⊤ 0 z F ⊤ 0 z . Thus, the remaining part of ṗ is obtained by cancelling this projection:

ṗ = a c h + (I 3 -c h c h ⊤ ) ṗ (17)
where the value of a does not have any interest for the sequel, but can be related to α by inserting the latter equation into the former. Now, concentrate on the part of ṗ lying in the epipolar plane. In (17), it is expressed in the camera frame, so we just need to bring it back to the microrobot frame, going backwards the camera intrinsic parameters, the orientation of the camera frame with respect to the micro-robot frame and compensating for the unknown scale factor in F, to get β:

β = F ⊤ 0 z F p ( 0 z × 0 h) ⊤0 R c K -1 (I 3 -c h c h ⊤ ) ṗ
(18) As a consequence, we have expressed 0 ż as a function of ṗ:

0 ż = ( 0 z × 0 h)( 0 z × 0 h) ⊤0 R c K -1 (I 3 -c h c h ⊤ ) -0 h c h ⊤ F ⊤ 0 z F p ṗ (19)
Consequently, we have the exact expression of the conversion of the image velocity into the laser beam velocity, without any matrix inversion, nor any explicit triangulation or scene structure knowledge. This expression only depends on the measurements ( 0 z and p), the fundamental matrix F and a reduced set of calibration parameters (K and c R 0 ). Geometry is always useful ! Now, we can come up to the control law, by enforcing a first order behaviour of the error in the image between the current and the desired projections of the laser spot:

ṗ = -λ( p -p * ) (20)

SIMULATION RESULTS

This control was implemented in a simulator and gives exponential decay of the image error (Fig. ??).

It shall be tested further with the introduction of noise in the sensor signals and the calibration parameters. Later, it shall be implemented on a basic hardware (standard camera + pan/tilt micropositionner from PI) before it is implemented in an endoscopic set-up.

DISCUSSION

In this paper, it was shown that resorting to geometry simplifies the eye-to-hand control law for a surgical laser (and any other application where a beam needs be accurately swept over a surface), by essentially removing the need for on-line estimation of the 3D surface. Actually, if a dedicated surgeon-robot interface is designed (?) to define the desired trajectory in the image, then the latter will geometrically contain a coherent description of the 3D surface, and thus, the 3D information is not purely and simply thrown away as it could seem but, rather, it is implicitly used.

Yet, the proposed proportional control in the image is essentially targeted at disturbance rejection pur-pose. It should be complemented by an additional term for trajectory tracking purpose ( p * (t)), including constraints on the tissue exposure to laser, in order to avoid any carbonization.

Further investigation of geometry, namely the trifocal geometry associated to a stereoscopic observation of the laser spot, is expected to further simplify the control and to increase its robustness, which is a key issue in the transfer of automation into actual clinical devices.

Also, many micromanipulators have a parallel kinematics architecture, which are known to be controllable without any joint sensing (?). Consequently, using the proposed multi-view geometric approach might enable simplified miniaturization of laser steering in an endoscopic set-up, because one can design steering parallel kinematics mechanisms without proprioceptive sensors. This is a very crucial investigation field, since endoscopic laser surgery faces very contradictory requirements in terms of sweeping range and frequency (yielding larger mechanisms) and of available space at the endoscopic tip.

Figure 1 :

 1 Figure 1: Current laryngeal laser surgery setup.

Figure 2 :

 2 Figure 2: Endoscopic laser microphonosurgery

Figure 3 :

 3 Figure 3: Schematic view of the laser steering system

Figure 4 :

 4 Figure 4: Analogy with stereoscopy.

Figure 5 :

 5 Figure 5: Time evolution of the image errors.

ACKNOWLEDGEMENTS

This work was supported by µRALP, the EC FP7 ICT Collaborative Project no. 288663 (http://www.microralp.eu), and by ACTION, the French ANR Labex no. "ANR-11-LABX-01-01" (http://www.labex-action.fr).