C C Cheah

G Oriolo

R Tatsambon
email: rtatsamb@irisa.fr

O Tahri
email: omartahri@isr.uc.pt

F Chaumette
email: chaumett@inria.fr

Distance-Based and Orientation-Based Visual

Servoing From Three Points Romeo Tatsambon Fomena, Member, IEEE, Omar Tahri, and Franc ¸ois Chaumette, Senior Member, IEEE Abstract-This paper is concerned with the use of a sphericalprojection model for visual servoing from three points. We propose a new set of six features to control a 6-degree-of-freedom (DOF) robotic system with good decoupling properties. The first part of the set consists of three invariants to camera rotations. These invariants are built using the Cartesian distances between the spherical projections of the three points. The second part of the set corresponds to the angle-axis representation of a rotation matrix measured from the image of two points. Regarding the theoretical comparison with the classical perspective coordinates of points, the new set does not present more singularities. In addition, using the new set inside its nonsingular domain, a classical control law is proven to be optimal for pure rotational motions. The theoretical results and the robustness to points range errors of the new control scheme are validated through simulations and experiments on a 6-DOF robot arm.

Index Terms-Three points, spherical projection, visual servoing.

I. INTRODUCTION

V ISUAL servoing consists of using feedback information provided by a vision sensor to control the motion of a dynamic system [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF], [START_REF]Visual Servoing via Advanced Numerical Methods[END_REF]. A vision sensor provides a large spectrum of potential visual features. However, if no planning strategy is developed, the use of some visual features in the control scheme may lead to stability problems in the case where the displacement that the robot has to achieve is very large [START_REF] Chaumette | Potential problems of stability and convergence in imagebased and position-based visual servoing[END_REF]. This justifies the need to design ideal visual features for visual servoing. By ideal, satisfaction of the following criteria is meant: local and, as far as possible, global stability of the system, robustness to calibration and modeling errors, nonsingularity, local-mimima avoidance, satisfactory trajectory of the system and of the features in the image, and, finally, maximal decoupling and linear link (which is the ultimate goal) between the visual features and the degrees of freedom (DOFs) taken into account.

Points are the most simple features that can be extracted from an image, both from a geometrical and an image processing points of view. This explains why most of the visual-servoing applications are based on image of points, for instance, visual homing [START_REF] Basri | Visual homing: Surfing on the epipoles[END_REF], autonomous navigation of mobile robots [START_REF] Burschka | Vision-based control of mobile robots[END_REF], [START_REF] Diosi | Outdoor visual path following experiments[END_REF], and autonomous stabilization of aerial vehicles [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF], [START_REF] Saripalli | Visually guided landing of an unmanned aerial vehicle[END_REF].

The image of points is at the center of lots of works devoted to approach an ideal-system behavior using a 3-D, a hybrid (i.e., 3-D and 2-D), or a 2-D visual-servoing method. In a 3-D visual servoing, the image of points and the knowledge of the object geometry serve to compute the relative pose of the object with respect to (w.r.t.) the camera, which is then used to elaborate visual features in 3-D Cartesian space [START_REF] Wilson | Relative end-effector control using cartesian position-based visual servoing[END_REF], [START_REF] Martinet | Vision based control law using 3d visual features[END_REF]. Estimating 3-D data or pose can be useful to ensure a global convergence when image noise and calibration errors are not present [START_REF] Chesi | Visual servoing for large camera displacements[END_REF]. However, small image noise could lead to large errors in the pose estimation [START_REF] Chaumette | Potential problems of stability and convergence in imagebased and position-based visual servoing[END_REF]. In hybrid visual servoing, which combines both 2-D and 3-D data, the 3-D data are generally obtained via the estimation of an homography matrix between the current and the desired images. This solution has been exploited in 2-1/2-D visual servoing, where a partially decoupled control scheme with no singularity in the whole task space and tolerating a coarsely calibrated system has been designed [START_REF] Malis | 2 1/2 d visual servoing[END_REF]. Using a coplanar set of points as a target, a decoupled homography-based visual servoing scheme that builds on the knowledge of the 3-D distance between two points has been proposed in [START_REF] Gans | Visual servoing to an arbitrary pose with respect to an object given a single known length[END_REF]. However, hybrid visual servoing is more sensitive to image noise (like 3-D visual servoing) than 2-D visual servoing, which uses directly features extracted in the image as control inputs. In 2-D visual servoing, simple perspective coordinates of image of points can be used to control a system [START_REF] Feddema | Automatic selection of image features for visual servoing of a robot manipulator[END_REF]; however, the resulting control law is highly coupled and has a limited convergence domain [START_REF] Chaumette | Potential problems of stability and convergence in imagebased and position-based visual servoing[END_REF]. Possible decoupling solutions exist for the motion along and around the optical axis, for instance, the cylindrical-coordinate approach [START_REF] Iwatsuki | A new formulation for visual servoing based on cylindrical coordinate system[END_REF] or the partitioned approach [START_REF] Corke | A new partitioned approach to image-based visual servo control[END_REF]. Another appealing solution to the decoupling issue is the use of momentinvariant theory. Using this approach, a combination of 2-D moments has been proposed for planar objects that are defined by points cloud [START_REF] Tahri | Point-based and region-based image moments for visual servoing of planar objects[END_REF]. Lately, spherical moments invariant to camera rotations, such as the area of a surface, have been used to design a partially decoupled control scheme for volumetric and planar objects defined by more than three points [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF]. Finally, in the case of a very particular target composed of a sphere marked with two points on its surface, a spherical-projection model has been used to propose a set of features providing a partially decoupled control system [START_REF] Fomena | Improvements on visual servoing from spherical targets using a spherical projection model[END_REF].

Inspired by the aforementioned work [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF], this paper exploits the geometric properties of the spherical projection of three points and proposes a new set of six features. An early version of this work appeared in [START_REF] Tatsambon Fomena | Visual servoing from three points using a spherical projection model[END_REF]. This paper extends that work by improving the theoretical developments and by validating the approach on a real robotic system.

Even if it is well known that the same image of three points corresponds to four different camera poses [START_REF] Horaud | New methods for matching 3d objects with single perspective views[END_REF], it is possible to control a 6-DOF robot using only three points in a neighborhood of the desired camera pose. The originality presented in this paper in comparison with [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF] is that the decoupling is obtained by three features invariant to rotations that are the Cartesian distances between the spherical projections of the three points. In addition, the three other features, which are designed to control rotation motions, are built using the angle-axis representation of a rotation matrix measured from the image of two points. The interaction matrix associated with the proposed angle-axis representation is valid for a general configuration between points, including the specific configuration between points proposed in [START_REF] Cowan | Geometric visual servoing[END_REF], where the target is a sphere marked with a tangent vector to a point on its surface.

Feature modeling and associated interaction matrices are detailed in Section II. In Section III, we compare our approach with the classical perspective coordinates of three points, which has been proven to present a singularity domain defined by a cylinder [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF]. In addition, due to the coupling in their interaction matrix, the perspective coordinates of points are not suited for pure camera rotations [START_REF] Chaumette | Potential problems of stability and convergence in imagebased and position-based visual servoing[END_REF]. A key contribution of this paper is the formal demonstration that our approach does not present more singularities and is optimal for pure camera rotations compared with the classical perspective coordinates of three points and the use of the coordinates of the center of gravity in [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF]. Concerning the singularity issue, the demonstration given in this paper is simple and easy compared with the demonstration using the perspective coordinates given in [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF]. The proposed control scheme depends on the ranges of points that are generally unknown in practice, as in the case of the classical perspective projection. In Section III, we also analyze the robustness of the control w.r.t. error on depth-value estimation. Finally, simulation results using the true depth values are presented in Section IV to validate the proposed approach in ideal conditions. In practice, the depth values could be estimated using structure from motion methods [START_REF] De Luca | Feature depth observation for image-based visual servoing: Theory and experiments[END_REF]. However, in order to show the robustness w.r.t. estimation errors, a very coarse approximation of the depth values, i.e., the desired values, is rather used in experiments on a 6-DOF eye-in-hand robotic system.

II. FEATURE MODELING

In this section, we design a set s sp = (s t , ξ) of six features to control the image of three points using a spherical-projection model. The set of features s t is invariant to camera rotations, while the set ξ is the angle-axis representation of a rotation matrix defined from the image of two points.

Let S(C,1) be the unit sphere of projection center in C; F c = (C, x, y, z) be the frame attached to the camera unique projection center C; P 0 , P 1 , and P 2 be a set of three points; and P i = (P ix , P iy , P iz ) be the vector coordinates of P i in F c .

We first recall that the interaction matrix L f related to a set of features f ∈ R n is defined such that ḟ = L f v c , where v c =(v, ω) ∈ se(3) is the instantaneous camera velocity [START_REF] Chaumette | Classification and realization of the different vision-based tasks[END_REF], v and ω are, respectively, the translational and the rotational velocities of the camera, and se(3) R 3 × R 3 is the Lie algebra of the Lie group of displacements SE(3).

A. Set s t Invariant to Camera Rotations

Let p i s be the spherical projection of points P i . We recall that the spherical projection of a point P i is defined by p i s = P i / P i . Note that p i s can easily be measured from a perspective image using formula p i s = p i p / p i p with p ip = (P ix /P iz , P iy /P iz ) or from any omnidirectional image with a unique projection center [START_REF] Geyer | A unifying theory for central panoramic systems and practical implications[END_REF]. Let d ij be the Cartesian distance between p i s and p j s [see Fig. 1(a)]. The analytical expression of d ij is given by

d ij = p i s -p j s = 2 -2p is p js . ( 1
)
Since a camera frame rotation preserves Cartesian distances, it is straightforward that feature d ij is invariant to camera rotation motions, as demonstrated below. From (1), the time variation of d ij is given by

ḋij = - 1 d ij p is ṗjs + p js ṗis . ( 2
)
The time variation of p k s is given by [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF] ṗk s = -

1 P k Γ p k s v + [p k s ] × ω (3)
where and[p k s ] × is the antisymmetric matrix related to vector p k s . Note that the range P k , which is generally unknown in practice, appears in the translation part of ṗk s similarly as the depth P kz appears in the well-known interaction matrix of the perspective image of a point.

Γ p k s = [p k s ] 2 × = I 3 -p k s p k s ,
Substituting (3) into (2), we immediately obtain the expression of the interaction matrix related to d ij

L d i j = 1 d ij 1 P j p is Γ p j s + 1 P i p js Γ p is 0 {1×3} .
(4) The bloc 0 {1×3} on the rotation component of the interaction matrix L d i j clearly shows the invariance property. This block has also been highlighted in [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF] when using invariants of spherical moments built from the image of at least four points, whereas in this paper, we use only three points. (a) Specific case presented in [START_REF] Cowan | Geometric visual servoing[END_REF] and (b) general case.

The set s t = (d 01 , d 02 , d 12 ) is thus formed by selecting the three distances between the spherical projection of the three points. Its interaction matrix is given by

L s t = [ L υ 0 ] (5)
with

L υ = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 1
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ .

B. Angle-Axis ξ Representation of a Rotation Matrix

From the spherical projections p i s and p j s , it is possible to determine a set of three features ξ such that its interaction matrix has the form

L ξ = [ -L ω (θ ξ , u ξ )L ω ,υ L ω (θ ξ , u ξ ) ] (6)
where L ω ,υ and L ω (θ ξ , u ξ ) will be defined later. The set ξ is based on two ideas. The first idea, as proposed in [START_REF] Malis | 2 1/2 d visual servoing[END_REF], concerns the θu (where θ is the angle of rotation, and u is the unitary axis of rotation) representation of the relative rotation between the current and desired camera frames. The second idea, as proposed in [START_REF] Cowan | Geometric visual servoing[END_REF], is about visual servoing from a sphere marked with a tangent vector where the configuration between the two points P i and P j in the object frame

F o = (O, x, y, z) is such that OP i ⊥ P i P j [see Fig. 2(a)].
As a major theoretical improvement of [START_REF] Tatsambon Fomena | Visual servoing from three points using a spherical projection model[END_REF], the set ξ is defined as the θu representation of the rotation matrix c R p i s

p * i s R c * , where c * R p * i s is the desired value of the matrix c R p i s = [v 1 v 2 v 3 ] defined as follows [see Fig. 1(b)]: ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ v 1 = p i s v 2 = Γ p i s (p j s -p i s ) Γ p i s (p j s -p i s ) v 3 = v 1 × v 2 (7)
where v 2 is the projection of vector p j sp i s on the orthogonal plane to p i s . From [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF], it is clear that c R p i s is not defined only if C, P i , and P j are aligned.

The tangent space Tc R p i s SO(3) to the element c R p i s ∈ SO(3) is identified with so(3) R 3 via right translation by

( c R p i s , c Ṙp i s ) → ( c R p i s , ζ), where ζ ∈ so(3) is such that [ζ] × = c Ṙp i s p i s R c . ( 8
)
In the specific configuration, where OP i ⊥ P i P j , from [START_REF] Saripalli | Visually guided landing of an unmanned aerial vehicle[END_REF], it has been shown in [START_REF] Cowan | Geometric visual servoing[END_REF] that

ζ = L ω ,υ v -ω (9)
where the expression of matrix L ω ,υ is given by

L ω ,υ = 1 P i (δ c v 1 v 3 + v 2 v 3 -v 3 v 2 ) ( 10
)
with δ c = -r 3 v 2 /r 3 v 1 , where r 3 = P i O/ P i O , with O the vector coordinates of the center of object frame O in the camera frame F c .

In this paper, we propose to compute ζ for a general configuration between the points, as shown in Fig. 2(b). With that generalization, matrix L ω ,υ is now given by (for detailed developments, see Appendix A)

L ω ,υ = 1 P i (δ p v 1 v 3 + v 2 v 3 -v 3 v 2 ) (11)
with δ p = (p i s p j s ) P j -P i P j Γ p i s (p j s -p i s ) . In addition, note here that the ranges P i and P j appear in the translation part of the interaction matrix. We will see in Section III-B how to deal with this issue.

In the case of the specific configuration OP i ⊥ P i P j [see Fig. 2(a)], it is possible to show that the general expression [START_REF] Chesi | Visual servoing for large camera displacements[END_REF] equals the specific expression [START_REF] Martinet | Vision based control law using 3d visual features[END_REF]. Indeed, plugging the expressions of v 1 and v 2 given in [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF] into δ c gives (after expansion) δ c Γ p i s (p j sp i s ) = (p i s p j s )r 3 p j s /r 3 p i s . By exploiting OP i ⊥ P i P j , it is easy to show that r 3 p j s /r 3 p i s = P i / P j . Therefore, δ c = δ p . Now, we use ζ to develop the interaction matrix related to ξ = θu( c R p i s p * i s R c * ). The expression of ζ given in (8) can be rewritten as

[ζ] × = d( c R p i s p * i s R c * ) dt ( c R p i s p * i s R c * ) (12)
since

d( c R p i s p * i s R c * )/dt = c Ṙp i s p * i s R c * + c R p i s p * i s Ṙc * = c Ṙp i s p * i s R c * because p * i s Ṙc * = 0. Plugging into (12) the Rodriguez formu- lation of c R p i s p * i s R c * c R p i s p * i s R c * = I 3 + sin(θ ξ )[u ξ ] × + (1 -cos(θ ξ ))[u ξ ] 2 × (13)
where θ ξ = ξ and u ξ = ξ/ ξ , we obtain after some developments (exactly as in [START_REF] Malis | Contributions à la modélisation et à la commande en asservissement visuel[END_REF])

ξ = -L ω (θ ξ , u ξ )ζ (14)
where

L ω (θ ξ , u ξ ) = -I + θ ξ 2 [u ξ ] × -1 - sinc(θ ξ ) sinc 2 (θ ξ /2) [u ξ ] 2 × ( 15
)
with sinc(x) = sin(x)/x. Finally, plugging the tangent space identification ( 9) into ( 14), we immediately obtain

ξ = -L ω (θ ξ , u ξ )L ω ,υ v + L ω (θ ξ , u ξ )ω (16)
from which we deduce the interaction matrix of ξ = θu( c R p i s p * i s R c * ) given in [START_REF] Diosi | Outdoor visual path following experiments[END_REF]. Note that it would also be possible to use the θu representation of matrix c * R p * i s p i s R c . In that case, by denoting ξ = θ ξ u ξ , we obtain

L ω (θ ξ , u ξ ) = I + θ ξ 2 [u ξ ] × + 1 - sinc(θ ξ ) sinc 2 (θ ξ /2) [u ξ ] 2 × .
(17) Parameters ξ and ξ can be used interchangeably since the corresponding control laws gives exactly the same result. In the following, we select the set ξ.

C. New Interaction Matrix

The new interaction matrix, which is obtained by stacking the two interaction matrices L s t and L ξ , is a lower block triangular square matrix given by

L s s p = L υ 0 -L ω (θ ξ , u ξ )L ω ,υ L ω (θ ξ , u ξ ) (18)
where matrix L υ is given by ( 5), matrix L ω ,υ is given by ( 11), and matrix L ω (θ ξ , u ξ ) is given in [START_REF] Iwatsuki | A new formulation for visual servoing based on cylindrical coordinate system[END_REF]. Let us mention that if p i s and p j s are the projections of two points lying on the surface of a spherical object, then it is possible to obtain an upper block triangular interaction matrix for this particular target using the θu representation of the relative orientation c * R c computed from [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF] [START_REF] Fomena | Improvements on visual servoing from spherical targets using a spherical projection model[END_REF]. Here, the partial decoupling is obtained from the distances between three image points, which we consider to represent a more general object.

To sum up, the new set s sp = (s t , ξ), built using only three points, is such that the decoupling is obtained by the set s t = (d 01 , d 02 , d 12 ); camera rotation motions are controlled using the angle-axis parameterization ξ of a rotation matrix measured from the image of two points. The interaction matrix related to ξ, which is proposed here, is valid for a general configuration between two points.

III. CONTROL ANALYSIS

Due to the lower block triangular form of L s s p , the determination of its singularity domain is shown, in this section, to be easier than in the case of the perspective projection of three points. We also analyze the stability of the control law.

We use the classical control law given by

v c = -λ L s s p -1 (s sp -s * sp ) (19)
where v c = (v, ω) is the camera velocity sent to the low-level robot controller, λ is a positive gain, and

L s s p -1 = L -1 υ 0 -L ω ,υ L -1 υ L -1 ω ( θ ξ , u ξ )
is the inverse of an approximation of the interaction matrix related to s sp . This approximation takes into account calibration errors, image-processing errors, and modeling errors w.r.t. ranges of points. We first analyze the control law in the ideal case, where the interaction matrix is perfectly known, and then in the case where the range values are approximated.

A. Ideal Case

Taking the inverse of the interaction matrix [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF] and plugging it into (19) leads to the ideal control law

v = -λL -1 υ (s t -s * t ) ω = -L ω ,υ v + λξ (20) since L -1 ω (θ ξ , u ξ )ξ = -ξ.
The domain of singularity of L s s p is given by the following theorem.

Theorem 1: The classical control method ( 20) is singular if and only if: the three points are aligned, or the camera optical center C lies on the cylinder of singularities, i.e., the cylinder, which is defined by the circumcircle passing through the three points and the normal to the plane on which the points lie (see Fig. 3).

The proof of Theorem 1 is given in Appendix B. Note that the given proof is easy and simple in comparison with the complex and difficult demonstration given in the case of the perspective coordinates in [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF].

Theorem 1 ensures that the new set s sp does not present more singularities than the perspective coordinates of the three points s pp = (p 0x , p 0y , p 1x , p 1y , p 2x , p 2y ), where

L s p p = [ L t L r ] (21)
with

L t = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ -1/P 0 z 0 p 0 x /P 0 z 0 -1/P 0 z p 0 y /P 0 z -1/P 1 z 0 p 1 x /P 1 z 0 -1/P 1 z p 1 y /P 1 z -1/P 2 z 0 p 2 x /P 2 z 0 -1/P 2 z p 2 y /P 2 z ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ and L r = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ p 0 x p 0 y -(1 + p 0 2 x ) p 0 y 1 + p 0 2 y -p 0 x p 0 y -p 0 x p 1 x p 1 y -(1 + p 1 2 x ) p 1 y 1 + p 1 2 y -p 1 x p 1 y -p 1 x p 2 x p 2 y -(1 + p 2 2 x ) p 2 y 1 + p 2 2 y -p 2 x p 2 y -p 2 x ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
. Indeed, using s pp , the classical control law (19) (with s pp instead of s sp ) has been shown to be singular when the three points are aligned or when the camera optical center belongs to the cylinder of singularities [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF].

With the decoupling property of s sp , it is also easier to determine the kernel of L s s p that characterizes the set of camera motions v c = 0, which leaves the image unchanged, i.e., ṡsp = 0. Indeed, from the expression of the interaction matrix given in [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF], we have ker(L s s p ) = {v c =(v, ω) ∈ se(3), L υ v = 0, ω = L ω ,υ v} [START_REF] Cowan | Geometric visual servoing[END_REF] which shows that we have to deal only with the 3 × 3 matrix L υ , contrary to the perspective projection of three points, where we have to deal with the larger and complex 6 × 6 matrix L s p p . Now, we analyze the control-law stability. We assume, in the following, that the interaction matrix never loses its rank during the servoing, i.e., the camera never crosses the cylinder of singularities, as shown in Fig. 3. Let e = s sps * sp be the error to regulate to zero. The closed-loop system equation (using the control law [START_REF] Fomena | Improvements on visual servoing from spherical targets using a spherical projection model[END_REF] in the ideal case) can be written as ė = -λL s s p L -1 s s p e = -λe, which means that the system is locally asymptotically stable (LAS). The system is not globally stable since we assumed the camera never crosses the cylinder of singularities; in addition, since with three points, four different camera poses exist providing the same points position in the image [START_REF] Horaud | New methods for matching 3d objects with single perspective views[END_REF], reaching the desired pose is only possible from a local neighborhood.

On one hand, the set s sp is perfectly suited to control camera rotations compared with the coordinates of the center of gravity used in [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF]. Indeed, for a pure rotation motion, since the value of s t is constant, i.e., s t = s * t , the expression of the control law (20) is given by v = 0, ω = λξ [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF] which means that the camera center follows a geodesic path.

On the other hand, as for classical perspective coordinates of points as soon as all points do not have the same depth, one disadvantage of the proposed approach is that a pure translation displacement also generates rotation motions. Indeed, from [START_REF] Tatsambon Fomena | Visual servoing from three points using a spherical projection model[END_REF], it is clear that, even if ξ = 0, ω = 0 as soon as s t = s * t . However, we will see in Section IV-A and B that the results obtained, in practice, for a pure translation motion do not deviate a lot from a straight-line trajectory in the Cartesian space and in the image space.

B. Approximation of Range Values

We now consider the usual case, in practice, where the true value of P i is not available. In that case, the closed-loop system equation using the control law [START_REF] Fomena | Improvements on visual servoing from spherical targets using a spherical projection model[END_REF] can be written as

ė = -λL s s p L s s p -1 e (24)
with

L s s p -1 = L -1 υ 0 -L ω ,υ L -1 υ L -1 ω (θ ξ , u ξ )
where L -1 υ and L ω ,υ depend on P i , i = 0, 1, 2. P i , which is the estimated value of P i , can be expressed as follows:

P i = |P iz |ρ i ( 25
)
with ρ i = (P ix /P iz ) 2 + (P iy /P iz ) 2 + 1, where P ix /P iz = p s ix /p s iz , and P iy /P iz = p s iy /p s iz are measured from the image of points. The interaction matrix (18) thus depends on the estimated depth value P iz .

In the case, where | P iz | = α|P iz |, i = 0, 1, 2, it is easy to show that the system is LAS for α > 0 and the singularity domain of L s s p ( P iz ) is exactly the singularity domain of L s s p (P iz ) since L s s p ( P iz ) = (1/α)L s s p (P iz ). This control robustness analysis is validated by the last simulation result presented in Section IV-A.

In the general case, the singularity and the stability analysis of the control using L s s p ( P iz ) are complex to establish and are left for future developments. Using a very coarse approaximation of the depth value P iz = P * iz , the control law built with either L s s p (P * iz ) or L s * s p (P * iz ) is well-known to be LAS [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]. In practice, the validation of the use of L s s p (P * iz ) and L s * s p (P * iz ) is shown in Section IV-B with perspective and fish-eye cameras, respectively.

Finally, in the case of pure rotations, it is clear from ( 19) that the control law is perfectly adequate, even in presence of errors on points depths since the depths of points are not involved. Indeed, we still have in that case v = 0, and ω = λξ, as in [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF].

To sum up, as a benefit compared with the use of classical perspective image of points and first-order spherical moments [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF], the new set has been theoretically shown to be optimal for camera rotation motions, even in the case of depths errors. In ideal conditions and, to some extent, in the case of depth errors, the singularity domain of a classical control using the new set of features has been theoretically characterized and shown not to be larger than the singularity domain of the classical perspective coordinates of points given in [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF]; the partial decoupling of the control thus makes the new set able to provide adequate camera motion in comparison with the classical perspective coordinates of points, as confirmed by results given in Section IV-A. Even though the stability of the control law is complex to analyze in the general case of depth errors, the control is LAS when using the desired depth values.

IV. RESULTS

This section illustrates the proposed theoretical results through simulations and real experiments. In simulation results, we assume that the point depths values are perfectly known. In first real experiments, using the current interaction matrix with a perspective camera, the point depths are coarsely approximated by their desired values. Finally, the LAS property is also validated using the desired interaction matrix with a fish-eye camera.

A. Simulation Results

In this section, we compare the new set s sp with the classical perspective coordinates of three points s pp using the simulator provided in ViSP [START_REF] Marchand | Visp for visual servoing: A generic software platform with a wide class of robot control skills[END_REF].

Note that since at least four points are required to determine either the set of features proposed in [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF] or the relative camera pose computed from a homography transformation in [START_REF] Gans | Visual servoing to an arbitrary pose with respect to an object given a single known length[END_REF], it is not possible to compare these methods with the new set of six features, which uses only three points. This comparison will be done in future works in which we plan to extend the present work to the case of more than three points.

We first present the case of camera rotation motions only; then, we consider only translation motions; finally, we present the case where both camera rotation and translation motions are involved.

In the object frame F o = (O, x, y, z), the coordinates of the three points are given by P 0 = r(0, 1, 0.30), P 1 = r( √ 3/2, -1/2, 0.30), and P 2 = r(-√ 3/2, -1/2, 0.30), with r = 0.5 m. The configuration of the three points describes an equilateral triangle, as shown in Fig. 4(a), and the circumcircle (of center O and radius r) of the equilateral triangle is the circular cross section of the cylinder of singularities. The pose of the object frame F o w.r.t. the desired pose of the camera frame F c * is set to the values c * t o = (0, 0, 2.35) m and θu( c * R o ) = (0, 0, 0) rad, which means that the desired pose of the camera is inside the cylinder of singularities, as shown in Fig. 4(b). Now, we present three experiments, where we consider only camera rotation motions. In the first experiment, we highlight both the partial decoupling of the control using the new set s sp and the coupling of the control using the classical set s pp . The orientation of the initial camera frame w.r.t. the desired camera frame has been set to θu( c * R c ) = (-0.20, 0.17, 0.79) rad. As expected, because of the partial decoupling, the new set s sp causes no translation displacement of the camera, while the classical set s pp does [see Fig. 5(a)]. Indeed, the coupling in the control law using the classical set s pp [see [START_REF] Horaud | New methods for matching 3d objects with single perspective views[END_REF]] generates undesired translation velocities with oscillations [compare Fig. 5(b) and(c)]. Using the classical set, all the features vary, which is not the case when using the new set where only the subset ξ varies since the subset (d 01 , d 02 , d 12 ) is invariant to rotations [compare Fig. 5(d) and(e)]. As expected also, the control using the new set s sp is ideal since the camera rotation velocities In the second experiment, we show the superiority of the new set s sp over the classical set s pp in the case of large rotation motions of the camera. The orientation of the initial camera frame w.r.t. the desired camera frame has been set to θu( c * R c ) = (0.21, -0.31, 0.47) rad. In this case, the control with the classical set s pp crosses the cylinder of singularities and converges toward another global minimum [see Fig. 6(a)]. Using the classical set, visual servoing fails because of the translation motions induced by the coupling in L s p p , whereas, as expected, the control using s sp converges ideally and exponentially (as before), as shown in Fig. 6(b) and(c).

In the third experiment, we validate the robustness of the control in the case of modeling errors, i.e., errors on points depths. We have introduced the following error of the depth estimation of points: P 0z = 0.5P 0z , P 1z = 0.3P 1z , and P 2z = 0.75P 2z . The relative orientation of the initial camera frame w.r.t. the desired camera frame is set to the same value, as in the second experiment described above. As expected, the robot displays exactly the same ideal behavior, as shown in Fig. 6(b) and(c), where only rotation motions are generated. Indeed, even in the case of modeling errors, the control scheme generates rotation motions only, as given by the expression [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF]. Now, we validate the new set for a translation motion and compare it with the classical set. The relative pose of the initial camera frame w.r.t. the desired camera frame is set to the value c * t c = (0.21, 0.31, -0.5) m. Using the new set, even if there is a little oscillation on v x (at the beginning of the servoing), which does not appear with the control using the classical set [compare Fig. 7(b) and (c)], the robot Cartesian trajectory is satisfactory [see Fig. 7(a)]. As expected, in both cases, all visual features vary and the errors decrease exponentially, as shown in Fig. 7(d) and(e).

In the next three experiments, we consider complex motions, i.e., motions made up of both rotation and translation displacements. In the first experiment, we validate the new set in the case of a motion, where the relative pose of the initial camera frame w.r.t. the desired camera frame is set to the following value: c * t c = (0.29, 0.16, -0.52) m and θu( c * R c ) = (0.21, -0.21, 0.30) rad. In this case, we have a better Cartesian trajectory (straight line) using the new set [see Fig. 8(a)]. In addition, camera velocities present no oscillation contrary to the velocities using the classical set [compare Fig. 8(b) and(c)].

In the second experiment, in order to validate the larger convergence domain of the control using the new set, we consider a relatively large displacement where the camera initial pose is very close to the boundary of the cylinder of singularities. The relative initial pose of the camera frame w.r.t. the desired camera frame is set to the following value: c * t c = (-0.29, -0.37, -0.48) m and θu( c * R c ) = (-0.26, 0.17, -0.52) rad. This particular position, which is close to a singularity of the control, induces a large condition number value (which is almost 70) of the block matrix L υ , as shown at the beginning of the servo in Fig. 9(a). The coupled control induced by the classical set causes the control to cross the cylinder of singularities and to converge toward another global In the third experiment, we validate the robustness of the new control to modeling errors. We have set the following error on the depth estimation of points: P iz = 0.5P iz , i = 0, 1, 2. The camera is initially positioned exactly the same as in the last experiment, i.e., near to the boundary of the cylinder of singularities. The results shown in Fig. 10(b) illustrate the control law convergence, despite a very high conditioning number (which is almost 200), due to modeling errors, of the block matrix L υ at the beginning of the servo, as shown in Fig. 10(a); in addi- tion, the robot Cartesian trajectory is satisfactory, as shown in Fig. 10(c).

To conclude, simulation results have shown that the new set is ideal for camera rotation motions; more importantly, using the new set, the convergence domain has been shown to be larger than in the case of the classical perspective coordinates.

B. Experimental Results

Now, we present five experiments that validate the use of our approach in practice. We use a 6-DOF robot arm equipped either with a perspective camera or with a fish-eye camera. The first four experiments use a perspective camera: the central point is located at (u 0 , v 0 ) = (320, 240) pixels and the couple of focal length values is given by (f u , f v ) = (600, 600) pixels/m. Experiments with that perspective camera have been carried out using L s s p ( P iz = P * iz ) in the control law [START_REF] Fomena | Improvements on visual servoing from spherical targets using a spherical projection model[END_REF]. In addition, the desired pose of the perspective camera is the same as for simulation results [see Fig. 4(b)], i.e., the desired camera pose is such that the image plane is parallel to the plane defined by the three points. In this case, using Thalès theorem with triangles (C, P 0 , P 1 ), (C, p 0s , p 1s ), and (25), the estimated value of the common desired depth P * z of points is given by

P * z = P * 1 -P * 0 ρ * 0 p * 1 s -p * 0 s .
In the first experiment, we consider a rotation between the current and the desired camera frames: c * t c = (0.0, 0.0, 0.0) m and θu( c * R c ) = (5 • , -7 • , 45 • ). The desired and initial images are shown in Fig. 11 (0.0 • , 0.0 • , 0.0 • ). The desired and initial images are shown in Fig. 12(a) and (b), respectively. The obtained results show image trajectories in Fig. 12(c) and the convergence of the control despite little noise on velocities, as shown in Fig. 12(d). Image trajectories are almost straight lines, which are satisfactory.

The third experiment illustrated in Fig. 13 shows, as expected, that our new control scheme performs adequately in the case of a rotation around the optical axis and a translation along the optical axis.

In the fourth experiment, we consider a rotation and a translation displacement between the current and the desired camera frames: c * t c = (0.0, -0.05, -0. Finally, we validate once more the control LAS property using an omnidirectional vision system, that is a fish-eye camera [see Fig. 15 the central point is located at (u 0 , v 0 ) = (315.61, 243.05) pixels, and the couple of focal length values is given by (f u , f v ) = (722.91, 721.65) pixels/m. We use the desired value L s * s p (P * iz ) of the interaction matrix in the control law [START_REF] Fomena | Improvements on visual servoing from spherical targets using a spherical projection model[END_REF]. Note that although the target is composed of four points, only three points are used for visual servoing. Fig. 15(c) and(d) shows the result, where we can see that the control converges once more with a satisfactory behavior.

To conclude, experimental results have demonstrated successful performances of the proposed control scheme, even in the case of calibration and modeling errors on points depths.

V. CONCLUSION AND FUTURE WORKS

In this paper, we have proposed a new set of six features for visual servoing from three points using a spherical-projection model. This new set includes distances between spherical projections of points, which allow to obtain a partially decoupled control scheme. In comparison with the classical perspective coordinates of three points, we have demonstrated that the new set does not present more singularities. Indeed, the singularity domain of the new set has been theoretically characterized. The singularities appear when either the three points are aligned or the camera optical center lies on the well-known cylinder of singularities. Using the new set, it is important to mention the simplicity of the determination of the cylinder of singularities in comparison with the complex demonstration given in the case of the perspective coordinates. The new set has been formally shown to be optimal for camera rotation motions compared with the classical perspective coordinates of points and the modern set using spherical moments. These theoretical results have been successfully validated through simulations and real experiments. Simulation results have shown that the convergence domain of a classical control method using the new set is larger than with the classical set. Experimental results on a 6-DOF robotic system have validated the use of our approach in the case of calibration errors and modeling errors on points depths. As future works, it will be interesting to extend the distance-based control scheme to the case of more than three points and to determine either a set of six visual features or a redundant set of features that provides decoupling properties. In the first case, the singularity domain of the control is the main problem, whereas in the second case, it will be necessary to find a set free of local minima.

APPENDIX A COMPUTATION OF THE TANGENT VECTOR ζ

Here, the detailed developments of the computation of the tangent vector ζ to c R p i s in SO(3) are presented using the framework proposed in [START_REF] Cowan | Geometric visual servoing[END_REF].

From (8), we immediately obtain

ζ = c Ṙp i s p i s R c × ( 26
ζ = c R p i s [ p i s R c c Ṙp i s ] × . ( 28
)
We have

p i s R c c Ṙp i s = ⎡ ⎢ ⎣ 0 v 1 v2 v 1 v3 v 2 v1 0 v 2 v3 v 3 v1 v 3 v2 0 ⎤ ⎥ ⎦ which leads to [ p i s R c c Ṙp i s ] × = -v 2 v3 -v 3 v1 v 2 v1 . ( 29
)
Note that the component of vector p i s R c c Ṙp i s × has been chosen in order to simplify the computations. By plugging ( 29) into [START_REF] Marchand | Visp for visual servoing: A generic software platform with a wide class of robot control skills[END_REF], we obtain

ζ = [v 1 v 2 v 3 ] ⎡ ⎢ ⎣ -v 2 v3 -v 3 v1 v 2 v1 ⎤ ⎥ ⎦ . ( 30
)
Since vector v 1 is the spherical projection of the point P i , from (3), we easily deduce the time variation of v 1 as v1 == -1

P i I 3 -v 1 v 1 v + v 1 ×ω. (31)
From (31), we obtain after some developments

v 3 v1 = - 1 P i v 3 v + v 2 ω (32)
and

v 2 v1 = - 1 P i v 2 v -v 3 ω. ( 33
)
Now, we present the detailed developments of the expression v 2 v3 . It worth mentioning that the computation of v 2 v3 is a contribution of this paper since this computation is valid for a general configuration of points [see Fig. 2(b)], including for the specific configuration between points [see Fig. 2(a)] in [START_REF] Cowan | Geometric visual servoing[END_REF]. From the expression of v 3 given in [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF], we have get v 3 M p i s p j s = 1 P i P j (p i s p j s ) -

P i P j v 3 (43)
i.e., v 3 is a left eigenvector of matrix M p i s p j s . Using (43), expression given in (42) simplifies to

v 2 v3 = - δ p P i v 3 v + v 1 ω ( 44
)
with δ p = (p i s p j s ) P j -P i P j Γ p i s (p j s -p i s ) . Finally, by substituting (32), (33), and (44) into (30), we obtain

ζ = L υ ,ω v -ω (45) with L ω ,υ = 1 P i (δ p v 1 v 3 + v 2 v 3 -v 3 v 2 ).

APPENDIX B PROOF OF THEOREM 1

Here, we give a proof of Theorem 1.

Proof:

The key element to the determination of the singularities of the classical control given in [START_REF] Tatsambon Fomena | Visual servoing from three points using a spherical projection model[END_REF] is the factorization of the determinant of the interaction matrix given in [START_REF] Tahri | Decoupled imagebased visual servoing for cameras obeying the unified projection model[END_REF].

From ( 18), since L s s p is a square triangular matrix, it is immediate to show that

L s s p = |L υ | |L ω (θ ξ , u ξ )|. ( 46
)
Since in the case where c R p i s is defined, i.e., when C, P 1 , and P 2 are not aligned, |L ω (θ ξ , u ξ )| = -1/sinc 2 (θ ξ /2), L ω (θ ξ , u ξ ) is nonsingular in the task space [START_REF] Malis | 2 1/2 d visual servoing[END_REF]. Therefore, we focus on the factorization of |L υ |. Focusing only on the factorization of |L υ | could also be seen by considering the tangent map relating the tangent space se(3) to the tangent space of features T s s p = T s t × Tc R p i s SO(3) given by

ṡt ξ = L υ 0 L ω υ -I 3 v ω . ( 47
)
Indeed, the determinant of the tangent map (47) is given by -|L υ |.

Using P 0 , P 1 , and P 2 , the block matrix L υ of the interaction matrix (18) can be rewritten as where α ij = 1/d ij P i P j , and k ij = 1 -( P i / P j ) cos(P i , P j ).

The determinant of L υ can be easily computed from the determinant of its transpose L υ . Indeed, from the fact that |L υ | = L υ , we have characterizes the cylinder of singularities [see Fig. 3] defined in [START_REF] Michel | Singularities in the determination of the situation of a robot[END_REF]. Using the expressions of k ij given in (48), it is possible to show after some developments that (51) is equivalent to (P 1 P 0 CP 0 )(P 0 P 2 CP 2 )(P 2 P 1 CP 1 )

|L υ | = L v x , L v y , L v z ( 49
+(P 0 P 1 CP 1 )(P 2 P 0 CP 0 )(P 1 P 2 CP 2 ) = 0 (52) Expression ( 52) is easily verified for the particular configuration of the cylinder of singularities illustrated in Fig. 3(a), where (P 2 P 1 CP 1 ) = (P 0 P 1 CP 1 ) = 0.

Let the point P be the orthogonal projection of the camera optical center C onto the plane defined by the three points P 0 , P 1 , and P 2 [see Fig. 3(b)]. Let F c be a frame centered in C and oriented such that z = CP/ CP . In F c , the coordinates of P are given by (0, 0, P z ), and points P 0 , P 1 , and P 2 have all the same z-component P iz = P z . We denote P i = (P ix , P iy , P z ) as the vector coordinates of P i in F c . Equation (52) still holds in F c since rotation preserves dot product. By expressing (52) in F c , expanding, and simplifying, we get the product of two determinants

P 0x P
The camera optical center C belongs to the cylinder of singularities iff P belongs to the circumcircle defined by the three points. The property that the point P of coordinates (0, 0, P z ) belongs to the circumcircle defined by the three points can be expressed by the following three-point formula for the circle [ which is clearly equivalent to expression (53). The cylinder of singularities described in Fig. 3 is thus characterized by the expression (53), which is equivalent to (51).

Fig. 1 .

 1 Fig. 1. Spherical projection of two points. (a) Distance between the spherical images of the two points and (b) components of the rotation matrix.

Fig. 2 .

 2 Fig. 2. Configuration of the two points in object frame F O = (O, x, y, z).(a) Specific case presented in[START_REF] Cowan | Geometric visual servoing[END_REF] and (b) general case.

Fig. 3 .

 3 Fig. 3. Cylinder of singularities. (a) Particular case and (b) general case.

Fig. 4 .

 4 Fig. 4. Configuration of three points in the 3-D space. (a) Equilateral triangle and circumcircle and (b) desired pose of the camera.

Fig. 5 .

 5 Fig. 5. Decoupled versus coupled control. (a) Camera Cartesian trajectories, (b) and (c) computed camera velocities (in meters per second and radians per second) using s p p and s sp , and (d) and (e) errors on s p p spp and s sp .

Fig. 6 .

 6 Fig. 6. Superiority for rotationmotions. (a) Cylinder of singularities is crossed by the control using the classical set s p p and (b) and (c) errors and computed camera velocities (in meters per second and radians per second) using s sp .

Fig. 7 .

 7 Fig. 7. Comparison in the case of a translation motion. (a) Camera Cartesian trajectories, (b) and (c) computed camera velocities (in meters per second and radians per second) using s p p and s sp , and (d) and (e) errors on s p p and s sp .

Fig. 8 .

 8 Fig. 8. Comparison in the case of a complex motion. (a) Camera Cartesian trajectories and (b) and (c) computed camera velocities (in meters per second and radians per second) using s p p and s sp .

Fig. 9 .

 9 Fig. 9. Larger convergence domain for the new set, where the cylinder of singularities is crossed by the classical set s p p . (a) Conditioning number of the block matrix L υ , (b) computed camera velocities (in meters per second and radians per second), and (c) camera Cartesian trajectories.

Fig. 10 .

 10 Fig. 10. Robustness w.r.t. modeling errors using the new set s sp . (a) Conditioning number of the block matrix L υ , (b) computed camera velocities (in meters per second and radians per second), and (c) camera Cartesian trajectories.

Fig. 11 .

 11 Fig. 11. Rotation displacement. (a) Desired image, (b) initial image, (c) errors, and (d) computed camera velocities (in meters per second and radians per second).

  (a) and (b), respectively. Fig. 11(c) shows the errors, and Fig. 11(d) shows the robot velocities, where we can see unexpectedly very small translation velocities. These translation velocities are caused by hand-eye calibration errors. Despite these unexpected small translation velocities, the convergence of the control proves the robustness of the new scheme w.r.t. calibration errors. In the second experiment, we consider only a translation displacement: c * t c = (-0.1, 0.05, -0.2) m and θu( c * R c ) =

Fig. 12 .

 12 Fig. 12. Translation displacement. (a) Desired image, (b) initial image, (c) image trajectories, and (d) computed camera velocities (in meters per second and radians per second).

Fig. 13 .

 13 Fig. 13. 45 • rotation around and 20-cm translation along the optical axis. (a) Desired image, (b) initial image, (c) image trajectories, and (d) computed camera velocities (in meters per second and radians per second).

  2) m and θu( c * R c ) = (5.0 • , -7.0 • , 15.0 • ). The desired and initial images are shown in Fig. 14(a) and (b), respectively. Fig. 14(d) shows the velocities and Fig. 14(c) shows the Cartesian trajectory, where we can see, once more, that the control converges satisfactorily.

  (a) and (b)]; the mirror-like parameter is d f = 1.71,

Fig. 14 .

 14 Fig. 14. Rotation and translation displacement. (a) Desired image, (b) initial image, (c) Cartesian trajectory, and (d) computed camera velocities (in meters per second and radians per second).

Fig. 15 .

 15 Fig. 15. Rotation and translation displacement. (a) Desired image, (b) initial image, (c) errors, and (d) computed camera velocities (in meters per second and radians per second).

  ) where [M] × is the vector associated with the antisymmetric matrix M. Using the property Sw = S[w] × S × ∀S ∈ SO(3) and ∀w ∈ R 3 (27) with settings S = p i s R c , and w = [ c Ṙp i s p i s R c ] × , from (26), we easily obtain

  k 10 P 0 + k 01 P 1 ) α 02 (k 20 P 0 + k 02 P 2 ) α 12 (k 21 P 1 + k 12 P 2 )

L

  v x = α 01 (k 10 P 0 + k 01 P 1 )L v y = α 02 (k 20 P 0 + k 02 P 2 ) L v z = α 12 (k 21 P 1 + k 12 P 2 ).Using the multilinear property of the determinant application, from (49), we obtain after some developments,|L υ | = α 01 α 02 α 12 (k 10 k 02 k 21 + k 01 k 20 k 12 ) |P 1 , P 0 , P 2 |(50) where α 01 α 02 α 12 = 0 since α ij = 0, i = 0, 1, j = 1, 2. From (50), it is clear that, if the three points are aligned, then |P 1 , P 0 , P 2 | = 0. Now, we show that in the case where the three points are not aligned, the expression k 10 k 02 k 21 + k 01 k 20 k 12 = 0 (51)

  Since the three points are not aligned, we have, on one hand,

	0y 1		P 0x P 0y P 2 0x + P 2 0y
	P 1x P 1y 1	.	P 1x P 1y P 2 1x + P 2 1y	= 0.
	P 2x P 2y 1		P 2x P 2y P 2 2x + P 2 2y
	P 0x P 0y 1
	P 1x P 1y 1	= 0.
	P 2x P 2y 1
	On the other hand, we must have
	P 0x P 0y P 2 0x + P 2 0y
	P 1x P 1y P 2 1x + P 2 1y	= 0.
	P 2x P 2y P 2 2x + P 2 2y

ACKNOWLEDGMENT

The authors gratefully acknowledge S. Hutchinson for his comments on earlier drafts of this paper, as well as the anonymous reviewers.

From (34), we immediately have

From the expression of v 2 given in [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF], we obtain after some developments

where u p i s p j s = Γ p i s (p j sp i s ). By substituting (36) into (35), we obtain

The detailed expression of u p i s p j s given by u p i s p j s = Γ p i s (p j sp i s ) = p j sp i s (p i s p j s )

which leads to up i s p j s = ṗj sṗi s (p i s p j s )p i s p j s ṗi s + p i s ṗj s .

(39) From (39), we obtain after some developments

with

By substituting (40) into (37), we obtain

By expanding the expression of v 3 given in [START_REF] Hamel | Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach[END_REF], we obtain v 3 = 1 Γ p i s (p j s -p i s ) (p i s × p j s ) from which it is straightforward to


Céline Teulière

Laurent Eck

Eric Marchand

Chasing a moving target from a flying UAV

This paper proposes a vision-based algorithm to autonomously track and chase a moving target with a smallsize flying UAV. The challenging constraints associated with the UAV flight led us to consider a density-based representation of the object to track. The proposed approach to estimate the target's position, orientation and scale, is built on a robust colorbased tracker using a multi-part representation. This object tracker can handle large displacements, occlusions and account for some image noise due to partial loss of wireless video link, thanks to the use of a particle filter. The information obtained from the visual tracker is then used to control the position and yaw angle of the UAV in order to chase the target. A hierarchical control scheme is designed to achieve the tracking task. Experiments on a quad-rotor UAV following a small moving car are provided to validate the proposed approach.

I. INTRODUCTION

The vision-based control of Unmanned Aerial Vehicles (UAVs) has become a very active field of research in the last decade [START_REF] Shakernia | Landing an unmanned air vehicle: Vision based motion estimation and nonlinear control[END_REF] [1] [START_REF] Saripalli | Visually guided landing of an unmanned aerial vehicle[END_REF] [6] [START_REF] Hérissé | A terrainfollowing control approach for a vtol unmanned aerial vehicle using average optical flow[END_REF]. Vision indeed provides a cheap, passive and rich source of information, and lowweight cameras can be embedded even on small-size flying UAVs. Until now, most of the efforts have been concentrated on developing vision-based control methods for autonomous take off, landing, stabilization and navigation, in which the visual information is usually obtained using a known model of a target [START_REF] Saripalli | Visually guided landing of an unmanned aerial vehicle[END_REF] or the environment [START_REF] Teulière | 3d model-based tracking for uav position control[END_REF], key images [START_REF] Courbon | Vision-based navigation of unmanned aerial vehicles[END_REF], or texture points for motion estimation [START_REF] Shakernia | Landing an unmanned air vehicle: Vision based motion estimation and nonlinear control[END_REF] [START_REF] Meingast | Vision based terrain recovery for landing unmanned aerial vehicles[END_REF] or optical flow computation [START_REF] Hérissé | A terrainfollowing control approach for a vtol unmanned aerial vehicle using average optical flow[END_REF].

In this paper we consider the specific task of chasing a moving object, in an unknown environment, and without any a priori model of the object (see figure 1). The reliability of the visual information is critical for the good realization of the vision-based control task. For autonomously performing such a task, one has to be able to robustly extract the object location from images despite difficult constraints: large displacements, occlusions, image noise, illumination and pose changes or image blurr.

Considerable work has already been done in visual tracking to address the aforementioned challenges, starting with designing an adequate representation for the object to be tracked. A simple and widely used way to describe an object is the image template, which stores luminance or color values, and their locations [START_REF] Hager | Efficient region tracking with parametric models of geometry and illumination[END_REF] [START_REF] Malis | Homography-based 2d visual tracking and servoing[END_REF]. Describing how the object looks like pixel-wise, image templates can accurately recover a large range of motions. However, they are very sensitive C. Teulière and L. Eck are with CEA, LIST, Interactive Robotics Unit, 18 route du Panorama, BP6, Fontenay aux Roses, F-92265 France firstname.name@cea.fr to some modifications in the object appearance due to pose changes, lighting variations, blurr or occlusions.

Density-based descriptors such as color histograms represent an attractive alternative for their low computational complexity and robustness to appearance changes [START_REF] Comaniciu | Real-time tracking of nonrigid objects using mean shift[END_REF] [START_REF] Pérez | Color-based probabilistic tracking[END_REF]. In the challenging UAV application context, strong simplifying assumptions are usually made in the vision algorithms. [2] [24]. In [START_REF] Azrad | Visual servoing of quadrotor micro-air vehicle using color-based tracking algorithm[END_REF] a color-based algorithm is used to track a fixed target and autonomously stabilize a UAV above it. Due to hardware limitations their proposed tracking approach is simplified and assumes that the target is clearly visible, without handling occlusions nor the presence of distractors of similar color.

In this paper, our objective is to provide a full vision-based system, using a color-based tracking method to robustly localize a moving object through frames and control the UAV to chase it. To our knowledge, this has never been done while considering potential loss due to occlusions, and estimating not only the position of the object but its rotation and scale changes in the image, which will allow us to control the UAV's attitude and yaw.

We consider a quad-rotor UAV (see figure 1 and7) equipped with a camera attached to its airframe, pointing downward, an inertial measurement unit (IMU), and a barometer. Except for the low level embedded attitude control, the computations are deported to a ground station. The data are transmitted between the ground station and the UAV through a radio transmission. Figure 2 gives an overview of the proposed system.

• The visual tracking system aims to provide an estimate of the relative position and in-plane rotation between the UAV and the object. To achieve this in a robust way, a color-based representation is chosen and the tracking is performed in the particle filtering framework. The tracking system is presented in section II.

• The control scheme is presented in section III. Estimations of the relative position and translational velocity between the UAV and the object, obtained by the vision system, are used as an input to the proposed control law. Experiments using the quad-rotor UAV are finally presented in section IV.

Onboard

II. OBJECT TRACKING ALGORITHM

The robust tracking of a moving object from a flying UAV, on long sequences, involves challenging constraints. In particular, the vision-based system has to face: large interframe displacements, due to both the UAV and the target motions, partial or full occlusions of the target (see figure 3-a and b), noisy images due to occasional erroneous transmission (see figure 3-c), background changes (see figure 3-d). The system also requires real-time capabilities.

A. Related works in color-based object tracking

Color-based object tracking has been extensively studied, since the successful application of the mean shift algorithm [START_REF] Comaniciu | Real-time tracking of nonrigid objects using mean shift[END_REF] to the tracking problem. The well-known drawback of choosing color histograms as a representation for the object is the loss of spatial information [START_REF] Hager | Multiple kernel tracking with ssd[END_REF], making difficult to track more complex motions than a simple translation. In the past few years different approaches have been proposed to tackle this issue, the main objective being to add some spatial information on the object to track while keeping the robustness of color histogram descriptors. In kernel-based methods, [START_REF] Hager | Multiple kernel tracking with ssd[END_REF] and [START_REF] Fan | Multiple collaborative kernel tracking[END_REF] proposed a Newton-like framework for using a set of multiple kernels. The spatial configuration between them allows to recover high-dimensional motion parameters. However, the search being deterministic, the algorithm can fail in case of total occlusion, presence of another object of similar color, or large interframe displacements.

Within another framework, [19] [17] propose to build a particle filter using the same kind of color-based similarity criterium to detect the translation and scale changes. The posterior density of probability of the object location is discretized in a set of weighted particles. The use of a probabilistic framework provides better robustness w.r.t. occlusions or presence of similar objects. To improve tracking accuracy, [19] [18] introduce the idea of dividing the object in several parts and to build a multi-part likelihood function, but the motion they estimate is limited to location and scale parameters. [START_REF] Maggio | Multi-part target representation for color tracking[END_REF] proposed a multi-part target representation for a better detection of scale and in-plane rotation parameters. More recently [14] [23] proposed to combine both deterministic and bayesian approaches using different multi-part target configurations. However, loss detection and initialisation or reinitialisation of the trackers are not mentionned.

B. Object representation

Object state: The goal of the tracking algorithm is to estimate the position of the object of interest through frames. The object is represented by a rectangle of fixed aspect ratio r = h w , which state is defined by its position c = (x, y) , orientation ψ and area A = hw (see figure 4). The state x k of the object in frame k is then given by

x k = (x k , y k , ψ k , 1 √ A k
) . This state will be used in section III as an input for the UAV control law. Object descriptor: Given the state x k of the object in frame k, and {l i } i=0..n k the pixel locations inside the rectangular boundaries, the color histogram is given by q(x k ) = {q u (x k )} u=1...m , m being the number of bins in which the color space is divided. For each bin u:

q u (x k ) = n k i=1 K(l i -c)δ u (b(l i )) (1)
where b(l i ) is the bin corresponding to the color of the pixel at location l i , δ designates Kronecker's function, and K is a kernel function centered in c, weighting image locations. In our case, K is the Epanechnikov kernel, which we suppose normalized so that m u=1 q u = 1.

Let q * = {q * u } u=1...m denote the reference histogram, determined in the tracking initialisation step. Then, the tracking process aims to find in each frame k, the candidate state x k which histogram q(x k ) is the "closest" to the reference histogram q * . To achieve this, a correlation criterion in the histogram space is provided by Bhattacharyya coefficient:

ρ(x k ) = ρ(q * , q(x k )) = m u=1 q * u q u (x k ). (2)
A candidate state x k for the object in frame k is then compared to the reference object using the Bhattacharyya distance:

d(x k ) = d(q * , q(x k )) = 1 -ρ(x k ). (3)
Multi-kernel representation: To improve the sensitivity of the object descriptor to different motions we use a generic multi-kernel representation. A set of identical kernels are positionned in the rectangular object.Weighting pixels locations with those kernels thus gives them a different importance according to their position in the object.

Formally, a state x k is associated to n h histograms {q j (x k )} j=1..n h computed with the kernels K j (l -c j ). The distances d j (x k ) = d(q * j , q j (x k )) are defined as in (3):

d m (x k ) = 1 n h n h j=1 d j (x k ) (4)

C. Tracking process

For the tracking to be robust to short occlusions of the object or noise due to transmission errors between the UAV and the ground station, we use the particle filtering framework. The proposed tracking scheme is based on the well-known CONDENSATION algorithm [START_REF] Isard | Condensation: conditional density propagation for visual tracking[END_REF]. The main idea is to represent the probability density function (p.d.f) p(x k | z 1:k ) of the state x k at frame k, by a finite set {(s

(i) k , π (i) k )} i=1..N of N samples, or particles, s (i) k associated with the weights π (i) k . Each particle s (i)
k represents a potential state for the object and z 1:k are the observations until frame k. For each new frame, the particles first evoluate according to a given dynamic model. Then, the likelihood of every particle is measured in the image and a weight is derived. The output considered is the weighted mean of the resulting set of particles. The particle set is updated by performing a random weighted draw among the particles. This resampling step promotes the best particles by duplicating them, to avoid degeneracy issues and keep a fair representation of the p.d.f.

Likelihood function: The likelihood function is derived from the distance defined in equation ( 4). The spatial information is yielded by computing n h histograms in the n h parts as in [START_REF] Pérez | Color-based probabilistic tracking[END_REF] [14] [START_REF] Teulière | A combination of particle filtering and deterministic approaches for multiple kernel tracking[END_REF]. Formally, the likelihood of a state x k is then defined by:

p(z k | x k ) ∝ exp(-λd m 2 (x k )) (5)
where λ is a constant parameter tuned empirically (a typical value is λ = 20).

For initialisation, the particles are sampled on a Gaussian distribution around the initial known position. The particles are propagated in the evolution step by a simple constant velocity model. The particle filtering output considered is the estimator of probability expectation:

E [x k ] = N i=1 π (i) k s (i) k .
The algorithm is summarized in figure 5.

Knowing the set of N particles {s } i=1..N . • Update: Using the observation z k the weight of each predicted particle is computed according to [START_REF] Comaniciu | Real-time tracking of nonrigid objects using mean shift[END_REF].

π (i) k ∝ p z k | x k = s k (i)
, with

N i=1 π (i) k = 1.
• Resampling by performing a random weighted draw of N particles from (s k

(i) , π (i) k ) i=1..N
, thus giving the new set: (s

(i) k , 1 N ) i=1..N
. Fig. 5. Color-based tracking with particle filtering.

D. Loss detection

In case of complete occlusions or temporary image losses (see figure 3-a andc) the distance measurement becomes irrelevant and the resampling step of the particule filter promotes the best particules in a wrong way. In such a situation, it is better to rely only on the evolution model. The tracker is considered to be lost when the minimal particle's distance d min (4) is above a given threshold d lim . For now, this threshold is tuned empirically, and refined in the beginning of the sequence, where we assume the object is not yet occluded, by taking d lim = 1.3d min . However it would benefit from an automatic adaptation for sequences with large illumination changes.

When the object is declared to be lost, the particle filter switches to a critical mode with no resampling step, until one particle gets under the threshold and the filter goes back to normal. This process allows the tracker to handle short occlusions or peaks of noise in the images. If the filter fails to recover, a reinitialisation procedure needs to be performed. This is discussed in the following section.

E. Automatic initialisation and reinitialisation

The tracking algorithm presented above assumes that the reference histograms of the object are available. They can be computed by selecting the target in the image in the beginning of the sequence as in [START_REF] Azrad | Visual servoing of quadrotor micro-air vehicle using color-based tracking algorithm[END_REF]. Here, to avoid this manual detection we provide the algorithm with an image of the target in which the model is computed.

Then the algorithm is able to automatically initialise by using a method similar to Camshift detection [START_REF] Bradski | Computer vision face tracking for use in a perceptual user interface[END_REF]. First, the reference histogram is used as a look-up table to build the image of probabilities (the "back projection" image) corresponding to the current image. This operation replaces the pixel values of the input image with the value of the corresponding bin of the histogram. Then, the resulting image represents the probability of each pixel to belong to the object (see figure 6). The histogram considered at this stage is the global histogram of the whole object, without multi-kernel division. The detection is achieved by running several Camshift searches from different initial windows resulting in several candidate positions for the object. The positions are then tested using the distance criterium ( 4) to determine which one corresponds to the object of interest.

The same process is used for reinitialisation in case of tracking failure (see section II-D). The reinitialisation can be requested manually by an operator from the ground station or automatically triggered when the tracking system remains lost after a given number of iterations.

III. UAV CONTROL

The tracking system presented in the previous section provides us with an estimate of the position, orientation and size of the object in the image. In this section we present the closed-loop control scheme used for the control the UAV using those estimates as measurements.

A. UAV Modelling

The UAV is represented by a rigid-body of mass m and of tensor of inertia I ∈ R 3×3 . Let us define the frame F c attached to the vehicle in its centre of mass, and assume it coincides with the camera frame (see figure 7). The position of the centre of mass of the vehicle relative to the world frame w p c is denoted by p. For simplicity of notation the rotation w R c of the body frame F c relative to F w = (e x , e y , e z ) is denoted by R. We also define the frame F p , centered in the centre of mass like F c , but whose axis are parallel to those of F w , so that p R c = w R c = R. Let v (respectively Ω) be the linear (resp. angular) velocity of the center of mass expressed in the world frame F w (resp.

in F c ). The control inputs to send to the vehicle are: T , a scalar input termed thrust or heave, applied in direction z c and Γ = [Γ x Γ y Γ z ] the control torques relative to the Euler angles.

Assuming the world frame is Galilean, Newton's equations of motion yield the following:

         ṗ = v m v = T Re z + mge z Ṙ = [Ω] × I Ω = -Ω × IΩ + Γ ( 6
)
where g is the gravity constant. The quad-rotor UAV is an underactuated system with 4 inputs. Its translational motion results from the rotations (pitch and roll). In this work we assume that the system's attitude is already controlled onboard with a separate high gain control loop [START_REF] Bertrand | Stability analysis of an uav controller using singular perturbation theory[END_REF]. Therefore, our control scheme acts as a controller sending orientation commands to a low-level controller which is responsible for robust flight.

B. Control scheme

Translational control: in our vision-based control scheme, the translational motion is controlled using the estimates of the location c = (x, y) of the object in the image plane obtained from the vision system. This location is expressed in the camera frame, that is, if (X c , Y c , Z c ) denote the unknown coordinates of the object in F c , then the estimate c from the tracker corresponds to x = Xc Zc and y = Yc Zc . As illustrated in figure 8, the displacement observed in the image plane is partly due to the UAV attitude. Note that it is independant of the distance Z c . We compensate this displacement using an estimate of the rotation matrix R computed onboard from the IMU, so that the position that we regulate to zero is the position c p = (x p , y p ) of the object in the projected frame F p . The relative position and velocity errors are defined by:

e cp = c * p -c p (7) e vp = v * p -v p (8)
where c * p is the desired position of the object in the image plane. In practice for the chasing task c * p = 0 since we want to see the object in the center of the image. The velocity v p is deduced from the differentiation of the position c p .

We use a hierarchical control. The inner-loop is a PI controller on the velocity, and the outer-loop a simple proportional control on the position:

v p * = -K p e cp (9)
The inner-loop on the velocity is required to ensure the stability of the system. It acts as a damping in the UAV control.

Yaw control: the yaw angle ψ is controlled by using a proportional controller:

Ω * z = -K pψ (ψ * -ψ) (10)
where K pψ denote the proportional gain. ψ * is the desired yaw angle, that is the desired orientation of the object in the image plane. It is set to zero in the experiments so that the UAV follows the rotations of the car. The yaw velocity Ω z is controlled onboard using gyrometers measurements.

Altitude: although the vision system also provides an estimate of the size of the object which can be taken as a depth measurement, we found that the velocity obtained by its differentiation was too noisy to be used directly in the altitude control. For the experiments the altitude is controlled onboard using the barometer measurements to remain constant.

IV. EXPERIMENTAL VALIDATION

This section presents the experiments conducted on the quad-rotor (X4-flyer) (figure 1). The UAV sends the images from its embedded camera to the ground station (PC) through a wireless analogical link at 2.4GHz. The data is processed on the ground station and the desired orientation and thrust are sent back to the quad-rotor vehicle. Onboard, the exponential stability of the orientation toward the desired one is ensured by a 'high gain' controller (running at 166Hz in the DSP) [START_REF] Guenard | A practical visual servo control for an unmanned aerial vehicle[END_REF]. On the ground station, the overall system (visual tracking and control computation) runs with a framerate of 20Hz. We tested the overall system on different sequences with a small colored car (figure 1). In all the experiments the RGB color space with 8 × 8 × 8 bins has been used. The particle filter runs with 200 particles.

A. Stabilization task

We first present a stabilization task performed while the target car was static, to validate the good behaviour of the control scheme. Figure 9 shows the resulting position error for the stabilization task: c p = (X p /Z p Y p /Z p ) .

In this experiment the measured altitude Z p was less than 2 meters, which means that the UAV was stabilized above the target with a maximal translation error of 10cm. Figure 9 shows that the maximum orientation error in stabilization is 3deg. a moving target: In this section we describe a longer sequence (about 10000 frames) in which the car is moving, with some turns and several complete occlusions (see figure 12). Figure 10 shows the trajectory of the car in the image.

Since the car's motion mainly occurs on the y axis, the position error is larger on this axis (up to 30cm, which remains very low). When a loss is detected, the estimate position is drawn in orange (figure 12-b). In the experiments, the occlusions are successfully detected, and the particle filter goes on predicting the car position with the constant velocity model, without resampling. When the car is detected again (the estimated position is drawn in green in the image frame), the particle filter goes back to its normal mode. Since the algorithm assumes that target velocity does not change much while it is occluded, its estimate can be wrong at the end of the occlusion in case of large acceleration or orientation change during the critical mode. However, without the resampling step the particles tend to cover a larger part of the state space and in most of our experiments the tracking algorithm succeeds in finding back the target. We also provide one example of loss after an occlusion where the autonomous reinitialisation procedure was necessary (figure 13). On condition that the target remains in the field of view, the reinitialisation allows the tracker to successfully recover from the loss. Since the barometer gives an absolute altitude measurement with a high variance (about 1 meter), the altitude control still requires some improvements. Its fusion with the estimate from the vision estimate which provides a more accurate relative information will be tested in future experiments. However, current experiments show that the tracking algorithm can handle scale changes (see figure 14). V. CONCLUSIONS In this paper, we proposed a full vision-based system, using a color-based tracking method to robustly track a moving object through frames and control a quad-rotor UAV to chase it. A multi-part tracker provides an estimate of the relative position and orientation between the UAV and the target in real time. The system has been successfully validated on a sequence with noise, full occlusions, turns and scale changes. The loss detection and reinitialisation system provided is efficient as long as the target remains in the field of view.

E

  Fig. 1. Quad-rotor UAV tracking a small vehicle and internal view from the embedded camera.

Fig. 2 .

 2 Fig. 2. Overview of the system.

Fig. 3 .

 3 Fig. 3. Examples of critical cases: full occlusion (a) and recovery (b), transmission noise (c) and background changes.

Fig. 4 .

 4 Fig. 4. Parameters of the state and rectangular representation.

1 }

 1 i=1..N with equal weights 1 N at step k -1: • Evolution of the particles according to a constant velocity model, giving a new set of particles: {s k (i)

Fig. 6 .

 6 Fig. 6. Example of back projection.

Fig. 7 .

 7 Fig. 7. Frame definitions.

Fig. 8 .

 8 Fig. 8. Attitude compensation: c is the position of the object that is estimated by the visual tracking algorithm, and cp is the corresponding position in the frame Fp where the rotation has been compensated.

Fig. 9 .

 9 Fig. 9. Trajectory of the center of the car in the image (pixel coordinates) and orientation error for the stabilization task.

Fig. 10 .

 10 Fig. 10. Trajectory of the center of the car in the image (pixel coordinates) for the chasing task.

Figure 11

 11 Figure 11 illustrates the yaw angle estimation and control with an example of turn.

Fig. 11 .

 11 Fig. 11. Example of turn: the tracking system properly estimates the rotation and the UAV follows it to keep the object with the same orientation in the camera frame.

Fig. 12 .

 12 Fig. 12. Example of complete occlusion.

Fig. 13 .

 13 Fig. 13. Example of target loss (a) and reinitialisation (b).

Fig. 14 .

 14 Fig. 14. Altitude changes.


Deformable Random Dot Markers

ABSTRACT

We extend planar fiducial markers using random dots [START_REF] Uchiyama | Random dot markers[END_REF] to nonrigidly deformable markers. Because the recognition and tracking of random dot markers are based on keypoint matching, we can estimate the deformation of the markers with nonrigid surface detection from keypoint correspondences. First, the initial pose of the markers is computed from a homography with RANSAC as a planar detection. Second, deformations are estimated from the minimization of a cost function for deformable surface fitting. We show augmentation results of 2D surface deformation recovery with several markers.

Index Terms: H.

INTRODUCTION

Fiducial markers have been utilized in a wide field of research. For example, the initialization of a camera pose in visual SLAM can be done with a marker to estimate a known scale. Studies on mixed/augmented reality have also been proposed such that a camera pose with respect to real world can be estimated to overlay virtual objects with geometrical consistency. Thanks to their simplicity of use, the stability of recognition and tracking process, up to now fiducial markers have been a useful tool and will continue to be utilized in the future as well.

The shape of planar fiducial markers was generally square with a black frame such as ARToolKit [START_REF] Kato | Marker tracking and HMD calibration for a video-based augmented reality conferencing system[END_REF] and ARTag [START_REF] Fiala | ARTag, a fiducial marker system using digital techniques[END_REF]. As another type of fiducial marker, random dot markers were proposed [START_REF] Uchiyama | Random dot markers[END_REF]. Compared with traditional square markers, the flexibility of design and the occlusion robustness were improved. However, random dot markers should still be planar as well as traditional markers. In order to enhance the functionality of fiducial markers, we propose an approach for recognizing and tracking nonrigidly deformable random dot markers.

RELATED WORKS

The world first AR system with fiducial markers was developed by Rekimoto [START_REF] Rekimoto | Matrix: a realtime object identification and registration method for augmented reality[END_REF]. The markers had a black and white coded square pattern inside a black frame and were recognized by decoding the pattern. The shape of binary pattern can be replaced with hexagon [START_REF] Uchiyama | MR platform: a basic body on which mixed reality applications are built[END_REF], circle [START_REF] Naimark | Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker[END_REF] and frame [START_REF] Wagner | Robust and unobtrusive marker tracking on mobile phones[END_REF]. Generally, such square markers are not able to be deformed because the pose of the markers is computed from the position of the four corners using homography.

Random dot markers have been developed to relax the constraints for the design and usability of fiducial markers [START_REF] Uchiyama | Random dot markers[END_REF]. Each dot is first extracted with binarization and utilized as a keypoint. The recognition and tracking of the markers are based on keypoint matching using local geometrical relationship of keypoints with LLAH (Locally Likely Arrangement Hashing) [START_REF] Nakai | Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval[END_REF]. It was experimentally confirmed that 40 random dots per marker were distinguishable enough for recognizing 1000 markers. Because the augmentation of a deformed surface can be useful for several AR systems, we extend this work to deformable random dot markers.

DETECTING AND TRACKING DEFORMATION

In this section, we describe the procedure of recognizing the markers and tracking their deformation.

Nonrigid Surface Recovery

Zhu and Lyu nicely formulated the problem of nonrigid surface recovery from keypoint correspondences such that the shape of 2D nonrigid surface could be computed by solving two linear equations with the finite number of iteration [START_REF] Zhu | Progressive finite newton approach to real-time nonrigid surface detection[END_REF]. The complexity of its computation is equivalent to one Newton optimization step. Because of low computational cost and the simplicity of the implementation, we incorporate this method into our approach.

As in [START_REF] Uchiyama | Random dot markers[END_REF], we first generate random dots without any overlay. In a keypoint database, the geometrical descriptors of each dot are stored. In addition, we decide the number of triangles and their size on a mesh as illustrated in Figure 2. The number of triangles is closely correlated with computational cost for recovering the shape.

Recognizing and Tracking Multiple Markers

For marker recognition, a marker is initially set as a plane and recognized by computing RANSAC based homography as in [START_REF] Uchiyama | Random dot markers[END_REF]. Once a marker is recognized, the detected plane is utilized as the reference plane for the Zhu's method [START_REF] Zhu | Progressive finite newton approach to real-time nonrigid surface detection[END_REF].

During tracking the deformation of a marker, the keypoints in a current image is matched with those in the previous image using LLAH [START_REF] Nakai | Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval[END_REF]. From these correspondences, we estimate the surface shape only with one iteration in the optimization of the Zhu's method because the deformation between consecutive images is small. This continuous estimation of the surface shape works as the tracking of marker deformation. As illustrated in Figure 3 (a) and (b), the marker shape can nonrigidly be deformed. We can consider that tracking fails when the number of inliers is lower than a threshold because the Zhu's method returns the inliers from the keypoint correspondences.

The overall procedure for recognizing and tracking multiple markers is as follows. First, all keypoints extracted in an image are matched with a keypoint database as marker recognition. To recognize multiple markers simultaneously, RANSAC based homography is computed for several candidates of markers that have enough correspondences as in [START_REF] Uchiyama | Random dot markers[END_REF]. Once markers are recognized, the keypoints in those markers are tracked by matching with the keypoints in the next frame using LLAH. For the remaining of the keypoints, that are not tracked in the image, marker recognition is again performed to find new markers. As illustrated in Figure 3 (c), multiple markers can simultaneously be recognized and tracked.

Note that the computational cost for recognizing one marker is not much affected by the number of markers registered in a database because LLAH utilizes a hash scheme for finding a keypoint correspondence from the database [START_REF] Nakai | Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval[END_REF]. The cost is mainly affected by the number of keypoints extracted in an image. This means that the cost increases when the number of markers captured in an image increases. Tracking the deformation of one marker can run over 30 frames per second on an Intel Core i7 M620 (2.67GHz).

CONCLUSION

We proposed an approach for recognizing and tracking multiple deformable fiducial markers using random dots. Because our marker system is based on keypoint matching, we utilized nonrigid surface recovery from keypoint correspondences as considered in [START_REF] Zhu | Progressive finite newton approach to real-time nonrigid surface detection[END_REF].

Other methods for nonrigid surface recovery such as [START_REF] Pilet | Fast non-rigid surface detection, registration and realistic augmentation[END_REF] and [START_REF] Gay-Bellile | Direct estimation of nonrigid registrations with image-based self-occlusion reasoning[END_REF] can also be applicable to our approach. One of the future works is the retexturing of the surface considering lighting conditions as in [START_REF] Zhu | Progressive finite newton approach to real-time nonrigid surface detection[END_REF]. The quality of retexturing random dot markers will be good because random dots can easily be removed by covering the dots with the color of the background. Also, we will develop some applications such that a virtual cloth can be overlaid on a white shirt with random black dot as an AR clothing system.

Figure 1 :

 1 Figure 1: Nonrigidly deformable random dot markers. (a) For an input image, (b) the deformed surface of a marker is recovered from keypoint correspondences using random dots, (c) texture is overlaid on the deformed surface.

Figure 2 :

 2 Figure 2: Mesh on a marker. A marker is represented by triangles with hexagonally connected vertices. The more the number of triangles increases, the more computational power is required.

Figure 3 :

 3 Figure 3: Augmentation results. A marker can nonrigidly be deformed as (a) and (b). (c) Multiple markers can simultaneously be recognized and tracked.