Philippe Martinet
email: philippe.martinet@lasmea.univ-bpclermont.fr

Djamel Khadraoui

Christian Thibaud

Jean Gallice

CONTROLLER SYNTHESIS APPLIED TO AUTOMATIC GUIDED VEHICLES

Keywords: Visual servoing, robot control, mobile robot, vehicles, modelling, vision

We h a ve b e e n i n terested in Automatic Guided Vehicles (AGV) for several years. In this paper, we synthesize controllers for AGV applications. Particularly, we are interested in road following and direction change tasks, and in analyzing the in uence of roll and pitch perturbations on vehicle behaviour. We use the bicycle as the kinematic vehicle model, and we c hoose the white band position of the road as the sensor signal. We de ne an interaction between the camera, which i s m o u n ted inside the vehicle, and the white band detected in the image space. Using this kind of interaction, we present h o w to use a pole assignment t e c hnique to solve the servoing task. We s h o w the simulation and experimental results (1/10 scale demonstrator) with and without perturbations.

INTRODUCTION

In the realm of intelligent systems for highways, development o f A GV is necessary to enable vehicles to drive automatically along the road. In fact, the requirement is for a controller that can maintain the position and the orientation of the vehicle with respect to the centre of the road and/or apply changes of direction. The problem of vehicle control using a camera has been given considerable attention by m a n y authors (Dickmanns and Zapp, 1987 Kehtarnavaz et al., 1991[START_REF] Raymond | Robust lateral control of highway v ehicles[END_REF][START_REF] Wallace | Progress in robot road following[END_REF][START_REF] Waxman | A visual navigation system for autonomous land vehicles[END_REF]. The work described in [START_REF] Jurie | A vision based control approach t o high speed automatic vehicle guidance[END_REF][START_REF] Jurie | High-speed vehicle guidance based on vision[END_REF] is among the most notable in lateral control using monocular vision. It consists of the reconstruction of the road using the 2D visual information extracted from the image processing system [START_REF] References Chapuis | Real-time vehicle trajectory supervision on the highway[END_REF]. In recent y ears, the integration of computer vision in robotics has steadily progressed, from the early \look and move" systems, to current systems in which visual feedback i s i n c o r p orated directly into the control loop. These techniques of vision based control are used to control holonomic robots in di erent domains [START_REF] Feddema | Visionguided servoing with feature-based trajectory generation[END_REF]Mitchell, 1989 Khadraoui et al., 1996 Papanikolopoulos et al., 1 9 9 1 P apanikolopoulos et al., 1993). The principle of this approach is described in gure 1. It is based on the task function approach [START_REF] Samson | Robot Control: The task function approach[END_REF], and many people have developed this concept applied to visual sensors [START_REF] Chaumette | La relation vision commande: th et application a des tâches robotiques[END_REF]Espiau et al., 1992 Hutchinson et al., 1 9 9 6 ) .

There are still few applications in mobile robots using this kind of approach. The main di culty is due to the presence of nonholonomic mechanical connections which limit robot movements [START_REF] Pissard-Gibollet | Asservissement visuel appliqu e a un robot mobile: etat de l'art et mod elisationcin ematique[END_REF].

In this paper, we propose a new technique with a visual servoing approach, in which c o n trol incorporates the visual feedback directly [START_REF] Khadraoui | Linear control of high speed vehicle in image space[END_REF]. In other words, this is speci ed in terms of regulation in the image frame of the camera. Our application involves controlling the lateral road position of a vehicle following the motorway white line. A complete 2D model of both the vehicle and the scene is then essential. It takes into account the visual features of the scene and the modelling of the vehicle.

The main purpose of this study is the development of a new lateral control algorithm. We propose a new control model, based on state space representation, where the elements of the state vector are represented by the parameters of the scene, extracted by vision. This approach w as experimented with a 1/10 scale demonstator.

MODELLING ASPECT

Before synthesizing the control laws, it is necessary to obtain the model of the vehicle and the one which expresses the interaction between the sensor and the environment.

Modelling the vehicle

It is useful to approximate the kinematic of the steering mechanism by assuming that the two front wheels turn slightly di erentially. Then, the instantaneous center of rotation can be determined purely by kinematic means. This amounts to assuming that the steering mechanism is the same as that of a bicycle. Let the angular velocity vector directed along z axis be called _ and the linear one directed along x axis called _ x.

Orientation equation: Using the bicycle model approximation (see gure 2 (a)), the steering angle and the radius of curvature r are related to the wheel base L by: tan = L r (1)

In gure 2 (b), we show a small portion of a circle S representing the trajectory to be followed by We assume that it moves with small displacements between initial curvilinear abscissa S 0 and a nal one S f such that:

1 r = l i m s!0 S = d dS = d dt dt dS (2)
where represents the orientation of the vehicle.

The dot of S includes the longitudinal and lateral velocities along the y and x axes respectively. I n fact, the rotation rate is obtained as:

_ = tan L p _ x 2 + _ y 2 (3)
Lateral position equation: In order to construct this equation, we treat the translational motion assuming that the vehicle moves with small displacements between t and t + t. In the case of a uniform movement during a lapse of time t, t h e vehicle moves through distance d = V t taking V as a constant longitudinal velocity (see gure 3). We express: 8 > < > :

S = ; lim t!0 x t+ t ; x t V t = ; _ x V C = _ y V (4)
In these expressions, C and S represent t h e trigonometric functions cosine and sine.

The approximation to small angles gives us the relation between the di erential of the lateral coordinate x and the lateral deviation with regard to , expressed as follows:

( _ x = ;V _ = V L (5)

Modelling the scene

The present section shows how to write the equation of the projected line in the image plane, using perspective projection. The scene consists of a 3D line and its projected image is represented by a 2 D line. Figure 4 shows the frames used in order to establish this relation. We use: R i = ( O x i y i z i ) as the frame attached to the 3D line R s = ( O x s y s z s ) as the sensor frame xed to the camera We take i n to account:

h the camera height the inclination angle of the camera the orientation of the vehicle Any 3 D point p i = ( x i y i z i 1) T related to the workspace can be represented by its projection in the image frame p p = ( x p y p z p ) T by the relation:

p p = Mp i (6)
The matrix M represents the homogeneous calibration matrix of the camera expressed in the frame R i such as:

M = P 0 c C 0 u C a R ;1 2 R ;1 1 T ;1 (7)
where:

P 0 c C 0 u C a translates p s = ( x s y s z s ) T in p p .

P 0 c C 0 u takes into account the intrinsic parameters of the camera (f x = fe x , f y = fe y ), and C a realizes an exchange coordinates frame. T, R 1 and R 2 characterize the extrinsic parameters of the camera. T takes into account the two translations of the camera: the height h and the lateral position x. R 1 and R 2 represent the lateral orientation and the inclination of the camera respectively. The expressions of the di erent matrices used are given as follows:

P 0 c C 0 u = " fx 0 0 0 0 fy 0 0 0 0 1 0 # C a = 2 6 4 1 0 0 0 0 0 ;1 0 0 1 0 0 0 0 0 0 3 7 5 T = 2 6 4 1 0 0 x 0 1 0 0 0 0 1 h 0 0 0 1 3 7 5 R 1 = 2 6 4 C ;S 0 0 S C 0 0 0 0 1 0 0 0 0 1 3 7 5 R 2 = 2 6 4 1 0 0 0 0 C ;S 0 0 S C 0 0 0 0 1 3 7 5
We note that the second translation and the third rotation are considered as null, and that x (lateral position) and (orientation) are the system variables.

Developing relation 7, we obtain the following expression of M: Eliminating y i from the equations 8, we obtain:

X = f x f y xC ; hS S hC Y + f x xS + hS C hC (9)
Considering that and are small (< 10 deg) and if we neglect the product term , the second order Taylor approximation gives us a new expression of X:

f x x f y h Y + f x x h + + O( 2 ) + O( 2 )(10)
The equation of the line expressed in the image frame is given by the following relation:

X = aY + b (11)
where (a b) are the line parameters expressed by: 8 > < > :

a = f x x f y h = 1 x b = f x x h + = 2 x + 3 (12)
We express the lateral position x and the orientation of the vehicle in order to de ne an interaction relation between the 3D position of the vehicle and the 2D parameters of the line in the image plane. We h a ve:

8 > < > : _ x = h f y f x _ a = 1 _ a _ = ; f y f x _ a + 1 f x _ b = 2 _ a + 3 _ b ( 13
)
with: f x = 1300pu, f y = 1911pu, V = 20km=h , L = 0 :3m and h = 0 :12m (in our 1/10 scale demonstrator).

CONTROL ASPECT

In this section, we show h o w to synthesize controllers with a pole assignment t e c hnique. First, we establish the state equation of the system, and we de ne a controller with di erent c haracteristics. We v erify by simulation the system behaviour without perturbations. We then introduce some perturbations, and analyse the simulation results. Secondly, w e modify the control law b y appending an integrator and present simulation results.

3.1 Pole assignment approach 3.1.1. Controller design Here, we present the application of the pole assignment t e c hnique when the state model is expressed directly in the sensor space. In our case, the sensor space is the image plane. The controller design is based on the kinematic model of the vehicle. We use the (a b) parameters of the 2D line in the image plane as the state vector. We drive the vehicle with the action on the wheel angle . W e c hoose b as the ouput parameter of the system and use the results of the vehicle and scene modellings to obtain the following equation:

;V = 1 _ a (V= L ) = 2 _ a + 3 _ b ( 14
)
The state vector, denoted by s = ( a b) T , is equal to the sensor signal vector in the state space representation. Developing, we h a ve the following state model of the system: _ s = As + B b = Cs (15)

with : A = 2 6 6 6 6 4

; V 2 1 ; V 3 1 V 2 2 1 3 V 2 1 3 7 7 7 7 5 B= 2 6 4 0 V L 3 3 7 5 C = 0 1
The visual servoing scheme is then as shown in the gure 5. where K = k 1 k 2 ] a n d k are the gains of the control law obtained by identifying the system to a second order system characterized by ! 0 and . For theses control gains, we obtain:

8 > > > > > < > > > > > : k 1 = L! 0 (2 2 V ; 1 ! 0 ) V 2 k 2 = 2 L 3 ! 0 V k = 1 L 3 ! 0 2 V 2 2 (17)
We note that, as expressed by these relations, the higher the velocity V , the smaller are the gains.

Simulation results

To v alidate this control law, we use a simulator developed with Matlab. Figure 6 shows the visual servoing scheme used in this simulator. We c hose ! 0 = 2 rd=s and = 0 :9 in order to x the behaviour of the system. In this case, we h a ve no perturbations ( is xed at -7 degrees).

The second set of results takes into account a perturbed angle . W e obtain the following response in b (see gure 8): We can see a static error in b, w h e n w e i n troduce the perturbations, and some oscillations appear when the angle increases. In this case, slows down from 0:9 t o 0 :6 and increases the overtaking, but the main contribution to the oscillations is due to the data ow latency.

In the next section, we express the static error in order to analyze the simulation and experimental results.

Closed l o op static error estimation

In this part, we analyse the behaviour of the vehicle when the extrinsic parameters of the camera are perturbed. In this case, equations 13 shows us that only 1 and 2 are a ected by t h i s k i n d o f perturbations.

; V ( 2 + 2 ) 1 + 1 ; V 3 1 + 1 V ( 2 + 2 ) 2 ( 1 + 1 ) 3 V ( 2 + 2 ) 1 + 1 3 7 7 7 7 5
Figure 9 represents the general visual servoing scheme, using the pole assignment approach. In this case, we can establish the general expression of the static error 1 by using the Laplace p transform by: Hence we obtain:

1 = h 1 + C (A ; BK) ;1 Bk i b 1 (20)
After some developments and approximations, we can write the nal relation of the static error as follows:

1 = " 1 ; 1 + 2 2 1 + 2V !0 2 1 2 2 # b 1 (21)
The static error becomes null, if the following expression is veri ed:

2 2 = = 0 (22)
We observe that, in the absence of perturbations in the parameter, we h a ve no static error. When is di erent from the reference value, we obtain an error which con rms all the simulation results. In the next section, we show t h e w ay to reduce this static error.

Pole assignment with integrator

In this section, we i n troduce an integrator into the control law in order to eliminate the static error in case of perturbations.

Controller design

The visual servoing scheme is presented by the gure 10 .

s - b* + - - i K ε δ 1 -1 b (pI -A) B C b K p
Fig. 10. Visual servoing scheme with integrator

In this case, we can express the control law b y the relation:

= ;k 1 a ; k 2 b ; K i Z ; b)dt (23)
where k 1 , k 2 and K i are the gains of the control law obtained by identifying the system to a third order system characterized by the following characteristic equation: (p 2 + 2 ! 0 p + ! 2 0 )(p + ! 0 ). The control law gains are given by:

8 > > > > > > > > > > < > > > > > > > > > > : k 1 = ; L! 0 ; 2 1 ! 0 2 ; 3 2 V + 1 ! 0 V 2 + L 1 2 ! 0 3 V 3 2 k 2 = 3L 3 ! 0 V K i = ; 1 L 3 ! 0 3 V 2 2 (24)

Simulation results

In the rst simulation, we h a ve no perturbations. Figure 11 illustrates the fact that the response time is correct. Secondly, w e i n troduce some perturbations in the angle. In fact, no static error persists during servoing, but some oscillations and problems of stability appear when is far from the reference value. Figure 12 shows these results.

EXPERIMENTAL RESULTS

In this section, we p r e s e n t di erent experimental results. The CCD camera is embedded on the end e ector of a cartesian robot and is connected to the vision system WINDIS.

Experimental site

To v alidate our approach, we programmed the control laws on our experimental robotic platform.

It is composed of a cartesian robot with 6 degrees of freedom (built by the rm AFMA Robot) and the parallel vision system WINDIS [START_REF] Martinet | Parallel architecture for visual servoing applications[END_REF][START_REF] Rives | A versatile parallel architecture for visual servoing applications[END_REF]. This whole platform is controlled by a VME system, and can be programmed in C language under the VxWorks real time operating system. Figure 13 presents the road built to a 1/10 scale. This road comprises three white lines. Using this kind of testbed allows us to use the visual servoing scheme illustrated in gure 14.

The vision system

Windis architecture comprises three basic modules (see gure 15). For each level, we i n troduced parallelism allowing us to reach video rate for most of the application tasks. The management o f s u c h a system is a tricky job and requires the use of a real time operating system. For ease of use, we c hoose to develop the whole system under VxWorks. The Windows Distributor extracts points from regions of interest and sends them through the bus Disbus. The DSP module (intermediate level processing) computes the (a b) parameters of the line from these selected points at video rate. In this implementation, we h a ve i d e n ti ed a data ow latency of three sample periods.

Experimental results comments

First, we experiment the control law with and without perturbations. As in simulation, we o btain a static error and some oscillations (see gure 16). In this experiment, we can verify the values of static error for di erent v alues of the parameter. For = ;8 w e h a ve 1 = 3 5 =34 = ;9 w e h a ve 1 = 4 9 =48

(measured value/ computed value). Figure 17 shows the results. We obtain the reduction of the static error in case of perturbation, but some oscillations and instability appears when is distant from the reference value of (-7 degrees).

CONCLUSION AND FUTURE WORK

Controllers based on a visual servoing approach, have been developed in this paper. We designed a controller with a pole assignment technique directly in the image space. After modelling the vehicle and the scene, we obtained equations which can be used to write the state model of the system. Visual servoing is performed well when there are no perturbations. When perturbations occur, a static error and oscillations appear. By introducing an integrator into the visual servoing scheme, we suppress the static error but amplify the oscillation problem.

In the future, we will investigate a robust control approach, which means de ning a controller which must be robust with regard to wide variations of parameter modelling. The choice of b as the ouput parameter of the system does not permit the control of the lateral position of the vehicle precisely when the perturbations appear, but insure a heading control. We look in the future how to control the system with the a parameter under

Fig. 1 .

 1 Fig.1. Visual servoing scheme.

Fig. 2 .

 2 Fig.2. Bicycle model. the vehicle. We assume that it moves with small displacements between initial curvilinear abscissa S 0 and a nal one S f such that: 1 r = l i m

Fig. 3 .

 3 Fig. 3. Kinematic modelling of the vehicle.

Fig. 4 .

 4 Fig. 4. Perspective projection of a 3D line.

  S f y S C ;fy C ;fy xS S + fy hC ;C S C C S ;xC S + hS # The pixel coordinates P = ( X = xp zp Y= yp zp ) T associated with each p o i n t o f t h e 3 D line (with x i = z i = 0), are expressed by: y i C C ; xC S + hS Y = f y ;y i S C ; xS S + hC y i C C ; xC S + hS (8)

Fig. 5 .

 5 Fig. 5. Visual servoing scheme with a pole assignment Finally, w e can express the control law b y t h e following relation: = ;k 1 a ; k 2 b + kb

Fig. 6 .

 6 Fig. 6. Visual servoing scheme of the simulator We use the kinematic model of the vehicle to simulate the behaviour of the vehicle, and the perspective projection relation to obtain the sensor signal s = ( a b) T . The rst results (see gure 7) illustrate the output behaviour of the system corresponding to an input value b = 100 pixels. We t a k e i n to account a data ow latency (three

  Fig. 8. b with perturbations ( = -7,-8,-9)

  Fig. 9. Static error Considering the state equations 15 and taking into account parameters perturbations, we h a ve:

  Fig. 11. b without perturbations ( = -7 )

  Fig. 12. b with perturbations ( = -7,-8,-9)

Fig. 13 .

 13 Fig.13. Road on a 1/10 scale. Figure13presents the road built to a 1/10 scale. This road comprises three white lines. Using this kind of testbed allows us to use the visual servoing scheme illustrated in gure 14.

Fig. 15 .

 15 Fig. 14. Visual servoing scheme The three basic modules the Window Distributor Subsystem, the WINPROC Window Processing Subsystem and the WINMAN Window Manager Subsystem.

  Fig. 16. b parameter: = -7, -8, -9 Secondly, w e modify the control law b y i n troducing an integrator in the visual servoing scheme.Figure17shows the results. We obtain the reduction of the static error in case of perturbation, but some oscillations and instability appears when is distant from the reference value of (-7 degrees).

  Fig. 17. b parameter: = -7, -8, -9

perturbations. We assume that experimentation on a real vehicle will be necessary to validate all of the primary results obtained in our laboratory.


P Martinet

N Daucher

J Gallice

M Dhome

Robot Control using Monocular Pose Estimation

Since the last years, we are interested in visual servoing and we h a ve d e v eloped many visual servoing applications using 2D visual features. The aim of this paper is to study how w e c a n i n troduce 3D visual features in a robot control loop. We consider a camera mounted on the end e ector of the manipulator robot to estimate the pose of the target object. The required positioning task is to reach a speci c pose between the sensor frame and a target object frame. Knowing the target object model, we can localize the object in the 3D visual sensor frame and obtain the pose between the camera and the target object at each iteration. To insure the visual servoing task, we h a ve d e v eloped two approaches. The rst approach is based on the task function approach i n troduced by Samson and Espiau at the end of 80's. We considered a 3D visual sensor which gives the 3D visual pose of an object in the sensor frame. In this case, the sensor signal (dimension 6) is composed of the position and the orientation of the frame object in the sensor frame. The second approach consists to split the trajectory between the current and the desired pose in a set of elementaries screwing tasks. At e a c h iteration, the situation between the sensor and the object is computed, and then the trajectory is updated.

INTRODUCTION

Sanderson and Weiss in 25] introduced an important classi cation of visual servo structures based on two criterias: space control and presence of joint feedback. So, in this classi cation we distinguish two main approaches:

Position Based Control : in this case, image features are extracted from the image and a model of the scene and the target is used to determine the pose of the target with respect to the frame attached to the camera.

Image Based Control : in image based control, the pose estimation is omitted, and the control law is directly expressed in the sensor space (image space). Figure 1 and 2 illustrated the both servoing scheme. These state of the art in the eld of visual servoing show that the Image Based Control has retained to be an alternative w ay t o t h e P osition Based Control approach. Generally, m a n y authors consider that Image Based Control approach is better than the other one, according to the following criteria: camera calibration, hand-eye calibration, robot modelling, scene and target modelling, ... , and also regarding to the processing time to compute the sensor signal.

It is clear that Image Based Control approach d o not need a precise calibration and modelling, because of the closed loop de ned in the sensor space. Many works 1], 4], 6], 9], 10], 12], 16], 19], 21], 22], 23] and many others, has been done in the case of the camera sensor and the 2D space. We recommand the reader to have a look on the review previously quoted and on 2] for an introduction on active perception.

The notion of Task Function introduced by Samson et al in 24], can be used to elaborate a control law i n the sensor space. According to this concept, Martinet et al in 20] introduce the notion of 3D logical sensor which deliver a 3D sensor signal by monocular vision at video rate. Recent progress in pose estimation, localization and 3D modelling 7], 15] shows that it is not an utopic way t o i n troduce 3D visual information in a closed loop control. Using this assumption, we c a n synthesize control laws using this kind of informations like w e do directly with the camera sensor. Another way to obtained the estimation of the pose, is to compute a Kalman lter using several visual features like presented in 27]. In this case, the control law is dened to reach a particular pose between sensor and object frames with a PD controller. In fact, few work 11], 20], 27] has been done using 3D sensor signal. However, we can remark that precise calibration and modelling are really usefull only in the case where the task to achieve is expressed in the cartesian space. If we learn the 3D reference signal in real conditions, like in Image Based Approach, we obtain the same good results in the logical sensor space.

The aim of this paper is to study how w e c a n introduce 3D visual features in a robot control loop. In the rst section of the paper, we discuss how to extract 3D visual features with a logical sensor. Knowing the object model, the logical sensor uses the De Menthon Algorithm 7] to localize the object in the 3D visual sensor frame. At each iteration we obtain the pose between the camera and the target object. In the second section, and the third section we present the two approaches developed in our laboratory. In the last section, we show the results obtained by simulation and experimentation with our robotic platform. We use a speci c object composed of four illuminated points in real experimentation. Finally we conclude and present some perspectives for future development in visual servoing.

POSE ESTIMATION BY MONOCULAR VISION

During the last ten years, many methods to locate 3D objects from one monocular image was proposed in the litterature. All of these approaches need some prerequisites, as the knowledge of the geometrical model of the observed object and matches between 2D image primitives and 3D model elements, to compensate the loss of data due to the projection of the 3D world on the image plane. Using these informations, the methods look for the spatial attitudes of the 3D model such that the matched model elements are projeted on the corresponding image primitives.

All of these approaches can be classi ed in function of the manipulated image primitives (points, straight lines, elliptical contours, limbs of curved surfaces ...) and the assumption about the projection of the real world on the image plane (orthographic, weak perspective, full perspective projection). Some methods give closed form solutions of the addressed inverse perspective problem, the others uses iterative processes to reach the solution.

The reader will nd in the following references an overview of the di erent kinds of approach 3], 7], 8], 14], 15], 17], 18], 26].

If up to a recent time, the computing time of such algorithms was prohibity to implement them during a real time closed loop control : it is no more the case. For example, the approach p r o p o s e d b y D. Dementhon 7] allows to compute the spatial attitude of a 3D object in a fraction of the image acquisition cycle.

FIRST APPROACH Introduction

In the last ten years, many people have become interested in how t o i n troduce external sensor signals directly into the control loop. Near the end of the 1980's, C. Samson and B. Espiau proposed a new approach, called "Sensor Based Control" 9], 24]. In this approach, any robot task can be characterized as a regulation to zero of a task function, where it de nes the interaction between the sensors and the robot environment. It is expressed with the following relation: e(r t ) = s(r t ) ; s ? ]

(1) where s ? is considered as a reference target sensor fea- ture to be reached in the sensor frame, and s(r t ) i s the value of sensor information currently observed by the sensors.

Figure 3 shows the 3D sensor based servoing scheme.

The interaction between sensor and scene is given in matrix form by: @s @r = L t s (2) This is called the interaction matrix which links the interaction between the robot and its environment. We where _ s is the time variation of s and T = ( V ) t is the velocity screw of the camera which englobes three translations and three rotations.

Considering an exponential convergence (for a task positioning) of all components of the task function (i.e. _ e = ; e, where is a positive scalar constant), we obtain the expression of the control law: C = L t+ s (4) where L t+ s represents the pseudo-inverse matrix of L t s .

The stability depends of the positivity o f t h e p r o duct term L t s :C. If L t ;1 s exists in all robot con gurations, this control law realize a perfect decoupling servoing scheme, and it is equivalent to consider the system as a juxtaposition of a set of closed integrator. Figure 4 represents the servoing scheme for each component S i of the sensor signal S. We c hoose the positif gain to insure the stability. The sensor signal S decreases as a rst order system with a constant of time = ;1 . In this case, the rst order steady state error is null (e 1 = 0).

Principle

We consider a scene with a 3D object and a wrist 3D sensory apparatus mounted on the end e ector of the robot. We de ne three homogeneous matrices transformation as follows:

M o is the homogeneous matrix transformation between an absolute frame attached to the scene, and the object frame R o M ct is the homogeneous matrix transformation between an absolute frame attached to the scene, and the sensor frame computed at each iteration R ct M c is the homogeneous matrix transformation between an absolute frame attached to the scene, and the sensor frame desired at the equilibrium R c . Figure 5 represents the scene with the di erent frames. The sensor signal S is a vector of dimension 6 and is composed of: X = AC] c the position of the absolute frame expressed in the sensor frame the orientation of the sensor frame relative t o the absolute frame attached to the scene. We h a ve: S = ( X ) t .

In the absolute frame, we h a ve the relation: AC] a = R c AC] c (5) where R c is the rotation matrix extracted from M ct . We use the exponential representation for the expression of the rotation. So, we write: R c = exp(AS( )) (6) where AS( ) represents the antisymetric matrix associated with the vector .

In our case, we h a ve ] a = R c ] c = ] c because the product AS( ) = 0 (cross product of two i d e n tical vectors).

Derivating relation 5, we obtain:

d AC] a dt = dR c dt AC] c + R c d AC] c dt (7)
In this equation, the rst term represents the translation velocity V of the robot expressed in the ab- solute frame. Using the de nition of R c , w e h a ve: dRc dt = AS( ) R c . In this expression, represents the rotation velocity v ector. Using the transpose matrix of R c , w e can write the following relation:

R c t V ] a = R c t AS( ) R c AC] c + d AC] c
dt [START_REF] Dhome | Determination of the Attitude of 3D Objects from a Single Perspective Image[END_REF] where:

the rst term R c t V ] a represents the translation velocity V of the sensor in the sensor frame the expression R c t AS( ) R c can be rewrite as ;AS( ). So, we h a ve: dX dt = V + AS( ) X ( 9) with X = AC] c . Using the relation AS( ) X = ;AS(X ) , w e can rewrite 9 as: dX dt = I 3 ;AS(X)] T (10) where T = ( V ) t is the kinematic screw expressed in the sensor frame. We h a ve also the following relation: d dt = (11) Derivating S = ( X ) t , from relation 3, 10 and 11, we obtain: _ S = L t s T = I 3 ;AS(X)] O 3 I 3 ]] T (12) We can give the expression of the inverse of L t s :

L t ;1 s = I 3 A S (X)] O 3 I 3 ]] (13)
The control matrix is given by C = L t ;1 s , and the system is equivalent to a set of decoupled integrators. In real case, if we consider an estimation of the sensor signal S = S est , the control matrix becomes:

C = I 3 A S (X est )] O 3 I 3 ]] ( 14
) and the product L t s C is not equal to but to L t s L t ;1 sest . In fact, we h a ve:

L t s C = I 3 A S (X est ; X)] O 3 I 3 ]] (15)
The orientation is decoupled and we obtain an exponential decay of the rotation angles. It is not the same case for the coordinate of the end e ector.

Convergence and stability of the control law

To insure the stability of the system, the product L t s C must be positive. In other terms, for all vectors Z = ( z a ) t of the state space (dimension 6) , we h a ve t o v erify the condition: Z t L t s C Z > 0 (16) and then that the scalar product < Z L t s C Z > is positive.

We can rewrite the scalar product < Z L t s C Z > as:

< Z Z> + < z A S (X est ; X) z > (17) In this new expression, the second term is null (triple product with two identical vectors) and the rst term represents the norm of the vector Z. Then, for all Z di erent to zero, we h a ve < Z L t s C Z > > 0 and the matrix L t s C is always positive which i n s u r e the stability of the system for all con gurations of the robot.

The steady state error is e = S ; S est becomes null at the end of the servoing and insures a perfect positioning task. The reference S must be learned by training, and then it compensate the error of modelling.

Modelling the closed loop control

To model the closed loop control, we h a ve to compute the evolution of the sensor signal S at each iteration. For that, we h a ve t o s o l v e the following relation: _ S = L t s T (18) for a given kinematic screw T. Considering that S = ( X ) t , from relation 12 we deduce: [START_REF] Martinet | Use of rst derivative of geometric features in Visual Servoing[END_REF] Then, we can solve the system of equation like:

_ = _ X = V + AS(;X) = V + AS( ) X
(t) = (t0) + R (t 0 )dt 0 X(t) = exp( R AS( (t 0 ))dt 0 ) X(t0)+ exp( R AS( (t 0 ))dt 0 ) R exp(; R AS( (t 00
))dt 00

)V (t 0 )dt 0 ( 20
)
If we consider a kinematic screw constant b e t ween time t 0 and t, w e obtain the following relation:

(t) = (t0) + (t ; t0) X(t) = exp(AS( ) (t ; t0)) X(t0)+ R t;t 0 0 exp(AS( ) t 0 )dt 0 V (21)
Using the Rodrigues formula, we h a ve: exp(AS( ) (t ; t0)) = cos( (t ; t0)) I3+ sin( (t ; t0)) AS(v) + ( 1 ; cos( (t ; t0))) v v t (22) In the following expression, AS( ) = AS(v), represents the norm of the vector and (v the unitary vector in the direction of ).

So, we can rewrite expression 21 like: X(t) = cos( (t ; t0)) I3 + sin( (t ; t0)) AS(v)+ (1 ; cos( (t ; t0))) v v t ] X(t0)+ sin( (t ; t0))= I3 + ( 1 ; cos( (t ; t0))= ) AS(v)+ (t ; t0 ; sin

( (t ; t0))= ) v v t ] V (23)

SECOND APPROACH Principle

The second approach is based on following assumption: the whole system including image acquisition and localization algorithm is taken as a black b o x d e l i v ering at video rate the pose parameters of an observed object.

Moreover, the proof of the convergence presented in the next paragraph is based on the hypothesis that the robot motion is perfect. The computation at each image shot of the target object pose permits to handle the behavior of the robot system. The pose parameters can be indi erently seen as the object pose in the camera reference system or as the pose camera relative to the object. These informations can be expressed as rigid transformation matrices of the form: M = R 3X3 T 3X1 0 1 where R is a rotation matrix and T a translation vector.

If, (see gure 6) M t denotes the object localization matrix in the reference system R ct of the camera at time t, a n d M the object desired pose matrix in R c , the C t command to be applied to the robot must transform R ct into R c .

In this way the coordinates X] m of a point X of the object de ned in the R m reference system are moved to X] ct in the reference system R ct following the formula: X] ct = M t X] m In the same way, X] c = M X] m where X] c are the coordinates of X in R c .

] m = M ;1 t X] ct X] m = M ;1 X] c Thus X] c = M t M ;1 ] ;1 X
] ct This relation expresses the fact that the matrix M t M ;1 ] ;1 is the pose matrix for reference system R ct relative t o R c .

Obviously the command transforming R ct in R c is:

C t = M t M ;1
The command is given by a rigid transformation matrix and is numerically performed as a (4 4) matrix product, the inversion of M being computed once o line before running the feedback c o n trol process.

The task funtion can be expressed relative t o t h e command law as: e] t = M ; M t = I ; C t ]M where I = (I) 3X3 (0) 3X1 0 1 On the other side, the pose to obtain expressed versus the current pose is: M = C ;1 t M t From the matrix C t , w e can extract the parameter vector X of the rigid transformation between R c and R ct . The inverse geometric model of the robot transforms this vector in an vector q of the corresponding joint v ariations. Last, a normalization relative to the sampling period t leads to the speed vector _ q to be applied to the motors controling the joints.

If the observed object is not moving, the minimization of the task function will be completed when the vector _ q will be zeroed (see gure 6), which corre- sponds to a command C t : C t = (I) 3X3 (0) 3X1 0 1 At this time M t = M and the regulation of the task function is done, the robot being located in the desired pose.

Theoretically, the form of the command law show that the task can be performed in one iteration, of course numerous factors prevent from sending the whole command directly to actuators of the joints the most important being the robot dynamic.

The rst factor naturally leads to consider only a fraction < 1 of the command:

C t = ( M t :M ;1 )
The computation of the fractional power of a rigid transformation matrix th is simple using a screwing representation (see gure 7 for more details). At each iteration (image shot), we compute the current pose M t from images primitives and the command C t to send to perform the task. At e a c h iteration, the M t computation will also permits to take i n to account the two kinds of following errors: errors induced by visual primitives extraction modelization errors in the robot and the acquisition system. The choice of is important and its value depends of the state of completion of the task. We will give in the following paragraph, a necessary condition for task convergence.

Convergence and exponential decrease of the task function Convergence Let us suppose that at time t in the servo control process the object is located at M t relative to the robot camera. The task function to regulate is: e] t = M ; M t The command C t will now bring the robot in a new pose given by: M t+1 = C ;1 t :M t Thus:

M t+1 = ( M M ;1 t ) M t To prove the theoretical convergence of the task, it su ces to show t h a t M t tends to M as t tends to in nity.

It is easy to prove b y recursion that: M t = ( M M ;1 0 )

2]0 2
We n d a g a i n t h a t i f = 1, (which is only to apply the whole command at the rst step) the task function is immediately satis ed. Extremal values = 0 a n d = 2 respectively correspond to a static behavior of the robot (M t = M 0 8t) and at a oscillating one (M t = M 0 if t is even, M t = ( M M ;1 0 ) 2 M 0 if t is odd).

Exponential decrease

We h a ve s h o wn that our command law insures the convergence of the task function. To be sure of good performance, it is quite necessary to know w h a t i s t h e rate of this convergence. We s h o w here that the error is exponentially decreasing with time.

Let us remember that the task function is given at time t by: e] t = M ; M t If we replace M t by its computed value (24), we obtain: e] t = M ; (M M ;1 0 ) 1;(1; ) t M 0 Thus: e] t = I ; (M 0 M ;1 ) ;1+(1; ) t M 0 M ;1 ]M And we h a ve the following relation: e] t = I ; (M 0 M ;1 ) (1; ) t ]M Time evolution of e] t depends of (M 0 M ;1 ) (1; ) t . This expression is nothing but a fraction of the screwing representation (M 0 M ;1 ). If 0 and 0 respectively characterize the translation norm and the rotation angle of the screwing representation (M 0 M ;1 ), the desired fraction will be represented by a rotation angle t , and a translation vector T t whose components will be determined by the following relations: t = ( 1 ; ) t 0 and T t = where the coordinates x c y c characterize a point o f t h e screwing axis in a particular coordinates system. The origin of this system corresponds to the origin of the reference system R ct (see gure 7). The z-axis is colinear to the screwing axis, the x-axis is colinear to the projection of the translation in a plane orthogonal to the screwing axis and the y-axis is obtained by a cross product. The screwing angle is now o b viously exponentially decreasing if 2]0 1 . For the translation, we will show that its norm is non increasing along time and majored by a decreasing exponential. Using elementary trigonometry, w e can write:

T t = 2 4
2 sin t 2 (x c sin t 2 + y c cos t 2 ) ;2 s i n t 2 (x c cos t 2 ; y c sin t 2 ) (1 ; ) t 0 3 5

The square of the norm of vector T t can be written: jjT t j j 2 = 4sin 2 t 2 (x 2 c + y 2 c ) + ( 1 ; ) 2t 2 0 The two conditions: 2 ]0 1 2 0 ] insure that the norm decrease to 0. Moreover as 8xj sin xj < jxj, w e get: jjT t j j 2 2 t (x 2 c + y 2 c ) + ( 1 ; ) 2t 2 0 Thus: jjT t j j 2 (1 ; ) 2t 2 0 (x 2 c + y 2 c ) + ( 1 ; ) 2t 2 0 jjT t j j 2 (1 ; ) 2t 2 0 (x 2 c + y 2 c ) + 2 0 ] that prove the property previously stated.

It is interesting to notice that the servoing behavior is quite independant of the starting position (the object must be visible). The above computations prove the convergence and decrease of the task function.

RESULTS

Simulation results: First approach

To v alidate the rst approach, we h a ve d e v eloped a s i m ulator in Matlab using the main relations previously developed. We represent the scene with two frames. The object frame is centered in the space, and materialized by a p yramid.

Figure 8 shows the 3D trajectory of the sensor frame during servoing. The initial S i and the nal S f = S value of the sensor signal are: S i = ( ;1:691 ;1:978 ;0:691 1:047 0 ;1:047) T S f = ( ;0:150 ;2:589 ;0:076 ;1:034 1:561 ;0:007) T The low l e v el image processing consists of a simple barycenter computation for each of the leds. From these four detected points, the Dementhon's algorithm 7] is used to locate the modelled object and thus the control laws can be computed. The sampling period is given by the image acquisition time, i.e. 40 ms.

First approach

We de ne three frames as follows: R S is an absolute frame attached to the scene, R C is a frame attached to the 3D sensor, and R O is the object frame. To test our approach, we implemented a 3D logical sensor on the "Windis" parallel vision system. This architecture, developed in collaboration with INRIA 1 in Sophia Antipolis (France), implements the concept of active windows. Figure 15 shows this architecture, where several active w i n d o ws (of varying size and position) are extracted from the image at video rate. Low level processing (3 3 o r 5 5 c o n volution) is then executed, and results are sent to the DSP modules. The DSP modules provide a geometric description of the required primitive i n e a c h window. The window m a nager controls all the architecture and computes the "3D logical sensor" features resulting from the geometric description. We use a six degrees of freedom cartesian robot, where the camera is embedded on the end e ector. The control law w as evaluated and implemented on our experimental site (see Figure 16). All software was written in C language and we use the VxWorks real time system environment. We h a ve tested the rst approach with various initial and nal situations of the visual sensor frame relating to the 3D object frame. The initial position of [START_REF] Allen | Real time visual servoing[END_REF] Institut National de Recherche en Informatique et Automatique the sensor is always X = ( ;253 ;101 1500) t (mm) in front of the object and the orientation = ( 0 0 0) t (deg). For the nal pose S = S f = ( X ) t we c hoose a speci c position X = ( 0 0 750) t (mm) and di erent o r i e n tation = ( X 0 0) t depending on the X angle (10 ;10 15 ;15 ;20) (deg). In each case, we estimate the interaction matrix at the equilibrium situation (L t s = L t s=s ? ). We obtained an exponential decay of the error signal and a convergence at around 200 iterations during all tests. We p r e s e n t the results for X = 15 deg. Figure 17 shows the 3D points trajectories during the visual servoing (which traduces the object trajectory in the camera frame). In Figures 18 and19, the position and orientation of the object are presented. The behavior is like a r s t order system. We noticed a little disturbance on the curves, due to the 3D logical sensor sensibility (less than 1 degree per second on the rotation velocities). This disturbance is due to the di culty to extract the Z position and the orientation of the object in the sensor frame in a dynamic sequence. To improve t h e results of the 3D logical sensor we think that we h a ve

Second approach

In the second approach, we h a ve c hosen to control the robot such as the desired pose of the tetrahedron (M ) gives a symmetric centered image. Actually, w e h a ve not realize a complete comparison between these both approaches, and we present n o results in this way. The particularity of this kind of method appears in the simplicity of the formalism the control laws depend only on the desired and current situations of the observed object. Then, from one application to another, only the pose algorithm has to be modi ed. For the future, we w ant to compare the two 3 D approaches developed in this paper.

Figure 2 :

 2 Figure 2: Dynamic imaged based look and move structure A bibliographic review on Visual Servoing has been done in 13], and particularly developed by P . Corke in 5]. A tutorial on Visual Servo Control has been organised by G. Hager et al 12] in 1996.These state of the art in the eld of visual servoing show that the Image Based Control has retained to be an alternative w ay t o t h e P osition Based Control approach. Generally, m a n y authors consider that Image Based Control approach is better than the other one, according to the following criteria: camera calibration, hand-eye calibration, robot modelling, scene and target modelling, ... , and also regarding to the processing time to compute the sensor signal.

  Figure 3: 3D servoing scheme

Figure 4 :

 4 Figure 4: Servoing scheme in ideal case

Figure 5 :

 5 Figure 5: Di erent frames used in modelling

Figure 6 :

 6 Figure 6: Control law computation

Figure 7 :

 7 Figure 7: Screwing representation of a rigid transformation

  1 ; cos t ) + y c sin t ;x c sin t + y c (1 ; cos t ) (1 ; ) t 0 3 5

Figure 8 :

 8 Figure 8: 3D trajectory of the sensor

Figure 9

 9 Figure 9 and 10 show t h e e v olution of sensor signal.

Figure 9 :

 9 Figure 9: Position of the sensorIf we i n troduce a noise of 30% (proportional to the magnitude of the sensor signal) directly on the sensor signal, we obtain the behavior represented on the gure 11. We can observe a little disturbance during servoing, but the convergence of the control is insured.

Figure 10 :

 10 Figure 10: Orientation of the sensor

Figure 12 :

 12 Figure 12: Position of the sensor with noise

Figure 13 :

 13 Figure 13: Orientation of the sensor with noise

Figure 14 :

 14 Figure 14: Object and sensor in the scene.

Figure 14

 14 Figure 14 represents the scene with the di erent frames used in this experimentation.To test our approach, we implemented a 3D logical

Figure 15 :Figure 16 :

 1516 Figure 15: Overview of Windis architecture

Figure 18 :

 18 Figure 17: Object trajectory

Figure 19 :

 19 Figure 19: Sensor signal: orientation

Figure 20 :

 20 Figure 20: The four leds evolution The rst scheme (see gure 20) shows the evolution in the image of the four barycentres relative t o a 3 D camera displacement around 1.2m.

Figure 22

 22 Figure22and gure 23 show the robot con gurations at the beginning and at the end of the process. Figure24and gure 25 characterize the evolution of the command applied to the camera and gure 21 shows the displacement of the optical center in the space during servoing. We can note the exponential decrease of these di erent v alues and a convergence before 200 iterations.

Figure 24

 24 Figure 22 and gure 23 show the robot con gurations at the beginning and at the end of the process.Figure 24 and gure 25 characterize the evolution of the command applied to the camera and gure 21 shows the displacement of the optical center in the space during servoing. We can note the exponential decrease of these di erent v alues and a convergence before 200 iterations. The last curves (see gure 26 and gure 27 ) are relative to the speed values applied to the robot ar-

Figure 21 :

 21 Figure 21: Camera optical center positions evolution

Figure 22 :

 22 Figure 22: Initial robot arm con guration

Figure 24 :

 24 Figure 24: Screwing angle evolution

Figure 25 :

 25 Figure 25: Screwing translation norm evolution

Figure 26

 26 Figure 26: Rotation parameters evolution

Figure 27 :

 27 Figure 27: Translation parameters evolution CONCLUSION Many people are interested in visual servoing. Until now, only 2D visual features has been considered. In this paper, we s h o w that 3D visual sensor elaborating 3D features at video rate can be used. We h a ve developed theorically two c o n trol laws and validated their e ectiveness in real conditions. Results seem to be satifactory in regard with the primitive extraction (image led barycenter computation) and pose estimation (De Menthon algorithm) algorithms.Actually, w e h a ve not realize a complete comparison between these both approaches, and we present n o results in this way. The particularity of this kind of method appears in the simplicity of the formalism the control laws depend only on the desired and current situations of the observed object. Then, from one application to another, only the pose algorithm has to be modi ed. For the future, we w ant to compare the two 3 D approaches developed in this paper.


D Khadraoui

C Debain

R R O U V Eure

P Martinet

P Bonton

J Gallice

Vision Based Control in Driving Assistance of Agricultural Vehicles

Keywords: Agricultural mobile vehicle, Vision-based linear control, Interaction matrix, Neural network, Camera kinematic modelling Principle of neural network training : : Creation of training data 20 Results with image processing algorithms

This article presents a real-time control system for an agricultural mobile machine (vehicle) based on an on-board vision system using a single camera. This system has been designed in order to help human beings in repetitive and di cult tasks in the agricultural domain. The aim of the robotics application concerns the control of the vehicle with regard to the reap limit detected in image space. The perception aspect in relation with the application has been described in previous work, and here we deal with the control aspect. We i n tegrate image features issued from the modelling of the scene in the control loop, in order to perform image-based servoing technique. The vehicle behaviour described here concerns bicycle and neural models, and three control laws are then synthesized. The rst and the second are modelling approaches and use an interaction between the scene and the image space. They are based on regulation of a task function. The third is a \black-box modelling" technique and is based on a neural network. Finally, experimental results obtained with these di erent c o n trol laws in di erent conditions are presented and discussed.

Introduction

The domain of agricultural work is in general extremely complex. Guiding an agricultural machine has more in common with driving a cross-country vehicle than with steering a high speed vehicle along a motorway, notably because of the nature of agricultural terrain. If we w i s h t o cover all agricultural situations which are today managed by h uman pilots, the guidance system must be particularly robust in order to adapt to the variety of natural situations encountered. Indeed, the extreme variety of the disturbances which could modify the behaviour of the vehicle is unknown in advance, and makes di cult and delicate the development and synthesis of servoing systems. Among these disturbances, we m a y note the problems of interaction between the wheels and the soil (sliding with wheels locked, wheel-spin and skidding), the slopes, and the intrinsic features of the vehicle. The appearance and disappearance of these disturbances are not foreseeable and they could evolve in time and depend on modi cations of the climatic conditions and the vehicle mass.

Today, the driving of mobile agricultural vehicles is in general entirely done by man. Steering control requires precision and dexterity and can require the driver's full attention in complex situations. It is tiring for the driver to keep focused on this single task, causing losses in cost and work quality. Automatic control of speed and steering allows the human, present o n t h e vehicle, to optimize use of the machine and the current agricultural task. His attention for driving is only necessary in exceptional situations. The development of a guidance system in the agricultural domain can also be conditioned by other criteria such as problems of cost and ease of implementation and utilization. The technological choices must therefore remain realistic in order to interest a manufacturer of agricultural machinery. F urthermore, the harsh conditions of agricultural work render this domain particularly demanding with regard to the equipment.

A signi cant amount of research w ork has been done in the domain of agricultural robotics. This subject covers a wide range of potential applications, from the inspection and/or picking of fruit or vegetables to the concept of an autonomous robot working in the agricultural environment. Computer vision can therefore be of signi cant v alue in order to automatically (or semi-automatically) drive a robotic apparatus which aims to replace human beings in repetitive or hard tasks in a natural environment [START_REF] Jarvis | Omniescient camera based outdoor mobile robot[END_REF][START_REF] Sandini | Visually guided operations in green houses[END_REF][START_REF] Derras | Vision guided mower for the upkeep of natural environment[END_REF]Amat et al. 1993Casals et al. 1993). Today, techniques of visual servoing are used to control robot manipulators (Chaumette 1990[START_REF] Espiau | A new approach to visual servoing in robotics[END_REF][START_REF] Khadraoui | Visual servoing in robotics scheme using a camera/laser-stripe sensor[END_REF] but there are still few applications in mobile robotics [START_REF] Pissard-Gibollet | Asservissement Visuel Appliqu e a u n Robot Mobile: etat de l'art et mod elisation cin ematique[END_REF][START_REF] Jurie | High-speed vehicle guidance based on vision[END_REF][START_REF] Khadraoui | Linear control of high speed vehicle in image space[END_REF]. For a mobile robot, the main problem in using these techniques is due to the presence of non-holonomic mechanical connections which limit robot movements. In this context, traditional visual control laws are in general synthesized by separating the vision aspect from the control module. Classical methods, which are implemented for agricultural vehicle guidance are based on vision in 3D space [START_REF] Derras | Segmentation texturale originale appliqu ee au guidage visuel d'un robot d'entretien d'espaces naturels[END_REF][START_REF] Debain | Development o f a visual guidance system for an upkeep robot of natural environments[END_REF][START_REF] Klassen | Guidance systems for agricultural vehicles[END_REF]. But with the vision-based control technique, the philosophy consists in reaching a particular con guration in the 2D image plane and not a situation between camera and object. So this approach has the advantage of avoiding the intermediary step of 3D estimation of the environment with regard to the robot. Problems related to the reconstruction of the 3D world are then eliminated and no explicit calibration of the camera is needed.

The studies presented in this paper were carried out on a mobile agricultural vehicle which has the steering wheels placed at the rear.

We present three methods of vision-based control integrating visual features directly into the control loop like represented in gure 1 : s is considered as a reference target image to be reached in the image frame, s is the value of the visual information currently observed by the camera and computed by image processing, G is the vector control gain to be synthesized by the task function approach or the neural approach, i is the control variable of the vehicle representing the steering angle of the machine, o is the set of outputs characterizing the machine's position and orientation. Two of the control laws are modelling approaches and based on a task function approach. The third is a \black-box modelling" approach and is based on neural networks. The main point of this paper is the presentation of experimental results using three di erent techniques of control.

Task Function Approach

We adopt the task function approach d e v eloped by [START_REF] Samson | Robot Control. The task function approach[END_REF][START_REF] Espiau | A new approach to visual servoing in robotics[END_REF]. This function can be thought of as representing a virtual kinematic constraint between the camera and the target. In this case, we explicitly use modellings of the scene and the vehicle in order to synthesize the control laws.

Modelling of the Scene

The modelling of the scene consists of the development o f a n i n teraction matrix which enables the system to relate the visual information to the di erent control inputs (Chaumette 1990). We d e v elop the problem considering that the camera used is modelled by the classical pinhole approximation.

Knowing the camera velocity screw T c de ned by three translational and three rotational velocities V = ( V x V y V z ) and = ( x y z ), we can relate it by means of :

_ s = L T s T c (1)
where L T s is the interaction matrix related to situation s (Chaumette 1990[START_REF] Hager | Visual Servo Control[END_REF].

The expression of L T s depends on the nature of the visual information contained in s and is obtained by u s i n g t h e w ell known equation of optical ow measurement t o 3 D structure and motion in the scene (Paul 1982[START_REF] Faugeras | Three Dimensionnal Computer Vision : A Geometric Viewpoint[END_REF] .

The scene is represented by a straight line for which w e nd an equation in the image frame of the camera. We express the position of the machine and its orientation according to the ( ) parameters of the line measured and we de ne the set of visual features : s = ( ) T (2)

The problem then consists in de ning the interaction matrix related to the desired position s = s . This represents the relation between the variation of the visual features and the velocity screw which de nes the set of machine displacements at the equilibrium position. From (1), we have :

_ s = L T js=s T c
(3) The equation of the plane containing the desired line and according to the camera frame is expressed as : y cos + z sin + h = 0 (4) where represents the angle of inclination of the camera (see Figure 2 Considering expression (1) and taking the desired features, such a s = and = , t h e n the interaction matrix at the equilibrium situation is given by : L

Vehicle Modelling

We use a machine which h a s t wo steerable wheels and two driven wheels (like a car). At a constant steering angle, this machine describes a circle. In order to model the displacement o f the machine, we consider a bicycle model which assumes the following : there are no exible parts, the vehicle moves on plane surface, there is no translational slip between the wheels and the surface, there is su cient rotational friction between the wheels and the surface.

We establish the general equations relative t o i t s b e h a viour taking into account its kinematic characteristics. It is useful to approximate the kinematics of the steering mechanism by assuming that the two rear wheels turn slightly di erentially. Then, the instantaneous centre of rotation can be determined purely by kinematic means. This amounts to assuming that the steering mechanism is the same as that of a bicycle. Let the angular velocity v ector directed along the y axis be _ and let the linear velocity directed along x axis be _

x. Using the bicycle model approximation (see Figure 3(a)), the steering angle and the radius of curvature r are related to the wheelbase L, as in (Kelly 1994) by: tan = ; L In Figure 3(b) we show a small portion of a circle S representing the trajectory to be followed by the machine. We assume that it moves with small displacements between an initial curvilinear abscissa S 0 and a nal one named S f such that: 8 > < > :

1 r = l i m s!0 S = d dS = _ _ S _ S 2 = _ x 2 + _ z 2 (7)
In fact, the rotational velocity is obtained as:

_ = ; tan L p _ x 2 + _ z 2 (8)
The lateral position denoted by x can be computed by assuming that the machine moves with small displacements. In the case of longitudinal motion along the z axis during a time interval t, the machine moves through distance z (see Figure 3). We express: 8 < : z = r sin x = r(1 ; cos )

Eliminating r from equations (9), we obtain:

x = z 1 ; cos sin (10) Without loss of generality w e can consider that the initial conditions are null since the frame is linked at the position S 0 and then x = x, z = z and = . W e compute the derivative over time of the lateral coordinate x of the machine given by (10) which depends on z and , as follows:

_ x = 1 ; cos sin _ z + z _ sin ] (11)
The approximation to small angles ( and are less than 7 o ) i s v alid in the case of our application and enables us to simplify equations ( 8) and ( 11). This gives the relation between the di erential of the lateral coordinate x and the lateral deviation with the steering angle by expressing the development of trigonometric equations to the second order. If we consider that the machine moves with constant longitudinal speed _ z = V and that _ x < < V , w e can write:

_ x = 2 ( _ z + z _ ) (12)
Taking into account the approximations below, we h a ve _ _ z r and then z r since the r is constant and the initial conditions are null (frame xed at initial position of the robot). We nally nd the kinematic model of the machine expressed by the following equations which are similar to those obtained by another method [START_REF] Khadraoui | Linear control of high speed vehicle in image space[END_REF]: 8 < :

_ = ; V L _ x = V (13)
However, we only control the wheel angle . W e can nd an equation linking to the vector (V _ ). In our application we consider that the velocity V is constant, when we h a ve (Ioos et al. 1988 Murray and[START_REF] Murray | Nonholonomic motion planning: steering using sinusoids[END_REF]

: = ; L V _ (14)

First Law : State Space Design

We treat here a single input linear system with output parameters ( ) (see Figure 1). To control such a model, a technique of pole assignment i s u s e d .

Here, we elaborate a state model which i n tegrates both the model of the machine and that of the scene [START_REF] Khadraoui | La Commande R ef erenc ee Vision pour le Guidage Automatique de V ehicules[END_REF]. In the context of our application, we assimilate the machine to a particular mobile robot which m o ves with limited degrees of freedom. It has non-holonomic constraints since the number of degrees of freedom of control, in our case, is less than the number of degrees of freedom of displacement : translation along x and z axes and rotation around the y axis. In the case where the desired situation is represented by a straight line centered in the image ( = = 0) and by considering equation (3), we can express the kinematic screw T c as :

T c = L T+ js=s _ s (15) Determination of the kernel of the interaction matrix (matrix of rank 2), permits us to constrain the lateral translation of the vehicle and its orientation by i n troducing a rank 2 matrix such a s :

T c = 2 4 1 0 0 0 0 0 0 0 0 0 1 0 3 5 L T+ js=s _ s (16)
Consequently, the matrix L T+ js=s is limited to the components that correspond to the lateral and the orientation movements. In fact, the interaction matrix is reduced to:

_ s = 2 4 l 11 l 12 l 21 l 22 3 5 0 @ V x y 1 A (17) with 8 > > > > > < > > > > > : l 11 = cos l 12 = ; sin l 21 = cos l 22 = ;(1 + 2 ) c o s
We remark that the velocities V x and y correspond to those expressed in (13) for the machine. We write: 8

< : V x = _ x = V y = _ = ; V L (18) and 0 @ _ x _ 1 A = 2 4 l 11 l 12 l 21 l 22 3 5 ;1 0 @ _ _ 1 A (19)
By integrating ( 19) over time we h a ve: 0

@ x 1 A = 1 l 2 4 l 22 ;l 12 ;l 21 l 11 3 5 0 @ 1 A + 2 4 k 1 k 2 3 5 ( 20
)
where k 1 and k 2 are the constants of integration, assuming that the matrix is rank 2. Using ( 18) and ( 20), we can easily express the velocities V x and y : 8 < :

V x = V l (;l 21 + l 11 ) + k 2 V y = ; V L (21)
with: l = l 11 l 22 ; l 21 l 12

By considering equations ( 17) and ( 21), we are led to the linear 2D model used for the steering control system. We i n troduce in the control loop the visual features and of the line. After development, the state representation of the system with initial conditions null (k 2 = 0 i f = 0 and = 0 ) i s g i v en by : 0 The continuous-time state-space form of the model ( 17) becomes:

@ _ _ 1 A = V l
_ s = A s + B ( 23
)
where:

s is the visual information vector to be computed at each iteration by image processing, is the control variable to be injected into the system at each step of the servoing task,

A and B are constant matrices.

The control law i s s y n thesized using a pole assignment technique by assimilating the behaviour of the system to a second order system having as a damping ratio and ! 0 as its frequency. T h e parameters and ! 0 can be chosen by using an LQ approach, when the control law = ;G s enables the poles of the closed-loop dynamics matrix to be placed at the desired locations. Finally, the gain matrix is expressed as follows:

G = ( g 1 g 2 ) (24)
with : 8 > > < > > :

g 1 = L ! 0 V 2 l (2 V l 21 ; ! 0 l 22 ) g 2 = ; L ! 0 V 2 l (2 V l 11 ; ! 0 l 12 ) (25)
2.4 Second Law : Regulation of two T ask Functions

In the modelling of the scene, we considered a visual primitive g i v en by t h e v ector s. Then we can de ne a task function e such a s :

e = C(s ; s ) (26)
e is a six dimensional vector which corresponds to the six degrees of freedom, s ; s is a vector of two components corresponding to the visual informations, and C is a matrix of six rows and two columns which i s t h e c o m bination matrix. The visual information used in our application is as shown in Figure 2 We can write: 26), we h a ve : e = C 1 ( ; ) + C 2 ( ; ) = e + e (28)

C = ( C 1 C 2 ) (27) From equation (
Our application focuses on servoing the robot with respect to a straight line, and for this purpose we consider that the line does not move with respect to the camera. This symplifying hypothesis is not satis ed for every trajectory. Nevertheless, we consider that it is realistic for a straight trajectory or for a piecewise-linear trajectory.

Adopting standard practice in vision-based control, we take the kinematic screw, representing the camera movements, as a control law [START_REF] Samson | Robot Control. The task function approach[END_REF][START_REF] Espiau | A new approach to visual servoing in robotics[END_REF]). In the case where the object stays xed on the camera frame, a possible control law i s :

T c = ; e (29)
We c hoose a similar control law g i v en by [START_REF] Debain | Lois de Commande pour le Contrôle et la Mobilit e de Machines Agricoles[END_REF] This amounts to introducing a supplementary gain on one part of the error, the one concerning the heading error. In fact, the di erent t ypes of drive of a mobile robot mean that we can give di erent degrees of importance to the lateral error and the heading error.

A precise but abrupt drive requires a lot of energy, f a vouring lateral error, whereas a smooth, non-precise drive emphasizes heading error. Consequently, w e rst impose an exponential decay on e and another on e by writing: _ e = ; ( e + e ) (31) The task function e is then based on an exponential function together with its decay w h i c h will be taken to be equal to the smaller of the two terms and . This behaviour imposes a second-order system with the response time and the overshoot conditioned by the and values.

By using the classical methods of the form of vision-based control, we can calculate the control matrix. We di erentiate the equation ( 28) as follows :

_ e = C _ s = C 1 _ + C 2 _ (32)
where and are scalar and are in R + and with (31), we obtain :

_ e = ; C 0 @ 0 0 1 1 A 0 @ ; ; 1 A (33)
Finally, ( 32) and ( 33) give :

C _ s = ; CB(s ; s ) (34) with B = 0 @ 0 0 1 1 A
As is given in (Chaumette et al. 1991), a possible value of C in order to satisfy the conditions of convergence near the desired position is

C = L T+ (s=s ) (35)
Here, L T+ is the pseudo-inverse of the interaction screw calculated for the equilibrium position s = s .

L T+

(s=s ) L T (s=s ) = I 6 (36) Near the the equilibrium position, the modelling of the scene gives us the following : _ s = L T (s=s ) T c (37) T c is the kinematic screw represented by six degrees of freedom of the camera. We can deduce from equations (34), ( 35) and (37) :

T c = ; L T+ (s=s ) B(s ; s ) (38)
T c gives us a velocity c o n trol of our camera and having the expression of y we deduce the value of the control input of our system expressed in the modelling of the vehicle (eqn. 14). It is given by : = ; L V h sin cos 1+h 2 ( ; ) + h 2 +cos 2 1+h 2 ( ; ) i (39)

3 Neural Approach

Neural Model

Using the lateral i and heading i errors de ned in the image reference at the equilibrium situation, the steering controller has to compute steering commands to obtain the minimum di erence between the vehicle and the line followed. We c hose a multilayer neural network because it has been used successfully in multiple areas such as classi cation, modelling and automation (LeCun 1987). The learning algorithm is the classical gradient method which aims at reducing the following error function [START_REF] Rumelhart | Parallel and Distributed P r ocessing[END_REF] Abbassi and Sayed 1991):

E rr = n X k=1 E k (40) with E k = P m 1 ( d k i ; r k i ) 2
, n the number of example, m the number of ouputs, d k i the expected output and r k i the real output. In our application, for an input ( i i ) the associated steering angle d i is unknown. This is known as the credit assignment problem. One way t o a void it is to perform neural training looking at the behaviour of an existing controller. In our application, driver behaviour is used to build a steering controller able to react as e ectively as the driver. The driver enables the system to know the correct output d i associated with an input ( i i ). The expected output d i is equal to the steering angle. In fact, at time (t = t i ), the input ( i i ) is computed and the steering angle d i is read. The set ( i i d i ) is then used by the back-propagation algorithm in order to modify the synaptic coe cients and minimize the overall error. In Figure 4, the driver is used as an ideal steering controller to perform neural network training.

The sets of examples ( i i d i ) are available at a regular rate. With the sampled data, the back-propagation algorithm used in the gradient method can lead to local minima (McInerney et al. 1989Dayho 1990). In order to avoid them, the learning step is divided into two parts (Figure 5 Step 1 is represented in Figure 5, where sets of data ( i i d i ) are stored in a database. In step 2, the learning set created in step 1 is used to perform the o -line neural network training. As there is no real time training, this method is called o -line neural network learning. The neural network can then be introduced into the control loop.

Choice of Neural Network

We opted for a multi-level gradient b a c k-propagation network. The implementation of this type of network with regular structure is relatively straightforward. We also notice that unlike classi cation applications, which generally require large networks, our application needs about ten neurons.

The choice of the size of network (number of levels and number of neurons per level) is still a di cult issue, since for a given problem we do not know the optimal network, just as for a given network we do not know the optimal number of neurons per level. The risk is double edged: a smaller number of neurons leads to insu cient modelling. too many neurons leads to excessive modelling which can reduce the performance of the network.

The nal choice of network in our guidance application was found experimentally. W e settled on a network with two inputs (heading and lateral position of the line detected in the image), one output (the steering angle) and two hidden levels. The rst has two neurons and the second has ve neurons (see gure 6).

Training the Network

The network learns from examples constituted by t h e h uman supervisor model. If the examples come from a domain of reduced functioning of the system, the network will not learn how t o respond correctly outside this domain. In the same way, if the supervisor provides erroneous information, the network will learn an erroneous model. In order to create the data from training, a scene composed of two straight lines and one curve is used. The radius of curvature r of this track to follow b y t h e v ehicle, de nes the maximum domain of controller functioning (see Figure 7). At the end of the training phase, the neural network can be introduced into the steering control loop. Apart from the problems related to the sensor, the performance obtained is similar to that achieved by a h uman pilot.

Experimental Results

Implementation Considerations

In order to test our three control laws, we laid out a trajectory composed of a bend, a step of one meter and a straight line as shown in Figure 9(a). A c a m e r a w as placed on the left side of the cabin (Figure 9(b)). This supplies an image of the white line whose parameters are transmitted to the control algorithm by the image processing task. These parameters are the distance (gap) and the angle (heading) of the line detected (Figure 2(b)).

In order to measure the quality of our control laws, we positioned a second camera in front of the vehicle so that the camera optical axis was perpendicular to the ground (Figure 9 This camera continuously measures the distance between the white line and the machine. A second image processing algorithm is used to enable us to reconstruct the trajectory of the machine. It is calibrated to be able to measure the performance of the guidance system with an accuracy of 5 cm.

For each c o n trol law, we carried out the same tests under the same conditions. The rst test consisted of following the trajectory at several speeds. We then added noise to the visual measurements. The noise corresponds approximately to what can be observed in a natural environment ( 16 pixels in ). Most often, it is induced by the terrain which is not perfectly at like bitumen, or else by the trajectory to be followed. The test is composed of four parts:

beginning of bend, following the bend (curve with constant turning radius), leaving the bend and response to a step of one meter, following the straight line.

The servoing task to be accomplished is : rst follow a bend with curvature r, then determine the response to a step of 1 meter distance between the two lines, and nally follow the line with good precision. The target is represented by the straight line centered in the image plane and characterized by = = 0 . We e m p l o yed an architecture composed of two VME microprocessor boards (Figure 10). all the cooccurent tasks which constitute the parallelized algorithm. To manage all the problems arising from the resources shared between the di erent processors, we c hose the OS9 operating system. From the di erent m e c hanisms for synchronization and communication between tasks o ered by this operating system, we made particular use of signals, communication pipes and data modules. Knowledge of a priori information and the parallelized algorithm enables us to achieve real time operation. The image processing results are sent to the personal computer which controls the steering angle. The range period of the image processing calculation time is about 200 ms, which is close to that encountered in a natural environment.

The experimental results obtained by the three control laws under the di erent conditions will be presented in the following sections.

Results using the rst Law (state space design)

In Figure 11 we present results with parameters adapted as a result of simulations in order to obtain good response to a second order system. We adjust the two parameters and ! 0 with regard to the average speed of the machine which is xed at 4 km/h . The results concern the input and the lateral position x of the machine obtained in real experimentation, and the features errors and used as visual informations. The experimental lateral position was reconstructed by using a second camera perpendicular to the scene taking into account t h e camera calibration. We note that the machine cut across the bend because the camera sees the scene 6.5 m in front of the machine. ! 0 (rd=s) V (km=h) 0.14 0.9 4 In Figure 12, we tested the robustness of the control law with regard to noise introduced in the measurement parameters and . W e remark that the task is achieved with good stability and robustness.

Finally, w e tested the robustness of the approach t o v ariation in speed. The control is synthesized at a xed velocity of 4 km/h and the machine is moved at di erent v elocities (4, 6, 8, 10 and 13 km/h). Figure 13 shows that the vehicle reaches the line and continues following it, despite the variation of its speed. But, we remark that increasing speed causes the controlled behaviour to oscillate at steady-state. This is usually an indication that the controller gains are too high.

Results using the second Law (regulation of two task functions)

The results shown in Figure 14 are obtained at a speed of 4 km/h. V (km=h) 0.3 2.6 4 The response time is about 20 s with no overshooting. There are some perturbations on the curves because the camera which measures the distance between the vehicle and the white line 20 vibrates considerably when the machine runs. The same image processing algorithm is used to control the machine and to measure the error. We will see that noise has no consequence on our system. Figure 15 presents results obtained in the same conditions, but introducing noise into the image processing ( gure 15). The same remark as for the rst law can be made in this case about bend-following performance.

It seems that there is no di erence between the two cases (with or without noise). In fact, the noise of the algorithm which is used to control the quatity of our system has more consequences than the e ect of noise introduced in the result of the image processing.

The test representation in Figure 16 was carried out at di erent speeds and without noise. Here, the response of the machine is reduced from the rst case (test at 4 km/h with adapted parameters). In fact, the machine has the same trajectory but needs less time to do it. The curves show that speed causes the same behaviour as in the rst case, since the gains become too high when the speed increases.

Results using the third Law (neural approach)

In this case, the training phase was carried out at a velocity of 4 km on a training track di erent from the test track (in particular, the radius of the bend is large). Figure 17 shows the results obtained respecting the conditions of training with regard to velocity. The machine follows the track with good stability. For the same longitudinal velocity of 4 km/h, random noise was added to the gap . The addition of this noise has no e ect on the performance of the vehicle in following the line, although the deterioration in the visual signals is clearly visible in the curves (see Figure 18). One possible interpretation of this phenomenon is that a high degree of precision is not required in the orders applied to the steering (wheel angle), because the machine itself constitutes a natural lter for the turning orders.

Three speed values were also tested (4 km/h, 6 km/h and 8 km/h) and gave us a good result. We did not nd a notable di erence between curves shown in Figure 19. We only note a small performance degradation at the beginning of the bend, when the speed increases. We observe a n o vershoot of 18 cm at 4 km/h, 25 cm at 6 km/h and 39 cm at 8 km/h. We h a ve presented a system that provides the driving assistance to an agricultural mobile machine in order to help human beings in repetitive and di cult tasks in a natural environment.

The system is based on a vision system using a single camera [START_REF] Derras | Segmentation non supervis ee d'images textur ees par champs de Markov : Application a l'automatisation de l'entretien des espaces naturels[END_REF]. Three vision-based control laws were tested successfully. F or each l a w, the control is directly speci ed in terms of regulation in the image space. It is noticeable that this approach has the advantage of avoiding the intermediate step of 3D estimation of the environment with regard to the machine. This enables us to introduce into the control loop features directly measured in the same space, which prevents us from making poor estimations of 3D measurements in the case of three dimensional control. In the regulation task function approach (the rst and second laws), we need to construct an interaction matrix related to the scene. We a l s o n e e d t o k n o w the vehicle model. The neural approach is a \black-box modelling" method and it avoids speci c modelling tasks such as explicit camera calibration and vehicle modelling.

With the rst law, the gains of the control law developed are adaptive with regard to the desired situation to be reached in the image space and to machine speed. We h a ve t o x t h e behaviour of the vehicle to a second order system characterized with known parameters (! 0 and ). So we do not need a phase of empirical gain research. We m a y note that the machine is modelled by a kinematic equation, and we t h e n s h o w that it is amply su cient for this kind of application. All the results obtained show good convergence and the robustness of our algorithm. They are good enough to demonstrate the feasibility o f s u c h an approach which can be extended for any other steerable, mobile, wheeled robot.

With the second law, we show that our theory can be used for many m a c hines. Our gains and are chosen by means of a desired response curve adapted to the machine. All our results show the robustness of our control algorithm. However, the regulation is done in movement and we do not solve the problem of controlling a non-holonomic machine with a nal speed equal to zero.

One advantage of the rst and the second control laws is due to their adaptability with regard to longitudinal velocity. But their disadvantage is that they do not react correctly in the case of a sharp bend. The principle reason is that the camera sees the scene approximately 6.5 m in front of the machine, and then anticipates its actions. Another reason is that we d i d not model the bend. In fact the rst and the second control expressions are the same, the only di erence being the method for selecting the gains. The choice of the parameters, ( , ! 0 ) f o r the rst, and ( , ) in the second permits to modify the pole location of the system's closed loop.

The third law gives correct results on driving track. The automatic guidance system rivals a h uman driver in performance. During the tests above 8 km/h, the neural network does not react su ciently quickly in the beginning of the bend, and then we h a ve a systematic check with loss of detection of the line. This is due rst to the modi cation of the dynamic characteristics of the vehicle with regard to the velocity used in the training phase. So the model built up by the neural network does not correspond su ciently to reality for higher speeds. Secondly, the system can not take su cient account of the heading information at the beginning of the bend. In fact, the beginning of the bend is translated rst by the modi cation of the gap of the line detected and then by v ariation of the heading. At this level, the best results can be obtained by c o n trolling the contents of the training data, or taking into account the velocity parameter V in the neural network, but this complicates the data training.

Working in an agricultural domain, with industrial partners, we h a ve often adopted a pragmatic approach. Our methods need not be highly sophisticated, but should be easy to implement on di erent v ehicles. We improve algorithms, and adapt them to each application, in order to obtain fast response and stability, with robustness with regard to speed and curve c hanges. With our present know-how, we can apply driving assistance systems to di erent situations and di erent v ehicles. To do that, we are developping robust image segmentation algorithms [START_REF] Derras | Segmentation non supervis ee d'images textur ees par champs de Markov : Application a l'automatisation de l'entretien des espaces naturels[END_REF][START_REF] Derras | Image processing and algorithms merging : Real time control of an upkeep machine for natural spaces[END_REF]. For instance, in agricultural applications like harvesting we use this kind of approach to extract the reap limit between harvested and unharvested areas modelled as a line whose parameters are used by the control laws previouly described. Figure 20 gives examples of reap limit detection for an agricultural application.

Figure 1 :

 1 Figure 1: Parts of the servoing chain

  (a)).

Figure 2 :

 2 Figure 2: 3D and 2D camera frames

  Figure 3: Vehicle kinematic modelling

  (b): represents the angle of the line detected in the image, represents the angle of the reference line, represents the distance with respect to O of the line detected in the image, represents the distance with respect to O of the line reference.

  ): T c = ; ( e + e ) (30) and two non-zero positive real values.

  Figure 4: Principle of neural network training

Figure 5 :

 5 Figure 5: Data storage (step 1) and learning set creation (step 2)

Figure 6 :

 6 Figure 6: Choice of neural network architecture

Figure 7 :

 7 Figure 7: Creation of training data

  (b)).

  Figure 9: Test procedure

  Figure 10: Parallel architecture

Figure 11 :

 11 Figure 11: First law: test at 4 km/h with adapted parameters and ! 0

Figure 12 :Figure 14 :

 1214 Figure 12: First law : test at 4 km/h with 16 pixels of noise in

Figure 15 :Figure 18 :

 1518 Figure 15: Second law : test at 4 km/h with 16 pixels of noise in

Figure 19 :

 19 Figure 19: Third law : test at di erent v elocities

Figure 20 :

 20 Figure 20: Results with image processing algorithms