David Dederscheck

Holger Friedrich

Christine Lenhart

Joachim Penc

Eduard Rosert

Maximilian Scherer

Rudolf Mester

Rahul Swaminathan

Vincenzo Caglioti

Running on Optical Rails: Theory, Implementation and Testing of Omnidirectional View-based Point-To-Point Navigation

published or not. The documents may come   L'archive ouverte pluridisciplinaire

Introduction

In this work, we present a method for guiding a mobile robot through a known environment, only relying on omnidirectional vision, and without use of any geometric information. Animals as well as humans use visual perception to a very large extent to orient themselves in their surroundings. When using visual information for guidance and navigation, most processing is related to recognition, association, and observing changes of similarity; it seems less plausible that geometry, e.g. triangulation, plays an explicit role in finding and following a known path. Our method is designed according to these principles. It has been very recently introduced in [START_REF] Friedrich | Optical rails. view-based point-to-point navigation using spherical harmonics[END_REF] as Optical Rails and tested using simulated data. In this article, we show the extension of our approach for successful implementation on a real robot vehicle. The method shares certain characteristics with landmark-based robot homing approaches [START_REF] Goedeme | Omnidirectional vision based topological navigation[END_REF][START_REF] Möller | Visual homing in environments with anisotropic landmark distribution[END_REF][START_REF] Argyros | Robot homing by exploiting panoramic vision[END_REF][START_REF] Franz | Where did I take that snapshot?[END_REF], view based homing [START_REF] Stürzl | Efficient visual homing based on Fourier transformed panoramic images[END_REF], and with those methods that use epipolar geometry [START_REF] Basri | Visual homing: Surfing on the epipoles[END_REF][START_REF] Makadia | Radon-based structure from motion without correspondences[END_REF]. It does, however, not require the use of artificial markers, laser scanners, SLAM, etc. In terms in alphabetical order of utilizing spherical harmonics, it is founded upon previous work of Makadia and Daniilidis [START_REF] Makadia | Radon-based structure from motion without correspondences[END_REF][START_REF] Makadia | Planar ego-motion without correspondences[END_REF]; the expansion into orthonormal basis functions for image signal representation leads to the essential innovation in Optical Rails: efficient differential pose change estimation and differential track following.

The representation of images in spherical harmonics (SH) is the key to the particular efficiency of Optical Rails: To control the motion of the robot, the gradient of a dissimilarity measure with respect to the motion parameters of the robot (translation, rotation) is obtained by comparing omnidirectional views. The computation of derivatives in image space is avoided by expanding the spherical image signals in basis functions; all further processing occurs only using the resulting low-dimensional coefficient vectors. Thus, with Optical Rails, pose derivatives can be obtained in a computationally inexpensive way using precomputed expressions. Another positive aspect is that by using a truncated SH expansion Optical Rails works well even without high resolution images or precision optics.

Optical Rails Concept

Imagine the desired course of the robot as a sequence of discrete waypoints, i.e. subsequent locations in the environment. Each of these waypoints is associated with an individual omnidirectional view. The position of the waypoints in space is neither determined nor necessary for steering the robot. Merely by comparing the view at its current location with the target view, the robot is able to drive towards the target waypoint. As soon as it gets close enough to its current target, the robot proceeds with the next waypoint in the sequence (waypoint handover ). In this way, the robot visits one waypoint after the other and thus follows the prerecorded path. Multiple paths can be interconnected and extended to a network in which the robot can move freely.

In our approach, all views are represented as an expansion into spherical harmonics (SH), analogously to the Fourier representation in the plane [START_REF] Makadia | Radon-based structure from motion without correspondences[END_REF][START_REF] Friedrich | Optical rails. view-based point-to-point navigation using spherical harmonics[END_REF]. The sets of coefficients a m of this expansion form view descriptors a. The dissimilarity Q between the current view and the target view can be computed from the difference of the two corresponding view descriptors.

Svoboda and Pajdla [START_REF] Svoboda | Motion estimation using central panoramic cameras[END_REF] perform motion estimation on panoramic images using correspondences of randomly scattered points. In our case, the differential change of the dissimilarity measure Q -for infinitesimal pose changes of the robot -corresponds to the desired robot motion to decrease dissimilarity. For the task of robot navigation, we may then assume that minimizing dissimilarity also reduces the distance between the positions. This incremental differential process is analogous to classical differential matching processes (a.k.a. 'differential image registration') as used by Lucas and Kanade [START_REF] Lucas | An iterative image registration technique with an application to stereo vision[END_REF].

For computing the differential change of the dissimilarity with respect to pose changes, we require a model for the effect of a pose change on the spherical image signal. An exact prediction can be made only for the trivial case of a pure rotation; for exactly predicting the effects of a 2D translation, we would require the depth structure of the scene. The changes w.r.t. translation and rotation are modeled by reprojecting the spherical signal on a virtual ceiling plane parallel to the ground plane (gnomonic projection) on which the robot moves.

A Model for Differential Pose Change Estimation

A natural choice for the dissimilarity measure Q between two views is the mean squared image signal difference obtained by integration across the hemisphere. Let s(θ, φ) be the view at the current -yet unknown -robot position p c , and a the associated view descriptor; s(θ, φ) is the view at the destination pose p d with the corresponding view descriptor ã. Due to the fact that the basis functions used here are orthonormal, the dissimilarity Q between the two views s(θ, φ) and s(θ, φ) can be directly computed from their view descriptors a and ã:

Q := π/2 θ=0 2π φ=0 (s(θ, φ) -s(θ, φ)) 2 • sin θ dφ dθ = ||a -ã|| 2 2 (1)
where sin θ is the Jacobian of integration in spherical coordinates θ, φ.

As we wish to minimize Q, we require the gradient -g = -∂Q/∂p c which represents the direction in which the robot should move (including rotation) to reduce the dissimilarity Q. Because s(θ, φ) is dependent on the current robot pose p c , but the exact relation is inaccessible due to the unknown depth structure, we need a model that approximates the effects of a pose change.

To that purpose, we project the spherical image signal onto a ceiling plane. Then differential translations and rotations of the projected signal correspond to motions of the robot underneath the ceiling plane. This way we coarsely approximate the underlying geometry to benefit from the now analytically determinable effects of the motion.

Using this model, we compute the partial derivatives of Q with respect to the components of the current pose vector p c and steer the robot in the direction of the destination.

Gnomonic Projection of a Hemispherical Signal. We project an image signal s(θ, φ) defined on a hemisphere onto a ceiling plane by using the gnomonic projection. Thus we obtain spherical coordinates from Cartesian coordinates on the projection plane by

G (x, y) = (θ(x, y); φ(x, y)) T θ(x, y) = arctan(1, x 2 + y 2 ) φ(x, y) = arctan(x, y) (2)
The inverse mapping from spherical coordinates to Cartesian coordinates on the projection plane is obtained by

G -1 (θ, φ) = (x(θ, φ); y(θ, φ)) T x(θ, φ) = tan(θ) • cos(φ) y(θ, φ) = tan(θ) • sin(φ) (3)
Integral of a Planar Projection of a Hemispherical Function. Since we aim to compute the derivative of Q in coordinates of the projection plane, we require the planar equivalent of our dissimilarity measure [START_REF] Argyros | Robot homing by exploiting panoramic vision[END_REF]. Thus we substitute θ, φ, dφ, dθ in (1) according to (2) and adjust the integration boundaries appropriately. The combined Jacobian w(x, y) for this transform is

w(x, y) = det |J| • sin(arctan(1, x 2 + y 2 )) = (1 + x 2 + y 2 ) -3 2 (4)
where det |J| is the Jacobian for the gnomonic transform G (x, y). We obtain the planar equivalent of Q for the projection:

Q = ∞ -∞ ∞ -∞ w(x, y) • [s(θ(x, y), φ(x, y)) -s(θ(x, y), φ(x, y))] 2 dx dy (5)
Differential Pose Change Estimation for 3 DoF Motion. Let b(x) and b(x) be the planar image signals acquired at the current pose p c and the destination pose p d of the robot, respectively. We define f (x, d) as an isometric Euclidean transform which represents the pose translation and rotation with the displacement parameter vector d:

f (x, d) = A(d) • x, A(d) =   cos ϕ -sin ϕ v 1 sin ϕ cos ϕ v 2 0 0 1   , d =   v 1 v 2 ϕ   , x =   x 1 x 2 1   (6)
We now regard the dissimilarity Q between the view at the destination pose b(x) and the transformed view at the current pose of the robot b(f (x, d)), i.e. to the latter the Euclidean transform f has been applied:

Q = A w(x) • b(f (x, d)) -b(x) 2 dx (7)
The partial derivative of Q with respect to the parameter vector d at d = 0 is:

∂Q ∂d i = A 2 • w(x) • b(f (x, d)) -b(x) • ∂b(f (x, d)) ∂d i - ∂ b(x) ∂d i dx d=0 (8) As b(x) does not depend on d, ∂ b(x) ∂di is 0. The derivative ∂b(f (x,d))
∂di is obtained by applying the generalized chain rule:

∂b(f (x, d)) ∂d i = ∂f1(x,d) ∂di , ∂f2(x,d) ∂di , ∂f3(x,d) ∂di ∂b(y) ∂y1 , ∂b(y) ∂y2 , ∂b(y) ∂y3 T y=f (x,d) (9)
where f i (x, d) are three individual components of the vector function f (x, d).

We perform the differentiation and substitute the results in [START_REF] Makadia | Radon-based structure from motion without correspondences[END_REF].

With

ξ 1 := -x 1 • sin ϕ+x 2 • cos ϕ and ξ 2 := -x 1 • cos ϕ-x 2 • sin ϕ we obtain: ∂Q ∂d = 2 Z A w(x) h b(f (x, d))-b(x) i" ∂b(y) ∂y 1 , ∂b(y) ∂y 2 ,ξ1 ∂b(y) ∂y 1 +ξ2 ∂b(y) ∂y 2 " T ˛y=f(x,d) dx ˛d=0 (10)
For these derivative terms depending on the parameter vector, we may now perform the transition to d = 0. As f (x, 0) is the identity transform, we obtain:

∂Q ∂d = A 2•w(x)• b(x) -b(x) • ∂b(x) ∂x1 , ∂b(x) ∂x2 , x 2 ∂b(x) ∂x1 -x 1 ∂b(x) ∂x2 T dx (11)
which represents the gradient g for continuous planar image signals.

Exploiting the Spherical Harmonic Representation. We represent image signals by linear combination of basis functions, which allows for drastic simplifications in the computation of gradient g. Let Yj be the gnomonically projected basis functions on the ceiling plane, i. e.

b(x) = j a j • Yj (x), b(x) = j ãj • Yj (x) ( 12
)
where a j are the coefficients of the linear combination, corresponding to the j-th entry of the view descriptor a. Substituting an image signal b(x) by j a j • Yj (x) in [START_REF] Möller | Visual homing in environments with anisotropic landmark distribution[END_REF] and correspondingly for b(x), results in

∂Q ∂d = 2 Z A w(x) 0 B B B @ P j P k (aj -ãj) Yj(x)ak ∂ Yk (x) ∂x 1 P j P k (aj -ãj) Yj(x)ak ∂ Yk (x) ∂x 2 P j P k (aj -ãj) Yj(x)ak " x2 ∂ Yk (x) ∂x 1 -x1 ∂ Yk (x) ∂x 2 " 1 C C C A dx (13)
Using the linearity of the integral operator, we obtain

∂Q ∂d = 2 0 B B B @ P j P k (aj -ãj)ak R A w(x) Yj(x) ∂ Yk (x) ∂x 1 dx P j P k (aj -ãj)ak R A w(x) Yj(x) ∂ Yk (x) ∂x 2 dx P j P k (aj -ãj)ak R A w(x) Yj(x) " x2 ∂ Yk (x) ∂x 1 -x1 ∂ Yk (x) ∂x 2 " dx 1 C C C A (14)
Since the integrals in the sums do not depend on any of the coefficients a i , we may precompute them. We obtain the precomputed coefficients u i (j, k):

u 1 (j, k) = ∞ x1=-∞ ∞ x2=-∞ w(x) • Yj (x) ∂ Yk (x) ∂x1 dx 2 dx 1 (15) u 2 (j, k) = ∞ x1=-∞ ∞ x2=-∞ w(x) • Yj (x) ∂ Yk (x) ∂x2 dx 2 dx 1 (16) u 3 (j, k) = ∞ x1=-∞ ∞ x2=-∞ w(x) • Yj (x) x 2 ∂ Yk (x) ∂x1 -x 1 ∂ Yk (x) ∂x2 dx 2 dx 1 (17)
With these integrals, the components of the gradient are computed using the following expression:

g(a, ã) := ∂Q ∂d =   j k (a j -ãj ) • a k • u 1 (j, k) j k (a j -ãj ) • a k • u 2 (j, k) j k (a j -ãj ) • a k • u 3 (j, k)   (18)
Note that the negative gradient -g represents a pose change of the camera, and this vector has to be transformed into the robot coordinate frame in order to provide steering information.

Spherical Expansion on a Plane

In the experiments performed here, image acquisition takes place with a wide angle lens; we obtain planar fisheye images with a field of view of approximately 160 • . For calibration of the lens and modeling the projection from spherical coordinates to the planar image coordinates, we employ the unified projection model as described in [START_REF] Mei | Single view point omnidirectional camera calibration from planar grids[END_REF], which can be applied to all single-viewpoint omnidirectional cameras. To obtain the calibration parameters, we use the Matlab calibration toolbox supplied by [START_REF] Mei | Single view point omnidirectional camera calibration from planar grids[END_REF].

Instead of projecting the image onto the unit sphere, we perform the SHexpansion directly in the image coordinates. To that end, we project the spherical basis functions onto a plane corresponding to the imaging process of the fisheye camera in order to obtain projected basis functions. The computation of these basis functions is time-consuming, but this needs to be done only once.

Let b(x, ŷ) be the discrete image signal of the camera. Using the calibrated model of the camera and the lens, we obtain discrete basis functions Ŷ m (x, ŷ) defined in discrete image coordinates x, ŷ, which conform to the property

a m = x,ŷ Ŷ m (x, ŷ) • b(x, ŷ) (19)
and thus are the planar equivalent of the SH expansion on the sphere. Inverting the projection function of the camera model enables us to precompute Ŷ m (x, ŷ) by sampling the spherical basis functions for each discrete image coordinate (pixel) and weighting them appropriately.

As the images acquired by the camera have a limited field of view, whereas our view descriptors represent a hemisphere of 180 • , we wish to interpolate the missing signal using a radial nearest-neighbor interpolation. However, instead of performing this interpolation as a preprocessing step, we include it in the process of image expansion by increasing the weight of the border pixels. We integrate over the area each interpolated border pixel would occupy on the sphere and obtain the integrative equivalent of interpolation. Hence, the relatively tedious interpolation of the input images is avoided.

Pose Tracking and Model-Based Motion Control

The navigation process in Optical Rails consists of a sequence of motions of the robot which are each governed by feedback control based on the gradient -g. We obtain -g by acquiring an image from the camera, performing an expansion into SH to obtain the view descriptor a of this image, and then using (18) to compute the gradient. We refer to this process as differential pose tracking. The extension to following an Optical Rail composed of multiple waypoints is achieved by performing a waypoint handover upon reaching the current destination.

Recall that -g gives a camera pose change direction to approach the destination based on the view taken at the current pose. We obtain the steering direction of the robot vehicle towards the destination pose by transforming the gradient -g into the vehicle coordinate system. We define the coordinate system transform M and the steering vector g s , which denotes the proper motion direction of the robot:

g s := ( g s,x lateral , g s,y forward , g s,ϕ rotation ) T , g s = M • (-g) (20)
Tracking and Steering Control for a Non-Holonomic Robot. If the robot follows a track under ideal conditions, it should constantly approach the next target in the forward driving direction while exactly locking to the track in lateral direction. This is particularly important for a non-holonomic vehicle, which can only move in two degrees of freedom. The motion of these vehicles (differential drive, car, tricycle) can be described by a translation rate u forward and a rotation rate u steering . If a trajectory of such a vehicle is subsequently followed using Optical Rails, we may assume that at each time the bearing (rotation) to the next waypoint is small, and that the lateral spacing to the next waypoint is close to zero. Thus, in principle it would be sufficient to determine the proper ratio of rotation and forward translation rate to follow the track. However, due to the varying visual structure in different areas of a real environment, the response of the gradient is not uniform in its individual components. Hence, such a ratio will be largely distorted, leading to a cumulative error of the trajectory and oscillations.

We use PI control to determine the steering rate while driving at a fixed forward speed. This is less susceptible to the influences of the changing sensitivity of the individual components of the steering gradient g s ; we obtain the rotational control value u rot by

u rot (t) = K rot,p • g s,φ (t) + K rot,i t k=t-∆t g s,φ (k) (21)
where K rot,p and K rot,i are weights for the proportional and integral control components, respectively. A typical issue when approaching a view is a deviation perpendicular to the driving direction, often experienced as a lateral divergence after passing through a curve. A continuous response of the lateral gradient component g s,x is indicative of this problem. Although the non-holonomic vehicle model does not allow direct lateral motions, a steering motion to the right while driving forward also moves the vehicle laterally to the right; we determine the lateral correction control value u lat by

u lat (t) = K lat,p • g s,x (t) + K lat,i t k=t-∆t g s,x (k) ( 22
)
with the weighting factors K lat,p and K lat,i as above. Finally, a weighted sum of the lateral correction and rotational control values determines the steering rate of the robot, whereas a constant forward velocity is used:

u forward = sign(g s,y ) • K forward , u steering = α 1 • u lat + α 2 • u rot (23)
To determine the motor control output for the robot drive system, the differential drive vehicle model is employed in connection with a calibration lookup table for motor speed linearization.

Recording a Track and Waypoint Selection

To record a track, greyscale images are acquired at a rate of approximately 5 images per second. For each image, the corresponding approximation in SH coefficients is stored in a list representing the raw data for Optical Rails.

Then, a waypoint selection scheme is applied to this sequence of view descriptors, which drastically reduces the number of reference views in the sequence. For the subsequent task of track following, the set of descriptors is reduced in such a way that subsequent waypoints have a minimum dissimilarity Q min . As views are typically recorded in rapid succession, we obtain a track T of approximately equi-dissimilar view descriptors ãi .

T = [ã 0 , ã1 , . . . , ãN ] (24)
Recall that in an area of convergence, closing in on a waypoint in terms of visual dissimilarity corresponds to approaching this waypoint geometrically; This is, however, on a different scale also depending on the structure of the environment.

The waypoint selection scheme ensures that waypoints which are recorded while the robot only moves on a very small scale are discarded. Therefore only distinct waypoints are kept, which is important for successfully following a track. Waypoints too similar lead to ambiguous handover; if the waypoints are too coarsely spaced, the radius of convergence of the gradient g can be exceeded.

Waypoint Handover

The task of waypoint handover in Optical Rails is to detect when the robot has reached its current destination and should steer towards the next waypoint. Without geometry, this decision has to be made based merely on visual information.

Let Q c be the dissimilarity between the view at the current position and view at the current destination. An intuitive criterion for handover is the dissimilarity measure Q c falling below a threshold T c (denoted here as condition I).

Q c < T c (25)
Even if we assume that all successive reference views on a prerecorded track have an equal dissimilarity Q, already minute (lateral or rotational) deviations from the prerecorded track can lead to handover problems. In a real environment the effect of small pose deviations on the dissimilarity measure can be very different.

There are regions where pose changes have little influence on Q c , whereas in other regions the same pose changes cause substantial changes in dissimilarity. This is caused by the different visual structure of the corresponding views in those regions. Therefore, relying only on a constant threshold is an insufficient condition, as it leads to premature handover in regions where the distance measure is insensitive to motions and causes the robot to diverge from the track. Conversely, if the threshold is chosen too restrictive, in motion sensitive regions no handover will occur at all and the robot will halt. We can, however, augment this criterion with additional conditions for handover in these motion sensitive areas. Let Q p be the dissimilarity between the view at the current position and the previous waypoint and Q n be the dissimilarity between the view at the current position and the next waypoint. We observe: While approaching the current destination, Q p increases while Q c and Q n decrease. At the current destination Q c is small; Q p and Q n are approximately equal (see Fig. 3). This is due to the utilized waypoint selection scheme (see Sec. 6), which provides approximately equal visual distance Q min between successive waypoints. We use the equal visual distance assumption as an additional condition (condition II) for waypoint handover, since it is less dependent on the visual structure of the environment. Therefore we set a small threshold T i on the difference of Q n and Q p , yet allow waypoint handover only if the dissimilarity Q c is not greater than T c,max .

|Q n -Q p | < T i ∧ Q c < T c,max (26)
However, if the visual appearance of the environment quickly changes from one waypoint to the next, equal visual distance between Q p and Q n cannot be assumed. In this case condition II would fail and handover would be missed. Beyond the destination, Q n decreases and Q c increases. Yet our waypoint selection scheme ensures densely spaced waypoints in these areas. Thus, if the robot is about as close to the current destination as it is to the next, handover can also be performed. We set an additional threshold T d on the difference between Q n and Q c and obtain condition III.

|Q n -Q c | < T d (27)
These three conditions lead to combined handover criterion, which can handle all typical situations.

H = Q c < T c condition I ∨ (|Q n -Q p | < T i ∧ Q c < T c,max ) condition II ∨ |Q n -Q c | < T d condition III (28)
8 Putting Optical Rails on a Real Robot

The theoretical concept of Optical Rails has already been extensively discussed [START_REF] Friedrich | Optical rails. view-based point-to-point navigation using spherical harmonics[END_REF]. This prior work uses a simulated robot moving in an artificial environment. While the theory governing the pose-change gradient g is a common foundation for the Optical Rails concept, the extension of the method to an application for a real vehicle -performing maneuvers in a laboratory environment -comprises substantial differences. This applies in particular to the task of obtaining the view descriptors in an efficient way; we use the Frobenius inner product of planar precomputed basis images and the raw camera image (Sec. 4). This new core concept -in concert with the efficient computation of g -has brought Optical Rails to full real-time capabilities.

Mobile Robot Platform

For the implementation of Optical Rails on a real robot, we use a proprietary robot platform based upon standard PC components with a six wheel beltdriven differential drive. The robot is equipped with a variety of different sensors connected via CAN-Bus (IR, accelerometer, ultrasonic), and stereo vision to serve as a multi purpose development platform in our labs. For Optical Rails, however, all these standard sensors are neither required nor used.

The robot runs a distributed software framework on the GNU/Linux operating system. Using a network interface server, the robot is controlled and transmits omnidirectional image data to a PC running Matlab where the Optical Rails algorithm (Sec. 8.2) operates.

Omnidirectional Camera System. For omnidirectional image acquisition, we added a hemispherical camera system to the robot, which is based upon a standard Firewire IIDC camera in connection with a door peephole lens (Fig. 4). We use automatic exposure mode to provide a basic level of robustness to global illumination changes. This relatively cheap imaging setup provides more than sufficient image quality for the vision task using SH, since high-frequency signals are disregarded.

Vehicle Model / Drive System. During track following, we perform motions with a rotational and translational rate governed by the PI control approach Let T be a prerecorded sequence of reference views ãi as introduced in Sec. 6. The robot is initially located at the pose of the first view a 0 in T . Track following takes place as follows:

1. i = 1 2. set the coefficient vector ãi as the destination view ã 3. grab the current image and calculate the coefficient vector a 4. compute the gradient g from a to ã and the dissimilarities Q p , Q c , Q n 5. transform the gradient g to the steering vector g s 6. issue the PI controller subroutine with g s as input value 7. translate the resulting rotation and translation rate into motor control values 8. if the combined handover criterion H is true, advance the destination to the next reference view in T (set i = i + 1) 9. If i > N , the final destination is reached and the robot stops. 10. Else (i ≤ N ) start over at 2. Note that for the case of approaching the last waypoint ãN , the dissimilarity measure Q n cannot be computed, which requires special treatment.

Results

We have tested Optical Rails by a number of experiments in our laboratory with different tracks to be followed. Here we present one of the most difficult tracks, in which the robot has to pass through a door (very rapidly changing views) and has to perform rather sharp turns. The result of this drive can be seen in Fig. 8.2 and is also supplied as a video, which shows the track as it is being recorded and as the robot drives this track autonomously. Even though the implementation includes transmission of the images via WLAN to a workstation performing the bulk of the computation in Matlab, we can process up to 5 input images per second. For the experiments, the processing of only one image per second (and thus issuing only one command to the robot per second) has proven to be sufficient. The utilized constants and thresholds for the Optical Rails algorithm were learned and fine tuned on a track from inside our laboratory, through the door, into the corridor. Without changing the parameters, the robot was able to follow different subsequently recorded tracks. Although we currently use precomputed coefficients u i (j, k) only up to SH order 4 and planar basis functions up to order 7, the results acquired so far are quite satisfactory and prove the Optical Rails concept to be applicable to a real robot.

Outlook

Our experimental results show that Optical Rails can be applied to a real robot in an office environment. To robustify the gradient in situations where the planar projection model is largely violated, we plan to incorporate the depth structure of the environment computed of consecutive snapshots. With such an extension, we expect Optical Rails to perform very well even in very challenging environments.

The present implementation allows for one way track following from the beginning of a track. However, for applications in a network of interconnected tracks it is not only important to reverse the direction: To begin driving from an arbitrary point, an ad hoc localization step and path planning have to be added.

To obtain a smoother driving behavior, we plan to use model-based trajectory planning in combination with the existing calibration of our robot vehicle.

In addition, it is important to provide illumination invariance to the system, since illumination changes also affect the dissimilarity between views. Corresponding procedures have been proposed in a paper on spherical harmonics [START_REF] Friedrich | View-based robot localization using illumination-invarant spherical harmonics descriptors[END_REF].

For universal applicability of the system an important task is to provide tolerance against occlusions. Suitable concepts based on statistical models and local

Fig. 1 .

 1 Fig. 1. From left to right: Y0,0, Y3,1, Y4,2 (upper row) and their weighted planar projection for our calibrated fisheye camera. The circle outline consists of the weights obtained by the integrative equivalent of interpolation.

Fig. 2 .

 2 Fig.2. Original (cropped) image signal (left) and its approximation using planar SH basis functions up to order 7 (right).

Fig. 3 .

 3 Fig. 3. Behavior of dissimilarity measures Qp (previous waypoint, dashed ), Qc (current waypoint, solid ), and Qn (next waypoint, dotted ) for a course along a track leading through a door (experimental results). Vertical bars indicate waypoint handovers.

Fig. 4 .

 4 Fig. 4. Door peephole camera system (left) and raw omnidirectional image (right)

Fig. 5 .

 5 Fig. 5. Experimental result of Optical Rails: following a prerecorded track. The approximate trajectory of the robot has been superimposed

8. 2

 2 The Optical Rails Algorithm.

Fig. 6 .

 6 Fig. 6. Comparison of the originally recorded teach-in sequence (upper images) with autonomous track-following using Optical Rails (lower images).

change detection already exist; the implementation still has to be performed. Yet, due to the holistic approach of Optical Rails, minor occlusions do not disturb the system.


Rui Shen
email: rshen@cs.ualberta.ca

Irene Cheng

Anup Basu

Multi-Camera Calibration Using a Globe

The need for calibration of multiple cameras working together in a network, or for the acquisition of free viewpoint video for 3D TV, is becoming increasingly important in recent years. In this paper we present a novel approach for calibrating multiple cameras using an ordinary globe that is usually available in every household. This method makes it possible to reduce multi-camera calibration to a level that is attainable by non-technical users. Our technique requires only one view of the globe for each camera, unlike earlier approaches that need multiple views of a calibration object. Previous methods that use spherical calibration objects can only calibrate the intrinsic parameters of cameras, but our technique is capable of calibrating both the intrinsic and extrinsic parameters simultaneously in a multi-camera setup. Experimental results verifying the accuracy of our algorithm are also presented.

Introduction

Camera calibration is an important step in computer vision in order to recover the geometry of a given camera (or cameras) [START_REF] Klette | Computer vision: three-dimensional data from images[END_REF]. Based on the dimension of the calibration object, most calibration techniques can be classified into four categories: 3D reference object-based calibration [START_REF] Agrawal | Camera calibration using spheres: A semi-definite programming approach[END_REF], which employs a calibration object that has precise 3D geometric features; 2D plane-based calibration [START_REF] Sturm | On plane-based camera calibration: A general algorithm, singularities, applications[END_REF], which uses a specially-designed planar pattern; 1D object-based calibration [START_REF] Hammarstedt | Degenerate cases and closed-form solutions for camera calibration with one-dimensional objects[END_REF], which employs a 1D stick with three or more marked points; and self-calibration [START_REF] Hartley | Self-calibration of stationary cameras[END_REF], which does not use a specific calibration object but determines a camera's intrinsic parameters by moving the camera in a static scene. Classical calibration techniques (e.g., [START_REF] Horaud | Linear camera calibration[END_REF]) require predefined patterns and static cameras, and often involve solving complicated equations. Active camera calibration [START_REF] Basu | Active camera calibration using pan, tilt and roll[END_REF] [START_REF] Borghese | Computing camera focal length by zooming a single point[END_REF] has been receiving more and more attention with the increasing use of active systems in various applications, such as surveillance and videoconference. Taking advantage of a camera's controllable movements, the camera can be automatically calibrated. In a multiple-camera setup, the relative positions and orientations of the cameras (i.e., extrinsic parameters) also need to be estimated. In such a scenario, self-calibration is not sufficient to retrieve both the intrinsic and extrinsic parameters simultaneously. Therefore, a calibration object is always employed [START_REF] Barreto | Wide area multiple camera calibration and estimation of radial distortion[END_REF] [START_REF] Caprile | Using vanishing points for camera calibration[END_REF]. We propose a multi-camera calibration algorithm using a globe as the calibration object. The widespread availability of globes makes our algorithm easy to apply in everyday environments. Several algorithms [START_REF] Agrawal | Camera calibration using spheres: A semi-definite programming approach[END_REF] [11] [START_REF] Zhang | Camera calibration from images of spheres[END_REF] using spherical objects as the calibration object are proposed by previous researchers. However, these algorithms use the contour of the spheres for calibration, and they can only calibrate for the intrinsic parameters in a multi-camera set up. Our algorithm utilizes the grids on the globe for calibration, and only one view of the globe from each camera is sufficient for calibrating both the intrinsic and extrinsic parameters in a multi-camera setup.

The remainder of this paper is organized as follows: Section 2 covers background theory related to multi-camera calibration. The proposed algorithm is discussed in Section 3. Experimental results are outlined in Section 4, before the work in concluded in Section 5.

Preliminaries

The pinhole camera model is employed. p = (x, y) T denotes the 2D projection of the 3D point P = (X, Y, Z) T on the image plane. Using homogeneous coordinates, p and P have the following relationship:

λ p = K[R | t] P (1)
where

K =   α s x 0 0 β y 0 0 0 1   is the camera calibration matrix; λ is the projection depth;
and R and t are the 3x3 rotation matrix and the 3x1 translation vector respectively; α and β are the focal lengths in pixels in the x-and y-direction respectively; and s is the skew. If the sensor element is rectangular, s becomes 0. γ = α β is called the aspect ratio. If the sensor element is quadratic, γ becomes 1, i.e., α = β. (x 0 , y 0 ) is the coordinates of the principal point on the image plane. The units of α, β and (x 0 , y 0 ) are all in pixels. The five parameters in K are called the camera intrinsic parameters; the three Euler angles in R and the three offsets in t are called the camera extrinsic parameters. The goal of our calibration algorithm is to determine the five intrinsic and the six extrinsic parameters; no radial distortion is considered. Camera calibration using 1D object was first proposed by Zhang [START_REF] Zhang | Camera calibration with one-dimensional objects[END_REF]. A stick with at least three marked points is used as the calibration object. The locations of the points on the stick and the length of the stick are known. When the stick moves with one point fixed, the camera's intrinsic parameters can be computed using six or more observations. Suppose there is a stick AB (A is the fixed point with depth z A ) with length L (i.e., B -A = L) and a third point C between A and B with C = λ A A + λ B B, the following equation can be obtained:

z 2 A h T K -T K -1 h = L 2 (2)
with

h = ã + λ A ( ã × c) • ( b × c) λ B ( b × c) • ( b × c) b ( 3
)
where K is the camera calibration matrix; ã, b and c are the images of points A, B and C respectively. From Equation (3), h for the set of points under every observation can be computed. Let: 2) can be rewritten as:

B = K -T K -1 =   B 11 B
v T x = L 2 (5) with v = (h 2 1 , 2h 1 h 2 , h 2 2 , 2h 1 h 3 , 2h 2 h 3 , h 2 3 ) T . ( 6
)
After the v's are computed, x can be obtained by solving the equation:

V x = L 2 1 (7)
Now that x is known, the intrinsic parameters can be computed easily from Equation (4). The critical motions of the 1D object are identified by Hammarstedt et al. [START_REF] Hammarstedt | Degenerate cases and closed-form solutions for camera calibration with one-dimensional objects[END_REF]. Let v j be the vanishing point in the jth image, then the motion is critical when the vanishing points of the stick lie on a conic ω so that:

∀ j v T j ω v j = 0 (8)
In 3D space, this means that the trajectory of the stick lies on a quadric surface. Wang et al. [START_REF] Wang | Multi-camera calibration with one-dimensional object under general motions[END_REF] applied 1D object calibration to a multi-camera setup, and proved that this object can take more general motions instead of constrained motions. However, one constraint still applies to the motions, i.e., the infinite points of the stick should not lie on a conic ω". Let V j be the infinite point of the stick A j B j in the jth image, i.e., Ṽj = Bj -Ã j , then the constraint can be expressed as the following equation NOT being satisfied:

∀ j V T j ω"V j = 0 (9)
Proposition 1: The motion constraint expressed in Equation ( 8) is equivalent to that in Equation ( 9). (See Appendix for proof.)

From Proposition 1, we can see that the only motion constraint that Wang et al. [START_REF] Wang | Multi-camera calibration with one-dimensional object under general motions[END_REF] have relaxed is that no point on the stick needs to be fixed during the motion. As stated in [START_REF] Hammarstedt | Degenerate cases and closed-form solutions for camera calibration with one-dimensional objects[END_REF], the critical motions do not depend on the actual position of the stick's fixed point, but only depend on the stick's orientation. Therefore, the direction vectors of the stick should not lie on a quadric surface during the motion.

Calibration Algorithm

We propose a technique to calibrate both single and multiple cameras from a single image of a globe (or sphere with evenly distributed surface points).

The intrinsic parameters of a single camera can be calibrated from the captured intersections of the constant latitude and longitude lines of a globe. Although the calibration object is now a 3D sphere, the actual computation of the cameras' parameters can mostly follow the way in which the 1D stick [START_REF] Zhang | Camera calibration with one-dimensional objects[END_REF] is used. A globe (sphere) with marked longitude-latitude intersections is shown in Figure 1(right). A is the center of the globe, which is also the fixed point. B i 's are the intersections distributed evenly on one great circle of the globe (15 • or 30 • apart from each other). Every diameter B i AC i of a great circle is equivalent to the 1D stick at a different orientation with λ A = 2.0 and λ B = -1.0. Note that now L equals the radius of the globe. In the case of a globe, at least three great circles (e.g., two meridians and the equator) should be used in order to avoid the degenerate cases and calibrate the cameras using only one view.

Figure 1(left) depicts the image of the globe; b i , c i , and a are the images of their corresponding spatial points respectively. Only b i 's can be obtained directly from the image, c i 's and a are not visible on the image. However, in order to calibrate the camera, the positions of all these image points need to be known. Normally, the 2D image of a 3D circle is an ellipse (if not degenerated to a line segment), which is a special case of a conic. However, in general, the center of the sphere does not project to the center of the ellipse. Using five points, the conic can be determined up to a scale factor. Let C 1 , C 2 , and C 3 denote the conics determined by the images of the equator circle and two meridians respectively. L 1 = C 1 C 2 is the line determined by the two intersection points of C 1 and C 2 , and L 2 = C 1 C 3 is the line determined by the two intersection points of C 1 and C 3 . a is determined by the intersection of L 1 and L 2 , i.e., a = L 1 L 2 . Once a is calculated, one can compute each c i by calculating the line-conic in-tersections. Let b i j denote the jth image point on the conic C i , and let b i j + λa denote a point on the line defined by b i j and a. Then, the other intersection of the line with

C i is c i j with λ = -(b T i j C i b i j )/2(b T i j C i a)
. From here, we can follow the method proposed by Zhang [START_REF] Zhang | Camera calibration with one-dimensional objects[END_REF] to recover the intrinsic parameters. However, part of the final calculation of the intrinsic parameters from x has some inaccuracies in [START_REF] Zhang | Camera calibration with one-dimensional objects[END_REF]. Here, we give more accurate equations:

y 0 = (x 2 x 4 -x 1 x 5 )/(x 1 x 3 -x 2
2 ) (10)

z A = x 6 -[x 2 4 + y 0 (x 2 x 4 -x 1 x 5 )]/x 1 ( 11
)
α = z 2 A /x 1 ( 12
)
β = z 2 A x 1 /(x 1 x 3 -x 2 2 ) (13)
s = -x 2 α 2 β/z 2 A = -x 2 β/x 1 ( 14
)
x 0 = sy 0 /β -x 4 α 2 /z 2 A = -(x 2 y 0 + x 4 )/x 1 ( 15
)
At least three great circles (one equator and two meridians) are needed to avoid degenerate cases, and at least five points of each great circle need to be visible to determine the conic. Therefore, at least fifteen points are needed. However, two points can be used twice, which are the intersections of the equator and the meridians. Hence, thirteen visible points are the minimal requirement for calibration.

Proposition 2: For a set of image points of a camera, if there exists an L that can be used to derive a K, then for any L > 0 the algorithm produces the same K. (See Appendix for proof.) Proposition 2 means that using a set of images, the radius L of the globe does not need to be known in order to get the correct calibration result. In addition, even if the value of the radius is assigned arbitrarily, the results are consistent. However, different assignments of L do affect the reconstruction, because different projection depths (z A ) are obtained.

Multi-Camera Calibration

In the previous section, the calibration of a single camera using a globe is discussed. Multiple cameras can also be calibrated simultaneously by capturing the positions of intersections of the constant latitude and longitude lines of a globe. Normally, one camera is selected as the reference camera, the coordinate system of which is assumed to coincide with the world coordinate system. The positions and orientations of other cameras are calculated relative to this reference camera. The spherical calibration object is shown in a two-camera case in Figure 2. Once the position and orientation of one camera relative to its adjacent camera are calculated, by combining the transformations from camera to camera, the position and orientation of one camera relative to any other camera can be obtained. Therefore, in this section we only discuss the two-camera case.

Suppose there are two cameras, Camera 0 with projection matrix K 0 [I | 0], and Camera 1 with projection matrix

K 1 R 1 [I | -O 1 ] = K 1 [R 1 | t 1 ],
where O 1 is the coordinate of Camera 1 's center in the world (Camera 0 's) coordinate system and t 1 = -R 1 O 1 . K 0 and K 1 can be calibrated using the method proposed in the previous section. Hence, the remaining problem is to calculate R 1 and O 1 . The projection expressed in Equation (1) can also be written as:

λ p = K[I | 0] Pcam (16)
where Pcam = R -RO

0 1 P (17)
is the position of point P in the camera's own coordinate system using homogeneous representation. When non-homogeneous representation is used,

P cam = R(P -O). (18)
Since z A is calculated together with K, z B i 's can be computed from the following equation as in [START_REF] Zhang | Camera calibration with one-dimensional objects[END_REF]:

z B i = -z A λ A ( ã × ci ) • ( bi × ci ) λ B ( bi × ci ) • ( bi × ci ) . ( 19
)
Then, the spatial points can be reconstructed in each camera's own coordinate system using a variation of Equation ( 16), which is

Pcam = λK - p 1 . ( 20
)
Since Bi,cam0 = I Bi = Bi , and Bi,cam1 = R 1 -R 1 O 1 0 1 Bi , we can derive the following equation:

Bi,cam1 = R 1 t 1 0 1 Bi,cam0 (21)
where

t 1 = -R 1 O 1 .
Once four or more such reconstructed pairs ( Bi,cam1 , Bi,cam0 ) are obtained, R 1 and O 1 can be calculated by:

R 1 t 1 0 1 = [ Bi,cam1 ][ Bi,cam0 ] -1 (22) O 1 = -R -1 1 t 1 . ( 23
)
If we use L as the unit for the measurement of relative positions, Equation ( 23) can be re-written as:

O 1 = - 1 L R -1 1 t 1 . ( 24
)
This completes the calibration of extrinsic parameters.

Proposition 3: For a set of image points of a camera, if there exists an L that derives a K, R, and O, then for any L > 0 the algorithm produces the same K, R, and O. (See Appendix for proof.)

Proposition 3 means that using a set of images, the radius L of the globe does not need to be known in order to get the correct calibration result. In addition, if two different radii are assigned separately, the results are consistent. However, the accuracy of the measurement of L does affect the calibrated position in physical units, e.g., centimeter or inch.

Compared to previous methods, one advantage of our algorithm is that only one image taken from each camera is sufficient for calibration of all the cameras, as long as enough intersections (minimally four) are visible simultaneously by any two adjacent cameras. Another advantage is that the correctness of the calibration result can be determined directly since the spherical points (including the sphere center) are reconstructed immediately. If the distances from each of those points to the sphere center all equal to L, then the points lie on a spherical surface. This reconstruction result validates the calibration result. However, one drawback of our algorithm is that the accuracy of the result strongly depends on the initial detection of the intersection points.

Experimental Results and Discussion

The proposed calibration algorithm is validated using both synthetic data and real images. As for synthetic data, a set of image points are generated by projecting a synthetic 3D globe model onto the image plane. To simplify the experimental process, the image of the globe center is assumed to be known. To test our algorithm under a single-camera setup, we use the following camera property: α = 1200, β = 1000, s = 1, x 0 = 400, and y 0 = 300. For the globe, the angle between each meridian is 15 o , L = 150, A = (0, 35, 1500) T . The experimental result computes exactly the same parameters as the assigned camera property. To test our algorithm under a two-camera setup, we use the following globe property: the angle between each meridian is 15 o , L = 200, A = (50, 10, 2000) T ; and the following camera properties: for Camera 0 (the reference camera), α = 1000, β = 1000, s = 1, x 0 = 400, and y 0 = 400; for Camera 1 , α = 1000, β = 800, s = 0, x 0 = 320, and y 0 = 240. The position and orientation of Camera 1 relative to Camera 0 is defined by the transformation in the following order: rotate -10 o about y-axis, rotate 10 o about x-axis, rotate 0 o about z-axis, translate by the vector (200, 100, -500) T = L(1, 0.5, -2.5) T . The experimental result computes exactly the same intrinsic and extrinsic parameters of the cameras.

Furthermore, the calibration algorithm for intrinsic parameters is also validated using real images. One image (image size: 2048 × 1536) used for testing is shown in Figure 3 (left) with twenty-two detected intersections marked as red circles and three fitted ellipses in blue. Here we employ the direct least-squares method [START_REF] Fitzgibbon | Direct least square fitting of ellipses[END_REF] for ellipse fitting. Twenty-two intersection points are used mainly for ellipse fitting, but it is not necessary to use all of them to calculate the intrinsic parameters in the steps afterward. Table 1 gives the calibration results. Case one uses all the twenty-two points for calibration, while case two uses fourteen points as marked in Figure 3 (right). Unlike other methods [START_REF] Zhang | Camera calibration with one-dimensional objects[END_REF] [START_REF] Wang | Multi-camera calibration with one-dimensional object under general motions[END_REF], which validate the calibration results by calculating the relative difference to the result obtained from the chessboard method [START_REF] Zhang | A flexible new technique for camera calibration[END_REF], our algorithm directly reconstructs the spherical points; therefore, we use the relative error in the reconstruction result as the measure of the correctness of the calibration result. The RMSE is the root mean squared error measured on the reconstructed sphere. We define the error as the normalized difference between the distance from a reconstructed point to the reconstructed center and the actual radius length L, i.e., E = | B i -A L -1|, where B i 's are the reconstructed spherical points and A is the reconstructed sphere center. Then, the RMSE is calculated as

RMSE = 1 N Σ N i=1 E 2 ,
where N is the total number of intersection points used. The reconstruction errors are around 2% in both cases, which reflects the error in the calibration results. As mentioned earlier, the accuracy of our algorithm depends largely on the initial detection of the intersection points. The error in the calibration results is caused mainly by the inaccuracy in the detection step. In addition, the fact that our algorithm does not take radial distortion into account may also contribute to the error.

In the current stage, the initial intersection points are picked manually. We are still working on automatically detecting those points. One possible method is to cover the grids on the globe, which are formed by the longitude and latitude lines, with alternate black and white blocks, a pattern that resembles the chessboard. A globe painted in this way will facilitate the detection of the intersection points. Alternatively, simply putting a marker on each of those inter- sections may also help the automatic detection. We are also working on using the proposed approach to calibrate a multi-camera setup, as shown in Figure 4, to be used to capture video for a free-viewpoint 3D display.

Conclusion

We proposed a new multi-camera calibration algorithm using a simple household object, a globe. The approach uses a single view of the globe to calibrate a camera, and does not need to move the globe to multiple locations. The effectiveness of the algorithm is validated through experiments on both synthetic data and real images. In the future, we will test our algorithm with noisy data to verify its robustness.

Proof: For the ith camera, suppose its intrinsic parameters (K i ) and extrinsic parameters (R i and t i ) are invariant when capturing the images. The vanishing point v i j is the image of the infinite point V i j , therefore:

∀ j ṽi j = K i [R i | t i ] Ṽi j . ( 25
)
Let the world coordinate system coincide with the ith camera's coordinate system, then Equation (25) can be rewritten as:

∀ j v i j = K i V i j . ( 26
)
If Equation ( 8) is satisfied, i.e., the motion is critical, then from Equation (26) the following equation is also satisfied:

∀ j v T i j ω v i j = V T i j K T i ω K i V i j = V T i j ω"V i j = 0. ( 27
)
This is exactly the same expression as Equation ( 9). Similarly, one can prove that if Equation ( 9) is satisfied, then Equation ( 8) is also satisfied.

Proposition 2: For a set of image points of a camera, if there exists an L that can be used to derive a K, then for any L > 0 the algorithm produces the same K.

Proof: Suppose L is scaled by a factor γ > 0, i.e., the radius of the globe used for calculation becomes L = γL. Note that the assignment of the radius does not affect the image points. From Equation [START_REF] Basu | Active camera calibration using pan, tilt and roll[END_REF], the current x =γ 2 x, i.e., x i = γ 2 x i for i ∈ [START_REF] Klette | Computer vision: three-dimensional data from images[END_REF][START_REF] Horaud | Linear camera calibration[END_REF]. Therefore, y 0 = y 0 , z A = γz A , α = α, β = β, s = s, and x 0 = x 0 using Equations [START_REF] Caprile | Using vanishing points for camera calibration[END_REF] to [START_REF] Fitzgibbon | Direct least square fitting of ellipses[END_REF]. Hence, K = K. Proposition 3: For a set of image points of a camera, if there exists an L that derives a K, R, and O, then for any L > 0 the algorithm produces the same K, R, and O.

Proof: We consider two cases: 1) the camera is the reference camera; and 2) the camera is not the reference camera.

Case 1: Because R = I and O = 0, the proposition can be proved directly from Proposition 2.

Case 2: Suppose L is scaled by a factor γ > 0. From Proposition 2, we know that as a consequence z A is scaled by γ, but K remains the same. z B i becomes z B i = γz B i from Equation (19). Therefore, B i,cam0 = γB i,cam0 = γ I(B i -0) = γB i and B i,cam1 = γB i,cam1 = γR 1 (B i -O 1 ) from Equation (20). Therefore, B i,cam1 = R 1 (B i,cam0 -γO 1 ). Using homogeneous representation, B i,cam1 = R 1 t 1 0 1 B i,cam0 , where t 1 = γt 1 . The calibration result of R 1 is not affected by the scale factor γ using Equation (22). Using Equation (24), the calibration result of O 1 is also not affected, as γ is eliminated from both the numerator and the denominator. Hence, the calibration result is independent of the assignment of L.

Fig. 1 .

 1 Fig. 1. (Left): The spherical calibration object; (right): image of a globe.

Fig. 2 .

 2 Fig. 2. The spherical calibration object in a multi-camera setup.

Fig. 3 .

 3 Fig. 3. Globe images with detected intersections and fitted ellipses.

Fig. 4 .

 4 Fig. 4. (Left and middle): different views of a globe captured by different cameras in a multi-camera set up (right).

  B 12 , B 22 , B 13 , B 23 , B 33 ) T , and let h = (h 1 , h 2 , h 3 ) T , and x = z 2

	12 B 13	
	B 21 B 22 B 23		(4)
	B 31 B 32 B 33
	with b = (B 11 , A b, then Equation (

Table 1 .

 1 The calibration results with real data.

	Cases	α	β	s	x 0	y 0	RMSE * min(E) * max(E)
	One 4448.99 5261.07 823.664 896.6 974.848 2.486	0.095	9.037
				(79.51 • )
	Two 4413.78 4931.82 439.022 1061.76 1051.12 1.905	0.002	4.918
				(84.32 • )

* : the unit for RMSE and E is percentage (%).

Appendix

Proposition 1: The motion constraint expressed in Equation ( 8) is equivalent to that in Equation [START_REF] Barreto | Wide area multiple camera calibration and estimation of radial distortion[END_REF].


Aubrey K Dunne
email: aubrey.dunne@eeng.dcu.ie

John Mallon

Paul F Whelan

Towards Dynamic Camera Calibration for Constrained Flexible Mirror Imaging

Flexible mirror imaging systems consisting of a perspective camera viewing a scene reflected in a flexible mirror can provide direct control over image field-of-view and resolution. However, calibration of such systems is difficult due to the vast range of possible mirror shapes and the flexible nature of the system. This paper proposes the fundamentals of a dynamic calibration approach for flexible mirror imaging systems by examining the constrained case of single dimensional flexing. The calibration process consists of an initial primary calibration stage followed by in-service dynamic calibration. Dynamic calibration uses a linear approximation to initialise a non-linear minimisation step, the result of which is the estimate of the mirror surface shape. The method is easier to implement than existing calibration methods for flexible mirror imagers, requiring only two images of a calibration grid for each dynamic calibration update. Experimental results with both simulated and real data are presented that demonstrate the capabilities of the proposed approach.

Introduction and Background

Flexible mirror imaging systems, recently introduced in [START_REF] Kuthirummal | Flexible mirror imaging[END_REF], are non-conventional catadioptric cameras typically consisting of a perspective camera viewing a scene reflected in a bendable mirror. The characteristics of the camera image can be continuously and non-linearly altered by directly flexing the mirror. This enables control over both the field-of-view of the image and the image resolution. By suitable flexing of the mirror, certain portions of the scene being imaged can be attributed more sensor resources than other portions, and the field-of-view can be easily altered as required. Flexible mirror imagers may thus be beneficial in active vision monitoring and security applications, where higher resolution could be obtained in image areas containing objects of interest without sacrificing fieldof-view (as would normally be the case in pan-tilt-zoom camera systems).

The introduction of flexible mirror imaging systems naturally leads to the requirement for methods for their calibration, so that tasks such as tracking in surveillance can be accomplished. Calibration of flexible mirror imaging systems is difficult firstly because the mirror deflection is generally unconstrained, and secondly because the camera configuration, and thus calibration, alters each time that the mirror is flexed. The calibration is intrinsically linked to the estimation of the flexible mirror surface itself, since knowledge of local mirror surface shape allows surface normals and thus reflected rays to be determined for given incident rays. However, since the mirror shape can change, catadioptric calibration methods that assume some prior knowledge of mirror shape, such as [START_REF] Tardif | Self-calibration of a general radially symmetric distortion model[END_REF][3] [START_REF] Scaramuzza | A Toolbox for Easily Calibrating Omnidirectional Cameras[END_REF], are not applicable, and rather methods that can estimate the mirror shape are required. Much work has been presented in the literature on the recovery of surface shape from images of a diffuse surface, using either structured light, or epipolar geometry. The recovery of specularly reflective surface shape has received relatively less attention from the vision community. This situation differs from the diffuse case since 'features' seen in the image are virtual features caused by the reflection, which do not obey the epipolar constraint. Halstead et al. [START_REF] Halstead | Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals[END_REF] presented one of the first techniques for determining specular surface shape from surface reflection in a single image. They used a bespoke conical calibration object and concentric camera to determine corneal surface estimates. Savarese et al. [START_REF] Savarese | Local shape from mirror reflections[END_REF] describe a method for specular surface recovery from a single image of a planar calibration target when at least two local orientations are available at each target point. Several approaches to estimating mirror shape from motion have also been presented. Swaminathan et al. [START_REF] Swaminathan | On the motion and appearance of specularities in image sequences[END_REF] examine the dependence on surface geometry of specularities in static scenes with constant velocity camera motion, whilst Roth et al. [START_REF] Roth | Specular flow and the recovery of surface structure[END_REF] go further by estimating the specular surface geometry from specular flow and using the result to improve surface estimation from diffuse flow. Oren et al. [START_REF] Oren | A theory of specular surface geometry[END_REF] recover information on surface profile from specular reflections whose paths overlap under constrained camera motions. However, all of these methods assume a fixed mirror shape with relative mirror scene motion, and so are not suitable for flexible imaging systems calibration.

Calibration methods for the general camera model such as [START_REF] Grossberg | A general imaging model and a method for finding its parameters[END_REF] and [START_REF] Sturm | A generic concept for camera calibration[END_REF] can be used to determine reflected ray directions and thus calibrate any catadioptric camera, but these methods are time consuming and when applied to a flexible mirror imager would require repeating the entire calibration process each time the mirror is flexed. Gonçalves and Araújo [START_REF] Gonçalves | Low-cost method for the estimation of the shape of quadric mirrors and calibration of catadioptric cameras[END_REF] also present a method for calibrating a catadioptric system consisting of a quadric mirror surface from a single image of a calibration target. A completely non-parametric method for specular surface estimation based on voxel carving was presented in [START_REF] Bonfort | Voxel carving for specular surfaces[END_REF], in which normal vectors are accumulated for each scene voxel, and the voxels with the normals in best agreement are considered to be on the specular surface. The method requires images of the reflective surface from many different viewpoints to achieve good results. All the above methods assume a fixed specular surface geometry. If they were to be applied to calibrate a flexible imaging system, the system would have to be completely recalibrated after each mirror flexion, which is not practical in the envisaged system. Kuthirummal and Nayar [START_REF] Kuthirummal | Flexible mirror imaging[END_REF], who introduced the concept of flexible mirror imaging with a nominally planar mirror surface, presented the only calibration method specifically for such systems of which the authors are aware. Their calibration is performed offline by acquiring an image of the mirror boundary, and an accompanying mirror surface measurement, for each mirror deformation that is likely to be imposed. They present results for a calibration with greater than 30, 000 image-estimate pairs. The one-to-one mapping between the mirror surface shape and a descriptor of its outline is then stored in a look-up table, so that once the mirror outline is visible in the image the calibration is available.

This paper focuses on the constrained case of a flexible mirror fixed along one edge that allows flexing in the plane perpendicular to this fixed edge. In practice, this system is easy to implement as it requires only a single point of deflection, yet it allows variation in field-of-view and image resolution across one dimension of the image. Such a setup has potential application in surveillance systems, where variations in the horizontal field-of-view are important but a fixed vertical field of view is typically adequate. The major contribution of this paper is the introduction of a novel method of calibration for this type of constrained flexible mirror imaging system. The principals of an alternative calibration method to [START_REF] Kuthirummal | Flexible mirror imaging[END_REF] are established, where the aim is to dynamically update the calibration rather than attempt to completely recalibrate after each mirror deflection.

The proposed calibration method takes its inspiration from the field of mirror design for catadioptric systems. Swaminathan et al. [START_REF] Swaminathan | Framework for designing catadioptric projection and imaging systems[END_REF] developed a linear method for determining the catadioptric mirror surface shape necessary to implement a desired scene to image map. The method uses constraints on the incident and reflected ray directions to determine the parameters of a B-spline surface model of the mirror. Similarly to [START_REF] Halstead | Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals[END_REF], the surface is determined directly from the set of surface normals. This method is adapted for the purpose of calibration in this paper, so that in conjunction with scene information obtained by imaging a calibration grid both before and after the mirror deflection, the new mirror surface can be estimated and thus the calibration determined. In comparison to the calibration method of [START_REF] Kuthirummal | Flexible mirror imaging[END_REF], the proposed method has the significant advantage of a much simpler calibration setup. It does not require a stereo rig for surface shape estimation, nor is there a large and laborious data collection procedure required. The proposed method is adaptive where [START_REF] Kuthirummal | Flexible mirror imaging[END_REF] is exhaustive. The mirror shapes that it can calibrate are not limited to the set of pre-calibrated deformations nor is there a requirement for the entire mirror boundary to be visible in the image, although unlike [START_REF] Kuthirummal | Flexible mirror imaging[END_REF] the presented method can not perform calibration in real time.

Section 2 presents the camera model and the method for mirror shape estimation. In Section 3 the calibration process, consisting of an offline primary stage and a dynamic stage that operates to update the calibration as the mirror flexes, are presented. Results for both simulated and real data demonstrating the performance of the calibration method are detailed in Section 4. Finally, directions for future work and conclusions are presented.

Camera Model

This paper deals with a constrained type of flexible mirror imaging system, in which a perspective camera images a scene reflected in a thin, flexible, nominally planar mirror surface, as used in [START_REF] Kuthirummal | Flexible mirror imaging[END_REF]. Furthermore, one edge of the flexible mirror is assumed to be fixed in position and in first derivative relative to the perspective camera, i.e. the mirror is clamped at one edge. It will be seen that this constraint removes the ambiguity in the location of the mirror in the world coordinate system. Flexing can occur in a direction perpendicular to the edge, and so the mirror can be modelled by a 2D curve in the plane. For simplicity it is assumed that the fixed edge of the mirror is vertical with respect to the perspective camera viewing it, and is perpendicular to the camera's principal axis.

Reflection of light rays from specular surfaces is governed by the law of reflection, which states that the angle of incidence is equal to the angle of reflection. Knowledge of the surface normal at any point on the surface allows the reflected ray to be calculated for any incident ray. Referring to Fig. 1, reflected light rays entering the perspective camera are termed viewing rays and are represented by unit vectors V l . The corresponding light rays that fall on the mirror from the scene are called scene rays and are represented by unit vectors V r . For any given mirror surface shape, the flexible imager can be modelled by the general camera model, which maps image pixels (u, v) to scene ray directions V r via a look up table. This model is non-parametric and so can effectively cope with the non-linearity and non-centrality of the flexible mirror imager.

Mirror Shape Model

The mirror surface is modelled by a B-spline curve with quadratic basis functions and an open uniform knot vector. The mirror spline model is

D(u) = K f i=1 c i f i (u) (1) = C T F (u) , (2)
where u is the horizontal image pixel coordinate, f i (u) are the spline basis functions, and K f is the number of basis functions. K f = 5 in this paper. As in [START_REF] Swaminathan | Framework for designing catadioptric projection and imaging systems[END_REF][START_REF] Kuthirummal | Flexible mirror imaging[END_REF], the complete mirror surface is described parametrically by the distances of the mirror points from the perspective camera centre, given by the spline model, as measured along the viewing rays V l of the primary optics. The perspective camera centre is assumed without loss of generality to be at the world coordinate origin. The mirror surface, S(u), is therefore given by

S(u) = D(u)V l (u) . (3)
Note that for the experiments with real data in Section 4.2, an additional Bspline that is a function of the vertical image pixel coordinate, and that has a single quadratic basis function, is incorporated into the mirror surface model. This is used to remove the vertical curvature that would otherwise result from Eqn. 3. The modification also accounts for any misalignment that exists in the experimental setup between the fixed mirror edge and the perspective camera, where the fixed mirror edge may not be perpendicular with respect to the camera principal axis.

Mirror Shape Estimation

The mirror surface is estimated from a set of linear equations as in [START_REF] Swaminathan | Framework for designing catadioptric projection and imaging systems[END_REF]. For known V l s and V r s, the surface normals, N (u), are given by

N (u) = V l (u) -V r (u) |V l (u) -V r (u)| . (4)
The tangents to the surface are given by the first derivative of the surface. Enforcing orthogonality between normals and the tangents leads to the following

dS(u) du • N (u) = 0 . (5)
Combining Eqns. 3 and 5 gives

(D ∂V l ∂u + V l dD du ) • N (u) = 0 . (6)
Expanding out with the B-spline basis functions and coefficients gives

dV l du • N (u)(C T F (u)) + V l • N (u)(C T dF (u) du ) = 0 , (7)
which is conveniently rewritten as

dV l du • N (u) F (u) T + (V l • N (u)) dF (u) T du C = 0 . (8)
From the pinhole model for the perspective camera V l is given by

[u f ] T |[u f ]|
, where f is the camera focal length, and thus

dV l du = 1 (u 2 + f 2 ) 3 2 f 2 -uf . ( 9
)
Each image point and corresponding mirror surface normal allow an equation of the form of Eqn. 8 to be formed. By stacking at least K f such equations into a matrix a homogeneous equation of the form Ax = 0 is obtained. This can then be solved using the SVD with equilibration to determine the B-spline coefficients. Note that A must contain at least one row for an image point in each knot interval.

Mirror Edge Constraint

The above solution for the B-spline coefficients is up to scale. For complete calibration it is necessary to remove the scale ambiguity. The fixed mirror edge constraint imposed in Section 2 provides the required constraint and so must be incorporated into the mirror shape estimation equations. Assume that a solution for the mirror surface and scale is available for some position and deflection of the mirror (see Section 3). In this case, for an image pixel, u e , viewing the fixed edge of the mirror the spline value D(u e ) is known. This value is constant for all mirror deflections so that after a deflection of the mirror the spline equation associated with u e is

D(u e ) = F ′ (u e ) T C ′ , (10)
where the prime indicates new spline coefficients and basis functions. A least squares solution for C ′ can thus be extracted in a subspace spanned by K f -1 basis vectors as

C ′ = C P + C B Φ , (11)
where C P is the particular solution, C B is a matrix of basis vectors, and Φ is the new vector of unknowns. Finally, C ′ is incorporated into Eqn. 8, and after some rearrangement the new equation to be solved that incorporates the edge constraint is

GC B Φ = -GC P , (12)
where

G = ∂V l ∂u • N (u) F ′ (u) T + (V l • N (u)) dF ′ (u) T du . (13)
A solution to Eqn. 12 can be determined using standard least squares techniques.

Calibration

The goal of calibration is to determine the mapping between viewing rays V l and scene rays V r . This mapping is given by the mirror surface, so that calibration reduces to the estimation of the mirror surface. The calibration consists of two stages: (1) calibration of primary optics and determination of initial mirror surface, and (2) dynamic updating of calibration after the mirror is flexed. The primary calibration stage is only executed once, whereas the dynamic stage is applied after each mirror flexion.

Primary Calibration

Primary calibration aims is to determine a complete calibration of the imager for some initial mirror position and shape. It employs two basic calibration techniques. Firstly the perspective camera is calibrated using a standard method such as [START_REF] Zhang | A flexible new technique for camera calibration[END_REF]. This information is used to both remove radial distortion from all subsequent perspective camera images, and to enable the V l s, which remain fixed, to be calculated as described in Section 2.2. The complete flexible mirror imager is then calibrated as a general camera for the initial mirror shape and position using general calibration [START_REF] Grossberg | A general imaging model and a method for finding its parameters[END_REF], which gives the mapping between u and V r for that configuration. Although general calibration is a time consuming process, it only needs to be carried out once. Interestingly, the primary calibration stage could be performed in a single step, by calibration of the perspective camera, if the initial mirror shape is known to be planar. Once the V l s and V r s are known, their intersection points on the mirror surface are estimated so that the mirror position can be determined. The position estimate is found by selecting the points that minimise the distance to V l and V r for each corresponding pair of rays.

Dynamic Calibration

Any deflection of the mirror requires the calibration to be updated, which is equivalent to estimating the updated mirror shape, S(u) ′ . In Section 2.2 it is shown that knowledge of the scene and viewing rays is sufficient for reconstructing the mirror surface. The viewing rays V ′ l for S(u) ′ can be directly determined from the location of the features in the second image as in Section 2.2, but the V ′ r s are unknown. To determine them some information about the scene is required, and this is acquired by imaging a calibration grid, placed in the imager field-of-view, both before and after mirror deflection. The pose of this grid is estimated from the first image, which has already been calibrated (either in primary calibration stage or in the previous application of dynamic calibration). A non-central pose estimation technique, such as [START_REF] Sturm | A generic concept for camera calibration[END_REF], is necessary to estimate the pose. The estimate of the calibration grid pose allows the 3-space locations of the grid corner scene points, P (u), to be determined.

Referring to Fig. 1, it is clear that if the scene points P (u) are at infinity, then V ′ r is equal to V r for the corresponding image points. This infinite scene assumption allows us to approximate V ′ r as V r for distant scene points, and enables a linear estimation of the mirror surface using Eqns. 3, 5 and 12. The linearly estimated result, Ŝ(u) ′ , is used to initialise a non-linear minimisation to determine the final mirror surface estimate, S(u) ′ . For the minimisation V ′ r is represented as

V ′ r (u) = P (u) -Ŝ(u) ′ (14) = P (u) -D(u) ′ V l (u) ′ (15)
so that the estimated surface normal, N (u), is

N (u) = V l (u) ′ -[P (u) -D(u)V l (u) ′ ] . (16)
Non-linear minimisation is then applied to minimise the following cost function

Γ (D(u) ′ ) = d Ŝ(u) ′ du • N (u) | N (u)| ( 17
)
This gives the solution, S(u) ′ , for which the viewing ray for the image of each scene point P (u) maps to the scene ray passing through that point. Once the calibration has been updated (S(u) ′ determined), the mirror can be flexed again and dynamic calibration reapplied using the V r s estimated in the last iteration. Note that although there is an inherent bias due to the infinity assumption, it is found that for small mirror deflections the linear estimate is sufficient for the convergence of the non-linear estimation. Large mirror deflections should be carried out in stages, with dynamic calibration applied at each stage.

Experimental Results

Experiments were carried out using simulated data, to evaluate the effect on performance of configuration variations, and using real data, so as to characterise the overall performance of the calibration method.

Simulated Experiments

Experiments with simulated data were conducted for variations in image noise, scene depth (distance between mirror and calibration grid) and mirror curvature.

The simulation results are presented in Fig. 3, where the datapoints in each plot are the average of 100 random trials. Row 1 shows the error plots for increasing additive Gaussian image noise. Additive image noise has a greater impact on the final surface estimate than on the linear estimate, since the error due to the infinite scene assumption far outweighs the error introduced by the noise.

The final surface estimate error is linearly proportional to the noise. The error plots for increasing scene depth are shown in row 2. As expected, the error decreases as the calibration grid moves further from the mirror, in line with the infinite scene assumption. Note that for large scene depths (> 3000mm) the final error becomes independent of the scene depth (infinity assumption is effective approximation). The error plots for increasing mirror curvature, κ, are shown in row 3. It can be seen that the error in the linear surface estimate increases with increasing curvature due to increasing weakness of the infinite scene assumption.

The error in the final mirror surface estimate is larger than the maximum error for either of the other two simulated experiments principally due to the poor linear estimate.

Real Experiments

The calibration method was evaluated for a real flexible imaging system set up as described in Section 2. A flexible plastic mirror1 was attached to a sheet aluminium substrate in order to improve rigidity. Deflections were applied to the back of the substrate using optical translation stages with 0.01mm resolution.

In the first set of experiments, high-accuracy comparative reference measurements of the mirror surface were obtained using a 3D laser scanner2 . Fig. 4 shows the estimated and reference mirror surface before and after a mirror deflection of approximately 7.98mm. It can be seen that the primary calibration error is of similar magnitude to the error for the final mirror surface estimate after flexing, indicating that the accuracy of the dynamic calibration is very sensitive to the primary calibration. The primary calibration error can be attributed to the higher order components of the mirror surface, caused by mirror defects, that are not modelled by Eq. 2. The distance from the mirror surface to the calibration grid was approximately 280mm, and as a result the linear estimate of the flexed mirror surface in Fig. 4 is relatively inaccurate, although the final estimate is still good.

A second set of experiments were conducted with the same flexible mirror imager configuration as above in order to separately asses the primary and dynamic calibration stages. The overall accuracy of the primary calibration and the effect of the surface estimation using B-splines were evaluated by comparing directly measured scene rays with scene rays estimated from the model. Approximately 160 pixels with corresponding V l s were chosen uniformly from the image, and the errors in the angles between the measured V r s and the V r s determined by reflecting these V l s in the estimated mirror surface were measured. The mean and standard deviation of these errors were 0.5552 o and 0.3285 o , respectively. Note that all angular errors are measured in the horizontal plane (u direction), in accordance with Eqn. 3.

After primary calibration, the mirror was deflected in three stages, with a 5mm deflection at each stage, and dynamic calibration was applied at each stage. Assessment of the accuracy of each dynamic calibration was made by again comparing directly measured V r s with V r s estimated from the calibrated model. An image of an active grid (spatio-temporally varying calibration patterns displayed on a flat-screen monitor [START_REF] Dunne | Efficient generic calibration method for general cameras with single centre of projection[END_REF]) was captured by the imager. The grid was then translated 100mm away from the mirror and imaged again. At each position, the world pose of the active grid was determined by reflection in a planar mirror [START_REF] Bonfort | General specular surface triangulation[END_REF]. Knowing the two poses of the active grid, and decoding the active grid data, the V r s for any image pixel can be determined. For 168 uniformly selected points in the image, their associated directly determined V r s were compared to the V r s estimated from the calibration data. The magnitudes of the angular errors are shown in Fig. 5, where the maximum directly determined ray deflections for stages 1, 2 and 3 were 4.1366 o , 8.4113 o , and 12.9171 o respectively. The mean and standard deviation of the angular errors, and the estimated deflection of the mirror determined from the calibration data, are shown in Table 1 for each stage of deflection. The estimated deflection values show good agreement with the actual deflections, with the maximum error being 7.76%. The reprojection errors at each stage for 160 corner points on a 252 × 261mm chessboard grid are also shown in Table 1. Note that the magnitude of the reprojection error is dependent on the distance of the calibration grid from the mirror (scene depth). Nevertheless, there is no apparent trend in any of the results in Table 1 to indicate that there is significant cumulative error in the dynamic calibration.

Finally, a distortion correction experiment was performed for the calibrated camera. Exactly perspectively correcting an image of a non-planar scene taken by a flexible mirror imager is not possible in the general case since the image is multi-perspective. However, by selecting a suitable synthetic plane a nearperspective image can be formed. Fig. 6 shows a face image from the flexible mirror imager both before and after such approximate perspective correction, in which the non-linear distortion of the face is seen to have been removed.

Future Directions

The calibration method presented in this paper applies to a constrained flexible mirror imager in which the mirror is deflected in only one dimension. Current work is focussing on generalising the framework to allow the imager to be calibrated for unconstrained deflections. Further work will examine auto-calibration for flexible mirror imagers, whereby existing scene points rather than calibration grid points are used in the dynamic calibration stage. This is a difficult problem as the unknown depth of scene points introduces a further degree of freedom into the calibration task.

Conclusion

Flexible mirror imaging systems provide unique benefits over existing imagers because they allow the field-of-view and the image resolution to vary within the image. However, the only existing method for specifically calibrating such systems operates offline by directly acquiring 3D mirror surface shape measurements for a very large set of possible mirror deformations. This paper investigates the fundamentals of a novel alternative method for the calibration of flexible imaging systems that does not require a significant offline stage. The proposed method facilitates the dynamic update of the current calibration after each mirror deflection. The flexible mirror, fixed in position along one edge and flexed in a single dimension, is modelled by a B-spline curve. A primary calibration stage determines the optical configuration for an initial mirror position. After the mirror flexes, dynamic calibration, requiring only a single image of a planar grid in each deflected mirror position, is applied to update the calibration. Simulated results demonstrate the performance of the method under variations in image noise, scene depth and mirror curvature, and validate the use of the linear estimate for initialising the non-linear minimisation process. Experiments with a real flexible imaging system demonstrates good results when compared to ground truth data. The presented principles of dynamic flexible mirror calibration and the proposed calibration method are important steps towards complete unconstrained calibration of flexible imaging systems.

Fig. 1 .

 1 Fig. 1. Flexible mirror imaging system.

Fig. 2 aFig. 2 .

 22 Fig. 2. Proposed flexible mirror imaging calibration method.

Fig. 3 .

 3 Fig.3. Error in linear and final mirror surface estimates for varying parameters. The simulated camera has focal length 26mm and image size 516 × 656 pixels. The flexible mirror is 420mm in length and is oriented at 67.5 o to the camera. Unless otherwise stated, the mirror deflection is 16.7mm, the scene depth is 1650mm and noise with 1 pixel standard deviation is added.

Fig. 4 .

 4 Fig. 4. Mirror surface estimates for real data before and after flexing.

Fig. 5 .

 5 Fig. 5. Angular errors in the calibrated Vrs across the image at each stage of deflection. Quadrilateral on ground plane indicates mirror boundary in image. Fixed mirror edge is rightmost side of quadrilateral.

Fig. 6 .

 6 Fig. 6. a) Face image from flexible mirror imager after 12mm deflection; b) Nearperspective face image with non-linear distortion removed using calibration data. Note the large distortion at top right of the image that was not captured by the calibration

Table 1 .

 1 Measured and estimated mirror deflections, and u direction angular errors between estimated and directly determined Vrs, for each deflection stage (SD = standard deviation).

	Stage No Deflection magnitude (mm) Angular error ( o ) Reprojection error (mm)
		Measured	Estimate		Mean (SD)	Mean (SD)
	1		5.0		4.7104	0.3578 (0.1751)	0.5122 (0.3027)
	2		5.0		4.6121	0.1088 (0.0833)	0.4446 (0.2486)
	3		5.0		4.7414	0.2318 (0.1477)	0.3992 (0.2582)
	500 -0.2 0 0.2 0.4 0.6 ) o Angular error (	1000	pix	100	600 pix 1100 0.2 0 0.4 0.6 500 -0.2	1000 pix	100	0 0.2 0.4 0.6 1100 pix 600 -0.2 500	1000 pix	100	1100 pix 600

Plastic wing mirror replacement mirror from www.carpointeurope.com

The non-specular back surface of the mirror was scanned instead of its specular surface, since a mirrored surface cannot be accurately measured by a laser scanner.

Acknowledgements

This research is funded by the Irish Research Council for Science, Engineering and Technology: funded by the National Development Plan.