Pablo Arrighi
email: pablo.arrighi@univ-amu.fr

Gilles Dowek
email: gilles.dowek@inria.fr

Free fall and cellular automata

published or not. The documents may come

Introduction

Three reasonable hypotheses-homogeneity in time and space, bounded velocity of propagation of information, and bounded density of information-lead to the thesis that physical phenomena can be described and simulated with cellular automata. This implication has in fact been formalized into a theorem both in the classical [START_REF] Gandy | Church's thesis and principles for mechanisms[END_REF] and the quantum case [START_REF] Arrighi | The physical Church-Turing thesis and the principles of quantum theory[END_REF], albeit in flat space.

Further evaluating this thesis leads to the project of selecting specific physical phenomena and attempting to describe them with cellular automata. In this work, we consider a particle upon which a constant force is applied-as induced by the first order approximation of a gravitational field. We do so in three different settings: Newtonian physics, Special Relativity, and General Relativity. We seek to capture each of these motions as a Cellular Automaton. The results are very different for these three theories.

Motion in cellular automata

Recall that the configurations of a 1D cellular automaton are functions from Z to a finite set of states Σ, which includes a distinguished quiescent state q. The evolution of the cellular automaton is a function F mapping configurations to configurations. It has to be causal and homogeneous, that is there must exist a radius r and a local function f such that for all i, (F (δ)) i = f (δ i-r , ..., , δ i-1 , δ i , δ i+1 , ..., δ i+r ).

Consider a temporal step ε, a spatial step ∆, and a discrete motion ỹ, that is a function from εN to ∆Z. A configuration δ is said to represent a particle at position k∆ if δ k = q and for all i = k, δ i = q. A transition function F represents a discrete motion ỹ if there exists an initial configuration δ such that for all k, F k (δ) represents a particle at ỹ(kε). A standard reference on cellular automata for constructing signals approximating different functions is [START_REF] Mazoyer | Signals in one-dimensional cellular automata[END_REF].

In each of the following three sections we proceed by first calculating the continuous motion y(t), that is the position as a function of time. We then construct the cellular automata for ỹ, if it exists. The differences between the three cases are highlighted.

Free fall in Newtonian physics

We begin with the standard case of Newtonian physics. The choice of units and notations will carry through in the next sections. In Newtonian physics, the gravitational force applied by a body of mass M upon a particle of mass M at a distance d is

F = G M M d 2
Multiplying both sides by G/c 4 , where c is the speed of light, introducing notations m = (G/c 2 )M which is the mass of the body expressed in meters, m = (G/c 2 )M which is the mass of the particle expressed in meters, and f = (G/c 4 )F which is the force expressed as a scalar without dimension, we get

f = m m d 2
Consider a particle whose initial distance to the body is R and initial velocity is zero and let y be such that d = R -y, we have

f = m m (R -y) 2 (1)
To define free fall, we approximate this force by

f = m m R 2 (2)
that is, introducing the notation g = m/R 2 f = m g For example, the mass of the Earth is M = 5.97 10 24 kg, so m = (G/c 2 )M = 4.42 10 -3 m. The radius of the Earth is R = 6.37 10 6 m, so g = m/R 2 = 1.09 10 -16 m -1 . Note that gc 2 = 9.81 ms -2 as expected.

When such a constant force is acting on a particle of mass M , its acceleration A is given the equation

M A = F
Multiplying both sides with G/c 4 and introducing the notation a = A/c 2 , which is the acceleration of the particle expressed in m -1 , we get

m a = f = m g thus a = g (3)
from which we get v = gt where v = V /c is the velocity expressed as a scalar with no dimension and t = cT is time expressed in meters, and

y = 1 2 gt 2
Thus the spacetime trajectory of this particle is a parabola. It is easy to prove that no cellular automaton can simulate such a motion: as the velocity of the particle increases linearly with time, the difference between y at some time step and at the next time step increases linearly with time. Thus, the evolution is not local. Moreover, to be able to compute y at the next time step from y at some time step, we need to know the velocity of the particle and it is then natural to express this velocity as part of the state of the cell. But then, as velocity is not bounded, the state space cannot be kept finite, even if velocity is defined with a finite precision.

Constant force in Special Relativity

In Special Relativity, neither of these problems occurs: velocity is bounded, hence the evolution is local. And if the velocity is known with a finite precision, a finite state space suffices. Still, another worry remains. If the velocity at some time step is computed from the velocity at the previous one, and both velocities are approximate, errors can accumulate. As we shall see, it is possible to circumvent this problem, and have a non divergent discretization of the trajectory of the particle.

In Special Relativity, the proper acceleration [START_REF] Taylor | Spacetime physics[END_REF] of a particle is

A = 1 1 -V 2 /c 2 3
dV dT dividing both sides by c 2 , we get

a = 1 √ 1 -v 2 3
dv dt

We assume that the force is as in Newtonian physics:

m a = f = m g, so a = g, that is 1 √ 1 -v 2 3 dv dt = g
This assumption, however, is now better understood as "constant force" than "free fall". We get

dv dt = g 1 -v 2 3 . (4)
Solving this equation, we get

v = gt 1 + (gt) 2 (5)
as the reader may check by differentiating ( 5) and comparing the result with (4) with v substituted by [START_REF] Inverno | Introducing Einstein's Relatvity[END_REF]. Then

y = 1 g ( 1 + (gt) 2 -1).
But, to prepare the case of General Relativity, we can also introduce a proper time τ such that

dt dτ = 1 √ 1 -v 2 .
Like v = dy/dt, we can introduce the velocity w = dy/dτ and we have

w = dy dτ = dy dt dt dτ = v √ 1 -v 2
and then

dw dτ = dw dv dv dt dt dτ = 1 √ 1 -v 2 3 g 1 -v 2 From w = v/ √ 1 -v 2 , we get v = w/ √ 1 + w 2 , thus dw dτ = g 1 + w 2 (6)
which is the equation of motion in terms of proper time.

In the same way, we have

dt dτ = 1 + w 2 (7)
y = 1 g ( 1 + (gt) 2 -1)
as expected.

Note that the velocity w = sinh(gτ ) goes to infinity when τ does. But the mapping from coordinate time to proper time τ = (1/g) arsinh(gt) slows down in such a way that the velocity v = gt/ 1 + (gt) 2 remains bounded by 1. Hence the particle never goes faster than light.

The spacetime trajectory of the particle is a branch of the hyperbola of equation (gy

+ 1) 2 -(gt) 2 = 1
Thus, in Special Relativity, the spacetime trajectory of a particle upon which a constant force is applied is not a branch of a parabola, but a branch of an hyperbola and the problem of modeling the motion of such a particle, with a cellular automaton, boils down to that of the approximability of a branch of hyperbola.

The branch of hyperbola

y = 1 g ( 1 + (gt) 2 -1)
has an asymptote

y = t - 1 g with whom the difference is y -y = 1 g ( 1 + (gt) 2 -gt) = 1 g( 1 + (gt) 2 + gt)
As expected, y -y decreases and goes to 0, when t goes to infinity. Moreover, if working with a space accuracy ∆, the hyperbola and its asymptote become indistinguishable at a time θ verifying

∆ = 1 g 1 + (gθ) 2 -gθ that is at time θ = 1 -(g∆) 2 2g 2 ∆
Consider an integer N and let ∆ = (1/g)/N . As N can be taken as large as we wish, ∆ can be taken as small as we wish. Consider the discretization of spacetime with a temporal and spatial step ∆. Consider the function ỹ from ∆N to ∆Z mapping every k∆ smaller than θ to the rounding of y(k∆) in ∆Z and every k∆ larger than θ to y (k∆

) = k∆ -(1/g) = (k -N )∆.
Let us construct a one-dimensional cellular automaton which represents the discrete motion ỹ. Set the state space Σ = {q, 0, ..., L -1, ∞}, with L = θ/∆ . Let us denote by c(k, σ) the configuration such that all cells are in state q except the cell k which is in state σ. If σ ∈ {0, ..., L -1}, the cellular automaton maps c(k, σ) to either c(k, σ + 1) or c(k + 1, σ + 1)-assuming (L -1) + 1 = ∞depending on whether ỹ((k + 1)∆) -ỹ(k∆) is equal to zero or to ∆, and c(k, ∞)

to c(k + 1, ∞).
Note that the internal state can be seen as a clock, the state k corresponding to the time k∆. It can also be seen as a representation of the momentum, as the momentum p = m v/ √ 1 -v 2 = m w = m gt grows linearly with time, the state k representing the momentum km g∆. The state ∞ corresponds to the case where momentum is large enough, so that its influence on velocity can be neglected, and the motion of the particle can be approximated by a uniform motion at the speed of light.

The number of states needed to simulate the spacetime trajectory is

l = 2 + θ ∆ = 1 2g 2 ∆ 2 + 3 2
If we assume that the number of bits that can be encoded in a cell of length ∆ is ∆/ρ, for some distance ρ, then, to encode log 2 (1/(2g 2 ∆ 2 ) + 3/2) bits, we need a cell of size ∆ such that

log 2 ( 1 2g 2 ∆ 2 + 3 2 ) ≤ ∆/ρ that is ∆/ρ -log 2 ( 1 2g 2 ∆ 2 + 3 2 ) ≥ 0
The function ∆/ρ -log 2 (1/(2g 2 ∆ 2 ) + 3/2) is monotonic in ∆, so this equation can be numerically solved. For example, if g = 1.09 10 -16 m -1 and ρ = 1.6 10 -35 m, this equation boils down to ∆ ≥ 5.11 10 -33 m = 320ρ

Indeed, if we take ∆ = 320ρ, a cell can encode 320 bits and l = 1.54 10 96 = 2 320 . So, with an accuracy of the order of magnitude of 10 -33 m, constant force in Special Relativity does not require a particle to contain more than a few hundred bits.

Free fall in General Relativity

In General Relativity, the gravitational effect of a body of mass M at a distance d = R -y is described by the metric tensor

g tt 0 0 -1 gtt where g tt = 1 -2m/(R -y).
The motion of a particle is described as a function mapping its proper time τ to a point in spacetime t(τ ) y(τ ) . The equations of this motion are [START_REF] Inverno | Introducing Einstein's Relatvity[END_REF]:

d 2 t dτ 2 + 2Γ
which are, respectively, the equation of motion in terms of proper time and that describing the relation between coordinate time t and proper time τ . "Constant force due to free fall" would make for a non-standard concept in General Relativity. On the one hand, a constant force of non-gravitational origin could indeed be applied in flat space and lead to the exact same computations as Special Relativity. On the other hand, free-falling could just mean following a geodesic trajectory in some more or less complicated metric-although not in a constant one. Indeed, making g tt constant as in the approximation of Equation ( 1) into ( 2) becomes an over-approximation, as the geodesics then become linear. So, we define this "first order approximated free fall" as the first non-trivial approximation of the metric tensor, that is we take a linear approximation of g tt as

g tt = 1 - 2m R - 2m R 2 y = 1 - 2m R - 2gy
where g = m/R 2 is the acceleration of gravity, as before. Introducing

y 1 = (1 -(2m/R))/2g we get g tt = 2g(y 1 -y)
In the same way, we approximate

dg tt /dy = -2m/(R -y) 2 by -2m/R 2 = -2g.
The equations of motion then become

dw dτ = g (9)
dt dτ = 1 2g(y 1 -y) 2g(y 1 -y) + w 2 (10)
Note the differences and similarities with the cases of the previous settings.

The equation describing the relation between coordinate time and proper time, that is Equation (10) or (8) does coincide with that of Special Relativity, that is Equation [START_REF] Mazoyer | Signals in one-dimensional cellular automata[END_REF], in the flat spacetime case when g tt = 1. But the equation of motion, that is Equation (9), coincides not with Special Relativity, that is Equation ( 6), but with Newtonian physics, that is (3). Integrating Equation (9), we get

w = gτ and y = 1 2 gτ 2
Equation (10) then becomes

dt dτ = 1 2g(y 1 -(1/2)gτ 2 ) 2g(y 1 -(1/2)gτ 2 ) + (gτ ) 2 dt dτ = y 1 2g 1 y 1 -(1/2)gτ 2
Integrating it, we obtain

t = 1 g artanh(τ g 2y 1 ) τ = 2y 1 g tanh(gt)
and finally

y = 1 2 gτ 2 = y 1 (tanh(gt)) 2
Note that the velocity

v = dy dt = 2gy 1 tanh(gt)(1 -(tanh(gt)) 2 ) = (1 - 2m R ) tanh(gt)(1 -(tanh(gt)) 2 )
is bounded by 1, hence the particle never goes faster than light. Like in Special Relativity, the velocity w = gτ goes to infinity when τ does and the mapping from coordinate time to proper time

τ = 2y 1 g tanh(gt)
slows down in such a way that the velocity v is bounded by 1. Moreover, unlike in Special Relativity, when t goes to infinity, τ has a finite limit 2y 1 /g. Thus, an infinite amount of coordinate time corresponds to a finite amount of proper time. As a consequence, with respect to coordinate time, after an acceleration phase, the particle decelerates and has a limit position y 1 .

The distance to the limit at time t is

y 1 -y = y 1 (1 -(tanh(gt)) 2 )
As expected, y 1 -y decreases and goes to 0 when t goes to infinity. Moreover, if working with a space accuracy of ∆, the position and its limit become indistinguishable at a time θ verifying

∆ = y 1 (1 -(tanh(gθ)) 2 )
that is at time

θ = 1 g artanh( 1 - ∆ y 1 )
Consider a distance ∆ that can be taken as small as we wish. Like in the case of Special Relativity, consider the discretization of spacetime with a temporal and spatial step ∆ and the function ỹ from ∆N to ∆Z mapping every k∆ smaller than θ to the rounding of y(k∆) in ∆Z and every k∆ larger than θ to y 1 .

Let us construct a one-dimensional cellular automaton which represents the discrete motion ỹ. Set the state space Σ = {q, 0, ..., L -1, ∞}, with L = θ/∆ . If σ ∈ {0, ..., L -1}, the cellular automaton maps c(k, σ) to either c(k, σ + 1) or c(k + 1, σ + 1)-assuming (L -1) + 1 = ∞-depending on whether ỹ((k + 1)∆) -ỹ(k∆) is equal to zero or to ∆, and c(k, ∞) to itself.

The number of states needed to simulate the spacetime trajectory is

l = 2 + θ/∆ = 2 + 1 g∆ artanh( 1 - ∆ y 1 )
If we assume that the number of bits that can be encoded in a cell of length ∆ is ∆/ρ, for some distance ρ, then, to encode this amount of information, we need a cell of size ∆ where

log 2 (2 + 1 g∆ artanh( 1 - ∆ y 1 )) ≤ ∆ ρ that is ∆ ρ -log 2 (2 + 1 g∆ artanh( 1 - ∆ y 1 )) ≥ 0
This function is monotonic in ∆, so this equation can be numerically solved. For example, if m = 4.42 10 -3 m, R = 6.37 10 6 m, and ρ = 1.6 10 -35 m, we get g = 1.09 10 -16 m -1 and y 1 = 4.57 10 15 m. This equation boils down to ∆ ≥ 2.69 10 -33 m = 168ρ

Indeed, if we take ∆ = 168ρ, a cell can encode 168 bits and l = 1.92 10 50 = 2 168 . So, with an accuracy of the order of magnitude of 10 -33 m, General Relativity also does not require a free falling particle to contain more than a few hundred bits.

Conclusion

Newtonian physics and Relativity completely differ with respect to the possibility modelling free fall within a cellular automaton. Such a simulation is not possible for Newtonian physics, while it is possible both in Special-constant force-and General Relativity-geodesics in a linearly approximated metric. The simulation can be very accurate with a reasonable number of internal states: a few hundred bits suffice to achieve an accuracy of the order of magnitude of 10 -33 m. So, as far as free fall is concerned, Relativity is completely consistent with the hypotheses of a bounded velocity of propagation of information and of a bounded density of information, unlike Newtonian physics.

In this work, we made explicit these accurate cellular automata, by exploiting the asymptotes to the trajectory, that exist both in Special and General Relativity. There was no need to use auxiliary signals as in [START_REF] Mazoyer | Signals in one-dimensional cellular automata[END_REF]. We have proved the existence of such cellular automata, but made no attempt to design "natural" ones: the local rules use the solutions of the equations of motion in order to know whether the particle should move, or not. Moreover, there was clearly no attention paid to covariance. The design of more natural automata is of course of prime importance. In the case of General Relativity for instance, the metric at each point ought to be carried by the corresponding cell: we began to address this question both in the classical case [START_REF] Arrighi | Discrete geodesics[END_REF], and, building upon [START_REF] Di Molfetta | Quantum walks as massless dirac fermions in curved space-time[END_REF], in the quantum case [START_REF] Arrighi | Quantum walks in curved spacetime[END_REF].

  Note that adding this third equation permits to drop the first, because differentiating the third equation and using the second to replace d 2 y/dτ 2 by -(1/2)(dg tt /dy)(g tt (dt/dτ ) 2 -(1/g tt )(dy/dτ ) 2 ) yields the first. Using this third equation, the second can also be simplified to

		dt dτ	=	1 g tt		g tt + w 2
									t yt	dt dτ	dy dτ	= 0
	d 2 y dτ 2 + Γ y tt (	dt dτ	) 2 + Γ y yy (	dy dτ	) 2 = 0
	where			Γ y tt =	1 2	g tt	dg tt dy
		Γ y yy = -	1 2	1 g tt	dg tt dy
	Γ t yt = Γ t ty =	1 2	1 g tt	dg tt dy
	are the non-zero Christoffel symbols corresponding to this metric tensor, that is
	d 2 t dτ 2 = -	1 g tt	dg tt dy	dt dτ	dy dτ
	d 2 y dτ 2 = -	1 2	dg tt dy	(g tt (	dt dτ	) 2 -	1 g tt	(	dy dτ	) 2 )
	to which we can add a third equation expressing that τ is a proper time
	g tt (	dt dτ	) 2 -	1 g tt	(	dy dτ	) 2 = 1
				d 2 y dτ 2 = -	1 2	dg tt dy
	Thus, introducing the velocity w = dy/dτ , the equations of motion boil down
	to the two equations
				dw dτ	= -	1 2	dg tt dy

Acknowledgements

The authors thank David Janin for useful discussions on this paper, and Alejandro Péres for indications on General Relativity. This work has been funded by the ANR-12-BS02-007-01 TARMAC grant. Pablo Arrighi is also a member of IXXI, where this research was partially conducted.


Don Joven Agravante

Giovanni Claudio

Fabien Spindler

Franc ¸ois Chaumette

Visual servoing in an optimization framework for the whole-body control of humanoid robots

Keywords: Visual servoing, optimization and optimal control, humanoid robots

In this paper, we show that visual servoing can be formulated as an acceleration-resolved, quadratic optimization problem. This allows us to handle visual constraints, such as field of view and occlusion avoidance, as inequalities. Furthermore, it allows us to easily integrate visual servoing tasks into existing whole-body control frameworks for humanoid robots, which simplifies prioritization and requires only a posture task as a regularization term. Finally, we show this method working on simulations with HRP-4 and real tests on Romeo.

I. INTRODUCTION AND BACKGROUND

V ISUAL servoing is a method of control that directly incorporates visual information into the control loop [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]. It has been shown that it is effective in various robotics applications. In particular, it is often used for gaze control and grasping with humanoids [START_REF] Mansard | Visually-Guided Grasping while Walking on a Humanoid Robot[END_REF]- [START_REF] Claudio | Vision-based manipulation with the humanoid robot Romeo[END_REF]. On the other hand, optimization is a process that seeks to obtain the minimum (or maximum) of a particular objective under some constraints [START_REF] Nocedal | Numerical Optimization[END_REF]. Optimization algorithms have proven to be a good framework for control, especially on complex systems such as humanoid robots. This is evidenced by various groups, working on humanoids, adapting similar optimization-based approaches [START_REF] Saab | Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints[END_REF]- [START_REF] Feng | Optimizationbased Full Body Control for the DARPA Robotics Challenge[END_REF]. The purpose of this paper is to show how visual servoing can be effectively used within these optimization frameworks. In doing so, we can gain the advantages of the optimization approaches (e.g., handling multiple tasks and constraints), yet retain the main research results of the visual servoing community. Specifically, we want to keep using the various visual features and the corresponding Jacobians that are well suited to be used in control, such as those collected and implemented in [START_REF] Marchand | ViSP for visual servoing: a generic software platform with a wide class of robot control skills[END_REF].

In the literature related to visual servoing, the problem of enforcing constraints has been one of the key issues because of the need to keep features in the field of view and avoid occlusions. In the state of the art, these are usually handled by firstly planning a path/trajectory in image-space and then tracking the result with visual servoing [START_REF] Mezouar | Path planning for robust image-based control[END_REF]- [START_REF] Kazemi | Randomized Kinodynamic Planning for Robust Visual Servoing[END_REF]. One way to do the planning phase is with optimization [START_REF] Hafez | Path planning approach to visual servoing with feature visibility constraints: A convex optimization based solution[END_REF]. So by using optimization for visual servoing itself, we can proceed without the planning phase but still handle constraints. This was one of the main ideas in [START_REF] Allibert | Predictive Control for Constrained Image-Based Visual Servoing[END_REF] which is probably the closest related work. In [START_REF] Allibert | Predictive Control for Constrained Image-Based Visual Servoing[END_REF], visual servoing with constraints is formulated as a Model Predictive Control (MPC) problem that is solved in an optimization framework. This was validated in a simulation of a free-flying camera and four image points for features. Compared to [START_REF] Allibert | Predictive Control for Constrained Image-Based Visual Servoing[END_REF], we firstly pose a novel second-order model. This is important to be consistent with dynamics. Secondly, we do not use predictive control and the cost function is formed differently such that we can use quadratic optimization, avoiding the complexities of using non-convex optimization. MPC implies an additional computational cost since the state and control vectors are multiplied by N previewed steps. We choose to avoid this for now. Meanwhile, the cost function formulation allows us a wider choice of convergence properties as motivated in Sec. IIA. Thirdly, we use the model to form inequalities for both the field of view constraint and occlusion avoidance. Finally, we show that our formulation integrates well to existing task objectives and constraints used to control a humanoid platform. There are also other works similar to [START_REF] Allibert | Predictive Control for Constrained Image-Based Visual Servoing[END_REF]. Notably, [START_REF] Garcia | Vision-guided motion primitives for humanoid reactive walking: Decoupled versus coupled approaches[END_REF] had similarly framed the visual servoing problem in an MPC and optimization context. This was integrated inside of an existing MPC for walking pattern generation (WPG) of a humanoid. Being an MPC formulation of visual servoing, our novelty claims relative to it are the same except for application to a humanoid platform. In regards to this, we create the visual servoing objectives and constraints directly in the whole-body controller while [START_REF] Garcia | Vision-guided motion primitives for humanoid reactive walking: Decoupled versus coupled approaches[END_REF] does it in the WPG. Historically, the WPG is solved ahead of time and separately so that it can produce reference trajectories for the whole-body controller to track [START_REF] Kajita | Biped walking stabilization based on linear inverted pendulum tracking[END_REF]. Recently, [START_REF] Sherikov | Whole body motion controller with long-term balance constraints[END_REF] has shown that both problems can be solved together. Leveraging this result, we argue that formulating visual servoing as just another task/constraint for the whole-body controller is better. Not only does it ensure consistency with actuation constraints (e.g., joint limits), it also simplifies the overall framework for task prioritization, i.e., both walking and visual servoing are just another set of tasks/constraints in the same optimization problem.

The rest of the paper is structured as follows. Sec. II defines the base formulation. We then show how inequalities can naturally represent the field of view and occlusion constraints in Sec. III. Simulation results are presented in Sec. IV and Sec. V shows some tests on a real humanoid robot. Finally, Sec. VI concludes and outlines some possible future works.

II. BASE FORMULATION

Classically, visual servoing techniques use a first-order motion model of the visual features and the Moore-Penrose pseudoinverse to solve the system of equations, formally:

ė =L e v, (1)
v = -λ L + e e, (2)
where e is an error vector of the chosen visual features, v the velocity of the camera, L e is the visual feature's Jacobian (or interaction matrix) with L + e the pseudoinverse of its estimate (denoted by the hat), and finally λ is a gain to be tuned. Notice the two parts: the model ( 1) and the control law [START_REF] Mansard | Visually-Guided Grasping while Walking on a Humanoid Robot[END_REF]. A primary idea of this paper is that re-using ( 1) is beneficial while (2) can be done more generally by optimization. Using the pseudoinverse for velocity-resolved control is not unique to visual servoing. In fact, the exact same method can be found in general robotics literature as instantaneous inverse kinematics. A useful parallel can be drawn between the pseudoinverse and optimization, as explained in [START_REF] Kanoun | Kinematic Control of Redundant Manipulators: Generalizing the Task-Priority Framework to Inequality Task[END_REF], where an equivalent optimization problem can be created. For example, (2) is the same as:

v = argmin v v 2 subject to L e v = -λe. (3)
In fact, several parallels can be drawn between classical robot control approaches and optimization, as presented in [START_REF] Malis | Improving vision-based control using efficient second-order minimization techniques[END_REF].

Contrary to [START_REF] Malis | Improving vision-based control using efficient second-order minimization techniques[END_REF], which focused on a novel method to solve the optimization problem, this paper concentrates on the problem construction while leveraging optimization solvers such as those explained in the reference text [START_REF] Nocedal | Numerical Optimization[END_REF].

To start detailing the base formulation, let us first generalize e to be any task vector in the operational space, then we define the function, f e , that maps the joint positions q to this space:

e = f e (q). (4)
A classical example is using Cartesian space to define e and the function f e is known from forward kinematics. Assuming f e is twice differentiable, we can define:

ė =J e q, (5)
ë =J e q + Je q,

where J e is known as the task Jacobian. Note how (5) has a similar form to (1). Furthermore, (6) has a well known form and can be used for effective Cartesian space control in an optimization framework [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF], [START_REF] Bouyarmane | Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations[END_REF]. Returning first to [START_REF] Nocedal | Numerical Optimization[END_REF], it can be related to (1) by expanding into:

ė = L e J p q, (7)
by setting J e = L e J p . Here, J p is the Jacobian of a corresponding robot frame p. For example, if p is the camera frame, then [START_REF] Koenemann | Whole-body model-predictive control applied to the HRP-2 humanoid[END_REF] is equal to [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]. Continuing, (6) then becomes: ë = L e J p q + L e Jp q + Le J p q.

We now have [START_REF] Koenemann | Whole-body model-predictive control applied to the HRP-2 humanoid[END_REF] and [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF], in the same form as ( 5) and [START_REF] Saab | Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints[END_REF]. With these, we can formulate an optimization problem consistent with the framework of [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF].

A. Quadratic programming objective

Recall that a general optimization problem can be written as finding x such that:

argmin x f o (x) subject to f c (x) ≤ 0, (9)
where f o (x) is the objective function and the inequality f c (x) ≤ 0 is the constraint which is infinitely more important than the objective. A constrained quadratic programming (QP) problem can be formulated when:

f o (x) = 1 2 x Qx + c x, (10)
f c (x) =Ax -b. (11)
For example, (3) is a QP by defining x = v. The objective function can then be formed like [START_REF] Kuindersma | Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot[END_REF] where Q is an identity matrix and c is a vector of zeros. For the constraint, the general form of ( 9) is an inequality. But we can form equality constraints, such as that in (3), with inequalities by creating artificial upper and lower limits which are equal. Doing this for (3), we can recover the form of [START_REF] Johnson | Team IHMC's Lessons Learned from the DARPA Robotics Challenge Trials[END_REF] where A = [ L e -L e ] and b = [-λe λe ] . However, most QP solver interfaces can explicitly handle equalities, so this is not always needed. For example, active set strategies [START_REF] Nocedal | Numerical Optimization[END_REF] can benefit from knowing there is one equality that is always active rather than two inequalities to regularly check. Once we have a well-formed QP, it can be solved reliably [START_REF] Nocedal | Numerical Optimization[END_REF]. To simplify the explanations that follow, we define the argument of the optimization, x = q, being consistent with the accelerationresolved framework [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF].

A QP is useful since we can define objectives as Euclidean norms. For example, a commonly used cost function is that of a tracking control objective:

f o (x) = 1 2 k(e des -e) + b( ėdes -ė) + (ë des -ë) 2 , ( 12
)
where e des , ėdes , ëdes define a desired trajectory in the task space and k, b are gains to tune. Note that this is a design choice. This corresponds to the design choice of ė = -λe in the basic visual servoing control law of (2). The advantage of ( 12) is that it corresponds to a mass-spring-damper system and can be tuned as such. For example, normally we set b = 2 √ k for a critically damped behavior. A particular case of [START_REF] Feng | Optimizationbased Full Body Control for the DARPA Robotics Challenge[END_REF] consists in positioning all the joints such that e = q and J e = I, along with defining ėdes = ëdes = 0 so that:

f o (x) = 1 2 k(q des -q) -b q -q 2 . ( 13
)
We will refer to this as a posture task similar to [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF]. For humanoid robots having many joints, this is used as a lowpriority task to make sure the QP solution is unique.

For visual servoing, we substitute ( 7), ( 8) into ( 12) to recover the form of [START_REF] Kuindersma | Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot[END_REF] where:

Q =J p L e L e J p , c = -J p L e (k(e des -e) + b( ėdes -L e J p q) + ëdes -L e Jp q -Le J p q). (14)
Note that this differs from (3) because the control law is used as an objective function instead of a constraint. However, if we use a slack variable, s, in (3) for relaxing the constraint, then the constraint effectively becomes an objective:

v = argmin v,s v 2 + w s 2 subject to L e v -s = -λe, (15)
where w is a weight used to adjust the priority. The slack variable trick can be applied to any constraint, including inequalities [START_REF] Kanoun | Kinematic Control of Redundant Manipulators: Generalizing the Task-Priority Framework to Inequality Task[END_REF]. So the difference between an objective and constraint is effectively only the priority. The design choice of lessening the importance of the visual servoing solution fits humanoid robots that already have several constraints that are more important. Another change is the use of the posture task [START_REF] Marchand | ViSP for visual servoing: a generic software platform with a wide class of robot control skills[END_REF] in place of the velocity norm objective. Lastly, note that adding several independent QP objectives together results in the same QP form of ( 10) so ( 15) can be written as a QP similarly to (3).

B. Particularities of visual servoing

To use [START_REF] Mezouar | Path planning for robust image-based control[END_REF], we need to detail some variables. Firstly, e is defined as one of the visual servoing features from the literature -e.g., point, line, circle, image moments, luminance, etc. [START_REF] Marchand | ViSP for visual servoing: a generic software platform with a wide class of robot control skills[END_REF]. These come together with a definition of the corresponding interaction matrix, L e . Recall that we can stack the features and Jacobians [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]. Although this is possible, a better way in the optimization framework is to define a separate task. This allows better handling of prioritization -whether weights [START_REF] Bouyarmane | Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations[END_REF], a hierarchy [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF], or both are used. Next, a robot body part, p, (or an associated surface) is selected to be the servo end point. This similarly comes with the Jacobian, J p . Note that a slight modification of the Jacobian is needed in the case of eye-to-hand systems as opposed to eye-in-hand systems that servo the camera body as illustrated in [START_REF] Chaumette | Visual servo control, Part II: Advanced approaches[END_REF]. Lastly, we are missing the definition of Le . An approximation can be made that the term Le J p q is negligible in the context of ( 14). However, it is possible to obtain Le as shown next.

One of the most common and simplest image-based features is the point. It is defined by: e

= x y = X/Z Y /Z , (16)
where the 3D coordinates of the point in reference to the camera frame are {X, Y, Z}, where Z is the depth. The equivalent pixel-space coordinates are easily obtained with the camera intrinsic parameters [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]. Its corresponding Jacobian is:

L e = -1 Z 0 x Z xy -(1 + x 2 ) y 0 -1 Z y Z 1 + y 2 -xy -x . (17)
Taking the time derivative we get:

Le = Ż Z 2 0 ẋZ-x Ż Z 2 ẋy + x ẏ -2x ẋ ẏ 0 Ż Z 2 ẏZ-y Ż Z 2 2y ẏ -ẋy -x ẏ -ẋ . ( 18
)
The image point derivatives { ẋ, ẏ} can be obtained from [START_REF] Koenemann | Whole-body model-predictive control applied to the HRP-2 humanoid[END_REF] while Ż is obtained by:

Ż = 0 0 -1 -yZ xZ 0 v, (19)
which comes from the spatial velocity definition. Another example comes from the class of pose-based features. The relative translation feature is defined as:

e = d t p , (20)
which is a 3-dim vector corresponding to the translation of the robot body part, p, with d, the target/desired frame used as the reference. Its Jacobian is:

L e = d R p 0 , (21)
which contains the corresponding rotation matrix. The derivative of a rotation matrix can be associated to the angular velocity placed in skew-symmetric matrix form [ω] × such that:

Le = ḋR p 0 = (-[ω] × d R p ) 0 . (22)

III. INEQUALITY FORMS: FIELD OF VIEW MAINTENANCE AND OCCLUSION AVOIDANCE

In visual servoing, we often want to formulate additional tasks relating to the visibility of features. These can be broadly classified into two. The first is to ensure that features remain within the field of view. In fact, a control law driving the feature error to zero does not explicitly prevent the features from leaving the field of view during the transition. Secondly, there are often other objects (or even the robot's other body parts) that have the possibility to block the field of view causing an occlusion, which we want to avoid. In these two cases, there is no specific target, making it difficult to formulate the problem as an equality, as the case in Sec. II. However, both of these can be easily formed as inequalities. Recall also that with slack variables the inequality does not need to be a strict constraint [START_REF] Kanoun | Kinematic Control of Redundant Manipulators: Generalizing the Task-Priority Framework to Inequality Task[END_REF]. The question then becomes that of correctly prioritizing the different tasks, which we leave up to the designer of specific use cases.

Maintaining the field of view implies that the feature of interest, e fov , should remain inside some defined image bounds such that:

e ≤ e fov ≤ e,

where e, e symbolizes the lower and upper limits respectively. Contrarily, avoiding occlusion can be formulated by defining e occ as some features related to the occluding object. We then desire to keep it outside of certain image bounds such that:

e occ ≤ e or e ≤ e occ . (24)
These behaviors can be formulated similarly. Recall that inverting the inequality amounts to simply negating both sides and that the task vector can be stacked. So without loss of generality, the following explanations only show a single direction of the FoV constraint, that is: e ≤ e.

A. Base inequality formulation

Any appropriate task vector can be used to formulate an inequality, as it is with the objective function. Recall that we want to have a second-order form. For this, a second-order approximation can be defined:

e k+1 = e k + ėk ∆t + 1 2 ëk ∆t 2 , ( 25
)
where ∆t is a time step from discretization. We can then constrain this by: e k+1 ≤ e.

We can get the linear form needed by first substituting [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF] into [START_REF] Chaumette | Visual servo control, Part II: Advanced approaches[END_REF]. Doing so also removes the need for the subscript k, which we drop to be concise. Next, we use [START_REF] Koenemann | Whole-body model-predictive control applied to the HRP-2 humanoid[END_REF][START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF] in [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF] to obtain the joint space expression. Finally, recalling that we use x = q, we recover the form of [START_REF] Johnson | Team IHMC's Lessons Learned from the DARPA Robotics Challenge Trials[END_REF] where:

A = 1 2 L e J p ∆t 2 , b =e -e k -L e J p q∆t - 1 2 ∆t 2 L e Jp q + Le J p q . ( 27
) Almost all other terms were detailed before. The only thing left to define are the limits, in this case e. For example, let us first define e to be image points as in [START_REF] Shen | Visual Servoing Path Planning for Cameras Obeying the Unified Model[END_REF]. This is a common and versatile definition because it can be extended by sampling the object of interest with several different points. Since we are concerned with visibility, the limits are best described in pixel space. For example, to define x such that it corresponds to the image border in pixel space, u, we have:

x = u -c x f x , (28)
where c x is the principal point, f x is the focal length, both of which are obtained from calibration of the intrinsic camera parameters. This can be done similarly for y and lower limits. Finally, note that (26) can be viewed as a 1-step preview horizon, making it similar to the form used in [START_REF] Allibert | Predictive Control for Constrained Image-Based Visual Servoing[END_REF]. Because of this, it also has the same disadvantages of not being very stable in a numerical sense. The solution in [START_REF] Allibert | Predictive Control for Constrained Image-Based Visual Servoing[END_REF] is to extend the preview horizon. This improves performance but has the disadvantage of extra computational cost (particularly since the number of constraints is increased). Differently, we improve the performance by slightly reforming (26) as shown next.

B. Augmenting the behavior by avoidance functions

Usually it is better to avoid a hard constraint rather than wait for it to activate (often violently). We can replace the hard constraint in (26) by an avoidance function, f (e, e). Another improvement can be made by constraining only the update (i.e., velocity and acceleration). Using both we have:

ėk ∆t + 1 2 ëk ∆t 2 ≤ f (e, e). ( 29
)
Using a parametrization of the avoidance function commonly used in [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF]:

ėk ∆t + 1 2 ëk ∆t 2 ≤ ξ e k -e s e i -e s -ξ off if e k > e i , (30)
where e s is a safety bound, such that e s = e -δ , while e i is an interactive bound which defines the activation boundary of the constraint, so e i = e -i where i > δ , finally ξ is a tunable gain for the avoidance behavior and a small offset ξ off ensures that the robot is moving away from the constraint direction instead of keeping still (zero update). We can now change the QP with this. Note that A retains its form. Then:

b =ξ e k -e s e i -e s -ξ off -L e J p q∆t -1 2 ∆t 2 L e Jp q + Le J p q .

(31)

Finally, note that the constraint activation condition needs to be handled explicitly in the implementation, by adding the constraint when e k > e i and removing it otherwise.

IV. SIMULATION RESULTS

This section shows some representative examples of our verification tests with simulations using the HRP-4 robot model with a 5 ms control loop. These are shown as part of the accompanying video (sped up only due to video length). In all of these, we have some essential whole-body control tasks in addition to the visual servoing tasks described. The required tasks can be generally described as:

• maintaining balance (e.g., dynamics consistency, center of mass control) • actuation limits (e.g., joint position and torque limits) • maintaining contacts (e.g., null contacting body acceleration, keeping within the static friction cone, unilaterality of force) • self-collision avoidance • default posture task (e.g, Eq.( 13)) Because whole-body control is still a very active area of research, different teams use various formulations as can be seen in some examples from the literature [START_REF] Saab | Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints[END_REF]- [START_REF] Feng | Optimizationbased Full Body Control for the DARPA Robotics Challenge[END_REF]. Here, we are using the same formulation as [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF]. The objective functions are combined using a weighted sum:

f total (x) = n i=1 w i f i (x). (32)
A guideline to tune the weights to produce a pseudo-hierarchy is:

w i min(f i (x)) > w i-1 max(f i-1 (x)), (33)
where task i has a higher priority than task i -1, and min() and max() represent minimal and maximal function values according to the expected/acceptable task error values. Typically, we have prioritized: (1) center of mass control, (2) visual servoing, and (3) posture, for the objectives while actuation limits, maintaining contacts, dynamics consistency and self-collision avoidance are explicit constraints. The interested reader can refer to [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF] for more implementation and technical details on the QP (e.g., solvers and runtime). However, the visual servoing tasks presented here can be adapted easily to fit with other optimization-based whole-body control frameworks. Furthermore, a strict hierarchy [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF] is also possible instead of ( 32) and (33).

A. Gaze with occlusion avoidance

In this simulation, we show how the gaze can be controlled simply by centering a single point feature, defined by ( 16), in the image. Since the feature is defined in image space without extrapolating the object pose, the method falls into Image-Based Visual Servoing (IBVS). Fig. 1 shows this demonstration. The feature is the 2D projection of the usercontrolled interactive marker. Additionally, a simulated wall serves as an occluding object to the left of the robot. For simplicity, we assume prior knowledge of its location. To define the occluding features, the wall edge closest to the robot is sampled with point features. The task limits are the edge of the image border (this is then augmented with the safety and interactive margins). In the accompanying video, the gaze control motion without occlusion is shown first by moving the object to the right of the robot. This corresponds to about 3 to 8 sec of the plot in Fig. 2. Contrarily, moving the object to the left of the robot can result in an occlusion by the wall as seen in Fig. 1. This corresponds to around 9 sec onwards of the plot in Fig. 2. The occlusion avoidance forces the robot to lean forward, noticeably using the torso and leg joints to gaze while maintaining balance (Center of Mass control). Additionally note how a small part of the wall enters the bottom-left corner of the simulated image inset in Fig. 1 (gray triangle on the bottom left). This portion is in between the sampled points. Although adding more points can always be done, it is also possible to use other image features such as line segments, that can better represent the object.

B. Hand servoing with modeling errors

Visual servoing is well-known for its robustness to some modeling errors. To show that this is still the case, we simulated a constant offset of 0.5 radian on the right shoulder pitch. This can be seen in Fig. 3. There are two different robot models here. The opaque one is the real robot model. The transparent one is the wrong robot model which has the offset in the right arm pose representing the mistaken internal knowledge. In this simulation, we used the wrong information for all the computations requiring the robot model, e.g., the robot Jacobians in [START_REF] Mezouar | Path planning for robust image-based control[END_REF]. However, we update the visual servoing task (error and feature Jacobians) with the real pose, simulating the information provided from a visual pose estimate. For the control, we use Pose-Based Visual Servoing (PBVS) on the right wrist with the translation feature of (20) and a

C. Combining with walking

Walking (and other locomotion modes) amounts to controlling the floating base of the humanoid while maintaining balance. This implies that contact states (footsteps) are handled. In this demonstration, we are using a WPG implementation with a reference velocity as an input [START_REF] Agravante | Walking pattern generators designed for physical collaboration[END_REF], [START_REF] Herdt | Online walking motion generation with automatic footstep placement[END_REF]. To use this together with our visual servoing tasks, a simple but effective method consists of defining the WPG inputs as a function of the visual servoing task errors. For example, if we have a PBVS task for the right hand and an IBVS gaze task (as in the previous simulations) then we can define the WPG reference as:

ċxref =k v (t xPBVS ), ċyref =0, θref = -k θ (θ gaze ),
where the WPG input reference velocities are ċxref , ċyref , θref , while t xPBVS is the translation part of the hand PBVS task corresponding to the x axis of the WPG frame (local frame on the robot), θ gaze is the yaw angle of the gaze frame relative to the current WPG frame and k v , k θ are gains to tune. The idea is that the hand PBVS will guide walking forward. Walking sideways is not used. The gaze IBVS orients the robot such that it faces straight at the object. Finally, note that bounds are needed for t xPBVS and θ gaze that are used in the coupling. When a bound is exceeded, we use: e = e max(e) where e is the unbounded error, max(e) is the largest limit violation and e is the result used. This preserves the vector direction. Fig. 5 shows a screenshot from the demonstration. Fig. 6 shows the relevant WPG control inputs: ċxref , θref . The start of the plot of Fig. 6 from time 0 to around 4 sec shows the clipping of the control due to the bound of 0.3 m/s on ċxref . After this, up to around 35 sec, we show how the tasks converged to a fixed goal. Next, from around 35 sec onwards, we see how the tasks converged when the interactive goal was moved. Furthermore, there is a small oscillation (especially when the task is close to converging). This is due to the conflicting tasks (since we are not using strict hierarchies). Specifically, the visual servoing task designed in this paper conflicts with the Center of Mass servoing with references generated by the WPG. Although the coupling laws designed here seek to resolve this conflict, having a separate solver for the WPG means it cannot be fully resolved in this manner.

Although this ad hoc coupling is effective in this demonstration, it has some clear drawbacks. The purpose here was to show further that visual servoing can be used as just another whole-body task implying that it can work together seamlessly with balance control and changing contact states. For a more rigorous integration of walking, we think that methods such as [START_REF] Sherikov | Whole body motion controller with long-term balance constraints[END_REF] which consider all the conflicting tasks together could be a more suitable approach to the problem. However, since this requires a larger effort, we have opted to leave this out of the scope of this work. For validation on a real robot, we are using Romeo, a 37 DOF humanoid from SoftBank Robotics. In these tests, we used the left eye camera with a resolution of 320x240 at 30 Hz.

For the interface to Romeo, we used a velocity controller provided by SoftBank Robotics running at 10 Hz. The computed acceleration commands are numerically integrated to provide the required velocity commands.

A. Gaze control with Romeo

In this test, Romeo detects and tracks a moving visual target where an IBVS gaze task is used to keep the target in the center of the image (see Fig. 7). In this case, the circular target is a single marker from the open-source marker-based localization system called WhyCon [START_REF] Krajník | A Practical Multirobot Localization System[END_REF]. Fig. 8 shows the IBVS task error throughout the test. We can see that the absolute task error always remains less than 0.3 despite the target being moved (in this case, no feedforward prediction of the motion was added either). Because of modeling errors, (i.e. intrinsic and extrinsic camera parameter errors and Romeo's mechanical model errors), we had to prevent overshooting by increasing the damping ratio of our control law (12) from 1 (used in simulations for a critically damped convergence) to 2.7, such that b = 5.4

√

k. Furthermore, along with the common tasks listed at the beginning of Sec. IV, two tasks were added for show. Although these do not help the visual servoing tasks in any way, they subjectively improve what appears in the video and screenshots. Firstly, an orientation task for the head is added to minimize the unnatural-looking head tilting. Secondly, another IBVS task is defined so that the right eye tracks the target as well (the eyes are independently actuated).

B. Grasping with Romeo

In this test, Romeo has to grasp a box (see Fig. 9). To do this, PBVS is used to move the hand towards a desired pose that is defined relative to the pose of the box. The box is also moved after the initial convergence, as seen in the video as well as the plot of Fig. 10 after around 15 sec. Simultaneously, IBVS is used for the gaze, as in Sec. V-A, this time keeping a point relative to both the box and hand as a target, similar to [START_REF] Agravante | Visual servoing for the REEM humanoid robot's upper body[END_REF], [START_REF] Claudio | Vision-based manipulation with the humanoid robot Romeo[END_REF]. Here, it is necessary to visually track both the hand and box. The ViSP [START_REF] Marchand | ViSP for visual servoing: a generic software platform with a wide class of robot control skills[END_REF] library is used for these. Specifically, we used the ViSP blob detection and pose estimation algorithm on the hand marker and the Model Based Tracker (MBT) for the box. This demonstration turned out to be particularly challenging with Romeo. Throughout the tests, we faced a problem of reduced stiffness in the knee joints, imposed by an internal temperature protection of the motors, to which we do not have access. This caused unwanted oscillations whenever it occurs. This can be noticed in the accompanying video towards the end of the grasping demonstration. This adds even more perturbations (on top of the camera and mechanical modeling errors). In spite of these, visual servoing made it possible for Romeo to succeed in the tasks.

VI. CONCLUSION AND FUTURE WORK

In this paper, we showed how visual servoing can be formulated as a quadratic optimization problem. Furthermore, we formulated both equality and inequality constraints. Moreover, by defining an acceleration resolved form, we were able to easily integrate the visual servoing tasks into an existing whole-body control framework for humanoids. Results were then shown with simulations on HRP-4 and then real tests on Romeo. For future work, we can envision using the visual servoing tasks in more challenging scenarios. Some specific areas to be improved in this regard are: the combination with walking, and adding feedforward prediction for target motion tracking. Both of these were briefly outlined here. We also envision using other image features and visual constraints to improve the challenges on real robot platforms such as Romeo.

VII. ACKNOWLEDGEMENT

This work was supported in part by the BPI Romeo 2 and the H2020 Comanoid projects (www.comanoid.eu).

Fig. 1 .

 1 Fig. 1. A demonstration showing a gazing task using IBVS.

Fig. 2 .

 2 Fig. 2. Simulation data of the IBVS gazing task error discussed in Sec.IV-A

Fig. 3 .

 3 Fig. 3. Final pose of a PBVS task for the right wrist with a simulated modeling error. The opaque robot represents the real robot model while the wrong robot model is transparent. The error is introduced in the right shoulder joint. The task error plot of Fig 4 shows convergence to the target.

Fig. 4 .

 4 Fig. 4. Simulation data of the PBVS hand task error discussed in Sec.IV-B, the goal was moved 3 times after initial convergence

Fig. 5 .Fig. 6 .

 56 Fig. 5. A demonstration showing walking together with a hand PBVS task and an IBVS gaze

Fig. 7 .

 7 Fig. 7. A demonstration showing gaze control with Romeo using IBVS.

Fig. 8 .

 8 Fig. 8. Data from Romeo of the IBVS gazing task error discussed in Sec.V-A

Fig. 9 .

 9 Fig. 9. A demonstration showing Romeo grasping a box using IBVS for the gaze and PBVS for servoing the hand.

Fig. 10 .

 10 Fig. 10. Data from Romeo of the hand PBVS task error discussed in Sec.V-B

supported in part by the BPI Romeo 2 and the H2020 Comanoid projects (www.comanoid.