Olivier Sigaud

Camille Salaün

Vincent Padois

On-line regression algorithms for learning mechanical models of robots: a survey

Keywords: Adaptive and learning systems, Adaptive control, Mechanical models

With the emergence of more challenging contexts for robotics, the mechanical design of robots is becoming more and more complex. Moreover, their missions often involve unforeseen physical interactions with the environment. To deal with these difficulties, endowing the controllers of the robots with the capability to learn a model of their kinematics and dynamics under changing circumstances is becoming mandatory. This emergent necessity has given rise to a significant amount of research in the Machine Learning community, generating algorithms that address more and more sophisticated on-line modeling questions. In this paper, we provide a survey of the corresponding literature with a focus on the methods rather than on the results. In particular, we provide a unified view of all recent algorithms that outlines their distinctive features and provides a framework for their combination. Finally, we give a prospective account of the evolution of the domain towards more challenging questions.

Introduction

A major trend in robotics research is a shift from industrial applications, where everything can be planned in advance, to service, intervention, and exploration applications where robots must be able to address a large diversity of tasks while reacting to unforeseen situations [START_REF] Sigaud | Introduction[END_REF] -in particular when they have to interact with humans. In front of these versatility requirements, the robots themselves get more and more complex. Due to the presence of humans, they must also be less dangerous, thus lighter, which results in stiffness limitations [START_REF] Zinn | Playing it safe[END_REF]. Taking all these constraints together, the accurate low-level control of these systems for achieving elementary tasks is an increasingly difficult matter, not to mention higher level decision making problems.

In this context, efficient control methods are generally modelbased and call upon mechanical models of the robot [START_REF] Chung | Motion control[END_REF]. For instance, when the robot is redundant with respect to a task, a forward model of its velocity kinematics is mandatory to make profit of the redundancy. Besides, a model of the dynamics is required when the dynamics effects can generate disturbances that are larger than what local PID controllers with a limited stiffness can reject instantaneously. However, the accuracy of * corresponding author, phone: (+33).1.4427.8853, fax: (+33). 1.4427.5145 model-based control directly depends on the accuracy of these models and of the sensors used to close the control loop.

Up to a certain extent, mechanical engineering knowledge can help to provide such models. Indeed, the kinematic equation of a rigid poly-articulated plant is easy to obtain given a precise knowledge of its geometry. Similarly, its dynamic equation can be written as the combination of well-modeled factors such as the inertia of the plant, Coriolis and centrifugal effects as well as external forces (including gravity). But, for instance, the geometry of the mechanical chain changes when the robot is using a tool or when its structure is altered accidentally. And, at the dynamics level, there are additional unmodeled effects such as joint and cable frictions or backlashes [START_REF] Featherstone | Dynamics[END_REF] that are often difficult to model accurately. Furthermore, some of these effects are inherently non-stationary.

In that context, on-line adaptation of models is a valuable strategy because it can capture non-stationarities in the mechanical properties of the plant.

Two approaches in that respect are the on-line parameter identification step of adaptive control [START_REF] Siciliano | Robotics: modelling, planning and control[END_REF], which tunes parameters of a given model given its mechanical structure [START_REF] Ljung | System identification: theory for the user[END_REF] and model learning, which identifies mechanical models using supervised learning methods without any knowledge of the structure. Whereas classical identification methods generally assume a rigid body model and then incorporate some enhance-ments to take non-linear phenomena such as friction or flexibilities into account, the model learning approach makes no assumption on the structure and includes all phenomena in a general function built out of experimental data.

This paper provides a survey of state-of-the-art methods in the supervised learning approach. A related survey paper has been published recently in Nguyen-Tuong & Peters [START_REF] Nguyen-Tuong | Model learning for robot control: a survey[END_REF]. That previous publication covers many topics in that domain, from the different control methods that make use of the learned models to the presentation of specific robotics applications that demonstrate the feasibility of the methods. Another related and recent survey paper is Hoffmann et al. [START_REF] Hoffmann | Body schema in robotics: A review[END_REF]. It investigates the notion of body schema and helps discriminating between different kinds of representation that the controller of a robot can have of its body. It covers the vision processing questions related to the acquisition of such representations (e.g. Jagersand [START_REF] Jagersand | Image-based predictive display for high dof uncalibrated tele-manipulation using affine and intensity subspace models[END_REF]) and more general work on the acquisition of a body schema or a body image that directly relate visual features to mechanical properties of the plant [START_REF] Hersch | Online learning of the body schema[END_REF][START_REF] Mansard | Jacobian learning methods for tasks sequencing in visual servoing[END_REF][START_REF] Martinez-Cantin | Body schema acquisition through active learning[END_REF][START_REF] Sturm | Unsupervised body scheme learning through self-perception[END_REF]. It also investigates somewhat superficially the relationships to computational neurosciences issues related to model learning in animals and humans.

Here, in the terminology of Hoffmann et al. [START_REF] Hoffmann | Body schema in robotics: A review[END_REF], we focus on mechanical models that are either forward or inverse models, without addressing all the other points listed above. More precisely, we present the local and incremental regression algorithms used for learning mechanical models so as to provide a unifying view through a study of their similarities and differences. In particular, we highlight the features that make these algorithms unique, and investigate whether they may be combined into a new generation of methods.

The paper is organized as follows. In Section 2, we give some background knowledge about the mechanical models of robots. In Section 3, we present machine learning methods that are used to learn these models. Each time we present a method, we give an overview of the applications to robot model learning it has given rise to. All these methods are discussed in Section 4. Section 5 is devoted to a prospective presentation of the evolution of the field. Finally, we summarize our main messages in Section 6.

Mechanical Models of Robots

Before presenting the methods used to learn mechanical models in Section 3, in this section, we explain what these models are. The control methods based on such models are not covered in the paper, we refer the reader to Nguyen-Tuong & Peters [START_REF] Nguyen-Tuong | Model learning for robot control: a survey[END_REF], Salaün et al. [START_REF] Salaün | Control of redundant robots using learned models: an operational space control approach[END_REF] or to other papers cited in Section 3 to learn more about these control methods.

Joint space and task space

Rigid poly-articulated systems are defined by a finite number of rigid bodies interconnected by ideal joints. An ideal joint is a kinematic relationship between two rigid bodies.

The n parameters chosen to describe the configuration of a robot along with the joint equations are sufficient to describe the pose of each body composing it. These parameters can be gathered to form the vector of configuration parameters q also named generalized coordinates. The configuration of a robot is thus defined on an n-dimensional space called joint space or configuration space.

Though configuration parameters are often chosen so that they can be intuitively related to the motion of the actuators of the plant, it is very difficult to use them to describe most of the tasks of a robot. In order to facilitate the description of these tasks, an alternative space can be used, which is often called task space or operational space.

A unitary task consists in positioning the coordinate frame of a body, for instance an end-effector, relatively to a reference coordinate frame. This task can be described with a vector of task space parameters ξ of size m ≤ 6, since six parameters are sufficient to locally span the space of positions and orientations.

Joint space to task space mappings

The joint space to task space mapping can be described at three different levels. At the geometric level, the forward kinematics model is a non-linear function f such that ξ = f (q).

If the robot is redundant with respect to the task, there is an infinite number of possible inverses for f . In that case, there is no simple method to span the set of possible solutions at the geometric level and the mapping is often described at the velocity level by the Jacobian matrix J (q) = ∂ f (q) /∂q such that ξ = J (q) q.

(1) J (q) is an m × n matrix and thus can be inverted using linear algebra techniques. In the redundant and non-singular cases, i.e. rank (J (q)) = m and m < n, there exists an infinity of possible generalized inverses of J (q) (see Ben Israel & Greville [START_REF] Israel | Generalized Inverses: Theory and Applications[END_REF]). Given the redundancy, a system can make profit of internal motions that do not induce any perturbation on the task to achieve other tasks. The resulting trajectory of the plant depends on the specific inverse that has been chosen. Different possible redundancy resolution schemes can be applied, depending on the compatibility of the tasks or constraints, which have to be solved. We refer the reader to Salaün et al. [START_REF] Salaün | Control of redundant robots using learned models: an operational space control approach[END_REF] for a formal treatment of this topic. Conversely, in the case of parallel plants, the inverse model is unique whereas there can be an infinity of forward models.

Forward and Inverse Dynamics

The forward dynamics mapping relates forces applied on the plant, among which the applied m-dimensional control input vector Γ, to the acceleration q. The Dynamics (D) of a plant is described as

q = A -1 (q) (Γ -Γ ext -b (q, q) -g (q) -ǫ (q, q)), (2)
where A (q), b (q, q), g (q), ǫ (q, q) and Γ ext are respectively the n × n inertia matrix of the plant, the vector of Coriolis and centrifugal effects, the vector of gravity effects, the vector of unmodeled effects and the torques resulting from external forces applied to the plant.

At the dynamics level, the state of the plant is s = q T , qT T thus the induced dimensionality is larger than at the kinematics level where it is q.

The Rigid Body Dynamics (rbd) of the plant corresponds to the dynamics of the system with rigid parts and ideal joints. It is given by [START_REF] Angeli | Fast and incremental method for loop-closure detection using bags of visual words[END_REF] where the term corresponding to unmodeled effects is removed. More generally, (2) can be written q = D q, q, Γ, Γ ext .

(3)

Similarly, the inverse dynamics (ID) is described as

Γ = A (q) q + b (q, q) + g (q) + ǫ (q, q) + Γ ext , (4)
or, more generally,

Γ = ID q, q, q, Γ ext . (5)
In any given state s, the joint space to joint space mapping represented by [START_REF] Baranes | R-IAC: Robust intrinsically motivated exploration and active learning[END_REF] gives the torques to be applied by the actuators of the robot to obtain the desired acceleration q at the next step.

Inverse Dynamics in the task space

In Khatib [START_REF] Khatib | Dynamic control of manipulators in operational space[END_REF], Khatib formulated an alternative expression of the dynamics of the plant in the task space

F = Λ (q) ξ + µ (q, q) + p (q) + ǭ (q, q) + F ext (6) Γ = J (q) T F (7)
where Λ (q), µ (q, q), p (q) and ǭ (q, q) are the projection in the task space of the inertia matrix, the Coriolis and centrifugal forces, the gravity forces and the unmodeled effects forces. F ext is the vector of external forces applied to the plant at the end-effector level. These equations relate the task space acceleration vector ξ, with a dimension less than or equal to 6, to the joint torques vector, with a dimension equal to n.

In general, inverse dynamics in the task space is a mapping from R 2n × R m to R n whereas inverse dynamics in joint space is a mapping from R 3n to R n . Poly-articulated systems have a task space dimension which is usually smaller or equal to the dimension of the joint space. Thus, the model of the inverse dynamics in the task space is generally smaller than the combination of the kinematics model with the standard inverse dynamics model.

Regression of Mechanical Models

In supervised learning, a class of machine learning techniques, a supervisor provides input/output pairs that can be used to learn a relationship between the corresponding training data. When the output is discrete, the classification problem consists in associating the presented input to an output value, its class. In contrast, when the output is continuous, the regression problem consists in finding a function that approximates as accurately as possible a latent function that describes the input/output relationship. Formally, given a set of k training samples S = {x i , y i } k i=1 , where x i ∈ X ⊆ IR l is a vector of dimension l and y i ∈ Y ⊆ IR the regression problem consists in approximating a latent function f : X → Y with a function f taken in some predetermined family so as to minimize some measurement of the approximation error.

For instance, the Least Square (ls) regression method considers the family of linear functions f (x) = β T x where β is a weight vector minimizing the quadratic error

min β ||y -Xβ|| 2 , (8)
where

y = [y 1 , . . . , y k ] T and X = [x 1 , . . . , x k ] T is a k × l matrix.
Learning a mechanical model of a plant is a regression problem where training samples are obtained from the state and controls of the plant along time.

A global view of the regression methods presented in this paper is given in Fig. 1. The next section describes early regression methods used to learn such models, from Artificial Neural Networks (anns) to Locally Weighted Regression (lwr) methods. Then we compare several state-of-the-art methods that all incrementally split the input space into regions and perform regression on local models associated to these regions.

In order to insist on what distinguishes these methods from their competitors and before discussing the impact of these differences, we focus on the following questions for all algorithms. How are the regions defined? How does the algorithm perform local regression? How do the region boundaries evolve? How does the algorithm add (or delete) a region? How does it compute the global output?

Artificial Neural Networks

This section is dedicated to anns and their evolution towards lwr methods. anns are a wide class of non-linear function approximation tools. They are declined under different forms such as Multi-Layer Perceptrons (mlps) [START_REF] Rosenblatt | The perceptron: A probabilistic model for information storage and organization in the brain[END_REF], Self-Organizing Maps (soms) [START_REF] Kohonen | Self-Organizing Maps[END_REF] or Radial Basis Function networks (rbfns) [START_REF] Cybenko | Approximation by superpositions of a sigmoidal function[END_REF]. Other forms of neural networks such as population coding are not covered in this paper because they are less relevant to the introduction of the subsequent sections, although they have been applied to robot model learning (see e.g. Butz et al. [START_REF] Butz | Exploiting redundancy for flexible behavior: Unsupervised learning in a modular sensorimotor control architecture[END_REF]). For a classical survey about learning mechanical models with anns, we refer the reader to Jordan & Rumelhart [START_REF] Jordan | Forward models: Supervised learning with a distal teacher[END_REF].

In all the sections below, α is the learning rate, n n is the number of neurons of a network and n r is the number of regions.

Multi-Layer Perceptrons

An mlp approximates the latent function f through a directed graph of artificial neurons whose nodes convey a non-linear function and edges convey a weight that is tuned by the algorithm. The excitation level v i (x) of each neuron i is calculated as

v i (x) = n n i=1 β i x i
where x i is the input of neuron i and β i is the associated weight. The activation function y i = f (v i ) uses the excitation level to determine if neuron i fires or not and is its output. Classical activation functions are y i = tanh(v i ) or the sigmoid function

y i = (1 + e -v i ) -1 .
The learning process consists in computing the error of the global network output to update all weighted connections. The weights can be updated by back-propagating through the network an error δy i between the desired value and the real output of each neuron i

δβ i j = -α ∂δy i ∂β i j . (9)
where β i j is the weight between input x i and neuron i. mlps are used to learn the inverse kinematics (e.g. Ahmad & Guez [START_REF] Ahmad | On the solution to the inverse kinematic problem[END_REF], Barhen et al. [START_REF] Barhen | Neutral learning of constrained nonlinear transformations[END_REF], Pourboghrat [START_REF] Pourboghrat | Neural networks for learning inverse-kinematics of redundant manipulators[END_REF]), the Jacobian matrix [START_REF] Lee | Robot kinematic control based on bidirectional mapping neural network[END_REF] or the forward kinematics model of a robot [START_REF] Brüwer | A network model for the control of the movement of a redundant manipulator[END_REF][START_REF] Nguyen | Neural network architectures for the forward kinematics problem in robotics[END_REF]. More recently, Sang & Han [START_REF] Sang | The estimation for forward kinematic solution of stewart platform using the neural network[END_REF] and Sadjadian & Taghirad [START_REF] Sadjadian | Numerical methods for computing the forward kinematics of a redundant parallel manipulator[END_REF] use ann methods to learn the forward models of parallel robots which involves highly coupled non-linear equations that are difficult to model analytically.

Self-Organizing Maps

A som is another type of ann. It is trained using unsupervised learning to produce a low-dimensional discrete representation of the input space. An important feature of soms is that they can preserve the topological properties of the input space.

Barretto et al. [START_REF] Barretto | Self-organizing feature maps for modeling and control of robotic manipulators[END_REF] provide a good survey of the application of soms to robot identification problems. Most authors use these methods in the context of visuo-motor control, apart from Walter & Schulten [START_REF] Walter | Implementation of self-organizing neural networks for visuo-motor control of an industrial robot[END_REF] and Wang & Zilouchian [START_REF] Wang | Solutions of kinematics of robot manipulators using a kohonen self-organizing neural network[END_REF] who respectively learn the inverse kinematics of a six dofs industrial robot Puma 562 and the forward kinematics model of a simple plant. In this context, the evaluation is based on the accuracy of the model itself rather than on its control capabilities.

In the visuo-motor case, Ritter et al. [START_REF] Ritter | Topologyconserving maps for learning visuo-motor-coordination[END_REF] and Martinetz et al. [START_REF] Martinetz | Three-dimensional neural net for learning visuomotor coordination of a robot arm[END_REF] use soms to learn a mapping between the three-dimensional end-effector position measured by two cameras and joint positions on a three dofs robot. More recent work in the domain can be found in Kumar et al. [START_REF] Kumar | Visual motor control of a 7dof redundant manipulator using redundancy preserving learning network[END_REF].

Furthermore, specific soms called Growing Neural Gas are capable of expanding in the relevant dimensions of a domain.

They have been combined with locally weighted learning methods presented hereafter [START_REF] Flentge | Locally weighted interpolating growing neural gas[END_REF]. The resulting algorithm is endowed with interesting properties for incremental function approximation in a large domain, but to our knowledge it has not been applied to the identification of mechanical models. Thus, we do not investigate the topic in more detail.

Radial Basis Function Networks

Radial Basis Function Networks (rbfns), illustrated in Fig. 2, can be seen as a simplification over mlps, retaining only one layer and using a Radial Basis Function (rbf) as activation function.

An rbf φ is a function whose output value ŷ depends only on the distance r i from its center c i to the input value x. A standard rbf is a Gaussian function defined as

φ(r i ) = 1 2πσ i 2 e - r 2 i 2σ i 2 ( 10
)
where σ i 2 is the variance of the Gaussian function i which determines its size and shape. The values of all σ i are often set to a unique value σ. To perform regression with rbfns, the output of all functions are weighted and summed together to get the global network output

ŷ = n n i=1 β i φ(r i ) (11)
where the β i are weights.

In rbfns, the weights are updated incrementally with different methods such as gradient descent based on the error between the output of the network and the desired value. As it can be considered as a one layer ann, the back-propagation can be written as In Sun & Scassellati [START_REF] Sun | Reaching through learned forward model[END_REF][START_REF] Sun | A fast and efficient model for learning to reach[END_REF], an rbfn is used to learn the forward kinematics of a robot. Each basis function is derived to obtain a Jacobian matrix, using the Resolved Motion Rate Control (rmrc) [START_REF] Whitney | Resolved motion rate control of manipulators and human prostheses[END_REF] framework to generate trajectories. In Lin & Goldenberg [START_REF] Lin | Neural-network control of mobile manipulators[END_REF], an rbfn is used to learn an inverse dynamics model of a 4 dofs manipulator in three dimensions.

δβ i = -α(y i -ŷi )φ(r i ). (12)

Locally Weighted Regression

Locally Weighted Regression (lwr) methods [START_REF] Atkeson | Using locally weighted regression for robot learning[END_REF] use Gaussian features like rbfns, but the weights are approximated with the ls method (see Fig. 3).

The main difference between rbfns and lwr methods lies in the introduction of a linear model associated to each rbf and used to perform ls regression. Each rbf defines a region of validity for the corresponding linear model.

More precisely, around each point x, the contribution of each receptive field to the global output is computed with a Gaussian function [START_REF] Bicchi | Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity[END_REF]. Thus lwr globally approximates non-linear functions by combining local linear models.

Figure 3: In lwr, x is the input of a latent function. A Gaussian, centered on x, weights stored point with respect to their proximity to x. A regression is then computed using the weighted points [START_REF] Cohn | Active learning with statistical models[END_REF].

Compared to anns or rbfns, lwr benefits from the power of linear estimation methods. But lwr needs to retain all training data to perform the ls approximation, which may be infeasible in large spaces given memory limitations. However, the linear models β can also be computed incrementally, using the Recursive Least Square (rls) algorithm. This extension to lwr gave rise to rfwr [START_REF] Schaal | Receptive field weighted regression[END_REF], that is an incremental local regression method avoiding storage of data. In parallel, lwr was improved in another direction by lwpls [START_REF] Schaal | Local dimensionality reduction[END_REF], that uses the Partial Least Square (pls) [START_REF] Elden | Partial least-squares vs. Lanczos bidiagonalization-i: analysis of a projection method for multiple regression[END_REF][START_REF] Wold | Soft modelling by latent variables: the non-linear iterative partial least squares (NIPALS) approach[END_REF] projection mechanisms to neglect irrelevant dimensions in the input space.

Locally Weighted Projection Regression

Locally Weighted Projection Regression (lwpr) [START_REF] Vijayakumar | Locally weighted projection regression: An o(n) algorithm for incremental real time learning in high dimensional space[END_REF] is probably the most popular learning method for learning mechanical models of robots. As described in Schaal et al. [START_REF] Schaal | Scalable techniques from nonparametric statistics for real time robot learning[END_REF], it can be seen as combining the properties of lwpls and rfwr. Indeed, it performs simultaneously incremental regression of local linear models like rfwr and input projection like lwpls. Furthermore, the shape of the region corresponding to each linear model is also tuned incrementally. lwpr provides a fast, incremental and reasonably accurate approximation, thus it is a good reference for comparisons.

Definition of regions.

In lwpls, rfwr and lwpr, a region is called a receptive field. Like its ancestors, lwpr uses Gaussian functions φ i (x)

φ i (x) = e - 1 2 (x -c i ) T W i (x -c i ) ( 13
)
where the positive distance metric W i of the Gaussian delimits a region which is updated during learning to match the training data.

Local regression mechanism. As in lwr, each region has a corresponding local linear model β i which is used to predict a local scalar output y i relative to an input vector

x ŷi (x) = β i x. ( 14
)
lwpr inherits from the capacity of lwpls to reduce the input dimensionality using pls. Some projection is made on the input vector to model high dimensional functions only on their most relevant directions. Moreover, like rfwr, the linear model approximation is performed efficiently using an incremental version of pls called nipals [START_REF] Wold | Soft modelling by latent variables: the non-linear iterative partial least squares (NIPALS) approach[END_REF]. Several other ways to perform dimensionality reduction in the context of local linear regression are discussed in Vijayakumar et al. [START_REF] Vijayakumar | LWPR: A Scalable Method for Incremental Online Learning in High Dimensions[END_REF].

Evolution of regions.

The distance metric W is adapted when new data are received to improve the coverage of the input space. It is computed as W = M T M where M is an uppertriangular matrix to ensure W is symmetric and positive definite. M is modified recursively and individually to regulate the size of the regions with the update law

M = M -α ∂C ∂M (15)
where C is a minimized cost function. This cost function is more complex than the weighted mean square error criterion in lwr. It is computed by a leave one out cross-validation method and includes a penalty term that prevents the population of models from collapsing. The reader can refer to Schaal et al. [START_REF] Schaal | Scalable techniques from nonparametric statistics for real time robot learning[END_REF] or Vijayakumar & Schaal [START_REF] Vijayakumar | Locally weighted projection regression: An o(n) algorithm for incremental real time learning in high dimensional space[END_REF] for a detailed presentation of the incremental version of the algorithm. Note that, in contrast with the distance metrics, the centers c i of the regions are not adapted. Their position depends on the mechanism used to add a new region.

Adding new regions. The rule for adding a new region is the following. For a given input x, the Gaussian function φ i (x) defining each region can be seen as a level of activation of the local model corresponding to this region. So, if, for the current x, the level of activation of no region is above a threshold w gen , a new region is created with x as center and an initial D init as distance metrics. The value of w gen plays a major role in the number of regions created for a given approximation problem.

Global model approximation. The approximation performed by lwpr is the sum of local linear models weighted by their respective Gaussian weights

ŷ(x) = 1 Φ (x) n r i=1 φ i (x) ŷi (x) (16)
where

Φ (x) = n r i=1 φ i (x).
Furthermore, the gradient of the latent function, differentiated with respect to the input (and not with respect to time) and the confidence bounds on the predicted output are also provided through an incremental algorithm [START_REF] Klanke | A library for locally weighted projection regression[END_REF].

Applications to robot model learning. lwpr is certainly the most used algorithm in the context of learning mechanical models of a plant. As a result, it is used as a basis for comparisons in most if not all the papers presented in the next sections. Hereafter, we list a few of the most impressive applications of lwpr to learning mechanical models of robots.

In D'Souza et al. [START_REF] Souza | Learning inverse kinematics[END_REF], an inverse kinematics model is learned with lwpr. The plant is an atr humanoid robot [START_REF] Kawato | Internal models for motor control and trajectory planning[END_REF] with 26 actuated joints among which only 4 are relevant for the learning algorithm. An end-effector is controlled in Cartesian space. Even though 22 of these dimensions are irrelevant, the dimension of the input (ξ, q) is 29 and the dimension of the output q is 26. The model is learned along a task space trajectory, which is much easier than learning in the whole space. The method solves redundancy by minimizing a cost function that depends on the configuration. Doing so, no inversion is involved and singularity problems are avoided.

In Schaal et al. [START_REF] Schaal | Scalable techniques from nonparametric statistics for real time robot learning[END_REF], Vijayakumar & Schaal [START_REF] Vijayakumar | Local dimensionality reduction for locally weighted learning[END_REF][START_REF] Vijayakumar | Locally weighted projection regression: An o(n) algorithm for incremental real time learning in high dimensional space[END_REF], the authors use lwpr to learn the inverse dynamics model of a seven dofs anthropomorphic Sarcos robotic arm, again along a task space trajectory. The trajectory is followed with a task space PD controller. Some noise is added, making the plant explore and learn an envelope of possible configurations around the trajectory. The input of the learning algorithm is (q, q, q) of dimension 21 and the output is the torque Γ of dimension 7. The number of training data points is 5 10 4 and the model contains 260 regions.

In Vijayakumar et al. [START_REF] Vijayakumar | LWPR: A Scalable Method for Incremental Online Learning in High Dimensions[END_REF], the same approach is used to learn the inverse dynamics model of the atr humanoid robot. The input dimension is 90 and the output dimension is 30. Here, 7.5 10 6 samples and 2200 regions are required to obtain an accurate model. To our knowledge, this is the largest space in which a function has been learned with lwpr, but once again the function is learned along a trajectory in a small sub-part of the input space. Moreover, the number of joints used to control the arm is much smaller than the number of joints taken as input of the learning algorithm. Besides, since learning 30 parallel lwpr models would be computationally too expensive, the authors choose to learn one single model with 30 dimensional outputs projected, thanks to the pls algorithm, onto the input data. Indeed, as a consequence of using the pls regression, it is possible to learn one single model with a vector output instead of a scalar one.

More recently, lwpr was used in Peters & Schaal [START_REF] Peters | Learning to control in operational space[END_REF] to learn the inverse dynamics in the task space of an anthropomorphic Sarcos arm and a Mitsubishi PA-10 robot. For both plants, the input dimension is 17 and the output dimension is 7. As underlined in Section 2.4, those dimensions have to be compared to the size of two separate models which can be learned independently: an inverse velocity kinematics model which links a 3 dimensional space to a 7 dimensional space and an inverse dynamic model which links a 21 dimensional space to a 7 di-mensional space. The inverse dynamics in the task space is also learned with lwpr in Sun de la Cruz et al. [START_REF] De La Cruz | Learning inverse dynamics for redundant manipulator control[END_REF], where the system deals with redundancy by choosing a solution that minimizes the torque output used to control the plant.

Finally, in Mitrovic et al. [START_REF] Mitrovic | Adaptive optimal control for redundantly actuated arms[END_REF], the inverse dynamics model of a two dofs planar arm actuated with three artificial agonistantagonist pairs of muscles is learned with lwpr in the whole planar space. The input is (q, q,u) where u is the activation of muscles and the output is ( q, q). The input dimension is dim(q, q,u) = 10 and the output dimension is dim( q, q) = 4. The authors use an optimal feedback control loop based on the ilqg algorithm [START_REF] Li | Optimal Control for Biological Movement[END_REF] to find the muscular activations that minimize the muscular input while performing some reaching movements. After a learning period generating 1.2 10 6 training points, the model is composed of 852 regions and reaches three different targets.

Extended classifier system for function approximation

The xcsf algorithm is a regression method that shares many similarities with lwpr but comes from Learning Classifier Systems (lcss), a family of adaptive rule-based systems first proposed by Holland [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF] that combines reinforcement learning mechanisms [START_REF] Sutton | Reinforcement learning[END_REF] and genetic algorithms [START_REF] Goldberg | Genetic Algorithms in Search, Optimization and Machine Learning[END_REF].

The immediate ancestor of xcsf is xcs [START_REF] Wilson | Classifier Fitness Based on Accuracy[END_REF], an efficient accuracy-based lcs designed to solve discrete classification problems and sequential decision problems. xcs finds accurate solutions to a wide range of problems with high probability in polynomial time [START_REF] Butz | Computational complexity of the XCS classifier system[END_REF]. xcsf [START_REF] Wilson | Function approximation with a classifier system[END_REF][START_REF] Wilson | Classifiers that Approximate Functions[END_REF] is an evolution of xcs towards function approximation.

As any lcs, xcsf manages a population P of rules, called classifiers. These classifiers contain a condition part and a prediction part. The condition part defines the region of validity of a local model whereas the prediction part contains the local model itself. xcsf is a generic framework that can use different kinds of prediction models (linear, quadratic, etc.) and can pave the input space with different families of regions (Gaussian, hyper-rectangular, etc.). In this paper, we only consider the case of linear prediction models and Gaussian regions. In this case, the structure of the function approximation built by xcsf is similar to the one built by lwpr.

Definition of regions.

An important specificity of xcsf is that it distinguishes a prediction input space and a condition space. The prediction input space, noted X, corresponds to the input used to compute the linear models with regression. The condition space, noted Z, is paved with Gaussian classifiers, corresponding to the regions of lwpr. A classifier defines a domain φ i (z) in the following way

φ i (z) = e - 1 2 (z -c i ) T W i (z -c i ) ( 17
)
where c i is the center of the Gaussian i and W i is a positive distance metric, inverse of the squared variance of Gaussian i. The input of the condition space, z ∈ Z, defines the space where the Gaussian influences the global prediction.

The match set M is the subset of all reliable classifiers in the population P for which φ i (z) is above a threshold φ 0 1 .

Local regression mechanism.

Each classifier has a corresponding linear model β i which is used to predict a local output vector y i relative to an input vector x, provided an augmented state

x aug = x T 1 T
to deal with the presence of an offset β 0 . The linear model is updated using the rls algorithm, the incremental version of the ls method. If a classifier i makes an error δy i = yŷi , it is updated with δβ i = δy i x T i α.

Evolution of regions.

A specificity of xcsf is that it uses a genetic algorithm (ga) instead of a gradient-based method to update its population P. As a result, there is no evolution of a region per se, but the ga rather uses a fitness function to add new regions similar to the old ones after applying mutation operators and deleting some regions. Here we explain how the fitness is obtained.

The prediction error of a classifier i approximates the mean absolute deviation of its prediction using the Widrow-Hoff delta rule [START_REF] Widrow | Adaptive switching circuits[END_REF]

δǫ i = α |δy i | -ǫ i
where ǫ i is the mean absolute error of the classifier i at the previous time step. Then the classifier accuracy is determined by the scaled inverse of ǫ i . Classifiers with an ǫ i below a threshold ǫ 0 are considered trustworthy. Finally, the fitness value is derived from the classifier accuracy relative to all others in the match set M (see Butz & Herbort [17], Butz et al. [START_REF] Butz | Toward a theory of generalization and learning in XCS[END_REF] for algorithmic details).

Adding and deleting regions. In xcsf, a classifier is created when there is no classifier in the match set corresponding to an input. This mechanism is called covering, it creates a matching condition which corresponds to that input with a randomly generated condition space. But classifiers can also be created by an evolutionary component based on a niched steady-state ga [START_REF] Wilson | Classifiers that Approximate Functions[END_REF] improved with a set-size-relative tournament selection [START_REF] Butz | Analysis and improvement of fitness exploitation in XCS: bounding models, tournament selection, and bilateral accuracy[END_REF]. This favors the recombination of efficient classifiers by crossover and mutations. A ga is run if the time since the last ga invocation exceeds a threshold θ GA to balance non-uniform problem-space sampling.

The reproduction of classifiers is performed within the match set M, whereas the deletion of inaccurate classifiers is performed over the whole population P. This combination of mechanisms ensures a good generalization capability [START_REF] Butz | Toward a theory of generalization and learning in XCS[END_REF][START_REF] Wilson | Classifier Fitness Based on Accuracy[END_REF]. In particular, general classifiers are reproduced more often as they match the input data more often than the others.

In more detail, the ga selects in the current match set M two classifiers based on their relative predictive accuracy with respect to the other classifiers. The centers c new i of their offspring are moved within an interval c ic new i ≤ -2W i ln φ 0 where φ 0 is the threshold used to generate the match set M. The shape of the ellipsoids, defined by σ i which is the square-root of W i , is also randomly increased or decreased within a limited interval. 1 This threshold is named θ m in Butz & Herbort [START_REF] Butz | Context-dependent predictions and cognitive arm control with XCSF[END_REF] Deletion takes place in the population when the number of classifiers in P reaches the maximum n max P . Supernumerary classifiers are deleted from P with a probability proportional to an estimate of the size of the match sets that they occur in. If a classifier is sufficiently experienced and its fitness F is significantly lower than the average fitness of classifiers in P, its deletion probability is further increased [START_REF] Kovacs | Deletion schemes for classifier systems[END_REF].

Finally, two additional processes that further optimize the population, namely subsumption and compaction, are described in Butz & Herbort [START_REF] Butz | Context-dependent predictions and cognitive arm control with XCSF[END_REF].

Global model approximation. The global population P partitions the input space into a set of overlapping prediction models. xcsf generates the global approximation based on the current match set M. The learning output ŷ is given for a (x, z) pair as the weighted sum of the linear models of each classifier i in

M ŷ (x, z) = 1 Φ (z) n M i=1 φ i (z) ŷi (x) (18)
where

Φ (z) = n M i=1 φ i (z)
and n M is the number of classifiers in the match set M.

In sum, the evolutionary mechanism is designed to evolve partitions in which linear approximations are maximally accurate. Indeed, xcsf strives to evolve complete, maximally accurate, and maximally general population of local approximations [START_REF] Butz | Bounding the population size in XCS to ensure reproductive opportunities[END_REF]. A more complete description of xcsf can be found in Butz et al. [START_REF] Butz | Toward a theory of generalization and learning in XCS[END_REF] or in Butz & Herbort [START_REF] Butz | Context-dependent predictions and cognitive arm control with XCSF[END_REF].

Applications to robot model learning. In Butz et al. [START_REF] Butz | Learning sensorimotor control structures with XCSF: redundancy exploitation and dynamic control[END_REF], Butz & Herbort [START_REF] Butz | Context-dependent predictions and cognitive arm control with XCSF[END_REF], Stalph et al. [START_REF] Stalph | Controlling a four degree of freedom arm in 3d using the XCSF learning classifier system[END_REF], the authors use xcsf to learn the forward kinematics model of a four dofs simulated arm in order to model motor adaptation in arm reaching experiments. They invert the model with classical analytical methods in order to control an end-effector in three dimensions while controlling the redundancy. They choose three different secondary tasks in the joint space which consist in minimizing angular velocities, avoiding joint limits or specifying a global posture.

Support Vector Regression

In Support Vector Regression (svr), the regression extension of Support Vector Machines (svms), it is assumed that the latent function f (x) can be parametrized as f (x) = φ(x) T w, where φ is a feature vector mapping the input x into some high dimensional space and w is the corresponding weight vector [START_REF] Drucker | Support vector regression machines[END_REF][START_REF] Nguyen-Tuong | Incremental sparsification for real-time online model learning[END_REF][START_REF] Smola | A tutorial on support vector regression[END_REF]. The weight vector w can be represented as a linear combination of the input vectors in the feature space with coefficients β i , i.e. w = n f i=1 β i φ(x i ), where n f is the number of features.

Given this representation, the prediction ŷ of an input x is

ŷ = f (x) = n f i=1 β i < φ(x i ), φ(x) >, = n f i=1 β i k(x i , x) (19)
where <, > denotes the dot product between two vectors and k(x, x ′ ) is a kernel function. A standard kernel is the Gaussian function

k(x, x ′ ) = σ 2 e - 1 2 (x -x ′ ) T W(x -x ′ ) , (20)
where W denotes the kernel width. By comparing [START_REF] Calinon | Robot programming by demonstration[END_REF] with ( 16) and ( 18), one can see that the global structure of the approximation in svms is different from the one used in lwpr and xcsf. Whereas the former is a linear combination of kernel functions, the latter is a kernel-weighted combination of linear models.

The standard way to obtain the parameters β i of the models in svms if through a quadratic programming optimization process based on all training samples [START_REF] Smola | A tutorial on support vector regression[END_REF]. There also exists a ls formulation called ls-svm [START_REF] Suykens | Least squares support vector machines[END_REF][START_REF] Suykens | Least squares support vector machine classifiers[END_REF]. In Fumagalli et al. [START_REF] Fumagalli | Learning to exploit proximal force sensing: A comparison approach[END_REF], the authors are interested in obtaining an accurate estimation of internal forces in the arm of the humanoid robot James. The arm of the robot is equipped with a force/torque sensor that estimates a total force from which the internal and external forces have to be determined. This in turn requires the estimation of a dynamics model, for which the authors compare ls-svm and a batch neural network approach to the analytical derivation of the model from cad parameters. The results show that ls-svm converges faster than the neural network approach, even though the final performance is similar.

An important parameter for computational complexity is the number of support vectors, hence algorithms like ν-svr use a specific meta-parameter to control this number [START_REF] Schölkopf | New support vector algorithms[END_REF]. Given its availability in the libsvm library [START_REF] Chang | LIBSVM: a library for support vector machines[END_REF], this method has been used as a baseline for performance comparisons in the context of comparisons for model learning methods [START_REF] Cederborg | Incremental local online Gaussian mixture regression for imitation learning of multiple tasks[END_REF][START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF].

LoSVR

The focus of this survey being incremental and local approaches, we must mention that an svm approach endowed with these properties and called losvr is presented in Choi et al. [START_REF] Choi | Local online support vector regression for learning control[END_REF]. losvr is built on a previous incremental svm algorithm presented in Ma et al. [START_REF] Ma | Accurate on-line support vector regression[END_REF]. It is used to learn the inverse dynamics model of a simulated 2 dofs arm and then a real robot. However, the addressed problem is rather small and the authors are expecting poor generalization capabilities from their system, so we do not discuss it further in this paper.

Random Features Regularized Least Squares

An important concern in locally weighted regression methods is the model complexity, expressed in terms of the number of regions. A standard way to deal with this concern is called regularization. It consists in adding a term related to model complexity in the cost function that is optimized by the approximation method.

For instance, in Regularized Least Squares 2 (rgls), the ls cost function (8) becomes 2 Not to be confused with Recursive Least Squares min β

λ 2 ||β|| 2 + ||y -Xβ|| 2 , (21)
where λ > 0 is a regularization parameter that balances the trade-off between complexity and accuracy.

The optimal β can be obtained by setting the gradient of (21) to 0, which gives

β = (λI + X T X) -1 X T y. ( 22
)
Kernel Regularized Least-Squares (krgls) 3 is the extension of rgls to non-linear functions using the same kernel trick as in svms [START_REF] Rifkin | Regularized least-squares classification[END_REF]. Like gpr, the problem of krgls is its cubic cost in the number of samples.

In Rahimi & Recht [START_REF] Rahimi | Random features for large-scale kernel machines[END_REF], the authors propose to approximate the kernel matrix with a set of random features, giving rise to Random Features Regularized Least Squares (rfrls), an algorithm that converges to krgls in O 1 √ D , where D is the number of features on which the kernel is projected.

Finally, in Gijsberts & Metta [START_REF] Gijsberts | Incremental learning of robot dynamics using random features[END_REF], the authors present an incremental version of rfrls (named irfrls hereafter) that introduces the incremental estimation method of rls in rfrls. The complexity of this method is in O D 2 , thus it is independent of the number of samples. Furthermore, the accuracy versus time complexity trade-off can be easily tuned by changing the number of features D. Experiments on several databases confirm that krgls is competitive with respect to gpr and that rfrls and irfrls can be set close to krgls with a small enough processing time.

Gaussian Processes Regression

In svr (Section 3.4), one is looking for a unique vector of weights β such that the approximation

f (x) = n f i=1 β i φ i (x)
is as close as possible to the latent function f . Bayesian linear regression methods provide an alternative basis to this problem. Instead of just looking for a specific vector β, they look for the posterior distribution over β given all the approximation samples and some prior. Given the resulting distribution, the best model can be determined either by averaging over the distributions of models or by selecting the Maximum A Posteriori (MAP) element of the distribution. As a result, such methods provide a more accurate but also more expensive way of finding an appropriate model.

As clearly explained in Williams [START_REF] Williams | Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond[END_REF], Gaussian Processes Regression (gpr) methods are mathematically equivalent to Bayesian linear regression. The main assumption in gpr is that the data presented to the learning process is corrupted by Gaussian noise with zero mean and variance σ 2

n . An important property of Gaussian vectors is the following. If n ≥ 1 and X = [X 1 , . . . , X n , X n+1 ] is a Gaussian vector of average µ = 0 and whose variance can be decomposed in the following way:

= K k k T k * ,
where K is a covariance matrix, k is a vector and k * is a scalar, then

E( f (x n+1 )|Y 1 = y 1 , . . . , Y n = y n ) = k T K -1 y and Var( f (x n+1 )|Y 1 = y 1 , . . . , Y n = y n ) = k * -k T K -1
k where y T = (y 1 , . . . , y n ), E(.) denotes the expectancy and Var(.) denotes the variance.

If the latent function f is assumed to be a Gaussian process, a function C : (x, x ′ ) → E( f (x), f (x ′ )) must be chosen to estimate the covariance matrix K out of data. Once this function is chosen and K is estimated, the approximation is obtained by just generating a Gaussian random variable of average k T K -1 y, and of variance k *k T K -1 k (see Williams [START_REF] Williams | Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond[END_REF] for details).

The advantage of gpr methods over Bayesian linear regression is that one does not need to define a set of feature functions over the input space. But, given the necessity to compute K -1 , standard gpr has O(n 3 ) computation and O(n 2 ) storage cost, where n is the number of samples used for approximation. Thus, as Williams [START_REF] Williams | Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond[END_REF] points out, gpr methods can be preferred only if n is not too big with respect to the number of feature functions that would be required to perform Bayesian linear regression.

Practical approaches to GPR

Given the cubic cost in the number of samples, there are two practical approaches to gpr. The sparse approach consists in limiting n by choosing adequately the points to remember and forgetting the rest. A good overview of this approach can be found in Quiñonero Candela & Rasmussen [START_REF] Quiñonero Candela | A unifying view of sparse approximate Gaussian process regression[END_REF]. For instance, it has been used recently in Sparse On-line Gaussian Processes (sogp), in the context of learning control policies from demonstrations [START_REF] Grollman | Sparse incremental learning for interactive robot control policy estimation[END_REF]. The mixture of experts (moe) approach rather consists in splitting the global domain of the latent function into smaller regions. In moe methods, the regions are called experts. Indeed, if one splits the function domain into k regions, the standard gpr complexities are k.O(n 3 i ) and k.O(n 2 i ) respectively, where n i is the number of samples that fall in region i ∈ {1, ..., k}.

This approach is also used to learn a policy from demonstration in Woods et al. [START_REF] Woods | Incremental Nonparametric Bayesian Regression[END_REF], that describes an incremental version of Infinite Mixture of Gaussian Process Experts (imgpe) [START_REF] Meeds | An alternative infinite mixture of Gaussian process experts[END_REF] and compares it to lwpr. From the analysis of the authors themselves, even if their algorithm benefits from the Bayesian linear regression properties in contrast with lwpr and provides more accurate results is some contexts, it is still too slow to be used in an on-line robotics set-up. To our knowledge, this system has not been applied to learning mechanical models of a robot. The system presented below can be seen as a better compromise following this line of research.

LGP

Local Gaussian Processes (lgp) is a system that combines the accuracy of gpr methods on one region with the good capability of lwpr to split the latent function domain into regions [START_REF] Nguyen-Tuong | Local Gaussian processes regression for real-time model-based robot control[END_REF][START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF][START_REF] Nguyen-Tuong | Model learning with local Gaussian process regression[END_REF][START_REF] Nguyen-Tuong | Real-time local Gaussian processes model learning[END_REF]. Many mechanisms being identical to those of lwpr, we only give a brief presentation, insisting on the points where both systems differ.

In lgp, regions are defined as in lwpr. Furthermore, the mechanism for adding a new region in lgp is identical to the one used in lwpr: a region is added if the distance of the current point to all activated region is more than a threshold w gen .

By contrast, the mechanisms for the evolution of regions differ. In lwpr, the parameters W determining the shape of the regions evolve whereas the centers of all regions are fixed. In lgp, the parameters W are fixed whereas the centers are moving each time a region receives a new point, the center being the barycenter of all the points associated to the region.

The linear approximation provided in each region is based on a local gpr method. In that context, the covariance function used is the standard Gaussian kernel function [START_REF] Quiñonero Candela | A unifying view of sparse approximate Gaussian process regression[END_REF].

In order to compute the global model output, instead of using the models of all regions as done in lwpr, lgp uses only the M regions whose center is closest to the current point.

In order to prevent the k.O(n 3 i ) computation cost to get too expensive, once a threshold is reached, the algorithm removes the points that bring the least information. In [START_REF] Nguyen-Tuong | Incremental sparsification for real-time online model learning[END_REF], lgp is augmented with an on-line sparsification algorithm inspired from what is used in Kernel Recursive Least-Squares (krls) [START_REF] Engel | The kernel recursive leastsquares algorithm[END_REF]. This additional process addresses the same issue as sogp, i.e. the incremental selection of samples that are used to train the local Gaussian processes.

In terms of application to robot model learning, in Nguyen-Tuong et al. [START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF], the authors compare the performance of lgp with lwpr, standard gpr and ν-svr (see Section 3.4). The comparison is performed on learning the inverse dynamics in the task space of a seven dofs Sarcos arm. It shows that lgp is competitive and provides a good trade-off between lwpr and gpr.

Finally, in Nguyen-Tuong et al. [START_REF] Nguyen-Tuong | Real-time local Gaussian processes model learning[END_REF], the authors also show that using lgp could give rise to better results than just using the parametric model provided by the robot manufacturer of a torque controlled manipulator.

Gaussian Mixture Models

A mixture model is a probabilistic model for representing the presence of sub-populations within an overall population, without requiring that an observed data-set should identify the subpopulation to which an individual observation belongs. In that context, the Expectation-Maximization (EM) algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF] can be used to make statistical inferences about the properties of the sub-populations given only observations on the global population.

The EM algorithm is an iterative method for finding maximum likelihood or MAP estimates of parameters in statistical models, where the model depends on unobserved latent variables. EM alternates between performing an expectation (E) step and a maximization (M) step. The E step computes the expectation of the log-likelihood evaluated using the current estimate for the latent variables. The M step computes parameters maximizing the expected log-likelihood found on the E step. These estimates are then used to determine the distribution of the latent variables in the next E step, up to convergence. In the mixture model context, the latent variable in the EM algorithm is the identity of the sub-population to which a specific data point should be attributed.

The specific computation of the E step and the M step depends on the underlying model. In Gaussian Mixture Models (gmms), the population is a predetermined collection of Gaussian regions, thus the function approximation is built as in rbfns [START_REF] Ghahramani | Learning nonlinear dynamical systems using an EM algorithm[END_REF].

ILO-GMR

In the context of machine learning methods for robotics, gmms are used mostly as an alternative to gpr methods for learning policies by demonstration (see Calinon [START_REF] Calinon | Robot programming by demonstration[END_REF] for a review). The corresponding methods are called Gaussian Mixture Regression (gmr), they are cubic in the number of samples, like gpr.

ilo-gmr, presented in Cederborg et al. [START_REF] Cederborg | Incremental local online Gaussian mixture regression for imitation learning of multiple tasks[END_REF], is one such method. However, one section of the paper is devoted to the comparison of ilo-gmr with lwpr, gpr, ν-svr, gmr, and lgp, to the problem of approximating the inverse velocity kinematics model of a Sarcos robot arm using the database provided by the authors of Nguyen-Tuong et al. [START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF]. The results show that ilo-gmr performs close to gpr and slightly better than lgp and standard gmr.

With respect to standard gmr, the main advantage of ilo-gmr lies in an efficient partitioning of the set of samples into regions based on an algorithm similar to kd-trees described in Angeli et al. [START_REF] Angeli | Fast and incremental method for loop-closure detection using bags of visual words[END_REF]. This allows one to break the cubic complexity of gmr and to quickly access the output for a given input.

In all other respects, ilo-gmr is identical to standard gmr, thus the model is defined as a fixed collection of Gaussian regions whose weights are trained with an EM algorithm.

Incremental Mixture of Linear Models

All the methods presented so far are approximating mechanical models as a mapping between an input space and an output space. This mapping is a function that returns a unique output for a given input. It can represent a forward model or an inverse model depending on which variables are used as input and which are used as output. Moreover, two functions may represent the inverse of each other if part of the input and the output are exchanged in the learning process.

In contrast with this standard approach, the method presented in Lopes & Damas [START_REF] Lopes | A learning framework for generic sensory-motor maps[END_REF] approximates a manifold in the space spanned by all input and output variables. This manifold can represent both a forward and an inverse model at the same time. It corresponds to a subspace that is compatible with the constraints generated by the considered model. Formally, if we call ν a [x T y T ] T point in the global space, the manifold can be represented by a constraint H(ν) = 0 that imposes the restriction that the point ν must comply with the model.

For instance, let us consider the kinematics model. The concatenation of any q vector and the corresponding ξ vector of task space position defines a point in a q × ξ space. All such points are lying on the manifold that characterizes the kinematics of the corresponding plant.

In practice, the authors are trying to approximate this manifold using a gmm approach. They try to estimate the probability of a point ν belonging to the manifold, i.e.,

p(H(ν) = 0|ν 1 , ν 2 , • • • , ν N ).
Definition of regions. More precisely, the manifold verifying H(ν) = 0 is approximated with a collection of regions that are defined as Gaussian functions with a center c m and a local covariance matrix C m . The probability of a data point ν i belonging to a region m is noted as p(ν i |m).

Evolution of regions.

The evolution of the parameters c m and C m of these regions is driven by an incremental implementation of the EM algorithm (see Lopes & Damas [START_REF] Lopes | A learning framework for generic sensory-motor maps[END_REF] for the equations realizing this incremental implementation).

Adding and deleting regions. The system starts with only one region and adds new regions when necessary. More precisely if, for some point ν, p(ν|m) is below a threshold p gen for all regions, then a new region n is added centered on this point ν and with a covariance initialized to maximize p(ν|n).

The system also computes an adjacency matrix between all pairs of regions based on the idea that if, for a point ν, p(ν|m 1 ) and p(ν|m 2 ) are both above a threshold, then m 1 and m 2 are very likely to be adjacent.

Global model approximation.

The global probability of a data point ν i belonging to the manifold verifying H(ν) = 0 results from a combination of the probabilities of all local regions. The difficulty here is that the system must provide either a forward or an inverse model depending on the need of the controller. Thus it is designed to answer to a query: given a specification of some coordinates of a point, what are (if any) the points lying on the manifold that match this specification?

For a given query ν q , after simple mathematical calculations, the system can provide for each region m the corresponding answer νam with a degree of confidence given by p(ν|m), where ν = [ν T q νT am ] T . But, in many cases, there will be either several discrete possible answer values or a continuous region corresponding to an infinity of such potential answers. Thus, a simple weighted average over the answers of all reliable regions does not provide a satisfactory solution.

Instead, the system uses the adjacency relationship described above to group together the answers of connected component and it provides one averaged answer and the corresponding degree of confidence for each group. By doing so, a set of answers may sample a continuous space of solutions.

Applications to robot model learning. To our knowledge, this system has not been applied so far to a robot. It was just compared with lwpr in simulated experiments. For the sake of easy reference, we call this system imlm (for Incremental Mixture of Linear Models) below.

Discussion

All the systems studied in Section 3 have been applied to learning the mechanical model of some plant. However, many other regression algorithms exist that could be applied to this problem. In particular, the regression methods used to learn a control policy from demonstration can generally be used as such to model identification problems. In that category, we already mentioned sogp and imgpe, for instance. We did not present this broader class of systems because it is well covered in Argall et al. [START_REF] Argall | A survey of robot learning from demonstration[END_REF], Calinon [START_REF] Calinon | Robot programming by demonstration[END_REF].

In this section, we focus on the incremental and local regression algorithms used for learning a mechanical model of robots presented in more details above. We do not discuss anymore the anns approaches that are not local because they do not fall into the unified view that is presented below. In practice, we focus on lwpr, xcsf, irfrls, lgp, imlm and ilo-gmr. The purpose of this section is to compare the systems by first providing a unifying view before discussing their strengths and weaknesses from the presentation of their properties. This part of the discussion is summarized in Table 1. Finally, we investigate from the unifying view whether their specific features could be combined into a new generation of systems.

Performance comparisons

Several papers present a performance comparison between some of the systems reviewed here [START_REF] Cederborg | Incremental local online Gaussian mixture regression for imitation learning of multiple tasks[END_REF][START_REF] Gijsberts | Incremental learning of robot dynamics using random features[END_REF][START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF][START_REF] Nguyen-Tuong | Real-time local Gaussian processes model learning[END_REF]. These comparisons are made possible thanks to the availability of a Sarcos and a Barrett arm database first published in Nguyen-Tuong et al. [START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF]. The papers generally compare the accuracy of the models, measured with a standard nmse calculation and sometimes the prediction time as a function of the number of samples.

However, the corresponding results are to be taken with care. Indeed, most of the algorithms are sensitive to meta-parameter tuning and there is no standard way to accurately perform this tuning so that all algorithms are given a fair chance. As a consequence, the same algorithm is attributed different performances in different papers.

A striking outcome of a survey of these results is that, despite its popularity, lwpr is generally less accurate than most of its competitors. This paradox can be explained by the fact that lwpr obtained impressive results when learning along a trajectory for very large plants whereas in the comparisons listed above it is used for learning over the whole space for smaller plants. We refer the reader to the list of papers above for other performance comparisons.

A unifying view

From the survey of the systems presented in Section 3, a unifying view emerges. We consider four aspects of these systems: the structure of the model, the way this structure evolves or not, the local regression method used and finally the meta-parameter tuning issue.

Structure of the models

If we consider Gaussian regions and linear models for xcsf, and Gaussian kernels for irfrls, then all the systems listed above include a Gaussian component and a linear component in their model. However, two broad classes of different structures must be distinguished. In lwpr, lgp and xcsf, the model is made of a Gaussian weighted combination of linear models, whereas in irfrls, imlm and ilo-gmr, it is made of a linear combination of Gaussian kernels. In that respect, the first class of systems is endowed with a more flexible structure, as already pointed out about lwr in Section 3.1.4.

Evolution of the structure

Table 2 summarizes in what respect the structure of the model can evolve in different systems. The systems are organized from the less flexible one, irfrls, where regions are generated randomly once and for all to the most flexible one, xcsf, where all elements of the structure can evolve.

Regression method

All the algorithms listed above perform local regression based on the minimization of an ls criterion, but the way to System iRFRLS ILO-GMR

LGP LWPR IMLM XCSF Addition of regions

• • • • Deletion of regions • Moving centers • • • • Moving shapes • • • •
Table 2: Capabilities of the systems to evolve the structure of the model. A • indicates that the system is endowed with the corresponding capability perform this minimization differs. As clearly stated in Nguyen-Tuong & Peters [START_REF] Nguyen-Tuong | Incremental sparsification for real-time online model learning[END_REF], standard svr approaches rely on a batch quadratic programming optimization process to get the linear parameters whereas gpr gets them through a matrix inversion (see Section 3.5). irfrls is an svr algorithm, but the batch optimization is replaced by an incremental version known as the QR algorithm in the field of adaptive filtering [START_REF] Sayed | Adaptive Filters[END_REF]. ilo-gmr relies on a standard EM algorithm. Finally, xcsf uses standard rls whereas lwpr relies on the more sophisticated nipals algorithm.

Tuning meta-parameters

In order to approximate an arbitrarily complex function in a large space, one must deal with a trade-off between the accuracy of the approximate model and the number of regions used to perform this approximation. Dealing with this trade-off is a matter of meta-parameter tuning. Different algorithms use a different strategy in that respect.

• In irfrls, the only meta-parameter is the number of Gaussian kernels. This makes it very easy to tune and very fast, but this approach should meet its limitations when many kernels are required

• In lwpr and lgp, the indirect parametrization is based on setting in advance the size of the regions through the w gen parameter. This is a poor compromise since one must try a w gen value, monitor the number of regions and the accuracy, and try again as long as the number of regions is too large or the accuracy too low.

• imlm sets a threshold p gen on the accuracy of the local models. Using this threshold provides guarantees in terms of accuracy, at the expense of computational complexity if a segmentation into many regions is necessary to reach the threshold.

• xcsf rather sets a maximum number of regions to perform the approximation. Such parametrization provides a good grip on the computational complexity of the process, at the expense of accuracy. But xcsf also uses a threshold on the global accuracy of prediction. Indeed, if the maximum population size is reached, the algorithm tries to improve individual classifiers in the population until the accuracy threshold is reached. However, if both constraints are not set with care, the process may diverge due to incompatible evolutionary pressures.

Key features of systems

The unifying view presented above reveals some of the features on which the systems can readily be compared. We can now highlight the strengths and weaknesses of these systems and exhibit the features that provide them with a unique advantage over their competitors.

LWPR

There are several reasons for the popularity of lwpr. First, a good documentation and an efficient and versatile code are available [46]. Second, lwpr has been convincingly applied to the identification of diverse models (kinematics, velocity kinematics and dynamics, forward and inverse) of large mechanical systems such as humanoid robots. This success was possible because lwpr is fast, reasonably accurate and because it performs dimensionality reduction in the prediction space of each local model. This dimensionality reduction feature is unique to lwpr among the systems we presented. Apart from its very appealing features, lwpr also suffers from several important limitations.

First, it comes with many meta-parameters and its performance is sensitive to their tuning. To temper this criticism, Klanke et al. [START_REF] Klanke | A library for locally weighted projection regression[END_REF] provides a useful "cookbook" to help determine how the meta-parameters should be tuned.

A second problem with lwpr is initialization. Because the centers of the region do not move during the learning process, lwpr is sensitive to the samples received in the first iterations. Thus, in order to get a good prediction accuracy with lwpr, it is necessary to start with an initialization stage that must ensure a good coverage of the latent function domain. However, this initialization stage can be difficult to realize in a concrete robotics set-up if the system is not endowed with the model it is supposed to learn.

Third, all the impressive results published on plants with many dofs have been obtained along some specific trajectory in the state space of the plant which makes the problem easier but also less relevant. It is easier because learning along a trajectory dramatically reduces the domain that has to be split into regions. With lwpr, covering a domain requires a number of regions that is roughly exponential in the number of dimensions of the domain: the dimensionality reduction property provided by nipals is applied to the local model in each region but does not result in a significantly lower number of regions. Furthermore, experiments show that the performance of lwpr collapses as soon as the number of regions gets above a few tens of thousands. The problem is also less relevant because controlling a plant along an articular trajectory prevents making profit of redundant dofs. To face this difficulty, the learning mechanism has to be endowed with a good extrapolation capability, which is generally not the case with the local methods studied in this paper in general and with lwpr in particular.

XCSF

xcsf shares a lot of similarities with lwpr in the structure of the learned models. However, it differs in the algorithms used to update this structure and in the meta-parameters that constrain the algorithms. A specific comparison between both systems has shown that xcsf generally outperforms lwpr in standard function approximation problems, even if lwpr usually converges faster [START_REF] Stalph | A comparative study: Function approximation with LWPR and XCSF[END_REF]. Among other things, xcsf can be tuned more intuitively and its evolutionary component can modify the centers of the approximation regions. Given that xcsf is available in Java [119], the main reason for its lesser use in the model learning community is probably its lesser maturity in the field.

But the unique feature of xcsf is the condition space versus prediction space distinction. This feature can result in important savings in the context of learning mechanical models of a plant. We illustrate these savings in the context of learning the forward velocity kinematics model and the inverse dynamics model.

According to (1), learning a forward velocity kinematics model with lwpr requires the joint positions and velocities q and q as input and the task velocity ξ as output. But, taking a closer look at the equation ξ = J (q) q, one can see that it relates linearly ξ to q for different contexts based on q. Thus one can use the values of q to define the condition space and the corresponding relation between ξ and q to learn the local models in the prediction space.

Instead of segmenting a (q, q) space into regions to approximate the forward velocity kinematics function, one just needs to segment a (q) space. As a result, the condition space is twice smaller with xcsf than with any other local regression method.

The same is true when one considers an inverse dynamics model. Equation ( 5) describing the dynamics of a polyarticulated system can be reformulated Γ = A (q) q + B (q, q) .

(

) 23
This equation can be viewed as a linear relation between q and Γ with dependencies on q and q Γ = A (q) B (q, q) q 1 .

In this context, B (q, q) can be seen as an offset. Thus, similar to the velocity kinematic case, one can learn an inverse dynamics model by providing at each time step a state (q, q) as condition parameters and a ( q, Γ) pair as input to the prediction space. This approach is less expensive in terms of dimensionality than the standard one. Indeed, one needs to span (q, q) instead of (q, q, q) to learn the model. As a result, the space is 1/3 smaller.

In Section 3.1.4, we noted that the major improvement in function approximation techniques brought by lwr has consisted in separating a linear model from the radial basis function that delimits the corresponding region of influence. In xcsf, the idea is put one step further: since those entities are separated, they do not need to be based on the same dimensions, thus the global input vector can be split into a condition part and a prediction part. The autonomous determination of which input should be used in the condition part, in the prediction part or just ignored is a matter for future research.

IRFRLS

The main feature of irfrls is its constant computational complexity. This property results from the segmentation of the latent function domain into a set of random kernels, which is the simplest possible mechanism. The number of kernels is the only meta-parameter that needs to be tuned. On the data sets presented in Nguyen-Tuong et al. [START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF] and other data sets extracted from the James and iCub humanoid robots, irfrls obtains a surprisingly good performance compared to its competitors. However, the random kernel approach obviously results in a suboptimal structure of the model, which may become critical when the model has to be very large. This system being very recent, more work is required to better determine its empirical properties, but given its simplicity, it is a good starting point for a new family of systems.

4.3.4.

LGP lgp was designed after lwpr and has been compared favorably to its ancestor in several contexts [START_REF] Nguyen-Tuong | Learning inverse dynamics: a comparison[END_REF][START_REF] Nguyen-Tuong | Real-time local Gaussian processes model learning[END_REF]. One difficulty with lwpr being meta-parameter tuning, a strong incentive for calling upon gpr methods is that they generally come with much less meta-parameters. Moreover, by splitting the domain of approximation into regions, lgp breaks down the computational complexity of standard gpr methods. It is still slower than lwpr, but it also inherits the greater accuracy of gpr. However, some meta-parameters inherited from lwpr for the segmentation into regions such as w gen are still difficult to tune.

Because it performs local regression based on a gpr process, lgp is strongly driven towards the use of sparse data for each region. Sparsification is the main feature brought into this survey by lgp.

Along this line of research, one may wonder if using standard Bayesian linear regression instead of gpr in a mixture of experts approach would not result in a more efficient approach, the features necessary for Bayesian linear regression being provided by the segmentation into regions.

IMLM

As explained in Section 2.2, the forward velocity kinematics model of a redundant plant is unique whereas there exists an infinity of corresponding inverse models. Conversely, for a parallel plant, the inverse velocity kinematics model is unique whereas there is an infinity of forward models. A first appealing property of the approach of Lopes & Damas [START_REF] Lopes | A learning framework for generic sensory-motor maps[END_REF] is that it can store all such models from data without knowing in advance if the model is unique nor losing the information corresponding to the potential infinity of solutions.

A second appealing property is that no matrix inversion is necessary to retrieve either a forward or an inverse information from the stored data. Intuitively, in the case of kinematics, one may provide a vector q or a vector ξ and retrieve the complementary part by looking for all points in the manifold that share the specified coordinates.

Nevertheless, retrieving the required information from such a manifold is not straightforward. The manifold being approximated by a set of Gaussian regions, the system returns one value for each region whose probability of belonging to the manifold is above a threshold. As a consequence, when there is an infinity of correct answers to a query, the system only returns a set of discrete answers sampling the domain of correct answers. Furthermore, the accuracy of the sampling is sensitive to the size of the regions.

Finally, when there are several correct answers, one must take additional constraints into consideration to choose a specific answer among the possible ones. Despite a first solution proposed in Lopes & Damas [START_REF] Lopes | A learning framework for generic sensory-motor maps[END_REF], this point is still an open problem.

ILO-GMR

ilo-gmr is a straightforward extension of gmr systems designed to perform learning from demonstration. It is a simple approach, whose main drawback is that the number of kernels have to be determined in advance. However, an interesting feature of ilo-gmr is its use of kd-trees to organize the regions and the corresponding samples so as to quickly access them. An extension of gpr with kd-trees has also been proposed in Shen et al. [START_REF] Shen | Fast Gaussian process regression using kd-trees[END_REF].

Combining Features from Several Systems

In the previous section, the key features of the studied systems have been highlighted. Now, we discuss whether these features could be combined in order to build a new generation of systems.

Dimensionality Reduction

The incremental dimensionality reduction property provided to lwpr by the nipals algorithm is a key feature of lwpr. It is particularly adequate for learning models along trajectories in spaces with many dimensions, some of which being irrelevant along the given trajectory. xcsf learns the same kind of linear models as lwpr, using rls instead of nipals. It seems that nipals could replace rls in xcsf without major modifications of the rest of the algorithm. Whether a similar dimensionality reduction property could be introduced into gpr or gmm based methods such as lgp and imlm is a more difficult open issue.

Condition Space versus Prediction Space Distinction

As outlined above, lwpr, lgp and xcsf both build their model as a Gaussian weighted combination of linear models. As a consequence, the distinction between prediction space and condition space used in xcsf could certainly be transferred to lwpr and lgp. In contrast, such a transfer would be much less immediate for all the other systems that use a different structure for the approximation of the latent function.

Learning the Input-Output Manifold

Learning the input × output manifold is unique to imlm. All the other methods are designed to approximate a function rather than a manifold. Anyway, even if replacing the gmm foundation of that system by other regression methods such as the one used in xcsf or lgp is probably an option, more work is required on top of imlm in order to efficiently use its unique feature.

Meta-parameter optimization

From Section 4.2.4, the meta-parameters used in imlm and xcsf look easier to tune than the one used in lwpr and lgp, with a slight advantage to xcsf. Whether the approach used in xcsf can be transposed to the other systems is an open question.

Alternatively, one may think of dealing with the accuracy versus number of regions trade-off with a multi-objective function optimization approach. Such an approach would generate a population of non-dominated solutions among which a very accurate solution, a very sparse one or any compromise between both extrema could be chosen by the user.

Sparsification and fast access to samples

It seems that the sparsification algorithm proposed in lgp is not specific of this algorithm in any way. Thus, it could probably be included in most of the other systems, giving rise to immediate computational savings.

Similarly, the fast access to samples brought by the use of kd-trees in ilo-gmr could certainly be imported in most if not all of the systems listed in this survey.

A new system that incorporates these properties as well as some of those that we listed in the previous sections remain to be built, giving rise to a new generation of more efficient systems.

Towards more advanced systems

Most of the experimental work presented so far consists in learning mechanical models of a robot without any interactions with its environment. In this context, the resulting benefit is limited. Indeed, the mechanical structure of a robot is generally almost perfectly known through the cad model, thus a good engineering work and an eventual additional parametric identification process can suffice to get an accurate model.

Obviously, the methods presented in this paper are becoming more advantageous when they address the robotics contexts outlined in the introduction. Future robots will interact physically with unknown objects and will even do so with unknown tools given the fast development of dexterous manipulation capabilities [START_REF] Bicchi | Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity[END_REF][START_REF] Grupen | A survey of general-purpose manipulation[END_REF][START_REF] Kaneko | Development of multifingered hand for life-size humanoid robots[END_REF]. Even more convincingly, they will interact with human users that are highly unpredictable. In all these cases, a model provided in advance cannot generally account for unspecified interactions. In this section, we first stress the challenges raised by these more difficult contexts of use, then we survey some new lines of research that start addressing them.

New Challenges

To face the challenges of interactions with the environment, the next generation of systems will have to address problems with many more dimensions, they will have to adapt faster, and they will have to memorize several models corresponding to different contexts of use, together with the capability to switch between these models appropriately.

Challenge 1: Plants with more dimensions

Among the algorithms surveyed here, lwpr is the only one that has been used on a plant as complex as a humanoid. But only one arm of the robot was moving and, above all, learning was performed along a specific trajectory. Learning kinematics and dynamics model of a more than 10 dofs robot over the whole space is still beyond the capabilities of all state-of-theart methods. However, for a humanoid engaged in a complex everyday-life mission, having such a complete model is necessary. A key property for addressing these larger problems without facing hard computational complexity challenges is generalization. Future systems should definitely be able to extrapolate the models they are learning beyond the domain where they were trained, which is generally not the case of the local learning methods surveyed in this paper.

Challenge 2: Faster adaptation

A barely addressed issue in all the papers surveyed here is speed of adaptation. Indeed, when a robot interacts with different objects or users, one would like the model to adapt to a new context of interaction fast enough to deal immediately with this new context. For instance, if a robot is pouring water from a container, the weight is changing quickly. Or, if a robot is in physical interaction with an user, the force exerted by the user may vary at any moment.

It is hard to accurately present the state-of-the-art on that topic in the absence of any systematic study, but the methods presented in this survey are not equivalent in that respect and are probably slower than what is required for the kind of nonstationary interactions just described above.

Challenge 3: Switching between contexts

If a robot has to physically interact with several users or objects, eventually with some tools, the systems described in this paper must be enriched with two capabilities: the capability to learn different models corresponding to the different contexts of interaction and the capability to switch between these models appropriately. Otherwise, the system would spend a lot of time re-learning the same models back and forth. An additional requirement is that the system is able to recognize which model it should be using in a given context or to determine that a new model is necessary.

New Lines of Research

Corresponding to the above challenges, two of the lines of research below consist in incremental improvements of the existing systems whereas the third line consists in an extension of the addressed domain. However, each of the two first lines contributes to all challenges simultaneously.

Line 1: Combining knowledge and regression

A way to increase the speed of adaptation and to deal with larger plants is to rely more on expert knowledge about the plant. Indeed, the model learning approach does not benefit from prior knowledge of the rbd model of the plant, though this knowledge is generally available. Recently, the authors of Nguyen-Tuong & Peters [START_REF] Nguyen-Tuong | Using model knowledge for learning inverse dynamics[END_REF] combined the strengths of rbd identification and machine learning methods, using gpr for learning the difference between the parametric mapping and the observed behavior.

Their method provides a higher model accuracy and better generalization for unknown trajectories compared to rbd identification and gpr-based methods. Furthermore, it shows a good learning performance even on sparse and noisy data.

Though their results are promising, it is still unclear how prior knowledge helps learning a robot dynamics, notably depending on the control law used to actuate the robot. Furthermore, their semi-parametric regression approach does not alleviate the methodological difficulty of performing identification, which, in most cases, is not straightforward.

A different approach to the same problem is presented in Sun de la Cruz et al. [START_REF] De La Cruz | Online incremental learning of inverse dynamics incorporating prior knowledge[END_REF], where the authors incorporate prior knowledge of the rbd model of the plant in lwpr by initializing all linear models with a first order approximation of the known rbd. The application to a simple simulated system convincingly improves the learning and generalization performance.

Other methods based on visual processing such as Hersch et al. [START_REF] Hersch | Online learning of the body schema[END_REF], Mansard et al. [START_REF] Mansard | Jacobian learning methods for tasks sequencing in visual servoing[END_REF], Martinez-Cantin et al. [START_REF] Martinez-Cantin | Body schema acquisition through active learning[END_REF], Sturm et al. [START_REF] Sturm | Unsupervised body scheme learning through self-perception[END_REF] incorporate prior knowledge of the structure of the body of the robot. These methods are surveyed in Hoffmann et al. [START_REF] Hoffmann | Body schema in robotics: A review[END_REF]. The promising line of research corresponding to all these works should be pursued whatever the model learning method used.

With a very different perspective, the system presented in Ulbrich et al. [START_REF] Ulbrich | Rapid learning of humanoid body schemas with kinematic bezier maps[END_REF] learns the forward kinematics model of mechanical systems using Bezier curves. Their method is based on the insight that the forward kinematics model of a mechanical system with rotational joints mainly calls upon trigonometric functions. It spans efficiently the parametric space of such functions in order to get an accurate model from sparse data.

Though this method does not rely on a parametric model of the robot in the classical sense, it is based on a strong assumption about the structure of the model. On the one hand, this assumption makes the parameter tuning more efficient. On the other hand, it makes the method less general. In particular, it can only be efficient when learning the forward kinematics model of a plant whose dofs all result from rotational joints. The extension to prismatic joints and to dynamics must call upon additional assumptions.

Line 2: Active learning

Different ways to span the input/output space can result in a large difference in the time necessary to learn an accurate model over the whole space. Thus it can increase both the speed of adaptation and the size of the problems that can be addressed. In Machine Learning, choosing appropriately the set of learning examples so as to increase the learning speed is called active learning. Active learning of a mechanical model is a difficult topic because it generates a "chicken and egg" problem. Indeed, in order to choose the samples that feed the learning process, it is necessary to know how to drive the plant towards the corresponding states. But to do so, having an accurate model is generally required. A way to escape this difficulty consists in calling upon model-free control methods such as articular PID controllers to drive the system towards areas of interest. So far, there are few attempts to design specific active learning methods for learning mechanical models of robots, exceptions being Baranes & Oudeyer [START_REF] Baranes | R-IAC: Robust intrinsically motivated exploration and active learning[END_REF][START_REF] Baranes | Maturationally-constrained competence-based intrinsically motivated learning[END_REF], Martinez-Cantin et al. [START_REF] Martinez-Cantin | Body schema acquisition through active learning[END_REF], Robbel [START_REF] Robbel | Active Learning in Motor Control[END_REF], Saegusa et al. [START_REF] Saegusa | Active learning for sensorimotor coordinations of autonomous robots[END_REF]. Given the growing interest in active learning in general, important progress can be expected in this domain in the near future.

Line 3: Switching models and sharing between models

In Petkos et al. [START_REF] Petkos | Learning multiple models of non-linear dynamics for control under varying contexts[END_REF], Petkos & Vijayakumar [START_REF] Petkos | Load estimation and control using learned dynamics models[END_REF], the authors address the problem of switching between models raised in Section 5.1.3. They rely on lwpr combined with an EM algorithm to learn elementary models and use a latent variable to account for the different unknown contexts of interaction. So far, their work is just a proof of concept. It should be extended with perception processes to help the system recognizing the different contexts based on richer information than just the articular state of the robot. Along a different line of research, given the infinity of inverse velocity kinematics models of a redundant plant, a different model can be used to achieve each different task in a different way. Thus, addressing a complex mission composed of several tasks can also be seen as a problem of learning several models that must be memorized in parallel. Some authors are using this approach and learn mechanical models in the context of multiple tasks based on gpr. More interestingly, since the inverse dynamics model is unique but may be involved in the realization of diverse tasks at the kinematic level, one can see the problem of learning this inverse dynamics model through a first task and using it for other tasks as a transfer learning problem [START_REF] Chai | Multi-task Gaussian process learning of robot inverse dynamics[END_REF][START_REF] Yeung | Learning inverse dynamics by Gaussian process regression under the multi-task learning framework[END_REF]. On top of those ideas, a next challenge will consist in generalizing what is learned in interaction with one object to interaction with a class of similar objects.

Conclusion

The on-line modeling of robotics system with regression methods is a very active field. In this paper, we surveyed in particular local and incremental algorithms currently used to solve this problem, providing a unifying view and comparing their features. Among other things, we have shown that, despite its popularity and efficiency for some problems such as learning along a trajectory in a large domain, lwpr suffers from several important drawbacks that justify the important effort dedicated to looking for a better algorithm.

Furthermore, we have shown that several systems come with interesting additional features, such as the condition space/prediction space distinction in xcsf or the capability to learn both the forward and inverse model into a unique manifold in imlm. We highlighted that there is still room for large performance improvements in this domain just by combining some the features of these algorithms. Further ideas such as the multi-objective optimization of the trade-off between the accuracy and the number of regions might also give rise to new lines of research. Finally, we are convinced that the application of such combinations of algorithms should help addressing in the near future the "new frontier" of model learning, such as the physical interaction with unknown objects, tools and users.

Figure 1 :

 1 Figure 1: Historical view of several lines of research on supervised learning algorithms giving rise to incremental and local regression methods to infer mechanical models

Figure 2 :

 2 Figure 2: Approximation (doted) of a sinus function (top in black) with an rbfn (bottom in cyan).

Table 1 :

 1 Summary of the discussion of the systems

	System	LWPR	XCSF	iRFRLS	LGP	IMLM	ILO-GMR
	Based on	LWR	LCSs	SVR	GPR + LWR	GMMs	GMR
	Main feature	Dimensionality reduction	Prediction vs. condition space distinction	Constant computa-tional complexity	Sparsification	Bidirectional mapping	Use of kd-trees for fast access
		Available in
	Other assets	C++ and matlab with cookbook. Fast.	Available in Java. Flexible	Simple. Easy tuning	Less meta-than LWPR parameters	No matrix required inversion	Simple
		Widely used
	Drawbacks	Hard to tune. Initialization stage required	Complex. Sensitive to tuning	No improvement of features	Slower than LWPR	Less mature, not used on real robots	Fixed number of features
	Favorite context of use	Large systems, along a trajectory, with dynamics	Learning on whole space	Learning on whole space	Sparse data, requiring accuracy	Redundant and parallel systems	Learning on whole space

Again, not to be confused with krls

Acknowledgments

This work is supported by the French ANR program (ANR 2010 BLAN 0216 01), more at http://macsi.isir.upmc.fr