Kazuya Otani
email: kazuya.otani@inria.fr

Karim Bouyarmane
email: karim.bouyarmane@loria.fr

Serena Ivaldi
email: serena.ivaldi@inria.fr

Generating Assistive Humanoid Motions for Co-Manipulation Tasks with a Multi-Robot Quadratic Program Controller

Human-humanoid collaborative tasks require that the robot take into account the goals of the task, interaction forces with the human, and its own balance. We present a formulation for a real-time humanoid controller which allows the robot to keep itself stable, while also assisting the human in achieving their shared objectives. We achieve this with a multirobot quadratic program controller, which solves for human motion reconstruction and optimal robot controls in a single optimization problem. Our experiments on a simulated robot platform demonstrate the ability to generate interaction motions and forces that are similar to what a human collaborator would produce.

I. INTRODUCTION

Robots have traditionally been used in isolation, e.g. within cages in factories. This is because robots were only capable of repeating pre-programmed behavior, and did not have any means of adjusting to the environment around them. In recent years, improved sensing capabilities and safer hardware have allowed robots to move into shared spaces with humans, such as factory floors and homes. In many cases, the robots do not replace humans; they complement human capabilities and relieve them of arduous tasks. Comanipulation systems combine the reasoning abilities of human operators with the precision and power of robots, which results in increased productivity and human comfort.

An ideal co-manipulation robot would be able to:

1) Learn general categories of motions to be performed 2) Adapt to slight variations in conditions (e.g. human or object position) 3) Actively aid the human in achieving shared goals In this paper, we propose a humanoid robot controller for physical human-robot interaction (pHRI) tasks that addresses the third point: a controller that can produce optimal motions in real-time, considering both the robot's stability and human assistance goals. This is achieved via a multi-robot quadratic program controller, which maintains an internal representation of the human's whole-body dynamics along with those of the robot and any other manipulated objects. We demonstrate in simulation experiments that the controller can generate interaction motions/forces similar to what a human partner would produce. Fig. 1. In collaborative tasks, the humanoid robot must reason about: goals of the task, interaction forces with the human, and its own balance. A human collaborator equipped with wearable sensors can provide further information about the human movement during the task.

A. Related Work

Control algorithms for physical human-robot interaction (pHRI) have several modalities available for the robot's perception of the human: vision (e.g. gesture/posture detection), sound (e.g. voice commands), bio-signals (e.g. EMG), and force/pressure sensors (e.g. force-torque on end effectors). Of these modalities, force-torque sensors (haptic data) have been used most often, due to the simplicity of the underlying mechanism. The haptic data is usually used to regulate the robot's impedance (resistance to motion caused by external forces) in order to generate compliant behavior [START_REF] Kosuge | Control of a robot handling an object in cooperation with a human[END_REF] [START_REF] Ikeura | Variable impedance control of a robot for cooperation with a human[END_REF].

Hybrid methods using multiple sensing modalities have been shown to improve capabilities in certain tasks. Early human-humanoid interaction work by Yokoyama et al showed that visual and haptic feedback, along with voice commands, allow a humanoid robot to collaboratively carry a large object with a human [START_REF] Yokoyama | Cooperative works by a human and a humanoid robot[END_REF]. Agravante et al use a similar approach to collaboratively carry a table while balancing a free-rolling ball on the table [START_REF] Agravante | Collaborative human-humanoid carrying using vision and haptic sensing[END_REF]. They use a decoupled approach which separates manipulation and balance control, while the human is sensed as an external force.

A limitation in most existing controllers for physical human-robot interaction is that the representation of the human state is vastly simplified as an external force or end effector poses. The whole-body configuration and the dynamics of the human are rarely considered, which limits the types of interaction that are possible. For example, it is impossible to reason about the human's ergonomics (e.g. posture, exertion) without at least a kinematic model of the human body.

Among the works that do consider a full human model, there have been applications in: optimizing human comfort in selecting handoff configurations [START_REF] Bestick | Implicitly assisting humans to choose good grasps in robot to human handovers[END_REF], offline trajectory optimization for predicting the motion of an exoskeleton-assisted human [START_REF] Millard | Predicting the motions and forces of wearable robotic systems using optimal control[END_REF], finding configurations for a fixed-base manipulator that minimize human exertion in a co-manipulation task [START_REF] Kim | Anticipatory robot assistance for the prevention of human static joint overloading in human-robot collaboration[END_REF]. These methods are used for higher-level planning and/or only consider kinematic data. The work in this paper was developed to test our hypothesis that there are certain tasks, such as co-manipulation and balance assistance, in which having a full-body dynamics model of the human is useful in providing optimal interaction motions.

For humanoid control, quadratic program (QP) controllers have become ubiquitous in recent years due to their robustness, speed, and flexibility. QP controllers allow intuitive encodings of constraints and objectives in an optimization problem that can be solved at real-time control rates. Early QP control work of Abe et al [START_REF] Abe | Multiobjective control with frictional contacts[END_REF] and De Lasa et al [START_REF] De Lasa | Feature-based locomotion controllers[END_REF] demonstrated multi-objective control for generating natural movements in computer animated characters. In the recent DARPA Robotics Challenge, a majority of the teams relied on a QP for their low-level full-body controller [START_REF] Bouyarmane | Exploring humanoid robots locomotion capabilities in virtual disaster response scenarios[END_REF], [START_REF] Feng | Optimization based controller design and implementation for the atlas robot in the darpa robotics challenge finals[END_REF], [START_REF] Kuindersma | Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot[END_REF].

Our approach makes use of the multi-robot QP (MRQP) controller introduced by Vaillant et al for animated characters [START_REF] Vaillant | Multi-character physical and behavioral interactions controller[END_REF], and applied to real robots in [START_REF] Bouyarmane | Multirobot and task-space force control with quadratic programming[END_REF]. The MRQP extends QP-based humanoid controllers to consider the combined dynamics of multiple robots.

We use a QP controller to map the motion capture data onto a simulated human; the use of a task-based controller and an approximate model for the reconstruction of human motions has been shown to be quite reliable in prior works. In [START_REF] Khatib | Robotics-based synthesis of human motion[END_REF], a controller that seeks to minimize position error (against human markers) and muscular effort on a musculoskeletal model are shown provide a good match with motion capture data. In [START_REF] Maurice | Virtual ergonomics for the design of collaborative robots[END_REF], a QP controller is used to simulate human motion and is empirically shown to produce similar trajectories to real humans for multiple motions.

B. Contributions

Our contribution is a formulation for reconstructing the dynamics of human motion and solving a humanoid control problem in a single optimization, using a multi-robot QP controller to explicitly model the whole-body dynamics of both the human and the robot. This allows for real-time control of a humanoid robot during co-manipulation tasks, generating motions that simultaneously keep the robot balanced and assist the human. We demonstrate that the controller can assist the human in symmetric co-manipulation tasks, as well as in maintaining balance.

The rest of this paper is organized as follows: Section II reviews the formulation of QP and MRQP controllers. Section III outlines our approach for modeling human dynamics and physical interaction in a MRQP. Section IV demonstrates the advantages of our controller through experiments in simulation. Section V discusses the results and limitations of this method. Section VI draws conclusions and discusses possibilities for future work.

II. PRELIMINARIES

A. Weighted QP control QP control for humanoid robots consists in solving, at each control time step, the following optimization problem:

min q,τ,λ k w k ||ÿ k -ÿd k || 2 , (1)
subject to:

              
dynamics constraints contacts no-slip (with environment) forces within friction cones joint position, velocity, torque limits collision avoidance

(2)

The decision variables of this problem are the configuration acceleration q, where q denotes the configuration of the robot including its 6D free-floating base, τ are the actuation joint torques, and λ the coefficients of contact forces along linearized friction cones. Using a linear approximation of the friction cones at the contact points, and by carefully writing the joint limits and collision-avoidance constraints to be linear in the decision variable vector, the optimization problem above is a QP [START_REF] Bouyarmane | Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations[END_REF].

y k denote quadratic objectives that encode desired motion, which we will call "tasks", mapped from the configuration space to any kind of 'operational space' with Jacobians, as shown in [START_REF] De Lasa | Feature-based locomotion controllers[END_REF]. ÿd k is the desired acceleration of the task, which can be derived from a desired trajectory or set-point of the task. w k is the weight of the task.

The QP controller is often at the bottom of a planning hierarchy, whose top layers produce higher-level plans using simplified models of the robot [START_REF] Feng | Online hierarchical optimization for humanoid control[END_REF] or multi-contact plans with nonlinear optimization [START_REF] Bouyarmane | Humanoid robot locomotion and manipulation step planning[END_REF].

B. Multi-Robot QP control

In [START_REF] Bouyarmane | Multirobot and task-space force control with quadratic programming[END_REF], the QP control framework is extended to control systems of multiple robots interacting among each other or with a dynamic environment. Let n be the number of such entities. In a collaborative robot-robot co-manipulation scenario, n = 3 (the two robots and the manipulated object).

The MRQP framework is based on the assumption that each entity i (i = 1 . . . n) can be modeled with the general dynamics equation of motion

M i qi + N i = p J T i,p f i,p + S T i τ i , (3)
which accounts for all possible actuated/non-actuated/underactuated cases through the control selection matrix S i . The general equation (3) also accounts for all possible fixedbase/floating-base cases through the inclusion (or not) of the the base configuration in q. We can thus use this equation to model humanoid robots (possibly multiple ones), as well as floating objects or articulated passive environments (e.g. doors).

The n entities physically interact with each other by exchanging contact forces that come in action-reaction pairs, according to Newton's second law. Each contact force f i,p on entity i is either applied by the fixed inertial environment, or is applied by one of the other entities j and appears with an opposite sign in that entity's equation as f i,p = -f j,p .

By rearranging the forces, we can keep exactly one representative of each action-reaction pair (f i,p , f j,p ) as decision variables of the MRQP, showing that there exists a permutation matrix Ψ such that we can combine all the equations of motion of the n entities in one single equation:

M (q)q + N (q, q) = J T 0 F 0 + (J 1 -Ψ T J 2 ) T F 1 + Sτ (4)
where q, τ , N denote respectively the stacked vectors of generalized configurations, actuation torques, and nonlinear effects respectively, M and S are the stacked block-diagonal matrices of M i and S i block-diagonal elements respectively, F 0 is the stacked vector of fixed inertial environment contact forces with corresponding stacked block-diagonal Jacobian matrix J 0 , F 1 the stacked vector of single representatives of action-reaction pairs, J 1 and J 2 are Jacobian matrices corresponding to the contact points between the different entities of the multi-robot system (see detailed derivations of these matrices in [START_REF] Bouyarmane | Multirobot and task-space force control with quadratic programming[END_REF]). It can also be demonstrated that the no-slip contact constraints between the different entities and with the environment can be written as J 1 q = ΨJ 2 q and J 0 q = 0.

Once the combined dynamics are formed, the formulation is the same as a single-robot QP controller, with the addition of the constraints for contacts and collision-avoidance between the entities. Denoting λ 0 and λ 1 as the coefficients along the linearized friction cone generators of F 0 and F 1 respectively:

min q,τ,λ 0 ,λ 1 k w k ||ÿ k -ÿd k || 2 , (5)
subject to:

                  
combined dynamics equation contacts no-slip (with environment) forces within friction cones joint position, velocity, torque limits collision avoidance contacts no-slip (between robots)

This optimization problem can be solved at real-time rates (200Hz) to control multiple robots. With an interface that simplifies the specification of desired behavior as quadratic objective tasks, the multi-robot QP allows for the generation of complex interaction behaviors from easily-interpretable objectives. The tasks can be written for the combined system (e.g. combined CoM) or benefit from the reduced representation to imply desired behavior through a task on a single robot (e.g. task on the position of the co-manipulated object that drives all the robots in contact with the object). In previous works, this has been used for collaborative carrying between two robots, environment object (box, door) manipulation, and adaptive human-to-humanoid motion retargeting.

In this paper, we consider three entities (n = 3): humanoid robot, human (using the method described in the next section), and a co-manipulated object.

III. METHODS

A. Human model

We incorporate a whole-body dynamics model of the human into the multi-robot QP as one entity of the system whose dynamics are modeled by equation [START_REF] Yokoyama | Cooperative works by a human and a humanoid robot[END_REF]. The simulated human tracks the real human's motions, which the robot then uses as a virtual approximation of the real human to reason about the human-robot system's combined dynamics.

In formulating our multi-robot QP, we assume that the whole-body configuration of the human is known. To capture human motion we use an Xsens MVN inertial motion capture suit [START_REF]Xsens MVN[END_REF], a wearable system with 17 wireless IMUs embedded in a lycra suit, which tracks motion at 240Hz. IMU data are mapped onto the motion of a calibrated biomechanical model with 23 segments, connected via 22 threedimensional joints. The motion capture system provides: link positions/orientations and joint angles, along with their first and second time derivatives. In the future, we envision that the motion capture suit could be replaced with a visual pose detection system such as [START_REF] Shotton | Real-time human pose recognition in parts from single depth images[END_REF].

The Xsens motion capture data is mapped onto our simulated human model, which is a 22 joint rigid body tree model, described in a URDF format. The limb lengths and masses are parametrized as a function of the subject's height and weight, based on average anthropometric coefficients [START_REF] Chaffin | Occupational biomechanics[END_REF], with further customization possible to match a subject's specific measurements.

B. Human motion reconstruction

To accurately reconstruct human motion with our MRQP, we set high-weight motion tracking objectives on the human "robot", which take motion capture data as time-varying setpoints.

We use a configuration task (objective on desired generalized configuration qdes ) to track the joint angles of the human's motion. On top of that, we set human motion tracking tasks on end effector pose (position and orientation tasks), since this is the most important feature of human motion for applications involving physical interaction (these are the links most often in contact with the human and external environment). These two tasks combine to produce a good approximation of the motion capture data on our simulated human.

To integrate the human motion reconstruction into a multirobot QP, we investigated two methods (illustrated in Fig- ure 2).

1) Single MRQP: Perform human motion reconstruction directly in the multi-robot QP, ensuring that the simulated human tracks the real human's motion well by setting human motion tracking task weights to be significantly higher than robot's task weights. This is the "soft priority" approach. 2) Cascaded QP-MRQP: Have a single-robot QP for human motion tracking with the simulated human (QP1), and a multi-robot QP for calculating the robot's motion considering the human's dynamics (QP2). The solution for the human motion qhuman from QP1 is set as an equality constraint on the human model's motion in QP2. This is the "hard priority" approach.

Method 1 has the advantage that it is simpler to implement and is less computationally expensive (it only solves one QP). The disadvantage of this method is that it requires tuning of the weights to trade off between rigidly tracking human motion (so the controller has an accurate tracking of the real task performed by the human) and achieving the robots tasks. In practice, we found that this method could reconstruct the human motion with reasonable accuracy (see Section IV and Figure 4).

Method 2 decouples the human model's motion from the optimization for the robot's motion, which allows stricter tracking of the motion capture data by the simulated human. However, in our experiments we found that it was less stable. We often saw the QP fail because of violated constraints, especially for fast movements. Accumulated numerical errors caused drift between the simulated human's states in QP1 and QP2, which meant that large q commands would cause differences in end effector motions large enough for other constraints (friction cone, no-slip contact) to be violated. In this context, Method 1 can be interpreted as a way to allow some "slack" in the human motion tracking to make the QP solvable.

For the reasons above, our implementation used Method 1 for integrating human motion tracking into the MRQP framework.

Collaboration between humans and robots often involves co-manipulation of objects (e.g. collaborative pick & place of large/heavy objects). In those cases, the objects also need to be modeled and added to the QP problem. Following the multi-robot QP framework, objects are modeled as independent entities with their own dynamics equations. We make the assumption that the geometrical model and inertial parameters of the manipulated object are known.

C. Robot objectives

The quadratic objectives in the MRQP controller define the metrics that the robot seeks to minimize in its motions. The tasks that we use in our approach can be separated into two categories: individual objectives for the robot, and interaction objectives that define the robot's behavior as a function of the human's motion.

For the robot's individual tasks, we set objectives that keep the robot balanced and in a natural posture away from singularities.

Balance is encoded in a CoM task, which encourages the robot to keep its CoM above the center of its support polygon, and a CoM bound constraint, which limits pdes CoM to ensure that the ground projection of the robot's CoM does not go outside its support polygon, defined in hyperplane representation {p CoM | Ap CoM ≥ b}. This is implemented as a damping behavior that slows down the CoM as it nears the boundaries of the convex hull:

ḋ + d∆t ≥ -ξ d -d s d i -d s ( 7
)
with d is the distance between p CoM and the nearest hyperplane, d i is the interaction distance at which damping turns on, d s is the security distance (minimum distance by which to stay inside convex hull), ξ the damping coefficient. We include a posture (full configuration space) task on the robot, whose reference is set to a rest posture q 0 with a low weight. This task ensures that the optimization is wellconditioned. In practice, it also helps generate more naturallooking movements.

D. Interaction objectives

The tasks described above encourage the robot to be individually balanced. We then add objectives that define the desired interaction between the robot and the human. These objectives define the robot's motion as an implicit function of the human's motion, which is solved for by the MRQP at each timestep.

The first interaction task is a regularization/minimization objective on the robot's contact forces with other entities (subvector of λ in Equation ( 5)), to avoid unrealistic behaviors in which the humanoid leans on/pushes other entities excessively.

For the experiments in this paper, we assumed that the human and the robot are performing a symmetric motion in which they face each other and perform mirrored versions of each other's motions. To achieve this, we set the desired pose of the robot's end effector to be a mirrored version of the human's end effector pose.

To derive the mirrored pose, we perform mirroring operations on the transform from the human's reference frame to their end effector T H hand H ref , which is then projected from the robot's reference frame to T R hand R ref .

The reference frames in world frame {T H ref W , T R ref W } are chosen to be at the midpoints between each robot's feet at initialization, with x, y, z axes pointing in the forward, left, and up directions respectively. We defined a "mirroring" operation T mirrored = Mirror(T Human ) on position and orientation: q H = {w, x, y, z} → q mirrored = {z, y, x, w}

x mirrored = x H z mirrored = z H y mirrored = y H if no contact y R 0 + (y H 0 -y H ) if contact (8)
where y 0 are the coordinates of the end effectors when the contact is initially established. The mirroring rule for the y direction amount to "mirror when there's no contact, move together when there is contact". The desired robot end effector position is calculated with:

(T R hand W ) des = Mirror(T H hand H ref ) • T R ref W (9)
Some other interaction tasks that can be used are:

• Collision avoidance between robots • Minimize distance/orientation errors between human and robot end-effectors (e.g. for a handoff) • Simulated human joint torque -induce the robot to carry more load

IV. EXPERIMENTS

To demonstrate the effectiveness of our pHRI framework, we performed several experiments in full dynamics simulation with an HRP-4 humanoid robot. We show that our controller can generate realistic interaction motions/forces for symmetric placement and balance assistance tasks.

These experiments were done using recorded motion of a human during a human-human collaborative task: only one partner was equipped with an XSens MVN motion capture suit [START_REF]Xsens MVN[END_REF], as the second partner is replaced by the robot in our simulations. Obviously this is not completely analogous to using this pHRI controller in real life; the limitations are discussed in Section V.

A. Symmetric placement task

The first experiment is a collaborative pick-and-place experiment, in which the human and robot work together to move a pole from one side of their bodies to the other. This task shows a simple application of the mirroring heuristic for generating robot follower motion. The pole is modeled as a floating-base unarticulated robot whose dynamics are incorporated into the combined dynamical system when contact is established. This is a simple motion that could possibly be replicated using other pHRI controllers. Even a simple impedance control framework would allow this motion, provided that the human exerts the extra torque needed to "push" the robot in the right direction. However, our controller reduces the amount of force the human needs to exert by encoding an assumption about the desired motion in the "mirroring" task.

In the next section, we will demonstrate a behavior that is only possible with pHRI formulations that consider the whole-body human model.

B. Balance assistance

In this experiment, the human leans their CoM outside of their own support polygon while holding onto a pole together with the robot. In the initial recording of the human-human motion, this required the partner (who the robot replaces) to pull back on the pole, keeping the human in balance.

The simulation results (depicted in Figures 5 and6) show that our controller calculates the effect that the human's leaning will have on the combined system, and generates a motion in which the robot leans back and braces itself to pull on the barbell/human. This assistive behavior emerges from the robot's individual CoM task, as well as the human model's motion tracking task.

This experiment shows the advantage gained from modeling the whole-body dynamics of the human. Other approaches that utilize a less complete representation of the human would have a hard time cleanly formulating the robot's assistive behavior in this situation.

C. Implementation details

In this paper, we omitted the mathematical descriptions of the QP objectives and constraints for conciseness. We follow the constraint formulation presented in [START_REF] Bouyarmane | Multirobot and task-space force control with quadratic programming[END_REF] (Section II.B), and details of the tasks we use can be found in [START_REF] Bouyarmane | Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations[END_REF].

Below is a complete list of the tasks used in the experiments, along with their relative weights: Contact change events were manually annotated in the recorded data, and handled via the method described in our prior work on motion retargeting [START_REF] Otani | Adaptive whole-body manipulation in human-to-humanoid multi-contact motion retargeting[END_REF]. In co-manipulation scenarios, we assume that the contact changes are simultaneous for the human and robot.

V. LIMITATIONS OF TESTING PHRI IN SIMULATION

Our experiments were done in simulation, using prerecorded motion of a human during a human-human collaborative task. Thus, our experiments show how a robot running our controller would react to a predetermined human motion. This means that motions which actively affect the human's motion (pushing/pulling CoM, encouraging human to lower arms, etc.) were excluded from these experiments. The experiments shown here demonstrate our controller's ability to generate motions/forces that are similar to what the human partner executed during the initial recording of the human-human motion. We also assume that the robot knows the goals of the task a priori, e.g. symmetric motion with the human.

To apply our framework in real pHRI scenarios, the robot's low-level motor control must allow some compliance so that the human can influence the robot's motion, and to account for small errors in the MRQP's human motion reconstruction (shown in Figure 4). The MRQP controller outputs a set of control setpoints {q des , τ des , λ des }; in our simulation experiments, the HRP-4 was controlled with joint position control by integrating qdes twice to get q des . To add compliance to the robot's motions, we can use a motor torque controller that takes q des and τ des as inputs.

VI. CONCLUSIONS AND FUTURE WORK

This paper presents a formulation of humanoid control for pHRI tasks, using a multi-robot QP to model the whole-body dynamics of the human. We show in simulation experiments that the controller can generate assistive motions for accomplishing collaborative tasks. This method is highly flexible and is easily adapted to varying robot morphologies, as well as different motion objectives.

Our future work will focus on implementing this controller on a real robot for physical experiments. Some additional avenues for future work are described below:

1) Integration with methods for predicting human intent: With our current approach, we make a priori assumptions about the type of motion the human will want to make. Integration of intent prediction algorithms into our framework would allow the robot to be more flexible; it won't have to make strong assumptions on the type of motion to be performed. Instead, the robot can be trained to perform a diverse set of collaborative motions, and infer the human's intended motion online. Thereafter, the robot will be able to actively help the human achieve the task with motion-specific interaction objectives. ("Intent" can consist of goal configurations, trajectories, speed, etc.; some these are demonstrated in [START_REF] Maeda | Learning interaction for collaborative tasks with probabilistic movement primitives[END_REF].)

2) Better modeling of human dynamics, reactions to external forces: To get a more accurate dynamics model of the human body than a rigid body tree, we can use neuromuscular human simulations which simulate individual muscles and tendons [START_REF] Delp | Opensim: open-source software to create and analyze dynamic simulations of movement[END_REF]. However, these simulations are too computationally expensive to run in real-time. One way around this is to have a surrogate model for the neuromuscular simulation (e.g. a neural network) which can be used as an inexpensive, real-time approximation to the full model and run in real-time as part of the QP controller.

Fig. 2 .

 2 Fig. 2. Diagram showing the difference between our two methods for human motion reconstruction. In both methods, the motion capture data is mapped onto our parametrized 22-joint human model, which is then used to compute the robot's controls.

Fig. 3 .

 3 Fig. 3. Steps of Experiment 1. See attached video for a better view.

Fig. 4 .

 4 Fig. 4. Tracking errors: difference of end effector (top) and CoM (bottom) position between human motion capture data and our simulated human model in Experiment 1. Results shown for both methods of human motion reconstruction. As expected, the Cascaded QP-MRQP method tracks closer. The Single MRQP method tracks within 10cm error.

Fig. 5 .

 5 Fig. 5. Robot assisting human with balance in Experiment 2. Forces on robot and human shown in red.

Fig. 6 .

 6 Fig. 6. Interaction forces (forces felt by the robot) in the x-direction (forward/backward) on the robot's right end effector during Experiment 2. Our controller generates realistic interaction forces that keep the collaborative task balanced.

• 4 •

 4 Robot -Individual * Center-of-mass: 100 * Posture: 1 -Interaction * Right end effector: 50 * Contact force minimization: Human (tracking tasks) -Center-of-mass: 200 -Posture: 60 -Left end effector: 140 -Right end effector: 60

ACKNOWLEDGEMENTS

This work received funding from the European Commission (GA no. 731540, H2020 project "An.Dy"). The authors would like to thank the IDH team in LIRMM and JRL for their support in using the multi-robot QP framework, and Pauline Maurice for implementing the human URDF model generator.


Intelligent Human-Robot Collaboration with Prediction and Anticipation

Serena Ivaldi * †

Collaborative robots need to safely control their physical interaction with humans. However, in order to provide physical assistance to humans, robots need to be able to predict their intent, future behaviour and movements. We are currently tackling these questions in our research within the European H2020 Project AnDy 1 .

Collaborative robotics technologies are rapidly spreading in manufacturing and industry, in the lead platforms of cobots and exoskeletons. The former are the descendants of industrial manipulators, capable of safely interacting and co-existing (i.e., sharing the same workspace) with operators, while the latter are wearable robotics devices that assist the operators in their motions. The introduction of these two technologies has changed the way operators may perceive interaction with robots at work: robots are no longer confined to their own areas; instead, they are sharing space with humans, modifying workstations, and influencing gestures at work (see Figure 1).

The major concern when introducing these technologies was to ensure safety during physical interaction. Most of the research in co-botics over recent decades has focused on collision avoidance, human-aware planning and re-planning of robot motions, control of contact, safe control of physical collaboration and so on. This research has been funded by the European Commission in several projects, such as SAPHARI 2 and CODYCO 3 , and contributed to the formulation of ISO norms on safety for collaborative robots, such as the ISO/TS 15066:2016 4 .

With the introduction of the new collaborative technologies at work, however, it has become clear that the problem of collaboration cannot be merely reduced to the problem of controlling the physical interaction between the human and the robot. The transition from robots to cobots, motivated largely by economic factors (increased productivity and flexibility) and health factors (reduction of physical stress and musculo-skeletal diseases), raises several issues Figure 1: The recent trend in collaborative robotics technologies in industry: from industrial robots working separately from humans, to cobots able to coexist and safely interact with operators. The advanced forms of cobots are exoskeletons, wearable devices that provide physical assistance at whole-body level, and more anthropomorphic collaborative robots that combine physical interaction with advanced collaborative skills typical of social interaction. from psychological and cognitive perspectives. First, there is the problem of technology acceptance and trust in the new technologies on the part of the operators. Second, there is the problem of achieving a more advanced form of interaction, realised with a multi-modal system that takes into account human cues, movements and intentions in the robot control loop, that is able to differentiate between work-related intentional and non-intentional human gestures, make appropriate decisions together with the human, and adapt to the human.

If we observe two humans collaborating, we quickly realise that their synchronous movements, almost like a dance, are the outcome of a complex mechanism that combines perfect motor control, modelling and prediction of the human partner and anticipation of our collaborators actions and reactions. While this fluent exchange is straightforward for us humans, with our ability to read our human partners, it is extremely challenging for robots.

Take, for example, two humans collaborating to move a big, bulky, heavy couch. How do the two partners synchronise to lift the couch at the same time, in a way that does not result in a back injury? Typically, the two assume an ergonomically efficient posture, ensure a safe haptic interaction, then use a combination of verbal and non-verbal signals, to synchronise their movement and move the couch towards the new desired location. While this collaborative action could be done in principle exclusively exchanging haptic cues, humans leverage their other signals to communicate their intent and make the partner aware of their status, intention and their upcoming actions. Visual feedback is used to estimate the partners current posture and effort, non-verbal cues such as directed gaze are used to communicate the intended direction of movement and the final position, speech is used to provide high-level feedback and correct eventual mistakes.

In other words, collaboration undoubtedly needs a good physical interaction, but it also needs to leverage social interaction: it is a complex bidirectional process that efficiently works if both humans have a good idea of the model of their partner and are able to predict his/her intentions, future movements and efforts. Such a capacity is a hallmark of the human central nervous system that uses internal models to plan accurate actions as well as to recognise the partners.

But how can these abilities be translated into a collaborative robotic system? This is one of the questions that we are currently addressing in our research, funded by the European H2020 project AnDy. AnDy involves several European research institutes (IIT in Italy, Inria in France, DLR in Germany, JSI in Solvenia) and companies (XSens Technologies, IMK automotive GmbH, Otto Bock Healthcare GmbH, AnyBody Technology). The main objective of the AnDy project is to create new hardware and software technologies that enable robots not only to estimate the motion of humans, but to fully describe and predict the whole-body dynamics of the interaction between humans and robots. The final goal is to create anticipatory robot controllers that take into account the prediction of human dynamics during collaboration to provide appropriate physical assistance.

Three different collaborative platforms are studied in AnDy: industrial cobots, exoskeletons and humanoid robots. The three platforms allow researchers to study the problem of collaboration from different angles, with platforms that are more critical in terms of physical interaction (e.g., exoskeletons) and more critical in terms of cognitive interaction (e.g., cobots and humanoids).

The main objective of exoskeletons is to provide physical assistance and reduce the risk of work-related musculo-skeletal diseases. It is critical that an exoskeleton is safe, assistive when needed, and transparent when not required. One of the challenges for an exoskeleton is the detection of current and future human activity and the onset of the kind of activity that requires assistance. While in the laboratories this can be easily detected by using several sensors (e.g., EMG sensors, motion tracker markers), it is more difficult to achieve in the field with a reduced set of sensors. Challenges for the acceptance of this kind of technology include a systematic evaluation of the effects of the exoskeleton on the human body, in terms of movement, efforts, ergonomics, but also on the perceived utility, trust towards the device and cognitive effort in using it. In a recent paper [START_REF] Maurice | Ethical and social considerations for the introduction of human-centered technologies at work[END_REF], we listed the ethical issues related to the acceptance of this technology.

For a collaborative robot (manipulator or more complex articulated robot such as a humanoid), the problems are similar in terms of physical interaction and safety. The cobot needs to be able to interact safely with the human and provide assistance when needed. Typically, cobots provide strength and endurance (e.g., they can be used to lift heavy tools and end-effectors) that complement human dexterity, flexibility and cognitive abilities in solving complex tasks. In AnDy we are focusing on the type of assistance that can help improve the ergonomics of the human operator at work. To provide suitable assistance, here the robot needs to be able to perceive human posture and efforts, to estimate the current task performed by the operator and predict future movements and efforts. Again, this is easily achieved in laboratory settings with RGB-D cameras, force plates and EMG sensors, but it is more challenging, if not impossible, to do in real working conditions such as in manufacturing lines with several occlusions and reduced external sensing. In AnDy, we exploited wearable sensors for postural estimation and activity recognition, which was also possible in a real manufacturing line [START_REF] Malaisé | Activity recognition with multiple wearable sensors for industrial applications[END_REF]. For the problem of predicting the future intended movement, we proposed describing the problem as an inference over a probabilistic skill model given early observations of the action. At first we leveraged haptic information, but rapidly developed a multi-modal approach to the problem of predicting human intention [START_REF] Dermy | Multi-modal intention prediction with probabilistic movement primitives[END_REF]. Inspired by the way humans communicate during collaboration, we realised that anticipatory directed gaze is used to signal the target location for goal-directed actions, while haptic information is used to start the cooperative movement and eventually provide corrections. This information is being used as input to the robot controller, to take into account the prediction of human intent in the planned robot motions.

This research was performed with the humanoid robot iCub, an open-source platform for research in intelligent cognitive systems. iCub is equipped with several sensors, that make it valuable for conducting human-robot interaction studies. Humanoid platforms such as iCub may seem far from industrial applications; however, many collaborative robots are now being equipped with a head and sometimes have two arms, which makes them more and more anthropomorphic and very close to a humanoid (see Figure 1). In this sense, operators may be driven to interact with them in a different manner than from the one they use with cobots or manipulators: the simple addition of a head with a face displaying information about the robot status, or moving along with the human, may create the illusion of a more intelligent form of human-robot interaction that goes beyond physical assistance. Expectations may increase, both in terms of the complexity of the interaction and the capacity of the system to properly react to the human and communicate its status. When such interactions occur, and they involve collaborative tasks or decision-making tasks, we believe that it is important to take a human-centred approach and make sure that the operators trust the system, learn how to use it, provide feedback and finally evaluate the system. As roboticists, we often imagine that humans wish to interact with intelligent systems that are able to anticipate and adapt, but our recent experiments show that when humans see the robot as a cognitive and social agent they tend to mistrust it [START_REF] Gaudiello | Trust as indicator of robot functional and social acceptance. an experimental study on user conformation to icub answers[END_REF]. Our take-home message is that we need to develop collaborative robotics technologies that are co-designed and validated by the end-users, otherwise we run the risk of developing robots that will fail to gain acceptance and adoption.

* This work was funded by the European Project An.Dy (no. 731540 H2020-ICT-2016-1).


Ahmad Mazyad

Fabien Teytaud

Cyril Fonlupt

Generating Term Weighting Schemes through Genetic Programming

Term-Weighting Scheme (TWS) is an important step in text classification. It determines how documents are represented in Vector Space Model (VSM). Even though state-of-the-art TWSs exhibit good behaviors, a large number of new works propose new approaches and new TWSs that improve performances. Furthermore, it is still difficult to tell which TWS is well suited for a specific problem. In this paper, we are interested in automatically generating new TWSs with the help of evolutionary algorithms and especially genetic programming (GP). GP evolves and combines different statistical information and generates a new TWS based on the performance of the learning method. We experience the generated TWSs on three well-known benchmarks. Our study shows that even early generated formulas are quite competitive with the state-of-the-art TWSs and even in some cases outperform them.

Introduction

Text Classification (TC) aims to automatically assign a set of predefined categories to a text document based on their content. TC is an important machine learning problem that has been applied to numerous applications such as spam filtering [START_REF] Tretyakov | Machine learning techniques in spam filtering[END_REF], language identification [START_REF] Zissman | Comparison of four approaches to automatic language identification of telephone speech[END_REF], authorship recognition [START_REF] Stamatatos | A survey of modern authorship attribution methods[END_REF], sentiment analysis [START_REF] Kumar | Sentiment analysis on twitter[END_REF], and so on. Generally, the TC approach is to learn an inductive classifier from a set of predefined categories. This approach requires that documents are represented in a suitable format such as the Vector Space Model (VSM) representation [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF].

In a VSM, a document d j is represented by a term vector d j = (w 1,j , w 2,j , ..., w t,j ) where each term is associated with a weight w k,j .

The weight represents how much a term contributes to the semantics of a document. The method which assigns a weight to a term is called Term Weight-ing Scheme (TWS).

Numerous TWSs exist and we introduce the most famous in Section 2. They are generated according to human a priori and mathematical rules. TWSs are usually simple mathematical expressions. Unfortunately, depending on the application, it is not easy to know a priori which TWS will be effective.

As expression discovery may naturally be addressed by genetic programming [START_REF] Cazenave | Nested monte-carlo expression discovery[END_REF], we are interested in this paper to study the effectiveness of Genetic Programming (GP) generated formulas and their aspects. We are also interested to know if a stochastic evolutionary process with no information about the complexity, the shape and the size of the expression can find at least competitive discriminative TWS.

The paper is organized as follows: Section 2 presents the TWSs and related works. In section 3 we present Genetic Programming and how it is applied to TWS. Section 4 presents the experiments and the results, and then we conclude in section 5.

Term Weighting Schemes

Text Classification (TC) is a supervised learning task. Hence, the training data consists of a set of labeled documents D = ((d 1 , l 1 ), ..., (d N , l N )), such that d j is the term vector of j-th document, l j is its label and N is the total number of training documents. As in VSM representation, a document d j is represented by a term vector d j = (w 1,j , w 2,j , ..., w k,j ) where w i,j is a weight assigned to the term t i of the document d j and determined by the TWS.

Statistical Information

Generally, a multi-labeled classification task is turned into several distinct singlelabel binary task, one for each label, using the binary relevance (BR) transformation strategy. That is, given the list of labels L = {l 1 , l 2 , ..., l m }, the original data set is transformed into m different data sets D = {D 1 , D 2 , ..., D m }. For each data set D k , documents having the label l k will be tagged as the positive category c k , and the rest as the negative category c k . Weights are then computed independently for each binary data set.

Based on the BR transformation, given a term t i and a category c k , TWS could be expressed using statistical information a, b, c and d obtained from the training data:

• a is the number of documents that contain the term t i and belong to the positive category c k . • c is the number of documents that contain t i and do not belong to c k .

• d is the number of documents that do not contain t i and do not belong to c k .

Using these statistics, the inverse document frequency (idf) is generally expressed as idf (t i , D) = log N |d∈D:t∈d| could also be expressed as idf (t i , D) = log N a+c where N = a + b + c + d is the total number of documents in the training data. Besides the statistics described above, Table 1 shows different statistical information that could be extracted from the training data.

Term Weighting Schemes

Generally, TWSs combines two of three factors pointed out by Salton et al. in [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF] that are believed to improve both recall and precision:

• Term Frequency (TF) factor : The TF factor is used to capture the relative importance of terms in a document. Table 2 lists four different TF factors.

• Collection Frequency (CF) factor : Also called term discrimination. The importance of words in a document (TF factor) does not provide enough discrimination ability. A common word like 'The' is frequent in almost all documents, and then it could not separate a group of documents from the remainder of the collection. Hence a discrimination factor is needed to favor those terms that are concentrated in a few documents of the collection. This corresponds to the second part of the previous formula.

Main known CF factors are presented in Table 3.

TWSs could be divided into two sets depending on whether they make use of available information on document membership (Supervised TWSs) or not (Unsupervised TWSs).

or log(2 + a * d b * c ) rf log(2 + a max(1,c) ) icf log( C Ct ) ig ( a N × log a×N (a+b)(a+c) ) + ( c N × log c×N (c+d)(a+c) ) +( b N × log b×N (a+b)(b+d) ) + ( d N × log d×N (c+d)(b+d) )
Unsupervised TWSs [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF][START_REF] Jones | A statistical interpretation of term specificity and its application in retrieval[END_REF][START_REF] Sebastiani | Machine learning in automated text categorization[END_REF][START_REF] Feldman | The text mining handbook: advanced approaches in analyzing unstructured data[END_REF] are generally borrowed from Information Retrieval domain [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF][START_REF] Jones | A statistical interpretation of term specificity and its application in retrieval[END_REF] and adopted for TC [START_REF] Mazyad | A comparative study on term weighting schemes for text classification[END_REF][START_REF] Debole | Supervised term weighting for automated text categorization[END_REF][START_REF] Mazyad | Information gain based term weighting method for multi-label text classification task[END_REF].

Term Frequency-Inverse Document Frequency (TF-IDF) is the most famous term weighting method, proposed by Jones in [START_REF] Jones | A statistical interpretation of term specificity and its application in retrieval[END_REF]. This method combines the TF factor and the CF factor and can be formally defined as below:

w i,j = tf i,j × log N N t . (1)
with, w i,j is the weight of the term t i in the document d j , tf i,j = f i,j is the term frequency represented by the raw count of t i in d j , and log N Nt is the inverse document frequency (idf). Besides the raw count (f t,d ) representation of tf , their exist numerous other variants such as binary representation (w i,j = 1 if the term t i occurs in the document d j and 0 otherwise), log(f i,j ) + 1, f i,j / t ∈d f t ,d . All these variants are also used as TWS on their own [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF][START_REF] Mazyad | A comparative study on term weighting schemes for text classification[END_REF][START_REF] Debole | Supervised term weighting for automated text categorization[END_REF][START_REF] Deng | A comparative study on feature weight in text categorization[END_REF]. The inverse document frequency has also a number of variants such as log(N/N t ) + 1, log((N -N t )/N t ) [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF].

Supervised TWSs makes use of available information on the membership of training documents by replacing the unsupervised idf component in TF-IDF by another supervised component. Debole et al. and Deng et al. in [START_REF] Deng | A comparative study on feature weight in text categorization[END_REF][START_REF] Debole | Supervised term weighting for automated text categorization[END_REF] are the first to take advantage of such information by combining the unsupervised TF component with different supervised term discrimination component: χ 2 (TF-CHI), which makes a test of independence between a term and a category. χ 2 alongside with other supervised feature selection metrics, has been tested in several papers, as a term weighting methods for text categorization. For example, Deng et al. in [START_REF] Deng | A comparative study on feature weight in text categorization[END_REF], replaces the idf factor with χ 2 factor, claiming that TF-CHI is more efficient than TF-IDF. In contrast, in a similar test, Debole et al. in [START_REF] Debole | Supervised term weighting for automated text categorization[END_REF], compare TF-IDF with three supervised term weightings, namely, TF-CHI, Odds Ratio (TF-OR) and Information Gain (TF-IG). The authors have found no consistent superiority of these new term weighting methods over TF-IDF; Information Gain TF-IG which measures the amount of information obtained for category prediction by knowing the presence or absence of a term in a document [START_REF] Yang | A comparative study on feature selection in text categorization[END_REF][START_REF] Debole | Supervised term weighting for automated text categorization[END_REF][START_REF] Deng | A comparative study on feature weight in text categorization[END_REF]; Gain Ratio (TF-GR) first used in a feature selection method defined as the the ratio between the information gain of two variables and the entropy of one of them [START_REF] Debole | Supervised term weighting for automated text categorization[END_REF]; Odds Ratio (TF-OR) first used as a feature selection methods by Mladeni'c et al. [START_REF] Mladeni'c | Feature selection for classification based on text hierarchy[END_REF]. It is a measure that describes the strength of association between two random variables. A comparative study on term-weighting for TC is made by Deng et al. in [START_REF] Deng | A comparative study on feature weight in text categorization[END_REF]. The study shows a good performance of TF-OR but is outperformed by TF-GR; Relevance frequency (TF-RF) proposed in [START_REF] Lan | Supervised and traditional term weighting methods for automatic text categorization[END_REF], measures the distribution of a term between positive and negative category, and favors those terms that are more concentrated in positive category than in negative categories; Inverse Category Frequency (TF-ICF) is a new supervised TWS proposed by Wang et al. in [START_REF] Wang | Inverse category frequency based supervised term weighting scheme for text categorization[END_REF]. The measure aims to favor those terms that appear in fewer categories. Several comparative studies on these TWSs for both term-weighting and feature selection has been reported in [START_REF] Yang | A comparative study on feature selection in text categorization[END_REF][START_REF] Lan | A comprehensive comparative study on term weighting schemes for text categorization with support vector machines[END_REF][START_REF] Mladeni'c | Feature selection for classification based on text hierarchy[END_REF][START_REF] Deng | A comparative study on feature weight in text categorization[END_REF][START_REF] Mazyad | A comparative study on term weighting schemes for text classification[END_REF][START_REF] Forman | An extensive empirical study of feature selection metrics for text classification[END_REF]. A new and different approach for term-weighting based on (TF-IG) have been proposed for multi-labeled classification task in [START_REF] Mazyad | Information gain based term weighting method for multi-label text classification task[END_REF]. The method computes a score based on all categories and then subtracts it from the original TF-IG weight. The idea is to take into consideration the weights of terms not only in terms of positive and negative categories but also in terms of every single category. Similar approaches have been proposed to learn TWSs via GP in [START_REF] Cummins | Evolving general term-weighting schemes for information retrieval: Tests on larger collections[END_REF][START_REF] Cummins | Evolving local and global weighting schemes in information retrieval[END_REF][START_REF] Cummins | Evolved term-weighting schemes in information retrieval: an analysis of the solution space[END_REF][START_REF] Trotman | Learning to rank[END_REF][START_REF] Oren | Reexamining tf. idf based information retrieval with genetic programming[END_REF][START_REF] Fan | The effects of fitness functions on genetic programming-based ranking discovery for web search[END_REF], however, these studies have focused on information retrieval problem. For TC, a similar approach proposed by Escalante et al. in [START_REF] Escalante | Term-weighting learning via genetic programming for text classification[END_REF]. However our study differs in two ways: first, Escalante et al. try to generate new TWSs by combining existing TWS, and secondly, they learn a single TWS for each data set whereas we learn a TWS for each category in a data set. In our work, we generate TWSs by combining statistical information at a microscopic level to evolve new TWSs. We also extends the study on the thematic TC. We hope this leads into more robust non human based TWSs.

Genetic Programming

Evolutionary computing is based on Darwin's theory of "survival of the fittest". The main scheme of evolutionary algorithms is to evolve a population of individuals that are randomly generated. Each individual represents a candidate solution that undergoes a set of genetic operators that allow to mix and alter partial solutions. One of the key features of evolutionary algorithms is that they are stochastic schemes.

Introduction

GP belongs to the family of evolutionary algorithms. It was first proposed by Cramer [START_REF] Cramer | A representation for the adaptive generation of simple sequential programs[END_REF] and then popularized by Koza [START_REF] Koza | Genetic programming: on the programming of computers by means of natural selection[END_REF]. Unlike genetic algorithms where the aim is to discover a solution, the goal of GP is to find out a computer program that is able to solve a problem.

In GP, a set of random expressions that usually represent computer programs are generated. As in all evolutionary computation algorithms, this set of programs will evolve and change dynamically during the evolution. What makes GP suitable for a number of different applications is that these computer programs can represent many different structures, such as mathematical expressions for symbolic regression [START_REF] Searson | Gptips: an open source genetic programming toolbox for multigene symbolic regression[END_REF], decision trees [START_REF] Koza | Concept formation and decision tree induction using the genetic programming paradigm[END_REF], programs that control a robot [START_REF] Koza | Genetic Programming II, Automatic Discovery of Reusable Subprograms[END_REF][START_REF] Lewis | Genetic programming approach to the construction of a neural network for control of a walking robot[END_REF] to fulfill a certain task or programs that are able to predict defibrillation success in patients and so on.

The quality of a candidate solution (i.e. a program) is usually assessed by confronting it with a set of fitness cases. This step is usually the most time-consuming step as the programs may get huge and several thousands of candidate programs are usually evaluated at each generation. These computer programs will undergo one or several evolutionary operators that will alter in a hopefully beneficial way. The most classical evolutionary operators are usually the crossover operator that allows the exchange of genetic material (in our case subtrees) and the mutation operator that allows a small alteration to the program.

In the most conventional GP approach, programs are usually depicted by trees. In GP terminology, the set of nodes are split into two sets, inner nodes of the tree are drawn from a set of functions while the terminal nodes (leaves) are drawn from a so-called terminal set. Depending on the problem, the set of functions can be mathematical functions, boolean functions, control flow functions (if,...), or any functions that may be suitable to solve the given problem. The terminal set is usually the set of inputs of the problem, e.g., parameters and constants for symbolic regression problems, sensors for robot planning and so on.

When the stopping criterion is reached, the best individual is returned, otherwise, the loop continues and the best individuals are selected (according to their fitness). There exist numerous ways for selecting the population, the mutation and the crossover operators. This is beyond the scope of this paper and the reader can refer to [START_REF] Koza | Genetic programming: on the programming of computers by means of natural selection[END_REF][START_REF] Karakus | Function identification for the intrinsic strength and elastic properties of granitic rocks via genetic programming (gp)[END_REF] for more information.

Evolving Term Weighting Scheme using Genetic Programming

A CF factor is a combination of statistical information. It is intended to measure the discriminative power of a term, i.e. it tells how much a term is related to a certain category. These statistics combined by means of mathematical operators and functions. We are interested in automatically evolving a CF factor (an individual) using GP. In our approach, the learned CF factor combined to the TF factor forms a term weighting method.

In our context of automatically evolving term weighting methods, an individual is a combination of the function set that is built with simple arithmetical operators (+,-,*,/,log,...) and the terminal set (constant values and inputs to our problem).

Tables 1 show the statistical information used as terminal set for generating formulas (the function set) which represent CF factors. As it can be seen, the function set is made of very simple arithmetical functions while the terminal set includes to the best of our knowledge all the statistical information used to build a TWS.

As previously mentioned, programs (generated TWS) are depicted as trees. In this problem, the terminal nodes consist of statistical information extracted from training data, while the inner nodes are a set of defined operators that combines the statistical information to form a new TWS.

Figure 1 describes the representation graphically and Table 4 shows the parameters used in the genetic programming algorithm.

Terminals and Function Set

In this study, we try to generate new TWS by evolving the CF factor and then combines it with the TF factor. The CF factor is a combination of constants, statistical information (N , N t ,...), and mathematical operators. Hence we define the terminals as the statistical information shown in Table 1. Regarding the mathematical operators, they are defined as one of the following (+, -, /, * , √

x, log1(x) = log(1 + x) and log2(x) = log(2 + x)).

We should note that the statistical information has different types (single value, vector, and matrix). For instance, the number of documents in the training data N is a constant (single value), the number of documents that contains a term t is a vector containing the number of documents for each term and finally, the number of documents that belongs to a category cat and contains a term t is a matrix. Operations on these different types of statistical information are taken care of by Eigen1 library using element-wise transformations.

Genetic Operators

In GP, a set of individuals is initialized and then evolved according to a set of genetic operators. At first, we randomly generate a random size individuals with a max size of twenty genes (the max size could be overpassed during the cross-over operation). As for genetic operators, we use the elite selection and reinsertion, a subtree crossover with a probability of 0.85 and one point mutation with a probability of 1/size of the individual. Figure 2 describes graphically the course of evolution of individuals in this study.

Fitness Function

Generally, the performance of a TWS is assessed on known benchmark by evaluating a classification model on VSM representation of this TWS. Numerous evaluation metrics exist to evaluate the classification models such as f 1 measure. Evaluating the classification model is a vital step that affects the performance of the GP. However, it could be very time-consuming. Hence, it is important to choose a good and fast machine learning algorithm. LibLinear [START_REF] Fan | Liblinear: A library for large linear classification[END_REF] is an open source library for large-scale linear classification. It supports logistic regression and linear support vector machines.

In our study, once a new individual is generated, we perform a 3-fold crossvalidation on the training data which generates three disjoint subsets. We use two subsets as the training set and one subset as the test set. The process is repeated three times using each time different subset for testing. The performance is measured using the f 1 measure. The average classification performance is used as the fitness function. The f 1 measure considers both precision p (true positive over true positive plus false positive) and recall r (true positive over true positive plus false negative) and can be formally defined as f 1 (p, r) = 2rp r+p .

Experiments and Results

This section presents an empirical evaluation of the proposed approach. The goal of this study is to assess the effectiveness of the generated TWSs and compare their performances to standard TWSs.

Experimental setup

In our experiments, we have used three widely well-known benchmarks in TC: Reuters-21578 Benchmark Corpus 2 , Oshumed Benchmark Corpus 2 and the 4 Universities data set also called Webkb 3 . The Reuters-21578 data set is one of the most used test collection for TC research. We use the well-known "Apte-Mod" split [START_REF] Joachims | Text categorization with support vector machines: Learning with many relevant features[END_REF]. This split includes 10788 documents from the reuters financial service, divided into a training set of 7769 documents and a test set of 3019 documents. The data set is highly skewed, the smallest category contains only 2 documents and the biggest contains 3964 documents. Documents in this data set belongs to one or more categories. This version of the data set contains ninety categories, however, in our experiments, we report results only for the largest ten categories. Oshumed dataset is extracted from the Oshumed 1 collection compiled by William Hersh. It includes 13,929 medical abstracts [START_REF] Debole | Supervised term weighting for automated text categorization[END_REF]286 for training and 7,643 for testing) from the MeSH categories of the year 1991. Each document in this data set belongs to one or more categories from 23 cardiovascular diseases categories. Webkb data set contains WWW-pages collected from computer science departments of various universities in January 1997 by the World Wide Knowledge Base (Webkb) project of the CMU text learning group. In this experiment, we kept only the four largest categories ("student", "faculty", "course" and "project"), and we split it into three random folds where two folds are used for the training set and one fold for the test set. For all three data sets considered in the experiments:

• A default list of stop words, punctuation and numbers are removed. • Lower case transformation is applied.

• Porter's stemming is performed.

Furthermore, for each experiment, a binary transformation is applied. That leads to multiple distinct single-label binary task, one for each label (see Section 2.1). Each task could be treated as an independent experiment with its own data set.

As mentioned above, each data set has been split into training and test subsets. Table 5 shows, for each data set, the number of documents in the training and test subsets, the number of classes, the number of terms, the size of smallest category and the size of the largest category.

TWSs are evolved using the training subset (see Section 3.2.3). Finally, the test subset is used to evaluate the performance of the generated TWS. And finally, for each data set, we report the f 1 measure (see Section 3.2.3).

In order to obtain more reliable results, we have performed 20 runs on each task. After having evaluated the generated TWSs, we report the performance average and standard deviation over the 20 runs. In addition, we report the maximum and minimum f 1 score obtained across the 20 runs (for each run, only the last generated TWS is taken into account).

Tables 6,7 and 8 show the results obtained by the generated TWSs and the best baseline using linearSVM.

Table 9 shows the average classification performance of the generated TWSs on the test subset of the training data (Validation) and the performance on the test data (Test). The goal of this experiment is to show us if further learning is possible or to warn us of eventual overfitting.

Table 10 shows the average classification performance of a random learned TWS for a single-label binary task on the complete data set. This is important in order to know whether our GP-Based TWS has good generalization performances.

Results

First, a fast study of the Tables 6,7 and 8 shows that the best baseline TWS is different for each binary task. Therefore, a multi-labeled task requires different TWSs for each category. Using different TWSs could lead to better results. However, the problem is to recognize the best TWS for a specific task. Finding the TWS by cross-validation does not obligatorily return the best TWS. Regarding Reuters-21578, the generated TWSs and the baseline schemes have similar performances. However, on Oshumed and Webkb data sets, the GP-Based TWSs outperforms the best baseline schemes.

From Table 9 , we can see that the performance of generated TWSs on the test subset of the training data during the cross-validation (See Section 3.2.3) are very similar to the performance on the test data. In addition, the standard TWSs have different results. That is interesting as it suggests that there is no overfitting and that further learning can improve the performance.

From Table 10, we can see that the average performance (macro-f 1 ) of the generated TWSs outperforms the best baseline on the three corpora which means that the three learned TWS have good generalization performance.

Finally, compared to the results obtained in [START_REF] Escalante | Term-weighting learning via genetic programming for text classification[END_REF] on Reuters-21578 and Webkb, we have similar results. Note that, in [START_REF] Escalante | Term-weighting learning via genetic programming for text classification[END_REF], they used Reuters-10 data set which contains, only documents from the top 10 categories of the Reuters-21578 data set, whereas we use Reuters-21578 "ModApte" split which contains documents from 90 categories. This makes our task harder. This kind of generation is new, and we can conclude that :

• Different data sets require a different formula. This means that having a good generic formula is really hard to find.

• Within a corpus, it is even better to use a different formula for each category. The hard task is to find the best for each one.

• Genetic programming is able to find very good formulas which outperform standard formulas given by experts in the literature.

• Eventually, even if the generated formula is specific to a given category, results show that the best formula for one category is generic enough to be good (but not best) for other categories.

We propose the following further works:

• Parallelization of the GP algorithm in order to find better formula (as the test performance is close to the learning performance, this means that there is still room to improve the learning).

• As said previously, further learning can be done in order to find more generic and robust formula.

Figure 1 :

 1 Figure 1: Representation for the idf component log(1+ N Nt ) in Genetic Programming.

Figure 2 :

 2 Figure 2: The generic diagram of evolutionary algorithm as applied in our study.

Table 1 :

 1 Statistical information (Terminals) used to evolve a TWS. Number of documents that do not contain t N cat Number of documents in the positive category cat N cat Number of documents that do not belong to cat • b is the number of documents that do not contain t i and belong to the positive category c k .

	Label Description
	N	Total number of documents
	C	Number of categories
	C t	Number of categories that contain the term t
	N t	Number of documents that contain t
	N t

Table 2

 2

		: Term Frequency Factors
	TF	Description
	1.0	1 for terms present, 0 otherwise
	raw frequency #times a term occurs in a document
	log(1 + tf )	log normalization
	0.5 + 0.5 tf max tf	double normalization 0.5

Table 3 :

 3 Six traditional CF factors. Given a term t and a category cat, N stands for the total number of documents, a is the number of documents that contain t and belong to cat, c is the number of documents that contain t and do not belong to cat, b is the number of documents that do not contain t and belong to cat, d is the number of documents that do not contain t and do not belong to cat, C is the number of categories and C t is the number of categories that contains documents containing the term t.

	CF Defined by
	idf log( N Nt ) N * (a * d-b * c) * (a * d-b * c) χ 2 (a+c) * (b+d) * (a+b) * (c+d)

Table 4 :

 4 Parameters used in our genetic program.

	Parameter	Value
	Population Size	100
	Max Individual Size	20
	Number of generations 100 Function set +, -, /, * ,	√ x,
		log1(x) = log(1 + x),
		log2(x) = log(2 + x)
	Terminal set	a, b, c, d, N , Nt, verlineNt,
		Ncat, Ncat, C, Ct
	Mutation	Type	OnePointMutation
		Probability 1/indvidual size
	CrossOver	Type	SubtreeCrossover
		Probability 0.85

Table 5 :

 5 Statistics on the selected data sets used for our experiments (training/test).

		Reuters	Oshumed	Webkb
	# documents	7769/3019 6286/7643 2803/1396
	# classes	90	23	4
	# terms	26000	30198	7890
	smallest category 1/1	65/70	336/168
	largest category	2877/1087 1799/2153 1097/544

Table 6 :

 6 Classification performance on top 10 categories of Reuters-21578 obtained with the generated TWSs and the best baseline. We put in bold the best results between our generated TWSs and the best baseline of the standard

	TWSs.
			GP		Best Baseline
	Label	f1	Min	Max	f1	Baseline
	earn	98.34±0.09	98.24 98.54 98.38 tf.idf
	acq	96.93±0.23	96.55 97.54 97.10 tf.idf
	money-fx 79.60±0.50 78.16 80.45 78.63	tf.idf
	grain	94.25±0.63 93.10 95.22 93.43	tf.rf
	crude	90.01±0.81 88.27 90.94 88.24	tf.rf
	trade	79.10±1.21 77.69 80.18 78.03	tf.rf
	interest	75.16±0.50	74.45 76.19 76.19 tf.idf
	ship	80.52±1.54 77.84 82.93 78.95	tf.or
	wheat	88.11±1.26	86.12 90.96 90.20 tf.chi
	corn	92.80±0.27	90.83 93.94 93.91 tf.chi
	Average	87.48±0.70 86.13 88.69 87.30

Table 7 :

 7 Classification performance on Oshumed data set obtained with the generated TWSs and the best baseline.

			GP		Best Baseline
	Label	f1	Min	Max	f1	Baseline
	C01	68.19±1.00 65.91 70.71 64.36	tf.or
	C02	41.28±1.20 38.45 43.51 36.38	tf.or
	C03	76.54±3.28	72.03 81.21 78.23 tf.or
	C04	80.06±1.48	77.67 81.72 80.06 tf.chi
	C05	59.48±0.20 59.05 60.59 52.85	tf.or
	C06	73.99±1.29 71.49 75.76 71.44	tf.or
	C07	41.40±3.35 34.86 47.45 32.6	tf.or
	C08	63.97±2.51 59.13 67.69 61.34	tf.or
	C09	53.75±2.63 50.85 58.43 48.00	tf.or
	C10	57.00±2.33 51.05 59.53 50.2	tf.rf
	C11	67.78±1.06 65.52 69.23 66.67	tf.or
	C12	76.72±1.10 73.52 78.25 72.86	tf.or
	C13	66.48±0.47 64.72 67.92 63.70	tf.or
	C14	80.08±0.39 79.22 80.55 77.11	tf.idf
	C15	65.98±0.71 64.16 67.20 61.53	tf.chi
	C16	33.54±0.89 31.14 35.41 28.00	tf.or
	C17	64.85±0.90 61.87 66.87 59.24	tf.chi
	C18	61.21±1.50	57.50 65.12 61.22 tf.or
	C19	41.60±2.04 38.23 45.01 39.84	tf.or
	C20	71.61±0.28 70.96 72.07 69.62	tf.or
	C21	65.55±0.32 64.18 67.56 64.37	tf.chi
	C22	10.31±0.12 8.33	14.37 4.21	tf.or
	C23	46.77±0.08 45.59 47.20 46.15	tf.idf
	Average 59.48±1.26 56.76 61.89 56.08

Table 8 :

 8 Classification performance on Webkb data set obtained with the generated TWSs and the best baseline.

			GP		Best Baseline
	Label	f1	Min	Max	f1	Baseline
	student	90.29±0.50 89.05 90.90 90.11 tf.rf
	faculty	86.62±0.15 85.69 87.81 86.21 tf.rf
	project	80.82±0.64 77.48 81.76 80.25 tf.rf
	course	94.47±0.34 93.86 96.08 93.56 tf.rf
	Average 88.05±0.41 86.52 89.14 87.53

Table 9 :

 9 Average classification performance for validation phase and test phase.

		Reuters	Oshumed	Webkb
	Validation 89.15±0.42 59.74±0.9	87.74±0.31
	Test	87.48±0.70 59.48±1.26 88.05±0.41

Table 10 :

 10 Average classification performance of random TWS learned for a single-label task on its corresponding data set and the best baseline. The selected TWS is randomly chosen between the best generated TWSs for each category. Best Baseline Reuters-21578 * * C * / / a c N log2 c C C * C * ( a c * N * log(2 + c)) 86.88 85.92 tf.rf Oshumed / d / + N t log2 C t a

	GP-Based	Baseline

http://eigen.tuxfamily.org/

ConclusionIn this paper, we have studied the benefits of using genetic programming for generating term-weighting schemes for text categorization. Unlike previous studies, we generate formula by combining statistical information at a microscopic level.


Serena Ivaldi
email: serena.ivaldi@inria.fr

come

Introduction to the Special Theme on

Human-Robot Interaction

Serena Ivaldi * and Maria Pateraki †

This special issue addresses the state of the art of human-robot interaction (HRI), discussing the current challenges faced by the research community for integrating both physical and social interaction skills into current and future collaborative robots.

Recent years have seen a proliferation of applications for robots interacting physically with humans in manufacturing and industry [START_REF] Zanchettin | Collaborative robots in the workplace [from the guest editors[END_REF], from bimanual cooperation in assembly with cobots (i.e., industrial manipulators for collaboration) to physical assistance with exoskeletons. These applications have driven research in many fundamental topics for collaboration, such as shared task allocation, synchronisation and coordination, control of contacts and physical interaction, role estimation and adaptive role allocation during collaboration, learning by demonstrations, safe control, etc [START_REF] Ajoudani | Progress and prospects of the human-robot collaboration[END_REF]. All the developments in these areas contribute to the success of the Industry 4.0, whose elite platforms are essentially cobots and exoskeletons.

At the same time, research in social robotics has made tremendous progress in understanding the behaviour and the intricacy of verbal and non-verbal signals exchanged by robots and humans during interaction, highlighting critical aspects such as trust, mutual awareness and turn-taking [START_REF] Thomaz | Computational humanrobot interaction[END_REF]. These studies were initially motivated by the increased assistance and service robotics application, ranging from the introduction of robots in malls and shops to hospitals and homes, but are now becoming crucial for the acceptance of new intelligent robotics technologies in other industrial domains, such as manufacturing.

The human-robot-interaction (HRI) research community is thus advancing both physical and social interaction skills for robots. The proof of the convergence of both skills are the new industrial robots such as Baxter and Sawyer, where compliant arms such as in cobots are coupled with a face emulating referential gaze and social behaviour, to facilitate collaboration with humans.

The European Commissions Strategic Research Agenda for Robotics acknowledges the importance of robotics. With their increased awareness and ease of use, robots represent the dawn of a new era as ubiquitous helpers improving competitiveness for business and quality of life for individuals. Their role is expected to continuously expand beyond their traditional role in the man-ufacturing industry, providing significant short to medium term opportunities in areas such as agriculture, healthcare, security and transport, while in the longer term robots are expected to enter almost all areas of human activity, including the home. Along this line, the European Commission highlights HRI as one of the key technology areas in robotics with greatest impact guaranteeing project funding of 66 million EUR for 2018-2020. A large number of national and European projects are active in this area and a selection of these can be found referenced in the articles on these issue. Besides some of the current challenges in human-robot interaction and the approaches to tackle these challenges in real applications are presented in this special issue.

Key challenges on human robot collaboration are discussed in several papers. Buoncompagni et al. addresses main research questions for HRC in smart factories, advocating an AI-based approach to develop intelligent collaborative robots and Ivaldi is focused on the prediction of the human partner, currently developed within the EU-funded H2020 project AnDy.

Topics related to conversational and dialog systems are addressed in Agirre et al. presenting relevant research work in dialog systems for industry aiming to improve the natural language interaction between humans and robots. On the same topic Schindler et al. describe a conversational system that faciltates HRI thanks to a context-aware approach based on audio-analysis, which has been successfully exploited in various application areas.

Manufacturing-oriented papers such as those by Kaiser and Horvath aim to support HRC scenarios in their respective areas. Kaiser used simulation tools to design collaborative assembly systems and to support the planning tasks, whereas Horvath describes a context-aware multimodal interface effectively utilised within SYMBIO-TIC H2020 project.

Assistive The articles in this special theme not only provide a panorama of the ongoing European research in the field, but highlight the intrinsic multidisciplinarity of the theme. Even in industrial sectors such as manufacturing, it is clear that the problem of introducing collaborative robots cannot be merely reduced to the problem of ensuring safety and controlling their physical interaction with the humans. A multitude of sub-problems must be taken into account for collaborative robots to be accepted and widely adopted: from rethinking the whole system software and hardware architecture to enabling natural communication. The diversity of challenges and topics addressed in the special theme illustrates the several challenges for human-robot interaction and collaboration.

  robots and healthcare applications within the context of HRI are discussed in Cesta et al., Hindriks et al. and Efthimiou et al.. Cesta et al. , present a cognitive architecture combining human perception and AI techniques to infer knowledge about the status of a user and the environment and plan personalised assistive robot actions for elderly people. Hindriks et al. reports on their first experiments on a social robot that supports collection of patient data in a hospital, to reduce the workload of nurses. Efthimiou et al. are developing a multimodal user-centred HRI solution that encourages trust and acceptance of assistive robots for elderly people. State-of-the art research in social HRI is presented in Schellen et al., Evers , Mokios et al. and Ribino et al. . Schellen et al. highlights the importance of social attunement in interactions with artificial agents, exploiting methods from experimental psychology and cognitive neuroscience to study social cognitive mechanisms during HRI. The research is partially funded by the starting ERC grant InStance. Evers designs socially intelligent robots for several applications, from service to education. As part of EU-funded FET projects TimeStorm and Entiment, Mokios et al. address the open challenge of time perception in HRI to enable fluent HRI. Ribino et al. argue that robots acting with humans following social norms may improve their acceptance and the dynamics of HRI by proactively reasoning in dynamic normative situations.