Je voudrais remercier en premier lieu mon directeur de thèse Olivier Stasse qui m'a guidée et accompagnée durant ma thèse. Je le remercie pour sa disponibilité et son aide qui m'ont permis de réaliser mes travaux, même dans des domaines nouveaux pour l'équipe. Merci de m'avoir soutenue constamment pendant ces trois années assez particulières. Merci également à Sébastien Boria, mon tuteur entreprise, et Damien Van Damme pour leur soutien côté entreprise. Je vous remercie pour vos échanges d'idées et votre aide afin que ma thèse se passe sans encombre au sein d'Airbus. Je remercie Philippe Fraisse et Vincent Padois d'avoir accepté d'être mes rapporteurs et de m'avoir apportée beaucoup d'éléments de réflexion et de correction sur mon manuscrit. Merci pour votre lecture approfondie et de votre bienveillance pendant ma soutenance. Merci aussi à Christian Ott, Serena Ivaldi et Andrea Del Prete pour avoir fait partie de mon jury de thèse et pour vos retours constructifs lors de ma soutenance. Merci encore à Andrea pour son soutien sur TSID et merci à Vincent Bonnet pour son aide pour mon premier papier

In this chapter, some of the figures presented were realized for the Work-Package (WP) reports in the scope of the joint-laboratory Robots For the Future of Aircraft Manufacturing (ROB4FAM). They are indicated to be part of these reports by noting WP in their captions. Moreover, Louise Scherrer has kindly authorized us to use some of her figures from her Master report Scherrer [START_REF] Scherrer | Reactive walking for the humanoid robot pyrène[END_REF] to illustrate some principles

Introduction

Robotics consists in the study of machines able to perform tasks that were previously executed manually. In an industrial context, it is often repetitive and draining tasks, which may lead to safety and health issues (such as musculoskeletal disorders). Thus, machines, called robots, are studied in terms of design and control in order to execute these types of industrial applications. These operations belong to semistructured environments, often known a-priori, and their realizations by a robot is called automation. Generally, industrial robots can be separated in two groups: one where the robot is used for mass production and needs only to have satisfying execution repeatability to efficiently execute tasks. And another one where the industrial process favors robots with high position accuracy over repeatability capability; and even versatility to be able to realize different tasks. In aircraft industry, a robot which can react and adapt to unknown environments and unplanned events is desirable. Indeed, some parts of the aircraft production process are closed to unstructured environments and adaptation of the solution would allow to maintain the production process even when failure occurs, which is often a point of concern. However, adaptability is rarely completely achieved in industry, as the robot needs to process the information given by its sensors to react accordingly to a planned or unplanned situation. Moreover, it is not always the most important or prioritized part of the process. For instance, when the industrial process consists in manufacturing a huge amount of identical products, the predictable quality of the repeatability process is more relevant than the ability to adapt. In general, a robot is programmed to execute a task, and various programs can be implemented to allow the robot to switch between them and fit with the manufacturing process. For instance, a robot can first drill a hole, then change its end-effector (versatility of the solution) and switch automatically to another task program such as deburring the hole. However, this type of programming cannot be called "adaptable" as it does not use actively the robot sensors to react to unplanned event. This scenario is currently the most common in industry Siciliano et al. [2]. Online path and solution modifications for the robot when unplanned events occur often require high computation resources and more complex algorithms.

In the industrial context, the robots are commonly rigid manipulators with six Degrees of Freedom (DoFs) but also parallel robots (Stewart Gough platform) or hybrid robots (for instance, a robot with 3 parallel DoFs and 2 serial DoFs). In particular, these robots are fixed to the ground or to a fixed structure integrating auxiliary axis (to add redundancy to the system). However, they can also be assembled on a mobile base to augment their reachable workspace. This design choice comes from the structured working environment where they perform tasks. Considering that the environment is well identified, the level of autonomy needed by the robot is less important compared to the expected quality results (such as the precision or repeatability). However, when the adaptability criterion becomes dominant, advanced robotics solutions are needed. For instance, if a human operator is not available or is not safe in the working environment, how can a robot still perform the task ? Some disaster or emergency-response scenarios have been addressed in the DARPA (Defense Advanced Research Projects Agency) Robotics Challenge (DRC) [START_REF] Spenko | The DARPA Robotics Challenge Finals: Humanoid Robots To The Rescue[END_REF] (June 2015), which presents complex tasks in dangerous, degraded, human-engineered environments. These scenarios included for instance walking on debris or remove them from a blocked entryway, climbing industrial ladders and walking across industrial walkways or using a tool to break through a concrete panel. Even for non disaster scenarios, adaptability is a desirable ability to safely work in such environments and alongside humans.

In the scientific domain, researches on advanced robotics solutions lead to more diverse type of robots: from rigid to compliant, from manipulators to drones, from quadruped and biped to humanoids. In particular, the literature presents multiple solutions on humanoid robots as they are complex systems designed to operate in the same environment than humans and perform the same tasks. Humanoid robots were the most represented type of robots in the DRC, indeed, one of their interesting purposes is to achieve tasks that can be dangerous for humans. The humanoid robots offer two main challenges to the literature, represented by their two main capabilities: the locomotion and the manipulation. By addressing the potential issues of the robotic domain on these complex systems, the literature proposes exhaustive solutions that cover most of the simpler systems' needs. In particular, a solution developed for a humanoid robot can be transposed to an industrial 6 DoFs robot by simplifying the formulation of the problem. This is why, in this thesis, we focus on designing solutions for a humanoid robot to then transfer them onto an industrial manipulator robot arm.

ROB4FAM

On the 21st May 2019, a joint lab between Airbus Operations SAS and the Laboratory for Analysis and Architecture of Systems (LAAS) is inaugurated. It is named ROB4FAM and creates a collaboration between the Airbus Commercial Aircraft Mechatronics and Robotics team and the LAAS-CNRS Gepetto team. Its goal is to develop innovative robotics solutions to achieve reactive manufacturing operations in a manufacturing plant environment, while evolving closed to human operators. The idea is to deploy the scientific toolset developed by the Gepetto team on industrial robots, so that these robots can take their environment into account to detect anomalies (uncertainties or environment changes) and adapt to them, continuously (in real time). Moreover, the proposed solutions must ensure that the robots' behaviors are stable, safe and robust in order to interact with the surrounding environment and in particular humans.

The robotics platforms used by the Gepetto team during my thesis are two robots made by the PAL-Robotics company: the humanoid robot TALOS and the wheeled manipulator robot TIAGo (see Fig. 1.4). One advantage of these two robots is that they are built aiming modularity and cross platform compatibility. The hardware and software of the robots are close enough such that the developed solutions on TALOS can be easily transferred to TIAGo (see Section 1.4). After the validation on these research platforms, the implemented solutions must be adapted and deployed on their industrial counterpart, such as the Middle Size Drilling Robot (MSDR), see Chapter 6.

My thesis is part of the ROB4FAM joint lab, focusing on the implementation of a torque whole-body control on the humanoid robot TALOS for manufacturing operations. The results of this joint-lab are open-source, under the license BSD-v2.

Objectives and Contributions of the Thesis

This thesis is a CIFRE, realized within the Airbus Operations SAS company in partnership with the LAAS. It is one of the two first thesis launched in the scope of the ROB4FAM joint-lab. The thesis tries to answer the issue of robotics wholebody control for manufacturing operations. One major concern is to ensure the safety of the operation for the robot and for its environment, for instance humans. Indeed, the current robotic systems implemented in factories are separated from the workers by safety fences. It requires a huge space compared to what a human needs and when a failure occurs it is necessary to stop the production chain to check the problem in the cell. A smaller and safer solution is thus targeted, which must be adaptable to the robotic platform and able to react to unplanned events. Thus, the thesis focuses on the development of innovative solutions on the more adaptable robots available in the laboratory, in particular the humanoid robot TALOS (see Section 1.4). This robot has high capabilities, can be controlled in torque and is commercially available; a part of the realized works contribute toward the robot evaluation on locomotion and multi-contact scenarios.

The aim is for the robot to succeed manufacturing operations such as drilling, tightening or deburring. Each of these operations requires high precision and a force application using a specific tool. Therefore, during this thesis the question of which type of control to use has been addressed. Applying a specific amount of force can be achieved by different methods, the most common being position control using force sensors and direct torque control. The chosen solution has moreover to respect the safety concern. To resume, this thesis address the following problems:

How to efficiently control a humanoid robot during manufacturing operations: using torque or position control; How to ensure the safety of the solution, by studying the stability and convergence of the solution toward an equilibrium.

To answer these problems and fulfill the objectives this thesis proposes the following contributions: firstly the design of a controller protecting the robotic system using a Model Predictive Control (MPC), secondly the benchmarking and implementation of three different humanoid robots control architectures, thirdly the application of one of this controller in a context of human-robot collaboration. And finally, the design of a novel passive whole-body controller, validated through a force application simulation, with the proof of its stability to ensure the safety and robustness of the solution. [4], aiming to create a successor to the robot HRP-2 Kaneko et al. [START_REF] Kaneko | Humanoid robot hrp-2[END_REF] developed by the National Institute of Advanced Industrial Science and Technology (AIST). The goal was to create a robot capable of dexterous bi-handed manipulation which can involve high payload, with better locomotion performances and with size and mass better suited to work in human environment. This is why TALOS is taller and heavier than HRP-2 (see Fig. 1.1) and has torque and temperature sensors.

In details, the robot is 1.75m tall, weights around 100kg and has electric actuators. It has 32 Degrees of Freedom (DoFs) with the following distributions of its joints: two legs of 6 DoFs, a waist of 2 DoFs, two arms of 7 DoFs with a 1 DoF gripper each and a head of 2 DoFs. The rigid chain actuators of TALOS are composed of a brushless motor, that can be controlled in current by the software, connected to a harmonic drive and a torque sensor attached to its corresponding link (except for the head and the grippers). The motor and joint positions are measured by two high-precision encoders (19 bits or 524, 288 counts per revolution), one encoder just after the motor and one encoder on the link after the harmonic drive. In addition, there are temperature sensors for each joint of the robot. The kinematic structure of TALOS is depicted in Fig. 1.2. Each depicted frame is located at the origin of a joint. All joints are revolute. The base link of the robot is located close to its center of mass, in the waist. As every humanoids robots, TALOS is not groundfixed to the environment, it is an under-actuated system. This under-actuated part, called floating base, represents the position of the robot in the world and relies on external contact forces to control its motion. It is a significant difference compared to fixed industrial manipulator robots (without auxiliary axis, such as the scenario considered for the MSDR).

TALOS has an Inertial Measurement Unit (IMU) in its torso, providing filtered measures of the orientation, the angular speeds and the linear accelerations at 1kHz. Force/Torque sensors are integrated in the ankles and wrists of the robot. TA-LOS possesses two Ubuntu operating system computers (18.04), one for the control Figure 1.2: Kinematic structure of TALOS Scherrer [START_REF] Scherrer | Reactive walking for the humanoid robot pyrène[END_REF] (patched with the Linux Real-Time Preempt) and one for the vision processes. The control computer is connected to all the actuators and sensors of the robot by an EtherCAT bus [START_REF]Industrial communication networks-fieldbus specifications-part 3-12: data-link layer service definitionpart 4-12: datalink layer protocol specification-type 12 elements[END_REF]. It uses the middle-ware Robot Operating System (ROS) Stanford Artificial Intelligence Laboratory et al. [7] and reaches a control-loop frequency of 2kHz. This high-frequency allows to obtain a quick reaction time necessary for real time complex behaviors. Because of the ROS architecture it is possible to use MoveIt! Coleman et al. [START_REF] Coleman | Reducing the barrier to entry of complex robotic software: a moveit! case study[END_REF], Chitta et al. [9] to plan the robot motions, and the Gazebo simulator Koenig and Howard [10] to simulate dynamically the robot TALOS. PAL-Robotics has additionally implemented a specific simulator for their robot which simulates its actuators dynamics.

The robot TALOS of the Gepetto team is named Pyrène and is the first of the TALOS production series of the PAL Robotics company. Therefore the subsequent iterations have been improved, in particular the torque sensors and the feet have been updated. Pyrène has been eventually revised following these updates but still has some differences with the newest version. Indeed, some changes have been done during the production process of the robot (like the torque sensors hosting and the cabling) and thus cannot be replicated on Pyrène. Moreover, the robot manufacturer parameters calibration was not satisfying, and the actuators parameters are not disclosed (see Chapter 3). Pyrène head has been modified following the request from the Gepetto team to include a LIDAR and a second camera in addition to the color and depth Orbbec Astra camera primarily mounted (see Fig. 1.3).

1.4.1.a The flexibility on the hip of TALOS

Humanoid robots often have flexible or compliant components. For instance, the actuators stiffness of the robot WALKMAN Negrello et al. [START_REF] Negrello | Walk-man humanoid lower body design optimization for enhanced physical performance[END_REF] can be directly tuned, creating an intended flexibility. Another example of humanoid robot with compliant material is HRP-2 [START_REF] Kaneko | Humanoid robot hrp-2[END_REF]. It includes a bush rubber in the ankle in order to smooth impacts. In the robot TALOS, a non-intended flexibility on the hip link has been observed and impacts meaningfully the control of its legs and, therefore, its balance and locomotion. Indeed, this flexibility (not modeled in the simulator) leads to errors in the landing positions of the feet on the real robot. However, the deflection is not directly measurable by the encoders and cannot be directly modified. This subject is more detailed in the Appendix 4.

Wheeled Manipulator robot TIAGo

The standard robot TIAGo used by ROB4FAM is constituted of four main components: the head, the torso, the arm and the mobile base (see Fig. 1.4). TIAGo's head has 2 DoFs and is equipped with stereo microphones, a speaker and an RGB-D camera. The torso of the robot supports the arm and the head, and is equipped with an internal lifter mechanism to change the height of the robot achieving a minimum and a maximum height of 110cm and 140cm respectively. The arm of TIAGo is composed of 7 DoFs with 3 DoFs for the wrist and its end-effector is a 5 finger under-actuated hand. A Force/Torque sensor is integrated on the end-point of the wrist. Finally, TIAGo mobile base has a differential drive mechanism and contains an on board computer, batteries, power connector, laser-range finder, three rear sonars, a user panel, a service panel and two WiFi networks to ensure wireless connectivity. The IMU is mounted at the center of the mobile base and can be used to monitor inertial forces and attitude.

Because TIAGo has a mobile base, the balancing issues that the humanoid robot encounters are not as critical. Indeed, the wheels allow a stable way of locomotion in opposition to legs. However, they will not allow the robot to reach some areas, The developed solution on the humanoid robot TALOS can be transferred on TIAGo thanks to the similarity of the software and hardware of the arms. Indeed, in both cases it is possible to control the actuators directly using current commands or positions commands. Moreover, using the software stacks of the Gepetto team, the solutions are modular because they take into account the Unified Robot Description Format (URDF) of the robot, thus the kinematics and dynamics of the robot are computed based on it. The relevant part of the solution for the TIAGo is the control of the arm, but using the complete solution validated on TALOS allows to have a safe, stable and robust result (see Chapter 5).

Middle Size Drilling Robot

Airbus manufacturing engineering teams are working on light automation concepts. For this, they are looking for lighter robots than the ones currently used in the factories. Indeed, some of the classical industrial robotics cells design can reach 20 tons in total including the high payload robot, the heavy end-effector and the infrastructure requested to integrate them (see Fig. 1.4.3). This type of solution is disproportionate considering that the final goal is to drill holes about 4, 8mm. Moreover, this huge industrial setup is hard to move, thus has a limited travel workspace. It is a fairly high cost investment which is technically complex, not flexible and with a huge blocking point: its dependency to the proprietary implementations. Implementing a more suitable solution is hard, due to the closed source nature of its design. Indeed, there is often no access to the mechanical and inertial parameters of the robot and it is difficult to interface with the proprietary software provided by the robot manufacturer, or even by the robot cell integrator.

The automated fuselage structure assembly line of Hamburg Finkenwerder is presented in Fig. 1.4.3, as an example of this type of heavy solution. The manipu- As an alternative to the typical aeronautic heavy drilling robotic cells, the MSDR has been designed by Airbus to be a medium size solution. The robotic arm of the platform was built by the company Fanuc (the M -800 model), see Fig. 1.6. It is a 6 DoFs manipulator robot, it weights 820kg, has a payload of 60kg and is fixed on the ground. Similarly to the humanoid robot TALOS, the MSDR has double encoders, one on the motor side and one at the output of the harmonic drive. This secondary encoders is added to compensate eventual deflections or backlash on the axis. This simpler, lighter and cheaper solution has a work envelope of 2.2m and has been thought to realize autonomous manufacturing operations in areas where heavy robotic form factor does not fit or has reachability issues. An example of these areas is closed to the fuselage stiffeners, some holes need to be drilled but the end-effector is too big to correctly fit in the remaining space and accurately drill them.

The MSDR end-effector is equipped with several sensors to improve its capabilities to react to the environment: it has stereo-vision cameras, laser sensors, force sensors and an IMU. The laser sensors are used to measure distances in order to ensure the normality of the Tool Center Point (TCP). Moreover, because the MSDR has been designed with the Airbus specifications, the mechanical aspects and hardware are known. In addition, even if the software and programming language of Fanuc are proprietary, the software openness is high: the software implementations are indeed accessible. Moreover, the end effector behavior with in line path modification is left to the Airbus team, which can thus freely implement the desired behavior (under the constraint of the programming language of the robot). From TALOS to the MSDR, the gap is more important than from TALOS to TIAGo. In particular, because the MSDR is fixed to the ground, it has no underactuation part. However, it is possible to use the MSDR with an auxiliary axis mounted under its base (as it is the case with robots in Hamburg) to add redundancy to the robot. This new structure add an under actuated part of one DoF to the robot, but has not been considered during my thesis. Similarly than for TIAGo, the relevant part of the solution is the control of the arm, adding the guarantee to have a safe, stable and robust result (see Chapter 5). In addition, even if the Fanuc implementations are accessible, it will be necessary to create an interface between the proprietary software and our solution. Indeed, our controller needs to read the robot sensors and actuators values in real time (at least 1kHz), and to sends its commands to the actuators. A solution is discussed in the Chapter 6.

Scientific Context

The Gepetto team of the LAAS aims at analyzing and generating the motion of anthropomorphic systems. The key difficulty of motion generation and control of anthropomorphic systems comes from the redundancy of their tree-like structure with respect to most positioning tasks, the natural instability of the bipedal posture, and the under-actuation of their spatial displacement. The team studies this problem by following an interdisciplinary approach focused on three research objects: the humanoid robot, the numerical mannequin and the human body.

One objective is to provide humanoid robots with motion autonomy by developing methods and software for motion planning and control. As said before, humanoid robotics presents two main challenges: the complexity of the robot mechanical system (more than 30 DoFs) and the physical interaction between the robot and the real world.

The research activities of the Gepetto team can be described in three levels:

The fundamental research level concerns theoretical developments related to system modeling and motion generation. The modeling part includes the mechanics of robotics systems, the mathematics of new representations and operators, and the recording and analysis of the human motion. The motion generation part ranges from the global trajectory planning to the local movement control (considered under different kinds of constraints).

The integration level integrates the theoretical developments in software packages. These ones are maintained and made accessible to the whole robotics community (using standard formats and tools, see Section 1.6).

The application level concerns the contribution to different domains such as assistive robotics, industrial robotics (specifically the factory of the future), actuator design, human movement imitation and understanding.

My thesis is part of the motion generation research studies of the team and in particular the control stage. For a given mission and a given robot, a control vector has to be computed and sent to the robot motors in a timely manner. This problem is in general complex. More precisely, considering the problem of moving to find an object in an unknown environment is N P -hard Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF]. The ROB4FAM joint-lab focuses on the following operational challenges:

Climbing stairs with an heavy handled tool (designed for humans)

Manufacturing tasks applied on a real structure such as drilling or screwing Climbing stairs imposes to switch from one contact to another in order to make the humanoid robot TALOS move while handling an heavy object. This implies taking into account balance with multiple contacts, the geometry and the dynamical effect of the tool to handle. The robot has to take into account possible drift due to contact slip, spring back effect from the material part and vibrations when performing the tasks. On the other hand, drilling and screwing mostly involve whole body motion to perform a force control based behavior.

In order to produce a feasible motion, the robot has to decide where to interact with the environment to be able to move and perform its actions. Considering the climbing case, the robot has to locate where to put its feet and possibly its hand. Once this is decided, it needs to find a feasible motion for each of its bodies to generate the overall motion. When a feasible solution is found (i.e. avoiding collision, joint limits, ...) this solution needs to be executed on the robot. This is usually not trivial as discrepancies between the model and reality will affect the result. It is therefore necessary to modify these motions and to create a final control command for the robot in order to make sure that the plan is followed. These three stages: global trajectory planning, local planning for control, control of the robot (illustrated in Fig. 1.7) are the ones considered by the Gepetto team to realize motion generation. This approach is generic and allows to formulate solutions for different robots and scenarios. However, the way the problem is solved is dependant on the robot and the scenario, meaning that a solution cannot be reused on a different robot or Finding a collision free trajectory from the starting configuration to an admissible set of goals Extract a sequence of controls to be played on the robot Execute the sequence of controls to realize the behavior Figure 1.7: Overall approach of the motion generation problem (WP) scenario, a specific solution will be needed. For instance, using the same problem formulation for stairs climbing and drilling will not work on both scenario; neither will the use of the same problem formulation for drilling using TALOS and the MSDR work. Nevertheless, they will be similar, because the approach uses generic formulation.

Let us focus on the example of the control formulations of an humanoid robot and a manipulator. The formulation used by the team is called the Stack-of-Tasks (SoT) and consists in realizing several Tasks at the same time to achieve a goal. These Tasks can for example be to move the end-effector at some desired position or to maintain the Center of Mass (CoM) at its position to keep the balance of the robot. Moreover, these Tasks can be prioritized to decide which action is more important than another (which can then be not perfectly achieved) or set as constraints, meaning that they must be fulfilled. In Fig. 1.8 is presented an example of the formulation to control a humanoid robot TALOS. Compared to the one for two robot manipulators shown in Fig. 1.9, one can see the similarities of the tasks involved. The h values represent the Task functions to be optimized by the control formulation to achieve the B desired values. In both cases the formulations involve Tasks on the CoM, on the arms-tips and on the feet-bases. Their prioritization is the same, with the most important Tasks at the bottom and the least one at the top. At the lowest is set a constraint as an example, the collision detection constraint to avoid the robot bodies to collide:

d(B i , B j ) ≥ (1.1)
for (B i , B j ) ∈ Collisions where Collisions is the set of body pairs for which collisions should be checked.

Using the Gepetto team approach and tools allows this thesis to propose implementations of controllers for humanoid robots which are versatile and can be used for industrial robots.

Frameworks and Libraries

In this section are quickly described the frameworks and libraries implemented by the Gepetto team to solve the problem of motion generation which have been used and expanded during my thesis. The advantage of these tools is that they are based on the same mathematical library Pinocchio Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF], and can interact with each others using the dynamic-graph package [START_REF]dynamic-graph[END_REF].

Pinocchio Library

Pinocchio Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF] is a library which instantiates the state-of-the-art Rigid Body Algorithms for poly-articulated systems based on revisited Roy Featherstone's algorithms Featherstone [START_REF] Featherstone | Rigid Body Dynamics Algorithms[END_REF]. Pinocchio efficiently computes the dynamics (and derivatives) of a robot model, or of any articulated rigid-body model (avatars in a simulator, skeletal models for bio-mechanics, etc.). It provides the analytical derivatives of the main Rigid-Body Algorithms like the Recursive Newton-Euler Algorithm or the Articulated-Body Algorithm. The library is open-source, mostly written in C++ with Python bindings, and distributed under the BSD licence. It has been used in this thesis for all the computations of the rigid body kinematics and dynamics.

Stack-of-Tasks (SoT) Framework

The main framework used in this thesis is the SoT Mansard et al. [START_REF] Mansard | A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: The stack of tasks[END_REF], it is a C++ Software Development Kit implementing a control architecture for redundant robots and more specifically for humanoid robots. The SoT framework is a collection of software packages handled by cmake and pkg-config. The rigid body kinematics and dynamics is computed using the Pinocchio library Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF].

1.6.2.a Dynamic Graph

The dynamic-graph package is used to connect computation nodes, entities together using a Graphical System Design (GSD), akin to what Simulink © does MathWorks [START_REF]Simulation and model-based design[END_REF]. Entities are connected through input and output signals. This package implements on-demand signal computation and an efficient caching mechanism to avoid useless data re-computation. This allows fast computation of signal values, which is a critical point for real-time control.

1.6.2.b ROS-Control SoT

This package encapsulates the SoT graph in the ros-control framework of the middleware ROS Stanford Artificial Intelligence Laboratory et al. [7]. The intent is to make it generic and adapted to any robot through rosparam. Thus, this package allows to have a ros-control architecture compatibility. It is particularly useful because the ros-control framework provides a generic API for controllers and robot hardware. For instance, one can choose to use the SoT controller or the one provided by the robots manufacturer, PAL-robotics, in the same way. Moreover, with this compatibility, Figure 1.10: ROS-Control framework with the ROS-Control SoT interface it is possible to use the Gazebo simulator of ros-control. The software developed for the real robot can then be tested in simulation with the same architecture. However, one issue of the generic ROS API is highlighted when specific components are used in the robot hardware. Indeed, if the actuators have specific sensors, such as temperature ones as the robot TALOS does, it is not handled by the architecture. Thus, the roscontrol-sot package [18] has been modified to take these sensors into account. This package is represented by the dark blue blocks in the SoT Framework scheme of Figure .1.11. It creates an abstract interface to communicate with the hardware interface of ros-control, its integration in the whole ros-control framework is displayed in Fig. 1.10 in purple, based on [START_REF] Rodríguez | Ros control, an overview[END_REF].

1.6.2.c SoT Core

SoT-core [20] is the package in charge of the control architecture. The package implements hierarchical control using the task-based inverse kinematics described in Mansard et al. [START_REF] Mansard | A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: The stack of tasks[END_REF] (see Section 4.2). The robots can be loaded from their URDF model. The rigid body dynamics is provided through the Pinocchio library Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF]. It has been used in this thesis to compare hierarchical position control to weighted position and torque control in Section 4.2.

1.6.2.d Overall scheme for real-time control

In the Fig. 1.11 is presented the general SoT flow chart, where the SoT-Core purple blocks can be replaced by another task-based controller (for instance a controller using TSID) and the Planner block can be any planning algorithm giving the appropriate references. Real-time control system are usually driven by a cyclic computational node which needs to send a control reference value to each motors of a robot. To compute this control reference values, sensor values need to be provided. In the SoT, special entities called Device are used to provide an abstract interface to the hardware. The Device has specific inputs which contains the control vector. This control vector is the result of a computation solving a control problem,

Task Space Inverse Dynamics (TSID) Library

TSID is a C++ library for optimization-based inverse-dynamics control based on the rigid multi-body dynamics library Pinocchio Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF]. It implements a non-strict hierarchical control using tasks functions Prete et al. [START_REF] Del Prete | Implementing torque control with high-ratio gear boxes and without joint-torque sensors[END_REF]. It has been used in this thesis to create the weighted position and torque control schemes in Section 4.2 and have been modified to include a passive constraint in Chapter 5.

Summary of the Chapters

The thesis is organized as follows: Chapter 2 presents a state-of-the-art on the different research domains encountered in this thesis. A particular attention is given to the whole-body control and the force control methods, which are the core of the thesis problem.

Chapter 3 details the first work achieved during this thesis toward the protection of the robotic system. It initially consists in identifying and modeling the robot chain drive and then in using it in a MPC to send safe torque commands. It has led to the publication Ramuzat et al. [22].

Chapter 4 presents the different works realized while investigating the differences and performances of whole-body control architectures for humanoid robots. Firstly, the developed control architectures during this thesis are detailed, a focus on the hip flexibility control of the robot TALOS can be found in Appendix 4. Moreover, the chosen force control to realize the manufacturing operations is explained. The targeted application is the drilling of a metal plate, requiring high precision and the application of huge forces.

Secondly, three controllers are compared: one solving torque commands and two position commands. The aim is to choose the most appropriate scheme for the robot TALOS, these controllers are compared in terms of tracking error, energy consumption and computational time by using Gazebo simulations of the robot walking on flat horizontal ground, tilted platforms, and stairs. This work has led to the publication Ramuzat et al. [23].

Thirdly, the efficiency of the previous whole-body torque controller is exposed in a context of human-robot collaboration. The goal is to make the robot proactively walk alongside a human. The results have been obtained during a collaboration with another PhD student of the Gepetto team, Isabelle Maroger and have been published in Maroger et al. [24]. This work combines an optimal control prediction model of the human trajectory, a Walking Pattern Generator (WPG) based on non-linear MPC and the previous real-time whole-body torque controller.

Finally, the last section describes the experiments realized on the real robot, using the presented controllers. This section discusses the different difficulties encountered and the approaches we tried to implement to solve them, leading to the publication Ramuzat et al. [START_REF] Ramuzat | Benchmarking whole body controllers on talos[END_REF].

Chapter 5 details the second work realized during this thesis toward the safety of the robotic system. It describes a new passivity-based inverse dynamics (ID) controller, based on the torque controller of Chapter 4 and the passivity theory. The control approach allows to achieve a safe multi-contact scenario on a torque controlled humanoid robot, using a global energy tank. This work has led to the publication Ramuzat et al. [START_REF] Ramuzat | Passive inverse dynamics control using a global energy tank for torque-controlled humanoid robots in multi-contact[END_REF].

Chapter 6 describes the expected challenges that will be encountered when transposing the obtained results from the TALOS robot to the MSDR or another industrial robot. It highlights the benefits and drawbacks of my complex and state-ofthe-art solution compared to the classical proposal for manipulator robots.

Finally the last chapter presents the conclusions of my works and the perspectives and future aims. In this chapter, some parts of the state-of-the-art presented were elements of the works realized for the Work-Package (WP) reports in the scope of the jointlaboratory ROB4FAM. Some of the figures presented are thus indicated of part of these reports (WP). The presented section on human-robot collaboration is part of the paper [24] realized in collaboration with Isabelle Maroger. Moreover, Louise Scherrer has kindly authorized us to use some of her figures from her master report Scherrer [START_REF] Scherrer | Reactive walking for the humanoid robot pyrène[END_REF] to illustrate some principles and the Walking Pattern Generator of Naveau et al. [START_REF] Naveau | A reactive walking pattern generator based on nonlinear model predictive control[END_REF].

Publications

Humanoid Robot Model

Robot Model

Our experimental platform is a robot TALOS from the PAL-Robotics company, it is 1.75m tall weighting 100kg and has electric actuators. It has n j = 32 joints and a floating base of n b = 6 DoFs, we denote n = n j + n b , and define the configuration space, called C-space. This space can be decomposed in three parts, leading to the following representation:

C = C position × C orientation × C joints C = R 3 × SO(3) × Q (2.1)
where R 3 represents the coordinates of origin of the floating base frame B and SO( 3) is the Lie group representing the rotation matrices that define the orientation of this frame relative to the world frame W . Q is the subspace of the n j -joint space, based on the Denavit-Hartenberg Bejczy [START_REF] Antal | Robot arm dynamics and control[END_REF] representation of the joint angles q j ∈ R n j . We express the coordinates of the robot q ∈ C as:

q =    x b R b q j    (2.2)
with the under-actuated coordinates of the floating base

b = [x b R b ] T ∈ R 3 × SO(3) (x b
the position and R b the orientation of the base frame B relative to the world frame W ) and the Denavit-Hartenberg Bejczy [START_REF] Antal | Robot arm dynamics and control[END_REF] representation of the joint angles q j ∈ R n j . Then, we can express the velocity of the robot v, composed of the linear velocity of the floating base (the derivative of the floating base position), the angular velocity of the floating base (which is not the derivative of the floating base orientation) and the joint angular velocities of the robot (the derivatives of q j ). Thus, v is not the direct derivative of q, the two vectors does not have the same size, as R b is a rotation matrix or a quaternion of size 4, whereas the angular velocity of the floating base is a angular velocity vector of size 3. We denote v as:

v =    ẋb ω b qj    (2.3)
with the under-actuated velocities of the floating base [ ẋb ω b ] T ∈ R 3 × R 3 and the joint angular velocities of the robot qj ∈ R n j . There exist a mapping between the two representations M :

[x b R b ] → [x b θ b ] with θ b ∈ R 3 ( θb = ω b ).
The acceleration of the robot a is then simply the derivative of v.

The equation of the robot dynamics can be written as:

M (q)a + C(q, v)v + g(q) h =    0 0 τ    N T τ + τ ext J T c F (2.4)
M ∈ R n×n the symmetric and positive definite inertia matrix, C ∈ R n×n the Coriolis matrix and g ∈ R n the gravity vector. q j ∈ R n j is the joint configuration of the robot. a, v, q ∈ R n are the accelerations, velocities and positions of the joint configuration of the robot including the base (free-flyer). The free-flyer information are estimated with a base-estimator from the configuration, IMU and force sensors of the robot (we use in the thesis the one persented in Flayols et al. [START_REF] Flayols | Experimental evaluation of simple estimators for humanoid robots[END_REF]). τ ∈ R n j are the joint torques of the actuators and τ ext ∈ R n are the external torques. N is a selector matrix associated to the actuated joints

N = [0 n b , 1 n j ], such that N T ∈ R n×n j .
A notable property of the dynamic model is that the choice for the Coriolis matrix C is not unique. A particular choice is to define this matrix such that:

1 2 Ṁ (q) = C(q, v) + C(q, v) T 2
Siciliano et al. [2]. Indeed, with this formulation, the matrix Ṁ (q) -2C(q, v) is skew-symmetric, thus their quadratic form is null, i.e. for any vector x ∈ R n :

x T [ Ṁ (q) -2C(q, v)]x = 0 (2.5)
It is an important property that is often used for simplifications.

Using the force contacts F , we can write that τ ext = J T c F = nc i=1 J T i F i with n c the number of contacts and J i the Jacobian of the contact point x i c (with the contact frame V i ) according to the robot coordinate vector q:

J i = [J i u J i a ] = T (V i , B) ∂x i c ∂q j (2.6)
where J i u is the "Jacobian" of the under-actuated part: it is the transformation T between the contact frame and the base frame Siciliano et al. [2]. And J i a is the classical Jacobian of the actuated part.

By denoting qj , qj the velocity and acceleration of the joints and ẋb , ω b the linear and angular velocities of the floating base frame we can decompose Eq.2.4, to have Henze et al. [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF]:

M (q)    ẍb ωb qj    a + C(q, v)    ẋb ω b qj    v + g(q) = N T τ + τ ext (2.7)
If all the external forces act at the end-effector frames V i with i = 1...ψ, with ψ the number of end-effector frames, then we can write:

τ ext = ψ i=1    I 0 xbi I J iT a    F i (2.8)
with J i a the actuated Jacobian matrix of the end-effector i making the contact point x i (see Eq.2.6),

F i = [f T i τ T i ]
T the effector wrench. xbi is the cross-product matrix of the vector x bi = x i -x b : the configuration-dependent lever arm between the endeffector frame V i and the base frame B (defining J u = T (V i , B) the transformation of Eq.2.6).

For balancing it is interesting to replace the floating base frame by the CoM one which has the same orientation as the base link. The equations are similar with c, w com , x com,i instead of x b , ω b , x bi and using the transformation matrix T:

   ċ ω com qj    =    I -x b,com J b,com 0 I 0 0 0 I    T    ẋb ω b qj   
(2.9)

We obtain the following simplified equation Wieber et al. [START_REF] Wieber | Modeling and control of legged robots[END_REF], denoting

v com = [ ċT ω T com ] T : M vcom qj + C v com qj + m g 0 0 = 0 τ + τ ext (2.10) We have M (q) ∈ R (n j +n b ) × (n j +n b ) , C(q, v) ∈ R (n j +n b ) × (n j +n b ) , τ ∈ R n j and τ ext ∈ R (n j +n b ) .
Here one can notice that the gravity vector g is projected as the robot weight in its base frame. It is a major difference compared to fixed based robots where this simplification is not possible.

Contacts properties

Using friction model of Coulomb Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF], the slippage is avoided when Henze et al. [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF], Ramos et al. [START_REF] Ramos | Dancing humanoid robots: Systematic use of osid to compute dynamically consistent movements following a motion capture pattern[END_REF]:

f ≤ µ |f ⊥ | (2.11)
with f and f ⊥ the tangential and normal components of the contact forces (in our case for each of the i th end-effector). µ is the coefficient of friction, an empirical property of the contacting materials. Similarly a constraint can be added on the torque to limit the friction:

τ ≤ µ |τ ⊥ | (2.12)
To avoid the interpenetration of the end-effector and the surface at the contact points x c (for instance between the foot and the ground) the Signorini conditions have to be met, called the complementarity constraints:

f ⊥ ≥ 0 ẍc⊥ ≥ 0 ẍc⊥ f ⊥ = 0 (2.13)
These equations can be simplified by fulfilling the condition of positivity to guarantee the contact:

f ⊥ ≥ f min ⊥ ẍc⊥ = 0 ⇐⇒ f min ⊥ ≥ 0 J c a + Jc v = 0 (2.14)
because ẋc⊥ = J c v, with J c the Jacobian matrix of the contact point x c (as defined in Eq.2.6).

Finally to prevent the end-effector from tilting, the Center of Pressure (CoP) p ∈ R 3 , can be restricted as follow to lie in the contact surface:

p ⊥ ∈ p min ⊥ , p max ⊥ (2.15) p 1 f 1,z p 2 f 2,z c f 1,y f 1,z f 1,x f 1 p 1 Figure 2.1: (left)
The HRP-2 humanoid robot walking on a non-flat terrain. On each contact p i a wrench (f i , τ i ) is applied. For sake of clarity all the components are not represented. This figure is part of the WP reports.

(right) A graphical representation of the Coulomb Friction constraint applied to the right foot of the humanoid robot with

f 1 = [f 1,x f 1,y f 1,z ] T and µf 2 1,z > f 2 1,x + f 2 1,y
(Eq.2.11).

These constraints define the allowed force at the contact points, which represents a friction cone defined by an axis along the surface normal and a semi-angle θ = arctan µ (see Fic.2.1 for an illustration of the friction cone).

Task Space and Workspace

It is common to distinguish two spaces from the configuration space of the robot (C-space linked to q). These spaces are denoted the task space and the workspace (usually defined for the end-effectors of the robot).

The task space is a space in which a task performed by the robot can be naturally expressed. For instance, to control the 3D position of the CoM of the robot, the task space would be R 3 . However, if one want to control the posture of the robot (the positions of each joint angles), then the task space corresponds to the configuration space. The task space is linked to the configuration space with the Jacobian matrix associated to the task Orin et al. [START_REF] Orin | Centroidal dynamics of a humanoid robot[END_REF]. One can switch between one space to another using the relation: ẋ = J a qj where x is the task point position in the task space, qj the velocities of the joint configuration and J a the Jacobian of the task point according to the robot state vector J a = ∂x ∂q . Using the Eq.2.6, we can also define the relation ẋ = Jv with v the velocity vector of the robot including its floating base.

The workspace defines the reachability space of the robot, i.e the set of configurations reachable by the robot end-effectors. This definition is independent of any task and is characterized by the robot structure.

A point in the task space or in the workspace may be achievable by more than one robot configuration, meaning that the point is not a full specification of the robot's configuration. Conversely, some points in the task space may not be reachable at all by the robot. By definition, however, all points in the workspace are reachable by at least one configuration of the robot Lynch and Park [35].

In this thesis, we use and focus on solutions designing the desired motions of the robot in the task spaces rather than in the configuration one. This approach is called the task-function approach Khatib [START_REF] Khatib | A unified approach for motion and force control of robot manipulators: The operational space formulation[END_REF], Samson et al. [START_REF] Samson | Robot control. the task-function approach[END_REF], Escande et al. [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF]. The objectives to be performed by the robot are expressed in their respective task spaces, using reference trajectories given by the motion planning (for instance the CoM, end-effectors, feet or head desired trajectories). The planning algorithm computes references respecting the reachability space of the robot.

Centroidal Dynamics

The under-actuated part of the whole-body dynamics of a robot is called the centroidal dynamics, that is the dynamics of the CoM. It is possible to project the entire robot dynamics on it. Considering the robot as a rigid body, one can use the Newton-Euler equations of motion to couple the variations of the centroidal momentum with the contact forces Orin et al. [START_REF] Orin | Centroidal dynamics of a humanoid robot[END_REF] (using Eq.2.10):

mc = i f i + mg = lc mc × (c -g) + L = i (p i -c i ) × f i + τ i = kc (2.16)
with c, ċ, c the CoM position, velocity and acceleration

, L = k [R k I k ẇk - R k (I k w k ) × w k ]
and g = [0, 0, -9.81] T , where R k ∈ SO [START_REF] Spenko | The DARPA Robotics Challenge Finals: Humanoid Robots To The Rescue[END_REF] is the rotation matrix between the k th body frame and the inertial coordinate frame, I k its inertial matrix, w k its angular velocity, m is the mass of the robot, f i ∈ R 3 the vector of contact forces at contact point i, p i ∈ R 3 their positions and τ i ∈ R 3 their contact torque (represented at the inertial coordinate frame). l c and k c ∈ R 3 are the linear and angular momentum around the CoM. The operator × denotes the cross product between two terms. Solving this problem can be complex mostly due to the term c × c. Because of this term, the dynamics of the system is not linear.Over a receding horizon this term generates polynomials which might be either convex or concave. In general solving such problem is known to be N P -Hard Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF].

Several questions need to be answered to make a robot evolve on any terrain based only on these equations: How to choose the contact location p i ∈ R ? What are the CoM trajectory c ∈ R 3 and the equivalent forces trajectories fulfilling Eq.2.16 ?

How to take into account constraints on L, p i , f i ?

In order to simplify the problem one might consider the resulting wrench of the external forces f ext acting on the CoM and its application point p, called the CoP.

mc = f ext + mg mc × (c -g) + L = p × f ext + τ ext (2.17) with c = [c x c y c z ] T , c = [c x cy cz ] T , L = [ Lx Ly Lz ] T .
This is equivalent to pose:

f ext = nc i=0 f i p × f ext = nc i=0 p i × f i (2.18)
This model is also called the free wheel model, or the inertia wheel model.

Zero Moment Point

The Zero Moment Point (ZMP) has first been introduced in Surla et al. [START_REF] Surla | Biped Locomotion: Dynamics, Stability, Control and Application[END_REF]. In 2D, the ZMP is a point where the moment of the ground reaction force is zero (giving the name of the point), while in 3D it specifies the point where the total of horizontal inertia and gravity forces is null Kajita et al. [START_REF] Kajita | ZMP and Dynamics[END_REF]. The concept assumes planar contact area (p z = 0) with high friction to avoid feet sliding.

The ZMP definition is linked to the support polygon one, which corresponds to a convex hull, the smallest convex set including all the contact points. By definition, the ZMP is the application point p when p is enclosed in the support polygon, otherwise p still exists (it is the CoP) but the system is not guaranteed to be in dynamic equilibrium.

In Fig. 2.2 is presented an illustrative example with the robot standing on flat floor with two contact points: p lef t , p right . They correspond to the application points of the left and right force resultants (in their friction cones). The resulting wrench of the external forces acting on the CoM is applied on a point inside the support polygon, it is thus the ZMP.

Linear Inverted Pendulum Model

Let us consider the total wrench applied to the robot and the ZMP p where it is applied. When a robot has its contact point on the floor we can set its altitude to zero, i.e. p z = 0. Then, considering the previous model, it is equivalent to a robot with a flying wheel at the top of an inverted pendulum. Thus Eq. 2.17 gives:

                   -c z cy + c y (c z + g) + Lx m = p y (c z + g) c z cx -c x (c z + g) + Ly m = -p x (c z + g) c x cy -c y cx + Lz m = p x cy -p y cx (2.19)
In general, to simplify the problem, L is neglected: the moment induced by the limbs of the robot on its CoM is considered much smaller then the one induced by the contact forces Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF].

The Linear Inverted Pendulum Model (LIPM), introduced in Kajita et al. [START_REF] Kajita | Real-time 3d walking pattern generation for a biped robot with telescopic legs[END_REF], can be obtained by considering an artificial constraint on the motions of the pendulum. That is, the motions of the center of mass of the biped robot is limited to a Figure 2.2: Scheme of the contact forces at points p i for each foot and their contact wrench cones and the ZM P in the support polygon. Scherrer [START_REF] Scherrer | Reactive walking for the humanoid robot pyrène[END_REF] Figure 2.3: Link between the ZMP (in blue), the support contact positions (in green and red) and the CoM (in orange) Scherrer [START_REF] Scherrer | Reactive walking for the humanoid robot pyrène[END_REF] CHAPTER 2. STATE OF THE ART plan whose normal vector is k x k y -1 T and z intersection is z c . This constraint can be formulated as :

c z = k x c x + k y c y + z c (2.20)
and by successive derivation :

cz = k x cx + k y cy (2.21)
To retrieve the equations of the LIPM of Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF] it is common to simplify the problem by considering the planar constraint to be horizontal, i.e. k x = k y = 0 and then cz = 0. In addition to that hypothesis, the part of Angular Momentum (AM), depicted as L previously, is neglected in front of the influence of the center of mass motion (induced by the contact forces).

Thus, from the eq. 2.19, the following linear relationship can be written between the CoM and the ZMP:

       p x = c x - z c g cx = c x - c z g cx p y = c y - z c g cy = c y - c z g cy (2.22)
It is important to notice that the two equations are decoupled. Therefore, using the LIPM formulation, one can solve separately the equations along the x and the y axis.

Since the ZMP position is imposed by the contact forces and contact positions, one can use the LIPM equations to deduce the CoM trajectory from the ZMP one (see Fig. 2.3 for an illustration). This is a common implementation in WPG Naveau et al. [START_REF] Naveau | A reactive walking pattern generator based on nonlinear model predictive control[END_REF], Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF], Herdt et al. [START_REF] Herdt | Walking without thinking about it[END_REF], creating the path of the CoM to respect the constraints on the contact forces, thus maintaining the contacts necessary for a balanced motion. Moreover, keeping the ZMP inside the support polygon with a certain margin is also often implemented in WPG, to obtain a safer balancing of the robot (reduce the risk of fall).

Equilibrium Criteria

2.2.3.a Divergent Component of Motion

We recall that the problem considers the planar constraint to be horizontal, i.e. cz = 0 and thus c z = constant. One can rewrite the eq.2.22 by noting ω = g/c z as follows:

cx = ω 2 (c x -p x ) (2.23)
And, setting X = [c x ċx ] T in eq.2.23, one can obtain the following system of equations:

Ẋ = 0 1 ω 2 0 X + 0 -ω 2 p x = AX + b (2.

24)

This matrix A can be decomposed as :

A = 1 2 1 1 -ω ω -ω 0 0 ω 1 -1 ω 1 1 ω = 1 2 P -ω 0 0 ω P -1 (2.25)
It is then possible to make a change of variables:

X =    ζ x ξ x    = P -1 X =    1 -1 ω 1 1 ω       c x ċx    =    c x -ċx ω c x + ċx ω    (2.26)
The dynamics of the LIPM can therefore be divided in two: the upper part which is naturally converging, and the lower part which is diverging. The diverging part is called the Divergent Component of Motion (DCM), and its definition is given by:

ξ x = c x + ċx ω (2.27)
The DCM has been introduced first in Pratt et al. [START_REF] Pratt | Capture point: A step toward humanoid push recovery[END_REF] under the name "Capture Point", while the presentation given here is the one provided by Takenaka et al. [START_REF] Takenaka | Real time motion generation and control for biped robot-4th report: Integrated balance control[END_REF] from Honda, and popularized by the work of Englsberger et al. [START_REF] Englsberger | Three-dimensional bipedal walking control based on divergent component of motion[END_REF] on the humanoid robot TORO.

2.2.3.b CoM Admittance Control

The net contact wrench (that is the contact wrench expressed at the level of the CoM) can be computed indirectly by defining the ZMP position that will produce the said contact wrench. Under the assumptions of the LIPM, one can retrieve the following two coupled first-order equations Englsberger et al. [START_REF] Englsberger | Three-dimensional bipedal walking control based on divergent component of motion[END_REF], Takenaka et al. [START_REF] Takenaka | Real time motion generation and control for biped robot-1st report: Walking gait pattern generation[END_REF]:

ċ = ω(ξ -c) ξ = ω(ξ -p) ξ = c + ċ ω (2.28)
with p, ξ respectively the ZMP and DCM and ω = g/c z .

These equations show that the DCM diverges from the ZMP, while the CoM converges to the DCM. Thus, the DCM can be controlled to control the CoM deviations and stabilize the system Englsberger et al. [START_REF] Englsberger | Three-dimensional bipedal walking control based on divergent component of motion[END_REF], Kajita et al. [START_REF] Kajita | Biped walking stabilization based on linear inverted pendulum tracking[END_REF], Sugihara [START_REF] Sugihara | Standing stabilizability and stepping maneuver in planar bipedalism based on the best com-zmp regulator[END_REF], Mesesan et al. [49]. This strategy is called the admittance control of the CoM.

From a CoM reference trajectory provided by a planner, a reference DCM can be defined as well. This reference, along with a measure of the DCM (based on the estimation of the base position of the robot) can be used in a Proportional-Integral (PI) controller. This controller has been presented in Caron et al. [START_REF] Caron | Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control[END_REF] (the integral term is used to eliminate the steady-state error):

ξ = ξ * + K P dcm (ξ * -ξ) + K Idcm (ξ * -ξ) (2.29)
with ξ * the desired DCM given by the planning, ξ the estimated DCM and K P dcm , K Idcm the proportional and integral gains. In terms of ZMP (see Eq.2.28), the obtained control law is Caron et al. [START_REF] Caron | Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control[END_REF]:

p * = p ref -1 + K Pdcm ω (ξ ref -ξ) + K Zdcm ω (p ref -p) - K Idcm ω (ξ ref -ξ) (2.

Angular Momentum

As said before, in the LIPM formulation, the Angular Momentum (AM) L is neglected. However, when a human dynamically walks, its AM is non-null and keeping it to 0 can lead to instability. In Kajita et al. [START_REF] Kajita | Resolved momentum control: Humanoid motion planning based on the linear and angular momentum[END_REF], one of the objective is to consider the AM part generated by the contact transition. A controller may thus aim to have a task regulating the AM of the robot. Indeed, commonly a posture task is defined to keep the uncontrolled joints of the robot at fixed positions, in particular on the upper-body. However, this does not allow motions of the arms to help the walk. Using an AM task fixes this issue, as the controller solution to generate this momentum is in general making the robot arms move Lee and Goswami [START_REF] Lee | A momentum-based balance controller for humanoid robots on non-level and non-stationary ground[END_REF].

Using the equation Eq.2.16, the centroidal dynamics is therefore defined by h c = [l c k c ] T ∈ R 6 . In Wensing and Orin [START_REF] Wensing | Generation of dynamic humanoid behaviors through task-space control with conic optimization[END_REF], the task formulation of the centroidal dynamics control is given by h c = A G (q j ) qj where q j , qj are the joint position and velocity vectors of the robot and A G is the Centroidal Momentum Matrix Orin et al. [START_REF] Orin | Centroidal dynamics of a humanoid robot[END_REF].

The linear and AM tasks dynamics are given by the following equations:

lc = m [c * + K Dcom ( ċ * -ċ) + K Pcom (c * -c)] kc = k * c + K Pam (k * c -k c ) (2.32)
with lc and kc the commanded rates of change of the linear momentum and angular momentum. K Pcom , K Dcom the proportional and derivative gains of the linear momentum task (CoM position) and K Pam the proportional gain for the AM task.

The AM task has been successfully implemented in Lee and Goswami [START_REF] Lee | A momentum-based balance controller for humanoid robots on non-level and non-stationary ground[END_REF] and used in the Section 4.4 in the controllers based on TSID. Another formulation of this task can be obtained by expressing the robot dynamics equation at the centroidal momentum level in the Quadratic Programming (QP) Koolen et al. [START_REF] Koolen | Design of a momentum-based control framework and application to the humanoid robot atlas[END_REF].

Actuator Model

In this part is described briefly how the dynamic model of electric actuators is formulated in the literature. Part of this description will be used in the Chapter 3 to model the elbow chain actuation of TALOS. An actuator controls a motor which is followed by a gear reduction system (often a wave generator), itself fixed to a link, see Fig. 2.3 for the TALOS actuator. The motor delivers forces or torques that cause the robot's links motion.

General formulation

The overall robot dynamic and its actuators are governed by the following equations Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF]:

M a + Cv + g = τ + DK -1 τ + τ ext I m θj + τ + DK -1 τ = τ m -τ f τ = K(θ j -q j ) (2.33)
with q j ∈ R n j , θ j ∈ R n j the link and motor side positions respectively. The gear reduction ratio is defined as r j = θ j q j for actuator j. High ration reduces the coupling effect between the joints W. Khalil [START_REF] Khalil | Modeling, identification and control of robots[END_REF]. τ m is the motor torque. We recall that v = [ ẋb , ω b , qj ] T with ( ẋb , ω b ) the linear and angular velocity of the robot base, qj the velocity of the actuated joint. τ f represents the friction (see the following section). K = diag(k i ) ∈ R n j ×n j the diagonal, positive definite joint stiffness matrix and D = diag(d i ) ∈ R n j ×n j the diagonal, positive semi-definite joint damping matrix. I m = diag(i i ) ∈ R n j ×n j is the matrix of rotor inertia.

In general, for rigid robot, K → ∞, thus the last equation of Eq.2.33 is not taken into account and the positions of the joint and the motor are considered equal (θ = q j ). And then K -1 → 0, modifying the first equation of Eq.2.33.

Friction

A simple approximation of the frictions τ f model can be expressed the following way:

τ f = F v qj + F s sign( qj ) (2.34)
where sign(.) is the function returning the sign of a variable.

Two kinds of frictions can take place all over the mechanisms: viscous frictions F v and dry frictions F s . This model has some short comings in the sense that it creates oscillations when the speed is close to zero. In this case, Coulomb frictions might appear and when they are not symmetric, leads to hysteresis.

Parameters Identification

Parameters identification of the actuator model consist in identifying all the parameters previously defined as I m , D, K, F v , F s but also:

The geometrical parameters of the robot, the gear reduction ratio r j for actuator j, defined by r j = θ j q j , the motor constant relating current i j to motor torque τ m,j for each actuator j such that τ m,j = K m,j i j .

the Center-Of-Mass position for each link c j , This is realized by generating trajectories which are maximizing the observability of the parameters W. Khalil [START_REF] Khalil | Modeling, identification and control of robots[END_REF], called optimal exciting motions (OEM) Bonnet et al. [START_REF] Bonnet | Self-generation of optimal exciting motions for identification of a humanoid robot[END_REF]. On TALOS it is possible to measure the joint positions, the motor positions, the current and the external torque (q j , θ, i, τ ext ). From this, the gear ratio r j and the motor constant K m,j can be identified.

For I m , D, K, F v , F s a common approach is to solve a Least Square Problem (LSP), based on Eq.2.33. Note that most of the non-linear parts of the equations are related to quantities which can be measured. Therefore while the equations are non-linear, the Least Square Problem is not. More precisely all the parameters n par (for each actuator) to be identified are put together in a vector of free variables called X ∈ R n×npar . All the measurements n meas are stack together to obtain a vector Y . The theoretical relationship is then Y = AX with A ∈ R nmeas×npar . A large number of measurements will make this equality over-constrained, thus to identify the parameters, the following LSP is solved Gautier and Jubien [START_REF] Gautier | Force calibration of KUKA lwr-like robots including embedded joint torque sensors and robot structure[END_REF]:

min X ||Y -AX|| (2.35)
When the measured variables are not precise or compromised, it is possible to use a Ransac (RANdom SAmple Consensus) algorithm or another iterative method to still estimate the parameters of the model.

Actuator Control

In general, the whole body controller provides a control vector u to the robot in two possible forms (see Section 2.5):

q * j , q * j , or q * j a desired position or velocity or acceleration for each actuator, τ * a desired torque for each joint.

However, the real control variable that is provided by the power electronics of the robot is the motor current. Thus, one have to add a layer of control creating a relationship between the output of the controller and the motor current Lynch and Park [35]. For the first form of u one can use a position control, while for the second form one will need to implement a torque control. In the following sections, for each motor j the current is noted i j .

2.3.4.a Position control

The position control is usually done by a simple Proportional Integral Derivative (PID) embedded in the electronic board of the robot:

i j = P j (q * j -q j ) + I j t 0 (q * j (h) -q j (h))dh + D j ( q * j -qj ) (2.36)
with q * j , q j the desired and measured joint j positions, P j the proportional gain, D j the derivative gain and I j the integral one. Proportional gains for this kind of control are typically very high (10000 for P j ), with I j often very small (0.1) or equal to zero. This is making the robot extremely rigid, but it is quite simple to deploy.

On TALOS and TIAGO this low level control is realized at 10 kHz by the Elmo board sending the current to the motors.

2.3.4.b Torque control

On the other hand to have a robot which is compliant, one would like to control the torque applied τ j and not the position.

There is a non linear relationship between the joint torque and the motor current which depends on the actuator model (see Section 2.3):

f :R → R (2.37) i j → τ j (2.38)
Once the chain actuation is modeled and its parameters identified, one can express this relationship in a low-level controller to directly send current commands from desired torques to the robot. Chapter 3 presents one implementation of such a controller for the elbow of the TALOS robot.

Motion and Locomotion Planning

During this thesis, two distinct planning approaches have been used to compute the reference trajectories used by our controllers: a Walking Pattern Generator WPG Stasse et al. [START_REF] Stasse | Modular architecture for humanoid walking pattern prototyping and experiments[END_REF], [START_REF]jrl-walkgen[END_REF] and a multi-contact locomotion framework Carpentier et al. [START_REF] Carpentier | Multi-contact locomotion of legged robotsin complex environments -the loco3d project[END_REF], [START_REF]multicontact-locomotion-planning[END_REF]. Because motion planning is not the core of our research subject, only this two approaches are described in this section. They have been chosen because they are part of the algorithms developed by the Gepetto team and can thus easily be connected to our architectures. But, we could have used any other methods providing reference trajectories for the CoM and the feet. Moreover, the two used approaches meet the requirements and needs of the thesis context: allowing to plan real-time locomotion solution for the first approach and multi-contact scenarios for the second one. Finally they are state-of-the-art solutions. We quickly present their position in the literature in this section.

Walking Pattern Generator

The generation of a stable humanoid robot gait, on flat floor without obstacles, is a commonly tackled issue in robotics. Introduced by Kajita et al. [START_REF] Kajita | Real-time 3d walking pattern generation for a biped robot with telescopic legs[END_REF], the LIPM is widely used to approximate the non linear dynamics of a humanoid robot CoM during gait as a function of the CoP. Indeed, the CoP is a relevant variable to control as the CoP has to stay within the support polygon of the robot at all time to ensure the humanoid robot balance Wieber [START_REF] Wieber | On the stability of walking systems[END_REF].

In Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF], the authors present a CoP preview control which computes the CoM of the humanoid robot over a prediction horizon from imposed footsteps [START_REF] Naveau | A reactive walking pattern generator based on nonlinear model predictive control[END_REF], Giraud-Esclasse [START_REF] Giraud-Esclasse | Towards a reactive motion generation on exteroceptive feedback for generalized locomotion of humanoid robots[END_REF]. The global problem in the green rectangle (wholebody trajectory for the full preview future) is currently impossible to solve in milliseconds. However, it is possible to solve the blue box: the whole-body problem for the nearest future, as well as the linearized problem at the CoM level (orange box), over the full preview.

given to the controller. In Wieber [START_REF] Wieber | Trajectory free linear model predictive control for stable walking in the presence of strong perturbations[END_REF], a new formulation of this CoP preview control using a Model Predictive Control (MPC) scheme is proposed. Nowadays, this MPC approach is the most commonly adopted method to design a WPG for humanoid robots Faraji et al. [START_REF] Faraji | Versatile and robust 3d walking with a simulated humanoid robot (atlas): A model predictive control approach[END_REF], Griffin and Leonessa [67], Scianca et al. [START_REF] Scianca | Mpc for humanoid gait generation: Stability and feasibility[END_REF]. Moreover, the original MPC has been improved to achieve automatic footstep placements by Herdt et al. [START_REF] Herdt | Walking without thinking about it[END_REF]. Even non-linear reformulations of this MPC (thus a NMPC) were proposed in Naveau et al. [69], Caron and Kheddar [START_REF] Caron | Dynamic walking over rough terrains by nonlinear predictive control of the floating-base inverted pendulum[END_REF].

Using these formulations, it is possible to solve the centroidal problem over a preview horizon. Then, these given references are used in a whole-body controller to compute position, velocity or torque commands for the complete joint configuration for the instantaneous future. This approach is depicted in Fig. 2.5, the F F frame corresponds to the free-flyer frame, or the base frame, thus the under-actuated part of the robot where the centroidal dynamics acts.

The planning method used in this thesis to compute the reference trajectories can be detailed as follows and is presented in Fig. 2.6: First, the WPG uses a Non-linear Model Predictive Control (NMPC) Stasse et al. [START_REF] Stasse | Modular architecture for humanoid walking pattern prototyping and experiments[END_REF], [START_REF]jrl-walkgen[END_REF] to compute a reference trajectory for the CoM, the ZMP, the waist orientation and the feet of the robot. The implementation uses the centroidal dynamics proposed in Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF]. The NMPC is sub-sampled internally at 200Hz, which represents a new trajectory planned over the entire horizon every 5ms. Then, a dynamic filter Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF] is used in order to take into account the influence of the whole body on the CoM trajectory and to correct it. Indeed, the LIPM neglects the torque induced by the limbs ( L) over the movement of the CoM and in the case of TALOS (which weighs around 100 kg) it leads to too important errors to maintain a balanced walking. The filter is computed with the Recursive Newton-Euler Algorithm Featherstone [START_REF] Featherstone | Rigid Body Dynamics Algorithms[END_REF] implemented in the Pinocchio library Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF]. It modifies the CoM trajectory to take into account the momentum generated by the limbs motion.

Finally, this algorithm provides desired trajectories for the ZMP (z * or ZM P ref in the scheme), the CoM (c * or C ref in the scheme), the waist orientation (W in the scheme) and the feet (p * i or LF, RF in the scheme), for a given set of foot steps or a desired velocity. The desired DCM ξ * is deduced from the desired CoM and desired ZMP trajectories. A complete description of the WPG algorithm can be found in Scherrer [START_REF] Scherrer | Reactive walking for the humanoid robot pyrène[END_REF], Naveau et al. [START_REF] Naveau | A reactive walking pattern generator based on nonlinear model predictive control[END_REF].

The trajectories used in this thesis for straight walk simulations (Sections 4.4 and 5.4) have been computed using this method, with pre-defined set of foot steps. The complete framework of the WPG associated with control schemes is presented in Appendix 1, in the Fig. 6.4. A modified version of the WPG is used for the planning of human like trajectories to achieve human-robot collaboration in Section 4.5. The new WPG is based on the same method presented here and in Naveau et al. [START_REF] Naveau | A reactive walking pattern generator based on nonlinear model predictive control[END_REF], the main differences are detailed in 4.5.3.b.

Multicontact-locomotion-planning

The multicontact-locomotion-planning framework decomposes the global locomotion problem in several sub-problems solved sequentially, as shown in Fig. 2.7. This approach aims at driving the motion planning problem by injecting knowledge on [START_REF]multicontact-locomotion-planning[END_REF] the system.

The first stage aims at finding a guide trajectory from the starting position to the goal one; making sure that the robot is in contact with the environment and that the trunk is not in collision with the environment.

The second stage consists in finding contact stances for which there are quasistatic stable poses.

The third stage consists in finding a centroidal trajectory which is dynamically balanced for the chosen set of contacts.

The fourth stage consists in generating a whole body consistent trajectory.

Each transition from one stage to the next one assumes that the latter is able to realize the input. This decomposition breaks down the problem into lower dimensional problems. Each of them is solved in an efficient manner with real-time time answer. The framework gives the possibility to chose which algorithms is used to solve each sub-problem. A more precise description of this approach is provided in Carpentier et al. [START_REF] Carpentier | Multi-contact locomotion of legged robotsin complex environments -the loco3d project[END_REF], [START_REF]multicontact-locomotion-planning[END_REF].

In our case, we use the following solution: Given the initial and final poses of the robot, the reachability plan and a contacts sequence (stage one and two) are computed using the method SL1M (Sparse L1-norm Minimization) presented in Tonneau et al. [START_REF] Tonneau | SL1M: Sparse L1-norm Minimization for contact planning on uneven terrain[END_REF]. It is a convex relaxation of the mixed integer (MI) programming approach for planning contact sequences for legged robots. Then, the centroidal dynamics (see Section 2.2) is optimized using two convex relaxations based on trust regions Ponton et al. [START_REF] Ponton | On time optimisation of centroidal momentum dynamics[END_REF]. Similarly to the WPG dynamic filter method, it takes into account the momentum generated by the swing leg thanks to iterations between a kinematic whole-body formulation and the centroidal dynamic optimization. In contrast, when solving Eq. 2.16, it does not assume that L = 0. Finally, as in the WPG approach, the steps placements and CoM trajectory are given to a whole body controller in charge of generating the joints trajectories (or torques depending on the robot low-level control).

The trajectories used in the platforms and stairs simulations have been computed using this open-source framework multicontact-locomotion-planning [START_REF]multicontact-locomotion-planning[END_REF].

Real-time Whole Body Control

Introduction

The goal of this section is to report the existing solutions on real-time whole body controller. From a mathematical viewpoint, a real-time whole body controller aims to find a control vector u ∈ R n j for each motor of a robot (n j = 32 for TALOS, and n j = 10 for TIAGo), for a given set of tasks, at high frequency (for instance 1kHz). The controller is said to be instantaneous and corresponds to the blue block of Fig. 2.5.

In all the thesis, we assume that u can take only three values: u = v, i.e. the control problem is solved considering the free variables as being velocities. u = a, i.e. the control problem is solved considering the free variables as being accelerations. u = τ , i.e. the control problem is solved considering the free variables as being joint torques.

For the first and second case, the robot is position controlled and the low-level servo control transform the commanded joint positions into motor currents with a PID at high frequency (typically 2 kHz for TALOS, see Section 2.3.4). This means that the output of the optimization is integrated one or two times before being sent to the low level system of the robot. In general, for locomotion scenarios, those two controls give the most efficient walk, achieving the fastest results with less tracking errors along the trajectories Caron et al. [START_REF] Caron | Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control[END_REF].

Torque control, in another hand, requires a low level controller able to transform the desired joint torques into motor currents (see Sections 2.3.4,2.3). To perform this transformation, the low level controller needs the model of the robot's chain actuation. This model can be given by the robot's manufacturer, or the manufacturer can directly provide the low-level controller. However, this model often presents inaccuracies and the identification of the robot inertial parameters and the motor's chain drive is required to correct it. This issue is partially why torque control was less popular until recently, the transition from the simulations (which often do not model the actuators dynamics) to the real robot proving to be much harder than position control Mesesan et al. [START_REF] Mesesan | Dynamic walking on compliant and uneven terrain using dcm and passivity-based whole-body control[END_REF], Englsberger et al. [START_REF] Englsberger | Torque-based dynamic walking -a long way from simulation to experiment[END_REF].

Whole-body controller 2.5.2.a Introduction: Task Function

The most widely-used technique to generate whole-body motions is to design the motion in a space dedicated to the task, rather than directly at the whole-body level (i.e. in configuration space, see Section 2.1.3). Indeed, it is easier to design the reference motion in the task space, and then transcript this reference to the configuration space.

Once the planning has computed reference trajectories for the CoM and feet positions, they are given to the controller as desired features noted s * . The task functions Khatib [START_REF] Khatib | A unified approach for motion and force control of robot manipulators: The operational space formulation[END_REF], Samson et al. [START_REF] Samson | Robot control. the task-function approach[END_REF], Escande et al. [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF] are defined as errors e between the current features s and the desired ones and are often non-linear. For instance, the geometric relationship between the configuration space and the Euclidean space of the end effector (hands or feet) is non-linear. In particular, there can be multiple configurations possible to reach a desired end effector position.

Therefore, the task functions are usually computed as an optimization problem:

min q ||s -s * || (2.39)
with the relation between s and q defined by the differentiable function f such that:

f : C → S q → s (2.40)
with C the configuration space and S the task space (for example the Special Euclidian group SE( 3)).

This problem is solved iteratively using a gradient descent approach Chevallereau and Khalil. [START_REF] Chevallereau | A new method for the solution of the inverse kinematics of redundant robots[END_REF]. Once expressed as an optimization problem, it is interesting to see that this opens up an incremental approach where constraints can be added, and various formulations of equivalent problems can be considered.

To generate a complex motion on a humanoid robot, several tasks have to be combined, sequentially Mansard et al. [START_REF] Mansard | Using a memory of motion to efficiently warm-start a nonlinear predictive controller[END_REF] or simultaneously. The simultaneous execution of two tasks on a robot can be achieved in two ways: by setting respective weights between the tasks, or by imposing a strict hierarchy. Recently, a solution has been proposed to handle both strict and non-strict priorities of an arbitrary number of tasks Liu et al. [START_REF] Liu | Generalized hierarchical control[END_REF]. These first two solutions have been implemented in the Gepetto Stack-of-Task (SoT) software (see Section 1.6):

The library implementing the task formulation in velocity domain using a strict hierarchy as in Mansard et al. [START_REF] Mansard | A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: The stack of tasks[END_REF], Mansard and Chaumette [START_REF] Mansard | Task sequencing for sensor-based control[END_REF] is SoT-Core [20],

The library implementing the task formulation in torque using weights and inequalities is SoT-Torque-Control [79] (C++ wrapper of TSID [80] using the dynamic-graph [START_REF]dynamic-graph[END_REF]).

2.5.2.b QP Formulation: Task space velocity or acceleration control

Introduction

Generally these optimization problems involving multiple tasks and constraints are solved using Quadratic Programming. A QP is a convex optimization problem with a convex objective function (cost function) which is quadratic and constraint functions which are affine Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF]. A famous unconstrained QP is the Least Square Problem (LSP), for instance the Eq.2.39 is a LSP, which has an analytical solution (given by using pseud-inverse).

However, a QP with equality and inequality constraints is not easily solved. One method to solve these optimization problem is to use active set solvers Nocedal and Wright [START_REF] Nocedal | Numerical Optimization[END_REF]. These solvers use the Lagrangian multipliers formulation and the Karush-Kuhn-Tucker (KKT) conditions Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF], Nocedal and Wright [START_REF] Nocedal | Numerical Optimization[END_REF]. They project the free variables of the problem in the null space of the equality and inequality constraints. Then, the objective function is minimized in this subspace. Such solvers guarantee that the constraints are properly fulfilled, but might be computationally expensive, in particular when the matrix of the linear constraints is ill-conditioned.

Another way to cope with the constraints is to reformulate a new optimization problem with all the constraints as tasks inside the objective function. Therefore, the constraints might not be fulfilled. This new problem is mathematically equivalent to the one with constraints but it means that the related tasks might not be completely performed Hofmann et al. [START_REF] Hofmann | Exploiting angular momentum to enhance bipedal center-of-mass control[END_REF]. This can be helpful for tasks that are subject to the environment disturbances.

Therefore, an important question is to decide which constraints need to strictly be respected (equality constraints), and which constraints can be relaxed (cost function). In general the constraints related to the physical laws are strictly enforced because any violation would induce damages to the robot. Constraints related to tasks such as the position of the right hand might be performed at best. It is important to notice that some tasks, such as foot contact with the ground, while being subject to modifications from the environment, may have a strong impact on the robot if they are not fulfilled correctly.

Mathematical formulation for velocity control

Let us simplify the motion equation based on the rigid body dynamics (see Eq.2.4) when there is no contacts:

M a + h = N T τ (2.41) with h = C(q, v)v + g(q).
If we have a task e regulating a feature s, the dynamics of the task can be imposed, for instance as an exponential decay (last equation):

e = s s * ė = ṡ -ṡ * ė = -λe = -λ(s -s * ) (2.42)
with the difference operator in the task space, for instance the operator of the Lie group. λ the control gain of the task, which can be constant, or adaptive in order to control the speed of convergence to the desired feature. The task is realized when ė reaches 0 or is close to 0.

Using the previous function f such that s = f (q) one can define the relationship between ṡ and v using the Jacobian of s Lynch and Park [35] and the formulation given in Eq.2.6 for the under-actuated part:

ṡ = [J u J a ]v = Jv (2.43)
One can notice that the Jacobian for the task error e is the same one as its feature s, because s * depends only on the time and not on the configuration. It is thus possible to compute v with Eq.2.43 as:

v = J † ṡ (2.44)
with J † the Moore-Penrose pseudo-inverse of J, since J can be non square and thus not invertible.

It is possible to reformulate Eq.2.42 to obtain the dynamics of the feature and then the desired joint velocities:

ṡ = ṡ * + ė ⇔ Jv = ṡ * -λ(s s * ) ⇔ v = J † ( ṡ * -λ(s s * )) (2.45)
One can introduce a slack variable w (an implicit optimization variable) to express the realization of the task, then the solver will optimize this variable instead of the error directly, allowing a relaxation of the problem if no solution is found (from Eq.2.42):

w = ė + λe w = Jv -( ṡ * -λ(s -s * )) (2.46)
A simple formulation of the QP problem can then be expressed as:

min v,τ w 2 s.t. M a + h = N T τ (2.47)
Where the free variables are the robot velocity v and the torque τ .

Mathematical formulation for acceleration/torque control

In this thesis, we also used the formulation of the task errors at the second order to achieve acceleration control. In Eq.2.42 one can define the dynamics of the second derivative of the error ë (third equation) Lynch and Park [35]:

ë = s -s * s = Ja + Jv ë = -2λ ė -λ 2 e (2.

48)

And thus, similarly as before one can obtain the desired joint acceleration and the slack variable w as:

a = J † (s * -Jv -2λ ė -λ 2 e) w = ë + 2λ ė + λ 2 e w = Ja + Jv -(s * -2λ ė + λ 2 e) (2.49)
Leading to the following QP:

min a,τ w 2 s.t. M a + h = N T τ (2.50)
Where the free variables are the accelerations a and the torque τ . It is possible to work only on one type of variables (to achieve acceleration or torque control on the robot) by substitution.

Formulation with contacts

If the system is in contact with the environment, its dynamics must account for contact forces f ext . If contacts are soft, measured/estimated contact forces fext can be easily included: min

a,τ w 2 s.t. M a + h = N T τ + J T c fext (2.51)
with J c the Jacobian of the contact points x c according to the robot state vector, as defined in Eq.2.6.

But if contacts are rigid, they constrain the motion. They can be implemented as nonlinear functions, which are differentiated twice:

x c = 0 ⇔ Contact point does not move J T c v = 0 ⇔ Contact point velocity is zero J T c a + Jc v = 0 ⇔ Contact point acceleration is zero (2.52)
This representation leads to the following optimization problem:

min a,f,τ w 2 s.t.   J c 0 0 M -J T c -S T     a f τ   =   -Jc v -h   (2.53)
These optimization problems represent Equality-Constrained Least-Square Problem (ECLSP) because they only have equality constraints. Unconstrained LSP can be directly solved using the Moore-Penrose pseudo-inverse Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF] (if J is surjective). For instance, for a fully actuated robot (q = q j , v = qj , a = qj ), if the desired feature does not move (s * = 0) and using Eq.2.49 one can find the solution: qj = -J † 2λ ė + λ 2 e + J qj (2.54)

The ECLSP can also be resolved with pseudo-inverse by using null-space projector Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF]. But the main benefit of QP solver is that they can handle inequality-constraints. It is mainly used to define boundaries of the system such as torque, velocity or joint limits; and also friction cones for the contacts:

τ min ≤ τ ≤ τ max q min j ≤ q j ≤ q max j qmin j ≤ qj ≤ qmax j f min ≤ f ≤ f max (2.55)

2.5.2.c HQP and Weighted Sum

Complex robots are typically redundant with respect to the main task they must perform, this redundancy can be used to execute secondary tasks. This multiobjective optimization can be achieved by setting respective weights between the tasks, or by imposing a strict hierarchy between them (cascade of QP or HQP) Kanoun et al. [START_REF] Kanoun | Kinematic control of redundant manipulators: Generalizing the task-priority framework to inequality task[END_REF].

We assume that the robot must perform N tasks, each defined by a task function and its optimization variable w i :

g i = w i 2 (2.56)
In the following we chose to use the formulation of Eq.2.49 for w i .

Weighted Sum

The easiest strategy is to sum all functions using user-defined weights µ i :

min a,f,τ N i=0 µ i g i s.t.   J c 0 0 M -J T c -S T     a f τ   =   -Jc v -h   τ min ≤ τ ≤ τ max q min j ≤ q j ≤ q max j qmin j ≤ qj ≤ qmax j f min ≤ f ≤ f max (2.57)
This problem remains standard computationally-efficient (compared to LSP). But, finding proper weights is hard and too extreme weights can lead to numerical issues. It is easier to find the priority of a task than its weight, but a cascade of QP can quickly become too computationally expensive if computed in a naive way. For this reason several teams working on humanoid robots are using this formulation for their controllers:

In Koolen et al. [START_REF] Koolen | Design of a momentum-based control framework and application to the humanoid robot atlas[END_REF], the IHMC whole body controller is described. The tracking of the feet position and the foot contact wrenches tasks are considered as strict equalities, whereas the momentum rate is tracked at best inside the cost function.

In Bouyarmane et al. [START_REF] Bouyarmane | Multi-robot and task-space force control with quadraticprogramming[END_REF], the JRL/IDH team is using strict inequalities to maintain the physical constraints: joint limits, velocity limit, acceleration limit, and strict equality to maintain the dynamical consistency. All the other tasks are tracked at best in the cost function.

The weighted approach tends to imply new gains when considering new applications. Finding these new gains is not straightforward and often require timeconsuming tuning and expertise. However some procedures exist to automatically tuned these gains, such as in Pucci et al. [START_REF] Pucci | Automatic gain tuning of a momentum based balancing controller for humanoid robots[END_REF] for a momentum based controller or in Teshnehlab and Watanabe [START_REF] Teshnehlab | Self tuning of computed torque gains by using neural networks with flexible structures[END_REF] by using learning approach. Penco et al. [START_REF] Penco | Learning robust task priorities and gains for control of redundant robots[END_REF] proposes a multi-objective optimization approach to learn the gains and also the priorities (strict and soft) of a controller.

Lexicographical Hierarchy

The alternative strategy is to use strict priorities to order the task functions, which means to solve a cascade of QP problems:

g * i = min a,f,τ g i s.t.   J c 0 0 M -J T c -S T     a f τ   =   -Jc v -h   g j = g * j ∀j < i τ min ≤ τ ≤ τ max q min j ≤ q j ≤ q max j qmin j ≤ qj ≤ qmax j f min ≤ f ≤ f max (2.58)
The following works have investigated the use of strict inequalities on humanoid robots:

In Hoffman et al. [START_REF] Hoffman | Multi-priority cartesian impedance control based on quadratic programming optimization[END_REF], the IIT team of WALKMAN presents a scheme to perform Cartesian Impedance Control in a hierarchical manner. The new task at level i is performed at best, while preserving the integrity of previous tasks. The dynamics is preserved through its formulation as a task equality.

In Henze et al. [START_REF] Henze | An approach to combine balancing with hierarchical whole-body control for legged humanoid robots[END_REF], the DLR team describes a whole body controller to perform contact control. In this work, the authors use 4 levels of hierarchy. The first level is used to maintain contact with the environment and to make sure that the planned external wrench is applied to the robot. The second level contains the tracking tasks of the end-effectors not used for the balancing. The third level is used to control the CoM and the fourth level is used to control the posture. This controller is moreover passive (see Section 2.6.3)

To illustrate this last solution with inequalities, let us take two tasks with priority order e 1 ≺ e 2 and bounds e 1 , e 1 and e 2 , e 2 respectively. When there is no contacts, the expression of the Lexicographical QP applied on inverse kinematics (see the previous velocity formulation of Section 2.5.2.b: ė -Jv = w) can be written as Saab et al. [START_REF] Saab | Dynamic whole-body motion generation under rigid contacts and other unilateral constraints[END_REF]:

H 1 = min v,w 1 w 1 2 s.t. ė1 * ≤ J 1 v + w 1 ≤ ė1 * H 2 = min v,w 2 w 2 2 s.t. ė1 * ≤ J 1 v + w * 1 ≤ ė1 * ė2 * ≤ J 2 v + w 2 ≤ ė2 * with w * 1 = arg min w 1 H 1 (2.59)
A naive implementation would be to solve the second problem H 2 in the same space than H 1 , i.e. v ∈ R n . However by working in the base of the null space of H 1 , the computation becomes extremely efficient Nocedal and Wright [START_REF] Nocedal | Numerical Optimization[END_REF]. In Section 4.4 is presented a hierarchical controller which exploit this specific structure to keep its control frequency higher than 1kHz in average with 4 hierarchy levels (see Section 4.4.4). Note however that when the tasks are compatible the projection in null space is useless. In addition, when a constraint is activated it means that the robot is working at one of its limit which is rarely desirable. The main interest of strict inequalities is the guarantee that a constraint is never violated and that a task of lower priority will not interfere.

Once the QP solver find a solution (using for example the active sets method Nocedal and Wright [START_REF] Nocedal | Numerical Optimization[END_REF]), the desired joint torques/accelerations or velocities are sent to the robot. However, most robots such as TALOS or HRP-2 ultimately controls their motor in current (electric motors). Thus the low-level controller must transform these commands into motor in current, this can be done by two general approaches: position control or torque control (see Section 2.3.4).

Recent Humanoid Robots Whole Body Controllers

In this section, a short presentation of recent humanoid robots and their control schemes is given. A review on the hardware design of humanoid robots can be found in Ficht and Behnke [START_REF] Ficht | Bipedal humanoid hardware design: A technology review[END_REF]. The aim is to highlight the particularities of the robot TALOS and investigate the promising control schemes compatible for our robot. PAL Robotics demonstrated, during IROS 2018, an accurate whole-body balancing control on TALOS PAL-Robotics [START_REF]Talos -torque controlled balancing[END_REF], however it is only position controlled.

In the Table .2.1 are presented some specifications of 6 humanoid robots and a biped. The robot Gazelle is presented in this section because it can be entirely torque controlled and achieves great results while walking on uneven terrain because of its hardware composition and control Jeong et al. [START_REF] Jeong | Design and control of the rapid legged platform gazelle[END_REF]. One has to notice that the TORO robot has more DoFs than the other robots because its 39 DoFs include its hands (6 DoFs each). The only robot presented here which does not have electric actuation is the Atlas robot, with hydraulic actuation using valve current (however other hydraulic robots exist such as HYDROïD from LISV and Hydra from the AIST). The HRP-4 robot has no torque sensors to achieve torque control but may be controlled in current.

The robot WALK-MAN (see Fig. 2.8) was first presented by the IIT in 2015 and has then been updated to reach the presented final version. The humanoid robot has been design to operate in damaged buildings and hostile environment Tsagarakis et al. [START_REF] Tsagarakis | WALK-MAN: A highperformance humanoid platform for realistic environments[END_REF]. It can be torque controlled. Recently a two strict hierarchy whole-body controller has been implemented, using energy shaping Subburaman et al. [START_REF] Subburaman | Online falling-over control of humanoids exploiting energy shaping and distribution methods[END_REF] to control a falling-over scenario. It is an interesting strategy because it allows to control the energy of the system, as energy shaping is an approach close to the one of the passivity Folkertsma and Stramigioli [96]. TORO (see Fig. 2.8) was made specifically for torque control Englsberger et al. [START_REF] Englsberger | Overview of the torque-controlled humanoid robot toro[END_REF] by the DLR, it was first presented in 2013 but the project begun in 2010. This humanoid robot is robust to walking, balancing and multi-contact scenarios, Henze et al. [START_REF] Henze | An approach to combine balancing with hierarchical whole-body control for legged humanoid robots[END_REF] presents a passivity-based whole body controller to perform multicontact balancing. This work has led this thesis to investigate the passivity theory to ensure the stability of the whole-body controllers. Englsberger et al. [START_REF] Englsberger | Torque-based dynamic walking -a long way from simulation to experiment[END_REF] presents a weighted whole-body controllers with DCM control on TORO, and Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF] describes an implementation proving the Lyapunov and passivity stability even for conflicting sub-tasks.

The HRP-4 robot (see Fig. 2.8) is the follow up of the HRP-2 robot designed by the Kawada Industries and AIST. It is a light-weighted robot which is designed to collaborate with humans, and able to react to external forces. The recent papers are from the common laboratory of the CNRS-AIST JRL realized at the LIRMM. In Caron et al. [START_REF] Caron | Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control[END_REF] the HRP-4 robot succeed stairs climbing in Airbus Plant, using a whole-body admittance controller with strict hierarchy. The CoM admittance control used in this paper is similar to what is described in Section 2.2.3.b. The stabilization of the control is based on the LIPM and feet wrench distributions. The robot is controlled in position, while the QP is solved for accelerations free variables, thus the command is integrated two times. In Bouyarmane et al. [START_REF] Bouyarmane | Multi-robot and task-space force control with quadraticprogramming[END_REF] is presented a weighted whole body controller achieving task force control with multi-robot. It also uses admittance control and the robot is controlled in position.

TORO and HRP-4 have been used in Kheddar et al. [START_REF] Kheddar | Humanoid robots in aircraft manufacturing[END_REF] to achieve some manufacturing tasks in an assembly line. The results of a complete solution embedding visual tracking, localization, planning and control is impressive. One can notice that HRP-4 is used for tasks which do not require force application and have limited contact. When multi-contact is needed or to realize the manufacturing applications, the torque controlled robots TORO and HRP-2Kai are used. Compared to what is achieved, we want to answer the first limiting factor encounter in this study, which is the security of the robotic platform. The repeated falls and weakness of the humanoid robots lead to damages during the experiments. Thus, we want to ensure the stability of our solution while conserving the robot capabilities to achieve demanding tasks (such as applying huge torque for drilling).

The robot JAXON (see Fig. 2.9) from the JSK laboratory is also a humanoid robot designed for torque control, to perform disaster relief assistance. In Shirai et al. [START_REF] Shirai | Design and evaluation of torque based bipedal walking control system that prevent fall over by impulsive disturbance[END_REF] is presented a whole body torque controller which can absorb impulsive disturbances. Associated with an online walking controller, the framework demonstrates high robustness to strong collision with the robot's legs. This robot is however the heaviest.

The robot Atlas (see Fig. 2.9) from Boston Dynamics and the MIT is a humanoid robot using state-of-the-art hardware which demonstrates human-level agility. However, the hardware specifications of this robot are not disclosed. In Koolen et al. [START_REF] Koolen | Design of a momentum-based control framework and application to the humanoid robot atlas[END_REF] is detailed a momentum based whole body torque control, using a soft hierarchy. This formulation controls the AM of the robot and achieve impressive results with the robot while walking on different terrains (tilted cylinder blocks, rubble) and fast on flat one (0.43m/s). These results confirmed our interest to add a control on the AM of the robot for our control schemes.

Finally, the robot GAZELLE (see Fig. 2.9) was recently designed by the KAIST, aiming to build a fast and reliable biped platform for walking experiments. As said before, the paper Jeong et al. [START_REF] Jeong | Design and control of the rapid legged platform gazelle[END_REF] presents the results obtained with this new platform. The controller split the control between a CoM task and ankles force tasks. It controls the damping of the CoM and then solves the Inverse Kinematics to obtain the desired joint configuration; while using the F/T sensors to regulate the torques on the ankles and send directly a torque command for them. This architecture allows the biped robot to walk fast on uneven terrain and to be robust to external pushes.

Compared to the robots in the literature, TALOS has average capabilities, however, it is the only robot commercially available (and not too expensive). It is possible to implement torque or position control, and it has been built to be able to sustain heavy load for a long duration and to apply high torques/forces to complete manufacturing operations.

Stability Analysis: Convergence toward an equilibrium 2.6.1 Introduction

Humanoid robots are meant to act in the same environment than humans, interacting with it, but this environment can be unknown and subject to disturbances. Even performing simple tasks in fixed environment can lead to instabilities, the authors encountered this problem while testing a classical torque control scheme using inverse dynamics on the robot TALOS on a simple postural task. After some repetitions of a sinusoidal motion on the robot arm, the system diverged brutally and blocked some of its harmonic drives. The gains tuned in simulation (simulating the rigid chain actuators) were too high for the real robot. Thus, even with gains tuned on a proper simulator modeling the actuators, we cannot expect the solution to remain stable (i.e. remains in a bounded distance from an equilibrium). To provide a safe and reliable interaction with the environment and possibly humans, it is necessary to ensure the stability of the robotic system. This problem is commonly tackled in the community using two strategies: the Lyapunov analysis or the passivity-based analysis. In this section, we first present briefly the Lyapunov theory and recent works proving the Lyapunov stability of humanoid robots whole-body controllers. Then, the passivity theory is described with more details as this analysis is used in the Chapter 5 to prove the stability of our new controller.

Lyapunov Theory

The Lyapunov theory consists in analysing the stability of an equilibrium, looking if the solution remains in a bounded distance from the equilibrium or converges asymptotically (Asymptotic stability) or exponentially (Exponential stability) Bacciotti and Rosier [START_REF] Bacciotti | Liapunov Functions and Stability in Control Theory[END_REF]. This analysis ensures that the controller will not find a diverging solution. We have the following definitions for local and global strict asymptotic stability Bacciotti and Rosier [START_REF] Bacciotti | Liapunov Functions and Stability in Control Theory[END_REF]:

Definition 1 (Lyapunov local asymptotic stability). For a system with the state space model of ẋ = f (x) (x ∈ R n ) and an equilibrium state x * = 0. If there exists a Lyapunov function V : R n → R such that: V is continuous, has continuous first derivatives, is locally positive definite and its time derivative is locally negative definite, i.e:

                   V (0) = 0 V (x) ∈ C 1 on B V (x) > 0 ∀x ∈ B\{0} V (x) = ∇V (x).f (x) V (x) < 0 ∀x ∈ B\{0} (2.60)
where B is the local neighborhood of the origin. Then the equilibrium is proven to be locally asymptotically stable (local Lyapunov stability).

Definition 2 (Lyapunov global asymptotic stability). For a system with the state space model of ẋ = f (x) (x ∈ R n ) and an equilibrium state x * = 0. If there exists a Lyapunov function V respecting the local Lyapunov stability (Definition 1), with B = R n and radially unbounded then the equilibrium is proven to be globally asymptotically stable (global Lyapunov stability).

We recall, V radially unbounded if:

x → ∞ =⇒ V (x) → ∞
From these definitions, one can obtain the exponential stability by majorating V by an exponentially converging function.

In the control literature, Lyapunov functions are often built based on the relative kinetic and potential energy of the system to prove its stability. Recently, Nava et al. [START_REF] Nava | Stability analysis and design of momentum-based controllers for humanoid robots[END_REF] proposed a momentum-based QP with two strict hierarchies and proved its Lyapunov stability by using integral gains in the task functions. In a similar way, Cisneros et al. [START_REF] Cisneros | Robust humanoid control using a qp solver with integral gains[END_REF] adds an integral term to the computed torque while using it in the constraints of their QP to prove the Lyapunov stability. Finally, Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF] has designed a new controller creating separate Lyapunov functions for each task and proving the overall stability and passivity of the system.

Passivity Theory

The passivity-based analysis consists in investigating the energy flows within a system Schaft [START_REF] Schaft | L2-Gain and Passivity in Nonlinear Control[END_REF]. It is also possible to directly control the system energy without proving the passivity, which requires an analysis of the energy derivative of the system. For instance, Joseph et al. [START_REF] Joseph | Towards x-ray medical imag-ing with robots in the open: safety without compromising performances[END_REF] proposes a formulation of a kinetic energy constraint in a QP to prevent a manipulator robot from transferring dangerous amounts of this energy during an impact. Yet, it has been proven in Camlibel et al. [START_REF] Camlibel | Mathematical Control Theory I: Nonlinear and Hybrid Control Systems[END_REF] that when a robot interacts with its environment, a passively-controlled robot is a necessary condition for stability Folkertsma and Stramigioli [96]. Using the derivative of the energy in the system, the passivity can be analyzed to ensure that the system converges to the desired equilibrium or to detect which components have an unbounded energy consumption.

2.6.3.a Definition

The principle of passivity comes from the concept of dissipative dynamical system Willems [START_REF] Willems | Dissipative dynamical systems part I: General theory[END_REF], in particular proving that the internal power of the system is less than or equal to the power transferred to the system through its port. This concept is associated with the Port-Hamiltonian System theory; which is an energy-based formulation of the physical systems Schaft [START_REF] Schaft | L2-Gain and Passivity in Nonlinear Control[END_REF].

The Hamiltonian H is the sum of the potential and kinetic energy of the system. Using the Poisson framework representation (input-state-output representation), one can obtain the following formulation:

For a state vector x ∈ R n , u, y ∈ R m : ẋ = J(x) -R(x) ∂H ∂x (x) + g(x)u y = g T (x) ∂H ∂x (x) (2.61) with J(x) = -J T (x), R(x) = R T (x) ≥ 0.
g represents the interconnection, and therefore effect, of the port variables on the state variables and vice versa. The matrix J is a power-continuous interconnection and is skew-symmetric, whereas R models pure resistive losses of the system. The latter property is obtain by taking the time derivative of the Hamiltonian:

Ḣ = ∂H T ∂x (x) ẋ = ∂H T ∂x (x) J(x) -R(x) ∂H ∂x (x) + ∂H T ∂x (x)g(x)u = - ∂H T ∂x (x)R(x) ∂H ∂x (x) + y T u (2.62)
using the fact that J is skew-symmetric ( ∂H T ∂x (x)J(x) ∂H ∂x (x) = 0). Thus, Ḣ is the power supplied through the port y T u, minus the power lost to friction, quadratic on R(x) Schaft [START_REF] Schaft | L2-Gain and Passivity in Nonlinear Control[END_REF].

Using the dissipative definition, a system is defined as a passive system Schaft [START_REF] Schaft | L2-Gain and Passivity in Nonlinear Control[END_REF] if it respects the following definition: Definition 3 (Passive System). A system with the state space model of ẋ = f (x, y) with initial state x(0) = x 0 ∈ R n , input vector u ∈ R m and output y = h(x, u) is said to be passive, if there exists a positive semi-definite function H : R n → R + , called storage function, such that:

H(x(T )) -H(x(0)) ≤ T 0 y T (t)u(t)dt (2.63) ∀y : [0, T ] → R m , x 0 ∈ R n and T > 0.
One can see that by choosing H = H, the storage function is the energy in the system and the supply rate y T u is the power transferred to the system through its port. If R = 0, there is no dissipation and the system is conservative.

From the Definition. 3, the passivity of the system can equivalently be proven by finding an appropriate storage function H(x) respecting the property 1:

Property 1 (Passivity Storage Function). A system with the state space model of

ẋ = f (x, y) with initial state x(0) = x 0 ∈ R n , input vector u ∈ R m and output
y = h(x, u
) is said to be passive, if there exists a positive semi-definite function H : R n → R + , called storage function, such that:

Ḣ(x) ≤ y T (t)u(t) ∀(x, y) ∈ R n (2.64)
This is why the storage functions are defined as potential and kinetic energy in the literature (for Lyapunov and Passivity analysis). Using the equation of the robot dynamics Eq.2.4, on can define a controller sending torque commands τ to the robot. From a Port-Hamiltonian System point of view, the controller is thus interconnected to the power port of the robot {v, τ }: the robot receives an input τ and outputs a velocity v onto the environment (see the Fig. 2.10 for a simple scheme example). The following section describes a simple analysis of the interconnected systems of a robotic system.

2.6.3.b Analysis of a robotic system

A robotic system can be decomposed in three components: the robot, the controller and the environment. It is often assumed that the environment of the robot can be described by a passive mapping (v → -τ ext ), in connection with impedance control Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF], Ott et al. [START_REF] Ott | On the passivity-based impedance control of flexible joint robots[END_REF] (green block in the block in the Fig. 2.10, see Section 2.7.3). Then, because the group robot-environment is passive, to prove the overall passivity of the system, one has to prove that the group controller-robot is passive (blue block in the Fig. 2.10, with τ or q as control variable). Indeed, the interconnection of passive systems leads to a general passive system Folkertsma and Stramigioli [96]. A second solution is to directly prove that the system {robot+controller} is passive with respect to the environment (red block in the Fig. 2.10).

2.6.3.c Energy Tank

The energy tanks are artificial energy storing elements that keep track of the energy naturally dissipated by the system agents Giordano et al. [START_REF] Robuffo | A passivity-based decentralized strategy for generalized connectivity maintenance[END_REF]. The concept of virtual energy tank is to give an energy budget to the controller and to change the behavior of the controller in function of it. The tank monitors the energy flow of the system to guarantee the passivity. The energy stored in this tank can be re-used to accomplish different goals without violating the passivity of the system. By augmenting one control architecture with a virtual energy tank Folkertsma and Stramigioli [96], it is possible to limit the energy generated by the system and then make it passive. For instance, in the case of a controller with multiple tasks, one can introduce an energy tank in the framework to store the energy dissipated by these tasks. The tank will then be used to transfer the stored energy when a task needs more energy to be completed. On the contrary, if the tank is empty, it will be used to deteriorate the tracking of this task to ensure the passivity of the system Folkertsma and Stramigioli [96]. For instance, if a robot lifts a mass its actuators consume potential energy (the energy flows through them) thus this energy is removed from the tank. In contrary, if someone pulls the mass down while the robot lifts it, the actuators extract energy from the system and the energy flows back to the tank. When the tank is empty, the controller is forbidden to inject energy into the system: the robot stops lifting the mass and the system is guaranteed to be passive Folkertsma and Stramigioli [96]. The energy tank is commonly defined as a virtual storage element with flow variable ṡ and effort variable s such that:

E tank = 1 2 s 2 (2.65)
The flow variable ṡ is often defined by the damping energy of the tasks Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF], Dietrich et al. [START_REF] Dietrich | Passivation of projection-based null space compliance control via energy tanks[END_REF][START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF] and then integrated to find the energy value in the tank. It is possible to create a global tank monitoring the energy of all the tasks, or one tank for each task Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF]. According to the value of the tank, the tracking of a task (or all tasks in the case of a global tank) is deteriorated to ensure the passivity of the system. This can be achieved by different ways:

By using a Dirac structure to link the flow, effort and control variables Dietrich et al. [START_REF] Dietrich | Passivation of projection-based null space compliance control via energy tanks[END_REF][START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF] (this method is not used in our case because of the complexity of the structure that will be involved)

By modifying the dynamics of the controller states according to the energy in the tank Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF] By computing coefficients based on the tank value to directly regulate the tasks Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF], Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF].

An elegant way of using the first method on manipulators has been presented for multi-tasks in a strict-hierarchy using null-space projection in Dietrich et al. [START_REF] Dietrich | Passivation of projection-based null space compliance control via energy tanks[END_REF]. Moreover, regulating the size and flow of the energy tank are challenges that have been expressed in the domain Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF], Tadele et al. [START_REF] Shiferaw Tadele | Combining energy and power based safety metrics in controller design for domestic robots[END_REF]. Providing too much energy to the system may lead the controller to diverge from the equilibrium or to over-accumulate this energy. Likewise, an uncontrolled sharp release of the tank energy might lead to dangerous motions Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF].

2.6.3.d State-of-the-art

For manipulator robots with flexible joints, Ott et al. [START_REF] Ott | On the passivity-based impedance control of flexible joint robots[END_REF], Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF] present a passive framework using impedance control. Another method, successfully implemented in Giusti et al. [START_REF] Giusti | On the combined inverse-dynamics/passivity-based control of elasticjoint robots[END_REF], couples Inverse Dynamics (ID) and passivitybased control by modifying the ID formulation to exploit the passivity-properties of the robot model. However, using these approaches on humanoid robots is not straightforward. Indeed, humanoids robots are not ground-fixed to the environment, they are underactuated systems. This under-actuated part, called floating base, prevents from using the full feedback linearization of the underlying system Acosta and Lopez-Martinez [START_REF] Acosta | Constructive feedback linearization of underactuated mechanical systems with 2-dof[END_REF]. Moreover, it relies on external contact forces to control its motion. Besides, the penalization of the tracking tasks used in Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF], Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF] can compromise the balance of the robot. Thus, passivity-based control on humanoid robots is still an open-problem. In Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF], the method of Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF] has been extended for a fully actuated humanoid robot for limit cycle control. Thus, this solution does not guarantee the passivity when constraints are activated in the QP due to the under-actuation of the humanoid robot.

The passivity method using impedance control has been first implemented on humanoid robots by Cheng et al. [START_REF] Cheng | Cb: A humanoid research platform for exploring neuroscience[END_REF] and has lately been extended on the DLR robot TORO, leading to impressive results in multi-contact scenarios in Henze et al. [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF]. Finally, Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF] has designed a new controller respecting the Lyapunov stability and proving the passivity of the system even for conflicting subtasks, for fully actuated robots. This controller covers the complete range between Inverse Dynamics (with non-strict hierarchy) and Proportional Derivative with feedforward (PD+) Control. However, as in Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF], for under-actuated robot, if a constraint becomes active, the passivity is no longer guaranteed. Nonetheless, it achieves great results with the humanoid robot TORO.

Force Control for Manipulation

Introduction

In order to perform an operation involving interactions of the robot arm with the environment it is needed to implement a specific control. Force control consists in using the force information given by the robot sensors in the controller. It allows to increase the robot adaptability to uncertain environments and to monitor the contact forces. Thus, force feedback can be used to enforce the safety of the control, avoiding huge contact forces which may lead to damages. This is particularly important for manufacturing and assembly tasks such as drilling or deburring where the amount of force required to perform the operation is important. Moreover, the controller used for these operations must also control the position of the arm or mechanical tool used by the robot. Indeed, the manipulation requires contact with the environment at precise position. Thus, the motion of the robot arm is constrained by this contact, constraints represented by the interaction forces.

In this section are presented the most common force control schemes present in the literature. These schemes have been mostly developed for manipulator robots and then adapted to robots with more DoFs. In particular, for humanoid robots, the solutions presented can be used on the end-effectors of the robot, for instance as specific tasks in the whole-body controller.

In the case of the robot TALOS, the forces are measured by the force sensors connected to the wrists and ankles of the robot. These force sensors can accurately provide forces along six DoFs. This vector F m consists of the three orthogonal force components and the three orthogonal torque components operating at the endeffector with respect to the fixed Cartesian coordinate system (named the world W ). Similarly, the position of the end-effector x in the constraint or task space is a six DoFs vector with three orthogonal displacements (distance between the endeffector and the world) and three angles (orientation of the end-effector with respect to the world). The Denavit-Hartenberg Bejczy [START_REF] Antal | Robot arm dynamics and control[END_REF] representation of the joint angles are set in the variable q j and the generalized joint torques are given by τ .

To achieve manipulation operations with both position and force requirements the researchers have implemented solutions which create a dynamic relationship between position and force variables in addition to the control of one variable. Indeed, it is not possible to simultaneously impose force and position values on the environment. The two most common solutions are dual, one performing a direct closed-loop force control, the other performing an open-loop force control, and are the following Siciliano et al. [2]:

The admittance control: Commands a force reference and builds a relationship between the end-effector motion and the error from the reference force Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF], Keemink et al. [START_REF] Arvid | Admittance control for physical human-robot interaction[END_REF], Yu and Perrusquía [START_REF] Yu | Simplified stable admittance control using end-effector orientations[END_REF]. This solution allows to control the contact force to a desired value, with an openloop position control (see Fig. 2

.11).

The impedance control: Commands a position reference and builds a relationship between the external forces and the error from the reference position Lu and Meng [START_REF] Lu | Impedance control with adaptation for robotic manipulations[END_REF], Anderson and Spong [START_REF] Anderson | Hybrid impedance control of robotic manipulators[END_REF], Albu-Schaffer and Hirzinger [START_REF] Albu-Schaffer | Cartesian impedance control techniques for torque controlled light-weight robots[END_REF], Schindlbeck and Haddadin [START_REF] Schindlbeck | Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks[END_REF]. This solution allows to control the position of the end-effector to a desired value, with an open-loop force control (see Fig. 2.12). The resulting impedance can be linear or nonlinear (if there is a force feedback).

These two control strategies are complementary, they implement opposed solutions to achieve the same goal and thus behave well in different situations: the two controllers exhibit opposite stability characteristics depending on the stiffness of the environment. The admittance control is better suited for interactions with soft environments or operations in free space (see Section 2.7.2) and the impedance control is better suited for dynamic interactions with stiff environments (see Section 2.7.3).

Both methods aim to implement a task-space relationship following a mechanical impedance Siciliano et al. [2]:

∆x = x m -x * F ext = M x ∆ẍ + D x ∆ ẋ + K x ∆x (2.66)
where x * , x m are the desired and measured positions of the end-effector, F ext the aimed external force measured at the end-effector, M x is a mass matrix, D x a damping matrix and K x a stiffness matrix. Because the measurement of the acceleration ẍm can be noisy, the mass term is often neglected and set to 0.

Others strategies have been implemented to combine the advantages of the admittance and impedance control or of the position and force control. The first one is called the hybrid position/force control Anderson and Spong [START_REF] Anderson | Hybrid impedance control of robotic manipulators[END_REF], Raibert and Craig [START_REF] Raibert | Hybrid Position/Force Control of Manipulators[END_REF], and more recently the hybrid admittance-impedance control Ott et al. [START_REF] Ott | A hybrid system framework for unified impedance and admittance control[END_REF], Kim et al. [START_REF] Kim | Weighted hybrid admittance-impedance control with human intention based stiffness estimation for human-robot interaction[END_REF] (see Section 2.7.5). The second one is the parallel force/ position control Chiaverini and Sciavicco [START_REF] Chiaverini | The parallel approach to force/position control of robotic manipulators[END_REF], Siciliano [START_REF] Siciliano | Parallel force/position control of robot manipulators[END_REF] described in Section 2.7.6.

Admittance Control

In the admittance control approach the robot is position controlled. The environment is assumed to behave as an impedance system, with elastic and dissipative components while the robot behaves as an admittance system. This method uses closed-loop force control and open-loop position control. It presents interesting properties when creating contact with compliant surfaces (direct control on the force and little use of position control) or when the environment is used to guide the end-effector. A simple control scheme is presented in Fig. 2.11. In this scheme the mass term of Eq.2.66 is neglected and the desired force F * to apply is related to a reference position

x ref = K c (F * -F m )
where K c is the compliance (or stiffness) matrix Salisbury [START_REF] Salisbury | Active stiffness control of a manipulator in cartesian coordinates[END_REF]. Thus, Eq.2.66 is simplified in Siciliano et al. [2]:

K x K c (F * -F m ) = D x ∆ ẋ + K x ∆x (2.67)
And the close-loop can be done by a PD+ on the end-effector velocity (see Eq. 2.68). The PD+ control can be of different forms, it can take an inner close-loop in position control with the term proportional on the position error ∆x = x * -x m (modified admittance control) or not.

In the example given in the Fig. 2.11, we set D x = I and choose the gains of the PD+ to be K x K c = D adm , K x = P adm . Thus we obtain the following equations and notations:

e f = F * -F m ẋ * = ẋm + D adm e f -P adm ∆x (2.68)
with F * , F m respectively the desired and measured forces at the end-effector, ẋ * , ẋm respectively the desired and measured velocities of the end-effector, P adm , D adm the proportional and derivative gains for the admittance control, q * , q m respectively the desired and measured joint positions, q * , qm respectively the desired and measured joint velocities and i the motor currents.

Then in the example, the desired end-effector velocity is saturated in the example and then v * , q * are computed using the Inverse Kinematics. The position controller to obtain the motor current is usually done by a simple PID:

i = P (q * -q m ) + I t 0 (q * (h) -q m (h))dh + D( q * -qm ) (2.69)
with P, I, D the respective proportional, integral and derivative gains. Proportional gains for this kind of control are often very high, and integral ones small or equal to zero. This is making the robot extremely rigid, but the solution is simple to deploy.

An overview of admittance control for physical interactions with human can be found in Keemink et al. [START_REF] Arvid | Admittance control for physical human-robot interaction[END_REF], which proposes guidelines to achieve high-performance control. A passive based admittance controller is proposed in Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF] for flexible joint robots, by choosing appropriate gain values. In Fonseca et al. [START_REF] Fonseca | Task-space admittance controller with adaptive inertia matrix conditioning[END_REF], the authors propose a task-space admittance controller adapting the inertia matrix online to avoid its ill-conditioning.

Robot Position Controller

+ - i v * , q * IK PD+ ẋ * sat PID q m ,𝓿 m ẋ * e f q m ,𝓿 m Forward Kinematics + - F m F * x m , ẋ m x *

Impedance Control

As opposed to admittance control, the impedance approach assumes that the environment has inertia or kinematic constraints, which given force inputs produce motion as outputs. This means that the environment acts as an admittance and thus the robot controller is defined as an impedance for consistency. When in rigid contact with the environment, the impedance approach allows to control the position of the contact, ensuring steady contact forces (depending on the rigidity of the surface) which can be regulated through the open-loop, or directly if a feedback loop is added.

Using impedance control, the robot can be directly torque controlled because the outputs of the impedance scheme are forces. Using the Jacobian matrix of the contacts J one can obtain the desired torques τ * from the desired forces Albu-Schaffer and Hirzinger [START_REF] Albu-Schaffer | Cartesian impedance control techniques for torque controlled light-weight robots[END_REF]. A simple impedance scheme is presented in Fig. 2.12 where a mass-spring-damper model is used to achieve the mechanical impedance described in Eq.2.66. Using the equation of the dynamics (see Eq.2.4) adapted in the Cartesian space, one can link F * and F ext Siciliano et al. [2]. Indeed, using the system dynamics, a solution which respects Eq.2.66 is simply given by τ * = J T F * Lu and Meng [START_REF] Lu | Impedance control with adaptation for robotic manipulations[END_REF] with the following equations:

∆x = x * -x m F * + F ext = M x ẍm + C x ẋm + g x F ext = M x ∆ẍ + D x ∆ ẋ + K x ∆x τ * = J T (M x ẍm + C x ẋm + g x -F ext ) (2.70)
with g x = J -T g and for simplification we choose the mass matrix M x to be equal to the natural Cartesian inertia Λ = J -T M J -1 .

Then, the desired torque is given to a torque controller which computes the motor currents using the chain actuator models (see Chapter 3 for an example on Robot Torque Controller the elbow joint of TALOS). As opposed to the previous position control, torque control leads to compliant behaviors but is more complex to deploy on the robot.

+ - Actuation Model 𝜏 m 𝜏 * i q m ,𝓿 m , a m Mass-Spring-Damper Model F ext = M x Δẍ + D x Δẋ + K x Δx F * = M x ẍ m + C x ẋ m + g x -F ext 𝜏 * = J T F * Forward Kinematics & Dynamics x m , ẋ m , ẍ m x * , ẋ * , ẍ * J, g x
Most of the impedance controllers neglect the mass term and only use a feedforward in velocity and do not regulate the error Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF], Schindlbeck and Haddadin [START_REF] Schindlbeck | Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks[END_REF], Ott et al. [START_REF] Ott | A hybrid system framework for unified impedance and admittance control[END_REF]. One reason is that, with this simpler formulation, it is possible to prove the passivity of the controller Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF], Schindlbeck and Haddadin [START_REF] Schindlbeck | Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks[END_REF]. Moreover it is possible to add an outer force close loop to regulate the forces Schindlbeck and Haddadin [START_REF] Schindlbeck | Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks[END_REF], Ott et al. [START_REF] Ott | A hybrid system framework for unified impedance and admittance control[END_REF].

Stiffness/Compliance Control

Another solution to perform indirect force control using a closed-loop position control is the compliance (or stiffness) control Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF], Albu-Schaffer and Hirzinger [START_REF] Albu-Schaffer | Cartesian impedance control techniques for torque controlled light-weight robots[END_REF], Salisbury [START_REF] Salisbury | Active stiffness control of a manipulator in cartesian coordinates[END_REF], Ajoudani et al. [START_REF] Ajoudani | Natural redundancy resolution in dual-arm manipulation using configuration dependent stiffness (cds) control[END_REF]. In this control scheme the position error is related to the contact force through a mechanical stiffness set in a specific matrix. If the contact forces exerted by the end-effector on the environment are not null while regulating its position, at the equilibrium the end-effector behaves like a generalized linear spring with compliance K c with respect to the equivalent force F :

F = K c ∆x (2.71)
K c is the Cartesian stiffness matrix associated to the forces F . The stiffness control consists in designing this matrix to achieve the desired compliant control. A simple Cartesian stiffness control scheme is detailed in Fig. 2.13. As opposed to the impedance control, the external forces behavior is chosen to respect the dynamics fixed by the stiffness matrix of Eq.2.71. Then the desired forces F * are computed using a PD control with a gravity compensation Siciliano et al. [2], Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF]:

F * = K c ∆x + D x ẋm + g x (2.72)
As in the impedance control, the output control can be torques which are transformed into motor currents by a torque controller.

To use the Cartesian stiffness matrix in the joint space, it needs to be transformed Salisbury [START_REF] Salisbury | Active stiffness control of a manipulator in cartesian coordinates[END_REF], Albu-Schaffer et al. [START_REF] Albu-Schaffer | Soft robotics: what cartesian stiffness can obtain with passively compliant, uncoupled joints?[END_REF]. The stiffness matrix in the joint space depends on the Jacobian J and thus on the robot configuration. And consequently the transform operator also depends on it. This mapping was first expressed in a simple form, by computing the stiffness matrix around the equilibrium position. However, this simplification removed the dependency of the stiffness matrix to J. Specifically, the terms representing the product of the changes in geometry and external forces of the stiffness matrix were removed Albu-Schaffer et al. [START_REF] Albu-Schaffer | Soft robotics: what cartesian stiffness can obtain with passively compliant, uncoupled joints?[END_REF].

Robot Torque Controller + - Actuation Model 𝜏 m 𝜏 * i q m ,𝓿 m Cartesian Matrix Stiffness K c F ext = K c Δx F * = K c Δx + D x ẋ m + g x 𝜏 * = J T F * Forward Kinematics & Dynamics x m , ẋ m J, g x x * , ẋ *
Recent works have been improving the stiffness control by regulating the task space stiffness in function of the robot configurations Ajoudani et al. [START_REF] Ajoudani | Natural redundancy resolution in dual-arm manipulation using configuration dependent stiffness (cds) control[END_REF][START_REF] Ajoudani | On the role of robot configuration in cartesian stiffness control[END_REF], while solving the redundancy of the manipulator. In Albu-Schaffer and Hirzinger [START_REF] Albu-Schaffer | Cartesian impedance control techniques for torque controlled light-weight robots[END_REF] is presented an impedance control enhanced by the addition of a local stiffness control. And a passive based control has been implemented for flexible joint robots in Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF].

Hybrid Control

During interaction tasks, the environment imposes constraints on the end-effector, defining a constraint frame orthogonal to the task Siciliano et al. [2]. If the environment is rigid and there is no friction, the DoFs of the end-effector can be constrained in velocity (no translation or rotation along an axis) or in force (no force application along a direction or torque around an axis); these are called natural constraints because they are determined by the task geometry. Thus, the task space can be divided in two sub-spaces: velocity and force control, since both velocity and force cannot be controlled along any given direction. Moreover, the end-effector can only be controlled using the variables that are not already used by the natural constraints. The reference values of these variables define the artificial constraints which depend on the type of control chosen to realize the task Raibert and Craig [START_REF] Raibert | Hybrid Position/Force Control of Manipulators[END_REF]. The hybrid force/position or force/motion control consists in not controlling the variables imposed by the natural constraints to avoid conflicts and over-specification of the problem.

The structure of the hybrid control can be described as the following: two controllers are implemented, one acting on the force variables and another on the position/velocity variables. These two controllers can act on the whole system (on each DoF and tasks) and the final control is composed of a mix of the two control laws depending on the selected behavior for the tasks. This composition is realized using complementary selection matrices Siciliano et al. [2], Anderson and Spong [START_REF] Anderson | Hybrid impedance control of robotic manipulators[END_REF], Raibert and Craig [START_REF] Raibert | Hybrid Position/Force Control of Manipulators[END_REF]. A simple example of such a scheme is presented in Fig. 2.14.

Robot Selection Controller F m F * q i q m ,𝓿 m , a m Impedance / Position Control Forward Kinematics & Dynamics x m , ẋ m , ẍ m x * , ẋ * , ẍ * J, g x Position Controller F * f + - F m F * Admittance / Force Control v * , q * PID q m , 𝓿 m Selection Matrix / Weights Force Controller F * e f x m , ẋ m x *
In this example, the chosen control input is the forces F * :

F * = S q F * q + S f F * f (2.73)
with S q , S f the respective selection matrix for the position and force control. Therefore, a task can be assigned specifying a desired force F * and a desired velocity ẋ * . This method can be used for velocity input in the same fashion, using a force controller to transform the desired forces from the position/velocity or impedance control in desired velocity. A more elegant way of handling this composition is to rewrite the equations 2.68 and 2.70 in terms of acceleration. Using the inverse dynamics control law, one can decouple the force and velocity variables to use the acceleration as control input for both the force and position/velocity controllers, simplifying the control scheme Siciliano et al. [2].

The first implementations of hybrid position/force control did not account for the dynamic coupling between the robot and the environment Raibert and Craig [START_REF] Raibert | Hybrid Position/Force Control of Manipulators[END_REF]. In our scheme (Fig. 2.14) it corresponds to the case where the force control and position control blocks are not admittance and impedance ones. The drawback of this early method was that the structure of the problem needed to be changed accordingly to the task phases. Indeed, it can be necessary to switch from force to velocity/position control quickly, but ensuring a stable transition is hard. It mostly depends on the precision of the geometric environment model and planning. Contact loss and unexpected impacts were typically difficult to handle by these controllers because the selection matrices did not take into account these disturbances.

Then, hybrid methods have been developed to take into account the dynamics of the interactions, such as using impedance control in Anderson and Spong [START_REF] Anderson | Hybrid impedance control of robotic manipulators[END_REF]. More recent researches on hybrid control tries to couple admittance and impedance control using switching or weighting variables instead of selection matrices Ott et al. [START_REF] Ott | A hybrid system framework for unified impedance and admittance control[END_REF], Kim et al. [START_REF] Kim | Weighted hybrid admittance-impedance control with human intention based stiffness estimation for human-robot interaction[END_REF]. The former paper presents an adjusting switching ratio depending on the duty cycle for fixed environment stiffness whereas the second one adjusts its control distribution weights according to the stiffness changes. In our illustrative scheme (Fig. 2.14), it can correspond to the case where the force control and position control blocks are admittance and impedance ones and the selection controller uses ratio or weights.

Robot Parallel Controller F m F * x i q m ,𝓿 m , a m Impedance / PD Position Control Forward Kinematics & Dynamics x m , ẋ m , ẍ m J, g x F * f + - F m F * PI Force Control Force Controller F * + + x * , ẋ * , ẍ *

Parallel Control

The parallel force/position scheme aims to combine the characteristics of the impedance and admittance schemes; controlling both position and force variables as the hybrid control schemes. It has been first presented to answer the issue of robustness with respect to the environment uncertainties of hybrid control Chiaverini and Sciavicco [START_REF] Chiaverini | The parallel approach to force/position control of robotic manipulators[END_REF]. This solution is achieved by implementing two controllers acting in parallel, avoiding conflicts by prioritizing the force control over the position one Chiaverini and Sciavicco [START_REF] Chiaverini | The parallel approach to force/position control of robotic manipulators[END_REF], Siciliano [START_REF] Siciliano | Parallel force/position control of robot manipulators[END_REF]. The dominance is achieved by implementing a PI force control loop in parallel to a PD position control loop. Thus, in a steady state situation it is possible to have a constant position error while the force error is equal to zero. This approach is close to the hybrid impedance control of Anderson and Spong [START_REF] Anderson | Hybrid impedance control of robotic manipulators[END_REF] but does not rely on selection strategy. In both approaches the task planning provides force or position references along suitable task space directions. However, the parallel control is robust in case of planning errors in particular during contacts thanks to the force dominance rule Siciliano [START_REF] Siciliano | Parallel force/position control of robot manipulators[END_REF].

The Fig. 2.15 illustrates the concept of parallel control, under the following set of equations Chiaverini and Sciavicco [START_REF] Chiaverini | The parallel approach to force/position control of robotic manipulators[END_REF], Siciliano [START_REF] Siciliano | Parallel force/position control of robot manipulators[END_REF] with force control input (acceleration input can be defined similarly):

F * = F * f + F * x F * f = K f e f + K i t 0 e f dt F * x = M x ẍ * + D x ∆ ẋ + K x ∆x (2.74)
One can choose a simpler impedance control as explained before by simply setting

F * x = K x ∆x -D x ẋm Siciliano et al. [2].
In the Chapter 4 is presented the solution implemented for our whole-body controller to achieve a force operation task. It is close to a parallel force/position scheme with the addition of Coulomb contact constraints (see Section 4.3.4).

Model Predictive Control

It is possible to create controllers which consider a time horizon on a short time window instead of an instantaneous control. This type of control is called Model predictive control (MPC) Rawlings et al. [START_REF] Rawlings | Model Predictive Control: Theory, Computation and Design[END_REF] and is becoming an important field of research in robotics because it can handle non-linearity, constraints and system dynamics Huba [START_REF] Huba | Selected Topics on Constrained and Nonlinear Control[END_REF]. The MPC is part of the branch of applied mathematics called Optimal Control. This method is able to predict the future behavior of the robot over a receding finite-time horizon, using the mathematical model of the system dynamics Rawlings et al. [START_REF] Rawlings | Nonlinear model predictive control: A tutorial and survey[END_REF], Findeisen and Allgöwer [START_REF] Findeisen | An introduction to nonlinear model predictive control[END_REF]. Because of its prediction capability, MPC can for instance explicitly take into account stability. Passivity can be implemented by using the value function as a storage function and the optimal feedback as an output of the system, or by adding a sate constraint over the trajectory to respect the passivity inequality Raff et al. [START_REF] Raff | Nonlinear Model Predictive Control: A Passivity-Based Approach[END_REF]. It is also possible to explicitly change the tasks which are activated inside the time horizon and to ensure state smoothness. Thus, using MPC, one can formulate and solve a sequence of controllers Todorov et al. [START_REF] Todorov | Mujoco: A physics engine for model-based control[END_REF] to handle complex problems. Recent MPC formulations are based on explicit discrete variables of the problem, in order to be able to choose contact phases Dai et al. [START_REF] Dai | Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot[END_REF].

The two main problems of the MPC approach are its computational cost and its limited time preview window (which is short, typically 2 -3 s). Indeed, as the Optimal Control Problem (OCPs) are often non-convex due to the robot model non-linearity, they lead to high computational load and to local minimum Boyd and Vandenberghe [START_REF] Boyd | Convex Optimization[END_REF]. This is why the MPC has first been applied on simplified model Kajita et al. [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF], Kim et al. [START_REF] Kim | Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control[END_REF] or systems with few DoFs Neunert et al. [START_REF] Neunert | Fast nonlinear model predictive control for unified trajectory optimization and tracking[END_REF], Grandia et al. [START_REF] Grandia | Feedback mpc for torque-controlled legged robots[END_REF]. A first solution was to use the OCP to re-plan trajectories at low frequency while the robot was controlled by another controllerNeunert et al. [START_REF] Neunert | Fast nonlinear model predictive control for unified trajectory optimization and tracking[END_REF].

For a system state x ∈ R n and a control u ∈ R m , such that ẋ = f (x, u), one can formulate the OCP as follows Mayne [START_REF] Mayne | A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems[END_REF], Dantec et al. [START_REF] Louis Dantec | Whole Body Model Predictive Control with a Memory of Motion: Experiments on a Torque-Controlled Talos[END_REF]:

min x,u T -1 t=0 l t (x(t), u(t))dt + l T (x(T )) s.t. x(0) = x 0 ∀t ∈ [0, T ], ẋ(t) = f t (x(t), u(t)) (2.75)
with T the horizon time where is made the prediction, l t , l T are the running and terminal cost functions and x 0 is the initial state.

The cost functions are used to define the robot tasks, to regularize the state and control trajectories. The system constraints (such as the joint, velocity or torque limits) can be added in the cost as well or be put as constraints of the problem. Thus, the cost functions are often non-linear. During an iteration of control, the robot state is estimated and a new OCP is solved over the finite horizon between the estimated state and the future shifted from one sampling period.

In order to efficiently use a whole-body MPC directly as the main robot controller one will want to tackle the following problems:

The computational time: To solve this problem one will need to use an OCP algorithm working at high frequency Carpentier et al. [START_REF] Carpentier | The pinocchio c++ library -a fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF], Featherstone [START_REF] Featherstone | Rigid Body Dynamics Algorithms[END_REF], or to rewrite the OCP to obtain a more efficient formulation in terms of computation.

The number of iterations before convergence: To reduce this number, one can use a convex formulation Majumdar et al. [START_REF] Majumdar | Recent scalability improvements for semidefinite programming with applications in machine learning, control, and robotics[END_REF], or a warm start.

To answer the first issue, shooting methods like Differential Dynamic Programming (DDP) Tassa et al. [START_REF] Tassa | Receding horizon differential dynamic programming[END_REF] have been introduced. In Bertsekas [START_REF] Bertsekas | Dynamic Programming and Optimal Control[END_REF], Tassa et al. [START_REF] Tassa | Control-limited differential dynamic programming[END_REF], Xie et al. [START_REF] Xie | Differential dynamic programming with nonlinear constraints[END_REF] are presented efficient solvers able to deal with non linear costs, constraints and dynamics (see the following Section 2.8.1). Because of this problem of computational time, it was not possible to connect the MPC directly to the low-level servo controller Englsberger et al. [START_REF] Englsberger | Torque-based dynamic walking -a long way from simulation to experiment[END_REF], Herzog et al. [START_REF] Herzog | Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid[END_REF] until Dantec et al. [START_REF] Louis Dantec | Whole Body Model Predictive Control with a Memory of Motion: Experiments on a Torque-Controlled Talos[END_REF]. Indeed, to connect the two controllers it is necessary to evaluate the MPC at a compatible high frequency (at least 100Hz).

To find a good warm start for the second issue, Diehl et al. [START_REF] Diehl | A real-time iteration scheme for nonlinear optimization in optimal feedback control[END_REF] proposes to use the solution from the previous iteration, but this heuristic is not robust to disturbances. Mansard et al. [START_REF] Mansard | Using a memory of motion to efficiently warm-start a nonlinear predictive controller[END_REF] uses the learning approach and specifically offline reinforcement learning (see Section 2.9) to find a warm start. In Dantec et al. [START_REF] Louis Dantec | Whole Body Model Predictive Control with a Memory of Motion: Experiments on a Torque-Controlled Talos[END_REF] is presented the first successful experiment implementing whole-body model predictive control with state feedback on a torque-controlled humanoid robot, running at 100Hz. The control scheme is able to do whole-body target tracking, control the balance in front of strong external perturbations and avoid collision with an external object.

Differential Dynamics Programming Formulation

The DDP algorithm considers the OCP of Eq.2.75 in discrete form and denotes the control and state variables along the time horizon T as U = {u 0 , u 1 , ..., u T -1 } and X = {x 0 , x 1 , ..., x T , U }. To minimize the cost function of Eq.2.75, the DDP uses the "Bellman's principle of optimality" to transform the general problem into a sequence of simpler sub-problems. Indeed, this principle states that the remaining decisions of an optimal policy constitute an optimal policy with respect to the first decisions resulting state (in the absence of disturbances) Bellman [START_REF] Bellman | On the theory of dynamic programming[END_REF]. For the DDP algorithm, it is used to proceed backward in time from T -1 to 0 while evaluating the optimal cost function at every node in the discretized state-time space. The evaluation is performed using a value function V based on the Bellman equation:

V i (x i ) = min u l i (x i , u i ) + V i+1 (f (x i , u i )) (2.76)
One can note Q, the function of the small variations of the value function around the state and command of iteration i. Q is locally approximated by a quadratic function (Gauss-Newton approximation) as follows Tassa et al. [START_REF] Tassa | Control-limited differential dynamic programming[END_REF]:

Q(δx, δu) ≈ 1 2    1 δx δu    T    0 Q T x Q T u Q x Q xx Q xu Q u Q ux Q uu       1 δx δu    (2.77)
Where the subscripts denote the partial derivatives of the terms for clarity, for instance f x = ∂f (x,u) ∂x . Using this approximation, one can obtain the algorithm of the DDP, detailed in the Appendix 2, see Algorithm 1. The optimal control local policy updating u (l.25 -26) is defined as in Forget et al. [START_REF] Forget | Implementation, identification and control of an efficient electric actuator for humanoid robots[END_REF], by:

δu * = arg min δu Q(δx, δu) = -Q -1 uu (Q u + Q ux δx) (2.78) α (l.25
) is a backtracking search parameter, set to 1, which can then be iteratively reduced (not presented in the algorithm for clarity) Tassa et al. [START_REF] Tassa | Control-limited differential dynamic programming[END_REF]. An illustrative scheme of the DDP algorithm can be found in Fig. 2.16.

It is possible in some cases to linearize this formulation and approximate the cost function to its quadratic form along the state trajectory Li and Todorov [START_REF] Li | Iterative linearisation methods for approximately optimal control and estimation of non-linear stochastic system[END_REF] to obtain an iterative linear quadratic regulator (iLQR). In the Chapter 3, we present a MPC using this DDP-iLQR formulation to control the elbow of TALOS in torque using the identified model of the actuator.

Learning approach

This thesis uses a model base approach to solve the different algorithms, relying on the kinematics and dynamics models of the robot. Another method is to use machine learning techniques, which propose new solutions in the robotics field, such as control, modeling or planning. We briefly expose their principles in this section. Two major approaches are Reinforcement learning Sutton and Barto [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF] and Deep Learning LeCun et al. [START_REF] Lecun | Deep learning[END_REF]. The former is a methodological approach which aims at finding a policy π to maximize a reward function assuming a value function. The value function V (x, u) based on state and control provides a way to evaluate the cost to achieve a behavior. It can use a parametric functional space to find an appropriate control law and/or the model to be controlled Sutton and Barto [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF], Jemin Hwangbo and Hutter [START_REF] Siegwart | Control of a quadrotor with reinforcement learning[END_REF]. Deep learning is a supervised approach which has recently gained a large popularity in computer vision LeCun et al. [START_REF] Lecun | Deep learning[END_REF]. A current method is to use Convolutional Neural Network (used in Deep Learning) in the context of Reinforcement Learning Esteban et al. [START_REF] Esteban | Learning deep robot controllers by exploiting successful and failed executions[END_REF]. Interestingly, in order to discover the policy a large part of these works are using MPC Todorov et al. [START_REF] Todorov | Mujoco: A physics engine for model-based control[END_REF]. More generally, the assumptions related to optimal control are used in the different learning approaches Bottou and Bousquet [START_REF] Bottou | Optimization for Machine Learning, chapter The Tradeoffs of Large Scale Learning[END_REF].

An important part of the learning method is to generate an action and to evaluate its corresponding reward. The exploration of the controlled system and its specific environment is in general time consuming and very costly on a real robot. For this reason simulations are often used in a pre-exploration phase, which can require high computing performance. For instance, Reinforcement Learning can be used on the physical parameters of the robot to learn a model in simulation (see Section 2.3). The training of the control policy has to satisfy the physical constraints such as the joint limits. The main problem is then to deploy the trained policy on the physical system. Indeed, the methodology implies to perform some run on the real robot to take into account the differences between the simulation model and the real one. Recent works (Hwangbo et al. [START_REF] Hwangbo | Learning agile and dynamic motor skills for legged robots[END_REF], Tan et al. [START_REF] Tan | Sim-to-real: Learning agile locomotion for quadruped robots[END_REF]) demonstrate great results on quadruped robots, however, it is unclear how this can be realized on humanoid robot which are much more unstable than quadruped robots. This impossibility to realize numerous tests on the real robotic platforms has lead to develop specific learning algorithms which require few training data, called one-shot and few-shot learning Vinyals et al. [START_REF] Vinyals | Matching networks for one shot learning[END_REF], Snell et al. [START_REF] Snell | Prototypical networks for fewshot learning[END_REF], James et al. [START_REF] James | Task-embedded control networks for few-shot imitation learning[END_REF].

Human-Robot Collaboration

Interactions between humanoid robots and humans raise great challenges as humanoid robots are complex systems due to their numerous DoFs and natural instability. During collaborative interactions, the tasks are generally performed by the human while the robot is passively following or assisting K. et al. [START_REF] Kosuge | Dynamic control for robot-human collaboration[END_REF]. However, by anticipating and predicting the human behavior and motions the robot would be able to perform its task more efficiently. To achieve this aim, cobotic applications require a good knowledge of the human behaviour. For example, to make a human and a humanoid robot perform a co-navigation or a co-manipulation task, a model of human walking trajectories is essential to make the robot follow or even anticipate the human movements Maroger et al. [START_REF] Maroger | Inverse Optimal Control to Model Human Trajectories During Locomotion[END_REF].

In this section is presented a short state-of-the-art on the prediction processes and human-likeness trajectory planning methods used to proactively interact with humans. This section focuses on the literature used in the paper [24] realized in collaboration with Isabelle Maroger, presented in Section 4.5.

Proactive human-robot interactions

To proactively collaborate with a human, a humanoid robot needs to predict, or at least guess, its partner's future actions Jarrassé et al. [START_REF] Jarrassé | Slaves no longer: review on role assignment for human-robot joint motor action[END_REF]. The first experiment where a humanoid robot proactively interacts with a human was performed in Bussy et al. [START_REF] Bussy | Proactive behavior of a humanoid robot in a haptic transportation task with a human partner[END_REF]. In this work, motion primitives like Stop, Side, Turn and Walk/Turn were used to generate the robot locomotion according to the human velocity. Since then, collaborative tasks are often aimed to be as proactive as possible in order to smooth the human-robot interactions.

For example, in Otani et al. [START_REF] Otani | Generating assistive humanoid motions for co-manipulation tasks with a multi-robot quadratic program controller[END_REF], a co-manipulation task is aimed. To achieve this goal, the authors design a robot controller which can generate optimal motions in real time equivalent to those generated by a human. As this controller takes into account a whole-body dynamics of the human, it allows a more proactive interaction between the humanoid-robot and the human. Furthermore, proactive co-navigation tasks have already been well studied. Indeed, in Teja S. and Alami [START_REF] Teja | Hateb-2: Reactive planning and decision making in human-robot co-navigation[END_REF], the authors propose a reactive trajectory planner for robots which takes into account the human predicted motions and goals to handle human-robot co-navigation. The estimation of the human movement is based on its current velocity and three different navigation modes are presented to make the robot move forward without colliding with the human. In this work, the goal is to smoothly avoid the human partner.

In Lanini et al. [START_REF] Lanini | Interactive locomotion: Investigation and modeling of physically-paired humans while walking[END_REF] and Lanini et al. [START_REF] Lanini | Human intention detection as a multiclass classification problem: Application in physical human-robot interaction while walking[END_REF], the authors target a collaborative handling task while walking, where the humanoid robot identifies its human partner's intentions. To guess future human motions, a training of a multiclass classifier of human intentions was performed using measurements collected from a humanhuman handling collaboration. The results of this training, namely the optimal features to discriminate a set of human motions, was then used to predict human intentions. Machine learning Aarno and Kragic [START_REF] Aarno | Motion intention recognition in robot assisted applications[END_REF] Kelley et al. [START_REF] Kelley | Understanding human intentions via hidden markov models in autonomous mobile robots[END_REF] Li et al. [START_REF] Li | Data driven models for human motion prediction in human-robot collaboration[END_REF] and probabilistic state machines Awais and Henrich [START_REF] Awais | Human-robot collaboration by intention recognition using probabilistic state machines[END_REF] have already been studied to guess human intentions. In Maroger et al. [24], an OCP model is proposed to predict the human partner's future trajectory (briefly described in Section 4.5). A real-time approach is more complex but gives more versatility for future works. Indeed, optimisation problems allow to incrementally add obstacles or new constraints by adding new terms to the cost function.

Human trajectories during gait

Healthy human gait is a well studied field in bio-mechanics Winter [START_REF] Winter | Biomechanics and motor control of human gait[END_REF]. However, the focus of those works is rarely on the modeling and the simulation of human walking trajectories. Nevertheless, in robotics, some authors studied this problem to generate human-like paths for a robot to follow. Thus, models of human walking trajectories using parametric curves called clothoids Raković et al. [START_REF] Raković | Human-inspired online path planning and biped walking realization in unknown environment[END_REF] or OCP models Papadopoulos et al. [START_REF] Papadopoulos | Generation of human walking paths[END_REF], Arechavaleta et al. [START_REF] Arechavaleta | On the nonholonomic nature of human locomotion[END_REF], Mombaur and Laumond [START_REF] Mombaur | From human to humanoid locomotion -an inverse optimal control approach[END_REF] have been designed.

In Maroger et al. [START_REF] Maroger | Walking human trajectory models and their application to humanoid robot locomotion[END_REF], the authors assessed those models according to a metrics evaluating the linear and angular distances between human trajectories and generated trajectories. The most human-like model was the one introduced in Mombaur and Laumond [START_REF] Mombaur | From human to humanoid locomotion -an inverse optimal control approach[END_REF] which approximates the human by a holonomic system instead of a non-holonomic one. Indeed, a holonomic model allows lateral and oblique motions which better fits human locomotion. This is why a new OCP model, inspired from Mombaur and Laumond [START_REF] Mombaur | From human to humanoid locomotion -an inverse optimal control approach[END_REF], is defined in Maroger et al. [START_REF] Maroger | Inverse optimal control to model human trajectories during gait[END_REF]. This model gives an accurate approximation of human behaviour. This OCP model is detailed and used in the Section 4.5. In this chapter are detailed the results obtained in the publication Ramuzat et al. [22].

Chapter 3 Identification, Modeling and Protection of the Robotic Chains

Introduction

Having torque control on a humanoid robot is interesting for safe interaction with the environment and for its potential impact on cobotic applications. Historically, most of the humanoid robots are controlled in position and few robots, such as the TORO robot Englsberger et al. [START_REF] Englsberger | Overview of the torque-controlled humanoid robot toro[END_REF] or the DYROS-JET robot Park et al. [START_REF] Park | System design of humanoid robot dyros-jet[END_REF], are torque controlled. Recently, a new generation of humanoid robots, including the TALOS robot, have shown the possibility to use their actuators either in position or in torque control modes Stasse et al. [START_REF] Stasse | Talos: A new humanoid research platform targeted for industrial applications[END_REF]. Several whole-body control architectures have been proposed (Cisneros et al. [START_REF] Cisneros | Robust humanoid control using a qp solver with integral gains[END_REF], Khatib et al. [START_REF] Khatib | Whole-body dynamic behavior and control of human-like robots[END_REF]) to compute and follow desired joint torques. However regulating joint torques is still an opened problem, which requires a low level controller able to follow a desired joint torque (see Section 2.3.4). In particular, some humanoid robots do not provide access to their low level controller (HRP-2 for instance), as well as a majority of industrial robots. In this situation, Khatib et al. [START_REF] Khatib | Torque-position transformer for task control of position controlled robots[END_REF] proposed a controller which takes in input a desired torque and provides a desired position to reflect the desired torque at the actuator. In a similar spirit, Prete et al. [START_REF] Del Prete | Implementing torque control with high-ratio gear boxes and without joint-torque sensors[END_REF] implemented a torque control and inverse-dynamics control on the HRP-2 robot, originally designed for position control. They estimate the joint torques from the force sensors and the IMU by using the method proposed for the iCub robot by Nori et al. [START_REF] Nori | icub whole-body control through force regulation on rigid non-coplanar contacts[END_REF].

In this chapter, we present a solution which takes advantage of the capabilities of the commercially available TALOS robot. Its industrial standard EtherCat bus allows to gather numerous information about the actuators at a very high frequency (∼ 2kHz). Thus, information related to the chain drive, the motor and joint positions or the joint torques can be read simultaneously. We would like to be able to generate extreme but safe motions, such as carrying heavy object or applying huge forces (∼ 600N) for manufacturing operations (like drilling), without breaking or causing damages to the motor. This event may occur if a large torque is sustained for an extended period of time or when the robot is carrying a large additional payload. To avoid it, the current drawn in an actuator can be limited by adapting its motion. Moreover, the motor current can be saturated by the joint controller if the desired joint torque is above its limit.

Ideally, this could be done using a MPC (see Section 2.8). The use of a DDP as MPC allows to cope with the actuator flexibility while meeting the control loop timing constraints. However, such a controller requires the knowledge of the inertial parameters of the robot, of the motor chain drive and to have a well calibrated joint torque sensor. As said before, these parameters may not be provided by the manufacturer of the robot or may be inaccurate. Indeed, for TALOS, the chain drive parameters are not disclosed, to protect industrial conception, and they do not take into account the element modifying the inertia (such as cabling, glued elements or protection covers). Fortunately, the literature in system identification of serial manipulators proposes methods that allow to identify the joint drive gains and inertial parameters at once Bonnet et al. [START_REF] Bonnet | Self-generation of optimal exciting motions for identification of a humanoid robot[END_REF], Gautier and Jubien [START_REF] Gautier | Force calibration of KUKA lwr-like robots including embedded joint torque sensors and robot structure[END_REF], Gautier and Briot [START_REF] Gautier | Global identification of joint drive gains and dynamic parameters of robots[END_REF] (see Section 2.3).

Alternatively, Reinforcement Learning algorithms can be used on the physical parameters of the robot to learn a model in simulation. The training of the control policy beware to satisfy the physical constraints such as the joint limits. The main problem is then to deploy the trained policy on the physical system. Indeed, the methodology implies to perform some run on the real robot to take into account the differences between the simulation model and the real one. Recent works (Hwangbo et al. [START_REF] Hwangbo | Learning agile and dynamic motor skills for legged robots[END_REF], Tan et al. [START_REF] Tan | Sim-to-real: Learning agile locomotion for quadruped robots[END_REF]) demonstrate great results on quadruped robots, however, it is unclear how this can be realized on humanoid robot which are much more unstable than quadruped robots.

In this context, the objective of this chapter is to push the TALOS robot to its limits during joint torque control. For that purpose, we identify the joint chain drive of the elbow joint and the corresponding inertial parameters and use a DDP approach to protect the system (see Section 2.8.1). We organize the chapter as follows: Section 3.2 presents the methods used to create the mechanical and thermal model of the chain actuation. In Section 3.3 are described the mathematical process to identify the parameters of these two models. Section 3.4 details the optimal control scheme which takes into account the identified models to protect the robotic system. Finally, Section 3.5 presents the simulations and experiments used to identify the parameters and validate the controller on the robot TALOS.

Modeling of the Rigid Chain Actuators

Mechanical Model

The inverse dynamics model of the investigated system is used to calculate the motor torque τ m and joint torques τ j as a function of the joint and/or motor positions, velocities and accelerations. It is usually calculated using Newton-Euler equations W. Khalil [START_REF] Khalil | Modeling, identification and control of robots[END_REF]. The rigid chain actuators of TALOS are composed of a brushless motor, that can be controlled in current, connected to a harmonic drive and a torque sensor attached to its corresponding link (see Fig. 2.3). In this study the investigated sub-system is composed of the chain actuation of a single joint as represented in Fig. 3.1. The motor and joint positions are measured by two high-precision encoders (19 bits or 524,288 counts per revolution) and are considered equal thanks to the harmonic drive high stiffness. Consequently, in this chapter, the variables q, q and q will be used to refer to the joint position, velocity and acceleration, respectively (we do not consider the under-actuation of the robot). The total inverse dynamics

τ m = (I m + I j )q + g M Y sin q + M X cos q + τ f m + τ f j τ j = I j q + g M Y sin q + M X cos q + τ f j (3.1)
and

τ f m = F vm q + F sm sign( q) + of f m τ f j = F vj q + F sj sign( q) + of f j (3.2)
where I m is the rotor inertia of the motor, I j is the corresponding link inertia expressed at the joint level, F vm and F sm are the motor viscous and dry frictions, F vj and F sj are the viscous and dry frictions at the joint level, of f m and of f j are the offsets of the torque motor and joint, M X and M Y are the first moment of inertia expressed at the joint level and g is the gravity.

The motor torque τ m is related to the current by:

τ m = RK m i m (3.3)
where R is the gear ratio, K m is the joint drive of the manufacturer and i m is the input current of the motor. Moreover, we have a similar relationship between the real applied torque at the joint, τ j , and the torque measured by the sensor ,τ js :

τ j = K j τ js (3.4)
where K j is the gain between the two quantities.

We can express the simple dynamic of our system as:

q = 1 I RK m i m -F v q -F s sign( q) + of f ) -(M Y sin q + M X cos q) g I (3.5)
where

I = I m + I j , F v = F vm + F vj , F s = F sm + F sj and of f = of f m + of f j .

Actuator Thermal model

The robot TALOS has temperature sensors for each of its joint. In this section is described the method used to obtain a model of the motor temperature evolution.

Let us consider R the winding resistor which increases linearly with the temperature of the motor T :

R = R T A (1 + α Cu (T -T A )) (3.6)
where T A is the ambient temperature, R T A is the armature resistance at ambient temperature and α Cu is the temperature coefficient of copper resistance.

We can now consider the Joule power losses together with the actuator model presented in the previous section: In this model, some terms are neglected due to their small impact on the considered problem: The friction losses because the type of motor is DC brushless, the re-magnetization losses and the Eddy current losses.

P J = Ri 2 m = R T A (1 + α Cu (T -T A ))i 2 m (3.7)
We consider that casing conducts the temperature well enough. Therefore, the outer side of the casing (where we measure the temperature) is considered to have the same temperature as the windings, i.e. R T A = R th . This allows us to have a very simple model of the motor temperature evolution, depicted in Fig. 3.2.2:

C th Ṫ = P J - 1 R th (T -T A ) (3.8)
with C th is the thermal capacity, R th is the thermal resistance. We set R th = R.

Identification of the Rigid Chain Actuators

Identification of chain drive and segment inertial parameters

Usually the segment mass, center of mass and inertia are provided by the robot's manufacturer with a relatively good accuracy, whereas the friction and chain drive parameters are unknown. The provided current gain drive K m is known to have 10 to 15% of inaccuracy Gautier and Briot [START_REF] Gautier | Global identification of joint drive gains and dynamic parameters of robots[END_REF]. Finally, the joint torque sensor should be calibrated prior to be used for control applications. Thus, we propose to use the inverse dynamics model and a total least square approach Gautier and Jubien [START_REF] Gautier | Force calibration of KUKA lwr-like robots including embedded joint torque sensors and robot structure[END_REF] to identify the system described in Fig. 3.1 and its chain drive at once. To do so, two experiments were performed with and without using an additional payload.

When the inertial parameters of the dynamic model are expressed at the joint level, the model is linear, thus Eq.(3.1) becomes:

τ m τ j = RK m i m K j τ js = WΦ = cos(q) sin(q) D D cos(q) sin(q) 0 D Φ (3.9)
where D = [q q sign( q) 1] and W is the so-called regressor matrix. As quickly explained in Section 2.3.3, we need to create a vector containing all the parameters to be identified. This vector is φ, it contains the inertial parameters to be identified and is defined as

φ = [M X M Y I m F vm F sm of f m F vj F sj of f j ].
The TALOS robot embeds motor current and joint torque sensors, thus the regressor matrix W is full rank and the parameters can be identified separately.

To identify the current drive and the torque sensor gains, it is necessary to insert them into the vector φ Gautier and Jubien [START_REF] Gautier | Force calibration of KUKA lwr-like robots including embedded joint torque sensors and robot structure[END_REF]. To do so, two regressor matrices that contain observations from two experiments have to be considered: W 0 for the experiment without payload, and W 1 for the one with a known payload at the end-effector. The Eq.(3.9) can be reformulated as:

     -W 0 i m 0 0 0 -W 0 0 τ js 0 0 -W 1 i m 0 -W up -W kp -W 1 0 τ js -W up -W kp              φ RK m K j φ up φ kp         = 0
(3.10)

W tot Φ tot = 0 (3.11)
where W up and W kp are the observation matrices corresponding respectively to the unknown and known payload inertial parameters.

As noted by Gautier et al. Gautier and Jubien [START_REF] Gautier | Force calibration of KUKA lwr-like robots including embedded joint torque sensors and robot structure[END_REF] W tot is a full rank matrix because of the measurement perturbations. Therefore, the system described in Eq. (3.11) is modified to the closest compatible one with respect to the Frobenius norm:

Ŵtot Φtot = 0 (3.12)
where Ŵtot is the closest rank deficient matrix from W tot and is calculated using the singular value decomposition of

W tot = U SV T .
The solution Φtot is given by the last column of V and is scaled Gautier and Jubien [START_REF] Gautier | Force calibration of KUKA lwr-like robots including embedded joint torque sensors and robot structure[END_REF] using the known position of the center of mass of the payload along the Y-axis (see Fig. 3.1).

The success of the inertial parameters identification relies on maintaining them excited using optimal motions Bonnet et al. [START_REF] Bonnet | Self-generation of optimal exciting motions for identification of a humanoid robot[END_REF]. An exciting joint trajectory is defined using a double S-curve spanning the whole range of motion of the elbow joint. The duration of each phase (acceleration, constant velocity and deceleration) of the S-curve is set to be equal. The velocity plateau is increased by steps of 20% up to its maximal value. A video illustrating the exciting joint trajectories of the arms is available at the following link: https://peertube.laas.fr/videos/watch/ae07428c-f25a-4637-a998-7e6885db3986.

Joint derivatives are obtained to fulfill the regressor matrix using centered differences from the measured joint positions. All the recorded data are low-pass filtered at 5Hz (Butterworth filter, zero-phase lag order 5) and the number of sample is decimated to reduce the noise influence Gautier [START_REF] Gautier | Dynamic identification of robots with power model[END_REF]. The S-curve motions are considered exciting since the condition number of the base parameter regressor matrix is low (cond(W b ) = 36).

Differential Dynamic Programming Optimal Control Scheme

This section presents the optimal control scheme used to find the control sequences required to perform the desired motion. In Section 2.8 is presented the DDP formulation (see Algorithm.1), we use a modified version here. In this chapter, we use the formulation given in Forget et al. [START_REF] Forget | Implementation, identification and control of an efficient electric actuator for humanoid robots[END_REF], Kumar Hari Shankar Lal Das et al. [START_REF] Kumar | Controlling a multi-joint arm actuated by pneumatic muscles with quasi-ddp optimal control[END_REF], which maximizes the performance of the desired motion with respect to the control under the actuator physical limitations constraints.

DDP State space representation

In state space, we denote the state vector x = [q, q] and the command vector u = [i m ].

We then represent the direct dynamics model (Eq.3.5) as the following:

ẋ = f (x, u) = f (q, q, i m ) =   q 1 I RK m i m -F v q -F s sign( q) + of f ) -(M Y sin q + M X cos q) g I   (3.13)

Treatment of Constraints

In comparison to the work achieved in Forget et al. [START_REF] Forget | Implementation, identification and control of an efficient electric actuator for humanoid robots[END_REF], we do not solve a box QP problem to bound the command vector. As introduced in Kumar Hari Shankar Lal Das et al. [START_REF] Kumar | Controlling a multi-joint arm actuated by pneumatic muscles with quasi-ddp optimal control[END_REF], we have instead considered the joint and actuator mechanical limits as constraints on the state and control space in the cost function formulation. The retained constraints functions are expressed by the following equations, referred as 'exponential barrier', at each time-step t:

max(c t ) = 1 -λ(c max -c t ) min(c t ) = 1 -λ(c t -c min ) C s (c t ) = e λmax(ct) + e λmin(ct) (3.14)
where c min , c max are the corresponding lower and upper mechanical limits on angular position, velocity and torque, i.e. c t ∈ {x(t), u(t)}. λ is a positive constant which defines the smoothness of the function. The higher it is, the quicker the cost will increase when approaching the limits (depending on the difference between the limit and the current state).

With these constraints, the total cost function will increase and reach a very high cost near the limits, keeping the system safe, in its mechanical bounds.

The control problem formulation is expressed as an open-loop control u = u(t, x) ∈ U which can minimize or maximize a cost function along a given time interval t ∈ [0, T ] and from an initial state x(0) = x 0 value. The most generic expression for the cost function can be written as follows:

J(x 0 ) = C f + C r + C s (3.15)
where

C f = h(x(T )) is the final cost, C r = T 0 c(x(t), u(t, x(t))
)dt is the integral of the running cost c(x, u) which encapsulates the task objectives and C s is the cost value imposed by the constraints.

For a non-linear dynamics Eq.(3.13) and a non-quadratic cost Eq.(3.15), optimal control solutions can be obtained using full DDP. However, as DDP is computationally expensive, an iterative LQR (iLQR) approach is considered Li and Todorov [START_REF] Li | Iterative linearisation methods for approximately optimal control and estimation of non-linear stochastic system[END_REF]. The iLQR method is relying on linearizing the dynamics and approximating the cost function to quadratic form along the x trajectory. This control approach is briefly summarized in the next section.

Iterative Linear Quadratic Regulator (iLQR)

To mitigate the DDP computational time it is possible to locally linearize the dynamics and approximate the cost function to quadratic form around the current point in state space (along the x trajectory). This method is called the iterative linear quadratic regulator (iLQR) approach Li and Todorov [START_REF] Li | Iterative linearisation methods for approximately optimal control and estimation of non-linear stochastic system[END_REF] and we use the same formulation as in Kumar Hari Shankar Lal Das et al. [START_REF] Kumar | Controlling a multi-joint arm actuated by pneumatic muscles with quasi-ddp optimal control[END_REF].

The iLQR is initialized with a nominal control sequence and the corresponding state trajectory (x 0 , u 0 ). The dynamical system is then linearized as in Eq.(3.16) and the cost function is approximated by the quadratic form Eq.(3.17), eventually a local LQR problem is solved. Using this solution, the states and the control sequence are improved iteratively.

We denote the dynamic partial derivatives f u = ∂f (x,u) ∂u and f x = ∂f (x,u) ∂x . And the cost partial derivatives l u = ∂J(x,u) ∂u ,

l uu = ∂ 2 J(x,u) ∂u 2 , l x = ∂J(x,u) ∂x , l xx = ∂ 2 J(x,u) ∂x 2 , l xu = ∂ 2 J(x,u) ∂x∂u , h x = ∂h(x,u) ∂x and h xx = ∂ 2 h(x,u) ∂x 2 . δ x = f x δx + f u δu (3.16) ∆J = h T x δx(T ) + δx T (T )h xx δx(T ) + T 0 l T x δx + l T u δu + δx T l xx δx + δx T l xu δu + δu T l uu δu dt (3.17)
At every iteration, Eq.(3.16) and Eq.(3.17) are solved and (δx, δu) are deduced from the resolution of a modified Ricatti-type system. Then the new improved sequence is generated by x ← x + δx and u ← u + δu. As for a classical DDP, it is a succession of backward and forward phases (see Section 2.8.1). When ∆J ≈ 0, the iLQR converges thus giving an optimal control sequence u * ∈ U and the corresponding optimal state trajectory x * .

State space partial derivatives

This section details the state space partial derivatives needed by the iLQR algorithm, obtained from eq. (3.13). The sign() function is not differentiable in 0, therefore we have chosen to approximate the sign( q) term by an hyperbolic tangent: tanh (µ q), where µ = 1000. We obtain the following new equation for the dynamic of the system:

f (x, u) =       q 1 I RK m i m -F v q -F s tanh (µ q) + of f ) -(M Y sin q + M X cos q) g I       (3.18)
The partial derivative in u can be directly computed because the function is linearly dependent in i m :

f u = 0.0 1 I RK m T (3.19)
Concerning the derivative in x, we have non-linear dependencies between q and q, and q and q. Moreover the relationship between q and q is not explicit. Therefore we use the spatial finite difference discretization of the equation to obtain f x at each iteration i:

x i+1 = f (x i , u i ) (3.20) f x =           f ( q i + h/2 qi , u i ) -f ( q i -h/2 qi , u i ) h f ( q i qi + h/2 , u i ) -f ( q i qi -h/2 , u i ) h          
(3.21)

Cost function

This section presents the cost function used in our system and the cost partial derivatives needed by the iLQR algorithm, obtained from the equation (3.13). Considering the actual state vector x, the desired state vector x * , the actual command vector u and the actual torque on the elbow motor τ m , we use the following cost function: The weighting matrix Q, W, R and P have been chosen to give a hierarchy from the most important error to control to the lesser one. Using the lexicographical order we have: q τ m , τ m q, q q, q q u. Where c, c are the corresponding lower and upper mechanical limits. The tracking of the trajectory (in position) is the prioritized task, followed by the one controlling the torque bounds.

J = (x -x * ) T Q (x -x * ) + u T R u +C T s (x) W C s (x) + C T s (τ m ) P C s (τ m ) ( 3
We obtain the following cost partial derivatives l u = ∂J(x,u) ∂u ,

l uu = ∂ 2 J(x,u) ∂u 2 , l x = ∂J(x,u) ∂x , l xx = ∂ 2 J(x,u) ∂x 2 , l xu = ∂ 2 J(x,u) ∂x∂u : l u = 2Ru + 2 dC s (τ m ) T du W C s (τ m ) (3.24) l uu = 2R + 2 d 2 C s (τ m ) T du 2 W C s + dC s (τ m ) T du W dC s (τ m ) du (3.25) dC s (τ m ) du = λ(τ m ) 2
RK m e λ(τm)max(τm) -e λ(τm)min(τm) (3.26)

d 2 C s (τ m ) du 2 = λ(τ m ) 4 (RK m ) 2 C s (τ m ) (3.27) l x = 2Q (x -x * ) + 2 dC s (x) T dx W C s (3.28) l xx = 2Q + 2 d 2 C s (x) T dx 2 C s + dC s (x) T dx W dC s (x) dx (3.29) dC s (x) dx = diag  
λ(q) 2 e λmax(q) -e λmin(q) λ( q) 2 e λmax( q) -e λmin( q)   (3.30)

d 2 C s (x) dx 2 =         λ(q) 4 C s (q) 0.0 0.0 0.0 0.0 0.0 0.0 λ( q) 4 C s ( q)         (3.31)
with λ(τ m ) = 0.5, λ(q) = 10 and λ( q) = 1. These parameters have been chosen accordingly to the explanation on the λ of the section 3.4.2. The 'exponential barrier' on the position bounds will be sharper than the two others. It also depends on the difference between the limit and the current state. For instance the maximal difference in current is 12A whereas the one in position is 2.35rad, then the λ(τ m ) do not need to be as big as the λ(q).

Results

Identifying the thermal model

The room temperature (26 • C during the experiment) is never the same as the ambient temperature for the motor. Indeed, the robot drives release a lot of heat, heating the motor to approximately 43 • C even when it is unused. The experimental setup is the following: the robot is controlled in position, a sinusoidal command is sent to the elbow actuators and a heavy charge is put on the arm of the robot (Fig. 3

.6).

As represented in Fig. 3.3, during the heating sequence of the experiment, the identified function fits the measured temperature evolution. On the other hand, Figure 3.3: Thermal parameter identification in the cooling part, the estimated and measured curves tend to diverge. The heat of the drives may cause the problem. Indeed, the temperature of the drives also increases during the experiment. But the drives and the motor are really close to one another, meaning that the temperature of the drives can not be set aside while studying the temperature of the motors: it makes the motors cool down slower. Nevertheless, results show that holding 20 kg at 5 cm of the elbow is not a problem for this specific joint in terms of temperature. As the chain actuators are similar for each joint of the robot TALOS, one can conclude that overheating the actuators of the robot during experiments is unlikely. Overall the identified segment inertial parameters are similar to the ones provided by the manufacturer. These parameters can be considered well identified due to their relatively low standard deviation σ % . The relative standard deviation gives in % a confidence index on the reliability of the identification of each parameter. See Presse and Gautier [START_REF] Presse | New criteria of exciting trajectories for robot identification[END_REF] for the detail of σ % calculation. The joint torque sensor was well calibrated since its identified gain was K j =1.015. The total joint drive gain RK m value is not disclosed, because of a confidentiality agreement with the robot manufacturer, but a difference of 16% with the value provided by the manufacturer was found.

Identifying the chain drive and inertial parameters

Controlling the actuator

The iLQR is implemented as described in Forget et al. [START_REF] Forget | Implementation, identification and control of an efficient electric actuator for humanoid robots[END_REF] and completed by the 'exponential barrier' constraints in the cost function. The cost functions and model dynamics are evaluated using the identified parameters (Table .3.1). This implementation is first validated in simulation with the use of Gazebo and then tested on the real TALOS robot. The noticeable contribution of our solution compared to Forget et al. [START_REF] Forget | Implementation, identification and control of an efficient electric actuator for humanoid robots[END_REF] remains in this last point, the torque control is implemented on the robot and achieve a satisfying real time control (with more protections on the mechanical limits of the robot).

The experimental setup is the following: the robot is controlled in torque with high gains except for the elbow actuator. A sinusoidal command for the elbow actuator is sent to the iLQR algorithm which computes a resulting signal respecting the limits. This signal is then sent to the robot. A heavy charge is finally put incrementally on the arm of the robot, at 5cm of the elbow joint, until reaching 34kg (see Fig. 3.6). The load moment arm when the load is perpendicular to the robot arm is equal to 16.67 N.m. Otherwise it is expressed similarly to the mass of the arm at the center of mass (see Eq.(3.13)): LY sin q + LX cos q with LY and LX the first moment of inertia of the load.

The iLQR algorithm is executed on the robot with a 15ms preview window, a 3.3ms discretization and has an execution time of 300µs (using an Intel CPU i7-3612QE @ 2.1 GHz). On a standard laptop such as an Intel CPU i7-8850H @ 2.6 GHz it can be executed with a longer preview window (100ms) and with a faster control frequency (1ms discretization) in a smaller execution time (200µs).

3.5.3.a Simulation

Fig. 3.7 presents the action of the exponential barriers on the position joint limits (see Eq.3.14). A sinusoidal desired trajectory is given to track, but reaches the lower angular position limit: -2.35rad. The computed trajectory shows a plateau before reaching the limit, demonstrating the activation of the protection in the iLQR. We can notice the small oscillations at the top of the sinusoide, due to the dry frictions when the angular velocity reaches zero (see Fig. 3.11). This causes a small delay in the computed trajectory. Fig. 3.8 depicts the tracking results on two cases: first without additional load and second by adding a 30 kg load to the forearm. In the first case, the trajectory follows the desired one with a small delay (∼0.1s), which may be causes by the dry friction as explain above. This bias can be removed by increasing the state constraint gains but the system will loose compliance. In the second case, the delay with the desired trajectory increases, in particular during the ascending phase of the sinusoidal command (the robot raises its arm and the load). Indeed, to match the desired trajectory quicker, the torques needed are high and the iLQR algorithm limits them. Nevertheless, in both cases the overall motion shape is respected.

The control trajectories depicted in Fig. 3.9 show how the limitation of 6 A (set a-priori) is implemented with the exponential barrier given by Eq.3.14. The cyan trajectory represents the current command when there is no additional load and it does not reach the limit. The green trajectory represents the current command when the robot carries a 30kg load, it shows that the command reaches a plateau (around -5.6A) before the limit. It explains the delay of the computed angular trajectory of Fig. 3.8, the tracking of the desired joint trajectory is degraded because the current is limited. This is due to the iLQR action to preserve the actuator, i.e. the exponential barrier is activated by the iLQR algorithm.

In comparison, Fig. 3.10-bottom depicts the control going over the current limit in the loaded case when the current exponential barrier is not enabled in simulation. In this configuration the joint position tracking is better, as shown in Fig. 3.10-up, but the actuator reaches its current limit.

3.5.3.b Experiments

Fig. 3.11 presents the action of the exponential barriers on the position joint limits on the real robot. As in simulation, the desired trajectory reaches the lower angular position limit: -2.35rad. The computed trajectory shows a plateau before reaching the limit, demonstrating the efficiency of the protection in the iLQR. Notice the oscillations at the beginning of the sinusoidal movement (when the arm is raising), the arm of the robot has difficulties to perform a smooth trajectory. As thought in the simulations, this is the consequence of the dry frictions, when the angular velocity q is approaching zero, the dry coefficient F s increases to reach the Breakaway friction value (the sum of the Coulomb and static frictions). This behaviour can create the delay noticed in the simulations. This is not currently taken into account in our model but would be in the future. Fig. 3.12 shows a comparison between the desired joint trajectory and the ones obtained without and with a 34kg load (we increase the load to have a better demonstration of the activation of the current exponential barrier). As expected, the trajectory without any load has a small delay and oscillates when the angular velocity is around zero. The trajectory obtained when the robot is carrying the additional 34 kg displays greater oscillations due to the load movements. It also presents a bigger degradation of the trajectory than in simulation, the sinusoidal movement is reduced (stopped at -1.7rad instead of -1.9rad) because large torques are necessary to achieve it (but are prohibited by the iLQR algorithm, see Fig. 3.13).

The Fig. 3.13 presents the control trajectories computed by the iLQR without and with the load. In the first case (cyan line), the command is far from the limit and do not activate the barrier. In the second case (green line), as in simulation, the command reaches a plateau before the limit, around -5.3 to -5.6A. The current is more reduced than in simulation, inducing a more degraded state trajectory in Fig. 3.12. The plateau is not as smooth as in the simulation, due to the computation time which is quite high, leading to picks in the calculation. Indeed, in these experiments the computation time of the overall control system is around 1 to 1.5ms. Compared to the control frequency of the robot, which is 1kHz, this duration is large.

A video describing the experiments on the robot can be found at the following location: https://youtu.be/YNoSnU7w4FY.

Conclusion

In this chapter we presented the actuator model, the identification and the control of the TALOS robot elbow joint. The identification results of the chain drive and of the inertial parameters at once proved to be accurate with low standard deviation and physical consistency of the parameters. Using the identified model and a iLQR approach to avoid reaching its limits, we have demonstrated that the robot is able to carry a load up to 34kg with a sinusoidal motion at low speed. As expected, it is not possible to use this algorithm in the main CPU as it takes 300µs for one actuator, nevertheless, we validated the efficiency of the solution. The extension of this work would be to identify all the actuators of the robot and to implement the solution over high-performance dedicated and embedded electronics board attached to each actuator. The work presented in this chapter details the publication Ramuzat et al. [22]. This work has focused on the identification and modeling of the rigid chain actuators of the robot and how to use these models to protect the system using a specific low level joint-controller. The following chapters must deal with the higher problematic of whole-body control which provides the desired joint-torque needed by the low-level controller. This whole-body controller must compute a command following reference trajectories given by a planning algorithm; regulating the errors coming from the robot model and environment.

Introduction

One of the major issue involving the whole-body motion and control of humanoid robots is their locomotion. Bipedal locomotion of humanoid robots is still considered as a difficult problem, because of the complexity of the robots dynamics, the numerous constraints of the motion and the unknown environment. Indeed, to make a humanoid robot stand and walk it is necessary to find a solution involving balancing, contacts creation or removal and contacts stability while performing motions adapted to the environment. To achieve the locomotion of such complex systems, the problem can be decomposed in four stages (see Fig. 2.7): guide planning, contacts sequence generation, centroidal trajectory optimization and whole-body control. Some solutions for the planning stages have been presented in Section 2.4. Most of the trajectory planning methods use the centroidal dynamics to generate consistent behaviors for a legged robot. In addition, the concepts of Divergent Component of Motion (DCM) Englsberger et al. [START_REF] Englsberger | Three-dimensional bipedal walking control based on divergent component of motion[END_REF] and Capture Point (CP) Pratt et al. [START_REF] Pratt | Capture point: A step toward humanoid push recovery[END_REF] associated to reduced dynamics models such as the Linear Inverted Pendulum (LIPM) Kajita et al. [START_REF] Kajita | Biped walking stabilization based on linear inverted pendulum tracking[END_REF] allow to simplify the trajectory generation. Our work focuses on the last stage of the problem: the whole-body control. Most control architectures for legged locomotion are either torque or position controlled.

In the recent literature there is a growing number of implementation of torque control algorithms to solve locomotion problems Lee and Goswami [START_REF] Lee | A momentum-based balance controller for humanoid robots on non-level and non-stationary ground[END_REF], Koolen et al. [START_REF] Koolen | Design of a momentum-based control framework and application to the humanoid robot atlas[END_REF], Englsberger et al. [START_REF] Englsberger | Overview of the torque-controlled humanoid robot toro[END_REF], Herzog et al. [START_REF] Herzog | Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid[END_REF]. Indeed, due to the intrinsic compliance of the torque control formulation, it is more suitable for interactions with humans and for multi-contact problems where external interactions and several contact points are needed. However, the transition from the simulations to the real experiments are harder due to the necessity to take into account the model of the actuation chain of the robot Ramuzat et al. [22]. This chapter details the works realized to create whole-body controllers able to cope locomotion problems. Indeed, locomotion and manufacturing operations both consist in creating contacts and applying forces in a certain manner while balancing. The capabilities designed for the controllers to realize locomotion with the robot TALOS are the same one needed to achieve the force operations. Moreover, the Gepetto team core subject of study is the anthropomorphic motions, thus the team has already numerous solutions of complex locomotion trajectories. This allows us to quickly test our implementations on these problems. This chapter presents the different aspects and steps needed to create a complete solution. For instance, the DCM approach can be used in the control stage, for admittance control on the CoM Mesesan et al. [START_REF] Mesesan | Dynamic walking on compliant and uneven terrain using dcm and passivity-based whole-body control[END_REF], Caron et al. [START_REF] Caron | Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control[END_REF] to improve the stability of the robot (as described in Section 2.2.3.b). We successfully implement three controllers, two position controlled and one torque controlled.

We then investigate the differences and performances of our torque and position control schemes to choose the most appropriate one for the robot TALOS. We test the most promising controller on a specific application of human-robot collaboration for navigation. Finally, the results of the experiments performed on TALOS are presented.

The chapter is organized as follows: Sections 4.2 and 4.3 detail the formulations used for the position and torque QPs used by our whole-body controllers. Then, the next sections explain the process implemented to benchmark the controllers: Section 4.4 details the three task-space whole-body control schemes to be compared. Section 4.4.2 presents the planning methodologies and the state estimator used for the simulations. Section 4.4.3 describes the energy criteria used for the controllers benchmarking. Section 4.4.4 presents the simulations results and Section 4.4.5 discusses them. In Section 4.5 is detailed the application of our torque controller in the scope of a human-robot co-navigation, this work was realized in collaboration with Isabelle Maroger. Finally, the last section describes the intermediate experiments realized on the real robot and the difficulties we faced, using the presented controllers.

Inverse Kinematics Quadratic Program

The first controller created during this thesis is a Hierarchical Quadratic Programming (HQP) (see Section 2.5.2.c). It solves a task-based inverse velocity kinematics described in Mansard et al. [START_REF] Mansard | A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: The stack of tasks[END_REF] and outputs a velocity command. It implements a strict hierarchy between the tasks and has been implemented using the package SoT-Core [20]. For the remaining of the manuscript, the inverse velocity kinematics is directly called inverse kinematics or IK.

Motion Task

In this controller, the task errors e to be reduced in the cost function are implemented as velocity-based tracking laws in the Lie group SE [START_REF] Spenko | The DARPA Robotics Challenge Finals: Humanoid Robots To The Rescue[END_REF]. Having the robot coordinate vector q and its velocity v as control input, a task-function is a derivable function x(q) whose space is named the task-space. And the task errors e ∈ R l are expressed as in the Section 2.5.2.b: ė(q, t) = ẋ(q) -ẋ * (t) ẋ(q) = Jv (4.1) with x * , ẋ * the desired position and velocity of the task, l is the dimension of the task error (for instance, for the 3D acceleration error on the CoM l = 3), J = [J u J a ] = [T (S, B) ∂e ∂q ] = [T (S, B) ∂x ∂q ] the Jacobian according to the robot coordinates vector. J u is the "Jacobian" of the under-actuated part: it is the transformation T between the task/error frame S and the base frame B (see Eq.2.6).

A first-order linear dynamics is imposed on these errors (see Section 2.5.2.b): ė(q, t) = K P (x(q) x * (q)) ⇔ ẋ(q) = ẋ * (t) + K P (x(q) x * (q)) (4.2)

with the difference operator of Lie group.

Contact Constraints

In SoT-Core, the contact constraints are implemented as tasks with the higher priorities. The velocity of the task is null and the position is kept constant, i.e. for a contact point x c and its Jacobian J c :

x i c = x i+1 c ẋc = 0 ⇔ J T c v = 0 (4.3)

Quadratic Programming Formulation

Using the similar formulation as Section 2.5.2.b and Section 2.5.2.c, with g i = w i for each task i ∈ N , it is possible to formulate the QP formulation of our controller as follows:

g * i = min v,g i g i s.t. J T c v = 0 g j = g * j ∀j < i (4.4)
with J c the Jacobian associated to the c contact points X c = {x c1 ...x cc }. Here we differentiate the contact tasks from the other one but they can be also put as function g 0 with the higher priority 0. This strict hierarchy can be solved by using the orthogonal projection operator P on the null-space of the Jacobian J. The tasks with lower priorities are solved at best in the remaining space of the previous tasks solutions. With J † the pseudo-inverse of J, one can write for N tasks such that e 1 ≺ e 2 ≺ ... ≺ e N :

q = J † 1 e 1 + N k=2 J † k ( k-1 l=1 P l )e k (4 .5)
In SoT-Core, this QP is solved using the method presented in Mansard and Chaumette [START_REF] Mansard | Task sequencing for sensor-based control[END_REF].

Inverse Dynamics Quadratic Program

The

Motion Task

Most of the task-function errors of TSID are expressed as motion task errors e. They are implemented as acceleration-based tracking laws in the Lie group SE(3). Having the robot coordinates vector q and the acceleration a as control input, a task-function is a second-order derivable function x(q) whose space is named the task-space. And the motion task errors e ∈ R l (with l the dimension of the error task for instance 3 for the CoM task) are expressed as in the Section 2.5.2.b: ë(q, t) = ẍ(q) -ẍ * (t) ë(q, t) = (Ja + Jv) -ẍ * (t) (4.6) with x * , ẍ * the desired position and acceleration of the task, J the Jacobian according to the robot state vector as defined previously and in Eq.2.6.

The following dynamics is imposed on these errors (see Section 2.5.2.b):

ë(q, t) = K P (x(q) x * ) + K D ( ẋ(q) -ẋ * (t)) ⇔ ẍ(q) = ẍ * (t) + K P (x(q) x * (t))+ K D ( ẋ(q) -ẋ * (t)) (4.7)
with the difference operator of Lie group. To simplify notations, in the following, we will use the usual difference operator instead of and we will not write the state or time dependencies of the terms. Using Eq.4.6 and Eq.4.7 leads to the generic formulation for a motion task-function:

J O a y = ẍ * -Jv + K P e + K D ė o (4.8)

Angular Momentum Task

The Angular Momentum (AM) dynamics is expressed in TSID using the equation defined in Section 2.2.4, recalled here:

kc = k * c + K Pam (k * c -k c ) (4.9)
with kc the commanded rates of change of the angular momentum and K Pam the proportional gain for the AM task. The linear component follows the motion task defined in Eq.4.8, called CoM task.

Contact Constraints

The inverse dynamics of the robot is solved in rigid contact with the environment Herzog et al. [START_REF] Herzog | Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid[END_REF], thus, these contacts constrain the motion. They are implemented in TSID as nonlinear functions, which are differentiated twice (with J c the Jacobian at the contact point). The Eq.2.52 is recalled here for clarity:

c(q) = 0 ⇔ Contact point does not move J T c v = 0 ⇔ Contact point velocity is zero J T c a + Jc v = 0 ⇔ Contact point acceleration is zero (4.10)
The equation of the dynamics is formulated as an equality constraint with τ ext replaced by the forces applied at each contact, and these forces are optimized (see Eq.4.14). In this chapter, we use the unilateral rectangular plane contact formulation of TSID. The contact is defined as a 6D motion task (Eq.4.8), an inequality constraint on the force bounds and a direct force task:

T T f = f * (4.11)
with the force-generator matrix T T ∈ R 6×12 mapping the forces at the vertices of the contact surface, f ∈ R 12 (four 3D forces), to a 6D resultant force at the contact point (center of the surface). f * ∈ R 6 the desired force for the contact. T creates the map between the two representations: τ contact = J T T T f . However in our formulation we do not use this direct force task, letting the QP optimize freely the forces at the contact, except for the Cartesian Force-Contact case.

Cartesian Force-Contact task

The context of my thesis is the validation of a solution for manufacturing operations. These operations, mainly drilling or tightening, need precision while applying a certain amount of force on a structure. To achieve these operations with our developed whole-body controllers we need to implement a new force task. The targeted application is the drilling of a metal plate, requiring the application of huge forces (∼ 600N). Following the study of the state-of-the art realized on the force control for manipulations (see Section 2.7), and based on the formulation of the tasks in TSID, we choose to implement a parallel force/position control. This task is composed of two parts: the force application and the position control of the contact. Thus, it combines two tasks, a motion task defined in Eq.4.8 for the control of the position, and a force task for the application of the force.

The force task is defined as a Proportional Integral with a feed-forward (PI+) and an anti-windup, inspired by Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF]:

e f = F * -F ext T T F O F F y = F * + P F e f + I F t 0 e f ds o (4.12)
with F * , F ext ∈ R 6 , respectively the desired and measured or estimated external wrenches and P F , I F ∈ R 6×6 the task gains. T F is the force-generator matrix defined as in the previous paragraph and F F is the constrained wrench.

In TSID, as explained in Eq.2.52,4.11, the forces in the equation of the dynamics are expressed at the contacts. Thus, the force task is linked to its appropriate contact task (see previous paragraph, Section 4.3.4). In the library, the task is called taskcontact-force, it implements the force task of Eq.4.12 and the position control part of the task is done by the 6D motion task of the associated contact (Eq.4.8).

Thus, we implemented a parallel force/position control which also respects the coulomb constraints as the position control is done by the contact constraint of the previous section (Section 4.3.3). Considering that our system has c contacts and the Cartesian Force-Contact task, we can replace τ ext by:

τ ext = c k=1 J T k T T k f k -J T F T T F F F (4.13)
which is in fact the J T c F term defined in Eq.2.4.

Quadratic Programming Formulation

TSID sums all the task functions g i (y) using user-defined scalar weights λ i , ∀i ∈ N tasks, subject to constraints such as the rigid contacts and the dynamics. It leads to the same formulation defined in Eq.2.57 replacing ω i with the tasks functions (Eq.4.8):

g i (y) = O i y -o i 2 min y=[a,f ] N i=0 λ i g i s.t.   J c 0 0 M -J T c -N T     a f τ   =   -Jc v -h   τ min ≤ τ ≤ τ max q min j ≤ q j ≤ q max j qmin j ≤ qj ≤ qmax j f min ≤ f ≤ f max (4.14)
Where the free variables are the acceleration a and the force F . The QP is formulated to solve the acceleration and the forces and to then retrieve the torque using Eq.2.4. It has only two strict layers: the constraint layer (priority 0) and the cost layer (priority I).

The solver used in TSID is eiquadprod [198], it implements the algorithm of Goldfarb and Idnani Goldfarb and Idnani [START_REF] Goldfarb | A numerically stable dual method for solving strictly convex quadratic programs[END_REF] for the solution of a convex QP problem by means of a dual method.

For x = [a, F ], the problem is of the form :

min x 1 2 x T Gx + g 0 x s.t. CE T x + ce 0 = 0 CI T x + ci 0 ≥ 0 (4.15)
with G the symmetric Hessian of the Cholesky factorization of the cost function, g 0 the Lagrange multipliers of the cost function, CE the generalized matrix of the equality constraints and ce 0 their associated vectors and similarly CI the generalized matrix of the inequality constraints and ci 0 their associated vectors.

Comparison of the Control Schemes

This section focus on the implementation and comparison of three real-time wholebody controllers using the task-function approach Samson et al. [START_REF] Samson | Robot control. the task-function approach[END_REF], Escande et al. [START_REF] Escande | Hierarchical quadratic programming: Fast online humanoid-robot motion generation[END_REF]. The objectives to be performed by the robot are expressed in their respective task spaces, using reference trajectories given by the planning. Complex motions combine several nonlinear tasks and constraints. Quadratic Programming problems (QP) are instantaneous optimization techniques used to solve such nonlinear problems, employing the whole-body kinematics or dynamics of the robot. In this section two types of QP are compared, a Hierarchical QP which imposes a strict hierarchy between the tasks Hoffman et al. [START_REF] Hoffman | Multi-priority cartesian impedance control based on quadratic programming optimization[END_REF], Henze et al. [START_REF] Henze | An approach to combine balancing with hierarchical whole-body control for legged humanoid robots[END_REF], Herzog et al. This section intends to follow the benchmarking of humanoid robots control architectures Romualdi et al. [START_REF] Romualdi | A benchmarking of dcm based architectures for position, velocity and torque controlled humanoid robots[END_REF]. It contributes toward the implementation and comparison of three whole-body control schemes: two using position control associated with DCM and CoM admittance controls and one using torque control. The first one is based on a Hierarchical QP using Inverse Kinematics (denoted IK in the section), the second and the third one use a weighted QP with Inverse Dynamics and an Angular Momentum (AM) regularization task (denoted respectively TSID position and TSID torque). They are evaluated in Gazebo simulations on three locomotion problems: walking on flat terrain, walking on uneven terrain and climbing stairs, on the criteria of trajectory tracking, energy consumption, passivity and computational cost. In particular, the proposed torque control implementation allows the TALOS robot to reach a 0.6m/s walk on flat floor, which is the highest walking velocity achieved on the TALOS robot.

Control Schemes

4.4.1.a Lexicographical Quadratic Programming

The first controller used is a Hierarchical Quadratic Programming (HQP) task-based inverse kinematics described in Mansard et al. [START_REF] Mansard | A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: The stack of tasks[END_REF] (see Sectin 4.2).

Inverse Kinematics QP: IK -This control scheme is based on a DCM controller (Eq.2.30), a CoM admittance controller (Eq.2.31) and a Lexicographic QP solving the inverse kinematics of the robot. The authors have implemented this scheme in The tasks used during the simulations are defined in Table .4.1 (the priority 0 is the highest one). The respective task gains are defined in Table 4.3. The weights and gains have been chosen through trials and errors with an a priori heuristic. This control scheme is presented in the Fig. 4.2, the IK block in purple represents the Lexicographic QP solving the inverse kinematics of the robot. The Eq.2.30 is implemented in the DCM Ctrl blue block and the Eq.2.31 in the CoM Admittance Ctrl one. A complete scheme of the framework is presented in Appendix 3, in the Fig. 6.5.

4.4.1.b Task Space Inverse Dynamics (TSID)

In this section are described the two schemes based on TSID (see Section 4.3) compared on locomotion challenges on the robot TALOS.

Inverse Dynamics WQP: TSID Position -This control scheme is based on a DCM controller (Eq.2.30), a CoM admittance controller (Eq. implements an AM task, which regulates the angular momentum to 0, using the formulation of Eq.2.32. The authors have implemented this controller using the TSID [80] library in the same package than the controller TSID Torque, with the DCM and CoM admittance controllers.

The tasks considered during the simulations are defined in Table .4.2. The priority 0 corresponds to a constraint and the priority I to a task in the cost function. The respective task gains are defined in Table 4.3. The weights and gains have been chosen through trials and errors with an apriori heuristic. This control scheme is presented in the Fig. 4.2, the TSID block in purple represents the WQP solving the inverse dynamics of the robot. The Eq.2.30 is implemented in the DCM Ctrl blue block and the Eq.2.31 in the CoM Admittance Ctrl one. A complete scheme of the framework is presented in Appendix 3, in the Fig. 6.5.

Inverse Dynamics WQP: TSID Torque -This control scheme is based on a WQP solving the inverse dynamics of the robot (with an AM regularization task, using the formulation of Eq.2.32), as shown in Fig. 4.3. From the desired acceleration computed by the QP, TSID retrieves the associated torque by using the robot equation of the dynamics. The authors have implemented this controller using the TSID [80] library in the open-source package [START_REF]talos-torque-control[END_REF].

The tasks considered in the simulations are the same as TSID position, with different gains (see Table 4.3). A complete scheme of the framework is presented in Appendix 3, in the Fig. 6.5.

4.4.1.c Remark on the state feedback

For position control, it is needed to integrate the result of the QP (one time for IK and two times for TSID position, see Fig. 4.2) to obtain the desired command. To avoid instabilities, the control loop of both QP use these integrated values in the next iteration instead of the measured ones. The measured position and velocity of the robot are only used to compute the CoM, DCM and ZMP for the admittance control in the position schemes. In contrary, the torque control scheme uses the measured values at each iteration of the QP (see Fig. 4.3) and in particular the position and velocity of the robot base (or free-flyer).

Locomotion Planning and Estimator

In this section the two locomotion planning methods presented in Section 2.4 have been used. First, the WPG has computed the trajectories for the walk on flat floor simulations (20cm and 60cm), with pre-defined set of foot steps. The WPG is plugged to the whole body controller using the dynamic-graph package in order to get the next reference point of the trajectories in real-time. Secondly, the multicontact-locomotion-planning is used to obtain the trajectories for the platforms and stairs simulations. Here the references are computed offline and then a file containing the trajectories is read during the simulation.

Finally, for every simulations, the estimation of the base (free-flyer) of the robot has been realized using the approach presented in Flayols et al. [START_REF] Flayols | Experimental evaluation of simple estimators for humanoid robots[END_REF]. The free-flyer information are estimated using robot joints configurations and velocities, the IMU and the force sensors of the robot. It allows the whole-body controller to have access to the whole state vector, including the position and velocity of the base, necessary to compute the CoM on an under-actuated system.

Energetic Comparison Criteria

4.4.3.a Energy cost

Based on Torricelli et al. [START_REF] Torricelli | Benchmarking bipedal locomotion: A unified scheme for humanoids, wearable robots, and humans[END_REF], a relevant criterion to compare the energy consumption of the control schemes is the cost of transport. It can be computed as the energetic cost of transport C et using the whole mechanical work of the actuation system E m or as the mechanical cost of transport C mt using only the positive one E m+ . [START_REF] Mansard | A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: The stack of tasks[END_REF]) with m the mass of the system, g the gravity constant, D the distance traveled by the system and τ i , ω i the respective torque and velocity of each robot joint for all (N ) joints.

C et = E m mgD C mt = E m+ mgD E m = T 0 N i=0 |τ i (t) ω i (t)|dt (4.

4.4.3.b Passivity Gait Measure

Another interesting energetic criterion is the ability to minimize joint torques to increase the passivity of the walk Torricelli et al. [START_REF] Torricelli | Benchmarking bipedal locomotion: A unified scheme for humanoids, wearable robots, and humans[END_REF]. The Passivity Gait Measure (PGM)Mummolo and Kim [START_REF] Mummolo | Passive and dynamic gait measures for biped mechanism: formulation and simulation analysis[END_REF] quantifies the passivity of a biped walking motion:

P GM = 1 - RM S(τ sa ) RM S(τ tot ) (4.17) RM S(τ tot ) = T 0 N i=0 τ i (t) 2 dt T (4.18)
where RM S is the Root Mean Square along the period of time T , τ sa stands for the torque on the stance ankle joint and τ tot for the torque on all robot joints.

Simulation Results

The simulations presented in this section have been made using Gazebo.

A video illustrating the simulations is available at the following link: https://peertube.laas.fr/videos/watch/4b5d3a5b-2355-47a0-8197-f41ed4f885c6.

The chosen simulations are walking on flat or uneven terrains and stair climbing.

Based on Torricelli et al. [START_REF] Torricelli | Benchmarking bipedal locomotion: A unified scheme for humanoids, wearable robots, and humans[END_REF], they cover different aspects of locomotion skills for a stationary environment with and without unexpected disturbances.

4.4.4.a Straight walk of 20 cm steps

In the simulation, the robot executes 6 steps forward at 0.2m/s and a final step (traveled distance of 1.2m). The time distribution is 0.9s for single support phase and 0.115s for double support phase (leading to steps of approx. 0.20m). The controllers have also been successfully tested on a faster walk with single/double support time of 0.711/0.089s. The Fig. 4.4 presents a comparison of the three control schemes on their estimated ZMP, on the sagittal (x-axis, top curves on the figure) and lateral (y-axis, bottom curves) planes only, because the desired height of the CoM is constant. Similarly Fig. 4.5 provides the results of the DCM estimation and tracking. Fig. 4.6 shows the forces applied on the ground along the z-axis on the left foot. The tracking of the CoM and the feet are accurately followed by the three controllers (tracking error lesser than 1cm).

The two position controllers achieve similar results, tracking correctly the ZMP reference of Eq. 2.30 and thus the DCM, with an average error of 2cm (see Table 4.4). Noticeably, the torque control presents a ZMP which is close to the position control results in Fig. 4.4 even though there is no explicit control on the ZMP nor the DCM. In the Tables presenting the error on the ZMP, for the torque scheme, the estimated ZMP is compared to the desired ZMP (from the planning). In particular, in the lateral plane, the error is quite low, 1cm in average. The Fig. 4.6 illustrates the ground impacts problem in position control compared to the better foot landing observed in torque control. Indeed, each time the left foot comes into contact with the ground (1.5s, 3.5s,...), the IK and TSID position schemes show peaks in the foot force (∼ 400N) which are avoided in TSID torque. This explains also the peaks in the ZMP errors (around 15cm) because during an impact the foot bounces on the ground. The force oscillations of the IK and TSID position controllers when the foot is in the air are due to the high control gains on the ankle (PID gains of the low-level position control in Gazebo), it is mainly noises.

4.4.4.b Straight walk of 60 cm steps in torque control

In Mesesan et al. [START_REF] Mesesan | Dynamic walking on compliant and uneven terrain using dcm and passivity-based whole-body control[END_REF] the humanoid robot TORO successfully performed a walk on flat terrain with a step length of 55cm (single/double support time of 1.1/0.4s). In the following simulation, the torque controller is pushed to its limits to show its capability to achieve a similar result. The robot TALOS executes 6 steps forward of 0.6m/s and a final one to go back to the initial position. The time distribution used is of 0.9s for single support phase and 0.115s for double support phase (leading to steps of approx. 60cm). 4.5); the ZMP and DCM estimations. The feet tracks well the desired trajectories along the y-axis (maximum error of 6mm) however, along the x-axis, they show some delay (maximum error of 6cm). Thus, it induces greater tracking errors on the x-axis for the CoM (peaks of 5cm along the x-axis and 1.5cm along the y-axis). One can notice that the DCM and ZMP along the x-axis are more stable, whereas along the y-axis they present large oscillations (which are caused by the feet impacts on the ground when landing).

In Fig. 4.8, the AM behavior is shown along the three axes. The AM task minimizes the momentum to zero. The x and y momentum components are the most solicited, leading to the inclination of the torso forward and backward and to important moves of the arms to compensate the delay of the CoM and succeed the 60cm steps. The authors observed that without this AM task, the walk cannot be achieved.

Axis Average Standard deviation Peaks CoM

x-axis 0.018m 0.013m 0.050m y-axis 0.004m 0.003m 0.015m Left Foot

x-axis 0.014m 0.013m 0.063m y-axis 0.001m 0.001m 0.005m Right Foot x-axis 0.016m 0.016m 0.063m y-axis 0.001m 0.001m 0.006m

4.4.4.c Walk on the tilted platforms: Uneven terrain

In this third simulation, the robot walks on tilted platforms which represent uneven terrain (Fig. 4.1). This walk is achieved using the multicontact-locomotion-planning trajectories (see Section 2.4.2). The framework ensures the stability of the controllers on non-flat terrain when the feet are tilted.

Figure 4.9 illustrates the tracking performance of the controllers. The ones in position present the largest oscillations as TSID torque is the most stable (see Table 4.6). Both the IK and the torque control show oscillations at t ≈ 18s; it corresponds to the worst case where the robot has its two feet tilted to keep its balance on two opposite platforms leading to small slippages of the feet (this behavior can be observed in the linked video). These oscillations are larger in the case of the IK scheme. Figure 4.9 illustrates the tracking performance of the controllers. The ones in position present the largest oscillations as TSID torque is the most stable (see Table 4.6). Both the IK and the torque control show oscillations at t ≈ 18s; it corresponds to the worst case where the robot has its two feet tilted to keep its balance on two opposite platforms leading to small slippages of the feet (this behavior can be observed in the linked video). These oscillations are larger in the case of the IK scheme.

Finally the same result of the 20cm walk on the contact forces is obtained in this simulation, as described by Fig. 4.10, there are impacts on the platforms in position control. Due to the high gains on the DCM to avoid the slippage of the robot, the IK control presents bigger peaks of force. Increasing the gains on the feet only generates more instability, but raising the ones on the DCM and admittance control lessen the oscillations (at the cost of a more rigid behavior).

4.4.4.d Climbing Stairs

In the last simulation the robot is climbing 6 stairs of 10cm height and 30cm long (see Fig. 4.1). The trajectories are planned with the multicontact-locomotion-planning. Fig. 4.11 shows the ZMP evolution of each controllers, where the result is similar to the uneven terrain simulation. The TSID torque scheme behave significantly better than the others, with a ZMP matching the one planned (errors lesser than 1cm, see Table 4.7). Noticeably, the IK scheme presents higher oscillations at the end of the move in the lateral plane. The robot ends displaced on the right compared to the desired trajectories, due to slippages of the feet when it finishes to climb a stair (shown in the linked video).

4.4.4.e Energy cost and Passivity Gait Measure

The results obtained for the cost of transport of the four simulations are presented in the Table 4.8, depending on the control scheme. The results obtained for iCub in Romualdi et al. [START_REF] Romualdi | A benchmarking of dcm based architectures for position, velocity and torque controlled humanoid robots[END_REF] are also presented for comparison (computed using Eq.4.16), as the human ones. The lower the energy consumption is, the better, and similarly, getting closer to the human cost of transport is an improvement.

0
Compared to the results obtained on iCub, the control in torque has a similar cost for the 20 cm steps simulation. However, the cost of the position controllers presented in this section is higher, because of their higher gains. The human efficiency is closer to the torque control, walking with a C et around 0.2J/kg/m Collins et al. [START_REF] Collins | Efficient bipedal robots based on passive-dynamic walkers[END_REF]. Noticeably, the energy costs in torque for the tilted platforms and stairs trajectories are still less important than the simpler walk in position; the C mt never exceeds 1, even for the 60 cm walk. Overall, the controller TSID position consumes less energy than the IK.

The Passivity Gait Measure comparison of the different simulations is reported in Table 4.9 for three gait stages: Single Support (Single S. corresponding to the stance ankle), Double Support (Double S.) and Flying Foot (Flying F. where the foot has no contact with the ground). The human results is given as an indicator Mummolo and Kim [START_REF] Mummolo | Passive and dynamic gait measures for biped mechanism: formulation and simulation analysis[END_REF], the robot behavior is expected to be similar during double support and flying foot phase where the ankle should be passive.

The results of the position control schemes show a behavior which is the opposite of the human one. The passivity of the ankle is higher during the stance phase because of the control of the ZMP which minimizes the ankle torque. And it is weaker during the double support and flying phases, due to the high PID gains of the low-level position control.

The control scheme in torque shows much more passive behavior (except on the stance foot), with a completely passive foot during the flying phase. During the double support phase, the ankle is almost passive (P GM ∼ 0.9) which is close to the human result. These results are better than the one expected in Mummolo and Kim [START_REF] Mummolo | Passive and dynamic gait measures for biped mechanism: formulation and simulation analysis[END_REF], where the torque controlled robot has a higher control on its stance ankle (P GM = 0.2).

Finally, on the uneven terrain, the double support phase corresponds to the worst case where the robot has its two feet tilted to keep its balance on two opposite platforms. This leads to a greater actuation than on flat floor (decreasing the passivity). Similarly, the stance phase corresponds to the left support phase on the final platform (highest slope), also leading to a bigger actuation of the ankle.

4.4.4.f Execution time of the control schemes

The computational time obtained during the execution of one control loop of the three schemes are presented in Table 4.10, according to the simulations. The computational time of the IK is better due to the computational efficiency of the null space projectors of the tasks. Exploiting this specific structure allows it to keep its control frequency higher than 1kHz in average with 4 hierarchy levels.

In TSID this method can only be used once because it is composed of two strict layers: the constraints and the cost.

Discussion

For the PGM results of the position schemes, the authors believe that adding an admittance control Caron et al. [START_REF] Caron | Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control[END_REF] on the ankle orientation may improve the results. If added, one can expect an increase of the actuation of the ankle during the stance phase, leading to a smaller PGM value.

In general the IK scheme presents higher oscillations and slippages when adding contacts. The authors think that this issue is mitigated in TSID position because it separates the feet task into a contact task and a tracking task. It allows to have different gains depending on the context (contact or not), indeed, the TSID schemes have higher gains for tracking tasks than for the contact ones.

One major point to discuss is the transition from the simulations to the real experiments. For torque control, in the Gazebo simulator, the joint torque control is almost perfect because the dynamics of the motor is completely neglected. However, not taking these dynamics into account will lead to unrealistic and dangerous behaviors on the real robot. The authors are currently testing the simulations on the simulator of the TALOS constructor, PAL robotics, which models the actuator dynamics of the robot. To do so, the torque control scheme is plugged to the constructor low-level controller, which computes a new command respecting the actuator dynamics. Only some tuning of the task gains seems to be needed to achieve good results.

Another concern is that the real robot is subject to imperfections such as errors in the chain actuation model, sensor noise, limited torque bandwidth or delays. In the presented implementation some actions against the possible problems have been implemented Englsberger et al. [START_REF] Englsberger | Torque-based dynamic walking -a long way from simulation to experiment[END_REF], such as adding filters on the signals retrieved from the robot and using the force sensors on the feet to improve the robustness of the base-estimator. Moreover, a PD+ controller has been implemented for the torque controller, to stiff the ankles, improving the quality of the feet landing. However, in simulation, this PD+ only increases the rigidity because of the feedback in position and velocity, and decreases the PGM, thus it has not been presented.

For position control, the simulations are dependent of the low-level control PID of the Gazebo plugin. These gains however have been modified to fit the real behavior of the motors. Thus, the position controllers should not need important modifications to be tested on the real robot. Yet, because the robot TALOS presents a flexibility in the hips which is not measured by the encoders, the simulations cannot be entirely realized on the real robot. The Appendix 4 describes the solutions developed to compensate this non-measurable hip flexibility of the robot TALOS (mentioned in Section 1.4). This flexibility leads to errors in the landing positions of the feet which cannot be compensated without a proper identification and modeling of the flexibility. In torque control however, this issue is mitigated because the flexibility is considered by the control system as external disturbances. Nonetheless, to achieve the experiments, it will be necessary control this flexibility; the solution presented in Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF] is currently tested on the robot TALOS and our controllers will be adapted to take it into account. It is important to mention that the final real robot implementations will require slightly different gains and weights.

Conclusion

Three whole-body control implementations are compared in this chapter. Two of them are position based (with DCM and CoM admittance control): a Lexicographic QP using inverse kinematics and a WQP using TSID with an AM task. The last one is a WQP using TSID in torque with an AM task. They are evaluated in Gazebo on flat, uneven terrains and stairs climbing; on the criteria of trajectory tracking, energy consumption, passivity and computational cost.

In general, both position control schemes present the same results, with less energy consumption and higher passivity for the TSID position controller. A better tuning of the tasks gains may improve its results on the ZMP tracking.

On the other hand, the TSID torque controller shows better results in terms of smoothness of the trajectory tracking, energy consumption, passivity of the walkwithout impacts and can achieve a 60cm walk with steps of 1s in simulation. This confirms the high capabilities of a torque control scheme coupled with an angular momentum regularization (see for instance Atlas in DARPA robotics challenge Koolen et al. [START_REF] Koolen | Design of a momentum-based control framework and application to the humanoid robot atlas[END_REF]). In average, the TSID controllers reach the 1kHz of control loop, necessary for real-time control, nonetheless, the IK scheme has the best computational time.

For our future work, we plan to control the hip flexibility of TALOS based on the identification work done in Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF], so that we can evaluate the three controllers on the real robot.

Application on Human-Robot Collaboration

for Navigation

Introduction

This section exposes the efficiency of the previous whole-body torque controller in a context of human-robot collaboration. It presents the results obtained during a collaboration with another PhD student of the Gepetto team, Isabelle Maroger, where the robot proactively walks alongside a human. The resulting paper Maroger et al. [24] is part of a French project, called ANR-COBOT, which aims at a collaboration between a human and a TALOS humanoid robot to carry and move a table . To smoothly achieve such a handling task, a good knowledge of the human dynamics while walking with a table is needed to allow the robot to anticipate and proactively follow the human movements. For now, the problem of walking with a table has been put aside and the issue has been reduced to the proactive tracking of a walking human by a humanoid robot. Thus, the paper focuses on the real-time prediction of human walking trajectory and its coupling with a Walking Pattern Generator (WPG) for a TALOS humanoid robot.

The paper presents a prediction model of human trajectory during gait and its coupling with a WPG which generates a TALOS humanoid robot CoM and footsteps along the predicted trajectory. This coupling allows a proactive tracking of a walking human by a humanoid robot, instead of a passive one, in order to ease every potential interactions during gait. The whole framework is assessed in simulation on Gazebo, as Fig. 4.12 shows, and is described on Fig. 4.13. This framework is designed to help a humanoid robot to predict the spontaneous human trajectory during gait according to its real-time CoM observation. This aims to allow the robot to move in accordance with its human partner. Thus, if the human accelerates, slows down or stops, the robot should act similarly while following the same trajectory as its partner.

This section focuses on the part of the paper corresponding to my work. It consists in the adaptation of the torque whole-body controller presented in Section 4.4.4 to successfully simulate the coupling of the WPG with the prediction model on the robot TALOS. It is represented by the green box. To achieve the simulations with the human-like trajectories, tasks gains and weights tuning were necessary and some of the trajectories cannot be performed because of their particularities. Some of them include wide lateral steps which lead to important swing of the torso, this behavior imbalance the robot and is the blocking point preventing our controller to achieve all the trajectories in simulation.

Scenarios

Three scenarios are considered in the paper:

1. The robot walks behind the human at a constant distance d behind .

2. The robot and the human act synchronously so that no distance divides them. Experimentally, this scenario can be achieved by putting the robot and its human partner side by side to show that the robot is mimicking the human motions without any delay.

3. The robot walks ahead of the human at a constant distance d ahead .

In all those scenarios, the robot is trying to follow the same trajectory as the human.

Those scenarios are represented on Fig. 4.14: the green and yellow curves display the whole past human trajectory, in stippled green the unknown future human trajectory and in purple the predicted trajectory.

Framework Description

4.5.3.a Prediction Model

To make the robot anticipate the human behaviour during gait, a model which allows to predict where a human is going using its past trajectory is needed. The model used in Maroger et al. [24] is based on the OCP model described in Maroger et al. [START_REF] Maroger | Inverse optimal control to model human trajectories during gait[END_REF] which had shown to generate paths which fit well human CoM trajectories. The OCP model is solved with a DDP solver Tassa et al. [START_REF] Tassa | Control-limited differential dynamic programming[END_REF] from the Crocoddyl library Mastalli et al. [START_REF] Mastalli | Crocoddyl: An efficient and versatile framework for multi-contact optimal control[END_REF].

The prediction process needs a human trajectory as an input. We used trajectories of 10 healthy subjects walking from 40 starting positions to one goal position in front of a table collected as part of a study of human walking trajectories Maroger et al. [START_REF] Maroger | Walking human trajectory models and their application to humanoid robot locomotion[END_REF] Maroger et al. [START_REF] Maroger | Inverse optimal control to model human trajectories during gait[END_REF]. To simulate a human walking, the recorded trajectory was sent at a given rate, simulating the human velocity, to the prediction process thanks to a ROS framework Stanford Artificial Intelligence Laboratory et Two parameters are fundamental in the prediction process: the amount of measurements needed before being able to correctly predict the human future trajectory and the sliding window size where the prediction process is done.

4.5.3.b Walking Pattern Generator

Then, as the future trajectory of the human can now be computed thanks to the prediction model described in the previous section, only the robot gait remains to be generated to perform a proactive tracking of a human. In Maroger et al. [24], a new WPG is designed to generate the CoM trajectory and the footsteps for a TALOS humanoid robot along the predicted trajectories. It is based on the Non-linear Model Predictive Control (NMPC) developed in Naveau et al. [69] which simultaneously solves the CoM and footsteps position and orientation problems. There are two main differences between the two NMPC:

The controller of Maroger et al. [24] is designed to track a reference trajectory rather than a reference velocity which implies a different cost function in comparison with Naveau et al. [69].

A terminal constraint on the Capture Point (CP) has been added in Maroger et al. [24] to avoid the internal instability issue in the problem formulation.

4.5.3.c Coupling of the Prediction Process and the WPG

The purpose of the paper Maroger et al. [24] is to design a whole architecture that allows a humanoid robot to smoothly follow a human who walks to an a priori unknown goal while predicting his future trajectory. This tracking is aimed to be proactive as a prediction process estimates the human future trajectory so that the future steps of the robot planned by the WPG are already in the right direction. This may cancel the reaction time of the robot and make the tracking smoother. The coupling of the prediction process and the WPG for a TALOS humanoid robot described in Sec.1.4 is a first step towards this purpose. This coupling can be described as follows, it corresponds to the blue boxes represented in Fig. 4.13. First, a pre-recorded human trajectory is streamed and retrieved by the prediction process. Then, it sends the predicted trajectory to the WPG which computes, thanks to a NMPC, the robot CoM and feet positions and orientations over the preview horizon with respect to the human velocity (the CoM orientation is in fact the robot base one). All those data are shared through a ROS framework and displayed on the RViz software as shown on Fig. 4.15 for a specific trajectory. In this figure, the chosen scenario is the second one on Fig. 4.14, namely the synchronization of a robot and a human. In the next section, the last steps of the process are detailed, namely the whole-body controller (the green box on Fig. 4.13) and the simulations on the TALOS robot on Gazebo are presented.

All the results and simulations resulting of this coupling are reproducible as all the libraries are open-source and the source code and the data are available on: https://github.com/imaroger/human_walking_trajectory_prediction.

4.5.3.d Whole-body controller

The whole-body controller computes a stable command from the reference trajectories and the actual state of the robot. In the paper Maroger et al. [24], we used the controller described in Section 4.4.1.b: it is a Weighted Quadratic Program (WQP), which solves the inverse dynamics of a robot in rigid contact with the environment Herzog et al. [START_REF] Herzog | Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid[END_REF]. It has been successfully tested on the humanoid robot TALOS in simulations in Ramuzat et al. [23], the results are presented in Section 4.4.4. The controller takes as inputs the CoM position, the base (free-flyer) orientation and the feet positions and orientations reference trajectories of the WPG. It implements task functions with them, as acceleration-based tracking law (see Section 4.3). The controller optimises its cost function using these tasks, prioritised with weights, while respecting constraints such as the robot dynamics and feet contacts. The weights and gains used in the paper are the same as the one of the torque controller presented in Section 4.4.4, except for the CoM and Angular Momentum (AM) tasks gains: the CoM proportional gains were raised to 800 and the AM ones decreased to 2.5. This gains tuning were needed to make the controller successfully perform the trajectory wide side steps found by the WPG.

Moreover, the orientation of the free-flyer given by the WPG was changing a lot and impacted the walking. As it was computed to be the mean of the feet orientations on each component in the WPG of Maroger et al. [24], we implemented it directly as this mean in our torque controller given the reference feet values. This solved the problem and we had a smoother orientation of the free-flyer, however the robot was thus less pivoting its waist than the human during the simulations.

The interest of this controller is that it implements an AM regularisation task, which allows to control the angular momentum part generated by the contact transition Kajita et al. [START_REF] Kajita | Resolved momentum control: Humanoid motion planning based on the linear and angular momentum[END_REF]. In particular, this task is fundamental here because the WPG feet reference trajectories contain wide lateral steps. These steps cannot be dynamically performed without the AM task in torque with our controller because the robot structure is not rigid, as opposed to position control. The wide lateral steps (in particular the ones followed by other lateral steps) lead to swings of the torso which are difficult to compensate without arms motions. Introducing the AM task solves this problem because to reduce the centroidal angular momentum the arms are naturally solicited to create motions around the torso to avoid the swings.

Simulation Results

The simulations have been realized using Gazebo on a standard laptop (Intel CPU i7-8850H @ 2.6GHz). The reference trajectories from the WPG are registered in files which are read by the whole-body controller during the simulation. The controller computes the desired torque for all the joints of the robot at 1kHz and sends this command to the simulated robot in Gazebo.

The simulation result of the same trajectory presented in Fig. 4.15 is shown in Fig. 4.12, where the robot is synchronized with the human. The CoM, base and feet references are well followed by the controller, their tracking are presented in Fig. 4.16, allowing the robot to successfully perform the motion. In particular, as explained in the previous paragraph, there are wide lateral steps to perform while keeping balance. There are 3 steps from 2.75m to 3m on the x-axis (with a total length of 25cm for the 3 steps) which have lengths of 20cm, 10cm and again 20cm on the y-axis (lateral plane). And there is a backward step from 3.9m to 3.75m on the x-axis which has a length of 22cm on the y-axis close to the end of the motion. These steps of around 20cm on the lateral plane are the ones needing the AM task and which are difficult to succeed for our torque controller.

The Figure .4.17 illustrates the results obtained for the first scenario, where the robot walk behind the human, here with a distance of 30cm. Similarly to the previous simulation, the references trajectories are correctly followed as presented in Figure .4.18. In this predicted trajectory there are less wide lateral steps, leading to a more stable motion. The whole prediction process and other simulations (in particular one with the scenario 3 where the robot walks ahead of the human) are shown in the video available at https://youtu.be/hu-cuUYl-58.

Discussion

Human walking trajectory model. The model used to predict the human trajectory depends a lot on the chosen human trajectory model. The model has been optimised in Maroger et al. [START_REF] Maroger | Inverse optimal control to model human trajectories during gait[END_REF] to generate smooth trajectory of the human CoM. That is to say that it does not take into account the oscillations of the CoM due to the steps. So, another model of human gait taking the steps into account could be consider in case of the prediction model needs to be more precise. Moreover, the chosen human trajectory model has been shown to fit well the average human behaviour while its performance could be poorer when apply to individual subjects. However, when testing the prediction model over trajectories of numerous subjects, it seems to achieve its goals whoever the subject is Maroger et al. [24].

Role of the parameters in the whole process performance. The prediction process efficiency depends a lot on the sliding window size where the prediction process is done and on the amount of measurements needed to reach a good prediction. Indeed, a greater amount of measurements and a smaller window size lead to a more precise prediction. However, the process is still able to predict the human behaviour when the amount of measurements is low, even if this prediction is less accurate Maroger et al. [24]. Yet, for the whole-body controller using a large sliding window size leads to a more stable motion, indeed small a window size creates more variations. A numerical assessment of the prediction model is presented in Maroger et al. [START_REF] Maroger | Metrics to assess a human trajectory prediction model during gait[END_REF]. On the other hand, the precision of the human tracking performed by the WPG relies on the chosen scenario and on the potential chosen distance to the human. The greater the distance is when the robot walks behind the human, the cleaner the footsteps are, because the part of the predicted trajectory used by the WPG is mostly in the the measured part. On the opposite, the greater distance is when the robot is ahead of the human, the messier the footsteps are, because all those generated footsteps are based on the prediction which can vary a lot according to the measurements.

Toward a full online application on the real robot. The test on the TALOS robot is an ongoing work. Torque-based walking has been successfully realized. However this walking pattern generator has some limited speed. Developments are under way to reach a sufficient speed to realize a human-humanoid robot interaction.

Conclusion

In this section is presented the work realized in collaboration with Isabelle Maroger in a context of human-robot collaboration. The aim is to ease the tracking of a human by a humanoid robot by helping the robot to predict, and thus anticipate, its partner behaviour. To this end, three contributions have been made in Maroger et al. [24]: First, a prediction OCP model is designed based on a human trajectory model, which generates the future trajectory of a human from its recent past trajectory. Then, a WPG is designed to generate a TALOS humanoid robot CoM and footsteps along the predicted trajectory in real time. Finally the CoM and footsteps trajectories resulting from the WPG are given to a torque whole-body controller which computes a stable command for the robot to follow. This whole framework has been successfully tested in real-time simulation. Thus, in this work, we achieve to develop and assess the capabilities of a whole framework aiming to control a simulated TALOS humanoid robot in order to make him proactively walk along with a human partner.

Experiments realized on the real robot using the controllers

In this section are presented the results we succeeded to achieve on the real robot TALOS and the difficulties we encountered. These experiments are intermediate steps toward transferring the whole simulated results on the real robot. We detail the blocking points preventing us to successfully achieve these complete experiments.

Position Control

4.6.1.a Static stabilization

Using the whole body admittance control and the stabilizer described as the IK scheme in the Section 4.4, the team obtained good results for balancing during quasi-static moves and standing position. Indeed, the admittance control at the CoM allows a quick reaction when applying external perturbations such as pushing the robot. Fig. 4.19 presents the reactive balancing of the TALOS humanoid robot when it is pushed from the front and from the side while standing on one foot. A video about this experiment is available at the following link: https://peertube.laas.fr/videos/watch/2dec7dba-cc57-4df4-8f10-a7d387404301

In the video push-recovery experiments are performed while the robot is standing on both feet and with one foot raised. One can notice that the robot is more stable with both feet on the ground, nonetheless, the IK scheme allows a good stabilization at the CoM level. The stabilizer correctly achieves the balance of the robot: it controls the DCM such that the CoM does not diverge and applies correct contact wrenches to avoid falling (neither slipping, nor presence of too much forces on one foot which imbalance the robot). It is important to underline that the admittance control is only implemented on the CoM, thus the robot is stiff on its upper parts while being more compliant on its lower parts (in particular the hips and ankles). This is why in the video pushing the robot arm produced motions on the whole robot and in particular its CoM.

The robot can achieve tasks with its upper body while external perturbations occur and keep its balance. It can also stabilize itself when non-dynamic trajectories are asked to the legs, or with no contact with the ground (for instance execute a swing on its foot). The difficulties appear when dynamic tasks are asked and involve the creation of contacts with the ground, typically during walking.

4.6.1.b Dynamic stabilization

The dynamic stabilization of the robot is an ongoing work. The actual implementation of the stabilizer should allow the robot to achieve this goal, however this is compromised by the flexibility in the hip of the robot TALOS (see Appendix 4). By tuning the gains of the admittance controller, the team managed to achieve once a straight walk of 20cm using the WPG reference trajectories of Section 4.4.4.a. The video of this successful motion is available at the following link: https://peertube.laas.fr/videos/watch/b56d80ed-7c6c-46a7-8750-fdb7ea6d1636

Latter on, we successfully achieved a repeatable on spot walking which is quite stable (See Fig. 4.20). The video of this on spot walking is available at the following link: https://peertube.laas.fr/videos/watch/1a920902-c75f-4fb0-a638-33bb9b48d649. One can notice that the left wrist of the robot is tilted, indeed, its absolute and relative encoders did not send the same value. Thus, when controlling its position, the wrist had an abnormal behavior as its returned position was not the good one. We had to deactivate its control for the experiment and then fix the offset of the relative encoder.

In both videos the impacts on the ground are large and lead to instabilities, in particular slippage (which can also be caused by the flexibility in the hip). The robot has to move its upper body to compensate for them, because of that, at the end of the 20cm walk the robot almost fall. These impacts are partly due to the wrong positions of the feet when making contacts with the ground. The flexibility in each hip of the robot cannot be measured by the encoders, then it is creating an error between the positions given by the encoders and the real ones. These displacements at the hips are small, but when transferred at the feet positions it can lead to errors of up to 5cm. Thus, the controller is assuming a false position of the feet, and the robot enters in contact with the ground at a wrong position (even at the wrong moment, sooner if the displacement is in the direction of the walk or later in the opposite case). This is creating the large impacts and slippage, which prevented us to achieve a successful walking; this is why compensating this flexibility is necessary. In the next subsection, the experiment realized to compensate it with a fixed value is presented.

An additional way to cope with the stabilization problem would be to reschedule the footsteps and their location according the landing time.

4.6.1.c Fixed compensation of the flexibility

As described in the Appendix 4 we first tried to compensate the flexibility by using a feed-forward on the commanded position of the hip taking into account the torsional stiffness and the measured torque. However, because of the noises on the torque sensors, we had to filter it, which leads the compensation to be applied with delays. We also tried to activate this compensation only on single support phases and not on double support ones to avoid accumulation of internal efforts (on double support the robot will try to correct its hip position while having its feet in contact with the ground and thus not moving, leading to this accumulation of energy). Even with such modifications the results were not enough to successfully perform repeatable walk.

Thus, we then tried to impose a fix compensation of the flexibility without taking into account the measured torque. With a leg of 1m weighting 20kg, we fixed the compensation on the hip to ∆q hip = 20 K ≈ 20 973 = 0.021 rad. Only a repeatable one step froward walk in position control has been successfully achieved with this method, see the video at the following link: https://peertube.laas.fr/videos/watch/08db3177-372b-43cc-85da-2009a267b5c9.

As the robot locomotion was not the core subject of my thesis and that we suspect the flexibility would be a less important issue for the robot during torque control; we stopped the work on the flexibility until the postdoctoral researcher Nahuel Villa came in the team. Then, he properly modeled this flexibility, as described in Appendix 4 and has recently achieved walking results on the robot using a CoP whole-body controller.

Torque Control

4.6.2.a PAL robotics low-level controller

To achieve torque control on the real robot, as explained in the Section 2.3.3 and in conclusion of the Section 4.4.4, it is needed to transform the joint torque commands to motor currents. We decided to use the PAL robotics constructor low-level controller, which computes new commands respecting the robot actuators dynamics. This low-level controller is a proprietary black-box, which use a ros-control hardware interface to communicate with the robot (see Fig. 1.10). To interface our control scheme (based on the SoT with the WPG), we had to create a new version of the roscontrol-sot package, presented in Section 1.6.2.b. Indeed, our control scheme needs no more to communicate directly with the robot but with the PAL robotics controller, which implements different functions and formulations. One of the major difficulty is that the proprietary code source is not available, we only had access to its C++ headers and some basic tutorials. Developing this interface to keep all the functionalities implemented in the roscontrol-sot package (for instance to keep the recording of the logs and creating all the necessary signals needed by the SoT in the dynamic-graph structure), take us months of work (including the following remark).

Moreover, the robot has a modified operating system called ferrum (which is an equivalent to ubuntu). In order to have exactly the same environment as the one on the robot to test our codes, we created a Docker [START_REF] Merkel | Docker: lightweight linux containers for consistent development and deployment[END_REF] container. Installing the SoT packages on this environment was not trivial as some packages had conflicting dependencies with the PAL robotics packages. Finally, we succeeded to test in this Docker container our interface and our torque controller using the PAL simulator available on ferrum. An additional difficulty is that the simulator (based on RViz) is slowed down when using our control scheme; i.e. the displayed behavior is estimated as five times slower than the real one. Therefore, a small and slow oscillation seen in the simulator is in reality a high frequency one on the robot and can lead to dangerous behaviors.

One has to note that, in Dantec et al. [START_REF] Louis Dantec | Whole Body Model Predictive Control with a Memory of Motion: Experiments on a Torque-Controlled Talos[END_REF], the MPC is not embedded on the robot and is interfaced with the PAL robotics low-level controller via ROS topics. This simpler architecture choice was made because it is a stand-alone package (no SoT or dynamic-graph framework) and does not imply sending commands at high frequency (200Hz). ROS topics may induce latency and do not allow to send high frequency commands leading to real-time issues. To avoid the troubles I encounter to interface my control scheme with the PAL robotics low-level controller, the CoP whole-body controller of Nahuel Villa (not relying on the SoT nor dynamicgraph framework) compensating the flexibility was directly implemented in the PAL robotics control scheme.

Tasks

Priority Weight Feet contacts 0 -Posture regularization in half-sitting I 10 4.12: Tasks gains of the torque control scheme for the posture task.

4.6.2.b Experiment Results on a Posture Task

Once we achieved satisfying results on the PAL robotics simulator, we tested the classical formulation of our torque controller using inverse dynamics on the real robot on a simple postural task. The tasks weights and gains used are presented in the Tables.4.11 and 4.12, as the "Fail" experiment.

After few repetitive sinusoidal motions on the robot arm, the system diverged brutally and blocked two of its harmonic drives: the waist and the right shoulder (we pushed the emergency button but the robot reached anyway the harmonic drive blocks). The Fig. 4.21 presents the result of the fail. After investigation it seems that the gains tuned in simulation (which simulates the actuation chains) were too high for the real robot. Thus, tuning the gains even on a proper simulator with the model of the actuators is not enough to ensure a safe solution. We know that some tuning is always necessary on the real robot, but we wrongly assumed that the solution would remain quite stable. Thus, to provide a safe and reliable interaction with the environment and possibly humans, we have looked for a way to ensure the system stability. The video of the failed experiment is available at this link: https://peertube.laas.fr/videos/watch/31fa2562-ba13-4043-a996-c2b8d5b21f4a. Unfortunately, it was not possible to repair the robot at the laboratory because the right shoulder and the torso were preventing the back cover of the robot to be removed (which needed to be removed to access the shoulder harmonic drive). Thus the robot had to be send back to PAL robotics premises for repair.

By lowering the gains value on the posture task, as presented in the Table.4.12, we succeeded to have a stable and compliant behavior of the robot. A video showing this compliant behavior is available at the following link: https://peertube.laas.fr/videos/watch/e9d8948d-08d5-4de9-8f42-2986fbbf0242, and depicted by the Fig. 4.22. At the end of the video, the robot is falling because the contacts on the feet have been disturbed (the feet moved) breaking the constraint of the QP. This small success encourage us to add the CoM task for further tests. Unfortunately, this task does not work on the real robot, instead of correcting the CoM error the QP seems to make it diverge. It is a behavior that is not appearing in the simulator, where the experiment is working. After investigations, it may be due to incorrect torque offsets used in the simulator and not up-to-date PAL robotics packages in our Docker container. However, after updating these packages, our interface with the PAL robotics low-level controller did not work anymore. As we were working on the passivation of our control scheme (see the next Chapter), we put the experimental parts in standby and will try to fix this problem by the end of my thesis.

Conclusion of the Chapter

In this chapter we presented the evaluations realized to achieve an efficient wholebody controller for the humanoid robot TALOS, able to follow correctly the reference trajectories given by the planning in real-time while keeping a low energy consumption. We detailed the force task implemented to achieve manufacturing operations. Finally, we prove the adaptability our torque controller by using it in different locomotion contexts such as walking in collaboration with a human, using a similar gait.

However, these evaluations have been made only on locomotion problems in simulations. Because the torque controller has proven to be adaptable, energy efficient and capable of quite high speed locomotion we choose to focus on this solution. Yet, when we tried our torque implementation on the real robot the system eventually diverges and the robot reaches its limits abruptly, blocking its harmonic drives. The investigation of the problem leads to decrease the controller tasks gains, however the question of the stability of our scheme was raised. Recalling the results obtained by Henze et al. [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF] and other passive controllers, we decide to consider a way to make our controller respect the passivity of the system. Thus, the following chapter presents the works realized in order to analyse the energy flow and passivity of our system. We detail the achieved results which modify the previous torque controller with a global energy tank to force the passivity of the system. The new scheme is tested in a multi-contact scenario employing the designed force task of Section 4.3.4 as a first step for manufacturing operations. In this chapter are detailed the results obtained in the submission Ramuzat et al. [START_REF] Ramuzat | Passive inverse dynamics control using a global energy tank for torque-controlled humanoid robots in multi-contact[END_REF].

Introduction

Humanoid robots are complex systems designed to operate in similar environment than humans. One of their interesting purposes is to be able to achieve tasks that can be dangerous or too demanding for a person. This aim often relies on two main capabilities of the humanoid robots: the locomotion and manipulation. However, to achieve such a goal, these robots must interact with their environments (workspace, tools, other robots, even humans), which are mostly unknown and/or subject to disturbances. Even performing a simple task in a known and/or fixed environment can lead to instabilities. As stated in the Section 4.6 of the previous chapter, the authors encountered this problem while testing a classical torque control scheme using inverse dynamics on the robot TALOS on a simple postural task. After some repetition of a sinusoidal motion on the robot arm, the system diverged brutally and blocked some of its harmonic drive. Thus, to provide a safe and reliable interaction with the environment and possibly humans, we have looked for a way to ensure the system stability.

As presented in the Section 2.6, the two most common ways to prove the stability of a system are the Lyapunov or the Passivity analysis. Because of the results presented in Henze et al. [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF] and of the proof on the passivity as a necessary condition for stability during robot interactions Folkertsma and Stramigioli [96], Camlibel et al. [START_REF] Camlibel | Mathematical Control Theory I: Nonlinear and Hybrid Control Systems[END_REF]; we chose the passivity analysis. This chapter details the analysis and results obtained toward the passivation of the torque controller presented in the previous Section The developed scheme is based on the mathematical adaptation of the passivity method introduced by Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF] and is adapted for under-actuated humanoid robots, using the non-hierarchical multi-objectives controller TSID. To the best of the authors knowledge, no proof of passivity on an ID controller using energy tank on an under-actuated robot has been published or implemented yet.

Our contribution to the literature consists in a passive whole-body controller using inverse dynamics (as opposed to Henze et al. [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF] which uses a PD+ structure) with energy tank (as opposed to Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF]) which regulates the task gains. The concept of energy tank is rarely used on humanoids robot: it is used in Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF] with a strict hierarchy of two tasks and the proof of stability is only done on fully actuated robots. Our control scheme takes advantage of the energy tank to modulate our tasks gains, in order to respect the passivity of the system. It is similar to what is done in Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF], where the task gains are functions of the current task inertia. However, our final control scheme has a constraint to respect the passivity inequality, not implemented in Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF]. Indeed, because humanoid robots are under-actuated, the passivity is no longer guaranteed when constraints become active in the QP (such as the torque limits for instance). It is the case for the works presented in Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF] and Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF], but in our scheme, this case is dealt with the imposed passivity constraint. Therefore, our contribution is two-fold: the task gains of our whole-body controller are non-constant and regulated by an energy tank, and our scheme implements a constraint to respect the passivity even when other constraints become active in the QP. The classical inverse dynamics formulation is transformed in a non-linear problem which first computes the energy tank and regulating coefficients ). The multi-contact scenario is the first step toward a manufacturing operation such as drilling. It consists in applying a definite amount of force on a block (representing a part to manufacture) while maintaining a fix position; then this block is removed and the robot must keep its balance (see Fig. 5.1). The removing of the block simulates a sudden lost of contact which can be due to slippage or, more importantly in the case of drilling, when the drill bit exits the hole (this can occur latter or sooner than what is planned and is introduced by a rapid decrease of the interaction force going down to 0). We organize the chapter as follows: Section 2.6.3 recalls briefly the passivity theory presented in Section 2.6.3 while applying it to our system. Then, Section 5.3 describes the mathematical process to add a global energy tank and obtain the passivity proof in TSID. Finally, Section 5.4 presents the simulations used to validate the robustness of the controller and Section 5.5 discusses the obtained results.

Passivity Theory

Recall of the definitions

Section 2.6.3 details the definition of the passive system, which is a dissipative Willems [START_REF] Willems | Dissipative dynamical systems part I: General theory[END_REF] system respecting the following definition. We just recall here the Property 1 on the derivative of the storage function Schaft [START_REF] Schaft | L2-Gain and Passivity in Nonlinear Control[END_REF] (we remove the state or time dependencies of the terms for clarity):

A system with the state space model of ẋ = f (x, y) with initial state x(0) = x 0 ∈ R n , input vector u ∈ R m and output y = h(x, u) is said to be passive, if there exists a positive semi-definite function H : R n → R + , called storage function, such that:

Ḣ ≤ y T u ∀(x, y) (5.1)
The passivity is a stability criterion based on the power flow exchanged by the components of the system. H(x) is often chosen as the energy of the system, then, the system internal power ( Ḣ: derivative of the energy in the system) is lesser or equal to the power transferred to the system through its port (y T u). This property will be used in the following sections to prove the passivity of our control scheme. Then, let us recall the definition of a virtual energy tank, as the control framework will be augmented by a global one to store the energy dissipated by the tasks. The tank can be used to transfer the stored energy when a task becomes active (the task needs more energy to be completed). In opposition, when the tank is empty, it can be used to deteriorate the tracking of this task to ensure the passivity of the system. The energy tank is defined as a virtual storage element with flow variable ṡ and effort variable s such that Giordano et al. [START_REF] Robuffo | A passivity-based decentralized strategy for generalized connectivity maintenance[END_REF], Dietrich et al. [START_REF] Dietrich | Passivation of projection-based null space compliance control via energy tanks[END_REF][START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF]:

E tank = 1 2 s 2 (5.2)

Analysis of our System

Our system is composed of three components: the robot, the controller and the environment. As in Section 2.6.3, we assume that the environment of the robot can be described by a passive mapping (v → -τ ext ), in connection with impedance control Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF], Ott et al. [START_REF] Ott | On the passivity-based impedance control of flexible joint robots[END_REF]. Then, the group robot-environment is passive, so we have to make the group controller-robot passive. As in Ott et al. [START_REF] Ott | On the passivity-based impedance control of flexible joint robots[END_REF], the input of our controller is the velocity of the robot, with the joint angular velocity measured by the encoders and the free-flyer velocity calculated by the baseestimator. And the output of the controller is the control joint torque τ . Because the interconnection of passive systems leads to a general passive system we have two solutions: we can impose the passive mapping (-N v → τ ) between the robot and the controller, then because the group robot-environment is passive the overall system will be passive. Or, we can impose the passive mapping (v → τ ext ) between the system {robot+controller} and the environment, see Fig. 5.2.

Note that because N T ∈ R n×n j is the selector matrix we have (taking the pseudoinverse)

N † ∈ R n j ×n = N T , N †T = N and N †T v = N v = qj by definition (but v = N T qj ).
Then, as described in Section 2.6.3 (see Garofalo and Ott [START_REF] Garofalo | Passive energy-based control via energy tanks and release valve for limit cycle and compliance control[END_REF]), we have to find a storage function H such that: In the following, we prove the passivity of the system by building Ḣ. Then we augment this function to obtain a Ḣext satisfying Eq.5.4.

Ḣ ≤ -(N †T v) T τ = -qT j τ = -v T N T τ (5.

Formulation using Energy Tank in TSID

Choosing E tank for a set of tasks

In this part we look at the proof of the passivity for a set of N motion tasks (see Eq.4.8). In this set of tasks, there must be at least a postural task regularization and one contact task with the environment. We consider here only one level of hierarchy (the cost, priority I), and assume that several tasks at this level can be put together as if it was only one task.

For one motion task x (see Eq.4.8), we define S the potential energy of the task:

S = 1 2 e T ΛK P e Ṡ = ėT ΛK P e + 1 2 e T ΛK P e Λ = (JM -1 J T ) † J † = U D -1 V T (5.5)
where K P is the proportional gain of the task regularization in the controller (Eq.4.8), J † is the Moore-Penrose pseudo-inverse of J, with U , D and V respectively the left singular vectors, the diagonal matrix of singular values and the right singular vectors of J (using the Singular Value Decomposition), e = (x -x * ) and Λ is the semi-positive definite Cartesian space inertia matrix (multiplication property of the positive semi-definite matrix M , see Prop.2). ΛK p is semi-positive definite if K p has equal positive elements (see Table.4.3) or under some specific conditions described in an open access document Ramuzat [START_REF] Ramuzat | On the semi-positive definition of the product of a positive diagonal matrix and a symmetric semi-positive definite matrix[END_REF] which have been added in the Appendix 5 of the manuscript. Then, because S = e T ΛK P e is a quadratic form of ΛK p , S is always positive.

We then define the storage function for one task as:

H = S + E tank (5.6)
Then, for a set of tasks we define H ∈ R ( N 0 1)×1 as:

H =   H 0 ... H N   , Ḣ =   Ḣ0 ... ḢN   (5.7)
And accordingly, the vectors and matrix J ∈ R L×n , K P , K D ∈ R L×L , E = X -X * ∈ R L , the respective stack of vectors and matrix of their non-bold value for the N tasks, with L = N i=0 l i . From Eq.4.8 we obtain the following formulation for the set of motion tasks:

Ẋ = Jv Ja = Ẍ * -Jv -K P E -K D Ė (5.8)
It is important to notice here that J ∈ R L×n is a skinny matrix, i.e. L ≥ n and is full rank. Indeed, as said before, the set of tasks contains at least a postural (acting on n j DoF) and a contact task (acting on 6 DoF). As n = n j + 6 in the case of a humanoid robot with a 6 DoF floating base, we have L ≥ n. Thus, the generalized task space is greater than or equal to the joint space, leading to an over-determined problem. Moreover, the pseudo-inverse of J can then be written as Ben-Israel and Greville [START_REF] Ben | Generalized Inverses: Theory and Applications[END_REF]: J † = (J T J) -1 J T . And then by definition:

J † J = (J T J) -1 J T J = I (5.9)
Finally, we can rewrite Λ = (JM -1 J T ) † . By definition: (AB) † = B † A † if A is full column rank and B full row rank Greville [START_REF] Greville | Note on the generalized inverse of a matrix product[END_REF]. As J is skinny, it is a full column rank matrix and J T is a full row rank matrix. Moreover, M is square and invertible, thus a full rank matrix. Then, M -1 J T is also a full row rank matrix, leading to the fulfillment of the definition, we obtain:

Λ = (JM -1 J T ) † = J †T M J † (5.10)
Then, we can define the potential energy for the set of tasks S as:

S = 1 2 E T ΛK P E Ṡ = ĖT ΛK P E + 1 2 E T ΛK P E Λ = J †T M J † J † = (J T J) -1 J T (5.11)
Following the same reasoning as for one task, ΛK P semi-positive definite as K P has equal positive elements along the tasks; thus S is always positive. Using the equation of the dynamics (Eq.2.4) and the formulation of TSID for τ ext (Eq.4.13), we have:

-v T N T τ = -v T M a + Cv + g -c k=1 J T k T T k f k -J T F T T F F F (5.12)
As the problem is over-determined, we can replace a by the Eq.5.8. The joints accelerations of the robot can be determined from the tasks accelerations. We obtain:

-v T N T τ = -v T M J † Ẍ * -Jv -K P E -K D Ė +Cv + g -c k=1 J T k T T k f k -J T F T T F F F (5.13)
We can notice that, using the formulation of J † and Λ of Eq.5.11 and Eq.5.9, we have:

J T Λ = J T J †T M J † = (J † J I ) T M J † = M J † (5.14)
And thus the Eq.5.13 becomes:

-v T N T τ = -v T J T Λ Ẍ * -Jv -K P E -K D Ė +Cv + g -c k=1 J T k T T k f k -J T F T T F F F (5.15)
By definition Ẋ = Jv and thus ẊT = v T J T . The previous equation is simplified in:

-v T N T τ = -ẊT Λ( Ẍ * -Jv) + ∆ ẊT ΛK P E + ẊT ΛK D Ė -v T Cv -v T g +v T c k=1 J T k T T k f k + v T J T F T T F F F ∆ = Ṡ + Ẋ * T ΛK P E -1 2 E T ΛK P E (5.16)
We know that S is positive, thus we want to choose the dynamics of our tank to contain the remaining terms (of which we do not know the sign):

Ėtank =

Damping of the tasks

ẊT ΛK D Ė + Ẋ * T ΛK P E -1 2 E T ΛK P E -ẊT Λ( Ẍ * -Jv) +v T -Cv -g + c k=1 J T k T T k f k + J T F T T F F F (5.17)
With this definition, Ḣ = Ṡ+ Ėtank = -v T N T τ , ensuring the passivity of the system for the power-port {-N v, τ }.

Including protection and valve

Once we have the desired formulation of the dynamics of the tank, we define α, β, γ, the coefficients that connect the tank to the system and regulate the tasks. They are used to bound the energy in the tank and to limit the power transferred to the system.

Let E max tank , E min tank be the upper and lower bounds of our tank and P low the power transfer limit. We have E tank = 1 2 s 2 , thus, Ėtank = s ṡ and we build ṡ to take into account the valves and satisfy Ṡ + Ėtank ≤ -v T N T τ . ṡ is separated in two components: ṡσ that we can control, linked to the tasks and ṡφ linked to the external components and thus cannot control (gravity, Coriolis and forces terms).

ṡσ = 1 s βγ ẊT ΛK D Ė + Ẋ * T ΛK P E - 1 2 E T ΛK P E -ẊT Λ( Ẍ * -Jv) + v T J T F T T F F F B σ (5.18) ṡφ = - 1 s v T Cv + g - c k=1 J T k T T k f k B φ (5.19) ṡ = (1 -α)( ṡσ + ṡφ ) (5.20)
Like in Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF], to control the flow transferred from the tank to the tasks, we choose:

γ =    P low B σ if B σ < P low ≤ 0 1 else (5.21) β =    0 if (E tank ≤ E min tank ) & (γB σ -B φ < 0) 1 else (5.22)
And like in Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF], we choose α to be:

α =    1 if (E max tank ≤ E tank ) & (βγB σ -B φ > 0) 0 else (5.23)
α is the overflow valve, meaning that when α = 1 the tank can be filled, otherwise the tank is already full and it cannot be filled more Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF]. β is the opposite valve, it controls the output flow of the tank, if β = 0 the tank is empty (or at its lowest value) and the controller cannot use it to fulfill the tasks Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF]. Finally, γ regulates the energy flow transferred from the tank to the controller. It avoids sharp release of the tank energy to the system, P low is an empirical power threshold defining when the transfer should be regulated. When B σ (the part of the tank derivative energy we can impact) is lesser than this threshold (negative: the system takes energy from the tank), γ takes the value of P low B σ Schindlbeck and Haddadin [START_REF] Schindlbeck | Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks[END_REF]. Then 0 ≤ γ ≤ 1. To avoid discontinuities in the coefficient equations, the transitions between their two states are smoothen using the same approach as in Kronander and Billard [START_REF] Kronander | Passive interaction control with dynamical systems[END_REF]. The impact of the energy tank is propagated to the tasks with the parameters β, γ. They are introduced in the task functions such that we have new references a βγ and F βγ F :

Ja βγ = Jβγa = βγ ẍ * -Jv -K P e -K D ė T T F F βγ F = T T F βγF F = βγ F * + P F e f + I F t 0 e f ds (5.24)
And thus the parameters appear naturally in the previous equations when replacing the acceleration and the force by a βγ and F βγ F in the equation of the dynamics. The Eq.5.16 is then transformed in:

-v T N T τ = βγ ẊT ΛK D Ė + Ẋ * T ΛK P E + v T J T F T T F F F - 1 2 E T ΛK P E -ẊT Λ( Ẍ * -Jv) s ṡσ -v T Cv + g - c k=1 J T k T T k f k s ṡφ +βγ Ṡ (5.25)
And then, the terms of the energy tank derivative appear ( Ėtank = s(1-α)( ṡσ + ṡφ )), as the one of the potential energy of the tasks. One can notice that E tank is only obtained by integrating Ėtank over time, which can add drift. However, note that E tank is only used to check if the tank is empty or full, its value is never used to change the scheme, as opposed to Ėtank which is used (partially) in γ and in the passivity constraint. Thus, we think that the drift issue on the energy tank is not critical.

5.3.2.a Proof with valves for the power port {-N v, τ }

The role of α is clear when analysing the derivative of the storage function H in order to prove the passivity of the system. Choosing H = S + E tank we have:

Ḣ = Ṡ + Ėtank Ḣ = βγ ĖT ΛK P E + 1 2 E T ΛK P E + s ṡ Ḣ = βγ ∆ -Ẋ * T ΛK P E + 1 2 E T ΛK P E + s(1 -α)( ṡσ + ṡφ ) Ḣ = βγ ∆ -Ẋ * T ΛK P E + 1 2 E T ΛK P E + (1 -α) βγ ẊT ΛK D Ė + Ẋ * T ΛK P E- 1 2 E T ΛK P E -ẊT Λ( Ẍ * -Jv) + v T J T F T T F F F +(1 -α) -v T Cv + g -c k=1 J T k T T k f k (5.26)
Replacing the Eq.5.25 in Eq.5.26 we finally obtain:

Ḣ = -v T N T τ -α -v T Cv + g -c k=1 J T k T T k f k -α βγ ẊT ΛK D Ė + Ẋ * T ΛK P E- 1 2 E T ΛK P E -ẊT Λ( Ẍ * -Jv) + v T J T F T T F F F = -v T N T -α -B φ + βγB σ ≤ -v T N T τ (5.27)
Indeed because of Eq.5.23, if α = 0 then βγB σ -B φ > 0, thus: -α βγB σ -B φ ≤ 0 Then from Eq.5.27 we can conclude the passivity of the system for a set of N tasks with respect to the power port {-N v, τ } = {-qj , τ }.

5.3.2.b Proof with valves for the power port {v, τ ext }

We can also prove the passivity of the overall closed-loop dynamics with respect to the power port {v, τ ext }. Let us augment the storage function H with the kinetic energy of the tasks and the gravity potential to obtain H ext :

H ext = S + E tank + 1 2 ẊT Λ Ẋ + V g (q)
(5.28)

We have by definition, g(q) = ∂V g (q) ∂q T , leading to:

∂V g (q) dt = ∂V g (q) ∂q ∂q dt = ∂q dt ∂V g (q) ∂q T = v T g(q) (5.29)
Let us write the formulation of the equation of the dynamics for one task in the Cartesian space:

Λẍ + µ ẋ = J †T (-g + N T τ + τ ext ) (5.30)
With µ = J †T CJ † -Λ JJ † . Then, for a set of N tasks we have, Z designating the stack of µ:

Λ Ẍ + Z Ẋ = J †T (-g + N T τ + τ ext ) (5.31)
By taking the derivative of Eq.5.28 and substituting Eq.5.31, we can write:

Ḣext = Ṡ + Ėtank + v T g + 1 2 ẊT Λ Ẋ+ ẊT J †T v T (-g + N T τ + τ ext ) -Z Ẋ (5.32)
Using the fact that the Coriolis matrix is defined such that: 1 2 Ṁ (q) = C(q,v)+C(q,v) T 2 , we have ẊT ( 12 Λ -Z) Ẋ = 0 (skew-symmetric property, recall of the eq.2.5). Finally we obtain, using the Eq.5.27: (5.33) Delivering the proof of passivity for the port {v, τ ext }.

Ḣext = Ṡ + Ėtank ≤-v T N T τ +v T N T τ + v T τ ext Ḣext ≤ -v T N T τ + v T N T τ + v T τ ext Ḣext ≤ v T τ ext

Modified passive QP formulation

The classical inverse dynamics formulation is transformed in a non-linear problem with two stages. It is a sort of alternating minimization problem. The first stage (1) computes the energy tank dynamics and the regulating coefficients and then the second stage (2) solves the QP modified by these coefficients. One can notice that the first stage finds the coefficients for a torque which does not take into account the constraints of the QP. Thus as in Englsberger et al. [START_REF] Englsberger | Mptc -modular passive tracking controller for stack of tasks based control frameworks[END_REF], if a constraint became active, the passivity is no longer guaranteed because the computation used for the energy tank is not the same as the one of the QP. This is why we add a new constraint in the QP formulation, similarly to Joseph et al. [START_REF] Joseph | Towards x-ray medical imag-ing with robots in the open: safety without compromising performances[END_REF], enforcing the inequality Ḣ ≤ -v T N T τ to maintain the passivity in any situation. The general (2) min

y=[a,f ] N i=0 λ i O i y -βγo i 2 s.t.   J c 0 0 M -J T c -N T     a f τ   =   -Jc v -h   τ min ≤ τ ≤ τ max q min j ≤ q j ≤ q max j qmin j ≤ qj ≤ qmax j f min ≤ f ≤ f max Ḣ ≤ -v T N T τ
(5.34)

Simulations

In this section are detailed the simulations realized on Gazebo to demonstrate the validity of our implementation. The tests are performed on a humanoid robot TA-LOS. The chosen simulations are a force application task performed by the left hand and a walk of 20cm steps on flat ground. These simulations allow to cover multicontact and locomotion skills in stationary environment. The set of tasks considered during the first simulation are presented in the Table .5.1, and in the Table .5.3 for the second one. The task gains are defined in Table 5.2. The authors have implemented this controller using the TSID [80] library in the open-source package [START_REF]talos-torque-control[END_REF].

The design parameters of the global energy tank are set as follows: E max tank = 5J, E min tank = 0.1J and P low = -0.5W. They are set as low values in order to keep small the contribution of the tank energy compared to the overall energy of the system. In the simulations, we use the robot base-estimator of Flayols et al. [START_REF] Flayols | Experimental evaluation of simple estimators for humanoid robots[END_REF]. The base information is estimated using robot joints configurations and velocities, the IMU and the force sensors of the robot. For the force task simulation, no planning methods were used as the robot remains in the same configuration. The force references were created by linear interpolation between the actual and desired force to be applied on the block. The reference trajectories of the 20cm step walk were computed with the multicontact-locomotion-planning framework [START_REF]multicontact-locomotion-planning[END_REF] (see Section 2.4).

Pushing task with unplanned contact removal simulation

In this simulation the robot makes a contact between a tool and a surface and applies a 30N force along the z-axis. In Gazebo, the tool is simulated by a cylinder and the surface is a simple square block, shown in Fig. 5.1. This set-up is a first step toward more complex operations, requiring contact and a force application, such as drilling.

It exposes the problem of unexpected broken contact that can be due to slippage, disturbances or when the drill bit exits the hole. In this simulation, the reference force is raised in two steps, the first one reaches 10N to stabilize the contact, in Fig. 5.4 some oscillations can be seen in the first slope, leading to some instability in the tank coefficients in Fig. 5.5. Then it is set to 30N, and at 32.3s the block is removed. The energy tank is initialized at 3J.

As shown in Fig. 5.4, the measured force along the z-axis falls to 0N when the block is removed. We first try a simulation without the energy tank and the passivity constraint. As a result, the robot quickly falls and cannot react in time to try to balance itself or to remove the force task when detecting the loss of the contact. On the contrary, with the formulation presented previously, when the block is removed the robot does not fall immediately. Indeed, as presented in Fig. 5.5, the α and γ coefficients are triggered. With the γ closes to 0, the force task is highly penalized and thus the hand slowly comes down. As a result, the system has the time to detect the loss of the contact and remove the task, leading to the pick in the consumption of the energy tank at 34.4s. By removing the task, the robot goes back to its initial position because of the postural task and thus raises its hand, going against the gravity and consuming energy. On can notice a continuous growth of energy during the steady state between 20 and 30 s. We think that this growth is due to the remaining error of the postural task which desired position is set to the half-sitting of the robot and not the contact configuration.

Because the energy constraint is enforced in the TSID constraints, the QP ensures that the passivity holds, i.e. Ḣ ≤ -v T τ . The Fig. 5.6 presents the evolution of the potential function derivative Ḣ during the removal of the block (at 32.2s) and the task. We also display the results without energy tank, when the robot falls. We can see that the mechanical power of the system is controlled in the passive implementation case whereas it presents peaks in the simple case, leading to the fall. One can notice that around second 34 -35 the tank energy slope is much steeper than -0.5 W (corresponding to the value of P low ). This is because the task is removed and the robot goes back to the postural task position (straighten up), acting against the gravity. Keep in mind that Ėtank = s(1 -α)( ṡsigma + ṡphi ), thus while ṡsigma is regulated by γ, ṡphi is not as we cannot control it. Thus, the high values of Ḣ correspond to the power delivered by the gravity, Coriolis and contact forces terms.

For the moment to remove the task we use a hand-programmed emergency action, but it should be implemented in a better way in the QP. Or for instance, having a finite-state-machine which would tell the QP which tasks should be added/removed in function of the energy tank parameters. The video of the simulation is available at the following link: https://peertube.laas.fr/videos/watch/f32023d8-dda8-45b6-8c22-d1cd1a0285a5

Walk of 20cm steps

In this simulation the robot walks 1 meter forward on flat floor with 20cm steps, starting with the left foot (shown in Fig. 5.1). The first and last steps are only 10cm long, and the first motion to swing the CoM of the robot is done in quasi-static. The double support duration is of 0.2s and the single support duration is of 1.2s. The set of tasks and gains used are the same as the previous simulation without the Cartesian Force-Contact task and with feet tasks motions during the single support stages. For clarity, the set of tasks are given in the Table In the Fig. 5.7 is presented the energy tank evolution during the simulation, with the regulating coefficients. One can see that at the beginning of the motion during the quasi-static swing the energy tank is emptied, leading the β parameter to decrease to 0 around the 10th seconds. The oscillations of the parameters lead the robot to oscillate as well because the β, γ parameters are used in the task errors formulation (see Eq.5.27). This behavior can be seen in the Fig. 5.8 and 5.9 that respectively depict the CoM tracking along the y-axis (lateral plan) and the feet tracking along the z-axis (height of the steps, the actuators being positioned in the ankles, on the ground the sensors give a position of 0.107m). As a result, the potential function derivative Ḣ is really noisy around the 10th seconds (see Fig. 5.10) and must have activated the constraint in the QP. To avoid this concerning behavior one can increase the budget of the energy tank (set at its maximum value, 5J, at the beginning of the motion), or try to replace the quasi-static part by a dynamic one to see if the tank is emptied. We think that the better solution should be to design the energy tank and P low parameters in function of the reference trajectories Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF].

Except from the concerning beginning, the robot achieves the walking motion without trouble with a good tracking of the reference trajectories, as shown in the Fig. 5.8 and 5.9. This motion can be seen as a repetitive sinusoidal one on the axis presented for the CoM and feet. This is reflecting on the energy tank behavior, as it is filled and then emptied in a sinusoidal manner. The increases of energy in the tank correspond of the motions taking the CoM in the middle of the support polygon and thus, the feet in double support. Reciprocally, the tank energy decreases when the robot CoM reaches the edge of the support polygon above the supporting foot. This behavior can be explained by two facts: First, in double support the feet motion tasks are removed, leaving only the contact motion tasks which present small displacement errors if any. The feet do not move, thus there is also no error or task velocity nor acceleration (no damping energy). This typically means that Ṡcontact = 0 and ṡσ contact = 0 (see Eq.5.18), thus the contacts task motions do not contribute much to the storage function, as opposed to the feet motion tasks which consume energy due to the small tracking delay. Secondly, the postural regularization task is set to the half-sitting position. Thus, when the robot swing on one side and another it is the furthest to its reference posture, the middle position is the closest to it, leading to this sinusoidal behavior of the energy tank. One may try to change the postural reference following the walk to see the evolution of the behavior.

The values of Ḣ oscillate a lot but remain under the passivity bound. Moreover, one can retrieve the sinusoidal pattern in it linked to the tasks behaviors. Looking at γ in the Fig. 5.7, even if it oscillate its is always greater than 0.5 (except at the beginning). Thus the tasks are not too penalized and the robot can perform the walk. This means that the oscillations of Ḣ are either due to the non controllable part of the energy tank ( ṡφ ) or from the potential energy of the tasks Ṡ. Looking at the behavior of Ṡ for each tasks, the sinusoidal pattern seems to correspond to the feet potential energy while the oscillations come from the potential energy of the CoM when adding contact. This was expected, as the CoM reflects the action of the contact forces at the centroidal level (linked with the CoP). The behaviors of Ḣ with respect to ṠCoM and Ṡfeet is presented in Fig. 5.11 to illustrate this comment.

The video of the simulation is available at the following link: https://peertube.laas.fr/videos/watch/7b5c3ae2-10d1-48da-a2ba-5d497e9c0249

This simple locomotion simulation presented does not show an application where the passivity approach is better than the classical one. But it demonstrates its capabilities on complex tasks as locomotion. The authors think that a walk where some parts of the ground are lower than planned can be a good application where the passivity scheme would behave better than the classical one. For instance, a walk where the robot goes down an unplanned step platform. The CoM and feet heights of the robot will then be wrong and the QP will try to correct them. Without the passivity, the classical approach would raise the CoM and accumulate errors on the feet, trying to make contact higher than the ground, potentially leading to divergence. With the passivity scheme and energy tank, the authors think that the tasks would be penalized but that the overall walking motion would be achieved.

Discussion

We first discuss the positivity of S in order to have H positive. In this paper, K p has equal elements in function of the tasks to achieve this result. However, this is restrictive, one may want to find properties on ΛK p to achieve the positivity with different gains [START_REF] Ramuzat | On the semi-positive definition of the product of a positive diagonal matrix and a symmetric semi-positive definite matrix[END_REF] or to reformulate S = 1 2 e T K 1 2

P ΛK 1 2

P e to keep the symmetry. For the first solution, we analysed the behavior of S by changing the gains values and experimentally found that for the posture task, if the gains are equal along the kinematics chains (arms, legs, torso and head), the quadratic form of ΛK p is positive. But for now, we do not have the proof of this result, we think that it comes from the structure of the mass matrix M which must be symmetric along the different kinematics chains of the robot. Thus, it may be possible to find a proof depending on the structures of M , J, Λ and K p to have the positive definition of S. It would be interesting to investigate this lead as it keeps the presented mathematical formulations of E tank and the proofs. For the second solution, one can reformulate S as S = 1 2 e T K 1 2

P ΛK P e to make it symmetric. Then, one must replicate the changes along the set of equations and update E tank accordingly. In our point of view, it should not change the overall proofs of the section.

During the first simulation, we observe oscillations on the α and γ coefficients due to the oscillations of the tool on the surface and after removing the force task. This behavior is mitigated thanks to smoothness functions Kronander and Billard [START_REF] Kronander | Passive interaction control with dynamical systems[END_REF] but can be improved. Indeed, in Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF] the P low value is not fixed but varies in function of the task progression, leading to a smoother γ because it has been learnt and built in coherence with the task to achieve.

One problem of the presented formulation is to maintain the robot balance: if the coefficient β falls to 0, each task will be degraded to a zero tracking. Thus, nothing will keep the balance of the robot and it may fall. In the walking simulation, this leads to oscillations and motions of the legs that deteriorate the walk. The passive scheme has no mechanism to go back to a balancing state; in particular, when the energy consumption comes from external perturbations (such as walking on debris) it cannot handle the robot recovery. In general, to find a new solution respecting the robot equilibrium, it is necessary to re-plan new reference trajectories taking into account the disturbances. To palliate this problem, one can choose to not multiply the β coefficient on the postural task and setting the reference posture to the current one, in order to have the robot only compensates the gravity force when β = 0. However, this will prevent the postural task to be regulated by the energy tank and may lead to a non-passive behavior. Moreover, this solution is unreliable if the robot has a reference posture not guaranteeing the balance. Using a reference computed on-line by the planning to ensure the balance fix this last issue. We think that a better solution would be to design the energy tank maximum value like P low , in function of the reference trajectories Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF]. Then the case were β falls to 0 should be rare, only caused by dangerous external disturbances.

Another point of concern is the addition of the passivity constraint in the QP which may lead to a too constrained problem. For instance, a case may arise where the passivity needs a torque which exceeds the limits imposed on it by the QP. The two constraints will be in conflict and lead to oscillations in the solution. To solve this problem it will be necessary to take into account the constraints of the QP in the formulation of the energy tank. It may also be possible to reformulate the problem in order to compute the two stages of our formulation in Eq.5.34 simultaneously. Otherwise, it will be necessary to relax the passivity constraint. This is a limit of the passivity formulation at control level; the authors think that on-line planning can help for this kind of issue. For instance, using Model Predictive Control with a constraint on the passivity may be a future aim.

It is important to keep in mind that the passivity approach is first of all a way to ensure the stability of the system, in a global way. One can add monitoring mechanisms to detect the ill conditioned behaviors of the robot to protect the system. However, these mechanisms are dependant of previously identified dangerous cases. With the passivity stability, even cases where the monitoring processes do not detect the diverging tasks/motions/behaviors are taken into account. In particular, one can use the regulating coefficients of the energy tank in the monitoring mechanisms to improve the handling of the diverging cases.

Finally, the transition from the simulations to the real experiments is not straightforward. In the Gazebo simulator, the joint torque control is almost perfect because the dynamics of the motors are neglected. However, not taking these dynamics into account will lead to unrealistic and dangerous behaviors on the real robot. Moreover, the real robot is subject to imperfections such as errors in the chain actuation model, sensor noise, limited torque bandwidth or delays Englsberger et al. [START_REF] Englsberger | Torque-based dynamic walking -a long way from simulation to experiment[END_REF]. In the presented implementation, the signals retrieved from the robot are filtered and the force sensors on the feet are used to improve the robustness of the base-estimator. The authors are currently testing the simulation on the simulator of the TALOS constructor, PAL robotics, which models the actuator dynamics of the robot.

Conclusion

In this chapter a novel control formulation for the passivation of an inverse dynamics framework is presented. It involves a global energy tank which monitors the exchange of energy between a set of tasks and regulates their gains. This method is close to the passive hierarchical impedance control relying on a strict hierarchy and null-space projection of Dietrich et al. [START_REF] Dietrich | Passive hierarchical impedance control via energy tanks[END_REF]. It adds a protection valve for power flow variations and is transposed in a case of a non-strict hierarchy on an underactuated humanoid robot. The robustness of the controller is evaluated in simulations on Gazebo through a multi-contact scenario and 20cm walk involving six and five tasks. Thus, the proposed method extends the repertoire of multi-objectives non-hierarchical controllers for applications requiring physical interactions with the environment or a human. Its passive formulation ensures a stable, safe and robust behavior of the system.

For our future works, we plan to improve the behavior of the coefficients, in particular, the one on the power flow regulation can be computed as in Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF], to vary in function of the task progression. The impact of these coefficients on the balance of the robot will be evaluated, indeed, for now, the robot may falls if all the system is penalized to respect the passivity. Finally, our new control formulation will be evaluated on the real robot.

In this chapter our passive torque controller implementing a contact-force task has been successfully tested in simulation on the humanoid robot TALOS. However, one can wonder how to apply this solution to an industrial context and industrial robots such as the MSDR. The transfer of the solution is not straightforward, as stated in the context of this thesis (see Section 1.4.3), the MSDR has no ROS interface, thus one will be needed to plug our solution to the low-level controller of the robot. The transfer of our state-of-the-art controller onto the MSDR industrial robot is discussed in the next chapter.

Introduction

This chapter describes the challenges that may arise when we will transfer the passive torque controller presented in the previous section on an industrial robot. The industrial robot MSDR considered has been presented in Section 1.4.3, as it is the solution designed by Airbus to achieve autonomous manufacturing operations (see Fig. 6.1).

As stated before, the first issue to tackle is the difference of model between the robot TALOS and the MSDR: one is under-actuated while the other is not because it is fixed to the ground (for the initial setup considered in this thesis). Thus, the relevant part of our control solution can be seen as the application of a passive contact-force task on the robot arm. Using our formulation of the passivity, the control formulation will guarantee safe, stable and robust behaviors of the robot onto the environment. One can thus formulate a new set of tasks in our previous torque controller to take into account only the relevant parts of the complex problem presented on the humanoid robot.

Secondly, the MSDR uses the Fanuc proprietary software architecture which does not implement a ROS interface. The difficulty here lies in reading sensors and writing commands (joint position or torque vector) within a guaranteed deterministic time boundary in order to update the system information in real time. The frequency needed to achieve a correct behavior is at least 250Hz, our torque scheme achieves a control loop at 1kHz. Thus, one will need to implement an interface to connect the low-level control board of the MSDR to our high level solution.

Once these issues are solved, one can highlights the benefits and drawbacks of our state-of-the-art solution compared to the classical proposal for manipulator robots. The proposal on how our solution will work on the MSDR is done on a drilling process for a fuselage metal plate. This Chapter is organized as follows: The first Section 6.2 details the change to be made in the formulations of the robot model, controller set of tasks and control law and how it will affect the results. Then, Section 6.3 presents an abstract method to create the interface between the robot low-level and our controller. Finally, the application proposal of our passive scheme compare to a classical drilling process is done in Section 6.4

Whole Body Control Formulation

Robot Model

Because the MSDR is fixed to the ground, it has no under-actuated part. It can nonetheless be put on a forklift truck to be moved in the fuselage cell. Then, a state estimation is necessary to check the base position of the robot and correct possible errors by updating the frame placements. In some specific setup, additional auxiliary axis mounted within the end-effector might be considered as source of under actuated parts. They are not considered here as the concept behind the MSDR is closely related to simple end-effector without additional axis modifying its absolute position.

Consequently, in this chapter, the variables q, q and q will be used to refer to the joint position, velocity and acceleration, respectively (as there is no under-actuated part).

The MSDR robot has stiffer joints (by design) than traditional industrial robots, as opposed to the flexible manipulators of Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF]. Then, the equation of the robot dynamics (Eq.2.4) can be re-written as:

M (q)q + C(q, q) q + g(q) h = τ + τ ext J T c F (6.1)
with M ∈ R n j ×n j the symmetric and positive definite inertia matrix, C ∈ R n j ×n j the Coriolis matrix and g ∈ R n j the gravity vector. q ∈ R n j is the the Denavit-Hartenberg Bejczy [START_REF] Antal | Robot arm dynamics and control[END_REF] representation of the robot joint angles. q, q ∈ R n j are the accelerations and velocities of the joint configuration of the robot. τ ∈ R n j are the joint torques of the actuators and τ ext ∈ R n j are the external torques.

The main difference from the humanoid robot formulation is that the accelerations a is replaced by the joint accelerations q and thus the section matrix N disappears.

Tasks

Priority Weight End-Effector contact I 100 Cartesian Force-Contact I 50 Posture regularization I 0.1 Table 6.1: Set of tasks that can be implemented for the MSDR using TSID.

Set of Tasks

As explained in the Section 1.5, the stack of tasks used for a humanoid robot and for manufacturing robots are similar (see Fig. 1.9). Because the base of the robot is fixed, one can neglect to add a task keeping the base joint at the same position. Moreover, controlling the CoM of classical robot manipulators is usually not a major concern. Indeed, for robot manipulators, static stability is ensured since they are fixed to the ground and that the gravity effects on the joint torques are generally compensated for, either actively or using passive mechanical devices Albu-Schäffer et al. [START_REF] Albu-Schäffer | A unified passivity-based control framework for position, torque and impedance control of flexible joint robots[END_REF], Schindlbeck and Haddadin [START_REF] Schindlbeck | Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks[END_REF]. Unfortunately, most of the time these compensations are implemented by the industrial robots manufacturers without providing access to their algorithms or intrinsic parameters.

The order of priority of the tasks is set using their weights, indeed, each task is put in the cost function (see Table .6.1). One can set the contact task as a constraint but it can be avoided as we will implement additional constraints on the force and it can over-constrain the problem. Nonetheless, the contact task is set as the higher priority (with the higher gain) as it is necessary to remain in contact with the plate to drill and the position precision is important. Then, the force task is set as the second most important task to follow accurately the force profile in order to successfully drill the plate. Finally, a posture regularization task can be added to control the remaining DoFs of the robot. Table .6.1 presents the tasks that seem appropriate to implement to realize a force manufacturing operation.

The end-effector contact task is necessary to control the position of the cutting tool while drilling. Indeed, even if the robot is quite stiff, spring effects can still appear in the joints of the robot and the robot parts can wrap while applying huge forces. Moreover, spring back effects can be propagated due to the mechanical properties of the aircraft part material. This can lead to deflect in the position of the end-effector. In the actual industrial process, two strategies are implemented to tackle this issue unplanned displacement: the use of a clamping force and slippage threshold.

Clamping force: To avoid any unexpected movement during drilling operation the end effector must be stuck against the aircraft part all along drilling cycle. To achieve this, the robot must apply a clamping thrust against the aircraft part. The clamping force is dependant on the stiffness of the surface. However, in the process of clamping, the robot can slip over the surface.

Slippage avoidance:

As stated Eq.2.11, to avoid the end-effector slippage along the parallel axis of the surface, it is necessary to have a the norm of the perpendicular force greater or equal than the norm of the normal one times the friction coefficient of the surface. However, this formulation leads to huge application forces which can damage the surface and/or the end-effector. Thus an empirical slippage threshold is chosen in practice.

Therefore, the end-effector contact task will be modified to take these implementations into account as constraints in the QP formulation (replacing the Coulomb one). Note that this task still implements the motion task of Eq.4.8 in the cost function to control the position of the end-effector. Moreover, the reference position of this task can be computed using the visual information of the robot (cameras, lasers) to adjust locally the exact position of the hole.

Quadratic Programming Formulation

The formulation of the QP in TSID is modified a little for the MSDR. Using the passive controller of Section 5.3.3, we can adapt the solution to a fully actuated robot with the previous contact constraints. The obtained QP, for the i ∈ {0, .., 3} previously defined tasks and their weights λ i is expressed as follows:

Find α, β, γ, Ḣ s.t. Eq.5.18 -5.23, 5.26

min y=[q] 3 i=0 λ i O i y -βγo i 2 s.t.   J c 0 0 M -J T c -N T     q F τ   =   -Jc v -h   τ min ≤ τ ≤ τ max F min ≤ F ≤ F max |f slip | ≤ |F EE ⊥ | ≤ |f max clamp | q min ≤ q ≤ q max qmin ≤ q ≤ qmax Ḣ ≤ -v T N T τ (6.2)
with F EE the force at the end-effector, f max clamp and f slip the max clamping force and slippage threshold defined previously. We add constraints on the angular limits in position with q min , q max and in velocity qmin , qmax . Then the free variables of the problem are the joint accelerations q. The equations 5.18-5.23 and 5.26 can be solved for the MSDR robot as well has for any humanoid robot, one just has to replace the v variable by q and to remove the N matrix. Indeed, because there is no free-flyer the velocity vector v only contains q (the other variables are null) and there is thus no need of the selection matrix N . The calculations remain the same otherwise and lead to the same result for the energy tank.

Interface between the robot low-level and our controller

Once the problem is re-formulated for the MSDR, the QP can compute a torque command respecting the passivity of the system. However, it needs the robot actuators angular position and velocity and the forces measured at the end-effector level. This data must be retrieved in a timely manner to be able to execute the QP in real time. In the case of the MSDR, to have access to this data it is necessary to use the Fanuc software.

In the control scheme implemented on the TALOS robot, the data transfer is performed by the ros-control framework of the middle-ware ROS Adolfo Rodríguez Tsouroukdissian [START_REF] Rodríguez | Ros control, an overview[END_REF]. Recalling the scheme of the Fig. 1.10, the roscontrol-sot package can communicate with the hardware interface of ros-control to retrieve the lowlevel information. Thus, to solve the issue of communication between the hardware of the MSDR and our control scheme, one can create the ROS robot hardware interface of the MSDR to communicate with the Fanuc software and retrieve the information. Tutorials exist online to implement your own hardware interface, it is however important to notice that the data of the MSDR can be internal sensors embedded within the robot (encoders, current, force sensors) or external sensors of the end-effector (lasers for normality, cameras, IMU, force sensors).

One issue that may arise is the importance of the delay between the reading of the sensors/actuators data by the Fanuc software and its transmission to the interface and to the controller. And vice-versa, if the delay between the command sent by the controller toward the Fanuc software and its application on the robot is too important it can lead to precision issues. One has to notice that in any case, the communication within the robot manufacturer's controller interfaces is mainly best effort and not real-time (∼ 250Hz), as it is not the prioritized process. However, to reduce the delays, it can be appropriate to have the Fanuc software implements a quick regulation control loop which takes the high-level controller input at a low frequency depending of the delay (for instance 500Hz). Its role will be to regulate the high-level commands for the low-level until reaching the defined time to read the new high-level control input. This regulating controller can for instance be achieved by a PD+ (as the controller used by the PAL-Robotics company) or a Regulation Sensitivity and Tracking (RST) polynomial controller (more common in industry for digital control). The RST is a robust control able to track reference trajectories as well as to regulate the control during disturbances. The PID is a particular form of RST, with a particular choices of the polynomials R, S and T.

In Fig. 6.2 is presented an example of complete solution incorporating all the elements from the planning to the control of the robot. In yellow are depicted the the visual processes to detect the holes (object recognition) and localize the metal plate and the robot (using for instance the Simultaneous Localization And Mapping (SLAM) approach). In red are presented the planning algorithms, which follows the stages detailed in Fig. 2.7: first plan the motion, then find the contact poses and finally optimize the trajectory with a MPC. The motion can be decomposed in several sub-motions or states, which can be organized in a finite-state-machine. The state will present different constraints or tasks such as the presence or absence of contact for instance. The current state is thus given to the whole-body controller to implements the appropriate set of tasks and constraints to achieve the desired motion. The controller is in the black rectangular box in the figure, with the characteristic necessary to ensure the passivity of the system (energy tank). Finally, the interface with the Fanuc software and the low-level controller is presented in blue between the robot and the controller.

Drilling Process Solution

Here we present a typical drilling process on a fuselage. Between two reference preholes (R1, R2, pre-hole means that the holes have not the final drill diameter) drilled by an operator, ten holes are performed by the machine, with a reference axis frame based on R1 and R2 (see Fig. 6.3). The drilling process follows the steps detailed in the Table.6.2, illustrated in the Fig. 6.3.

The position and orientation adjustments are done with the vision system and the normality correction with the lasers embedded in the end-effector. The current Figure 6.3: Drilling operation with 10 holes between two reference holes (R1, R2).

Step Description 0 Initial position: MSDR moved to R1 1

Adjust the position and check the normality between the TCP and the surface 2

Move the TCP to R2 robot program is a point to point offline path planning executed without modifications at run time (outside of the offsets adjustments on R1, R2). This nominal process correspond to a coordination between tasks and resources (robot, humans and fuselage) which is sequential. When a problem arises, the recovery process is typically the following:

The machine has failed to perform the task with the requested quality An operator needs to apply a repair process, before the machine restarts, or at the end of the machine process.

Which means that, as presented in the context section, the operator and the robot cannot currently work together, the machine has to be stopped for an human to act, and failure in the machine program stop the process. Moreover, in order to restart the robot production process, its program must be modified and re-validated offline. This leads to delays in the production chains.

The targeted goal is to achieve a concurrent coordination, where the robot can adapt from the planning if an error occurs to correct its task or skip the problematic hole completely to continue the process (even onto another part of the plate). Then, the operators can fix the issue while the robot continue the drilling process. It can be resumed as follows:

All points are considered in the robot program and on operators tasks backlog (to be fixed only if the robot does not succeed to complete the task) When the robot moves to a point where tasks are performed, it skips the task to the next point or to the next subgroup If the robot is not able to realize its task with success, the task is re-planned or aborted and skipped, and the robot continue to the next target.

With the proposed solution of Fig. 6.2, involving visual feedback, planning, finitestate machine and reactive control, it is possible to realize this targeted concurrent coordination.

Another point of interest is the safety of the robotic solution, indeed, with the concurrent solution (and even with the sequential one), the operators can perform manual tasks very close to the robot. Thus, the collision shall be avoided to ensure human safety. For now, the solution implemented is to progressively decrease the speed of the robot when the robot get closer to humans or the fuselage. What can be used in our proposed solution is the reactive planning of "collision free" path that can be computed on demand, while executing the initial path. Moreover, using our passive torque controller allows to benefit from the natural compliance of torque control and from the passivity of the system. This means that, subject to unplanned disturbances, the control will be penalized to ensure the safety of the whole system (environment, humans and robot), see Chapter 5. Thus, the tasks performed will be degraded or aborted in favor of the security. However, the qualification of this safety process for production will be difficult.

Conclusion

In this chapter is presented the two challenges that we think may arise when we will transfer our passive torque controller onto the MSDR. The reformulation of the optimization problem to match the robot fully-actuated model and new force constraints has been presented. Then, the issue of interfacing the hardware and low-level control of the robot to our solution has been discussed, a ROS hardware interface between the Fanuc software and our control scheme is possible. Finally, a proposal of complete control scheme using some of the Gepetto team works and the passive controller has been presented. Its application on a drilling process has been explored and the benefit of it stated. In particular, our solution improve the adaptability, reactivity and safety of the robotic solution.

The future works consist in implementing the proposed solutions on the MSDR robot and test them on the drilling process. This will however require a consequent amount of time to create a self-sufficient "tool-box" from the Gepetto team software. Moreover, the certification of the programs termination (analysis of time complexity) and robustness is an important security process for Airbus, which is not currently achievable. Despite this blocking point, Airbus is interested to transfer the developed solution of this thesis on the MSDR. A first step could be to directly implement a simple passive PD+ control using the Fanuc software. A second step may be to implement the complete solution as a "tool-box" in the scope of the joint-lab ROB4FAM.

Conclusion Summary

This thesis aims to answer the issue of robotics whole-body control for manufacturing operations. During the development of my solutions, two major objectives had to be considered, the safety of the operation for the robot and for its environment, and the capabilities of the whole-body controller to achieve force application while keeping its balance.

A state-of-the-art is first presented in the Chapter 2. It details the robot model and centroidal dynamics formulation used in my thesis, the different type of control designed on humanoids robots for locomotion and manipulation scenarios and the stability criteria considered in my works.

Secondly, a process for identifying and modeling the elbow chain actuator of the robot TALOS is described in the Chapter 3. This model and the identified parameters are used to design a low level current controller to protect the robotic system using a MPC. The process of identification can be applied on any rigid chain actuator (for any robot) to retrieve its inertial parameters, it is only subject to correctly model the chain actuation. Even if the MPC is computationally slow, it allows TALOS to carry high payload (up to 34 kg on one arm) while realizing motions and protects the system. The Chapter 4 details the three whole-body controllers implemented in this thesis, two in position and one in torque. Their respective tasks functions and formulations are presented to achieve locomotion and manipulation scenarios. This chapter compare them in simulations on complex locomotion problems such as stair climbing and walking on non flat terrain. These simulations have yet to be tested on the real robot using the modeling of the hip flexibility of TALOS. The intermediate steps performed on the real robot are detailed as well as the blocking points preventing us to successfully achieve the experiments. Because the torque controller has proven to be energy efficient, capable of quite high speed locomotion and adaptable to uncommon scenarios (such as navigating in collaboration with a human, while using similar gait); I chose to focus on this solution for the next developments of the thesis.

Yet, using this controller on the real TALOS robot leads to instabilities in the solution on the simplest scenario. Thus, to realize the objective of safe operations, stability analysis and in particular the passivity analysis methods are investigated in the Chapter 5. I implemented a novel formulation for my torque control scheme to obtain a passive system (a dissipative system), ensuring a stable, safe and robust behavior of the system. My whole-body control scheme embeds an energy tank monitoring the exchanged power in the system. The tank regulates the controller tasks gains in function of the input-output system power. This new scheme is tested in two simulations. First on a multi-contact scenario employing a parallel position/force task as a first step for manufacturing operations. Secondly on a simple locomotion scenario, a walk of 20cm steps on flat ground.

Finally, the Chapter 6 presents the expected difficulties to consider when transferring the passive solution from the humanoid robot TALOS to the manipulator robot MSDR. The two main aspects to consider are the reformulation of the optimization problem to match specific force constraints and the creation of an interface between the Fanuc software and my controller. This chapter also highlights the benefits that my solution and, in a more global approach, the planning and visual methods of the Gepetto team, can bring. The team hopes to improve the adaptability, reactivity and safety of the industrial robotics solution.

Future Work Identification and Modeling of the Whole Chain Actuation

The extension of the work presented in the Chapter 3 would be to identify all the actuators of the robot and to implement the solution over high-performance dedicated and embedded electronics board attached to each actuator of the robot. However, due to the sanitary crisis this identification process has been delayed.

Moreover, with the model and parameters of each of the actuator it will be possible to implement a torque controller mapping the desired joint torques of the QP controller to motor currents and send them directly to the robot. This way, there will be no need to use the PAL-Robotics company low-level controller to try my simulations on the real robot. For now, this proprietary controller is a black box and I encounter some problems to interface my complete solution with it even if it uses a ros-control architecture, as presented in Section 4.6. This is why, currently, even the simulation of the contact-force has not be tested on the real robot, while the flexibility of the robot is not a blocking point. It is a known difficulty of torque control implementations to successfully transfer the simulation results on the real robot. I should have started to work on the interface with the PAL-Robotics lowlevel controller sooner to solve this problem. Or, it might have been quicker to try to create a simple PD+, validated in the PAL-Robotics simulator, to transform my desired joint torques to motor currents and completely bypass the black-box controller.

Interfacing my Controller with the Flexibility Compensation Model

To finalize the work presented in the Chapter 4 and test the walking simulations on the real robot, it would be necessary to implement the flexibility compensation of Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF] in my controllers. In this paper, the hip bending is estimated from the commanded torque and the elastic deflection (θ) is added to the measured hip configuration q hip . The same is done for the derivative of the deflection. The new values of the hip configuration and joint velocities are then replaced in the coordinate and velocity vectors q and v and are used by the base-estimator and the controller. Because the implementations done in Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF] have been made using TSID, it should be easy to join my controllers with them and then test the simulations on the real robot. Be aware that for torque control, the previous point still holds and the problem with the PAL-Robotics low-level controller must be solved. Furthermore, in Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF], the modified coordinate and velocity vectors q and v taking into account the deflection are also used in a locomotion planner to compute coherent reference trajectories. An interesting future step would be to use this planner in tandem with my controllers to adequately take into account the hip flexibility of the robot TALOS.

Finally, with the modified torque controller using the flexibility compensation, one would want to continue the collaboration with Isabelle Maroger on the ANR-COBOT project. The aim would be to test the whole developed framework for human-robot co-navigation in real time on the robot TALOS. In particular, using the contact-force task added in my torque controller and the passivity implementation, it would be amazing to have TALOS successfully carry and move a table in collaboration with a human. Thus, the stability of the system would be guaranteed and it would be possible to control the amount of force the robot has to apply on the table (currently only the control of the robot hands positions is possible).

Improving the Energy Tank Shaping and Regulating Coefficients

To improve the results obtained in the Chapter 5 on the passivation of the control scheme, one would revise the definitions of the budget of the energy tank and of the regulating coefficients. Indeed, for now, the robot may falls if all the system is penalized to respect the passivity, because the coefficients β, γ are acting on every tasks. However, as discussed before, the passivity proof will not hold if these coefficients only act on some of the tasks. One can try to implement an energy tank for the tasks responsible of the robot balancing, such as the CoM, the feet contacts and the posture ones and another tank for the other tasks. Then, if the activation of the system does not come from the balancing tasks, they will not be penalized and the robot would keep its balance.

Moreover, it would be interesting to design the budget and the maximal value allocated to the energy tank in function of the reference trajectories. The upper bound of the tank might then increase or decrease in function of the planning. For instance, while moving on flat floor the robot does not need too much energy, but if it has to climb an obstacle the necessary amount may rise. Thus, one would increase the maximum value of the tank a little before-hand to store energy to anticipate the climb. Similarly, the coefficient on the power flow regulation γ can be computed as in Shahriari et al. [START_REF] Shahriari | Power flow regulation, adaptation, and learning for intrinsically robust virtual energy tanks[END_REF] to vary in function of the task progression, close to what I described before for the maximal value of the tank, using the reference trajectories. Finally, one would want to use the regulating coefficients of the energy tank in higher level monitoring mechanisms to improve the handling of the diverging cases.

Reformulating the QP Formulation or Developing the Passivity Criterion on a MPC

Another way to improve the results obtained in the Chapter 5 would be to reformulate the alternate minimization problem of Eq.5.34 to solve the energy parameters concurrently to the complete optimization problem (subject to constraints). Then, the passivity as a constraint in the QP will not be necessary as the energy parameters will be optimized on the same problem formulation than the QP (including the constraints of the system). For now, the addition of the passivity constraint in the formulation can lead to a too constrained problem and it is possible that it enters in conflict with another constraint and lead to oscillations in the solution.

A second possibility could be to implement the passivity criterion in a MPC, for instance the one developed by Dantec et al. [START_REF] Louis Dantec | Whole Body Model Predictive Control with a Memory of Motion: Experiments on a Torque-Controlled Talos[END_REF] which has achieved great results recently. The passivity can be implemented by using the value function of the Optimal Control as the storage function and constraining it. Or it is possible to reformulate the problem and to add a sate constraint over the trajectory to respect the passivity inequality Raff et al. [START_REF] Raff | Nonlinear Model Predictive Control: A Passivity-Based Approach[END_REF].

Testing the Complete Approach in a Factory

Finally, one of the future goal would be to bring the TALOS and TIAGo robots to an Airbus factory and test in real conditions the overall approach including localization, visual-servoing and control. This aim was first expected by the end of my thesis, but due to the sanitary crisis, we accumulate delays and could only successfully test the localization and navigation stack of the robot TIAGo. Once some of the previous points are fixed, one would try to demonstrate the developed solutions in the industrial environment. This is a future goal for the ROB4FAM joint-lab, which could be achieved before or in parallel with the implementation of the results on the MSDR. ∆V ← value > 5:

while ∆V > do 6:

V (T ) ← l T (x T )

7:

V x (T ) ← l T x (x T )

8:

V xx (T ) ← l T xx (x T ) Backward Phase

9:

for i = T -1; i ≤ 0; i = i -1 do 10:

Q x ← l x + f T x V i+1
x 11:

Q u ← l u + f T x V i+1 u 12: Q xx ← l xx + f T x V i+1 xx f x + V i+1 x f xx 13: Q uu ← l uu + f T u V i+1 xx f u + V i+1
x f uu 14:

Q ux ← l ux + f T u V i+1 xx f x + V i+1
x f xx 15: 16:

∆V i ← -1 2 Q u Q -1 uu Q u 17: V i x ← Q x -Q u Q -1 uu Q ux 18: V i xx ← Q xx -Q ux Q -1 uu Q ux 19: 20: k i ← -Q -1 uu Q u 21: K i ← -Q -1 uu Q ux 22:
end for Forward Phase 23: x(0) ← x 0 24: for i = 0; i ≥ T -1; i = i + 1 do 25: û(i) ← u(i) + αk i + K i (x(i) -x(i))

26:

x(i + 1) ← f i (x(i), û(i)) The upper-scripts of l, f, k, , K, V correspond to the iteration numbers (i = 0 to T ), while for x, u this number is in subscript.

The backward pass is initialized with the value function set to the terminal cost and its derivatives (l.6 -8). Then Q is filled according to the Jacobian and Hessian of its pseudo-Hamiltonian form (l.10 -14), and V updated (l. 16 -18). The terms k and K correspond respectively to the feed-forward and feedback terms of the optimal control local policy updating u (l.25 -26) defined by Eq.2.78. The new position command is then defined by: ∆q hip = τ hip K q hip cmd = q hip des + ∆q hip (6.3)

Sot torque control framework for position control

However, the noise in torque sensors requires filtering, which introduces important delays. Unfortunately, previous experiences with such a technique carried out with the robot gave poor performances. Only a repeatable one step walk experiment on TALOS in position control was achieved using fixed flexibility compensation (see Section 4.6). In torque control however, this issue is mitigated because the flexibility is considered by the control system as external disturbances. Nonetheless, to achieve robust experiments, it will be necessary to take into account this flexibility which cannot be compensated without a proper identification and modeling.

With Model

A second method has been tested to control the hip flexibility in Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF], relying on the use of a specific model for the flexibility. It uses the standard model defined by Nakaoka et al. [214], introducing passive joints in the waist-leg connection, where the link cross-section is reduced, as the link deflections of the robot concentrate there. The torque on each passive joint is related to its deflection θ as a spring-damper:

τ f = -k f θ -d f θ, (6.4)
with link stiffness k f and damping d f coefficients.

As the stiffness is coming for the vertical linkage, only the flexibility along pitch and roll deflections is modeled, which produce the main impact on foot placement. As a result, the model for the robot TALOS is augmented to reach 42 DoFs composed by 32 actuated joints, 4 elastic passive joints and the robot free-flyer. The obtained results in Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF] are quite satisfying, see the video at the following link: https://peertube.laas.fr/videos/watch/9a3c5258-e5b7-49a5-a153-02e804a06f65.

With this modeling a new value of the stiffness flexibility has been identified. It is important to notice that between the identification experiments the legs have been unmounted and a backlash in the knees have been corrected. The second experiment consists in lifting a leg and putting the CoM at the center of the support foot, to match the CoP position computed from the F/T sensors, it allows to identify the static stiffness. For the left foot, without compensation, the CoP does not equal the CoM, it is naturally going toward the outside of the robot. With the compensation and a stiffness set at 3000 Nm/rad, the CoP and CoM nearly match. With a lower stiffness, the CoP goes toward the inside of the robot. This observation is confirmed by checking the torque measured by the ankle F/T sensors. There should be no torque around the X-axis once the reference is reached, but there is ∼ 20Nm toward the outside of the robot without compensation. Using the flexibility compensation model with a stiffness of 3000 Nm/rad the torque is lowered to ∼ 5Nm and decreasing the stiffness makes the torque goes to the opposite direction. Thus, the second estimation of the stiffness flexibility found a value of ∼ 3000 Nm/rad on the left leg. For the right leg a value of ∼ 4750 Nm/rad has been identified.

It is a work in progress to interface the controllers developed in this thesis and the compensation model implemented in the paper Villa et al. [START_REF] Nahuel | Addressing flexibility in biped locomotion with robust control and closed-loop model-predictive control[END_REF].

It corresponds to a change of basis of K p is a real symmetric semi-positive definite matrix. Thus its eigenvalues are real and non-negatives (Definition.4) and so are the ones of Λ because of Eq.6.6. Then we have proven our first result: the eigenvalues of ΛK p are real and non-negatives.

Semi-positive definition of ΛK p

In this part we look at a way to prove the semi-positive definition of ΛK p by studying its symmetric part in the Toeplitz decomposition: Definition 6 (Toeplitz decomposition). Every square matrix A can be decomposed uniquely as the sum of two matrices U and V , where U is symmetric and V is skew-symmetric.

A = U + V = 1 2 (A + A T ) + 1 2 (A -A T ) (6.7)
In our case A = ΛK p and U = 1 2 (ΛK p + (ΛK p ) T ) is symmetric.

We recall the fact that the quadratic form of a skew-symmetric matrix equals to zero. Indeed, by definition V T = -V and thus x T V x = (x T V T x) T = -x T V x which holds only if it equals to zero. Thus, the quadratic form of x T ΛK p x is the same one of x T U x, i.e:

x T ΛK p x = x T ( ΛK p + (ΛK p ) T 2 )x (6.8)

One can thus prove the semi-positive definition of the symmetric matrix U to prove the semi-positive definition of ΛK p . Indeed, if x T U x ≥ 0, ∀x ∈ R n \{0}, using Eq.6.8, we obtain x T ΛK p x ≥ 0, ∀x ∈ R n \{0}: proving of the semi-positive definition of ΛK p .

One way to prove the semi-positive definition of U is to look at its eigenvalues. Because U is symmetric, if its eigenvalues are non-negatives then U is semi-positive definite (see Definition 4). This gives our second result: if U has non-negative eigenvalues, then ΛK p is semi-positive definite.

On the eigenvalues of U

One may notice that we have further information on the eigenvalues of U with respect to the ones of ΛK p . Using the following theorem of Fan on matrices [START_REF] Zhang | Matrix theory: basic results and techniques[END_REF] (Chapter 10, Theorem 10.28):

Theorem 1 (Eigenvalues majorization of Ky Fan). Let A be an n × n matrix with eigenvalues λ 1 (A), ..., λ n (A) and R e λ i (A) their real parts. Then:

n i=1 R e λ i (A) ≤ n i=1 λ i
A + A T 2 (6.9)

Then, because the eigenvalues of ΛK p are real and non-negatives (as proven in the first part), we have using the Theorem 1:

0 ≤ n i=1 λ i (ΛK p ) ≤ n i=1
λ i (U ) (6.10) Thus, we know that the sum of the eigenvalues of U is non-negative, however to prove the semi-positive definition of U it is needed to prove that all its eigenvalues are non-negatives.

Glossary

Résumé

Les robots humanoïdes ne sont pas encore capables de travailler en sécurité dans un environnement fait pour l'Homme et d'effectuer les mêmes tâches. L'objectif de cette thèse est d'étudier des algorithmes de contrôle corps-complet pour robots humanoïdes afin d'effectuer des opérations de productions de structures avions telles que le perçage. Cette thèse s'inscrit dans le cadre du laboratoire commun entre le LAAS-CNRS et la société Airbus Operations SAS. Les travaux et études présentés appartiennent aux domaines scientifiques de l'optimisation pour le contrôle de systèmes dynamiques et de l'analyse de leur stabilité.

Les contributions majeures de cette thèse sont la conception et l'implémentation de nouveaux algorithmes de contrôle temps-réel pour des robots humanoïdes, intégrés sur le robot TALOS. Les applications présentées traitent de la locomotion du robot et de la réalisation d'opérations nécessitant l'application de force. Premièrement, les paramètres de la chaîne d'actionnement du robot sont identifiés afin de commander directement le robot en courant tout en garantissant la protection du système. La solution proposée confirme les capacités du robot TALOS contrôlé en couple lors de la manipulation d'une charge élevée. Ainsi, des algorithmes de contrôle temps-réel en couple pour le corps-complet du robot sont ensuite étudiés. Trois contrôleurs sont implémentés et comparés, deux en position et un en couple. Leur analyse est effectuée sur des scénarios de locomotion complexes en utilisant plusieurs critères comme le suivi de trajectoires et la dépense énergétique. Le contrôleur corps-complet en couple est validé en simulation et ses avantages par rapport aux schémas en position confirment le choix de l'implémenter pour des opérations de productions. Cependant, les premiers tests réalisés sur le robot réel ont conduit à une dangereuse divergence de la solution. C'est pourquoi, une analyse de stabilité a été effectuée pour assurer la sécurité et la robustesse de la solution pour des applications industrielles. La solution développée est basée sur la théorie de la passivité, augmentant le contrôleur en couple avec un réservoir d'énergie contrôlant les transmissions d'énergie au sein du système. Ce nouveau contrôleur passif est validé en simulation sur des scénarios de locomotion et de multi-contacts avec l'environnement. Ce dernier met en place une tâche d'application de force comme première étape vers des opérations de productions.

Les résultats de cette thèse ont été intégrés dans la suite logicielle Stack-of-Tasks du LAAS-CNRS.

Mots clés : Contrôle corps-complet, Robots humanoïdes, Contrôle en force/ couple

Abstract

Humanoids robots are not yet able to safely work in similar environment than humans and to perform the same tasks. The objective of this thesis is to study whole-body control algorithms for humanoids robots in order to perform aircraft manufacturing operations such as drilling. This thesis is part of the joint-lab between the LAAS-CNRS laboratory and the Airbus Operations SAS company. The presented research and innovative studies rely on recent advances on control theory, optimization and stability analysis.

The main contributions of this thesis are the design and implementation of new real-time controllers for humanoid robot, integrated on the robot TALOS. The illustrated applications are locomotion and force task operations. First, the chain actuation parameters of the robot are identified in order to directly control the robot in current while having low level protection mechanisms. The proposed solution confirms the capabilities of the robot TALOS to achieve torque control in complex scenario with high payload. Thus, real-time whole-body torque controllers are then investigated. Three controllers are implemented and benchmarked, two in position and one in torque. Their comparison is performed on complex locomotion scenarios using several metrics including trajectory tracking and energetic criteria. The whole-body torque controller is validated in simulations and its advantages compared to the position schemes confirms the decision to implement it for manufacturing operations. However, the first tests realized on the real robot led to a dangerous divergence of the solution. Thus, a stability analysis was performed to ensure the safety and robustness of the solution for industrial applications. The developed solution is based on passivity theory, augmenting the whole-body torque control scheme with a global energy tank monitoring the power flow of the system. This new passive scheme is validated in simulations on locomotion and multi-contact scenarios. The latter involves force applications as a first step toward manufacturing operations.

The results of this thesis have been integrated in the Stack-of-Tasks framework of the LAAS-CNRS.

Keywords: Humanoid Whole-Body Control, Humanoids Robots, Force/Torque Control

viii LIST OF FIGURES 2 . 78 3. 5 82 3. 8 85 4. 1 95 4. 2 117 Chapter 1 Context of the thesis Contents 1 . 1 3 1. 3 4 1. 4 4 1. 4 . 1 6 1. 4 . 2 8 1. 5 1 . 6 14 1. 6 . 1 15 1. 6 . 3 16 1. 7 16 1. 8

 27858288519521171113344441642851614611563167168 16 Scheme illustrating the DDP algorithm . . . . . . . . . . . . . . . . . 63 3.1 Scheme of the retained mechanical model. . . . . . . . . . . . . . . . 69 3.2 Equivalent electric model of the simple actuator thermal model. . . . 71 3.3 Thermal parameter identification . . . . . . . . . . . . . . . . . . . . 77 3.4 Results of the fitting of motor and joint torques used for the identification process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results of the joint torques identification . . . . . . . . . . . . . . . . 80 3.6 Experiment where TALOS is holding 34kg at 5 cm of the elbow joint. 813.7 Simulated state trajectory illustrating the angular exponential barrier. Comparison between simulated trajectories with and without load. . . 82 3.9 Simulated control trajectories with and without an additional load. . 83 3.10 Simulated state and control trajectories in loaded case but without control input limitation. . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.11 Experiments -State trajectory illustrating the angular exponential barrier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.12 Experiments -State trajectories with and without a 34kg load. . . . . 85 3.13 Experiments -Control trajectories with and without a 34kg load. . . Walking on Uneven Terrain and Climbing Stairs. . . . . . . . . . . . Position control schemes: IK and TSID. The OR block is used to activate only one controller at a time. . . . . . . . . . . . . . . . . . . 96 4.3 TSID torque control scheme. . . . . . . . . . . . . . . . . . . . . . . . 97 4.4 ZMP estimation of the 20 cm step walk. . . . . . . . . . . . . . . . . 101 4.5 DCM estimation of the 20 cm step walk. . . . . . . . . . . . . . . . . 101 4.6 Z-axis left foot force of the 20 cm step walk. . . . . . . . . . . . . . . 102 4.7 Feet, CoM, DCM and ZMP of the 60 cm step walk. . . . . . . . . . . 103 4.8 AM behaviour during the 60 cm step walk in torque. . . . . . . . . . 103 4.9 ZMP estimation of the tilted platforms simulation. . . . . . . . . . . . 104 4.10 Z-axis left foot force of the tilted platforms simulation. . . . . . . . . 105 4.11 ZMP estimation of stairs climbing. . . . . . . . . . . . . . . . . . . . 106 4.12 Simulation on Gazebo and RViz of the robot executing a predicted trajectory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.13 Description of the whole framework of the Human-Robot Collaboration for Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.14 Representation of the 3 scenarios of the Human-Robot Collaboration for Navigation study . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.15 RViz simulation of a collaborative navigation using the NMPC. . . . 115 4.16 Tracking of the CoM and Feet trajectories in the Gazebo simulation where the robot is synchronized with the human (scenario 2). . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 ROB4FAM . . . . . . . . . . . . . . . . . . . . . . . . . . . Objectives and Contributions of the Thesis . . . . . . . . The robotics platforms: TALOS, TIAGo and MSDR . . Humanoid robot TALOS . . . . . . . . . . . . . . . . . . 4 1.4.1.a The flexibility on the hip of TALOS . . . . . . . Wheeled Manipulator robot TIAGo . . . . . . . . . . . . 7 1.4.3 Middle Size Drilling Robot . . . . . . . . . . . . . . . . . Scientific Context . . . . . . . . . . . . . . . . . . . . . . . 10 Frameworks and Libraries . . . . . . . . . . . . . . . . . . Pinocchio Library . . . . . . . . . . . . . . . . . . . . . . 14 1.6.2 Stack-of-Tasks (SoT) Framework . . . . . . . . . . . . . . 14 1.6.2.a Dynamic Graph . . . . . . . . . . . . . . . . . . 14 1.6.2.b ROS-Control SoT . . . . . . . . . . . . . . . . . 14 1.6.2.c SoT Core . . . . . . . . . . . . . . . . . . . . . . 15 1.6.2.d Overall scheme for real-time control . . . . . . . Task Space Inverse Dynamics (TSID) Library . . . . . . . Summary of the Chapters . . . . . . . . . . . . . . . . . . Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Figure 1 . 1 :

 11 Figure 1.1: From HRP2 to TALOS.

Figure 1 . 3 :

 13 Figure 1.3: Pyrène head with the LIDAR and the two cameras.

Figure 1 . 4 :

 14 Figure 1.4: Left: Humanoid robot TALOS. Right: Manipulator robot TIAGo

Figure 1 . 5 :

 15 Figure 1.5: Airbus automated fuselage structure assembly line at Hamburg

Figure 1 . 6 :

 16 Figure 1.6: Middle Size Drilling Robot.

Figure 1 . 8 :

 18 Figure 1.8: Parallel between the control architecture of a humanoid robot and manipulator robots: TALOS stack of tasks (WP).

Figure 1 .

 1 Figure 1.9: Parallel between the control architecture of a humanoid robot and manipulator robots: Manipulators stack of tasks (WP).

Figure 1 .

 1 Figure 1.11: SoT framework

Figure 2 . 4 :

 24 Figure 2.4: TALOS Actuator illustration: two modular-magnetic-encoders surround the AC-motor connected to the Harmonic Drive. This figure is part of the WP reports.

Figure 2 . 5 :

 25 Figure 2.5: Scheme of the locomotion problem for humanoid robots, based on Naveau et al.[START_REF] Naveau | A reactive walking pattern generator based on nonlinear model predictive control[END_REF], Giraud-Esclasse[START_REF] Giraud-Esclasse | Towards a reactive motion generation on exteroceptive feedback for generalized locomotion of humanoid robots[END_REF]. The global problem in the green rectangle (wholebody trajectory for the full preview future) is currently impossible to solve in milliseconds. However, it is possible to solve the blue box: the whole-body problem for the nearest future, as well as the linearized problem at the CoM level (orange box), over the full preview.

Figure 2 . 6 :

 26 Figure 2.6: Scheme illustrating the Walking Pattern Generator and the Dynamic Filter Scherrer [1].

Figure 2 . 7 :

 27 Figure 2.7: Planning a guide (a) Finding contacts poses (b) MPC on Centroidal dynamics (c) Whole-body control (d)[START_REF]multicontact-locomotion-planning[END_REF]

Figure 2 . 8 :

 28 Figure 2.8: Left: WALK-MAN. Center: TORO. Right: HRP4.

Figure 2 .

 2 Figure 2.9: Left: JAXON. Center: Atlas. Right: Gazelle.

Figure 2 .

 2 Figure 2.10: Simple Analysis of the passivity of a robotic system.

Figure 2 . 11 :

 211 Figure 2.11: Simple Admittance Control Scheme.

Figure 2 . 12 :

 212 Figure 2.12: Simple Cartesian Impedance Control Scheme.

Figure 2 . 13 :

 213 Figure 2.13: Simple Stiffness Control Scheme.

Figure 2 . 14 :

 214 Figure 2.14: Example of Hybrid Control Scheme.

Figure 2 . 15 :

 215 Figure 2.15: Example of Parallel Control Scheme.

Figure 2 .

 2 Figure 2.16: Scheme illustrating the DDP algorithm: first the backward pass is solved to update the value function and then the forward pass is executed to obtain the new state and control values.

Contents 3 . 1 68 3. 2 70 3. 3 73 3. 4 . 1 DDP 75 3. 5 76 3. 5 . 1 76 3. 5 . 2 77 3. 5 . 3 81 3. 6

 316827037341755765176527753816 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . Modeling of the Rigid Chain Actuators . . . . . . . . . . 69 3.2.1 Mechanical Model . . . . . . . . . . . . . . . . . . . . . . 69 3.2.2 Actuator Thermal model . . . . . . . . . . . . . . . . . . Identification of the Rigid Chain Actuators . . . . . . . . 71 3.3.1 Identification of chain drive and segment inertial parameters 71 3.4 Differential Dynamic Programming Optimal Control Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . State space representation . . . . . . . . . . . . . . 73 3.4.2 Treatment of Constraints . . . . . . . . . . . . . . . . . . 73 3.4.3 Iterative Linear Quadratic Regulator (iLQR) . . . . . . . 74 3.4.4 State space partial derivatives . . . . . . . . . . . . . . . . 74 3.4.5 Cost function . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying the thermal model . . . . . . . . . . . . . . . . Identifying the chain drive and inertial parameters . . . . Controlling the actuator . . . . . . . . . . . . . . . . . . . 79 3.5.3.a Simulation . . . . . . . . . . . . . . . . . . . . . 79 3.5.3.b Experiments . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Figure 3 . 1 :

 31 Figure 3.1: Scheme of the retained mechanical model.

Figure 3 . 2 :

 32 Figure 3.2: Equivalent electric model of the simple actuator thermal model.

Fig. 3 .

 3 Fig.3.4 shows the results of the least square identification. It shows the fitting of the measured motor and joint torque when no payload was used. The corresponding Root Mean Square Difference (RMSD) was 0.6 N.m showing an excellent fitting. Fig.3.6.c shows the estimate of joint torque from the current of the motor that is of importance for the proposed dynamic controller. Table1details the comparison between the identified parameters and their values as provided by the manufacturer.

Figure 3 . 4 :

 34 Figure 3.4: Results of the fitting of motor and joint torques used for the identification process.

Figure 3 . 5 :

 35 Figure 3.5: (a) Velocity profile used to excite the dynamics system. (b) Estimate of the joint torques from motor current and the identified model. (c) Estimation of the difference between the joint and motor torque.

Figure 3 . 6 :

 36 Figure 3.6: Experiment where TALOS is holding 34kg at 5 cm of the elbow joint.

Figure 3 . 7 :

 37 Figure 3.7: Simulated state trajectory illustrating the angular exponential barrier.

Figure 3 . 8 :

 38 Figure 3.8: Comparison between simulated trajectories with and without load.

Figure 3 . 9 :

 39 Figure 3.9: Simulated control trajectories with and without an additional load.

Figure 3 .

 3 Figure 3.10: Simulated state and control trajectories in loaded case but without control input limitation.

Figure 3 .

 3 Figure 3.11: Experiments -State trajectory illustrating the angular exponential barrier.

Figure 3 .

 3 Figure 3.12: Experiments -State trajectories with and without a 34kg load.

Figure 3 .

 3 Figure 3.13: Experiments -Control trajectories with and without a 34kg load.

Figure 4 . 1 :

 41 Figure 4.1: Walking on Uneven Terrain and Climbing Stairs.

[ 153 ]

 153 , and a weighted QP which sets weights to prioritise the tasks Koolen et al.[START_REF] Koolen | Design of a momentum-based control framework and application to the humanoid robot atlas[END_REF], Bouyarmane et al.[START_REF] Bouyarmane | Multi-robot and task-space force control with quadraticprogramming[END_REF], Cisneros et al.[START_REF] Cisneros | Robust humanoid control using a qp solver with integral gains[END_REF].

Figure 4 . 2 : 1 :

 421 Figure 4.2: Position control schemes: IK and TSID. The OR block is used to activate only one controller at a time. Tasks Priority Feet tracking 0 CoM height tracking I CoM lateral-sagittal tracking II Waist orientation III Posture regularization in half-sitting IV

  2.31) and a WQP solving the inverse dynamics of the robot. Compared to the previous controller,

Figure 4 . 3 :

 43 Figure 4.3: TSID torque control scheme.

Figure 4 . 4 :Figure 4 . 5 :Figure 4 . 6 :

 444546 Figure 4.4: ZMP estimation of the 20 cm step walk.

Figure 4 .

 4 Figure 4.7 presents the results obtained on the tracking of the feet and the CoM (see Table4.5); the ZMP and DCM estimations. The feet tracks well the desired trajectories along the y-axis (maximum error of 6mm) however, along the x-axis, they show some delay (maximum error of 6cm). Thus, it induces greater tracking errors on the x-axis for the CoM (peaks of 5cm along the x-axis and 1.5cm along the y-axis). One can notice that the DCM and ZMP along the x-axis are more stable, whereas along the y-axis they present large oscillations (which are caused by the feet impacts on the ground when landing).

Figure 4 .

 4 Figure 4.7: Feet, CoM, DCM and ZMP of the 60 cm step walk.

Figure 4 . 8 :

 48 Figure 4.8: AM behaviour during the 60 cm step walk in torque.

Figure 4

 4 Figure 4.9: ZMP estimation of the tilted platforms simulation.

Figure 4 .

 4 Figure 4.10: Z-axis left foot force of the tilted platforms simulation.

Figure 4 . 12 :Figure 4 . 13 :

 412413 Figure 4.12: Simulation on Gazebo (left) of the robot executing a predicted trajectory. The CoM, footsteps (desired in red, real in blue) and human trajectories (in green) are also displayed in RViz (right) for comparison.

Figure 4 . 14 :

 414 Figure 4.14: Representation of the 3 possible scenarios of Maroger et al. [24]: at the top, the robot walks behind the human, on the middle they are synchronized, at the bottom the robot walks ahead of the human

Figure 4 . 15 :

 415 Figure 4.15: The robot CoM (in red) and footsteps (past steps in grey, current support foot in red and future support foot in green) are generated from the current predicted human trajectory (in purple)

Figure 4 . 16 :

 416 Figure 4.16: Tracking of the CoM and Feet trajectories in the Gazebo simulation where the robot is synchronized with the human (scenario 2).

Figure 4 . 17 :

 417 Figure 4.17: Simulation of the robot executing a predicted trajectory behind the human. The CoM, footsteps (desired in red, real in blue) and human trajectories (in green) are displayed in RViz but have also been realized in Gazebo.

Figure 4 .

 4 Figure 4.18: Tracking of the CoM and Feet trajectories in the Gazebo simulation where the robot is behind the human (scenario 1).

Figure 4 .

 4 Figure 4.19: Experiments -Push recovery of the TALOS robot with one foot raised.

Figure 4 .

 4 Figure 4.20: Experiments -On spot walking with the TALOS robot.

Figure 4 . 21 :

 421 Figure 4.21: Failed experiment using torque control for a postural task.

Figure 4 . 22 :

 422 Figure 4.22: Successful experiment using torque control for a postural task.

128 5. 2 130 5. 3 141 5. 5 145 5. 6

 1282130314151456 . . . . . . . . . . . . . . . . . . . . . . . . . . Passivity Theory . . . . . . . . . . . . . . . . . . . . . . . . 129 5.2.1 Recall of the definitions . . . . . . . . . . . . . . . . . . . 129 5.2.2 Analysis of our System . . . . . . . . . . . . . . . . . . . . Formulation using Energy Tank in TSID . . . . . . . . . 131 5.3.1 Choosing E tank for a set of tasks . . . . . . . . . . . . . . 131 5.3.2 Including protection and valve . . . . . . . . . . . . . . . 133 5.3.2.a Proof with valves for the power port {-N v, τ } . 135 5.3.2.b Proof with valves for the power port {v, τ ext } . . 135 5.3.3 Modified passive QP formulation . . . . . . . . . . . . . . 136 5.4 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.4.1 Pushing task with unplanned contact removal simulation 138 5.4.2 Walk of 20cm steps . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Figure 5 . 1 :

 51 Figure 5.1: Simulations with Passivity: Left: Contact-Force Task -Right: Walk of 20cm steps.

3 )Figure 5 . 2 :

 352 Figure 5.2: Port-based modeling of the subsystems and their relative power variables.

Figure 5 . 3 :

 53 Figure 5.3: TSID torque control scheme with global energy tank.

Figure 5 . 4 :

 54 Figure 5.4: Reference and measured forces of the pushing task.

Figure 5 . 5 :

 55 Figure 5.5: Energy tank and coefficients evolution during the force simulation.

Figure 5 . 6 :

 56 Figure 5.6: Energy derivative Ḣ and its bound -vN T τ during the removal of the bloc and the task. The result when no energy tank is added is also presented.

  .

  5.3 and the gains of the feet tasks motions are: K P = [100, 100, 100]; K D = [5, 5, 5], see Table.

  5.2.

Figure 5 .

 5 Figure 5.7: Energy tank and coefficients of the passive walk of 20cm steps.

Figure 5 . 8 :

 58 Figure 5.8: CoM tracking on the y-axis of the passive walk of 20cm steps.

Figure 5 . 9 :

 59 Figure 5.9: Feet tracking on the z-axis of the passive walk of 20cm steps.

Figure 5 .

 5 Figure 5.10: Ḣ constraint of the passive walk of 20cm steps.

Figure 5 .

 5 Figure 5.11: Ḣ evolution with respect to ṠCoM and Ṡfeet , on the passive walk of 20cm steps.

Figure 6 . 1 :

 61 Figure 6.1: How to transfer the solution from TALOS to the MSDR ? Example of the process of drilling a metal plane.

Force_EEFigure 6 . 2 :

 62 Figure 6.2: Whole control scheme for the MSDR using our solution.

Appendix 2 : 2 :U 3 :X

 223 Differential Dynamics Programming Algorithm Algorithm 1 DDP Solver 1: function DDPSolver(x 0 , U 0 ) ← random U 0 = {u 0 , ..., u T -1 } ← {x 0 , U } 4:

U

  ← {û(0), ..., û(T -1)} 31:X ← {x 0 , x(1), ..., x(T ), U } 32: return {X, U } 33: end function

Figure 6 . 6 :

 66 Figure 6.6: Framework of sot-torque-control in position mode.

Figure 6 . 8 :

 68 Figure 6.8: Scheme to illustrate the identification of the flexibility

1 2 p

 12 . Because the eigenvalues (denoted λ) are invariant to change of basis we have:λ(ΛK p ) = λ(K

Journal

  Maroger et al. [24] I. Maroger, N. Ramuzat, O. Stasse and B. Watier, "Human Trajectory Prediction Model and Its Coupling With a Walking Pattern Generator of a Humanoid Robot". In IEEE Robotics and Automation Letters with IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) option, October 2021, vol. 6, no. 4, pp. 6361-6369.

Ramuzat et al. [25] N. Ramuzat, O. Stasse and S. Boria, "Benchmarking Whole Body controllers on TALOS". In Frontiers Robotics and AI, January 2022. Ramuzat et al. [26] N. Ramuzat, S. Boria and O. Stasse, "Passive Inverse Dynamics Control using a Global Energy Tank for Torque-Controlled Humanoid Robots

  [START_REF] Henze | Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios[END_REF] with p ref , ξ ref the respective ZMP and DCM reference values, given by the planning. And K Zdcm the proportional gain on the ZMP.

	CHAPTER 2. STATE OF THE ART
	Finally, this desired ZMP is used into a CoM admittance control as Caron et al.
	[50]:
	c * = cref + K Padm (p -p * )	(2.31)
	with K Padm the proportional gain on the ZMP for the admittance control. The two
	position control schemes presented in the Section 4.4 use this stabilization formula-
	tion.

Table 2 .

 2 1: Comparison of Humanoid robots specifications.

		TALOS WALK-MAN	TORO	HRP-4 JAXON	Atlas	Gazelle
	Date	2018	2018	2014	2010	2015	2016	2020
	Actuation Electric	Electric/	Electric/ Electric Electric Hydraulic Electric
			Compliant	Compliant
	DoFs	32	32	39	34	32	28	13
	Size	1.75m	1.85m	1.74m	1.51m	1.88m	1.50m	1.30m
	Weight	100kg	102kg	76.4kg	39kg	127kg	75kg	60kg
	Torque	22	22	25	0	32	28	13
	sensors
	F/T	Ankle/	Ankle/	Ankle	Ankle/ Ankle/	Ankle/	Ankle
	sensors	Wrist	Wrist		Wrist	Wrist	Wrist
	IMU/	1	2	2	1	1	1	1
	Accelero.
	Cameras	2	1	3	4	1	2	0
	LIDAR	1	1	0	0	1	1	0
	/Laser

Table 3 .

 3

		CAD ID	σ %
	M X	-0.08 -0.11 2.1
	M Y	1.1	1.1	0.2
	I j	0.34 0.33	9.9
	F Sj	0	0.55	1.3
	F V j	0	0.47 10.3
	Of f j	0	-0.27 3.4
	I m	0.21 0.45	9.5
	F Sm	0	4.0	0.1
	F V m	0	5.1	0.7
	Of f m	0	0.86	7.9

1: Results of the identification process and comparison with manufacturer data.

Table 4 .

 4 2: Set of tasks used for the TSID position and torque control schemes.

	AM ref Feet ref Waist ref
	Pattern Generator	dCoM ref CoM ref	𝑞 𝓿 base v base q TSID	Estimation 𝜏*	Forces IMU Encoders Robot

Table 4 . 3

 43

	Tasks Gains	IK	TSID position	TSID torque
	(20cm|stairs) (20cm|stairs) (20-60cm|stairs)
	K pcom	100	1000	20|12
	K dcom	-	300	3
	K pcomH	100	1000	-
	K dcomH	-	300	-
	K pwaist	300	100	100
	K dwaist	-	20	20
	K pcontacts	1000	30	30-100|30
	K dcontacts	-	11	11-0|11
	K pfeet	1000	2000	1200|500
	K dfeet	-	20	12
	K pam	-	10	10
	K pposture	100	see below	see below
	K dposture	-	2 K pposture	2 K pposture
	K pcomAdm	15|45	12	-
	K pdcm	8|25	8	-
	K idcm	1	1	-
	K zdcm	1	1	-
	TSID Gains		Legs	Torso
	K pposture	[10, 5, 5, 1, 10, 10]	[100, 100]
			Arms	Head
	K pposture	[50, 10, 10, 10, 50, 10, 10, 10] [100, 100]

: Tasks gains of the control schemes. tilted platforms and stairs simulations use the same gains.

Table 4 .

 4 4: ZMP error of the 20 cm step walk simulation.

	Control Scheme Axis Average Standard deviation Peaks
	IK	x-axis 0.019m	0.022m	0.131m
		y-axis 0.022m	0.026m	0.150m
	TSID	x-axis 0.028m	0.025m	0.142m
	position	y-axis 0.025m	0.027m	0.138m
	TSID	x-axis 0.026m	0.021m	0.078m
	torque	y-axis 0.011m	0.014m	0.078m

Table 4 . 5

 45

: CoM and Feet error of the 60 cm step walk.

Table 4 . 6

 46

	Control Scheme Axis Average Standard deviation Peaks
	IK	x-axis 0.021m	0.024m	0.278m
		y-axis 0.016m	0.018m	0.118m
	TSID	x-axis 0.012m	0.017m	0.197m
	position	y-axis 0.015m	0.019m	0.127m
	TSID	x-axis 0.013m	0.021m	0.107m
	torque	y-axis 0.005m	0.006m	0.058m

: ZMP error of the tilted platforms simulation.

Table 4 .

 4

	0.5 1.0 1.5 2.0 ZMP position (m)	Sagittal/Lateral ZMP estimation IK Sagittal/Lateral ZMP estimation TSID Sagittal/Lateral ZMP estimation TSID Torque Sagittal/Lateral reference ZMP for CoM Admittance Sagittal/Lateral Desired ZMP
	0.0
		10		20 Time t (s)	30	40
		Figure 4.11: ZMP estimation of stairs climbing.
	Control Scheme Axis Average Standard deviation Peaks
	IK		x-axis 0.022m	0.026m	0.257m
			y-axis 0.015m	0.017m	0.151m
	TSID	x-axis 0.009m	0.013m	0.151m
	position	y-axis	0.012	0.015m	0.119m
	TSID	x-axis 0.008m	0.006	0.049m
	torque	y-axis 0.006m	0.005m	0.047m

7: ZMP error of the stairs simulation.

Table 4 .

 4 8: Results of the specific cost of transport.

	Control Simulation	E m	E m+	C et	C mt
	Scheme		[J]	[J]	[J/kg/m] [J/kg/m]
	Human	-	-	-	0.2	0.05
	iCub
	position	20cm	-	-	-	0,49
	torque	20cm	-	-	-	0.26
		20cm	1983.9 1359.3	1.68	1.15
	IK	platforms 5418.7 3769.2	3.7	2.6
		stairs	7249.5 2145.3	4.1	1.2
	TSID	20cm	2324.5 764.1	1.97	0.65
	position platforms 5377.5 1413.6	3.6	2.0
		stairs	6812.6 2059.6	3.8	1.2
		20cm	521.8 259.3	0.44	0.22
	TSID	60cm	3147.2 1583.8	0.89	0.45
	torque	platforms 1378.6 668.5	0.93	0.45
		stairs	1861.1 1205.5	1.1	0.68
		Simulation Double S. Single S. Flying F.
	Human	50cm	1.0		0.6	∼ 1.0
		20cm	0.35		0.89	0.24
	IK	platforms	0.27		0.85	0.31
		stairs	0.46		0.86	0.36
	TSID	20cm	0.37		0.74	0.37
	position platforms	0.27		0.86	0.30
		stairs	0.55		0.86	0.34
		20cm	0.93		0.87	1.0
	TSID	60cm	0.87		0.79	1.0
	torque	platforms	0.87		0.8	0.91
		stairs	0.97		0.89	1.0

Table 4 .

 4 9: Results of the PGM on three gait stages.

	Control Scheme Simulation 20cm (60cm) Platforms Stairs
	IK	Average	0.5ms	0.7ms	0.6ms
		Peaks	2ms	4ms	4ms
	TSID	Average	1.2ms	1.2ms	1.2ms
	position	Peaks	4.5ms	4.3ms	4.2ms
	TSID	Average	1ms (1.4ms)	1.2ms	1.1ms
	torque	Peaks	2.8ms (6ms)	5ms	5.5ms

Table 4 .

 4 10: Comparison of the execution time.

Table 4 .

 4 11: Set of tasks for the torque control scheme on the posture task.

	Experiments Gains	Legs	Torso
	Fail	K pposture	[800, 800, 800, 800, 800, 800]	[1000, 1000]
	Success	K pposture	[50, 50, 50, 50, 50, 50]	[100, 100]
			Arms	Head
	Fail	K pposture [800, 800, 800, 800, 800, 800, 800, 800] [100, 100]
	Success	K pposture	[50, 50, 50, 50, 50, 50, 50, 50]	[10, 10]
	Table

Table 5 .

 5 1: Set of tasks used in the contact simulation.

	Tasks Gains				K P	K D	K I
	CoM Waist Feet Contacts		[50, 50, 50] [100, 100, 100] [30, 30, 30, 30, 30, 30] 2 [5, 5, 5] 2 √ K P √ K P	---
	Feet Motions			[100, 100, 100,	[5, 5, 5,	-
	(walk simulation)				100, 100, 100]	5, 5, 5]	-
	Force-Contact (force simulation) [10, 10, 10, 10, 10, 10] AM [10, 10, 10]	-√ 2 K P	0.5K P -
	Posture	Legs	Torso	Arms	Head
	K P	[80, 80, 80, [80, 80] [80, 80, 80, 80, [80, 80]
	K D	80, 80, 80] 2 √ K P	2 √	K P	80, 80, 80, 80] 2 √ K P	2 √	K P

Table 5 .

 5 2: Tasks gains of the control scheme.

Table 5 . 3

 53

: Set of tasks used in the walking simulation.

Table 6 .

 6 2: Set of steps performed by a manipulator robot for drilling operation.

	3	Repeat step 1 for R2 and move to 1st hole
	4	Drill 1st hole
	5	Drill remaining 9 holes with step 4
	6	Move the TCP to R1
	7	Repeat step 1 and drill R1 to final diameter
	8	Move to R2 and repeat step 7 for R2

  RST Regulation Sensitivity and Tracking. 154 SoT Stack-of-Tasks. 12, 14, 15, 123 TSID Task Space Inverse Dynamics. xi, 15, 16, 38, 91-94, 96, 128, 129, 152, 153, 160 URDF Unified Robot Description Format. 8, 15 WPG Walking Pattern Generator. 17, 28, 33-36, 98, 111, 113, 114, 116, 118-120, 123 ZMP Zero Moment Point. 26, 28-30, 34, 35, 98, 100, 102, 105, 108, 109

Remerciements

The

Appendix 3: Frameworks of the different controllers

Sot talos balance framework for position control

Appendix 4: TALOS Hip Flexibility Identification and Control

As presented in Sec.1.4.1.a, the robot TALOS presents a flexibility at its hip. In this section is presented the solutions implemented in this thesis to try to compensate it while using a position controller.

Flexibility Identification

In this section is described the first experiment realized to identify the flexibility caused by the supporting structure of the TALOS hip and not only by the actuator compliance (gearbox, torque sensor ...). This flexibility is located not only at the hip roll-pitch, but also on the yaw, which means that part of it can not be observed with the joint encoders. To estimate the flexibility, the feet distances were collected with a motion capture system. The robot was rigidly controlled in position, and by applying forces on the feet, the whole legs were deflected during the experiment.

The experiment is described in Fig. 6.8, where the flexibility on the hip is denoted K (1 or 2 in function of the leg side), d (in mm) is the measured feet distances between what is measured by the motion capture and the feet positions calculated with direct kinematics. α is the angle of the deflection, τ is the torque measured by the sensors. We applied manually a force F to bring the feet closer and record the torques sensors measurements, the feet distances collected by the motion capture and the feet positions calculated with direct kinematics.

The flexibility is calculated under the following set of hypothesis:

The torque at the hip attachment is approximated to the roll-pitch torque measurement at the sensor, since the level arm between the two is low compared to the leg. Thus

The flexibility on both legs are approximated to be the same:

We noticed during this experiment that the obtained distance between the measured and computed feet positions d is proportional to the measured torques (τ 1 + τ 2 )/2. We thus performed a linear regression to find the slope parameter linking the measured torques to the feet errors. This gave the estimated hip flexibility around the X-axis to 1.027 mm/Nm which corresponds to 973.201 Nm/m. Approximating the leg of TALOS to 1m long (level-arm), it corresponds to a torsional stiffness of 973.201 Nm/rad.

Flexibility Control Without Model

Using the previous estimated flexibility, it is possible to compute the hip deformation not measured by the encoder. This deformation ∆q hip can then be used to modify the position control at the hip pitch and roll joints to take the flexibility into account.

Appendix 5: On the semi-positive definition of the potential energy S

This Appendix details the results presented in the open-access document Ramuzat [START_REF] Ramuzat | On the semi-positive definition of the product of a positive diagonal matrix and a symmetric semi-positive definite matrix[END_REF].

Problem statement:

We have Λ a real symmetric semi-positive definite matrix and K p a real positive diagonal matrix (thus also symmetric). We defined in Chapter 5 the potential energy of a task as S = 1 2 e T ΛK P e. The product ΛK p is a square matrix but non-symmetric. We want to prove that ΛK p is semi-positive definite with respect to the Definition 5.

If the elements of the diagonal K p are equal, i.e. K p = aI with a ∈ R + \{0}, then the product ΛK p = aΛ is a real symmetric semi-positive definite matrix. Thus, for a diagonal K p with equal elements we have directly x T ΛK p x ≥ 0 ∀x ∈ R n \{0}.

For different elements on the diagonal, we have the following results:

-The eigenvalues of ΛK p are real and non-negatives

has non-negative eigenvalues, then the quadratic form of ΛK p is semi-positive definite Proof. Let us recall some useful matrix definitions and properties: Definition 4 (Real symmetric semi-positive definite matrix). A is a real symmetric matrix (its eigenvalues are thus real): A is semi-positive definite ⇐⇒ all its eigenvalues are non-negatives.

Definition 5 (Quadratic form semi-positive definition). A matrix

If A is a real semi-positive definite matrix, then B T AB is semi-positive definite for any matrix B.

Property 3 (Matrix congruent to a symmetric matrix). Any matrix congruent to a symmetric matrix is again symmetric: If A is a symmetric matrix then so is B T AB for any matrix B.

Proof that the eigenvalues of ΛK p are real and non-negatives

Because K p is diagonal and positive we can write K p = K