Alexis Jensen
email: alexis.jensen@irisa.fr

Thomas Chatagnon
email: thomas.chatagnon@irisa.fr

Niloofar Khoshsiyar

Daniele Reda
email: dreda@cs.ubc.ca

Michiel Van De Panne

Charles Pontonnier
email: charles.pontonnier@irisa.fr

Julien Pettré
email: julien.pettre@irisa.fr

Physical Simulation of Balance Recovery after a Push

Keywords: Computing methodologies → Real-time simulation physics-based simulation, biomechanics, animation, controller-based, experimental data, balance recovery

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Simulating the behaviour of humans during physical interactions with their environment has been a long-standing goal in computer animation. In this work, we are interested in simulating how humans respond to moderate to strong local pushes and realistically estimating their global motion after such a perturbation. This is of interest, for example, in understanding how motion can propagate from one person to another and generate large scale motion in crowds, but for now we consider the case of a single human subject to an external force applied to his or her body.

When standing still, humans rest on their two feet in an upright position. On a flat surface, static balance is achieved by keeping the projection of the centre of mass (CoM) on the ground within the limits of the Base of Support BoS (the convex hull of the footfloor contact points). After an external push, several movement strategies are known in humans to achieve balance recovery [START_REF] Kuangyou | A unified approach for revealing multiple balance recovery strategies[END_REF]. We focus here on stepping strategies that occur after moderate to strong push forces. Stepping strategy is to extend the BoS, and therefore combines the control of the position of the feet together with the one of the CoM to satisfy the balance conditions.

Thus, to understand better the global motion resulting from external perturbations, our goal is to dynamically control the motion of a character to simulate steeping strategy for balance recovery. Our approach is to control the joint angles of the character to drive the projected CoM towards the center of the BoS, while simultaneously repositioning the feet as necessary to adjust the configuration of the BoS according to the instantaneous motion of the CoM. The control scheme is derived from an experimental study observing stepping strategy for balance recovery [START_REF] Chatagnon | Stepping strategies of young adults undergoing sudden external perturbation from different directions[END_REF]. The results of this study were used to provide generalized equations of the feet behavior during stepping and to optimize the model parameters to enhance the overall response with regard to the experimental data.

The current work proposes two main contributions. The first contribution introduces a balance assessment method controlling the stepping policy of the character with regard to the time remaining before the CoM reaches the boundaries of the BoS (Time to Base of Support TtBoS). The second contribution is a novel foot stepping strategy leading to a steady state after a balance loss. These two contributions are combined with state-of-the-art physics-based human simulation, then tuned and validated using the same experimental data that inspired the control of the character. The end result is a simulator that can accurately reproduce the behavior of a human being pushed, for any push force or duration.

STATE OF THE ART

The field of physics-based human simulation is a long standing subject involved with the domains of biomechanics, robotics and dynamic simulation. This section covers the main approaches that emerged over the years, with an additional focus on the literature concerning balance recovery and the use of data in the field.

Optimisation based models. Optimisation methods are based on defining a set of costs to be minimized, which converges towards the simulated human performing a given task. Balance in these methods is part of the optimisation cost. These methods compute the best future trajectory, and as such are qualified as feedforward approaches. An example is the use of the zero moment point (ZMP) to follow [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF]] along a trajectory. Predictive optimisation models [START_REF] Shen | Human-like balance recovery based on numerical model predictive control strategy[END_REF]] have addressed the simulation of standing balance, leading to works studying stepping after large pushes [START_REF] Kudoh | Stepping motion for a human-like character to maintain balance against large perturbations[END_REF]]. Optimisation based models have been often used in robotics, providing robust results by translating the problem into solvable equations [START_REF] De Lasa | Feature-based locomotion controllers[END_REF]], sometimes mixed with motion capture [START_REF] Muico | Contact-aware nonlinear control of dynamic characters[END_REF]]. The quality of the results comes at the cost of long computation times and a lack of flexibility or reactivity in the character's responses, since the optimisation problem must be defined in advance.

Control based models. Control based methods aim to adjust the posture of the body to achieve a defined goal, often by controlling the position of the CoM. Unlike optimisation methods, they rely on the analysis of current and past states to achieve their goal. Controller-based methods rely on a set of controllers, that act typically on joints to move the character. While Proportional-Derivative (PD) joint controllers are mainly used, some works have use musclebased control approaches [START_REF] Lucia | Muscle-based control for character animation[END_REF][START_REF] Geijtenbeek | Flexible muscle-based locomotion for bipedal creatures[END_REF]. This type of control-based approach can track target motions, whether they are authored [START_REF] Yin | Simbicon: Simple biped locomotion control[END_REF] or computed [START_REF] Coros | Generalized biped walking control[END_REF]. The strength of controller-based methods is their ability to respond and adapt to external perturbations.

To control the CoM, various techniques have been studied in the field. While previous works have used real-time inverse dynamics [START_REF] Herzog | Balancing experiments on a torque-controlled humanoid with hierarchical inverse dynamics[END_REF]], many rely on the inverse pendulum model [START_REF] Tsai | Real-time physics-based 3d biped character animation using an inverted pendulum model[END_REF]] (IPM), a simplified model of lower body equilibrium that can easily simulate the trajectory of the center of gravity. Previous studies have extended the IPM with intelligent foot placement using capture points [START_REF] Pratt | Capture point: A step toward humanoid push recovery[END_REF], or with inverse kinematics and state machines [START_REF] Benjamin | Push recovery by stepping for humanoid robots with force controlled joints[END_REF]. However, the simplicity of such models struggles against sudden perturbations, such as a hard push.

Since controlling the position of the CoM is closely related to balance, control methods must implement strategies for balance management for the simulation to work. In the literature, a three step balance recovery method has emerged [START_REF] Aftab | Ankle, hip and stepping strategies for humanoid balance recovery with a single Model Predictive Control scheme[END_REF]]. The strategy is rooted in human behavior, with low-intensity impulse recovery that does not require stepping and involves only the ankles and hips motion when possible. It relies directly on previous step triggering works [START_REF] Stephens | Humanoid push recovery[END_REF]] on the basis of the CoM movement. The core of the balance recovery simulation is balance assessment, the relationship between which and feet placement has been studied in several papers, with concepts such as the Extrapolated CoM (XCoM) [START_REF] Schulz | Can initial and additional compensatory steps be predicted in young, older, and balance-impaired older females in response to anterior and posterior waist pulls while standing[END_REF] or the Base of Support (BoS) [START_REF] Al Hof | The condition for dynamic stability[END_REF]. The BoS has already been used to compute a threshold for step triggering [START_REF] Pai | Thresholds for step initiation induced by support-surface translation: a dynamic center-of-mass model provides much better prediction than a static model[END_REF]], and the XCoM has been used in a simple control balance model [START_REF] At | The 'extrapolated center of mass' concept suggests a simple control of balance in walking[END_REF].

Learning based models. In recent years, traditional controllers have given way to deep learning, reinforcement learning, and other neural network based methods of character control. Balance in learning model is part of the reward system, without being the main focus. These works are able to reproduce tasks by imitating motion capture data [START_REF] Peng | Deepmimic: Example-guided deep reinforcement learning of physics-based character skills[END_REF], and are sometimes able to adapt the choice of task to the environment [START_REF] Peng | Amp: Adversarial motion priors for stylized physics-based character control[END_REF]]. Some work focus on continuous adaptation of the learned movement [START_REF] Xie | Allsteps: curriculum-driven learning of stepping stone skills[END_REF]]. Another approach relies on reinforcement learning to generate control without the need for motion capture, but often falls short in terms of adaptability [START_REF] Howell | Predictive sampling: Real-time behaviour synthesis with mujoco[END_REF]. In general, learning-based approaches are the best at seamlessly reproducing captured motions, which may include complex interactions [START_REF] Lee | Interactive character animation by learning multi-objective control[END_REF][START_REF] Starke | Local Motion Phases for Learning Multi-Contact Character Movements[END_REF].

Data parameter optimisation. In the aforementioned models, motion capture is often used directly as a standard to mimic. This paper takes a different approach, with a Proportional Derivative (PD) joint control strategy whose parameters are optimized using experimental data. PD control optimisation is based on gain tuning and has been widely studied in the field of robotics, both with [START_REF] Iffah | Optimization of the PID-PD parameters of the overhead crane control system by using PSO algorithm[END_REF][START_REF] Gaing | A particle swarm optimization approach for optimum design of PID controller in AVR system[END_REF][START_REF] Zamani | Design of a fractional order PID controller for an AVR using particle swarm optimization[END_REF]] and without [Jagodnik and van den Bogert 2010] using the particle swarm optimisation (PSO) method, which is used in this paper.

Optimisation of bipedal walking simulations has also been studied in the domain, with similarly structured work [START_REF] Jack | Optimizing walking controllers[END_REF]] based on the Simbicon paper. Metrics to compute the effort of human walking have been established [START_REF] Jack M Wang | Optimizing locomotion controllers using biologically-based actuators and objectives[END_REF], and along other metrics have been used in an optimisation scheme [START_REF] Geijtenbeek | Flexible muscle-based locomotion for bipedal creatures[END_REF] for gait simulation. Such metrics allow the evaluation of the produced motion and torques for this paper's output simulation.

Our work. This paper studies the stepping strategy required for balance recovery after pushes, with a focus on accurately representing reality. For these reasons, we opt for a control based approach. The lack of responsiveness of optimisation based methods clashes with our intention of responding to perturbations, while learning based approaches involve black boxes that naturally do not allow an interpretation of the model to validate its internal accuracy. Our approach is based on the 'Generalized Biped Walking Control' paper [START_REF] Coros | Generalized biped walking control[END_REF], which based on the Simbicon controller [START_REF] Yin | Simbicon: Simple biped locomotion control[END_REF]]. We extend the system with inverse kinematics similar to previous works [START_REF] Benjamin | Push recovery by stepping for humanoid robots with force controlled joints[END_REF]. The inverse kinematics are based on the goal feet position, a strategy that has been used in works related to trajectory step planning (the aforementioned ZMP [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF]] and capture points [START_REF] Pratt | Capture point: A step toward humanoid push recovery[END_REF]).

Previous models lack an adequate response to high intensity push, which is the core of our contribution. We introduce a novel stepping strategy to handle strong perturbations, based on feet positioning. In contrast to other works where target motions are generated from predefined state machines or motion capture, our method is able to assess balance and compute proper feet placement in a human-like manner. Our general approach makes it possible to experiment with variations in the push intensity, duration, position, and in the morphology of the subject without changing the model. Our novel feet based balance assessment and recovery are both anchored in observations of recent experimental data dedicated to balance recovery. Previous works have established a link between step characteristics and COM velocity, through experimental analysis of pushing scenarios [START_REF] Li | Experimental study on pedestrian contact force under different degrees of crowding[END_REF][START_REF] Zhang | Predicting foot placement for balance through a simple model with swing leg dynamics[END_REF]].

Here we used a similar approach based on our experimental data [START_REF] Chatagnon | Stepping strategies of young adults undergoing sudden external perturbation from different directions[END_REF] .

This paper presents how the same data are used to tune the parameters of our model and validate its output motions. By using the data for optimisation, we are able to not only reproduce the human balancing process, but also to better explain its mechanism.

OVERVIEW

Figure 2 depicts the method used to simulate balance recovery, with a visual representation of the character being pushed and a flowchart of the simulation iteration. We follow the SI metric system for forces, distances and velocities. The legend of the figure shows the correspondence between the colors of the flowchart boxes and the main components of the proposed method: the Locomotion System that triggers steps to recover balance, the control of the Foot Trajectory after steps are triggered, and finally the Full-Body Control of the Center of Mass (CoM) position. The following paragraphs provide details about each of these components, starting with the controlled character itself.

Character. The character of the simulation is a poly-articulated kinematic chain of cuboid limbs, except for a capsule head. A total of 14 limbs are attached at 11 joints. Of all the joints, 4 have 1 Degree of Freedom (DoF) while the others have 3, for to a total of 25 DoFs. The size and mass of the limbs in the body follow Winter's [START_REF] David | Biomechanics and motor control of human movement[END_REF]] table, scaled by the character's total height and weight.

Joint PD controller. Actuation of the motion for the character is mainly done with traditional Proportional Derivative (PD) controllers. For every joint, a unique set of Kp and Kd gains is provided handling all the DoFs of the joint. As seen in Figure 2 close-up 1, the desired and actual angles 𝜃 𝑑 and 𝜃 are compared on a local, limb-aligned basis. Considering also the angle velocity 𝜃 𝑑 and 𝜃 , the PD controller outputs a torque 𝜏 that actuates the joint to the target angle, according to the classic PD controller equation (1).

𝜏 = 𝐾 𝑝 .(𝜃 𝑑 -𝜃 ) + 𝐾 𝑑 .( 𝜃 𝑑 -𝜃 ) (1)
Simulation iteration. At the beginning of the simulation iteration, the current state of the character is evaluated by the Locomotion System (Section 4.1) in red. This process is based on the analysis of the Center of Mass (𝐶𝑜𝑀), the Expected Center of Mass (𝑋𝐶𝑜𝑀) and the Base of Support (𝐵𝑜𝑆). If both feet are planted (i.e., the character is not walking), balance is assessed. If necessary, a step is triggered which puts the character in the walking state.

Depending on this analysis, a step may be necessary, which is represented by the Foot Trajectory component (Section 4.2), shown in blue in Figure 2. This involves first computing a goal position coordinates for the swinging foot on the ground 𝑋𝑍 plane, here called 𝑋 𝑑𝑒𝑠𝑖𝑟𝑒𝑑 (𝑋 𝐷 ) and 𝑍 𝑑𝑒𝑠𝑖𝑟𝑒𝑑 (𝑍 𝐷 ), which will lead the character to balance. To reach this goal, a trajectory is computed to be followed during the next simulation iterations. An overall motion is generated for the legs using inverse kinematics, which follows the trajectory of the swinging foot. Each joint in the body has default target angles, some of which are overwritten by the inverse kinematics process during walking.

These goal angles are provided to the PD controllers, producing a torque 𝜏 at each joint of the body. All the forces applied in our method by the Full-body Control (Section 5) are shown in green.

To support the motion of the body, the Jacobian transpose method is used to compute the torques for the three type of forces, 𝐹 𝐶𝑜𝑀 , 𝐹 𝑉 𝐸𝐿 and 𝐹 𝐺𝑅𝐴𝑉 . An example is shown in Figure 2 close-up 2, where the force of the limb 𝑖, 𝐹 𝐺𝑅𝐴𝑉 𝑖 , is applied through two torques 𝜏 1 and 𝜏 2 using the elbow and shoulder joints, according to their Jacobian matrix with respect to the CoM of each limb.

All of these torques are then merged and integrated in the physics engine (using the Dantzig-Wolfe [Dantzig and Wolfe 1960] decomposition solver applied to Mixed Linear Complementary Problems), for a new iteration to begin. This process can be perturbed by an external force 𝐹 𝑃𝑈 𝑆𝐻 , directly applied to the body during the integration.

Locomotion System

The first objective of the simulation is to accurately determine whether it is necessary or not for the character to take a step. This section introduces how the current state of the simulation is analyzed, leading to a walking state if necessary. It also explains the state machine that the model follows, to alternate stepping between legs to recover balance.

Balance assessment using Time to Base of Support. The purpose of this work is for the character to recover and maintain balance, with or without locomotion. As such, the first concept to be clarified is the definition of balance : what factual observation can be made to determine whether a character is balanced or not. For this purpose, this work is based on an experimental dataset of participants undergoing external perturbations [START_REF] Chatagnon | Stepping strategies of young adults undergoing sudden external perturbation from different directions[END_REF].

More specifically, the step triggering condition described in the following work [START_REF] Chatagnon | Stepping strategies of young adults undergoing sudden external perturbation from different directions[END_REF], is implemented in our et al. 2005], or the Time to BoS boundary [START_REF] Schulz | Can initial and additional compensatory steps be predicted in young, older, and balance-impaired older females in response to anterior and posterior waist pulls while standing[END_REF]] (here referred as TtBoS). On one hand, the concept of MoS relies on the analysis of an augmented representation of the CoM relying its velocity called XCoM, computed with the following equations :

𝑋𝐶𝑜𝑀 = x CoM + x CoM 𝜔 0 with 𝜔 0 = √ 𝑔/𝑙 (2) 𝑀𝑜𝑆 = (𝑢 𝑚𝑎𝑥 -𝑋𝐶𝑜𝑀). x CoM || x CoM || (3)
Here, 𝑥 𝐶𝑜𝑀 represents the CoM position, 𝑔 is the gravity constant, 𝑙 is the length of the legs and 𝑢 𝑚𝑎𝑥 is the closest intersection with the BoS in the direction of the XCoM from the CoM. Figure 3 illustrates these concepts and the relationships between them.

On the other hand, the concept of TtBoS uses the Distance to BoS (DtB) together with the instantaneous velocity of the CoM, as written in the following equations :

𝑇 𝑡𝐵𝑜𝑆 = 𝐷𝑡𝐵 || x CoM || (4) 𝐷𝑡𝐵 = (𝑢 𝑚𝑎𝑥 -x CoM ). x CoM || x CoM || (5)
The DtB represents the distance between the CoM and the nearest boundary of the BoS in the direction of CoM velocity, while the TtBoS is the time it would take for the CoM to reach the BoS at its current velocity. Based on experimental data, a threshold value T for the Time to BoS is defined, below which the character is considered to be unbalanced in the future, without the possibility to recover without taking a step.

Leg switch state machine. Using the previously defined threshold T, the simulation is now able to detect whether the character should step or not. This allows us to define a simple state machine which is the locomotion model of the character, as shown in Figure 4. The base state of the character is the Standing (S0) state. In this state, the character' rigidity is sufficient to maintain its CoM in the BoS, and steps are not required. When the threshold T is crossed, a step is initiated. This changes the state of the character to Walking with the right (S1) or left leg (S2) depending on the direction the CoM is moving toward.

While stepping, the characters converge toward a balanced state. However, if at the end of the step when both feet are in their intended position, the character is still considered as unbalanced (the TtBoS is still below T ), then another step is initiated. The swinging leg is switched, and the process is repeated until the character is considered as balanced at the end of a step. Thus, it simply returns to the Standing state, where the internal forces are sufficient to keep the balance.

Foot trajectory

Computation of the foot trajectory. When the character enters a Walking state, first are computed the end goal position's coordinates, 𝑋 𝐷 and 𝑍 𝐷 , for the foot of the current swinging leg. Not only must the final position be computed, but the StepTime the step takes to be completed is also critical for a quick yet cohesive movement.

To calculate these key factors, equations derived from experimental data are used. Both the foot end position and stepping time have been found to be closely related to the value of the CoM velocity when a step is triggered, the scalar denoted x CoM projected . Similar to the threshold for the time to BoS, those parts of the model are anchored in the observation of experimental data that supports the simulation, leading to the following equations and parameters:

(𝑋 𝑑 , 𝑍 𝑑 ) = 𝑥 𝐹𝑜𝑜𝑡 𝑝𝑟𝑜 𝑗𝑒𝑐𝑡𝑒𝑑 + 𝑎 1 .ℎ𝑒𝑖𝑔ℎ𝑡 . x CoM projected (6) 𝑆𝑡𝑒𝑝𝑇𝑖𝑚𝑒 = 𝑎 2 + 𝑏 2 . x CoM projected (7)
In order to reach the target position (𝑋 𝐷 , 𝑍 𝐷 ) in StepTime seconds, a trajectory must be established that will move the foot to the next position, as shown in Figure 5. The problem can be divided into two parts : the projected trajectory of the foot on the ground over time, and the height of the foot over time. The projected position can be easily computed by linear interpolation between the initial and final position, as seen on the left of Figure 5. For the height, the model approximates it using a spline proportional to the step distance, resulting in a 3D trajectory like the right part of Figure 5. Swinging leg inverse kinematics. By default, each joint in the character's body has a target angle of 0 for each DoF while being subjected to gravity. When a step is taken, those goal angles are overwritten for the lower body to be in the correct position. These angles are computed at each simulation iteration when in the Walking state, following the previously defined trajectory. The angles are computed using the Cycling Coordinate Descent (CCD) algorithm [START_REF] Song | A fast inverse kinematics algorithm for joint animation[END_REF], as shown in Figure 6. The first step is in the coronal plane, to compute the rotation around the 𝑋 axis 𝜃 𝑠𝑡𝑟𝑖𝑑𝑒 which relies on the single DoF of the knee to simplify the inverse kinematics to a plane. The goal in blue follows along the trajectory, represented by the dashed line. This is shown in the second part, where by knowing the length of the leg's limbs and the position of the goal, the position of the knee can be deduced and the angles 𝜃 ℎ𝑖𝑝 and 𝜃 𝑘𝑛𝑒𝑒 are calculated.

Standing leg inverse kinematics. To support the stepping motion, the standing leg must contribute to the balance. The goal position of the CoM is estimated to be between the final positions of the two feet when they are projected onto the ground. The standing leg is given goal angles with the foot not moving and the CoM in the correct position, using the same CCD method but in reverse (where the end effector is now the hips).

FULL-BODY CONTROL

Throughout the body, motion at this point is generated by PD controllers of the joints provided with goal angles. These goal angles are set to 0 by default. Thus, the angles of the whole body result only from the weight of gravity on the limbs, which is counteracted by the PD controllers. Below the waist, the goal angles are redefined when stepping. However, this leaves the rest of the body in its default state, which is both inefficient and incorrect. When pushed, every part of the body is involved in the balance recovery, albeit to different degrees. This section deals with the calculation of the other applied torques throughout the whole body.

Set of applied torques. In total, four different types of torques can be applied in the simulation. The first is from the PD controllers, which generate torques from goal angles. The other three are gravity compensation, velocity compensation and CoM driving, which are computed using jacobian transpose. Gravity compensation compensates for the constant effort required to resist gravity, to reduce the load on the PD controllers for finer control. The goal of velocity compensation is to involve the whole body in counteracting the push by converging zero CoM velocity. Finally, CoM driving assists the legs' PD controller in their stepping motion by pushing the ground-projected CoM toward the center of the feet.

Torque computation. The method used in order to compute the torques is the jacobian transpose, a classic control method in robotics [START_REF] Samuel R Buss | Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods[END_REF]]. Figure 7 illustrates the application to limb 𝑖 of a force 𝐹 𝐺𝑅𝐴𝑉 𝑖 at the position 𝐶𝑜𝑀 𝑖 through two torques, one for the lower arm (𝜏 1 in the figure) and one for the upper arm (𝜏 2 in the figure). Each torque is computed for a joint 𝑘, with the goal of acting on a limb 𝑖 and as such with respect to its 𝐶𝑜𝑀 𝑖 position with:

𝐽 𝑘 = 𝛿𝑃 𝐶𝑜𝑀 𝑖 𝛿𝜃 𝑘 (8)
By transposing the Jacobian of the joint 𝑘, a torque can be computed that will apply the equivalent of a force F (𝐹 𝐺𝑅𝐴𝑉 𝑖 in the figure) at the position 𝐶𝑜𝑀 𝑖 using :

𝜏 𝑘 = 𝐽 𝑇 𝑘 𝐹 𝐺𝑅𝐴𝑉 𝑖 (9)
The implementation of this process and the relevant force values to add to the system (namely the following gravity and velocity compensation) are described in detail in the paper 'Generalized Biped Walking Control' [START_REF] Coros | Generalized biped walking control[END_REF], which is the main inspiration for this paper. Gravity compensation. The first torques applied are gravity compensation. A set of forces is computed, one for each limb of the upper body. The position of force application is the CoM of the limb i, with the force following the equation:

𝐹 𝐺𝑅𝐴𝑉 𝑖 = -𝑔.𝑚 𝑖
(10) The joints between the corresponding limb and the hips each have a torque computed. In the end, every limb 𝑖 of the upper body have gravity compensation applied, with each limb 𝑖 involving a different set of joints 𝑘 using jacobian transpose to do so. For example, the shoulder joint not only compensates for the gravity of the upper arm limb, but also the lower arm limb, since this joint affects both.

Velocity compensation. The next force applied is velocity compensation, at the CoM of the character, with the corresponding equation :

𝐹 𝑉 𝐸𝐿 = -𝑎 𝑉 𝐸𝐿 . x CoM (11)
𝑎 𝑉 𝐸𝐿 is a dimensionless tuning parameter to control the intensity of the resulting torques. This force is applied only once at the CoM of the character, but the torques are computed for each joint between the standing foot and the head.

CoM driving. In addition to the previous set of forces, another force called the CoM driving contributes to balance. It is applied in the same way as velocity compensation, from the standing foot to the head, and only once at the CoM of the character using:

𝐹 𝐶𝑜𝑀 = 𝑎 𝐶𝑜𝑀 .(x CoM desired -x CoM ) ( 12
)
𝑎 𝐶𝑜𝑀 is another dimensionless tuning parameter, once again regulating the final torques values. It is a mixture of the two previous forces: it supports the PD controllers like the gravity compensation and involves the whole body like the velocity compensation.

Force purpose and equilibrium. The three forces applied to the system are expressed in the same way as the PD controllers : by torques at the joints of the body. This means that at each joint, several torques can be summed to get the final applied torque. In order for this summation to work, each torque value must be carefully tuned in order to ensure a balanced contribution from the controllers, through gains (𝐾 𝑝 , 𝐾 𝑑 ) or tuning parameters (𝑎 𝑉 𝐸𝐿 ,𝑎 𝐶𝑜𝑀 ).

PARAMETER OPTIMISATION

The goal of the simulation is to mimic the reaction of a human to physical perturbations, more specifically a push. As seen in the previous sections, this simulation relies heavily on observations made from the experiment data [START_REF] Chatagnon | Stepping strategies of young adults undergoing sudden external perturbation from different directions[END_REF]. The experiment can be summarized as follows : A person is pushed with a pole with a certain intensity for a certain time, and in response takes steps or not at a certain distance. The experimental data consists of markers positions, which are processed with CusToM [START_REF] Muller | CusToM: a Matlab toolbox for musculoskeletal simulation[END_REF] to compute reference joint angles and CoM trajectories using inverse kinematics.

As shown in Figure 8, the trigger for the experiment is the force applied to the body. Pushes of varying angle, force and time were explored.

Several parameters introduced into the simulation can be adjusted to improve the realism of the results. Therefore we have developed an optimisation procedure to adjust these parameters in order to make the generated simulations comparable to the experimental data.

Particle Swarm Optimisation. The optimisation process uses the PSO (Particle Swarm Optimisation) algorithm [START_REF] Kennedy | Particle swarm optimization[END_REF]. This process involves exploring a set of parameters in a bounded multi-dimensional domain. It uses multiple particles that influence each other, by defining a personal and global best and gravitating towards an optimal multi-dimensional position.

More specifically, PySwarm [pys [n. d.]] is used along the simulation. After many iterations, the particles naturally gravitate towards a global minimum result according to the defined cost function.

Joint optimisation. The first optimisation step adjusts the joint PD controller gains. The goal is to find the best gain values that allow an accurate tracking of the goal angles without introducing unnecessary or unrealistic forces. Total cost is computed by summing the results over a set of scenarios, covering weak to small pushes, with the 𝐶 𝑗𝑜𝑖𝑛𝑡 cost function of a single push with a total duration of 𝑡 𝑚𝑎𝑥 is :

𝐶 𝑗𝑜𝑖𝑛𝑡 = 𝑊 𝑎𝑛𝑔𝑙𝑒 ∫ 𝑡 𝑚𝑎𝑥 0 𝐸 𝑎𝑛𝑔𝑙𝑒 (𝑡)𝛿𝑡 𝐻 𝑎𝑛𝑔𝑙𝑒 +𝑊 𝑒 𝑓 𝑓 𝑜𝑟𝑡 ∫ 𝑡 𝑚𝑎𝑥 0 𝐸 𝑒 𝑓 𝑓 𝑜𝑟𝑡 (𝑡)𝛿𝑡 𝐻 𝑒 𝑓 𝑓 𝑜𝑟𝑡 (13)
Where for each error, a dimensionless weight 𝑊 and a threshold value 𝐻 are defined. Below the threshold, the integrated value is replaced by 0. This means that if the integrated value of the angle error is low enough it is ignored. With a heavy weight, this means that simulation that do not follow the goal angles are heavily penalized, while all simulation below the threshold compete to minimize the effort.

The cost function is based on two main error functions, inspired by works on human effort [START_REF] Geijtenbeek | Flexible muscle-based locomotion for bipedal creatures[END_REF], integrated over the duration of the simulation. For effort minimization 𝐸 𝑒 𝑓 𝑓 𝑜𝑟𝑡 (𝑡), we use the current rate of metabolic expenditure [START_REF] Jack M Wang | Optimizing locomotion controllers using biologically-based actuators and objectives[END_REF] where 𝑖 is for each joint, and 𝑗 is for each DoF of each joint .

𝐸 𝑎𝑛𝑔𝑙𝑒 (𝑡) = ∑ 𝑖 ∈ 𝐽 𝑜𝑖𝑛𝑡𝑠 ∑ 𝑗 ∈𝐷𝑜𝐹 ||𝜃 𝑖,𝑗 𝑑𝑒𝑠𝑖𝑟𝑒𝑑 (𝑡) -𝜃 𝑖,𝑗 (𝑡)|| 2 (14) 𝐸 𝑒 𝑓 𝑓 𝑜𝑟𝑡 (𝑡) = ∑ 𝑖 ∈ 𝐽 𝑜𝑖𝑛𝑡𝑠 ∑ 𝑗 ∈𝐷𝑜𝐹 ||𝜏 𝑖,𝑗 (𝑡)|| 2 (15)
Model optimisation. The first optimisation concerned the ability of the body to follow joint angle goals without too much effort. This second optimisation aims at tuning the generation of these angle goals, along with tuning the full-body control forces. The goal is for the trajectory of the CoM generated by the simulation to resemble the general human response to the same push. To do this, the corresponding 𝐶 𝑚𝑜𝑑𝑒𝑙 cost function is :

𝐶 𝑚𝑜𝑑𝑒𝑙 = ∫ 𝑡 𝑚𝑎𝑥 0 𝐸 𝑡𝑟𝑎 𝑗 (𝑡)𝑑𝑡 (16)
It relies on a trajectory error function, that directly compares the experimental data and the simulation side by side. The experimental data, while very useful, also show a great deal of variability in individual response and between characters. For the same push and character, the final position shows high inconsistencies. Comparing the velocity over time, rather than the absolute positions, eliminates the differences at the starting and final positions between the simulation and the data, focusing on the stepping motion.

𝐸 𝑡𝑟𝑎 𝑗 (𝑡) = ||𝑣 𝐶𝑜𝑀 𝑑𝑒𝑠𝑖𝑟𝑒𝑑 (𝑡) -𝑣 𝐶𝑜𝑀 (𝑡)|| 2 (17)
For this optimisation, the data compared to the simulation is limited to perpendicular pushes to the back. For each iteration, the same set of pushes is analyzed for consistency. This set of pushes pans across the entire range of push intensity and duration, from small pushes with no step to multi-step responses.

A total of 6 parameters are optimized in this second step. The first two parameters are those from the Jacobian-based forces 𝑎 𝑉 𝐸𝐿 and 𝑎 𝐶𝑜𝑀 . Closely related to the force is a 𝑎 𝑃𝑈 𝑆𝐻 tuning parameter, a conversion gain of the push for the simulation.

The remaining 3 parameters are related to the feet. The stickiness S of the feet is the first : the simulation alternates each foot between total complete freedom and a grounded state with restricted motions to avoid the physics engine's foot sliding. The grounded state is activated when the foot does not belong to the swinging leg and touching the ground. The stickiness when grounded ranges from 0 to 1, where 0 means no possible movement and 1 complete freedom of movement. It is followed by the Height of the Spline 𝐻 𝑠 , a dimensionless tuning parameter of step height over time. Finally, a filter is applied to avoid micro-step, tuned by the threshold 𝑇 𝑓 in our optimisation. Steps with a stepTime under 𝑇 𝑓 are removed.

RESULTS AND DISCUSSION

In order to validate the direction of our approach, four aspects are scrutinized. The first evaluation aims to check whether the simulation is capable of recovering balance in the boundaries of our data, with a supplementary look beyond the experimental pushes values. The second metric is the torques produced at the joints of the model. The goal is for the model's produced torque to be under the human maximum in the same condition. By comparing the values with real human standards, the physical soundness of the control model can be tested.

The last two analyses are based on reproducing each experimental push in the simulation, allowing direct data comparison. This is done by normalizing the real trajectories in accordance to height and weight, creating pairs of real and simulated results for a push force and duration. The first comparison performed is the step triggering, to verify that the simulation triggers step consistently with the data. To do this, we use a precision test, where 'Positive' and 'Negatives' correspond to the simulation requiring a step, while 'True' and 'False' come from the experimental data needing to step. The second criterion is the CoM trajectories over time, being compared visually and statistically between experimental and simulated data. The difference between every pair of simulated and real trajectories were analyzed using an SPM [START_REF] Pataky | One-dimensional statistical parametric mapping in Python[END_REF]] t-test, to determine if there are notable differences between them.

Validation

The first qualitative metric explored in Figure 9 is the ability of the character to recover balance after receiving pushes of varying strength and duration. By default, we set our character to an average height and weight (1.70m and 70kg), pushed from a perpendicular angle to the back. A push is applied at every 20N (between 0 and 300 Newtons) and 0.2s (between 0 and 2 seconds), only once due to the deterministic nature of the simulation. Note that this range of impulse goes beyond the experimental study on which we based our work, as shown in Figure 9. The blue and red regions correspond to complete success or failure. Mixed responses indicate the area in which the character fell for pushes of lower intensity than pushes with successful balance recovery. The model is able to recover balance after a physical perturbation as seen in Figure 9, within the boundaries of the experimental conditions but also beyond. Falling naturally occurs for strong pushes, where balance is naturally not recoverable. While the lack of data does not allow to appreciate the experimental balance recovery limits, the response of the model in experimental conditions is in accordance to the data, thus validating the model for this range of pushes. An interesting phenomena also appears, as strong but very long pushes yield more balance recovery : the push is so long that it begins to serve as support for the character, that simply steps along the push.

Physical Plausibility

The following section aims at verifying whether the generated torques values are within the range of human torques. The measurements are made for the lower body joints, and compared with an empirical study [START_REF] Dennis E Anderson | Maximum voluntary joint torque as a function of joint angle and angular velocity: model development and application to the lower limb[END_REF]]. Figure 10 illustrates the results. For each type of torque, the maximum torque (up axis) observed in humans for a given angle (right axis) and an angular velocity value (bottom axis) is represented as a surface. Each color corresponds to the torque measured over the duration of a low (green), medium (blue) and high (red) impulse push. The surface represents the torque generation capabilities for the corresponding angle and angular velocity values reported from Anderson et al [START_REF] Dennis E Anderson | Maximum voluntary joint torque as a function of joint angle and angular velocity: model development and application to the lower limb[END_REF]]. For every lower body joint, the measured torques are below the human maximum. The only exception is plantar flexion, with values well above the recorded human limits.

Hip extension

Hip flexion The computation of torques in the simulator is done without prior knowledge, but still manages to produce cohesive values, underlying the physical cohesiveness of the model. The only exception is the plantar flexion, which suffers from the foot contact model of the simulator that produces unrealistic contact forces and high torques. This is not a problem in our case because the interactions between the feet and the ground are simplified, and the generated torque is mostly bypassed by our contact model. However, this means that the dispersion of energy into the ground is inaccurate, which could be a clue for further progress of this work, which will be discussed in the conclusion section.

Knee extension Knee flexion

Plantar flexion Dorsiflexion

Experimental vs simulated data

In this subsection, the simulation results are confronted and compared with the experimental data computed from measured human pushes.

Step triggering. For each pair of real and simulated pushes, the correspondence of step triggering is tested. The resulting values are a sensitivity of 0.88, a precision of 0.81, and an overall accuracy of 0.74.

The step triggering condition based on the use of the XCoM and BoS is able to detect the same balance as the experimental data with an accuracy of 0.74. The relatively low accuracy is due to the high variation in the original data. However the high sensitivity of 0.88 shows that the necessity of stepping is well detected, rather taking a step than risking a fall.

CoM trajectory. Figures 11 and12 showcase a study of the CoM trajectory over time in the direction of the push. Only stepping motions are examined in these figures. The color gradient corresponds to the intensity of the push for each of the simulated (red) and real (blue) motions. Once the step is triggered, Figure 11 shows the similarity in the position over the duration of the step, which is directly related to both the simulated StepTime and feet goal position. The mean values and variance in Figure 12 have been studied to detect discrepancies. From a statistical standpoint, the hypothesis of the SPM t-test is never rejected, supporting that our model is able to reproduce accurately the real motions of human balance recovery.

As a complementary analysis, various cases of different push angles, subject height and subject weight with corresponding morphology were tested in our simulation. A supplementary video is provided to demonstrate those configurations, and while no data is used to validate the results, this serves as an example of the potential uses of this simulation.

Baseline comparisons

We provide a qualitative comparison with two baseline methods: (a) RL: a reinforcement learning based neural network policy, trained using proximal policy optimization (PPO) [START_REF] Reda | Physics-based Motion Retargeting from Sparse Inputs[END_REF]; and (b) MPC: an online model predictive control method [START_REF] Howell | Predictive sampling: Real-time behaviour synthesis with mujoco[END_REF]. The RL policy is conditioned on the current state as well as a target state that consists of the position and velocity of the head, root, hands, and feet. The policy is trained to imitate walking and standing clips, consisting of about 10 minutes of data. At inference time, the policy is tracking a reference clip that consists of a fixed standing pose. The character is simulated using the GPU-based ISAAC simulator. The MPC policy uses a receeding horizon optimization of 0.35 s, as simulated using MuJoCo. We experiment with sampling-based method and iLQG methods, which both optimize the torque-based action trajectory over the horizon. A multithreaded implementation allows for interactive experimentation with the MPC method, opeerating at 10 to 50% of real-time. While the default MuJoCo-MPC implementation [START_REF] Howell | Predictive sampling: Real-time behaviour synthesis with mujoco[END_REF]] comes with both a standing and walking controller, we find that the walking controller, which also stands when at rest, is most robust to pushes and thus we use this as for our baseline comparison.

The RL baseline produces results that are more qualitatively similar to the real push results than the MPC results. However the results from our proposed controller are qualitatively more similar to the real world results, taking into account the number, length, and timing of the resulting steps. We note that one possibly way to improve on the RL results would be to use an adversarial motion prior (AMP), which can be shown to be a feasible way to guide RL to solve a motion task while respecting the available motion styles. In principle, a set of the captured push responses clips could then be used as the data to train an AMP policy. However, this type of approach remains tricky to train, given that the discriminator capability and the RL state distributions are both changing over time.

CONCLUSION

Contributions. This paper introduces a new control-based model which, by taking direct inspiration from real push data, aims to accurately recreate the human response to a physical perturbation. With relevant feet positioning and forces in the body, real life stepping behaviors and trajectories can be observed in the simulator.

As new contributions to the domain this paper not only introduces a new balance assessment method but also a new adaptive foot trajectory computation strategy, and validated their robustness against novel experimental data in the context of physics-based human simulation.

Limitations. Our method shows its limitations mainly through the use of the default collision system for the contact foot model. The behavior of the character upon hitting the ground is far from reality, as shown by the torque analysis of the physical plausibility study. Contact with the ground simply nullifies the foot's acceleration and velocity, and then locks the foot in position to serve as support. This is something we could aim to improve, as this is an important interaction where the energy from the push is dissipated into the ground.

Another limitation concerning the foot trajectory is the legcrossing issue. When stepping to the side, the other leg becomes an obstacle to the step and must be taken into account in the foot trajectory. While the issue does not appear in perpendicular pushes, pushes from the side often results in our simulation with the legs colliding and a fall. While the current solution is to disable solid collisions, this does not represent reality and should be replaced by a better foot trajectory computation, aware of the space available.

Finally, while the produced trajectories are comparable to the experimental values, it lacks the inherent variability of human behavior. Figure 11 is a prime example of how the simulation's results is representative of reality, but too rigid, as the trajectories corresponding to one or two steps create two sets of end positions.

Perspectives. Adding variability using stochastic approaches to step goals and time could lead to better trajectories capable of recreating the natural variability of the experimental data, using similar distributions to those observed in the data. Variations in push angle and morphology of the character could be studied in more detail. The video linked to this paper showcases the simulation results with different push angles, height and weight. Entailing a new foot trajectory computation system and perhaps more adaptive torque computation, this could be a way of generating high variability data that could not otherwise be captured.

The variability could also lie in the surrounding environment of the simulation. A spatially aware character might be able to generate feet target position adapted to the surrounding, thus avoiding obstacles or supporting itself with other objects. This opens the way for complex balance resolution, where all parts of the body especially the hands can be used to recover balance.

With characters of different morphologies and a model that is spatially aware of the surroundings, a new type of behavior could be observed in physics-based human simulation. Multi-character physics-based interactions could be simulated, taking the push simulation to the level of a crowd simulation. Any crowd configuration could then be simulated by adding complex environments and objectives for each person in the simulation.

The simulation proposed in this paper is based on a character modeled from realistic anthropometric data and produces biomechanically plausible results, which makes it possible to consider its future use for fall simulations, or the instantiation of large scale physics-based crowd simulations.

Figure 2 :

 2 Figure 2: Character being pushed, with two close-up of internal systems (Left). Classification of the internal concepts and flowchart steps (Middle). Flowchart of the simulation iteration (Right).

Figure 3 :

 3 Figure 3: Step detection process when pushed

Figure 4 :

 4 Figure 4: Locomotion state machine

Figure 5 :

 5 Figure 5: Step Computation in 2D and 3D

Figure 6 :

 6 Figure 6: Two-step Inverse Kinematics

Figure

  Figure 7: The Jacobian forces applied

Figure 9 :

 9 Figure 9: Balance recovery for varied impulse, with the range of intensity and duration covered experimentally delimited

Figure 10 :

 10 Figure 10: Measured torques of the simulation in respect (green for valid, red for over the maximum) to recorded human maximum torques (surface)

Figure 11 :

 11 Figure 11: Trajectory over time of the normalized experimental data (blue) and simulation (red)

Figure 12 :

 12 Figure 12: Mean trajectory and variance over time of the normalized experimental data (blue) and simulation (red)

FEET MOTION STRATEGYThe core of this work is based on the simple premise that humans, after receiving a push, may need to take steps in order to recover balance. At the core of this recovery is the contact between the character and the ground: their feet. This section describes the two parts of the model that deal with the placement of the feet, respectively the Locomotion System and the Foot Trajectory.

ACKNOWLEDGMENTS

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No [899739] (CrowdDNA), as well as French government ANR CONTINUUM project under grant agreement No [ANR-21-ESRE-0030]