Anastasia Bolotnikova
email: a.bolotnikova@softbankrobotics.com

Sébastien Courtois

Abderrahmane Kheddar

Allison M Okamura

Multi-contact Planning on Humans for Physical Assistance by Humanoid

Keywords: Physical Human-Robot Interaction, Physically Assistive Devices, Humanoid Robots, Robot Companions

published or not. The documents may come

I. INTRODUCTION

T HE world is facing an ageing population problem [1].

Our objective is to use a humanoid technology to partly compensate for the lack of workforce in caregiving for the elderly and frail. We want to enable a humanoid to assist in daily-life motions of frail or ageing persons, in well-identified situations, to help support the person's autonomy [START_REF] Niemelä | Robots as social and physical assistants in elderly care[END_REF]. To help in walking safely, sit-to-stand transfer, getting out or in bed, etc. The assistance consists mainly in providing a support by physical contacts, through which extra-torque or guidance are provided to the human for such tasks. Contrarily to some trend in this domain [START_REF] Onishi | Generation of human care behaviors by human-interactive robot riman[END_REF], [START_REF] Mukai | Development of a nursing-care assistant robot riba that can lift a human in its arms[END_REF], a robot whatsoever is not expected to hold with full power a person that still has some capabilities of motion. We rather see a humanoid as a companion that could play the role of a reconfigurable static or moving hurdle to provide contact support to a person to help with motion tasks.

Indeed, a humanoid robot can potentially be used as a reconfigurable and mobile multi-functional assistive support structure. One such platform can be used to assist in various tasks, unlike simpler robots that are designed for a specific Fig. 1: Safe and feasible humanoid robot posture for pHRI. task. Moreover, a friendly anthropomorphic appearance of the robot can result in overall better usability due to higher likeability [START_REF] Staffa | Recommender interfaces: the more human-like, the more humans like[END_REF], more intuitive communication [START_REF] Mitzner | Identifying the potential for robotics to assist older adults in different living environments[END_REF], [START_REF] Torta | Evaluation of a small socially-assistive humanoid robot in intelligent homes for the care of the elderly[END_REF] and easier gain of user trust and acceptance of the technology [START_REF] Li | A cross-cultural study: Effect of robot appearance and task[END_REF].

We develop multi-contact planning that works directly on a human point cloud. The central question of multi-contact planning is the computation of a feasible multi-contact configuration of a humanoid robot. We answer the question: how to compute a feasible robot posture in contact with a human, and plan contact locations on a surface of a human body part, while accounting for human safety and comfort, as well as robot structure and assistance task inherent constraints? (Fig. 1)

We build additional constraints to the posture generator (PG) in [START_REF] Brossette | Multicontact postures computation on manifolds[END_REF] formulated as non-linear optimization on non-euclidean manifolds. Multi-contact planning follows care-givers guidance to build and initialize PG, which includes the robot, assistance task and human inherent constraints and objectives. We present newly integrated PG constraints that plan contact locations on a surface of a human body part specified in the assistance task description. We fit a Non-Uniform Rational B-Spline (NURBS) surface, trimmed by a NURBS curve, on a segmented point cloud, that represents a human body part, as acquired from the embedded robot's camera, and use it to formulate geometric contact constraints. Additionally, we use collision avoidance with human point cloud constraints to ensure physical human-robot interaction (pHRI) safety. More explicitly, our contributions are:

1) A construction and initialization of PG for pHRI from an intuitive description of an assistance task (Sec. III); 2) A formulation of constraints for contact location planning on a human point cloud surface (Sec. III-A); 3) A human point cloud processing for contact planning and collision avoidance PG constraints (Sec. III-B); 4) An evaluation of our method in pHRI tasks (Sec. IV).

II. RELATED WORKS

Multi-contact planning has been addressed in various works and is currently a hot topic in humanoid research. Yet, it has never been extended to human physical assistance. To our best knowledge, our work is the first to consider such a perspective. At the heart of multi-contact planning, there is a so-called PG that generates on-demand (i.e. requests from the search strategy) the possible contact candidates.

Multi-contact planning methods treated mostly interaction with static and rigid environments, for which exact models are available. Existing methods address multi-contact planning by building a search tree [START_REF] Escande | Planning contact points for humanoid robots[END_REF] or using a cascade coupling [START_REF] Tonneau | An efficient acyclic contact planner for multiped robots[END_REF], both based on a frequent inference of the PG (viewed as generalized inverse kinematics). First, the contact surfaces are elected and the corresponding contact posture is found, if exists, that realizes these contacts. Other methods embed contact planning directly with the motion problem formulation. For instance, in [START_REF] Arreguit | Fast multi-contact whole-body motion planning with limb dynamics[END_REF] the contact sequence is predefined, the rigid and static environment is modelled by flat circular surfaces, and the contact locations of each humanoid endeffector are part of the optimization variables, so they are free to move inside the predefined contact surfaces. Similarly, the contact locations on flat surfaces, and also the contact sequence itself, have been incorporated into an optimization problem by the use of mixed-integer optimization [START_REF] Deits | Footstep planning on uneven terrain with mixed-integer convex optimization[END_REF], [START_REF] Ponton | A convex model of momentum dynamics for multi-contact motion generation[END_REF], non-linear trajectory optimization [START_REF] Dai | Whole-body motion planning with centroidal dynamics and full kinematics[END_REF] or augmentation of the contact creation related decision variables [START_REF] Mordatch | Discovery of complex behaviors through contact-invariant optimization[END_REF]. Those are some of the most outstanding works in multi-contact motion planning, but so far none have ever addressed contact planning on a human body for assistance.

One could be tempted to extend previous multi-contact planning to pHRI. In practice there are interesting simplifications such as the fact that (i) a human present a closed-form almost know surface for contact planning, and (ii) assistance must follow recommendations from geriatric, care-givers and doctor professionals, which means that the type of contact to achieve a given assistance are known and must be followed. However, there are also some difficulties such as the fact that (i) human is articulated, and (ii) its surface is varying with clothes and deformations which require using direct perception to plan contacts, e.g. point clouds. The PG on point cloud has been explored in preliminary experiments using plane segmentation for stair climbing [START_REF] Oßwald | From 3d point clouds to climbing stairs: A comparison of plane segmentation approaches for humanoids[END_REF], and in multi-contact navigation planning in flat and rigid surfaces environments [START_REF] Brossette | Pointcloud multi-contact planning for humanoids: Preliminary results[END_REF]. In pHRI, however, basic plane fitted on a point cloud cannot wellrepresent human body surfaces. The inclusion of a trimmed NURBS surface into the PG, in our work, allows to achieve high flexibility for the modelling of a surface for contact location planning. As a result, our framework can handle a wide range of various pHRI tasks.

We use a single RGBD camera and human links location 2D probability heatmaps, form OpenPose library [START_REF] Cao | Realtime multi-person 2d pose estimation using part affinity fields[END_REF], to perform human link point cloud segmentation (Sec. III-B). The output of this segmentation is used to construct a parametric surface for contact location planning and convex hulls of human body parts for collision avoidance constraints (Sec. III-A). There are works dedicated to the problem of the semantic meaning of human body parts in a point cloud. For instance, the method to fit an entire 3D human body model to the point cloud was proposed in [START_REF] Kwok | Volumetric template fitting for human body reconstruction from incomplete data[END_REF]. This method, however, requires two RGBD cameras, a set of precomputed human body templates and takes around 30s to complete. A faster method for human pose estimation via skeleton fitting on a point cloud acquired by multiple RGBD cameras was proposed by [START_REF] Barros | Real-time human pose estimation from body-scanned point clouds[END_REF]. This method, first, roughly initializes human skeleton on a point cloud. Then, points in the cloud are assigned the limb class to which they are closest to. This method is reported to perform under 1 s, and could potentially be used in our framework for point cloud segmentation. However, it requires two RGBD sensors calibrated setup, whereas we opt to use one RGBD sensor for lower set-up complexity and potential use of the method with the on-board robot camera. The machine learning-based labelling of point cloud elements into human body part classes was proposed in [START_REF] Buys | An adaptable system for rgb-d based human body detection and pose estimation[END_REF]. The output of this method could also potentially substitute OpenPose probability heatmaps in our framework (Fig. 4). However, the performance of this method on our sample data has shown to be far less superior compared to that of the OpenPose, which made it unusable in our work.

In the following Sec. III, we describe in details our proposed methodology. Our method requires a point cloud from a single RGBD camera and an intuitive description of the pHRI task, to generate a safe and feasible whole-body posture suitable for the task, and find appropriate optimal contact locations on a finely defined human body part surface, fitted directly on the human point cloud. To the best of our knowledge, no PG framework has such functionality, which is critical in pHRI.

III. PROPOSED METHOD

In the world, where robots will be working with human caregivers for assisting patients in secondary care tasks, it is paramount that high-level intuitive human commands can be translated into low-level robot motion planning and control objectives. It is especially critical to make sure that there is no increased workload of caregivers due to the introduction of robots into their workspace, a problem that often goes unnoticed in the process of introduction of new technology [START_REF] Niemelä | Robots as social and physical assistants in elderly care[END_REF]. Indeed, in caregiving sector, assistance know-how and practices should be instructed to the robot from health professionals knowledge and practices [START_REF] Khatib | Robots in human environments: Basic autonomous capabilities[END_REF]. Our objective is to enable the robot to generate safe feasible postures for engaging in pHRI as instructed from the high-level instructions given to the robot by non-(robotics) experts.

For a given assistance task instruction, here we denote it as T , a humanoid robot has to simultaneously plan its optimal floating base location (i.e. position and orientation) and joints configuration that we call robot posture q. Additionally, the robot needs to autonomously plan the suitable, appropriate and feasible contact locations on the patient body, where assistive forces should be applied. The computed posture must enable the robot to supply or resist required contact forces to assist human as required while ensuring its own equilibrium, human comfort and safety. For solving such a task, we formulate a non-linear optimization problem described by eq. 1.

1 min q,u,v,f φ(q, u, v, f ) (1a) s. t.                                   
joint and torque limits self-collision avoidance environment-robot collision avoidance environment-robot contacts equilibrium and friction cone constraints human-robot collision avoidance

f min i ≤ f c i ≤ f max i ∀i ∈ [1, m] P e i = S i (u i , v i ) ∀i ∈ [1, m] u i , v i ∈ Ω C i ∀i ∈ [1, m] (1b) (1c) (1d) (1e) (1f) (1g) (1h) (1i) (1j)
Here, φ is a cost function -i.e. a function of the decision variables, to minimize (e.g. distance to a desired posture, torques, contact forces, etc.). The explicit expression of φ is given in Sec. IV, eq. 5a. The constraints eq. 1b-1h have been previously discussed in [START_REF] Brossette | Multicontact postures computation on manifolds[END_REF] and are described in detail in [START_REF] Brossette | Viable multi-contact posture computation for humanoid robots using nonlinear optimization on manifolds[END_REF]. Some are bounds on the decision variables and others are more general bounds. The constraints eq. 1i-1j, which enable the robot to plan contact location on a surface of the human body part, are introduced in this work, in Sec. III-A.

An assistance task T , for instance, could be the following: T = {Lightly contact the patient on the left arm with your right hand and look at her/his face}. Such description of the task determines the structure of PG optimization problem, it's constraints and objectives. More precisely, it dictates the total number of robot-human contacts m, robot end-effector to be used for making contact P e i , the patient body parts to contact S i (eq. 1i) (trimmed by Ω C i if necessary (eq. 1j)), class of force bounds that is appropriate for the task f min i , f max i (eq. 1h), and robot head orientation objective (see eq. 5a).

To autonomously plan the contact locations on a surface of a human body part, as dictated by T , we add eq. 1i to the problem, which restricts the contact to lie on the NURBS surface S i fitted to the point cloud of the segmented human body part. To allow our framework to equally handle any arbitrary point cloud or special-cases, we use additional curve enclosure constraint, eq. 1j, that trims away areas in the parametric space of S i that are not suitable for contact (e.g. not covered by the point cloud, sensitive or delicate areas of the human body surface). The NURBS surface parameters u i and v i are then stacked into our PG on manifold decision variables u and v, respectively. The robot contact forces f c i are here stacked into a PG decision variables vector f . Depending on the problem and its size, non-linear optimization is not deterministic in general, see discussions in [START_REF] Brossette | Multicontact postures computation on manifolds[END_REF].

Nevertheless, we suggest that the information that we extract from T and the information of the human perception in the environment (Sec. III-B), allows us to have very good initial guesses. This allows the solution, when it exists, to be found relatively fast. We bring practical examples of such task-aware PG initialization in the Sec. IV. As in any gradient descend approaches, there is no guarantee that the solution is a global optimum nor that it could be systematically found when it exists. In the latter case, we perturb the initial guess.

A. Contact constrained to a surface fitted on a point cloud

In this section, we detail the constraints for planning a contact location on an point cloud surface (eq. 1i-1j).

Given a point cloud D = {p k |k = 0, • • • , K} and an initial guess of the control points locations P init ij of the NURBS surface S, the control points position update c ij , that fits S onto D, can be computed via quadratic optimization that minimizes the Euclidean distance between points p k and corresponding closest points projected onto surface as S(u k , v k ), see eqs. 2a-2b.

min cij k p k - i j N i,b (u k )N j,r (v k )(P init ij + c ij ) i j N i,b (u k )N j,r (v k ) 2 (2a) 0 ≤ u k , v k ≤ 1 u k , v k ∈ R (2b)
The parameters b and r denote surface order, in directions U and V of the surface parametric space, respectively (see Fig. 3). The nonrational B-spline basis functions are denoted as N x,y (z). The number of control points can be either predefined or adjusted in the fitting process [START_REF] Piegl | The NURBS book[END_REF], [START_REF] Dimitrov | Non-uniform b-spline surface fitting from unordered 3d point clouds for as-built modeling[END_REF], [START_REF] Flöry | Fitting curves and surfaces to point clouds in the presence of obstacles[END_REF].

Example of a point cloud D and a fitted NURBS surface S are shown in Fig. 2 (left). This figure also illustrates the next issue we need to address, which is the four-sided nature of the NURBS surface. Since NURBS parametric space is four-sided, surface fitted to an arbitrary point cloud likely needs to be trimmed by fitting a constraining closed curve C that encloses the point cloud, thus defining a subspace of the surface parametric space Ω C ⊂ U V that is suitable for making a contact. The control points of C are found by, first, projecting 3D points p k into U V ⊂ R 2 space, to get corresponding 2D points g k . Given the initial guess of curve control points location P init i , so-called, footpoint parameter t k is computed for every g k , so that point C(t k ) on the curve is the closest point to g k and n t k is curve normal at this point. The constraining curve fitting process consists in finding curve control points position update values c i by solving the optimization problem eqs. 3a-3b [START_REF] Flöry | Constrained curve fitting on manifolds[END_REF].

min ci k w k g k -i N i,s (t k )(P init i + c i ) u N i,s (t k ) 2 (3a) (g k -C(t k )) T • n t k ≤ 0 ( 3b
)
where s is the curve order and w k are the point's weights, which are lower for the interior points and higher for the points which are closest to the curve. The points g k , projected onto the U V space of S, along with fitted constraining curve C, are shown in Fig. 2 (left). Note that if D represents a rather four-sided real surface, the fitting of C may not be required, as the surface underlying such a point cloud will already be well-defined by S (i.e. nothing to "trim out" with C).

After S and C control point values are adjusted to fit D, the goal is to find such values of u * and v * so that the contact point location P e ∈ S(u * , v * ) is on the area of NURBS surface covered by the point cloud (i.e. u * , v * ∈ Ω C ), while satisfying all the other PG constraints eq. 1b-eq. 1h.

The fitted surface and the constraining curve can now be incorporated into the PG formulation. We consider here only one contact for the purpose of clarity. The end-effector frame for contact is expressed as P e = {p e , R e = ( x e , y e , z e )}, with frame position p e and orientation R e w.r.t. the world frame. We add surface parameters as additional decision variables u and v, and add the constraints eqs. 4c-4j to the PG problem.

min q,u,v,f φ(q, u, v, f ) (4a)
s. t. eq. 1b-eq. 1h (4b)

p s = i j N i,b (u)N j,r (v)P ij i j N i,b (u)N j,r (v) (4c) 0 ≤ u, v ≤ 1 u, v ∈ R (4d) ([u v] -C(t uv )) T • n tuv ≤ 0 (4e) du = ∂S(u, v) ∂u dv = ∂S(u, v) ∂v (4f) z s = du × dv y s = z s × du (4g) R s = du |du| , y s |y s | , z s |z s | (4h) # » p s p e • R s = (0, 0, 0) (4i) z e • R s xy = (0, 0) z e • R s z ≥ 0 (4j)
The constraints eqs. 4c-4e ensure that the contact point p s lies on the authorized surface area. The constraints eqs. 4f-4h compute the surface contact frame orientation R s at point p s with Z-axis aligned with the surface normal. The constraints eqs. 4i-4j align the robot's end-effector frame with the surface contact frame in 3 translational directions and 2 orientational Fig. 3: Curve enclosure constraint on U V space.

axes. The robot is free to choose its contacting end-effector orientation only around the surface normal. These constraints are illustrated in Fig. 2 (right); specifically eq. 4e is illustrated in Fig. 3. Here, the variable t uv ∈ [0 1], from the curve parametric space, is a footpoint parameter of [u v] point, computed on previous optimization iteration, such that C(t uv ) is the closest point on the curve to [u v] and n tuv is the curve's normal at this point.

The solution to eq. 4 is an optimal whole-body robot posture and the contact location on the point cloud surface, approximated by the trimmed NURBS surface, that satisfies joint and torque limits, maintains robot statically stable, keeps interaction forces inside the friction cones, avoids collisions and satisfies contact force bounds.

In the following Sec. III-B, we detail the human point cloud segmentation that supplies the input D for the construction of the contact constraints of the proposed PG framework. We also describe how the point cloud segmentation is used to construct human-robot collision avoidance constraints of PG (eq. 1g).

B. Processing of human point cloud

Our proposed robot-human contact planning, described in Sec. III-A, can be used for assistance pHRI tasks, as long as, the point cloud to contact D (e.g. human shoulder, human back) is properly segmented out from an entire scene observed by an RGBD camera. Here, we present the point cloud segmentation pipeline that supplies input to our PG. The overview of the entire pipeline is shown in Fig. 4.

First, an RGB image is processed by a two-branch multistage convolutional neural network (CNN) from the OpenPose library [START_REF] Cao | Realtime multi-person 2d pose estimation using part affinity fields[END_REF]. This CNN predicts confidence maps (CM) for 25 main human body keypoints, by assigning the likelihood of the presence of a particular human body part to every image pixel. Simultaneously, CNN predicts, so-called, Part Affinity Fields (PAF), which encode the location and orientation of human body parts in the 2D image.

In our work, we use CM and PAF to compute human body parts 2D masks for point cloud segmentation. We threshold PAF and CM of all the body parts to consider only high likelihood pixels (≥ σ = 40%), which are assigned a 100% likelihood after thresholding. The pixels with likelihood below σ are assigned 0% likelihood. As a result, we obtain black and white images that represent 2D masks of human body parts.

We combine all resulting 2D masks to obtain 16 masks for body part segmentation from point cloud (different total number of masks can be used depending on the use-case). We dilate the resulting masks to remove small holes and expand the borders. The head mask is augmented by adding an ellipse of estimated head width and height around the face centre. The torso mask is augmented with a polygon that connects 3-4 visible torso keypoints (assuming that at least 3 torso keypoints are visible). Further, subtraction of body parts masks that are likely to be occluding torso (e.g. arms, forearms, hands) is performed on the torso mask.

Once the masks for all individual body parts are computed, a depth image and camera intrinsic parameters are used to compute 3D point cloud of the entire scene. We apply human body parts 2D masks on a point cloud to segment 16 subclouds which contain only those 3D points that are likely to belong to each particular human body part.

The result of the segmentation is used to select a sub-cloud D that, according to predefined task T , is to be used for establishing a contact. This sub-cloud D is filtered, downsampled made fit a NURBS surface S and a trimming curve C, using NURBS algorithms from Point Cloud Library [START_REF] Piegl | The NURBS book[END_REF], [START_REF] Rusu | 3d is here: Point cloud library (pcl)[END_REF], and building contact constraints eqs. 4c-4j.

All segmented sub-clouds are used to create (strictly) convex hulls of human body parts for human-robot non-desired collisions avoidance (eq. 1g). The PG's collision avoidance is implemented using efficient GJK distance algorithm for proximity queries [START_REF] Escande | A strictly convex hull for computing proximity distances with continuous gradients[END_REF]. The cloud D, representing a body link for the contact creation, is also used to define collision avoidance constraints with all robot links but the end-effector used for contact as specified in T .

There are ongoing efforts in the 3D computer graphics and vision communities to provide directly reliable 3D pose and joint configuration of humans in any posture, see e.g. [START_REF] Xu | Monoperfcap: Human performance capture from monocular video[END_REF]. Shall this be one day readily available in reasonable computation time and reliability, we can simply replace our pipeline with it, eventually pre-fit a personalized NURB on it, and use it as an input for our PG. It won't make our PG faster but we will gain in the perception side (i.e. the construction of the PG problem). We exemplify how our method performs on sample point clouds in the following Sec. IV.

IV. EXPERIMENTAL RESULTS

We evaluate the performance of our method for whole-body posture generation and contact location planning on a human point cloud using three sample pHRI scenarios in the context of human care and assistance (Sec. IV-C-IV-E).

A. Implementation details

The PG framework [START_REF] Brossette | Multicontact postures computation on manifolds[END_REF], that we use and extend for the pHRI use-case in our work, is highly versatile and is not robot specific. Given just a URDF file description of Pepper robot, that we use as a platform in our work, all the basic PG constraints (eqs. 1b)-1f) are automatically constructed.

The novel PG constraints (eqs. 4c)-4j, and 1g) are constructed based on the results of the point cloud processing in each particular scenario. The only robot specific parts of the PG formulation in our scenarios is a frame constraint for Pepper mobile base, which is free to move only in XY plane and around Z-axis of the world reference frame, and 3 contact constraints with the ground one per each robot wheel.

The RGBD data for each scenario was collected and then processed offline. We present the resulting safe and feasible multi-contact postures for pHRI, computed by our proposed method, visualized in RViz together with the corresponding point clouds and trimmed contact NURBS surfaces in the bottom row of Fig. 5). The top row plots in Fig. 5 show cost function eq. 5a convergence, for each scenario, that indicate the optimality of the resulting PG solutions. The convergence criterion of PG is thoroughly described in [START_REF] Brossette | Viable multi-contact posture computation for humanoid robots using nonlinear optimization on manifolds[END_REF] (see p. 78).

B. Results of surface and constraining curve fitting

We assume that, after segmentation and filtering, a point cloud accurately represents the underlying real surface. We consider the trimmed NURBS surface to well represent the actual surface of the human body when the average squared fitting errors, eq. 2a and eq. 3a, are below 1 2 mm and 0.005 2 , for surface and curve, respectively. Fitting the constraining curve with a lower tolerance threshold is significantly slower and more importantly useless. The human body is compliant already, and in the online experiments, the person might move a bit too. Thus, the robot must be controlled to reach the person and established the contacts in closed-loop. In the continuation of our work, the robot's configuration, that is computed by our PG, will serve as a target for the closed-loop QP [START_REF] Bouyarmane | Quadratic programming for multirobot and task-space force control[END_REF] controller, that will achieve the desired contacts and postures at best using online perception and measured contacts [START_REF] Bolotnikova | Contact observer for humanoid robot pepper based on tracking joint position discrepancies[END_REF], [START_REF] Bolotnikova | Compliant robot motion regulated via proprioceptive sensor based contact observer[END_REF]. In our controller, contacts will be made using guarded motion to absorb surface uncertainties. For example, when a motion supporting contact is required on the patient's back, it won't be required at mm precision.

C. Scenario 1: Attracting human's attention

We consider a use-case where robot attracts a human's attention by performing a light touch. The task for the robot is T 1 = {Lightly contact the patient on the left upper limb with your right hand and look at her/his face}. The points of the human left upper limb are extracted from the point cloud for fitting a trimmed NURBS surface for contact constraint with the robot's right end-effector. Other segmented sub-clouds are used to create convex hulls for collision avoidance.

Since we know the task to perform a priori, we can do a task-and human-aware initialization of PG. We initialize the robot posture with the mobile base in front of and facing the surface to contact (as detected from RGBD data), the right endeffector slightly raised and turned to be prepared for a contact, and the left end-effector in a downward position. We denote such robot configuration as preferred posture q pref , which can be dictated for specific classes of assistance tasks from human knowledge and expertise.

We use q pref in the PG cost function to keep the final result to be close to the preferred posture. We also define the robot camera orientation objective, to ensure that it is oriented towards the human head sub-cloud average point h avg . Finally, the force bounds, which can be defined by medical professionals, for the 'light' human-robot contact interaction forces are set to f min = {-0.05, -0.05, 0.5}, f max = {0.05, 0.05, 3.0} in our scenarios. The final form of such the PG is eq. 5.

min q,u,v,f w p q -q pref 2 + # » p cam h avg # » p cam h avg • R cam z -1.0 2 (5a)
s. t. eq. 1b-eq. 1h eq. 4c-eq. 4j

where p cam and R cam z are the translation and Z-axis orientation of the robot camera optical frame, respectively, w.r.t. the world reference frame. The vector w p contains the weights of the preferred posture objective for each robot joint. The elements of this vector are set to 1, except for the mobile base and neck joints, which are set to 0, to let the robot freely plan mobile base and head position and orientation.

The solution of eq. 5, a safe and feasible robot posture for the pHRI task T 1 , is shown with annotations of all objectives and constraints in Fig. 1. Another view of the same scene, that better illustrates the result of contact location planning on a point cloud surface, is shown in the bottom left of Fig. 5.

D. Scenario 2: Initiating assistance for sit-to-stand transfer

The second scenario consists in initiating a process of assistance for sit-to-stand transfer. Note that a suitable strategy for assistance in such scenario may vary from patient to patient. We assume that a suitable strategy is to initiate two contacts. The first contact is closer to the patient's shoulder for applying a pushing force forward and upward. The second contact is closer to the hand of the patient, which would allow to control human's forward movement by resisting force applied on the robot end-effector by the patient. The same trimmed NURBS surface is used to formulate both PG contact constraints. However, different initial points in the surface parametric space are used for decision variables initial values, one closer to the shoulder and another closer to the hand. The command of the assistance task for this scenario given to the robot is T 2 = {Lightly contact the patient on the left upper limb near the shoulder with your right hand, and near the patient's hand with your left hand, while looking at her/his face}. The objective function in this scenario has the same form as eq. 5a, with two sets of contact constraints with trimmed NURBS surface and a different task-specific q pref (with both end effectors raised and turned in preparation for the contact). The resulting posture and contact locations are shown in the bottom middle of Fig. 5.

E. Scenario 3: Checking for responsiveness

In our last presented scenario, the robot is required to check if a person, lying on a bed, is responsive. The task given to the robot is T 3 = {Lightly contact the patient on the right upper limb with your left hand and look at her/his face}. The human right upper limb point cloud is segmented out and used for surface and curve fitting, which are then used to define the contact constraints between human right upper limb and robot left end-effector. We reuse the objective function of scenario 1, with a different value of h avg , as the human head is now in a different location in the scene. The robot left end-effector is slightly raised and turned in q pref . The resulting whole-body posture and optimal contact location are shown in the bottom right part of Fig. 5.

F. Discussion, limitations and future work

The task complexity, commutation times of each part of our method and the total number of PG solver iterations for each scenario are presented in Tab. I. All computations are performed on the GeForce GTX 1050 Ti GPU.

The convex hull for collision avoidance between the robot and the bed, that a person is sitting or lying on, is defined and added manually to the simulation scene in all scenarios. Ideally, such convex hulls should be computed automatically.

We also assume that a person is well detectable on a 2D image by OpenPose with the pixel probability threshold of 40%. Otherwise, the human point cloud cannot be well segmented and a safe robot posture cannot be computed.

In our future work, the proposed method will be optimized for the online (re)planning of the robot motion in pHRI scenarios. We will use the output of proposed PG to control the robot in experiments, where it engages in physical contact with people, for assistance in motion [START_REF] López | Compliant control of a humanoid robot helping a person stand up from a seated position[END_REF]. The user reaction to such interaction will be analyzed and used in the following steps of development of motion planning methods and a 'robotiquette' for pHRI in close proximity [START_REF] Dautenhahn | Socially intelligent robots: dimensions of humanrobot interaction[END_REF].

Lastly, the proposed method will be incorporated into the motion synthesis framework to compute robot trajectories accounting for the type of motion that human and robot must undergo while maintaining or switching contacts [START_REF] Escande | Planning contact points for humanoid robots[END_REF]. This must be done to guarantee that the computed postures outputted are indeed suitable for a particular a priori known assistance in motion task all along the motion path.

V. CONCLUSION

We proposed new constraints for generating safe and feasible multi-contact robot postures for pHRI tasks. We presented the details of the proposed pHRI specific contact constraints, that allow a robot to autonomously plan a feasible optimal contact location on human body parts. The implementation of the human point cloud processing, that generates the input for contact location planning and human-robot collision avoidance constraints, is presented and evaluated in three sample pHRI scenarios. In the future, we will work on integrating the PG with the QP controller and making a series of experiments with real patients.

Fig. 2 :

 2 Fig. 2: NURBS surface and curve fitting to point cloud (left), trimmed contact surface constraint representation (right).

Fig. 4 :

 4 Fig. 4: Point cloud processing pipeline to supply input for the posture optimization problem formulation for pHRI task.

Fig. 5 :

 5 Fig. 5: PG convergence plots for each scenario (top), computed robot postures in contact with human point cloud (bottom).

TABLE I :

 I The computation time of each phase of the method and the number of solver iterations for each test scenario.

	ACKNOWLEDGMENT

Matrices and vectors are in bold; scalars in non-bold lower-case; descriptive functions in calligraphic font.

We thank L. Marande, M. Arlandis, M. Busy, M. Caniot, M. Besombes and A. Laflaquière at SoftBank Robotics for the help with data collection and edits. We thank A. Escande and P. Gergondet for the help with getting into the PG.