Rahaf Rahal
email: rahaf.rahal@irisa.fr

Amir M Ghalamzan-E

Firas Abi-Farraj
email: firas.abi-farraj@irisa.fr

Claudio Pacchierotti
email: claudio.pacchierotti@irisa.fr

Paolo Robuffo Giordano

F Abi Farraj

P Robuffo

Haptic-guided Grasping to Minimise Torque Effort during Robotic Telemanipulation

Keywords: Teleoperation, Shared Control, Grasping, Haptics, Post-grasp manipulation

Teleoperating robotic manipulators can be complicated and cognitively demanding for the human operator. Despite these difficulties, teleoperated robotic systems are still popular in several industrial applications, e.g., remote handling of hazardous material. In this context, we present a novel haptic shared control method for minimising the manipulator torque effort during remote manipulative actions in which an operator is assisted in selecting a suitable grasping pose for then displacing an object along a desired trajectory. Minimising torque is important because it reduces the system operating cost and extends the range of objects that can be manipulated. We demonstrate the effectiveness of the proposed approach in a series of representative real-world pick-and-place experiments as well as in a human subjects study. The reported results prove the effectiveness of our shared control vs. a standard teleoperation approach. We also find that haptic-only guidance performs better than visualonly guidance, although combining them together leads to the best overall results.

INTRODUCTION

Currently-available autonomous robots are still far from the reliability and safety required in many applications, such as medical robotics [START_REF] Panesar | Artificial intelligence and the future of surgical robotics[END_REF] or hazardous waste management [START_REF] Pardi | Planning maximum-manipulability cutting paths[END_REF]. In these industrial settings, robotic teleoperation has become an effective tool to combine the experience and cognitive capabilities of a human operator with the precision, strength, and repeatability of a robotic system. Indeed, robotic teleoperation has proven effective in many applications, such as in nuclear waste decommissioning, minimally-invasive surgery, demolition, and tissue palpation. However, controlling a dexterous robotic manipulator can be rather complicated as well as cognitively and physically demanding [START_REF] Talha | Towards robotic decommissioning of legacy nuclear plant: Results of human-factors experiments with tele-robotic manipulation, and a discussion of challenges and approaches for decommissioning[END_REF][START_REF] Rahal | Caring about the human operator: haptic shared control for enhanced user comfort in robotic telemanipulation[END_REF]. In this respect, shared control algorithms have been investigated as one of the main tools for designing effective yet intuitive robotic teleoperation systems, helping operators in carrying out increasingly challenging tasks. This approach enables to share the available degrees of freedom of the robotic system between the operator and an autonomous controller. Applications include surgical suturing, industrial pick-and-place, mobile robotics, and industrial robotics. Implementations of shared control can use variable admittance control or active constraints, often enforced through haptic force feedback guiding the operator along the preferred directions of motion.

The use of haptic guidance in shared control has been referred to as haptic shared control [START_REF] Abbink | Haptic shared control: smoothly shifting control authority? Cognition[END_REF] or, alternatively, as virtual fixtures, active constraints or haptic guidance [START_REF] Abi-Farraj | A haptic shared-control architecture for guided multi-target robotic grasping[END_REF] . Haptic shared control in robotic teleoperation has been employed to, e.g., guide the human operator towards a reference position or along a given path [START_REF] Ly | Intent-aware predictive haptic guidance and its application to shared control teleoperation[END_REF], to avoid certain areas of the robot workspace, to teach the operator a manual task [START_REF] Zhu | Humanrobot shared control for humanoid manipulator trajectory planning[END_REF], to modulate the level of robotic guidance [START_REF] Izadi | Impedance modulation for negotiating control authority in a haptic shared control paradigm[END_REF], based on predictions of human motion intentions (Xi Fig. 1: Teleoperation setup. The human operator uses a Virtuose 6D grounded haptic interface (Haption, France) to control a 7 degree of freedom (DoF) Panda robotic manipulator (Franka Emika, Germany), used as a 6-DoF manipulator. The user receives haptic feedback through the haptic interface and visual feedback from an LCD screen et al., 2019), or to ease the grasp of irregularly-shaped objects [START_REF] Yang | Human-machine shared control: New avenue to dexterous prosthetic hand manipulation[END_REF].

However, haptic shared control has rarely been used to optimise a relevant metric over the future consequences of a local operator's action. [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF] studied this problem and proposed a shared control algorithm able to guide the human operator towards a grasping pose that minimises the proximity to robot singularities during the execution of the object future trajectory. Although the preliminary results were quite promising, we also realized that just directly providing the haptic cues may sometimes result confusing for the human operator since these cues (that follow the gradient of the chosen cost function w.r.t. the current pose) can easily end up guiding the operator towards unfeasible grasping poses.

In this work, inspired from the works of [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF]; [START_REF] Pardi | Choosing grasps to enable collision-free post-grasp manipulations[END_REF]; Selvaggio et al. (2019a); [START_REF] Parsa | Haptic-guided shared control grasping: collision-free manipulation[END_REF], we present a haptic shared control approach that helps human operators grasping objects in a way that minimises the torque exerted by the robotic manipulator over a future post-grasp trajectory. Besides considering a torquerelated metric, our system further improves on the previous work by making sure that the provided haptic cues will always guide the user towards a feasible grasping pose whereas, as explained, the cues proposed by [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF] do not have guarantee of pointing towards feasible grasps. This is obtained by first using a geometric grasping algorithm to find several good grasp poses on the target object, and by then evaluating, for each grasping candidate, the torques needed by the remote robot for picking up the object and moving it along a pre-planned trajectory. In addition, we propose to convey this torque-related information using haptic guidance, visual guidance, or a combination of the two to human operators, which helps them to choose a grasp candidate corresponding with the minimum robot torque effort. As a final contribution, we also demonstrate the feasibility and effectiveness of the proposed approach in a set of representative pick-and-place tasks and by conducting a human subjects study where we compare the performance of the above three feedback modalities.

The contributions of our paper can then be summarised as follows:

present a haptic shared control technique selecting the best local grasping pose to minimise the robot torques over a future post-grasp trajectory; devise three feedback techniques to guide the operator toward the best local grasp pose: haptic-only, visual-only, and combined visuo-haptic guidance; carry out real-world trials using three representative objects having different shapes, dimensions, and weights; carry out an extensive human subjects evaluation enrolling 15 subjects, using statistical tools to compare the considered feedback conditions over six metrics.

The proposed framework is compatible with any robotic teleoperation system and it only requires the 3-dimensional model and an estimate of the dynamic parameters of the target object(s). In this paper, as a proof of concept, we consider a teleoperation system (Fig. 1) composed of a 6-DoF grounded haptic interface and a 7-DoF robotic manipulator equipped with a parallel jaw gripper.

RELATED WORKS

Robotic grasping and manipulative movements are key elements to build a reliable robotic solution for many industrial setups. In this regard, robotic teleoperation has proven effective in many applications, such as in nuclear waste decommissioning [START_REF] Talha | Towards robotic decommissioning of legacy nuclear plant: Results of human-factors experiments with tele-robotic manipulation, and a discussion of challenges and approaches for decommissioning[END_REF], minimally-invasive surgery (Selvaggio et al., 2019a), demolition [START_REF] Corucci | Toward autonomous robots for demolitions in unstructured environments[END_REF], tissue palpation [START_REF] Pacchierotti | Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery[END_REF], and needle insertion [START_REF] Meli | Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation[END_REF]. However, controlling a dexterous robotic manipulator can be rather complicated as well as cognitive and physically demanding [START_REF] Rahal | Caring about the human operator: haptic shared control for enhanced user comfort in robotic telemanipulation[END_REF]. Teleoperating a robot with complex and nonintuitive kinematic poses a high cognitive load on human operators [START_REF] Talha | Towards robotic decommissioning of legacy nuclear plant: Results of human-factors experiments with tele-robotic manipulation, and a discussion of challenges and approaches for decommissioning[END_REF]. While conservative industries do not sufficiently trust the autonomous systems, the emerging autonomous technologies can potentially ease the teleoperation and help reduce the cognitive load on the human operator.

Simple robotic manipulation tasks can be classified in two categories: (i) synthesising grasp poses, and (ii) performing manipulative movements. There is a bulk of research on synthesising grasp poses from given single/multiple point clouds, e.g., by sim-to-real learning [START_REF] James | Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks[END_REF], deep learning via domain randomisation [START_REF] Tobin | Domain randomization and generative models for robotic grasping[END_REF], deep learning that learns hand-eye coordination for grasping unknown objects [START_REF] Levine | Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection[END_REF] and probabilistic generative models of grasping configurations [START_REF] Kopicki | One-shot learning and generation of dexterous grasps for novel objects[END_REF]. Other works, e.g. [START_REF] Varley | Shape completion enabled robotic grasping[END_REF], reported improved performance in synthesising grasping configuration for unknown objects by shape completion, e.g., via primitive shape fitting to point cloud [START_REF] Baronti | Primitive shape fitting in point clouds using the bees algorithm[END_REF]. However, state-of-the-art autonomous grasping methods currently do not provide proof of stability/success of the grasping actions which is demanded by conservative industries, such as nuclear waste decommissioning. For instance, [START_REF] Kopicki | Learning better generative models for dexterous, single-view grasping of novel objects[END_REF] presents an approach to improve some of these shortcomings as reported in [START_REF] Kopicki | One-shot learning and generation of dexterous grasps for novel objects[END_REF]: this approach can partially generalise across different working conditions, unknown objects and unknown environments with success rate of 87.8%. Although the improved performance reported in [START_REF] Kopicki | Learning better generative models for dexterous, single-view grasping of novel objects[END_REF] (i.e. < 25% improvement in the success w.r.t. the result reported in [START_REF] Kopicki | One-shot learning and generation of dexterous grasps for novel objects[END_REF]) is significant, the obtained success rate is still far from 100% which is demanded by conservative industries [START_REF] Talha | Towards robotic decommissioning of legacy nuclear plant: Results of human-factors experiments with tele-robotic manipulation, and a discussion of challenges and approaches for decommissioning[END_REF].

The works above are only concerned with (i) reach-tograsp, (ii) forming grasping configuration on an object surface and (iii) lift the object [START_REF] Pardi | Choosing grasps to enable collision-free post-grasp manipulations[END_REF]. Nonetheless, the robot may need to deliver complex manipulative movements after making a stable grasp. Other lines of research studied the planning and control of manipulative movements [START_REF] Ratliff | Chomp: Gradient optimization techniques for efficient motion planning[END_REF] to deliver complex (e.g. post-grasp) actions. For instance, a robot may need to grasp an air paint spray gun and follow a specific trajectory to deliver the quality painting. There is a range of different manipulative movements, e.g., they can be a simple pickand-place task [START_REF] Ghalamzan | Task-relevant grasp selection: A joint solution to planning grasps and manipulative motion trajectories[END_REF], suturing with a non-invasive surgical robot (Selvaggio et al., 2019a), robotic painting or robotic cutting [START_REF] Pardi | Planning maximum-manipulability cutting paths[END_REF]. The optimisation based manipulation planning algorithm [START_REF] Ratliff | Chomp: Gradient optimization techniques for efficient motion planning[END_REF][START_REF] Zucker | Chomp: Covariant hamiltonian optimization for motion planning[END_REF][START_REF] Schulman | Motion planning with sequential convex optimization and convex collision checking[END_REF] can encode the environmental constraints into a cost function where it can be used to find the optimum trajectory in a given environment. While the robot can optimally perform a task using trajectories computed by optimisation based approaches, they are usually computationally very expensive [START_REF] Zucker | Chomp: Covariant hamiltonian optimization for motion planning[END_REF] and may not be suitable if the robot needs to quickly adapt to the changes in its environment. Robot learning from demonstrations (LfD) [START_REF] Abbeel | Autonomous helicopter aerobatics through apprenticeship learning[END_REF] was proposed to overcome this shortcoming and quickly generalise demonstrated trajectories. For instance, [START_REF] Peternel | Robotic assembly solution by human-in-the-loop teaching method based on realtime stiffness modulation[END_REF] proposed a human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. The haptic feedback is used both as a means for improving teacher demonstrations and as a human-robot interaction tool in [START_REF] Rozo | A robot learning from demonstration framework to perform force-based manipulation tasks[END_REF]. [START_REF] Kormushev | Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input[END_REF] presented method to learn positional and force profiles and reproduce robot force interactions in a human-robot interaction setting.

The two lines of research, namely grasp synthesising and manipulative movements planning, have been studied mostly in isolation. However, synthesised grasping configurations may impose some constraints on feasible post-grasp manipulative movements. For example, a certain grasping configuration could lead to reaching singularities or joint limits in the post-grasp task. Moreover, the desired manipulative movements may limit the choice of suitable grasping configuration, e.g., the screwing task determines that a robot must form a stable grasp on the handle of a screwdriver. In this regard, [START_REF] Detry | Task-oriented grasping with semantic and geometric scene understanding[END_REF] select the grasping pose with the maximum corresponding affordance utility value.

There are a few numbers of studies on jointly considering the problem of grasping an object, and delivering desired post-grasp manipulative movements. For instance, [START_REF] Pardi | Choosing grasps to enable collision-free post-grasp manipulations[END_REF] and [START_REF] Ghalamzan | Task-relevant grasp selection: A joint solution to planning grasps and manipulative motion trajectories[END_REF] studied how the choice of a grasp pose can be used to avoid singularities and collision during manipulative movements. Two-phase optimisations were used in [START_REF] Horowitz | Combined grasp and manipulation planning as a trajectory optimization problem[END_REF] to generate the contact necessary for making a stable grasp on an object and to find the optimal object path that can be followed, given the optimal grasping configuration. In contrast, [START_REF] Vahrenkamp | Bimanual grasp planning[END_REF][START_REF] Vahrenkamp | Manipulability analysis[END_REF] studied the optimal grasps resulting in a maximum manipulability at initial grasping configuration. Similarly, [START_REF] Mavrakis | Analysis of the inertia and dynamics of grasped objects, for choosing optimal grasps to enable torque-efficient post-grasp manipulations[END_REF][START_REF] Mavrakis | Safe robotic grasping: Minimum impact-force grasp selection[END_REF] studied the selection of the gasping configuration yielding the best safety value and torque efforts during the post-grasp manipulative movements. These works, however, assume a reliable planner can generate several precise/stable grasp poses where there does not yet exist a reliable autonomous system fully trusted by conservative industries for different scenarios, e.g., for an arbitrary object in a variety of lighting conditions, making this an open research challenge [START_REF] Zhou | Visual robotic object grasping through combining rgb-d data and 3d meshes[END_REF].

The conservative industries only trust fully teleoperated robotic systems despite the significant recent advancements in autonomous grasping systems [START_REF] James | Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks[END_REF][START_REF] Levine | Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection[END_REF][START_REF] Kopicki | Learning better generative models for dexterous, single-view grasping of novel objects[END_REF]. In fact, a very basic teleoperation system is still the only means of performing many remote manipulation tasks. For example, the majority of robots deployed in the nuclear industry are still teleoperated by a human [START_REF] Bogue | Robots in the nuclear industry: a review of technologies and applications[END_REF]. Teleoperation may become very complex and impose a high cognitive load on a human operator [START_REF] Talha | Towards robotic decommissioning of legacy nuclear plant: Results of human-factors experiments with tele-robotic manipulation, and a discussion of challenges and approaches for decommissioning[END_REF]. The human operator may not be fully aware of consequences of commanded manipulative movements (Selvaggio et al., 2019a;[START_REF] Pardi | Choosing grasps to enable collision-free post-grasp manipulations[END_REF][START_REF] Ghalamzan | Task-relevant grasp selection: A joint solution to planning grasps and manipulative motion trajectories[END_REF][START_REF] Detry | Task-oriented grasping with semantic and geometric scene understanding[END_REF][START_REF] Mavrakis | Analysis of the inertia and dynamics of grasped objects, for choosing optimal grasps to enable torque-efficient post-grasp manipulations[END_REF][START_REF] Mavrakis | Safe robotic grasping: Minimum impact-force grasp selection[END_REF][START_REF] Parsa | The impact of motion scaling and haptic guidance on operators' workload and performance in teleoperation[END_REF], hence, they may use trial and error procedures, e.g. procedures of reach-to-grasp movements, grasping, moving, dropping the object (because a constraint has occurred), changing the robot configuration to avoid the local constraint, and so on. Controlling a complex robotic system using human inputs alone is often difficult and may require special skills or dedicated training [START_REF] Franchi | Shared control: Balancing autonomy and human assistance with a group of quadrotor uavs[END_REF]. Hence, assisted teleoperation has been proposed to facilitate the teleoperation by, for example, allowing an autonomous algorithm to perform parts of the task or by providing the operator with visual and haptic guidance [START_REF] Abi-Farraj | A visual-based shared control architecture for remote telemanipulation[END_REF]. This can potentially reduce the workload of the human operators and improve their performance [START_REF] Boessenkool | A task-specific analysis of the benefit of haptic shared control during telemanipulation[END_REF].

Several assisted teleoperation frameworks have been proposed to tackle the problem of grasping or manipulation. For example, [START_REF] Achibet | The virtual mitten: A novel interaction paradigm for visuohaptic manipulation of objects using grip force[END_REF] introduced a paradigm for visual-haptic manipulation of objects and [START_REF] Cipriani | On the shared control of an emg-controlled prosthetic hand: Analysis of user prosthesis interaction[END_REF] discussed the impact of different force-feedback-based control strategies on the operator's performance during grasping. Similar studies for different manipulation tasks are presented in [START_REF] Boessenkool | A task-specific analysis of the benefit of haptic shared control during telemanipulation[END_REF][START_REF] Nelson | Robotic manipulation using high bandwidth force and vision feedback[END_REF]. In these works, the haptic feedback provided to the operator aims at transmitting the forces (which are sensed through tactile and/or force sensors on the remote-arm) to the operator. In contrast, [START_REF] Masone | Semi-autonomous Trajectory Generation for Mobile Robots with Integral Haptic Shared Control[END_REF] proposed an approach in which the operator is informed about the feasibility of modifying an intended trajectory. However, these assisted teleoperation approaches are used only for solving either reach-to-grasp (RtG) or post-grasp manipulative movements (PGMM). Neither of those facilitates planning jointly for both RtG and PGMM. Hence, a selected grasp by a human may not be optimal for post-grasp motions. In other words, the human operators are not informed about the consequences of their actions in a receding time horizon, i.e. about the quality of their preferred grasp pose in terms of control effort or singularity over the post-grasp motions.

Haptic-guided shared control has recently caught the attention of the researchers to improve the telemanipulation experience which helps joint solution of RtG and PGMM. Applications include surgical suturing (Selvaggio et al., 2019a), a simple pick-and-place [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF], mobile robotics [START_REF] Bruemmer | Shared understanding for collaborative control[END_REF][START_REF] Cognetti | Perception-aware human-assisted navigation of mobile robots on persistent trajectories[END_REF], and industrial robotics [START_REF] Selvaggio | Haptic-based shared-control methods for a dual-arm system[END_REF][START_REF] Selvaggio | A shared-control teleoperation architecture for nonprehensile object transportation[END_REF]. Implementations of shared control can use variable admittance control [START_REF] Duchaine | General model of humanrobot cooperation using a novel velocity based variable impedance control[END_REF] or active constraints [START_REF] Rahal | Haptic shared-control methods for robotic cutting under nonholonomic constraints[END_REF][START_REF] Pacchierotti | Steering and control of miniaturized untethered soft magnetic grippers with haptic assistance[END_REF], often enforced through haptic force feedback guiding the operator along the preferred directions of motion. For instance, in [START_REF] Abi-Farraj | Humanoid teleoperation using task-relevant haptic feedback[END_REF], the haptic feedback cues are considered to avoid reaching the constraints of keeping the balance of the humanoid robot. Selvaggio et al. (2019b) present a passive task-prioritised shared-control method for remote telemanipulation of redundant robots. The proposed method fuses the task-prioritised control architecture with haptic guidance techniques to realise a shared-control framework that keeps the user away from the dangerous region of the workspace. [START_REF] Adjigble | An assisted telemanipulation approach: combining autonomous grasp planning with haptic cues[END_REF] presented a shared control system in which if the operator follows the desired trajectory generated by a grasp planner, zero force would be fed back to the leader robot, and if he/she deviates from the desired path, a returning force would be proportional to the geometric distance. [START_REF] Abi-Farraj | A haptic shared-control architecture for guided multi-target robotic grasping[END_REF] used haptic information to guide the user smoothly and continuously as the user switches from a grasp candidate to the next one, or from one object to another one, avoiding any discontinuity or abrupt changes. Selvaggio et al. (2019a) compute the optimal grasping configuration in a manifold of grasping configurations around a needle where the robot does not face any singularities or joint limits. [START_REF] Singh | Haptic-guided teleoperation of a 7-dof collaborative robot arm with an identical twin master[END_REF] used haptics (1) to feedback the force sensed at the remote-arm and (2) to inform the human operator of collision in robot workspace. In [START_REF] Rahal | Caring about the human operator: haptic shared control for enhanced user comfort in robotic telemanipulation[END_REF], haptic force cues are generated to guide the users towards successful completion of the task, along with directions that improve their posture and increase their comfort. Moreover, the effect of inaccuracy in haptic shared control on human performance is studied in [START_REF] Van Oosterhout | Haptic shared control in tele-manipulation: Effects of inaccuracies in guidance on task execution[END_REF]. [START_REF] Peternel | Human arm posture optimisation in bilateral teleoperation through interface reconfiguration[END_REF] propose a method for improving the human operator's arm posture during bilateral teleoperation. [START_REF] Parsa | Haptic-guided shared control grasping: collision-free manipulation[END_REF] presents a shared control informing the operator of future expected collision by haptic force cues. However, the works above do not consider the joint effort of the manipulator during manipulative movements. Prior work [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF] studied how the user can benefit from the predictive singularity cost computation. This is based on a Task-Oriented Velocity manipulability (TOV) cost function. The user can then select a grasping configuration so that the robot avoids singularities during manipulative movements. However, this optimisation was performed without accounting for whether the resulting optimised end-effector pose was compatible with the main task of grasping the object as the force cues based on TOV cost can disagree with the movements toward the object to grasp. Moreover, neither of the works (i) use a combination visual-haptic guidance, nor (ii) use a discretised grasping manifold to provide a space of possible grasp poses.

The present work proposes a shared control system for addressing the above shortcomings and informing the operator about the grasp choice with the minimum torque effort -using Task Oriented Torque Effort (TOTE) -that would be exerted by the robot during the manipulative movements. A grasp pose yielding the minimum joint effort is a very important factor as minimum joint effort correlates with the minimum robots running cost. It may also yield a safe distance to min/max joint effort limits imposed by the electrical motors at each joint. This can also result in an extended lifetime of the joint motors. We illustrate the effectiveness of our proposed visual-haptic shared control for grasping an object by a series of experiments with the real robotic system shown in Fig. 1 (consisting of a Haption device and a Panda robotic arm) and a comprehensive human subject study.

SHARED CONTROL METHOD

Whenever an operator grasps an object using a teleoperated robotic manipulator, the grasping actions can be divided into three phases (see Fig. 2): (a) reach-to-grasp, when the remote robot moves towards the object to be grasped; (b) grasp, when the robot securely grasps the object with its end-effector; and (c) post-grasp movements, when the manipulator carries out In this work, we decide to leave the human operators free to approach and grasp the target object according to their preferences and to then switch to the robot autonomy as soon as the object is grasped. In this way, the operator participates to the part of the task which is most demanding in terms of cognition while leaving the rest to the robot autonomy. While the operator approaches the object, we provide feedback information about the torque necessary for the robot during delivering the object in the post-grasp autonomous movement phase. In fact, as shown in Fig. 3, for the same autonomous post-grasp manipulation defined for the object, two different grasp poses lead to different robot configurations, resulting, in turn, to different torques required by the motors.

We summarise the architecture of the control system in Fig. 4 and describe it in details in the following Sections. A grasping algorithm generates a dense distribution of feasible grasp poses using the 3-dimensional model of the object. Any data-driven grasping approaches, e.g. [START_REF] Kopicki | Learning better generative models for dexterous, single-view grasping of novel objects[END_REF], can be employed for this purpose, and we used GraspIt! [START_REF] Miller | Graspit! a versatile simulator for robotic grasping[END_REF]) in this work. The torque-related metric is then calculated for each grasp pose considering the object inertia matrix and the planned postgrasp trajectory. Finally, this information is used at runtime to guide the user towards the grasp poses locally minimising the torque metric. Combining this information with their own experience and perception of the environment, the human operators can make the choice on how to grasp the object informed about the future expected cost.

System details

In our teleoperation setup, we consider a 7-DoF manipulatorremote arm-and a 6-DoF haptic device-master arm-(see Fig. 5). We fix joint-3 of the Panda robot to reduce it to 6-DoF, as this allows us to avoid complexities of the inverse kinematics of redundant robots in Sec. 3.2. As such, the robot acts and is referred to as a 6-DoF serial manipulator.

Let us define three reference frames (see Fig. 5): F g ∈ SE(3), attached to the robot end-effector, F o ∈ SE(3), attached to the object centre of mass, and the robot base frame F r ∈ SE(3). These frames are also shown in Fig. 2, along with F gr representing the robot end-effector frame at the chosen grasping configuration. F m ∈ SE(3) is the base frame of the master device, w.l.o.g. taken parallel to the robot base frame F r ∈ SE(3), and F w ∈ SE(3) is the world frame. The configuration of the haptic interface and its Cartesian velocity are defined in F w by x m ∈ R 6 and v m ∈ R 6 , respectively. The device is modelled as a gravity-pre-compensated generic mechanical system,

M(x m )v m +C(x m , v m )v m = τ h f + τ hu + Bv m , (1)
where M(x m ) ∈ R 6×6 is the positive-definite symmetric inertia matrix, C(x m , v m ) ∈ R 6×6 are the Coriolis/centrifugal terms, τ h f , τ hu ∈ R 6 are the haptic feedback and the human operator's forces, respectively; B ∈ R 6×6 is a damping matrix for stabilizing the system. The velocity of the robot is defined as v g ∈ R 6 , and a velocity-to-velocity coupling between the master and the remote system is implemented by setting v g = v m . We let o x g = { o t g , o R g } ∈ SE(3) represent the grasp pose, which is defined as the relative pose between the gripper and the object to grasp. As we have already mentioned, we consider the post-grasp trajectory to be assigned and carried out autonomously after the user grasps the object. For instance, considering a pick-and-place task, we generate this trajectory based on the initial position of the object and a given target location. The trajectory is defined in the world frame as

w x o (t) = { w t o (t), w R o (t)} ∈ SE(3), 0 ≤ t ≤ 1,
with t being a time parametrisation such that t = 0 is the starting point and t = 1 is the endpoint of the trajectory. From the planned postgrasp trajectory, we calculate the corresponding trajectory for the robot end-effector w.r.

t. F w , w R g (t) = w R o (t) o R g w t g (t) = w t o (t) + w R o (t) o t g . ( 2
)
Using the inverse kinematics (IK) solver we then obtain the joint space trajectory θ (t, o x g ) ∈ R 6 , which is function of the object trajectory over time w x o (t) and of the grasp pose o x g . We want to inform the human operator about the quality of a candidate grasp pose o x g w.r.t. the torque that the robot would exert when moving the object after the grasp. We call this metric Task-Oriented Torque Effort (TOTE), and it is function of the previously calculated desired joint trajectory. Our shared control system provides the operators with feedback information guiding them towards poses that minimise the TOTE index and are also feasible grasp poses.

Manipulator dynamics under load

To evaluate Task Oriented Torque Effort (TOTE) for a given grasp pose, we need to calculate an augmented equation of motion which accounts for the grasped object as well as for the dynamics of the manipulator. Assuming a secure and steady grasp, the inertia tensor can then be expressed in a frame attached to the robot end-effector, as follows:

M o ( o x g ) = E -T ( o x g )M ob j E -1 ( o x g ), (3)
where E( o x g ) ∈ R 6×6 is the matrix transforming the linear and angular velocities of the object CoM to generalized velocities in the frame F g attached to the end-effector [START_REF] Murray | A mathematical introduction to robotic manipulation[END_REF][START_REF] Mavrakis | Safe robotic grasping: Minimum impact-force grasp selection[END_REF]. We can then compute the grasped object inertia matrix and, hence, the governing equation of motion of the augmented robot and grasped object in the joint space is:

M o ( o x g , θ ) = J T (θ )M o ( o x g )J(θ ) , (4)
where J(θ ) ∈ R 6×6 is the the robot Jacobian. We compute the effect of the object on the Coriolis and gravitational term of the robot dynamic equation using join-space robot governing equation of motion (JSR-GEM) and from (3) and (4), the following holds:

C o ( o x g , θ , θ ) = 1 2 n ∑ k=1 ∂ M o,i j ∂ θ k + ∂ M o,ik ∂ θ j - ∂ M o,k j ∂ θ i θ . (5)
The gravitational term of the dynamics of the grasped object in the robot joint space can be defined using JSR-GEM:

N o ( o x g , θ ) = ∂V o ( o x g , θ ) ∂ θ , ( 6
)
where V o ( o x g , θ ) = mgh o ( o x g , θ ) and h o ( o x g , θ ) can be computed using the forward kinematics of the robot. The resulting equation of motion accounting for the dynamics of the system including the external object dynamics can then be written as

M( o x g , θ ) θ + C( o x g , θ , θ ) + N( o x g , θ ) = τ( o x g , θ , θ , θ ), (7)
where variables with a bar notation refer to the system augmented with the external object. τ( o x g , θ , θ , θ ) ∈ R 6 represents the joint torques needed to perform the target manipulative action while

M( o x g , θ ) = M o ( o x g , θ ) + M(θ ), C( o x g , θ , θ ) = C o ( o x g , θ , θ ) +C(θ , θ ),
and

N( o x g , θ , θ ) = N o ( o x g , θ , θ ) + N(θ , θ ).

Task Oriented Torque Effort (TOTE) cost function

The grasp pose o x g specifies the joint space trajectory of the robot needed to perform the post-grasp manipulation (see eq. ( 2)) and affects the dynamics of the post-grasp manipulative motions, which includes the robot and the grasped object. As a consequence, the grasp pose ultimately affects the robot joint torques, τ( o x g , θ , θ , θ ) (hereafter referred to by τ( o x g ) for simplicity) needed to move the robot (and the object) over the desired trajectory. Torques τ(t, o x g ) expected at each time t of the trajectory are defined following (7) as

τ(t, o x g ) = M o x g , θ (t, o x g ), θ (t, o x g ) + C o x g , θ (t, o x g ), θ (t, o x g ) + N ( o x g , θ (t, o x g )) . (8)
Let H( o x g ) be a cost function defining the torque effort the robot exerts to perform the post-grasp manipulation, i.e., moving the object along a pre-defined trajectory in our case. Of course, this cost function can be defined for any other type of post-grasp manipulation, depending on the task at hand. Given an object and a desired trajectory to follow w x o (t), we can finally define task oriented torque effort (TOTE) cost function H( o x g ) as

H ( o x g ) = T 0 τ(t, o x g )dt, ( 9
)
where T is the duration of the post-grasp manipulation task.

NAVIGATION INFORMATION

During reach-to-grasp, we would like to guide the operator towards grasp poses locally minimising the TOTE index (which corresponds with the minimum torque effort during post-grasp manipulative motions), using a combination of haptic and visual navigation feedback. The objective of our haptic navigation cues τ h f is to guide the user towards the direction minimising the TOTE metric H( o x g ). A first approach to achieve this objective would be to provide force cues along the negative gradient of H( o x g ) w.r.t. optimization variable o x g , as done in [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF]. However, this method might lead to undesired behaviours as it does not consider where the location of feasible grasp poses on the target object, i.e., the cues could lead towards a pose that does minimize H( o x g ) but from which it is impossible to grasp the object (or the 'optimal' pose could even end up lying inside or above the object itself). In this work, we instead combine the gradient descent approach with knowledge regarding the feasible grasp poses. From the 3-dimensional model of the object, we use GraspIt! to retrieve a dense set of grasp poses. Then, for each pose, we calculate the corresponding end-effector and joint post-grasp trajectories, excluding those for which no feasible inverse kinematic solution exists. This step leaves us with a set of n feasible grasp poses o x i where 0 ≤ i ≤ n. Finally, we compute the TOTE for each feasible grasp pose o x i , called H i ( o x i ). This information is used to inform the user about the direction to follow so as to minimise the robot effort during the post-grasp action.

The most straightforward way to provide the user with a meaningful feedback may be to drive her/him towards the grasp pose corresponding with the minimum cost. However, the operator may still consider that the grasping pose corresponding ot the minimum value of TOTE is not the best choice because of other constraints important to task completions (or her/his own judgment). In general, robotic grasping/manipulation may involve different application specific objectives, e.g., the manipulation may be subject to a suitable affordance of the object or collision-free movements (just to name a few). The goal, therefore, is to provide the users with a local, continuous and informative guidance such that they get informed on how to move in the vicinity of their current gripper pose so as to minimise the torques needed for the post-grasp action. To achieve this, we do not rely on the cost function value alone in the design of the haptic cues, but also take into consideration the distance of each proposed grasp pose to the current gripper pose.

To this end, we define the roto-translational distance between any grasp candidate o x i and the current gripper pose o x g as

| o x i -o x g | = || o p i -o p g || + µ | g θ i |, ( 10
)
where g θ i ∈ [-180, + 180] is the angular part of the angleaxis representation of g R i = g R o o R i , and µ > 0 is used to properly scale the angular component of the distance with respect to the linear one (µ = π/180 in our experiments, which indicates an equal weight for angular and translation distances).

Finally, the force cues are calculated as the weighted average of multiple force vectors guiding the user towards nearby grasp poses having a low cost. The lower H i ( o x i ), the stronger the force guidance; moreover, each force vector is weighted inversely to the distance between its grasp pose o x i and the current gripper pose o x g , i.e.,

τ h f = 1 n n ∑ i=1 H( o x g ) -H i ( o x i ) 1 + k| o x i -o x g | m   o p i -o p g || o p i -o p g || g ∆ i   if H( o x g ) > H i ( o x i ) (11)
where g ∆ i is the axis part of the angle-axis representation of g R i = g R o o R i , and k and m are positive control gains (k = 6 and m = 8 in our experiments).

Visual navigation guidance

In addition to the haptic feedback described above, we provide the user with a visual representation of the different possible grasping poses to give them a preliminary idea of how "good" the different grasps are, and allow them to plan ahead their grasping pose in a way to make the post-grasping manipulation more efficient for the robot. This is achieved by color coding the grasping configurations, as shown in Fig. 6, where grasps are represented with different shades of green depending on their TOTE cost values. In Sec. 6, we study the impact of haptic, visual and viso-haptic feedback on the human performance in an comprehensive human subject tests, and the effect of combining them together on the user performance.

PICK-AND-PLACE EXPERIMENT

To test the effectiveness of our proposed shared control approach, we carried out a first pick-and-placement experiment on three different objects.

Experimental setup

The experimental setup is shown in Fig. 1. The master side is composed of a 6-DoF grounded haptic interface (Virtuose 6D, Haption, France); while the remote side is composed of a 7-DoF robotic manipulator (Panda manufactured by Franka Emika) with one joint fixed, and therefore used as a 6-DoF manipulator. An LCD screen is placed in front of the human operator (i.e. subjects). The master interface is placed next to the remote robotic manipulator to provide the subjects with a direct view on the remote workspace. This simplifies our study as we avoid any complexities caused by the choice of camera views and screens showing the remote workspace.

The environment is composed of three different objects placed on a table in front of the robot: WO: a 275-g wooden object made of two rectangles having 10×2.6×4 cm and 11×2.6×9 cm dimensions (right of Fig. 7). LA: a 11×11×15 cm rectangular lamp of 1958 g with an handle (center of Fig. 7); LS: an 228-g L-shaped object made of two 14×1.2×6.5 cm rectangles (left in Fig. 7);

These objects have been chosen for their similarity with common objects used in nuclear decommissioning scenarios [START_REF] Abi-Farraj | A haptic shared-control architecture for guided multi-target robotic grasping[END_REF], following a discussion within the European H2020 project "Robotic Manipulation for Nuclear Sort and Segregation" (RoMaNS). The use of a broader set of objects will be the subject of future work (see Sec. 8).

Task and methods

One expert operator carried out the pick-and-place task. The operator was asked to control the robotic manipulator with the grounded haptic interface to grasp the considered objects. We visualise the post-grasp trajectory before the start of the experiment and explain to the subjects that the aim of the experiments is to choose a grasping pose minimising the overall torque exerted by the robot during autonomous post-grasp motions. Once the object was grasped, the autonomous system moves the remote robot to pick the object up and deliver it at the planned pose, following a pre-planned autonomous trajectory. The pose of the object relative to a frame attached to robot hand is fixed throughout the post-grasp trajectory. We employed the shared control method described in Sec. 3. We used velocity-to-velocity mapping for masterslave commanding. Since the workspace of the master device is smaller than that of the remote robot, we used a clutch button on the master interface to decouple the motion of the master and remote systems, allowing the user to reposition the haptic device in a comfortable position before controlling again the robot [START_REF] Abi-Farraj | A haptic shared-control architecture for guided multi-target robotic grasping[END_REF]. We considered two navigation feedback modalities: (T) standard human-in-the-loop teleoperation, where the operator receives no guidance about suitable grasp poses; (H) our haptic shared control teleoperation approach, where the subject receives haptic guidance toward grasps minimising H( o x g ), as described in Sec. 4.1. The post-grasp trajectory was designed to include both a translational and rotational component and is different for each of the 3 objects, as shown in the video available at https://youtu.be/B3YCOZPpynQ.

The human operators started by grasping "WO". As described in Sec. 3, the system generated the set of feasible grasp poses, calculating for each of them the TOTE cost H( o x g ) over the post-grasp trajectory. Then, haptic feedback guides the user towards the pose locally minimising the cost, in case of H. Fig. 6a shows H( o x g ) for the feasible grasps on the object used in our experiments. The feasible grasping candidates are colour coded where the darker the green colour of the grasping candidate, the higher the corresponding cost value. The user, then, grasps object LA, receiving the same type of guiding feedback. This object represented a particular challenge for the system, as it is quite heavy and, if not handled correctly, might require torques too high for the robot to exert during the post-grasp trajectory. Again, Fig. 6b illustrates H( o x g ) for the feasible grasps on this object using different shades of green. Finally, the user grasped object LS and Fig. 6c shows H( o x g ) for the feasible grasps on this object.

It is important to highlight that the navigation feedback guides the operator towards the pose locally minimising H( o x g ). This behaviour still enables the human operator to decide from where to approach the object, combining the knowledge of the system about H( o x g ) with the experience and additional environmental information brought by the operator. For example, from Fig. 6c, we can see that H( o x g ) is equally low if grasping the object from its right or left hand side. The user might prefer one or the other considering information unavailable to the system, e.g., one part looks damaged/corroded or it is too close to other objects.

Robustness against object parameters

Our method needs a rough 3-dimensional model of the object and of the corresponding approximated inertial properties. The rough 3D model of the object can be obtained using depth cameras mounted on the robot end-effector for example and the object mass and inertia parameters can be estimated using robot pushing actions, as per our previous work [START_REF] Abi-Farraj | A haptic shared-control architecture for guided multi-target robotic grasping[END_REF] and [START_REF] Mavrakis | Estimating an object's inertial parameters by robotic pushing: A data-driven approach[END_REF]. These obtained 3D model and inertia parameters may be rough approximations of the real values: hence, we propose in this section a discussion on the effects of these uncertainties on our shared control method.

Starting with the effect of inertial properties, we observe that at low velocities, the torques on the robot joints Table 1: Average error between the measured joint torques τ g and the calculated ones using eq. ( 8) (with and without the effect of velocity and acceleration) for a sample task for our experiments Calculated (eq. ( 8 We also show the curves for two grasping poses, with a high and low TOTE value τ(t, o x g ) could be approximated solely using the gravity vector, N( o x g , θ (t, o x g )), as the other two components are negligible. To verify this observation, we move an object of 1 Kg, grasped by the gripper of the Panda robot, along a sample predefined trajectory with low velocity similar to the ones we used in our experiments. We compare the measured torques over the post-grasp trajectory to the calculated torques using eq. ( 8), with and without the effect of the velocity and acceleration components. The results are summarised in Table 1, and show that the effect of adding these terms is, indeed, negligible. We thus used this approximation to simplify the calculation of the TOTE in eq. ( 9) that is used to compute the haptic feedback. Higher velocity tasks should of course include the dynamics components in the calculations.

We also tested the effect of accuracy in the mass estimation on the cost function calculation offline, using the object LA used in our experiments. Fig. 9a shows the value of the cost function H ( o x g ) for different estimations of the object mass: its actual mass m, 0.5m and 2m. For each case, we show the value of the cost function for the grasping poses with the highest and the lowest TOTE value. Even though the value is different for each mass estimation, we notice that the three cases lead to the same ranking of the grasping poses in terms of cost function value. We can also see in Figs. 9b,9c and 9d that the evolution of the torque norm over time steps in the post-grasp trajectory has the same trend. As such, an approximation of the mass of the object would lead to the same generated shared control forces in terms of direction, even if the strength of the force is different. Since in our setting the mass of the object is known, we used the exact value of the mass in the experiments.

Results

We registered the grasp pose chosen by the operator and the torques exerted by the robot over the post-grasp trajectory in the two different control modalities (T vs. H-full teleoperation and the haptic shared control) for the three considered objects (WO vs. LA vs. LS).

As expected, the grasp pose chosen by the user differed between T and H modalities. In condition T, the operator chose the most intuitive grasp poses, shown in Figs. 8a,8c,8e, which are usually along top edge/surface line of the object. However, the robot may require a large amount of torque effort to deliver desired manipulative movements during the corresponding post-grasp manipulative movements and those grasps are not the ones with the corresponding minimum H( o x g ). In condition H, the shared control algorithm guides the user to grasp poses shown in Figs. 8b, 8d, 8f, which are less intuitive but more efficient in terms of H( o x g ).

After forming successful grasps, the operators stop teleoperating the remote robot and an autonomous controller picks up the object and moves it along a planned trajectory. Fig. 10 shows the evolution of the L2 norm of the joint torques ∥τ g ∥ over time during the post-grasp trajectory, for the three objects in case of H and T control mode. As expected, torques are minimised when the user is guided towards grasps with low H( o x g ).

It is interesting to see that our haptic-guided shared control allow the operator to freely choose different local optimal solution-it is shown in Fig. 10b that the operator can chose left and right hand side grasp pose, shown with H left and H right, which have H( o x g ) value lower than the intuitively selected grasp pose, shown with T.

HUMAN-SUBJECTS STUDY

The experiments above showcase the features of our proposed haptic-guided shared control. To further illustrate the efficiency of our approach, we present a series of human subject tests showing the effectiveness of our approach to minimise the TOTE cost in comparison with the basic teleoperation setup.

We consider again the robotic system described in Secs. 3, 5.1 and shown in Fig. 1. In this section, we only consider object WO (right hand side of Fig. 7) because it does not present any handling risk for novice users-the lamp mass, e.g., may be dropped by novice and damaging the experimental setupand also provide a larger area of grasp choices for the user. This provides a good example to compare the use of basic teleoperation and our proposed haptic-guided shared control teleoperation setups.

Task and feedback conditions

Similar to the experiments in Sec. 5, participants were asked to control the motion of the robotic manipulator to grasp the object, following the guidance information provided by the system. We describe the experiments and the task for the user, i.e. the users need to teleoperate reach-to-grasp and form a stable grasp where the post-grasp movements are performed autonomously. They were also informed that we would ideally like to have the minimum joint torque efforts during the entire experiments including post-grasp movements. The task started when the manipulator moved for the very first time, and it was considered successful completion when the robot completed the planned post-grasp trajectory. We consider four different ways of enabling the user to control the system for completing the grasping task, T: basic teleoperation, where the subject receives no haptic guidance about suitable grasp poses. H: our proposed haptic shared control approach, where the subject receives haptic guidance toward grasp poses locally minimising H( o x g ), i.e., the torque the robot needs to exert along the post-grasp trajectory. V: visual guidance, where the subject is shown a graphical representation of the scene indicating the value of H( o x g ) for each grasp pose using a colour scale (see Fig. 6a). HV: a combination of H and V, where the subject receives haptic guidance toward grasp poses minimising H( o x g ) and is shown a graphical representation of the scene indicating the value of H( o x g ) for each grasp pose using a colour scale.

Conditions T and H are the same already employed in Sec. 5. Users always started by conditions T, so to not be influenced by the guidance. The other three conditions H, V and HV were performed in a random order. In all conditions, the user controls all the DoF of the robotic manipulator through the haptic interface, as described in Secs. 3 and 5.1.

A video showing trials in all experimental conditions is available in the video.

Participants

Fifteen right-handed subjects (average age 27.6, 12 males, 3 females) participated in the study. Seven of them had previous experience with haptic interfaces, three had previous experience with robotic teleoperation in general, none had ever tried the proposed shared control technique before. The experimenter explained the procedures and spent about two minutes adjusting the setup to be comfortable before the subject began the experiment. Each subject then spent about three minutes practising the control of the system before starting the experiment.

Results

To evaluate the effectiveness of our telemanipulation system and the usefulness of the proposed shared control approach, we recorded (i) the completion time, (ii) the mean torque exerted by the manipulator joints during the post-grasp phase, (iii) the peak torque exerted by the manipulator joints during the post-grasp phase, (iv) the linear trajectory followed by the robotic end-effector, and (v) the angular motion of the robotic end-effector. Moreover, immediately after the experiment, subjects were also asked to report (vi) the effectiveness of each feedback condition in completing the given task using bipolar Likert-type eleven-point scales.

To compare the different metrics, we ran one-way repeatedmeasures ANOVA tests (first five metrics) and Friedman tests (last metric) on the data. Four control modalities (standard 2 summarises this experiment.

Fig. 11a shows the average task completion time. One user took significantly more time than any other to grasp in the H condition (41.96 s, five times the average of the other users). We did not consider this outlier in calculating the mean and 95% confidence interval showed in the figure. However, we considered it in the following statistical analysis. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated (χ 2 (5) = 16.255, p = 0.006). The one-way ANOVA test with a Greenhouse-Geisser revealed a statistically significant change in the task completion time across the conditions (F(1.868, 26.148) = 3.572, p = 0.045, a = 0.05). Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between conditions T vs. V (p = 0.004) and T vs. HV (p = 0.012). The Bonferroni correction is used to reduce the chances of obtaining false-positive results when multiple pair-wise tests are performed on a single set of data.

Fig. 11b shows the mean torque applied by the manipulators' joints along the post-grasp trajectory. Data passed the Mauchly's Test of Sphericity. The one-way ANOVA test revealed a statistically significant change in the task mean torque across the conditions (F(3, 42) = 160.778, p < 0.001, a = 0.05). Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between all conditions (T vs. H, p < 0.001; T vs. V, p < 0.001; T vs. HV, p < 0.001; H vs. V, p = 0.002; H vs. HV, p < 0.001; V vs. HV, p < 0.001).

Fig. 11c shows the peak torque applied by the manipulators' joints along the post-grasp trajectory. Data passed the Mauchly's Test of Sphericity. The one-way ANOVA test revealed a statistically significant change in the task peak torque across the conditions (F(3, 42) = 108.291, p < 0.001, a = 0.05). Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between all conditions (T vs. H, p < 0.001; T vs. V, p < 0.001; T vs. HV, p < 0.001; H vs. V, p = 0.002; H vs. HV, p = 0.004; V vs. HV, p < 0.001).

Fig. 11d shows the linear trajectory described by the robot end-effector during the task. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated (χ 2 (5) = 11.728, p = 0.039). The one-way ANOVA test with a Greenhouse-Geisser correction revealed no statistically significant change in the task linear trajectory across the conditions (F(2.085, 29.187) = 0.677, p > 0.05, a = 0.05).

Fig. 11e shows the summed angular motion described by the robot end-effector during the task. Data passed the Mauchly's Test of Sphericity. The one-way ANOVA test revealed no statistically significant change in the angular trajectory across the conditions (F(3,42) = 2.203, p > 0.05, a = 0.05).

At the end of the experiment, we asked the participants to rate the perceived effectiveness of the three conditions in guiding them in comparison to T (i.e., H vs. V vs. HV), asking "How do you rate the effectiveness of condition [XX] in completing the given task?" for each condition. The responses were given using bipolar Likert-type scales that ranged from 0 to 10, where a score of 0 meant "very low" and a score of 10 meant "very high". The same question was also used in [START_REF] Maisto | Evaluation of wearable haptic systems for the fingers in augmented reality applications[END_REF][START_REF] Meli | Combining wearable finger haptics and augmented reality: User evaluation using an external camera and the microsoft hololens[END_REF] for evaluating the subjective/perceived effectiveness of haptics in robotic teleoperation. Fig. 11f shows the perceived effectiveness of the four experimental conditions. A Friedman test showed a statistically significant difference between the means of the three feedback conditions (χ 2 (2) = 13.192, p = 0.001). The Friedman test is the non-parametric equivalent of the more No statistically significant difference between conditions.

Angular motion Angular motion

No statistically significant difference between conditions. Perceived effectiveness Perceived effectiveness H vs. HV p = 0.004 V vs. HV p = 0.032

Most effective condition (chosen by subjects)

Thirteen subjects out of fifteen chose HV, two chose V.

popular repeated-measures ANOVA. The latter is not appropriate here since the dependent variable was measured at the ordinal level. Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between H vs. HV (p = 0.004) and V vs. HV (p = 0.032).

We also asked whether subjects were tired at the end of the task, using again bipolar Likert-type scales that ranged from 0 to 10, where a score of 0 meant "not tired" and a score of 10 meant "very tired". Results showed that subjects were not particularly tired at the end of the task (mean ± stand. dev., 2.07±1.03).

Finally, thirteen subjects out of fifteen found condition HV to be the most effective at completing the grasping task. Two subjects preferred condition V. Quite surprisingly, no subject indicated H as their preferred condition.

DISCUSSION

Shared control is becoming a popular technique to improve the performance and intuitiveness of robotic telemanipulation systems. Our work aims at improving the human operator's awareness of some future manipulation related metrics, which might be non-intuitive or difficult to predict at the master side during teleoperation. As a proof of concept, we evaluated the torques exerted by the robot over a future trajectory set to be autonomously executed after the object is grasped via teleoperation. This scenario is relevant, e.g., in the sort and segregation of dangerous waste, where it is impor-tant to leave operators free to choose which object and how to grasp while providing them as much information as possible on the constraints and objectives relevant to the remote system and environment. Minimising the torque expenditure is not only relevant because it reduces the energy used by the robot-hence reduces the operation cost-but also because it extends the range of objects the system can manipulate. Indeed, as the robotic system can only provide limited torque, knowing where and how to grasp enables the manipulator to handle, e.g., heavier objects. It might be argued that expert operators know well the system dynamics and therefore can already estimate how the robot torques will evolve over a given post-grasp trajectory. However, especially when handling dangerous waste, operators do not have a direct and clear view of the environment and it might be rather difficult to estimate such a complex metric just from looking at the system from a distant window. Even if the operator has a good view of the environment, as in our experimental setup, receiving intuitive haptic cues can significantly reduce the cognitive load and speed up the process, which is already an important result. For example, the Sellafield (UK) nuclear site stores 140 tonnes of civil plutonium and 90,000 tonnes of radioactive graphite [START_REF] Pearce | Shocking state of world's riskiest nuclear waste site[END_REF]. The robotic systems in use provide teleoperation capabilities through primitive master consoles (e.g., passive joystick or teach pendants), making the process too slow for processing the material in a reasonable time, hence the need for faster-teleoperation solutions. Finally, increasing the intuitiveness of these robotic systems is expected to flatten the learning curve, enabling more operators to become proficient in a shorter time (subjects in Sec. 6 were all novices).

Of course, our framework can be adapted to consider different metrics of predicted costs, e.g., relevant for the postgrasp phase, such as cost of maximising the robot workspace in a certain direction at the end of the manipulation or positioning the robot body to optimise the user's viewpoint on future actions. As long as the set of feasible grasp poses can be assigned with relevant cost values, our approach can be adapted with very little efforts.

First, we evaluated our shared control technique in a preliminary pick-and-place experiment on three different objects. We asked experienced users to grasp each object using classic teleoperation (T) in which they received no guidance about suitable grasp poses, and our haptic shared control (H) in which they received haptic feedback towards poses locally minimising H( o x g ). Shared control H takes into account all the feasible grasp poses evaluated by the grasping algorithm, weighted by their distance to the current pose of the robot. In this way, the feedback is never abrupt and is gently updated as the robot moves around the object. This characteristic enables the user to always know where the best local grasp is, while still being able to move away if needed, which is paramount for many applications. We want to leave operators free to ultimately choose where to grasp because they might pick up on some information unknown to the system. Results of this first experiment show that robot torques during the autonomous part of the task are lower when using the haptic shared control approach.

After this preliminary experiment, we carried out a human subject study enrolling fifteen human subjects. We tested the performance of four experimental conditions: classic teleoperation (T), haptic shared control (H), visual guidance (V), where each grasp pose is color-coded to indicate its H( o x g ) value, and combined visuo-haptic guidance (HV), which provides both haptic and visual feedback as in H and V. As a measure of performance, we evaluated the time-tocompletion for each experiment, robot mean and peak torque, end-effector trajectory length and angular motion, as well as users' perceived effectiveness. Results showed that, in all the considered metrics but three (time-to-completion, linear and angular motion), the proposed visuo-haptic guidance HV outperformed more classic teleoperation T. Moreover, the great majority of subjects preferred HV over the other approaches. Results also show that in the two most relevant metrics, mean and peak torque, both haptic approaches (H and HV) perform significantly better than any other, proving the effectiveness of haptic guidance in optimising the robot torque during telemanipulation. However, as expected, providing users with this type of guidance led to a longer time-to-completion, which, of course, needs to be taken into consideration. In this respect, it is interesting to notice that condition H requires 12% and 31% less time than V and HV, respectively. At the same time, it requires 8% less torque than V and only 9% more torque than HV. In some scenarios, such an improvement in time-to-completion might be worth a small increase in torque demands. Moreover, of course, longer completion time might lead to increasing user's fatigue, especially if using the system for long periods. To minimize user effort, we will consider including ergonomics-related consideration in the computation of the costs [START_REF] Rahal | Caring about the human operator: haptic shared control for enhanced user comfort in robotic telemanipulation[END_REF] Finally, the role of visual guidance merits special attention. While it clearly makes the task longer to complete (both V and HV), it is very well appreciated by users, who chose HV and V as the most preferred conditions. This preference is not unexpected, as humans are generally rather used to follow visual navigation feedback (e.g., turn-by-turn car navigation systems based on road signs), while they are not used to follow haptic navigation cues at all. For this reason, we expect more training to significantly improve the performance of the haptic modality in all the considered metrics.

It is also interesting to notice that human operators had a good direct view of the environment. We can safely expect that, in conditions with degraded visual feedback on the robot and remote environment, haptic feedback will play an even bigger role with respect to the manipulation performance [START_REF] Pacchierotti | Cutaneous haptic feedback in robotic teleoperation[END_REF].

We also want to highlight that our approach is independent from the grasping algorithm. Although here we used GraspIt! to generate the grasp poses, our framework is capable to work on top of any other similar approach. For the same reason, as long as the algorithm can generate feasible grasp poses, our framework is expected to work on objects of any shape which are, e.g., sensed by RGB-D sensors.

Finally, as in our target application most waste consists of single rigid bodies, the proposed framework currently does not support the handling of deformable objects. This feature could be added by considering the uncertainty on the relative pose of the robot hand w.r.t. the object surface.

CONCLUSION AND FUTURE WORKS

This paper presented an innovative haptic shared control technique for robotic telemanipulation that provides the human operator with information on how to grasp an object for optimising a metric accounting for the consequences on the future object motion, which could be unintuitive or difficult for the operator to predict. For example, in this work, we considered minimising the torques the manipulator will exert on the object to carry out an autonomous manipulation after the grasp. This metric is important because it helps reducing the system operating costs while extending the range of objects it can manipulate.

Exploiting a 3-dimensional model of the object, the system identifies the feasible grasp poses and then, using a rough approximation of the object inertial properties, it evaluates for each grasp pose the torques needed by the robot for performing the post-grasp manipulative action (i.e., in our case, moving the grasped object along a pre-planned trajectory). With this information, during the teleoperation the system can provide the human operator with navigation guidance towards the grasp that locally minimises the expected torque effort. While this work focused on minimising the expected torques, the framework can be easily adapted to consider any other type of metric which is able to provide a cost for the set of feasible grasps. Similarly, while our post-grasp manipulation consisted in picking the object up and placing it somewhere else, the framework can be adapted to consider to any other type of manipulation action.

We demonstrated the effectiveness of the proposed approach in a series of representative real-world experiments as well as a human subjects study enrolling 15 subjects. The results proved the effectiveness of our shared control techniques vs. standard human-in-the-loop teleoperation in most performance metrics. Moreover, haptic-only guidance performed better than visual-only guidance, although combined visuo-haptic guidance led to the best overall results.

Future work will study how the strength of the haptic guidance links to task performance. For example, we could adapt the guidance strength to the operator's performance or experience, e.g., a system could use a stiff navigation approach (i.e., less freedom for the operator) when operated by novices, while it could employ a soft navigation approach (i.e., more freedom for the operator) when operated by experts. This approach could be also useful when teaching new operators, employing different levels of autonomy according to the operator's experience and learning process. Of course, as the framework supports different cost functions and postgrasp manipulation actions, we will also consider how to extend our experimental evaluation in this direction, considering what might be relevant for other fields of application (e.g., medical robotics). In this respect, we also plan to compare the proposed techniques with other grasping and guidance methods currently employed in these different scenarios of application. Finally, we plan to extend our experimentation to consider a broader range of environments, moving closer to the type of objects we can find in high-impact target scenarios, e.g., in nuclear waste storage sites. In this respect, we should also consider the capabilities of the employed robotic system, e.g., maximum applicable grasping and lifting forces, with respect to the considered objects in the environment, so as to always ensure a feasible manipulation.

Fig. 2 :

 2 Fig. 2: Three phases of grasping an object: (a) reach-to-grasp-moving the robotic arm towards the object to be grasped; local coordinate frames are attached to the end-effector (red), centre of mass (CoM) of the object (blue) and the selected grasp pose (green); (b) grasping the target object, and (c) performing the required post-grasp manipulation; the past local frames attached to the CoM are shown with blue dashed lines. In this paper, the first two phases are carried out via teleoperation, while the last one is performed autonomously. During the teleoperation, we provide visual-haptic feedback about the grasp poses minimising the robot torque during the future autonomous manipulation phase

Fig. 3 :

 3 Fig. 3: Two choices of grasp pose shown in (a) and (c). The choice of grasp pose and the corresponding robotic arm configuration determines the following joint space trajectory of the robotic arm for the very same object trajectory. (b) and (d) show two shots of the robotic arm (corresponding to the grasping choices shown in (a) and (c), respectively) for moving the object to the predefined pose. The joint configurations during motion are different in the two cases, as well as the torques exerted by the robot for moving the object (due to different gravity, Coriolis and acceleration terms)

Fig. 4 :

 4 Fig. 4: Block diagram summarising the shared control framework. A set of feasible grasps { o x i } is generated by a grasping algorithm. Then, we calculate the torque cost function H( o x i ) at each of these poses to design the visuo-haptic feedback τ h f to guide the human user. Finally, the operator controls the remote robot in velocity using the grounded haptic interface

Fig. 5 :

 5 Fig. 5: We consider several coordinate frames including (i) the world reference frame, and coordinate frames attached to (ii) the handle of master-arm, (iii) the end-effector of the remote-arm, base of the remotearm, and (iv) the object

Fig. 6 :

 6 Fig. 6: Feasible grasps and TOTE cost H( o x g ) for the three objects used in our experiments. The grasping candidates are colour-coded according to the corresponding cost value. Lighter green corresponds to a lower cost function

Fig. 7 :

 7 Fig. 7: Objects used in our grasping experiments. From left to right, an L-shaped wooden object (LS), a lamp (LA), and a b-shaped wooden object (WO). These objects have been chosen following a discussion within the European H2020 project "Robotic Manipulation for Nuclear Sort and Segregation" (RoMaNS), which considered them as good representatives for the sort and segregation of nuclear waste

  Fig. 8: Grasp poses teleoperated in T (fully teleoperated) and H (hapticguided shared control teleoperated) mode of control system: 8a, 8c and 8e grasp poses for WO, LS, and LA chosen by the user in T control mode; 8b, 8d and 8f grasp poses for WO, LS, and LA chosen by the user in H control mode

  actual mass: ∥τ g ∥ [Nm] vs. time [s] half the mass: ∥τ g ∥ [Nm] vs. time [s] double the mass: ∥τ g ∥ [Nm] vs. time [s]

Fig. 9 :

 9 Fig. 9: (a) Plot of H ( o x g ), for grasps with high and low TOTE, based on different mass approximations. The x-axis represents the multiplier of the mass parameters used in the calculations, i.e., half the actual mass, the actual mass, and twice the actual mass. (b)-(d) Evolution of the norm of the joint torques ∥τ g ∥ for the actual mass (b), half the mass (c) and double the mass (d). We also show the curves for two grasping poses, with a high and low TOTE value

  Fig. 10: Time [s] vs. norm of the joint torques ∥τ g ∥ [Nm] during the post-grasp trajectory for T and H control mode and three objects: 10a WO (corresponding with 8a and 8b), 10b LS (corresponding with 8c and 8d), 10c and LA (corresponding with 8e and 8f)

Fig. 11 :

 11 Fig. 11: Human subjects experiments: Mean and 95% confidence interval of (a) completion time, (b) mean torque and (c) peak torque over the post-grasp trajectory, (d) linear trajectory and (e) angular motion of the end effector for the four conditions (T, H, V, HV), and (f) subjective perceived effectiveness for conditions H, V, HV

Table 2 :

 2 Summary of the experiment

	Task	Control the haptic teleoperation system to grasp and
		lift an object.
	Participants 15 subjects (12 males, 3 females)
	Conditions T (standard teleoperation), H (haptic shared control),
		V (visual guidance), HV (haptic shared control and
		visual guidance)
	Statistical analysis (one-way rm ANOVA or Friedman test)
	Completion time Completion time
		T vs. V	p = 0.004	T vs. HV p = 0.012
	Mean torque Mean torque
		T vs. H	p < 0.001	T vs. V	p < 0.001
		T vs. HV p < 0.001	H vs. V	p = 0.002
		H vs. HV p < 0.001	V vs. HV p < 0.001
	Peak torque Peak torque
		T vs. H	p < 0.001	T vs. V	p < 0.001
		T vs. HV p < 0.001	H vs. V	p = 0.002
		H vs. HV p = 0.004	V vs. HV p < 0.001
	Linear trajectory Linear trajectory

This project was partially funded by EU H2020 RoMaNS, 645582, and EPSRC EP/M026477/1, National Centre for Nuclear Robotic.