C Pontonnier

C Livet

A Muller

A Sorel

G Dumont

N Bideau

Ground Reaction Forces and Moments Prediction of Challenging Motions: Fencing Lunges

Keywords: dynamics, optimization, GRF, motion, contact

Introduction

Classically, inverse dynamics studies of motion are based on a bottom-up approach measuring the Ground Reaction Forces and Moments (GRF&M) with force platforms. The field of study is constrained to the contact zones of these platforms, diminishing the ecological properties of the motion to be studied and constraining drastically the analysis. Such a limitation is critical for sports motion analysis, since motions to be studied can be quite large in amplitude and varied in their contacts with the ground. GRF&M prediction offers the opportunity to enlarge the field of study without additional measure to be made. Optimization-based GRF&M prediction methods are using a set of contact points defined under the feet to generate forces acting on the body. Motion analysis is used to define at any moment the points in contact with the ground and an optimization routine finds the minimal set of forces to be applied on these points satisfying the dynamical equilibrium of the whole body. If such methods have been validated on daily living motions [START_REF] Fluit | Prediction of ground reaction forces and moments during various activities of daily living[END_REF]) and some classical sports motions [START_REF] Fluit | Prediction of ground reaction forces and moments during various activities of daily living[END_REF]) such as running, vertical jump or side-cut motions, their capacity to predict GRF&M on complex sport motions is still to be proved. Lunge motion is a fundamental attack of modern fencing, asking for a high level of coordination, speed and accuracy. It consists in an explosive extension of the front leg accompanying an extension of the sword arm. In such motions, the direction of action and the way feet are orientedguard position -are particularly challenging for a GRF&M prediction method. The current study aims at evaluating an optimizationbased GRF&M prediction method on a large set of fencing lunge data.

Methods

Experimental data

The experimental data used in the current study is extracted from another one dealing with uncertainty during lunge motions [START_REF] Sorel | Studying fencing lunge accuracy and response time in uncertain conditions with an innovative simulator[END_REF]. 10 of 11 male regional to national level fencers (5 right handed, 5 left handed, aged 22±3 y.o., height 1 Corresponding author: charles.pontonnier@ens-rennes.fr 1.78±0.06m, weight 75±8kg) participated to this study, giving their informed consent after a presentation of the protocol (validated by the INRIA national ethics comitee). They performed a total of 120 fencing lunges each, under various conditions. 1 fencer of the original cohort was excluded from the present study due to unexploitable force platform data. Whole body motion of the fencers was recorded thanks to a VICON optoelectronic system (24 cameras, 250 Hz). A set of 49 markers derived from the ISB recommendations was used. Each foot was placed under an AMTI Force platform (1000 Hz), enabling the measure of GRF&M under each foot independently during the whole motion. Lunges start-end were extracted with the method proposed in [START_REF] Sorel | Studying fencing lunge accuracy and response time in uncertain conditions with an innovative simulator[END_REF]. A whole body biomechanical model developed in the CusToM library [START_REF] Muller | CusToM: a Matlab toolbox for musculoskeletal simulation[END_REF]) was used to perform the inverse kinematics analysis and the GRF&M prediction. Geometry of the model was scaled to the subjects [START_REF] Muller | A simple method to calibrate kinematical invariants: application to overhead throwing[END_REF]) and masses and inertias were obtained from (Dumas et al. 2006).

GRF&M prediction and analysis

For the total of 1200 lunges extracted from the experimentation, a classical GRF&M prediction method was applied. A set of 𝑁 𝑓 = 14 contact points per foot was defined. At each frame, a Sequential Quadratic Programming (SQP) method found the solution of the following minimization problem:

min 𝐹 ∑‖𝑭 𝒊 ‖ 2 2𝑁 𝑓 𝑖=1 𝑠. 𝑡. { 𝑀(𝒒)𝒒̈+ 𝑪(𝒒, 𝒒̇) + 𝑮(𝒒) + 𝝀 + 𝑬 = 𝟎 ∀𝑖 ∈ [1,2𝑁 𝑓 ], 𝑭 𝒊 < 𝑭 𝒊 𝒎𝒂𝒙
Where, by isolating the whole-body biomechanical model, 𝑀(𝒒) is the inertia matrix, 𝑪(𝒒, 𝒒̇) is the centrifugal and Coriolis force vector, 𝑮(𝒒) is the gravity force vector, 𝝀 is the generalized internal force vector, 𝑬 is the generalized external force vector (computed from the contact force at each contact point 𝑭 𝒊 ) and 𝑭 𝒊 𝒎𝒂𝒙 the vector containing the maximal force available for the contact point 𝑖. At each frame, a contact point was considered as active if it was close to the floor and almost without motion. The distance and velocity thresholds were respectively 0.02 m and 0.8 m/s. When a contact point was respecting the thresholds, the associated force was limited to 0.4 Body Weight and had to respect the Coulomb's law of friction. A friction coefficient of 0.5 was used here [START_REF] Fluit | Prediction of ground reaction forces and moments during various activities of daily living[END_REF]. Predicted forces and moments of each foot were compared to the measured ones in terms of RMSE, RMSE normalized to body mass, rRMSE and Pearson's correlation coefficient. 1: at the beginning of the lunge, a static phase in guard position presents an important 𝐹 𝑦 value of opposite value at each foot. Such a stabilization scheme, ensuring a good balance of the fencer, is not well predicted by the algorithm. Indeed, the minimization tends to find the minimal set of forces ensuring equilibrium, and therefore does not predict forces arising from specific motor control strategies, generally compensated in the whole-body dynamic equilibrium. This issue is a future way of improvement, by adding motor control features in the cost function/the constraints of the optimization scheme, such as balance or stiffness control.

Results and discussion

In conclusion, such a method is already quite relevant to study specific and challenging motions in a less constrained field, even if improvements can be made in the definition of the optimization problem to be solved.

Figure 1 -

 1 Figure 1 -Predicted vs. measured GRF : sample trial

Table 1 -

 1 mean±std RMSE, rRMSE, RMSE/BW and r for all the trials and all the subjects.Results presented Table1are very similar to some of the results presented in[START_REF] Fluit | Prediction of ground reaction forces and moments during various activities of daily living[END_REF]. The vertical forces 𝐹 𝑧 are accurately estimated with a particularly low rRMSE. Most of correlation coefficients are high, excepted for the medio-lateral force 𝐹 𝑥 and the vertical moment 𝑀 𝑧 . 𝐹 𝑥 is particularly low during the whole motion -see RMSE/BM, explaining this issue, whereas 𝑀 𝑧 is poorly predicted due to the bad estimation of the antero-posterior component 𝐹 𝑦 . This issue is particularly visible in Figure

		Left foot / Right foot
	RMSE	rRMSE	RMSE/BM	r
	(N or	(%)	(N/kg or
	N.m)		N.m/kg)
	𝑭 𝒙 12.1±3.8	21.8±10.8	0.16±0.05	0.64±0.24
	11.2±3.6	20.0±7.7	0.15±0.04	0.69±0.23
	𝑭 𝒚 71.9±11.9	17.2±3.4	0.97±0.20	0.89±0.04
	71.0±11.2	15.5±2.6	0.96±0.18	0.90±0.04
	𝑭 𝒛 66.5±68.6	7.0±6.0	0.85±0.77	0.96±0.07
	65.8±68.5	6.8±4.8	0.84±0.77	0.96±0.08
	𝑴 𝒙 89.1±78.5	5.7±2.8	1.14±0.88	0.98±0.03
	81.1±76.5	7.4±5.9	1.05±0.87	0.94±0.14
	𝑴 𝒚 15.2±7.7	7.3±2.7	0.20±0.08	0.97±0.03
	17.8±9.5	6.4±2.1	0.23±0.12	0.98±0.02
	𝑴 𝒛 31.3±12.0	32.2±13.5	0.41±0.13	0.50±0.26
	29.7±11.1	26.3±20.7	0.40±0.17	0.68±0.30


Firas Abi-Farraj

Claudio Pacchierotti

Oleg Arenz

Gerhard Neumann

Paolo Robuffo Giordano

A Haptic Shared-Control Architecture for Guided Multi-Target Robotic Grasping

Keywords: shared control, teleoperation, active constraints, virtual fixtures, grasping !

Although robotic telemanipulation has always been a key technology for the nuclear industry, little advancement has been seen over the last decades. Despite complex remote handling requirements, simple mechanically-linked master-slave manipulators still dominate the field. Nonetheless, there is a pressing need for more effective robotic solutions able to significantly speed up the decommissioning of legacy radioactive waste. This paper describes a novel haptic shared-control approach for assisting a human operator in the sort and segregation of different objects in a cluttered and unknown environment. A 3D scan of the scene is used to generate a set of potential grasp candidates on the objects at hand. These grasp candidates are then used to generate guiding haptic cues, which assist the operator in approaching and grasping the objects. The haptic feedback is designed to be smooth and continuous as the user switches from a grasp candidate to the next one, or from one object to another one, avoiding any discontinuity or abrupt changes. To validate our approach, we carried out two human-subject studies, enrolling 15 participants. We registered an average improvement of 20.8%, 20.1%, 32.5% in terms of completion time, linear trajectory, and perceived effectiveness, respectively, between the proposed approach and standard teleoperation.

INTRODUCTION

Robotic teleoperation has been central in the technological advances of the past century, with applications ranging from space and deep ocean exploration to manipulating contaminated material in hazardous environments. However, teleoperating a multi-degree-of-freedom (multi-DoF) robot can be challenging for several reasons. First, it is hard to make good judgment of depth when looking at the remote environment from behind standard monoscopic screens [START_REF] Kaufman | Sight and mind: An introduction to visual perception[END_REF]. Moreover, the simultaneous coordination of rotational and translational motions in a 6-DoF environment is cognitively demanding. In this respect, it has been observed that humans tend to heavily rely on translations when given command of a 6-DoF robotic system [START_REF] Ware | Rotating virtual objects with real handles[END_REF], [START_REF] Zhai | Quantifying coordination in multiple dof movement and its application to evaluating 6 dof input devices[END_REF]. Rotations are usually overlooked and avoided except if utterly necessary. The same studies also observe an "incremental" behavior when the operator needs to actuate a rotational motion: users actuate one rotation direction at a time instead of controlling all 3 rotations together, as they usually do when commanding translations. This behavior is in line with interesting psychological studies arguing that human subjects are incapable of mentally rotating objects in 3-dimensional space [START_REF] Parsons | Inability to reason about an object's orientation using an axis and angle of rotation[END_REF]. To address these limitations, shared control has • 1 F. A. Farraj, C. Pacchierotti, and P. Robuffo Giordano are with the CNRS, Univ Rennes, Inria, IRISA -35042 Rennes, France. {firas.abi-farraj, claudio.pacchierotti, prg}@irisa.fr been investigated as one of the main tools to design complex but intuitive robotic teleoperation systems, helping operators in carrying out several increasingly difficult robotic applications, such as robotic grasping, cutting and precise positioning tasks [START_REF] Kofman | Teleoperation of a robot manipulator using a vision-based human-robot interface[END_REF], [START_REF] Park | Structured beam projection for semiautomatic teleoperation[END_REF], [START_REF] Pedemonte | Visualbased shared control for remote telemanipulation with integral haptic feedback[END_REF]. Shared control makes it indeed possible to share the available degrees of freedom of a robotic system between the operator and an autonomous controller. The human operator is in charge of imparting high level, intuitive goals to the robotic system that are translated to motor commands by an autonomous controller. How to implement such division of roles between the human operator and the autonomous controller highly depends on the task, robotic system, and application.

This paper presents a novel haptic shared-control architecture for assisting a human operator in grasping objects of different shapes while assuming no previous knowledge of the environment. A depth camera reconstructs the scene and a grasping algorithm generates potential grasp candidates. Based on haptic feedback, the user is guided toward suitable grasping poses. The paper tackles the problem of simultaneous multi-target haptic guidance in the full six dimensional case. Additionally, the user is provided with haptic feedback regarding the presence of potentially dangerous singularities and joint limits of the manipulator. The algorithm uses a combination of vibrotactile and kinesthetic feedback to help the user identifying the source of the force cues (guidance away from singularities and joint limits vs. attraction toward potential grasp candidates). Human subjects experiments are performed to evaluate the overall architecture.

The rest of the paper is organized as follows. Sec. 2 provides a review of related literature on haptic shared control, focusing on manipulation tasks and multi-target scenarios. Sec. 3 describes the teleoperation system; Sec. 3.1 details how the algorithm generates the grasp candidates from the point cloud; Sec. 3.2 describes the shared control algorithm as well as how the active constraints are generated and enforced; Sec. 4 details the two experimental evaluations, which are then discussed in Sec. 5. Finally, Sec. 6 concludes the paper and comments on potential future perspectives.

RELATED WORK AND CONTRIBUTIONS

In this section, we present a review of relevant previous work on shared control, virtual fixtures and haptic guidance architectures. We start by discussing different ways of implementing haptic shared control for general human-robot interactions. We then focus on related work with respect to our target application of teleoperated robotic sorting of nuclear waste and haptic guidance in the presence of multiple potential targets.

Haptic shared control

Haptic feedback and guidance have been shown to play a significant and promising role in shared-control applications. For example, haptic cues can provide the user with information about the state of the autonomous controller and its current plan; or they can be used to gradually limit the degrees of freedom available to the human operator, according to the difficulty of the task or the experience of the user [START_REF] Steele | Shared control between human and machine: Using a haptic steering wheel to aid in land vehicle guidance[END_REF], [START_REF] O'malley | Shared control in haptic systems for performance enhancement and training[END_REF], [START_REF] Crandall | Characterizing efficiency of human robot interaction: a case study of shared-control teleoperation[END_REF]. The dynamic nature of active haptic constraints enables to design very flexible robotic systems, which can easily and rapidly change the division of roles between the user and the autonomous controller. Boessenkool et al. [START_REF] Boessenkool | A task-specific analysis of the benefit of haptic shared control during telemanipulation[END_REF] called this approach "haptic shared control", defining it as a technique in which "an assisting system continuously communicates a calculated ideal control input to the human operator by forces (in a passive or active way)". Similar haptic-enabled shared control approaches have also been refered to as virtual fixtures, active constraints, haptic guidance or guiding, and virtual guiding forces. Already in 2012, Abbink et al. [START_REF] Abbink | Haptic shared control: smoothly shifting control authority?[END_REF] reviewed several implementations of haptic shared control in the literature, arguing that it is a promising approach to meet common design guidelines for human-automation interaction, especially for automotive applications. Later on, Boessenkool et al. [START_REF] Boessenkool | A task-specific analysis of the benefit of haptic shared control during telemanipulation[END_REF], [START_REF] Boessenkool | Haptic shared control improves tele-operated task performance towards performance in direct control[END_REF] investigated haptic shared control as a way of supporting operators while performing a teleoperated bolt-and-spanner task. Based on the available sensory information at the slave side, the haptic system calculated the ideal control action, which was then provided to the operator as guiding force feedback. More recently, Hong and Rozenblit [START_REF] Hong | A haptic guidance system for computer-assisted surgical training using virtual fixtures[END_REF] proposed a haptic shared control architecture for surgical training. The trainee was guided away from "forbidden regions" using haptic guidance of a varying magnitude according to the trainee's proficiency level. Under different names, haptic shared control has been successfully used in several applications and tasks, e.g., to guide the operator toward a reference position [START_REF] Mulder | Sharing control with haptics: Seamless driver support from manual to automatic control[END_REF], [START_REF] Marayong | Effect of virtual fixture compliance on human-machine cooperative manipulation[END_REF], [START_REF] Passenberg | Towards realtime haptic assistance adaptation optimizing task performance and human effort[END_REF], [START_REF] Bowyer | Active constraints/virtual fixtures: A survey[END_REF], [START_REF] Abayazid | Experimental evaluation of comanipulated ultrasound-guided flexible needle steering[END_REF], to avoid certain areas of the environment [START_REF] Bowyer | Active constraints/virtual fixtures: A survey[END_REF], [START_REF] Meli | Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation[END_REF], and for training on manual tasks [START_REF] Li | Negative efficacy of fixed gain error reducing shared control for training in virtual environments[END_REF], [START_REF] Marchal-Crespo | The effect of haptic guidance on learning a hybrid rhythmic-discrete motor task[END_REF].

It is also possible to take advantage of the richness of haptic stimuli to provide multiple pieces of information through the haptic sensory channel (e.g., guidance and interaction forces with the environment). For example, Ramos et al. [START_REF] Peon | Vibrotactile stimuli for augmented haptic feedback in robot-assisted surgery[END_REF] combined haptic feedback and sensory substitution via vibrotactile stimuli in a teleoperated needle insertion task. They provided the human operator with vibrotactile feedback to render navigation cues and kinesthetic feedback to reproduce the mechanical properties of the tissue being penetrated. Similarly, Pacchierotti et al. [START_REF] Pacchierotti | Steering and control of miniaturized untethered soft magnetic grippers with haptic assistance[END_REF], [START_REF] Ongaro | Evaluation of an electromagnetic system with haptic feedback for control of untethered, soft grippers affected by disturbances[END_REF] presented a teleoperation system for controlling the motion of magnetic microgrippers. It provides operators with navigation and interaction information through kinesthetic and vibrotactile feedback, respectively.

Sort and segregation of waste

In this work we present an innovative haptic shared control system, designed to help a human operator grasp objects in a cluttered environment. Our target application has been inspired by the European H2020 project on "Robotic Manipulation for Nuclear Sort and Segregation" (RoMaNS). The nuclear industry is, indeed, one of the most exciting fields of application for robotic telemanipulation, and the management of nuclear waste is a prominent issue for several countries in the world [START_REF] Dittmar | Nuclear energy: Status and future limitations[END_REF], [START_REF]Radioactive Waste in Perspective[END_REF]. RoMaNS targets the Sellafield (UK) nuclear site, which is the largest nuclear site in Europe with roughly 13,000 people working in a 6 km 2 compound. It stores nearly all the radioactive waste generated by the UK's 15 operational nuclear reactors, including 140 tonnes of civil plutonium [START_REF] Wade | Sellafield plutonium a multi-layered problem[END_REF] and 90,000 tonnes of radioactive graphite [START_REF] Pearce | Shocking state of world's riskiest nuclear waste site[END_REF]. An estimation of the remaining cost of decommissioning and clean-up of the Sellafield site alone amounts to 47.9 billions GBP [30], with an increase of 90% from 2010. Sort and segregation of the stored nuclear waste is the first step toward the decommissioning of the site. Only at Sellafield, 69,600 m 3 of legacy intermediate level waste need to be placed into 179,000 storage containers [START_REF] Marturi | Towards advanced robotic manipulation for nuclear decommissioning: A pilot study on tele-operation and autonomy[END_REF], [START_REF] Shaukat | Visual classification of waste material for nuclear decommissioning[END_REF]. To avoid wastefully filling expensive high-level containers with low-level waste, many old legacy containers must be cut open, and their contents "sorted and segregated". However, current robotic systems designed for such a task provide teleoperation capabilities through primitive master consoles (e.g., passive joystick or teach pendants), making the process too slow for processing large amounts of material in a reasonable time. In this respect, Park et al. [START_REF] Park | Enhanced teleoperation for d and d[END_REF] proposed an architecture with different levels of shared autonomy for controlling popular tools in a deactivation and decommissioning environment (e.g., a circular saw,or a robotic hand). Active constraints provided guidance about the predetermined actions to be completed. Nakes et al. [START_REF] Noakes | Telerobotic sensor-based tool control derived from behavior-based robotics concepts[END_REF] combined concepts from behavior-based systems with telerobotic tool control to achieve efficient semi-autonomous teleoperation of a reciprocating saw and a powered socket tool. Leeper et al. [START_REF] Leeper | Strategies for human-in-the-loop robotic grasping[END_REF] proposed several human-in-the-loop strategies for grasping an object in a cluttered scene using visual interfaces. One of the most interesting strategies was a "point-and-click" Graphical User Interface. The user chooses a point from the scene and the grasping algorithm generates potential grasp candidates around that point. The user can then choose a suitable grasp candidate and modify it, if needed. Finally, an autonomous controller generates and actuates the trajectory from the current gripper pose to the target one. More recently, Ghalamzan et al. [START_REF] Ghalamzan | Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions[END_REF] proposed an interesting approach which also accounts for the postgrasp phase. The operator is in full control of the robotic manipulator and is able to choose any grasping pose. At the same time, active constraints guide the user in the direction which optimizes manipulability over the post-grasp trajectory. Abi-Farraj et al. [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF] proposed a shared-control architecture for assisting a human operator in grasping waste using a 6-DoF robotic manipulator. The autonomous algorithm is in charge of controlling 2 DoF of the robotic manipulator, keeping the gripper always oriented toward the object to grasp. The gripper is therefore constrained to move on the surface of a virtual sphere centered on the object; and the human operator can move the gripper across the surface of the sphere and toward/away from the object (i.e., changing the radius of the sphere). Although this approach proved to be quite efficient and robust [START_REF] Abi-Farraj | User evaluation of a haptic-enabled shared-control approach for robotic telemanipulation[END_REF], it has two significant limitations. First, it can only consider one object at a time: the user has to choose the object to grasp at the beginning of the task and cannot intuitively switch to another one in the scene. Second, the algorithm always keeps the gripper oriented toward the object's center of mass, which may not be the best way of grasping the considered object. The grasping will fail if the object needs to be grasped otherwise. These two major limitations are directly addressed by the shared-control algorithm presented in this paper.

Multi-Target Haptic Guidance

In a sort and segregation scenario, the system faces multiple objects to be grasped and manipulated in a cluttered environment. Of course, the same object may have multiple grasp candidates, depending on its geometry and position in the scene. This scenario highlights the need for a multitarget grasp guidance architecture, able to assist operators while leaving them with as much freedom as possible during the grasp. However, up to our knowledge, such an architecture is yet to be proposed.

Multi-target haptic guidance has been studied for a variety of applications, ranging from haptic computer menus [START_REF] Stephen | Solving multi-target haptic problems in menu interaction[END_REF], [START_REF] Oakley | Guidelines for the design of haptic widgets[END_REF], to assisting a human in playing a musical device [START_REF] Berdahl | Forcesensitive detents improve user performance for linear selection tasks[END_REF], and virtual reality applications [START_REF] Dennerlein | Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues[END_REF], [START_REF] Vanacken | Multimodal selection techniques for dense and occluded 3d virtual environments[END_REF]. For example, Vanacken et al. [START_REF] Vanacken | Multimodal selection techniques for dense and occluded 3d virtual environments[END_REF] investigate a multi-modal feedback system for target acquisition in densely-populated environments. Haptic and auditory feedback are employed to inform the user about the existence of a target in their immediate vicinity. No directional information is provided to the user by either haptic or audio channels, and results do not show any significant improvement of such a multimodal system when compared to using visual feedback only. Dennerlein and Young [START_REF] Dennerlein | Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues[END_REF] show that providing haptic guidance in a point-and-click task increased the user's performance and reduce the muscoskeletal loading during mouse use. Menelas et al. [START_REF] Menelas | Non-visual identification, localization, and selection of entities of interest in a 3d environment[END_REF] address the use of audio and haptics to reduce the visual load during interaction tasks within virtual environments. The user is charged with the acquisition of a target of interest in an environment containing multiple distractors. Results show that the use of haptic feedback, along with visual and auditory cues, provide a more efficient and effective interface for the user. Multi-target haptic guidance has also been studied for desktop user interfaces [START_REF] Stephen | Solving multi-target haptic problems in menu interaction[END_REF], [START_REF] Oakley | Guidelines for the design of haptic widgets[END_REF]. A haptic device is used as a mouse for hovering over and selecting items from a computer menu. The authors tackle multi-target issues arising from such a scenario and propose different ways for optimizing the performance of the operator. Results show that haptic augmentation of complex widgets may reduce performance, although carefully designed feedback can result in performance improvements. Berdahl and Kontogeorgakopoulos [START_REF] Berdahl | Forcesensitive detents improve user performance for linear selection tasks[END_REF] propose a haptic guidance architecture for assisting the user in linear selection tasks. The user is provided with haptic guidance for selecting fundamental frequencies from a continuous range. A different application is proposed in [START_REF] Robles | Integration framework for nasa nextgen volumetric cockpit situation display with haptic feedback[END_REF], where the authors propose a haptic-based framework for the integration of force feedback in a NASA NextGen Volumetric Cockpit Situation Display. The user is provided with haptic guidance for object selection and route manipulation while acting on a 2D screen interface. The user is guided away from obstacles and toward target objects. A modified Newton's gravitational law equation generates the attractive force, but the overlap between attraction fields is not directly addressed. Multiple virtual fixtures are also considered in [START_REF] Selvaggio | Enhancing bilateral teleoperation using camerabased online virtual fixtures generation[END_REF], where the user is assisted with haptic feedback for controlling the tip of a serial manipulator. Only translations are considered and a spring-like force is implemented to guide the operator toward the closest target. A passive switching mechanisim allows the operator to safely switch from one target to another through virtual damping injection. A different approach is proposed in [START_REF] Raiola | Co-manipulation with a library of virtual guiding fixtures[END_REF] for handling multi-target fixtures when controlling a serial manipulator. The authors implement a control scheme in which all fixtures are simultaneously active, but they are scaled according to the probability that their corresponding task is chosen. These probabilities are calculated using data from previous demonstrations and can, therefore, be prone to errors. Moreover, the authors consider only 3-DoF of the system without accounting for orientations in the architecture. Other approaches have also been implemented where the intention of the user is interpreted using Hidden Markov Models (HMMs) and trajectory distributions from previously demonstrated data to activate/deactivate the fixtures accordingly [START_REF] Li | Recognition of operator motions for real-time assistance using virtual fixtures[END_REF], [START_REF] Aarno | Adaptive virtual fixtures for machine-assisted teleoperation tasks[END_REF]. A smooth switching mechanism is then used to turn fixtures on and off.

The multi-target haptic guidance scenario has been studied thoroughly in literature for a wide range of applications as we have just described. However, some limitations can be highlighted. In the vast majority of the described literature, two approaches are considered for designing the attractive force cues: (i) a spring-damper system attracting the user toward the goal and (ii) a magnet-like force which increases as the user gets closer to the object. The latter design (magnet-like) is especially used when targets are close to each other as it allows the user to switch intuitively between them [START_REF] Stephen | Solving multi-target haptic problems in menu interaction[END_REF], [START_REF] Oakley | Guidelines for the design of haptic widgets[END_REF], [START_REF] Berdahl | Forcesensitive detents improve user performance for linear selection tasks[END_REF], [START_REF] Dennerlein | Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues[END_REF]. However, the magnet-like design has only been used for the two or three dimensional case. In applications where more DoFs are considered, a virtual spring-damper system has always been employed to guide the user to the different targets, while an intention recognition algorithm has been in charge of predicting the user's intention and scaling or activating/deactivating fixtures as needed to avoid noise and distracting forces [START_REF] Raiola | Co-manipulation with a library of virtual guiding fixtures[END_REF], [START_REF] Li | Recognition of operator motions for real-time assistance using virtual fixtures[END_REF], [START_REF] Aarno | Adaptive virtual fixtures for machine-assisted teleoperation tasks[END_REF]. However, such an implementation is prone to error when the intention of the user is not inferred properly and the on/off switching of the guidance can generate confusion.

Contribution

The shared-control architecture proposed in this paper aims at presenting a more general approach than our previous work in [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF], enabling the human operator to intuitively handle multiple objects with different shapes. We propose a multi-target grasp guidance architecture for the full six dimensional case. The proposed architecture employs concepts from previous literature on the two-and threedimensional case (magnet-like haptic cues) and generalizes them to the full six-dimensional case to handle translations and rotations simultaneously. Unlike previous literature tackling the 6-DoF scenario, we do not rely on probabilistic models and intention recognition for ensuring the smooth switch between the different targets. This is rather assured by the particular design of the attractive cues.

First, a point cloud scan of the target environment is used to find potential grasp candidates for all the objects in the scene. These grasping poses are then used as attraction points to provide haptic guidance to the human operator. Dynamic active constraints gently steer the operators toward feasible grasping poses, enabling him to intuitively navigate the environment and safely switch between different grasping candidates placed on the same or on a different object. Moreover, the operator is provided with additional guiding cues to ensure that he complies with certain safety constraints of the system (e.g., workspace, joints, and singularity limitations). To enable the operator to differentiate between these two haptic cues (guiding toward a grasping pose vs. proximity to unsafe configuration), we use kinesthetic and vibrotactile feedback. Active constraints providing grasping guidance are enforced by conveying kinesthetic feedback through a 6-DoF grounded haptic interface; active constraints enforcing the safety limits are conveyed via kinesthetic feedback provided through the same 6-DoF haptic interface and vibrotactile feedback provided by a custom haptic bracelet. A picture of our robotic setup is shown in Fig. 1.

METHODS

Fig. 2 illustrates the proposed framework. The human operator commands the system by applying a force τ h to a grounded haptic interface. He is then guided toward the potential grasp candidates (τ g ) while being kept away from possibly unsafe kinematic configurations of the system (τ c ). This information is provided to the operator via a combination of kinesthetic and vibrotactile stimuli, provided through the grounded haptic interface and a vibrotactile bracelet, respectively. The user is also provided with a visual representation of the scene showing the point cloud, the current gripper pose, and the grasp candidates (see Fig. 4). No information about the guidance is provided through visual feedback. Finally, the pose of the master device x m and its velocity v m are mapped into velocity commands v s driving the slave telemanipulator. Details on how these quantities are calculated are reported in the following sections.

To demonstrate the feasibility and effectiveness of the proposed approach, we employed the robotic system shown in Fig. 1. The master side consists of a Haption Virtuose 6D haptic interface, a high performance grounded device with three active translational DoF and three active rotational DoF. The slave side is composed of an Adept Viper s850 6-DoF serial manipulator equipped with a Sake EzGripper and an Intel Realsense SR-300 RGB-D sensor. However, the proposed shared control approach can also be used with similar master-slave robotic systems. The remote environment is composed of different objects placed on a small table in front of the manipulator.

Point Cloud and Grasp Pose Generation

At the beginning of the task, we generate potential grasp candidates using a point cloud model of the scene (see Fig. 3 and4). The same point cloud is also used during the teleoperation to provide visual feedback to the human operator.

To retrieve a comprehensive view of the environment, we attached a RGB-D camera to the end-effector of our robotic manipulator, as shown in Fig. 1, and we performed an automated scanning of the scene. The scanning routine iteratively builds a point cloud by driving the robot to 18 different pre-programmed positions around the scene. At each position, a new point cloud is recorded and merged with the previous ones using the Iterative Closest Point (ICP) algorithm [START_REF] Besl | Method for registration of 3-d shapes[END_REF].

The final point cloud is then used to find 6-DoF grasp candidates. To do so, we employed the grasp pose detection (GPD) algorithm [START_REF] Gualtieri | High precision grasp pose detection in dense clutter[END_REF], [START_REF] Pas | Grasp pose detection in point clouds[END_REF], which provides us with a list of grasp poses scored according to their predicted performance. However, we cannot directly use the grasp candidates produced by GPD algorithm, as they are often cluttered and include poses that are difficult or impossible to reach with our robotic system. Therefore, we performed an additional filtering on the candidates generated by GPD.

Firstly, we discarded all grasps that were not within the workspace of our robot. Then, we selected the most promising candidates, based on their GPD score and distribution over the objects (i.e., we do not consider grasp candidates too close to each other). For the scene shown in Fig. 1a, the number of remaining grasp poses was N g = 9. Throughout the paper, we will refer to the grasp poses by x i = (p i , R i ), 1 ≤ i ≤ N g , where p i ∈ R 3 and R i ∈ SO(3) represent the corresponding gripper position and orientation in a fixed world frame W, respectively. The set of grasp candidates after filtering is shown in Fig. 4.

Haptic shared-control architecture

The human operator is given control over the full six DoF of the manipulator through a position-force bilateral teleoperation coupling. He commands the motion of the slave robot by applying forces τ h on the master handle, and, at the same time, he receives haptic feedback about potential grasp candidates τ g and the presence of possibly unsafe configurations of the system τ c (see Fig. 2).

When the operator is far from any object, he only receives haptic feedback regarding the presence of possibly unsafe kinematic configurations, e.g., joint limits and singularities. As he gets within a pre-defined distance d lim from any grasp candidate, he also starts to receive haptic cues guiding him toward the closest suitable pose. If the haptic feedback is due to the proximity to a kinematic constraint, the operator is provided with kinethetic and vibrotactile feedback. On the other hand, if the cues are guiding the operator toward a grasping pose, we only provide kinesthetic feedback.

Master/slave coupling

We consider the frames of reference M, attached to the end-effector of the master interface, S, attached to the endeffector of the slave manipulator, and W, a fixed world frame (see Fig. 1). Let the pose of frame M w.r.t. W, expressed in W, be denoted by

x m = (p m , R m ) ∈ R 3 × SO(3). Similarly, let x s = (p s , R s ) ∈ R 3 × SO(3)
represent the poses of S w.r.t. W, always expressed in W. Finally, the translational and rotational velocities of M and S are defined in

W as v m = ( ṗT m , ω T m ) T ∈ R 6 and v s = ( ṗT s , ω T s ) T ∈ R 6
, respectively. The master interface is modeled as a generic, gravity precompensated mechanical system,

M (x m ) vm + C(x m , v m )v M = τ + τ h , (1)
where M (x m ) ∈ R 6×6 is the positive-definite and symmetric inertia matrix (in the "task" coordinates x m ), C(x m , v m ) ∈ R 6×6 accounts for the Coriolis/centrifugal terms, and τ , τ h ∈ R 6 are the feedback and human forces applied at the master handle, respectively. Haptic feedback τ accounts for three components,

τ = τ c + τ g + Bv m , (2)
where τ c are the haptic cues informing the operator about the kinematic constraints of the system (see Sec. 3.2.2), τ g the haptic cues guiding the operator toward the proposed grasp candidates (see Sec. 3.2.3), and B ∈ R 6×6 a damping factor which improves the stability of the system.

As for the slave, we assume that its end-effector can be controlled in velocity (as most industrial manipulators, including our Viper robot). The slave is coupled to the master with a classical cartesian coupling modulo a rototranslational scale, such that

v s = λ ∆p m -∆p s R s s θu s,d + v m , (3)
where λ is a control gain, ∆p m = p m -p m,0 and ∆p s = p s -p s,0 , with p m,0 and p s,0 representing the respective positions of the master and the slave at the beginning of the experiment. Similarly for the orientation, s θu s,d is the angle-axis representation of the relative rotation between the desired and the current orientation of the slave s R s,d = s0 R T s m0 R m , where s0 R s is the current orientation of the slave w.r.t. its orientation at the beginning of the experiment and m0 R m is the current orientation of the master w.r.t. its orientation at the beginning of the experiment. Therefore, the second term of (3) ensures a good responsiveness of the system, while the first term prevents drifts.

Haptic guidance to avoid kinematic constraints

We use kinesthetic and vibrotactile haptic feedback to keep the human operator away from possibly dangerous configurations of the system. While such constraints differ from one system to another, a generic approach is to express these constraints as a suitable cost function H(q). The gradient of H(q) w.r.t. the joint configuration vector ∂H(q)/∂q can then be used to generate the haptic feedback τ c provided to the operator. In this work, we considered manipulator's joint limits and singular configurations as the two kinematic constraints which may negatively impact the execution of the task. Of course, other constraints may be taken into account, such as torque limits or collision avoidance.

3.2.2.1 Joint limits: Inspired by the preliminary implementations of [START_REF] Pedemonte | Visualbased shared control for remote telemanipulation with integral haptic feedback[END_REF], [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF], [START_REF] Huo | The jointlimits and singularity avoidance in robotic welding[END_REF], the cost describing the proximity to the limits of a joint i ∈ [1, . . . , 6] is defined as

h i,JL (q i ) = k JL                          tan π 2 1 - q i,max -q i q th 2 if q i,max -q th < q i < q i,max , tan π 2 1 - q i -q i,min q th 2 if q i,min < q i < q i,min + q th , 0 otherwise, ( 4
) where q i is the i-th joint value, (q i,min , q i,max ) are the i-th min/max joint limits, k JL is a positive constant, and q th is the range (from the limit) in which the cost function is active. As shown in Fig. 5a, the joint cost h i,JL (q) is zero in the middle of the joint range and it grows to infinity at the joint limits.

3.2.2.2 Singularities: The singularity of a serial manipulator results in a loss of rank in its jacobian matrix. The determinant of this jacobian, which goes to zero when the jacobian loses rank, can thus be used as a measure of the robot's proximity to a singular configuration. Inspired by the preliminary implementations of [START_REF] Huo | The jointlimits and singularity avoidance in robotic welding[END_REF], [START_REF] Yoshikawa | Analysis and control of robot manipulators with redundancy[END_REF], the cost describing the proximity to singularities can be defined as

h s (q) = k s            tan π 2 1 - det(J (q)) 2 δ 2 if 0 < det(J (q)) 2 < δ, 0 otherwise. ( 5
)
where k s is a positive gain, and δ is a positive threshold. As shown in Fig. 5b, the singularity cost h s (q) grows to infinity as det(J (q)) 2 → 0, and it goes gradually to zero as det(J (q)) 2 → δ.

The total cost function H(q) is then calculated as

H(q) = i h i,JL (q) + h s (q) , (6)
and the derivative of H(q) w.r.t. time can be written as

Ḣ(q) = ∂H(q) ∂q q = ∂H(q) ∂q J † v s . (7)
The gradient descent direction v s,d , which ensures the minimization of H, is therefore defined as

v s,d = - ∂H(q) ∂q J † T , (8)
and the force cues τ c can be evaluated as

τ c = -ν c R s ∂H(q) ∂q J † T , (9)
where ν c is a gain defined to fit the master device force and torque specifications.

As described earlier, the human operator receives information on the proximity to system constraints through kinesthetic and vibrotactile haptic feedback. In this work, we provide the additional vibrotactile cues through a haptic armband [START_REF] Aggravi | Design and Evaluation of a Wearable Haptic Device for Skin Stretch, Pressure, and Vibrotactile Stimuli[END_REF], which is composed of four Precision Microdrives 307-100 Pico Vibe 9mm vibration motors, an Arduino Mini Pro 3.3 V, a 3.7 V LiPo battery, and a RN-42 Bluetooth 2.1 antenna. The electronics and battery are embedded into a 3D-printed case, and the same is done for each motor. The devices have the form of an elastic wristband with a VELCRO strap. When worn, the motors are positioned evenly around the arm, at 90 degrees from each other. All the motors are driven in the same way, and the magnitude of the commanded vibrations grows with the norm of τ c ,

τ v = ν v ||τ c ||, (10)
where ν v is a positive gain chosen to fit the vibration range of the bracelet.

Haptic guidance toward suitable grasping poses

In addition to providing information about the proximity to dangerous configurations of the robot, we also provide haptic guidance toward suitable grasping poses. Whenever the robotic gripper comes close to an object, the data generated by the grasping algorithm is used to assist the operator in approaching a suitable grasp candidate. The usual approach when handling 6-DoF multi-target scenarios consists in implementing a virtual spring between the current pose of the robot and the different target poses [START_REF] Selvaggio | Enhancing bilateral teleoperation using camerabased online virtual fixtures generation[END_REF], [START_REF] Raiola | Co-manipulation with a library of virtual guiding fixtures[END_REF], [START_REF] Li | Recognition of operator motions for real-time assistance using virtual fixtures[END_REF], [START_REF] Aarno | Adaptive virtual fixtures for machine-assisted teleoperation tasks[END_REF], [START_REF] Masone | Interactive Planning of Persistent Trajectories for Human-Assisted Navigation of Mobile Robots[END_REF].

In this case, the operator receives a force which increases as the gripper moves away from the target pose and decreases as it approaches it. This technique has two downsides: (i) the user can be stuck between two neighbouring poses, and (ii) the user can be distracted by forces pushing her/him toward other targets. As described in Sec. 2, to resolve these issues, researchers have often used probabilistic approaches or simple switching mechanisms. We get inspiration from previous work on 2D digital screens (see Sec. 2) to propose a different approach, designed to always guarantee a continuous and smooth behavior as the user switches between grasp candidates without relying on probabilistic models. To this end, we consider a force profile that increases as the user gets closer to the target pose. This approach, akin to a "magnetic" behavior, ensures that the direction of the guiding force always points toward the closest suitable pose. Moreover, it also enables to consider all the target grasping poses at once. Each of them will contribute to the force feedback according to their distance from the current pose of the gripper: near poses will exert a stronger influence in the force feedback, while far poses will exert a feebler influence. As we detail below and as it is shown in Fig. 6, this combined approach ensures the continuity and smoothness of the received haptic feedback.

Guidance for one grasping pose.

The proposed haptic guidance consists of a 6-DoF force feedback, divided into linear and angular parts. Each grasping pose contributes to the overall resulting force according to its linear and angular distance from the current pose of the gripper. For the linear force contributions, let

∆ i,p = p i -p s ||p i -p s ||
denote the normalized translation from the current gripper position p s to a given grasping position p i in the world frame. The contribution of this grasping position to the linear part of the guiding force should be directed along ∆ i,p , with a positive magnitude that monotonically approaches zero as we increase the distance from the grasping position. We choose a linearly decreasing scaling factor

k p ( s d i ) = 1 - 1 d max s d i
that equals zero when the euclidean distance s d i = ||p ip s || between the gripper position and the grasp position equals a threshold d max . When the euclidean distance s d i is larger than d max , the respective grasp position is ignored by setting its force contribution to zero (see Fig. 6a). On the other hand, in the close vicinity of a target grasping position, we linearly decrease the force contribution back to zero to avoid any abrupt changes in the direction of the force. This is achieved by using the scaling factor

k p,0 ( s d i ) = ( 1 d 0 - 1 d max ) s d i instead of k p ( s d i )
, where d 0 is a small positive distance threshold at which the force contribution has its maximum value. The linear force contribution τ i,p of grasping position i is thus a piecewise continuous linear function given by

τ i,p = ρ p          k p,0 ( s d i ) ∆ i,p if s d i < d 0 k p ( s d i ) ∆ i,p if d 0 < s d i < d max 0 if s d i > d max , (11)
where the constant factor ρ p controls the maximum force. Fig. 6a shows the profile of ||τ i,p || vs. s d i , with threshold values d 0 = 0.005 m and d max = 0.07 m. It is important to recall here that a derivative term is continuously active to assure stability (see ( 2)).

For the angular torque contributions, let s θ i ∈ [-180, 180] and ∆ i,r be the angular and axial parts of the angle-axis representation

( s θ i , ∆ i,r ) of s R i = R T s R i , re- spectively.
Analogously to the linear case, we define scaling factors to regulate the torque contribution as a function of the angular distance between each grasping pose and the gripper, s θ i . The scaling factors are given by

k r ( s θ i ) = 1 - 1 θ max s θ i and k r,0 ( s θ i ) = ( 1 θ 0 - 1 θ max ) s θ i ,
where θ max is the farthest angular distance after which the haptic guidance is activated and θ 0 is the threshold angular distance where the maximum torque is attained.

The angular torque contribution of grasping position i is thus defined as

τ i,θ = ρ θ          k r,0 ( s θ i ) ∆ i,r if s θ i < θ 0 k r ( s θ i ) ∆ i,r if θ 0 < s θ i < θ max 0 if s θ i > θ max , (12)
where ρ θ controls the maximum torque. As above, having θ 0 > 0 prevents any abrupt change in the direction of the force when the gripper is on the target pose. We cannot consider the linear and angular components as two separate and independent contributions to the haptic feedback, because it is important to account for the current gripper orientation ( s θ i , ∆ i,r ) in the generation of the linear force cues τ i,p , and vice-versa. Instead, the rototranslational distance ( s θ i , s d i ) between the gripper and a (c) ||τ i,θ || vs. s d i , s θ i Fig. 6. Linear and angular force profiles for one grasping pose i. This approach considers a force profile that increases as the user gets closer to the target pose, akin to a "magnetic" behavior. Thresholds dmax and θmax indicate the distance after which the haptic guidance is activated; thresholds d 0 > 0 and θ 0 > 0 prevent any abrupt change in the direction of the force when the user is very close to the target pose.

grasp candidate is to be considered altogether. To this end, accounting for [START_REF] Abbink | Haptic shared control: smoothly shifting control authority?[END_REF] in [START_REF] Boessenkool | A task-specific analysis of the benefit of haptic shared control during telemanipulation[END_REF], we get

τ i,p = ρ p                              k p,0 ( s d i ) ∆ i,p if s d i < d 0 & s θ i < θ 0 k p,0 ( s d i )k r ( s θ i ) ∆ i,p if s d i < d 0 & θ 0 < s θ i < θ max k p ( s d i ) ∆ i,p if d 0 < s d i < d max & s θ i < θ 0 k p ( s d i )k r ( s θ i ) ∆ i,p if d 0 < s d i < d max & θ 0 < s θ i < θ max 0 if s d i > d max || s θ i > θ max (13)
A similar approach can be used to generate the guiding torques τ i,θ accounting for (11) in ( 12)

τ i,θ = ρ θ                              k r,0 ( s θ i ) ∆ i,r if s θ i < θ 0 & s d i < d 0 k r,0 ( s θ i )k p ( s d i ) ∆ i,r if s θ i < θ 0 & d 0 < s d i < d max k r ( s θ i ) ∆ i,r if θ 0 < s θ i < θ max & s d i < d 0 k r ( s θ i )k p ( s d i ) ∆ i,r if θ 0 < s θ i < θ max & d 0 < s d i < d max 0 if θ i > s θ max || s d i > d max ( 14
) Fig. 6b and Fig. 6c show the behavior of τ i,p and τ i,θ as a function of s d i and s θ i , with threshold values d 0 = 0.005 m, d max = 0.07 m, θ 0 = 5 deg and θ max = 170 deg. If the gripper is far away from any target pose, i.e., s d i > d max or s θ i > θ max , the operator does not receive any force feedback. Then, as the gripper is driven closer to a grasp candidate, both translation and orientation feedbacks increase. Finally, to avoid any abrupt change in the direction of the force, the feedback goes back to zero when the gripper is at the target pose. The choice of threshold values d 0 , d max , θ 0 and θ max is system-dependent and rather delicate, as it must ensure a smooth and safe force feedback. For example, choosing d 0 = θ 0 = 0 would result in keeping the magnetic effect active until the gripper is exactly on the target pose (see Fig. 6a). This behavior may lead to abrupt changes in the direction of the force when s d i is close to zero (and, therefore, p i -p s may change direction very fast). To choose the right parameters for our system and target application, we asked 2 expert operators to repeatedly carry out a pickand-place task, changing the abovementioned parameters at runtime to make the teleoperation as intuitive, safe, and comfortable as possible. Finally, we asked them to find a consensus on the parameters' values and we used those in our implementation (see Sec. 4 for details).

Guidance for multiple grasping poses. Equations ( 13) and ( 14) describe the linear and angular components of our guiding feedback for a generic grasping pose x i , respectively. However, as mentioned before, an interesting feature of our approach is that we can consider all the grasping candidates at once. A straightforward way to calculate the total haptic guidance feedback is

τ g = [τ T g,p , τ T g,θ ] T = i τ i , (15)
where τ i = [τ T i,p , τ T i,θ ] T represents the force and torque cues associated with grasp candidate x i , as defined in ( 13), [START_REF] Hong | A haptic guidance system for computer-assisted surgical training using virtual fixtures[END_REF].

As an example, Fig. 7 shows the behavior of the linear part of the guidance, as defined in [START_REF] Mulder | Sharing control with haptics: Seamless driver support from manual to automatic control[END_REF], when the gripper moves between two grasp candidates. For simplicity, we assume that the gripper x s and the grasp candidates x 1 , x 2 are all placed along the x axis, with p 1 = [0, 0, 0] T m, p 2 = [0.03, 0, 0] T m, and p s moving between [-0.1, 0, 0] T m and [0.1, 0, 0] T m (see Fig. 7a). As the gripper moves from [-0.1, 0, 0] T m to [0.1, 0, 0] T m, the linear haptic cues τ g,p guide the user first toward x 1 and then toward x 2 , as expected. However, since [START_REF] Mulder | Sharing control with haptics: Seamless driver support from manual to automatic control[END_REF] sums up all the poses contributions, the haptic guidance around x 1 and x 2 will not go to zero exactly at the grasping poses (see red dots in Fig. 7b). In fact, in both cases, the haptic guidance is slightly shifted toward the other grasp candidate, as the attraction force τ 2,p toward grasp candidate x 2 is active also in the vicinity of x 1 , and viceversa. This behavior happens only when grasping poses are closer than d 0 (see Fig. 7b).

To avoid this undesired behavior, whenever the gripper gets very close to a target pose, we progressively fade out the contributions of the other grasp candidates. To this end, we define the rototranslational distance between any grasp candidate x i and the gripper pose x s as

|x i -x s | = s d i + µ s θ i , (16)
where µ > 0 is used to properly scale the angular component of the distance with respect to the linear one. We also denote by x d the grasp candidate closest to the gripper pose x s such that

|x d -x s | = min i (|x i -x s |). (17)
Finally, τ g in ( 15) is adjusted to account for the case where the gripper gets very close to the target pose i.e., |x d -x s | < d 0 + µθ 0 , such that

τ g =            τ d + i =d |x d -x s | d 0 + µθ 0 τ i if |x d -x s | < d 0 + µθ 0 i τ i otherwise , ( 18
) where τ d = [τ T d,p , τ T d,θ
] T is the haptic guidance due to the closest grasping pose x d .

Fig. 7c shows the refined behavior of the linear part of the guidance, as defined in [START_REF] Bowyer | Active constraints/virtual fixtures: A survey[END_REF]. Now, as the gripper moves to a grasp candidate, haptic cues are exactly centered on that pose.

EXPERIMENTAL EVALUATION

To evaluate the effectiveness and viability of our sharedcontrol approach in sorting different types of objects (see Sec. 2.2), we carried out two pick and place experiments, enrolling fifteen human subjects. In the first experiment, we consider only one simple object to grasp, comparing the proposed shared-control approach vs. standard teleoperation and a state-of-the-art shared-control technique [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF]. As described in details in Sec. 4.3, in this first experiment, we chose to consider only one simple object because of the limitations of [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF], which is only able to grasp objects of regular shape. In the second experiment, we considered multiple objects of different shapes and dimensions, comparing the proposed shared-control approach vs. standard teleoperation only. The first experiment enabled us to compare the proposed approach with both standard teleoperation and a state-of-the-art shared-control technique, while the second experiment enabled us to evaluate the proposed approach in a more complex and realistic scenario. These tasks have been chosen following a discussion within the RoMaNS project, which considered them as good representatives of sort and segregation of nuclear waste.

Experimental setup and task

The experimental setup is shown in Fig. 1, and it is described at the beginning of Sec. 3. Depending on the experiment, the remote environment is composed of either one or four objects placed on a table in front of the manipulator: one cardboard container (filled with foam) (14×4×4 cm, 80 g, left object in Fig. 3), one wooden cube (4×4×4 cm, 20 g, top object in Fig. 3), and two L-shaped wood pieces (one composed of two 14×1.2×6.5 cm rectangles, 125 g, center object in Fig. 3; and one made of 10×2.6×4 cm and 11×2.6×9 cm rectangles, 280 g, right object in Fig. 3). The selection of these objects has followed an internal discussion with expert operators. Objects with similar cubic and parallelepipedlike shapes have been often used in telemanipulation [START_REF] Abi-Farraj | User evaluation of a haptic-enabled shared-control approach for robotic telemanipulation[END_REF], [START_REF] Averta | Incrementality and hierarchies in the enrollment of multiple synergies for grasp planning[END_REF], [START_REF] Pacchierotti | Cutaneous haptic feedback in robotic teleoperation[END_REF]. A bin for dropping the grasped items is placed on the right hand side of the manipulator. To enable the (c) Linear forces when fading out farther contributions (see eq. ( 18)). operator to see the environment, the master interface is placed two meters in front of the slave robot. Since the workspace of the master interface is smaller than that of the slave robot, we used a button on the master interface as a clutch. Whenever the button is pressed, the motions of the master and slave systems are decoupled (both rotations and translations). This approach allows the user to start the motion at an arbitrary position, then pause, move to a more comfortable or suitable position, and then resume control of the robot [START_REF] Abi-Farraj | User evaluation of a haptic-enabled shared-control approach for robotic telemanipulation[END_REF], [START_REF] Bimbo | Teleoperation in cluttered environments using wearable haptic feedback[END_REF]. Of course, clutching can be avoided by acting on the scaling of the robot motion with respect to the master interface. However, in the following experiments, no motion scaling is used.

We asked the human subjects to use the master interface to control the motion of the slave manipulator. The task consisted in grasping the object(s) placed in front of the robot and placing them into the bin.

Representative repetition of the sorting task

Before starting with our human subject experiments, we carried out a representative repetition of the sorting task, employing all the four objects. Fig. 8 shows the results of this preliminary run. Fig. 8a shows the roto-translational distance |x m -x s | between the current gripper pose x s and the closest grasp candidate x m vs. time. Fig. 8b and Fig. 8c show the linear and angular guiding cues provided to the operator vs. time, respectively. As expected, the haptic feedback increases as the gripper approaches a potential grasp candidate (i.e., when d 0 < s d i < d max ). Moreover, to avoid any abrupt change in the direction of the force, whenever the gripper is very close to the target pose (i.e., s d i < d 0 ), the force slowly diminishes (see Sec. 3.2.3 and Fig. 6). This behavior leads to a convex U shape for both the linear and angular force graphs as |x m -x s | approaches zero. An exception to this 'U-shaped' behavior is at t = 41 s, where the linear cues τ p look different. In fact, in this case, the user was not entirely satisfied by the grasp pose proposed by the architecture, and therefore he adjusted it to one which he redeemed more convenient. This behavior is desirable, as we want to guide the human user, but also leave him free to make the final decision. It is also interesting to notice that the operator received linear haptic feedback at t = [73, 77] s and t = [90, 95] s (denoted in red rectangles), although the gripper was not in the proximity of any grasp candidate (see Figs. 8a and8b). This behavior can be explained by the sudden increase of the cost function H, shown in Fig. 8d, meaning that the robot approached one of the system constraints (e.g., joint limits).

Experiment #1: pick and place of one object

Goal: compare the proposed shared-control approach vs. standard teleoperation and a state-of-the-art shared-control technique in an environment composed of one regularshaped object.

Experimental setup and feedback conditions

We consider the robotic system described in Sec. 4.1 and shown in Fig. 1. For this experiment, the remote environment is composed of only one object, i.e., the wooden piece shown on the right hand side of Fig. 1. Participants were required to control the motion of the robotic manipulator and gripper to grasp the object, lift it from the ground, and place it in the bin. They were asked to complete the pick-and-place task as fast as possible. The task started when the manipulator moved for the very first time and it was considered successfully completed when the object was released in the bin.

We consider three different ways of commanding the motion of the robot through the haptic interface: T: classic teleoperation, where the subject controls all the 6 DoF of the manipulator and receives no haptic guidance about suitable grasping poses. SF: a standard constrained shared-control approach [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF], where the subject controls only 4 DoF of the manipulator and the remaining 2 DoF are controlled by an autonomous controller. SH:

our proposed haptic shared control approach, where the subject controls all the 6 DoF of the manipulator and receives haptic guidance about suitable grasping poses and proximity to possibly unsafe configurations of the system.

Each subject carried out six randomized repetitions of the grasping task, two for each experimental condition. A video showing trials in all experimental conditions is available as supplemental material and at https://youtu. be/Bb4M3UjwAGY. 4.3.1.1 Condition T: In this condition, the subject is in full control of the manipulator's 6 DoF. The master/slave coupling is the same as that described in Sec. 3.2.1. However, the user receives haptic feedback only about the proximity to system's constraints. No haptic guidance toward suitable grasping poses is provided. The term τ g is thus removed from eq. ( 2), which becomes

τ = τ c + Bv m , (19)
where τ c accounts for joint limits and singularities, and B is a damping matrix (see Sec. 3.2). 4.3.1.2 Condition SF: In this condition, we used the standard shared-control approach of [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF]. An autonomous controller controls a subset n = 2 of the gripper DoF, with the objective of assisting the operator in approaching and grasping the target object. The operator then acts in the null space of the autonomous controller, steering the remaining 6 -n DoF. Fig. 9 shows the setup at hand, where two reference frames are considered: O : {Ø o ; x o , y o , z o }, attached to the object to be grasped, and S : {Ø s ; x s , y s , z s }, attached to the gripper of the slave manipulator. In [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF], the model of the object was assumed to be known and its pose was retrieved at run time using the image from the camera attached to a second slave manipulator. However, in this work, both the model of the object and its pose are retrieved using the depth camera after reconstructing the scene (see Sec. 3.1).

Let s p o represent the object position in S and d = s p o the gripper/object distance. This shared-control strategy aims at autonomously keeping the gripper approaching direction z s oriented toward the object. This behavior is achieved by having the autonomous controller steer

p = s p o s p o ∈ S 2 , (20)
to a desired value p d = z s , ensuring the desired alignment.

To this end, let J p be the jacobian linking the variation of p, ṗ, to the velocity of the gripper v s such that

ṗ = J p v s . (21)
J p will then be

J p = - 1 d P p [p]× ∈ R 3×6 , (22)
where P p = Ipp T , and [•] × is the skew-symmetric operator. Finally, the "primary" task, ensuring the gripper orientation toward the object, is defined as

v s = kJ † p (p d -p) . ( 23
)
The operator is then given command over the 4dimensional null space of (23), N = [n 1 . . . n 4 ] ∈ R 6×4 , where n 1 . . . n 4 are a basis spanning N and chosen to be a comprehensible interface for the human operator such that

n 1 = p 0 , n 2 = 0 p , n 3 = -[p] × e y -P p e y /d , n 4 = [p] × e x P p e x /d , (24)
where e x = [1 0 0] T and e y = [0 1 0] T . This basis is shown in Fig. 10. Therefore, two directions are blocked on the master interface, and the operator is given command over four directions only (see Fig. 10) to command the null space N . The resulting full control law is

u A = kJ † p (p d -p) + N Λ, (25)
Fig. 9. System architecture and main reference frames attached to the gripper and the object to be grasped. Fig. 10. Motion directions controlled by the human operator in [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF] projected on the slave and the master sides.

where Λ = [λ 1 ...λ 4 ] ∈ R 4 are the operator's velocity commands. Moreover, similarly to what is described in Sec. 3.2.2, the operator receives haptic cues informing him about the proximity to system's constraints. More details on this shared-control approach can be found in [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF]. 4.3.1.3 Condition SH: In this condition, the subject is in full control of the manipulator's 6 DoF. He receives haptic guidance regarding suitable grasping poses and proximity to possibly unsafe configurations of the system. This condition is our proposed approach, and it is described in details in Sec. 3.

Participants

Fifteen right-handed subjects (average age 26.4) participated in the study. Five of them had previous experience with haptic interfaces. None of the participants reported any deficiencies in their visual or haptic perception abilities. The experimenter explained the procedures and spent about two minutes adjusting the setup to be comfortable before the subject began the experiment. Each subject then spent about three minutes practicing the control of the telemanipulation system before starting the experiment.

Results

To evaluate the effectiveness of our system in grasping the considered object, the usefulness of the proposed sharedcontrol approach, and the effectiveness of haptic stimuli in such a task, we recorded (i) the completion time, (ii) the linear trajectory followed by the robotic end-effector, and (iii) the angular motion of the robotic end-effector. Moreover, immediately after the experiment, subjects were also asked to report the effectiveness of each feedback condition in completing the given task using bipolar Likert-type elevenpoint scales. To compare the different metrics, we ran oneway repeated-measures ANOVA tests on the data. The control modality (standard teleoperation vs. constrained shared control of [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF] vs. our haptic-enabled shared control, T vs. SF vs. SH) was the within-subject factors. All data passed the Shapiro-Wilk normality test. Figure 11a shows the average task completion time. Data passed the Mauchly's Test of Sphericity. The one-way ANOVA test revealed a statistically significant change in the task completion time across the conditions (F(2, 28) = 7.183, p = 0.003, a = 0.05). Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between T vs. SF (p = 0.027) and T vs. SH (p = 0.042). The Bonferroni correction is used to reduce the chances of obtaining false-positive results when multiple pair-wise tests are performed on a single set of data. Figure 11b shows the linear motion covered by the robotic gripper during the task. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated (χ 2 (2) = 17.415, p < 0.001, a = 0.05). The oneway repeated-measure ANOVA with a Greenhouse-Geisser correction revealed a statistically significant change in the linear motion across the conditions (F(1.151, 16.110) = 8.319, p = 0.001). Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between T vs. SF (p = 0.032) and T vs. SH (p = 0.031). Figure 11c shows the average angular motion covered by the robotic gripper during the task. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated (χ 2 (2) = 6.306, p = 0.043). The one-way repeated-measure ANOVA with a Greenhouse-Geisser correction found no statistically significant change in the angular motion across the conditions. Fig. 11d shows the perceived effectiveness for the three experimental conditions. A Friedman test showed

Gripper

Sphere of motion

Object to grasp

Center of mass Fig. 12. The standard constrained shared-control approach of [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF] (SF) is not suitable for grasping objects of irregular shapes, as the gripper is always oriented toward the center of mass. For example, using the SF technique, it is not possible to grasp the L-shaped wood pieces considered in Experiment #2. For this reason, we did not include SF in our second experimental evaluation.

a statistically significant difference between the means of the four feedback conditions (χ 2 (2) = 6.536, p = 0.038, a = 0.05). The Friedman test is the non-parametric equivalent of the more popular repeated-measures ANOVA. The latter is not appropriate here since the dependent variable was measured at the ordinal level. Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between T vs. SH (p = 0.041). Finally, thirteen subjects out of fifteen found the two shared-control conditions to be the most effective at completing the grasping task: nine preferred condition SH while four preferred SF.

Summary of results: SH outperforms T in all metrics but angular trajectory; SF outperforms T in completion time and linear trajectory; SH was the most preferred condition.

Experiment #2: pick and place in a cluttered scenario

Goal: compare the proposed shared-control approach vs. standard teleoperation in a more complex environment composed of four objects having different shapes.

Experimental setup and feedback conditions

This experiment considers the same robotic system as in Sec. 4.3. The remote environment is now composed of the four objects described in Sec. 4.1 and shown in Fig. 3. Participants were asked to control the motion of the robotic manipulator and gripper to grasp the four objects, one by one, lift them from the ground, and place them in the bin, acting as fast as possible. The task started when the manipulator moved for the very first time, and it was considered successfully completed when the last object was released in the bin.

Since the shared-control approach of [START_REF] Abi-Farraj | A visualbased shared control architecture for remote telemanipulation[END_REF] (SF) can only consider one object at a time, it is not suitable for this second experiment. Moreover, as the SF approach orients the gripper toward the object's center of mass, it is also not suitable for grasping objects having an irregular shape (see Fig. 12), such as those considered here. For this reason, this experiment considers only two ways of commanding Each subject carried out four randomized repetitions of the grasping task, two for each experimental condition. A video showing trials in all experimental conditions is available as supplemental material and at https://youtu. be/Bb4M3UjwAGY?t=1m49s.

Participants

The same fifteen subjects who participated in the first experiment also participated in this second experiment.

Results

We considered the same metrics as in Sec. 4.3. Since here we only consider two conditions (T vs. SH), we ran pairedsamples t-test to compare the metrics. All data passed the Shapiro-Wilk normality test. Figure 13a shows the average task completion time. The paired-samples t-test revealed a statistically significant change in the task completion time across the conditions (T vs. SH, t(14) = 3.176, p = 0.007, a =0.05). Figure 13b shows the linear motion covered by the robotic gripper during the task. The paired-samples ttest revealed a statistically significant change in the linear motion across the conditions (T vs. SH, t(14) = 2.464, p = 0.027, a = 0.05). Figure 13c shows the angular motion covered by the robotic gripper during the task. The paired-samples t-test did not reveal a statistically significant change in the linear motion across the conditions. As before, immediately after the experiment, subjects were also asked to report the effectiveness of each feedback condition in completing the given task using bipolar Likert-type elevenpoint scales. Fig. 13d shows the perceived effectiveness for the three experimental conditions. A related-samples Wilcoxon signed rank test showed a statistically significant difference between the means of the feedback conditions (Z = 2.887, p = 0.004, a = 0.05). Moreover, thirteen subjects out of fifteen found condition SH to be the most effective at completing the grasping task. Finally, eleven subjects out of fifteen found the action of the vibrating bracelet useful (average score 7.2±2.4 out of 10).

Summary of results: SH outperforms T in all metrics but angular trajectory; SH was the most preferred condition; the vibrotactile feedback was deemed useful.

DISCUSSION

To demonstrate the feasibility and effectiveness of our shared-control approach, we conducted two remote telemanipulation experiments enrolling fifteen human subjects. The first experiment consisted in picking up one object and placing it inside a bin. We tested the performance of the proposed shared-control system (SH) against a more classic teleoperation approach (T), in which the user is able to freely control all the DoF of the manipulator, and a standard constrained shared-control approach (SF), where the control of the robot's DoF are strictly divided between the human and an autonomous algorithm. As a measure of performance, we considered the average completion time, trajectory length, angular motion, and perceived effectiveness. Results showed that, in all the considered metrics but one (angular trajectory), our proposed shared-control approach significantly outperformed the more classic teleoperation architecture. Moreover, all subjects preferred one of the two shared-control architectures with respect to teleoperation. However, in this first experiment, we did not find any significant difference between the two shared-control approaches (SH vs. SF). This result means that, if we are dealing with only one simple object, our proposed approach may not significantly improve the task performance with respect to previous methods. However, SH can consider multiple objects at once, something which is not possible in SF, where a particular object must be chosen beforehand. This gives the operator the flexibility to judge, on the fly, the sequence of objects to be picked. Moreover, SH guides the operator toward feasible grasp poses, while SF guides only toward the object's center of mass. This behavior prevents SF from grasping objects having an irregular shape (see Fig. 12), which is often the case in our sort and segregation scenario.

In the second experiment, we considered the same experimental setup and task as before but with four different objects to move. We tested the performance of our sharedcontrol system (SH) only against classic teleoperation (T) (omitting (SF) due to the limitations described earlier). As a measure of performance, we considered again the average completion time, trajectory length, angular motion, and perceived effectiveness. In all the considered metrics but angular motion, SH outperformed T. Moreover, all subjects but two found our shared-control approach more effective than classic teleoperation.

These results are in accordance with the literature. For example, Vogel et al. [START_REF] Vogel | Flexible, semiautonomous grasping for assistive robotics[END_REF] compared a shared-control scheme for grasping using virtual fixtures vs. standard teleoperation. Participants were asked to grasp different objects in a simulated environment using a SpaceMouse interface. Completion time and total trajectory decreased by approximately 23% and 31%, respectively, when using the shared-control approach. With respect to our work, the gripper orientation is fixed and autonomously controlled based on the selected grasp pose. Raiola et al. [START_REF] Raiola | Co-manipulation with a library of virtual guiding fixtures[END_REF] presented a set of virtual fixtures which they demonstrated in a pick-and-place industrial task. Completion time decreased by approximately 15% compared to standard teleoperation. With respect to our work, Raiola et al. used a probabilistic approach to predict user's intention and dynamically select the most suitable virtual guide.

It is important to also notice that none of our subjects was experienced in using the experimental setup. In fact, the recorded significant difference between shared control vs. teleoperation might change in the presence of experienced users. This is something we plan to extensively study in the future, since all the operators in our target scenario at the Sellafield nuclear site are skilled and experienced. Moreover, we want to highlight that our approach is totally independent from the grasping algorithm. Although here we used the GPD algorithm to generate the grasp poses, our approach is expected to work on top of any other similar technique. For this reason, as long as the considered grasping algorithm is able to generate feasible grasp poses, our shared-control approach is expected to work on objects of any shape.

For all our experiments we used clutching to enable the operator to move the slave robot within a larger workspace than the master device. Given a grounded haptic interface and a grounded slave manipulator, it is always possible to define an appropriate scaling factor between master and slave velocities such that the operator does not need clutching. However, as the difference between the master and slave workspaces increases, this mapping requires higher gains, resulting in a telemanipulation system which is very hard to control, since the operator's accuracy in positioning the slave arm is degraded. For example, the custom rig at the National Nuclear Laboratory (UK) is composed of (i) the same grounded haptic interface we are using in this paper, and (ii) a 500-kg-payload Kuka KR 500 manipulator. Although it is theoretically possible to map the workspace of the Virtuose 6D to the (much) larger workspace of the KUKA robot, this would result in very high motion gains (i.e., a small movement of the master interface would cause a big movement of the slave robot). For this reason, we decided to use the clutch in our implementation. No subject reported any confusion in using this control of the robot. However, if the user clutches a lot, he or she may steer the master interface in a configuration very different than that of the slave robot. This could eventually lead to confusion. In this respect, we are also interested in understanding how to best tune the master-slave motion scaling factor, with the final objective of finding a good trade-off between high precision of movement, low need of clutching, and intuitiveness. Finally, since we are using kinesthetic feedback, we are somehow limiting the control capabilities of the human user, who is not able to freely move the robot wherever they prefer. For example, a high kinesthetic force would prevent a user to go toward the workspace limits, even if he wants to. To address this point, we are studying new ways of providing guidance information to the operators using only ungrounded stimuli, with the objective of pro-viding them with information about what the autonomous controller thinks they should do, but without reducing their capabilities to control the motion of the robot. A possible approach is to employ a wearable device instead of the grounded Virtuose interface.

CONCLUSIONS AND FUTURE WORK

This paper presents a novel haptic-enabled shared-control architecture, whose objective is to ease the manipulation of objects in cluttered environments. In particular, it tackles the multi-target haptic guidance problem for the six dimensional case. It proposes a solution which ensures a smooth and continuous haptic feedback without relying on probabilistic estimates or intention recognition models.

We have been investigating this problem due to our involvement in a European collaborative project, whose objective is to find more intuitive and effective ways to manage nuclear waste. Specifically, we are working to improve the effectiveness of robotic teleoperation in the sort and segregation of nuclear waste at the Sellafield site (UK). A faster and more efficient sort and segregation of the stored waste is in fact the first step toward the decommissioning of the site. To this end, we considered a human operator using a master haptic interface to control a robotic arm equipped with a gripper and a depth camera. A point cloud scan of the environment is used to retrieve potential grasp poses of the objects in the scene. These poses are then translated into kinesthetic and vibrotactile haptic cues, guiding the user towards suitable grasps as well as keeping him away from system constraints.

The paper details the theoretical methods of this approach, presents an implementation for a 6-DoF robotic teleoperation system, and a principled human subject evaluation in a real environment. Results showed that the proposed shared-control technique outperformed standard teleoperation in several recorded metrics showing, in particular, a 20% decrease in the task completion time. Moreover, it showed no significant difference with respect to a standard constrained shared-control technique. However, the newly proposed technique outperforms its predecessor in its applicability to more general scenarios and a wider range of objects.

In the future, we are planning to study how the strength of haptic guidance affects the task performance, e.g., a system could use a stiff guidance approach (i.e., less freedom for the operator) when it is operated by novices, while it could implement a soft guidance approach (i.e., more freedom for the operator) when it is operated by experts. This flexible approach could be also useful when teaching new operators, employing different levels of autonomy according to the operator's experience. Along this line of research, we will also study the role of the threshold values in the system's performance and how the strength of haptic guidance affects their choice. We will also investigate the effects of employing different cost functions and/or control gains on task performance. To this end, we plan to carry out more human subject experiments in real scenarios. Finally, we also intend to quantify the impact of cutaneous haptic feedback on the overall performance of the users as well to test different types of ungrounded haptic stimuli [START_REF] Chinello | Design and evaluation of a wearable skin stretch device for haptic guidance[END_REF].

Fig. 1 .

 1 Fig. 1. The experimental setup showing the slave robotic arm on the top and the master haptic arm on the bottom.

Fig. 2 .

 2 Fig. 2. A schematic illustration of the proposed architecture. The operator uses the master interface to control the slave robot and receives haptic feedback based on the grasp guidance and system constraints. Additionally, the visual interface shows grasp candidates produced by the grasp pose detection (GPD) algorithm.

Fig. 3 .Fig. 4 .

 34 Fig. 3. A picture of the objects used in the experiment.

Fig. 5 .

 5 Fig. 5. The potentials associated with the joint limits and singularities of the manipulator.

  ||τ i,p || vs. s d i ||τ i,p || vs. s d i , s θ i

s

  (a) The gripper xs moves first toward x 1 and then toward x 2 . Linear forces when summing up all the contributions (see eq. (15)).

Fig. 7 .

 7 Fig. 7. The figures show the linear haptic cues guiding the user toward grasping poses x 1 and x 2 as a function of the position of the gripper in a mono-directional case. They highlight the difference between a simple summation of the forces of attraction to all grasping poses (Fig. (b)) and the adopted solution, which centers the attraction force on the closest grasp candidate (Fig. (c)).

Fig. 8 .

 8 Fig. 8. Sample experiment for picking and placing several objects in a cluttered scene. (a) Roto-translational distance between the gripper and the closest grasp candidate, (b) linear force received by the operator, (c) angular torques received by the operator, and (d) the evolution of the cost function describing the constraints.

  Angular trajectory (deg).

Fig. 11 .

 11 Fig. 11. Experiment #1. Mean and 95% confidence interval of (a) completion time, (b) linear trajectory length, (c) angular motion, and (d) perceived effectiveness of the three feedback conditions are plotted.

  Angular trajectory (deg).

Fig. 13 .

 13 Fig. 13. Experiment #2. Mean and 95% confidence interval of (a) completion time, (b) linear trajectory length, (c) angular motion, and (d) perceived effectiveness of the three feedback conditions are plotted.

O. Arenz and G. Neumann are with the Lincoln Center for Autonomous Systems, University of Lincoln -Lincoln, LN6 7TS, United Kingdom. oleg@robot-learning.de, GNeumann@lincoln.ac.uk

O. Arenz is also with the Computational Learning for Autonomous Systems, Technische Universität Darmstadt -64289 Darmstadt, Germany. • This research has received funding from Rennes Métropole, under the "Allocation d'Installation Scientifique (AIS) 2017" programme; from the European Commission Horizon 2020 Research and Innovation Programme under Grant Agreement No 645582 of project "RoMaNS"; and from the CNRS, under the 2018 "Projet Exploratoire Premier Soutien -Jeunes chercheuses et jeunes chercheurs" programme.

Although users already expressed an appreciation for the received vibrotactile information, an objective performance evaluation is necessary. The same need exists for the visual interface used to assist the user in the task.

Firas Abi-Farraj is a post-doctoral researcher of the Centre national de la recherche scientifique (CNRS) at Irisa and Inria, Rennes, France. He received his PhD degree from the University of Rennes 1 in 2018. He spent 6 months of 2018 visiting the Institute of Robotics and Mechatronics which is part of the German Aerospace Center (DLR) in Munich, Germany. His research currently focuses on Human-Robot Interaction (HRI) and more particularly on developing intuitive interfaces to help human operators efficiently command complex robotics systems (ex. dual arm systems and humanoid robots). To this end, he employs software and hardware components from various domains of robotics including haptics, force control, humanoid robotics, visual servoing, reactive trajectory planning, and manipulation.