Claudio Pacchierotti

Federico Ongaro

Frank Van Den Brink

Changkyu Yoon

Domenico Prattichizzo

David H Gracias

Sarthak Misra

Steering and Control of Miniaturized Untethered Soft Magnetic Grippers With Haptic Assistance

Keywords: microrobotics, haptics, soft robotics, magnetic control, micro teleoperation, grasping

published or not. The documents may come   L'archive ouverte pluridisciplinaire

Steering and control of miniaturized untethered soft magnetic grippers with haptic assistance C. Pacchierotti, F. Ongaro, F. van den Brink, C. Yoon, D. Prattichizzo, D. H. Gracias, S. Misra Abstract-Untethered miniature robotics have recently shown promising results in several scenarios at the microscale, such as targeted drug delivery, microassembly, and biopsy procedures. However, the vast majority of these small-scale robots have very limited manipulation capabilities, and none of the steering systems currently available enable humans to intuitively and effectively control dexterous miniaturized robots in a remote environment. In this paper, we present an innovative micro teleoperation system with haptic assistance for the intuitive steering and control of miniaturized self-folding soft magnetic grippers in 2dimensional space. The soft grippers can be wirelessly positioned using weak magnetic fields and opened/closed by changing their temperature. An image-guided algorithm tracks the position of the controlled miniaturized gripper in the remote environment. A haptic interface provides the human operator with compelling haptic sensations about the interaction between the gripper and the environment, as well as enabling the operator to intuitively control the target position and grasping configuration of the gripper. Finally, magnetic and thermal control systems regulate the position and grasping configuration of the gripper. The viability of the proposed approach is demonstrated through two experiments involving twenty-six human subjects. Providing haptic stimuli elicited statistically significant improvements in the performance of the considered navigation and micromanipulation tasks.

Note to Practitioners-The ability to accurately and intuitively control the motion of miniaturized grippers in remote environments can open new exciting possibilities in the fields of minimally-invasive surgery, micromanipulation, biopsy, and drug delivery. This article presents a micro teleoperation system with haptic assistance through which a clinician can easily control the motion and open/close capability of miniaturized wireless soft grippers. It introduces the underlying autonomous magnetic and thermal control systems, their interconnection with the master haptic interface, and an extensive evaluation in two real-world

I. INTRODUCTION

T HE field of untethered miniature robotics has advanced significantly with several demonstrations of robotic tasks, such as controlled micropositioning [START_REF] Solovev | Catalytic microtubular jet engines self-propelled by accumulated gas bubbles[END_REF], [START_REF] Woods | Towards a micropositioning system for targeted drug delivery in wireless capsule endoscopy[END_REF], [START_REF] Marino | Robust electromagnetic control of microrobots under force and localization uncertainties[END_REF], pickup and delivery of micro-objects, cells, and molecules [START_REF] Rakotondrabe | Presentation, force estimation and control of an instrumented platform dedicated to automated micromanipulation tasks[END_REF], [START_REF] Balasubramanian | Micromachine-enabled capture and isolation of cancer cells in complex media[END_REF], [START_REF] Kim | Evaluation of telerobotic shared control strategy for efficient single-cell manipulation[END_REF], [START_REF] Zhang | Targeted cargo delivery using a rotating nickel nanowire[END_REF], biopsy [START_REF] Gultepe | Biopsy with thermally-responsive untethered microtools[END_REF], and drilling into soft tissue [START_REF] Matteucci | A compact and disposable transdermal drug delivery system[END_REF], [START_REF] Xi | Rolled-up magnetic microdrillers: towards remotely controlled minimally invasive surgery[END_REF]. Further, small-scale robots have also been applied in environmental science [START_REF] Guix | Superhydrophobic alkanethiol-coated microsubmarines for effective removal of oil[END_REF], [START_REF] Soler | Self-propelled micromotors for cleaning polluted water[END_REF], sensing [START_REF] Kuralay | Self-propelled carbohydrate-sensitive microtransporters with built-in boronic-acid recognition for isolating sugars and cells[END_REF], [START_REF] Orozco | Molecularly imprinted polymer-based catalytic micromotors for selective protein transport[END_REF], and drug delivery [START_REF] Li | Dynamics modeling and analysis of a swimming microrobot for controlled drug delivery[END_REF], [START_REF] Nelson | Microrobots for minimally invasive medicine[END_REF], [START_REF] Malachowski | Stimuli-responsive theragrippers for chemomechanical controlled release[END_REF].

Among the diverse applications presented in the literature, transportation and assembly at the microscale are important for biomedical applications. For example, Sanchez et al. [START_REF] Sanchez | Controlled manipulation of multiple cells using catalytic microbots[END_REF] demonstrated the transport of animal cells using catalytic selfpropelled microrobots, and Martel et al. [START_REF] Martel | Using a swarm of self-propelled natural microrobots in the form of flagellated bacteria to perform complex microassembly tasks[END_REF] completed a microassembly task using a swarm of flagellated bacteria. More recently, Gao et al. [START_REF] Gao | Artificial micromotors in the mouses stomach: A step toward in vivo use of synthetic motors[END_REF] reported an in vivo study of zinc-based artificial micromotors in a living organism using a mouse model. They demonstrated that the acid-driven propulsion in the mouse stomach effectively enhances the binding and retention of the motors as well as of cargo payloads on the stomach wall. Moreover, they reported that the body of the micromotors gradually dissolved in the gastric acid, releasing the carried payload and leaving no toxic residue behind.

Although very promising, prior micromanipulation tasks using untethered small-scale robots have been quite simple, since most of these robots have very limited manipulation capabilities [START_REF] Solovev | Catalytic microtubular jet engines self-propelled by accumulated gas bubbles[END_REF], [START_REF] Gao | Water-driven micromotors[END_REF], [START_REF] Marino | Robust electromagnetic control of microrobots under force and localization uncertainties[END_REF]. In order to expand the possible applications of untethered miniaturized robots and allow them to facilitate more complex (and useful) tasks, researchers have started to focus on the study and development of devices with gripping modalities. Being able to simultaneously grasp, move, and release an object at small scales would expand the applicability of such robots to a multitude of tasks, including minimally invasive surgery and targeted biopsy. It is noteworthy that robotic grippers have already been guided in in vivo applications [START_REF] Platt | Vision and task assistance using modular wireless in vivo surgical robots[END_REF], [START_REF] Harada | A reconfigurable modular robotic endoluminal surgical system: vision and preliminary results[END_REF], [START_REF] Lehman | Dexterous miniature robot for advanced minimally invasive surgery[END_REF], but the grippers used were significantly larger centimeter-sized devices. One of the reasons for the relatively large size of these devices is the need to integrate batteries to power their electromechanical actuators [START_REF] Randhawa | Microchemomechanical systems[END_REF].

As an alternative to battery and antenna powered devices, stimuli responsive materials are expected to play an important role in the creation of untethered self-assembling and selffolding microrobots [START_REF] Randhawa | Microchemomechanical systems[END_REF], [START_REF] Stuart | Emerging applications of stimuli-responsive polymer materials[END_REF]. Self-folding stimuli-responsive (c) Detail of the master system.

Fig. 1. Haptic-enabled micro teleoperation system. The image-guided algorithm tracks the position of the miniaturized soft gripper in the remote environment using a high-resolution camera and a Fourier-descriptors-based algorithm. A 6-DoF grounded haptic interface then provides the human operator with haptic stimuli about the interaction of the gripper with the remote environment. At the same time, it enables the operator to intuitively control the reference position of the gripper. Finally, the magnetic control algorithm steers the gripper toward the reference position defined by the operator, and a Peltier element regulates the temperature of the distilled water where the gripper is floating.

soft robots can in fact regulate the grasping and transport of objects by converting chemical and biochemical signals into mechanical, optical, electrical, and thermal signals, and vice-versa. These types of robots are playing an increasingly important role in a wide range of applications and scenarios, such as drug delivery, diagnostics, tissue engineering, as well as biosensors, microelectromechanical systems, coatings, and textiles [START_REF] Syms | Surface tension-powered self-assembly of microstructures-the state-of-the-art[END_REF], [START_REF] Leong | Tetherless thermobiochemically actuated microgrippers[END_REF]. For example, Gultepe et al. [START_REF] Gultepe | Biopsy with thermally-responsive untethered microtools[END_REF] recently succeeded in performing a biopsy procedure in the gastrointestinal tract of live pigs with unguided, distributed metallic microgrippers. These microgrippers were designed to stay open at cold temperatures, and then close after approximately 10 minutes of exposure to body temperature. The microgrippers were injected into the bile duct or esophagus of the animals using a catheter that was introduced from the mouth. The grippers were able to retrieve sample tissue with a gripper retrieval rate as high as 95%. In order to reduce any risks associated with these devices being left in the body, microgrippers could potentially be fabricated using biodegradable and biodissolvable materials such as polymers and hydrogels. The soft tissue-like mechanical properties of these materials could also facilitate delicate, flexible, and safe grasp capabilities [START_REF] Ilievski | Soft robotics for chemists[END_REF], [START_REF] Breger | Self-folding thermo-magnetically responsive soft microgrippers[END_REF]. Moreover, soft grippers could be more suitable for medical procedures, since the sturdy grasp of rigid grippers may damage, or even destroy, biological cells and tissues. Among these soft materials, we found hydrogels -gels in which the swelling agent is water -very promising. Hydrogels are a class of porous softmaterials composed of a network of polymer chains, which can be biocompatible and/or biodegradable. Moreover, they can operate in aqueous environments and recent advances in soft-lithography, photopatterning, and 3D printing allow them to be structured with microscale resolution.

In this work, we used soft bilayer miniature grippers where a hydrogel film is paired with a rigid non-swelling polymer SU-8. The two layers are photopatterned with UV light to form devices shaped like hands. The hydrogel is then doped with biocompatible iron oxide to allow it to respond to external magnetic fields. As demonstrated by [START_REF] Breger | Self-folding thermo-magnetically responsive soft microgrippers[END_REF] and detailed in Sec. II, this procedure endows the soft grippers with stimuli-responsive, self-folding, and magnetic properties. The temperature-induced differential swelling stress in the bilayer structure enables the grasping configuration of the gripper through temperature control, while the iron oxide allows wireless control of the positioning of the grippers through weak magnetic fields. As we illustrate, the capability to wirelessly control both the spatial position of the soft grippers and their grasping configuration significantly increases the dexterity of these agents, expanding their applicability and effectiveness with respect to other smallsize robots [START_REF] Solovev | Catalytic microtubular jet engines self-propelled by accumulated gas bubbles[END_REF], [START_REF] Gao | Water-driven micromotors[END_REF].

Given their promising features and potential reach, one of the challenges to facilitate complex manipulation tasks is the precise and quick control of these miniature grippers. Moreover, for reasons of responsibility, safety, and public acceptance, it is beneficial to provide a human operator with intuitive means to directly control the motion of such grippers, especially for medical scenarios [START_REF] Troccaz | Semi-active guiding systems in surgery. a two-dof prototype of the passive arm with dynamic constraints (PADyC)[END_REF], [START_REF] Jakopec | The hands-on orthopaedic robot acrobot: Early clinical trials of total knee replacement surgery[END_REF], [START_REF] Pacchierotti | Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback[END_REF]. In this case, the operator needs to receive sufficient information about the status of the gripper and about its interaction with the remote environment. This is possible through different types of feedback information that flow from the remote scenario to the human operator. Haptic feedback is one piece of this information flow, and it has been proven to be a valuable tool in robotic teleoperation [START_REF] Okamura | Methods for haptic feedback in teleoperated robotassisted surgery[END_REF], [START_REF] Quek | Sensory substitution and augmentation using 3-degree-of-freedom skin deformation feedback[END_REF], [START_REF] Pacchierotti | Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems[END_REF], [START_REF] Jarc | Robot-assisted surgery: an emerging platform for human neuroscience research[END_REF]. Its benefits have been proven in cardiothoracic procedures [START_REF] Kennedy | A novel approach to robotic cardiac surgery using haptics and vision[END_REF], microneedle positioning [START_REF] Salcudean | Performance measurement in scaled teleoperation for microsurgery[END_REF], telerobotic catheter insertion [START_REF] Kazi | Operator performance in surgical telemanipulation[END_REF], palpation [START_REF] Pacchierotti | Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery[END_REF], cell injection [START_REF] Pillarisetti | Evaluating the effect of force feedback in cell injection[END_REF], [START_REF] Wang | Autonomous zebrafish embryo injection using a microrobotic system[END_REF], and even micromanipulation [START_REF] Pacchierotti | Haptic feedback for microrobotics applications: a review[END_REF], [START_REF] Kim | Task-based and stable telenanomanipulation in a nanoscale virtual environment[END_REF], [START_REF] Mehrtash | Bilateral macromicro teleoperation using magnetic levitation[END_REF], [START_REF] Bolopion | A review of haptic feedback teleoperation systems for micromanipulation and microassembly[END_REF], [START_REF] Pacchierotti | Intuitive control of self-propelled microjets with haptic feedback[END_REF], [START_REF] Ghanbari | Haptic microrobotic cell injection system[END_REF]. For example, Ghanbari et al. [START_REF] Ghanbari | Haptic microrobotic cell injection system[END_REF] used active constraints to haptically assist the operator in penetrating a cell at the desired location, and Mehrtash et al. [START_REF] Mehrtash | Bilateral macromicro teleoperation using magnetic levitation[END_REF] developed a magnetic micromanipulation platform able to provide a resistive force every time the microrobot encountered a stiff object. More recently, Pacchierotti et al. [START_REF] Pacchierotti | Intuitive control of self-propelled microjets with haptic feedback[END_REF] presented a teleoperation system with haptic feedback that enables a human operator to intuitively control the positioning of a self-propelled microjet in 2D. Providing haptic information significantly improved the performance and the perceived realism of the considered positioning tasks. However, due to the limited manipulation capabilities of microjets, the authors did not carry out any complex manipulation task.

In this study, we present an innovative micro teleoperation system with haptic assistance for steering and control of miniaturized self-folding magnetic soft grippers in 2D space, shown in Fig. 1. It enables a human operator to intuitively and accurately control the motion of a soft gripper in the remote environment while providing him/her with compelling haptic stimuli about the interaction between the soft gripper and the environment.

An image-guided algorithm tracks the position of the controlled gripper using a high-resolution camera and a Fourierdescriptors-based algorithm, as described in Sec. III-A. A 6 degrees-of-freedom (6-DoF) grounded haptic interface then provides the human operator with haptic stimuli about the interaction between the controlled gripper and the remote environment, as well as enabling him/her to intuitively control the reference target position and temperature of the gripper, as described in Sec. III-B. Finally, magnetic and thermal control systems regulate the positioning and temperature of the selected gripper using the magnetic forces generated by six electromagnetic coils and a Peltier element, respectively, as described in Sec. III-C. In order to test the effectiveness of our teleoperation system and to understand the role of haptic assistance for such an application, we carried out two experiments in a real scenario. The first one, described in Sec. IV-A, evaluates the steering capabilities of the proposed teleoperation system for a path following task. The second experiment, described in Sec. IV-B, evaluates the steering capabilities of our system for a pick-and-place task. A video of a soft gripper moving in free space using an Omega 6 interface is available as supplemental material.

A preliminary version of this work has been presented in [START_REF] Ongaro | Evaluation of an electromagnetic system with haptic feedback for control of untethered, soft grippers affected by disturbances[END_REF]. However, while [START_REF] Ongaro | Evaluation of an electromagnetic system with haptic feedback for control of untethered, soft grippers affected by disturbances[END_REF] mainly focused on the magnetic positioning of the soft grippers, this paper addresses in detail the image-based tracking algorithm, the closed-loop haptic and thermal controls, and, most importantly, it introduces an experimental evaluation in two paradigmatic scenarios, involving twenty-six subjects.

II. MINIATURIZED, STIMULI-RESPONSIVE, SELF-FOLDING, SOFT ROBOTIC GRIPPERS

Miniature soft polymeric grippers composed of SU-8 and poly N-isopropylacrylamide-acrylic acid (pNIPAM-AAc) were patterned via a photolithographic approach (see Fig. 2a), which is described in detail in [START_REF] Breger | Self-folding thermo-magnetically responsive soft microgrippers[END_REF]. Briefly, a water soluble polyvinyl alcohol (PVA) sacrificial layer was deposited onto a silicon (Si) substrate by spin coating at 1000 rpm and dried at 150 • C for 1 minute. To achieve rigid and non-swellable polymeric panel structures, SU-8 solution was coated onto the PVA layer by spin coating at 2000 rpm and pre-baking at 70 • C for 1 minute, 115 • C for 3 minutes, and 70 • C for 1 minute. The baked SU-8 layer was then photopatterned using a dark field mask with 180 mJ/cm 2 UV light (365 nm) exposure and then post-baking at 70 • C for 1 minute, 115 • C for 3 minutes, and 70 • C for 1 minute (step I in Fig. 2). Uncrosslinked material was removed by immersing in SU-8 developer for 1 minute and washed with 1 second acetone and 10 seconds isopropyl alcohol (IPA), then dried using compressed air (step II in Fig. 2). Further, to pattern the second thermal-and magnetic-responsive swellable thin hydrogel layer, 0.8 mL pNIPAM-AAc solution mixed with 5% w/w biocompatible iron oxide (Fe 2 O 3 ) nanoparticles and deposited atop of the SU-8 patterns. After waiting 2 minutes to get a uniform thickness, the second continuous pNIPAM-AAc layer was aligned with SU-8 panel layer in non-contact mode using a spacer and then photopatterned using a second dark field mask with 40 mJ/cm 2 UV light exposure (step III in Fig. 2). Unreacted parts were removed by washing with acetone and IPA, and then dried using compressed air (step IV in Fig. 2). Finally, the photopatterned structures were immersed in DI water overnight to completely dissolve the PVA sacrificial layer and release the untethered soft grippers (step V in Fig. 2).

The untethered SU-8/pNIPAM-AAc grippers show anisotropic swelling and deswelling between layers when triggered by a thermal stimulus. Initially, the grippers are in a two dimensional (2D) flat state at room temperature (RT). On heating, the grippers close spontaneously due to significant volumetric shrinkage of the pNIPAM-AAc layer which undergoes a phase transition from a hydrophilic to hydrophobic state above its lower critical solution temperature (LCST). Reversibly, on cooling down, the self-folded grippers open back to their 2D flat state as the pNIPAM-AAc layer swells on reversal of the transition from a hydrophobic to hydrophilic state. This spontaneous and environmental thermal response of the pNIPAM-AAc layer is able to open and close the soft grippers reversibly, as shown in the bottom half of Fig. 2. Further, by incorporating magnetic Fe 2 O 3 nanoparticles, the soft grippers can be manipulated from afar using precisely controlled magnetic fields, as detailed in Sec. III-C.

III. MICRO TELEOPERATION SYSTEM

Figure 1a shows how the tracking, haptic, and control systems are interconnected. Table I summarizes the features and parameters of the system.

A. Tracking system

To precisely track the position of the controlled soft gripper in the remote scenario, we placed a high-resolution camera above the Petri dish hosting the environment, as shown in Fig. 1. The camera is a Blackfly 1.4 MP Color GigE PoE camera (Point Grey Research Inc., Richmond, Canada), mounted on a Mitutoyo FS70 microscope unit (Mitutoyo, Kawasaki, Japan) using a Mitutoyo M Plan Apo 2 / 0.055 objective. It has an adjustable zoom with a maximum of 24X, a frame rate of 125 fps, and it is mounted on a linear stage to enable precise focusing. A CCD sensor is used for recording, with a pixel width and height of 5.50 µm, providing a resolution up to 0.50 µm.

Each frame is first converted to HSV colorspace, then its saturation channel (S) is filtered using a median filtering technique, and finally a binary frame is obtained using an Fig. 3. Tracking algorithm. Each frame registered by the high-resolution camera is first converted to HSV colorspace, then its saturation channel (S) is filtered using a median filter, and a binary frame is obtained using an adaptive threshold. The binary frame is then used to find the contours of the image, and Fourier descriptors are used to detect the centroid of the microgripper and measure how much the microgripper has closed. Finally, a Kalman filter is used to deal with uncertainties in the tracking. The tracking system runs at 50 Hz. adaptive threshold method. The resulting binary frame is used to find the contours of the image, and Fourier descriptors are then employed to detect the centroid of the microgripper and measure its extent of closure. Finally, a Kalman filter is used to deal with uncertainties in the tracking. The flow chart of the tracking algorithm is shown in Fig. 3, and each part of the algorithm is described below in detail.

1) Color space selection: Light changes in the background and glares due to the reflection of the light with the gripper's SU8-layer (see Sec. II) make it very hard to robustly track the grippers in RGB colorspace. The RGB color model is in fact an additive model in which red, green, and blue light are added together, and it is not trivial to separate the intensity information from the color information, making the model quite sensitive to changes in the light condition [START_REF] Süsstrunk | Standard RGB color spaces[END_REF]. To address this problem, we look into the HSV colorspace, which separates the light intensity information from the color information. Since we are only interested in the shape of our gripper, and we want to avoid the aforementioned issues related to the change of light conditions, we disregard the H-and V-channels and we focus our analysis on the S-channel only. The two top pictures in Fig. 3 shows the original RGB image (left) and the HSV image (right) in RGB colorspace, where the H channel is shown in red, the S channel is shown in green, and the V channel is shown in blue.

2) Median filtering: After converting the RGB image to HSV color space and extracting the S-channel, we apply a median filter to reduce noise. The median filter is a non-linear digital filtering technique that replaces each pixel value with the median of n × n neighboring entries, where n is an integer representing the kernel size of the filter [START_REF] Pratt | Image feature extraction[END_REF]. The mid right picture in Fig. 3 shows the S-channel of the HSV image after the median filtering.

3) Adaptive threshold: Due to vignetting, different areas of the image show different light conditions. In order to be able to compare these areas, we employ an adaptive thresholding technique. Once the S-channel of the HSV image is extracted and the noise is reduced, the filtered image frame F f (x, y) is converted to a binary frame F b (x, y) using an adaptive threshold,

F b (x, y) = 255 if F f (x, y) > T (x, y) 0 otherwise, (1)
where T (x, y) is the threshold matrix, defined as the mean of a neighborhood around the considered pixel, weighted using a Gaussian distribution. Pixels equal to zero are then colored as black, while pixels equal to 255 are colored as white. The mid left picture in Fig. 3 shows the image after the abovementioned adaptive thresholding process.

4) Contour detection:

From the binary image, we compute the contours of all the visible blobs using a contour detection algorithm based on the technique presented by Reeb et al. [START_REF] Reeb | Sur les points singuliers dune forme de pfaff completement intégrable ou dune fonction numérique[END_REF] and then further developed by Bajaj et al. [START_REF] Bajaj | The contour spectrum[END_REF] and Carr et al. [START_REF] Carr | Simplifying flexible isosurfaces using local geometric measures[END_REF]. Each blob is described by a sequence of points connected to each other by segments. This sequence is found using the method proposed by Suzuki et al. [START_REF] Suzuki | Topological structural analysis of digitized binary images by border following[END_REF], in which detected objects are represented as trees. In each tree, the root is a sequence describing the outer contour, and the children are sequences describing inner contours of successive levels. Once the contours are computed, blobs whose contour is smaller than our gripper are removed, leaving out only our target gripper. The bottom picture in Fig. 3 shows the image after the abovementioned contour detection process.

5) Fourier descriptors:

Fourier descriptors are used to find the centroid of the tracked gripper and to measure its extent of closure. In order to calculate the Fourier descriptors of the given contour (see subsection above), its coordinates are uniformly resampled and mapped to the imaginary plane,

z(s) = x(s) + iy(s), (2)
where x and y are the resampled contour coordinates (see Fig. 3), i is the imaginary unit, and s is the running arc length [START_REF] Van Der Heijden | Image based measurement systems: object recognition and parameter estimation[END_REF]. After this mapping, we apply the discrete Fourier transform,

Z k = 1 N N -1 n=0 z n e -i2πnk N , (3)
where z n are the contour points of eq. ( 2), and N is the number of points. The more points we consider, the better we approximate the contour. In this work, we chose N = 32, since it provides a good trade-off between contour fidelity and runtime speed. In general, the implementation is most efficient when N is a power of 2 [START_REF]On the discretization error of fourier descriptors of planar closed curves[END_REF]. The centroid of the contour is then defined as (x c , y c ) = (Re(Z 0 ), Im(Z 0 )) [START_REF] Van Der Heijden | Image based measurement systems: object recognition and parameter estimation[END_REF].

We use Fourier descriptors to also evaluate the extent of closure of the gripper. As detailed in Sec. IV-B, in fact, this information is used to provide the human operator with information about the status of the grasping procedure. First, we normalize the Fourier descriptors to make them invariant to translations and rotations of the gripper [START_REF] Huang | A content-based image retrieval system[END_REF]. The objective of this normalization is to isolate the information in the descriptors pertaining to size, position, and orientation. Normalizing the position is achieved by setting Z 0 to zero. The normalization with respect to orientation (and size) is achieved by scaling and rotation of the contour such that the amplitude of the first harmonic Z 1 becomes 1 and its phase is zero [START_REF] Van Der Heijden | Image based measurement systems: object recognition and parameter estimation[END_REF].

After this normalization, we are ready to evaluate the 6-fold symmetry, perimeter, and area of the gripper. In fact, from the point of view of the camera, a completely open gripper looks like a 4-mm-wide star with 6 symmetric spikes (see Fig. 4a), while a completely closed gripper looks like a 1-mm-wide disk (see Fig. 4e). Our approach is therefore twofold. At the same time we try to recognize the 6 symmetric spikes and to measure the perimeter/area ratio of the gripper. Combining these two pieces of information provides us with a reliable measure of how much the considered gripper has closed.

To recognize the spikes, we recall that

If a contour is M -fold symmetric such as z(s) = z(s)e ( 2πni M )
, where n is an integer, then only the harmonics 1 ± M m, where m is an integer, are nonzero [START_REF] Van Der Heijden | Image based measurement systems: object recognition and parameter estimation[END_REF].

For our gripper, M = 6, therefore only harmonics 1 ± 6m, m = 0, 1, 2 should be non-zero. Figure 4 shows three examples of images recorded by the high-resolution camera and their corresponding Fourier descriptors |Z k |. Figure 4a shows a completely open gripper, Fig. 4c shows a half-closed gripper, and Fig. 4e shows a completely closed one. From Fig. 4b, we can see that the difference between descriptors Z 7 , Z -5 , Z 13 , Z -11 and the others is quite large for the open gripper. This difference decreases for the other two grippers, whose descriptors are shown in Figs. 4d and4f. We did not include Z 1 in the graph, since after the normalization |Z 1 | = 1. We measured the 6-fold symmetry of the gripper by calculating the difference between the mean of normalized harmonics 1 ± 6m and the mean of the other normalized descriptors,

6 F S = 2 5 2 m=-2 |Z (1+6m) | - 1 32 16 k=-15 |Z k |. (4)
A high value of this metric indicates that the gripper is 6-fold symmetric (i.e., open in our case).

As mentioned before, in addition to evaluating the 6-fold symmetry of the gripper, we also consider its perimeter/area ratio. Since this ratio is larger for an open gripper than for a closed one, the ratio can be directly considered as a measure of the extent of closure of the gripper. The perimeter P and the area A of the gripper are estimated as

A = π 16 k=-15 k|Z k | 2
(5)

P 2 = 4π 2 16 k=-15 k 2 |Z k | 2 (6)
where Z k are the non-normalized Fourier descriptors [START_REF] Van Der Heijden | Image based measurement systems: object recognition and parameter estimation[END_REF]. We combine metrics 6 F S and P/A to define a new metric that evaluates the extent of closure of the gripper, 6) Kalman Filter: Finally, a Kalman filter is used to deal with uncertainties in the tracking. For this purpose, we modeled a gripper as,

C = 6 F S 6 F S,max 2 + P A A max P max 2 , (7)
x k = F x k-1 + w k , (8)
where x k ∈ R 4×1 is the state variable, F ∈ R 4×4 is the transfer matrix, and w k ∈ R 4×1 is the process noise. Noise w k is modeled as a random variable with Gaussian distribution

N (0, Q), where Q = E[w k w T k ] is the covariance matrix. Measurements z k ∈ R 2×1 are then given by z k = Hx k + v k , (9)
where v k ∈ R 2×1 is the measurement noise, that is again assumed to have Gaussian distributions N (0, R)

for covariance matrix R = E[v k v T k ].
Since we measure only the position of the gripper, z k and H ∈ R 2×4 are simply represented as

z k = z x z y , H = 1 0 0 0 0 1 0 0 . ( 10
)
The position of the gripper as estimated by the Kalman filter is considered our final gripper position p m ∈ R 2 . This position vector is an input of the control and haptic systems as depicted in Fig. 1a. The tracking system runs at 50 Hz.

7) Evaluation:

In order to evaluate the performance of our tracking system, we registered four videos of four different robotic grippers moving in the remote environment, for a total of 281 seconds (see video available as supplemental material). We converted the videos to JPEG images at 5 fps, for a total of 1405 images. The centroid of the gripper as tracked by the tracking algorithm and the reference point were not shown.

Fifteen participants took part in the experiment, including ten males and five females. None of the participants reported any deficiencies in their visual abilities. The task consisted in looking at the images extracted from the videos and clicking at the centroid of the gripper. The difference between the centroid as estimated by the subjects and the centroid as estimated by our tracking algorithm provides a measure of performance of our tracking algorithm. Each experiment lasted for approximately 30 minutes. The algorithm showed an accuracy in tracking the gripper's centroid of 106 ± 30 µm. Similar subjective evaluation approaches are common in the literature [START_REF] Heath | A robust visual method for assessing the relative performance of edge-detection algorithms[END_REF], [START_REF] Bowyer | Empirical evaluation techniques in computer vision[END_REF].

B. Haptic system

The haptic rendering system is composed of a 6-DoF Omega haptic interface (Force Dimension, CH). The interface has six degrees of freedom, three for translation and three for rotations, as shown in Fig. 1c. The interface is constructed in such a way that translations and rotations are decoupled from each other. Translational degrees of freedom are active, while rotational degrees of freedom are passive. This haptic interface is also equipped with active gravity compensation to improve the teleoperation experience and reduce the operator's fatigue. In this work, we use the Omega 6 as an impedance haptic interface. We measure the position of the end-effector, controlled by the human operator, to set the reference target position of the soft gripper. At the same time, through the same end-effector, we provide the operator with haptic stimuli from the remote environment. The scaling factor between master and slave positions is 0.2 in all directions, i.e., moving the end-effector of the Omega interface of 10 cm moves the gripper's reference position of 2 cm. The force to be provided is evaluated according to the feedback condition considered, as detailed in Sec. IV, and it is a combination of kinesthetic, frictional, and vibrotactile stimuli. Kinesthetic and frictional stimuli mainly stimulate Merkel (SA I) and Ruffini (SA II) receptors, while vibrotactile stimuli mainly stimulate Pacinian receptors (FA II). Although Omega interfaces have been often used in the literature to provide (kinesthetic) force feedback, they are also known to be effective in producing vibrations at 25 -200 Hz by commanding sinusoidal forces to their end-effectors [START_REF] Pacchierotti | Intuitive control of self-propelled microjets with haptic feedback[END_REF], [START_REF] Pacchierotti | Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback[END_REF], [START_REF] Abayazid | Experimental evaluation of co-manipulated ultrasoundguided flexible needle steering[END_REF]. The haptic control loop runs at 1 kHz.

Since we control the soft grippers in 2-dimensional space, the translational motion of the Omega interface is constrained on its x-y plane (see Fig. 1c). Force f z (t), provided by the Omega interface along the z axis, is defined as

f z (t) = -k b,k (p r,z (t) -p z,plane ), (11)
where k b,k = 1000 N/m, p z,plane is the location of the x-y plane along z, and p r,z (t) is the current position of the end-effector of the Omega in the z direction. The penshaped end-effector of the Omega is also equipped with a programmable button, which is used during the experiments to activate/deactivate the temperature control on the Petri dish (see Sec. III-C). When the temperature control is active, the pen-shaped end-effector enables the human operator to control the target temperature of the environment. The end-effector can rotate, and its rotation is used to set the target temperature driving the Peltier element to

t pe = θ r 60 + 36 , (12)
where θ r ∈ [0 • , 320 • ] is the rotation of the pen-shaped endeffector (see Fig. 1c). More details on how the temperature control works can be found in Sec. III-C. As mentioned in Sec. II, controlling the temperature of the remote environment enable us to close and open the soft grippers.

C. Control system

Given the current position of the gripper, as estimated by the tracking algorithm, and the commanded reference position, the control system moves the gripper using an array of six orthogonally-oriented iron-core electromagnets, with the objective of steering it toward the reference point. Moreover, a Peltier element regulates the temperature of the water where the controlled gripper is floating, enabling the control of the grasping configuration of the small robot. The reference position and temperature are controlled by the operator through the haptic interface, as detailed in Sec. III-B.

1) Position control: The position control is implemented as a Multi-Input Multi-Output (MIMO) Proportional Integral Derivative (PID) control. The input of the regulator is the current position of the gripper p m (t) ∈ R 3 , as evaluated by the tracking algorithm, while its output is a vector of electromagnetic forces F ∈ R 3×1 to be applied on the gripper to move it toward the reference point. The direction and magnitude of F depend on the direction and magnitude of the position error and on the previous state of the system, as detailed in [START_REF] Khalil | The control of self-propelled microjets inside a microchannel with timevarying flow rates[END_REF], [START_REF] Khalil | Magnetic-based motion control of paramagnetic microparticles with disturbance compensation[END_REF]. Force F needs then to be mapped to input currents for the six electromagnets. In this respect, it is possible to map the commanded currents into electromagnetic forces exerted on the grippers as

F(p m ) = (m • ∇)B(p m ) = Λ(m, p m )I ; ( 13
)
where B(p m ) ∈ R 3×1 and I ∈ R 6×1 are the magnetic fields produced by the electromagnets and their currents at point p m , respectively; m ∈ R 3×1 is the magnetic dipole moment of the grippers; and Λ(m, p m ) ∈ R 3×6 is the actuation matrix, that maps the input currents into magnetic forces. B(p m ) was estimated using a finite elements simulation, and it was then validated using a three-axis Hall magnetometer (Senis AG, Baar, Switzerland). Finally, the magnetic dipole moment of the grippers m was estimated using the U-turn technique [START_REF] Bahaj | An alternative method for the estimation of the magnetic moment of non-spherical magnetotactic bacteria[END_REF].

Since we control 2 degrees of freedom (x and y positions) with 6 electromagnets, the position-control system is overactuated. Consequently, different matrices can be used to map F to I.

In this work we used the Moore-Penrose pseudoinverse of Λ (Λ + ) [START_REF] Penrose | A generalized inverse for matrices[END_REF],

I = Λ + (m, p m )F(P) . ( 14
)
We tuned the PID controller using the Ziegler-Nichols method [START_REF] Ziegler | Optimum settings for automatic controllers[END_REF]. The magnetic positioning system runs at 100 Hz.

The performance of the closed-loop position control was assessed in 120 motion control experimental trials, which consisted of point-to-point positioning tasks (40 trials) and tracking of circular and square trajectories (80 trials). The circular trajectories had a radius of 3 mm, while the square trajectories had a side of 6 mm. Eight different grippers were used. The grippers moved with an average velocity of 721±132 µm/s and with a positioning error of 115±104 µm (mean ± standard deviation). Figure 5 shows a representative point-to-point positioning trial. A video of the experiment is available as supplemental material.

2) Temperature control: The self-folding capabilities of the soft grippers can be controlled using thermal stimuli. For this purpose, we regulate the temperature of the water wherein the grippers are suspended using a Peltier element, which is able to produce both positive and negative temperature differentials between its two faces. The element is directly attached to the bottom of the Petri dish, as shown in Fig. 1b. Moreover, in order to compensate for Joule effects, an aluminum heat-sink is attached to the outer face of the element. A closed-loop temperature control is implemented using an Arduino Uno board and a thermometric probe. The regulator reads the water temperature from the probe and receives the target temperature from the haptic interface (see Sec. III-B). These inputs are then fed to a proportional controller that determines the current for the Peltier element. The system is able to reach the reference temperature at approximately 10 • C per minute.

3) Implementation and stability: The integrated micro teleoperation system is managed by a real-time GNU/Linux computing machine (Debian 8 with Linux Kernel 3.16). The camera and the Omega 6 interface are connected to the computer via Ethernet and USB connections, respectively. Electromagnets are powered by Elmo Whistle 1/60 servo controllers (Elmo Motion Control, Petach-Tikva, Israel). Master and slave systems are connected using a simple direct coupling scheme, as summarized in Fig. 1a. Unlike passive coupling, direct coupling does not reduce the information reliability and response time [START_REF] Ni | Haptic Feedback Teleoperation of Optical Tweezers[END_REF]. However, it does not guarantee the unconditional stability of the system. System parameters are reported in Table I.

In order to guarantee the stability and safety of the teleoperation system, we implemented the passivity controller presented by Franken et al. [START_REF] Franken | Bilateral telemanipulation with time delays: a two-layer approach combining passivity and transparency[END_REF] (see also Sec. I). The control algorithm is able to guarantee stable behavior of bilateral telemanipulation systems in the presence of time-varying destabilizing factors, We considered four target trajectories and eight feedback conditions (a video is available as supplemental material).

such as hard contacts, relaxed user grasps, stiff control settings, and/or communication delays. The architecture is split into two separate layers. The hierarchical top layer, named Transparency Layer, aims at achieving the desired force sensation, while the lower layer, named Passivity Layer, ensures the passivity of the system. The operator and the environment impress a movement to the master and slave systems, respectively. The Transparency Layer displays the desired force behavior by computing the torques to be applied to the operator and to the environment. The Passivity Layer checks how the action planned by the Transparency Layer influences the energy balance of the system. If the passivity condition is not violated, the planned action can be directly applied to both sides of the system. However, if loss of passivity is detected, a scaled control action is applied to preserve stability. Separate communication channels connect the layers at the slave and master levels so that information related to exchanged energy is separated from information about the desired behavior. The parameters used in our implementation of this control strategy are the same employed in [START_REF] Pacchierotti | Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback[END_REF], [START_REF] Pacchierotti | Intuitive control of self-propelled microjets with haptic feedback[END_REF]. Passivity control was only used to regulate the kinesthetic stimuli provided by the haptic device (see Sec. IV), since vibrotactile and frictional stimuli do not affect the stability of the control loop [START_REF] Pacchierotti | Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems[END_REF].

IV. EXPERIMENTAL EVALUATION

In order to evaluate the effectiveness of our integrated system and understand the role of haptic assistance for such application, we carried out two sets of experiments. The first one, described in Sec. IV-A, evaluates the steering capabilities of the proposed teleoperation system for a path following task. The second experiment, described in Sec. IV-B, evaluates the steering capabilities of our system for a pick-and-place task. In both experiments the environment is filled with distilled water, and a soft gripper with a diameter of 4 mm is used.

A. Experiment #1: Path following

The first experiment aims at evaluating our teleoperation system in a path following task, as sketched in Fig. 6a and demonstrated in the video available as supplemental material. A frame of the video is shown in Fig. 6b. Table II summarizes the methods and results of this experiment.

1) Subjects: Sixteen subjects (14 males, 2 females, age range 22 -31 years) took part in the experiment. Six of them had previous experience with haptic interfaces, and no subject reported any deficiency in the perception abilities. The experimenter explained the task, asked the subjects to sign an informed consent, and then spent about two minutes adjusting the setup to be as comfortable as possible. No practice trial was allowed. In order to avoid providing undesired auditory cues, participants were isolated from external noises through a pair of noise-canceling headphones.

2) Methods: The task consisted of steering a soft gripper in the remote environment along a predetermined trajectory, being as fast and precise as possible. According to the feedback condition considered, the subject is provided with visual, kinesthetic, frictional, or vibrotactile assistance about the error in following the target trajectory (see below). We considered four different trajectories,

y = 13.6 -3.2 sin π 6 (1 + 6i) - x 2 (1 + j) , i = 0, 1 j = 0, 1
Trajectory 1 with i = 0 and j = 0 is shown in Fig. 6b. Each subject made sixty-four randomized repetitions of the path following task, with eight repetitions for each feedback condition proposed. We considered eight different feedback conditions, that are combinations of these four types of feedback: visual feedback (Vs), kinesthetic feedback (K), vibrotactile feedback (Vb), frictional feedback (Pc), and position-error feedback (Pe).

When visual feedback (Vs) is provided, the target trajectory is shown on the screen (as in Fig. 6b).

When kinesthetic feedback (K) is provided, the Omega interface provides an attractive kinesthetic force that keeps the user close to the target trajectory. This force f k (t) ∈ R 2 , provided by the Omega along its x and y axes, is where

f k (t) = -k k (p r (t) -p t,r ) -b k ṗr (t), (15)
k k = 1000 N/m, b k = 5 Ns/m, p r (t) ∈ R 2
is the current position of the reference point as controlled by the user through the haptic interface, and p t,r is the point on the target trajectory closest to p r (t).

When vibrotactile feedback (Vb) is provided, the Omega provides a vibration as soon as the user moves the end-effector out from the target trajectory. The amplitude of the vibration increases as the end-effector of the Omega moves away from the target trajectory. This force f vb (t) ∈ R 2 , provided by the Omega along its x and y axes, is

f vb (t) = k vb (p r (t) -p t,r ) sin (2πf vb t) sin (2πf vb t) , (16)
where k vb = 50 N/m and f vb = 150 Hz. The vibration frequency was chosen to fit the haptic interface specifications and to maximally stimulate the Pacinian corpuscle receptors [START_REF] Cholewiak | Sensory and physiological bases of touch[END_REF].

When frictional feedback (Pc) is provided, the Omega increases its friction when the user moves the reference point away from the target trajectory. This force f pc (t) ∈ R 2 , provided by the Omega along its x and y axes, is Finally, when feedback about the position error (Pe) of the gripper is provided, the Omega provides an attractive kinesthetic force that keeps the reference point close to the controlled gripper. This force f i (t) ∈ R 2 , provided by the Omega along its x and y axes, is

f pc (t) =     
f i (t) = -k i (p r (t) -p m (t)), (18)
where k i = 200 N/m, and p m (t) is the current position of the gripper as evaluated by the tracking system. We combined the abovementioned types of feedback in four conditions:

1. Visual feedback (Vs), 2. Visual feedback + Vibrotactile feedback (Vs + Vb), 3. Visual feedback + Frictional feedback (Vs + Pc), 4. Kinesthetic feedback (K), and we tested all these conditions with and without providing feedback about the position error between the soft gripper and the reference point, ending up with 4 (feedback conditions) × 2 (with and without position-error feedback) × 4 (target trajectories) = 32 different experimental conditions. The experiment lasted approximately 30 minutes.

A view of the remote environment is always provided to the subjects through the high-resolution camera and a LCD screen (see Fig. 1). The Omega 6 haptic interface is always used to provide the controller with the gripper's reference position.

Figure 7 shows the position of the reference point (blue) and of the tracked gripper (red) along the x and y axes for target trajectory 1 (see Sec. IV-A2). Average trajectory along the x and y axes (solid lines) ± standard deviation (patches) along the y axis is shown for each feedback condition. The target trajectory is plotted in black.

3) Results: We measure (1) task completion time, (2) error in following the target trajectory with the gripper, (3) distance between the reference point and the gripper, and (4) perceived effectiveness of each feedback condition. To compare the different metrics, we ran two-way repeated-measures ANalysis Of VAriance (ANOVA) tests on the data shown in Fig. 8 [START_REF] Faraway | Practical regression and anova using r[END_REF]. Presence of the position-error feedback (Pe) and type of feedback condition (Vs, Vs + Vb, Vs + Pc, and K) were treated as within-subject factors.

Figure 8a shows the average completion time for the eight experimental conditions. The data passed the Shapiro-Wilk normality test and the Mauchly's Test of Sphericity. Sphericity was assumed for variables with only two levels of repeated measures. The two-way repeated-measure ANOVA revealed a statistically significant change in the completion time for position-error information (F (1, 15) = 19.223, p = 0.001, partial η 2 = 0.562) and feedback condition (F (3, 45) = 9.438, p < 0.001, partial η 2 = 0.386). Post hoc analysis with Bonferroni adjustments revealed a significant increase in the completion time for conditions providing visual and vibrotactile feedbacks on the error in following the target trajectory (Vs + Vb vs. Vs, p = 0.013; Vs + Vb vs. Vs + Pc, p = 0.003; Vs + Vb vs. K, p = 0.002). We used the Bonferroni correction to reduce the chances of false-positive results when multiple pair-wise tests are performed on a same set of data.

Figure 8b shows the average error in following the target trajectory with the gripper for the eight experimental conditions. It is calculated as the mean over time of e m = p m (t)-p t,m , where p m (t) is the position of the gripper as evaluated by the tracking algorithm, and p t,m is the point on the target trajectory closest to p m (t). The data passed the Shapiro-Wilk normality test. Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated for the feedback condition variable (χ 2 (5) = 34.744, p = 0.039). A Greenhouse-Geisser correction was applied to tests involving data that violate the sphericity assumption. The two-way repeated-measure ANOVA revealed a statistically significant change in this error for position-error information (F (1, 15) = 163.579, p < 0.001, partial η 2 = 0.916) and feedback condition (F (2.065, 30.983) = 347.793, p < 0.001, partial η 2 = 0.959). Post hoc analysis with Bonferroni adjustments revealed a significant difference in the error between all conditions (Vs vs. Vs + Vb, p < 0.001; Vs vs. Vs + Pc, p < 0.001; Vs vs. K, p < 0.001; Vs + Vb vs. Vs + Pc, p < 0.001; Vs + Vb vs. K, p < 0.001; Vs + Pc vs. K, p < 0.001).

Figure 8c shows the average distance between the reference point and the gripper for the eight experimental conditions. It is calculated as the mean over time of d rf,m = p r (t)p m(t) . The data passed the Shapiro-Wilk normality test and Mauchly's Test of Sphericity. The two-way repeated-measure ANOVA revealed a statistically significant change in the distance for position-error feedback only (F (1, 15) = 827.144, p < 0.001, partial η 2 = 0.982).

After this analysis, we also tested whether the distance between the reference point and the gripper has any relationship with the measured error in following the target trajectory. Our hypothesis is that it is more effective to control the motion of the gripper when the reference point is close to it, leading to smaller errors. A Pearson's product-moment correlation was run to assess the relationship between this distance and the error in following the target trajectory with the gripper. Preliminary analyses showed the relationship to be linear with variable normally distributed, as assessed by Shapiro-Wilk test, and there were no outliers. There was a strong positive correlation between the distance of the reference point and the gripper and the gripper trajectory error (e m : r(14) = 0.819, p < 0.001), confirming our hypothesis.

Immediately after the experiment, we also measured users' experience. Participants were asked to rate the effectiveness of each feedback condition in completing the path following task using a slider that ranged from 0 to 10, where a score of 0 meant "very low" and a score of 10 meant "very high". Figure 8d shows the perceived effectiveness for the eight experimental conditions. The data passed the Shapiro-Wilk normality test and Mauchly's Test of Sphericity. The two-way repeated-measure ANOVA revealed a statistically significant change in this error for position-error information (F (1, 15) = 14.695, p = 0.002, partial η 2 = 0.495) and feedback condition (F (3, 45) = 61.712, p < 0.001, partial η 2 = 0.804). Post hoc analysis with Bonferroni adjustments revealed a significant difference in the error between feedback conditions Vs vs. Vs + Vb, p < 0.001; Vs vs. K, p < 0.001; Vs + Vb vs. Vs + Pc, p < 0.001; Vs + Vb vs. K, p < 0.001; and Vs + Pc vs. K, p < 0.001.

Finally, subjects were asked to choose the condition they preferred the most. Condition K+Pe was preferred by fourteen subjects and condition Vs was preferred by two subjects.

B. Experiment #2: Pick-and-place

The second experiment aims at evaluating our teleoperation system in a pick-and-place task, as sketched in Fig. 9a and demonstrated in the video available as supplemental material. A frame of the video is shown in Fig. 9b. Table III summarizes the methods and results of this experiment.

1) Subjects: Ten subjects (9 males, 1 female, age range 24 -33 years) took part in the experiment, all of whom were right-handed. Five of them had previous experience with haptic interfaces. None reported any deficiencies in their perception abilities. The experimenter explained the task, asked the subjects to sign an informed consent, and then spent about two minutes adjusting the setup to be as comfortable as possible. No practice trial was allowed. In order to avoid providing undesired auditory cues, participants were isolated from external noises through a pair of noise-canceling headphones.

2) Methods: The task consists of picking up a polystyrene microbead from the ground and dropping it off at a predetermined location. An obstacle is placed in the middle of the remote environment at a known location (see Fig. 9). The pick up and drop-off locations were chosen randomly in the left and right hand sides of the environment, respectively. The microbead was placed on a small pedestal to make the pick up easier.

Each subject made six randomized repetitions of the pickand-place task, with two repetitions for each feedback condition proposed:

• Kinesthetic + Position-Error feedback (K + Pe), where kinesthetic force is used to render both the position error between the controlled gripper and the reference point, and the collisions between the reference point and the obstacle,

• Vibrotactile + Position-Error feedback (Vb + Pe), where kinesthetic force is used to render the position error between the controlled gripper and the reference point, and vibrotactile feedback is used to render the collisions between the reference point and the obstacle,

• No force feedback (N).

In condition K+Pe, the Omega haptic interface provides the subject with kinesthetic feedback about the collisions of the reference point with the obstacle and about the position error between the gripper and the reference point. Kinesthetic force f k,2 (t), responsible for rendering collisions of the reference point with the obstacle, is evaluated according to the popular god-object model [START_REF] Zilles | A constraint-based god-object method for haptic display[END_REF], and the obstacle is modeled as a springdamper system:

f k,2 (t) = -k k,2 (p r (t) -p r,proxy (t)) -b k ṗr (t). ( 19
)
k k,2 = 1000 N/m, b k = 5 Ns/m, p r (t) ∈ R 2 is the current position of the reference point as controlled by the subject through the haptic interface, and p r,proxy (t) ∈ R 2 is the virtual location of the haptic interface (or god-object), placed where the haptic interface point would be if the haptic interface and the object were infinitely stiff (i.e., on the surface of the obstacle in our case) [START_REF] Zilles | A constraint-based god-object method for haptic display[END_REF]. The more the user pushes the endeffector of the haptic interface toward the obstacle, the more force the system will provide. On the other hand, kinesthetic force f i (t), responsible for rendering the position error between the gripper and the reference point, is evaluated as in eq. ( 18). In this condition the subject feels an opposite force when trying to penetrate the obstacle and when moving the reference point away from the gripper. Moreover, in addition to the above mentioned kinesthetic forces, the operator is also provided with a 200-Hz 50-ms-long vibration burst when the tracking system considers the gripper completely close and ready to move the microbead away from its pickup location. This information prevents the operator from moving the gripper too early, when the grasping procedure is not yet complete and the bead has therefore not been safely grasped. The tracking algorithm considers a gripper completely close when C < 1.30, where C is the measure of the extent of closure of the gripper (see Sec. III-A).

In condition Vb+Pe, the Omega haptic interface provides the subject with vibrotactile feedback about the collisions of the reference point with the obstacle, and kinesthetic feedback about the position error between the gripper and the reference point. Vibrotactile feedback f vb,2 (t) is calculated according to the god-object model, and the obstacle is modeled as a spring system:

f vb (t) = k vb (p r (t) -p r,proxy (t)) sin (2πf vb t) sin (2πf vb t) , (20)
where k vb = 80 N/m and f vb = 150 Hz. The position error is again rendered as in eq. [START_REF] Sanchez | Controlled manipulation of multiple cells using catalytic microbots[END_REF]. In this condition the user feels a vibration every time that the reference point enters in contact with the obstacle. Moreover, as in condition K+Pe, the user feels an opposite force when moving the reference point away from the gripper and a 200-Hz 50-ms-long vibration burst when the tracking system considers the gripper completely close.

In condition N, the system provides no information about the position error between the gripper and the reference point, and no information about the collisions between the reference point and the obstacle.

In all three conditions, the closing and opening of the gripper is managed by controlling the temperature level of the remote environment through the Peltier element, as described in Sec. III-C. The target temperature is set by rotating the pen-shaped end-effector of the Omega interface, as detailed in Sec. III-B. A view of the remote environment is always provided to the subjects through the high-resolution camera and a LCD screen (see Fig. 1). The Omega 6 haptic interface is always used to provide the controller with the microgripper's reference point.

3) Results: We measure (1) task completion time, (2) length path of the gripper, (3) distance between the reference point and the gripper, (4) error in placing the microbead at the requested drop-off location, and (5) perceived effectiveness of each feedback condition. To compare the different metrics, we ran one-way repeated-measures ANOVAs on the data shown in Fig. 10. Type of feedback condition (K+Pe, Vb+Pe, and N) was treated as within-subject factors.

Figure 10a shows the average completion time for the three experimental conditions. The data passed the Shapiro-Wilk normality test and the Mauchly's Test of Sphericity. The oneway repeated-measure ANOVA did not reveal any statistically significant difference between the means.

Figure 10b shows the average length of the path of the gripper for the three experimental conditions. The data passed the Shapiro-Wilk normality test and Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA revealed a statistically significant difference between the means (F (2, 18) = 20.518, p < 0.001, partial η 2 = 0.695). Post hoc analysis with Bonferroni adjustments revealed a significant difference in conditions K+Pe vs. N (p = 0.002) and Vb+Pe vs. N. (p < 0.001).

Figure 10c shows the average distance between the reference point and the gripper for the three experimental conditions. The data passed the Shapiro-Wilk normality test and Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA revealed a statistically significant difference between the means (F (2, 18) = 134.252, p < 0.001, partial η 2 = 0.937). Post hoc analysis with Bonferroni adjustments revealed a significant difference in conditions K+Pe vs. N (p < 0.001) and Vb+Pe vs. N (p < 0.001).

Figure 10d shows the average error in placing the microbead at the requested drop-off location for the three experimental conditions. The data passed the Shapiro-Wilk normality test and Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA did not reveal any statistically significant difference between the means.

Immediately after the experiment, we also measured users' experience. Participants were asked to rate the effectiveness of each feedback condition in completing the pick-and-place task using a slider that ranged from 0 to 10, where a score of 0 meant "very low" and a score of 10 meant "very high". Figure 10e shows the perceived effectiveness for the three experimental conditions. The data passed the Shapiro-Wilk normality test and Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA revealed a statistically significant difference between the means (F (2, 18) = 22.275, p < 0.001, partial η 2 = 0.712). Post hoc analysis with Bonferroni adjustments revealed a significant difference between all conditions (K+Pe vs. Vb+Pe, p = 0.011; K+Pe vs. N, p = 0.001; Vb+Pe vs. N, p = 0.022).

Finally, subjects were asked to choose the condition they preferred the most. Condition K+Pe was preferred by six In order to test the effectiveness of our system, we carried out two sets of experiments, involving 26 subjects. The first one aimed at evaluating the steering capabilities of the proposed teleoperation system in a path following task (see Sec. IV-A), while the second one aimed at evaluating the steering capabilities of our system in a pick-and-place task (see Sec. IV-B). In both experiments, regardless of the feedback condition considered, all subjects were able to successfully complete the given task. Moreover, both experiments showed that haptic force feedback provides a statistically significant improvement of performance and perceived effectiveness.

In the first experiment, we provide the human operator with information about the error in following the target trajectory, and about the position error between the gripper and the reference point. Providing haptic stimuli (vibrotactile, kinesthetic, or frictional) elicited statistically significant improvements in the task performance with respect to providing only visual feedback in all the considered metrics except completion time. Moreover, providing information about the position error between the gripper and the reference point encouraged the user to keep the reference point close to the gripper, enabling him or her to control the motion of the robot more accurately. Keeping the reference point close to the gripper was in fact proven to have a statistically significant positive effect in reducing the error of the gripper in following the target trajectory. Although conditions providing kinesthetic feedback (K and K+Pe) outperformed all the others, stability of teleoperation systems with kinesthetic feedback can be significantly affected by communication latency in the loop, hard contacts, relaxed grasps, and many other destabilizing factors which dramatically reduce the effectiveness of this type of feedback in teleoperation [START_REF] Hashtrudi-Zaad | Transparency in time-delayed systems and the effect of local force feedback for transparent teleoperation[END_REF]. For this reason, we have decided to also include the evaluation of other types of feedback that do not affect the stability of teleoperation systems, i.e., visual, vibrotactile, and frictional [START_REF] Pacchierotti | Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems[END_REF]. Among these intrinsically "safe" conditions, the combination of visual and vibrotactile feedback (Vs+Vb) showed the best performance in all the considered metrics except completion time. This condition was also considered by the subjects the second most effective after the ones providing kinesthetic feedback. These considerations are particularly relevant for all those fields and applications were the safety of the system is a paramount and non-negotiable requirement, as in the biomedical applications envisaged in Sec. I. Finally, it is interesting to note that two out of 16 subjects chose condition Vs as the preferred one. These two subjects complained that the feedback about the position error between the gripper and the reference point (Pe) slowed down the task. Although they did not like conditions providing position-error feedback, they still found them very effective in achieving small trajectory errors.

In the second experiment, we provide the human operator with haptic stimuli about the collisions between the reference point and the remote environment, and about the position error between the gripper and the reference point. Providing kinesthetic feedback (K+Pe) statistically significantly improved the performance of the task with respect to the other conditions in all the considered metrics except completion time. Similarly, providing vibrotactile feedback (Vb+Pe) statistically significantly improved the performance of the task with respect to not providing force feedback (N) in all the considered metrics except completion time. As mentioned before, although kinesthetic feedback performed the best, vibrotactile feedback is still very promising, since it performed better than providing no force feedback at all and it does not affect the stability of teleoperation systems. Finally, although the preferred condition was K+Pe, subjects particularly appreciated the capability of condition Vb+Pe to enable the discrimination between the force due to the position-error feedback (kinesthetic) and the one due to the collision with the obstacle (vibrotactile). The stability of the teleoperation system is guaranteed by the time-domain passivity algorithm presented in Sec. III-C3. The controller reduces the force applied to the system every time a loss of passivity is foreseen. Nonetheless, in our teleoperation experiments, mostly thanks to small communication delays, the passivity algorithm seldom acted on the ideal forces computed by the Transparency Layer.

In the near future, we will focus on investigating the practical translational aspects of the proposed micro teleoperation system for biomedical applications. In fact, in the current form, our soft magnetic grippers are still large for many medical applications. For example, for biopsy applications, since the sizes of many cells are in the range of 5-100 microns, ideal sizes for these grippers would range from 5 µm to 1 mm [START_REF] Gultepe | Biopsy with thermally-responsive untethered microtools[END_REF]. Such microgrippers could be introduced through endoscopes, needles, or catheters and be used to access smaller conduits in the body such as the circulatory, urinogenital, lymphatic systems, and mammary ducts. For this reason, we are currently investigating the miniaturization of the bilayer soft gripper used in this study. We are also studying the use of other self-folding grippers, fabricated using metallic materials and inorganic materials. It is noteworthy that elsewhere, ultrasmall untethered grippers capable of capturing even single red blood cells have been described [START_REF] Malachowski | Self-folding single cell grippers[END_REF], which highlight feasibility of further miniaturization. In this respect, we believe that it is important to retain the soft structure of the grippers, as they could be more suitable for medical applications. In fact, the sturdy grasp of rigid grippers may damage, or even destroy, biological cells and tissues.

In addition to exploring smaller grippers, we are also planning to substitute the high-resolution camera with an ultrasound imaging system, in order to be able to track the position of the controlled gripper in biological fluid. In this respect, Sanchez et al. [START_REF] Sanchez | Magnetic control of self-propelled microjets under ultrasound image guidance[END_REF] presented an algorithm for the closed-loop control of micromotors using feedback extracted from B-mode ultrasound images, and Khalil et al. [START_REF] Khalil | The control of self-propelled microjets inside a microchannel with timevarying flow rates[END_REF] demonstrated the effectiveness of a wireless magnetic control system in steering the same micromotors against fluidic flows. We are also investigating the possibility of using targeted onspot heating techniques, such as induction or high-intensity focused ultrasounds. In this way, we would be able to trigger the self-folding capability of the grippers without heating the surrounding environment. Moreover, this targeted heating approach would significantly speed up the closing and opening processes. Of course, the safety of all the above mentioned approaches will have to be assessed for use in biomedical applications involving humans and other living beings. Finally, we also plan to study the effects of passivity-related reductions of haptic stimuli in micromanipulation applications.

  (a) Interconnected haptic-enabled micro teleoperation system. (b) Detail of the slave system.

Fig. 2 .

 2 Fig. 2. Soft gripper fabrication and operation. (a) Schematic illustration of the process used to photopattern SU-8 and pNIPAM-AAc soft grippers. (b) Optical images showing the grippers closing on heating and opening up on cooling.

Fig. 4 .

 4 Fig. 4. Three images of the same soft gripper and their corresponding normalized Fourier descriptors. As the gripper closes, the difference between descriptors Z 7 , Z -5 , Z 13 , Z -11 and the others descreases. We did not include Z 1 in the graph, since |Z 1 | = 1, and plotting it would have made the vertical |Z k | scale too large.

Fig. 5 .

 5 Fig. 5. Position control: A representative point-to-point positioning trial. The reference position of the gripper was suddenly changed and the trajectory of the gripper was registered. Solid lines show the position of the gripper in (a) space and (b) time. The commanded reference positions are shown as circles in (a) and as a dashed line in (b).

Fig. 6 .

 6 Fig.6. Experiment #1: Path following task. Subjects are required to steer a soft gripper along the predetermined trajectory, being as fast and precise as possible. We considered four target trajectories and eight feedback conditions (a video is available as supplemental material).

Fig. 7 .

 7 Fig. 7. Experiment #1: Path following task. Average trajectory of the reference point (blue) and the tracked gripper (red) along the x and y axes (solid lines) ± standard deviation (patches) along the y axis is shown for each feedback condition for target trajectory 1 (black dashed line).

  Error in following the trajectory with the soft gripper (mm). Distance between the reference point and the soft gripper (mm).

Fig. 8 .

 8 Fig.8. Experiment #1: Path following task. Completion time, error in following the trajectory with the soft gripper, distance between the reference point and the soft gripper, and perceived effectiveness for the eight considered feedback conditions are plotted (mean and 95% confidence interval). A video of the experiment is available as supplemental material.

  -b pc ṗr (t) if moving away from the target trajectory, 0 otherwise, (17) where b pc = 20 Ns/m.

Fig. 9 .

 9 Fig.9. Experiment #2: Pick-and-place task. The task consists of picking up a microbead from the ground and dropping it off at a predetermined location. Subjects are able to move, close, and open the soft gripper through the haptic interface. Moreover, through the same interface, they receive haptic stimuli about the interaction of the soft gripper with the environment (a video is available as supplemental material).

  Completion time (s).

  Path length of the soft gripper (mm).

  Drop-off error (mm).

Fig. 10 .

 10 Fig.10. Pick-and-place experiment. Completion time, path length for the soft gripper, distance between the reference point and the soft gripper, placing error, and perceived effectiveness for the three considered feedback conditions are plotted (mean and 95% confidence interval). A video of the experiment is available as supplemental material.

TABLE I TELEOPERATION

 I SYSTEM DETAILS AND PARAMETERS

	Tracking system
	Hardware	Blackfly 1.4 MP Color GigE PoE camera
		+ Mitutoyo FS70 microscope
	Refresh rate	50 Hz
	Accuracy	106 ± 30 µm
	Haptic system
	Hardware	Omega.6 6-DoF haptic interface (transla-
		tional DoF active, rotational DOF passive)
	Refresh rate	1000 Hz
	Workspace	160×110 mm (translations limited to 2-
		DoF)
	Positioning system
	Hardware	Six electromagnets powered by Elmo
		Whistle 1/60 servo controllers
	Refresh rate	100 Hz
	Positioning accuracy	115

± 104 µm (2.9% of the body length) Velocity 721 ± 132 µm/s Grasping system Hardware Conrad TECI-1703 Peltier element 3.9 W Refresh rate 50 Hz Open/close average elapsed time 10 s Interconnection Hardware real-time GNU/Linux machine Stability control time-domain passivity control, adapted from [69] Master-slave position scaling factor 0.2

This project (ROBOTAR) has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation programme (Grant Agreement #638428). The research has also received funding from the European Union's FP7/2007-2013 (Grant Agreement #601165 of project "WEARHAP"). We also acknowledge funding from the National Institutes of Health (NIH) under Award Number R01EB017742 which supported design, fabrication and characterization of the grippers. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

The authors also thank Dr. Stefano Scheggi for his help in setting up the tracking system.