Aude Bolopion

Member, IEEE Hui Xie

Sinan Haliyo

Stéphane Régnier

D Sinan Haliyo

Haptic Teleoperation for 3-D Microassembly of Spherical Objects

Keywords: Haptic interface, bilateral coupling, virtual guide, microassembly, dual-tip gripper

come

I. INTRODUCTION

M ANIPULATION of objects of less than ten micrometers is a challenge, as it shares many difficulties with nanomanipulation. Among them, the visual feedback from the optical microscope is limited and does not enable the accurate positioning of tools and objects. Due to the scale reduction, adhesion forces become predominant over gravitational ones [START_REF] Fearing | Survey of sticking effects for micro parts handling[END_REF]. Tools to manipulate these objects must be carefully designed [START_REF] Menciassi | From "macro" to "micro" manipulation: models and experiments[END_REF]. Fully automated micromanipulation is difficult to achieve, given the high influence of environmental parameters and the lack of repeatability. Moreover, it would result in a lack of flexibility in the overall system. Haptic feedback appears as a promising solution to provide assistance to operators [START_REF] Bukusoglu | Haptic manipulation of microspheres using optical tweezers under the guidance of artificial force fields[END_REF], in particular for AFM-based manipulation [START_REF] Guthold | Controlled manipulation of molecular samples with the nanoManipulator[END_REF], [START_REF] Ferreira | Virtual reality and haptics for nanorobotics[END_REF]. Solutions to assist micromanipulations through haptic feedback are the first steps towards intuitive nanomanipulations.

To provide high quality force feedback, haptic coupling schemes have been developed to handle micro and nanoscale specificity [START_REF] Boukhnifer | H ∞ loop shaping bilateral controller for a two-fingered tele-micromanipulation system[END_REF], [START_REF] Boukhnifer | Wave-based passive control for transparent micro-teleoperation system[END_REF]. In particular, passivity controllers, widely used in macroscale teleoperation [START_REF] Anderson | Bilateral control of teleoperators with time delay[END_REF], [START_REF] Hannaford | Time-domain passivity control of haptic interfaces[END_REF], have been adapted to microscale in order to preserve long-range attraction forces such as van der Waals interactions [START_REF] Kim | Task-based and stable telenanomanipulation in a nanoscale virtual environment[END_REF], [START_REF] Onal | A scaled bilateral control system for experimental one-dimensional teleoperated nanomanipulation[END_REF]. Based on these haptic coupling schemes, several teleoperated micro-and nanoscale tasks are reported in the literature. Early examples deal only with feeling the shape of substrates or objects [START_REF] Hollis | Toward a tele-nanorobotic manipulation system with atomic scale force feedback and motion resolution[END_REF]. [START_REF] Marliere | Multi-sensorial interaction with a nano-scale phenomenon: The force curve[END_REF] reports a haptic implementation of a approach/retract task of an AFM probe. The first remote tasks inducing a modification of the sample are indentations (e.g. direct patterning on a substrate [START_REF] Li | Videolized" atomic force microscopy for interactive nanomanipulation and nanoassembly[END_REF]). Tasks involving pushing/pulling or cutting objects are also of primary interest [START_REF] Guthold | Investigation and modification of molecular structures with the nanoManipulator[END_REF], [START_REF] Sitti | Teleoperated touch feedback from the surfaces at the nanoscale: modeling and experiments[END_REF]. As only two measurements -bending and torsion -are directly available from an AFM cantilever, 3D haptic feedback of nanoscale interactions between the tool and the object can only be achieved by the use of contact mechanics models [START_REF] Sitti | Teleoperated touch feedback from the surfaces at the nanoscale: modeling and experiments[END_REF]. Such models are used for 3D haptic feedback in surface indentation and touching micro-objects [START_REF] Li | Development of augmented reality system for AFM-based nanomanipulation[END_REF], [START_REF] Vogl | Augmented reality user interface for an atomic force microscope-based nanorobotic system[END_REF], [START_REF] Onal | Teleoperated 3-D force feedback from the nanoscale with an atomic force microscope[END_REF]. However, so far, no manipulation tasks have been reported. In addition to transmitting micro-and nanoscale interactions, haptic feedback is used as an enhancement for user assistance, with virtual guides for pushing and pick-and-place by adhesion tasks [START_REF] Ammi | Robotic assisted micromanipulation system using virtual fixtures and metaphors[END_REF]. In this case, haptic feedback is used to keep the user's motion on a specified path.

All the above mentioned works use a single AFM cantilever and static measurement. Consequently, only one controllable contact point is available to the user. Using the AFM in contact mode implies also some limitations compared to using it in dynamic mode. The dynamic mode, where the force measurement is obtained through variations on the amplitude or frequency of a vibrating probe, is generally considered of finer quality than the static contact mode, where the measured force is directly proportional to the deflection of the probe [START_REF] Sitti | Survey of nanomanipulation systems[END_REF]. In addition, a frequent time-consuming factor is the preliminary scan, which is required especially in the case where vision quality is poor (for objects smaller than few micrometers), contact mechanics models are used for 3D feedback, or path planning is required to implement virtual guides.

This work validates the feasibility of using haptic feedback for 3D manipulation. A two-layer pyramidal structure based on four ∅5 ± 1µm microspheres is selected to illustrate the study since these objects are commonly considered at this scale [START_REF] Saito | Micro-object pick and place operation under SEM based on micro-physics[END_REF]. This paper is based on the analysis of haptic coupling schemes we did previously [START_REF] Bolopion | Analysis of stability and transparency for nanoscale force feedback in bilateral coupling[END_REF]. The first results of 2D teleoperation of ∅50µm microspheres with haptic feedback using rolling were obtained in [START_REF] Bolopion | 2D micro teleoperation with force feedback[END_REF]. This current report deals with 3D microassembly in ambient conditions of microspheres ten times smaller. The system uses two independent AFM probes to collaboratively grasp and position each object, as reported in [START_REF] Xie | Three-dimensional automated micromanipulation using a nanotip gripper with multi-feedback[END_REF]. Since this setup enables 3D manipulation of different objects with force measurement [START_REF] Xie | A versatile atomic force microscope for 3D nanomanipulation and nanoassembly[END_REF], the proposed method could be adapted to different objects provided that the equation of their shape is known. Teleoperation through haptic feedback is extensively used in every step of the operation. Since we have chosen to concentrate on the feasibility of a fully teleoperated 3D manipulation, automated manipulation is not considered even though good results could be obtained. The applications of these two modes are complementary: automated manipulation is designed to perform repeated tasks efficiently. Teleoperation provides high flexibility and enables specific manipulations of individual objects, such as placing them on a TEM grid for physical properties analysis. As the proposed approach relies on the operator to increase the flexibility of the system, the prescan step is avoided. It is replaced by a user-guided initial exploration. This exploration allows for online calculation of virtual guides, helping the operator to correctly align the dualtip gripper with respect to the manipulated object, even in the case of poor visual accuracy. Additionally, different feedback schemes are presented for pick-and-place of microspheres.

Compared to existing teleoperation systems at the micro/nano scale, this work presents two main contributions. Haptic feedback based on dynamic mode AFM data is presented for the first time. This mode is most often used for micro/nano manipulation as the measurement of the amplitude of the oscillations provides accurate information about the position of the object. From these data, the grasping point is determined based on a least mean square algorithm. Next, two different haptic feedback methods are proposed for two probes based pick-and-place operations. The first one provides information on the measured interaction forces directly, while the second one assists the user in improving dexterity and avoiding collision. These feedbacks are proposed for the same task, but they correspond to different applications (comprehension of physical phenomena or safe and efficient manipulation). This paper is organized as follows. The experimental setup and the manipulation protocol are described in Section II and Section III, respectively. Haptic feedback based on dynamic mode measurements and online construction of virtual guides to accurately align the grippers to microspheres is discussed in Section IV. Section V explains and analyses in detail the pick-and-place experiments, and Section VI the construction of a two-layer pyramid. A table summarizing the notations used in this paper can be found in Section VIII.

II. EXPERIMENTAL SETUP

Detailed specifications of the manipulation setup are discussed in [START_REF] Xie | Development of a flexible robotic system for multiscale applications of micro/nanoscale manipulation and assembly[END_REF]. A brief summary is given here. The micromanipulation platform is depicted in Fig. 1. The AFM gripper is equipped with an optical microscope, and two sets of nanopositioning devices and optical levers to coordinate two AFM cantilevers with protruding tips (namely, Tip I and Tip II, Nanosensors ATEC-FM) facing each other, forming a dual-tip gripper. Tip I is fixed on an X-Y -Z motorized micropositioning stage, while Tip II is actuated by an open-loop X-Y -Z piezotube mounted on a X-Y -Z manual stage. A closed-loop X-Y -Z nanostage is used to actuate the I. Each cantilever disposes of its own optical lever, comprising a laser source and a four-quadrant photodiode. Data acquisition occurs at 500-800 Hz for static force sampling and at 600 kHz for amplitude through a NI 6289 DAQ card.

Cantilevers can be used in two different modes: tapping and static. For the tapping mode, a piezoceramic excites each probe at its natural frequency. The amplitude of the resulting oscillations is measured through the variations of the voltage output on the photodiode:

A = β • ∆V ( 1
)
where A is the amplitude measurement, β = 10 -6 m • V -1 is a calibrated conversion factor, and ∆V is the differential voltage response of the photodiode.

In static mode, the normal force applied on the cantilever F is measured directly from the output voltage of the photodiode:

F = k n S n ∆V (2)
where k n = 2.8N • m -1 is the normal stiffness of the cantilever, and S n = 8 • 10 -7 m • V -1 is the sensitivity of the optical levers.

An Omega haptic interface, manufactured by Force Dimension 1 is provided for intuitive user control of the manipulator. This master arm is a 3 degrees of freedom (DoF) device. The user manipulates the handle and the resulting position p h is scaled down to be used to control the actuators (nanostage and piezotube). The haptic force F h sent to the user through the haptic interface is based on measurements from the two photodiodes (V I and V II ). As represented by the switches S 1 and S 2 in Fig. 1, different translators and feedbacks are used at each step of the microassembly. The next sections detail the use of the haptic interface to interactively perform a microassembly task.

III. 3D MICROASSEMBLY PROTOCOL At microscales, complex strategies must be used to manipulate objects [START_REF] Savia | Contact micromanipulation -survey of strategies[END_REF], [START_REF] Sausse-Lhernould | Electrostatic forces in micromanipulations: review of analytical models and simulations including roughness[END_REF]. Dual-tip manipulation has certain advantages over other 3D manipulation techniques. These alternatives are adhesion-based pick-and-place or monolithic two-finger grippers. The former requires mechanically complicated strategies for an accurate placing on the substrate [START_REF] Haliyo | Controlled rolling of micro objects for autonomous micro manipulation[END_REF]. The latter suffers from the lack of force sensing of the gripper [START_REF] Heriban | Robotic micro-assembly of microparts using a piezogripper[END_REF] and from the ill-controlled gripper/object interactions, resulting in serious drawbacks in a scenario requiring precision, such as the one proposed here. On the other hand, in dual-tip manipulation the contact area is extremely small, and hence gripper/object adhesion is a minor issue. As both probes provides AFM-grade force sensing, it is possible to monitor all interactions, including contacts, adhesion forces and primarily the grasping force. However, these advantages come at the cost of augmented complexity of the overall manipulation process. Different delicate steps are required to position each tip correctly and to coordinate the pick-and-place of the object. An overview of the complete manipulation scenario is depicted in Fig. 2.

In order to grasp the object between two tips, each contact point has to be aligned with the center of the manipulated sphere. Given the relative sizes of manipulated objects, AFM cantilevers and their protrudent tips, a vision-based control scheme under an optical microscope does not provide sufficient resolution and precision. An initial AFM scan would give additional information for correct positioning, but that is a time-consuming step and comes at the risk of disturbing the manipulation scene.

The approach proposed here is based on a user-driven exploration of the manipulated object. Haptic feedback allows the operator to feel when he/she touches the object while freely exploring the manipulation area. Note that during this operation the vertical position of the probes are constrained to a few micrometers above the substrate and the operator controls only the horizontal motion. The data recorded during this exploration is processed online, which generates a virtual guide to pull the user to the optimum contact point. The user "feels" and sequentially adjusts the contact force for both tips, ensuring an adequate grip on the object. In the third phase of the manipulation, both grippers are immobilized on both sides of the object and the operator controls the motion of the sample holder, while still receiving haptic feedback calculated from the output of the two probes.

The choice of the particle to be manipulated is made using a top view optical microscope (from an Olympus BX50WI Steps of dual-tip pick-and-place manipulation. The manipulation area and a coarse positioning of the tips are determined using the optical microscope. The operator then sequentially places each tip on both sides of the object using the Omega, controlling respectively the sample holder through the nanostage and Tip II through the piezotube. In both steps haptic feedback is provided based on the amplitude variations of each tip in tapping mode.

Once the object is held between the two tips, the lift-off and placing on the substrate are achieved by haptic control of the sample holder. Contact mode measurements are used to provide the force feedback.

microscope). The coarse positioning of the tips is also performed using this visual feedback. They are moved using the micropositioning modules (manual and motorized stages) described in Table I. The positioning at the correct height is achieved by automated detection of the substrate. The user then selects the task to be realized using a user interface. This protocol could be simplified by automated transitions between these steps. In that case, the completion of the pick-and-place operation would be determined by the user's decision, to give him or her the possibility of picking up again the sphere to bring it elsewhere. Tip-alignment phases, including the haptic feedback and virtual guide generation and pick-and-place phases with two different haptic schemes are detailed in the following.

IV. ASSISTED GRIPPER ALIGNMENT

The alignment of each tip is a user-driven process. The operator moves the tip while receiving haptic feedback derived from amplitude measurements of the AFM probe. During the initial exploration and prior to the generation of virtual guides, the haptic feedback is only on the x axis (Fig. 3(a)). As the operator manually scans the surface of the to-be-manipulated object, the data are recorded to reconstruct its shape and create the virtual guide. This virtual guide generates the haptic feedback along the y axis, pulling the tip to the calculated grasp line y 0 , parallel to the x axis, and crossing the sphere's center.

(b) (c) A 0 A t A 0 A t (d) t 1 t 2 t n 1 t A 2 t A Tip

A. Tapping Mode Measurements

In tapping mode, each probe is excited at its natural frequency. At constant z position above the substrate and away from objects, this results in oscillations at constant amplitude, noted as A 0 . While approaching an object, starting from a few hundreds of nanometers, the tip contacts the object intermittently and the amplitude A t decreases until a minimum value A CP is reached at full-contact between the tip and the object.

Fig. 3 illustrates the principle of object detection from amplitude variations. The tip is first set to a given z position above the substrate h 0 . This step is achieved in an initial phase and the user controls the motion only in the (x, y) plane parallel to the substrate. While the tip moves on the grasp direction of the gripper, the x axis, amplitude decreases until contact (Fig. 3(b)). On the y axis, perpendicular to the grasp direction, both tips must be aligned with the center of the sphere. This matches the minimum of amplitude along the y axis, at a fixed x position (Fig. 3(c)).

B. Haptic Feedback for Tip Alignment

To align the gripper with the sphere and bring it to contact the visual feedback from the optical microscope does not provide sufficient resolution. Haptic force aims to compensate for this lack of visual feedback. The haptic coupling used is depicted in Fig. 4 2 . Each tip is sequentially aligned on the 2 More information about this control scheme can be found in [START_REF] Bolopion | Analysis of stability and transparency for nanoscale force feedback in bilateral coupling[END_REF]. This remark also applies for the coupling depicted in Fig. 10. grasp line and brought to contact.

1) x axis:

The haptic feedback along the x axis should provide the following information:

• R 1 : force null when the tip is far from the object • R 2 : increasing force as the tip approaches the object • R 3 : increasing force as the tip applies a force on the object According to the variation of amplitude described previously, the following haptic feedback F hx is proposed and satisfies requirements R 1 -R 3 :

F hx = -α a (A -A 0 ) if A > A CP -α a (A -A 0 ) + k x (x -x CP ) else (3)
where α a is a scaling factor. The amplitude A 0 is measured at the beginning of the experiment while the tip oscillates at its natural frequency. Two cases are distinguished:

• before contact (first equation): as the amplitude is decreasing while the tip approaches the object, an increasing repulsive force is sent to the user so that he or she is aware of the presence of the object • while in contact (second equation): a spring k x between the position of the contact point x CP and the current position of the tip x is added to the feedback of the first equation. It simulates the force applied by the tip to the sphere. The contact point location x CP is set when the amplitude measurement reaches A CP at full contact. A CP is an arbitrary threshold, set according to the conclusions given in [START_REF] Xie | Three-dimensional automated micromanipulation using a nanotip gripper with multi-feedback[END_REF].

2) y axis: The force perceived along the y axis must enable the user to align the tips with respect to the sphere, on the grasp line. The haptic feedback along the y axis is not available before all the points have been recorded and the computation of the virtual guide by Equation ( 7) is achieved. During this exploration in search of the y 0 position, the x axis haptic feedback is provided to the user so that he or she perceives the sphere's location. When y 0 is computed, a haptic feedback F hy simulating a spring k y between y 0 and the current position y of the tip is sent to the user:

F hy = k y (y -y 0 ) (4)

C. Virtual Guide Generation

During the initial exploration in tapping mode, n contact points (x i t , y i t ), with their matching amplitudes A i t (i = 1..n) are collected. The contact position data are acquired only if the actual amplitude A t is in the [15%A 0 , 70%A 0 ] interval to avoid false positives. In order to define the z i t coordinate for each contact point (x i t , y i t ), the approximation z i t = A i t /2 is proposed. This is a relative position since the cantilever is oscillating around the z position h 0 , set manually. The calculation of the z 0 coordinate of the sphere is thus relative to h 0 , and is not accurately known. However, as the only parameter useful for haptic feedback is the y 0 coordinate, this approximation is acceptable.

Fig. 3(d) represents the n points (t 1 , ...t n ) recorded during the exploration process. These points are used to reconstruct the shape of the manipulated sphere, calculate the grasp line, and provide the haptic feedback along the y axis. With prior knowledge of the shape of the object, and the n recorded points, the sphere can be reconstructed from the surface equation:

(x -x 0 ) 2 + (y -y 0 ) 2 + (z -z 0 ) 2 = R 2 (5)
where R is the radius of the sphere, and x 0 , y 0 and z 0 are the coordinates of its center. This can be written as:

x 2 + y 2 + z 2 -C a x -C b y -C c z +C d = 0 (6)
where:

C a = 2x 0 , C b = 2y 0 , C c = 2z 0 and C d = x 2 0 +y 2 0 +z 2 0 -R 2 .
Finding the coefficients C a , C b , C c , and C d allows us to define the coordinates at the center of the microsphere and its radius. To do so, the equation which best fits the n recorded points is determined using a least mean square algorithm. The solution of the following system of equations gives the coordinates of the sphere's center:

         x 1 t y 1 t z 1 t -1 x 2 t y 2 t z 2 t -1 x n t y n t z n t -1                   C a C b C c C d          =          (x 1 t ) 2 + (y 1 t ) 2 + (z 1 t ) 2 (x 2 t ) 2 + (y 2 t ) 2 + (z 2 t ) 2 ... (x n t ) 2 + (y n t ) 2 + (z n t ) 2          (7)
The coefficients C a , C b , C c and C d can be deduced from:

C a C b C c C d T = M -1
•Y where T is for the matrix transposition and:

M =     ∑(x i t ) 2 ∑ x i t y i t ∑ x i t z i t -∑ x i t ∑ x i t y i t ∑(y i t ) 2 ∑ y i t z i t -∑ y i t ∑ x i t z i t ∑ y i t z i t ∑(z i t ) 2 -∑ z i t -∑ x i t -∑ y i t -∑ z i t n     Y =     ∑ x i t [(x i t ) 2 + (y i t ) 2 + (z i t ) 2 ] ∑ y i t [(x i t ) 2 + (y i t ) 2 + (z i t ) 2 ] ∑ z i t [(x i t ) 2 + (y i t ) 2 + (z i t ) 2 ] -∑[(x i t ) 2 + (y i t ) 2 + (z i t ) 2 ]    
The position of the grasping point along the y axis is then computed as y 0 = C b 2 . Note that as all the points are on the same side of the sphere along the x axis (x < 0 for Tip I and inversely for Tip II), the calculated x 0 coordinate may be inaccurate. However, as stated above, the only parameter used for virtual guide is y 0 . The generation of this virtual guide depends on the number of points n and their respective positions. An empirical analysis to define a minimum value for n and the distribution of the points recorded is presented below along with experimental analysis.

D. Experimental Validation of Tip Alignment

Manipulated objects are microspheres, with a diameter of 4 -6µm. An image of a sphere from an optical microscope, and one from an AFM scan are presented in Fig. 5(a) and (b). Before aligning the tips with the spheres, they are first positioned manually at the correct vertical position (around 500 -600nm) above the substrate. [START_REF] Bolopion | Analysis of stability and transparency for nanoscale force feedback in bilateral coupling[END_REF] proposes a haptic feedback solution for this step. Each tip is then sequentially positioned by the operator at each side of the object. For Tip I, the user actually controls the nanostage transporting the sample holder. As the nanostage includes a closed-loop position controller, the Omega supplies directly set-point values for its motion. For the alignment of Tip II, the piezotube actuator is used. As this actuator is in open-loop, it does not provide accurate positioning. However, coupling the piezotube with a haptic interface is equivalent to a closed-loop force-feedback scheme with the operator as the controller. All experimental data presented below are acquired using the nanostage and provides accurate position information. The piezotube, although lacking precise position measurements, gives qualitatively similar results.

1) x-Axis Haptic Feedback: Experimental results acquired while moving the tip along the x axis and contacting the microsphere are depicted in Fig. 6. The position of the tip is represented in Fig. 6(a), the amplitude measurement is depicted in Fig. 6(b), and the haptic feedback in Fig. 6(c). In area 1, the tip is away from the sphere and the feedback is null. In area 2, the user distinctly perceives the haptic feedback as the tip approaches the sphere and intermittent contact starts. An additional feedback is transmitted when an effort is applied by the cantilever on the sphere in area 3, increasing the sensation of stiffness.

Compared to using direct force measurement from a cantilever in static mode, the tapping mode amplitude measurement provides a better sensitivity on the x axis. In static mode, as the measurement direction is almost aligned with the probes' length, the equivalent stiffness is extremely high compared to k n on the z axis. Hence, a static detection on the x axis would only occur when a quite important force is already applied on the object. In contrast, the use of the tapping mode allows earlier detection of the object, as only intermittent contact with the object produces a detectable signal. This allows users to be aware of the object's presence and prevents their involuntarily pushing.

2) Generation of the Virtual Guide: As stated above, the preliminary exploration provides the data points to generate the virtual guide and the associated y axis feedback. The influence of two parameters -the number of points n used in Equation [START_REF] Boukhnifer | Wave-based passive control for transparent micro-teleoperation system[END_REF], and the minimum distance between two points (noted d) -is experimentally explored in order to optimize the virtual guide. Table II summarizes the different trials. Each experiment (for a given (n, d) couple) is repeated 5 times. Fig. 7 compares a reference AFM scan and y 0 obtained by shape reconstruction, for the considered (n, d) couples. The standard deviation is displayed in Fig. 7(c). As all these results are user dependent -since users are free to choose any trajectory for the initial exploration -they should be treated only qualitatively. The first observation is that increasing the number of points n has little impact on the accuracy of the virtual guide if these points are close to each other. Moreover, if n is set too high, more points on the edges of the semi-sphere are required.

As the position data at these locations are less accurate, the standard deviation is higher. On the other hand, setting a minimum value for d forces data points to be more evenly distributed on the surface and leads to a better estimation (Fig. 8) as supported by the decrease of the standard deviation. Fig. 7 also shows that except for small d values, increasing the number of points highly increases the time-cost of the virtual guide generation. Since only the hemisphere facing the tip is accessible and the tip is constrained in the vertical direction, setting d to a minimum value limits the maximum number of points that can be acquired. Choosing d = 0.3µm and n = 12 is a good trade-off between precision of the results and time-cost of the guide generation. These values are selected for the following manipulations. 3) y-Axis Haptic Feedback: The y axis feedback is effective as soon as the virtual guide is generated. Its value is calculated using Equation (4). Fig. 9 represents the force perceived by the user. The position of the tip as well as y 0 are also given. The haptic feedback on the y axis helps the user to align the tip with respect to the sphere, as y 0 is at the equilibrium point of the virtual spring.

Precise positioning is achieved since contact information is transmitted to the user through the x axis of the haptic device F hx , and alignment is ensured thanks to the haptic force F hy .

V. PICK-AND-PLACE WITH HAPTIC FEEDBACK

After the object is grasped between the two tips, it will be lifted from the substrate, transported to the target location, and placed on the substrate. Two different haptic feedbacks are proposed for this task. The first one renders to the user directly the forces measured by both probes, with proper scaling. The second one calculates a virtual guide using these measurements to assist the user to lift-off the object to a vertical position set sufficiently high to avoid any contact with other objects or the substrate. It also ensures that the placing is voluntary. In both cases, as depicted in Fig. 10, the Omega haptic device is used for position control of the sample holder through the nanostage, while the two tips holding the microsphere are immobilized. The force data are obtained in static mode, from the deflection of each probe measured directly on photodiodes using Equation ( 2). Haptic feedback is rendered along the vertical ascending z axis. Hence, a positive value results in a force pushing the haptic handle upwards, away from the substrate; it is hence called "repulsive", while a negative value pulls the handle downward towards the substrate ("attractive").

The manipulation is carried out in ambient conditions, at 20 • C, and relative humidity of 48%.

A. Haptic Feedback of Nanoscale Interactions

This first haptic feedback returns to the user the nanoscale interactions of the pick-and-place operation as faithfully as possible. It is synthesized from force responses of Tip I and Tip II. As detailed in [START_REF] Xie | Three-dimensional automated micromanipulation using a nanotip gripper with multi-feedback[END_REF], adhesive forces F ao between the sphere and the substrate can be estimated as:

F ao = F I + F II ( 8
)
where F I (resp. F II ) is the force applied to Cantilever I (resp. II). Hence, the haptic force rendered to the user is computed as:

F f e hz = α f (F ao -F 0 ) = α f [(F I -F I 0 ) + (F II -F II 0 )] (9)
where F 0 = F I 0 + F II 0 is the force measured when the tips are holding the sphere before lift-off and it is naturally proportional to the grasping force applied by the tips to the object. Removing this offset allows the user to discard the grasping force which is not useful for pick-and-place. Moreover, in the case where the grasped object is lost hazardously during the lift-off, the measured forces F I and F II will fall back to zero, and Equation ( 9) will give a negative value, pulling back the probes to the substrate. A force amplification factor α f is used to scale the measured forces and the haptic force sent to the user. The nominal value used here is α f = 2.0 • 10 6 . This coefficient is set considering the magnitude of nanoscale interactions that should be felt by the user (in particular the pull-off force). Detailed discussion on the definition of this parameter can be found in [START_REF] Bolopion | Analysis of stability and transparency for nanoscale force feedback in bilateral coupling[END_REF]. Fig. 11 represents the haptic feedback during a pick-and-place operation of a 5µm sphere from a glass substrate and the insert depicts forces measured from probes. The curve's starting point is the contact state between the microsphere and the substrate. As the nanostage moves down (hence the object held by the tips is lifted), probes are bent down measuring negative forces (inset i). During the pickup, when the nanostage position reaches around -900 nm, the microsphere pulls off the substrate with a minimum force of -1125 nm overcoming the adhesion. Note that after the pull-off, the measured force falls to -550 nm, and not to the pre-pick-up null value (inset ii). Actually, as the tip/object contact points are in the lower hemisphere, during the lift-off the object slides slightly down, Measured normal force responses from both microcantilevers during the pick-and-place manipulation of a microsphere: (i) Pick-up occurs; (ii) The microsphere is detached from substrate after the pull-off; (iii) The microsphere snaps into the substrate; (iv) The gripper/microsphere pulls-off; (v) The gripper snaps into the substrate; (vi) Manipulation ends with slight bending of the microcantilevers. increasing the grasping force 3 . During the transport phase, a change on the force can be noted. This is again due to the sliding of the sphere in the gripper. This hypothesis is backed up by the approx. 0.2µm difference seen between pick-up and touch down positions along the z axis.

During the placing operation the microsphere snaps-in the substrate (inset iii). As the object is pushed to the substrate, between (iii) and (iv) the contact force compensates the grasping force, until the tips pull off the sphere (inset iv) and slide down from the object to the substrate (inset v). At this point, it is sufficient to move apart both tips along the y direction to release the sphere from the gripper and achieve the operation. Note that since the contact area at object/tip interface is much smaller than at the object/substrate interface due to the sharp tips used, the problem of the object adhering to a probe is limited.

B. Haptic Feedback Providing Assistance

The haptic scheme presented above allows the user to feel the nanoscale interactions, especially the adhesion and the well-known pull-off phenomena. However, it is arguable if this feedback has a positive effect on manipulation dexterity. It may be more interesting to conceal these effects of which the operator is unfamiliar with and to replace the haptic feedback with a virtual guide. An assistive haptic feedback, with a positive effect on the dexterity, should fulfill following requirements: 3 Both pick-up and unintentional loss of a sphere yield a negative force feedback until the pull-off (either the pull-off of the sphere from the substrate, or the tips from the sphere). After the pull-off, in case of picking-up the sphere, the force feedback remains negative due to the grasping force. In case of an unintentional loss of the sphere, the cantilevers will go back to their neutral position, i.e. with no bending. In that case, the haptic feedback sent to the user will become positive since the zero haptic feedback corresponds to the initial grasping force applied on the sphere. The user will thus be able to distinguish the pick-up and the unintentional loss of the sphere.

• the user does not have to use great effort to lift the sphere off • the sphere should be kept at a given z position during transport to avoid contact • the sphere should be placed voluntarily only, not because the operator touches the substrate accidentally • while placing the sphere, the effort applied should be as little as possible to enable an easy release of the tips. Sufficient haptic information should be provided so the user can effectively feel that the placing has been achieved. The proposed approach is based on the use of the opposite of the measured interaction force as haptic feedback. As such, for example, the pull-off force will result in a positive force on the haptic handle, actually pushing the held object away from the substrate. The expected perception is comparable to releasing a pressed keyboard button.

As previously, the haptic feedback is computed from force measurement from both photodiodes. To fulfill the requirements stated above, the output of the photodiode is converted into the haptic force F as hz :

F as hz = -α f (F ao -F 0 ) if z < z PO -α f (F ao -F 0 ) + k z (z -z PO ) else (10)
where α f is a force amplification factor, F is the force measured from the photodiode, F 0 is the grasping force as above; z is the position on the vertical axis and z PO is the vertical position of nanostage corresponding to the end of pull-off phenomena. Its value is detected online during the manipulation from the sudden drop in force measurement; k z is the stiffness of the virtual spring of the haptic guide which will effectively pull the object above z PO and keep it at a constant vertical position. The result of this feedback scheme is depicted in Fig. 12 for a pick-and-place operation on the substrate. A repulsive force, proportional to the opposite of the measured adhesive forces, assists the user to lift the object during the pull-off. Immediately after the pull-off phenomena, the measured force does not fall back to its initial value, due to the sliding of the sphere between the tips as explained above. Thus, a residual portion of the repulsive force remains at z ≥ z PO . As the spring k z is activated at z PO , this force is counterbalanced by the virtual guide at z D , above z PO . Then users can freely move the sphere above the substrate in the horizontal plane while the spring analogy of the virtual guide ensures that they keep a relatively constant position on the z axis. To place the sphere on the substrate the user has to counteract the spring k z between z D and z PO and the residual repulsive force below z PO . This condition ensures that the object is not be placed by mistake. When the force measurement falls below F 0 , the vertical motion is automatically stopped, so that no additional force is applied as the sphere reaches the substrate. This facilitates the releasing of the tips and protects fragile objects and the gripper. As this results in the haptic force becoming null, the user clearly discerns the placing. For haptic feedback, the parameters are set to: α f = 2.5 • 10 6 , k z = 1 • 10 6 N.m -1 . Forces higher than 3N are truncated.

The perception of this haptic feedback is equivalent to pushing a keyboard button: the object is kept at the vertical position z PO during the transfer and the user has to voluntarily push it back to the substrate to place it. Table III details each step of the pick-and-place operation using this virtual guide with phase transitions and user perception.

Using this scheme the pick-and-place operation is made easier and safer as the haptic feedback helps the user to perform the given steps correctly.

C. Comparison of Two Haptic Feedback Rendering

Two haptic feedback methods are presented in this paper for pick-and-place operation. However, the resulting feedback in the users is different (see Table IV). Haptic feedback of nanoscale interactions aims at transmitting physical phenomena. The user directly feels the adhesive forces, the pull-off phenomena, and the contact with the substrate. It improves the understanding of these interactions. The haptic feedback for assistance is not designed to be related to the physical forces but to facilitate the manipulation operation. The choice of the method will thus depend on user needs and the specificities of the task.

VI. CONSTRUCTION OF A TWO-LAYER PYRAMID

In order to validate 3-D manipulation capabilities of the haptic system, a microstructure is built. The example of a two-layer pyramid composed of four nylon microspheres with diameter of 4-6µm is chosen [START_REF] Saito | Micro-object pick and place operation under SEM based on micro-physics[END_REF]. Microspheres were deposited on a freshly cleaned glass substrate. An area of interest for the experiments was selected under an optical microscope with 20× optical magnification. Fig. 13 shows the selected area and the inset shows the assembly sequence. After a coarse positioning under the optical microscope, the

Step

Haptic feedback Position of the tips: Eq. used A: Lift-off begins.

F 0 is acquired, F as hz is null. z < z PO : 1

A-B: The sphere is pulled to overcome adhesive forces.

F as hz is repulsive and increases: it helps the user lifting the sphere. The pull-off is detected, the position z PO is acquiried, the feedback switches from Eq. 1 to Eq. 2. The feedback switches from Eq. 2 to Eq. 1.

z = z PO : 2 → 1 E-F: The sphere is moved towards the substrate.

F as hz decreases. z < z PO : 1 F: The sphere is placed on the substrate: F = F 0 .

The vertical motion is stopped. F as hz = 0 z < z PO : 1

TABLE IV COMPARISON OF HAPTIC FEEDBACK METHODS FOR PICK-AND-PLACE

OPERATIONS

Step

Direct haptic feedback of nanoscale interactions

Haptic feedback providing manipulation assistance

The sphere is lifted-off.

The haptic feedback is attractive, and corresponds to adhesive forces.

The haptic feedback is repulsive to help the user to lift the sphere.

The sphere is above the substrate.

A residual haptic force attracts the user. It corresponds to the slide down of the sphere in the gripper.

A spring attracts the user to an equilibrium position.

The sphere is placeddown.

The haptic feedback is attractive and decreases. It is null when the sphere is on the substrate.

The haptic feedback is repulsive. The forces the user must counteract ensure a voluntary placingdown. The feedback is null when the sphere is on the substrate.

total manipulation, which includes the alignment of the gripper and the pick-and-place operation, takes less than five minutes per sphere. Since this manipulation is based on the operator, this time mainly depends on users' skills. It is given only as an indication here. The 3D micropyramid is built using pick-and-place manipulation. Fig. 14 shows the 3-D micromanipulation process. Fig. 14(a) and Fig. 14(b) are captured when the first sphere is picked and placed. Fig. 14(c) shows the second sphere manipulated and released at its final position. The first layer is finished after the third microsphere has been manipulated by pushing (Fig. 14(d)). Fig. 14(e) and Fig. 14(f) describe the transport of the last microsphere which completes the assembly. The ultimate result is shown in Fig. 14(f), and with a magnification of ×100 in Fig. 14(g). Fig. 15 shows the haptic feedback of nanoscale interactions during the manipulation of the fourth sphere. As this sphere is put back above the others, the placing occurs ∼ 4µm above the initial position.

In the case where assistive feedback is used for manipulation of the fourth sphere, two points are considered:

• The given vertical position at which the sphere is maintained with zero force feedback must be higher than the first layer of the spheres. This is adjusted by changing the value of the stiffness k z . To increase the z position above the substrate, this stiffness should be decreased. • While placing the sphere, the z position will not become lower than z PO . The force feedback will not decrease before placing the sphere. When the measured force will be such that F ≤ F 0 (the sphere is deposited onto the pyramid), the vertical motion of the sphere will be stopped and the force feedback will become null. In these circumstances assistive haptic feedback produces results similar to the ones presented in Fig. 12. Moreover, if the height at which the sphere should be placed is known, Equation 10 can be adjusted by tuning k z so that haptic feedback is identical to the one transmitted when the sphere is placed down on the substrate.

VII. CONCLUSION

Haptic feedback is used to perform a 3D microassembly operation using AFM. Although a complex tool composed of two cantilevers as a dual-tip gripper has to be used to perform accurate grasping and pick-and-place at this scale, users unfamiliar with the setup were able to carry out the task, as haptic feedback ensures a high intuitiveness of the setup. To sequentially place the two tips in the grasp line and in contact with the object, both cantilevers are used in dynamic mode. This approach enables sufficient feedback to detect the sphere even if the measurement direction is almost aligned with the probe. Data acquired during this exploration are processed online to compute virtual guides which pull the user to the grasp line. For the pick-and-place step, haptic feedback allows us to either faithfully render microscale interactions or provide assistance to the operator for improved dexterity and collision avoidance. These feedbacks are based on force measurements from both cantilevers in static mode. A two-layer pyramid is built from four ∅5 ± 1µm spheres in ambient conditions for experimental validation of the usability of the overall setup.

This paper concentrates on the feasibility of a teleoperated system with haptic feedback for 3D AFM-based manipulation.

To design an efficient setup, it is necessary to clearly define which tasks could be automated, and which ones require the high flexibility of teleoperation, depending on the application. User-based tests should then be carried out to measure the performance of the proposed systems. A user evaluation will also compare the pick-and-place haptic feedback schemes in terms of efficiency and perception. In this paper, the example of a pyramid is chosen to illustrate the analysis. The methodology presented could be extended to other type of objects, such as carbon nanotubes or nanowires. In particular, the virtual guide along the y axis could be adapted to non-spherical objects by taking the object geometry into account.

The system presented is a first step towards haptic feedback nanomanipulation. Prior to using it, though, a real-time virtual reality scene should be generated and displayed in 3D, along with virtual guides, to compensate for the complete lack of visual feedback. to amplitude • A, F: Amplitude and force measurement from the photodiodes, respectively • A 0 , F 0 : Amplitude when the tip is oscillating at its natural frequency, force when the tip is placed in contact with the sphere before the grasping operation

VIII. NOMENCLATURE

• F I 0 , F II 0 : Force output of Photodiode I, II when the tip is placed in contact with the sphere before the grasping operation • A i t : Amplitude of the i th recorded contact point during the tapping mode exploration • A CP : Residual amplitude at full contact • F ao : Adhesion forces between the sphere and the substrate Coordinates:

• x 0 , y 0 , z 0 : Coordinates of the sphere center • x i t , y i t , z i t : Coordinates of the i th recorded contact point during the tapping mode exploration

Fig. 1 .

 1 Fig. 1. AFM gripper-based telemicromanipulation system: (left) The haptic device providing a user interface to control the 3D microassembly with realtime haptic feedback; (right) The dual-probe gripper comprises two AFM cantilevers with protruding tips for pick-and-place micromanipulation.

  Photo I and II: force orfe hz F

Fig. 3 .

 3 Fig.3. A schematic diagram of haptic exploration by local scan of the lower semi-microsphere using a oscillating cantilever: (a) Top view of desired grasp configuration; (b) Front view shows the tip tapping the microsphere while approaching on the x-axis; (c) Side view shows the tip tapping the microsphere when scanning on the y-axis with a fixed x position; (d) n contact points recorded with random x i t and y i t positions with matching amplitudes A i t .

Fig. 4 .

 4 Photo I Photo IIA I

Fig. 5 .

 5 Fig. 5. (a) An optical image of a microsphere under 100× optical magnification (the scale bar represents 5µm). (b) An AFM image scan on the microsphere.

Fig. 6 .

 6 Fig. 6. Haptic feedback along the x axis while exploring the to-bemanipulated sphere: (a) Tip position; (b) Measured tip oscillation amplitude; (c) Force sent to the user. For the haptic feedback, the coefficients are set to: α a = 3 • 10 6 N.m -1 , k x = 10 • 10 6 N.m -1 and a CP = 0.1µm.

Fig. 7 .Fig. 8 .

 78 Fig. 7. Results of the virtual guide generation: (a) Reference scan at several x positions from the sphere (the dash line represents the grasp line y 0 ); (b) Estimated location of the grasp line; (c) Standard deviation; (d) Time needed to complete the localization of y 0 .

Fig. 9 .Fig. 10 .

 910 Fig. 9. Haptic feedback along the y axis while aligning the tip with the sphere: position of the tip and force computed with k y = 10 • 10 6 N.m -1 .

  Fig. 11.Measured normal force responses from both microcantilevers during the pick-and-place manipulation of a microsphere: (i) Pick-up occurs; (ii) The microsphere is detached from substrate after the pull-off; (iii) The microsphere snaps into the substrate; (iv) The gripper/microsphere pulls-off; (v) The gripper snaps into the substrate; (vi) Manipulation ends with slight bending of the microcantilevers.

ZFig. 12 .

 12 Fig. 12. Haptic feedback assistance for pick-and-place operation: (A) The pick-and-place operation begins, the sphere is on the substrate; (B-C) The sphere pulls-off; (D) The sphere is maintained above the substrate at a given vertical position; (E) The sphere is moved toward the substrate; (F) The sphere is placed down on the substrate.

z < z PO : 1 B

 1 -C: Pull-off happens.

z

  = z PO : 1 → 2 C-D: The sphere is detached from the substrate.

F

  as hz is repulsive, the spring k z starts to counterbalance it. z > z PO : 2 D: The sphere is manipulated at around z D . F as hz = 0: the virtual spring counterbalances the residual repulsive force at z D (> z PO ). z = z D : 2 D-E: Placing operation begins.

  F ashz is repulsive and increases: the user pushes down the sphere towards the substrate.z D > z > z PO : 2 E:The sphere reaches to z PO .

Fig. 14 .

 14 Fig. 14. Teleoperated 3-D microassembly demonstration of a micropyramid: (a-d) Four photos intercepted from the assembly process of the first layer of the micropyramid; (e and f) Assembly process of the second layer (the fourth microsphere) of the micropyramid. The top view photos (a)-(f) are captured under magnification of 20×. (g) Microassembly result magnified 100×.

Fig. 13 .

 13 Fig. 13. Task planning of the micropyramid assembled by four microspheres. The right-side diagram shows the assembly protocol, in which microspheres 1, 2, and 4 are assembled by pick-and-place manipulation while microsphere 3 is pushed to its target position.

Fig. 15 .

 15 Fig. 15. Haptic feedback during the microassembly operation of the fourth microsphere: (i) Pick-up occurs; (ii) The microsphere is detached from the substrate; (iii) The microsphere snaps into the first layer of microspheres; (iv) The gripper/sphere pulls-off; (v) Grippers snap into the first layer of microspheres.

•

  F op , F h : User and haptic force, respectively • H(s): Haptic device transfer function • F hx , F hy , F f e hz , F as hz : Haptic force on x, y, z (either for feeling or assistance) axes • α a , α f , α d : Amplitude, force and displacement scaling factor, respectively • k x , k y , k z : Haptic stiffness for the x, y and z force feedback • n, d: Number of points and distance between two points for the virtual guide generation • z d : Equilibrium position for transport when the haptic feedback is providing assistance Photodiode: • V I , V II : Voltage output of Photodiode I, II • A I , A II : Amplitude output of Photodiode I, II • F I , F II : Force output of Photodiode I, II • k n : Stiffness of the cantilever • S n : Sensitivity of the photodiode • β : Calibrated conversion factor of the photodiode from voltage

  • p h = [x h , y h , z h ]: Coordinates of the haptic handle • p n = [x n , y n , z n ]: Coordinates of the nanostage • p p = [x p , y p , z p ]: Coordinates of the piezotube • z PO : z position of the pull-off • x CP : x position of the tip at full contact • h 0 : z position of the tips while aligning the gripper

TABLE III ASSISTANCE

 III BASED ON HAPTIC FEEDBACK FOR PICK-AND-PLACE OPERATION

This work was supported in part by the ANR (French Agency of Research) through the PACMAN Project, and the NANOROL Project under Grant No. PSIROB07-184846. The authors are with Institut des Systèmes Intelligents et de Robotique, Université Pierre et Marie Curie -Paris 6, CNRS UMR 7222, 4 place Jussieu, 75005 Paris, France. {bolopion, xie, haliyo, regnier}@isir.upmc.fr

http://www.forcedimension.com