Yannick Wend

Kuni Zoetgnande
email: yannick.zoetgnande@univ-rennes1.fr

Jean-Louis Dillenseger
email: jean-louis.dillenseger@univ-rennes1.fr

A generic interpretable fall detection framework based on low-resolution thermal images

Keywords: Thermal images, Fall detection, Stereo vision, Deep learning

In this paper, we addressed the particularly challenging problem of fall detection using very low resolution thermal images. We proposed a new method for fall detection only based on the matches and a determined threshold. By classifying a pair of matched points on the ground or not on the ground, we could easily determine how many percent of the shape of a person is on the ground. Thus, we could determine if there is a fall or not. The experiments show that the method is able to classify features of human silhouette as one the ground or not on the ground.

Introduction

According to the French National Institute of Demography, the number of elderly people will continue to increase and will double by 2050 (for example, the number of people over 75 years old will increase from 6 million in 2020 to 12 million in 2050). In this context, even if infrastructures are created to welcome them, it becomes essential to find solutions allowing the elderly to live at home, as long as possible and with as much autonomy as possible.

In this context, falls are to be monitored in particular, since they are the first cause of mortality, apart from disease, for individuals over 75 years of age and therefore represent a real societal issue. This surveillance concerns two aspects: detection and prevention of falls. Detection of a fall allows for rapid assistance to be given to the individual. Prevention reduces the number of falls and delays the onset of the first fall. In our case, prevention consists in monitoring the activity of the person and deducing signs of fragility by analyzing the evolution of this activity.

One solution is to propose a new low-cost device based on thermal sensors to prevent the risk of falls by analyzing the activity of seniors. These types of sensors have been chosen because they meet different characteristics of acceptability by the monitored persons : the device is purely passive, these sensors allow a day and night operation and ensure the anonymity of the observed persons.

As the price of the device is an important criterion for the deployment in the living places of the elderly, it was chosen to use low-cost thermal cameras. The main drawback, however, is that lowcost cameras have poor image resolution (80 × 60 pixels in our case), and the images themselves are poor in information. The information is purely two-dimensional, yet fall detection and/or activity tracking require an accurate estimation of the person's pose in the room. So we decided to associate two cameras in stereoscopic way.

Our contributions are two-fold:

• A complete interpretable framework that allow to performs 3D vision from very low resolution thermal image

• An hybrid solution (vanilla machine learning + deep learning) for fall detection

The papers is structured ad follows: Section 2 describes the device we used to detect falls and recognise activities, Section 3 presents a new calibration method for thermal cameras, Section 4 details our stereo-vision pipeline with features extraction and matching, Section 5 presents a generic framework for fall detection and Section 6 concludes the paper and presents some perspectives. In order to detect falls, we decided to combine two thermal cameras in stereo mode. But if thermal cameras with a good resolution are relatively expensive, recently, some manufacturers have proposed very low cost thermal cameras, such as the Lepton 2 from FLIR, which we chose in our study. On the other hand, these low-cost cameras have a very low resolution (80×60 pixels), produce noisy images with drifts in the level of values over time (the cameras correct this drift from time to time, which has the effect of an abrupt temporal jump in the values of the image) and do not have a temperature calibration.

Stereo vision device

The acquisition system is composed of a pair of FLIR lepton 2.5 cameras (Table 1). The horizontal field of view is 51°and the diagonal field of view is 63.551°. The maximum frame rate of the cameras is eight frames per second. The two cameras are not synchronized. They are placed in parallel (their optical axes are parallel) in order to favor the field of view. The distance between the two cameras (the base line) was defined at 16 cm. The device will be placed high up (on the ceiling or on a wall just below the ceiling) and directed so as to monitor an entire room.

The two cameras are controlled by a micro-controller card. This card allows to recover the images on a PC by USB and in the future to make the interface with the embedded processing card.

Camera calibration

Conventionally, the calibration step consists in finding the parameters of a calibration model from real points given by a calibration grid. In our case, we chose the pinhole camera model and we used the functions of the OpenCV library to estimate the different parameters of the model. However, we were faced with two problems: 1) making a calibration grid suitable for the physical properties measured by the sensor and its resolution and 2) ensuring that despite the low resolution of our cameras, the process calibration is robust, i.e. precise and reproducible.

Calibration grid

The calibration grid must give temperature information. As the calibration process can be quite long, we opted to manufacture a grid whose points are actively heated. We have placed 36 bulbs, in

Robust calibration

The robustness of the calibration will depend, on the one hand, on the precision of the estimation of the points of the grid in the images and, on the other hand, on the number of different acquisitions of this grid.

Due to the low resolution, we replaced the pixel-precise point extraction module offered in OpenCV by our own method where the center of the light halos is located to sub-pixel precision. The other parameter that will influence the accuracy of the calibration is the number of grid images used to estimate the model parameters. Too small a number not only results in an inaccurate but also non-reproducible calibration. In this case, the parameters estimated by two successive calibrations can be quite different. On the other hand, too large a number of images considerably increases the handling during the acquisition of the grid. We carried out a bootstrap-type study in order to assess the evolution of the precision and reproducibility of the calibration as a function of the number of images. We concluded that for our protocol, the acquisition of 35 image pairs was necessary and sufficient to provide an accurate and robust calibration.

Extraction and matching of feature points in subpixel precision

Once the cameras have been calibrated, 3D vision requires 3 steps:

1. the features extraction from the two images 2. the features matching to estimate the disparity between the two images 3. the triangulation to estimate the 3D position as a function of the disparity.

A good reconstruction then depends on the number of features extracted from the images and on the precision of the estimate of the disparity between the points seen by the two images. In order to maximize these two criteria, we have proposed the following diagram (Fig. 3). Fig. 3. Suite of treatments for matching to a sub-pixel precision: 1) robust features extraction; 2) robust matching to pixel precision; 3) more accurate estimation of the disparity at subpixel precision.

Features extraction

As we mentioned previously, thermal images are very little textured compared to images from cameras working in the visible spectrum. It is therefore much more difficult to extract a large number of landmarks (corners, edges) from these images. In addition, the low resolution makes these landmarks much more blurry, which leads to a much less precise localization. The literature gives us several classical methods for RGB images: the Harris corner detector [START_REF] Harris | A Combined Corner and Edge Detector[END_REF], KLT [START_REF] Tomasi | Shape and motion from image streams: a factorization method[END_REF], FAST [START_REF] Rosten | Faster and better: a machine learning approach to corner detection[END_REF], BRIEF [START_REF] Calonder | BRIEF: Binary Robust Independent Elementary Features[END_REF], the phase congruency [START_REF] Morrone | Feature detection from local energy[END_REF][START_REF] Kovesi | Image Features from Phase Congruency[END_REF]. Some authors have proposed to adapt or simplify these methods for the extraction of landmarks in the case of thermal images (but on images of greater resolution). Hajebi et al. demonstrated in their paper that phase congruency could extract more characteristic points than other [START_REF] Hajebi | Sparse disparity map from uncalibrated infrared stereo images[END_REF] methods. We therefore explored and adapted this method to our images.

In [START_REF] Kovesi | Phase Congruency Detects Corners and Edges[END_REF], the authors modified a bit the equation to circumvent the previous version drawbacks:

PhaseCong 2 (x) = W (x) ∑ n A n [ϒ n -Ψ n ] -T ∑ n A n (x) -ε (1)
where

ϒ n = cos(φ n (x) -φ (x)) ; Ψ n = sin(φ n (x) -φ (x)), W (x)
is the frequency spread weighting, T is a noise threshold and ε is a small value (eg. 1-e3) to avoid division by 0.

The equation ( 1) can be extended to the two-dimensional image domain by applying it on several orientations θ after filtering the image by a bank of Log-Gabor filters. To reduce the computation cost due to the number of orientations and scales of the Log-Gabor filters bank, we implemented the variant proposed by [START_REF] Wang | Image feature detection based on phase congruency by Monogenic filters[END_REF] using a monogenic filter instead of Log-Gabor. The monogenic signal is a Riesz transform concatenated with a 2D signal.

As results of the 2D extension, we get a set of PC(θ ), the phase congruency at orientation θ . The features are then estimated by combining all the PC(θ ). This is done by computing the following values [START_REF] Kovesi | Phase Congruency Detects Corners and Edges[END_REF]:

a = ∑ (PC(θ )cos(θ )) 2 (2) b = 2 ∑ (PC(θ )cos(θ ))(PC(θ )sin(θ )) (3) c = ∑ (PC(θ )sin(θ )) 2 (4)
Combining a, b and c gives a hint of the strength of the feature. More precisely, the maximum moment M and the minimum moment m can be estimated by:

M = 1 2 (c + a + 2 b 2 + (a -c) 2 ) (5) m = 1 2 (c + a -2 b 2 + (a -c) 2 ) (6)
These moments are used to characterize the features. Higher is the maximum moment more significant will be the feature, and higher is the minimum moment more probably this feature point will be a corner. Because of the lack of information in images, we only took M into accounts. So a feature was considered significant if M was higher than a threshold γ.

In short, phase congruency reflects the behavior of the image in the frequency domain. It has been noted that contour or corner type elements have several of their frequency components in the same phase. The idea is then to seek the congruence of phases at different orientations and scales.

For each pixel, a maximum moment is estimated by combining the different congruences. A pixel having a maximum moment greater than a threshold γ will be considered as a feature.

An evaluation carried out on our low resolution images made it possible to demonstrate that: the phase congruency made it possible to extract more features than the other classical techniques and that the phase congruency was insensitive to sudden temporal variations in the intensity of the images . We also found that by varying the value of the threshold γ, we could obtain either features which are robust but few in number, or many features but at the expense of robustness.

Features matching

The calibration of the cameras makes possible to rectify the images. The matching is then simplified by finding similar points along the epipolar line. Similarity is estimated using the Lades similarity measure [START_REF] Lades | Distortion invariant object recognition in the dynamic link architecture[END_REF] conducted in a 5 × 5 window. The matching is then confirmed using consistency criteria between the images (constraints of uniqueness or order of the points along the line, similar orientation of the landmarks, bijectivity of the matching between right and left views, etc.). We visually assessed the suggested matches for 15 pairs of images and found less than 1% of incorrect matches.

Estimate the disparity at a subpixel level

Matching allows you to estimate the disparity for each pair of points to pixel precision. This precision is however insufficient because we showed that in the configuration of our assembly, a matching error of 1 pixel led to an uncertainty of more than 50 cm in depth. Phase correlation is a classic method of estimating a translation by finding the position of the maximum of the correlation peak. The idea is then to model this peak using a cardinal sine in our case and to estimate the position of the maximum at a sub-pixel level by fitting the model to the data. After estimating the best computational window size for the correlation, we evaluated our method on high resolution thermal images and on images acquired by our cameras. On high-resolution images, more than 99% of the dots were matched with an accuracy of less than 0.5 pixels. However, the match rate would decrease if higher precision was desired: 90% of matched pixels with an error less than 0.25 pixels, 55% with an error less than 0.1 pixels and 33% with an error less than 0.05 pixels. We observed identical behavior on our low-resolution images with match rates of 97%, 83%, 55% and 34% for errors less than 0.5, 0.25, 0.1 and 0.05 pixels, respectively.

3D reconstruction

As we had no ground truth we simply compared the 3D reconstruction of the points extracted, matched and reconstructed by triangulation using our method (phase congruency and phase correlation) with those reconstructed using more classical functions provided by OpenCV (features extraction using ORB [START_REF] Karami | Image matching using SIFT, SURF, BRIEF and ORB: performance comparison for distorted images[END_REF], matching using KNN) (see Fig. 4). We clearly see that our method reconstructs more 3D than the classical method. If we look at the distribution of the depth in z of the reconstructed 3D points for a person who is about 3 m from the cameras: ORB + KNN places the points at 1523.94 ± 1612.76 m while our method gives 3648.10 ± 256.43 m. Our measurement seems more precise and provides points with less dispersion.

We also applied this method to hot spots placed on the ground. The idea was then to determine the ground plane in order to detect falls by analyzing the distance between the points extracted from people and the ground plane. Unfortunately, even though our method gave more precise 3D points, the scatter of the points was too large to estimate the ground plane robustly.

Fall detection

Classical stereovision methods based on 3D reconstruction have not been able to reconstruct the ground plane, nor to estimate the pose of the person from our pairs of low-resolution thermal images. This, despite the fact that we had developed a robust calibration method, and pattern extraction and matching at a sub-pixel level of precision. We then thought of another strategy: is it possible to define the relationship between a 3D point and its projections on the two images of the stereo pair, not analytically by a model, but by learning?

The idea is then to use a convolutional neural network or a more classic classifier and to teach it whether a 3D point seen by the two images of the pair is on the ground or not.

The fall detection procedure is as follows (Fig. 5):

1. A matching procedure. We have taken the procedure described in the chapter "Detection of falls by stereoscopic reconstruction" with:

(a) features extraction by phase congruency;

(b) A simple segmentation of these points to keep only the points potentially belonging to a silhouette. This segmentation is easy on thermal images because of the heat given off by people;

(c) features matching by simple similarity measurement and some consistency criteria between the images. Note that, since we have not calibrated the cameras, we cannot use epipolar geometry. On the other hand, due to the lack of information in thermal images, there are ultimately relatively few points to match.

2. The classification of the paired points in "3D point on the ground" / "3D point above the ground" by inference of the network or the classifier;

3. The detection of the fall by an analysis of classified points.

It should be noted that we do a fall detection based on the analysis of static images. We do not involve a temporal analysis at this level.

The key points of the procedure are then the choice of the network or the classifier, the learning of this network / classifier and the analysis of the classified points.

Dataset

The objective is to find out if a pair of matched points is on the ground or above. For this, we moved a hot spot (a lit bulb) on the floor and in the room space above the floor and we acquired the corresponding images (Fig. 6). The lamp is easy to segment as it is usually the 'brightest' object. We therefore have a first learning base composed of pairs of images of the lamp associated either with a class " 3D point on the ground " or with a class " 3D point above the ground ".

In order to simplify the learning, we also extracted the 2D position of the bulbs in the images (center of gravity in gray level in an ROI around the bulbs). We therefore have a second learning base made up of pairs of 2D coordinates associated either with a class "3D point on the ground" or with a class "3D point above the ground".

Our learning strategy allowed us to have a balanced data set containing as many points on the ground as there are non-ground points.

Choice of network or classifier.

We have explored two strategies depending on the information on features points: 2D coordinate pair or thermal image pair.

If 2D points are presented as their 2D coordinates in images, a simple SVM type classifier is sufficient. If the 2D points are in the form of thermal images, we have proposed a model based on deep learning, inspired by DenseNet (DGD) [START_REF] Huang | Densely connected convolutional networks[END_REF]. This solution has the advantage of associating the characteristics of the thermal image (resolution, noise, halo, etc.) in the learning process. For a person on the ground, we assume that a large portion of the outstanding points will be classified 'on the ground'. Likewise, for a person standing, sitting, or even lying in a bed or on a sofa, only a small part of the features points (those of the feet for example) should be classified on the ground. A simple threshold on the percentage of points classified on the ground should be sufficient to detect or not the fall.

Results

For training, we acquired 6000 images of the lamp on the ground and 6000 images of the lamp above the ground. This acquisition is done in less than half an hour because it suffices to move the lamp on the ground or in space with the cameras in video mode (8 fps). Such an acquisition must be done once for a given configuration of cameras.

The performance of our fall detector has been tested on four databases. Each database was made up of images acquired from a different person. In these images the people were in one of the following configurations: standing, walking, sitting, lying on a bed or a sofa or fallen on the floor. These databases were not balanced in the sense that there were relatively few drops compared to other activities.

First, we compared our two classifier strategies: SVM on 2D coordinates and DGD on image data. For this we applied our classifiers to the training data in the form of 5 replications of a 2-block cross-validation (5x2cv). For each replication, we randomly shuffled the data. However, we have learned and tested SVM and DGD with the same data respectively. The median classification rate of DGD (0.976) slightly exceeds that of SVM (0.965) (p = 0.00028). Taking the image information into account provides a slight gain in classification precision. However, this gain seems a little marginal compared to the complexity of the implementation of the DGD. We then set the parameters of our method (the phase congruency threshold which acts on the number of features and the decision threshold on classified points) on data of persons. As we knew the ground truth fall/no fall , we were able to evaluate and optimize the different settings in terms of sensitivity, specificity, precision and F1-score. The results obtained showed very good performance with scores greater than 0.9 for the various indicators.

On the other hand, we found that our method was able to distinguish a person lying down, a person on the ground. This case is generally difficult to discriminate for single-camera methods, even based on deep learning (Fig. 7).

Conclusion

In this paper, we presented a framework to perform fall detection for very low-resolution thermal images. The framework is composed of three main parts: Points classification, Stereo-pipeline and a threshold-based fall detection. We prove that phase congruency is not only adapted to feature extraction for thermal image but also that stereo matching performed on phase congruency magnitude space provide more and well distributed matches than a well-traditional method that is ORB.

We compared a linear SVM method with a network witch we had derived from DenseNet. By classifying if a point is on the ground or not, we could determine the percentage of an object which is on the ground. So by using an appropriate threshold, we could perform fall detection.

Fig. 1 .

 1 Fig. 1. The stereo system composed of two Lepton 2 cameras set in parallel .

Fig. 2 .

 2 Fig. 2. Calibration grid

( a )Fig. 4 .

 a4 Fig. 4. 3D points re-projected on an image plane.

Fig. 5 .

 5 Fig. 5. Fall detection procedure

Fig. 6 .

 6 Fig. 6. Left view of the image of a lamp

( a )

 a Person after falling to the ground. (b) 76.66 % points classified on the ground.

( c )

 c Person lying on a sofa (d) 0% points classified on the ground.

Fig. 7 .

 7 Fig. 7. Difference between a person on the floor (a and b) and a person lying on a sofa (c and d)

Table 1 :

 1 Characteristics of Lepton 2.5

	Dimension	8.5 x 11.7 x 5.6 mm
	Resolution	80 (h) x 60 (v) pixels
	Pixel size	17 µm
	Field of view	51°(h) x 37.83°(v)
	Thermal sensitivity <50 mK
	Accuracy	± 5°C
	Frame rate	9 Hz
	Dynamic range	-10 to 140°C
	Price	<200$

Acknowledgments

This work was part of the PRuDENCE project (ANR-16-CE19-0015) which has been supported by the French National Research Agency (ANR).


Marwa Zahid
email: marwa.zahid@ced.uca.ma

Youness Hadouch

Daoud M'barek Amjoud

Mohamed Mezzane

Khalid Gouné

Abdelhadi Hoummada

Anna G Alimoussa

Brigita Razumnaya

Zdravko Rožič

Kutnjak

M'barek Amjoud

Daoud Mezzane

Mohamed Gouné

Khalid Hoummada

Abdelhadi Alimoussa

Anna G Razumnaya

Brigita Rožič

Zdravko Kutnjak

Enhanced near-ambient temperature energy storage

Keywords: Lead-free, sol-gel process, energy storage, electrocaloric effect, ambient temperature, barium stannate titanate. Declarations

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction:

Earlier studies and applications using ferroelectric properties of perovskite structural materials were carried out with lead-based materials such as lead-titanium zirconate (PZT) [1] and magnesium niobate-lead-titanate (PMN-PT) [2]. Unfortunately, due to lead toxicity [3], these lead-based materials can be harmful to the environment or human health, hence the need to develop clean and environmentally friendly lead-free material with improved ferroelectric properties.

Barium titanate (BaTiO3, BT) and its derivatives were the first lead-free perovskite materials studied and used for dielectric capacitors since 1950 [4]. These materials are eco-friendly and can be used to develop new lead-free materials with high piezoelectric, electrocaloric and energy storage properties [5].When doped with isovalent cations, the electrical and dielectric properties of BT tend to improve. Many systems have been studied over the years, including the BCZT (Ba0.85Ca0.15Ti0.9Zr0.1O3) system, which consists of calcium cation substitution on the A site and zirconium cation substitution on the B site [6],and BST (Ba1-xSrxTiO3) where the strontium is doped in the A site [7].The BTS (BaTi1-xSnxO3) system, on the other hand, is designed by inserting tin cations Sn 4+ in the B site [5], [8], [9], [10]. Nonetheless, most of these materials tend to achieve significant energy storage and electrocaloric properties only around the Curie temperature when the ferroelectric to paraelectric transition occurs, i.e. in a narrow temperature range, limiting their applications. The emphasis is primarily on lead-free materials with diffuse phase transition that leads to a wider temperature range near ambient temperature, where the energy storage and the electrocaloric properties are improved [11].

In this regard, the Curie temperature has been reported to decrease from 120°C for the undoped BT system, to 40°C, 41°C and 37°C in the BTS systems [5,9,12]. As a result of the coexistence of the four phases, cubic, tetragonal, orthorhombic, and rhombohedral (R, O, T, and C) at the quadruple point near the ambient temperature (AT), defined as 25 °C by the International Union of Pure and Applied Chemistry (IUPAC), the BTS11 could achieve an extremely high dielectric constant, a high piezoelectric coefficient, enhanced energy storage, and a significant electrocaloric effect [9]. However, the dielectric peak remained noticeably sharp in all studied compositions. Gao et al. [12] conducted the study of the Ba(Ti1-xSnx)O3 ferroelectric material system with various compositions, and came to the conclusion that one aspect that requires to be addressed further is the poor temperature stability of the BTS system properties.

The most commonly used method for elaborating dense and performant BTS ceramics is the solid-state method, which produces non-homogeneously doped powders with a high risk of contamination and agglomeration, it also necessitates a high calcination temperature (1250 °C) [8]. However, the sol-gel method, which is also commonly used in thin film synthesis [13] [14][15] [16], is a simple and precise method that can produce a more uniform and fine powder with better stoichiometry control, resulting in improved dielectric and ferroelectric properties. and it also requires lower heating temperatures [17].

In the present study, structural, dielectric, ferroelectric and electrocaloric properties were investigatedin BaTi0.89Sn0.11O3ceramic (BTS11) and found to be enhanced around ambient temperature in a broad temperature range. Therefore, the BTS11ceramic can be a suitable candidate for ambient temperature energy storage and frigorific applications.

Experimental details:

Chemicals:

In order to prepare the BTS11 ceramics, barium acetate Ba(CH3COO)2, titanium isopropoxide (C12H28O4Ti) and tin chloride (SnCl2.2H2O) were used as raw materials, whereas acetic acid and 2-Methoxyethanol were used as the solvents. All reagents and solvents were of chemical grade were and used without any further purification.

Experimental methods:

The XRD patterns were obtained by X-ray diffraction using the Panalytical X-Pert Pro with Cu-Kα radiation (λ = 1.54059 Å) at room temperature ranging from 20° to 80°, while the surface morphology of the ceramic was examined using a scanning electron microscope (SEM, VEGA 3-Tescan). The density of the ceramic sample was determined by the Archimedes method. The micro-Raman Raman spectra were excited by the polarized radiation of an argon laser with λ = 514 nm and were measured using a Renishaw in Via Reflex Raman spectrometer equipped with an Edge-filter. Temperature-dependent micro-Raman measurements were performed by aLinkam THMS600heating and freezing stage with a temperature stability of 0.1 K. The dielectric properties temperature dependence was measured using an impedance analyzer (HP 4284A) in the temperature range of 20to 80°C, and at frequency ranges of 20 Hz-100 kHz. Differential scanning calorimetry (Perkin Elmer Jade DSC) was used to point out the different phase transitions in a temperature range of -20 to 100°C. The polarization-electric field (P-E) hysteresis loops were performed by CPE1701, PloyK, USA, with a high voltage power supply (Trek 609-6, USA) in a silicone oil bath at 100 Hz in the temperature range of 5-120 °C. Finally, the electrocaloric study was carried out by the indirect method using the previously recorded P-E hysteresis loops. A silver paste was used to electrode the BTS11 ceramic pellet for the dielectric and ferroelectric measurements.

Synthesis procedure:

BaTiSn11 ceramic was prepared via a sol-gel route. A suitable amount of barium acetate (Ba(CH3COO)2) was dissolved in glacial acetic acid, while tin chloride (SnCl2.2H2O) was dissolved separately in 2-methoxyethanol.

The two solutions were then mixed at room temperature. The mixture was then stirred for 1 hour after a stoichiometric volume of titanium isopropoxide was added. The solution was then poured drop by drop with ammonia until it formed a clear gel, which was dried overnight before being calcined at 1000 °C for 4 hours. The resulted powder was then pressed under uniaxial compression in pellets of 8 mm diameter and 0.4mmthickness that were sintered at 1350°C for 7h.All chemicals are of analytical grade and used without further purification.

Results and discussion:

Structural and microstructural analysis:

The XRD of the BTS11 calcined powder and the sintered ceramic are displayed in figure 1(a,b).Both spectra show a pure perovskite structure with no heterogeneous peaks, indicating that Ti 4+ has been completely replaced by Sn 4+ in the BTS11 solid solution.The enlarged view of the calcined powder's 45° peak reveals a distinct splitting, indicating the presence of the tetragonal phase (Figure 1-c). The presence of the orthorhombic phase is indicated by the formation of a triplet at 65° (Figure 1-d) [18][19] [20]. However, the enlarged viewsof the XRD peaks centred at 2θ=45° and 2θ=65° of the sintered BTS11 ceramicpresented in figure 1-e and figure 1-f respectively, indicate the presence of a single characteristic peak with a small shoulder towards lower angles. The merger of ( 200) and (002) peaks at 2θ=45°, and ( 202) and (220) peaks at 2θ=65° indicates the dominant cubic phase in the BTS11 ceramic at room temperature. Given the similarity of the total energies between the cubic phase (nonpolar phase) and the tetragonal, rhombohedral, and orthorhombic phases (polar phases), polar phases can be randomly generated, formed and embedded in the cubic phase matrix [21]. Thus, the Rietveld refinement of the dominant cubic phase with the space group Pm3m was performed using Fullprof suite software (figure 2-a).Table 1 compares the refined structural parameters of BTS11 ceramic at room temperature to those reported in the literature. As one can see in the table 1, the BTSx ceramics exhibit different phase structures depending on the tin composition(x), usually the cubic phase is remarked when x exceeds 10% [13], [19], [17], and the tetragonal or the orthorhombicphase is remarked when x takes a value below 10% [24], [25], [10]. Thus, a dominant phase of cubic is found in our BTS11 ceramic, which can be further demonstrated by Raman and DSC measurements. The SEM micrograph of the sintered BTS11 ceramic is presented in figure 2-b. The sintered ceramicexhibits coarse and large grains along with some smallparticles,resulting ina high density of 5.91g/cm 3 and an averagegrain distrubution of 18.5 µm.The small particles found in the BTS11 ceramic fill the gaps between the larger grains, improving densification and,eventualy,enhancing the dielectric and ferroelectric properties [26].

The DSC and the temperature dependence of Raman spectroscopy were performed for the BTS11sintered ceramic to acquire more information about the phase transitions.

The DSC curve of our sample presented in figure 3-b shows a unique, small and large endothermic peak near ambient temperature between 28°C and 41°C.Indeed, as reported by Gao et al. [12], when the composition xof Sn dopant increases in BTSx, the transition enthalpy atthe Curie temperature decreases gradually, and when it changes to the multi-phase point (x=10.5), the heat flow peak becomes nearly invisible, indicating that first-order transition changes to the near second order. Thus, the presence of only one small endothermic peakindicates that the three phase transitions (TR-O, TO-T and TT-C) are all overlapped in this large temperature range. Hence,due to the characteristics of the multi-phase boundary, highdielectric constant and ultra-high piezoelectric properties can be expectedin this temperature range [9,12].

Raman spercroscopy is a versatile method to probe the short range modes since itsstructural informations refers to a single TiO6 octahedra, whereas the symmetry from XRD is due to a group of unit cells of more than 10 nm 3 [27].

Thus, to confirm the multiphase co-existence in the BTS11, the temperature dependence of Raman spectroscopy was performed and presented in figure 3-a.Note that at 0 °C, the spectral response contains the polar modes with E and A1-symmetries typical for the tetragonal phase of the BaTiO3 ceramics. Although the spectra change rather gradually with temperature, we observe several important features that are interpreted as the coexistence of different phases characteristic for the BaTiO3 in a narrow temperature range. Below 30 °C, we note the presence of both bandsA1(TO1) at 150cm -1 and A1(TO3) at 250cm -1 ,characteristic of the tetragonal and cubic symmetries.Moreover, the relative intensities of E(TO2) and A1(LO2)/E(LO) bands at 305 and 715 cm -1 , respectively, which are more intense below 30°C,indicate the presence of the tetragonal phase. The band at 190 cm -1 assigned to the orthorhombic phase is also visible up to 30°C [28]. Besides, the three modes A1(TO1), A1(TO3)

and E(TO1) that identify the rhombohedral structure are also present up to 30°C. The disappearance of the characteristicmodes of all ferroelectric phases at 180cm -1 A1(TO1), 305cm -1 E(TO2), and 715cm -1 A1(LO2)/E(LO) above 30°C confirms thatthe ferroelectric-paraelectric phase transition happens around this temperature. Thus, the BTS11 ceramic above 30°C startstransitinginto cubic paraelectric phase.

For additional support of the latter conclusion, the thermal variation of FWMH of the A1(TO3) Raman mode has also been investigated and displayed in figure 3

Dielectric analysis

Figure 4 illustrates the temperature dependence of the dielectric constantεr and the dielectric loss tanδ at various frequencies in the BTS11 sample from 20 to 80°C. For example, the dielectric constant at 1kHz reaches a maximum value of εrmax=15460 and a dielectric loss of tanδ=0.055 at the Curie temperature that happens to be at very nearambienttemperature (Tc=28°C) which is far less from the temperature reported in other works of BTS elaborated by the classicsolid-state method [30][31](Table2). The high value of εr may be due to the tricritical phenomenon occuring at the multi-phase point (x=11%) as well asthe elaboration method, the sol-gel method,which results in homogeneous powder particles and higher density BTS11 ceramic (Table 1).

The presence of only one anomaly, centered at 28 °C, suggests the existence of the multiphase (R-O-T-C) in the large temperature range of 20-42°C as predicted by the differential scanning calorimetry and the Raman spectroscopy.

The critical behavior of our BTS11 sample was quantified using the three equations below:

0 0 1 ( ) r T T T T C ε - =  (Eq.1) m dev m T T T ∆ = -(Eq.2) ( ) 1 1 (1 2) m r m T T C γ γ ε ε - - =   (Eq. 3)
Fig. 4 Temperature dependence of dielectric constant and the dielectric loss at various frequencies of the BTS11 ceramic sintered at 1350 °C / 7h.

Where T0 is the Curie temperature, C is the Curie-Weiss constant, ε m is the maximum dielectric constant and Tm is the corresponding temperature, and γ is the degree of relaxation.

The degree of relaxation γ determines the type of the phase transition, it takes 1 as value when the material is a normal ferroelectric, and 2 when it is a relaxor, while it takes a value in between (1<γ<2) when there is the presence of an incomplete-diffused phase or relaxor-type phase transition [30,32]. The fitting results of the third equation (Eq.3) displayed in figure 5-b were used to quantify the value of the degree of relaxation. The summary of the dielectric results of our BTS11is displayed in Table2 with a comparison of the dielectric properties at 1 kHz of other BTSx ceramics. As we can see in Table 2, our BTS11 ceramic exhibits a high dielectric constant and a broad diffuse peak. In BTS11and BTS10.5ceramics prepared by solid-state and reported S. by Merselmiz et al. [5] and Z.

Luo [33] respectively, a high dielectric constants (17390) at Tm of 41 °C for BTS11 and (72000) at Tmof28 °C for BTS10.5 were reached. Those enhanced dielectric properties are explained by the fast thermodynamic energy provided by the presence of (C+R+T+O) multiple phase boundaries in the BTS ceramics. However, despite their high dielectric constant, sharp dielectric peaks were observed. The relaxation degree γ of our ceramic reached 1.428 which shows a rather broad (diffuse) dielectric constant peak compared to 1.34 reported by Merselmiz et al. reported important γ values of 1.536, 1.44 and 1.5 for a BTS13, BTS5 and BTS10 respectively but the dielectric constants were relatively low [23], [30], [34]. Hence, the enhanced dielectric properties found in our BTS11 ceramic synthesized by a sol-gel process is mainly due to the quasi quadruple point where all phases coexist.

Table3Comparison of the dielectric properties at 1 kHz of the BTSx ceramics.

Method

Ferroelectric properties and Energy storage performances:

The electric field dependent polarization performed at different temperatures from 10 to 100 °C under the applied field of 35kV/cm and a frequency of 100 Hz are shown in figure 6. The maximum value of the polarization (Pmax=14.72µC/cm 2 ), the remanent polarization (Pr=2.99µC/cm 2 ), the charge storage density (Qc=Pmax-Pr =11.73µC/cm 2 ) and the coercive field (Ec= 1.01 kV/cm) of the BTS11 ceramic arefound near ambient temperature in the range of 10 to 40 °C. Meanwhile, the P-E hysteresis loop is observed to be slim and gets even slimmer with the increase of the temperature from 10 to 100 °C due to the ferroelectric-paraelectric phase transition. The ferroelectric behavior can be noticed in a broad temperature range, even above the Curie temperature (Tc=28°C) up to 60°C, due to the diffuse phase transition confirmed by the dielectric measurements.

Fig. 6 P-E hysteresis loops of the BTS11 ceramic from 10 °C to 100 °C.

The temperature-dependent variation of Pmax, Pr, Qc and Ec is shown in figure 7-a. Pmax and Pr decrease with increasing the temperature due to the ferroelectric-paraelectric phase transition, whilst the coercive field takes a minimum value at 20°C, then stays approximately the same up to 80°C,after which it increases quickly with the increase of the temperature due to the conductive mechanism usually observed at high temperature and low frequencies [8].

The energy storage performance of the BTS11 ceramic was further investigated using three equations 4, 5 and 6.

The total energy density Wtot was calculated by integrating the electric field from 0 to Pmax, whilst the recovered energy density Wrec was determined fromthe integral of the electric field from Pr to Pmax. Finally, the lost energy density Wloss was the difference between Wtot and Wrec. (Eq. 5)

loss tot rec W W W = - (Eq. 6)
At ambient temperature, Wtot, Wrec and Wloss are169.8 mJ/cm 3 , 110.3 mJ/cm 3 , and 59.5mJ/cm 3 , respectively. The thermal evolution of the energy storage properties is shown in figure 7-b. As we can observe from the figure, Wtot decreases with the temperature increase, whereas Wrec keeps an approximately constant value of 120 mJ/cm 3 in the temperature range of 30 °C-70 °C, and then it starts decreasing gradually with increasing temperature. It is worthy to note that the enhanced thermal stability in the BTS11 ceramic makes our quasi quadruple ferroelectric material a solid candidate for energy storage applications in a large temperature domain from the ambient to 80

°C [12].

The efficiency is also estimated using equation 7.

(%) 100 100

rec rec tot rec loss W W W W W η = × = × + (Eq.7)
The efficiency is a crucial parameter that helps determine the energy storage performance of the sample, as shown in figure 7-a. Initially, η increases quickly with increasing the temperature to78% at the ambient temperature ( 25°C), then it increases very slowly to reach a maximum value of 84% at 60°C, and finally it starts decreasing gradually at 90 °C proving significant thermal stability of the BTS11 ceramic. Table 3 sums up some energy storage properties of other barium titanate based ceramics. The results concerning the BTS, BST and BCZT ceramics reported in previous works (Table 3)are conducted under a high electric field and/or at high temperature, which is not a requirement for energy storage devices, the main goal is to obtain a slim hysteresis loop preferably at very near ambient temperature which is the case of our material, which makes it a good candidate for energy storage applications [12], [36], [5], [20].

Indirect electrocaloric measurements

The electrocaloric measurements in the BTS11 sample were done indirectly using the maxwell equation (Eq. 8), which exploits the hysteresis loops' thermal evolution. The evolution of the polarization P as a function of the temperature is shown in figure 8-a.P reaches a maximum value at 10 °C then decreases with a further increase in temperature due to the transition from the ferroelectric phase to the paraelectric phase, as previously mentioned. The enhanced ΔT values of 0.86 and 0.64 K are obtained at 41 and 37 °C under 35 and 25 kV/cm respectively. As commonly reported in the literature, interesting electrocaloric response is usually observed at the quasi quadruple point where the total number of polar states is significant (12 polar states in an O phase, 8 polar states in a R phase, Fig. 8 Thermal evolution of (a) the polarization and of (b) the adiabatic temperature of the BTS11 ceramic at different applied electric fields. and 6 polar states in a T phase),given that the entropy of an electrocaloric materiel depends on the total number of the polar states [39].Thus, in the quasi quadruple BTS reported in our work and also in the results reported by Z. Luo et al. [33], high adiabatic temperature is noticed under low electric field at low temperature. When the tin content is slightly below the quasi quadruple point (x<10.5), higher temperature is required to reach a high ΔTmax.

For instance, in BTS8 and BTS10 systems reported by W. [43]. Meanwhile, in Ba0,95Sr0,05(Ti0,95Zr0,05)0.75Sn0.25O3,ΔT reaches only 0.1 K under 10 kV/cm at 90 °C [44], and in Ba0.85Ca0.15Ti0.90Hf0.10O3, the adiabatic temperature attains0.74Kunder 35 kV/cm but at high temperature (123°C) [45]. Therefore, the BTS11 is a promising ambient temperature electrocaloric material compared with other lead free system.

Furthermore, the electrocaloric responsivity ζ calculated by equation 9 (Figure 9) is used for further analysis of the electrocaloric behavior of the BTS11ceramic. The maximum value of ζ (0.24K.mm/kV) is obtained at near ambient temperature, and its evolution as a function of the temperature in a broad temperature range is similar to the adiabatic temperature evolution. These results make our BTS11 material a potential candidate for practical nearambient temperature frigorific applications.

/ T E ξ = ∆ ∆ (Eq. 9) Table 4 summarizes the electrocaloric properties in our sample in comparison with similar and other lead-free materials at different applied electric field values.

Conclusion

To summarize, we have investigated the structural, dielectric, ferroelectric, and energy storage and electrocaloric properties at very near ambient temperature in the eco-friendly BTS11 ceramic synthesized by the sol-gel method.

As a result, a high dielectric constant of 15460, high recovered energy of 122 mJ/cm 3 with 79% efficiency and a significant electrocaloric effect (ΔT=0.86 K, ζ=0.24K.mm/kV under 35 kV/cm)are obtained near ambient temperature with remarkable thermal stability due to the overlap of the three phases transition. Hence, the BTS11 ceramic can be considered an excellent candidate for energy storage and frigorific applications near ambient temperature.

Fig. 2

 2 Fig.2 (a) Refined XRD patterns of the sintered BTS11 ceramic and (b) SEM micrograph the BTS11 sintered ceramic.

Fig. 1

 1 Fig.1 XRD patterns of (a) the calcined BTS11 powder and its the enlarged views of 45° XRD peak (c) and 65° peak (d), (b) XRD patterns of the sintered BTS11 ceramic, and its enlarged views of 45° XRD peak (e) and 65° peak (f).

  Fig.3 (a) Temperature dependence of Raman spectra (b) FWMH of the A1(TO3) Raman mode and the DSC curve of the sintered BTS11 ceramic.

[ 5 ]

 5 in a BTS11 elaborated by solid-state. However, M. Cernea et al, M. Kumari et al. and Md. J. Ansaree et al.

Fig. 5

 5 Fig.5 (a) Thermal variation of 1/𝜀𝜀 𝑟𝑟 and (b) plot of modified Curie-Weiss law at 1 kHz of the sintered BTS11

Fig. 7

 7 Fig.7 Thermal evolution of (a) Pr, Pmax, Qc, and Ec and (b) the energy storage performance of the BTS11 ceramic.

Figure 8 -

 8 b depicts the thermal evolution of the adiabatic temperature of the BTS11 sample. The one thermally stable curve observed also confirms the existence of the multiphase as predicted by the DSC and Raman and confirmed by the dielectric measurements and shows the thermally stable electrocaloric performance in BTS11. The maximum value of ΔTmax under 12 kV/cm is reached at30 °C, whereas under 35 kV/cm, ΔTmax is attained at 41 °C, which is commonly observed in ferroelectric materials[39].

Fig. 9

 9 Fig.9 Temperature dependence ξ at different applied electric fields.

Table 1

 1 Refined structural parameters for the BTS11 ceramic at room temperature in comparison with other BTSxceramics.

	Peparation	Com		Calcinatio	Sintering	Cell	Densit	Refe
	method of	opsit		n	temperat	volum	y	renc
	BTSx	ion	Structure	Temperat	ure (°C)	e(Å 3 )		es
		(x)		ure	and time
				T°C/time	(h)
				(h)
	Sol-gel	11	Cubic	1000/4	1350/7	65.122	5.91	This
						4		work
			Pm-3m
	Low-	13	Cubic	800/2	1300/2	-	-	[23]
	temperatur
	e aqueous
	synthesis
	Solid-state	5	Tetragonal	1130/2	1330/2	-	5.7	[24]
			P4mm
	Solid-state	6.5	Orthorhom	1050/4	1350/4	-	5.2	[25]
			bic Amm2
	Solid state	10	Tetragonal	1100/4	1350/4	64.68	-	[10]
			P4mm
	Sol-gel	20	Cubic	1000	1350/2	-	5.43	[17]
	Solid state	12	Cubic	1100/2	1370/5	64.76	-	[22]

Table 4

 4 Comparison of the energy storage properties of our BTS11 ceramic with other barium titanate based ceramics.

	Ceramic	Method	Wtot	Wrec	η	E	T	References
			(mJ/cm 3 )	(mJ/cm 3 )	(%)	(kV/cm)	(°C)
	BaTi0.89Sn0.11O3	Sol gel	151	117	78	35	30	This work
	BaTi0.89Sn0.11O3	Sol gel	104.9	71.3	67.9	25	28	This work
	BaTi0.895Sn0.105O3	Solid	31	-	-	10	37	[12]
		state
	BaTi0.89Sn0.11O3	Solid	92.7	84.4	91.04	25	60	[5]
		state

  Kayaphan et al. and S. Qiet al., a ΔTmax of 0.52 and 0.48Kwere reached under an applied field of 20 and 15 kV/cm at 58 and 64 °C respectively[40],[41]. Furthermore, H. Zaitouni et al. yielded an adiabatic temperature of 0.14 K in a BST system but at 108 °C[42], which is quiet comparable with what has been reported by H. Mezzourh et al. in the BCZT system

Acknowledgments

The authors gratefully acknowledge the generous financial support of the European Union Horizon 2020 Research and Innovation actions MSCA-RISE-ENGIMA (No. 778072) and MSCA-RISE-MELON (No. 872631) and ARRS project J1-9147 and program P1-0125 and CNRST Priority Program PPR 15/2015 and the ministry of Science and Higher Education of the Russian Federation, grant agreement No 075-15-2021-953.

Funding

This work was supported by CNRST Priority Program PPR 15/2015, by the European Union's Horizon 2020 research and Innovation actions MSCA-RISE-ENGIMA (No. 778072) and MSCA-RISEMELON (No. 872631) and ARRS project J1-9147, and by the ministry of Science and Higher Education of the Russian Federation, grant agreement No 075-15-2021-953.

Availability of data and material

Not applicable

Code availability

Not applicable

Conflicts of interest/Competing interests

Not applicable

Ethics approval

Not applicable

Consent to participate

We confirm that all authors mentioned in the manuscript have participated in, read and approved the manuscript and have given their consent for the submission and subsequent publication of the manuscript.

Consent for publication

We confirm that all the authors mentioned in the manuscript have agreed to publish this paper.

Relevance Summary

In the submitted manuscript, I am reporting, the successful synthesis of barium stannate titanate ceramics by solgel method, and the study of their structural, dielectric, ferroelectric, energy storage, and electrocaloric properties.

In addition, we confirm that this work is original and has not been published elsewhere, nor is currently under consideration for publication elsewhere.


Sophia Sakr

Barthélemy Cagneau

Thomas Daunizeau

Stéphane Régnier

Sinan Haliyo

Haptic Remote Control Interface for Robotic Micro-Assembly at Low Frequency

Microassembly of submillimetric objects is still a manual process in most industries. Manufacture of MOEMS (Micro-Optical-Electrical-Mechanical System) based sensors, or watchmaking and even micro-surgery relies very heavily on the motor skills of an operator handling specialized tools. We propose here a preliminary work on bilateral coupling between a macro-tweezers and a micro-tweezers using a user interface for a microrobotic system which ambitions to combine the precision of the robot with the expertise of the craftsman. It strives to hide the scale-change between the microcomponents and the user's workbench, by providing a scaled-up tabletop image of the robot's sample-holder, co-located with a hand-held tool. This active tweezers mimics the traditional tool in form and function, albeit augmented with feedback. Its motions are tracked to drive the microgripper. Users may hence directly pick-up a micro-object from the image, while feeling the grasp of the microgripper on their fingers.

I. INTRODUCTION

Precision manipulation of millimeter-range components is an intricate task, still mostly done manually. For example in watchmaking, the know-how and dexterity of craftsmen and their experience are unique assets. As in most cases in surgery, full-automation without operator intervention is currently unreachable. However, the requirements of selfdiscipline and long training make new operators difficult to recruit. Moreover, novel MOEMS components (Micro-Optical-Electrical-Mechanical System) appear with dimensions below the human capabilities. Consequently, robotic solutions are called for assist the operator and push down the dimensions of assemblies.

Teleoperation is a robotic technique first proposed for macroscale industrial applications [START_REF] Vertut | b. teleoperation and robotics: applications and technology[END_REF]. Coupled with haptic feedback, an operator can achieve the task better and faster, or intervene in remote, and potentially hostile environments while taking advantage of his learned motor skills [START_REF] Wildenbeest | The impact of haptic feedback quality on the performance of teleoperated assembly tasks[END_REF]. This strategy is particularly adapted to mentioned applications. However, the idea of combining the skills of the operator to the dexterity of a robotic system in teleoperation does not always hold, especially in the case of micromanipulation. The user interface generally consists of 2D screens, of input *This work was supported by the French National Research Agency through the ANR-Colamir project (ANR-16-CE10-0009), and partially by the French government research program "Investissements d'avenir" through the Robotex Equipment of Excellence (ANR-10-EQPX-44). 1 Sophia Sakr, Thomas Daunizeau, Stephane Regnier and Sinan Haliyo are with Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique (ISIR), F-75005 Paris, France {sophia.sakr, sinan.haliyo}@sorbonne-universite.fr France barthelemy.cagneau@uvsq.fr devices like a mouse, joystick, buttons, and eventually of a generic haptic device [START_REF] Millet | Haptics and graphic analogies for the understanding of atomic force microscopy[END_REF], [START_REF] Bolopion | Haptic feedback in teleoperation in micro-and nanoworlds[END_REF]. A skilled technician considers unconsciously their tools as an extension of themselves and it has an important impact on the workflow [START_REF] Miller | Sensing with tools extends somatosensory processing beyond the body[END_REF]. Teleoperation interfaces hence imply a fundamentally different workflow from manual manipulation and do not translate well, especially in the case of a senior technician. This human-machine interface issue is also at the heart of a general problem of insertion of robotics in microassembly workshops or in medical applications.

Consequently, the problem has been investigated in the literature form different angles. A simple approach is to equip a grounded haptic device with an 'active gripping extension' instrumenting the two-finger pinch, as in some commercially available interfaces [START_REF] Tobergte | The sigma.7 haptic interface for mirosurge: A new bi-manual surgical console[END_REF] or in prototypes designed as a part of complete teleoperation chain [START_REF] Li | Haptic device with gripping force feedback[END_REF]. However, grounded devices are quite cumbersome and do not sit well on a microassembly workbench. Other solutions like wearable devices [START_REF] Pierce | A wearable device for controlling a robot gripper with fingertip contact, pressure, vibrotactile, and grip force feedback[END_REF], [START_REF] Choi | Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality[END_REF] or gloves [START_REF] Hinchet | Dextres: Wearable haptic feedback for grasping in vr via a thin form-factor electrostatic brake[END_REF] are proposed to feedback the gripping force to human hand. These approaches strive to emulate the hand grasping directly as the remote object, whereas precision manipulation uses tools. In addition, a tool can be easily put down momentarily, whereas it is more tedious to take off a wearable device. Hand-held solutions have hence a particular allure.

Some works adopt well-known designs such an augmented joystick or a gamepad, hence they are solely used as remote controllers [START_REF] Choi | Claw: A multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality[END_REF]. On the other hand, traditional tools of the precision work, tweezers or forceps, are among humanity's oldest instrument. Nowadays, they still retained their function and form. There are very few changes between modern specimens and the ones found by archaeologists in Egyptian pyramids or in Mesopotamia [START_REF] Adamson | Surgery in ancient mesopotamia[END_REF]. A robotic evolution of this design would tackle the acceptability issue while retaining the operator's expertise. A prototype based on real tweezers augmented with a voice-coil actuator is proposed. However, it is designed as an extension to a grounded haptic device [START_REF] Rizun | Mechatronic design of haptic forceps for robotic surgery[END_REF]. Tweezers with voice-coils are also explored [START_REF] Burdet | Design of a haptic forceps for microsurgery training[END_REF], [START_REF] Ng | Stiffness perception during pinching and dissection with teleoperated haptic forceps[END_REF] but this particular actuation exhibits some limitations, especially in heat dissipation.

Lately, we have developed compact hand-held active tweezers designed from the ground for microassembly [START_REF] Sakr | An ungrounded master device for tele-microassembly[END_REF], [START_REF] Daunizeau | Mechatronic forceps[END_REF]. This device offers a good compromise between force output, power requirements and thermal behavior by the mechatronic choices. It is designed to be used either as a replacement of classical tweezers with assisted force control on gripping or as a master device for remote operation with a microassembly robot. This manuscript focuses on this latter usage. The setup including the master device and the slave microassembly robot are presented next. A particular coupling strategy is crafted to work around the limitations of the communication layer and is then implemented. A grasping operation demonstrates the proposed teleoperation chain.

II. MICROASSEMBLY STATION

Fig. 1 depicts the different components of the microteleoperation setup used in this paper. The master tool is a hand-held active tweezers. The slave system is a microassembly station 1 with 4 DoF and an instrumented microgripper.

A. Hand-Held Active Tweezers as Master Tool

The master device is a tweezers-shaped ungrounded interface which measures 14.5cm and weights 51.2g (Fig. 2). It has a single active degree of freedom which corresponds to the grasping, driven by a coreless motor (DCX10L, Maxon Motor) coupled through a backlash-free mechanism. This motion is reversible, letting the user operate the tweezers as a traditional one, with a possible haptic assistance from the motor. All electronic boards are embedded. It includes a wide range of sensors: strain gauges at the clamped end of the tweezers' branches allow for opening and contact detection. A force sensor under the fingertip collects the force applied by the user.

Device position and orientation are tracked externally by vision. In that respect, 2 infra-red cameras (Prime 13W, Optitrack) are placed at approximately 2m from the working space on both sides and a 3D-printed marker is mounted on the tweezers (Fig. 1.b). The tracking uses the constructor's own closed source software and occurs at 120Hz and has a precision of 0.1mm [START_REF] Sakr | An ungrounded master device for tele-microassembly[END_REF].

B. 4 DoF microManipulator as Slave Device

The microassembly station is a 4 DoF robot composed of a sample-holder mounted on 3 piezoelectric stick-slip stages (2 translations xy and 1 rotation ω). The microgripper is mounted on a vertical piezoelectric stick-slip stage z.

Translations have a resolution around 100nm while rotation exhibits a resolution of 500µ • . Furthermore, it is possible to change manually the orientation of the gripper around x axis. In the future it could be considered to add the two other rotations to get a 6 Dof micro-manipulator and reach the dexterity of a craftsman. Two cameras (top and side views) provide visual feedback (IDS UI-3590CP,18 Mpx 21fps). The microgripper is composed of 2 fingers consisting in piezoelectric beams with 2 strain gauges on the bottom and outer faces at their clamped end (fig. 5). The grasp is controlled by the voltage applied to both fingers. The gripping force f m u and the contact with the substrate are measured thanks to four gauges. The output of those sensors is not linear and presents a hysteresis as depicted in Fig. 4.

C. High-Level Control interface of the Microassembly Station

The microassembly station relies on a main controller unit and appropriate APIs to communicate and control individual entities of the system. Typically, all stick-slip actuators can be controlled, as well as microgripper input voltage. It feeds back the position as reported by internal encoders and the tension measured by a strain gauge at the microgripper.

Although quite straightforward to the interface, this controller does not allow for low-level functionalities and is limited by a frequency below 50Hz to communicate. We have also observed random jitter affecting the communication frequency by 40% in some conditions. Moreover, the microgripper input and output are measured in Volts. Those are not straightforward to convert to Newtons because of the hysteresis and lack of proper knowledge of physical parameters and low-level access.

III. TRAJECTORY CONTROL

The motion of the gripper, excluding the grasp, is controlled with the motion of the master device. An overhead projector projects the top-view camera in front of the user on his workbench, considerably enlarging the image (fig. 1

.b).

Side-view is shown on the monitor in front of the user.

The position of the master tool's tip is obtained from Optitrack system. A short calibration using 3 pre-defined points on the workbench gives the homothecy between the user's workspace and the robot's.

For motion control, the operator's position with respect to the sample holder is extracted. The microgripper is brought to this position using the robot's internal controllers. Only the rotation around the vertical axis is respected. Note that actually the movement is executed solely by the sampleholder, but its motion is compensated in the video feed before being displayed to the user, so from his point of view only the microgripper moves.

The only feedback during the process is visual, from top and side-view cameras. To alleviate the burden of controlling the 3 spatial translations simultaneously, the vertical axis is constrained. The microgripper is kept a few centimeters above the substrate so it can easily be positioned above the sample without risking a collision. The vertical motion is controlled explicitly when the operator requires it, with the master tool, or from the classical user interface. A single DoF haptic device is planned for this motion, relying on the vertical force sensing of the microgripper [START_REF] Ousaid | Feeling what an insect feels[END_REF].

This remote control strategy, as well as an alternative ratemode coupling, are explained in detail in [START_REF] Sakr | A handheld master device for 3d remote micro-manipulation[END_REF].

IV. HAPTIC COUPLING OF MICROGRIPPING

The system is composed of the microgripper described previously and used as the slave device whereas active tweezers operate as the master. The objective is to couple the gripping of microscale objects with the master tool to take advantage of the user's motor skills for force control of the grasp. Furthermore, in order to evaluate the impact of haptic feedback, two different modes of operation are compared:

1) The first one only relies on visual feedback. The opening of the microgripper controls the opening of the active tweezers with appropriate scaling.

2) The second one provides the user with force feedback from the microgripper. It is thus necessary to propose a controller to achieve force rendering on the master side.

A. Low Rate Communication Issue

Haptic devices for teleoperated gripping has already been explored [START_REF] Haliyo | Tele-manipulation by adhesion of micro objects[END_REF], [START_REF] Li | Haptic device with gripping force feedback[END_REF], [START_REF] Bolopion | Haptic feedback in teleoperation in micro-and nanoworlds[END_REF], [START_REF] Khurshid | Effects of grip-force, contact, and acceleration feedback on a teleoperated pick-and-place task[END_REF], [START_REF] Choi | Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality[END_REF]. Bilateral haptic coupling can be modeled as a two-port system and is prone to instabilities when passivity criteria is not met [START_REF] Bolopion | Analysis of stability and transparency for nanoscale force feedback in bilateral coupling[END_REF]. Additionally, a sampling frequency close to 1 kHz is expected for better results [START_REF] Reynier | Summary of the kinesthetic and tactile function of the human upper extremities[END_REF], [START_REF] Hayward | Haptic interfaces and devices[END_REF].

This requirement poses a serious issue with the current slave system, whose communication protocol is not designed for such use. The latency and bandwidth are well below mentioned values and initial measurements indicate that the average frequency is around 40Hz. This low bandwidth issue has been widely studied in the field of teleoperated systems [START_REF] Lum | Teleoperation in surgical robotics-network latency effects on surgical performance[END_REF]. Although some approaches exist to teleoperate at these frequencies, their trade-off is of a considerable complexity with a degraded perception because of the added damping.

We propose here an alternative approach to enable force feedback manipulation at low rate communication. The challenge is to render physical interactions at the master's side based on minimal data exchange. The proposed solution is to locally compute force feedback in the active tweezers loop as soon as contact detection is triggered at the slave's side. This clearly relies on minimal information exchange since only binary [contact / non-contact] information is required.

However, despite its apparent simplicity, several issues still need to be addressed. The first one is to detect the grasp in real-time from the slave's side. The second one is to make force rendering as most realistic as possible on the active tweezers. Naturally, a transparent rendering cannot be achieved because information like forces and positions are not shared in real-time.

We assume that providing binary grasp information through a predefined stiffness variation would provide sufficient additional stimuli for better user efficiency [START_REF] Kim | Multirate haptic rendering using local stiffness matrix for stable and transparent simulation involving interaction with deformable objects[END_REF].

B. Grasp Detection

The microgripper is controlled with an integrated proprietary microcontroller. It handles the tension applied to each arm and collects the strain gauges information through a Wheatstone bridge. It communicates with on-demand TCP packets via ethernet and keeps the command voltage constant between requests. This protocol is not adapted to real-time control as it does not care to minimize the latency nor to keep the periodicity. The approach proposed is to estimate the state [grasping / not-grasping] of the gripper by observing the output signal with respect to the input set-point.

The input voltage is roughly proportional to the microgripper's opening, f c . The output signal, stemming from the strain gauges, gives the deformation torque at the clamped end. When the beam is activated, its behavior is of a clampedfree cantilever. While in contact with an object, considering a perfect symmetry, the system is akin to a clamped-pinned beam. For the same load, this results in a higher strain and the clamped end as depicted in fig. 5.

Observing the variation of the difference of output signal f mu and input f c , it's been pointed out its time-derivative drops sharply at contact. On the opposite direction, a sharp increase occurs. A threshold t g value is empirically determined and grasp state is set accordingly:

if d dt ( f c -f mu ) < -t g → grasp (1) d dt ( f c -f mu ) > t g → release (2)

C. Stiffness Variation on Active Tweezers

Haptic perception of contact is interpreted as a change in the stiffness [START_REF] Laycock | A survey of haptic rendering techniques[END_REF]. Additionally, during fine manipulation with tweezers, Human's motor control loop is referenced on force, rather their fingers opening. At contact, a clamped-free to clamped-pinned transition occurs resulting in a rise of the apparent stiffness of the tweezers.

This behavior can be reproduced using the actuator of the master tweezers. Writing the contribution of natural stiffness of the device as k n × θ (with θ the opening angle), the actuator can be controlled to provide a desired apparent overall stiffness k d such as :

(k n + k a ) × θ with k d = k a + k n =⇒ W a = k a × θ
with W a the motor torque and k a a parameter which can be positive or negative. This simple torque control is quite straightforward to implement. Note that choosing a negative k a between -k n and 0 gives a lower apparent stiffness.

However, it is necessary to modify the apparent stiffness of the gripper without causing a discontinuity on the force rendered to the user. Assuming that in no-grasp state only the natural stiffness of the mater tweezers is felt and at grasp the apparent stiffness switches to k c , an offset value W o f f has to be set in order to keep torque W a (θ c ) at the same value, with θ c the contact position.

W a (θ c ) = k n × θ c = k c × θ c +W o f f (3) =⇒ W o f f = (k n -k c ) × θ c (4)

D. Coupling of grasping

Fig. 6 shows the scheme implemented to control the opening of the microgripper through the master tweezers and to modify its stiffness at contact. It relies on a Proportional-Integral controller that aims to regulate the opening of the active tweezers. The opening O p is monitored and compared to O p re f which denotes the initial opening of the active tweezers. The PI controller prevents the user to completely close the gripper by modifying the stiffness of the active tweezers. It tends to maintain constant the opening and, as a consequence, increases the force feedback. The gains define the dynamical change of the stiffness.

Remarkably, the control scheme in Fig. 6 does not rely on continuous data exchange. More precisely, it does not require continuous force measurement at the slave's side. However, Fig. 7. Three trials to grasp a screw with the master and the slave devices. Without feedback, the user closes the tweezer at its maximum, exerting unnecessary load on the microgripper. the output voltage f mu of the force sensor located on the piezoelectric beam is added to the motor's input through a gain G. This signal is used to settle between the forces arising at microscale and those applied to the user. Moreover, the reference used to close the gripper is also updated at the same low frequency to drive the microgripper tracking the master tool.

To summarize, the stiffness is controlled in real-time and at high frequency (around 3kHz) with the PI controller on the master side. Meanwhile, the forces arising between the microgripper and the object is added at a lower frequency (around 40Hz) to render effects that occur at microscale. The current master opening O p is sent to the microgripper as an input command.

E. Experimental Results

The experiment consists in grasping a screw with a 1.4mm radius. The active tweezers is controlled with the control scheme presented in Fig. 6.

In a first experiment, force feedback is disabled and the user grasps the screw and releases it. The results are presented in Fig. 7-a (dashed lines). Without the change in the apparent stiffness, hence without sensing the contact, the operator closes the master tool to its maximum, where he gets, at last, a stimulus by the touching of both branches. Meanwhile, the microgripper is pushed to its maximum power, exerting unnecessarily excessive force on the gripped object.

In a second trial, the feedback controller is used. Measurements are plotted with continuous lines in Fig. 7-a. When the contact is detected, based on the derivative value of the output's strain gauge, force feedback is enabled and the reference opening O p re f is recorded. As expected, the opening is controlled with respect to O p re f and the controller limits its value by modifying the stiffness of the active tweezers.

In a third trial following immediately the second, the user quickly re-grips the object without moving it. The results are presented in Fig. 7-b. The same data are plotted as well as the output voltage of the master force sensor (dashed line). With the structure proposed in Fig. 6, we cannot expect that the forces measured at microscale correspond to those applied on the user (including a gain for the scale factor). Remarkably, the shapes of the signals that represent the forces (slave strain gauges and master force sensor) are very similar. It means that, despite its particular structure, the coupling proposed throughout this paper is suitable to render forces measured with the microgripper. The main difference is when the user intends to release the object. The signal of the master force sensor quickly decreases and a delay is observed before the output voltage of the strain gauge also decreases. This is mainly due to the low frequency update of the reference of the microgripper as well as its inner controller. Nevertheless, this latency goes mostly unnoticed by the user who relies on the haptic feedback on his fingers.

In Fig. 7-b, some disturbances appear on the opening of the active tweezers. This phenomenon, which was not observed in Fig. 7-a, seems to indicate that the user is closing the active tweezers too fast. These peaks suggest that the transient state while switching force feedback on or off needs to be better controlled.

Those experiments show that force-feedback is successfully enabled even if information is exchanged at a low rate. The stiffness of the master device is virtually changed with the controller and the opening is controlled to provide the user with forces.

V. DISCUSSION

This paper presents a haptic remote control interface for an industrial microassembly robot using an innovative master tool. The ambition here is to provide a "cockpit" very much like the traditional workbench of a manual microassembly technicians, as in watchmaking or jewelry. The objective is the uttermost transparency of the robotization, as to let the operator draw advantageously from their learned motor skills.

The interface provides a tabletop display of a microassembly robot's work area, enlarged to the user's scale thanks to the projection. They operate directly on the projected scene using the active tweezers, which mimics successfully classic tweezers in form and function.

The position of the master's tip is assumed by the microgripper, co-locating both scale-spaces. A previous work on motion control using optical tracking has shown that position control was more precise but with a combination of both position and speed, it might be possible to be more efficient [START_REF] Sakr | An ungrounded master device for tele-microassembly[END_REF].

Here, the haptic control of the grasping is detailed. Dealing with low frequency communication due to the microassembly robot's API, a strategy has been developed. It has been proven that the grasp detection between the micro-object and the tweezers was robust. It is computed based on the behavior of the microgripper's strain gauges. Then, results revealed that despite the low frequency of the communication, rendering a stiffness change without destabilizing the user nor the system is possible.

Note that the proposed coupling has some "sloppiness": the microgripper is poorly characterized, and no calibration has been sought to tie the input and output tensions to physical quantities. The hysteresis is observed but is not compensated by signal processing. Moreover, the strain gauges, by their positions on the beams do not provide realistically the force applied by the branches, which depends if an object is grasped (clamped-pinned) or not (clamped-free). Furthermore, the control scheme was tested only on hard objects and not soft. Future works would have to explore the manipulation of different kind of objects.

However, handing the control to Human, and providing them with sensory input, off-loads all these shortcomings to their motor control skills. They are instinctively capable of dealing with hysteresis, and of evaluating successfully the amount of their mechanical action.

This strategy seems to pay off, and its confirmation is underway in a user-evaluation campaign with operators specialized in manual microassembly. For the comparison's sake, a state-of-art large-bandwidth coupling will be implemented by gaining low-level access to microgripper. The overall interface will also be run against a traditional screenand-joystick setup, eventually extended with an Omega.7 type haptics device.

Fig. 1 .

 1 Fig. 1. Experimental setup: a. Microassembly station; b. Active tweezers as master device; c. Microgripper and the sample-holder. The assembly station provides the vertical motion of the microgripper and the sample-holder 3 planar DoF. User operates on top of a real-time projected and enlarged image of the sample-holder where its motion is compensated.

1Fig. 2 .Fig. 3 .

 23 Fig.2. The active tweezers used as the master device. A DC motor actuates the gripping in both ways through a perfectly reversible mechanism. Several sensors collect crucial information (not shown in the picture: Accelerometer, Hall sensor, motor encoder).

Fig. 4 .

 4 Fig. 4. Characterization of the microgripper. Piezoceramic actuation and strain gauge's output exhibit a predictable hysteresis.

Fig. 5 .

 5 Fig. 5. Different stage of the piezoelectric beam deformation according to conditions at the tip. a) piezoelectric beam not actuated. -b) piezoelectric actuated: it bends as a beam recessed/free -c) piezoelectric beam closing until an object: behavior can be approximate to a beam recessed/simply supported.

Fig. 6 .

 6 Fig.6. Control scheme to regulate the active tweezers stiffness.

Barthélemy Cagneau is with the Université de Versailles, Paris-Saclay, Laboratoire d'Ingénierie des Systèmes de Versailles (LISV), 78140 Vélizy,

ACKNOWLEDGMENT

We would like to thank warmly David Reversat from ISIR for continuous support, as well as Percipio Robotics for their help and assistance.