Dinmukhammed Mukashev

Student Members, IEEE Adilzhan Adilkhanov

Member, IEEE Zhanat Kappassov

Can A Vibrotactile Stimulation On Fingertips Make An Illusion Of Elbow Joint Movement?

With the proliferation of haptic interfaces, the vibrotactile capabilities are accessible to a substantial number of end-users allowing a more realistic experience with Virtual Reality. Currently, the primary use of this vibrotactile feedback is to provide additional support to visual interaction via prehensile object manipulation using fingers. Nevertheless, haptic stimuli can be also applied for non-prehensile interaction that involves movements of the elbow joint. In this paper, we have designed and evaluated a vibrotactile device to investigate the effect of haptic stimuli, applied onto the fingertips, on a sensation of elbow displacements. The vision-driven displacement produced by the pseudo-haptics effect is then amplified by the vibrotactile stimuli. The experimental platform consists of a voice-coil actuator and a force sensor for generating mechanical vibrations at fingertips. The efficacy of the approach was validated in experiments with human subjects. The results show that the combination of pseudo-haptic and haptic illusion effects can be used to render various soft and rigid virtual objects.

I. INTRODUCTION

Virtual Reality (VR) applications have been experiencing an increased interest in recent years. Research studies in human perception revealed that a deeper dive into the Virtual Environment (VE) is possible through the conjunction of AR/VR and haptic feedback as it reduces cognitive load and increases performance [START_REF] Sigrist | Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review[END_REF]. According to Hayward et al. [START_REF] Hayward | Haptic interfaces and devices[END_REF], the word "haptics" refers to the capability to sense a natural or synthetic mechanical environment through touch. Our tactile sensation is mediated by a combination of receptors inside our skin. In general, human haptic perception consists of two: kinesthetic haptic feedback and cutaneous (tactile) haptic feedback. Kinesthetic feedback refers to the sense of position and motion of one's body state mediated by a variety of receptors located in the skin, joints, skeletal muscles and tendons [START_REF] Hayward | Haptic interfaces and devices[END_REF]. Cutaneous feedback is related to the stimuli detected by low threshold mechanoreceptors under the skin within the contact area [START_REF] Pacchierotti | Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery[END_REF].

In this connection, multiple new paradigms in VR devices have surged in recent years. They include wearable and nonwearable haptic devices used for the interactions with the VE. Haptic devices are used to engage these feedbacks and make users have the feeling of touch (provide haptic illusions).

The authors are with the Department of Robotics and Mechatronics, School of Engineering and Digital Sciences, Nazarbayev University, Nur-Sultan, Kazakhstan.

This work was partially supported by Nazarbayev University Facultydevelopment competitive research Grant no. 110119FD45119, Ministry of Education and Science of the Republic of Kazakhstan grant no. AP09058050, and Institute of Smart Systems and Artificial Intelligence (ISSAI). Devices addressed to stimulate kinesthesia are typically grounded, bulky, mechanically complex, expensive and have a limited workspace. Traditionally, kinesthetic devices are able to provide clear force or torque to move a user's hand or resist motion [START_REF] Culbertson | Haptics: The present and future of artificial touch sensation[END_REF]. The most popular commercial examples are Phantom (Sensable Technologies, USA) and Omega (Force Dimension, Switzerland). They are widely used in industry and medicine for teleoperation tasks and other tool-based applications (e.g. manipulator hand, dental drill) [START_REF] Rodríguez Ramírez | Applications of haptic systems in virtual environments: A brief review. advanced topics on computer vision, control and robotics in mechatronics[END_REF].

It has been shown that to some extent it is possible to compensate for a lack of kinesthesia with the modulated cutaneous force technique, without significant performance degradation [START_REF] Pacchierotti | Improving transparency in teleoperation by means of cutaneous tactile force feedback[END_REF]. Cutaneous feedback can be displayed by mobile, lightweight, compact devices that can be wearable and mounted on a user's body. Specifically, fingertips are one of the most stimulated areas of the human body as they are involved in almost all our tactile interaction with the environment. Fingerpads rich with mechanoreceptors [START_REF] Westling | r. s. responses in glabrous skin mechanoreceptors during precision grip in humans[END_REF] are being mostly stimulated through cutaneous feedback to give sensations of surface softness, curvature, edges, and texture [START_REF] Chinello | A three revolute-revolute-spherical wearable fingertip cutaneous device for stiffness rendering[END_REF], and object's weight [START_REF] Schorr | Three-dimensional skin deformation as force substitution: Wearable device design and performance during haptic exploration of virtual environments[END_REF].

In recent human perception studies, it was shown that fluctuations, resembling the mechanical response of granular solids, provoke a sensation of limb displacement even during pressing with a finger on a stiff surface [START_REF] Terekhov | The brain uses extrasomatic information to estimate limb displacement[END_REF]. The mechanical response of granular solids was replicated using the haptic device described in our previous work [START_REF] Adilkhanov | Vibero: Vibrotactile stiffness perception interface for virtual reality[END_REF]. The developed device provoked the sensation of index finger and thumb movements during a tip grasp of the stiff device of an object seen in an HMD. In this work, we aim in table-top haptic feedback device that excites the feeling of elbow movement [START_REF] Gaudeni | Presenting surface features using a haptic ring: A psychophysical study on relocating vibrotactile feedback[END_REF], [START_REF] Williams | Body-mounted vibrotactile stimuli: Simultaneous display of taps on the fingertips and forearm[END_REF] were conducted to create a vice-versa simulation (e.g. illusion of fingertips perception while simulating wrist or forearm).

The newly introduced device shares the key hardware design and stimuli actuation principles with the one in [START_REF] Adilkhanov | Vibero: Vibrotactile stiffness perception interface for virtual reality[END_REF]. The changes in the system and the VE architecture are firstly introduced in the following section. Afterward, the experimental procedure with human subjects is presented in Section IV. The recorded data were processed with the necessary statistical tools such as one-way and two-way analysis of variance (ANOVA). We conclude the work by discussing results, arisen issues, and future improvements.

II. EXPERIMENTAL PLATFORM

The setup consists of three main parts: hardware including actuators and force sensors, software for obtaining data from the sensor, and a virtual reality interface.

A. Hardware

In order to produce vibrotactile feedback in response to a user's applied force, we integrated the six-axis force/torque sensor (Weiss KMS 40-C, Weiss Robotics, Germany). For our application, the sensor was securely attached to a basis (ThorLabs optical table) and was pressed only in one direction (vertically -Z-axis). The sensor has a sampling rate of 500 Hz and 16 bit resolution which is enough to modulate proper vibrations [START_REF] Culbertson | Haptics: The present and future of artificial touch sensation[END_REF]. The sensor is capable of operation up to 120 N. It is factory calibrated and provides the data directly in the SI unit. The rest of the hardware was mounted on top of the force sensor as it is shown in Figure 3.

The obtained data from the force sensor was handled by the Robot Operating System (ROS) for the further modulation of vibrations and the use by VR application. In ROS, we have employed the open-source package by Lorenz Halt 1 . Further, this data is sent to the processing board (Teensy USB Board, Version 3.6, PJRS, USA) and Unity PC.

Teensy board with two digital-to-analog converting (DAC) ports was used to obtain the data from the force sensor (through a special Arduino package called 'rosserial'), and The pair of the voice coil actuators were used to deliver tactile cues to a user's fingerpads. As it can be seen from Figure 3, two actuators were placed vertically on the sides of the device. Such structure of the device was designed to mimic a grip of rod or stick (as explained in Section II-B) with thumb and index finger. The analog signal from the Teensy board to the actuators was amplified by the class-D Type Amplifier (Adafruit Mono 2.5W Class D Audio Amplifier -PAM8302, Adafruit Industries, USA). The bandwidth range of vibration is from 90 Hz to 1000 Hz what allows to have a choice of different vibration modes.

The generated vibrations are spread well through the solid objects. Thus, four springs per motor were used to avoid vibration suppression and convey all tactile sensations to the fingertips (Figure 3). Each spring has a constant of k = 200 N/m what is enough to keep the actuators stationary in relation to the device's enclosing structure. Indeed, the shift under the maximum vertical load of 16.5 N was less than 1 mm. This shift can be neglected as it is quite less than the displacement of the hand, rendered in VR. To isolate and hold in place the voice coil, four springs were used for each motor by two from each side which adds up to a total of 8 springs for two motors.

The enclosing structure of the device was 3D printed on Ultimaker S5 (Ultimaker, Netherlands) with Polylactic acid (PLA) plastic filament. It has cutouts for index and thumb fingers (Figure 3) on the sides for better grip. To avoid the shift of two fingers towards each other, the 3D printed limiter part was mounted between two actuators.

B. Virtual Reality

The VR goggles (Oculus Rift, Oculus, USA) were used as a device to show the VE environment (Figure 4) to testee (Figure 1). For one part of the experiment, Leap Motion Controller (Ultraleap, USA) was used to simulate the real hand in the virtual environment. The environment was created using the Unity Real-Time Development Platform (Version 2019.1.7f1). All models (i.e monitor, hand, table, etc.) were loaded from Unity's Assets store which is available for any registered user. The Virtual environment contains an interior element to give the user sense of scale.

The Leap Motion Controller was used in one of the parts of the experiment. On the hand model given by lip motion, the virtual stick was attached. It had a collider (Invisible mesh of object shape used to simulate the physical collision events) component which interacts with a sphere's collider. Thus, the squeeze effect from the real hand movements was implemented.

III. METHODOLOGY

A signal from the Teensy board was sent to motors as sinusoidal function v where the component R was varied depending on the force rate, while values A, φ were constants:

v = A • sin(φ • R); (1)
The rate of force (R) was measured as a derivative of the applied force. Further, the force rate was normalized and filtered. Thus,

R = 1 -R norm (2)
Where R norm is a normalized rate. It is reverted since the number of rate increases, it should contribute to the increased frequency and because the φ is a number less than 1. The multiplied number should be larger to increase the resulting frequency as the force rate increases. φ was chosen to be 0.1 or 0.5 for different parts of experiments with a combination of different sampling rates from the sensor: dt = 10, 5, 2 (where it corresponds to sample every dt milliseconds). Also, the upper force limits were set to 6.5N , 11.5N , and 16.5N . Those three values were chosen to have an obvious separation in travel distances. As for vibrations, the same limits were set in the VE for the travel limit of the bar.

As the force reaches upper or lower force limits, the output v sets to 0. Similarly, upper and lower thresholds were set for the normalized force rate of change: lower limit 0.01 and upper limit 0.3.

For the stick displacement and ball deformation:

d = F/(10 • c) (3)
the value of coefficient c was left as variable to be able to adjust and fine tune the stick and ball movements. However, for the experiment it was set c = 10 for all modes and only thresholds for the force F were left as control of the bar travel. Also, the ball shape control was implemented similarly. The Z component of the scale of the ball was decreased by the value d while the X and Y components increased. Also, the ball was shifted by d/2 along the Z-axis to compensate for the squeeze (it was scaled from both sides with respect to the middle of the ball) and stay on the table.

IV. PERCEPTUAL EVALUATION The Collaborative Institutional Training Initiative (CITI)

Program's Health Information Privacy and Security course was completed by the authors prior to the experiment. After that, the protocol of the experiment (NU-IREC 357/06012021) was approved by the Institutional Research Ethics Committee (IREC) of Nazarbayev University (NU).

A. Participants

There were ten volunteers involved in the experiments. Subjects (7 male and 3 female) were chosen randomly regardless of their familiarity with technologies in the field of this research. None of the participants reported any physical injuries or mental disorders, specifically in visual or haptic perception abilities. Instructions were provided to each participant in written and verbal forms before the procedure started. Subjects were asked to remove gloves, watches, bracelets, and jewelry if any as it may cause discomfort during the experiment.

B. Procedure

Before the start of measurements, subjects were asked to put on the Oculus Rift VR HMD and try to squeeze the virtual ball without the haptic device, e.g. move the hand in front of the Leap Motion Controller (Ultraleap Co., UK). As the motion tracking sensor allows a free movement, this step was needed to give the subjects an opportunity to feel the real movement of the hand during the ball squeeze task. After several trials, participants responded a value of distance perceived keeping in mind diameter of the virtual ball.

Following, participants were asked to put on headphones that produce white noise at the volume that allows to eliminate sound from the vibromotors. After that, a trained instructor helped to place their forearm on a special support and hold the device with their index finger and thumb (Fig. 1). According to the instruction, participants had to push on the virtual ball with a bottom end of the virtual stick five times. The counter (from 5 to 0) was displayed on the screen of the virtual monitor in order to make a user's aware of his/her progress. The value of the counter changes every time a user reaches the minimal applied force threshold. When the counter drops to zero, there is shown a text "Report the number". This means that a participant has to tell verbally the average hand's displacement value perceived during these five squeezes. After that, the device's operational mode changes, the counter showing "5" appears on the virtual screen, and a participant has to repeat the task. Overall, there were 5 rounds of 13 operational modes. The modes were shuffled differently in these five rounds. The time required to complete the task is around 10 minutes.

C. Statistical Methods

The aim of this study is to evaluate how the vibrotactile stimulation elicited by our haptic device enlarges the visual perception of limb displacement. After the experimental data collection, the perceived displacement values of each testee were normalized by dividing all the raw values by the given testee's maximum responded value. Despite that the all participants were provided with dimensions of the virtual ball and had the opportunity to feel real movement of the hand, the responded values significantly differed from one subject to another. Thereby, the maximum value is 1, and the minimum is 0 as it is depicted on Figure 5. Only six operational modes were included into further analysis because there were no significant difference between the resultant effect of 3 vibrational levels (φ = 0.1, 0.1, 0.5 with dt = 10, 5, 2 respectively). Therefore, only (φ = 0.1 and dt = 10) vibrational coefficient was considered.

As it can be seen in Figure 5, the perceived movement of the elbow joint depends more on the applied force threshold -the higher the applied force, the more the stick flattens the ball. Considering the sizes of the objects, such deformation is easily visually detectable by users. Therefore, the difference in responses between different is more noticeable than than the difference within force factor groups. However, there is an effect of vibrotactile stimulation as the perceived displacement due to combination of visual and haptic stimuli is slightly greater than those without vibrations. In order to validate the significance of our results, we have performed ANOVA tests. Firstly, we decided to verify the effect of vibration within each of the three upper force threshold pairs. The pairwise one-way ANOVA (Table 1) was taken for the records by the factor of vibration level (e.g. enabled vibrations -"ON" and disabled vibrations -"OFF"). The difference of data in the first pair (from the left on Fig. 5) is statistically insignificant (' ': p = 0.219), and the same for the third pair (' ': p = 0.2930). However, for the pair of force threshold equal to 11.5 N the difference of data is meaningful ('**': p = 0.00133).

Further, in order to check the effect of each of the two parameters -vibration level and upper force threshold (F th = 6.5 N ; F th = 11.5 N ; F th = 16.5 N ) -we performed the two-way ANOVA test for the records. According to Table 2, if considering the combination of the two factors, it makes the result statistically significant with low probability of getting these records by chance (p value for the vibration factor = 0.00473 '**'; p value for the force threshold factor is less than 2e-16 '***').

D. Results

Compare to the effect on illusory finger displacements reported in figure ten in [START_REF] Adilkhanov | Vibero: Vibrotactile stiffness perception interface for virtual reality[END_REF], a similar vibrotactile stimulation has less effect on perception of illusory elbow displacements (Table IV

-C).

There are multiple reasons that could cause this discrepancy. First, the size of the ball in this work was larger than in the former one, which inclined the subjects to rely more in the visual cues (pseudo-haptics) rather than on the haptic cues. Second, according to human cortical homunculus, the number of tactile receptors varies dramatically in different parts of a human body and it is much higher in the fingers and palm side than in the elbow. Finally, the applied haptic stimuli may have a stronger effect in the middle band of the force range: the reports of the participants analyzed with one-way anova() reveal that for the lower (6.5 N) and higher (16.5 N) force threshold, the results are statistically insignificant and, therefore, the vibrotactile stimulation does not amplify the perception of the elbow displacement seen in HMD; in contrast, in the middle range of muscular efforts (force threshold of 11.5 N), there is a dependence on perceived elbow displacements: a larger limb displacement is sensed when the vibrotactile stimuli is applied.

In order to investigate this issue, we conducted experiments without the visual feedback (eyes closes). Similar to the resulted reported in [START_REF] Terekhov | The brain uses extrasomatic information to estimate limb displacement[END_REF] and in [START_REF] Adilkhanov | Vibero: Vibrotactile stiffness perception interface for virtual reality[END_REF], a longer stimulation time interval (i.e. a higher force threshold) causes a sensation of a larger limb displacements. This fact indicates that the size of the ball was big enough to strengthen the visual cues and, thus, the pseudo-haptic effect.

In overall, a two-way anova() (Table IV-C) demonstrates that the combination of visual and vibrotactile stimuli causes a sensation of elbow displacement perceived through VR application.

V. CONCLUSION

In this paper, we extended our previous work [START_REF] Adilkhanov | Vibero: Vibrotactile stiffness perception interface for virtual reality[END_REF] for nonprehensile object manipulation in the VE. Specifically, we described the capability of our new table-top haptic device to enhance the quality of perceived haptic clues representing the softness of an object being poked via a grasped object in the VE seen in a HMD.

Empirical results demonstrated that modulated vibrotactile stimuli made the subjects perceive a higher displacement in the middle range of applied forces. This makes us believe that a vibrotactile stimulation on fingertips causes a sensation of elbow joint displacement under finely tuned conditions when the pseudo-haptic effect does not prevail over the haptic illusion effect.

The experiments were performed using a custom made vibrotactile controller incorporating a six-axis force sensor and a voice-coil actuator. As for the future work, we aim in increasing the number of the degrees of freedom. All the experiments conducted in this work considered the perception of elbow displacement on a single axis only. The developed setup incorporates a six-axes force sensor that allows us to detect the lateral and rotational forces.

As an extension and application of this phenomenon, we can vary a softness of the objects sensed in the VE.

Fig. 1 .

 1 Fig.1. Experimental procedure: a user wearing VR goggles is using the haptic device to perform the task (seen on the screen).

1Fig. 3 .

 3 Fig. 3. Wireframe view of device in full assemble

Fig. 4 .

 4 Fig. 4. Pciture (a) view from the VE. Picture (b) same view with squeezed ball

Fig. 5 .

 5 Fig. 5. The result of the experiment with 10 subjects (each subject is indicated with different color). The colored vertical bars and dots denote the average and standard deviation for each subject on each operational mode, respectively. The average and standard deviation per operational mode are shown with boxes. The result of ANOVA test (p-value) is displayed on the top of the figure.

Table 1 .

 1

			One-Way ANOVA
	Factor	Force threshold	F value p value
	vibration 6.5N		1.532	0.219
	vibration 11.5N		10.91	0.00133
	vibration 16.5N		1.119	0.293
	Table 2. Two-Way ANOVA
	Factor		F value	p value
	vibration	8.101	0.00473
	force threshold	229.385	¡2e-16

ACKNOWLEDGEMENT

We thank Alexander Terekhov for the discussions on the methodology, Amir Berrezag for providing the the highquality voice-coil actuators, and Vincent Hayward for valuable discussions and lecture on haptics during visiting Tactile Robotics laboratory.


Togzhan Syrymova

Student Members, IEEE Yerkebulan Massalim

Yerbolat Khassanov

Members, IEEE Zhanat Kappassov

Vibro-Tactile Foreign Body Detection in Granular Objects based on Squeeze-Induced Mechanical Vibrations

Granular particles, filled within an elastic material, produce mechanical vibrations in structures or air when squeezed. This refers to structure-borne noise, is defined as a noise that occurs from the impacts of particles hitting each other due to their momentum. The momentum depends on both properties of particles and velocity of squeezing. Therefore, the structure-borne noise is highly correlated with the properties of particles. In this connection, we study a vibro-tactile sensor for detecting the mechanical vibrations from squeezing granular objects. Specifically, we explore machine learning solutions to detect foreign body within these objects using detected vibrations. We evaluated multiple learning approaches on a collected data set of 900 squeezing experiments across 15 different granular materials. In our experiments, the most successful method was convolutional neural network that achieved an accuracy of 91% on unseen test data. Remarkably, the foreign body was detected with a higher success rate for the majority of material types except salt and coffee granules.

I. INTRODUCTION

Humans can do reasoning on the quality of the material they touch. They actively move their fingers and manipulate objects in order to do such reasoning thanks to the somatosensory system. The renowned physiologist Sechenov called this phenomenon a "dark muscle sense" [START_REF] Liutsko | Proprioception as a basis for individual differences[END_REF]. For example, one can use vibrations from sliding over a textured surface to recognize objects [START_REF] Lederman | Hand movements: A window into haptic object recognition[END_REF] or to detect slippage [START_REF] Srinivasan | Tactile detection of slip: surface microgeometry and peripheral neural codes[END_REF]. Indeed, previous works have shown that humans can create haptic information about objects' physical characteristics based on the properties of the perceived vibro-mechanical signal. For example, Hayward et al. [START_REF] Giordano | Identification of walked-upon materials in auditory, kinesthetic, haptic, and audio-haptic conditions[END_REF] have shown that humans are able to classify walked-upon materials using the feedback from the ground. On the other work, Pittenger et al. [START_REF] Pittenger | Haptic and auditory information support perception of size: Fine granules[END_REF] demonstrated the ability of humans to guess the size of granules inside an opaque container by shaking it.

In robotics, the sense of touch is provided by various forcesensing technologies, including capacitive, piezoresistive, magnetic, and optical to name a few of them [START_REF] Kappassov | Tactile sensing in dexterous robot hands[END_REF], [START_REF] Massalim | Array of accelerometers as a dynamic vibro-tactile sensing for assessing the slipping noise[END_REF]. This sensing ability allows robots to preempt slippage [START_REF] Teshigawara | Highly sensitive sensor for detection of initial slip and its application in a multi-fingered robot hand[END_REF] and distinguish surface roughnesses [START_REF] Xu | Tactile identification of objects using bayesian exploration[END_REF]. Presumably, the ability to sense and process vibrations would also allow robots to This work was supported by the NU Faculty-development competitive research grants program "Variable Stiffness Tactile Sensor for Robot Manipulation and Object Exploration" 110119FD45119 and by the Ministry of Education and Science of the Republic of Kazakhstan grant for tactile sensing.

Authors are with the Dept. of Robotics and Mechatronics, Nazarbayev University, 53 Kabanbay Batyr Ave, Z05H0P9 Astana, Kazakhstan. Email: zhkappassov@nu.edu.kz.

Corresponding author: Z. Kappassov. recognize granular material types and detect irregularities within them [START_REF] Kudrolli | Size separation in vibrated granular matter[END_REF], [START_REF]Transient sound radiated by spheres undergoing an elastic collision[END_REF].

Unfortunately, vibro-mechanical signals produced by granular materials have attracted only limited attention from researchers. To the best of our knowledge, there are no general methods for recognizing granular particles based on tactile sensors. With regard to vibro-tactile recognition, majority of the studies have targeted surface textures [START_REF] Xu | Tactile identification of objects using bayesian exploration[END_REF], [START_REF] Fishel | Bayesian exploration for intelligent identification of textures[END_REF], [START_REF] Wettels | The Human Hand as an Inspiration for Robot Hand Development[END_REF], [START_REF] Liu | Surface material recognition through haptic exploration using an intelligent contact sensing finger[END_REF], whereas only a few and abrupt studies have targeted granular materials [START_REF] Clarke | Learning audio feedback for estimating amount and flow of granular material[END_REF], [START_REF] Nakamura | Multimodal object categorization by a robot[END_REF] despite the fact that they are abundant and commonly found in human-made products such as salt, sugar, soda, coffee, and others. These materials consist of small and rigid objects (particles or grains) of various shapes which can be classified using existing machine learning techniques [START_REF] Lepora | From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor[END_REF]. However, modeling the mechanics of induced vibrations over granular particles is challenging [START_REF]Transient sound radiated by spheres undergoing an elastic collision[END_REF].

In this work, we aim to make a step towards filling this gap. It is for the first time, the detection of a foreign body within a soft object filled with granular particles is explored (see Fig. 1). Specifically, we employ conventional and deep learning (DL) based approaches to classify the data acquired from squeezing 30 deformable objects corresponding to 15 different materials with and without a foreign body.

In the remainder of the paper, we review the vibrotactile based methods for object recognition in Sec. II. Section III describes the mechanics of granular vibrations and sensor design for measuring them. We introduce the silicone molding-based design and fabrication of the sensor in Sec. III-B.2. The utility of vibro-mechanical signal for the foreign object detection task is evaluated in Sec. IV. The last section concludes this work. II. RELATED WORK Within robot applications driven by tactile feedback, the classification of objects based on their deformability has gained momentum as there is an increasing demand for industrial and service robotics tasks in soft object manipulations [START_REF] Sanchez | Robotic manipulation and sensing of deformable objects in domestic and industrial applications: A survey[END_REF]. Object-wise, the taxonomy of soft objects is given by fabric sheets (e.g. clothes), soft cables (e.g. electric wires) and ropes (e.g. shoelaces), cast deformable objects (e.g. silicone rubber, sponge, and squishy toys), and soft granular objects that vibrate when squeezed (e.g. granulated latex pillows, snowballs, and bags filled with granules as salt, sugar, and etc.).

In the latter two cases, i.e. deformable objects, the sense of touch has a pivotal role as the softness of objects cannot be inferred based on their shapes, which could be easily acquired using vision sensors [START_REF] Vasquez | In-hand object shape identification using invariant proprioceptive signatures[END_REF]. Depending on a transduction technology of tactile sensors (sensed signals can be dynamic or static with respect to the time response and may represent an array of data, vector or scalar), these soft objects can be recognized from contact forces [START_REF] Liu | Surface material recognition through haptic exploration using an intelligent contact sensing finger[END_REF], contact pressure profile [START_REF] Pezzementi | Tactile-object recognition from appearance information[END_REF], and friction-induced vibrations [START_REF] Xu | Tactile identification of objects using bayesian exploration[END_REF].

Using contact forces, deformable objects were distinguished based on their stiffness properties [START_REF] Bscher | Augmenting curved robot surfaces with soft tactile skin[END_REF]. An alternative way was implemented using an array of pressure sensors detecting an area of contact [START_REF] Drimus | Design of a flexible tactile sensor for classification of rigid and deformable objects[END_REF]: when the sensing arrays was pushed against a soft object, the area of contact increased, whereas it did not change in the case of a rigid object. Inspired by this idea, we presented a magnetic tactile sensing array [START_REF] Kappassov | A series elastic tactile sensing array for tactile exploration of deformable and rigid objects[END_REF]. However, in such method, we assumed that the sensing surface was always larger than the object. In this connection, we proposed another method, classifying soft from rigid objects, that incorporated both a proprioceptive, i.e. joint effort (current) sensors and an exteroceptive, array of pressure sensors, sensing systems [START_REF] Mazhitov | Deformable object recognition using proprioceptive and exteroceptive tactile sensing[END_REF]. In the experiments, there were no soft objects filled with granular particles.

Indeed, we did not find tactile data sets of deformable granular objects as exemplified in Table I. Almost all of the sets include quasi-static pressure signals similar to grayscale images [START_REF] Sundaram | Learning the signatures of the human grasp using a scalable tactile glove[END_REF], [START_REF] Gao | Deep learning for tactile understanding from visual and haptic data[END_REF], [START_REF] Molchanov | Contact localization on grasped objects using tactile sensing[END_REF], [START_REF] Chebotar | Bigs: Biotac grasp stability dataset[END_REF]. In [START_REF] Chu | Robotic learning of haptic adjectives through physical interaction[END_REF], the data set consists of vibration signals from sliding over different textures.

To the best of our knowledge, the closest work exploits sound vibrations to estimate the amount of granular objects dropped while pouring them [START_REF] Clarke | Learning audio feedback for estimating amount and flow of granular material[END_REF]. DL methods were used as the computational means to perform the task. The pouringinduced vibrations were detected by a microphone.

In this work we developed a vibro-tactile sensor, which consists of an accelerometer and an absolute pressure sensor molded in silicone rubber, and collected a Vibro-Tactile Data Set 1 for 30 soft granular objects (one half is with a foreign 1 https://gitlab.com/roboticsNU/granular-objects-data-set-.git body with granules and another half is without the foreign body). When were squeezed, all granular objects caused mechanical fluctuations resulting in mechanical vibrations so that the granular objects with the foreign body could be distinguished by DL methods. [START_REF] Chebotar | Bigs: Biotac grasp stability dataset[END_REF] Signals from BioTac sensors using a threefinger robot hand Contact localization [START_REF] Molchanov | Contact localization on grasped objects using tactile sensing[END_REF] Signals from BioTac sensors of 18 rigid objects while grasping with a three-finger robot hand Sensors stitched in a glove [START_REF] Sundaram | Learning the signatures of the human grasp using a scalable tactile glove[END_REF] Data from an array of pressure sensors collected from human grasps Visual and haptic features learning [START_REF] Gao | Deep learning for tactile understanding from visual and haptic data[END_REF] Signals from both BioTac sensors and a camera while grasping rigid and soft non-granular objects Vibro-tactile features learning [START_REF] Chu | Robotic learning of haptic adjectives through physical interaction[END_REF] Signals from BioTac sensors from sliding over surfaces of 60 non-granular objects III. METHODS

A. Mechanical Vibrations of Granular Objects

When a solid object (e.g. one granular particle within many of the same kind) collides with another object, its surface starts to vibrate producing pressure changes in the surrounding medium (other objects). These changes are perceived: as sound if they are in the audible range and as tactile stimuli, if they occur at a contact point in a detectable range (i.e. up to 1 kHz [START_REF] Johansson | Coding and use of tactile signals from the fingertips in object manipulation tasks[END_REF]). Occurring vibrations are modulated by colliding objects' shapes and material. Therefore, one can identify these objects [START_REF] Giordano | Identification of walked-upon materials in auditory, kinesthetic, haptic, and audio-haptic conditions[END_REF].

There at least three sources of squeeze-induced vibrations in granular objects:

1) Impact Force-Induced Vibrations: The surface vibrations produced due to an impact force applied to a single granular solid particle. They can be modeled as a sum of exponentially descending waveforms [START_REF] Gaver | How do we hear in the world? explorations in ecological acoustics[END_REF] q impact (t) = n Φ n e -δnt cos w n t

where q impact (t) is the resultant vibration waveform at the time t after the impact, Φ n is the initial value of the partial when q impact (0) = n Φ n which is proportional to the impact force, e -δnt is the damping term that is related to the material of the particle (e.g., plastic and glass objects produce different sounds), cos w n t is the partial frequency term that depends on the size and configuration of the particle.

2) Acceleration Noise-Induced Vibrations: When one solid particle ("impactor") hits its neighbor ("impactee"). The simplified vibration model for spherical particlesderived based on Hertzian approach of elastic collision forces -is described in [START_REF] Koss | Transient sound radiated by spheres undergoing an elastic collision[END_REF] and given by the function q collision (r, θ, t) that depends on the distance between spheres centre and the locations of a sensing point r, the angle between collision normal and the direction towards the sensing point θ, and the time t as the function is different during and after the collision.

3) Slip-Induced Vibrations: There is the sliding noise due to the friction between interacting particles. Given the surface profile r(x) along horizontal axis and position x t at the time t, vibrations q(t) due to vertical displacements is given by q slide (t) = r (x t ) ẋ [START_REF] Yao | Design and analysis of a recoil-type vibrotactile transducer[END_REF].

The resultant vibration q(t) = q impact (t) + q collision (t) + q slide (t)

depends on various material constants and physical interaction properties. Moreover, in the mentioned models, the collision of a foreign body with uniform granular particles is not considered. Therefore, we apply DL methods (Sec. IV-B) to detect the foreign body within the granular particles. The accuracy of the method would depend on the performance of a sensor, which is described next.

B. Vibro-Tactile Sensor

In order to register the described above vibrations, our vibro-tactile sensor (Fig. 1 d) incorporates one vibration sensing module and one pressure sensing module.

1) Accelerometer: The vibration sensing module consists of a high-bandwidth-two-axes accelerometer (Analog Devices, ADXL203) with analog outputs. This accelerometer measures accelerations along vertical and horizontal axes tangential to the sensing surface.

The accelerometer has a physical bandwidth that exceeds 1 kHz. Such bandwidth allows us to capture vibrations generated by interactions of granular particles. An ordinary first-order filter (with the time constant τ RC = 1µs) was used as an anti-aliasing filter.

We used a Data Acquisition Module (DAQ, National Instruments, NI 6221) installed in a Ubuntu running Linux workstation (CPU Core i9-7900, 32 Gb RAM DDR4) to sample accelerometer readings at 4 kHz. The Kinetic distribution of Robot Operating System (ROS) was used to acquire signals and synchronize them with force sensing signals from the second sensing module.

2) Silicone Rubber Embedded Pressure Sensor: We embedded a pressure sensor into a silicone rubber (MPL115A2, NXP Semiconductors) to estimate the strength of squeezing an object [START_REF] Tenzer | The feel of MEMS barometers: Inexpensive and easily customized tactile array sensors[END_REF]. The pressure sensor is a micro electromechanical sensor (MEMS) technology-based barometer that measures absolute pressure. The pressure sensor was sampled at 100 Hz using a microcontroller unit connected with the same workstation via USB protocol.

We designed a custom fingertip holder using a rapid prototyping printer (Extended2+, Ultimaker) from PLA material with a deepening (6 mm depth) inside to embed a single vibro-tactile module from the accelerometer and pressure sensor and two deepenings (9 mm depth) at the bottom to fix fingertip to two teeth on the gripper. A silicone compound (SortaClear18, Smooth-on) -mixed in the ratio of 10:1 of parts A and B -was poured into a deepening inside the fingertip up to 4 mm level with barometer placed at the bottom. The mixture was preliminarily degassed in a vacuum chamber to remove air bubbles and cured with ultraviolet for 12 hours. The labor-intensive process took less than one day. All 3D-printed molds were firmly fixed using glue in order to get high precision and repeatability of the sensor geometry. The total weight of the sensor elastomer is around 6 g.

We calibrated the embedded into the silicone rubber pressure sensor's readings using a force sensor with the same contact area of silicon rubber for both calibration and experiment scenarios (Weiss WTS-40). The calibration results of the pressure sensor with average values from ten measurements are shown in Fig. 2. As can be seen from the graph, the range of the pressure sensor's sensed force is between 1 N and 15 N.

IV. EXPERIMENTS

Vibro-tactile sensor was installed onto a robot gripper (SCHUNK EGN 80, SCHUNK GmbH) controlled using ROS installed on another workstation at 100 Hz (with the same parameters as the workstation for the sensor). Computers were connected via Ethernet (Fig. 3).

The vibro-tactile data was collected by squeezing 30 objects (rubber balloons) filled with granular particles, where first 15 were distinct granular objects (Fig. 4) and other 15 were same granular objects with a foreign body inside (a solid ball of 2 cm diameter printed with FormLab Form 2 clear resin cartridge). The objects were removed from their original packages and placed into the balloons (Fig. 1 b andc) to avoid effects and noise caused by different packaging materials. All balloons filled with the materials were inflated with small portion of air to make them more elastic and of the same shape with average mass around 100 g.

A. Data Collection

For data collection squeezing and releasing motions were produced on the 30 objects. These motions were repeated ten times at three different velocities: slow (40 mm/s), medium (60 mm/s), and fast (80 mm/s). In total, there were 900 recording sequences with 2,900 samples in each sequence for accelerations in horizontal and vertical axes, and normal force readings (7,830,000 samples in overall). An example of the force history applied by the robot gripper and squeeze-induced vibrations of two balloons filled with nuts (one with a foreign body and one without it) are exemplified in Fig. 5.

The initial experiments showed that the vibrations induced at the stage of squeezing was more informative; whereas utilizing releasing signals were deteriorating the performance of the model. In this connection, DL methods fed by squeeze-

B. Foreign Body Detection

We split the data into three sets of train, dev and test using stratified sampling to ensure an equal proportion of materials with and without the foreign body. The train set was used to learn the parameters of the models, whereas the dev set was used to fine-tune the hyperparameters and select the best performing model which is then evaluated on the test set.

We explored both conventional and deep learning based machine learning techniques. The conventional techniques were adopted from Matlab's Classification Learner toolbox, which includes k-NN, Support Vector Machines (SVM), and other models. While the DL-based models were implemented using PyTorch. In particular, we implemented two different model architectures: feedforward neural networks (FNN) and convolutional neural networks (CNN). The FNN model consisted of two hidden layers with 512 units each and with ReLU activation functions. The CNN model consisted of three convolutional layers followed by two fully connected layers: the first layer had 512 units with ReLU acitvation and the second layer was a linear projection with sigmoid activation for binary classification. In the convolutional layers, we used four filters with a receptive field of 1 × 16 and stride of four. Both models were optimized using Adam algorithm with the initial learning rate set to 10 -5 and regularized using dropout. The number of training epochs was set to 10,000 and we evaluated the model after each epoch on the dev set. The models are trained on a single Tesla V-100 GPU on the Nvidia DGX machine.

The experimental results are shown in Table II. For the conventional techniques, we report only three best performing models out of 16. For the deep learning based techniques, we report both FNN and CNN. The best result was achieved by CNN which significantly outperformed other models. To boost the performance of other models, we pre-processed the raw input data using a fast Fourier transform (FFT). Although the performance of other models considerably improved, they were still inferior to CNN. Note that the CNN model didn't benefit from the pre-processed data as it can implicitly learn the high-level discriminative features from the raw input.

To analyze the impact of different materials on foreign body detection capability, we computed the accuracy of the CNN model for each material separately (see Table III). The material IDs from 1 to 15 correspond to nut, bean, lentils, peas, buckwheat, baking soda, salt, couscous, rice1, rice2, millet, corn croup, pearl barley, coffee, starch respectively. Interestingly, we found that for the majority of material types the foreign body could be detected with a high accuracy above 70% except for salt and coffee materials, IDs 7 and 14 in Table III. These findings suggest high robustness of foreign body detection systems based on squeeze-induced vibrations, which are modeled in Eq. ( 2), to different material types.

V. CONCLUSION

In this paper, we designed a mechanical vibration detection system consisting of a high-bandwidth accelerometer attached to a soft silicone-molded fingertip with an embedded absolute pressure sensor. We collected vibro-tactile signals from squeezing 15 different granular materials with and without a foreign body inside. The obtained dataset was used to identify the presence of a foreign body using both conventional and deep learning based approaches. The experimental results show the high efficacy of vibro-tactile signals for the foreign body detection task were 91% accuracy is achieved using CNN models. Moreover, the trained model was found to be invariant to the physical properties of granular particles.

As future work, we envision to enrich the sensor with other sensing modalities (e.g. temperature) and increase the sensitivity to vibrations. However, there are methods of breast lump detection like sonography that based on the screening test data [START_REF] Cheng | Automated breast cancer detection and classification using ultrasound images: A survey[END_REF], we believe that preliminary inspection of lumps in breasts by initial palpation [START_REF] Rodriguez-Ruiz | Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists[END_REF] may benefit from our vibro-tactile sensing approach.

Fig. 1 :

 1 Fig. 1: A foreign body mixed with granular particles changes properties of mechanical vibrations when an object filled with them is squeezed. (a) Robot gripper with a vibro-tactile sensor squeezing granular objects (b) with and (c) without the foreign body. (d) The vibro-tactile sensor.

Fig. 2 :

 2 Fig. 2: Pressure to Force map of the absolute pressure sensing barometer embedded in a silicone rubber.

Fig. 3 :

 3 Fig. 3: Robot Gripper Control and Data Acquisition Block Diagram.

Fig. 4 :

 4 Fig. 4: Granular objects used in experiments for foreign body detection.

1 )

 1 Squeeze: Gripper received a command (gripper position reference set point of 0 mm) to squeeze a balloon with granular objects. 2) Releasing: After gripper reached the reference set point, it received a release command towards open position (20 mm).

Fig. 5 :

 5 Nut squeezing results: (a) Force history, (b) Vibrations, and (c) Harmonics of vibrations. induced vibration signals by far outperformed the ones fed by the release-induced vibration signals and force signals, and thus are not present in the next section.

TABLE I :

 I Tactile Data Sets.

	Tactile Data Set	Description
	BioTac Grasp Stabil-
	ity Dataset

TABLE II :

 II Foreign Body Detection Results. SVM1 -Cubic SVM, SVM2 -Medium Gaussian SVM.

	ID	Model	Input feature Dev set (%)	Test set (%)
	1	SVM1	raw FFT	79 81	77 83
	2	SVM2	raw FFT	79 79	75 84
	3	k-NN	raw FFT	84 72	79 81
	4	FNN	raw FFT	88 89	74 87
	5	CNN	raw FFT	96 83	91 81

TABLE III :

 III Foreign Body Detection Results of CNN Model for Different Material Types.

	Material ID	1	2	3	4	5	6	7	8
	Accuracy	100 100	90	100 100	100 70 80
	Material ID	9	10	11	12	13	14	15
	Accuracy	90	80	100	90	100	70	90

ACKNOWLEDGEMENTS

The authors are thankful to Vincent HAYWARD for discussions on human perception aspects, especially haptics, Adilzhan ADILKHANOV (Nazarbayev University) for assistance in taking measurements.