Ameur Latreche
email: am.latreche@univ-skikda.dz

Ridha Kelaiaia
email: r.kelaiaia@univ-skikda.dz

Ahmed Chemori
email: ahmed.chemori@lirmm.fr

Adlen Kerboua
email: ad.kerboua@univ-skikda.dz

Reliability and validity analysis of MediaPipe-based measurement system for some human rehabilitation motions

The many epidemics that the world is witnessing necessitate the use of telehealth. Developing a website for telerehabilitation follow-up is appealing. Furthermore, the patient benefits financially from the elimination of costly equipment. The major purpose of this paper is to present a telerehabilitation website and to compare an artificial intelligence technique incorporated into the website called MediaPipe to measure range of motion with a universal goniometer and a digital angle ruler. Data was collected from around 50 healthy volunteers with the assistance of physical therapists. The MediaPipe-based shoulder measurement system's reliability is determined. To validate the approach, the 95% limits of agreement and mean difference between the MediaPipe and the two other devices are determined. The results indicate that the MediaPipe is reliable and valid for use in telerehabilitation with the studied movements. The method is then used to monitor a patient with a right patella fracture.

Introduction

The World Health Organization estimates that around 15% of the global population is disabled [START_REF] Organization | WHO global disability action plan 2014-2021: Better health for all people with disability[END_REF]. Some of these people require therapy for some periods of time, while others require it constantly. Telemedicine expands both patients' and medical staff's horizons and capabilities. By using telemedicine, several expenses may be avoided. Rehabilitation is crucial for patients who have undergone surgery on their lower or upper limbs, or sustained other injuries. Through rehabilitation sessions, the doctor should monitor the patient's gradual improvement by measuring his range of motion using a clinical goniometer. However, this device requires both hands, minimising the doctor's ability to stabilise the joint [START_REF] Lea | Range-of-motion measurements[END_REF]. Furthermore, a clinician's skills and experience influence the goniometer precision [START_REF] Akizuki | The effect of proficiency level on measurement error of range of motion[END_REF]. Another device, called the inclinometer, has been used for its advantages, compared to the goniometer, such as handiness and somewhat better reliability. However, the main disadvantage of the digital inclinometer is that it costs more than a universal goniometer. Therefore, to remotely measure and follow the progress of a patient's condition, various types of equipments with different functions have been developed, such as sensors like inertial measurement units (IMUs), which include accelerometers, magnetometers, and gyroscopes. The goal of using IMUs is to detect and measure human movements using the remarkable innovations in micro-electromechanical systems, which are lightweight and compact. By incorporating different algorithms for the desired activity, such as rehabilitation, motion tracking, gait analysis, gesture recognition, and handwriting algorithms, IMUs can regroup accelerations, magnetic signals, and angular velocities from human movements [START_REF] Kurban | Daily motion recognition system by a triaxial accelerometer usable in different positions[END_REF][START_REF] Li | Real-time 3d motion tracking and reconstruction system using camera and imu sensors[END_REF][START_REF] Weygers | Drift-free inertial sensor-based joint kinematics for long-term arbitrary movements[END_REF][START_REF] Baldi | Upper body pose estimation using wearable inertial sensors and multiplicative kalman filter[END_REF]. Many research studies have shown the accuracy of wearable IMUs when they are used to make joint angle measurements [START_REF] Teufl | Validity of inertial sensor based 3d joint kinematics of static and dynamic sport and physiotherapy specific movements[END_REF][START_REF] Cutti | Ambulatory measurement of shoulder and elbow kinematics through inertial and magnetic sensors[END_REF][START_REF] El-Gohary | Shoulder and elbow joint angle tracking with inertial sensors[END_REF]. The Kinect V1 or V2 devices are also often utilised in telerehabilitation studies [START_REF] Ozturk | A clinically feasible kinematic assessment method of upper extremity motor function impairment after stroke[END_REF][START_REF] Perez-Medina | ephort: towards a reference architecture for tele-rehabilitation systems[END_REF][START_REF] Sheng | An automated system for motor function assessment in stroke patients using motion sensing technology: A pilot study[END_REF]. However, the Kinect device is not very accurate. Unlike other technologies for ordinary citizens, its cost is relatively high, and it is not available to everyone. Furthermore, a research study in [START_REF] Naeemabadi | Evaluating accuracy and usability of microsoft kinect sensors and wearable sensor for tele knee rehabilitation after knee operation[END_REF] demonstrated that the Kinect sensor has some limitations when it comes to following some of the exercises involving the lower limbs, especially knee rehabilitation exercises. Thus, it is necessary to find a way for the doctor to remotely check on a patient's health that it works well and does not cost too much.

There are several examples in the literature of goniometers or digital goniometers being used to compare measurements from sensors or algorithms [START_REF] Çubukçu | Reliability and validity analyzes of kinect v2 based measurement system for shoulder motions[END_REF][START_REF] Wochatz | Reliability and validity of the kinect v2 for the assessment of lower extremity rehabilitation exercises[END_REF][START_REF] Cai | Validity and reliability of upper limb functional assessment using the microsoft kinect v2 sensor[END_REF][START_REF] Lin | Assessment of shoulder range of motion using a wearable inertial sensor network[END_REF]. Generally, a statistical study should be done to compare the measurement instruments and prove their reliability and validity. Reliability and validity are defined in [START_REF] Hammersley | Some notes on the terms 'validity'and 'reliability[END_REF] as follows: the agreement between two efforts to measure the same trait using a maximally similar method is the reliability. The validity is the agreement between two attempts to measure the same trait through maximally different methods. Çubukçuet et al. [START_REF] Çubukçu | Reliability and validity analyzes of kinect v2 based measurement system for shoulder motions[END_REF] provided the reliability and validity of the Kinect V2 device for shoulder motions, comparing it with a goniometer and an inclinometer. Additionally, Wochatz et al. [START_REF] Wochatz | Reliability and validity of the kinect v2 for the assessment of lower extremity rehabilitation exercises[END_REF] provided the reliability and validity of the Kinect for lower-limb movements. Vafadar et al. [START_REF] Vafadar | Interrater and intrarater reliability and validity of 3 measurement methods for shoulder-position sense[END_REF] studied the inter-rater and intra-rater reliability and validity of a laser pointer, an inclinometer, and a goniometer. Abd Elrahim et al. [START_REF] Elrahim | Inter-rater and intra-rater reliability of kinovea software for measurement of shoulder range of motion[END_REF] investigated the inter-rater and intra-rater reliability of Kinova software when it is used to measure the shoulder joint ROM in healthy individuals. This software is a video-based method for measuring the range of motion of joints using a virtual goniometer. By considering the above studies, we aim to find more effective solutions based on cutting-edge technologies that are available to the majority of people at a lower cost. In this research study, we focused on using the MediaPipe technique [START_REF] Lugaresi | Mediapipe: A framework for perceiving and processing reality[END_REF] in the telerehabilitation field to measure the range of motion. Through our in-depth research, we found that, so far, no studies have been conducted to confirm the reliability and validity of the MediaPipe technique in the measurement of the ROM. This is why we decided first to study the reliability of this technique and compare it with a goniometer and an angle ruler. The MediaPipe technique has been used in various research studies, such as those concerning yoga movements [START_REF] Garg | Yoga pose classification: a cnn and mediapipe inspired deep learning approach for real-world application[END_REF] and rehabilitation [START_REF] Baranyi | Ai technologies for machine supervision and help in a rehabilitation scenario[END_REF]. One of the most important features of the MediaPipe technique is that it can smooth out any noise from pose prediction or classification by applying an exponential moving average (EMA). By doing this, we can obtain the nearest pose cluster, calculate a probability for each of them, and utilise them for smoothing over time. Different methods have been used to filter raw data in object tracking, such as those of [START_REF] Ozturk | A real-time application of singular spectrum analysis to object tracking with sift[END_REF] and [START_REF] Hisham | Mono camera-based human skeletal tracking for squat exercise abnormality detection using double exponential smoothing[END_REF]. The main contributions of this paper are the following:

• Investigation of the reliability and validity of MediaPipe-based measurements.

• The discussion of MediaPipe's contribution to telerehabilitation by validating it through real patient cases.

The rest of the paper is structured as follows. Section 2 introduces the three methods used to measure the proposed motions. This section is divided into two subsections, where the first one introduces the technique of MediaPipe, the goniometer, and the angle ruler. The second subsection presents the data for the shoulder ROM, and a the statistical analysis is provided. Section 3 discusses the reliability and validity of MediaPipe. In Section 4, the main goal is to present the contribution of MediaPipe to telerehabilitation, with a patient case study. Section 5 is a discussion of the technique and the project. Section 6 concludes the paper and introduces some future directions.

Methods and Data of measuring ROMs

Technique and instruments used in measuring the range of motion

In this research project, a total of 50 volunteers have participated; their ages range from 21 to 42. Most of the data were taken at the University of Skikda. All participants were measured using three methods: MediaPipe with a camera on a computer, Dell Intel(R) Core(TM) i5-4310U CPU @ 2.00GHz, 2.60 GHz, and 4.00 GB RAM. The second instrument is a clinical goniometer (figure 1). The third is a digital instrument called Angle Ruler 50, a 360 • digital protractor 2 × 50 cm ADA (see figure 1). The measurement accuracy is about 0,3 • , power: 3V, 1 × CR 2032. Data are visualized on the built-in LCD digital display. This study measures four ranges of motion, shoulder abduction, adduction, flexion, and extension. Before beginning to measure the movements, we obtained consent from all volunteers to publish their data in this research. We had a physical therapist explain the method to each individual and take measurements. We first started by measuring the four movements with MediaPipe in three repetitions. This procedure aims to calculate the intraclass correlation coefficient (ICC) to prove the reliability of the technique. The physiotherapist did just one measurement with the goniometer and the angle ruler.

MediaPipe pose estimation technique

MediaPipe is a framework developed by Google for machine learning applications [START_REF] Lugaresi | Mediapipe: A framework for perceiving and processing reality[END_REF]. It predicts the location of 33 pose landmarks (see figure 2). This technique can forecast a full-body segmentation mask depicted as a two-class segmentation (human or background). Using the MediaPipe framework, inference models like TensorFlow and TFLite, as well as multimedia processing functions, can be constructed. With BlazePose, MediaPipe infers 33 3D landmarks on the whole body from RGB video frames. Its library output holds the coordinates of the user's major basis in the image. A function is programmed to get the data. Then, this function calculates the angles at each human body joint. The angle made between the two lines is calculated by using analytic geometry. For example, we have three points (show figure 3), ( 1, 1), ( 2, 2), and [START_REF] Akizuki | The effect of proficiency level on measurement error of range of motion[END_REF][START_REF] Akizuki | The effect of proficiency level on measurement error of range of motion[END_REF]. We suggest that and as skeletal structures, or two bones. The intersection between the lines and is on point . The angle between and can be determined as follows: Slope of is:

1 = ( 2 -1)∕( 2 -1). ( 1
)
BC slope is:

2 = ( 3 -2)∕( 3 -2). (2)
Now, we can determine the angle between and :

= ( 1 -2)∕(1 + 1. 2). (3)

Statistical analysis

A physiotherapist was responsible for interacting with the participants and taking the readings from the clinical goniometer and the angle ruler. Measurements were made by the three methods. The volunteers did the proposed movements playfully. The results obtained are compared with the normal interval values of [START_REF] Gill | Shoulder range of movement in the general population: age and gender stratified normative data using a community-based cohort[END_REF], which shows that all the results are on the normal average.

To obtain the statistical analysis, IBM SPSS Statistics version 26 was utilized. An ICC was applied to show if the MediaPipe is reliable. According to [START_REF] Shrout | Intraclas in assessing rater reliability[END_REF][START_REF] Shrout | Intraclass correlations: uses in assessing rater reliability[END_REF][START_REF] Landis | The measurement of observer agreement for categorical data[END_REF][START_REF] Zaki | Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: a systematic review[END_REF][START_REF] Li | Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging[END_REF], the ICC was assessed as follows: between 0.81 and 1.0 is "very good or excellent", and good if the result is inside the interval of 0.61 to 0.80. "Moderate" if it is between 0,41 and 0,60, and "fair" if the value is between 0,21 and 0,40. Finally, it is "poor" if it is below 0.20. The standard error of the measurement (SEM) and the minimal detectable change (MDC) are calculated to get the absolute reliability, as shown in the two next equations [START_REF] Stratford | Health status measures: strategies and analytic methods for assessing change scores[END_REF][START_REF] Stratford | Defining the minimum level of detectable change for the roland-morris questionnaire[END_REF] :

= . √ 1 - (4) = . √ 2 × 1.96 (5)
The StDev indicates standard deviation. Then, to prove MediaPipe validity, Bland-Altman analysis [START_REF] Bland | Statistical methods for assessing agreement between two methods of clinical measurement[END_REF] is effectuated. Limits of agreement (LOA) and mean bias metrics are calculated to evaluate the absolute accuracy of MediaPipe over the goniometer and the digital ADA.

Results

Figure 5 compares the ROM value results of the shoulder abduction, adduction, extension, and flexion, using the three methods. Different colors indicate each method of visualizing the differences. Figures 6,7, and 8 indicate the histogram graphics to show the distribution of the active range of motion of the shoulder movements cited previously.

The black curves represent the Gaussian distribution, or the normal distribution, which is a probability distribution that is symmetric about the mean. As indicated, there is no significant difference between the three methods, and this is what Table 1 shows.

Reliability results

Firstly, we did a test of the reliability of MediaPipe. All participants were measured at three different times. The MediaPipe reliability results of shoulder motion pose are noted in Table 2. It includes the pose of the proposed movements with Mean, StDev, SEM, MDC, and ICC values. Through the ICC, the presented AI method has excellent reliability because all the results are > 0.81 in the proposed movements.

Validity results

To get the validity of the MediaPipe, a 95% limit of agreement (LOA) between MediaPipe-based measurement and the two other instruments calculated, is calculated as follows: The results of Bland-Altman plots comparing the two instruments with the MediaPipe are obtained by the MediaPipe presented Table 3. In the shoulder extension, the mean differences of the UG from the MediaPipe technique were -0.01 • , -0,108 • in the adduction. -0,55 • in the abduction and -0,14 • in the flexion. In the extension, the mean differences of the angle ruler from the MediaPipe technique were -0.36 • . -0,552 • in the adduction, -0,6604 • in the abduction, and -0,14 • in the flexion. As shown in the Figure 9, the differences in the studied pose angles acquired by the two measurement methods are not distributed in a systematic way. However, a random distribution for all the studied motions. These values fall within a narrow range, whether comparing the MediaPipe with the goniometer or the MediaPipe with an angle ruler. So, they indicate a generally superior agreement. According to [START_REF] Bland | Statistical methods for assessing agreement between two methods of clinical measurement[END_REF], the validity analysis gives good results for all proposed motion poses. This study proved that there is no difference in calculating some ROMs utilizing the three different methods. The proposed new technology could help physiotherapists with telerehabilitation and be useful for clinical rehabilitation.

= ( ) + 1, 96 . (6) = ( ) -1, 96 . (7)

MediaPipe's contribution to telerehabilitation

Site web overview

The website has been created and developed for several reasons and factors. What the world experienced during the COVID-19 period and the variants that emerged like Delta and Omicron need to be considered in future scientific research and work to find practical solutions that help the patient during epidemics and crises in particular, and in general on all days. In addition, the patient often suffers from difficulty moving. Therefore, it is recommended to do rehabilitation exercises at home, and this is done by monitoring the patient via the site and remotely and accurately by watching the data in real-time. The third reason is to avoid transportation costs and other related costs. Whereas, in third-world countries, there is a difficulty in moving with the scarcity of transportation and the concentration of major hospitals and health centers in major cities. It makes it difficult for those living in villages and rural areas. For example, here in Algeria, most of the area is desert, and moving around is tiring, exhausting, and expensive. In addition to the scarcity of transportation, especially in desert areas, the patient must travel more than 200 km to reach the nearest hospital or health center. Another critical factor is hospital overcrowding, as the patient often waits more than a month to get the earliest appointment. In addition to all these reasons and factors, wearable sensors or Kinect sensors can be dispensed with through the website and by the method used. They are the most widely used equipment in telerehabilitation and thus save costs. In Algeria, this equipment is too expensive for most people to buy because it costs more than the basic wage of the average worker.

The proposed website focuses on artificial intelligence techniques to visualize the angles of the desired joints of the human body in real-time. Facilitating the use of the website is the first objective. The platform includes two stations; the first is for the patient, and the second is for the physician or doctor. The accounts are created by an admin (show figure 10). For the privacy of the data, each patient and doctor has an e-mail and password.

The doctor can schedule appointments and patients involved in rehabilitation sessions. The manager does all the technical procedures. Communication the doctor and the patient is done via the Internet. To facilitate the process for the patient, we relied on the simplicity of the way to use the site. Because we know we are dealing with different segments of society. In addition, there is a page on the site that helps explain how to use it to the patient. Also, there are no complications on the patient page; just two buttons to choose if the rehabilitation is in the lower or upper extremities, and a button to enter the meeting with the doctor (show figure 11).

When the meeting begins and after the conversations, the patient must stand in front of the computer or the phone, two or three meters away, and start exercising. Here the doctor can monitor the patient from a distance and see the values of the ROM in real time. On the screen and through the curve that shows the values for each frame. Then, the doctor can record the values, keep them, and compare them with the values of the upcoming sessions. After the meeting, the data can be kept in many formats; XLS, PDF document, CSV, JPEG, or PNG image (show figure 12).

Python and node.js were used as programming language, and Nginx 1.10.3 was utilized as the web server and inverse proxy. PostgreSQL is utilized as the primary data store for the web. Select 2, Moment.js 2.17.1, jQuery UI

Patient case study

A physical therapist often uses a goniometer to measure ROM and pursue a patient's condition in rehabilitation sessions. In this case, the doctor is often with the patient. Advanced tools such as wearable sensors or Kinect are required in telerehabilitation, but they are often expensive. In this research, a website is proposed. The MediaPipe library is integrated to show the ROM's value in real-time. Through this technique and from the data of several healthy people, it has been observed that a healthy person can reach 20 • of flexion. The patient finds it difficult to do the flexion, and this is why he needs rehabilitation sessions. Before starting to continue the rehabilitation process, approval was first obtained from the doctors in order to help and take their advice and directions. Also, the patient signed our request to allow us to take the photos, videos, and data needed for this study.

This study followed a 21-year-old patient injured in a traffic accident. He was operated on for a fractured right kneecap. His first day of rehabilitation took place on December 30, 2021. The blue graph (figure 13) represents the data of a healthy person. We use his data to facilitate the comparison between the healthy subject and the patient. Furthermore, to clarify the progress of the patient. From the first curve, we can see that the patient has difficulty bending his knee. The data shows that the flexion value is 142 • , the lowest value. After five days, we noticed no noticeable improvement in the patient, and that was due to the pain he was in somehow; he had not adopted and was unresponsive to rehabilitation exercises, which is what is observed through the curve (Record 2). On January 19,2022, we noticed a remarkable development in the bending process, and it was because the patient was attending massage and rehabilitation sessions even at home, which helped him achieve more degrees in the flexion "about 54 • ". In the fourth recording, a noticeable improvement and speed in the bending process were noticed compared to previous sessions. Here, the patient begins to reach essential stages of recovery, where he performs the flexion-extension process smoothly and at considerable speed. Here, the minimal value of the flexion is about 67 • . Then (Record 5), the patient reached about 42 • , which means a significant improvement. In the last session, the flexion reached 39 • . It was also noted that the pain had gone down a lot after the rehabilitation process. Finally, the technology helped the physiotherapist monitor the patient's condition remotely. Both sides strongly recommend this, and many doctors are interested in remote rehabilitation because it will make the operation much easier for the patient and cut down on the number of people in hospitals and the time it takes for patients to get to their appointments.

Discussion

The range of motion evaluation is a crucial component of musculoskeletal exploration [START_REF] Gajdosik | Clinical measurement of range of motion: review of goniometry emphasizing reliability and validity[END_REF]. A goniometer is often the most commonly used method by kinetherapists. But its use is manual only, and with the development of telerehabilitation techniques, it is better to search for modern methods that are more accurate and comfortable for the patient. This study reports the reliability and validity of the MediaPipe-based system for measuring four active shoulder ranges of motion. Compared with the clinical goniometer and the Kinect results cited in Table 4, and with the results obtained with MediaPipe, we can say that the MediaPipe technique has excellent reliability and is relatively accurate. The rehabilitation process often takes weeks. Through the proposed website, the doctor can monitor the condition of his patient remotely. It can also guide and correct the position. Often, the patient feels more comfortable doing rehabilitation at home than moving to a hospital or rehabilitation center. Also, through this work, the patient avoids many expenses such as transportation and other costs. It also avoids the trouble of traveling, as it desperately needs rest. The proposed technique is also inexpensive, and it is coughed up in terms of use. The most important point that we took into consideration is the protection of patient data, because, as we know, patient data is very sensitive and medical privacy is important. That is why we protect the data and accounts of all patients. There are some limitations that must be taken into consideration when we are doing the experiments. The most prominent of these is the misuse of technology by a segment of society, especially elderly people. It requires foreign intervention. The proposed technique has so far only been programmed to detect one person per frame. It may not be appropriate to use it when there are multiple people in the image frames.

From what we have noticed in our experience, we must choose the best position in front of the camera for good detection. An arbitrary position may have an effect on the pose estimation. Moreover, it is better to work with the application in good lighting conditions because experiments show noise introduction due to lighting conditions.

One of the main drawbacks is the instability of the internet. That is why we proposed another solution, which is to record the session through the development of a WebApp and then send the results to the doctor, who will in turn evaluate the results and send his feedback to the patient. Another limitation that can be added as future work is how these experimental results correlate to more complex movement types, for example, reach-to-grasp movements.

Conclusion

The goal of this article is to test the reliability of using this novel technique in telerehabilitation precisely. The ICC results demonstrate that the MediaPipe technique is excellent for studied motions. The 95% LOA results of the presented rehabilitation exercises concluded that the MediaPipe provides near results to the two other instruments. As stated by Bland-Altman analysis results, it is evident that the presented MediaPipe technique is valid for use in telerehabilitation.

Furthermore, this study presents a website for telerehabilitation to control rehabilitation at a distance using the technique of MediaPipe, the progress of which is an active range of motion (AROM) of patients in real-time. The results showed that it is an ideal idea to develop a home-based rehabilitation system using just a computer or tablet, so the cost is less than using a wearable sensor or Kinect. In the future, we plan to study and test the technique in different situations with different types of complex movement and arbitrary orientation.

Figure 1 :

 1 Figure 1: Instruments used in the measurement of ROM:A-Goniometer, B-Angle ruler.

Figure 2 :

 2 Figure 2: BlazePose model 33 human Poses [27].

Figure 3 :

 3 Figure 3: Angle joint calculation.

Figure 4 :

 4 Figure 4: Some pose estimation.

( a )

 a Comparison of shoulder abduction with three techniques. (b) Comparison of adduction with three techniques. (c) Comparison of shoulder extension with three techniques. (d) Comparison of shoulder flexion with three techniques.

Figure 5 :

 5 Figure 5: Comparison of the three methods of measurement in studied motions.

Figure 6 :

 6 Figure 6: Shoulder ROMs frequencies with MediaPipe method of measurement.

Figure 7 :

 7 Figure 7: Shoulder ROMs frequencies with goniometer method of measurement.

Figure 8 :

 8 Figure 8: Shoulder ROMs frequencies with angle ruler method of

Figure Bland -

 Bland Figure Bland-Altman plots a comparison between MediaPipe and UG and MediaPipe and an AR.

1. 11 . 4 ,

 114 jQuery. 1.11.1 were used as libraries in JavaScript. To track and control the range of motion, a library called MediaPipe was integrated with the website.

Figure 10 :

 10 Figure 10: Registration for patient account.

Figure 11 :

 11 Figure 11: Patient station view.

Figure 12 :

 12 Figure 12: Physiotherapist view during the session.

Figure 13 :

 13 Figure 13: RoM data of knee flexion extension of patient in six records

Table 1

 1 Data of four shoulder motions obtained using universal goniometer, angle ruler and MediaPipe.

	Movement Meth Mean	SE Mean StDev Min	Q1	Med	Q3	Max
	Abduction Go	177.76 • 0.48 •	3.41 •	160 •	177.5 •	179 •	180 •	180 •
		AR	177.86 • 0.45 •	3.20 •	161 •	177.07 • 178.95 • 179.65 • 180 •
		Me	177,20 • 0,49 •	3,49 •	160 •	177 •	178,50 • 179 •	180 •
	Flexion	Go	169.12 • 0.88 •	6.28 •	155 •	163.75 • 174 •	174 •	180 •
		AR	169.54 • 0.89 •	6.30 •	157.50 • 163.87 • 170 •	175.10 • 179.7 •
		Me	169,40 • 0,93 •	6,61 •	155 •	165 •	170 •	176 •	180 •
	Adduction Go	64.50 •	0.94 •	6.66 •	51.00 •	59.75 •	60 •	64 •	75.00 •
		AR	64.95 •	0.95 •	6.71 •	51.20 •	60.25 •	64.20 •	70 •	75.10 •
		Me	64.40 •	0.95 •	6.76 •	50.00 •	60 •	64 •	70 •	75.00 •
	Extension	Go	56.89 •	0.49 •	3.51 •	45.00 •	55.00 •	58.50 •	59.00 •	60.00 •
		AR	57.24 •	0.51 •	3.63 •	44.80 •	56.10 •	58.50 •	60.00 •	60.90 •
		Me	56.88 •	0.51 •	3.63 •	45.00 •	55.00 •	58.20 •	60.00 •	60.00 •

Abbreviation : Meth : method, Go : clinical goniometer, AR : Angle Ruler, Me : MediaPipe, SE Mean :standard error of the mean, StDev : Standard deviation, Min : minimum, Med : median, Max : maximum.

Table 2

 2 Reliability results of the MediaPipe based measurement system.

		ICC	StDev	SEM	MDC
	Abduction	0.968	2.11 •	0.377 •	1.046 •
	Adduction	0.99	5.778 •	0.5778 •	1.601 •
	Extension	0.99	4.296 •	0.4296 •	1.190 •
	Flexion	0.992	6.725 •	0.6725 •	1.864 •

Table 3

 3 Bland-Altman analysis results of shoulder joint data acquired utilizing Universal goniometer, angle ruler and MediaPipe.

		Goniometer vs MediaPipe	Angle ruler vs MediaPipe
	ROM	Mean bias	95% LOA	Mean bias	95% LOA
	Shoulder abduction	-0,55 •	-2,9436 • to 1,8308 •	-0,6604 •	-3,2865 • to 1,9657 •
	Shoulder adduction	-0,108 •	-1,774 • to 1,558 •	-0,552 •	-2,44 • to 1,34 •
	Shoulder extension	-0,01 •	-1,8916 • to 1,87 •	-0,36 •	-2,4327 • to 1,7127 •
	Shoulder flexion	-0,14 •	-2,5998 • to 2,3198 •	-0,14 •	-2,5998 • to 2,3198 •

Table 4

 4 MediaPipe ICC results compared to other devices and techniques.

		Mediapipe	UG	Kinect
	Movement	ICC	ICC	ICC
	Abduction	0.968	from 0.58 to 0.99 [38, 39, 40, 41]	0.86-0.98 [15]
	Adduction	0.99	0.98[42]	0.59 [17]
	Extension	0.99	0.91[42]	0.62 [15, 43, 44]
	Flexion	0.992	0.53 to 0.96 [39, 40, 41]	0.85-0.99 [15, 43, 44]

Acknowledgments

The authors thank all the patients and medical staff for participating in this study. It is noted that this research was supported by the Directorate General for Scientific Research and Technological Development (DGRSDT) in Algeria (Project PRFU No. A01L09UN210120220001).

Availability of data and materials

The data that support the findings of this study are available upon request from the authors.

Ethics approval and consent to participate

No indication of ethics is given.

Consent for publication

Permission has been taken from all patients and participants in this research project to share their data or photographs.

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Authors' contributions

All authors have been personally and actively involved in substantial work leading to the paper and will take public responsibility for its content.


Laurent Koessler
email: laurent.koessler@univ-lorraine.fr

Samuel Louviot

Jacek Dmochowski

Jean-Pierre Vignal

Jacques Jonas

Louise Tyvaert

Louis Maillard

Keywords:

In principle, epilepsy including recurrent seizures resulting from excessive neuronal discharges should be an ideal candidate for tDCS, expected to rebalance the cortical excitability. Transcranial direct current stimulation(tDCS) has been used in preclinical studies [START_REF] San-Juan | Transcranial direct current stimulation in epilepsy[END_REF] and some of them have been translated to clinical studies for reducing the seizure frequency. In drug-resistant epilepsy, occurrence of epileptic discharges significantly decreased after several tDCS sessions compared to shamcontrolled groups. Based on these results, a recent review proposed a level B recommendation ("probable efficiency") for tDCS in epilepsy [START_REF] Fregni | Evidence-based guidelines and secondary meta-analysis for the use of transcranial direct current stimulation in neurological and psychiatric disorders[END_REF]. However, the effect of tDCS on epileptic seizures in humans is largely unknown, due to difficulties in recording (unpredictability of seizure) and analyzing (noise and artifacts) scalp EEG signals during tDCS [START_REF] Liu | Immediate neurophysiological effects of transcranial electrical stimulation[END_REF]. To our knowledge, one only one study attempted to prevent the generation of seizures in epileptic patients using tDCS, and this approach was not effective [START_REF] Tekturk | The effect of transcranial direct current stimulation on seizure frequency of patients with mesial temporal lobe epilepsy with hippocampal sclerosis[END_REF]. Here, thanks to a rare opportunity of simultaneous tDCS and intracranial EEG (iEEG) recordings, we investigated the effect of tDCS on focal epileptic seizure with direct and online video-iEEG recordings in a drug-resistant epileptic patient. With this rare dataset, our aim was to demonstrate that ictal epileptic discharges and clinical symptoms recorded during tDCS were significantly different and reduced as compared to those recorded without tDCS.

MC is a female (19-year-old) suffering from refractory insuloopercular epilepsy. She gave informed consent to participate in this study (NCT03644732). Based on the presurgical evaluation, a video-iEEG recordings was planned to localize the epileptogenic zone (EZ) and the eloquent cortices around. The patient was stereotactically implanted with seventeen multi-contact iEEG electrodes (215 recording contacts) targeting the left hemisphere. HD-video iEEG recording was performed 20 hours a day for four days. Drug treatment remained stable during the iEEG evaluation. To target the EZ, a T7 (anode over the EZ [START_REF] Koessler | Automated cortical projection of EEG sensors: anatomical correlation via the international 10-10 system[END_REF]) -C6 (cathode) montage was chosen using two HD electrodes (12mm diameter). First, a bipolar transcranial alternating current stim-ulation (tACS) of 1mA amplitude was performed to quantify the induced electric field (EF) [START_REF] Louviot | Transcranial Electrical Stimulation generates electric fields in deep human brain structures[END_REF]. Second, a resting state session of transcranial direct current stimulation (tDCS) was performed to modulate the interictal epileptic discharges. The tDCS parameters were +1mA (T7) and -1mA (C6) and a 20-min total duration was set. Due to clinical considerations, TES were done the last day of the iEEG recording i.e., just before the removal of the intracerebral electrodes. The four-day of video-iEEG recordings were visually analyzed by the two epileptologists blind to the tDCS experiment. Datasets of iEEG signals were sorted into ictal epileptic discharges without tDCS (IED) and ictal epileptic discharges during tDCS (IED*). Same-sized epochs (12 seconds) were defined considering the mean seizure duration. FFT analysis of iEEG signals was performed using bipolar montage. The mean amplitude and standard deviation were computed for each frequency band ( ) of the different epochs. Then, a Friedman statistical test, with post-hoc pairwise Bonferroni corrections, was performed to compare all IED. The global percentage differences at the different frequency bands were calculated. In addition, the epileptogenicity of the brain structures was quantified using a semi-automatic calculation of the "epileptogenicity index" [START_REF] Bartolomei | Epileptogenicity of brain structures in human temporal lobe epilepsy: a quantified study from intracerebral EEG[END_REF] (EI). Finally, clinical symptoms during the seizures were retrospectively reviewed by two epileptologists blind to the tDCS experiment using HD video recordings.

During the four days of iEEG investigation, five spontaneous seizures occurred and were recorded in S′1-2 to 8-9 and O′5-6 to O′9-10 contacts (Fig. 1A; Supplementary Fig. 1) i.e., the posterior long insular gyrus and the basal part of the parietal operculum. Strikingly, IED durations were 35-37 seconds long whereas IED* was 12 seconds long (p < 0.001; Student's T-test). Mean EF magnitudes in the EZ ranged from 0.18 to 0.21 V/m. At the beginning of the seizures (Epoch A), no significant amplitude spectrum difference was found between IED* and all other IED. During epochs B and C, IED* amplitude distributions were significantly higher than the four IED in the delta, theta, and https://doi.org/10.1016/j.brs.2023.03.040 1935-861/© 20XX Note: Low-resolution images were used to create this PDF. The original images will be used in the final composition. alpha bands. In higher frequency bands (beta, gamma1 and gamma2), the amplitude spectrum distributions of the IED reversed and became significantly lower than the IED*. In the last segment D, no significant difference was found (Fig. 1B; Supplementary Table 1). The epileptogenic index of IED* was null in all bipolar contacts whereas from 4 to 5 bipolar contacts were significantly detected with EI > 0.3 for the four IED (Supplementary Fig. 2 andtable 2).

For the first four epileptic seizures, the first clinical symptom was facial reddening and the patient verbally warned. Then, she had a right upper limb tonic extension, a body rotation, a head flexion, a left lower and upper limb tonic extension. Finally, the patient smiled. The dura-tion of the clinical symptoms was 37, 40, 36, 36 seconds. For the seizure recorded during tDCS, the beginning and the end were like the first four epileptic seizures. However, strikingly, the major disabling motor symptoms disappeared (Supplementary Fig. 3). The duration of this seizure during tDCS (22 seconds) was significantly reduced (p = 0.005, Student's T-test).

Thanks to this rare dataset, we provide here original evidence of both electrophysiological and clinical epilepsy symptoms reductions during a single tDCS session at low intensity and for few minutes duration. Our report is important because it offers both fundamental insights in the intra-cortical effects of tDCS on epileptic discharges and

C O R R E C T E D P R O O F
therapeutic perspectives. The most crucial feature of the epileptic seizure, i.e., the low amplitude fast discharge, was completely cancelled and the most disabling and dangerous part of the clinical symptoms disappeared. Regarding the underlying mechanism of the tDCS-related seizure reduction, the main hypothesis would be the inhibition of the EZ i.e., a reduction of cortical hyperexcitability and thus of the ability of cortical tissue to produce a high-energetic fast epileptic discharge. Since the seizures during and without tDCS were identical during the pre-ictal period but then differed in terms of absence of the low-voltage fast discharge, the tDCS effect could be more precisely related to a neuromodulation of the interneurons. Indeed, several studies have shown that interneuron activity leads to the generation of low-voltage fast discharge observed at the seizure onset [START_REF] Cammarota | Fast spiking interneuron control of seizure propagation in a cortical slice model of focal epilepsy[END_REF][START_REF] Shiri | Interneuron activity leads to initiation of low-voltage fast-onset seizures[END_REF][START_REF] Wendling | Interictal to ictal transition in human temporal lobe epilepsy: insights from a computational model of intracerebral EEG[END_REF].

Fig. 1 .

 1 Fig. 1. A. iEEG signals in bipolar montage of selected contacts in the epileptogenic zone (S′ and O′ electrodes). The five epileptic seizures started with polyspike discharges in the band (period A). Then, for seizures 1-4, low amplitude fast discharges occurred in the band (B and C periods) and seizures stopped after around 36 sec. In contrast, seizure #5, recorded during tDCS, stopped after 12s and no amplitude fast discharges was observed. (Red arrow: end of the ictal epileptic discharges). B. Amplitude spectrum of the IED* (red line) compared to the IED (n = 4; gray lines) in the epileptogenic zone (O′5-10; S′1-9 iEEG contacts). The different frequency bands are colored (cyan: band; orange: band; purple: band; yellow: band; red: γ 1 band; green: γ 2 band). IED: ictal epileptic discharge; * mentioned that the discharges were recorded during tDCS. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Acknowledgements

We thank the patient MC for her willingness to take part in the research protocol during iEEG exploration. We thank Bruno Rossion for his comments on a previous version of this manuscript. This study was funded by grants from the Lorraine University of Excellence Initiative (France) and the City College of New-York (USA) (DrEAM mobility grant) and the Region Grand Est (France).

Data sharing

Original data of this study are available (https://doi.org/10.17632/ 3krd4hrk86.2).

Declaration of competing interest

JD has patent rights to HD-tDCS technology.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https:// doi.org/10.1016/j.brs.2023.03.040.


Trajectory planning issues in cuspidal commercial robots

Durgesh Haribhau Salunkhe 1 , Damien Chablat 1

and Philippe Wenger 1

Abstract-A cuspidal serial robot can travel from one inverse kinematic solution to another without crossing a singularity. Cuspidal robots ask for extra care and caution in trajectory planning, as identifying an aspect related to one unique inverse kinematic solution is not possible. The issues related to motion planning with cuspidal robots are related to the inherent property arising from the geometric design of the robot. The cuspidality property has not been considered in recent industrial 6R robots with a non-spherical wrist. In this work, cuspidality is illustrated with the JACO robot (gen 2, non-spherical wrist), a serial arm by Kinova Robotics which is deployed in various applications and is cuspidal in nature. A nonsingular change of solutions for the robot is provided to highlight the effect of cuspidal robots on the interference with the environment. The pose with multiple inverse kinematic solutions in an aspect is presented. Problems in choosing the initial solution of the path in cuspidal robots, and its consequence, is illustrated with an example path in the workspace of the JACO robot. The paper presents the importance of cuspidality analysis of 6R robots and the implications of neglecting it.

I. INTRODUCTION

Wrist-partitioned anthropomorphic serial robots have been conventional designs for 6R commercial robots. The inverse kinematics and the singularities of such robots can be easily determined. Since the determinant of the Jacobian matrix, J, factors into three components, the singularity-free connected regions in the joint space, named aspects, can be determined and visualized. From the analysis of these robots, it was assumed that every inverse kinematic solution of a serial robot lies in an aspect. This was proved to be incorrect, and counterexamples were presented in 1988 by Parenti-Castelli [START_REF] Parenti-Castelli | Position analysis of robot manipulators: Regions and subregions[END_REF] in two 6R robots and by Burdick in 1989 [START_REF] Burdick | On the Inverse Kinematics of Redundant Manipulators: Characterization of the Self-Motion Manifolds[END_REF] in 3R robots. Cuspidal robots are serial manipulators that can travel from one inverse kinematic solution (IKS) to another without crossing a singularity. The ability to follow a path effecting the change of inverse kinematic solutions without crossing a singularity is termed as cuspidality and the path is called a nonsingular change of solutions. The name 'cuspidal' originated from the existence of a cusp in the singularity locus in the workspace of cuspidal 3R robots [START_REF] Omri | How to recognize simply a non-singular posture changing 3-dof manipulator[END_REF]. Cuspidality exists in both serial and parallel manipulators [START_REF] Innocenti | Singularity-free evolution from one configuration to another in serial and fully-parallel manipulators[END_REF], [START_REF] Wenger | A Review of Cuspidal Serial and Parallel Manipulators[END_REF], but this paper focuses on serial manipulators only. Most commercial robots implemented in the past are non-cuspidal and the posture in which a non-cuspidal robot is operating can be easily identified with the signs of the factors of the determinant of J [START_REF] Borrel | A study of multiple manipulator inverse kinematic solutions with applications to trajectory planning and workspace determination[END_REF]. Instead, posture identification is not possible in cuspidal robots as the determinant does not usually factor and there 1 Nantes Université, École Centrale Nantes, CNRS, LS2N, UMR 6004, 44000 Nantes, France {durgesh.salunkhe, damien.chablat, philippe.wenger}@ls2n.fr are multiple solutions in one aspect [START_REF] Wenger | A New General Formalism for the Kinematic Analysis of All Non-redundant Manipulators[END_REF]. Cuspidal robots were first formalized in [START_REF] Wenger | A New General Formalism for the Kinematic Analysis of All Non-redundant Manipulators[END_REF] and later extensively studied in [START_REF] Wenger | Comments on "A classification of 3R regional manipulator geometries and singularities[END_REF]- [START_REF] Wenger | Cuspidal and noncuspidal robot manipulators[END_REF]. Different approaches were implemented to identify and classify 3R orthogonal robots (i.e. robots with three mutually orthogonal joint axes), based on cuspidality. Recently, it was proved that the existence of a cusp point is a necessary and sufficient condition for generic 3R robots to be cuspidal [START_REF] Salunkhe | Necessary and sufficient condition for a generic 3r serial manipulator to be cuspidal[END_REF]. A certified algorithm to decide the cuspidality of non-redundant revolute-jointed robots was presented [START_REF] Chablat | Deciding Cuspidality of Manipulators through Computer Algebra and Algorithms in Real Algebraic Geometry[END_REF], allowing a designer to verify his designs for cuspidality. The cuspidality analysis of 3R robots can be extended to wrist-partitioned 6R robots. This is attributed to the reason that the singularities in the position and the orientation for such robots are decoupled. While 3R cuspidal robots have been extensively studied, the cuspidality study of 6R robots with non-spherical wrist is still ongoing, while more and more such robots appear on the market [START_REF] Verheye | Why hasn't anyone heard of cuspidal robots?[END_REF].

The issues related to path following in cuspidal robots are not only related to the algorithms used for trajectory planning, but also due to the inherent property arising from the geometric design of the manipulator. As designers move away from the conventionally implemented wrist-partitioned anthropomorphic architecture, there is a high chance that such robots have completely different kinematic properties. The cuspidality property seems to have been slipped from the consideration of the current designers around the world [START_REF] Verheye | Why hasn't anyone heard of cuspidal robots?[END_REF]. Recently, several industrial 6R robots with a nonspherical wrist have appeared. To the best of our knowledge, the only 6R robots with a non-spherical wrist that have been analyzed for the cuspidal property, are the UR-series robots from Universal Robots. Computer algebraic tools were used to show that these robots have eight aspects with no more than one IKS per aspect [START_REF] Capco | Robots, computer algebra and eight connected components[END_REF], leading to a conclusion that such robots are non-cuspidal. This is because UR robots have three parallel joint axes, a geometric simplification that was identified by Pieper as a solvability criterion [START_REF] Pieper | The Kinematics of Manipulators Under Computer Control[END_REF]. In the absence of 3 parallel joint axes, 6R robots with a non-spherical wrist are likely to be cuspidal.

In this work, cuspidality is illustrated with the JACO robot (gen 2, non-spherical wrist) from Kinova robotics, which is used in various applications. The JACO robot was mainly designed for rehabilitation [18] and thus is employed near persons with disability and interacts with them in a few cases. Multiple IKS that belong to the same aspect are presented, and the implication of cuspidality on path planning is highlighted. The dangers of neglecting the cuspidality are detailed and the rationale behind limiting their use in collaborative environments is motivated too.

II. PRELIMINARIES

In this section, few definitions are presented for a complete understanding of the cuspidal robots. Joint limits and obstacles are not considered in the current study.

A. Recall of important definitions

Aspects: The aspects are the largest singularity-free connected regions in the joint space of a robot [START_REF] Borrel | A study of multiple manipulator inverse kinematic solutions with applications to trajectory planning and workspace determination[END_REF].

Cuspidal robot: A cuspidal robot is a robot for which there exists a path in the joint space connecting two IKS without crossing the locus of critical points (i.e., meeting a singularity) in the joint space [START_REF] Wenger | Changing posture for cuspidal robot manipulators[END_REF].

Nonsingular change of solutions:

A nonsingular change of solutions is a connected path between two IKS which does not cross the locus of critical points in the joint space. In the workspace, a nonsingular change of solutions forms a closed loop. Indeed, the robot trajectory ends up at the same pose it started from, but in a different joint configuration. The singularities of generic 6R robots depend on θ i , i = 2, ..5, and thus lie in a 4-dimensional space that cannot be visualized. It can be confirmed that a given change of solutions path is nonsingular if the determinant of the Jacobian matrix does not change sign along the path. Fig. 1 shows an example of a nonsingular change of solutions in a 3R robot. It can be seen that the path in the joint space never crosses the locus of critical points (shown in blue lines) and, accordingly, the determinant value does not change signs throughout the path.

B. Path repeatability of cuspidal robots

If a non-cuspidal robots is able to follow a prescribed closed path once, it can repeat the path in same direction multiple times. This is not always the case in cuspidal robots, where path repeatability depends on the initial IKS. This is well established for 3R robots: a closed path corresponding to a nonsingular change of solutions can be followed only once in the same direction, no matter which initial IKS is chosen [START_REF] Wenger | Uniqueness Domains and Regions of Feasible Paths for Cuspidal Manipulators[END_REF]. For 6R robots, the situation is even more delicate. It was reported in [START_REF] Parenti-Castelli | Position analysis of robot manipulators: Regions and subregions[END_REF] that some 6R robots can repeat a closed path corresponding to a nonsingular change of solutions, while Fig. 1: Example of nonsingular change of solutions in the joint space and the progress of determinant value on the path Robot parameters:

d = [0, 1, 0], a = [1, 2, 3 2 ], α = [-π 2 , π 2 , 0], path = (-3, -0.5) to (-0.742, 2.628).
others cannot. This property of 6R robots to repeat or not a nonsingular change of solutions path has further implications in trajectory planning. If the pose of the end-effector can be changed with manual intervention and the trajectory is not pre-planned, it is not possible to identify the initial choice of IKS. Due to this reason, the use of 6R cuspidal robots in an interactive environment may not be suitable as it can lead to interference between the robot and its environment.

III. KINEMATIC ANALYSIS

In this section, the kinematic analysis of the JACO robot is presented. The methodology implemented to get the IKS is briefly revisited, and instances of end-effector pose with varying IKS are provided.

A. Inverse Kinematics

Table I shows the DH parameter table for the JACO robot. Here, d i denotes the distance along the axis of i th motor, a i is the offset between i th and (i + 1) th frame and α i is the orientation of (i + 1) th the axis with respect to the i th axis. The parameter d 6 does not affect the inverse kinematic solution and thus can be kept as a variable. For the purpose of this paper, d 6 = 160 mm. The JACO robot has an offset in the wrist, and thus a direct decoupled analysis cannot be implemented. Inverse kinematics for such robots with an offset in the wrist can be obtained by previously established algorithms for generic 6R robot [START_REF] Raghavan | Inverse Kinematics of the General 6R Manipulator and Related Linkages[END_REF], [START_REF] Husty | A new and efficient algorithm for the inverse kinematics of a general serial 6r manipulator[END_REF] or with methods applicable specifically to offset in the wrist [START_REF] Trinh | A Geometrical Approach to the Inverse Kinematics of 6R Serial Robots With Offset Wrists[END_REF], [START_REF] Gosselin | Polynomial Inverse Kinematic Solution of the Jaco Robot[END_REF]. In this paper, the results are presented by using the methodology detailed in [START_REF] Gosselin | Polynomial Inverse Kinematic Solution of the Jaco Robot[END_REF] as it is specific to the JACO design and is easy to implement. Using this method, a 16-degree-polynomial in tan θ 1 2 is obtained by eliminating the other joint variables. This polynomial with rational parameters is solved with the Isolate function [START_REF] Rouillier | Solving zero-dimensional systems through the rational univariate representation[END_REF] in RootFinding library available in Maple 2020. Then, straightforward backpropagation is used to obtain the rest of the joint variables.

B. Number of IKS in the JACO robot

It is a well-known result that a wrist-partitioned 6R robot has at most eight solutions. This is attributed to the decoupling between the positions and orientation of the endeffector and thus can be solved independently. The first 3R chain forms a regional manipulator and its inverse kinematics can be solved with a four degree inverse kinematics polynomial or two quadratics in cascade. The wrist provides two solutions for each orientation and thus, yields maximum of eight (4 × 2) solutions for a wrist-partitioned 6R robot. This does not hold true for robots with an offset in the wrist such

IV. CUSPIDALITY ANALYSIS

In this section, it is shown that the JACO robot is cuspidal and a path corresponding to nonsingular change of solutions is presented. The twelve solutions are equally distributed in two aspects, and an IKS from one aspect can be connected to other five present in the same aspect.

A. Investigating connectivity

The solutions for a particular pose of the end-effector is given in table II along with the sign of the determinant of the Jacobian matrix, det(J), at the corresponding joint configuration. To search for a nonsingular change of solutions, a linear interpolation between two IKS with the same determinant value was evaluated. If the determinant value changed sign along the linear interpolation, then the path was divided in three parts and the two non-extreme points of the path were recalculated by using the Nelder-Mead approach [START_REF] Nelder | A Simplex Method for Function Minimization[END_REF], to find the possible nonsingular change of solutions. This is a working algorithm to determine cuspidality of any robot and is faster to implement when compared to a more robust certified algorithm in [START_REF] Chablat | Deciding Cuspidality of Manipulators through Computer Algebra and Algorithms in Real Algebraic Geometry[END_REF]. The advantage of using Nelder-Mead approach to determine the connectivity of the given IKS is that the method is versatile and can adapt to constraints easily. This implies that the working algorithm can be extended to find a nonsingular change of solutions with joint limits and collision constraints of the robot too. Figure 3 & 4 show the progress of det(J) along the joint paths obtained from the algorithm. It is apparent that there are six solutions in aspect det(J) > 0 and six solutions in aspect det(J) < 0.

B. Nonsingular change of solutions

The visualization of a nonsingular change of solutions is presented in this subsection to emphasize the impact of the choice of initial IKS on the robot's interaction with the environment. Figure 5 shows the end-effector path corresponding to the nonsingularchange of solution from VII to VIII (Fig. 4a). The end-effector path is a closed loop. The nonsingular change of solutions in 3R robots is well understood and can be anticipated [START_REF] Wenger | Uniqueness Domains and Regions of Feasible Paths for Cuspidal Manipulators[END_REF]. Indeed, the nonsingular change of solutions takes place strictly after crossing the locus of critical values (a workspace boundary associated to a singularity) and thus the trajectory passes through regions with different number of IKS [START_REF] Salunkhe | Necessary and sufficient condition for a generic 3r serial manipulator to be cuspidal[END_REF]. This behavior has not been proved in 6R robots, and thus it is unclear whether a nonsingular change of solutions implies a path that passes through different connected regions in the workspace. It is straightforward to conclude from fig. 5 that the kinematic properties along the desired path are a direct result of the choice of initial IKS.

V. ISSUES IN TRAJECTORY PLANNING

In this section, the challenges in planning a trajectory with a cuspidal robot are highlighted by using JACO robot. As shown in fig. 2, it is known that there are multiple connected regions with varying numbers of IKS. It is well established that when a workspace boundary associated with a singularity is crossed, at least two solutions disappear (resp. appear) if the robot move toward a region with less IKS (resp. with more IKS) [START_REF] Kohli | Workspace Analysis of Mechanical Manipulators Using Polynomial Discriminants[END_REF]. In classical path planning algorithms, a change of solution only occurs when det(J) changes sign. Thus, an end-effector path can be declared infeasible if det(J) changes In the case of robots, a jump to another IKS in the aspect can be experienced without being detected, and this jump results into going off the planned end-effector path. Such a behavior has been recorded in the MICO robot earlier but lacks a detailed explanation of the error [START_REF] Verheye | Why hasn't anyone heard of cuspidal robots?[END_REF]. It is surprising to note that the issue in path planning of the JACO Gen 2 robot has not been widely reported. The authors' best guess is that either most of the planning done using this robot is in the joint space or the solutions are just declared infeasible if a critical value is encountered. The issues in planning were discussed with the company that manufactures the robot.

A. Issue 1: Choice of initial IKS

In order to better explain the issue of choosing a suitable initial IKS, the case of a 3R robot is presented in detail. The workspace and singularities of 3R robots have been extensively presented in [START_REF] Wenger | Uniqueness Domains and Regions of Feasible Paths for Cuspidal Manipulators[END_REF], [START_REF] Salunkhe | Necessary and sufficient condition for a generic 3r serial manipulator to be cuspidal[END_REF], and are revisited briefly in this paper. Figure 6 shows the singularities in both the joint space and the workspace of an example of a 3R orthogonal robot with DH-parameters similar to the one mentioned in fig. 1. Figure 6b illustrates a closed loop path that starts from The dotted line represents the pose of the end-effector and the intersection of this line with the level set representation corresponds to one IKS. q i is the IKS in the joint space of the robot and the corresponding point on the level set representation is given as p i . Depending upon the initial IKS, q i , the path is either feasible or infeasible. It can be deduced from fig. 1, fig. 7 and fig. 8 that if the trajectory starts from initial IKS q 2 or q 3 , the closed loop path is infeasible. The path is feasible only if the initial IKS is either q 1 or q 4 . Closed-loop paths that do not result in a change of IKS can be repeated but this is clearly not the case of nonsingular solution changing paths. This dependency of path feasibility and repeatability on the initial IKS pose several challenges in planning trajectories of 6R cuspidal robots. Being aware of the cuspidal property of a robot, the choice of initial IKS may not be a problem if the complete trajectory to be followed and the repeatability condition for the path are known prior to execution. Thus, it is of great importance that such robots are strictly used in environments with pre-planned trajectories.

If deployed in collaborative areas (as is the present case for the Jaco robot), such robots are bound to error and can lead and11 show the time histories of the joint angles along the path. The dotted lines divide the path into four parts and represent the instances when the path crosses a region with a given number of IKS in the workspace. They are labeled as (i → j) denoting the change from a region with i solutions to the region with j solutions. The first and the fourth regions correspond to the path in a eight-solution region while the second and third regions correspond to the path in six-and four-solution regions, respectively. The blue color paths in each plot are the solutions in an aspect with det(J) > 0 while the red paths correspond to det(J) < 0. In fig. 10, T i , i = 1..8 is a trajectory corresponding to each initial IKS. It can be seen that if the robot starts the path from IKS corresponding to T 3 , T 4 , T 7 or T 8 , it soon meets a singularity. If the cuspidality of the robot is not taken into account, a sudden jump will take place to a solution available in the same aspect at the next discrete instance. For example, if the path is initiated from T 4 (resp. from T 3 ), after the first dotted line, (8 → 6), there will be a sudden jump to T 1 , T 5 or T 8 (resp. to T 2 , T 6 or T 7 ). In case of trajectory T 7 and T 8 , there will be a sudden jump after (6 → 4) to T 2 or T 6 and T 1 or T 5 , respectively. Some of these jumps are shown by green color lines in fig. 10. It is apparent that T 3 , T 4 , T 7 and T 8 are infeasible. Trajectories T 1 and T 5 whose initial IKS belongs to an aspect with det(J) < 0 necessarily correspond to a nonsingular change of solutions that are non-repeatable. The trajectory T 1 (resp. T 5 ) leads to change the IKS to the initial IKS of T 8 (resp. T 4 ). Trajectories T 2 and T 6 whose initial IKS belongs to an aspect with det(J) > 0 are the only two trajectories out of the eight possibilities that are repeatable and do not correspond to a nonsingular change of solutions.

Trajectories that pass through multiple regions with higher to demonstrate the issues in path planning is that it travels through multiple connected regions in the workspace. It suffices to say that the cuspidality analysis is essential particularly in the present scenario when designers choose unconventional designs. The issues in the trajectory planning of a cuspidal robot make it necessary that such robots are limited to applications involving a known environment and use of pre-planned trajectories.

VI. CONCLUSIONS AND FUTURE WORK

In this work, an example of a cuspidal commercial robot employed in applications involving human interaction is presented. This robot is close to conventional Puma-type robots but has a wrist offset. The kinematic analysis with the different IKS and the major difference from conventional wrist-partitioned robots has been detailed. The JACO robot was shown to be cuspidal with the help of connectivity analysis of the twelve IKS of the robot. A nonsingular change of solutions for the robot was then presented. The problems in trajectory planning arising from the innate kinematic property of the JACO robot was introduced. The issues related to the initial choice of IKS as well as the repeatability of a given path were detailed with an example trajectory. Many existing industrial robots like FANUC CRX series and Kinova Link 6 are cuspidal robots and thus the risks of deploying cuspidal robots in collaborative environments were also explained. In future, the cuspidality analysis of robots will be conducted considering joint limits and collision constraints. This will help analyzing practically feasible paths and their nature enhancing the trajectory planning of cuspidal robots. A path planning framework for cuspidal robots is also being designed to mitigate the issues presented in this paper.

Fig. 3 :

 3 Fig. 3: Progress of the det(J) for six nonsingular change of solutions in aspect with positive determinant value

Fig. 4 :

 4 Fig. 4: Progress of det(J) for six nonsingular change of solutions in aspect with negative determinant value

Fig. 5 :

 5 Fig. 5: Nonsingular change of solutions: IKS VII → VIII.

Fig. 6 :

 6 Fig.6: Singularities (blue lines) in the joint space and the workspace of a 3R orthogonal robot.

(a) Closed path starting from IKS q 1 (Fig. 7 :

 17 Fig. 7: An example of nonsingular change of solutions

Fig. 8 :

 8 Fig. 8: Same path as in fig. 7 but without changing solutions

Fig. 9 :Fig. 10 :

 910 Fig. 9: A closed trajectory crossing multiple connected regions in the workspace of JACO robot

  (a) Value of θ 2 along the path (b) Value of θ 3 along the path (c) Value of θ 4 along the path (d) Value of θ 5 along the path (e) Value of θ 6 along the path

Fig. 11 :

 11 Fig. 11: Plot for θ 2...6 of all solutions at discretized points along the path in fig. 9.

TABLE I :

 I The DH parameters of the JACO robot robot. A case of end-effector pose with twelve IKS was exhibited in[START_REF] Gosselin | Polynomial Inverse Kinematic Solution of the Jaco Robot[END_REF]. While, in the absence of joint limits and collisions, a pose in a nonsingular configuration of a wrist-partitioned 6R robot has either eight IKS or four IKS, it is shown that the Jaco robot can have two, four, six, eight, ten, or twelve IKS. It can be concluded from this finding that the workspace of the JACO robot is divided in multiple regions of varying number of IKS and thus the trajectory planning for such a robot is a non-trivial problem. This is because a desired path can cross singularities in the workspace and switch between multiple connected regions in the workspace with varying number of IKS. Figure2shows two different 2D slices in the workspace of the JACO robot with a fixed orientation and z-coordinate. The blue, yellow, dark orchid, green, magenta, and turquoise colors represent the end-effector pose with two, four, six, eight, ten, and twelve solutions, respectively. The slice in fig.2ahas regions with two, four, six, and eight solutions. It is worth noting that two voids appear in this slice. This is unusual in Puma-type robots. The slice in fig.2bhas four, six, eight, ten, and twelve solution regions.

	(a) Slice 1	(b) Zoomed section of Slice 2
	Fig. 2: Regions with different number of IKS in 2D slice
	(xy-plane) in the workspace of JACO. The fixed orientation
	as a quaternion (h) and fixed z-coordinate(mm) for the slices:
	Slice 1: h = 0.2565 + 0.033 î + 0.812 ĵ + 0.582 k, z = 560.56
	Slice 2: h = 0.984 + 0.004 î + 0.103 ĵ + 0.147 k, z = 257.94
	as the JACO
		i	d i (mm)	a i (mm)	α i (rad)	θ i (rad)
		1	275.5	0	π/2	θ 1
		2	0	410	0	θ 2
		3	13.3	207.3	-π/2	θ 3
		4	103.8	0	π/2	θ 4
		5	0	103.8	-π/2	θ 5
		6	d 6	0	0	θ 6

TABLE II :

 II Example enumeration of 12 IKS of a pose of the JACO robot.

	Orientation: h = 0.549 + 0.497 î + 0.423 ĵ + 0.522 k,
	position(x, y, z)(mm) = (140.49, 47.13, 324.876)
	IKS	θ 1	θ 2	θ 3	θ 4	θ 5	θ 6
	(sign(det))	(rad)	(rad)	(rad)	(rad)	(rad)	(rad)
	I (+)	3.0675	1.0545	1.3090	2.4283	-1.2305	-2.3002
	II (+)	2.4335	0.0936	1.5741	1.4311	2.3452	0.5391
	III (+)	-0.8579	3.0408	1.5721	-1.5912	2.1625	0.5390
	IV (+)	2.9132	0.2824	1.9297	-2.1648	-2.9685	-2.7165
	V (+)	-0.2812	2.0346	1.8631	-0.5833	-1.0917	-2.4130
	VI (+)	-0.2456	2.8156	1.3090	0.4882	-2.8301	-2.3003
	VII (-)	-3.1201	0.7082	1.4904	2.62	-1.9637	-1.8817
	VIII (-)	2.4730	0.0943	2.0281	-1.4916	-2.4244	2.4362
	IX (-)	-0.1583	2.7025	1.4699	-0.0656	-2.5402	-1.9078
	X (-)	-0.7501	1.9399	2.0268	-1.4270	0.6212	2.6291
	XI (-)	-0.8046	3.0466	1.1135	1.5103	-2.2697	2.4394
	XII (-)	2.5338	1.2022	1.1149	1.5839	0.6030	2.6322

ACKNOWLEDGMENT

The authors are supported by the joint French and Austrian ECARP project: ANR-19-CE48-0015, FWF I4452-N. The authors also thank Achille Verheyes, Freedom Robotics, for sharing his experience on trajectory planning with cuspidal robots in real world application.


Krishna Vamsi

Stanley Guda

Christine Mugisha

Damien Chablat Chevallereau

Chablat

V K Guda

S Mugisha

C Chevallereau

D Chablat
email: damien.chablat@cnrs.fr

ON HUMAN-COMPUTER INTERACTION, inPress. hal-04055526

Keywords: Safety, intermittent contact interface, human intention prediction, trajectory planning, human-robot collaboration, virtual reality

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction of a Cobot as intermittent haptic contact interfaces in virtual reality

Introduction of a Cobot as intermittent haptic contact interfaces in virtual reality 1 Introduction

Virtual reality technologies allow a user to get immersed in virtual worlds. Haptic technologies born from robotics have increased the immersion in these virtual worlds by providing the sensation of touch. In a Virtual Reality (VR) simulation, haptic interfaces allow a tangible and physical interaction with the virtual environment, but they must generally be constantly held in hand by the user and therefore do not allow objects to be touched in a natural way. At the same time, many applications require hands-on interaction without intermediaries. This is particularly the case for simulations that require tactile exploration of the physical properties of virtual objects.

Classical force feedback interfaces, also called here classic contact haptic interfaces (CHIs), are robotic systems allowing natural motion interactions with virtual or remote environments. They are used in several domains such as design, manufacturing, assembly, scientific visualization, entertainment, education, medicine, space, rehabilitation, micro manipulation and molecular biology. In all cases, they should provide adequate kinesthetic (force) feedback, contributing to enhance the sense of presence in the virtual environment.

With CHIs, the user is usually mechanically linked to the device's endeffector, typically a handle, whose movements, measured by the robot, are used to know the configuration (position and orientation) of his/her hand. This information is necessary to provide force feedback which is consistent with the virtual scene and the mechanical properties of the virtual object (VO) being touched. The mechanical link that is established when the user manipulates the haptic device has however a non-negligible influence since he/she experiences the friction, inertia and vibrations of the mechanical structure, even in free space where he/she is expected to feel nothing. Such unwanted sensations decrease the realism of the interaction since the user feels all the time the presence of the robot. In addition, the difference between free space and contact is less distinctively felt than in the real world.

In order to cope with these issues, several efforts can be made in terms of mechanical design, e.g. use of very lightweight and very stiff structures (even if an optimal trade off is difficult to attain) and more efficient transmission systems. Another approach consists in installing a force sensor at the level of the robot's end-effector in order to measure and compensate any resisting force in the direction of displacement. However, resisting forces can never be totally canceled and none of these approaches completely eliminates the feeling of the presence of the robot in free space.

Years of research on haptics, robotics, and interaction in VR have led to the development of a new generation of haptic devices called intermittent contact interfaces (ICI) [START_REF] Kim | Encountered-type haptic display for large vr environment using per-plane reachability maps[END_REF], [START_REF] Mcneely | Robotic graphics: a new approach to force feedback for virtual reality[END_REF], [START_REF] Araujo | Snake charmer: Physically enabling virtual objects[END_REF]. Intermittent-contact interfaces (ICIs) represent an original and promising approach aiming to cope with the aforementioned issues. Its principle consists in removing the mechanical link between the human operator and the force feedback interface during manipulations in free space and come at his/her contact only when force feedback is required. This solution implies the need to track and closely follow the user's movements in free space and to prevent him/her to move in the constraint direction when a VO is being touched. This way, the user doesn't feel any force in free space (perfect transparency) and the transitions from free space to contact are deemed to be felt more naturally as the robot really touches the user at the simulated contact moment. This approach aims to improve the realism of the interactions, however it suffers from several shortcomings. First, its efficiency has not yet been proven in terms of user safety. Second, even if IC interfaces are experimentally proven to be stable at low speeds, they tend to become oscillating at higher speeds.

Finally, despite the fact that a lot of tasks are performed by the mean of tools in the real world, most of existing ICIs focus on bare finger interactions and are therefore not optimal for simulating tool-mediated tasks.

User safety is central to the implementation of an ICI. To satisfy this constraint, we propose to use a cobot, for which we will develop specific path planning algorithms taking into account its low performance in terms of travel speed in the intrinsically safe mode, as well as all segments of the robot and the user's entire body in interference management. These algorithms will be based on a user activity prediction model that will allow both specifying a final desired location and the constraints to be respected when defining the path to reach this desired location. This paper summarizes the contribution of [START_REF] Guda | Contributions to utilize a cobot as intermittent contact haptic interfaces in virtual reality[END_REF] which introduces a series of research works aiming to leverage the usability of such a system. This research work is created to deal with the present limits in such a haptic device system.

Description of context

This work was developed under the frame of the French National Research Agency (ANR) LobbyBot Project. The LobbyBot project consisted in developing a system that could be integrated into an industrial application for automotive interior prototyping. The system is intended to be used to recreate an automotive cockpit for faster prototyping in VR for the Renault Group, a French automotive company. In this fast prototyping process, designers had to explore and evaluate the perceived quality of a virtual car interior with their sense of touch of different materials, shapes, and objects that could be arranged in a virtual automotive interior that could be easily configured in VR. This new paradigm was conceived as a means to save the Renault Group costs in budget and time for fabricating actual automotive cockpits that are exclusively used for prototyping purposes. The purpose of this project is the control of the robot to implement this ICI. Interaction techniques will allow managing any delays by the ICI in relation to the user. To increase the extent of sense of the form and materials can be restored by the use of an adapted pseudo-haptic return, and to follow a surface with the finger through appropriate sensory feedback. The integration of all the results into an ICI prototype will make it possible to validate the interest of the solution on an industrial case study. With current technologies, this industrial application (evaluation of the perceived quality of a virtual car interior) cannot be treated in a fast and low cost way.

Applications of ICIs

ICI applications range from industry (Figure 3a), entertainment (Figure 3b), medicine (Figure 3c), and research purposes (Figure 3d). In all these cases, users expect to "encounter" a surface to touch or manipulate in a Virtual Environment. In the case of industrial applications, these devices are considered for virtual prototyping that requires to have haptic feedback in several locations to recreate workspace or expected location for objects to be manipulated. In the case of entertainment, they are used to recreate elements that can come in contact with the users when interacting with a virtual environment [START_REF] Huang | Haptic-go-round: A surrounding platform for encounter-type haptics in virtual reality experiences[END_REF]. In the case of medicine, its often used for remote body-palpation and surgery practice [START_REF] Filippeschi | Encountered-type haptic interface for virtual interaction with real objects based on implicit surface haptic rendering for remote palpation[END_REF]. The use in research purposes often looks for leveraging the device's capabilities for rendering more complex surfaces and objects [START_REF] Kim | Synthesizing the roughness of textured surfaces for an encountered-type haptic display using spatiotemporal encoding[END_REF].

Challenges

In order to leverage the capabilities of ICI, several challenges must be addressed, these are addressed in different dimensions such as: User safety and security (a) Application in Industry by Lobbybot [START_REF] Posselt | Toward virtual touch: investigating encounter-type haptics for perceived quality assessment in the automotive industry[END_REF].

(b) Haptic-go-round from one direction to another while fishing in the virtual scene. [START_REF] Huang | Haptic-go-round: A surrounding platform for encounter-type haptics in virtual reality experiences[END_REF].

(c) The virtual palpation system. [START_REF] Filippeschi | Encountered-type haptic interface for virtual interaction with real objects based on implicit surface haptic rendering for remote palpation[END_REF].

Challenge 1. User safety and security

The concern to avoid unexpected collisions with users has been present ever since the early days of Human-Robot Interaction research. Measures for addressing this issue normally consider path-planning algorithms that help the robot to actively avoid the user in cases where both user and device could come into contact involuntarily. However, the use of path-planning often causes the device to move more slowly. This delay affects response time and user's perceived immersion in the virtual environment, and thus, it has been recognized as an issue to be addressed by the research community.

As an alternative to solutions exclusively relying on path-planning for avoiding collisions with users, we propose additional techniques like the placement of robot to minimize the workspace interaction, and creation of multiple zones for robot motion. Research concerning user safety for ICI systems needs to look for strategies that help to avoid any undesired contact that could break users perceived immersion in the virtual environment. The main objective remains to ensure the safety of the operator. With virtual reality, the operator has a modified perception of the robot and of the risk of collision. A visualization in the virtual world of the robot reduces the immersion but increases the safety. In the intersection zones between the space possibly scanned by the robot and the human, the robot (UR5) will be placed in cobot mode, i.e. it will stop and switch to gravity compensation mode if a contact effort is detected. Thus it cannot injure the operator. However, the speed of the robot is then limited.

Research Question 1. Robot placement Robot placement holds the promise of removing the need for a highly dedicated and structured workspace, as well as responding more quickly to environmental changes. Within the systems, dynamic and robust placements are crucial and strategically important, since they are often done in early stages in the process.

A complete set of operations consists in performing a specific task/operation by a robot on a set of task points. Often, after the robot returns to its starting configuration, a user is introduced to the system, and operations are performed. Since these cycles are repeated several times, it is very important that they are executed as fast as possible in order to increase immersion.

Once a set of specific points is assigned to a robot, the layout has limited freedom to optimize the robot workstation:

robot's base placement (translation and rotation); The orientation of robot TCP at each task points is important as it effects the immersion of the user. So the robot should be able to reach the task points with the required orientations. The study started with no information on the robot base location in the virtual environment. visiting order of the work-points; In this approach the user decides the order of interaction with the task points. To improve user safety , the robot has to be closer to the user hands near the task points and as far as possible to the human body (torso, head and especially neck). Major concerns were to restrict the operation of the robot to one aspect to avoid crossing singularities while performing the tasks. robot's home configuration in the station (six joints); initial approach is to achieve one single base location and have single aspect of robot to connect and move between all the task point.

The last three ones may be modified by changing the robot program, whereas the first has to be completely decided before installing the robot in the workspace.

Research Question 2. User model

There is no fixed technique or standard procedure to use a tracking system to explore user actions in VR. Human movement tracking systems are expected to generate real-time data that dynamically represent the pose changes of a human body (or a part of it), based on welldeveloped motion sensor technologies. Generic tracking systems employed within these systems adhere to the human body in order to collect movement information. These sensors are commonly categorized as mechanical, inertial, acoustic, radio, microwave, and magnetic-based. In such systems, the user is generally equipped with sensors on his hands, that give the sense of interacting with a virtual object. Additionally, these sensors don't give information about the position and orientation of the user's location in the system. Visual marker-based tracking is a technique where cameras are applied to track human movements, with identifiers placed upon the human body.

The whole idea of ICI is to increase immersion and remove the sensation of robot presence in free space. Since the robot has the plan to move between task points without colliding with the user. For safety concerns, the position of the user is very important. However, we cannot have complex tracking systems to locate the user. Normally the use of external cameras and markers are used to locate the user's position. But such systems increase the number of sensors attached to the user, which reduces the immersion. So the goal is to use less additional sensors as possible to give the best immersion experiences while getting the accurate location of the user for planning trajectories of the robot.

Challenge 2. User immersion experience

Haptic feedback for ICIs goal is to improve the immersion experience of users. The major idea is to eliminate the mechanical link so as to have perfect transparency in free space. But even trying to achieve this experience comes with the challenges like planning the robot motion between the VO being touched (without colliding with the user), and predicting which VO object the user intends to touch in the environment.

Research Question 3. Robot motion planning In the given scenario the user is immersed in Virtual reality. The user has no information regarding the motion of the robot. The robot's motion has to be safe for the user, experiencing the VR environment. It should avoid collision with the user and also any other obstacles in the environment. The robot's motion should take into consideration collision with the user based on the tracking system used. It's not only for the user's safety, but also to have a better immersion of the user, the robot must place a real object at the place where the user wants to touch the VO as quickly as possible, and before the user's hand reach the contact location.

Research Question 4. Target object detection. Since they touch the user only when force feedback is required, intermittent contact interfaces, and in particular close-tracking-type devices, aim to provide more realistic interactions with virtual environments than classical contact haptic interfaces (CHI). User intention prediction in ICI remains a challenge to be properly addressed by the research community.

Previous systems force the user to interact with the VO, selected by it. These systems have to use control algorithms and interaction techniques to make the haptic rendering as efficient as possible. For the new system, we want to give the user freedom to select the VO to interact and the robot system should adapt to align itself w.r.t to the VO. The goal of such a system is to predict the VO the user wants to interact with as soon as possible. However, for these new systems, the two solutions (control algorithms and interaction techniques) are not enough to render the haptic sensation.

Roadmap for the introduction of cobots

The main research area considered as a contribution to this work is improving user safety and immersive experience for users of a ICI haptic device in humanrobot interaction.

The user safety axis is improved by addressing four contributions. The first contribution (C1) consists of a set of safety techniques based on robot placement. The second (C2) creation of a user model for tracking user information, and the third (C3) techniques for robot trajectory planning. Then, the second axis immersive experience is improved by making sure the user does not lose the illusion of the environment. This aspect can be addressed by making sure the user does not wait a long time for the robot to reach the desired interactive location. This is achieved by addressing 2 contributions, the first contribution (C3) techniques in robot trajectory planning and the second contribution (C4) techniques for user goal predictions, designed to optimize the response time in ICI systems.

The relationship between the contributions and the research axis can be seen in Table 1.

Contribution

User Immersive safety experience (C1) Safety techniques based on robot placement.

x (C2) User model for tracking user information.

x (C3) Techniques for robot trajectory planning.

x x (C4) Techniques for user goal predictions.

x

Table 1: Contributions to the research axis A set of safety techniques for users based on robot positioning (C1) was presented in [START_REF] Guda | Safety in a human robot interactive: Application to haptic perception[END_REF]. In this paper, a design space for safety techniques was introduced using visuals restricting the interactive workspace of the robot and the user, in order to reduce potential unintended collisions. The dimensions of the workspace are defined by where the user wants to interact with the virtual environment as well as to ensure the safety of the user. Using this design of robot placement, a set of workspaces was developed to explore variations of the prop orientations. An evaluation focusing on the best solution for robot placement is done. Safety was evaluated and the ideal location of the robot was defined. The main constraint is that the robot end effector cannot touch the user's head (Figure 4). The model was designed to estimate the user location in the Robot Operating System (ROS) environment [START_REF] Guda | Contributions to utilize a cobot as intermittent contact haptic interfaces in virtual reality[END_REF]. The model created had to have features such as movement of both arms and also the mobility of the user around the hip. These major three moving parts provided better information about the user's location, improving the user's safety in using an ICI system. A user study was designed to assess and test the performance of these motions to test the accuracy and reliability of creating a user model. Results suggested that the model is accurate for the given scenario in locating the user in the environment. Only five sensors are needed to have a good representation of the upper limbs of the user (Figure 5). This property allows a simplified set up of users while guaranteeing their safety. Fig. 5: Structure for both arms to evaluate the joint position of the arms q i , the position of the hands, and the base position of the robot.

The helmet also adds additional information. The integration of the ROS node into the Unity simulation requires the implementation of a system architecture with elements under Windows and others under Linux (Figure 6). An approach for the trajectory planning of robots was introduced in [START_REF] Gutierrez | Trajectory planning in dynamics environment: application for haptic perception in safe human-robot interaction[END_REF]. The goal is to increase the immersive experience (C3) by reducing the time taken by the robot to reach the contact area. This approach renders large surfaces and multiple textures through the use of a rotating prop, that couples the prop's rotation and position with the users' hand position when exploring a textured surface in VR. A use-case scenario was designed for contextualizing this approach. Later, a user study was conducted to validate the approach haptic rendering performance. A novel approach for the prediction of user intention based on head gaze and hand position was introduced in [11] (Figure 7).

Four strategies are presented to predict where the user wants to interact with the virtual environment (C4) [START_REF] Guda | Motion strategies for a cobot in a context of intermittent haptic interface[END_REF] (Figure 8). The strategies include the hand position (A), the hand position and the gaze direction (B), the hand position and safe point to allow faster motions (C), and the hand position, the gaze direction, and the safe point to allow faster motions.

Each approach presented is evaluated on four key features: (1) time taken to prediction, (2) time taken by the robot to reach the contact area, (3) distance traveled by the robot, and (4) safety of the user. These approaches integrate an interaction technique for contact area selection and release.

A comparative analysis of data from all the trajectories shows that, if the objective is to maximize safety strategies (C) and (D) would be better. Both strategies (C) and (D) ensure safety by selecting safe points when the hand is The selection of the safe points means that the robot will have to travel a longer distance to reach the desired point. While strategies (A) and (B) select intermediate points that are inside the car and there are no safe points. So since the points are all inside the car, and the robot does not travel longer distances but has reduced velocity. Strategy (D) gives the second-best safety and, at the same time minimize, the time to detect/reach a desired point. Therefore it can be seen as the best strategy. The detection time for strategy (D) is the smallest because we used the gaze of the user to pre-select the points. This plays an important role in giving priority to visual information over information from the hand position.

The fastest detection time allows the robot to start moving to the desired point at the earliest time and reach the desired point the fastest.

A simple real-time demonstration of the above system using strategy (D) can be found here 1 .

Conclusions and future work

This paper summarized the context of the introduction of cobot for the implementation of a haptic interface with intermittent contact. The different challenges were presented in order to guarantee the safety of the user and improve the user's immersion. The various stages presented are invariants to be considered for the implementation of similar experiences.

A work of diffusion remains to be done to put on GitHub the functions allowing to define the optimal placement of the robot, the generation of trajectories according to the environment, the position of the user, and his intention for new applications. Moreover, a transfer of the software to ROS2 could improve security by guaranteeing the real-time aspect of the information exchanges.

Fig. 1 :

 1 Fig. 1: Conceptual scheme of the experimental platform

Fig. 2 :

 2 Fig. 2: The LobbyBot project setup.

  (d) Synthesizing the Roughness of Textured Surfaces for an Encountered-type Haptic Display[START_REF] Kim | Synthesizing the roughness of textured surfaces for an encountered-type haptic display using spatiotemporal encoding[END_REF].

Fig. 3 :

 3 Fig. 3: ICI applications

Fig. 4 :

 4 Fig. 4: Placement analysis of the base table of the robot.

Fig. 6 :

 6 Fig. 6: System's Architecture.

Fig. 7 :

 7 Fig. 7: Diagram of the inputs used to choose a robot movement strategy, hand, and head position with the gaze direction.

Fig. 8 :

 8 Fig. 8: Four strategies to detect human intention

Acknowledgment

This work was funded under the LobbyBot project: Novel encountered type haptic devices, ANR-17-CE33 [21]. The authors of the article thank the members of the project for their help in carrying out this work, Lionel Dominjon, Sandrine Wullens, Allexandre Bouchet, Javier Posselt, Maud Marchal, Anatol Lecuyer and Victor Rodrigo Mercado Garcia.

Data

sensor coordinates Data joint values Data sensor coordinates Unity Legend: output: input: Data Mannequin TF frames «Planning Scene Monitor» Update obstacles in the scene Data Obstacles in the scene «Move Group» Data Obstacles position Data desired pose (goal) «ROS control» ur modern driver