Pierre Vieyres

Laurence Josserand

Ma Juan Sandoval

Nicolas Morette

Aicha Cyr

Fonte

A predictive contr enhance distal robo Interactiv

Keywords: bilateral architecture, tele-ech robotic, GUI, force feedback I

Performing a robotized telemedicin networks brings forth two issues. One is transp enable the operator, e.g. the medical ultras safely and accurately perform bilateral te despite the long time delays inherent to the co To counter these effects, two strategies are com at the operator site, the rendering of the intera remote robotic systems with its environment (i. the control of the robot's orientation. The fir development of a new control architecture ba model providing an anticipated value of the d stiffness; it is complemented with a graphic us which provides the expert with the real-time the haptic probe with the robot's end effecto operated control. These combined strategies with an improved interactive tool for a tele-dia

INTRODUCTION

Robotized tele-echography has been dev mid-nineties to compensate for the insuffi ultrasound (US) experts and their regrouping care facilities centers, resulting in a gro medically isolated areas. Ultrasound imagi reliable and non-invasive technique us hospitals; however, it is an expert-depe meaning that the experts can only make a dia are able to combine anatomical knowledge orientation of the probe and its position on and the analysis of the ultrasound images. In shown that robotized tele-echography offe solution to making real time diagnoses for patients with results comparable to diagnos local echography. This robotized approach than the basic tele-ultrasound modality pres [START_REF] Kontaxakis | EU-TeleInViVo: an integrated portable telemedicine workstation featuring acquisition, processing and transmission over low-bandwidth lines of 3D ultrasound volume images[END_REF][START_REF] Chimiak | Architecture for a high-performance tele-ultrasound system[END_REF]. Several research teams [START_REF] Fichtinger | Robotic assistance for ultrasoundguided prostate brachytherapy[END_REF][START_REF] Masuda | Three dimensional motion mechanism of ultrasound probe and its application for teleechography system[END_REF][START_REF] Vieyres | A tele-operated robotic system for mobile teleechography : The OTELO project[END_REF][START_REF] Bassit | Kinematics of a dedicated 6 DOF robot for tele-echography[END_REF][START_REF] Najafi | A novel hand-controller for remote ultrasound imaging[END_REF][START_REF] Neubach | Ultrasound-Guided Robot for Flexible Needle Steering[END_REF][START_REF] Vilchis | A new robot architecture for tele-echography[END_REF][START_REF] Ito | Portable and attachable tele-echography robot system: FASTele[END_REF] have with various constraints and objectives (e.g needle insertion, prostate biopsy), tele-operat rol approach and intera l environment renderin otized tele-echography ve platform for robotized telechography arco Chiccoli, ril Novales, Orleans y Soteris Avgousti, Sotos V Telemedicine r Cyprus University 30 arch Kyprianou str ne act via specific parency in order to ound specialist, to ele-operation tasks ommunication link. mbined to improve, actions between the .e. the patient), and rst approach is the ased on an internal distant environment ser interface (GUI) relative position of r for a better teleprovide the expert agnosis. hography; medical veloped since the icient numbers of g in large medical owing number of ing is a low cost, sed routinely in endent technique, agnosis when they e with the current the patient's body n [START_REF] Arbeille | Realtime teleoperated abdom-inal and fetal echography in 4 medical centres, from one expert center, using a robotic arm & ISDN or satellite link[END_REF], Arbeille has fers radiologists a r remotely located ses using standard h is more reliable sented in the 90's been developing, g. ultrasound scan, ted robotized tele- echography systems using po standard communication netwo During the tele-echography pro positions the probe-holder se anatomical location and main (Fig. 1a). The medical expert, by Essomba et al. [START_REF] Essomba | Contribution to the Design of a Robotized Tele-Ultrasound System[END_REF], achiev of the remotely located robot of the haptic probe is two fo contact force applied by the patient's skin, the other is expert's gestures to be reprodu effector. The patient's ultr information fed back to the me in real time and provides a diag However, performing a specific networks raises two order to enable the operator specialist, to perform precis operation tasks despite time communication link (e.g. W bilateral control architecture i expert site the contact force be environment (i.e. the patient). robotized tele-echography via issues. One is transparency in r, i.e. the medical ultrasound ely and safely bilateral teledelays inherent in the chosen Wi-Fi, satellite); therefore, a is necessary to feedback at the etween the remote robot and its The second issue is knowledge of the accurate positioning of the robot's end-effector with respect to the haptic probe's current position, which effects the analysis of the received ultrasound images on which the expert relies to give a diagnosis; this information has to be embedded in the Expert Graphic User Interface (GUI) to provide better information about the remote robot's behavior. This paper addresses with these two issues and presents a new control approach to compensate for the long time delays of the communication links, to provide an anticipative force feedback to the expert site, combined with an interactive GUI to provide the expert with an accurate understanding of the orientations of the remote ultrasound probe in contact with the patient's body. The proposed combined approaches are designed to improve the performances of tele-echography robots when using long time delays communication links. The paper is organized as follows: the telemedicine communication protocols network is discussed in part II, part III presents the robotized tele-echography concept, and the needs for a protocol for its safe functioning with respect to patients. Part IV introduces the new control architecture developed to anticipate the contact force between the robot and its environment and the simulation results. Finally, the GUI virtual tool, that provided a complementary tool for the medical expert to give a diagnosis, is presented.

II. TELEMEDICINE COMMUNICATION NETWORK

Telemedicine applications can be categorized as requiring low, medium or high bandwidth transmission. The range of network choices for telemedicine in recent years have included wired communications technologies (plain telephone lines), ISDN and ADSL. Nowadays options include more modern technologies, digital land-lines or cellular/wireless, broadband networks such as broadband Integrated Services Digital Network (BISDN) with the Asynchronous Transfer Mode (ATM), as well as satellite networks, Wireless Local Area Network (WLAN) and Bluetooth, allow the operation of ambulatory and mobile telemedicine systems [START_REF] Pattichis | Wireless telemedicine systems: an overview[END_REF]. It should be noted that when considering telemedicine and telecommunications technologies, it is important to evaluate not only their capabilities and cost/performance trade-off but also the general technical development [START_REF] Reljin | Telecommunication Requirements in Telemedicine[END_REF]. Regarding the transmission of medical data there are no theoretical bandwidth requirements. The range and complexity of telecommunication technology requirements vary with the specificity and characteristics of the given telemedicine application; generally a lack of bandwidth is interpreted as a longer transmission time [START_REF] Sachpazidis | TeleHealth networks: Instant messaging and point-to-point communication over the internet[END_REF]. Table I gives the bandwidth needed for some telemedicine real time transmission. Standard GSM can only provide data-transfer speeds of up to 9.6 kbps, that only allows real-time small data transmission. GPRS and Edge theoretically allows transfer up to 171 Kb/s and 384 kb/s respectively, but all users in a communication cell share the same bandwidth. It can reliably be used for realtime small data transfer or at least a reasonable quality ambiance video [START_REF] Istepanian | Enhanced telemedicine applications with next generation telecommunication systems[END_REF]. The 3G cell phone technologies, based on UMTS may support up to 1.75Mb/s, and can support 384kb/s transmissions for medical images [START_REF] Mueckenheim | On system level modelling of UMTS downlink shared channel scheduling[END_REF]. Wi-Fi Lan (802.11g, 802.11n) can support from 54Mb/s up to 300Mb/s bandwidth,but must be very well controlled to avoid too many collisions and have real limitations in terms of mobility and coverage [START_REF] Zhang | Wireless telemedicine services over integrated IEEE 802.11/WLAN and IEEE 802.16/WiMAX networks[END_REF]. Nowadays, wired communications using dedicated digital lines with high baud rates are present in all hospitals and can be used for telemedicine applications between two hospitals. The connection to mobile or isolated area may use commercial links with high baud rate, such as ASDL, or high cost geostationary satellite communications.

In all cases, transmission using IP protocols is highly recommended. The protocol remains always the same, whatever the communication links used. Connection oriented protocols, such as Transmission Control Protocol (TCP/IP), can be used for basic connection (i.e. control data to ensure the devices connections) but is not suitable for images or videos transmission [START_REF] Slama | Experimental analysis of an internet-based bilateral teleoperation system with motion and force scaling using a model predictive controller[END_REF][START_REF] Xiaohui | The design and implementation of real-time Internet-based telerobotics[END_REF]. Intrinsically, this kind of protocol reduces the network rate when the transmission reaches the speed limitation of one of the link of the network chain due to the repetition of packet loss. Connectionless protocols such as User data Protocol (UDP) are better adapted for images or videos, and are not sensitive to the latter problem; the down side is that it does not offer any guarantees that information will be delivered to the final destination. For real-time telemedicine, the choice is simple: it is better to loose one image than to overload and block a communication link. In the robotized tele-echography application presented in this paper, we used a connection-oriented protocol to ensure the connection between the distant sites, and connectionless protocol for data/images transmission. When a control loop is required through the network (e.g. control a remote robot), data are sent using UDP. For this specific medical teleoperated robotic application, to avoid a mechanical divergence of the robot when data loss occurs and for the patient safety, data sent are not robot joint velocity or torque set points but position set points. Hence in case of data loss, the robot remains oriented in the last received data position. Specific protocol (e.g. hybrid connected-connectionless) are developed to control robots through the network, but are not normalized [START_REF] Wirz | Bidirectional transport protocol for teleoperated robots[END_REF]. Once the protocol is well defined for a given telemedicine application, the users have to face with the time varying delay inherent to the chosen communication link and its consequences on the stability and transparency properties for the tele-operated robotic system; Solutions to maintain transparency and stability properties during the teleechography medical act are discussed in the following sections.

III. PRESENTATION AND ISSUES OF THE ROBOTISED TELE-ECHOGRAPHY SYSTEM

A. The tele-echography platform

The system includes three parts linked to each other: the expert station (i.e. the operator), the patient station and the communication link that enables data exchange between the two stations (Fig. 2.).

• The expert (Master) station: the specialist uses a haptic fictive ultrasound probe to control the robot end-effector holding the real ultrasound probe. The haptic device allows the expert to feel the interactions between the distal environment (i.e. the patient's body) and the real ultrasound probe.

• The patient (Slave) station: the robot carrying the real ultrasound probe is positionned and maintained on the patient by an assistant. The robotic probe holder system reproduces the medical expert's gestures during the teleechography procedure thanks to the haptic device.

• The communication link: the communication network links the expert station to the patient station. Depending on the requirements and availability at the sites involved, a teleoperation system may use different types of communication links (e.g. WLAN, Satellite). However, these network links may generate transmission varying time delays, which hinder the overall robotic tele-operation and force feedback rendering at the expert site. In addition, a videoconferencing system is used for visual and auditory interactions between the expert, the patient and the assistant, and to transfer the ultrasound images from the patient station to the expert station.

B. Constraints of the tele-operated medical procedure

A robotic tele-echography system must provide the specialist with the best working conditions for a remote medical consultation and ensure safety for the patient. However, the use of the communication links between Expert and Patient stations introduces variable time delays that may lead to disturbances in the system stability and deterioration in the reference trajectory tracking generated by the expert [START_REF] Charron | Contribution à la commande bilatérale et à la gestion des configuration singulières pour le suivi de trajectoire d'un système téléopéré: application à la télé-échographie robotisée par satellite[END_REF]. Figure 3 shows different transmission delays round trip time measured using an Inmarsat satellite communications from different European locations. The average time of these transmission delays is about 1 second with an average variation of 200ms. These long time variable delays indicate a need the development of specific control architecture to maintain stability and transparency in the tele-operated system. This paper presents communication protocols to be used for a tele-operated procedure and the combined solutions developed to provide an interactive and efficient system for the medical expert to perform the tele-operated actions. A comparison of TCP and UDP protocols is done with reference to telecommunication link characteristics such as variable time delay, bandwidth and reliability, which are of great importance to carry out a tele-operated telemedicine task.

The protocol choice and the network will impact the development of the bilateral control architecture needed to ensure: the transparency of the system, i.e. the force exerted by the robot on the patient's body should be equivalent to the force fed back to the expert, and the trajectory accuracy of the medical gesture, whatever the transmission delays. Designing a GUI will allow the expert to compare the orientation of the fictive probe with respect to that of the ultrasound probe. This functionality offers reassurance to the specialist about the ultrasound image analysis taking into account the communication delays.

IV. BILATERAL CONTROL ARCHITECTURE

In order to preserve the stability and transparency of the system, we developed and compared two bilateral control architectures. The system is transparent under two conditions: the robot end effector holding the real probe reproduces the same orientations as the one generated by the expert manipulating the haptic probe (fictive probe) and the force rendered by the haptic probe of the expert is the same as the force exerted by the probe on the patient's body.

A. Wave variables architecture

Keeping in mind the objectives of stability and transparency in the tele-operated system, we used a modified passive bilateral control architecture (position/ force) added with a specific controller on the slave side, based on the (velocity/force) formalism of wave variables introduced by Niemeyer and Slotine [START_REF] Niemeyer | Designing force reflecting teleoperators with large time delays to appear as virtual tools[END_REF]. This architecture is referred to as passive as it has no internal energy generation and was used as reference architecture in this work. This ensures the stability of the system regardless of the values of time delays caused by communication links. This architecture is composed of three blocks (Fig. 4). The expert site comprises the operator whose force action is represented by F h , the model of the haptic probe M(s) which enables the remote control of the robot's orientations, the master controller C M (s) that provides the force feedback F mc to the haptic probe from the measured delayed environment force F ed , finally the incident wave generator constructs the incident wave u m . The patient site consists of the slave robot S(s) interacting with the distant environment E(s), the slave controller C S (s), and the reflected wave generator. Using the incident waves u s and the force of the environment, this generator provides the references in position x md and the reflected wave v s . The communication links transmit the incident and reflected waves between the expert site and the patient site. The delays inherent to communication links between the two sites are represented by T1 and T2, respectively. Figure 5 shows the internal structure of the incident and reflected wave generators. These generators are bijective operators defining u and v waves from a linear combination of the position x and the force f. b is the characteristic impedance of the wave variables and was initialized to the mean value of the body's stiffness as given in section B. The force F ed fed back to the operator (i.e. the medical expert) depends on the time delays, the received incident wave variable Us and the input position generated by the operator as follows:

The first term of the equation ( 1) allows an immediate rendering of the force back to the operator as soon as the teleoperated task starts. The second term is a reflection produced upon the transformation of waves and must be filtered before entering the wave generator at the patient site. The last term contains the delayed contact force between the robot and the environment. The simulation results show the position of the robot's end effector with respect to the input position x m of the haptic probe. x s represents a sinusoidal displacement of the ultrasound probe in contact with the patient's body (Fig. 6); x s signal is delayed by a delay time variable T 1 (0.7s). We can note a conservation of the amplitude and frequency of the input position. Figure 7 shows the anticipated force feedback F ed rendered to the expert in comparison to the force F e generated by the robot's end effector on the patient's body. In this case, it was considered a constant patient breathing period of 4.8 seconds. The results validate the anticipative ability of this architecture to ensure the transparency property of the tele-operated system even for long time varying delays.

B. Stiffness control architecture

The second bilateral control architecture proposed is based on the stiffness control (Fig. 8). This architecture is based on the theory of internal model control. The expert site uses the patient site model (i.e. robot and environment systems) and the haptic probe position to obtain an estimate of the robot's behavior, anticipated with respect to the reference of the expert time. In addition, the stiffness of the patient's body, with which the robot is in contact, is measured and adjusted with respect to the "expert time" reference. The position and stiffness adjusted to the time reference of the expert allow the reconstruction of the force feedback. The measured compliance Kmes(t) of the patient's body received at expert, given by equation 2, has an average stiffness Kmoy. A sinusoidal signal is added to represent the patient's breathing pattern. This model is a first approximation of the real phenomenon, including breathing, which is generally more difficult to characterize than by using a simple sine function. However, it is justified by the fact that our approach is robust as it uses an internal model. In addition, for Kmoy, we consider primarily the stiffness of the abdomen in humans, known to be relatively homogeneous (450 N/m + ou -50 N/m) [START_REF] Tortora | Principles of Anatomy and Physiology[END_REF] [START_REF] Sherwood | Fundamentals of Physiology : A Human Perspective[END_REF]. Hence, four parameters allow a full identification of the measured stiffness K mes ; Kmoy, Amp, T and Φ: the average stiffness, the breathing amplitude, the breathing period and its phase, respectively. Kmes(t) is reconstructed and referenced to the expert time and in real time, providing an estimate of the current stiffness K est . The time delays are known as they are evaluated also in real time hence avoiding a possible instability. The four parameters identification was carried out using basic calculation methods in order to validate the concept. However, to refine the results, the Levenberg-Marquardt algorithm was used [START_REF] Marquardt | An Algorithm for Least-Squares Estimation of Nonlinear Parameters[END_REF]. This minimization numerical algorithm works as an iterative procedure to minimize a criterion J (Equation ( 3)). In this case, the criterion J represents the square of the difference between the measured stiffness K mes and estimated stiffness K est .

The force feedback F mcf received by the expert is calculated using the estimated stiffness K est and the robot position shifted with respect to the expert time reference (fig. 9). The simulation results show the conservation of the amplitude and frequency of the reference position x m sent by the expert, and reproduced by the robot on the patient's body, where x s represents a sinusoidal movement of the ultrasound probe in contact with the patient's body (Fig. 10). As in the previous architecture, these signals are delayed by a variable delay time T 1 (mean 0.7s).

Considering a patient with a breathing period of 4.8 seconds, a breathing amplitude of 100N/m, a breathing phase of 0.52 rd and a K moy of 440N/m, figure 11 shows the anticipated force feedback F mcf compared with the force F e generated by the robot on the patient's body. This result demonstrates the good transparency provided by the proposed architecture. The results show that overall the two approaches anticipate the contact force between the robot and its environment, and thus satisfy the notion of transparency. However, we note that there are errors in amplitude in both cases of less than 15%, more pronounced in the case of wave variables. Figure 12 illustrates these errors between the input force sent from the robot and the forces rendered at the expert for each of the developed architectures (Fig. 12).

V. THE GRAPHIC USER INTERFACE AND VIRTUAL PROBES

When using communication links with variable long time delays, the medical specialists will always receives the patient's ultrasound images with a time lag T 2 with respect to the current position of the haptic probe they are manipulating. To provide the medical expert with a full rendering of the force feedback, it is very important that the expert knows exactly the position of the haptic probe in relation to the position of the robot's end-effector corresponding to echography images being received. In order to compensate for the time lag and improve the haptic rendering of the remote environment, the control architecture developed in the previous section is complemented with a dedicated graphic user interface (GUI) including a virtual probe. In this section, we present a joint 3-D model of the haptic probe and of the robot's end-effector, representing their respective orientations in a unique GUI in order to help the expert to perform his remote medical act with the best

A. Virtual haptic probe

The virtual haptic probe consists of three parts: the mobile element held by the medical expert, the end-tip of the probe representing the ultrasound sensor capturing the ultrasound images and one element joining the two previous ones. The blender is used to design the surface elements of the virtual probe. The final result of the virtual probe is showed in Fig. 13a andb.

B. Comparison with the robot's end-effector position

A model of the robot's end-effector holding the real ultrasound probe has also been designed using Blender. This second virtual model is added in the GUI to the virtual haptic probe model. The virtual haptic and the end-effector models orientations are extracted and displayed on the GUI, in real time, from data provided by the real haptic probe localization sensors and the robot's end-effector proprioceptive sensors, respectively. Figure 13 presents the GUI display of the two virtual probes 3-D models in their initial position (Fig 13a) and during the tele-echography act when positions are not synchronized due to the time lag generated by the communication link (Fig. 13b). This display is included in the general GUI dedicated to the medical expert.

C. The Graphic User Interface (GUI)

The GUI developed for the ANR-Prosit is divided in two subscreens: one for the technical part, which allows an engineer to set and to evaluate the various status of the local and remote systems such as connection protocols, tele-operated modes, proprioceptive sensors, data loss, this part is not presented here; the second GUI display is dedicated to the medical experts and help them to make a diagnosis when performing the tele-consultation. The so-called medical GUI is composed of three parts: the dynamic 2-D ultrasound image, a reconstructed 3D ultrasound image available post consultation, and the virtual probes display discussed in the previous section. Figure 14 represents a zoom of the medical GUI.

VI. CONCLUSION

The GUI improves the tele-operated act of the medical expert and offers in combination with the haptic input device a better rendering of the distal interactions between the robot and its environment (i.e. the patient). Simulation results validate the transparency quality provided by the two architectures for the tele-operated system; they provide the expert a force feedback equal to the force exerted by the robot. That is, the two architectures minimize the negative effects of variable transmission delays on the transparency of the system. It is necessary to validate experimentally the two control architectures will be shortly validated on the tele-echography platform via Inmarsat satellite link.

a

  Figure 1. (a) Positioning of the light patient (b) haptic probe held by the e ultrasound probe holder robot.

  This work was supported by PROSIT Electricity Cluster, Cyprus university r Cyprus and UK Inmarsat Satellite serv active GUI to ng during y oskarides, Takis Kasparis research laboratory of Technology (CUT) r. 3036 Limassol, Cyprus b t-weight (< 3 kg) Prosit 1 robot on the expert for the orientation of the remote ortable ultrasound devices via orks (i.e. internet and satellite). ocedure, a paramedical assistant erial robot on a predetermined ntains it on the patient's body using a haptic probe developed ves a position open-loop control orientations (Fig. 1b). The role ld: one is the rendering of the real ultrasound probe on the the recording of the medical uced by the remote robot's end rasound image is the only edical specialist who analyzes it gnosis.

Figure 2 .

 2 Figure2. General scheme of a robotic tele-echography system (from[START_REF] Charron | Contribution à la commande bilatérale et à la gestion des configuration singulières pour le suivi de trajectoire d'un système téléopéré: application à la télé-échographie robotisée par satellite[END_REF]).

Figure 3 .

 3 Figure 3. Transmission delays (round trip time) measured when using Inmarsat satellite link from (a) Bourges, France and from (b) Limassol, Cyprus.

Figure 4 .

 4 Figure 4. Scheme of the wave variables architecture.

Figure 5 .

 5 Figure 5. Internal architecture of the two wave generators.

Figure 6 .

 6 Figure 6. Comparison between the input position xm and the signal xs received by the robot's end effector using the wave variables architecture.

Figure 7 .

 7 Figure 7. Comparison between the force Fe generated by the robot's end effector in contact with the environment and the force Fed fed back to the expert site using the wave variables architecture.

Figure 8 .

 8 Figure 8. Scheme of the stiffness control architecture.

Figure 9 .

 9 Figure 9. Detail of the proposed internal model.

Figure 10 .

 10 Figure 10. Comparison between the input reference position xm and the signal xs received by the robot using the stiffness control architecture.

Figure 11 .

 11 Figure 11. Comparison between the input force Fe generated by the robot and the force feedback Fmcf at the expert site using the stiffness control architecture.

Figure 12 .

 12 Figure 12. Comparison between the errors of the force feedback ∆F WV (wave variables architecture -green dotted line) and ∆F SC (stiffness control architecture -blue dotted line). rendering of the distal interactions between the robot and the patient. The 3D model was built with Blender, an open source software used to design objects and 3-D animations; the GUI was developed using Qt, an open source software developed by Nokia company and designed for developing GUI. Within the GUI, the virtual probe and the virtual robot's end-effector, are orientated, in real time, thanks to the orientations data provided by the robotic Prosit; orientations are given with respect to Euler angles frame.

Figure 13 .

 13 Comparison between the virtual haptic probe (front purple model) and virtual end-effector (black model) position (a) initial position (b) during the tele-echography act under long time delay. a b Figure 14. Zoom on the graphic user interface (a) the virtual haptic probe and robot's end-effector orientations difference, (b) the 2-D ultrasound image received at the expert site.

TABLE I .

 I MINIMUM BANDWIDTH REQUIREMENT FOR REAL-TIME TRANSMISSION

		Minimum Bandwidth
	Data Type	Required for Real-
		time transmission
	Digital blood pressure	8 Kb/s
	Digital thermometer	8 Kb/s
	Oxygen saturation meter	8 Kb/s
	Electrocardiogram	16 Kb/s
	Ultrasound	320 Kb/s
	Dermatology (high resolution and color)	384 Kb/s
	Scanned x-ray	384 Kb/s
	Mammogram	384 Kb/s
	CT	384 Kb/s
	Video + audio conference	320+64 Kb/s

T ANR-08-CORD-017, the S2E2 Smart research Fund, Tototheo Group Cyprus, vices