Université De Strasbourg

Rafael Aleluia Porto

Thèse Dirigée

M De Mathelin

Michel Professeur

M Redarce

Tanneguy Professeur

M Tamadazte Brahim

M Nageotte Florent

M Zemiti

M Laroche

Modelling and control of robotic flexible endoscopes using machine learning

Keywords: robots continus, modélisation, apprentissage automatique, modèles hybrides, robotique médicale continuum robots, modeling, machine learning, hybrid models, medical robotics Nom : ALELUIA PORTO Prénom : Rafael

Modelling and control of robotic flexible endoscopes using machine learning

Résumé Dans le contexte des chirurgies mini-invasives, les endoscopes flexibles et les instruments chirurgicaux à câble sont des outils essentiels. Dans les systèmes robotiques, la transmission par câble permet de contrôler les effecteurs distaux à l'intérieur du patient à partir de moteurs situés du côté proximal. Cependant, des non-linéarités sont introduites en raison de l'interaction entre les câbles et les gaines à l'intérieur des outils endoscopiques. Dans cette thèse, un nouveau modèle cinématique inverse (MCI) qui peut prendre en compte des non-linéarités complexes est proposé pour les systèmes flexibles, ainsi qu'une extension pour tout type de système robotique. Le concept des techniques développées consiste à combiner la modélisation géométrique avec des techniques d'apprentissage automatique. Cela permet d'obtenir un MCI précis, efficace et capable de gérer les effets d'hystérésis. La précision est améliorée par rapport à d'autres approches basées sur l'apprentissage, avec une phase d'entraînement plus rapide, comme l'ont montré les expériences menées sur la plateforme STRAS.

The abdomen is inflated by using CO 2 to create space for the surgical gesture. The rigid endoscope is inserted through a trochar to allow visualization of the internal organs. . . . . . . . . . . . . . . . . . . 1.2 Timeline of major breakthroughs. Lichtleiter - [START_REF] Rathert | Philipp bozzini (1773-1809) and the lichtleiter[END_REF], Desormeaux -[Zada et al., 2012], Kelling - [START_REF] Litynski | Origin of laparoscopy: Coincidence or surgical interdisciplinary thought?[END_REF]], Schindler - [START_REF] Benedict | Examination of the stomach by means of a flexible gastroscope: A preliminary report[END_REF], Hirschowitz fiberscope - [START_REF] Wilcox | Fifty years of gastroenterology at the university of alabama at birmingham: a festschrift for dr. basil i. hirschowitz[END_REF] and video endoscope - [START_REF] Wheeler | Endoscopic apparatus[END_REF]. . . . . . . . . . . . . 1.3 Types of endoscopes used on different parts of the GI tract . . . . . 1.4 A: Seemingly normal endoscopic image of Barrett's esophagus. B: Endoscopic image of the same patient after the staining with methylene blue. The bottom arrow indicates the presence of an intramucosal adenocarcinoma, whereas the arrow on the top indicate a normal stained mucosa [START_REF] Song | Chromoendoscopy[END_REF]. . . . . . . . . . . . . . . 1.5 Several surgical tools used in endoscopy. Source : [START_REF] Storz | Systems for Gastroenterology 3rd edition[END_REF] . . 1.6 Illustration of endoscopic submmucosal dissection. 1 -marking the zone of interest with an electric knife. 2 -injection of a saline solution with a needle under the tissue, in the submucosa, to elevate the area of dissection and reduce the risk of perforation. 3 -Dissection of the tissue by following the marks done at step 1. 4 -Removing of the dissected tissue using a grasper. . . . . . . . . . . 1.7 Illustration of NOTES cholecystectomy with different entry points. 1.8 Schematic view of a flexible endoscope and a depiction of the different channels (adapted from [START_REF] Conner | Guidelines for Perioperative Practice: Edition 2016[END_REF]). . . . . . . . . 1.9 Experimentation in NOTES at IRCAD France. . . . . . . . . . . . .

6

LIST OF FIGURES 1.10 Schematic view of the R-scope [START_REF] Yonezawa | A novel double-channel therapeutic endoscope ("r-scope") facilitates endoscopic submucosal dissection of superficial gastric neoplasms[END_REF]. . . . . . . . 37 1.11 Real R-scope system [START_REF] Astudillo | Transgastric cholecystectomy using a prototype endoscope with 2 deflecting working channels (with video)[END_REF]. . . . . . . . . . . . . . 37 1.12 View of the distal part of the endoSAMURAI and its flexible arms [START_REF] Hussain | Natural orifice translumenal endoscopic surgery of the gastrointestinal tract[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 1.13 View of the operator interface of the EndoSAMURAI. The movement is transmitted mechanically from the interface to the distal part. [START_REF] Spaun | A multitasking platform for natural orifice translumenal endoscopic surgery (notes): a benchtop comparison of a new device for flexible endoscopic surgery and a standard dual-channel endoscope[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . 39 1.14 View of the distal part of the Anubiscope [START_REF] De Donno | A simulator for assessing control strategies for a novel flexible robot in noscar surgery[END_REF]. 39 1.15 Overall view from the Anubiscope system. It presents 110cm of passive flexible body and a 18.5 cm bendable tip. The wheels on the handle allow for controlling the bending of the distal part, with the possibility of locking them in place. Image from https://www. karlstorz.com/. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 1.16 View of the MASTER mounted on top of a conventional dual-channel endoscope [START_REF] Phee | Robot-assisted endoscopic submucosal dissection is effective in treating patients with early-stage gastric neoplasia[END_REF]. . . . . . . . . . . . . . . . . . . . . . 41 1.17 Clinical use of the MASTER. An endoscopist holds and controls the endoscope while a surgeon controls the robotized instruments [START_REF] Phee | Robot-assisted endoscopic submucosal dissection is effective in treating patients with early-stage gastric neoplasia[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 1.18 Robotic arm mounted on a conventional endoscope [START_REF] Lee | Robotic endoscopy system (easyendo) with a robotic arm mountable on a conventional endoscope[END_REF]. 42 1.19 Controller of the easyEndo being held together with a conventional endoscope [START_REF] Lee | Robotic endoscopy system (easyendo) with a robotic arm mountable on a conventional endoscope[END_REF] The robot independent mappings, on the other hand, depicts the shape of the robot in terms of the task space variables [START_REF] Webster | Design and kinematic modeling of constant curvature continuum robots: A review[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

LIST OF FIGURES

2.5 a: projection of the constant-curvature robot onto the bending plane. On this case, for a null rotation angle φ, the bending plane coincides with the xz plane. b: Depiction of the arc parameters, namely and arc length , angle of the bending plane φ and curvature κ [START_REF] Webster | Design and kinematic modeling of constant curvature continuum robots: A review[END_REF] 2.11 The experimental setup for measuring the 3D position of the flexible tools is similar to the one used for flexible endoscopes (view figure 2.9). The main difference is the position of the markers. . . 2.12 View of the inside of a flexible section actuated by antagonistic cables [START_REF] Bardou | Développement et étude d'un système robotisé pour l'assistance à la chirurgie transluminale[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.13 Static characteristic between the cable displacement and the bending angle on both axis of the same flexible endoscope. . . . . . . . 2.14 Illustration of the physical parameters either estimated or measured for compensating the elongation in the works from Sun et al. [START_REF] Sun | Elongation modeling and compensation for the flexible tendon-sheath system[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.15 Illustration of the lumped-parameter modeling approach applied to a catheter [START_REF] Jung | A modeling approach for robotic catheters: effects of nonlinear internal device friction[END_REF]. . . . . . . . . . . . . . . . . . . . . 2.16 Image of the robotic catheter with the position sensor embedded on its tip (left) and the actuation unit (right) [START_REF] Penning | Towards closed loop control of a continuum robotic manipulator for medical applications[END_REF]]. 2.17 Image of the robotic catheter with the position sensor embedded on its tip (left) and the actuation unit (right) [START_REF] Penning | Towards closed loop control of a continuum robotic manipulator for medical applications[END_REF]]. 2.18 Depiction of the endoscope tip, surgical instrument and markers used for the shape reconstruction [START_REF] Reilink | Image-based hysteresis reduction for the control of flexible endoscopic instruments[END_REF]. . . . . . . . 2.19 Depiction of the endoscope tip, surgical instrument and markers used for the shape reconstruction [START_REF] Cabras | Comparison of methods for estimating the position of actuated instruments in flexible endoscopic surgery[END_REF]. . . . . . . . [Xu et al., 2017a] on the XY-plane. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 3.20 2D trajectory using the approach of [Xu et al., 2017a]

Acronyms

F i #» v Vector v v Versor of #» v i #» v Vector #» v expressed in frame F i # » OP
Vector starting at point O and finishing at point

P i # » OP Vector # » OP expressed in frame F i #» u × #» v Cross product between #» u and #» v #» u • #» v Dot product between #» u and #» v O m×n
Null matrix of m lines and n rows I m×n Identity matrix of m lines and n rows i R mn Rotation matrix from frame F m to frame F n expressed in frame F i i T mn

Translation from frame F m to frame F n expressed in frame

F i M mn = m R mn m T mn O 1×3 1
Homogenous transformation matrix from F m to F n I would like to start thanking my advisor Florent NAGEOTTE for his outstanding mentoring throughout the thesis. The way you allowed me to develop my own ideas, while giving very pertinent inputs and aids to help my advancing, was one of the main factors that allowed the completion of this PhD. You are an excellent advisor, for which I would recommend to anyone wanting to start a PhD in medical robotics. I would like to also thank my director, Michel DE MATHELIN, because this thesis would not exist without his support.

To the members of my jury, Tanneguy REDARCE, Brahim TAMADAZTE, Edouard LAROCHE et Nabil ZEMITI, thank you very much for taking your time to review and evaluate this work. Special thanks to Nabil ZEMITI for being part of my follow-up committee from the start.

My fellow PhD colleagues, you were the best company I could have ever asked during these years. In no specific order of preference, I would like to thank Gaelle, Maciej, Kisoo, Antoine, François, Ajeethan, Jérémy, Quentin, Thibaut, Mouna, Oscar, Loïc, John and Benjamin (and if I forgot someone, I am deeply sorry, but you are also special) for all the talks, discussions and laughs shared on our workspace, during the coffee-breaks or PhD meetings. I would also like to express my gratitude to Dr Demangel and Dr Schmitt for their work during this last year. Without your help, I am not sure if I would have been able to actually finish this document by 2020.

Não posso deixar de mencionar o suporte dos grupos de amigos que pude encontrar desde a minha chegada a Estrasburgo -os intercambistas brasileiros. Vocês foram minha família passageira, que se renovava a cada ano, e estiveram presentes em diversos momentos que não pude estar com minha família de sangue. Sem vocês, esse período teria sido ainda mais duro.

Aos meus grandes amigos que estiveram comigo, seja presencialmente seja apenas conversando de longe, muito obrigado. Sem entrar em detalhes para não ocupar 17 páginas de agradecimento, muito obrigado, sem ordem de preferência, a Edgar Martins, Paula Coeli, Sayuri "Bertrand" Yamashita, Eleni Angelopoulou, Ana Filipa Santos Seiça, Thuy Tuyen Tran e Kristin Limbach.

Aos meus pais, José Humberto Reis Santos Porto e Maria das Graças Aleluia

LIST OF FIGURES

dos Santos Porto, muito obrigado por sempre acreditarem em mim. Vocês fizeram de tudo para que eu tivesse toda a educação que eu precisasse e sempre me apoiaram nas minhas decisões acadêmicas. Desde o meu primeiro intercâmbio em 2013, que eu sei que não foi fácil para vocês se separar do seu filho único, até a partida mais longa para o doutorado, vocês dois sempre estiveram ao meu lado. E por último, mas definitivamente não menos importante, muito obrigado a minha parceira Isadora Campos Torres e a nossa pequena criaturinha Paçoca Lorelai Campos Porto. Muito obrigado por todo suporte que me foi dado, em especial nesse último ano. Muito obrigado pelo seu amor. Muito obrigado por me aguentar nesse período tão difícil de confinamento e de fim de doutorado. Muito obrigado por ser essa pessoa tão fantástica, determinada, levemente ansiosa, mas especialmente com uma habilidade de trazer alegria onde está.

Déclaration sur l'honneur

Declaration of Honour J'affirme être informé que le plagiat est une faute grave susceptible de mener à des sanctions administratives et disciplinaires pouvant aller jusqu'au renvoi de l'Université de Strasbourg et passible de poursuites devant les tribunaux de la République Française.

Je suis conscient(e) que l'absence de citation claire et transparente d'une source empruntée à un tiers (texte, idée, raisonnement ou autre création) est constitutive de plagiat.

Au vu de ce qui précède, j'atteste sur l'honneur que le travail décrit dans mon manuscrit de thèse est un travail original et que je n'ai pas eu recours au plagiat ou à toute autre forme de fraude.

I affirm that I am aware that plagiarism is a serious misconduct that may lead to administrative and disciplinary sanctions up to dismissal from the University of Strasbourg and liable to prosecution in the courts of the French Republic.

I am aware that the absence of a clear and transparent citation of a source borrowed from a third party (text, idea, reasoning or other creation) is constitutive of plagiarism.

In view of the foregoing, I hereby certify that the work described in my thesis manuscript is original work and that I have not resorted to plagiarism or any other form of fraud.

Introduction Contents

Motivation and problem statement

In the past twenty years, the use of flexible endoscopes has developed tremendously. Indeed, the search for less invasive procedures has led to the development of endoluminal and transluminal approaches, also called natural orifice or no-scar surgery. For navigating in the digestive tract, flexible instruments are required in order to limit trauma to the tissues and associated pain. While flexible endoscopes are standard instrumentation for gastroenterology for decades, their use for surgical procedures remain highly problematic. Flexible endoscopes lack the features usually praised by surgeons, such as triangulation (the configuration where the surgical view and two instruments form an inverted pyramid), independent motions and stability of instruments and camera.

This observation has led medical material companies to develop platforms where several flexible instruments, endoscopes or guides are combined to provide independent motions and sometimes added degrees of freedom. Despite numerous attempts, these medical systems have however remained mostly unused and almost none of them are used in clinical routine. The main reason is the difficulty to control such instruments, due to non-intuitive control systems (wheels, handles, levers, locks), the necessity for several physicians to handle different parts of the system combined with a very limited workspace for collaboration near the entry point in the patient. Three paths have been followed since then to continue developing these minimally invasive procedures: specialized training in the use of conventional endoscopes, development of particular skills for a particular platform and robotization of flexible endoscopes. Several teams have initially followed this last path, like the Imperial College (Hamlyn Centre for Robotic Surgery) and the Korea Advanced Institute of Science and Technology (Telerobotics and Control Laboratory).

The interest for robotics was guided by the example of the da Vinci surgical system largely used in laparoscopic surgery. It showed that teleoperation of a medical robot handling articulated instruments can allow a single surgeon to handle camera and up to three surgical instruments to obtain improved dexterity and comfort. Several teleoperated systems based on flexible systems have been developed and brought to clinical or pre-clinical trials (MASTER from Nanyang university of Singapore, ViaCath from Endovia and University of Purdue), demonstrating very encouraging results. As of today, no such system has reached the market yet. This should become reality very shortly, with leaders in medical material, robotics and even web giants working together in this direction.

The ICube laboratory in Strasbourg has participated from early on to this research as a member of a consortium with Karl Storz (German company leader in endoscopy), IRCAD (Institut de Recherche sur les Cancers de l'Appareil Digestif, research and training center in Strasbourg). The efforts have led to the development of a platform called STRAS (acronym for Single port and Transluminal Robotic Assistant for Surgeons), which has been brought to pre-clinical trials for colorectal cancers treatments. The medical and technical results are very encouraging, with decreased operating times and the possibility for novice surgeons to easily perform complex operations normally reserved to experts when using conventional flexible endoscopes. Nevertheless, it was also noted that robotic flexible instruments and flexible endoscopes exhibit unwanted behaviors, which can disturb medical users. Cable-driven flexible systems, which constitute the core of the flexible endoscopes and instruments, are subject to strong non-linear behaviors, such as dead-zones and backlash. At some moments during teleoperation, the user may therefore experience what is felt as delays or lack of responsiveness.

The non-linearities still act as disturbances for the user. They prevent the accomplishment of any precise open-loop automatic motion, because the forward kinematic model of the robotic system is practically erroneous. It must be noted that these problems are not specific to the STRAS robotic system, but general to tendon driven systems, especially flexible ones, for which the cable path is not constant. The problem also exists on manual instruments, however, due to the difficult and non-intuitive control modes, these non-linearities only account for a limited part of the control issues.

Objectives

Improving the experience of medical users is an important goal for facilitating difficult procedures. Developing automatic or semi-automatic modes in teleoperated medical systems is also a valuable aim. These modes can relieve the user from difficult tasks and decrease the cognitive load during long procedures. Both objectives require to solve the aforementioned problems of non-linearities in cable-driven flexible systems. This is the main goal of this PhD thesis. From a robotic point of view, three main approaches can be envisioned:

1. Removing or limiting non-linearities by modifying the mechanical design of the instruments; 2. Modeling the non-linearities in order to take them into account into open-loop control; 3. Using closed-loop control by relying on distal information to remove the effects of the non-linearities.

The three approaches are complementary and can also be advantageously combined. Mechanical modification was not our goal, as we only have limited access to the instruments used in our robotic platform, instruments and endoscopes being produced by Karl Storz. Closing the loop requires embedded sensors, which are not readily available in clinical practice, at the exception of the endoscopic camera. Endoscopic cameras on the other hand have low framerates, which make them little adapted to obtain sufficient bandwidth to remove visible effects. Typically, compensating an unknown pure backlash from distal feedback would require a control loop running at a minimum

LIST OF FIGURES

frequency of 500Hz [START_REF] Bardou | Développement et étude d'un système robotisé pour l'assistance à la chirurgie transluminale[END_REF]. Moreover, providing consistent and robust information from endoscopic images remains a challenge [START_REF] Cabras | Comparison of methods for estimating the position of actuated instruments in flexible endoscopic surgery[END_REF]. In this thesis work, we therefore focus on the second approach, with the aim to improve modeling so as to ideally allow pure open-loop control, or alternatively improve feedback control.

Physical modeling of non-linearities of tendon-driven flexible endoscopes is a very difficult task. It has been shown in several works that even for quasistatic modeling, many physical parameters intervene, some of them are very difficult to know beforehand or even to measure in situ, such as the configuration of the sheaths in which cables run. On the other hand, black box approaches are not well suited either because the behavior is dependent on many state parameters. For the specific instruments of the STRAS robot, it was shown that the behavior is different between supposedly identical instruments. Moreover, the behavior changes over time and use, due to wears, change of tension and change of internal configurations of the system. Following these observations, in this work we try to tackle the problem of the quasi-static identification of the behavior of robotic endoscopic systems from data acquired on the system. Using data for modeling is nowadays referred to as data-driven approaches or machine learning techniques. At the same time, we aim at reducing the time needed for building datasets, with the idea that such identifications, or at least updates, will be needed before each use of the instrument. In the medical context, identification operations should be kept as quick as possible. We therefore propose to combine geometrical modeling based on standard simple models with data-driven models obtained from limited sets of data acquisition. This initial idea has led us to develop a more general framework with guidelines on how basic geometrical knowledge can advantageously be combined with machine learning to develop more accurate models at reduced cost of data acquisition.

Thesis overview

This manuscript presents the work done during my PhD work. The first chapter presents the medical context, with a focus on medical devices developments for endoluminal and transluminal surgery. Chapter 2 presents the basis of this work, in particular the STRAS robotic platform used as the experimental platform and the state-of-the-art on the modeling and compensation of non-linearities in robotic flexible instruments. Chapter 3 develops the proposed approach of combining models and data-driven approaches to model flexible instruments with a single bending direction and flexible endoscopes with two bending directions. Prediction capabilities and open-loop control possibilities are demonstrated on both systems. Chapter 4 proposes a generalization of the developed techniques with different conditions of application on the same material platform.

This work was funded in part by the CAMI labex and in part by the ICube laboratory. The CAMI labex is a laboratory of Excellence supported by the French "Investissements d'Avenir" funds and managed by the national research funding agency ANR under number ANR-11-LABX-0004-01. CAMI Labex, where CAMI stands for Computer-Assisted Medical Interventions, groups 6 research teams in France, working in the field of the assistance to medical procedures by mainly using technologies from the information. One of the aims is to augment the actions of medical users by improving the design or control of their tools. This PhD thesis inscribes in this general goal.

List of publications

Peer Reviewed International Conferences

Evolution of surgery techniques of the digestive tract

The surgical field is relatively new when we take into account the whole history of medicine. Due to the pain and the great risk of post-surgical infection, any incisions in the abdomen were malvised because the risk of fatality was very high until the middle of the 19th century. Thus, the work of a surgeon was quite limited until the popularization of the use of anesthetic agentsmainly attributed to William Thomas Green Morton in 1845 when he used ether to make a tooth extraction -and the introduction of antiseptic methods popularized by Joseph Lister in 1867 when he used carbolic acid to disinfect lesions and surgical tools.

Despite the many existing specialties, we will focus on advances in the area of gastrointestinal tract surgery. This type of surgery concerns organs that are mainly related to digestive processes, such as the esophagus, stomach, large intestine, small intestine and rectum. Therapeutic procedures performed on the gallbladder, liver, appendix or for treatment of hernias or even obesity are also part of this specialty. Nowadays, stomach cancer is the fifth most common type of cancer and the third most lethal according to GLOBOCAN statistics [START_REF] Bray | Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries[END_REF] with colorectum cancer being even worse (second most lethal and third most common), but in the 19th century stomach cancer was the leader of this statistic.

One of the first great specialists in gastrointestinal surgery was Theodor Billroth. He was the first to demonstrate experimentally, in an animal laboratory, that it is possible to perform gastrectomy -partial or complete removal of the stomach -without fatalities after the failed experiments of Jules Péan in 1879 and Ludwig Rydigier in 1880. In 1881, he performed the first successful gastrectomy to remove a carcinoma that occupied about 1/3 of the distal stomach volume [START_REF] Ellis | A history of surgery[END_REF]. This surgery was performed through a large incision in the patient's abdomen. This type of technique, now considered invasive, is also called laparotomy.

For a long time, laparotomy was the norm for performing surgeries due mainly to the difficulties related to the exploration and visualization of the internal organs. Even though some advances were already beginning to appear in the area of endoscopy during the 19th century, the tools were still too primitive and unsafe to be used. These limitations, however, did not prevent the parallel advancement of endoscopy.

The first reports of a functional device allowing internal visualization from a natural orifice is attributed to Philip Bozzini in 1806 [START_REF] Reuter | History of Endoscopy: An illustrated documentation[END_REF]. His invention, called the "Lichtleiter" -light conductor when translated -allowed the illumination of internal tissues by reflecting the light of a candle through a system of mirrors. Although functional for inspection and diagnosis, it was not possible to perform any type of therapy with this device.

The term "endoscopy" was only used many years later with the modification of the original "Lichtleiter" design by the french physician Antonin Jean

EVOLUTION OF SURGERY TECHNIQUES OF THE DIGESTIVE TRACT 25

Desormeaux in 1853 [START_REF] Desormeaux | De l'endoscope, instrument propre a éclairer certaines cavités intérieures de l'économie[END_REF]. His device, Desormeaux's endoscope, was still completely rigid such as the endoscopes used in laparoscopy today. In practical terms, this caused an impossibility of inspecting the upper gastrointestinal tract -great risk of hurting the esophagus and practically impossible to access the stomach through natural cavities except in very particular cases (for example, a reported case was done on a professional swordswallower [START_REF] Saxena | Essentials of pediatric endoscopic surgery[END_REF]).

The first reported case of a laparoscopy was performed on a dog by the german physician Georg Kelling in 1901. He was also the first one to introduce a flexible esophagoscope/gastroscope in 1932. Inspired by a human finger, it consisted of tubes playing the role of phalanges and was completely covered with Indian rubber [START_REF] Vecchio | History of laparoscopic surgery[END_REF]. However, his instrument was not very successful when inspecting rather obscure areas mainly because of difficulties regarding illumination.

In parallel and without being aware of Kelling's advances, Hans Christian Jacobaeus was the first to perform a laparoscopy on a human in 1910. He was also responsible for creating the term laparoscopy. His results were published soon after in 1912, after carrying out 109 laparoscopies in 69 patients [START_REF] Jacobaeus | Über Laparo-und Thorakoskopie[END_REF]. In 1932, the german physician Rudolf Schindler, in conjunction with the manufacturer Wolf, invented the first semi-flexible gastroscope [START_REF] Benedict | Examination of the stomach by means of a flexible gastroscope: A preliminary report[END_REF]. Although the first half was still rigid, the second half of this instrument allowed a maximum bending of about 30 o [START_REF] Gross | Technical evolution of medical endoscopy[END_REF]] by using cables to change its form. Its use was adopted globally thanks to the great increase in terms of safety for patients, especially at the time of insertion [Marks and Dunkin, 2013].

Other major advances that increased flexible endoscopy applications were developed in the 1960s. Harold Hopkins' partnership with Karl Storz was responsible for the foundation of modern endoscopy with the junction of an optical system of rod-lens with a bundle of fiber optics. This combination led to a considerable improvement in image quality that could not have been achieved before. Thanks to this technology, it was also possible to create the first fully functional flexible commercial endoscopy device known as the Hirschowitz fiberscope [START_REF] Hirschowitz | A personal history of the fiberscope[END_REF]. The device was still actuated by long cables used to bend its distal part.

Finally, possibly the biggest breakthrough in allowing flexible endoscopy to spread was the integration of CCD cameras into endoscopes in the late 1960s. Despite the low resolution of the images (100x100 pixels), the first flexible endoscope with a embedded digital camera was made in 1983 by Welch Allyn Inc [START_REF] Wheeler | Endoscopic apparatus[END_REF]. This made possible to acquire the images thanks to this camera at the tip of the endoscope, convert them into an electrical signal -easily transmitted inside the flexible endoscope -and finally be processed by an external device. This allowed a huge gain in space inside the endoscopes, which transformed them into the polyvalent surgical instruments we see today. Many major advances have still been made over the years, such as improved sensors, miniaturization of the size of endoscopes and introduction of robotic devices, but only the latter will be discussed in section 1.3.

Discussion

An important point to consider is why surgical techniques have evolved from laparotomy to laparoscopy and today converge more and more to flexible endoscopy. Laparotomy is considered invasive because a large incision at the level of the abdomen is necessary to perform the surgery. These procedures have several risks for the patient, mainly related to blood loss and postoperative infections. Recovery times and, consequently, hospitalization time are also generally longer when compared with minimally invasive surgeries with equally satisfactory results [START_REF] Bateman | Endoscopic versus laparotomy management of endometriomas[END_REF]. In aesthetic terms, laparoscopy and endoscopy are also much superior, offering the option of small scars or even none at all in some cases (discussed in the chapter 1.2.3).

On the other hand, there are certain difficulties that arise for the surgeon when the size of the incisions in the patient is reduced. Indirect vision, whether by digital cameras on current devices or by mirror systems on the first devices, causes a loss of depth perception. This can be solved by using stereo vision systems, which are not yet standard in flexible endoscopy due to the large exploration movements and these systems not always accepted by medical users -issues with motion sickness are still very prominent. There is a loss of mobility and dexterity of surgeons, as the surgical tools move through a trocar in the case of laparoscopic surgery or through small channels in en- [START_REF] Rathert | Philipp bozzini (1773-1809) and the lichtleiter[END_REF], Desormeaux -[Zada et al., 2012], Kelling - [START_REF] Litynski | Origin of laparoscopy: Coincidence or surgical interdisciplinary thought?[END_REF]], Schindler - [START_REF] Benedict | Examination of the stomach by means of a flexible gastroscope: A preliminary report[END_REF], Hirschowitz fiberscope - [START_REF] Wilcox | Fifty years of gastroenterology at the university of alabama at birmingham: a festschrift for dr. basil i. hirschowitz[END_REF] and video endoscope - [START_REF] Wheeler | Endoscopic apparatus[END_REF].

CHAPTER 1. MEDICAL CONTEXT doscopic surgery. Finally, there is also the loss of tactile sensation because the whole operation is performed "at a distance", without direct contact by the physician [START_REF] Veldkramp | Laparoscopic surgery versus open surgery for colon cancer: short-term outcomes of a randomised trial[END_REF]. These barriers, however, are gradually being broken down thanks mainly to the development of robotic systems.

Flexible endoscopy

In this chapter, we will analyze different types of existing flexible endoscopes, the surgical tools that can be used with these systems and their main applications. An analysis of the advantages and disadvantages of using these systems in the context of digestive system surgeries will also be done.

Types of flexible endoscopes

Despite the more usual use of flexible endoscopes for diagnostic purposes, there are already several techniques that can be used for therapy purposes.

Various types of endoscopes exist and are designed for optimal use in certain areas of the body. The main types and techniques associated with their uses are :

• Oesogastroduodenoscopy (EGD): diagnostic endoscopic procedure that visualizes the upper part of the gastrointestinal tract down to the duodenum by using a gastroscope. The organs usually observed are the oesophagus, stomach and duodenum.

• Duodenoscopy: special type of upper endoscopy that targets the papilla, biliary tract and pancreatic ducts. It is performed by a duodenoscope, which allows a lateral view instead of a frontal one. It also presents an elevator, device that allows an upward/downward movement on the distal part of the working channel, to better control endoscopic tools.

• Colonoscopy: endoscopic examination of the large bowel, including the rectum, colon and terminal ileum by the use of a colonoscope. The insertion point is usually the anus, which allows for a larger diameter (consequently, more tools can be used) compared to gastroscopes.

• Enteroscopy : procedure of using an endoscope for the direct visualization of the small bowel. It is usually a demanding technique given the length of the small bowel (4-6m on an adult) and the difficulty of access. They can be performed by using enteroscopes, which are much longer gastroscopes that allows reaching these more distant areas, or using a system with surgical balloons pull the intestine towards the user.

An illustration of this list can be seen in 1.4.

Diagnostic

One of the most common uses of flexible endoscopes is to diagnose "simple" diseases such as chronic gastro-esophageal reflux disease (GERD) -which can develop in more serious problems such as Barretts esophagus and esophageal adenocarcinoma [START_REF] Pohl | The role of overdiagnosis and reclassification in the marked increase of esophageal adenocarcinoma incidence[END_REF] -or even stomach ulcers. The great potential of these tools is the presence of functionalities such as high resolution cameras, the possibility to enlarge images with optical zoom during inspection or the use of chromoendoscopy techniques -addition of dyes into the GI tract in order to enhance some features.

The use of the endoscopic camera, however, is not the only way to diag- nose diseases with flexible endoscopes. The use of ultrasound probes (Endoscopic Ultrasound or EUS) to diagnose pancreatitis [START_REF] Hocke | Contrast-enhanced endoscopic ultrasound in the diagnosis of autoimmune pancreatitis[END_REF] or even pancreatic cancer [START_REF] Agarwal | Endoscopic ultrasound-guided fine needle aspiration and multidetector spiral ct in the diagnosis of pancreatic cancer[END_REF] is already proving to be a viable strategy for this type of disease. Other visualization methods also used are laser-based techniques, such as endomicroscopy [START_REF] Goetz | Advances of endomicroscopy for gastrointestinal physiology and diseases[END_REF] to perform biopsies of the GI mucosa or even Optical Coherence Tomography (OCT) [START_REF] Sivak | High-resolution endoscopic imaging of the gi tract using optical coherence tomography[END_REF].

Therapy

On the other hand, a strong trend in the use of these instruments for therapy has been emerging since the last decade. For treatment of diseases such as Barretts esophagus, there are possibilities with the use of radiofrequency ablation [START_REF] Ackroyd | Ablation treatment for barrett oesophagus: what depth of tissue destruction is needed[END_REF] both in focal and circonferential form [START_REF] Sharma | Circumferential and focal ablation of barrett's esophagus containing dysplasia[END_REF]. These techniques allow the removal or destruction of the affected tissue by applying heat in the region of interest. The depth and extent of these burns can be modulated with the electrical power and time of exposure.

With the use of endoscopic ultrasound, it is also possible to guide procedures for biliary drainage [START_REF] Kawakubo | Multicenter retrospective study of endoscopic ultrasound-guided biliary drainage for malignant biliary obstruction in j apan[END_REF] -insertion of a small plastic tube that allows the drainage of an obstructed bile duct -or for fine-needle aspiration [START_REF] Bang | A meta-analysis comparing procore and standard fine-needle aspiration needles for endoscopic ultrasound-guided tissue acquisition[END_REF] -insertion of a thin needle in a tissue with abnormal appearance. With the guide of an EUS, the risk associated with these procedures is greatly reduced.

From another point of view, flexible endoscopes have also been used in Source : [START_REF] Storz | Systems for Gastroenterology 3rd edition[END_REF] conjunction with simple surgical accessories such as knives, snares, forceps and needles to perform procedures previously performed either by laparoscopy or laparotomy. Techniques such as polypectomy [START_REF] Ferlitsch | Colorectal polypectomy and endoscopic mucosal resection (emr): European society of gastrointestinal endoscopy (esge) clinical guideline[END_REF], Endoscopic Mucosal Resection (EMR) [START_REF] Rösch | Attempted endoscopic en bloc resection of mucosal and submucosal tumors using insulated-tip knives: a pilot series (including videos)[END_REF] and Endoscopic Submucosal Dissection (ESD) [START_REF] Gotoda | Endoscopic submucosal dissection of early gastric cancer[END_REF] allow the removal of small or long pieces of tissue, including tumors, in a minimally invasive way and without external scars. Peroral Endoscopic Myotomy (POEM) is being performed for CHAPTER 1. MEDICAL CONTEXT treating achalasia -disease that prevents relaxation of the sphincter and an absence of contractions -and other motility disorders of the oesophagus [START_REF] Teitelbaum | Clinical outcomes five years after poem for treatment of primary esophageal motility disorders[END_REF]. Obesity has been treated by the placement of intragastric balloons [START_REF] Tate | Intragastric balloon treatment for obesity: review of recent studies[END_REF] or even by performing Endoscopic Sleeve Gastroplasty (ESG) [START_REF] Sharaiha | Endoscopic sleeve gastroplasty significantly reduces body mass index and metabolic complications in obese patients[END_REF]. All the above-mentioned surgeries can be performed within the concerned lumen, meaning that no incision is required, and everything is done within the organ itself. Another philosophy that is also applied with endoscopic procedures is the use of a natural orifice as an entry point, but then making an incision inside the patient in order to access other parts of the body. A notable technique that has been developed is called Natural Orifice Transluminal Endoscopic Surgery (NOTES) [START_REF] Bardaro | Development of advanced endoscopes for natural orifice transluminal endoscopic surgery[END_REF]. One of the most known applications of said technique is in cholecystectomy -removal of the gallbladder usually performed due to the presence of gallstones (see image 1.7) -either by going through the colon [START_REF] Pai | Transcolonic endoscopic cholecystectomy: a notes survival study in a porcine model (with video)[END_REF], the stomach [START_REF] Auyang | Human notes cholecystectomy: transgastric hybrid technique[END_REF] or even the vagina [START_REF] Zorrón | Notes transvaginal cholecystectomy: report of the first case[END_REF].

Benefits and drawbacks

As noted, different procedures can be completely replaced by options performed by a flexible endoscope and small surgical tools. In general, these options have several advantages, especially for the patient. However, in most cases, these benefits are converted into difficulties for surgeons.

The aesthetic benefits of endoluminal or even transluminal surgeries made with flexible endoscopes are evident. The non-presence of external scars, or even small scars in the case of SPA surgeries, is a great benefit for patient recovery [START_REF] Ikeda | Are there significant benefits of minimally invasive endoscopic thyroidectomy?[END_REF].

CHAPTER 1. MEDICAL CONTEXT

Another considerable advantage in surgeries performed by endoscopes is related to the reduction of post-operative recovery time [START_REF] Fuchs | Minimally invasive surgery[END_REF]. Several studies indicate that there is a great reduction in pain, a faster normality of peristaltic movements and, consequently, an enhancement in the overall recovery time [START_REF] Freeman | Comparison of pain and postoperative stress in dogs undergoing natural orifice transluminal endoscopic surgery, laparoscopic, and open oophorectomy[END_REF][START_REF] Pang | Comparison of safety and outcomes between endoscopic and surgical resections of small (≤5 cm) primary gastric gastrointestinal stromal tumors[END_REF][START_REF] Joo | Endoscopic versus surgical resection of gi stromal tumors in the upper gi tract[END_REF]. On the other hand, an increase in the overall surgery time was also observed depending on the procedure, increasing the strain on surgeons.

On the other hand, flexible endoscopes are not intuitive to use. The control of the bending of the tip, in manual use, is done by turning wheels present on the side of the endoscope handle. The rotation of the handles are transformed into a bending of the distal part of the endoscope, either on the up/down or the left/right direction. Specialized and long-term training is necessary to master the use of this equipment [START_REF] Crespin | Feasibility of adapting the fundamentals of laparoscopic surgery trainer box to endoscopic skills training tool[END_REF][START_REF] Ward | The learning curve to achieve satisfactory completion rates in upper gi endoscopy: an analysis of a national training database[END_REF], which are not an obligation on several medical diplomas across the world. This means that a huge part of recent-graduated surgeons are not trained to perform surgery with flexible endoscopes.

One problem that is emerging is the risk of transmission of Carbapenem-Resistant Enterobacteriaceae (CRE), also known as "superbugs". There are reports of outbreaks dating back 30 years [START_REF] Muscarella | Risk of transmission of carbapenem-resistant enterobacteriaceae and related "superbugs" during gastrointestinal endoscopy[END_REF] and mainly related to the use of duodenoscopes in the gastrointestinal tract [START_REF] Humphries | Superbugs on duodenoscopes: the challenge of cleaning and disinfection of reusable devices[END_REF]. Given the reusable nature of these surgical instruments, a great deal of attention is needed in the cleaning, disinfection and, in some cases, sterilization procedures. However, the complex structure coupled with the presence of several long tubes, makes cleaning procedures extremely difficult (see figure 1.8). Currently, the recommendations are for high-level disinfection, even for procedures of high complexity [START_REF] Petersen | Multisociety guideline on reprocessing flexible gastrointestinal endoscopes: 2011[END_REF][START_REF] Spaun | Use of flexible endoscopes for notes: sterilization or high-level disinfection?[END_REF]. To strongly reduce the risk, one option being studied is the utilization of single-use endoscopes [START_REF] Farr | Single-use, port deployable articulating endoscope[END_REF]Kenney, 2016] [Couvillon Jr, 2016].

Another great difficulty when doing procedures like NOTES is the cooperation between the surgeons involved. Each instrument used, counting the endoscope itself, should be controlled by a different surgeon. In the case of more complex operations, where the use of two-channel endoscopes is required and the tools have several degrees-of-freedom, at least 3 surgeons must cooperate at the same time, controlling different instruments in a non-intuitive manner. An illustration of the setup of an operating room during a manual NOTES operation can be seen in the figure 1.9.

Flexible systems for mini-invasive surgery

Given the difficulties mentioned in chapter 1.2.4, the use of endoscopy techniques in gastrointestinal surgery is still limited. The learning curve for manual use of these instruments is relatively steep, which discourages their use by surgeons with extensive experience in other techniques or by newly trained surgeons. One way to level the skill of these professionals is to make the use of these instruments more accessible -more intuitive to control, more accurate, and less dependent on external factors. One possible approach to accomplish this is the development of mechanical or robotic interfaces to optimize their use in certain surgical procedures.

Mechanical endoscopic systems

These systems have no electromechanical actuators in their structure. They are often designed with innovative mechanics, usually driven by cables, to simplify the use of the endoscope together with other surgical instruments or to give more dexterity and mobility to the surgeon.

R-Scope

This endoscope, developed by Olympus (Japan), was one of the first systems to be used for NOTES and ESD. This system has two channels for the use of surgical instruments -in the case of ESD, it is very common to use a grasper to hold the tissue and a knife to perform the dissection -with the particularity that each instrument can be moved independently of the endoscope along one direction. One allows a vertical movement of the surgical tool, while the other allows only a horizontal movement. It also has two bending sections. The proximal section can only bend in one plane (up/down), while the distal section can bend in two orthogonal planes (up/down and left/right). Once the configuration of the levers and handles has been tuned by the operator, it is possible to lock them in position to perform the procedures (see figs. 1.10 and 1.11).

Although it allows for greater dexterity than conventional dual-channel endoscopes, this system is still relatively limited. Instrument DOFs are not sufficient for the complete performance of surgical procedures, especially if there are complications. Given the positioning of the instrument channels, parallel to each other, triangulation of these tools is almost impossible to achieve. Besides the limitations from a surgical point of view, these changes in the design of the endoscope make it more complex to control. Therefore, the improvements brought by the device is usually overshadowed by its complexity of use [Spaun et al., 2009a].

EndoSAMURAI

Also developed by Olympus (Japan), EndoSAMURAI was thought to try to solve several limitations of R-Scope. The EndoSAMURAI has 2 independent arms (check figure 1.12), each with 5 degrees of freedom, and an interface completely different from those normally used in conventional flexible endoscopes. This system is very effective for performing sutures and pin transfer especially compared to dual-channel endoscopes [START_REF] Spaun | A multitasking platform for natural orifice translumenal endoscopic surgery (notes): a benchtop comparison of a new device for flexible endoscopic surgery and a standard dual-channel endoscope[END_REF]]. Another interesting feature is the presence of a third channel, which is shown to be multi-purpose -it can be used either for a third auxiliary instrument or for irrigation/suction during operation. Its interface, illustrated in the figure CHAPTER 1. MEDICAL CONTEXT 1.13 is only used after a previous positioning of the endoscope. Once properly positioned, it is possible to lock its configuration with the use of an over-tube and manipulate the arms without interference from the endoscope configuration. One of the major advantages over the R-Scope is the guaranteed triangulation of the instruments and their wide range of movements thanks to the additional degrees of freedom. Figure 1.12: View of the distal part of the endoSAMURAI and its flexible arms [START_REF] Hussain | Natural orifice translumenal endoscopic surgery of the gastrointestinal tract[END_REF].

Anubiscope

Another mechanical platform initially developed for application in NOTES is the Anubiscope, developed at the Institute for Research against Digestive Cancer (IRCAD-IHU, France) and produced by Karl Storz (Germany). This flexible endoscope is a little wider than conventional endoscopes, having 16mm of external diameter, but it has features that make it very versatile. Its distal part is composed of a shell-shaped structure, which is completely closed at the moment of body insertion. Once positioned, the "shell" opens and releases the access of 2 working channels for surgical instruments of up to 4.3 mm positioned on the left and right of the endoscopic camera. The outlet of the channels is slightly inclined in relation to the plane of the camera (about 10º outward), which guarantees the triangulation of the instruments. The instruments used with this platform have 3 degrees of freedom, being them transla-Figure 1.13: View of the operator interface of the EndoSAMURAI. The movement is transmitted mechanically from the interface to the distal part. [START_REF] Spaun | A multitasking platform for natural orifice translumenal endoscopic surgery (notes): a benchtop comparison of a new device for flexible endoscopic surgery and a standard dual-channel endoscope[END_REF]. Thanks to all these features, a great improvement in safety and efficiency of procedures such as ESD has been shown experimentally even by less experienced surgeons [START_REF] Diana | Endoluminal surgical triangulation: overcoming challenges of colonic endoscopic submucosal dissections using a novel flexible endoscopic surgical platform: feasibility study in a porcine model[END_REF]. On the other hand, the use of this platform requires the cooperation of at least 2 surgeons at the same time due to the large amount of degrees of freedom present (especially when adding the surgical instruments). This can increase the complexity of the operation, the mental load of surgeons, as well as the total cost of the operation.

Robotic platforms

Even though the purely mechanical solutions can solve several issues of flexible endoscopic surgery, they also introduce problems of manipulation because of their complexity. A possible solution to enhance the dexterity of surgeons without increasing the complexity of the manipulation is by using robotics. Several research groups have developed different robotized flexible endoscopic platforms.

Instead of creating a fully dedicated platform, some research groups developed robotic endoscopy systems which can be mounted on conventional endoscopes with motorized actuation. A known example of this kind of system is the Master and Slave Transluminal Endoscopic Robot (MASTER) from EndoMASTER Pte (Singapore). This system can be mounted on top of any conventional double-channel endoscope and enhances it by deploying two motorized instruments that can be remotely controlled. Up to 9 degrees of freedom can be controlled at the end effector [START_REF] Lomanto | Flexible endoscopic robot[END_REF]. Figure 1.16: View of the MASTER mounted on top of a conventional dualchannel endoscope [START_REF] Phee | Robot-assisted endoscopic submucosal dissection is effective in treating patients with early-stage gastric neoplasia[END_REF].

Figure 1.17: Clinical use of the MASTER. An endoscopist holds and controls the endoscope while a surgeon controls the robotized instruments [START_REF] Phee | Robot-assisted endoscopic submucosal dissection is effective in treating patients with early-stage gastric neoplasia[END_REF].

Another example is the easyEndo [START_REF] Lee | Robotic endoscopy system (easyendo) with a robotic arm mountable on a conventional endoscope[END_REF]. It presents 4 degrees of freedom for the endoscope (2 deflections and rotation are motorized while the translation is manual) and 3 for the instruments (deflection, rotation and translation). The endoscope and the robotic arm are controlled by the means of two small four-way joysticks, one for each part of the system. One of the CHAPTER 1. MEDICAL CONTEXT joysticks is supposed to be held together with the endoscope, to control the insertion manually and the other degrees of freedom with the controller. Figure 1.18: Robotic arm mounted on a conventional endoscope [START_REF] Lee | Robotic endoscopy system (easyendo) with a robotic arm mountable on a conventional endoscope[END_REF].

Figure 1.19: Controller of the easyEndo being held together with a conventional endoscope [START_REF] Lee | Robotic endoscopy system (easyendo) with a robotic arm mountable on a conventional endoscope[END_REF]. Some groups have also developed robotized versions of the mechanical solutions presented at section 1.3.1. An example is the STRAS platform, developed at Strasbourg, France, a robotized version of the ANUBISCOPE. This platform has been constructed with a modular approach -the system has been decomposed in several elemental subsets that perform complementary tasks :

• The cart allows for an easy displacement of the platform with passive wheels as well as an adjustment in the height and orientation of the endoscope (motorized) to allow its use in different contexts. • The cradle is a metallic structure that holds the handle of the endoscope and the T/R modules. It allows for a rotation and translation of the whole system with respect to the main axis of the endoscope.

FLEXIBLE SYSTEMS FOR MINI-INVASIVE SURGERY

• The endoscope module allows the motorized control of the endoscope's deflection. This is achieved by replacing the traditional wheels with an electromechanical one. • The instrument module also replaces the traditional handle of the surgical tools in order to remotely control the bending and the opening/closing of the grasper (when existent on the tool). Two of these modules can be used simultaneously, one for each channel of the ANUBISCOPE. • The T/R modules add two motorized degrees of freedom to each surgical tool. Usually achieved manually, these modules allow a rotation and translation along the the main axis of the instruments.

The details concerning the dedicated controlling interface and telemanipulated robot will be presented at section 2.1. Other systems have been developed as robotic systems from the beginning. This was for instance the case of the ViaCath syste developed by Endovia, which provided a two arms system with an endoscopic guide equipped with the endoscopic camera. This early system was probably the first robotic device aimed at NOTES surgery, and was supported by a major company in medical robotics (Endovia was bought by Hansen medical, owner of the Sensei and Magellan robotic catheters).

Cart

For now, there are no commercially available robotic flexible endoscopic systems aimed at surgical applications. The FlexMed system developed by Medrobotics on the basis of. HARP developed at Carnegie Mellon has been CE approved, but as a hyperarticulated endoscope with follow the leader capability, it is mainly aimed at navigation in the digestive tract. However, there are several products or soon to be products in the related field of single port CHAPTER 1. MEDICAL CONTEXT access surgery, also called single port laparoscopic surgery.

A robotic system from Intuitive Surgical (EUA) for performing Single-Port Access is the da Vinci SP. Its latest version is capable of passing a fully articulated camera along with 3 surgical instruments, each one of them with 7 degrees of freedom, through a 2.5cm cannula. These instruments are controlled through a dedicated console allowing the control of the instruments by the movements of the wrists and the fingers of the surgeon. Its first clinical use was in reported in 2014 in the context of single-port urulogic surgery [START_REF] Kaouk | A novel robotic system for single-port urologic surgery: first clinical investigation[END_REF]. A relatively similar system to the da Vinci SP, also used for Single-Port Access, is the SPORT ™Surgical System [START_REF] Seeliger | Enabling single-site laparoscopy: the sport platform[END_REF] manufactured by Titan Medical Inc (Canada). The access to the patient's body is done via an insertion tube with a diameter of 25mm, which is endowed with 3 channels. One is used to insert a flexible endoscope for 3D visualization and the

FLEXIBLE SYSTEMS FOR MINI-INVASIVE SURGERY

other 2 for passing multi articulated instruments. Each instrument moves with 7 degrees of freedom and its tips are replaceable and meant to be used a single time. The SPORT system, which was initially developed under the name IREP at Columbia University and Vanderbilt university by the team of N. Simaan, relies on continuous flexible instruments driven by superelastic tubes, whereas the DaVinci SP uses discrete rigid joints. [START_REF] Surgical | [END_REF] The main difference between SPA and endoluminal or NOTES surgery is that the insertion tube can be (and usually is) rigid and much shorter. This allows to use rigid transmissions such as rods, and guarantees that the path of the transmission is constant on the main part of their length. This limits non-linear behaviors and allows a more accurate control of the distal degrees of freedom. The fact that the ViaCath system development was discontinued because the control of the instruments by the user was assessed as too complex, mainly because of backlash and non-linearities in the transmissions, is a very representative example of the difficulty to handle the control of flexible bending instruments. In robotic systems such as the Sensei or Magellan from Hansen medical , the control issues are less critical because thee is a single instrument to control and its motions are much more restricted and much slower than for surgical tasks in the gastrointestinal tract.

Conclusion

The evolution in the quality of patient care has evolved drastically since minimally invasive surgery became a feasible option. Huge improvements have been achieved on the techniques employed, resulting in better post-operative outcomes. As seen in this chapter, the latest developments tend to still decrease invasiveness by relying as much as possible on natural orifices and by navigating in the natural lumen of the patient for reaching distant operating sites.

The possible visible advantages for the patients in terms of recovery time, infection rates or aesthetic results, unfortunately come with important drawbacks, which usually mainly concern the physicians : a greater workload, increased surgery time, reduced vision and dexterity, among other issues evoked in this chapter. This has justified the need of not only improving the techniques, but also improving the tools used to perform these surgical gestures.

The use of robotics in the context of flexible endoscopic surgeries has the potential of eliminating most of the drawbacks introduced by minimizing the incisions, while maintaining its advantages. The robotization of flexible endoscopes and instruments is not an easy task, with challenges on the design, actuation and control levels. If not intuitive, these robotic systems may cause more harm than good. The example of the Spider system, developed for single port laparoscopic surgery, and abandoned at the stage of certification obtention because of difficulties of control by surgical users, reminds the very high difficulty to provide useful tools for medical applications.

Even though some great results have been achieved on the use of the STRAS platform in the context of telemanipulated surgeries such as ESD [Zorn et al., 2017], there is still improvements to be made. Some automated gestures, like tissue tracking or compensation for physiological movements, are still very challenging to be performed and can improve even more the quality of the surgical gesture. The reasons why this is still challenging will be discussed in details in chapter 2.

CHAPTER 2. MODELING OF THE STRAS PLATFORM

Description of the STRAS platform

All the techniques proposed and developed in this document were tested on the STRAS robotic system initially developed for NOTES surgery and then adapted for endoluminal procedures. This section is focused on detailing the functioning of this platform, as well as providing details on its dimensions, degrees of freedom, workspace and operation modes.

Telemanipulated system

As explained earlier in the section 1.3.2, STRAS is composed of a main endoscope, a short version of the Anubiscope, and surgical instruments that pass through the internal channels of the endoscope. It is possible to control two instruments simultaneously thanks to the master interface designed in order to mimic the degrees of freedom of these tools. More details will be given in section 2.1.2. The bending degrees of freedom of the endoscope and the flexible instruments are actuated by means of antagonist cables running through sheaths inside the flexible body of the endoscope of instruments. Only the distal ends are controllable, with the tendons applying moments to the discrete vertebrae present in the tip of these systems and thus modifying their configurations.

The main endoscope is 55cm long and is wider than conventional endoscopes, with a diameter of 16mm. STRAS is mainly aimed at procedures in the lower digestive tract, for rectal, sigmoid and descending colon exploration and treatment (see figure 1.4). The distal controllable part has a length of 185mm, in addition to a rigid part of 48 mm where the endoscopic camera is located. It can be deflected in two orthogonal bending planes, both driven by pairs of cables arranged orthogonally. The platform itself also allows for a rotation and translation of the system as a whole, thus providing a total of 4 degrees of freedom to the endoscope.

The distal part is equipped with a mobile shell-shaped structure that allows for easier insertion in the patient's lumen as well as protection of the instruments and the camera, while leaving partial visibility for the surgeon. When opened, it provides access to 3 working channels for surgical instruments -two lateral channels located one in each "shell" (4.3mm of diameter) and a third one located in the center of the rigid part just below the camera (3.2mm of diameter). When the shell opens, it creates a deviation from the central channel. This offset ensures triangulation of the side instruments and prevents tissues and organs from falling into the camera's field of view.

The instruments used in the lateral channels are long and thin. The length of these instruments is usually 900mm, with a distal bending part of 18 mm (only this length can be bent) and a diameter of 3.5 mm. They are also driven by antagonist cables, but only one pair of cables is used, thus providing only one bending plane. They can also be rotated and translated in the channel, but these movements are provided by external actuation directly on the shaft The degrees of freedom of the cart are generally not used except during the insertion and removal phases of the endoscope. The other 10 degrees of freedom, however, can be used freely according to needs in terms of visualization and overall positioning (2 degrees of the endoscope + 2 degrees of the whole system) or in terms of tissues manipulation (3 degrees of each instrument, 2 instruments). The actuation has been dimensioned in order to provide the same capabilities as with the manual Anubiscope, Experiments have been performed in different configurations (straight, slightly bended and strongly bended endoscope) to ensure that the overall performance of the robot would not be affected by the shape of the passive body of the endoscope [START_REF] Nageotte | STRAS: A Modular and Flexible Telemanipulated Robotic Device for Intraluminal Surgery[END_REF].

Controlling interface

The STRAS platform presents a custom-made controlling interface allowing a single physician to control the flexible endoscope and both surgical tools. The interface has been made to be simple and intuitive to use, mimicking as closely as possible the movement of the surgical tools.

Two identical handles compose the controlling console, one for each surgical tool of the telemanipulated robot. The shaft ( 8) is attached to a bracket (10) by the means of a pivot joint (12). This bracket is then connected to a structure by the means of another pivot joint (13) and a translating joint (14), composing the 3 degrees of freedom of the interface. A four-way stick (23) and a trigger ( 22) are also present to control the endoscope and the cradle. See figure 2.2 for the illustration of each element of the interface. The link between the controlling interface and the telemanipulated robot are as follows :

1. The translation along the axis T E controls the translation of the tool; 2. The rotation with respect to the axis P E controls the rotation of the tool with respect to the channel axis; 3. The rotation with respect to the axis P M controls the bending of the tool; 4. The trigger ( 22) controls the opening and closing of the grasper of the tool when existent; 5. A four-way stick (23) that can be moved four discreet directions, indicated as North(N)-South(S) or East(E)-West(W), controls the bending of the endoscope along its orthogonal bending planes (N-S for one direction and E-W for the other) or the translation and rotation of the cradle.

The stick on one of the handles controls the endoscope while the other controls the cradle.

These relations are also illustrated at figure 2.3. The scaling factor for each joint is determined based on the mechanical end stops of each joint and and the range of motion of the controlling interface. These controlling interfaces have been designed after observing that standard commercial interfaces did not allow to provide a good control of the flexible instruments. In particular, two problems were identified. First, imperfections in the displacement of the instruments caused large discrepancies between the controlling interface and robot, causing difficulties for fine control. Second, kinematic singularities of the instruments (straight configura-CHAPTER 2. MODELING OF THE STRAS PLATFORM tion in particular) could not be handled easily with interfaces without equivalent singularities, even when trying to block motions actively, because the position of the singularities could not be mapped precisely and constantly on the master side. These problems are both linked to the actuation imperfections that will be discussed in the section 2.5.

The next sections will be focused on the modeling of the components of the telemanipulated robot, meaning the flexible endoscope and the flexible surgical tools.

Kinematic modeling of continuum systems

A flexible endoscope and the flexible surgical tools can be classified as being continuum systems. Contrary to most of robots used in the industry which are composed of a finite number of discrete joints that define their movement, a continuous robot is classified as an under-actuated system -it has theoretically an infinite number of local joints and can take virtually an infinity of shapes when actuated from an external source, but has a limited amount of actuators.

To model such systems, the usual approach consists in defining 3 spaces, each one of them containing different information about its shape [START_REF] Webster | Design and kinematic modeling of constant curvature continuum robots: A review[END_REF]. The task space represents the space where the task of the robot is defined and usually depicts the position and orientation of its terminal effector. The configuration space describes the shape of the robot's body, usually described by its central line. Finally, the actuator space illustrates the required actuation to be performed in order to change the shape and perform the task. On the physical system, it is normally found on the proximal side of the system. An illustration of these spaces as well as the mappings are shown in figure 2.4. Figure 2.4: Depiction of the 3 spaces used for modeling the geometry of constant-curvature robots. The robot-specific relations transform the robot's actuation into the variables describing its configuration in space. The robot independent mappings, on the other hand, depicts the shape of the robot in terms of the task space variables [START_REF] Webster | Design and kinematic modeling of constant curvature continuum robots: A review[END_REF].

Therefore, determining the direct geometric model is equivalent to determining the relations f specif ic and f independent , whereas the transformations

FLEXIBLE ENDOSCOPE WITH 2 BENDING PLANES

55

f -1 independent and f -1 specif ic constitute the inverse geometric model. A classic hypothesis usually used for this class of systems is the piecewise constant curvature -each link of the robot can be modeled as an arc of a circle defined by 3 arc parameters during bending. The parameters are :

• The curvature κ : defined as the inverse of the bending radius r, it denotes how far from a straight line the current configuration of the curved link is.

• The angle of the bending plane φ : angle which defines the plane all the points of the arc belong to.

• The arc length : denotes the mean length (central line) of the system.

It is also common to use the bending angle β as a parameter by replacing either the length or the curvature κ since the relation between these 3 parameters is linear :

β = lκ. ( 2

Flexible endoscope with 2 bending planes

The actuation on flexible endoscopes can be done with several configurations. As examples, there are simple endoscopes with only a pair of cables (which means only one bending plane), three independently actuated cables (non intuitive for manual use, normally only existent on robotic applications) or 2 pairs of cables. The structures using pairs of cables can be either antagonistic or independent, with antagonistic being the configuration found on most of the commercial endoscopes. In this section, we will focus on endoscopes with 2 orthogonal pairs of antagonistic cables, since it is the one used on the context of this thesis.

With respect to the general modeling of arcs given in figure 2.4, endoscopes are inextensible. This means that the length is constant and equal to the relation β κ .

To adapt for this type of system, instead of using the curvature κ, we use the bending angle β to constitute the configuration space. The main advantage of using the bending angle instead of the curvature is its linear dependency with respect to the actuation.

Direct geometric modeling

The distal end of a flexible endoscope, as shown previously in section 2.1, is composed of a flexible section and a rigid section. The flexible section has a length L f and will be considered inextensible longitudinally, meaning that length of the central line remains constant during bending. The rigid section has a length L d and is the part of the endoscope that embeds the endoscopic camera.

A frame denoted F 0 is attached at the proximal end of the flexible section at its central point O. This frame will be the base frame used as reference for defining the task. The #» i 0 and #» j 0 axis point to the directions where each pair of antagonistic cables are attached, while the #» k 0 is aligned with the tangent to the central line of the endoscope at point O (see figure 2.6). At the end of the rigid section, also at its central point, the frame F end is attached.

Our objective here lies in expressing the origin of the frame F end attached to the central point of the distal end O end in function of the arc parameters κ, φ and β in the base frame F 0 . To do so, let us consider the shape of the endoscope in the bending plane Π depicted in picture 2.6. By using a cylindrical parameterization of the task space, we can express the cartesian position of the end effector in terms of the distance to the z-axis ρ, the rotation θ around the z-axis of the frame F 0 and the depth d with :

ρ = x 2 0 + y 2 0 ; (2.2) θ = arctan2(y 0 , x 0 ); (2.3) d =z 0 (2.4)
where the function arctan2() is defined as : where 0 P = (x 0 , y 0 , z 0 ) is the position of the end effector expressed in the frame F 0 . The cylindrical parameterization helps in finding the inverse kinematic model in the next section.

arctan2(y 0 , x 0 ) =                arctan y 0 x 0 if x 0 > 0 and y 0 0 arctan y 0 x 0 + π if x 0 < 0
In the plane Π, showcased in figure 2.8, one can see that the distance to the z-axis ρ is a function of the bending angle β as :

ρ = L f β (1 -cos β) + L d sin β (2.6)
and the depth d :

d = L f β + L d cos β. (2.7)
The relation between the rotation θ with respect to F 0 and the angle of the bending plane φ is trivial, since both angles coincide. Thus :

θ = φ. (2.8)
The equations (2.6), (2.7) and (2.8) represent the robot-independent mapping, because regardless of the actuation methods, the shape of the flexible endoscope will be the same provided the hypothesis of constant curvature is respected.

CHAPTER 2. MODELING OF THE STRAS PLATFORM

The robot-dependent mapping is determined by taking into account the position of the pair of cables with respect to the endoscope. The attachment points on the proximal and distal end of the flexible section are shown in the figure 2.6 as black dots.

When the endoscope is not bent, meaning it is in an upright position, the length of the cables of each side is the same as the length of the flexible section L f . This way, we can define the actuation as being the change in length with respect to its reference straight length as :

∆L i = L f -L i (2.9)
with i = {x, y} indicating which pair is actuated based on its position with respect to the base frame F 0 . When one of the cables is pulled and its length is reduced, the corresponding antagonistic cable is released and its length is increased. Note that each cable has a different curvature κ i and, thus, a different radius of curvature R i . The radius of curvature for the cable i is :

R i = L i β (2.10)
as can be seen in figure 2.8. Since the radius of curvature is different for each cable, one can define a radius variation in the same sense as the length variation during bending as :

Cable attaching points

Φ ΔR 2 ΔR 1
∆R i = R f -R i (2.11)
with R f the radius of curvature of the flexible section. It should be noted, however, that while the length of the central line is always constant, the curvature depends on the configuration of the tool. By using equations (2.9), (2.10) and ( 2.11), one can determine :

∆L i = L f -L i = βR f -βR i = β∆R i (2.12)
By projecting the attaching points of the cables onto the bending plane Π, we can obtain (see figure 2.8) :

∆R x = D 2 cos φ (2.13)
and

∆R y = D 2 sin φ (2.14)
with D being the distance between the attaching points of the pair of cables. This value coincides with the diameter of the flexible section in our system.

CHAPTER 2. MODELING OF THE STRAS PLATFORM

By combining equations (2.12), (2.13) and (2.14), we can obtain :

∆L x = β D 2 cos φ ∆L y = β D 2 sin φ β 2 D 2 2 = ∆L 2 x + ∆L 2 y β 2 =       ∆L x D 2       2 +       ∆L y D 2       2 .
(2.15)

This way, we can define bending angles along each of the main planes. The bending angle along the x-z plane will be called β x and the bending angle along the y-z plane will be called β y . Their relation is then :

β 2 = β 2 x + β 2 y (2.16) with β x = ∆L x D 2
and

β y = ∆L y D 2
.

(2.17)

The only missing link is the relation between the angle of the bending plane φ and the actuation. By dividing equation (2.14) by equation (2.13) and using equation (2.12) one obtains : .18) Then, the relation between the angle of the bending plane φ and the actuation is :

tan(φ) = ∆L y ∆L x . ( 2
φ = arctan2(∆L y , ∆L x ). (2.19)
By using equation ( 2.17), one can also show that :

φ = arctan2(β y , β x ).
(2.20)

Inverse geometric modeling

In this section we focus on the obtention of the inverse robot-specific mapping and inverse robot-independent mappings. These mappings are necessary for obtaining the necessary actuation to achieve a desired pose when controlling the endoscope.

For the inverse robot-independent mapping, let us consider a desired point defined in the task space using a cylindrical parameterization P * = (ρ * , θ * ). Given the structure of the flexible endoscope and the availability of only 2

FLEXIBLE ENDOSCOPE WITH 2 BENDING PLANES

61 DOFs, it is not possible to enforce a desired depth d * given a desired distance to the z-axis ρ * .

The inverse relation between the desired rotation θ * with respect to F 0 and the desired angle of the bending plane φ * is still trivial :

φ * = θ * .
(2.21)

In order to find the desired bending angle β * to achieve the desired distance to the z-axis ρ * , it is necessary to numerically invert the equation (2.6) :

ρ * = L f β * (1 -cos β * ) + L d sin β * inv -→ β * (ρ * ). (2.22)
So, the inverse robot-independent mapping is determined by equations (2.21) and (2.22).

From the configuration space to the actuator space, one needs to determine the inverse robot-independent functions. Let's first determine the desired bending angles along each of the main planes β *

x and β * y in function of the desired configuration space. By replacing (2.20) in (2.16), one obtains :

β * 2 = β * x 2 + β * y 2 β * 2 = β * x 2 + (tan φ * β * x ) 2 β * 2 = β * x 2 (1 + tan φ * 2 ).
(2.23) So, the desired bending angle along the x-z plane is :

β * x 2 = β * 2 1 + tan φ * 2 .
(2.24) By using equation (2.16), it is then possible to determine the desired bending angle along the y-z plane as :

β * y 2 = β * 2 -β * x 2 .
(2.25)

Then, to determine the desired cable displacement on each axis, one just needs to invert equation (2.17), to obtain :

∆L * x = β * x D 2 and ∆L * y = β * y D 2 . (2.26)
This way, the inverse robot-specific mapping is a combination of equations (2.24), (2.25) and (2.26).

Figure 2.9: The experimental setup is composed of the STRAS platform equipped with Chilitag markers [START_REF] Bonnard | Chilitags 2: Robust fiducial markers for augmented reality and robotics[END_REF] on the tip and the external 3D position measuring system.

Discussion

In order to validate this model, some 3D trajectories have been performed with the STRAS platform to compare the measure data with the prediction. The experimental setup is shown in image 2.9.

The distal positions are provided by an external measurement system composed of two AVT Prosilica GC660 cameras mounted with F1.8/6.5-52 mm objectives. The fiducial markers are being used to increase the accuracy and simplify tracking tasks, which are necessary in order to make the 3D reconstruction of the tip of the tool. The accuracy of this system used with marker tracking has been assessed to [0.1, 0.1, 0.2] mm.

The RMS error obtained when validating this model was 18.3 mm, with a variance of 8.4 mm 2 and a maximum of 28.8 mm. These errors are unacceptable for automatic positioning of the endoscope. The source of these errors are varied and will mostly be discussed in section 2.5.

Flexible surgical tools

The flexible surgical tools, in terms of modeling, are very similar to flexible endoscopes. The schematic 3D view of these tools is basically the same as the one illustrated in figure 2.6, except for the cable attachment points -the surgical tools only have one bending degree-of-freedom, which means they only possess one pair of antagonistic cables. Another difference, which is crucial for its proper functioning, is the addition of other 2 joints -a prismatic and a 2.4. FLEXIBLE SURGICAL TOOLS 63 revolute joint allow this tool to translate in the direction of the z-axis of the channel frame and rotate around the same axis.

Direct geometric modeling

Considering the similarities between both systems, the robot-independent functions are almost identical for the flexible surgical tools and the flexible endoscopes. Equations (2.6) and (2.8) are still valid. There would be a difference if the orientation was taken into account, since another rotation would be added around #» k 0 in the base frame. However, the depth d can now be controlled given the additional degree-of-freedom of the surgical tools. By looking the projection of the tool at the bending plane (figure 2.10), one has :

d = t + L f + L d -∆d (2.27)
with t the translation applied by the prismatic joint and ∆d being the depth difference caused by bending. This variable can be calculated as :

∆d = L f 1 - sin β β + L d (1 -cos β).
(2.28)

The robot-specific functions, however, require more attention given the way that the degrees-of-freedom work in the surgical tool. For the rotation CHAPTER 2. MODELING OF THE STRAS PLATFORM and the translation, the relations are trivial since the joints directly operate these variables. This means that :

t = q trans
(2.29) and φ = q rot (2.30)

with q trans and q rot the joint coordinates for the translation and rotation joints respectively.

For the bending joint, taking into account that only one pair of antagonistic cables is present, it is equivalent as using the equations of the flexible endoscope considering that one of the orthogonal cable pairs is blocked and cannot change its length. For this document, we will consider that ∆L y is always null. This way, by adapting the development made in (2.15), one obtains:

β = ∆L D 2 (2.31)
with ∆L the actuation for the bending joint -there is no need for an index x or y considering there is only one bending direction.

Inverse geometric modeling

The inverse robot-independent functions shown for the flexible endoscope, concerning the task space variables ρ, with equation (2.22), and θ, with equation (2.21), are still the same. The the modification of the joints arrangement have no impact on these two variables.

The desired depth d * can be converted into a desired translation t * by inverting equation (2.27) provided that the desired bending angle β * has already been obtained by the means of equation (2.22). The reason why we fix the distance bending angle first is because it is the only parameter that can influence the distance to the z-axis ρ. On this case :

t * = L f + L d -∆d(β * ) -d * .
(2.32)

The inverse robot-dependent functions is quite simple to obtain. Since the prismatic joint and revolute joint directly control respectively the translation and the rotation of the tool with respect to the channel frame, one has :

q * trans = t * (2.33) and q * rot = φ * . (2.34)
For the bending joint, the same simplification used for the direct geometric model can be used here. By considering the equations of the flexible endoscope with ∆L y = 0 (consequently β y = 0), one obtains :

∆L * = β * D 2 . ( 2

Discussion

To validate the proposed model, some trajectories were performed with each degree of freedom separately to evaluate the overall 3D positioning error based on the prediction of the model. The experimental setup is shown on figure 2.11. It is the same external 3D position measuring system as used with the flexible endoscope, but with markers placed both on the tip of the flexible tool and on its base. This is done in order to reconstruct the channel frame F ch and to express the coordinates in this frame.

Figure 2.11: The experimental setup for measuring the 3D position of the flexible tools is similar to the one used for flexible endoscopes (view figure 2.9). The main difference is the position of the markers.

The comparison was performed on 250 points distributed onto the workspace. The assessment provided an RMS error of 8.9 mm with variance 22.1 mm 2 , and maximum error 17.2 mm. Similarly to the flexible endoscope, these large errors do not allow to perform automatic tasks. The main source of errors will be discussed on section 2.5.

Cable actuation

Cables are largely used as actuation on mechanical systems when the power source has to be placed remotely from the actual terminal organ. The main advantages of using cables are their low density, high resistance and high capacity to transmit forces over long distances [START_REF] Ou | Kinematic synthesis of tendon-driven manipulators with isotropic transmission characteristics[END_REF]. Many applications use cables as a means of motion transmission: parallel robotics -with applications for transporting loads on large workspaces [START_REF] Dallej | Vision-based modeling and control of large-dimension cable-driven parallel robots[END_REF] -, medical robotics -endoscopic systems as studied, laparoscopic instruments CHAPTER 2. MODELING OF THE STRAS PLATFORM [START_REF] Broeders | Robotics revolutionizing surgery: the intuitive surgical[END_REF], but also systems requiring MRI compatibility [START_REF] Pfeil | Observations and experiments for the definition of a new robotic device dedicated to ct, cbct and mri-guided percutaneous procedures[END_REF] -, rehabilitation robotics -with the use of exoskeletons [START_REF] Schabowsky | Development and pilot testing of hexorr: hand exoskeleton rehabilitation robot[END_REF] -among several others. These cables can be antagonistic (complementary extension/retraction actions) or independent (extension and retraction of each cable is done through separate mechanisms). The antagonist configuration is simple to implement, intuitive to use manually and can be actuated with only one motor per pair of cables when robotized. The independent configuration offers more options for applying cable tension and control strategies. Both are valid depending on context and application. In this manuscript, we will focus on discussing the issues on the antagonistic configuration, since this is the one used on both systems we are interested in.

As discussed previously, the geometric model of both the flexible endoscope and flexible surgical tools do not present the required accuracy for precise medical applications. There are many reasons for this kind of behavior:

• The model relies on the hypothesis of constant curvature during bending, which is usually not true. To achieve this shape, a constant tension distribution throughout the bending section is required [START_REF] Camarillo | Mechanics modeling of tendon-driven continuum manipulators[END_REF]. Due to friction, the tension on the base of the bending section is usually higher than at the tip;

• Ageing of the vertebrae that compose the bending section can also have a great influence on how the tension is being distributed. Some vertebrae can deteriorate faster than others, changing the maximum angle between two successive vertebrae. The increasing appearance of friction on the axis between vertebrae is also a factor that influences the tension distribution over the tendons.

• The cable transmission itself is subjected to several non-linearities [START_REF] Agrawal | Modeling of transmission characteristics across a cable-conduit system[END_REF]. The interaction between the long cables and the sheaths on the interior of the flexible bodies create internal friction that is really challenging to model and very difficult to predict given the large number of variables that have an impact on its behavior.

From the experience acquired on manipulating these systems, even though all the previous exposed reasons are valid and introduce non-linearities to the system, we will be focusing on the issues brought by the use of cable transmissions. Even though the hypothesis for constant curvature is usually not respected, its influences are minor compared to the non-linearities created by using cables. This may not necessarily be the case for applications with external loads, but these will not be discussed on this thesis. Figure 2.12 shows how usually the motion is transmitted from the proximal to the distal end of a flexible instrument. The motor is attached to a pulley that is rotated to change the length of the cables on each side of the Figure 2.12: View of the inside of a flexible section actuated by antagonistic cables [START_REF] Bardou | Développement et étude d'un système robotisé pour l'assistance à la chirurgie transluminale[END_REF].

controllable distal section. The cables are attached to the last vertebra and on the pulley, while the sheaths are generally attached in a fixed position at both ends of the flexible body. On the passive body of the flexible tool, the sheaths are usually free so as to avoid interfering with the instrument's flexibility.

As can be seen, the diagram of figure 2.12 and principles of the transmission are quite simple. However, two main non-linearities are usually introduced when using this kind of transmission :

• Backlash : the motion is not transmitted directly from the proximal side to the distal side at a change of direction. The consequence is the introduction of hysteresis between the movement of the motors and the configuration of the tool.

• Dead-zones : the motion between the proximal and distal part is not locally transmitted but without a change in direction. This non-linearity can for instance occur due to slack in the cables, which usually happens near the central configuration of the tool or endoscope.

These effects can greatly reduce the user experience with the tools. A time delay is perceived on every change of direction giving the impression of a slow and non-responsive system. They can also reduce the precision during manipulation, because the user has to compensate for these non-linearities on the fly. These characteristics are also tool-dependent, meaning that two tools, even though very similar in terms of construction, can behave differently. Some examples can be seen on figure 2.13, showing the behavior of the bending joint of several tools.

The static characteristics of the cable transmission greatly deviate from the linear behavior that is expected. Moreover, there is a high variability between the curves, even for the same instrument, supporting the hypothesis that it is far from trivial to model such systems. This issue has already been studied for several years and is usually referred to as "backlash compensation" -even though other non-linearities are often also taken into account. The next sec- tion is dedicated on the approaches developed over the years to try to solve this issue.

Control strategies for tendon-actuated systems

As seen in sections 2.3, 2.4 and 2.5, only describing the geometry is not enough neither to accurately predict the position of such systems nor to control them given the presence of non-linearities. For mitigating this issue, several approaches have been developed over the years based on different principles. A non-exhaustive list of backlash compensation methods applied to cable transmission is presented in this section.

Physical modeling

The first idea that comes to mind is to model the physical behavior of the tendon-driven transmission. This involves investigating the main causes, such as cable elongation, internal friction between the cables and the sheaths, the elastic effects of each vertebrae, etc.

An example is the work from [START_REF] Sun | Elongation modeling and compensation for the flexible tendon-sheath system[END_REF], in which the authors study the tendon elongation and tension transmission in a tendon-sheath system. The experiment consisted in fixing the proximal end of the tendon on a DC servo motor, the distal end on a spring system and applying different input tensions to observe the behavior of the tendon. Two load cells are also used, one near the DC motor and another near the spring system, in order to measure the input and output tensions. With these measures in addition to the 2.6. CONTROL STRATEGIES FOR TENDON-ACTUATED SYSTEMS 69 position measurement provided by an encoder, the authors aimed to validate an elongation model which is a function of a friction coefficient (estimated beforehand), a tendon property (also estimated beforehand) on a tendon with reduced length. An overview of these parameters can be seen on figure 2.14. Figure 2.14: Illustration of the physical parameters either estimated or measured for compensating the elongation in the works from Sun et al. [START_REF] Sun | Elongation modeling and compensation for the flexible tendon-sheath system[END_REF].

In the works from Jung et al. [START_REF] Jung | A modeling approach for robotic catheters: effects of nonlinear internal device friction[END_REF], the authors model a robotic catheter using a nonlinear lumped-parameter approach accounting for the internal friction of the cables. The idea of a lumped-parameter model is to model the bending section as a series of linear springs and dampers. An illustration of this approach can be seen on figure 2.15.

In order to incorporate the effects of internal friction on this lumpedparameter model, a Dahl friction model [START_REF] Dahl | A solid friction model[END_REF] is incorporated on each lumped-mass element. To identify this model, a total of 11 parameters must be either measured or estimated through a series of experiments or measurements. Some of these parameters are simple to obtain (length, inner and outer diameter, number of discreet elements), but others require some engineering design data (mass, stiffness and damp for the lumped-elements combined with the initial slope, exponent, constant steady-state friction and friction coefficient used on the Dahl friction model).

Another work based on compensating the elongation of the cables during manipulation comes from Xu et al. [START_REF] Xu | Motion compensated controller for a tendon-sheath-driven flexible endoscopic robot[END_REF]. Their work uses a feedforward compensation controller with an internal tension-transmission and elongation model. In order to predict the nonlinearities of the transmission, 9 parameters must be estimated or measured.

Even though their results are very interesting, their application on a flexible endoscope seems very difficult if not impossible. The Bowden cables are normally very long, these systems do not usually include force sensors on neither end and the knowledge about the cable disposition inside the passive body of the endoscope is generally required. These drawbacks make their use Figure 2.15: Illustration of the lumped-parameter modeling approach applied to a catheter [START_REF] Jung | A modeling approach for robotic catheters: effects of nonlinear internal device friction[END_REF]. very challenging on a real scenario.

External sensors

Another philosophy is to add sensors to the flexible system in order to measure its distal position (or depending on the sensor, even the shape of the flexible sections) in real time during use. This way, it becomes possible to use closed loop control strategies to compensate the non-linearities. The introduction of sensors can also be complementary to the use of a previously formulated model.

One of the first works seen in the literature showing interesting results is the work from [START_REF] Penning | Towards closed loop control of a continuum robotic manipulator for medical applications[END_REF]. Their work consisted on using a trak-STAR system (Ascension Technology, EUA), which is a electromagnetic localization system, in order to track the position of a catheter and implement a position control loop. The authors were focused on evaluating the improvement on accuracy and repeatability of the closed-loop system. Even though the results are not groundbreaking, it opened the way for several other works to be inspired and improve upon it. The illustration of how the sensor was integrated on their system is shown on figure 2.16. The work from Roesthuis et al. consisted of using Fiber Bragg Grating sensors in order to estimate the shape of flexible minimally invasive surgical instruments and implement a position control loop [START_REF] Roesthuis | On using an array of fiber bragg grating sensors for closed-loop control of flexible minimally invasive surgical instruments[END_REF]. These sensors allow for the measuring of axial strain on the area they are located, giving a local estimation of the curvature of the tool. By placing several arrays along the flexible instrument, it is possible to interpolate the points in between to reconstruct the 3D shape of the tool. Their results are impressive, achieving a reduction of the mean error by over 10 times depending on the trajectory being performed. Figure 2.17 shows the disposition of a set of arrays of fiber Bragg Gratting sensors on a flexible instrument.

Figure 2.17: Image of the robotic catheter with the position sensor embedded on its tip (left) and the actuation unit (right) [START_REF] Penning | Towards closed loop control of a continuum robotic manipulator for medical applications[END_REF].

CHAPTER 2. MODELING OF THE STRAS PLATFORM

The main issue of these strategies are mainly linked to the technological challenges that are imposed when integrating external sensors on miniature devices [START_REF] Liu | Large deflection shape sensing of a continuum manipulator for minimally-invasive surgery[END_REF]. All of these systems were custom made for tests in laboratory, but putting them in use for clinical applications is a much more complicated task. Not only there are issues with the miniaturization of these devices, but also all the aspects regarding safety and sterilization must be taken into account.

Endoscopic camera

Considering the challenges posed by integrating new external sensors to flexible surgical tools, a way to circumvent this issue is by using the sensors readily available on any flexible endoscopic device: the endoscopic camera. Mostly used for giving the physician an internal view of the interior of the patient, these cameras can be used as sensors for estimating the pose not only of the endoscope itself, but also from adjacent surgical tools.

In the works from Reilink et al., the author proposes an online hysteresis estimation by using only the endoscopic camera as measuring device [START_REF] Reilink | Image-based hysteresis reduction for the control of flexible endoscopic instruments[END_REF]. The approach proposed requires the estimation of the current configuration of the endoscopic tools by identifying fiducial markers on the endoscopic image and performing a reconstruction based on their location. A simple hysteresis model is then updated based on the difference between the estimated configuration of the tool and the prediction provided by the hysteresis model. Figure 2.18 shows the endoscope tip, the flexible tool and the location of the markers.

Figure 2.18: Depiction of the endoscope tip, surgical instrument and markers used for the shape reconstruction [START_REF] Reilink | Image-based hysteresis reduction for the control of flexible endoscopic instruments[END_REF].

Another similar approach, but with the addition of data-based learning methods, is presented in the works of Cabras et al. [START_REF] Cabras | Comparison of methods for estimating the position of actuated instruments in flexible endoscopic surgery[END_REF]. The approach is divided in two main steps : the first one is the detection of the instrument on the endoscopic image by the means of marker segmentation; the second is the 3D position estimation of the tip of the endoscope. The learning data is obtained by using two electromagnetic trackers (Aurora from NDI Medical), one used as a reference frame positioned at the endoscope and another used to measure the tip position. The setup is detailed on figure 2.19. Figure 2.19: Depiction of the endoscope tip, surgical instrument and markers used for the shape reconstruction [START_REF] Cabras | Comparison of methods for estimating the position of actuated instruments in flexible endoscopic surgery[END_REF].

Even though this is a great approach for dealing with hysteresis problems, it is still not enough if the clinical applications rely on compensating really fast movements -for example, the compensation of the peristaltic movements would still be a really challenging task to perform using only the information from the camera. This is mainly due to the quality of the images and frame rates that can be obtained with such devices (usually around 25 FPS or less). To implement efficient feedback control strategies, this sampling rate is too slow.

Conclusion

Performing automated tasks with a flexible endoscopic robot is still a real challenge. The actuation, typically through long cables, which allows the desired flexibility of the endoscopes and instruments, introduces a variety of non-linearities. These non-linearities have a large impact on the precision, speed and overall performance of the robot.

The use of sensors to compensate these issues is still a complex problem. Integration of sensors in a miniaturized form is always a technological challenge, especially when taking into account the constraints of the surgical context -the possibility of being disinfected or sterilized while maintaining its functionality, as well as the compatibility with other devices in the operating room is not an easy task to achieve.

Modeling these non-linearities has also been proven to be a complex feat. To obtain models, it is necessary, in most cases, to incorporate sensors (such as force sensors for measuring the tendons tension) that are usually not readily available on these platforms. Considering that these models can vary greatly on a single system, as shown experimentally for the flexible endoscope, and that they can vary over time, some other modeling strategies should be envisaged.

With all of this under consideration, machine learning might be an interesting candidate for identifying models. These approaches could rely on the use of external sensors, not embedded on the flexible endoscope nor the surgical tools, and could also be used to learn specific behaviors just before the medical procedure. The obtained models are also usually flexible, possibly allowing to take into account the variability observed between each surgical instrument.

In chapter 3, learning-based approaches will be studied. An overview of machine learning techniques applied into robotics will be presented in order to understand their capabilities and potential limitations. Then a novel technique combining the advantages of data-driven methods and classic kinematic modeling is proposed both for the flexible endoscopes as well as the surgical tools. Experimental results will also be shown to validate the proposed approach, but also to understand its shortcomings. Our aim is to achieve a precision of 1 mm in the positioning of the tool, accuracy typically expected by surgeons. The training phase (data acquisition and learning phase) should also be performed under reasonable time, so not to disturb the surgical flow. Considering the mean time for manual ESD procedures is of 66 minutes [START_REF] Arezzo | Systematic review and meta-analysis of endoscopic submucosal dissection vs endoscopic mucosal resection for colorectal lesions[END_REF], we are aiming to keep training time around 6 to 7 minutes, 10% of the overall average procedure time.

Chapter 3

Non-linearities compensation mixing machine learning and kinematic analysis

Using machine learning for robot control

In this section, we will focus on different control techniques that have been used for the control of flexible robots by relying on machine learning techniques.

Classification of machine-learning techniques

Machine learning algorithms can be classified based on several principles. The structure and type of data, the task to be performed and the complexity of the model to be learned are some of the criteria used for this categorization.

Supervised vs Unsupervised learning methods

The type of data available for training is extremely relevant when determining the type of task that can be performed and which algorithm to choose. These algorithms fall under the following categories :

Supervised learning: The objective is to predict the outputs from a list of independent inputs. Note that the basic characteristic of supervised learning systems is that the training data contains the desired response. In this case, we say that the data are labeled with the answers or classes to be predicted.

Unsupervised learning: The task of the algorithm is to group unlabeled examples, i.e. examples that do not have the class attribute specified. In this case, it is possible to use learning algorithms to discover patterns in the data.

In between these two categories, there is also semi-supervised learning. The great motivation for this type of learning is the fact that unlabeled examples exist in abundance and labeled examples are generally scarce. The idea of semi-supervised learning is then to use the labeled examples to obtain information about the problem and use them to guide the process of learning from unlabeled examples.

We are mainly interested in supervised learning approaches, since the training data can be easily annotated. Indeed, the searched models are relations between variables which can be measured (for example, the cable displacement and the distal position of the tools).

Classification vs Regression tasks

Another observation related to supervised learning concerns the type of class attribute. If it is continuous, the learning problem is known as regression and if the class is discrete, the problem is known as classification.

Our problems fall under the category of regression tasks. The relations to be learned involve continuous input and output spaces. Non-parametric models: The complexity of the model evolves with the amount of data available for training. These algorithms do not rely on assumptions about the shape of the underlying data.

In this work, both approaches will be studied.

State-of-the-art on robot control using data-driven approaches

In order to deal with most of the issues commented earlier in chapter 2.6, data-driven approaches based on machine learning start to appear. These approaches can be used to learn various types of non-linearities in a really powerful and versatile way. The models can be learned just before use, avoiding the need to integrate sensors to the system. They can also be used together with the endoscopic camera in order to predict the behavior of the tool in between the captured frames.

A recent work was done by Xu et al. on the control of an endoscopic device based solely on a data-driven approach [Xu et al., 2017a]. On their paper, they present a comparison between different supervised learning algorithms to learn a direct mapping from the task space (defined as the 3D position of the tip of the endoscope) to the actuator space (cable displacement controlling the bending on two orthogonal planes and a prismatic joint allowing the translation of the instrument). A visualization of both spaces is provided on figure 3.1. Their approach shows promising results on learning the non-linearities of flexible systems without any prior knowledge of the kinematics or construction of the robot. There are two main issues with the way it is implemented:

• The amount of data required for learning is relatively elevated. This can be translated in learning sessions that are incompatible with clinical workflow, because these models are time varying and need to be retrained before each use.

• It is not possible to model the backlash introduced by the cable transmission. When a system is subjected to hysteresis, the mapping between its 3D position and the actuation is non-unique. Depending on the previous positions of the tool, multiple actuation can achieve the exact same configuration. By searching a direct mapping between these two spaces without adding other variables, it is only possible to learn an average model of the hysteresis.

In order to reduce the length of the training procedure, it is possible to incorporate à priori knowledge concerning the kinematics of the robot. This idea has a similar philosophy from the works of Nguyen-Tuong and Peters, in which the authors use previous knowledge from the robot dynamics and learn the non-linearities with data-driven methods [START_REF] Nguyen-Tuong | Using model knowledge for learning inverse dynamics[END_REF]. This results in a semi-parametric model that incorporates the advantages of using classic dynamic modeling techniques with the ability of learning the non-linearities from the machine learning techniques.

In this chapter, we will show how the combination of prior kinematic knowledge and machine learning has been done to improve the positioning accuracy of flexible surgical tools and flexible endoscopes. From our knowledge, no prior work has been done in this scope at the beginning of the thesis. Some parallel works have appeared and will be discussed later on.

Flexible surgical tools

Results of the presented work in this section have been published in [Porto et al., 2019d].

Our work focuses on the control of robotic flexible instruments, such as those described on section 2.4. The 3 degrees-of-freedom -bending of the tip, rotation of the instrument and translation in the channel -are motorized. Following the same notation presented at 2.4, the motor position for each joint will be denoted q bend , q rot and q trans for the bending, rotation and translation joint respectively.

The objective considered here is to control the cartesian position of the tip of the instruments in the frame of the channel F ch , which describes the task space of the instrument. The reference position for the instrument in this frame will be noted

Rationale of the method

Since the inverse position kinematic of bending instruments is difficult to model precisely, it could advantageously be learned from the system itself. However, instead of using a black box technique, we propose to incorporate the coarse knowledge of the working of the instruments. The rationale of our proposed method rely on the following observations:

• Some of the non-linearities of the transmission, such as the internal friction, create hysteresis effects which must be taken into account.

• Learning the complete relation between motor positions and cartesian positions of the tip of the instrument over the whole workspace becomes time prohibitive when hysteresis effects (i.e. dependency from the previous motions) are considered, even for instruments with only 3 DOF. Indeed, each point of the workspace should be reached from at least 8 CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS different directions (combination of positive and negative speeds of each joint).

• Despite complex behaviors presented previously, one can assume that the transmissions from the proximal motors to the distal configuration are only weakly coupled, as confirmed by experimental measurements (see figure 3.3). Then it is possible to partly decouple the IKM, and to use a similar 3-steps procedure as in the conventional IKM, despite the non-linearities. This avoids learning the proximal to distal relations on the whole workspace. In figure 3.3, the static characteristic between q bend and ρ is shown for different configurations of the instrument as well as the theoretic model assuming constant curvature. The same forward-and-backward motion has been performed on the bending axis under different values of translation and rotation to validate the hypothesis that the DOFs are uncoupled. As can be seen, the curves change only very slightly. Even in the worst case, for which both rotation and translation have been changed, the RMS error, taking the initial setting as the reference, is only 0.68 mm. This shows that, even though there could be some coupling between the DOFs, this effect can be neglected.

For the considered instruments we use θ, ρ, d and ∆d as distal parameters. As shown before in section 2.4, they can be numerically obtained from ch P * with minimal models. Moreover, they can be linked to motors positions in a decoupled manner by using the 3-steps IKM presented earlier.

The global method can be briefly described as follows (see also figure 3.4):

Before use:

• Learn the relations between motor positions and distal parameters by using an external sensor. Namely :

1. The inverse relation f -1 : θ → q rot from θ to q rot 2. The inverse relation g -1 : ρ → q bend from ρ to q bend 3. The direct relation h : q bend → ∆d from q bend to ∆d 4. The inverse relation k -1 : t → q trans from t to q trans .

Because of the hysteresis, these relations are generally not functions. Therefore, other variables must be taken into account in order to describe the hysteresis branches.

At use time:

• From a desired position ch P * express the desired orientation θ * , the desired radius ρ * and desired depth d * .

• Use the learned models to compute the joint positions :

1. q * rot = f -1 (θ * ) 2. q * bend = g -1 (ρ * ) 3. q trans = k -1 (d * + h(q * bend )
). • Apply the desired joint positions to the robotic system.

Note that no sensor has to be used during this stage.

Learning the inverse kinematics

In order to obtain the needed relations described previously, we propose to use an Extreme Learning Machine. Several other methods have been tested on our system before, but this particular method had a great trade-off between training time, generalization and amount of hyperparameters to tune.

Extreme Learning Machine (ELM)

ELM is a supervised learning method based on a Neural Network with a single hidden layer. Let us consider N observations of input-output pairs (ξ i , ξ o ) = (x[k], y[k])| k=1,...,N . A standard Neural Network with M hidden nodes and an activation function h(x) can be written as:

ŷ[k] = M i=1 γ i h(ω i • x[k] + b i ) (3.1)
where ŷ[k] is the estimated output for input observation

x[k], Γ = (γ 1 , γ 2 , • • • , γ M )
is the output weight vector, ω i are the input weights and b i are the hidden layer biases. See figure 3.5 to see a graphical representation of each term.

Train

f -1 (φ) Train g -1 (ρ) Train h(q bend ) Train k -1 (d)
Express the trajectory using the distal parameters

Compute q * rot = f -1 (θ * ) Compute q * bend = g -1 (ρ * ) Compute q * trans = k -1 (d * + h(q * bend ))
Apply joint trajectory to the motors

Before use

At use time

Generate training dataset Express data on channel frame

Express data using the distal parameters Generate desired trajectory Figure 3.4: Flowchart of the proposed approach [Porto et al., 2019d].

In a standard Neural Network, the aforementioned parameters are usually computed using backpropagation and gradient-descent methods. In ELM, the input weights ω i and hidden layer biases b i are randomly chosen using a uniform distribution [START_REF] Huang | Extreme learning machine: a new learning scheme of feedforward neural networks[END_REF]. The problem of minimizing the norm of the error vector Y -Ŷ can therefore be rewritten in matrix form as :

HΓ = Y (3.2)
where H is known as the hidden layer output matrix. This problem can be solved as a linear least square minimization:

Γ = H † Y (3.3)
where H † is the pseudo-inverse of H. This algorithm provides good generalization at fast learning speed provided an adequate amount of hidden nodes [START_REF] Huang | Extreme learning machine: a new learning scheme of feedforward neural networks[END_REF].

Learning the hysteresis effects with ELM

For our method, the key element of the training is to model the hysteresis effects for each of the relations. For this purpose, for direct relations (from actuator space to distal parameters space), the data for training are built as

ξ i [k] = (q j [k], d j [k]) and ξ o [k] = α[k] (3.4)
where q j is the considered actuator position, α is the considered distal parameter (θ, ρ, ∆d or d) and d j [k] is an additional input data giving the displacement direction (coded as +1/ -1) of the considered joint j = {bend, rot, trans}.

Similarly, for inverse relations (from distal parameters space to actuator space) the data for training are built as

ξ i [k] = (α[k], d j [k]) and ξ o [k] = q j [k].
(3.5)

During the training stage, d j can be estimated using the joint positions as d j [k] = sign(q j [k]q j [k -1]). When using the trained model to generate the actuator trajectory, the desired trajectory of the distal space parameter α * is CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS used instead:

d j [k] = sign(α * [k] -α * [k -1]
) because the actuator position is not yet known. Both forms are equivalent and yield the same information, however, during the training phase, it is preferable to rely on the more accurate actuator measurement rather than the measured task space variable to eliminate the effect of measurement noise.

Experimental setup

The experimental setup can be seen in figure 2.11. It is the same setup used to validate the theoretical inverse kinematic model.

Training the models

A total of 700 points is used to train all the models (see figure 3.6). The required amount of data was determined by analyzing the learning curves of each model once the number of hidden nodes M was defined. M was determined for each model, by analyzing the prediction error for different numbers of nodes given a large dataset (N = 2000 points per model). This procedure allows to properly choose M for avoiding overfitting, and allows to reduce the amount of points used for training. For learning, the data have been divided in two subsets: a training set containing 90% of the data, and a testing set containing the remaining 10%. Before training, the dataset was centered and normalized. The prediction performance was evaluated on the validation set using the Normalized Root Mean Square Error (NRMSE) between the predicted data and the ground-truth.

Inverse rotation model (function f -1 )

The desired rotation θ * is transformed to the joint space through the use of this model, which was obtained from:

ξ i [k] = (θ[k], d rot [k]) and ξ o [k] = q rot [k].
(3.6)

To generate the training set, only the rotation is moved from -45 o to 45 o with steps of 1.8 o for fixed large bending and translation. The trained model (M = 50 nodes, validation fitting indicator 0.9331) and the training data (N = 300 points) can be seen in figure 3.7. Note that even if this motion is not cable-driven, a large hysteresis is observed with non-linear behaviors at the change of direction, due to friction of the instrument shaft in the endoscope channel.

Inverse radius model (function g -1 )

This model is necessary in order to find the bending joint position for a desired radius ρ * and was obtained from:

ξ i [k] = (ρ[k], d bend [k]) and ξ o [k] = q bend [k] (3.7)
where d bend is the displacement direction of the bending motor.

The training set is generated by fixing the translation and rotation and performing 3 forward-and-backwards motions of the bending joint: q bend ∈ [-3; 3]mm with steps of 0.12 mm. The trained model (M = 50 nodes, validation fitting indicator of 0.9832) and the training data (N = 300 points) can be seen in figure 3.9. As expected, one observes a complex static curve with Figure 3.10: Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data.

Direct bending to ∆d model (function h)

The computation of the bending to ∆d is necessary to properly calculate the translation to be applied to the instrument. It is obtained with:

ξ i [k] = (q bend [k], d bend [k]) and ξ o [k] = ∆d[k]. (3.8)
The training set is the same as the one used for learning the inverse radius model. The trained model (M = 50, validation fitting indicator 0.9727) and the original training data can be seen in figure 3.11.

Inverse depth to translation model (function k -1 )

The inverse depth to translation model is used to reach the desired translation d * . The input-output pairs are:

ξ i [k] = (d[k], d trans [k]) and ξ o [k] = q trans [k].
(3.9)

The set of points used to train this model is composed by 5 forward-andbackwards motions with q trans ∈ [5; 20]mm for fixed bending and rotation (N = 100 points). M = 10 nodes were used and the obtained validation fitting indicator was 0.9744 (see figure 3.13).

Experimental results

For assessing the proposed approach, we tested the behavior of one instrument of the STRAS robot for 2D and 3D trajectories and compared with different control techniques.

CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS

-4 -3 -2 -1 0 1 2 3
Motor position q bend (mm) Given a desired position in the task space, the trained models are used as described in section 3.2.1 to compute the actuators positions references. The chosen trajectories are ellipses, which mimic the typical contours of lesions to be dissected during endoscopic submucosal dissections (ESD) procedures in the digestive tract [Zorn et al., 2017]. They are executed on a point by point manner for assessing the static accuracy.

FLEXIBLE SURGICAL TOOLS

2D trajectory in the XY-plane

First we carried out trajectories involving only two DOF, bending and translation (ellipse in the XY-plane with 44mm and 6mm axes) in order to more easily understand the behavior of the proposed technique and compare it with other methods. The statistics on the measured errors are reported in table 3.1

CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS

(first column).

For the proposed method (table 3.1, first column) the 2D RMS error is only 0.55 mm, which is really promising. One can also notice the very good repeatability of the motion as it is run several times. This result shows that our proposed method is able to compensate the non-linearities of the system and that it generalizes well outside of the trajectories used during training. To show the importance of the backlash compensation, we trained the models defined in section 3.2.4 without taking into consideration the displacement direction, i.e. without using motion direction as inputs. The results are shown on figures 3.17 and 3.18 on the spatial and time domain respectively.

It can be observed that the backlash effect is very significant on the xdirection, which is controlled by the bending joint. Moreover, for the depth, the compensation of the bending by the translation cannot be properly executed because it relies on the prediction of ∆d from the bending joint. This finally results in large errors on both directions. The statistics are in table 3.1 (second column).

We also compared our method with the state-of-the-art method proposed in [Xu et al., 2017a]. The training dataset was composed of 1920 points, distributed on a [32 × 30] grid in the actuator space [q bend × q trans ] covering the same ranges as in our method. Forward and backward motions for the bending were performed for fixed values of the translation. The obtained model presented a validation fitting indicator of 0.6794 for the bending joint and of

3D trajectory

In figure 3.21, the desired trajectory is an ellipse with a major axis of 18 mm, a minor axis of 17.2 mm, tilted 24 o around the X-axis, therefore requiring the use of 3DOF. Figure 3.20: 2D trajectory using the approach of [Xu et al., 2017a] shown in the time domain. RMS (mm) 0.5 0.2 0.6 6.5 2.4 6.9 7.4 2.7 7.9 Var. (mm 2 ) 0.2 0.1 0.1 39.9 4.2 8.4 49.1 6.7 11.6 Max (mm) 1.6 0.8 1.7 12.9 4.3 12.9 14.0 6.3 14.1 becomes much more complex and the models do not generalize as well as in the 2D experiments.

Discussion

Experiments in the 2D case show that our proposed approach outperforms the approach from [Xu et al., 2017a] for positioning accuracy, while using a smaller training dataset. The obtained accuracy would allow the correct execution of automatic dissection in the bending plane of the instrument.

The errors without backlash compensation are comparable with the ones obtained from the method in [Xu et al., 2017a]. They are also comparable to the ones assessed when using the conventional inverse kinematic model (see section 2.4.3), only a bit lower. This can be explained by the fact that learningbased methods learn the actual shape of the relation between bending and cartesian positions, which differs from the theoretical one as seen on figure 3.3. Nevertheless, the main source of errors lies in the hysteresis effect, which is not learned.

The errors obtained with the method of [Xu et al., 2017a] are larger than the ones reported in the original article, where the trajectories used for training and experiments were similar (circles) and all the movements where made in one direction. On the contrary, in our experiments, as in practical use, the motion used during training is very different from the one executed during the testing task.

The experiment in the 3D case exhibits larger errors than in the 2D case. Nevertheless, the RMS error and the maximum errors have been reduced respectively 4-fold and 3-fold with respect to the conventional IKM as reported in section 2.4.3. Moreover, the obtained accuracy is similar to the accuracy of measurement reported in [START_REF] Cabras | An adaptive and fully automatic method for estimating the 3d position of bendable instruments using endoscopic images[END_REF] using the same instrument and the feedback from the endoscopic camera. This shows the interest of our approach for open-loop position control of the considered instruments.

Summary

The main contributions of the proposed approach are :

• The combination of kinematic knowledge with learning to simplify the structure of the trained models with respect to a black box approach [Xu et al., 2017a].

• This creates a great reduction in the amount of data required. In total, 700 points were necessary to train all of the models, whereas 20000 points were used in [Xu et al., 2017a]. This reduction makes the approach more suitable for medical applications, especially for single-use instruments, since the training is significantly faster.

• It allows taking hysteresis effects into account at a very low cost. Even though it would be possible to add the displacement direction to the input space on the approach proposed in [Xu et al., 2017a], it would also significantly increase the complexity of the model and the amount of data needed to generalize well. Each points of the workspace should be reached from 8 different directions in the training set, corresponding to all the combinations of directions for each axis.

Flexible endoscope

Results of the presented work in this section have been published in [Porto et al., 2019a] and [Porto et al., 2019c].

CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS

In this chapter, we show an extension of the method presented on section 3.2. The previous approach was adapted for cable-driven surgical instruments having 3 uncoupled degrees-of-freedom. However, as we have seen on section 2.3, the cable-driven flexible endoscopes we are dealing with have 4 cables, which are endowed with 2 coupled degrees-of-freedom.

Following the same notation presented at 2.3, the joint positions will be denoted ∆L x and ∆L y .

The objective considered here is to control the cartesian position of the tip of the flexible endoscope in the base frame F 0 , which describes the task space of the endoscope. The reference position for the instrument in this frame will be noted 0 P * = (x * , y * ) T . Only two coordinates are considered because there are only two degrees-of-freedom that can be controlled.

Rationale of the method -adaptation to flexible endoscopes

For this extension, we keep the hypothesis that the distal part of the endoscope has a constant curvature during bending -therefore describing an arc of a circle -but we now consider the non-linearities of the motion transmission. This implies that equations (2.2), (2.20) and (2.16) are still valid, but (2.17) is not.

By using β x and β y as our distal parameters, a decoupled link can be found between the configuration space and the actuation space, as shown before in section 2.3. The interest of using machine learning is to replace equation ( 2.17) by a relation obtained from the observation of the actual behavior of the cable transmission. The global method can be briefly described as follows (see also figure 3.24):

Before use:

• Learn the relations between motor positions and distal parameters by using an external sensor. Namely :

1. The inverse relation p -1 :

β x → ∆L x from β x to ∆L x 2.
The inverse relation r -1 : β y → ∆L y from β y to ∆L y

In the same way as before, these relations are generally not functions.

The displacement direction will also be taken into account to accurately describe the hysteresis behavior.

At use time:

• From a desired position 0 P * express the desired bending angles β * x and β * y .

• Use the learned models to compute the joint positions :

1. ∆L * x = p -1 (β * x ) 2. ∆L * y = r -1 (β * y ).
• Apply the desired joint positions to the robotic system.

Learning the inverse kinematics

A different learning algorithm has been used for the flexible endoscopes. The functions r -1 and p -1 were trained using a Gaussian Process Regression (GPR) algorithm [START_REF] Rasmussen | Gaussian processes in machine learning[END_REF]. This algorithm falls into the classification of non-parametric, supervised algorithms.

Gaussian Process Regression (GPR)

Gaussian processes (GP) are defined as following [START_REF] Rasmussen | Gaussian processes in machine learning[END_REF] :

"A Gaussian Process is a collection of random variables, any finite number of which have (consistent) joint Gaussian distributions."

Before use

Generate training dataset

At use time

Express data on base frame

Express data using the distal parameters

Train p -1 (β x )

Train r -1 (β y )

Compute ΔL x = p -1 (β * x )

Generate desired trajectory

Express the trajectory using the distal parameters

Compute ΔL y = r -1 (β * y )
Apply joint trajectory to the motors A simple interpretation of the definition used is that a gaussian process is an extension of normal distributions. Instead of having a fixed mean value and fixed variance, a GP is specified by a mean function m(x) and a covariation function k(x, x ) where x is the input vector. A GP can then be written as :

y ∼ GP (m, k) (3.10)
which means that the function y is a Gaussian Process distributed with a mean function m and a covariance function k.

In order to clarify the distinction between the Gaussian Process and the Gaussian Distribution, we will be using µ to indicate the mean value of the distribution and Σ to indicate the covariance matrix. This means that, if we evaluate the GP in the finite set of input points x i , the output will be a finite set of Gaussian Distributions with the following attributes :

µ = m(x i ), i = 1, 2, • • • , n and (3.11) Σ = k(x i , x j ), i = j = 1, 2, • • • , n • . (3.12)
When working with Gaussian Process Regression, a prior GP is going to be updated by using training data in order to obtain a posterior GP. The proper-Figure 3.25: Illustration of a Gaussian Process. ties of the posterior GP are shaped in the light of the data used for training [START_REF] Rasmussen | Gaussian processes in machine learning[END_REF].

By using the posterior process, it is possible to make predictions about data with the same properties as those used for training. For that, we will consider a training set composed of n input-output pairs. The input vector will be noted X, with its individual values being x i , and the output vector will be noted Y . Let us also consider a set of input points X * in which we want to predict the outputs Y *

Y * |Y ∼ N (µ * + Σ * Σ -1 (Y -µ) , Σ * * -Σ * Σ -1 Σ * ). (3.14)
By inspection, one can determine the posterior GP is :

Y * ∼ GP (m P , km P ), m P (x) = m(x) + Σ(X, x) Σ -1 (Y -m(X)) k P (x, x ) = k(x, x ) -Σ(X, x) Σ -1 Σ(X, x).
(3.15) with m P (x) and m P (x) the mean and covariance functions of the posterior process and Σ(X, x) is a vector of covariances between every training case and the prediction inputs.

The final aspect to address is the training of the GP, meaning the determination of the hyperparameters present in the mean and covariance functions. For so, the method that presents an optimal trade-off between the fitting of the data and complexity of the model is optimizing the log marginal likelihood [START_REF] Rasmussen | Gaussian processes in machine learning[END_REF]. This quantity is calculated as :

L = logp(y|x, λ) = - 1 2 log|Σ| - 1 2 (Y -µ(X)) Σ -1 (Y -µ(X)) - n 2 log(2π) (3.16)
where p(y|x, λ) is the probability of the data given the hyperparameters (expressed as the vector λ).

Learning the hysteresis effects with GPR

The learning procedure is very similar to the one presented previously on chapter 3.2.3. The input-output pairs for inverse relations are built as

ξ i [k] = (β i [k], d i [k]) and ξ o [k] = ∆L i [k] (3.17)
where i is the index of the considered joint (i = {x, y}).

The mean and covariance functions, m(ξ i ) and K(ξ i ; ξ i ) respectively, used in the Gaussian Process Regression were chosen as :

m(ξ i ) = β i D 2 (3.18)
which corresponds to the geometric model provided by (2.26) and

K(ξ i , ξ i ) = σ 2 o exp - (ξ i -ξ i ) 2 2l 2 + σ 2 n δ qq (3.19)
where σ 2 o is the output variance, l is the length parameter of the kernel, σ 2 n is the noise variance and δ qq is the Kronecker's delta.

The main interest of using GPR is that we can adapt the theoretical geometric model to the experimental data. While using ELM, a non-linear behavior is learned based solely on the data used for training and its domain of validity is restricted to a neighborhood of the observations. With GPR, predictions can be provided for points far from the observations based on the actual geometric model. This algorithm has not been applied to the surgical flexible tools (only applied to flexible endoscopes) mainly because it is not possible to obtain the analytical inverse kinematic model (equation (2.6) is non-invertible) to be used as the mean function.

Experimental setup

The experimental setup can be seen in figure 2.9. It is the same setup used to validate the theoretical inverse kinematic model.

FLEXIBLE ENDOSCOPE 103

Training the models

A total of 500 points is used to train all the models, 250 for each joint (see figure 3.6). The required amount of data was determined by analyzing the learning curves of each model once the other hyperparameters have been tuned, similarly to what have been done with the surgical tools.

To generate the training set, we create a trajectory for ∆L x as a triangular wave, varying from -5mm to 5mm with 50 samples for slope (0.2mm step of cable displacement) with ∆L y set to zero. The same trajectory is then applied to ∆L y , with ∆L x set to zero. From the 500 points, 80% were used for the training of the models, while the remaining 20% were used for validation purposes. Before training, the dataset was centered and normalized.

Inverse bending on X model (function p -1 )

We can obtain the cable displacement ∆L *

x necessary to achieve the desired bending on X β *

x by using this model. The input-output pair is chosen as :

ξ i [k] = (β x [k], d x [k]) and ξ o [k] = ∆L x [k]. (3.20)
The trained model (validation fitting indicator 0.995) and the training data can be seen in figure 3.27. It is possible to note that the hysteresis is very small and the static characteristic is almost linear. Inverse bending on Y model (function r -1 ) Similar to the model p -1 , this model is required to convert the desired bending on Y β * y into the cable displacement ∆L y . It was obtained from:

ξ i [k] = (β y [k], d y [k]) and ξ o [k] = ∆L y [k]. (3.21)
The trained model (validation fitting indicator of 0.976) and the training data can be seen in figure 3.29. Contrary to the behavior seen on the other joint, there is a strong hysteresis effect with the presence of dead-zones. This showcases the interest of using data-driven approaches, since both joints are identical construction-wise and are present on the same tool.

Experimental results

Analogously to the surgical tools, 2D trajectories have been performed with the approach proposed on this chapter and the approach from [Xu et al., 2017a] to evaluate the accuracy of the positioning. The trained models for our approach are used as described on the flowchart of figure 3.24.

The trajectory resembles an inf symbol on the XY plane. The displacement on the X direction varies from -68mm to 92mm, while the displacement on Y varies from -58mm to 58mm. Table 3.3 presents a summary of the results obtained while performing the described trajectory.

The RMS error obtained by using our proposed approach was 4.4 mm, which is a great improvement compared with 18.3 mm attained by the IKM. As can be seen on Figs. 3.27 and 3.29, the hysteresis effect is not the same on

CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS

both axis -the Y axis has a much larger hysteresis width and the presence of dead-zones. Consequently, without any compensation, the amplitude of the movement is much more constrained in comparison with the compensated one. The executed trajectory can be seen in figure 3.31. The same trajectory has been executed after training a 2D model of the endoscope using the approach from [Xu et al., 2017a]. The model has been trained by applying circular trajectories in the joint space -which should result in circles in the task space if there were no influence from the nonlinearities -with the same center and different radius. A total of 4,000 points divided in 13 circles traversed 4 times compose the training dataset. The radius of these circles are linearly distributed from -9mm to 9mm. The training dataset has been created in order to closely resemble the one used in [Xu et al., 2017a].

The trajectory performed using the approach from [Xu et al., 2017a] can be seen in figure 3.32. The precision obtained with this model is poorer than the one attained with our proposed approach. The RMS error obtained in our system was 11.8 mm, almost 3 times higher than the proposed method. This difference in performance was expected, considering that the model from [Xu et al., 2017a], in the way it is defined, cannot learn the hysteretic behavior observed in our system. It should also be observed that this poorer accuracy has been achieved with a much larger training dataset (4,000 points compared to only 500). With a 87.5% reduction of the training dataset, a superior precision was achieved by taking into account some knowledge about the geometry of the system.

Discussion

The proposed approach is capable of largely compensating the non-linearities introduced by the cable transmission in flexible endoscopes, as shown by experiments. A gain in precision of over 4 times has been achieved in comparison with classic kinematic modeling by incorporating a simple learning procedure. A reduction of the duration of the training phase was also achieved in comparison with other state-of-the-art approaches.

This work can still be improved. Other inputs could be added to our models in order to better handle the change in direction regardless of the current configuration of the endoscope. Updating the models online during use could also be of interest, because the hysteresis shapes can be subject to changes associated to the shape of the passive body of the endoscope. The use of the endoscopic camera as a sensor for the training phase instead of external cameras is also a possibility we are investigating. Finally, using this method to handle the non-linearities in a teleoperated context, in addition to its use for automatic movements, is an application that we are currently aiming to achieve.

Conclusion

In this chapter, two novel ways of incorporating machine learning in the modeling of tendon-driven flexible systems have been presented. These methods rely on the mixing of classic kinematic modeling with learning-based algorithms to accentuate their advantages and decrease their inconveniences.

However, some limitations exist when using the proposed approach with the presented experimental setup. Considering that we are using a stereoscopic camera setup to estimate the 3D position of the end effector and that fiducial markers are used to increase the precision of the estimation and simplify the tracking task, these fiducial markers must be seen by the cameras during the complete training steps. This limits the rotation angles that can be learned, because the markers can become invisible and the 3D position cannot be estimated anymore. This limitation can be overcame by using 3D markers instead of the current ones at the cost of increased post-processing time.

For the flexible surgical tools, the bending is also limited by kinematic singularities. If the bending of the end effector goes beyond around 100 o , the value of the radius in the bending plane starts to decrease. This means that the function mapping the joint position and the radius becomes non-invertible. This way, the inverse static characteristic cannot be computed directly. A possible solution would be to define two functions for the radius that are valid under different ranges of motor positions.

Another limitation comes from the input space used to model the hysteresis branches. The hysteresis compensation only works in a small neighborhood of the point in which the direction changes during training. If the change in direction occurs far from the one used in the training data, an overcompensation of the dead-zone can appear. This results in an overshoot on the concerned coordinate. This could potentially be solved by adding other relevant parameters to the input space, however it still remains an open question. The drawback of doing so is that the model will require much more points to generalize well, slowing down the learning phase.

Nevertheless, these limitations do not challenge the interest of the proposed methods. The gain in terms of precision using such approaches is significant. A reduction on the RMS error of at least 4 times and up to 16 times (compared to the IKM) has been achieved. Furthermore, a decrease on the amount of training examples of at least 87.5% and up to 96.5% has been obtained when compared to other state-of-the-art approaches like [Xu et al., 2017a]. This is promising for potential clinical applications with single-use instruments, since their model cannot be learned until moments before a surgical procedure.

The way machine learning has been incorporated for both the flexible surgical tools and flexible endoscopes is very similar. Albeit some hypotheses are different, the principle used was the same -isolate the influences of each joint and learn their behavior separately to generalize over the whole workspace.

From our knowledge, this has never been done before in the context of robot control and identification.

In the next chapter, the similarities between both approaches will be analyzed. A general framework is then proposed to mix classic kinematic modeling with machine learning.

CHAPTER 4. MULTILAYERED APPROACH FOR KINEMATIC MODELING OF COMPLEX SYSTEMS

Introduction

In chapter 3, it was shown how to incorporate machine learning techniques together with classical geometric modeling in order to obtain hybrid, accurate and fast-training models. As seen, the application of these techniques for surgical tools and flexible endoscopes is quite similar, but with some crucial differences.

The first difference is in the control objective -for the tools, we were interested in a 3D position control, while for the endoscope only the 2D position was relevant. There are also differences in terms of hypothesis. We assume that the distal part of the endoscope bends with constant curvature, but this hypothesis is not necessary for the surgical tools. For this reason, it was required to describe the system configuration (equations (2.8) and (2.16)) to decouple the endoscope joints. Despite all these differences, there are several points in common. The parameterization of the task space, the analysis of the robot's workspace and the use of machine learning are very close points -even identical in some waysbetween these two systems. Then a question arises: is it possible to find a sufficient generic line of reasoning that encompasses both solutions presented, and possibly other completely different robotic systems?

The objective of this chapter is to explain the generalization of the techniques presented in the chapter 3, and show how it can be applied in other robotic systems even outside the scope of this thesis. We called this framework "Multilayered approach for kinematic modeling". Several examples of systems will be analyzed, and experimental results will be presented. Finally, an analysis will be made about the effects of using this approach in function of the number of layers used for each system.

Rationale of the method

The main idea of this method is based on the use of modeling layers. The use of few layers indicates an opaque modeling approach, while the use of several layers details the particularities of the studied system and configures a transparent approach. An illustration of the method is shown in figure 4.1.

Modeling layers

A total of 6 layers were identified in order to generalize the combination of machine learning techniques together with classic modeling. Not all of them are mandatory. They will be explained in the following.

Layer 1: Hypothesis formulation

In this layer, all the assumptions are formulated: the way the robot moves, what non-linearities it is subjected to, the conditions in which the modeling is * *Although presented as a single layer before the task space, the hypothesis layer may influence every other layer described in the diagram. valid. These hypotheses have an influence on every other following layer and also on the number of layers that need to be used in order to properly model the system. For example, in the modeling presented in chapter 3.3, we assume that the endoscope is subjected to hysteresis effects (meaning that a given 3D position can be obtained by multiple joint coordinates) and that the robot describes an arc of a circle with constant curvature during bending. These hypotheses have an important impact on the modeling, both on the searched model and on the errors that can be introduced by making wrong assumptions. It is not advisable to search a direct mapping between the 3D position and the joint positions since it has not a unique solution and can results in requiring a large amount of data to accurately learn the behavior of the system. On the other hand, the actual shape of the endoscope might differ from the assumed constant curvature model, which introduces modeling errors.

This layer is mandatory and is probably the most important one given the impact it has on the precision and complexity of the models. It is not constrained to any particular space because it can influence every step of the modeling.

Layer 2: Task definition

This layer is located in the task space and is directly related to the task itself. It defines exactly the task to be performed by the robot. Is it a 2D or 3D positioning task? Should the orientation of the robot be taken into account? It is a mandatory layer that is usually easily determined.

As an example, the surgical tools task required a 3D positioning, whereas the flexible endoscope was defined by a 2D positioning. In both cases, the orientation of the robot was not taken into account. This may differ for different applications and is not an intrinsic characteristic of the system, but of the task.

Layer 3: Parameterized space

This is an optional layer also located in the task space. On this layer, a parameterization is sought to better describe the task taking into account the architecture of the robot. The choice can be related to the robot's workspace and the positioning of its actuators.

For example, the surgical tools have a cylindrical workspace. This makes the use of a cylindrical coordinate system a great candidate to describe the task. What makes this layer optional is the fact that not every architecture has an optimal parameterization and sometimes the cartesian description of the task is the optimal one. If we take a X-Y table as an example, there is no reason to search for a different parameterization other than the cartesian one to describe the task.

Layer 4: Kinematic analysis

This is an optional layer located between the configuration space and the task space. In this layer, we are interested in determining the existence and directions of impact of the joints on the task space without taking into account the actual geometry of the robot.

Let us use the surgical tool as an example to illustrate how the kinematic analysis layer works. As seen in chapter 3.2, each joint impacts the distal movement in a unique way. The bending joint, noted q bend , only impacts the distance to the origin ρ and the depth d. The rotation joint q rot only impacts the angle of the bending plane θ and the translation joint q trans only impacts the depth. Then, the kinematic analysis layer will be the set of relations between the task space and the joint space :

1. The inverse relation f -1 : θ → q rot from θ to q rot ; 2. The inverse relation g -1 : ρ → q bend from ρ to q bend ; 3. The direct relation h : q bend → ∆d from q bend to ∆d; 4. The inverse relation k -1 : y → q trans from d to q trans ; as has been seen in chapter 3.2.

In this layer, we search for the relations between the task space and the configuration space (or actuation space depending on the model) without expressing them mathematically. To illustrate this, let us consider an alternative surgical tool with a rigid pivot joint instead of a flexible bending joint (see figure 4.2).

In this alternative architecture, even though the way the instrument moves is different when the bending joint is actuated, we can still describe the exact same impacts as for the flexible tool. This means that, even though these tools are different and have different geometrical models, their kinematic analysis is identical. This layer is considered optional because on opaque models (such as black box), it is possible to completely skip it and link the task space straight to the actuation space.

Layer 5: Kinematic modeling

This optional layer is used when the geometry of the robot is taken into account. Instead of only putting in evidence the relations between the task space and the configuration space, the equations linking both spaces are here considered.

In the case of the approach presented for the surgical tool in chapter 3.2, this layer is completely skipped. The way the robot bends, whether it is by the use of a flexible body or a rigid pivot joint does not have an influence in The effects on the task space can be expressed with the same parameters, however the shape of the robot is different.

the learning phase. However, for the approach used for the flexible endoscope (chapter 3.3), this layer is essential for decoupling the overall bending angle into two orthogonal bending angles. By also taking the example shown in figure 4.2, if we wished to increase the amount of modeling layers and take into account the geometry of the robot, this layer would be different given the different nature of the joints. This is also an optional layer because it is possible to not take into account the geometry given the hypothesis made and the overall structure of the robot. However, this layer is linked with the kinematic analysis layer. If the kinematic modeling layer is chosen to be used, the kinematic analysis also is used, since the modeling corresponds to the mathematical or numerical expression of the relations contained in the kinematic analysis.

Layer 6: Machine learning

This last layer is located between the actuation space and a higher level space. This one can be the task space or the configuration space, depending on the number of layers used. It represents the mapping, obtained using machine learning algorithms, between the actuators and the variables of interest. This layer is mandatory for incorporating machine learning with kinematic modeling. If it were removed, the combination of the other layers would be equivalent to a conventional kinematic modeling (either analytic or numeric). The amount of layers between the task definition and this layer determines how transparent the modeling is and also how complex the final model is to use.

Black box techniques, also referred as opaque techniques, linking the task space straight to the machine learning layer, is very simple to implement. The hypothesis will shape the machine learning algorithm (which features should be taken into account for obtaining the model) and the task will determine which variables are being modeled. Only a single model needs to be trained, but it might be over-complex.

On the other hand, transparent approaches or glass box [START_REF] Holzinger | A glass-box interactive machine learning approach for solving np-hard problems with the human-in-the-loop[END_REF] techniques, linking the joints (present at the kinematic modeling layer) to the actuator space, might require the learning of several models as shown in chapter 3. The main advantage of doing so is that these models are usually simple, do not require much data to generalize well and are independent from one another. Models in the middle ground will be referred as translucent. These models are not direct links from the task space to the actuation space, but they are not much detailed. Usually the actual geometry of the robot is not taken into account, but an analysis on the impacts of each joint on the pose of the end effector is required.

Summary

A general overview of all the layers is presented below :

1. Hypothesis: This layer determines the overall validity of the model and the assumptions made; 2. Task definition: This layer is the one in which the task to be performed by the robot is determined; 3. Parameterized space: Depending on the architecture of the robot, it may be possible to find a parameterization better suited for specifying the task. However, this parameterization may not be simply specified or the cartesian parameterization may already be optimal one; 4. Kinematic analysis: The impacts between the configuration and task space are put in evidence without taking into account the actual geometry of the robot; 5. Kinematic modeling: The geometric relationship between the configuration and task space is determined; 6. Machine learning: This step is used to determine the mapping between the concerned space -whether it is the task or the configuration spacewith the actuation space.

To illustrate the layers presented beforehand and how these layers can be used in a practical way, we have applied the multilayered approach on several systems. Experimental results are mentioned for the systems available at our OF COMPLEX SYSTEMS facilities, but other systems are also discussed to illustrate the applicability of the technique. Before these examples, it is important to showcase how the relations that need to be learned are determined.

Determining the functions to be learned

A question may arise when defining the machine learning layer -which functions need to be learned and what are their dependency with respect to the actuation space ?

A useful tool that allows for determining these relations is the Jacobian matrix between the space being analyzed (task space or configuration space) and the actuation space. By definition, the Jacobian matrix expresses the sensitivity of each coordinate of the concerned space with respect to each actuator present in the robot as :

J = ∂P ∂q =                             ∂P 1 ∂q 1 ∂P 1 ∂q 2 . . . ∂P 1 ∂q n ∂P 2 ∂q 1 ∂P 2 ∂q 2 . . . ∂P 2 ∂q n . . . . . . . . . . . . ∂P m ∂q 1 ∂P m ∂q 2 . . . ∂P m ∂q n                             (4.1)
with P denoting the coordinates of the concerned space of dimension m and q denoting the actuation space of dimension n. Each non-null term represents how an actuator influences the displacement of the robot in that particular direction.

Let us take an example of a 3D positioning task of a robot possessing 3 actuators that are completely independent from one another. An example of such a robot could be a XYZ table. By analyzing its Jacobian matrix, one would obtain :

J =                      ∂x ∂q 1 ∂x ∂q 2 ∂x ∂q 3 ∂y ∂q 1 ∂y ∂q 2 ∂y ∂q 3 ∂z ∂q 1 ∂z ∂q 2 ∂z ∂q 3                      =                      ∂x ∂q 1 0 0 0 ∂y ∂q 2 0 0 0 ∂z ∂q 3                      . (4.2)
which implies the existence of 3 individual relations that can link the task space P to the actuator space q, each represented by a line of the Jacobian matrix. These relations would be :

1. x = f (q 1 ); 2. y = g(q 2 ); 3. z = h(q 3 ).

APPLICATION TO FLEXIBLE ENDOSCOPES WITH TWO BENDING PLANES 119

Other systems may present coupled joints -these will appear as Jacobian matrices that are not diagonal matrices. Let us take an example of system that requires a 3D positioning and roll orientation task with 4 actuators. After performing the kinematic analysis, the first two actuators have an influence on both x and y coordinates, while the third and fourth have independent actions respectively on the z coordinate and on the roll of the end effector. The Jacobian matrix for this system is :

J =                                    ∂x
                                   =                                    ∂x ∂q 1 ∂x ∂q 2 0 0 ∂y ∂q 1 ∂y ∂q 2 0 0 0 0 ∂z ∂q 3 0 0 0 0 ∂r ∂q 4                                    . (4.3)
The terms surrounded by the light blue box indicate that these two task coordinates are coupled and affected by the same actuators. Instead of learning two independent functions, one for x and another for y, this group indicates the presence of a function with two inputs and two outputs. The functions linking these spaces are :

1. (x, y) = f (q 1 , q 2 ); 2. z = g(q 3 ); 3. r = h(q 4 ).

In order to increase the efficiency of the training phase -equivalent to reducing the amount of data required for proper generalization -it is interesting to parameterize the task space or the configuration space to obtain a Jacobian matrix as close as possible to a diagonal form. It is also possible to achieve this diagonalization of the Jacobian matrix by the use of the hypothesis layer. An example will be shown later on with the flexible endoscope on how to do so.

Application to flexible endoscopes with two bending planes

In this section, we will rewrite the approach used by [Xu et al., 2017a], the approach presented at the section 3.3 and a new modeling approach using the formalism described previously.

Approach I -Direct link between the task space and actuator space

As previously discussed, this approach has been suggested in [Xu et al., 2017a]. An adaptation will be made in order to present it with the multilayered formalism.

Hypothesis:

The way machine learning was applied in [Xu et al., 2017a] assumes that each point of the task space can be reached by a unique actuation.

Task definition:

Instead of 3D positioning task of the endoscope, we will only perform a 2D positioning. For our particular system, those are the only degrees-of-freedom heavily affected by non-linearities. The pose of the end effector is then :

P = (x, y).

Parameterized space:

There is no need to parameterize the task space in this particular approach. This layer can be completely skipped.

Kinematic analysis:

No analysis is required because a direct link to the machine learning layer is searched. The Jacobian between the task space and the actuator space is :

J =              ∂x ∂∆L x ∂x ∂∆L y ∂y ∂∆L x ∂y ∂∆L y              . (4.4)
By looking at the Jacobian matrix, two functions could be defined for this particular approach :

1. x = f (∆L x , ∆L y ); 2. y = g(∆L x , ∆L y ).
However, considering that the influences on both functions are identical, a sole relation can be used. In this case : (x, y) = f (q bend , q trans ).

Kinematic modeling:

The geometry of the instrument is not important for this approach. This layer can be completely skipped.

APPLICATION TO FLEXIBLE ENDOSCOPES WITH TWO BENDING PLANES 121

Machine learning:

Only one function needs to be learned, being :

• The inverse relation f -1 : (x, y) → (q bend , q trans ) from (x, y) to (q bend , q trans ).

The results of using said model have been shown in 3.3.5.

Discussion

As shown before, this model does not generalize well mainly because of the hypothesis made. Instead of modeling the hysteresis branches, this approach ends with a "mean" model that is not accurate and requires a relatively high amount of data to train.

The accuracy can be improved by a change in the hypothesis. Taking into account the hysteretic loop can be done in a straightforward way, but it will wield in more data being required to have good generalization. Even though it is the simplest one to implement, the trade-off between its accuracy and efficiency is unbalanced.

Layer

Content of the modeling layer 1 -Hypothesis Each cartesian point is achieved by a single joint coordinate;

2 -Cartesian space P = (x, y)

-Machine learning

The inverse relation f -1 : (x, y) → (∆L x , ∆L y ) from (x, y) to (∆L x , ∆L y ); Table 4.1: Summary of modeling layers used for approach I on the flexible endoscope.

Approach II -Using the hypothesis of constant curvature while bending

This is the same approach presented in chapter 3.3.

Hypothesis:

The main assumptions regarding this particular method if that the flexible controllable part of the endoscope describes an arc of constant curvature during bending.

It is also assumed that the joints are not coupled and independent from one another.

CHAPTER 4. MULTILAYERED APPROACH FOR KINEMATIC MODELING OF COMPLEX SYSTEMS

Task definition:

Similarly to the previous approach, the goal is to perform a 2D positioning task. The pose of the end effector is then :

P = (x, y).

Parameterized space:

A suitable parameterization for this system, assuming constant curvature, is the polar parameterization. Its use greatly simplifies the mapping between the task space and the configuration space. The task space is then defined as :

P cyl = (ρ, θ) = ( x 2 + y 2 , arctan2(y, x)).

Kinematic analysis:

Both joints have an impact on the radius of the tool ρ and on the rotation of the tool θ. This means that the Jacobian matrix from the task space to the actuation space is:

J =              ∂ρ ∂∆L x ∂ρ ∂∆L y ∂θ ∂∆L x ∂θ ∂∆L y              (4.5)
implying that the searched relation is of the form :

(ρ, θ) = f (∆L x , ∆L y ).

(4.6)

In order to be able to define independent relations for each actuation, it is necessary to go further into the next layer.

No analysis is required because a direct link to the machine learning layer is searched. The Jacobian between the task space and the actuator space is :

Kinematic modeling:

To describe the configuration of the tool, we will be using equations (2.6), (2.8), (2.17) and (2.20). These equations allow to describe the task in terms of the orthogonal bending angles of the endoscope. Each orthogonal bending angle is only affected by its corresponding pair of cables. This means that the Jacobian from the configuration space to the actuation space is :

J =               ∂β x ∂∆L x ∂β x ∂∆L y ∂β y ∂∆L x ∂β y ∂∆L y               =              ∂β x ∂∆L x 0 0 ∂β y ∂∆L y              (4.7)
which implies the existence of two relations linking these two spaces :

4.4. APPLICATION TO FLEXIBLE ENDOSCOPES WITH TWO BENDING PLANES 123

• β x = h(∆L x );

• β y = k(∆L y ).

Machine learning:

The searched functions are :

• The inverse relation f -1 : β x -→ q x from β x to q x ; • The inverse relation g -1 : β y -→ q y from β y to q y .

The results of using said model have been shown in 3.3.5.

Layer

Content of the modeling layer 1 -Hypothesis Constant curvature during bending; No coupling between the joints; Cable transmission subjected to non-linearities.

2 -Cartesian space P = (x, y)

3 -Parameterization ρ = x 2 + y 2 θ = atan2(y, x)
4 -Kinematic analysis ρ = f (∆L x , ∆L y ); θ = g(∆L x , ∆L y ).

-Kinematic modeling

Equations (2.6), (2.17) and (2.20); β x = h(∆L x ); β y = k(∆L y ).

-Machine learning

The inverse relation h -1 : β x → ∆L x from β x to ∆L x ;

The inverse relation k -1 : β y → ∆L y from β y to ∆L y .

Table 4.2: Summary of modeling layers used for approach II on the flexible endoscope.

Approach III -Using the hypothesis of small bending angles

Machine learning can be incorporated a different way in the global modeling of the same system when its motions are limited by constrained spaces . This is actually the case on many applications in lower-endoscopy when using colonoscopes with a large diameter, since the tissues of the colon limit the movement amplitude during manipulation.

Hypothesis:

For this analysis, we will focus only on applications within constrained spaced, which means that the overall bending angle of the endoscope will remain small. This means that the displacement of the tool from its upright position will also be small. In other words:

β 1; β x 1; β y 1.
Akin to the previous approach, we assume is that the joints are not coupled and are completely independent from one another.

Finally, the cable transmission is subjected to several non-linearities. These effects create a hysteresis shape that shall be taken into account during modeling.

Task definition:

The pose of the endoscope will be defined only by its x and y coordinates, as was the case from the previous approaches. This means: P = (x, y).

Parameterized space:

For this approach, we will skip the parameterization of the cartesian space. The reason will become clear when the other layers will be analyzed.

Kinematic analysis:

When only small bending angles are considered, some approximations can be done in order to simplify the modeling of the endoscope. Let us, for now, assume that the endoscope bends with a constant curvature. This means that the distance from the origin ρ will follow the equation (2.6) :

ρ = L f β (1 -cos β) + L d sin β
and the angle of the bending plane φ will follow equation (2.20) :

φ = arctan2(β y , β x ).
with the bending angle β being a combination of its orthogonal components β x and β y (see equation (2.16). The other parameters are detailed in image 3.23.

This means that the cartesian position of the tip of the endoscope will be :

x = ρ cos φ and y = ρ sin φ. By replacing equations (2.6) and (4.9) in (4.8), one obtains the relations between the cartesian positions x and y and the orthogonal components of the bending angle β x and β y : Now, let us do a sensitivity analysis on the cartesian position based on the changes of the bending angles β x and β y . We are mainly interested in the effect of β y on the displacement along x and the effect of β x on the displacement along y. These partial derivatives are:

x = L d sin β - L f (cos β -1)
∂x ∂β y = ∂y ∂β y = β x β y β 4 β 2 d cos β + β sin β(L -d) + 2L(cos β -1) . (4.12)
Let us analyze equation (4.12). Figure 4.3 shows the function evaluations on a 50x50 grid with the values of β x and β y linearly spaced in the interval [-0.6, 0.6] rad. As can be seen, in the neighborhood of the point (β x , β y ) = (0, 0), the sensitivity tends to 0. This means is that the influence of β x on the displacement along the y direction and the influence of β y on the displacement along the x direction are negligible. By moving the endoscope in this neighborhood -which implies only applying small bending angles to the system -we can consider that the system is decoupled.

Figure 4.4 also illustrates the same phenomenon obtained from simulations assuming constant curvature. By applying the configuration path shown on the right, if the tool bends with a constant curvature, the tip of the endoscope will perform the trajectories shown on the left (solid line). The trajectories shown on the left with a dashed line would be performed by the endoscope in the case of no coupling. As can be seen, the difference between the curves is very small near the straight configuration, with a larger difference when the endoscope goes away from this central configuration.

Although this analysis has been done by assuming a constant curvature during bending, the same behavior has been observed on our system -which does not bend in such a way as has been discussed in chapter 2.3. From this point on, we will consider that the coupling is negligible.

In this case, one can learn the inverse relations that link the actuator space to the Cartesian space without doing any parameterization nor using kinematic modeling. This can be seen by the Jacobian between the Cartesian space and the actuator space : No further kinematic modeling is required. As previously shown, the knowledge about the shape of the endoscope during bending is not necessary to decouple each main displacement direction.

J =              ∂x ∂∆L x ∂x ∂∆L y ∂y ∂∆L x ∂y ∂∆L y              ≈             ∂x ∂∆L x 0 0 ∂y ∂∆L y            

Machine learning:

For finding the inverse kinematic model, two functions need to be learned. The searched functions are :

• The inverse relation f -1 : x -→ ∆L x from x to ∆L x ;

• The inverse relation g -1 : y -→ ∆L y from y to ∆L y .

Considering the hysteresis effects, we will take into account the displacement direction to be able to model both branches of the hysteresis. This means the input-output pair for the x direction is:

ξ i [k] = (x[k], d x [k]) and ξ o [k] = ∆L x [k] (4.16)
and for the y direction is:

ξ i [k] = (y[k], d y [k]) and ξ o [k] = ∆L y [k].
(4.17)

The data set used for training and validation was the same used on all other approaches. The learned models can be seen on figures 4.5 and 4.6.

The models were obtained by using the Extreme Learning Machine algorithm, since the relations between the X and Y positions of the endoscope and their respective cable displacement are non-invertible.

In order to generate the joint trajectory, the following steps should be followed :

• Generate the desired trajectory in the task space 0 P * = (x * , y * );

• Use the learned models to compute the joint positions :

1. ∆L * x = f -1 (x * ); 2. ∆L * y = g -1 (y * ).
By comparing the steps from this approach with the one illustrated at figure 3.24, it is possible to see reduction in the amount of steps for both the training phase and the use phase. However, the effort required in the kinematic analysis layer in order to simplify the modeling the models is much greater.

Results

The proposed approach has been assessed by performing trajectories defined on the XY-plane. The displacement on the X-direction was from -68mm to 72mm, while on the y-direction it was from -40mm to 50mm. The trajectory has a complex shape with several changes in direction both on the X and Y direction in order to evaluate the hysteresis compensation. A total of 268 points compose this trajectory. For this trajectory, the RMS error was 3.59mm, with a variance of 3.25mm 2 . These errors are larger than the ones obtained with the approach presented at section 4.4.2, but are still a huge improvement compared to both the Inverse Kinematic Model and the approach from [Xu et al., 2017a]. On the other hand these models are very simple to implement and train, since there is no need to estimate the bending angle during training nor during use. It is also worth noting that these results could be improved by constraining the amplitude of the movement of the endoscope -this way, the coupling between the joints is further reduced and the modeling errors can be further attenuated.

Layer

Content of the modeling layer 1 -Hypothesis Small bending angles; No coupling between the joints; Cable transmission subjected to non-linearities.

2 -Cartesian space P = (x, y)

4 -Kinematic analysis x = f (∆L x ); y = g(∆L y ).

-Machine learning

The inverse relation f -1 : x → ∆L x from x to ∆L x ;

The inverse relation g -1 : y → ∆L y from y to ∆L y .

Table 4.3: Summary of modeling layers used for approach III on the flexible endoscope.

Application to flexible surgical tools

The approach presented at the section 3.2 and an adaptation of the approach used by [Xu et al., 2017a] will be presented in this section.

Approach I -No hypothesis on curvature

This is carefully explained on chapter 3.2. By using the extension framework presented on this chapter, it is possible to further show the similarities and divergences with the approach applied to flexible endoscopes (see section 3.3).

Hypothesis:

The main assumption is that there is no coupling between the translation, rotation and bending joint on the instrument. This is important in order to perform a partial decoupling of the configuration space that will be done later on.

Similarly to the other modeling approaches, non-linearities are introduced by the antagonistic cable transmission. A hysteretic behavior is expected between the joint space and the configuration space.

Task definition:

We are interested in doing a 3D positioning of the tool with respect to the channel frame. This means that the pose will be defined as : P = (x, y, z).

Parameterized space:

The cylindrical parameterization will be used in order to partly decouple the task space. Considering the workspace of this tool is a shaped as a cylinder, this parameterization allows to better visualize the influence of each joint on the task space. The task space is then defined as :

P cyl = (ρ, θ, d) = ( x 2 + y 2 , arctan2(y, x), z).

Kinematic analysis:

In order to analyze the effects of each joint on the task space, let us take a look at figure 3.2. This figure best illustrates the movement of the tool with respect to the channel frame.

By actuating the bending joint, two effects can be seen -a change in the distance from the origin ρ and a change on the depth of the tool d. The rotation and translation joints only have one effect each -on the angle θ and on the depth d respectively. By taking these effects into account, the Jacobian between the task space and the actuation space is :

J =                     
                     =                      ∂ρ ∂q bend 0 0 0 ∂θ ∂q rot 0 ∂d ∂q bend 0 ∂d ∂q trans                      (4.18)
which would normally imply that 3 functions are required to reconstruct the 3D position of the tool, them being :

1. θ = f (q rot ); 2. ρ = g(q bend ); 3. d = k(q rot , q trans ).

Although correct in its format, the relation k may require far more resources than the other relations given the dimension of its input space. To solve this issue, a slightly different parameterization of the task space is then proposed.

The distance to the origin ρ and the rotation angle θ are kept unchanged. The depth, however, will be described as a combination of two different variables -the translation of the tool t and the difference in depth during bending ∆d, as shown in figure 3.2 and specified in section 2.4. The translation can only be affected by the translation tool, while the difference in depth is only modified during bending. This means that the Jacobian between the modified task space and the actuation space is :

J =                             
                             =                              ∂ρ ∂q bend 0 0 0 ∂θ ∂q rot 0 0 0 ∂t ∂q trans ∂∆d ∂q bend 0 0                              (4.19)
which implies the learning of 4 independent functions :

1. θ = f (q rot ); 2. ρ = g(q bend ); 3. ∆d = h(q bend ); 4. t = k(q trans ).

Kinematic modeling:

The knowledge of the actual shape of the surgical tool is not necessary for this approach. This layer can then be skipped. OF COMPLEX SYSTEMS

Machine learning:

The 4 functions specified on 3.2.1 need to be learned. The searched functions are :

• The inverse relation f -1 : θ → q rot from θ to q rot ; • The inverse relation g -1 : ρ → q bend from ρ to q bend ; • The direct relation h : q bend → ∆d from q bend to ∆d;

• The inverse relation k -1 : y → q trans from d to q trans .

The reason why the relation h is direct rather than inverse (in contrast to the other three relations) comes from the objective of the model itself. Considering the bending joint is the only one capable of changing the distance to the origin ρ, the change in depth during bending can be seen as a noise that needs compensation. This compensation can only be performed by the translation joint because it is the only other one capable of changing the depth of the tool.

The details on the training of these models and the results of their use are specified in section 3.2.

Discussion

This modeling approach is a great example of a translucent box approach. Even though some analysis had to be done in order to comprehend the effects of each joint on the task space, it is not necessary to go further into the geometric modeling. This approach presents a good trade-off between the hypotheses in which it is subjected and the simplicity of the models that need to be learned.

Approach II -Direct link between the task space and actuator space

A much more opaque approach, leaning towards a black box approach, can be applied to the surgical tools by adapting [Xu et al., 2017a]. Similarly to what has been shown in section 4.4.1, a direct link will be done to the machine learning layer without any consideration about the shape or the way the tool moves.

Hypothesis:

The only hypothesis required to apply the method from [Xu et al., 2017a] is that each point inside its workspace can be reached by a single actuation.

Task definition:

As an adaptation, we just applied this method to perform a 2D positioning task using the bending and the translation joint. The rotation joint will not be Table 4.4: Summary of modeling layers used for approach I on the surgical tools.

taken into account in this modeling. The pose of the end effector is then :

P = (x, y).

Parameterized space:

There is no need to parameterize the task space in this particular approach. This layer can be completely skipped.

Kinematic analysis:

Since a direct mapping from the task space and actuator space is sought, no further analysis is required. The Jacobian between the task space and the actuator space is :

J =             ∂x ∂q bend ∂x ∂q trans ∂y ∂q bend ∂y ∂q trans             . (4.20)
Similarly to the approach presented at section 4.4.1, two functions could be defined :

1. x = f (q bend , q trans ); CHAPTER 4. MULTILAYERED APPROACH FOR KINEMATIC MODELING OF COMPLEX SYSTEMS 2. y = g(q bend , q trans ).

These functions can be assembled in a single one since the influences are the same. Then : (x, y) = f (q bend , q trans ).

Kinematic modeling:

The geometry of the instrument is not important for this approach. This layer can be completely skipped.

Machine learning:

Only one function needs to be learned, being :

• The inverse relation f -1 : (x, y) → (q bend , q trans ) from (x, y) to (q bend , q trans ).

The results of using said model have been shown in 3.2.

Layer

Content of the modeling layer 1 -Hypothesis Each cartesian position can be reached by a single actuation coordinate;

2 -Cartesian space P = (x, y)

-Machine learning

The inverse relation f -1 : (x, y) → (q bend , q trans ) from (x, y) to (q bend , q trans ) Table 4.5: Summary of modeling layers used for approach II on the surgical tools.

Conclusion

In this chapter, a framework for mixing machine learning with kinematic modeling has been presented. This framework allows the incorporation of learning-based algorithms in different stages of modeling to obtain hybrid models. The complexity of the model, the amount of data required for training as well as the precision is highly dependent on the transparency of the modeling approach.

This framework has been applied on the systems available in the context of this thesis -flexible endoscopes and flexible surgical tools -with different choices of layers to illustrate their effect on the final model. Even though it has only been shown for these two systems that are relatively similar (tendondriven serial robots with continuous joints), it is not limited to this category of system. This framework can be easily applied to serial robots with discrete joints as well as parallel robots. For example, the SPIRITS robot [START_REF] Pfeil | Observations and experiments for the definition of a new robotic device dedicated to ct, cbct and mri-guided percutaneous procedures[END_REF] developed in ICube for needle insertions in interventional radiology would be a good candidate for trying this approach, considering the non-linearities introduced by its actuation.

From our experiments, some effects have been observed depending on the amount of layers chosen for the modeling. Opaque approaches, notably the one from sections 4.4.1 and 4.5.2, are prone to requiring a lot of data for training. This is expected, since no effort to decouple the effects of the joints on the task space is made. In order to generalize well, the training data must cover the whole workspace of the robot, multiple times depending on the inputs. For example, if the displacement speed must be taken into account, every point of the workspace should be reached with a different speed to properly train the models. The amount of data required increases exponentially with the amount of DoFs and additional features used for learning. It may also fall short in terms of performance if the hypotheses are not well chosen. The main advantage of this kind of approach is that it may take into account unspecified non-linearities.

Translucent approaches show the best trade-off between precision of the model and training effort. They usually require more assumptions than completely opaque approaches, but less than completely transparent ones. The models are much simpler than direct links from the task space to the actuation space and yet may take into account unspecified physical phenomena. It is the case of approaches of sections 4.4.3 and 4.5.1. Unfortunately, some systems may not be suitable for this kind of approach when there is strong coupling between the effects of joints on the parameters of the task space. For strongly coupled systems, to maintain efficiency on the training phase, it is generally required to go one step further into transparent approaches.

The approach of section 4.4.2 falls under the clear box category. All the modeling layers are used in order to decouple the influence of the actuation space into the configuration space. This approach is the closest to classic kinematic modeling, with the machine learning layer being introduced as a last step. The models to be learned are normally the simplest ones, but are subject to several assumptions. The main risk of using transparent approaches is the introduction of modeling errors that cannot be compensated by the learning algorithms. However, it can always be applied to any system in which the classic kinematic model is known -the mapping between the actuators and the joints is done with data-driven techniques, represented by the machinelearning layer.

Chapter 5

Conclusion

Flexible instruments and endoscopes are key tools for the surgery of the coming years, both in digestive surgery and in endovascular and cardiac surgery. Both domains share the same need for long slender instruments with the capability to navigate in a tortuous environment while limiting interactions. Both of them also have the need for dexterity for performing fine and precise surgical motions. In endoluminal surgery, the feedback provided to the surgeon is generally an endoscopic camera mounted at the distal tip of the endoscope. The large diameter of the lumen allows using several such instruments, with the general goal to realize bimanual operations.

Distal degrees of freedom are usually provided by cable-driven bending sections, with one or two bending planes, i.e. one or two degrees of freedom. These distal actions are combined with global displacements, namely translation and rotation of the shaft of the instrument to provide multiple degrees of freedom. However, the combination of flexibility and dexterity comes at the price of complex motion transmission from the proximal side outside of the patient to the distal side at the operation site. Cables are subject to friction, loss of tension, change of shape and path. This, combined with imperfections on the distal bending structure made of individual vertebrae or from a compliant skeleton, make the behavior of these instruments very complex. This is especially critical for robotized instruments because telemanipulation using intuitive interfaces make the impact of these non-linearities even more prevalent [START_REF] Allemann | Joystick interfaces are not suitable for robotized endoscope applied to notes[END_REF][START_REF] De Donno | Master/slave control of flexible instruments for minimally invasive surgery[END_REF]. Moreover, robotic instruments can be used to perform automatic motions and augment users capabilities. For such tasks, either an automatic closed control-loop or good models for open-loop control are required.

In this document, we have proposed an approach which consists in combining models and machine learning techniques to develop quasi-static models of flexible instruments. The rationale of this technique is that physical models cannot be easily identified in real practice given the large amount of variables needed and sensors required. Moreover, it has been shown that the 138 CHAPTER 5. CONCLUSION behavior of the flexible cable-driven instruments varies over time after use. This justifies to rely on data acquired on the system itself before actual use, which can be considered as more reliable than general models. However, for taking into account hysteresis effects which are prevalent in cable driven degrees of freedom, pure data-driven approaches would require a very large number of data and render them intractable for medical instruments. Therefore, we suggest to combine models and machine learning.

The general idea is to use the parts of the model that can be considered as reliable despite non-linear behaviors, in order to construct parts of the complete model, typically between the configuration space and the task space. Another feature is to rely on the knowledge of the system general working to decouple as much as possible the relations between the task space and more accessible spaces such as the actuation space or the configuration space. By doing so, learning from data can be performed in low-dimensional spaces, hence saving time and need of data. This approach was originally developed for flexible instruments with a single bending plane and then extended to two bending planes endoscopes. The capabilities of prediction of the obtained models have been tested in laboratory experiments, and the models have been used in open-loop control modes for both 2D tasks and 3D tasks. For 2D tasks, the accuracy has been shown to be very good (0.6 mm RMS error with maximum error of 1.7mm). These values are in the range of expected accuracy of surgeons, typically assessed at 1mm. They are largely improved with respect to state-of-the-art methods.

For 3D tasks, which involve the combination of 3 DoFs for the single bending plane instruments, qualitative and quantitative improvements have been observed, but the accuracy is less satisfactory than for 2D tasks. It can be argued that it is insufficient for a surgical task. However, it is at the same level of accuracy that was obtained for estimating the position of the same instrument using the feedback provided by the endoscopic camera. In the one hand, this means that an approach with a 3D feedback would not improve over this open-loop control. On the other hand the proposed approach also allows to have a fast response, not limited by the framerate of the endoscopic camera. Image-based control such as visual servoing could probably improve the accuracy parallel to the imaging plane but probably not in the depth direction. This speaks in favor of methods combining open-loop control and visual feedback.

Inspired by the work on the one bending plane instruments and the two bending planes endoscopes, a more general framework has been proposed for combining models and data-driven methods for modeling robotic systems. The key point is that learning directly from the actuation space to the task space is often intractable, while obtaining complete models is very challenging. This framework is especially interesting for systems with actuation means running over long distances (cables, super elastic tubes), for which accurate models are difficult to obtain. We have shown on the instruments and endoscopes available on the STRAS robot how the combination can be handled differently depending on the available hypotheses on the system working and on the workspace to cover. Models obtained from learning methods based on acquired data can only generalize well in conditions close to the ones of the training step. In laboratory experiments, the testing conditions were similar to the training conditions. In order to assess how the proposed approaches can be used practically, the proposed approach for modeling and open-loop control has also been tested in an in vivo experiment. The model of one of the instruments (the electrical hook in the right channel) has been obtained as developed in section 3.2 and 4.5.1. The training dataset has been obtained using the external stereoscopic system in the laboratory (see figure 2.11), prior to the procedure. In these in vivo experiments, the STRAS robotic system was mainly used for testing a new instrument specially developed in the lab, a steerable OCT catheter [START_REF] Mora | Steerable OCT catheter for real-time assistance during teleoperated endoscopic treatment of colorectal cancer[END_REF]. During the procedure, several elliptic trajectories were performed, similar to the ones used for testing in the laboratory experiments. In the in vivo environment, it was not possible to obtain the ground truth using external cameras. Endoscopic images where therefore acquired and processed off-line after the procedure. The trajectory was defined as an ellipse requiring motions in bending, rotation and translation. The desired trajectory was expressed in the endoscopic camera frame, by registering the frame of the instrument with the frame of the endoscopic camera. Figure 5.1 shows the qualitative results of this experiment.

We do not provide quantitative results as the 3D position of the instrument could not be measured. One can observe that the path performed with the proposed approach is much closer to the desired path than the one obtained with the standard geometrical model. Moreover, it seems qualitatively good, and well repeatable, with a good superimposition of two ellipses. This experiment confirms the laboratory results and in addition it allows to qualitatively validate that the model obtained beforehand and off-line can be used for open-loop control even after a normal medical manipulation of the endoscope -insertion and navigation of the endoscope, insertion of the instrument in the channel and its telemanipulation. Despite these promising preliminary results in vivo, in case of strong deformations of the endoscopic guide, it might be necessary to realize the identification process in situ, after reaching the operating area. In this case, external cameras cannot be used anymore, and the endoscopic camera becomes the only readily available sensor. The proposed approach requires 3D metric measurements of the tip of the instrument in order to reconstruct the distal parameters used for learning. This is of course more difficult to obtain with the embedded camera only. However, a recent work carried out in the team has shown that it is possible on a laboratory setup [START_REF] Poignonec | Towards in situ backlash estimation of continuum robots using an endoscopic camera[END_REF].

All the experiments have been realized in free space, without contact with the tissues. The proposed approach mainly aims for non-contact tasks such Figure 5.1: Trajectories performed in vivo using the proposed approach and the classic inverse kinematic model. as optical scanning or laser treatments. In case of strong interactions with tissues, it is known that the endoscopic instruments deform, which will impact the accuracy of the models based on data acquired without contact. Modeling the effect of interactions is a very complex problem, requiring the knowledge of the physical characteristics of both the instrument and the tissues. Datadriven approaches are not well-suited to handle these cases, as the deformation or displacement created by interactions are dependent on the type and direction of contacts. Measurement of deformations usually rely on embedded sensors [START_REF] Ahn | Measurement and characterization of soft tissue behavior with surface deformation and force response under large deformations[END_REF] possibly combined with proximal force measurement [START_REF] Ottensmeyer | In vivo data acquisition instrument for solid organ mechanical property measurement[END_REF]. Even state-of-the-art methods based on finite-elements models applied to catheters do not allow to predict accurately [START_REF] Kaladji | Prediction of deformations during endovascular aortic aneurysm repair using finite element simulation[END_REF]. In the case of contact, we envision the use of our models in combination with in situ feedback (typically endoscopic camera) to estimate the discrepancy between positions predicted by the models and the measurements.

Since the beginning of this thesis, other works using machine learning and kinematic knowledge for controlling surgical instruments have been developed. The works from [START_REF] Bibliography Baek | Hysteresis compensator with learning-based hybrid joint angle estimation for flexible surgery robots[END_REF] combine kinematic-based joint angle estimation with image-based joint angle estimation by the means of a Kalman Filter. Both estimation methods rely on neural networks for predicting the configuration of the instrument. The method was used for instruments with distal discrete joints, whose configuration is probably better estimated from proximal measurements that flexible bending instruments.

Another approach combining analytical and data-driven approaches for modelling the kinematics of robots is the approach from [START_REF] Cursi | Gphqp: Hierarchical quadratic programming for controlling a gaussian process regression model of redundant robot[END_REF]. On this work, the authors use Gaussian Process Regression combined with kinematics to obtain the forward kinematic model of the Micro-IGES robot.One of the main differences of this approach compared to the one presented at section 3.3 is the use of the predicted covariance provided by the GPR, which is not used with our method.

The appearance of similar methods further validates the proposed approaches and the need to improve the efficiency of learning-based algorithms. These hybrid techniques, although recent and few in sheer number, show a new trend in the control of flexible medical devices. Objectifs L'amélioration du ressenti des chirurgiens est un objectif important pour faciliter les procédures chirurgicales difficiles. Le développement de modes automatiques ou semi-automatiques dans les systèmes médicaux téléopérés est également un objectif envisageable. Ces modes peuvent soulager l'utilisateur de tâches difficiles et réduire la charge cognitive lors de longues procédures. Ces deux objectifs exigent de résoudre les problèmes de non-linéarité dans les systèmes flexibles à câble. C'est le but principal de cette thèse de doctorat. D'un point de vue robotique, trois approches principales peuvent être envisagées :

1. Supprimer ou limiter les non-linéarités en modifiant la conception mécanique des instruments ; 2. Modéliser les non-linéarités afin de les prendre en compte dans la commande en boucle ouverte ; 3. Utiliser la commande en boucle fermée en s'appuyant sur des informations distales pour éliminer les effets des non-linéarités.

Dans ce travail, nous essayons d'aborder le problème de l'identification quasi-statique du comportement des systèmes robotiques endoscopiques à partir des données acquises sur le système. L'utilisation de données pour la modélisation est aujourd'hui appelée "approches guidées par les données" "techniques d'apprentissage automatique" ou même "machine learning". Simultanément, nous souhaitons réduire le temps nécessaire à la construction des ensembles de données, avec l'idée que ces identifications, ou du moins des mises à jour, seront nécessaires avant chaque utilisation de l'instrument. Dans le contexte médical, les opérations d'identification doivent être aussi Chapitre 1

Compensation des non-linéarités : en combinant apprentissage automatique et analyse géométrique -Apprendre les relations entre les positions des moteurs et les paramètres distaux en utilisant un capteur externe. À savoir :

1. La relation inverse f -1 : θ → q rot de θ à q rot 2. La relation inverse g -1 : ρ → q bend de ρ à q bend 3. La relation directe h : q bend → ∆d de q bend à ∆d 4. La relation inverse k -1 : t → q trans de t à q trans . En raison de l'hystérésis, ces relations ne sont généralement pas des fonctions. Par conséquent, d'autres variables doivent être prises en compte afin de décrire les branches de l'hystérésis.

Au moment de l'utilisation :

-A partir d'une position souhaitée ch P * exprimer l'orientation souhaitée θ * , le rayon souhaité ρ * et la profondeur souhaitée d * .

-Utiliser les modèles appris pour calculer les positions des articulations :

1. q * rot = f -1 (θ * ) 2. q * bend = g -1 (ρ * ) 3. q * trans = k -1 (d * + h(q * bend )
). -Appliquer les positions articulaires souhaitées au système robotique.

On peut noter qu'aucun capteur ne doit être utilisé pendant cette étape.

entraînement des modèles

Un total de 700 points est utilisé pour entrainer tous les modèles (voir figure 1.3). La quantité de données nécessaires a été déterminée en analysant les courbes d'apprentissage de chaque modèle. Cette procédure permet de choisir correctement M pour éviter le sur-apprentissage (overfitting), et permet de limiter le nombre de points utilisés pour l'entraînement.

Résultats expérimentaux

Pour évaluer l'approche proposée, nous avons testé le comportement obtenu pour un instrument du robot STRAS pour des trajectoires 2D et 3D et nous l'avons comparé avec différentes techniques de commande.

Étant donnée une position souhaitée dans l'espace de travail, les modèles formés sont utilisés comme décrit précédemment pour calculer les consignes des positions des actionneurs. La précision est évaluée par un système de mesure externe le même que celui utilisé dans l'étape d'entraînement. Les trajectoires choisies sont des ellipses, qui imitent les contours typiques des lésions à disséquer lors des procédures de dissection endoscopique de la sousmuqueuse (ESD pour endoscopic submucosal dissection) dans le tube digestif [ Zorn et al., 2017]. Elles sont exécutées point par point pour évaluer la précision statique.

Trajectoire 3D

Dans la figure 1.4, la trajectoire souhaitée est une ellipse avec un grand axe de 18 mm, un petit axe de 17,2 mm, incliné de 24 degrés autour de l'axe X, ce qui nécessite l'utilisation des 3 degrés de liberté. Le tableau 1.1 donne les statistiques de l'erreur.

Discussion

Les expériences menées dans le cas 2D montrent que l'approche que nous proposons est plus performante que celle de [Xu et al., 2017] pour la précision du positionnement, tout en utilisant un ensemble de données d'entraînement plus petit. La précision obtenue permettrait l'exécution correcte de la dissection automatique dans le plan de flexion de l'instrument. L'expérience dans le cas 3D présente des erreurs plus importantes que dans le cas 2D. Néanmoins, l'erreur RMS et les erreurs maximales ont été réduites respectivement de 4 et 3 fois par rapport au modèle géométrique inverse classique.

Résumé

Les principales contributions de l'approche proposée sont les suivantes :

-La combinaison de connaissances cinématiques avec l'apprentissage permet la simplification de la structure des modèles formés par rapport à une approche boîte noire [Xu et al., 2017].

-Cela permet de réduire considérablement la quantité de données requises. Au total, 700 points ont été nécessaires pour former tous les modèles, alors que 20 000 points ont été utilisés dans [Xu et al., 2017]. Cette réduction rend l'approche plus adaptée aux applications médicales, en particulier pour les instruments à usage unique, puisque l'acquisition est nettement plus rapide.

-Elle permet de prendre en compte les effets de l'hystérésis à un coût très faible. Même s'il serait possible d'ajouter la direction de déplacement à l'espace d'entrée sur l'approche proposée dans [Xu et al., 2017], cela augmenterait aussi considérablement la complexité du modèle et la quantité de données nécessaires pour bien généraliser. Chaque point de l'espace de travail devrait être atteint à partir de 8 directions différentes dans l'ensemble d'entraînement, correspondant à toutes les combinaisons de directions pour chaque axe. 1.2 Endoscope flexible L'objectif considéré ici est de contrôler la position cartésienne du bout de l'endoscope flexible dans le repère de base F 0 , qui décrit l'espace de travail de l'endoscope. La position de référence de l'instrument dans ce repère sera notée 0 P * = (x * , y * ) T . Seules deux coordonnées sont prises en compte car il n'y a que deux degrés de liberté qui peuvent être contrôlés. Apply joint trajectory to the motors

Before use

Generate training dataset

Entrainement des modèles

Un total de 500 points est utilisé pour entrainer l'ensemble des deux modèles, 250 pour chaque articulation (voir figure 1.3). La quantité de données nécessaires a été déterminée en analysant les courbes d'apprentissage de chaque modèle une fois les autres hyperparamètres réglés, comme cela a été fait avec les outils chirurgicaux.

Pour générer l'ensemble d'apprentissage, nous créons une trajectoire pour ∆L x sous la forme d'une onde triangulaire, variant de -5mm à 5mm avec 50 échantillons pour la pente (pas de 0,2mm de déplacement du câble) avec ∆L y fixé à zéro. La même trajectoire est ensuite appliquée à ∆L y , avec ∆L x fixé à zéro.

-40 100

Résultats expérimentaux

La trajectoire à réaliser ressemble à un symbole infini dans le plan XY. Le déplacement le long de la direction X varie de -68mm à 92mm, tandis que le déplacement selon Y varie de -58mm à 58mm. Le tableau 1.2 présente un résumé des résultats obtenus lors de l'exécution de la trajectoire décrite. L'erreur RMS obtenue en utilisant notre approche proposée était de 4,4 mm, ce qui est une grande amélioration par rapport aux 18,3 mm atteints par le modèle géométrique conventionnel. La trajectoire exécutée peut être vue sur la figure 1.8.

Discussion

L'approche proposée est capable de compenser largement les non-linéarités introduites par la transmission par câble dans les endoscopes flexibles, comme le montrent les expériences. Un gain de précision de plus de 4 fois a été obtenu par rapport à la modélisation cinématique classique en intégrant une procédure d'apprentissage simple. Une réduction de la durée de la phase d'entrainement a également été obtenue par rapport à d'autres approches de l'état de l'art.

CHAPITRE 2. APPROCHE MULTICOUCHE POUR LA MODÉLISATION CINÉMATIQUE DES SYSTÈMES COMPLEXES 18

2. Définition de la tâche : Cette couche est celle dans laquelle la tâche à effectuer par le robot est définie (positionnement 2D, 3D, orientation ...) ; 3. Espace paramétré : En fonction de l'architecture du robot, il est possible de trouver un paramétrage plus adapté à la spécification de la tâche. Cependant, ce paramétrage peut ne pas être apparent ou le paramétrage cartésien peut déjà être optimal ; 4. Analyse cinématique : Les impacts entre la configuration et l'espace de travail sont mis en évidence sans tenir compte de la géométrie réelle du robot ; 5. Modélisation cinématique : Les relations géométriques entre les espaces de configuration et de tâche sont déterminées ; 6. Apprentissage automatique : Cette étape permet de déterminer la correspondance entre l'espace concerné -qu'il s'agisse de l'espace de tâche ou de l'espace de configuration -et l'espace d'actionnement.

Pour illustrer les couches présentées précédemment et la manière dont ces couches peuvent être utilisées de manière pratique, nous avons appliqué l'approche multicouche à plusieurs systèmes. Les résultats expérimentaux sont mentionnés pour les systèmes endoscopiques disponibles au laboratoire, mais d'autres systèmes sont également abordés pour illustrer l'applicabilité de la technique.

Application aux endoscopes flexibles à deux plans de flexion

Dans cette section, nous allons réécrire l'approche utilisée par [Xu et al., 2017] et une nouvelle approche de modélisation en utilisant le formalisme décrit précédemment.

Approche I -Lien direct entre l'espace de tâche et l'espace d'actionnement

Comme indiqué précédemment, cette approche a été suggérée dans [Xu et al., 2017]. Une adaptation a été faite afin de la présenter avec le formalisme à plusieurs couches (voir tableau ??).

Comme indiqué précédemment, ce modèle ne se généralise pas bien, principalement en raison de l'hypothèse formulée. Au lieu de modéliser les branches d'hystérésis, cette approche se termine par un modèle "moyen" qui n'est pas précis et qui nécessite une quantité relativement importante de données pour s'entraîner.

La précision peut être améliorée par une modification de l'hypothèse. La prise en compte de la boucle d'hystérésis peut être faite de manière simple, mais elle nécessitera davantage de données pour avoir une bonne générali- Table 2.1 -Résumé des couches de modélisation utilisées pour l'approche I sur l'endoscope flexible.

sation. Même si elle est la plus simple à mettre en oeuvre, cette approche ne permet pas un bon compromis entre précision et efficacité.

2.2.2

Approche II -Utilisation de l'hypothèse des petits angles de flexion L'apprentissage automatique peut être intégré de manière différente dans la modélisation globale du même système lorsque ses mouvements sont limités par des espaces contraints . C'est en fait le cas de nombreuses applications en endoscopie basse lors de l'utilisation de coloscopes de grand diamètre, car les tissus du côlon limitent l'amplitude des mouvements lors de la manipulation (voir le tableau ??).

Couche

Contenu de la couche de modélisation 1 -Hypothèse Angles de flexion petits ; Pas de couplage entre les articulations ; Transmission par câble soumise à des non-linéarités.

2 -Espace cartésien P = (x, y) 4 -Analyse cinématique x = f (∆L x ) ; y = g(∆L y ).

-Apprentissage automatique

La relation inverse f -1 : x → ∆L x from x to ∆L x ; La relation inverse g -1 : y → ∆L y de y à ∆L y .

Table 2.2 -Résumé des couches de modélisation utilisées pour l'approche II sur l'endoscope flexible.

Application aux outils chirurgicaux flexibles

L'approche présentée à la section 1.1 et une adaptation de l'approche utilisée par [Xu et al., 2017] seront présentées dans cette section. Cette approche a été expliquée en détails au chapitre 1.1. En utilisant le cadre d'extension présenté dans ce chapitre, il est possible de faire apparaître les similitudes et les divergences avec l'approche appliquée aux endoscopes flexibles (voir la section 1.2).

Cette approche de modélisation est un excellent exemple d'approche par boîte translucide (voir tableau ??). Même si une analyse a dû être effectuée afin de comprendre les effets de chaque articulation sur l'espace de travail, il n'est pas nécessaire d'aller plus loin dans la modélisation géométrique. Cette approche présente un bon compromis entre la fiabilité des hypothèses auxquelles elle est soumise et la simplicité des modèles qu'il faut apprendre.

Couche

Contenu de la couche de modélisation 1 -Hypothèse Pas de couplage entre les articulations ; Transmission par câble soumise à des non-linéarités.

2 -Espace cartésien P = (x, y, z)

-Paramétrisation

Paramétrisation cylindrique : ρ = x 2 + y 2 θ = atan2(y, x) d = z 4 -Analyse cinématique θ = f (q rot ) ; ρ = g(q bend ) ; ∆d = h(q bend ) ; t = k(q trans ).

-Apprentissage automatique

La relation inverse f -1 : θ → q rot from θ to q rot La relation inverse g -1 : ρ → q bend de ρ à q bend La relation directe h : q bend → ∆d de q bend à ∆d La relation inverse k -1 : d → q trans de d à q trans . Table 2.3 -Résumé des couches de modélisation utilisées pour l'approche I sur les outils chirurgicaux.

Approche II -Lien direct entre l'espace de tâche et l'espace d'actionnement

Une approche beaucoup plus opaque, s'apparentant vers une approche de type boîte noire, peut être appliquée aux outils chirurgicaux en adaptant les données de [Xu et al., 2017]. De même que ce qui a été montré dans la section 2.2.1, un lien direct sera fait avec la couche d'apprentissage automatique sans aucune considération sur la forme ou la façon dont l'outil se déplace (voir tableau ??). 6 -Apprentissage automatique La relation inverse f -1 : (x, y) → (q bend , q trans ) from (x, y) to (q bend , q trans ) Table 2.4 -Résumé des couches de modélisation utilisées pour l'approche II sur les outils chirurgicaux. Chapitre 3

Conclusion

Les instruments flexibles et les endoscopes sont des outils essentiels pour la chirurgie des années à venir, tant en chirurgie digestive qu'en chirurgie endovasculaire et cardiaque. Les deux domaines partagent le même besoin de disposer d'instruments longs et fins, capables de naviguer dans un environnement tortueux tout en limitant les interactions avec les tissus. Tous les deux types de chirurgie ont également besoin de dextérité pour effectuer des mouvements chirurgicaux fins et précis. En chirurgie endoluminale, le retour d'information au chirurgien est généralement fourni à partir d'une caméra endoscopique montée à l'extrémité distale de l'endoscope.

Dans ce document, nous avons proposé une approche qui consiste à combiner des modèles et des techniques d'apprentissage automatique pour développer des modèles quasi-statiques d'instruments flexibles. Le principe de cette technique est que les modèles physiques ne peuvent pas être facilement identifiés dans la pratique étant donné la grande quantité de variables nécessaires et de capteurs requis. De plus, nous avons démontré que le comportement des instruments flexibles à câble varie dans le temps après utilisation. Cela justifie l'utilisation des données acquises sur le système lui-même avant l'utilisation réelle, qui sont plus fiables que les modèles classiques génériques. Par contre, pour tenir compte des effets d'hystérésis qui sont courants dans les degrés de liberté des instruments à câble, les approches purement guidées par les données nécessitent un très grand nombre de données et les rendent impossibles à utiliser dans un contexte médical. C'est pourquoi nous suggérons de combiner les modèles géométriques classiques et l'apprentissage automatique.

Depuis le début de cette thèse, d'autres travaux combinant l'apprentissage automatique et la connaissance cinématique pour la commande des instruments chirurgicaux ont été développés. L'apparition de méthodes similaires valide davantage les approches proposées et la nécessité d'améliorer l'efficacité des algorithmes basés sur l'apprentissage. Ces techniques hybrides, bien que récentes et peu nombreuses, montrent une nouvelle tendance dans la commande des dispositifs médicaux flexibles.

4. 1 1

 11 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Rationale of the method . . . . . . . . . . . . . . . . . . . . . . 4.3 Determining the functions to be learned . . . . . . . . . . . . . 4.4 Application to flexible endoscopes with two bending planes . . 4.5 Application to flexible surgical tools . . . . . . . . . . . . . . . 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Illustration of typical laparoscopic cholecystectomy [U.S. National Library of Medicine -National Institutes of Health, 2020].

  . . . . . . . . . . . . . . . . . . . . . . . 42 1.20 View of the telemanipulated portion of the STRAS platform[Nageotte et al., 2020]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 1.21 View of the distal part of the da Vinci SP Gen 4 [Intuitive Surgical, 2020a]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 1.22 A: da Vinci SP Gen 4. B: surgeon console. [Intuitive Surgical, 2020a] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 1.23 Multi articulated instruments of the SPORT surgical system [Intuitive Surgical, 2020b]. . . . . . . . . . . . . . . . . . . . . . . . . . . 45 1.24 A: SPORT patient cart. B: SPORT surgeon workstation. [Intuitive Surgical, 2020b] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.1 Kinematic diagram of the telemanipulated robot [Nageotte et al., 2020]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.2 3D schematic view of the controlling interface and its kinematic diagram [de Mathelin et al., 2014]. . . . . . . . . . . . . . . . . . . 52 2.3 Mapping between the controlling interface and the telemanipulated robot [de Mathelin et al., 2014]. . . . . . . . . . . . . . . . . 53 2.4 Depiction of the 3 spaces used for modeling the geometry of constantcurvature robots. The robot-specific relations transform the robot's actuation into the variables describing its configuration in space.

3. 1

 1 Task space showing the measured 3D positions of the tip of the endoscope (left) and actuator space[Xu et al., 2017a]. . . . . . . . 3.2 Left channel frame . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Typical static characteristics between the motor controlling bending and the Cartesian positions of the flexible instrument, for different values of translation and rotation. . . . . . . . . . . . . . . . 3.4 Flowchart of the proposed approach [Porto et al., 2019d]. . . . . . 3.5 Graphical representation of an Extreme Learning Machine. . . . . 3.6 Training set shown in the channel frame. . . . . . . . . . . . . . . . 3.7 Inverse rotation model f -1 . . . . . . . . . . . . . . . . . . . . . . . LIST OF FIGURES 3.8 Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.9 Inverse radius model . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.10 Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 3.11 Direct depth variation during bending model h. . . . . . . . . . . . 90 3.12 Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.13 Inverse translation model k -1 . . . . . . . . . . . . . . . . . . . . . . 91 3.14 Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.15 2D trajectory on the XY-plane . . . . . . . . . . . . . . . . . . . . . 92 3.16 2D trajectory shown in the time domain . . . . . . . . . . . . . . . 93 3.17 2D trajectory without backlash compensation on the XY-plane. . . 93 3.18 2D trajectory without backlash compensation shown in the time domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 3.19 2D trajectory using the approach from

  shown in the time domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 3.21 3D trajectory performed in the task space. . . . . . . . . . . . . . . 96 3.22 3D trajectory using the proposed approach in the time domain. . . 96 3.23 Depiction of a flexible endoscope and its important modeling parameters [Porto et al., 2019a]. . . . . . . . . . . . . . . . . . . . . . 98 3.24 Flowchart of the proposed approach applied to flexible endoscopes. 100 3.25 Illustration of a Gaussian Process. . . . . . . . . . . . . . . . . . . . 101 3.26 Training set shown in the base frame. . . . . . . . . . . . . . . . . . 103 3.27 Learned model p -1 linking the bending angle on X β x and the cable displacement ∆L x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 3.28 Learning curve of the function p -1 . . . . . . . . . . . . . . . . . . . 104 3.29 Learned model r -1 linking the bending angle on Y β y and the cable displacement ∆L y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.30 Learning curve of the function r -1 . . . . . . . . . . . . . . . . . . . 105 3.31 2D trajectory performed by the flexible endoscope using the proposed approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 3.32 2D trajectory performed by the flexible endoscope using the approach from [Xu et al., 2017a]. . . . . . . . . . . . . . . . . . . . . . 107 4.1 General scheme of the approach highlighting the different modeling layers and spaces involved during the kinematic modeling. . . 113 4.2 Surgical tool with a pivot joint (right) instead of a flexible bending joint (left). The effects on the task space can be expressed with the same parameters, however the shape of the robot is different. . . . 4.3 Sensitivity analysis -Impact of β y on the x coordinate and β x on the y coordinate. The parameters used to generate this image were L f = 160mm and L d = 50mm, parameters of the ANUBISCOPE. . . 4.4 Comparison between the configuration space and cartesian space. 4.5 Learned model f -1 linking the displacement on X and the cable displacement ∆L x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Learned model f -1 linking the displacement on Y and the cable displacement ∆L y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 2D trajectory performed by the flexible endoscope under the hypothesis of small bending angles. . . . . . . . . . . . . . . . . . . . 5.1 Trajectories performed in vivo using the proposed approach and the classic inverse kinematic model. . . . . . . . . . . . . . . . . . .

1

 1 Porto, R. A., Nageotte, F., Zanne, P., and de Mathelin, M. (2019d). Position control of medical cable-driven flexible instruments by combining machine learning and kinematic analysis. In 2019 International Conference on Robotics and Automation (ICRA), pages 7913-7919. IEEE Porto, R. A., Nageotte, F., Zanne, P., and de Mathelin, M. (2019a). Backlash compensation in cable-driven flexible endoscopes using machine learning and kinematic analysis. In IEEE International Conference on Robotics and Automation (ICRA) Workshop-Open Challenges and State-of-the-Art in Control System Design and Technology Development for Surgical Robotic Systems Porto, R. A., Nageotte, F., Zanne, P., and de Mathelin, M. (2019b). Combining machine learning and kinematic analysis to control medical cable-driven flexible instruments. In Joint Workshop on Computer/Robot Assisted Surgery (CRAS) Peer Reviewed National Conferences Porto, R. A., Nageotte, F., Zanne, P., and de Mathelin, M. (2019c). Modeling the non-linearities of flexible endoscopes using machine learning. In Proceedings of SurgeticaEvolution of surgery techniques of the digestive tract . . . . . . 1.2 Flexible endoscopy . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Flexible systems for mini-invasive surgery . . . . . . . . . . . . 1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 CHAPTER 1. MEDICAL CONTEXT

Figure 1 . 1 :

 11 Figure 1.1: Illustration of typical laparoscopic cholecystectomy [U.S. National Library of Medicine -National Institutes of Health, 2020]. The abdomen is inflated by using CO 2 to create space for the surgical gesture. The rigid endoscope is inserted through a trochar to allow visualization of the internal organs.

1. 1 .Figure 1 . 2 :

 112 Figure 1.2: Timeline of major breakthroughs. Lichtleiter -[START_REF] Rathert | Philipp bozzini (1773-1809) and the lichtleiter[END_REF],Desormeaux -[Zada et al., 2012], Kelling -[START_REF] Litynski | Origin of laparoscopy: Coincidence or surgical interdisciplinary thought?[END_REF]], Schindler -[START_REF] Benedict | Examination of the stomach by means of a flexible gastroscope: A preliminary report[END_REF], Hirschowitz fiberscope -[START_REF] Wilcox | Fifty years of gastroenterology at the university of alabama at birmingham: a festschrift for dr. basil i. hirschowitz[END_REF] and video endoscope -[START_REF] Wheeler | Endoscopic apparatus[END_REF].

Figure 1 . 3 :

 13 Figure 1.3: Types of endoscopes used on different parts of the GI tract

Figure 1

 1 Figure 1.4: A: Seemingly normal endoscopic image of Barrett's esophagus. B: Endoscopic image of the same patient after the staining with methylene blue. The bottom arrow indicates the presence of an intramucosal adenocarcinoma, whereas the arrow on the top indicate a normal stained mucosa [Song et al., 2007].

Figure 1 . 5 :

 15 Figure1.5: Several surgical tools used in endoscopy. Source :[START_REF] Storz | Systems for Gastroenterology 3rd edition[END_REF]

Figure 1 . 6 :

 16 Figure 1.6: Illustration of endoscopic submmucosal dissection. 1 -marking the zone of interest with an electric knife. 2 -injection of a saline solution with a needle under the tissue, in the submucosa, to elevate the area of dissection and reduce the risk of perforation. 3 -Dissection of the tissue by following the marks done at step 1. 4 -Removing of the dissected tissue using a grasper.

Figure 1 .

 1 Figure 1.7: Illustration of NOTES cholecystectomy with different entry points.

Figure 1 . 8 :

 18 Figure 1.8: Schematic view of a flexible endoscope and a depiction of the different channels (adapted from [AORN. et al., 2016]).

Figure 1 . 9 :

 19 Figure 1.9: Experimentation in NOTES at IRCAD France.

Figure 1 .

 1 Figure 1.10: Schematic view of the R-scope [Yonezawa et al., 2006].

Figure 1 .

 1 Figure 1.11: Real R-scope system [Astudillo et al., 2009].

  tion (allowing the change of depth of the instrument), rotation (allowing the change of angle of the instrument) and bending of the distal part (causing a change in the orientation of the tip of the instrument as well as a change of depth).

Figure 1 .

 1 Figure 1.14: View of the distal part of the Anubiscope [De Donno et al., 2012].

Figure 1 .

 1 Figure 1.15: Overall view from the Anubiscope system. It presents 110cm of passive flexible body and a 18.5 cm bendable tip. The wheels on the handle allow for controlling the bending of the distal part, with the possibility of locking them in place. Image from https://www.karlstorz.com/.

Figure 1 .

 1 Figure 1.20: View of the telemanipulated portion of the STRAS platform[Nageotte et al., 2020].

Figure 1 .Figure 1 .

 11 Figure 1.21: View of the distal part of the da Vinci SP Gen 4 [Intuitive Surgical, 2020a].

Figure 1 .Figure 1 .

 11 Figure 1.23: Multi articulated instruments of the SPORT surgical system [Intuitive Surgical, 2020b].

2. 1 .

 1 DESCRIPTION OF THE STRAS PLATFORM 51 of the instruments. Therefore, each instrument has 3 degrees of freedom (graspers additionally have an opening and closing movement). The kinematic diagram of the STRAS platform is illustrated in figure 2.1.

Figure 2 . 1 :

 21 Figure 2.1: Kinematic diagram of the telemanipulated robot [Nageotte et al., 2020].

Figure 2 . 2 :

 22 Figure 2.2: 3D schematic view of the controlling interface and its kinematic diagram [de Mathelin et al., 2014].

Figure 2 . 3 :

 23 Figure 2.3: Mapping between the controlling interface and the telemanipulated robot [de Mathelin et al., 2014].

. 1 )Figure 2

 12 Figure 2.5: a: projection of the constant-curvature robot onto the bending plane. On this case, for a null rotation angle φ, the bending plane coincides with the x-z plane. b: Depiction of the arc parameters, namely and arc length , angle of the bending plane φ and curvature κ [Webster III and Jones, 2010].

Figure 2 . 6 :

 26 Figure 2.6: 3D view of the distal part of the flexible endoscope.

Figure 2 . 7 :

 27 Figure 2.7: View from the top (projection onto the x-y plane) of the flexible endoscope.

Figure 2 . 8 :

 28 Figure 2.8: Orthogonal projection of the flexible endoscope onto the bending plane Π.

Figure 2 .

 2 Figure 2.10: Orthogonal projection of the flexible endoscope onto the bending plane Π.

Figure 2 .

 2 Figure 2.13: Static characteristic between the cable displacement and the bending angle on both axis of the same flexible endoscope.

Figure 2 .

 2 Figure 2.16: Image of the robotic catheter with the position sensor embedded on its tip (left) and the actuation unit (right) [Penning et al., 2011].

Contents

3. 1

 1 Using machine learning for robot control . . . . . . . . . . . . . 3.2 Flexible surgical tools . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Flexible endoscope . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 CHAPTER 3. NON-LINEARITIES COMPENSATION MIXING MACHINE LEARNING AND KINEMATIC ANALYSIS

3. 1 .

 1 USING MACHINE LEARNING FOR ROBOT CONTROL Parametric vs Non-parametric methods Parametric models: The complexity of the model is unchanged based on the amount of training data. The training process usually revolves around finding the values of a finite number of parameters that optimizes a cost function, usually based on (but not limited to) the fitting of the data.

Figure 3 . 1 :

 31 Figure 3.1: Task space showing the measured 3D positions of the tip of the endoscope (left) and actuator space [Xu et al., 2017a].

Figure 3 . 2 :

 32 Figure 3.2: The considered cable-driven instrument passing through the left channel of the STRAS system. Top: top view, bottom: face view. Figure shows left channel frame, the degrees-of-freedom of the instrument, configuration variables and important construction parameters[Porto et al., 2019d].

  Figure 3.3: Typical static characteristics between the motor controlling bending and the Cartesian positions of the flexible instrument, for different values of translation and rotation.

Figure 3 . 5 :

 35 Figure 3.5: Graphical representation of an Extreme Learning Machine.

Figure 3 . 6 :

 36 Figure 3.6: Training set shown in the channel frame.

  Figure 3.7: Inverse rotation model.

  (a) Learning curve for the function f -1 . (b) Validation curve for the function f -1 .

Figure 3

 3 Figure 3.8: Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data.

Figure 3 . 9 :

 39 Figure 3.9: Inverse radius model g -1 .

Figure 3

 3 Figure 3.11: Direct depth variation during bending model h.

Figure 3 .

 3 Figure 3.12: Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data.

Figure 3

 3 Figure 3.13: Inverse translation model k -1 .

Figure 3 .

 3 Figure 3.14: Curves used to tune the hyperparameter of the ELM and set the amount of training data required for generalizing well outside of the training data.

Figure 3

 3 Figure 3.15: 2D trajectory on the XY-plane.

Figure 3

 3 Figure 3.16: 2D trajectory shown in the time domain.

Figure 3 Figure 3

 33 Figure3.17: 2D trajectory without compensation on the XY-plane.

Figure 3

 3 Figure3.19: 2D trajectory using the approach from[Xu et al., 2017a] on the XY-plane.

Figure 3 .Figure 3

 33 Figure 3.21: 3D trajectory performed in the task space.

Figure 3 .

 3 Figure 3.23: Depiction of a flexible endoscope and its important modeling parameters [Porto et al., 2019a].

Figure 3 .

 3 Figure3.24: Flowchart of the proposed approach applied to flexible endoscopes.

  . The joint distribution of the training set and the prediction set are : mean value of the training set (µ = m(X)), Σ the training set covariance (Σ = k(X, X )), Σ * the training-prediction set covariance (Σ * = k(X, X * ')) and Σ * * the prediction set covariance (Σ * * = k(X * , X * ')). Considering that the training set is known, our objective is to determine the conditional distribution of Y * given Y . This distribution is determined as :

Figure 3

 3 Figure 3.26: Training set shown in the base frame.

  Figure 3.27: Learned model p -1 linking the bending angle on X β x and the cable displacement ∆L x .

Figure 3 .

 3 Figure 3.28: Learning curve of the function p -1 .

  Figure 3.29: Learned model r -1 linking the bending angle on Y β y and the cable displacement ∆L y .

Figure 3 .

 3 Figure 3.30: Learning curve of the function r -1 .

Figure 3

 3 Figure 3.31: 2D trajectory performed by the flexible endoscope using the proposed approach.

Figure 3

 3 Figure3.32: 2D trajectory performed by the flexible endoscope using the approach from[Xu et al., 2017a].

Figure 4 . 1 :

 41 Figure 4.1: General scheme of the approach highlighting the different modeling layers and spaces involved during the kinematic modeling.

Figure 4 . 2 :

 42 Figure 4.2: Surgical tool with a pivot joint (right) instead of a flexible bending joint (left).The effects on the task space can be expressed with the same parameters, however the shape of the robot is different.

  quadratic relation between β x , β y and β, it is possible to show that :

Figure 4 Figure 4 . 4 :

 444 Figure 4.3: Sensitivity analysis -Impact of β y on the x coordinate and β x on the y coordinate. The parameters used to generate this image were L f = 160mm and L d = 50mm, parameters of the ANUBISCOPE.

Figure 4 Figure 4 . 6 :

 446 Figure 4.5: Learned model f -1 linking the displacement on X and the cable displacement ∆L x .

  Figure 4.7: 2D trajectory performed by the flexible endoscope under the hypothesis of small bending angles.

Contents 1 . 1 1 . 2

 1112 Outils chirurgicaux flexibles . . . . . . . . . . . . . . . . . . . . 7 Endoscope flexible . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.1 Outils chirurgicaux flexibles L'objectif considéré ici est de contrôler la position cartésienne du bout des instruments dans le repère du canal F ch . La position de référence de l'instrument dans ce cadre sera notée ch P * = (x * , y * , z * ) T . La méthode globale peut être brièvement décrite comme suit (voir également la figure 1.2) : Avant utilisation :

Figure 1 . 1 -

 11 Figure1.1 -L'instrument chirurgical actionné par câble passant par le canal gauche de l'endoscope. En haut : vue de dessus, en bas : vue de face. La figure montre le repère du canal gauche, les degrés de liberté de l'instrument, les variables de configuration et les paramètres de construction importants[Porto et al., 2019d].

Figure 1 . 2 -

 12 Figure1.2 -Organigramme de l'approche proposée[Porto et al., 2019d].

Figure 1

 1 Figure 1.3 -Ensemble de données d'entraînement exprimé dans le repère canal.

Figure 1

 1 Figure 1.4 -Trajectoire 3D effectuée dans l'espace des tâches.

CHAPITRE 1 .

 1 COMPENSATION DES NON-LINÉARITÉS : EN COMBINANT APPRENTISSAGE AUTOMATIQUE ET ANALYSE GÉOMÉTRIQUE 12

Figure 1 .

 1 Figure 1.5 -Illustration d'un endoscope flexible et de ses paramètres de construction [Porto et al., 2019a].

1. 2 . 1

 21 Justification de la méthode -adaptation aux endoscopes flexibles En utilisant β x et β y comme paramètres distaux, un lien découplé peut être trouvé entre l'espace de configuration et l'espace d'actionnement. L'intérêt de l'utilisation de l'apprentissage automatique est de remplacer la relation entre les déplacements des câbles et les angles de flexion par une relation obtenue à partir de l'observation du comportement réel de la transmission mécanique. La méthode globale peut être brièvement décrite comme suit (voir également la figure 1.6) : Avant utilisation : CHAPITRE 1. COMPENSATION DES NON-LINÉARITÉS : EN COMBINANT APPRENTISSAGE AUTOMATIQUE ET ANALYSE GÉOMÉTRIQUE 13 -Apprendre les relations entre les positions des moteurs et les paramètres distaux en utilisant un capteur externe. À savoir : 1. La relation inverse p -1 : β x → ∆L x de β x à ∆L x 2. La relation inverse r -1 : β y → ∆L y de β y à ∆L y De la même manière qu'auparavant, ces relations ne sont généralement pas des fonctions. La direction du déplacement sera également prise en compte pour décrire avec précision le comportement de l'hystérésis. Au moment de l'utilisation : -A partir d'une position souhaitée 0 P * exprimer les angles de flexion souhaités β * x et β * y . -Utiliser les modèles appris pour calculer les positions articulaires : 1. ∆L * x = p -1 (β * x ) 2. ∆L * y = r -1 (β * y ). -Appliquer les positions articulaires souhaitées au système robotique.

Figure 1 . 6 -

 16 Figure 1.6 -Organigramme de l'approche proposée appliquée aux endoscopes flexibles.

Figure 1 . 7 -

 17 Figure 1.7 -Ensemble de données d'entrainement exprimé dans le repère de base.

Figure 1

 1 Figure 1.8 -Trajectoire 2D effectuée par l'endoscope flexible selon l'approche proposée.

  f -1 : (x, y) → (∆L x , ∆L y ) de (x, y) à (∆L x , ∆L y ) ;

CHAPITRE 2 .

 2 APPROCHE MULTICOUCHE POUR LA MODÉLISATION CINÉMATIQUE DES SYSTÈMES COMPLEXES 20 2.3.1 Approche I -Sans hypothèse sur la courbure

Figure 2 . 1 -

 21 Figure 2.1 -Schéma général de l'approche mettant en évidence les différentes couches de modélisation et les espaces impliqués lors de la modélisation cinématique.

  . . . . . . . . . . . . . . . . . . 2.6 3D view of the distal part of the flexible endoscope. . . . . . . . . 2.7 View from the top (projection onto the x-y plane) of the flexible endoscope. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Orthogonal projection of the flexible endoscope onto the bending plane Π. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 The experimental setup is composed of the STRAS platform equipped with Chilitag markers [Bonnard et al., 2013] on the tip and the external 3D position measuring system. . . . . . . . . . . . . . . . . . 2.10 Orthogonal projection of the flexible endoscope onto the bending plane Π. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 3 .

 3 2 gives the statistics of the error. By adding the rotation, the task

	3.2. FLEXIBLE SURGICAL TOOLS				95
		40		Measured				Reference
	X (mm)	0 20
	-20	0	50	100	150	200	250	300
		60
	Y (mm)	50 55
		45	0	50	100	150	200	250	300

Table 3 . 1 :

 31 Statistics of the errors for the 2D trajectories for different position control methods

	Statistics	our method	w/o direction input [Xu et al., 2017a]
		X	Y	2D	X	Y	2D	X	Y	2D

Table 3

 3

	.2: Statics of the error signal regarding the 3D trajectory
	Statistics		Direction
		X	Y	Z	3D
	RMS (mm)	1.0339 1.0370 1.5793 2.1537
	Variance (mm 2 ) 0.8276 0.4907 2.4225 1.1023
	Max (mm)	2.4274 2.0245 4.4336 6.0239

Table 3

 3

	.3: Statistics of the errors for the 2D trajectories for different position
	control methods
	Statistics		our method			IKM		[Xu et al., 2017a]
		X	Y	2D	X	Y	2D	X	Y	2D
	RMS (mm)	2.9	3.3	4.4	4.0	17.8 18.3 5.3 10.5 11.8
	Var. (mm 2 ) 15.0 184.5 58.7 20.1 105.6 8.4 23.6 6.7 11.6
	Max (mm)	5.7	8.6	8.9	7.8	28.3 28.8 14.5 19.8 20.2

Contents

  Objectifs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Liste des publications . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

	Introduction

Table 1 .

 1 1 -Statique de l'erreur concernant la trajectoire 3D

	Statistique		Direction
		X	Y	Z	3D
	RMS (mm)	1.0339 1.0370 1.5793 2.1537
	Variance (mm 2 ) 0.8276 0.4907 2.4225 1.1023
	Max (mm)	2.4274 2.0245 4.4336 6.0239

Table 1 .

 1 2 -Statistiques des erreurs pour les trajectoires 2D pour différentes méthodes de commande de position

	Statistiques	notre méthode		IKM		[Xu et al., 2017]
		X	Y	2D	X	Y	2D	X	Y	2D
	RMS (mm)	2.9	3.3	4.4	4.0	17.8 18.3	5.3	10.5 11.8
	Var. (mm 2 ) 0.19 0.03 0.11 39.88 4.19 8.37 49.10 6.67 11.57
	Max (mm)	5.7	8.6	8.9	7.8	28.3 28.8 14.5 19.8 20.2

(a) Learning curve for the function g -1 . (b) Validation curve for the function g -1 .

Acknowledgements

Chapter 4

Multilayered approach for kinematic modeling of complex systems

Layer

Content of the modeling layer

-Hypothesis

No coupling between the joints; Cable transmission subjected to non-linearities.

2 -Cartesian space P = (x, y, z)

-Parameterization

Cylindrical parameterization:

-Machine learning

The inverse relation f -1 : θ → q rot from θ to q rot The inverse relation g -1 : ρ → q bend from ρ to q bend The direct relation h : q bend → ∆d from q bend to ∆d The inverse relation k -1 : d → q trans from d to q trans .

ÉCOLE DOCTORALE 269 Mathématiques, Sciences de l'Information et de l'Ingénieur