Elodie Paquet
email: elodie.paquet@ls2n.fr

Marceau Metillon
email: marceau.metillon@ls2n.fr

Kevin Subrin
email: kevin.subrin@ls2n.fr

Benoit Furet
email: benoit.furet@ls2n.fr

Stéphane Caro
email: stephane.caro@ls2n.fr

Path Generation for Foam Additive Manufacturing of Large Parts with a Cable-Driven Parallel Robot

Keywords: Additive Manufacturing, Cable-Driven Parallel Robot, Innovative Construction

In this paper, a framework for foam printing with a Cable-Driven Parallel Robot (CDPR) is described for building large parts. Compared with the traditional robotic systems, CD-PRs have a large workspace that can include the printing area. In addition, the potential reconfigurability of CDPRs is an asset to get rid of the collisions between the cables and the environment during the execution of the printing task. The printing feasibility is verified through the process identification where key parameters are used in the proposed control law. The features to be taken into account through a framework to make a successful printing with a CDPR are described. Finally, advantages and drawbacks of CDPRs for additive manufacturing are discussed and future work is presented.

Introduction

3D printing solutions are generally based on Cartesian robots with a 2D motion in the horizontal plane and a 1D motion for the support table, corresponding to a three degrees of freedom machines [START_REF] Barnett | Large-scale 3d printing with a cable-suspended robot[END_REF][2] [START_REF] Keating | Towards Site-Specific and Self-Sufficient Robotic Fabrication on Architectural Scales[END_REF]. For printing large parts, several materials can be used such as: plastic materials [START_REF] Paquet | Foam additive manufacturing technology: main characteristics and experiments for hull mold manufacturing[END_REF], polymer foams [START_REF] Barnett | Large-scale 3d printing with a cable-suspended robot[END_REF] [START_REF] Furet | 3D printing for construction based on a complex wall of polymer-foam and concrete[END_REF][5] concrete [START_REF] Viktor Mechtcherine | Large-scale digital concrete constructionconprint3d concept for on-site, monolithic 3dprinting[END_REF][6] [START_REF] Kaduk | Development of a computer vision based real-time feedback system for closed-loop control in 3d concrete printing[END_REF], or materials based on metallic particles [START_REF] Chesser | Design of a cable-driven manipulator for large-scale additive manufacturing[END_REF] [START_REF] Duty | Structure and mechanical behavior of Big Area Additive Manufacturing (BAAM) materials[END_REF]. The combination of automatic system and process has allowed multiple advances for large-scale 3D-printing, particularly house walls in construction fields. For instance, BatiPrint3D™ construction technology, composed of a Staubli poly-articulated arm and a B2A Systems Automatic Guided Vehicle (AGV), was proposed by Nantes University, France (Figure 1). In 2017, Yhnova demonstrator became a 95m 2 social dwelling built for the social landlord Nantes Métropole Habitat (NMH) [START_REF] Furet | 3D printing for construction based on a complex wall of polymer-foam and concrete[END_REF].

The 3D-printing demonstrator BatiPrint3D™ technology focuses on the construction of house walls through the deposition of two layers of polymer foam used as a formwork for a third concrete layer inside. To build large parts Figure 1. Additive foam manufacturing: BatiPrint3D, Yhnova's house 3D Printing with an AGV and a serial robot without interruption in the printing process that would be related to the concatenation of the different workspaces of a mobile robot, the use of CDPRs for additive manufacturing in a short term is conceivable [START_REF] Kaduk | Development of a computer vision based real-time feedback system for closed-loop control in 3d concrete printing[END_REF][10] [START_REF] Cabay | On the improvements of a cabledriven parallel robot for achieving additive manufacturing for construction[END_REF] [START_REF] Izard | Large-scale 3d printing with cable-driven parallel robots[END_REF]. CD-PRs are a potentially suitable replacement for very largescale applications [START_REF] Chesser | Design of a cable-driven manipulator for large-scale additive manufacturing[END_REF] [START_REF] Chawla | Workspace analysis and design of large-scale cable-driven printing robot considering cable mass and mobile platform orientation[END_REF]. They easily achieve a large workspace without requiring massive equipment and machinery [START_REF] Chesser | Kinematics of a cabledriven robotic platform for large-scale additive manufacturing[END_REF][START_REF] Gagliardini | Discrete reconfiguration planning for Cable-Driven Parallel Robots[END_REF]. This paper introduces a framework to adapt the 3D Printing process to a CDPR. A printing head with polymer foam for the production of large parts is managed by this architecture. The 3D foam printer performance is demonstrated through the construction of two different large parts, with an accuracy equal to 1 cm. The advantages and drawbacks of the novel 3D printer are then discussed. The paper is organized as follows: Section 2 introduces the process and the robotic architecture and a way to identify key parameters to adapt the process on a CDPR. Section 3 highlights the experimental validation and proposes a framework to fulfill the need. Section 4 draws some conclusions and future work.

Spraying End-Effector mounted on a CDPR

Spraying end-Effector

The process of deposition is managed by an air-actuated motor which controls the spraying nozzle enabling or disabling the foam deposition. Two tanks containing the Polyol and Isocyanate materials, respectively, are placed in the robotic cell and mixing them together. The polymer foam is then obtained. The mixture is performed in a static mixer and the compound is then sprayed on the surface of the slab or on the sub-layers as described in Figures 2 and5. The material expanses and acquires a sufficient stiffness in about six seconds. This time depends on the mix temperature and on the compound reactivity. The foam density is bounded between 35 kg/m 3 and 45 kg/m 3 . Its thermal conductivity is equal to 0.027 W.M.K and the Young modulus is equal to 7 MPa. The mean width of the wall created by foam spreading is around 72 mm obtained by adapting different parameters such as flow rates of Isocyanate and Polyol, the distance between the spraying nozzle, the speed of displacement of the nozzle. We now present the robotic architecture i.e. the CDPR which moves the platform.

Moving platform Cables

CDPR presentation

A CDPR is a robotic system composed of at least 6 cables, reeled in and out by winches, which connect together a frame and a MP. It belongs to a particular class of parallel robots where a MP is linked to a base frame using cables [START_REF] Viktor Mechtcherine | Large-scale digital concrete constructionconprint3d concept for on-site, monolithic 3dprinting[END_REF] [START_REF] Gueners | Cable behavior influence on cable-driven parallel robots vibrations: experimental characterization and simulation[END_REF]. Motors are mounted on a rigid base frame and drive winches. Cable coiled on these winches are routed through exit points located on the rigid frame to anchor points on the MP as shown in Figure 3. By controlling the cable lengths in a synchronous manner, the load can be steadily translated and rotated in a large space with a good dexterity and stability. CDPRs have a large workspace and reach high dynamic performance. Figure 3 depicts the overall architecture of the cable-suspended 3D foam printer with its main components, namely, winches, exit-points and the MP. The winches control the eight cable lengths, which move and actuate the MP. Cables are routed through exit-points located on the rigid frame and connected to anchor-points located on the MP. The printed part is located on the ground so that the MP can access to the top layer of the printed part. The extruder is controlled by an output of the controller of the CDPR. The 3D foam printer, named CRAFT and displayed in Figure 4, has the following dimensions: 2.4 m x 3.67 m x 2.76 m (l x b x h). Furthermore, it is reconfigurable and their different reconfiguration strategies were studied in [START_REF] Picard | Modeling and Robust Control of Cable-Driven Parallel Robots for Industrial Applications[END_REF], [START_REF] Zake | Stability analysis of pose-based visual servoing control of cable-driven parallel robots[END_REF], [START_REF] Marceau Métillon | A cable-driven parallel robot with full-circle end-effector rotations[END_REF].

Process identification

The maximum possible speed of the flow rate and thus the maximum axial speed of the foam spraying effector is 210 mm/s. This system provides relatively robust position control with a PID position controller and a dynamic positioning error equal to 0.05 mm. An important parameter for printing with the proposed device is the relationship between the spraying nozzle position and the foam flowrate. A model of the foam 3d printing process is expressed in terms of the nozzle height 𝑑 𝑑 , the velocity of the spray end-effector 𝑣 𝑑 and the layer height ℎ 𝑐 .

The flowrate and the geometrical dimension of the layer were found at different positions and at different deposition speed by measuring on different section of layers. Figure 6 represents the height of foam deposited during a certain Table 1 gives the measured layer height ℎ 𝑐 as a function of 𝑣 𝑑 and 𝑑 𝑑 . Figure 7 shows the relationship between the three variables ℎ 𝑐 , 𝑑 𝑑 and 𝑣 𝑑 governing the foam printing process. It is noteworthy that the printing process under study is non-linear. The foam printing process is modeled as follows:

𝑣 𝑑 = 𝑏 1 + 𝑏 2 ℎ 𝑏 3 𝑐 + 𝑏 4 𝑑 𝑑 𝑏 5 (1)
with 𝑏 1 = -1.743 , 𝑏 2 = 1.464, 𝑏 3 = -0.314, 𝑏 4 = 1.830 and 𝑏 5 = -0.054 being computed with ©Matlab fitnlm function from the values given in Tab. 1.

CDPR Control

The actuators of the CDPR are controlled using a computed torque feed-forward control scheme with a joint space feedback corrector. The integration of the model is performed inside the control law of the CDPR. The control scheme is shown in Fig. 8. The desired velocity of the spray end-effector 𝑣 𝑑 , namely the one of the moving-platform, along the deposition axis is obtained using Eq. ( 1). The desired moving-platform twist t 𝑑 is computed by projecting the deposition velocity profile onto the deposition axis. The desired moving-platform pose x 𝑑 and acceleration t 𝑑 are obtained using the desired twist first time integrative and derivative respectively. The Inverse Geometric Model (IGM) and the Inverse Kinematic Model (IKM) are used to define the desired joint position q 𝑑 and velocity q 𝑑 respectively. Using the measured joint position from the encoder readings, the joint position error e 𝑞 and the joint velocity error e 𝑞 are used to define a correction torque 𝚪 𝑃𝐼 𝐷 of the motor in a Proportional Integrative Derivative corrector. A friction compensation torque 𝚪 𝑓 is computed using a viscous and Coulomb friction model with the desired joint velocity to anticipate the actuation torque lost in friction. A feed-forward term accounts for the dynamic and static wrenches exerted on the moving-platform to determine a compensation torque 𝚪 𝐹𝐹 . The friction compensation torque, the PID corrector torque and the feed-forward torque are summed up to define the actuation torque 𝚪.

Experimental validation

First, some layers were printed using a polyurethane foam with a density of 35kg/m 3 . The first tests with the trajectory planner used in the CDPR allowed to test different parameters, in particular the printing speed versus its accuracy. The proposed framework for foam 3D printing with a CDPR is illustrated in Fig 9.

The developed measuring framework is used to improve the flatness surface and achieve the foam process through the implementation of a closed feedback loop. The principle is to keep the geometrical error along the z-axis smaller than 10 mm for one layer thickness. A first approach is to continuously vary the layer thickness proportionally to the measured 'z-error' by using the foam modeling described in Fig 7. This can be achieved either by defining the new operating parameters. It is also possible to adapt the nozzle spraying velocity 𝑣 𝑑 or to change the deposition height 𝑑 𝑑 of the nozzle online. The propose framework for foam printing with a CDPR is described in Fig. 9 for a dedicated high dimension parts to be printed. The quality of the foam to be deposited should satisfy some buildability (mechanical characteristics), geometry (shape) and flatness performance. Accordingly, the trajectory to be followed by the spraying end-effector should be defined and well followed. The first tests consisted in printing polylines in order to see the constitution of the robot to follow a straight trajectory, and to test the stacking of the layers. The CDPR well behaved at the tested speeds (up to 0.3m/s) in the sense that the trajectory remains linear along a line of 1m long and the height 𝑑 𝑑 remains constant so that the layer thickness is homogeneous at 32 mm, and the layer width is also constant at 70mm (Figure 10 a). After, this experimentation shows a good repeatability of the CDPR trajectories regardless of the printing location in the robot workspace (Figure 10c). We note some widths of non homogeneous layer section, due to the accumulation of material formed at the starting and stopping points of the lines (Figure 10 b). The second test involved a curved trajectory serving as the pattern for a curved wall. The pattern is 0.8m long and 0.25m wide with the oval shape.

The profile is set to the variables 𝑑, 𝑒 and 𝑟, which correspond to the width, length of the profile and the radius of the rounding (see Figure 9) in different frames such as : 𝑅 𝑖𝑚 𝑝 = ( 𝑂 𝑖𝑚 𝑝 , 𝑥 𝑖𝑚 𝑝 , 𝑦 𝑖𝑚 𝑝 , 𝑧 𝑖𝑚 𝑝 ): Printing frame whose origin is the first printing point. 𝑅 𝑠𝑡𝑟 = ( 𝑂 𝑠𝑡𝑟 , 𝑥 𝑠𝑡𝑟 , 𝑦 𝑠𝑡𝑟 , 𝑧 𝑠𝑡𝑟 ): Structure frame whose origin is the center of the printed structure. 𝑅 𝑝 = ( 𝑂 𝑝 , 𝑥 𝑝 , 𝑦 𝑝 , 𝑧 𝑝 ): Moving-plateform (MP) frame whose origin is the gravity center of the MP. 𝑅 𝑏 = (𝑂 𝑏 , 𝑥 𝑏 , 𝑦 𝑏 , 𝑧 𝑏 ): Base frame or Global frame related to the fixed frame.

To define the trajectory, the user defines:

-The desired time to move the effector from its initial position (at rest) to the 1st printing point The profile to be printed is defined in 𝑅 𝑖𝑚 𝑝 . The user places this local coordinate system in the structure frame 𝑅 𝑠𝑡𝑟 and then in the base reference regarding the robot 𝑅 𝑏 (Fig. 10). To print this experimentation, two types of geometries are used: lines and circular curves. The methodology used for the structure definition is the following : When the trajectory is defined, the moving platform is suspended in its initial position, which was identified by a laser-tracker. The moving platform goes from the center of the structure frame to the initial printing point. It starts with zero velocity and acceleration and arrives to point A with the desired tracking velocity 𝑣 𝑑𝑚𝑎𝑥 (0.3 m/s). The spraying effector based on the moving platform follows the predefined structure shape. There is a vertical transition from one layer to the following. The velocity norm is kept constant, along the printing phase. When printing is finished, the MP goes from the last printed point to the structure center. The path height and path width used for constructing the shape foam is 70 mm , respectively, the polyurethane foam characteristics. These dimensions were chosen based mainly on the objective to build as large an object as possible, in a reasonable amount of time and the quantity of foam in material tanks. With these dimensions, the shape wall is built in 10 layers and the flatness default is approximately 10 mm. However, other factors include a desire to maximize construction resolution and optimize foam gun performance.

Discussion on the results

A prototype system, built at Nantes University, is presented on the CRAFT platform, and data from this system shows its suitability for large-scale printing. In order to scale this out to full-size deployment there are, however, different challenges associated with workspace shape and make correction of flatness surface print are identified as targets for future research. The success of this system demonstrates the feasibility of CDPR for large additive manufacturing systems for construction field.

The proposed novel robotic solution allows the association of the CDPR with a polyarticulated robot, embedded on its mobile platform, on which various endeffectors dedicated to additive manufacturing end-effector based on expanding material, edge finishing tool, measurement/control tool would be mounted ( Fig. 12).

Many research works have been conducted on the design, modeling and control of CDPR over the last fifteen years because of their numerous potential applications, especially in industry, and their potential improved performances, especially in terms of accuracy. It should be noted that the national platform "XXL Robotics" of the Equipex + TIRREX project (Technological Infrastructure for Robotics Research of Excellence 2021-2028)1 will be located at LS2N, Nantes University, site Mlab XXL in Saint-Aignan de Grandlieu. This platform will include a reconfigurable cable-driven parallel robot of size 24m x 14m x 6m, equipped with an embedded robotic arm on its moving-platform. This experimental platform will allow us to tackle many industrial applications related to manufacturing robotics and construction robotics such as 3D printing of large parts, namely, adding material, but also removing material such as machining.

Acknowledgements

This work was supported by the ANR CRAFT project, grant ANR-18-CE10-0004, https://anr.fr/ Project-ANR-18-CE10-0004 and the EquipEx + TIR-REX project, grant ANR-21-ESRE-0015.

1https://www.ins2i.cnrs.fr/fr/cnrsinfo/ trois-nouvelles-infrastructures-equipex-pour-la-robotique-les-donnees-scientifiques-et-les

Figure 2 .

 2 Figure 2. The moving-platform with the spraying gun and its actuation system

Figure 3 .

 3 Figure 3. Schematic of a CDPR dedicated to additive foam manufacturing

Figure 4 .Figure 5 .

 45 Figure 4. A CDPR, named CRAFT, used for foam additive manufacturing at LS2N, Nantes, France

Figure 6 .

 6 Figure 6. Sectional view of printed foam samples as a function of the linear velocity 𝑣 𝑑 of the spraying end-effector for deposition distance 𝑑 𝑑 equal to 150 mm

  mm) deposition distance (mm) deposition speed (m.s -1 ) v d [m/s] d d [mm] h c [mm]

Figure 7 .

 7 Figure 7. Process modeling characterized by a response surface expressed in Eq.(1) obtained based on a non-linear multivariate regression of data given in Table.1

1

 1 Figure 7. Process modeling characterized by a response surface expressed in Eq.(1) obtained based on a non-linear multivariate regression of data given in Table.1

Figure 8 .

 8 Figure 8. Control scheme of the CDPR used for foam additive manufacturing, the trajectory planner being defined based on the foam additive model (1)

Figure 9 .

 9 Figure 9. The proposed framework for foam printing with a CDPR

Figure 10 .

 10 Figure 10. First experimentation polyline printing (a) , polyline with non homonogeneous layer section at the begin and end of layer (b), Layers overlay printing (c)

-

  The nominal speed of the effector for printing 𝑣 𝑑𝑚𝑎𝑥 (which should be kept during printing) -Number of layers to print -The height difference between two successive layers

Figure 11 .

 11 Figure 11. The shape trajectory inside the CDPR Workspace (a) , the printing shape trajectory (b)

Figure 12 .

 12 Figure 12. Construction of a foam shape part with the Craft CDPR

Figure 13 .

 13 Figure 13. (a) Concept of a new CDPR used as a foam large scale printer; (b) reconfigurable CDPR (Nantes University and IRT Jules Verne) used in TIRREX-XXL platform


Joris Guerin

Kevin Delmas

Jérémie Guiochet

Evaluation of Runtime Monitoring for UAV Emergency Landing

To certify UAV operations in populated areas, risk mitigation strategies -such as Emergency Landing (EL)must be in place to account for potential failures. EL aims at reducing ground risk by finding safe landing areas using onboard sensors. The first contribution of this paper is to present a new EL approach, in line with safety requirements introduced in recent research. In particular, the proposed EL pipeline includes mechanisms to monitor learning based components during execution. This way, another contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM) approaches within the context of a real-world critical system. A new evaluation methodology is introduced, and applied to assess the practical safety benefits of three MLRM mechanisms. The proposed approach is compared to a default mitigation strategy (open a parachute when a failure is detected), and appears to be much safer.

I. INTRODUCTION

Unmanned Aerial Vehicles (UAV) can be used in a variety of domains such as construction or structures visual inspection [START_REF] Watkins | Ten questions concerning the use of drones in urban environments[END_REF]. The Joint Authorities for Rulemaking of Unmanned Systems recently released a document called Specific Operations Risk Assessment (SORA) [START_REF]JARUS guidelines on Specific Operations Risk Assessment (SORA) v2.0, Guidelines[END_REF], providing guidelines for the development and certification of safe UAV operations. However, applying the detailed risk evaluation and mitigation procedure of the SORA does not allow to certify many useful UAV operations in urban environments [START_REF] Guérin | Certifying emergency landing for safe urban uav[END_REF]. This is mostly due to the risk of ground impact with third party in case of failure, which imposes very high integrity levels on urban UAVs. A potential solution is to introduce new ground risk mitigation strategies, such as Emergency Landing (EL). EL consists in using either external databases or on-board sensors to identify a safe landing area, and to reach it in a potentially degraded control mode. This paper deals with the former sub-problem, sometimes called EL spot detection [START_REF] Loureiro | Emergency landing spot detection algorithm for unmanned aerial vehicles[END_REF], which is simply referred to as EL from now on.

A review of recent EL approaches is presented in Section II-A, and shows that there is still no consensus about EL objectives. Recent research proposed to clearly define EL objectives for urban UAV, in the form of SORA requirements [START_REF] Guérin | Certifying emergency landing for safe urban uav[END_REF]. These requirements define the kinds of authorized areas for landing as well as the protocol for testing EL approaches. It also imposes the use of safety monitoring techniques for complex Machine Learning (ML) based components. The first contribution of this paper is to introduce a new approach to urban EL, presented in Section III, aiming to comply with such requirements. In short, we use a Convolutional Neural Network (CNN) to conduct Semantic 1 Université de Toulouse, 2 LAAS-CNRS, 3 ONERA, Toulouse, France.

joris.guerin@laas.fr, kevin.delmas@onera.fr, jeremie.guiochet@laas.fr Segmentation (SemSeg) on images collected from UAV onboard cameras. The segmentation maps are then processed to find valid landing zones. Finally, three runtime monitoring mechanisms from the literature are implemented to gain confidence in the final decisions.

Another contribution is to use EL as a case study to evaluate Machine Learning Runtime Monitoring (MLRM) techniques within the practical context of a critical autonomous system. A literature review about existing MLRM mechanisms (Section II-B) shows that their performance is rarely studied in terms of added safety at system level. Hence, a generic evaluation methodology is proposed (Section IV) to assess the safety benefits of MLRM approaches with respect to the whole system. This methodology is used to assess the quality of the proposed EL system, and especially of the three different MLRM approaches. The results obtained (Section V) represent a significant improvement compared to the default SORA mitigation strategy, i.e., using a parachute to decrease the UAV kinetic energy.

II. RELATED WORK

A. UAV Emergency Landing

This literature review focuses on EL techniques using images from on-board cameras. They can be divided into two main lines of work: 1) low-altitude approaches try to identify the right time to trigger a short range landing manoeuvre [START_REF] Chen | Robust autonomous landing of UAV in non-cooperative environments based on dynamic time camera-LiDAR fusion[END_REF], [START_REF] Lee | Landing area recognition using deep learning for unammaned aerial vehicles[END_REF], and 2) high-altitude approaches aim at finding a distant landing spot that will remain safe for a longer time range. This paper fits into the latter category, which is presented in further details. A first approach identifies areas in the image with low concentration of edges, and classifies them into building, bitumen, trees, grass or water using a Support Vector Machine algorithm [START_REF] Mejias | Classifying natural aerial scenery for autonomous aircraft emergency landing[END_REF], [START_REF] Mejias | A multi-layered approach for site detection in UAS emergency landing scenarios using geometry-based image segmentation[END_REF], [START_REF] Mejias | Forced landing technologies for unmanned aerial vehicles: towards safer operations[END_REF]. Similarly, another technique estimates depth in RGB images using a CNN, and selects flat surfaces for landing (roof-tops, roads or grass areas) [START_REF] Marcu | SafeUAV: Learning to estimate depth and safe landing areas for uavs from synthetic data[END_REF]. Finally, other approaches propose to split the image into small tiles, and to classify them into different categories. In [START_REF] Lai | A deep learning model for automatic image texture classification: Application to vision-based automatic aircraft landing[END_REF], tiles are classified into water, trees, grass, bitumen or building categories, whereas in [START_REF] Funahashi | Safety-level estimation of aerial images based on convolutional neural network for emergency landing of unmanned aerial vehicle[END_REF] and [START_REF] Klos | Ensemble transfer learning for emergency landing field identification on moderate resource heterogeneous kubernetes cluster[END_REF], image patches are manually labeled and a CNN is trained to predict their suitability for landing.

The previous paragraph illustrates that current EL research lacks well-defined, unified objectives. Although the common underlying goal in most papers is to avoid hurting people and damaging infrastructures, a formal definition of these goals is missing. For example, while transportation infrastructures are considered dangerous for landing by some papers [START_REF] Bleier | Risk assessment of flight paths for automatic emergency parachute deployment in UAVs[END_REF], others consider that flat areas, such as roads, are safe landing Fig. 1: Proposed architecture for Emergency Landing. The Core Model (CM) uses Semantic Segmentation to identify safe landing areas, while the Runtime Monitor (RM) is in charge of detecting CM errors to enhance the safety of the system. spots [START_REF] Loureiro | Emergency landing spot detection algorithm for unmanned aerial vehicles[END_REF], [START_REF] Marcu | SafeUAV: Learning to estimate depth and safe landing areas for uavs from synthetic data[END_REF]. To address this issue, a recent paper conducted a hazard analysis for urban EL, and used it to define specific requirements for EL in the style of the SORA [START_REF] Guérin | Certifying emergency landing for safe urban uav[END_REF]. This work concludes that a successful EL approach should avoid roads at all costs, and should not fall near pedestrians without a proper mitigation strategy to reduce the kinetic energy of the drone (e.g., parachute). In addition, The EL module should be tested extensively under different kinds of realistic potential threats (e.g., camera defect, meteorological conditions). Finally, runtime monitoring techniques should be implemented to ensure proper behavior of complex functions based on ML. These requirements are the guiding principles of both the design and test phases of the EL approach presented in this paper.

B. Runtime Monitoring of ML

Large ML models have introduced efficient solutions to computer vision problems that had long remained unsolved, including safety critical operations such as urban EL. However, due to their high complexity, conventional dependability techniques cannot guarantee safety [START_REF] Schwalbe | A survey on methods for the safety assurance of machine learning based systems[END_REF]. To gain confidence in these systems, new approaches are being developed to monitor ML components during execution, so as to detect uncertain behavior and trigger appropriate safety actions.

A complete literature review about MLRM mechanisms is not the purpose of this paper, but we present a short taxonomy of existing techniques to illustrate the general philosophy inspired from [START_REF] Ferreira | Benchmarking safety monitors for image classifiers with machine learning[END_REF]. A first family of approaches predicts misbehavior of a ML model by analyzing its inputs. For instance, reconstruction errors from auto encoders can be used to determine if an input is within the validity domain of the model [START_REF] Henriksson | Towards structured evaluation of deep neural network supervisors[END_REF], [START_REF] Stocco | Misbehaviour prediction for autonomous driving systems[END_REF], [START_REF] Cofer | Run-time assurance for learning-based aircraft taxiing[END_REF]. Another kind of Neural Network (NN) monitor models activation patterns of internal layers during training, and uses it to detect out-of-distribution (OOD) data [START_REF] Cheng | Runtime monitoring neuron activation patterns[END_REF]. Finally, other approaches focus on the NN outputs to detect OOD data at runtime. For example, the highest softmax activation can be used as an anomaly score [START_REF] Hendrycks | A baseline for detecting misclassified and out-of-distribution examples in neural networks[END_REF], and when possible, softmax values can even be calibrated using OOD data [START_REF] Liang | Enhancing the reliability of outof-distribution image detection in neural networks[END_REF]. We also mention that some works have implemented MLRM mechanism within the context of practical safety applications [START_REF] Cofer | Run-time assurance for learning-based aircraft taxiing[END_REF], [START_REF] Stocco | Misbehaviour prediction for autonomous driving systems[END_REF], [START_REF] Rahman | Run-time monitoring of machine learning for robotic perception: A survey of emerging trends[END_REF].

Similarly, our work studies the performance of three MLRM mechanisms within the context of a full EL architecture.

Regarding evaluation, several benchmark papers have already been published to assess the performance of different MLRM techniques. For instance, the work presented in [START_REF] Henriksson | Towards structured evaluation of deep neural network supervisors[END_REF] uses a benchmarking methodology based on three datasets, one for training the model, one for tuning the monitor and one for evaluation. Likewise, a complete benchmarking framework, relying on artificial generation of OOD data, is presented in [START_REF] Ferreira | Benchmarking safety monitors for image classifiers with machine learning[END_REF]. These approaches present the advantage of using different data sources to fit a monitor and to assess its performance. However, they do not study the behavior of monitors at system level. This paper addresses this limitation by introducing an evaluation pipeline that can be applied to critical systems embedding MLRM, allowing to measure the safety benefits of using such mechanisms.

III. EMERGENCY LANDING APPROACH

This work aims at finding safe landing zones in images collected from an on-board camera at high altitude (∼ 50m).

According to [START_REF] Guérin | Certifying emergency landing for safe urban uav[END_REF], a valid approach for safe EL is to:

1) Select an area far from busy roads.

2) Go over this area and open a parachute. Such an implementation with good assurance would guarantee that no fatality will occur. In this work, buildings are also not considered safe. Indeed, crashing into a facade can lead to catastrophic loss of control, and our approach cannot distinguish them from roofs. We note that this paper does not deal with minimizing the distance to the landing area, although this might be an interesting extension.

The proposed architecture is composed of two main modules (Figure 1). First, the Core Model (CM) processes the image to identify candidate areas and ranks them based on a custom landing quality score. Next, the candidates are validated by the Runtime Monitor (RM), supervising the ML components of the CM. In practice, if the best candidate is validated, the UAV goes to the selected area. Else, the candidate is discarded and the second best candidate goes through the monitoring step. This process continues until either a valid landing zone is found or all candidates are rejected. The outputs produced by the CM, and their associated monitoring status are illustrated in Figure 2. We underline that monitors are not fault free and can sometimes reject valid landing zones (see candidate 12 in Figure 2). The complete code for our approach is available [START_REF] Guerin | ANITI -UAV Emergency Landing[END_REF].

The scope of validity of our approach is defined by the specific settings of the UAVid dataset [START_REF] Lyu | Uavid: A semantic segmentation dataset for uav imagery[END_REF], which was used for development and testing. UAVid contains 200 images for training and 70 for validation, extracted from 27 video sequences collected in different urban environments at a height of around 50 meters. The camera angle is set to 45 degrees (oblique view) and the images are high resolution (4K). All 270 images are densely labelled for SemSeg, i.e., each pixel of each image is labelled with one of the following categories: building, road, static car, tree, low vegetation, humans, moving car, and background.

A. Core Model

Once the EL system is triggered, an image I is collected with the on-board camera. The CM processes I and identifies n candidate areas {c 1 , ..., c n }, which are ranked based on a custom landing quality score. This section presents the different steps composing the CM.

1) Semantic segmentation: The image I is first semantically segmented using a Multi-Scale Dilation network (MSDnet) trained on UAVid's training set. MSDnet is a large CNN model, which obtained the best results on UAVid [START_REF] Lyu | Uavid: A semantic segmentation dataset for uav imagery[END_REF]. Processing 4K images using MSDnet is prohibitively slow for such an emergency function (∼ 1min). Hence, the original image is resized to 1024×576, which reduces computation time significantly, while maintaining good segmentation results (visual evaluation). The HD image is used on small image patches as a MLRM mechanism (Section III-B).

2) Valid candidates identification: A pixel of I is considered valid for landing if no unsafe pixel (road, static or moving car, building) is present within a safety perimeter of radius R, dependent on UAV dimensions. To find the valid candidates, the segmentation map I seg is converted to a binary map representing the forbidden pixels I forbid . Then, the ground dimension of a given pixel I i,j is estimated using optics formulas and geometric projections (detailed computation in [START_REF] Guerin | ANITI -UAV Emergency Landing[END_REF]). To simplify computation, the ground dimension of each pixel belonging to row i is considered equal, and noted δ i . Then, the ground safety radius R can be converted in pixels for any I i,j : Ri,j = β.R/δ i , where β is a safety coefficient to account for the uncertainty on the UAV's height and camera angle. In our experiments, we use R = 2 meters and set β = 1.7, which guarantees that R > 2 meters for error rates up to 20% [START_REF] Guerin | ANITI -UAV Emergency Landing[END_REF].

In practice, to reduce processing time, I forbid is first divided into ten horizontal stripes of equal height. The two top stripes are removed as they contain the sky or unreachable areas (oblique view). For each remaining stripe, Ri,j is considered constant and is computed using the center row. This way, valid pixels are identified by simply convolving a square of side Ri,j on each stripe.

3) Representative candidates selection: The number N of candidates generated in the previous step is prohibitively high (up to tens of thousands). Thankfully, this huge list contains many neighboring pixels, representing similar areas. Hence, an algorithm is designed to retain only a small number N * (user defined) of representative candidates.

To do so, the first step is to identify independent regions of connected candidate pixels. This is done by applying DBSCAN [START_REF] Ester | A density-based algorithm for discovering clusters in large spatial databases with noise[END_REF] to the N initial candidates, represented by their XY coordinates. DBSCAN is a powerful unsupervised algorithm to find clusters with high density of data points. This step generates K clusters {C 1 , ..., C K }, and we note N k the number of candidates contained in C k .

Next, for each cluster C k , the objective is to select a number of candidate landing zones N * k proportional to its size:

N * k = max {1, round (N * × N k /N )}.
This way, each cluster has at least one representative, and k N * k ≈ N * . Next, the N * k representatives of C k are chosen so that their safe perimeters cover most of the cluster's area, while minimizing their mutual overlap. This is achieved by applying K-means clustering [START_REF] Macqueen | Some methods for classification and analysis of multivariate observations[END_REF] with N * k centroids to the data points of C k . Fitting K-means to densely connected data points generates centroids that are spread out across the region, and can thus be used as representative candidates for the cluster. Finally, if there is too much overlap between the safe perimeters of two centroids, only one of them is retained. A max overlap of 25% works well in practice to obtain a good coverage of the valid regions. The Scikit-learn implementations [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF] were used for both DBSCAN ( = 3) and K-means (default parameters).

4) Candidates ranking: The previous step generates the final set of n = k N * k candidates {c 1 , ..., c n }. Then, the candidates are ranked in increasing order based on a custom hazard function H. For a given candidate c, H is defined as:

H(c) = α × H s (c) + (1 -α) × H d (c), (1)
where H s (c) reflects the semantic categories present in c, H d (c) the distance from c to unsafe categories, and α ∈ [0, 1] is a weighting factor (α = 0.5 in our experiments, equal importance). To define H s , a hazard score s l is associated with each authorized category (the lower, the safer). Then, H s (c) is a weighted average of the pixels in the safe perimeter of c:

H s (c) = 1 max l (s l ) × l m l .s l , (2)
where m l is the proportion of pixels in c mapped to the l th category. H s (c) ranges between 0 and 1. The values used in our experiments for the s l 's are defined as follows: Within the authorized categories, trees are considered more risky as the top view does not allow to know what hides below (s tree = 3).

The background represents all objects from none of the 7 other categories. It is accepted because dangerous categories are supposed to be already removed. However, as we do not know precisely what it represents, it is given a higher score than the remaining categories (s background = 2). The presence of humans is not considered dangerous (parachute landing) (s human = 1) and low vegetation does not involve any hazard a priori (s vegetation = 0). The proposed scores are based on our understanding of the EL problem (see [START_REF] Guérin | Certifying emergency landing for safe urban uav[END_REF] for more details regarding the safety analysis of EL), however,

different
H d (c) = (d(c) -d max )/(R -d max ), if d(c) < d max 0, otherwise. (3)

B. Runtime safety monitoring strategies

Erroneous selection of a hazardous landing zone can have catastrophic consequences. Steps III-A.2 to III-A.4 were tested empirically on the ground truth label maps of the UAVid training set, and no error was detected. This preliminary study suggests that the EL pipeline is reliable, as long as the segmentation maps produced by MSDnet are correct. Hence, to gain confidence in the safety of our EL approach, three strategies are proposed to monitor the predictions of MSDnet at runtime. They are chosen because they can be easily adapted to the SemSeg setting (most existing MLRM approaches are designed for classification).

1) Local High Definition (LHD):

To gain confidence in the system without excessively increasing processing time, we use the full resolution on local patches around the monitored candidates. This way, richer information is provided to MSDnet, and the improved segmentation helps to detect unsafe categories, and reject candidates accordingly. The other two monitors are built on top of LHD.

2) Classification Hierarchy (CH):

The second MLRM mechanisms builds on the CFMEA safety analysis method [START_REF] Salay | A safety analysis method for perceptual components in automated driving[END_REF]. First, a classification hierarchy is built to reflect the risk level of different MSDnet prediction uncertainties. For example, in the hierarchy presented in Figure 3, if there is uncertainty between tree and human, the pixel is under-classified as safe, i.e. the closest common ancestor. Similarly, uncertainty between road and tree results in underclassification as any. This way, monitoring a candidate consists in under-classifying all of its pixels, and rejecting it if it contains either any or unsafe pixels. In our case, prediction uncertainty is assessed using a threshold τ on the softmax values, i.e., if several categories have softmax values greater than τ , the result is uncertain among these categories. Several values of τ are tested in our experiments.

3) Monte-Carlo Dropouts (MCD): The third monitoring technique leverages Monte-Carlo Dropouts [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF] (MCD) to improve uncertainty estimation. By maintaining dropout layers active at inference time (dropout rate ρ), MSDnet becomes non-deterministic, and running it on the same image multiple times enables to compute better statistical indicators of network uncertainty. In this work, the HD local patch around the monitored candidate is processed N MCD times by the stochastic MSDnet, and for each pixel, we note µ i and σ i the empirical mean and standard deviation of the softmax value associated with category i. Then, the following scores are computed: S MCD i = µ i +3.σ i , and they are used instead of softmax to obtain pixel classification (or under-classification when relevant). If N (µ i , σ i ) is actually a good proxy for the distribution representing MSDnet confidence, there is over 99.7% chances that the true value of the pixel's softmax score is below S MCD i . Several values of ρ and N MCD are tested in our experiments.

IV. EVALUATION PROTOCOL

This section presents a test protocol to evaluate the contributions of MLRM approaches to the safety of the whole system. It consists of three steps, inspired by FARM [START_REF] Arlat | Fault injection and dependability evaluation of fault-tolerant systems[END_REF], a methodology to evaluate fault-tolerance mechanisms.

A. Test scenarios

The first step consists in defining relevant test scenarios with respect to potential failures of the applications. Namely, MLRM mechanisms must be evaluated on data representing live operations conditions, and accounting for realistic deviation from the nominal behavior. This step is highly context dependent, and can take various forms, such as generating simulation environments or collecting/labelling datasets.

To evaluate the proposed EL architecture, we use the validation set of UAVid. Its images present flight characteristics similar to the training set (drone height, camera angle, etc.), but are extracted from different video sequences, which allows to assess generalization of our EL approach to Fig. 4: Evaluation perturbations. Examples to illustrate the synthetic image modifications used to evaluate the proposed Emergency Landing approach. The example image is from the UAVid validation set. different locations. In addition, different kinds of realistic image perturbations are generated to gauge the benefits of using MLRM mechanisms against faulty inputs. Five perturbations are selected from [START_REF] Hendrycks | Benchmarking neural network robustness to common corruptions and perturbations[END_REF] in order to represent plausible sensor defects, changes in external conditions and failures related to UAV dynamic (Figure 4).

B. Readouts

The system under test then runs on the evaluation scenarios defined above, and relevant execution variables -called readouts -are measured and stored in order to evaluate performance in the application context. To evaluate EL on a given image, the ground truth label map is used to compute the true hazard score Ĥ(c) of each candidate c:

Ĥ(c) = 1, if c ⊃ unsafe pixels, (1 -κ) × Ĥs (c), otherwise, (4)
where κ defines a gap to separate candidates containing unsafe categories from others (κ = 0.5 in our experiments), and Ĥs (c) is computed using Eq. 2 with ground-truth labels. Using this formula forces candidates containing unsafe categories (even small areas) to have high hazard scores. We also define the default action, which consists in turning off the motors and opening a parachute without moving. This degraded safety action corresponds to what would be executed in case of failure, without the EL system. It is also the action executed if no candidate is accepted by the monitor. The bottom center area of the image is used as a proxy for the default landing area (Figure 2), as the exact spot is not present in the image (UAVid uses oblique view).

In summary, the following readouts are collected while running our test procedure on a given image: 1) the rank and true risk score of each candidate, 2) the monitoring decision associated to each candidate, 3) the true risk score of the default action, 4) the monitoring times for each candidate.

C. Metrics

The readouts are then converted into metrics representing both the individual behavior of the MLRM mechanism under test, and the added safety resulting from its use.

1) Monitor's individual behavior: The role of a monitor is to assess the safety status of a candidate, i.e., safe or unsafe depending on whether it is free of unsafe categories. Hence, standard metrics from binary classification can be used to evaluate the performance of MLRM approaches. For EL monitoring, the following definitions hold: a true positive is a rejected unsafe candidate, a false positive is a rejected safe candidate, a true negative is an accepted safe candidate, and a false negative is accepted unsafe candidate. Then, the following metrics are considered (range [0, 1]):

• Precision (P): Proportion of rejected candidates that were actually unsafe. (High to avoid wasting time rejecting safe candidates). • Recall (R): Proportion of unsafe candidates that were rejected. (High to avoid catastrophic events). • FP rate: Proportion of safe candidates that were rejected. (Low to avoid discarding all safe candidates). Precision, recall and FP rate represent different dimensions under which the performance of a monitoring strategy can be evaluated. To simplify the interpretation of the results, two metrics aggregating performance scores are also reported:

• Matthews correlation coefficient (MCC): It accounts for all categories of the confusion matrix (range [-1, 1]). • F1-score: Harmonic mean of P and R (range [0, 1]).

2) Monitor's added safety: The safety added by a specific module of the EL pipeline is assessed by comparing the candidates chosen with and without this module. The difference between the two true hazard scores is called the safety gain G. If we note c 0 the default candidate, c the candidate chosen by the CM and c * the one chosen by the full pipeline, we have:

G CM = Ĥ(c) -Ĥ(c 0 ), (5)
G RM = Ĥ(c * ) -Ĥ(c), (6)
G * = Ĥ(c * ) -Ĥ(c 0 ) = G CM + G RM . (7)
These safety gains range between -1 and 1, and represent respectively the safety benefits brought by the CM, the additional benefits of using MLRM and the total benefits of the full architecture. To perform well under this metric, a MLRM mechanism should reject unsafe candidates (high recall), but should not be too conservative and reject too much safe candidates (low FP rate) or it might end up back to the default action and decrease CM results. Taking too much time to select the landing zone (low precision) can also be a source of hazard. Depending on the air traffic, it can increase the risk of collision or power failure. To evaluate monitoring time, the average overhead per image is also computed. V. RESULTS To the best of our knowledge, this work presents the first EL architecture addressing the requirements from [START_REF] Guérin | Certifying emergency landing for safe urban uav[END_REF]. Other existing approaches were not designed to actuate in densely populated areas, and comparing them with our results using our evaluation criteria would be unfair.

Regarding parameters of the different MLRM mechanisms, the following values were tested (chosen from preliminary experiments): τ ∈ {0.25, 0.125}, N M CD ∈ {10, 30} and ρ ∈ {0.5, 0.75, 0.9}. To improve readability, only results for τ = 0.25, N M CD = 10 and ρ = 0.9 are presented in Table I, as they always perform better or similar to other values. F1-scores are not reported here as they behave similarly to MCC. The complete results can be found in [START_REF] Guerin | ANITI -UAV Emergency Landing[END_REF].

We first underline that all safety gains are positive, for all perturbations. This means that, on average, the core model chooses landing zones that are better than the default action, and that all MLRM mechanisms tested improve the final landing zone selected.

Comparing the performance of the proposed MLRM mechanisms, MCD appears to have little effect on the results, while involving a huge computational overhead. This might be due to the fact that MSDnet only contains two dropout layers, which breaks the fundamental assumption that each layer is followed by dropouts [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF]. Except for brightness, it also seems that adding CH actually decreases LHD performance. Finally, we note that good monitors usually have low FP rate (except brightness). On the other hand, monitors that are too conservative (better recall but worse FP rate) tend to have lower safety gain.

VI. CONCLUSIONS

This paper presents a new approach to EL, which aims to address safety requirements identified in the recent literature. To this end, our architecture includes mechanisms to monitor ML components during execution. Hence, our work is also used as a practical use case to study the behavior of different MLRM approaches in the context of a real-world critical system. An extensive evaluation methodology is presented for generic systems embedding MLRM.

The results obtained suggest that existing MLRM mechanisms still lack robustness to ensure safety of critical ML systems. However, our work provides some insight about how to develop such technology. On the one hand, it shows that LHD alone contributes the most to improve EL safety, which suggests that successful monitors might be derived from richer information about the problem at hand. Even when some sensor information is too complex to cope with the main system implementation constraints, it might be useful for a monitor to focus on safety critical regions of the input space. On the other hand, it illustrates the importance of using system level metrics, such as the proposed safety gain, to properly assess the safety performance of MLRM mechanisms. Indeed, some monitors can perform well on individual metrics, but have limited interest for the whole system's safety (e.g., LHD+CH+MCD for shifted pixels).

ACKNOWLEDGMENT

Our work has benefited from the AI Interdisciplinary Institute ANITI. ANITI is funded by the French "Investing for the Future -PIA3" program under the Grant agreement n°ANR-19-PI3A-0004.

Fig. 2 :

 2 Fig. 2: Example of EL outputs. Circles are candidates identified by the CM, colors indicate RM status. The default action consists in opening a parachute without moving.

  implementation contexts might require different values based on specific hazard analyses. On the other hand, H d (c) is defined using d(c), the distance between the center of c and the closest forbidden pixel. If d(c) exceeds a certain threshold d max , then it is considered safe and H d (c) = 0 (we use d max = 3R). By construction, we know that d(c) ≥ R, and thus H d (c) can be brought to [0, 1] range as follows:

Fig. 3 :

 3 Fig. 3: Classification hierarchy. The classification tree for emergency Landing corresponding to UAVid categories.

TABLE I :

 I Results of the evaluation of the proposed EL system. Reported values are averaged across the entire evaluation dataset (70 images). Each perturbation is evaluated independently. Best results for each perturbation-metric pairs are in bold.

	Perturbation	G CM	Monitor	MCC	F 1	Precision Recall	FP rate	G RM	Overhead (sec)	G *
			LHD	0.711 0.814	0.813	0.816	0.105	0.144	2.12	0.646
	None	0.502	LHD+CH LHD+MCD	0.684 0.803 0.708 0.813	0.727 0.810	0.896 0.816	0.187 0.107	0.122 0.148	2.46 34.53	0.625 0.650
			LHD+CH+MCD	0.676 0.797	0.708	0.912	0.209	0.132	36.40	0.634
			LHD	0.698	0.863	0.863	0.863	0.165	0.144	0.62	0.541
	Brightness	0.396	LHD+CH LHD+MCD	0.697 0.695 0.862 0.870	0.816 0.861	0.930 0.863	0.251 0.168	0.164 0.144	0.79 37.13	0.560 0.541
			LHD+CH+MCD	0.698 0.870	0.807	0.944	0.271	0.143	33.75	0.539
			LHD	0.460 0.714	0.781	0.657	0.200	0.099	0.49	0.426
	Fog	0.327	LHD+CH LHD+MCD	0.484 0.760 0.456 0.712	0.739 0.779	0.783 0.657	0.302 0.203	0.091 0.099	0.71 28.94	0.418 0.426
			LHD+CH+MCD	0.485 0.767	0.726	0.813	0.334	0.067	30.07	0.394
			LHD	0.464	0.712	0.573	0.941	0.505	0.026	0.59	0.347
	Motion blur	0.321	LHD+CH LHD+MCD	0.469 0.712 0.464 0.712	0.564 0.573	0.966 0.941	0.537 0.505	0.016 0.026	0.64 33.35	0.337 0.347
			LHD+CH+MCD	0.466 0.711	0.563	0.966	0.541	0.008	23.83	0.329
			LHD	0.565	0.729	0.766	0.695	0.141	0.134	2.26	0.635
	Shifted pixels	0.500	LHD+CH LHD+MCD	0.544 0.738 0.565 0.730	0.680 0.764	0.808 0.699	0.254 0.143	0.053 0.124	3.12 36.37	0.553 0.625
			LHD+CH+MCD	0.543	0.739	0.672	0.821	0.267	0.053	42.23	0.553
			LHD	0.671 0.797	0.744	0.858	0.170	0.087	2.57	0.607
	Pixel trap	0.519	LHD+CH LHD+MCD	0.616 0.765 0.671 0.797	0.664 0.744	0.903 0.858	0.263 0.170	0.055 0.087	3.31 40.93	0.574 0.607
			LHD+CH+MCD	0.597 0.754	0.646	0.907	0.287	0.056	46.97	0.576