Rodolphe Dubois
email: rodolphe.dubois@protonmail.com

Alexandre Eudes
email: alexandre.eudes@onera.fr

Vincent Frémont
email: vincent.fremont@ls2n.fr

Sharing Visual-Inertial Data for Collaborative Decentralized Simultaneous Localization And Mapping

Keywords:

. Experiments showed that the proposed methods allow the agents to build, exchange and integrate consistent visual-inertial packets, and improve their trajectory estimation accuracy up to several centimeters.

Introduction

Collaborative robotics has gained much attention over the past years as it promises to make the traditional applications of mobile robotics more efficient and robust. Indeed, missions such as exploration, inspection and Search-And-Rescue (SAR) [START_REF] Queralta | Collaborative multi-robot search and rescue: Planning, coordination, perception, and active vision[END_REF] operations may benefit from the parallelization and distribution of tasks, the inter-robot mutual information derived from the fusing of their complementary observations, and the resiliency of the fleet in facing the loss of one of its agents. Such applications depend on the ability of the robots to navigate through an environment without prior knowledge on it, as aimed by Simultaneous Localization And Mapping (SLAM).

While single-robot SLAM has been extensively studied [START_REF] Cadena | Past, present, and future of simultaneous localization and mapping: Toward the robustperception age[END_REF], a current challenge is to extend it to collaborative frameworks. Collaborative SLAM (CSLAM) does not boil down to a mere extension of single-robot SLAM as it raises specific challenges to cope with. The first challenge is informational. The exchanged data should allow to spot inter-robot correspondences and consistently re-estimate the trajectories while faithfully reflecting the knowledge of the agents. Secondly, processing inter-robot interactions should not cut into the computational and memory resources required by the other processes running in parallel to the SLAM algorithm. Finally, robots should address communication issues stemming from the limited bandwidth and communication range. Despite those difficulties, CSLAM demonstrates an enhanced potential for various domestic, industrial and military applications. However, by relying on task integration in centralized architectures or task distribution in decentralized architectures, most proposed solutions undermine the data processing autonomy of the robots when it comes to fuse information from the other robots by exclusively relying on their own resources and data.

The goal of this paper is to design data sharing methods for collaborative decentralized visual-inertial SLAM, which enforce the agents' autonomy while coping with informational, communication and resource constraints. They aim at allowing each agent to process the information provided by its own sensors as well as the information received from the other agents independently, without the need for cloud computing resources. As contributions, we propose three ways of locally summarizing the visual-inertial information: condensed packets based on the computation of consistent virtual factors, pruned packets which select a subset of raw visual-inertial factors in a less conservative way (both packets being completed with raw 2D visual information), and rigid packets which convey more exhaustive visual-structural information. We also build a full collaborative SLAM architecture to organize their computation, exchange and integration. Dedicated to multi-robot navigation, those methods aim at enhancing the estimation accuracy of each robot. They were designed to be agnostic to the SLAM Front-End as long as it outputs a visual-inertial factor graph. We evaluate them on multi-robot scenarios built from the EuRoC dataset [START_REF] Burri | The EuRoC micro aerial vehicle datasets[END_REF] and our custom AirMuseum dataset [START_REF] Dubois | AirMuseum: a heterogeneous multi-robot dataset for stereo-visual and inertial Simultaneous Localization And Mapping[END_REF].

This paper extends our previous work [START_REF] Dubois | On data sharing strategy for decentralized collaborative visual-inertial simultaneous localization and mapping[END_REF] in three ways. First, it proposes multiple changes into the building of the communicated packets and their integration into the recipient robots' maps. Secondly, it brings a deeper focus into the underlying system architecture regarding the chosen communication policy and task allocation scheme. Finally, it provides a more exhaustive performance evaluation of the proposed methods in coping with the embeddability, communication and information constraints which are classically faced by multi-robot SLAM.

The paper is organized as follows. Section 2 first tackles the structure of multi-robot SLAM algorithms and reviews the main contributions which have been published in that field. Section 3 introduces the notations and mathematical notions which we will use throughout the paper. Sections 4, 5 and 6 detail the proposed common collaborative SLAM architecture regarding the retained task and data allocation scheme, the communication policy and the data association and fusion strategy. The next three sections 7, 8 and 9 propose three ways of building visual-inertial packets to be exchanged between robots, each one being introduced by a specific related work subsection. Finally, experiments are carried out and discussed in section 10.

Related works

Simultaneous Localization And Mapping

Simultaneous Localization And Mapping methods concurrently estimate a map of the observed environment and the trajectory the robot has run within it, and continuously refines them through local and global relocalization against the reconstructed map. Figure 1 depicts the general architecture of a SLAM algorithm. It first requires at least one exteroceptive sensor like a LiDAR [START_REF] Deschaud | IMLS-SLAM: scan-to-model matching based on 3d data[END_REF] or a camera bench [START_REF] Mur-Artal | ORB-SLAM: a versatile and accurate monocular SLAM system[END_REF] for visual SLAM (VSLAM), those latter being cheaper, lower power consuming and easier to embed on drones. A local odometry and mapping module interprets, abstracts and fuses such measurements to initialize locally consistent map and trajectory estimates. However, it inevitably drifts, what compromises the global consistency of the map. Hence, a correspondence detection module builds on place recognition algorithms [START_REF] Lowry | Visual place recognition: A survey[END_REF] to spot global loop closures when the robot observes previously mapped areas. Finally, a global inference module may globally and smoothly refine the estimates based on the spotted correspondences through Maximum A Posteriori (MAP) estimation. All those modules are flagged either as Front-End if they process incoming data at their reception rate, or Back-End processes otherwise.

The contributions of this article focus on Visual-Inertial (VI) SLAM, whose sensor bench enhances a monocular camera with an Inertial Measurement Unit (IMU) [START_REF] Mourikis | A multi-state constraint Kalman filter for vision-aided inertial navigation[END_REF][START_REF] Qin | Vins-mono: A robust and versatile monocular visual-inertial state estimator[END_REF]. Such coupling of low-cost sensors brings multiple advantages: not only does the IMU allow to scale the 3D model reconstructed from monocular images and align it with gravity vector, but it also improves the visual tracking, especially in the case of tight-fusion VI Odometry (VIO) [START_REF] Leutenegger | Keyframe-based visual-inertial odometry using nonlinear optimization[END_REF]. However, it requires a steady time-synchronization between the IMU and the camera, and enlarges the estimation problem with velocity and inertial biases states.

Front-End processes

Map

Trajectory

Estimates

Global inference

Probabilistic model

Back-End processes

Collaborative SLAM

As emphasized in [START_REF] Cadena | Past, present, and future of simultaneous localization and mapping: Toward the robustperception age[END_REF], single-robot SLAM has achieved high maturity and widened new research perspectives, one of which being CSLAM [START_REF] Saeedi | Multiple-robot simultaneous localization and mapping: A review[END_REF]. A CSLAM framework extends single-robot SLAM architectures along three axis: i) a task and data allocation scheme; ii) an adapted communication policy and iii) a matching and merging (or data association and fusion) strategy. The allocation scheme governs how each task and data is either centralized, decentralized or distributed. The communication policy supervises the topology, planning and content of inter-robot exchanges. Finally, the matching and merging strategy deals with the spotting of inter-robot correspondences and multi-robot inference.

Task and Data allocation scheme

A task (resp. piece of data) is centralized when a single agent is responsible for its completion (resp. storage). The latter aggregates and processes data collected by the others and eventually reports results and updates to them. On the contrary, the task is decentralized when it is performed by multiple agents. It is distributed if its computations are shared among several agents.

Centralized allocation schemes

Centralized allocation schemes rely on a central server which exhibits increased computational power and storage capacity. Though relieving the agents from complex Back-End tasks, such architectures might nonetheless undermine their data processing autonomy. Existing methods can thus be classified according to what extents the agents are integrated to the server. A first approach, coined by CoSLAM [START_REF] Zou | CoSLAM: Collaborative visual slam in dynamic environments[END_REF], reduces agents to mobile sensors and concentrates all mapping and localization tasks on the server. On the contrary, a second strategy grants more autonomy to the agents by providing them a full SLAM pipeline, while it delegates the detection of global interrobot correspondences and global inference operations to the server. Such approach has been adopted by C 2 TAM [START_REF] Riazuelo | C2tam: A cloud framework for cooperative tracking and mapping[END_REF], MOARSLAM [START_REF] Morrison | MoarSLAM: Multiple operator augmented rslam[END_REF] and CORB-SLAM [START_REF] Li | CORB-SLAM: a collaborative visual slam system for multiple robots[END_REF], which all build on client-server architectures. Finally, a third and intermediary strategy splits individual SLAM pipelines between the agents and the server but leaves critical VIO modules to agents. For instance, CSfM [START_REF] Forster | Collaborative monocular slam with multiple micro aerial vehicles[END_REF] spawns one thread handler per agent on the server which builds it associated map while a common place recognition module spots intra and inter-robot correspondences. However, CSfM does not provide any feedback to the agents, contrary to CCM-SLAM [START_REF] Schmuck | CCM-SLAM: Robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams[END_REF] and CVI-SLAM [START_REF] Karrer | CVI-SLAM: Collaborative Visual-Inertial SLAM[END_REF] for monocular and visual-inertial CSLAM. In the same way, Qin et al. proposed CoVins [START_REF] Qin | A general optimization-based framework for local odometry estimation with multiple sensors[END_REF] which extends Vins-Mono [START_REF] Qin | Vins-mono: A robust and versatile monocular visual-inertial state estimator[END_REF] into CSLAM, and Deustch et al. introduced the generic TeamSLAM [21] framework.

Decentralized allocation schemes

Decentralized allocation schemes seek increased flexibility for the fleet and enhanced autonomy for its agents. Each agent thus maintains its own map and is responsible for its full SLAM pipeline and the integration of the data received from the other agents (e.g. spotting of inter-robot correspondence and running global inference). Such approach has been implemented by several methods such as DDF-SAM [START_REF] Cunningham | DDF-SAM 2.0: Consistent distributed smoothing and mapping[END_REF], AUV-CSLAM [START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF] for underwater acoustic SLAM, Schuster et al. [START_REF] Schuster | Distributed stereo vision-based 6d localization and mapping for multi-robot teams[END_REF] for dense stereo-visual collaborative SLAM, Dubois et al. [START_REF] Dubois | On data sharing strategy for decentralized collaborative visual-inertial simultaneous localization and mapping[END_REF] and Sartipi et al. [START_REF] Sartipi | Decentralized visual-inertial localization and mapping on mobile devices for augmented reality[END_REF] for visual-inertial CSLAM. Decentralized allocation schemes are more conducive to the distribution of tasks and data which compensates for the absence of a central server. Such pooling of computational resources may target specific modules such as correspondence detection [START_REF] Cieslewski | Data-efficient decentralized visual SLAM[END_REF] and global correspondences consistency checks [START_REF] Lajoie | DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams[END_REF], distributed inference [START_REF] Lajoie | DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams[END_REF][START_REF] Choudhary | Distributed trajectory estimation with privacy and communication constraints: a two-stage distributed gauss-seidel approach[END_REF] and map storage [START_REF] Koch | Managing environment models in multirobot teams[END_REF][START_REF] Quraishi | Robustness to connectivity loss for collaborative mapping[END_REF].

Communication policies

Communication topology

The communication topology defines the set of robots each agent can communicate with. Though the adopted topology should match to the chosen allocation scheme, it still retains some degrees of freedom. In centralized topologies, some frameworks [START_REF] Zou | CoSLAM: Collaborative visual slam in dynamic environments[END_REF][START_REF] Forster | Collaborative monocular slam with multiple micro aerial vehicles[END_REF] restrict to unidirectional exchanges from the agents to the server while a second family [START_REF] Schmuck | CCM-SLAM: Robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams[END_REF][START_REF] Karrer | CVI-SLAM: Collaborative Visual-Inertial SLAM[END_REF] make the server provide feedback to the agents through bidirectional exchanges. On the opposite, decentralized topologies allow peer-to-peer exchanges. They certainly make multi-robot interactions more flexible, but also denser and unstructured. Special attention should be paid to handle communication losses and recoveries, and to prevent double-counting issues. Double-counting arises when a robot fuses received data while being unaware of its eventual correlations with its own estimates; this is likely to stem from cyclic data exchanges and result into inconsistent, overconfident and biased estimates.

Communicated packets

SLAM algorithms process three kinds of data: i) raw or pre-processed sensor outputs (like image keypoints and descriptors); ii) probabilistic models and iii) map and trajectory estimates. The communication policy dictates which of them are shared among the agents. Communication strategies can be classified according to the degree of processing involved in the exchanged data, and whether such data is communicated exhaustively or in a summarized way. In most centralized methods such as CSfM [START_REF] Forster | Collaborative monocular slam with multiple micro aerial vehicles[END_REF] and CVI-SLAM [START_REF] Karrer | CVI-SLAM: Collaborative Visual-Inertial SLAM[END_REF], agents communicate extensive information on each keyframe. It may include up to keypoints, descriptors, landmarks, inertial measurements, etc. Van Opdenbosch et al. [START_REF] Van Opdenbosch | Collaborative visual SLAM using compressed feature exchange[END_REF] proposed to communicate a selection of compressed visual binary features to share visual information. Schuster et al. [START_REF] Schuster | Distributed stereo vision-based 6d localization and mapping for multi-robot teams[END_REF] make robots exchange submaps augmented with rigid stereo-triangulated dense point clouds, pose estimates and a factor graph. A second strategy relies on marginalization and sparsification techniques, as in DDF-SAM [START_REF] Cunningham | DDF-SAM 2.0: Consistent distributed smoothing and mapping[END_REF], Lazaro et al. [START_REF] Lazaro | Multi-robot SLAM using condensed measurements[END_REF] using condensed measurements and AUV-CSLAM [START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF].

Communication planning

The communication policy schedules the sending of the packets. Two strategies have been coined in the literature. The first one is robot-centric as packets are broadcast based on intrinsic criteria such as the run distance. It ensures regular data exchanges which may occur keyframeper-keyframe as in CSfM [START_REF] Forster | Collaborative monocular slam with multiple micro aerial vehicles[END_REF] and DOOR-SLAM [START_REF] Lajoie | DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams[END_REF], or submap-per-submap as done by AUV-CSLAM [START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF]. The second strategy is altruistic as exchanged packets are generated based on the other agents' estimated knowledge. Such approach was notably investigated by Tian et al. [START_REF] Tian | Nearoptimal budgeted data exchange for distributed loop closure detection[END_REF]: preliminary exchanges of light metadata allow to identify potential inter-robot correspondences and subsequently compute the optimal exchange policy to minimize the communication cost and the induced labor division.

Matching & Merging Strategies 2.5.1. Inter-robot correspondences and loop closures

When a robot receives data from another robot, its first task is to spot eventual correspondences with its own observations. This step is critical for the mutual registration of the reference frames attached to the robots, and for the multi-robot inference as it enforces inter-robot mutual information. Correspondences may first stem from direct inter-robot observations, through marker detection [START_REF] Wang | AprilTag 2: Efficient and robust fiducial detection[END_REF] or appearance tracking [START_REF] Madhavan | Distributed cooperative outdoor multirobot localization and mapping[END_REF]. Additionally, indirect correspondences are spotted through visual-structural correspondences. Instantaneous indirect correspondences, which only compares the robots' current observations, are mostly used for collaborative localization [START_REF] Sartipi | Decentralized visual-inertial localization and mapping on mobile devices for augmented reality[END_REF][START_REF] Käslin | Collaborative localization of aerial and ground robots through elevation maps[END_REF]. Retrospective indirect correspondences may be spotted between any interrobot pair of keyframes upon the reception of data from other robots. Centralized architectures take advantage of the central server to spot such correspondences, while decentralized ones may distribute the task [START_REF] Cieslewski | Data-efficient decentralized visual SLAM[END_REF], make each robot build one visual database per other robot [START_REF] Lajoie | DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams[END_REF], or leverage preliminary metadata exchanges to circumscribe the search [START_REF] Giamou | Talk resource-efficiently to me: Optimal communication planning for distributed slam front-ends[END_REF].

Multi-robot inference

Though filtering-based multi-robot inference has first been proposed [START_REF] Howard | Multi-robot simultaneous localization and mapping using particle filters[END_REF], most CSLAM algorithms have taken advantage of the flexibility of factor graphs. They naturally extend to multi-robot frameworks, for which ones Kim et al. [START_REF] Kim | Multiple relative pose graphs for robust cooperative mapping[END_REF] proposed to use anchor nodes to handle multiple trajectories. Furthermore, factor graphs are prone to support distributed optimizations as proposed by Choudhary et al. [START_REF] Choudhary | Distributed trajectory estimation with privacy and communication constraints: a two-stage distributed gauss-seidel approach[END_REF] and DOOR-SLAM [START_REF] Lajoie | DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams[END_REF]. Decentralized methods should prevent double-counting which may compromise the consistency of the estimates. For such purpose, Cunningham et al. [START_REF] Cunningham | DDF-SAM 2.0: Consistent distributed smoothing and mapping[END_REF] coined anti-factors to subtract information while Paull et al. [START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF] send consecutive independent local packets. Multi-robot inference is also responsible for estimating the transformations between the reference frames attached to the robots. To that aim, Indelman et al. [START_REF] Indelman | Incremental distributed inference from arbitrary poses and unknown data association: Using collaborating robots to establish a common reference[END_REF] use Expectancy-Maximization (EM) optimization where latent variables are used to deactivate inconsistent inter-robot correspondences.

Autonomy in multi-robot frameworks

This literature review shows that the current trend for CSLAM to enhance efficiency and robustness is to develop either highly-integrated centralized architectures or highly-distributed decentralized frameworks. However, both approaches prevent the agents from independently fusing the information received from the other agents with the information provided by their own sensors by extensively resorting to cloud computing resources, and thus mitigate their autonomy. Indeed, they either depend on a central server whose failure may neutralize the whole fleet, or mutually depend on each other. To tackle this issue, we propose three data sharing methods for VI CSLAM which enforce the autonomy of the agents while addressing the information, network and resource constraints.

Notations

In the next sections, A t BC denotes a physical quantity t attached to C w.r.t. B expressed in A, where A, B and C are reference frames. SE n and SO n respectively denote the Special Euclidean Group and the Special Orthogonal Group of dimension n. We parameterize SE 3 using the composite Lie group SO 3 × R 3 . The operator ⊕ denotes the SE 3 pose product. We define the and operators:

δτ T 2 T 1 = log SO 3 (R 2 • R 1 ) ∨ t 2 -t 1 = δθ δp (1)
T 2 = δτ T 1 = exp SO 3 (δθ ∧ ) • R 1 δp + t 1 0 1 ( 2
)
where R (•) ∈ SO 3 and t (•) ∈ R 3 parameterize T (•) ∈ SE 3 , log SO 3 maps from SO 3 to its Lie algebra so 3 , the vee operator maps from so 3 to R 3 , and the exp SO 3 and hat operators respectively map reversely. Interested readers may refer to [START_REF] Ortega | A micro lie theory for state estimation in robotics[END_REF] for more details on operations on Lie groups. J f (θ) θ ( θ) denotes the Jacobian matrix of f (θ) w.r.t. θ evaluated at θ. Finally, the Mahalanobis norm of x ∈ R n , weighted by a symmetric positive-definite matrix A, is denoted x A .

Overview of the proposed architecture

In this paper, we propose three decentralized collaborative visual-inertial SLAM algorithms dedicated to multirobot navigation, respectively based on the exchange of condensed, pruned and rigid visual-inertial packets. We first assume that each robot estimates its own map and trajectory using a given VIO algorithm which outputs a visual-inertial factor graph. We make robots regularly exchange data packets which summarize successive, uncorrelated and local portions of their map, and append them with a selection of visual information. Such packets are used by the receiver to spot inter-robot correspondences, and estimate the trajectories with enhanced accuracy. A client-server architecture is also added: it allows the robots to optionally request additional visual information upon the spotting inter-robot correspondences in order to dynamically refine them.

Task and data allocation scheme. As illustrated in Figure 2, we use a fully decentralized and weakly distributed task and data allocation scheme: each robot is responsible for running its own SLAM algorithm and integrating the received packets into its own estimated map. Nonetheless, it yields an implicit labour division since each robot restricts to spotting correspondences against its own map.

Robot B

Each robot handles its own Front-End modules to build its map and spot loop-closures within its own trajectory. It also handles multirobot inference and reference frame anchoring. However, it only detects inter-robot correspondences against its own observations. Communication policy. The proposed decentralized communication policy, described in section 5, splits into two communication regimes. First, a regular communication policy makes robots regularly build and exchange data packets from successive uncorrelated local submaps to avoid double-counting issues. Each packet decouples the data for place recognition and trajectory estimation.

It also handles the exchange of spotted correspondences and direct inter-robot observations. A client-server architecture allows each robot to request additional visual information after spotting inter-robot correspondences. The communication policy is completed with a regularization policy which operates when two robots recover contact. Those latest get mutually anchored as soon as enough inter-robot correspondences between them are available. Upon reception, each new packet is used to spot interrobot correspondences with the receiver's past keyframes, and regularly matched against the receiver's new keyframes. Newly spotted correspondences trigger global multi-robot inference which only uses the factors attached to sent and received packets (see section 6).

Common communication policy

Regular communication policy

The regular communication policy supervises the exchange of three kinds of data: i) packets; ii) spotted correspondences and iii) requested visual frame information, as described in Figure 3. The computation and broadcasting of a new packet is triggered based on kinematic criteria, as soon as the robot estimates it has traveled a distance d ≥ d max since it computed its last packet, where d max can be set to several meters. The computed packet is then broadcast to all reachable neighbors. Each spotted correspondence is broadcast the same way. A correspondence message holds the IDs of the robots, the timestamp of their matched keyframes and the measurement itself. Sent packets and correspondences are associated with sending indexes, which are incremented for each newly broadcast packet or correspondence. Robots maintain and update a table which maps such indexes either to the correspondence's ID or a structure describing the spatial extend of the packet. Finally, when a robot spots a inter-robot correspondence, it can request additional visual information about the k neighboring frames to the matched one to spot more correspondences. As a response, the queried robot sends a selection of the keypoints and descriptors of the neighboring frames (see §5.3).

Regularization communication policy

In a decentralized framework with limited communication range, robots may occasionally lose contact between each other when covering large areas, and thus miss some valuable information broadcast in the meanwhile. When they recover contact, they should regularize their knowledge. For that purpose, we make each robot hold a reception bookkeeping. Robot i stores the index of the most recently received information from each other robot j = i. When two robots recover contact, they first exchange their current reception bookkeeping. They can then regularize their knowledge by sending to the other robot all the packets and all the correspondences with posterior indexes. For each robot, data must be sent in chronological order not to break the consistency of the reception history if the communication was to be lost again during the process.

Visual information selection

All presented methods resort to a keyframe and keypoint selection mechanisms. Given a sub-trajectory, the objective is to select a subset of the keyframes which covers the whole environment observed from that sub-trajectory, as described by Algorithm 1. As a second step, we select a Algorithm 1-Keyframe selection on a sub-trajectory Parameters n min : threshold on commonly observed landmarks; q min : threshold quotient of commonly observed landmarks; α : coefficient between 0 and 1; L i : set of observed landmarks from keyframe with index i; Input: I 1:N = {I 0 , ..., I N } the set of successive keyframes within the given sub-trajectory.

Output: S the set of selected keyframes.

S ← {I 0 , I N }; r ← 0;

for i ∈ {1, ..., N -1} do n ri ← |Lr ∩ L i |; q ri ← |Lr ∩ L i | |Lr| ; if n iN ≤ n min ou q iN ≤ q min then n iN ← |L i ∩ L N |; q iN ← |L i ∩ L N | |L N | ; if n ri ≤ α • n min ou q ri ≤ α • q min then S ← S ∪ {I i }; r ← i;
subset of the keypoints on the selected keyframes. We use a simple heuristic which involves to retain the n kp keypoints associated with the most observed landmarks, as they are likely to be associated to the most stable and informative descriptors. The track length is a good heuristic indicator of the salience of a keypoint, which depends on its distinctiveness, repeatability and detectability.

TR i B0 Ri ξB 0 TR i B1 Ri ξB 1 TR i B2 Ri ξB 2 TR i B3 Ri ξB 3 TR i B4
Ri ξB 4

Positions of the landmarks observed by robot Ri : Ri ∈L Inertial and visual factors encode stochastic constraints between pose , inertial and landmark states.

(a) Structure of a visual-inertial factor graph Robot i's reference frame Robot j's reference frame

TR i Rj TR i B0 TR i B1 TR i B2 TR i B3 TR i B4 TR j B0 TR j B1 TR j B2 TR j B3 TR j B4
The trajectories of robots i and j are estimated in their own reference frame. The pose offset between them is modeled as an anchor node [START_REF] Kim | Multiple relative pose graphs for robust cooperative mapping[END_REF] which is involved in inter-robot factors.

(b) Multi-robot factor graph

. Correspondence detection module

To spot indirect correspondences, we use the pipeline developed by Lynen et al. [START_REF] Lynen | Get out of my lab: Large-scale, real-time visualinertial localization[END_REF] and implemented in the Maplab framework [START_REF] Schneider | Maplab: An open framework for research in visual-inertial mapping and localization[END_REF]. Each spotted correspondence is broadcast to all reachable robots. When a correspondence is received from another robot, it is added to the map if both corresponding keyframes are known. Otherwise, it is buffered until missing keyframes from upcoming packets are received. When a new packet is received, the robot looks for correspondences between the received keyframes and its own keyframes. It also checks whether some buffered correspondences may be added to the map. Finally, when it adds a new keyframe to its own trajectory, it looks for loop closures with its own keyframes, as well as inter-robot correspondences with the keyframes received from the other robots. All spotted correspondences are broadcast to the other robots.

Global inference module

The trajectory is modeled as a discrete set of consecutive keyframes, each one being described by its pose T WB k ∈ SE 3 , its velocity W v WB k and its inertial biases B k b k , where W and B respectively denote the global and the body inertial frames. The environment is sparsely represented as a set of 3D landmarks i whose positions Bi l i are estimated w.r.t. their first observer keyframe B i . We encompass all those variables in the set Θ and denote their associated space H Θ . The knowledge carried by a set of measurements Z onto a set of variables Θ is modeled as a graph of factors [START_REF] Grisetti | A tutorial on graph-based SLAM[END_REF]. This is a bipartite graphical probabilistic model which encodes the likelihood p(Z|Θ) of the observations w.r.t. the estimated variables. Under the hypothesis of identically and independently distributed observations, it factors as:

p(Z|Θ) = z∈Z p(z|Θ) (3)
Each observation z ∈ Z yields a factor p(z|Θ) which appears in such decomposition. In the Gaussian case:

p(z|Θ) = N (ξ z (Θ); 0, Σ ξz ) ∝ exp - 1 2 ξ z (Θ) 2 Σ ξz ( 4
)
where N is the normal distribution, 0 is the null vector, ξ z (Θ) is the associated residual and Σ ξz its covariance matrix. Inference is performed by minimizing the resulting negative log-likelihood of the measurements:

Θ = arg max Θ∈HΘ p(Z|Θ) = arg min Θ∈HΘ z∈Z ξ z (Θ) 2 Σ ξz (5)
which yields a nonlinear least-square optimization problem over the residuals in the Gaussian case, classically solved using the Levenberg-Marquardt algorithm. In the visualinertial case, the associated factor graph has the form depicted by Figure 4a with inertial and visual factors which are defined below. In our multi-robot framework, we use the architecture advocated by Kim et al. [START_REF] Kim | Multiple relative pose graphs for robust cooperative mapping[END_REF], where each robot's baseframe is associated to a dedicated anchor node as illustrated by Figure 4b. Multi-robot inferences are triggered as soon as a minimum number of new correspondences have been spotted or if a minimum time duration has elapsed since the spotting of the latest new and unprocessed correspondence. In the next paragraphs, we detail the residuals for inertial, visual and relative pose factors.

Inertial factors

Each inertial measurements u k connecting keyframes K k and K k+1 yields an inertial residual:

ξ u k (Θ) =    T WB k+1 TWB k+1|k W v WB k+1 -W vWB k+1|k B k+1 b k+1 -B k b k    (6)
where TWB k+1|k , W vWB k+1|k are predicted using a RK4 integrated discrete-time dynamics of the robot from the pose and inertial states at time k. Inertial factors may also be derived from pre-integrated IMU factors [START_REF] Forster | On-manifold preintegration for real-time visual-inertial odometry[END_REF].

Visual factors

The observation z ij of landmark l j from keyframe K i yields a visual residual:

ξ zij (Θ) = z ij -π Bi T -1 WBi ⊕ T WBj • Bj l j (7)
where B j denotes the first observer keyframe of landmark l j . The function π Bi is the pinhole camera projection which maps a 3D point expressed in B i onto the camera image plane, and which depends on distortion, intrinsic and camera-IMU extrinsic parameters.

Relative pose factors

A relative pose factor z ij between keyframes K i and K j of robots R i and R j yields the residual:

ξ zij (Θ) = T -1 RiBi ⊕ T RiRj ⊕ T Rj Bj TBiBj ( 8
)
where TBiBj is the estimated relative pose, and T RiRj is the offset pose between the reference frames attached to robots R i and R j , associated to an anchor node [START_REF] Kim | Multiple relative pose graphs for robust cooperative mapping[END_REF].

7. Condensed Visual-Inertial Packets (CVIP)

Related works

A first strategy to communicate visual-inertial information is to summarize it into condensed packets. Marginalization techniques allow to bound the ever-increasing complexity of the SLAM inference problem by pruning variables without losing information. However, it yields a dense distribution as it induces cross-correlations between all the remaining variables. Sparsification is the process of approximating this dense distribution by a sparse one with less partial pairwise correlations yielded by a set of virtual factors and which result into the same MLE estimate. In the Gaussian case, it involves to compute the mean and covariance of the virtual factors of the new topology.

Ideally, the sparsification process should minimize the loss of information which is commonly quantified using the relative entropy i.e. Kullabck-Leibler divergence (KLD) w.r.t. the dense distribution. This first requires a judicious choice for the enforced topology. Invertible topologies such as spanning trees (e.g. Chow-Liu trees [START_REF] Chow | Approximating discrete probability distributions with dependence trees[END_REF]) allow closed-form sparsification as used for Generic Linear Constraints [START_REF] Carlevaris-Bianco | Generic node removal for factor-graph SLAM[END_REF] and Nonlinear Factor Recovery [START_REF] Mazuran | Nonlinear factor recovery for long-term SLAM[END_REF]. However, richer topologies may provide better approximations at the cost of requiring iterative minimization with the constraint of enforcing a positive-definite information matrix. For such case, Interior Point [START_REF] Eckenhoff | Decoupled, consistent node removal and edge sparsification for graph-based SLAM[END_REF], quasi-Newton [START_REF] Mazuran | Nonlinear factor recovery for long-term SLAM[END_REF] and block-coordinate descent [START_REF] Vallvé | Factor descent optimization for sparsification in graph SLAM[END_REF] methods can be used.

The sparsified distribution should ideally be consistent with the original one, meaning that it should not add artifactual information by artificially reducing the uncertainty on any subset of variables. In the Gaussian case, this yields a Loewner ordering constraint over the dense and sparsified covariance matrices, respectively denoted Σ d and Σ s : Σ d ≤ Σ s . Such constraint can be enforced with log-barriers or by truncating inconsistent eigenvalues in the spectrum of the sparsified covariance matrix resulting from the relaxed problem [START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF].

Marginalization and sparsification techniques are used in single-robot SLAM frameworks to simplify the inference problem, especially by pruning the oldest variables of optimization sliding windows [START_REF] Liu | Ice-ba: Incremental, consistent and efficient bundle adjustment for visual-inertial SLAM[END_REF]. In multi-robot frameworks, it provides a convenient means to communicate condensed packets by projecting the information over commonly observed landmarks [START_REF] Cunningham | DDF-SAM 2.0: Consistent distributed smoothing and mapping[END_REF][START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF][START_REF] Lazaro | Multi-robot SLAM using condensed measurements[END_REF]. Marginalization and sparsification steps may be coupled to minimize subsequent communication costs [START_REF] Paull | A unified resourceconstrained framework for graph SLAM[END_REF].

Packet computation

The proposed method is inspired from AUV-CSLAM [START_REF] Paull | Communicationconstrained multi-auv cooperative SLAM[END_REF] and relies on the exchange of consistently marginalized and sparsified packets, augmented with a selection of raw 2D visual information to spot inter-robot correspondences.

Local marginalization

We first perform a local visual-inertial bundle adjustment in a local gravity-aligned frame of reference L. The pose of the first keyframe is kept constant for gauge fixing. The optimized variables Θ include all the keyframe poses T and inertial states E, and the positions L of the observed landmarks. Z VI denotes the set of visual and inertial factors. The posterior distribution p(Θ|Z VI ) is locally approximated as a Gaussian distribution in the neighbourhood of the resulting MLE estimate Θ:

p(Θ|Z VI ) = N ξ Θ(Θ); 0, I Θ ZVI ( Θ) -1 (9)
where ξ Θ is the stacked residual w.r.t. the MLE estimates. I Θ ZVI ( Θ) is the observed Fisher Information Matrix (FIM):

I Θ ZVI ( Θ) = z∈ZVI J ξz Θ ( Θ) • Σ -1 ξz • J ξz Θ ( Θ) (10)
where ξ z , Σ ξz and J ξz Θ ( Θ) respectively denote the measurement residual, its covariance matrix, and its Jacobian matrix w.r.t. Θ evaluated at Θ. We marginalize p(Θ|Z VI ) to yield the following distribution on the pose variables T :

p (T |Z VI ) = N ξ T (T ); 0, I T ZVI ( T ) -1 (11)
where ξ T (T ) is the stacked absolute pose residual. The marginalized information matrix I T ZVI ( T ) is computed as the Schur's complement of the discarded variables:

I T ZVI ( T ) = I Θ ZVI ( Θ) RR - I Θ ZVI ( Θ) RM I Θ ZVI ( Θ) -1 MM I Θ ZVI ( Θ) MR ( 12
)
where the subscripts R and M respectively denote the sets of rows or columns associated to the retained and marginalized variables. The resulting distribution is equivalent to a single dense |T |-ary stacked absolute pose factor z T such that p z T |T p (T |Z VI ). The marginalization process is depicted by Figure 5a, compared to Figure 4a.

Consistent sparsification

The resulting marginalized factor p(z T |T ) is far too complex to be communicated as such. As a second step, we thus take advantage on sparsification techniques to consistently approximate it using a set Z S of uncorrelated Gaussian relative pose factors (as defined in equation ( 8) and shown in Figure 5b) between successive keyframes. Such virtual factors yield a distribution: [START_REF] Zou | CoSLAM: Collaborative visual slam in dynamic environments[END_REF] where ξ Z S (T ) is the stacked relative pose residual, and Σ ξ Z S |Z S is its block diagonal covariance matrix. The virtual factors must yield the same MLE estimate:

p(Z S |T ) = z∈Z S p (z|T ) = N ξ Z S (T ); 0, Σ ξ Z S |Z S
T = arg max T ∈H T p(Z VI |T ) = arg max T ∈H T p(Z S |T ) ( 14
)
whose locally associated Gaussian distribution is:

p(T |Z S ) = N ξ T (T ); 0, I T Z S ( T ) -1 (15)
with the following observed Fisher Information matrix:

I T Z S ( T ) = J ξ Z S T ( T ) • Σ -1 ξ Z S • J ξ Z S T ( T ) (16)
The covariance matrices of the virtual factors are found as the solution of an optimization problem which minimizes the information loss under a consistency constraint:

Σ * ξ Z S = arg min Σ ξ Z S ∈D+ D KL (p(T |Z VI ), p(T |Z S )) (17a) subjected to I T Z S ( T ) ≤ I T ZVI ( T ) (17b)
where D KL denotes the Kullback-Leibler divergence, D + is the set of block diagonal positive definite matrices which matches the desired sparsified topology. Mazuran et al. [START_REF] Mazuran | Nonlinear factor recovery for long-term SLAM[END_REF] showed that this problem, relaxed from the consistency constraint, has a closed-form solution as the Jacobian matrix J ξ Z S T ( T ) of the sparsified topology is invertible. The dense covariance matrix is first projected onto the space of the new relative pose factors Z S :

Σ ξ Z S |ZVI = J ξ Z S T ( T ) I T ZVI ( T ) -1 J ξ Z S T ( T ) (18)
The projected covariance matrix Σ ξ Z S |ZVI is then marginalized onto each new virtual factor z ∈ Z S to get its covariance matrix Σ ξz|ZVI . The transformations applied to the covariance matrices are illustrated by Figure 6. However, such factors are still correlated as each one summarizes the whole information of the original distribution.

We finally need to impose the consistency constraint (17b) from the solution of the relaxed problem. Classical methods [START_REF] Vandenberghe | Determinant maximization with linear matrix inequality constraints[END_REF] based on the solving of Semi-Definite Programming problems by interior point methods and which model the consistency constraint with log-det barriers are intractable in real-time. We therefore use a non-optimal method to enforce this constraint with less complexity. The consistency constraint can be reformulated with covariances:

I T Z S ( T ) ≤ I T ZVI ( T ) ⇔ Σ ξ Z S |Z S ≥ Σ ξ Z S |ZVI (19)
We compute the largest eigenvalue λ max of Σ ξ Z S |ZVI . This can be efficiently achieved using for instance the Lanczos algorithm [START_REF] Lanczos | An iteration method for the solution of the eigenvalue problem of linear differential and integral operators[END_REF]. Then, we iterate over each factor z ∈ Z S and perform an eigen-decomposition of its marginalized covariance matrix

Σ ξz|ZVI = Q • diag(λ z ) • Q with λ z
the vector of its eigenvalues and Q ∈ SO 6 . Two alternative truncation techniques are possible. The first one replaces by λ max each eigenvalue of λ z which is lower while the second expands λ z by a factor α ≥ 1 such that min(α • λ z ) = λ max . This preserves the relative uncertainties between the residuals. To account for the fact that each eigenvalue will indifferently contribute to the orientation and translation residuals, which exhibit different magnitudes and units, the dilatation is carried out in a normalized space, by working on the correlation matrix.

Finalizing the packet

The packet is completed by adding a selection of raw visual information following the procedure described in §5.3 and Algorithm 1. The final packet consists of the computed relative pose virtual factors and the communicated pruned visual frames. The camera model is appended to the first packet as it is required to estimate relative poses from spotted inter-robot correspondences. The computed relative pose factors are recorded in the map, and will be used for global inference along with the received factors.

Packet integration

Once the trajectory is initialized using the relative pose factors, we search for inter-robot correspondences. Interrobot correspondences are spotted by querying the received visual frames into the receiver's visual database as exposed in section §6.1. For instantaneous correspondences, the receiver's new keyframes are temporally stored into a local visual database of limited capacity, which we query with all the received visual frames when it gets full. The local visual database is then re-initialized. When a robot spots an inter-robot correspondence using a received visual frame, it can request additional visual information over the neighboring visual frames to the corresponding robot. It then receives the keypoints coordinates and descriptors associated to the highly covisible keyframes. To mitigate the communication cost of such requests, only a subset of the keypoints is communicated, using the same selection process as described in section §5.3. The dense covariance matrix is first projected onto virtual factors' space, an then marginalized onto each virtual factor's subspace.

Related works

A second strategy to synthetize visual-inertial information is to summarize the localization information through factor pruning. It seeks to approximate the original visualinertial distribution as compactly as possible without excessively compromising the reliability of the resulting MLE estimate in terms of accuracy and uncertainty. It does not face the consistency issue met by condensation techniques, and boils down to a NP-hard resource-constrained optimal subset selection problem :

Z * S = arg max Z S ⊆Z u(Θ, Z S ) s.t. c(Θ, Z S ) ≥ 0 ( 20
)
where u quantifies the utility of the set of selected factors Z S w.r.t. some interest variables Θ, and c encodes a selection constraint. The utility function should ideally promote the synergistic selection of informative factors while penalizing the joint selection of mutually redundant ones.

The information theory provides adequate tools to measure the redundancy between variables, such as mutual information [START_REF] Kretzschmar | Information-theoretic compression of pose graphs for laser-based SLAM[END_REF] and relative entropy [START_REF] Wang | Kullback-leibler divergence based graph pruning in robotic feature mapping[END_REF]. However, they result into complex metrics, which can be approached by simpler correlated heuristic, geometric [START_REF] Mur-Artal | ORB-SLAM: a versatile and accurate monocular SLAM system[END_REF] or statistical metrics [START_REF] Schmuck | CCM-SLAM: Robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams[END_REF]. Building on graph theory, Khosoussi et al. [START_REF] Khosoussi | Novel insights into the impact of graph structure on SLAM[END_REF] related the reliability of the MLE estimate to the number of spanning trees in the pose graph and designed a subsequent graph pruning strategy.

The second purpose of the utility function is to quantify how a subset Z S of factors contributes to the estimation of interest variables Θ like trajectory variables. Joint utility functions may once again benefit from informationtheoretic tools to derive landmark selection schemes which bound the localization uncertainty along the trajectory [START_REF] Choudhary | Information-based reduced landmark SLAM[END_REF]. Zhao et al. [START_REF] Zhao | Good feature selection for least squares pose optimization in VO/VSLAM[END_REF] coined a max-LogDet criteria to select landmarks which best condition the least square pose estimation problem. Finally, Dymzcyk et al. [START_REF] Dymczyk | Keep it brief: Scalable creation of compressed localization maps[END_REF] proposed an optimization framework for landmark sparsification under limited budget constraints in the context integer linear and quadratic programming.

In practice, it may be approximated as a sum of individual contributions. Landmarks are then ranked according to their information gain to spot the most informative landmarks for localization [START_REF] Cheein | Feature selection criteria for real time EKF-SLAM algorithm[END_REF] or collision-avoidance [START_REF] Mu | Two-stage focused inference for resource-constrained collision-free navigation[END_REF]. The utility of each landmark can be learnt using statistical regression models [START_REF] Dymczyk | Erasing bad memories: Agent-side summarization for long-term mapping[END_REF], based on simple criteria including tracking, reprojection and appearance. Dymzcyk et al. [START_REF] Dymczyk | The gist of maps-summarizing experience for lifelong localization[END_REF] used an iterative reduction algorithm alternating between sampling and scoring, based on the number of observer keyframes. In the context of Structure from Motion, Cao et al. [START_REF] Cao | Minimal scene descriptions from structure from motion models[END_REF] proposed a probabilistic Min-K-Cover (MKC) algorithm to ensure a probabilistic k-cover while maximizing the associated descriptor variance.

Packet computation

The proposed method resembles the previous one but relies on landmark pruning formulated as a set cover problem. It makes robots regularly exchange pruned packets. Each packet consists of raw inertial factors and selected landmark tracklines to convey the localization information on the trajectory poses, and of raw 2D visual information on selected keyframes.

Subgraph extraction

Contrary to CVIP, we consider a larger subgraph which includes some keyframes communicated in the previous packet but which have a covisibility ratio r ≥ r common with the first new keyframe, as illustrated in Figure 7. The objective is to mitigate border effects during the selection of the landmarks and enforce trackline overlapping between successive packets. Furthermore, no local optimization nor covariance extraction is needed.

Appending trajectory and inertial measurements

The first step is to append the trajectory and inertial information to the packet. For each keyframe K i , we append its current relative pose estimate T Ki-1K with its previous keyframe K i-1 . Only inertial factors posterior to the previous packet are actually appended despite of the subgraph's overlapping.

Landmark selection

Then, we select a subset of the landmarks which ideally maximizes the information on the pose of the keyframes. The first method models the selection as a set k-cover problem. It consists of computing a minimal subset of well-constrained landmarks such that each keyframe is covered with at least k landmark observations. This is a NPcomplete problem whose solution can be approximated using a greedy sampling and scoring algorithm. At each iteration, we select the landmark i which covers the most partially covered keyframes. In the case of multiple optimal landmarks, we select the one with the most observations. Each landmark is selected with its full trackline, which may result in supernumerary covers on some keyframes. Then, the selected landmark is deleted from the pool of candidates and the covering scores for keyframes and landmarks are updated. We also delete all the landmarks j which show redundancies to i which we quantify using the quantity R ij , inspired from [START_REF] Dymczyk | Keep it brief: Scalable creation of compressed localization maps[END_REF]:

R ij = 1 |K ij | k∈Kij min (d min , d(p k,i , p k,j )) d min ≤ R min (21)
where K ij denotes the set of the keyframes onto which ones they both project, and p k,• stands for their projection on the k th keyframe; d min is a threshold and normalization distance and R min ∈ [0, 1] is a user-defined threshold to characterize redundancy. The algorithm stops as soon as all keyframes are covered with at least k landmark observations or no remaining landmark covers a partially covered keyframe. We call this method PVIP-MKC. The iterative sampling and scoring method proposed by Dymczyk et al. [START_REF] Dymczyk | The gist of maps-summarizing experience for lifelong localization[END_REF] provides an alternative suitable solution for solving the above set cover problem as an integer-based optimization problem [START_REF] Dymczyk | Keep it brief: Scalable creation of compressed localization maps[END_REF]:

I * L S = arg min I L S ∈{0,1} N q • I L S s.t. A • I L S ≥ k (22)
where I L S is a binary indicator vector whose i th variable indicates whether the i th landmark is selected into L S or not. The i th coefficient of q is the inverse observer count for the i th landmark. The cover constraint is encoded by a visibility matrix A defined such that A ij = 1 if keyframe i observes landmark j and A ij = 0 otherwise. k is the target cover vector. As the optimization is already performed on the bounded extracted subgraph, no preliminary partitioning is needed. We refer to this approach as PVIP-ILP.

Finalizing the packet

The visual information is appended the same way as it was done for CVIP. The final packet includes the consecutive inertial factors, the selected landmark tracklines, as well the selected keypoints and descriptors. We might need to fuse landmarks and keypoints on the already received extra vertices. The current relative pose estimates between the consecutive keyframes are also appended to the packet. In the first packet, we also include the IMU and camera model coefficients which are needed for inference and the detection of inter-robot correspondences.

Packet integration

Upon reception, the trajectory is reconstructed from the relative pose estimates, which we then use to initialize the inertial states and triangulate the communicated landmarks. Inter-robot correspondences are spotted following the same process as CVIP. Global multi-robot inference is performed as a reduced visual-inertial bundle-adjustment over the selected landmarks. However, such inference is run as a Back-End process. To quickly impact new correspondences, we formulate a pose-graph relaxation problem which we build over the new correspondences and by enforcing priors on the current relative pose estimates along the trajectories and the previously included correspondences, and we only consider unit-covariance matrices.

Submap Visual-Inertial Packets (SVIP)

Related works

When sharing data with other robots, a third and more classical solution involves transmitting rigid submaps, holding extensive visual-structural information on the observed environment. Global consistency is then approached by mutually registering the locally consistent submaps. The sub-mapping approach has been used in decentralized collaborative SLAM: Schuster et al. [START_REF] Schuster | Distributed stereo vision-based 6d localization and mapping for multi-robot teams[END_REF] make robots exchange dense stereo-visual point clouds registered through ICP, Sartipi et al. [START_REF] Sartipi | Decentralized visual-inertial localization and mapping on mobile devices for augmented reality[END_REF] exchange and update current visualinertial submaps for collaborative localization and [START_REF] Dubois | Dense decentralized multi-robot SLAM based on locally consistent TSDF submaps[END_REF] exchange submaps associated to Truncated Signed Distance Functions for dense mapping.

Packet computation

The proposed method relies on the exchange of visualinertial submaps. In each packet, the localization information is summarized into a condensed factor graph using local marginalization and sparsification techniques as presented in §7. However, no consistency constraint was enforced on those packets. The particularity of SVIP lies in the way it communicates visual-structural information. We take advantage on the rigid point cloud associated to the submap to encode visual and structural information required for place recognition and correspondence characterization. The positions of well-constrained landmarks are expressed relatively to the body frame of the first keyframe of the submap. We associate each landmark with its most consensual (median) descriptor. Landmarks with high descriptor variance are ignored as they may be unstable or result from incorrect data association.

TLB0 L ξB0 TLB1 L ξB1 TLB2 L ξB2 TLB3 L ξB3 TLB4 L ξB4
Positions of observed landmarks B0 ˆ

∈L

Landmark median descriptors Landmark covisibility information

A SVIP packet includes virtual relative pose factors, and a rigid point cloud landmarks resulting from local optimization, along with descriptor and a landmark covisibility table as required to spot and filter correspondences. Finally, inspired by the structure of the summary maps described by Lynen et al. [START_REF] Lynen | Get out of my lab: Large-scale, real-time visualinertial localization[END_REF], we also append a landmark covisibility table: each landmark is associated with a list of the indexes of all its observer keyframes within the submap, but the coordinates of the associated 2D keypoints are not communicated. Such information is necessary to filter outlier candidate keyframes for correspondences. The final packet is represented in Figure 8. Once the submap is completed, we replace the portion of the original map with the computed submap but we keep the original frames whose 2D visual information is needed to spot correspondences.

Packet integration

The received submap is reconstructed and connected to the previously received submaps. Indirect retrospective inter-robot correspondences are spotted by querying a subset of the recipient robot's keyframes against the visual database associated to the received submap. Those query keyframes are selected along the recipient robot's trajectory using Algorithm 1. If a correspondence is found, then neighboring keyframes are also queried. Finally, the submap visual database is appended to the global visual database attached to the sender robot. When adding a new keyframe on the receiver's trajectory, it is queried against the visual databases of the other robots.

Performance evaluation

The proposed methods aim at enforcing the autonomy of the robots while meeting the requirements derived from informational, communication and embeddability constraints. While the autonomy regarding the detection of inter-robot correspondences and multi-robot inference is achieved by construction, the suitability of the proposed architectures and data exchange methods w.r.t. those constraints must be assessed.

Test scenarios

We evaluated the proposed methods on multiple multirobot scenarios. We first used the EuRoC dataset [START_REF] Burri | The EuRoC micro aerial vehicle datasets[END_REF] to build 4 multi-robot scenarios by synchronizing individual Machine Hall sequences, as detailed in table 1a. However, the EuRoC sequences were primarily designed for singlerobot SLAM, and consequently, trajectories are already self-sufficient in terms of loop closures. To properly stage the specific challenges of online collaborative SLAM, we designed the AirMuseum dataset [START_REF] Dubois | AirMuseum: a heterogeneous multi-robot dataset for stereo-visual and inertial Simultaneous Localization And Mapping[END_REF]. It consists in five heterogeneous multi-robot scenarios with aerial and terrestrial sequences whose properties are displayed in Table 1b. Their trajectories were jointly designed such that they accumulate a significant odometric drift which would be correctable online based on the temporal and spatial distribution of informative inter-robot direct and indirect correspondences. The additional objective was to evaluate the proposed methods in a heterogeneous context including terrestrial robots and test the limits of the proposed methods on more complicated scenarios than the one built from the EuRoC sequences (e.g. less correspondences, lower signal-to-noise ratio, IMU excitations and visual richness on the terrestrial sequences, lower IMU quality, etc.).

Implementation and simulation details

Simulations were carried out using the ROS middleware on an Inter R Xeon(R) W-123 CPU 3.60 GHz × 8 processor. We simulated the outputs of an elementary SLAM Front-End by using the open loop trajectory estimates of the Vins-Mono visual-inertial estimator. We then performed a KLT retracking [START_REF] Baker | Lucas-kanade 20 years on: A unifying framework[END_REF] over the trajectory and the input images. The output was then dumped into a .bag file with custom keyframe messages including visual frame information, landmark tracklines, inertial states and relative pose estimates between consecutive keyframes.

The simulation were carried out as a turn-by-turn process based on the chronological iteration of all the keyframes and direct observation ROS messages extracted from the provided .bag files as described by Algorithm 2 and Figure 9a. We implemented each robot as a ROS node with its own topic publishers and subscribers to simulate data sending and reception. Robot manage their own map using

Seq. Distance Duration Scenarios [m]

[s] 12 [START_REF] Zou | CoSLAM: Collaborative visual slam in dynamic environments[END_REF]

(b) AirMuseum multi-robot scenarios

Table 1: Properties of the multi-robot scenarios the Maplab framework [START_REF] Schneider | Maplab: An open framework for research in visual-inertial mapping and localization[END_REF], and handle dedicated modules to add new keyframes, perform detect intra and inter-robot correspondences, perform multi-robot inference, build, exchange and integrate packets and anchor reference frames. Figures 9b and9c give some example visuals of the respective simulations of the CVIP and the SVIP methods on the MH123 scenario.

The presented methods share some common parameters. The first one is the minimum traveled length min between two consecutive packets, which tunes the amount of information contained in the packets versus their computation complexity. It especially affects CVIP and SVIP since the processing requirements of the marginalization step drastically increase with the size of the packets. In the following simulation, we empirically set min = 5m as a good compromise. The second family of parameters include the covisibility ratio q min and threshold n min for the selection of the keyframes in CVIP and PVIP. The third one is the number n kp of keypoints and descriptors to retain on each selected keyframe, and which is used in the three methods. Since visual information accounts for most of the weights of the exchanged packets, those parameters should be tuned so that they allow to spot interrobot correspondences without degrading the communication cost too much. We empirically set q min = 20%, n min = 20 and n kp = 100. The last family of parameters is specific to PVIP and includes the landmark target cover k and the maximum allowed redundancy threshold R min . We set R min = 0.5. We experiment different values for k, which should be chosen large enough such that the keyframe poses are sufficiently constrained for BA. If not explicitly specified, k = 20.

Assessing embeddability constraints

The distribution of packet computation times on the EuRoC scenarios are shown in Figure 10a. We note that they range from a few tenth of seconds to a few seconds, which suits to real-time requirements. We note that SVIP and CVIP methods require more processing time than PVIP. This results from the local marginalization and sparsification process which require local inference and Fisher information matrix calculus and inversion. On the contrary, PVIP packets are exempted from any optimization step. We note that the ILP selection procedure outruns the MKC selection, what may be mostly explained by the need to first build a redundancy table of the R ij indices between pairwise covisible landmarks. Packet integration times are displayed in Figure 10b. CVIP packets are the fastest packets to be integrated, in a few tenth of milliseconds, as they just require to extend the trajectory with the exchanged relative pose factors and rebuild the visual frames. SVIP requires additional time to rebuild the landmark point-cloud. PVIP packets require more time to be integrated -but still less than a tenth of second -as they need to re-initialize the inertial states and re-triangulated the landmarks based on their tracklines. Multi-robot pose-graph inference durations are shown in Figure 11a. Once again, we note that such processes are compatible with real-time requirements as their processing times range from a tenth of second to a second. Pose-graph optimizations for CVIP and SVIP, and posegraph relaxation for PVIP, allow to quickly impact new intra and inter-robot correspondences than CVIP and PVIP. SVIP generally requires more time for pose-graph inference as it spots more correspondences. In the case of PVIP, pose-graph relaxations are completed with reduced visualinertial bundle-adjustements, which last from a few seconds to a few dozen of seconds, depending on the target cover, as shown in Figure 11b.

Assessing communication constraints

The second aspect to evaluate is the induced network load. The distribution of the weights of the computed packets are displayed in Figure 12a while Figure 12b decomposes the content of the packets. CVIP outputs the lightest packets, with a median weight around 60 kBytes, since they only include the computed relative factors, augmented with local selections of keypoints and descriptors on a subset of the keyframes and which explain 80% of the weight of the packets. SVIP and PVIP packets are much heavier, with median weights between 0.1 and 0.2 MBytes. Most of the weight of a SVIP packet is explained by its visual-structural information. Regarding PVIP, the weight of the packet split between its inertial factors and its visual frame information (which also include the tracklines of the selected landmarks). Packets are not the only items to be exchanged between the robots. Figure 14 shows the accumulation of ex-changed data over time in the three-robot scenario MH123 between the three robots. We notice that most of the induced communication load is explained by the broadcasting of the computed packets, whose bandwidth requirements scale linearly with the number of robots, apart from regularization processes. Figure 13 shows the averaged bandwidth consumption associated to each method. In all methods, the average consumed bandwidth barely exceeds 100 kB/s. We note that the broadcasting of the spotted correspondences explains a significant portion of the consumed bandwidth in SVIP, as sharing visual-structural information helps spotting more correspondences.

Assessing informational constraints

Finally, we need to evaluate how the proposed methods deal with the above-mentioned informational constraints. We first assess how well the computed packets manage to preserve the original visual-inertial information. For such purpose, we compute the Kullback-Leibler divergence between the distribution induced on the absolute pose estimates by the original visual-inertial factors Z VI and the one derived from the compute packet factors Z packet . Such results are reported in Figure 15.It shows that both PVIP MKC and ILP packets are the ones which best minimize the loss of information, with one order of magnitude of difference with CVIP and SVIP. The influence of the target cover appears explicitly.

Additionally, Figure 16 displays the distribution of the eigenvalues of the difference of the covariance matrices of the original and the packet distributions. The resulting packets are consistent if and only if all those eigenvalues are positive, and all the less conservative as they are low. Thus, even though SVIP, which implements a version of CVIP relieved from the consistency constraint, seems to better preserve the original information, it clearly appears that it double counts the original information in the resulting distribution, and thus adds artifactual information spread between correlated factors. Figure 16 also clearly shows the impact of the proposed normalization step in the sparsification process, which results into a "lower" covariance matrix difference. Finally, it unsurprisingly confirms that PVIP performs better than SVIP and CVIP not only in minimizing the information loss, but also in ensuring the consistency of the final packets. In particular, it is less conservative than CVIP.

Even though individual packets are consistent and independent from each other, inconsistencies may still occur in the global map since the correlations between the inter-robot correspondences whose underlying PnP problems involve common landmarks are ignored.

Finally, we evaluate the multi-robot estimation accuracies. Figure 17 reports the translation estimation errors for each robot in each scenario on its own trajectory in the single-robot case (red), and in the multi-robot cases for each method. Each robot also estimates the trajectories of the other robots, with very similar accuracies of their associated self-estimation. 1

Evaluation of the EuRoC scenarios

Results on the EuRoC scenario are reported in Figure 17a. As previously mentioned, EuRoC trajectories were intentionally made self-sufficient in terms in loop closures, which results into good single-robot accuracies, except on MH3 whose trajectory shows a much ampler kinematics.

First, we note that SVIP performs poorly on those scenarios, and significantly degrades the estimation accuracies with wider error distributions. This is particularly blatant in scenario MH45. Two factors may explain such poor performances. First, this may be explained by a poorer quality of the inter-robot correspondences spotted using the exchanged rigid visual-structural information, and whose relative pose factors are then expressed w.r.t. to the first vertex of the packet. This may especially happen when the matched section of the packet's point cloud relates to the packet's last keyframes. Furthermore, SVIP detects at once bunches of inter-robot correspondences and yields a resulting topology which enforces strong and sudden constraints solely on the first vertex of the spotted packet. This seems to compromise the online integration of such correspondences during the subsequent pose-graph optimization. It was notably the case on the MH45 scenario, what amplified an estimation error which could not be recovered afterwards.

Contrary to SVIP, CVIP shows encouraging results on all the scenarios as it generally allows to tighten the error distribution and reduce the average and median estimation errors. This is especially the case in scenario MH13 for Robot 3, which benefits from the integration of Robot 1's packets to reduce its median estimation error of nearly 14cm. However, Robot 1 fails to properly absorb Robot 3's estimation errors conveyed in its packets, which slightly degrades its estimation accuracy. This suggests that some additional regularization mechanisms should frame the integration of the received packets to specifically deal with prior disparities in estimation accuracy. Finally, PVIP performs better than CVIP and SVIP, as it improves the estimation accuracy for all the robots in all the EuRoC scenarios. It is notably the only methods which achieves to improve the accuracies for Robot 1 while its trajectory already benefits from several informative loop closures. Interestingly, we observe MKC and ILP versions perform similarly, and that the marginal utility of additional landmark cover is low as we achieve comparable performances with the different covers. PVIP clearly benefits from its two-level inference process with pose-graph relaxation and reduced visual-inertial Bundle Adjustement. Furthermore, the EuRoC scenarios provide an ideal framework for PVIP as it is a richly textured environment and IMU sensors are excited enough to make its biases observable during the packet integration.

Evaluation on the AirMuseum scenarios

Results on the AirMuseum scenarios are reported in Figure 17b. Such scenarios allow to evaluate the proposed methods in an heterogeneous context, and confront them with the specific challenges of terrestrial VI SLAM (i.e. more agressive kinematics with jerk movements, higher noise to vibration ratio, poorlier textured ground observations, less intra-robot loop closures, etc.). The reported single-robot estimation accuracies suggest the difficulty of those sequences to VI-SLAM.

A first observation is that SVIP performs better in improving the estimation accuracies than it did on the Eu-RoC scenarios. One reason may be that fewer inter-robot correspondences are spotted, what enforces a more favorable constraint pattern than in the EuRoC scenarios. Furthermore, the inconsistency of the packet allows to yield stronger relative pose factors, which result into less flexible trajectory estimates.

The CVIP method is the one which achieves the best performances on the terrestrial sequences, and generally allows significant gains in accuracy as well as a manifest tightening of the error distribution. However, its results on drone sequences are very contrastive. As in the Eu-RoC scenarios, the method shows difficulties in absorbing the estimation errors conveyed with the received packets, hence the need for additional regularization mechanisms during packet integration. Another observation from the terrestrial sequences is the trajectory estimation suffers from the loss of the absolute gravity information, which is entirely projected onto the 6DoF relative pose factors.

PVIP shows reasonable performances on most of the scenarios. However, it also exhibits significant failures, as in scenario 3 for robot B, and also on the drone sequences. In terrestrial sequences, this may be explained by the fact that some local portions of the trajectories may be visu- ally under-constrained, and that the inertial biases may not always be observable when re-initializing the inertial states on the reconstructed received packets. Results on the drone trajectory once again highlights the difficulty to properly absorb the estimation errors associated to the trajectories which are received and integrated within the map, without compromising the estimation accuracy on its own trajectory in the case of such disparities. PVIP has difficulties in areas with low visual quality.

Conclusion and perspectives

In this article, we aimed at providing data exchange solutions for collaborative visual-inertial SLAM which enhances the autonomy of the robots in fusing the data received from the other agents while meeting the requirements for the informational, communication and resource constraints. We proposed CVIP, PVIP and SVIP, respectively based on the exchange of condensed, pruned and rigid visual-inertial packets, and designed a common multirobot framework based on a decentralized and weakly dis-tributed task allocation scheme, a dual communication policy based on a regular and a regularization regime, and a matching and merging strategy which builds on the exchanged packets for the spotting of inter-robot correspondences and multi-robot inference.

The proposed methods were evaluated on the multirobot scenarios built from the EuRoC dataset and provided by the custom AirMuseum dataset. We showed that they allow the computation and the integration of consistent packets in real-time with a low induced bandwidth consumption in a decentralized architecture. The evaluation on the EuRoC scenarios showed that CVIP and PVIP allowed to enhance the estimation accuracy on the robot trajectories in most of the scenarios, while the SVIP method showed some limitations. The evaluation on the AirMuseum scenarios showed that they were able to positively impact the estimation accuracy on most of the terrestrial sequences, but also highlighted some limitations which draw perspectives for improvement. CVIP allows to share light packets which are easily fused within the receiver's map, but their computation is complex and conservative. PVIP provides a faster and more consistent way to broadcast the visual-inertial information which allows a more accurate two-level multi-robot inference, even though it requires more bandwidth. Finally, SVIP communicates exhaustive visual-structural information which helps to spot many inter-robot correspondences.

A perspective would be to make the integration of received packets more robust not to compromise the estimation accuracy on the other trajectories. This could be achieved prior to the multi-robot inference by adding some regularization mechanisms, for instance through a more extensive and systematic use of robust cost functions not to compromise the receiver's trajectory estimate to easily. Spotted inter-robot correspondences could also undergo a preliminary cycle analysis, as advocated by [START_REF] Estrada | Finding good cycle constraints for large scale multi-robot SLAM[END_REF] and done in DOOR-SLAM [START_REF] Lajoie | DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams[END_REF]. Finally, the tuning of the parameters could be made automatic, especially regarding the target cover of PVIP, which may adapt to the local topology of the summarized map.

Figure 1 :

 1 Figure 1: Common architecture of a SLAM algorithm

Figure 2 :

 2 Figure 2: Task and Data allocation scheme

Figure 3 :

 3 Figure 3: Communication policies

Figure 4 :

 4 Figure 4: Single and multi-robot factor graphs 6. Common Matching & Merging strategy 6.1. Correspondence detection moduleTo spot indirect correspondences, we use the pipeline developed by Lynen et al.[START_REF] Lynen | Get out of my lab: Large-scale, real-time visualinertial localization[END_REF] and implemented in the Maplab framework[START_REF] Schneider | Maplab: An open framework for research in visual-inertial mapping and localization[END_REF]. Each spotted correspondence is broadcast to all reachable robots. When a correspondence is received from another robot, it is added to the map if both corresponding keyframes are known. Otherwise, it is buffered until missing keyframes from upcoming packets are received. When a new packet is received, the robot looks for correspondences between the received keyframes and its own keyframes. It also checks whether some buffered correspondences may be added to the map. Finally, when it adds a new keyframe to its own trajectory, it looks for loop closures with its own keyframes, as well as inter-robot correspondences with the keyframes received from the other robots. All spotted correspondences are broadcast to the other robots.

  Positions of observed landmarks B0 ˆ ∈L Absolute pose factor p(z T |T ) (a) Marginalized graph: marginalization yields an equivalent nary factor z T over the remaining pose variables T encoding the likelihood p(T |Z VI ).

  Sparsified graph: sparsification approximates the dense distribution p(T |Z VI ) as p(T |Z S ) ∝ zs∈Z S p(zs|T ).

Figure 5 :

 5 Figure 5: Marginalization and Sparsification processes

Figure 6 :

 6 Figure 6: Solving the relaxed problem 8. Pruned Visual-Inertial Packets (PVIP)

  observed landmarks { Ri ˆ } ∈L Covisibility ≥ x%

Figure 7 :

 7 Figure 7: Structure of the underlying visual-inertial factor graph of a PVIP packet, to be compared with Figure 4a

Figure 8 :

 8 Figure 8: Content of a SVIP packet, compared with Figure 4a

  Multi-robot simulation architecture (b) CVIP simulated on MH123 (c) SVIP simulated on MH123

Figure 9 :

 9 Figure 9: Simulation architecture and visuals

Algorithm 2 -

 2 Multi-robot turn-by-turn simulation foreach msg in sorted messages do update clock(msg); robot ← robots[msg→robot id] if msg.type = keyframe then Process subscribers' callbacks queues robot.add received packets(); robot.add received correspondences(); robot.process frame requests and responses(); Add the new keyframe robot.add new keyframe to the map(msg); robot.compute new packet if required(); robot.detect intra robot loop closures(); robot.detect inter robot correspondences(); robot.anchor reference frames if possible(); robot.perform inference if required(); else if msg.type = direct correspondence then robot.direct correspondence topic.publish(msg); Run the turn of the robots which have stayed inactive more than a given time period;

  for PVIP-ILP and PVIP-MKC without significant differences w.r.t. the target cover.

Figure 10 :

 10 Figure 10: Distribution of packet computation and integration times

40

 40 (a) Multi-robot pose-graph optimization times (s) denotes the pose-graph relaxation introduced in §8.(b) Multi-robot reduced BA optimization times (s)Processing times are displayed for different target cover k.

Figure 11 :

 11 Figure 11: Distribution of processing times

Figure 12 :

 12 Figure 12: Packet sizes and decomposition

Figure 13 :

 13 Figure 13: Average bandwidth consumption in MH123.

Figure 14 : 10 PVIP-MKC 20 PVIP-MKC 30 PVIP-MKC 40 PVIP-ILP 10 PVIP-ILP 20 PVIP-ILP 30 PVIP-ILP 40 Kullback-Figure 15 :Figure 16 :

 1410203040102030401516 Figure 14: Cumulated exchanges (MBytes) timelines (s) in scenario MH123

Figure 17 :

 17 Figure 17: Distribution of the translation errors on the estimated trajectories

  Multi-robot scenarios built from the EuRoC dataset are given names of the form MH{XYZ} according to the involved sequences. For instance, MH12 denotes the scenario which consists of sequences MH1 and MH2.

				45 123
	MH1	80.6	182
	MH2	73.5	150
	MH3	130.9	132
	MH4	91.7	99
	MH5	97.6	111
	Scenarios		#1 #2 #3 #4 #5
	Duration	[s] 445 387 304 316 304
	Trajectory length
	Robot A	[m] 229 116 147 182 141
	Robot B	[m] 225 141 143 120 178
	Robot C	[m] 191 162 202 222 131
	Drone	[m] -	-203 215 204

(a) Multi-robot scenarios built from EuRoC

[START_REF] Burri | The EuRoC micro aerial vehicle datasets[END_REF]

Robots A, B and C are terrestrial Wifibots TM and the drone is a DJI-M100 TM . All robots carry a stereo camera bench and an IMU.

Note that, as described in section §10.2, the reported performances for single-robot SLAM are those of the SLAM algorithm whose Front-End builds on the the open-loop trajectory estimated by Vins-Mono, and whose Back-End builds on the loop-closure and optimization tools provided by Maplab[START_REF] Schneider | Maplab: An open framework for research in visual-inertial mapping and localization[END_REF]. We advocate it allows a fairer comparison between the mono and multi-robot scenarios which thus share the very same underlying SLAM algorithm. The performances of the actual Vins-Mono algorithms on the EuRoC and AirMuseum sequences are respectively reported in[START_REF] Qin | Vins-mono: A robust and versatile monocular visual-inertial state estimator[END_REF] and[START_REF] Dubois | AirMuseum: a heterogeneous multi-robot dataset for stereo-visual and inertial Simultaneous Localization And Mapping[END_REF].

Acknowledgements

This work was supported by the Direction Générale de l'Armement (DGA).


Statistical approach for the evaluation of an effective conductivity in a complex solid

Contact

lea.penazzi @mines-albi.fr J E M P 2 0 1 8 -2 2 6 4 3 9

(1)

(2)

(3) (4) (5) (6)

Methodology

Statistical method of Monte Carlo to solve combined conductive-radiative heat transfer in the solid phase of a porous medium.

Model

Solid spheres at the stationnary state (5)

∇ 2 T(x) = 0 (
+ A R,top ( ϕ π h + T R (Y R , Ω R ))
Linearized solid/void boundary condition

T(y) = p S T(y -δ b ) + p R T R (y) (6)
Test Case Temperature profile T m (z) of the bed of spheres :

T m (z) = 1 n n i=0 T i (x) (7)
Continuity of flux at the solid/void boundary :

λ e f f = ϕ dT m dz (8)
NB : λ e f f computation is not complete yet, the aim is hence to present the method.

Conclusion

A single Monte Carlo algorithm to solve combined conductive-radiative problem in a complex solid.

Realizations of the algorithm can be interpreted as a succession of conductive-radiative paths.

Perspectives :

-Validation of the algorithm -Computation of the temperature as a function of the effective conductivity (Symbolic Monte Carlo)

Notations (order of appearance) : x position in the solid ; y position at the solid/void boundary ; ω unit vector indicating the radiation direction ; I spectrally integrated specific intensity ; λ solid thermal conductivity ; n lunit vector normal to the surface at y ; emissivity ; σ Stefan-Boltzmann constant ; φ radiative net-flux density imposed at the surface of the bed of spheres.

)

  boundary condition : conduction/radiation -λ∇T .n = σT 4 (3) -∫ ∂H (n) ω .nI(y, -ω )dω Statistical formulation Solid spheres (with v i = (-1) i e i+1 2 + δv i ) (4) Discretization of space on an orthonormal basis (x, e 1 , e 2 , e 3 ) Vacuum T R = E(A R,bot T min + A R,C T(Y R ))

Key References R. Fournier & al.,

	Radiative, conductive
	and convective
	heat-transfers in a single
	Monte Carlo algorithm,
	Journal of Physics,
	(2016).
	C. Caliot, & al.,
	Combined
	conductive-radiative heat
	transfer analysis in
	complex geometry using
	the Monte Carlo method
	(2017).
	N. Sakatani, & al.,
	Compressional stress
	effect on thermal
	conductivity of powdered
	materials :
	Measurements and their
	implication to lunar
	regolith (2016).