Adrian Bojko
email: adrian.bojko@cea.fr

Romain Dupont
email: romain.dupont@cea.fr

Mohamed Tamaazousti
email: mohamed.tamaazousti@cea.fr

Herve Le Borgne
email: herve.le-borgne@cea.fr

Hervé Le Borgne

Learning to segment dynamic objects using SLAM outliers

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

I. INTRODUCTION

Simultaneous Localization and Mapping (SLAM) algorithms [START_REF] Mouragnon | Real time localization and 3d reconstruction[END_REF], [START_REF] Klein | Parallel Tracking and Mapping for Small AR Workspaces[END_REF] are frequently used in autonomous vehicles [START_REF] Rosinol | Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios[END_REF], augmented reality [START_REF] Besbes | An interactive augmented reality system: a prototype for industrial maintenance training applications[END_REF], [START_REF] Gay-Bellile | A mobile markerless augmented reality system for the automotive field[END_REF], [START_REF] Tamaazousti | The constrained slam framework for non-instrumented augmented reality[END_REF] and robotics [START_REF] Cheng | An Accurate Localization Scheme for Mobile Robots Using Optical Flow in Dynamic Environments[END_REF]. A SLAM localizes the camera in a static environment while reconstructing it [START_REF] Cadena | Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age[END_REF]. Visual SLAMs rely on image features and assume that the camera moves in a static environment [START_REF] Hartley | Multiple View Geometry in Computer Vision[END_REF] (static world assumption), although it is rarely met in practice.

An extreme case where the static world assumption is not true is the motion consensus inversion, which we define as a situation where the SLAM relies more on dynamic objects than on the static environment. This may happen when an object close to the camera moves. The SLAM effectively uses a frame of reference that is not the ground thus fails. Some works give examples of such situations [START_REF] Schorghuber | Slamantic -leveraging semantics to improve vslam in dynamic environments[END_REF], [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF] without further studying the problem. In this article, we specifically address this issue that is of high interest in practice.

Dynamic SLAMs aim to reduce the impact of dynamic objects by explicitly processing them. Most Dynamic SLAMs are constructed by improving an existing SLAM through different approaches. Geometry-based approaches detect motion at runtime [START_REF] Cheng | Improving monocular visual SLAM in dynamic environments: an optical-flow-based approach[END_REF] using geometrical algorithms, e.g., optical flow. These approaches compute the dominant motion of the image and may fail if there are motion consensus inversions.

Learning-based approaches use semantic masks [START_REF] Kaneko | Mask-SLAM: Robust Feature-Based Monocular SLAM by Masking Using Semantic Segmentation[END_REF]. Classes are defined at annotation time (before training), and any class that is not annotated is not recognized at runtime. Hybrid approaches try to compensate the limits of one approach with the advantages of the other. The first type combines geometrical and learned approaches at runtime [START_REF] Bescos | Dy-naSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes[END_REF], Fig. 1. Steps of our method. We only need one monocular sequence per dynamic object to make the SLAM robust and prevent major failures due to dynamic objects. We train one network for our Consensus Inversion dataset and one for TUM RGB-D.

but our experiments show that it does not prevent consensus inversions. The second type uses geometry during training and semantic segmentation at runtime. An example is [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF]: although their approach reduces the risk of consensus inversion, it needs training sequences recording the same location at different times, does not detect objects that do not move between training sequences and requires a full stereo camera + LIDAR setup.

Similarly to [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF], we use geometrical information (inliers and outliers) to localize dynamic objects and construct their masks, which we use to train a network. We integrate the trained network in an existing SLAM to make it Dynamic. However, our approach only needs one monocular sequence per dynamic object, while [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF] requires at least two and up to eight in practice. Moreover, contrary to [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF], we do not need stereo nor depth information, making our approach easier to implement and put into practice.

We use our method to make ORB-SLAM 2 [START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF] (Monoc-ular, Stereo and RGB-D) and LDSO [START_REF] Gao | LDSO: Direct Sparse Odometry with Loop Closure[END_REF] robust to dynamic objects without any manual annotations. In addition to our method, we created the dataset Consensus Inversion and created two new metrics, Penalized ATE RMSE and Success Rate, to quantify the robustness of Dynamic SLAMs.

II. RELATED WORK

Given a geometric SLAM algorithm, making it robust to dynamic objects is relevant to real-world applications since it generalizes the SLAM from static to dynamic environments. Keeping the original, non-learned modules of an existing SLAM is valuable as it limits generalization issues encountered by learned algorithms, e.g., end-to-end SLAMs [START_REF] Wang | End-toend, sequence-to-sequence probabilistic visual odometry through deep neural networks[END_REF]. For these reasons, we focus on Dynamic SLAMs that keep the original SLAM modules.

Such Dynamic SLAMs include a dynamic object masking module in their pipeline. Most have the masking module placed right after feature computation: they either remove features on dynamic objects or process them separately. Rarely, some Dynamic SLAMs start by masking dynamic objects then detect features only in unmasked areas of the image [START_REF] Kaneko | Mask-SLAM: Robust Feature-Based Monocular SLAM by Masking Using Semantic Segmentation[END_REF].

A. Geometry-based approaches

Geometry-based approaches do not use any kind of machine learning. [START_REF] Saputra | Visual SLAM and Structure from Motion in Dynamic Environments: A Survey[END_REF] surveys many non-learned Dynamic SLAMs and concludes with "handling missing, noisy, and outlier data remains a future challenge for most of the discussed techniques [...] Most techniques also have difficulty in dealing with degenerate and dependent motion.". Optical flow approaches [START_REF] Cheng | An Accurate Localization Scheme for Mobile Robots Using Optical Flow in Dynamic Environments[END_REF] compute pixel displacements between frames but may not work if dynamic objects occupy most of the scene or have an erratic motion. Depth maps approaches [START_REF] Sun | Improving RGB-D SLAM in dynamic environments: A motion removal approach[END_REF] use the additional depth information to identify salient objects but are limited by sensor range and resolution. Clustering/backgroundforeground approaches [START_REF] Li | RGB-D SLAM in Dynamic Environments Using Static Point Weighting[END_REF], [START_REF] Sun | Motion removal for reliable RGB-D SLAM in dynamic environments[END_REF] identify dynamic objects by grouping and assigning probabilities to points with similar motions but have high computational costs and do not work well with noisy or degenerate motions [START_REF] Saputra | Visual SLAM and Structure from Motion in Dynamic Environments: A Survey[END_REF]. Our experiments show that geometrical approaches do not completely prevent consensus inversions.

B. Learning-based approaches that use semantic masks

Learning-based approaches that use semantic masks [START_REF] Kaneko | Mask-SLAM: Robust Feature-Based Monocular SLAM by Masking Using Semantic Segmentation[END_REF], [START_REF] Wang | A Unified Framework for Mutual Improvement of SLAM and Semantic Segmentation[END_REF] remove features on dynamic objects at runtime according to their class (e.g. people and cars are usually masked) but are limited by the availability and scope of training data, not working with unknown objects [START_REF] Yu | DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments[END_REF].

C. Hybrid approaches

Hybrid approaches are divided in two types. The first type combines geometrical and learned algorithms at runtime [START_REF] Bescos | Dy-naSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes[END_REF], [START_REF] Brasch | Semantic Monocular SLAM for Highly Dynamic Environments[END_REF], [START_REF] Li | Stereo Vision-Based Semantic 3D Object and Ego-Motion Tracking for Autonomous Driving[END_REF], [START_REF] Schorghuber | Slamantic -leveraging semantics to improve vslam in dynamic environments[END_REF], [START_REF] Yu | DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments[END_REF], leveraging strong points while compensating weaknesses (as optical flow + semantics or depth map + semantics). However, there are still failure points common to the underlying algorithms, e.g., an object of unknown class causing a consensus inversion.

The second type uses geometrical approaches during training and learned approaches at runtime [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF]. During training, geometrical approaches are used to generate the masks of dynamic objects, which are then learned. At runtime it is the same as learning-based approaches. [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF] learns the appearance of dynamic objects by comparing 3D reconstructions over time but requires several traversals of the same environment at different times and a full stereo camera + LIDAR setup during training. Our approach is like [START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF] but we require only one monocular sequence per object, without needing additional traversals or sensors.

III. LEARNING TO SEGMENT DYNAMIC OBJECTS A. Overview

We outline in this section our approach, illustrated in Fig. 2, to generalize a feature-based SLAM into a Dynamic SLAM. Our goal is to protect the SLAM against the negative effects of dynamic objects in a given environment. We achieve this by training a segmentation network with two classes: static and dynamic, the latter being masked during the execution of the SLAM. Having detailed classes as car or person is unneeded since we always mask dynamic objects. We show that unconditional masking is more robust than geometry, which fails under consensus inversions. Among mask-based approaches, ours is the only one with automatic annotation and very low data requirements (one sequence per object).

SLAMs usually reject non-static features with methods as RANSAC [START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF]. We make the hypothesis that, if there is no motion consensus inversion, the sudden apparition of dense clusters of outliers characterizes violations of the static world assumption by dynamic objects. Thus, if dense clusters of outliers suddenly replace inliers it means that the inliers that became outliers were in fact dynamic features, indicating that there is a whole object violating the static world assumption rather than just isolated features.

Given a SLAM, we define example sequences as sequences that respect our hypothesis, i.e., make the SLAM generate clusters of outliers on dynamic objects when they move. In practice, example sequences are sequences in which dynamic objects are reconstructed by the SLAM and do not cause motion consensus inversions, e.g., a sitting person that stands up a couple meters away from the camera.

The steps of our approach are: 1) Outlier and inlier preprocessing: we use the SLAM to generate outliers and inliers. For non-deterministic SLAMs, we add a filtering step. 2) Mask creation and network training: we use the inliers and outliers to create the masks of dynamic objects. Then we use the masks to train a network. 3) SLAM Integration: we integrate the trained network right after the feature detection step of the SLAM.

B. Outlier and inlier preprocessing

Once initialized, a feature-based SLAM algorithm computes camera poses for each frame in roughly three steps:

1) 2D keypoint detection.

Fig. 2. Overview of our approach. We collect inliers and outliers from example sequences and use them to create masks of dynamic objects. We train a semantic segmentation network with the created masks and integrate it in the SLAM after the keypoint detection step. At runtime we infer the masks of any sequence and remove all features on dynamic objects.

2) 2D-3D matching between detected keypoints and known 3D map points + triangulation of new 3D map points. 3) Bundle adjustment: robust optimization of 2D-3D matches and camera poses. We save the coordinates of outliers and inliers of each frame right after the bundle adjustment: outliers are keypoints whose 2D-3D match was rejected, and inliers those whose 2D-3D match was not rejected. SLAMs may be non-deterministic due to multithreading or random functions (e.g,. RANSAC). So, we save inliers and outliers coordinates over several runs and merge them, filtering rarely observed coordinates as they tend to create spurious clusters when merged.

C. Mask creation and network training

This step is divided in mask creation network training. 1) Mask creation: this step consists in localizing dynamic objects with sliding windows, refining the windows into masks, and propagating the masks to the whole sequence.

Localizing dynamic objects with sliding windows: on every image of every sequence we use rectangular sliding windows of different sizes, at a stride, to evaluate how the inlier/outlier ratio changes.

Let w be a window on image p and w its corresponding window on image p . Then the outlier score S is: S = outlier density of w inlier density of w / outlier density of w inlier density of w

When a mapped object moves, many inliers become outliers, making the ratio S drop between consecutive frames. We consider that w contains a dynamic object if S is less than a threshold S max , set by the user.

We roughly compensate camera motion with the homography H = K.dR p,p .K -1 where K is the camera intrinsic matrix and dR p,p is the relative rotation between w and w . We apply H on window w to have both w and w match the same physical location. This approximation is easy to compute using a trajectory generated by the SLAM and was accurate enough in our experiments.

Refining sliding windows into single masks: we merge all bounding boxes that overlap on the same image. Then we project each merged bounding box on the past and future k frames and create image sequences with the content of these projected bounding boxes. The created sequences are a perfect fit for Unsupervised Video Object Segmentation (UVOS, methods that automatically segment salient/dynamic objects in videos) as there is no ambiguity on which object to segment. For each created sequence we apply the author's implementation of COSNet [START_REF] Lu | See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks[END_REF] (a State-of-the-Art UVOS network) on the central images, thus masking the dynamic objects inside the sliding windows.

Propagating single masks: now that we have a single accurate binary mask for every dynamic object, we can propagate them to past and future frames using semi-supervised video object segmentation. These methods track dynamic objects in videos but require very accurate initial guesses which we have thanks to the previous step. We apply the author's implementation of SiamMask [START_REF] Wang | Fast Online Object Tracking and Segmentation: A Unifying Approach[END_REF] (a State-of-the-Art network that is both lightweight and class-agnostic) towards past and future frames. The result is a set of binary masks, covering the whole sequence, for each dynamic object.

2) Network training: Our goal is to train a semantic network able to mask all dynamic objects simultaneously. First, we train one instance of DeepLabv3+ 1 [START_REF] Chen | Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[END_REF] (a State-of-the-Art semantic segmentation network) for every set of masks generated at the previous step. Then, we infer masks for every sequence and for every trained network. We superimpose the masks inferred on the same sequence and use all superimposed masks to train a final instance of DeepLabv3+. The computed model can be used to mask all dynamic objects of all sequences simultaneously.

D. SLAM Integration

We integrate the final DeepLabv3+ trained network after the feature detection module in the SLAM. We infer the mask of dynamic objects from the current image, then filter all features whose coordinates are on masked areas.

IV. CONSENSUS INVERSION DATASET AND NEW METRICS

To the best of our knowledge, there is no dataset nor appropriate metrics to test the robustness of SLAMs with respect to motion consensus inversions. Hence, we created the Consensus Inversion dataset, illustrated in Fig. 3, made of two subsets: Dynamic and Static. We used a MYNT EYE D1000-120 stereo camera at 1280x720 / 30Hz. IMU data is recorded but not used. All sequences are about 500 to 1000 images long.

A. Consensus Inversion / Dynamic

The camera is dynamic in the first subset. It includes two sequences with static objects (book/notes) and twelve with dynamic objects (dragon/dromedary/car, each object moves in three sequences). This subset covers different situations:

• Static: a static object close to the camera. Tests oversegmentation, i.e., if the SLAM masks a static object and fails for this reason (all image features are masked). • Easy: an object moves but does not cause consensus inversion. Tests standard SLAM robustness. • Hard: an object moves and causes motion consensus inversion. Tests SLAM robustness to consensus inversions. • Very hard: an object moves rigidly with the camera while very close to it. Tests the robustness to consensus inversion when detecting object motion is extremely difficult. We computed the ground truth and the ground truth tracking rate (% of tracked frames) using ORB-SLAM 2 stereo improved with our method and with the early stopping of bundle adjustment removed. We also computed the ground truth tracking rate in monocular mode.

B. Consensus Inversion / Static

The camera is static in the second subset. An object close to the camera starts moving and causes a consensus inversion: the SLAM must, however, not compute any motion. We made five sequences per dynamic object.

C. Metrics: Penalized ATE RMSE and Success Rate

The standard metrics to test SLAMs are the ATE RMSE (Absolute Trajectory Error) and the tracking rate (% of tracked frames). However, if tracking rates are too different the comparison of ATE RMSEs is biased: a SLAM that stops early might skip tricky parts of the sequence. Hence, we defined the Penalized ATE RMSE and Success Rate.

We consider that an ATE RMSE is invalid if: 1) it is unknown (e.g. when using reported results) or 2) the tracking rate is lower than r gt -∆ r,max , where ∆ r,max is a fixed threshold and r gt is the ground truth tracking rate (i.e. the expected % of tracked frames).

The Penalized ATE RMSE is computed in relation to other SLAMs in case of invalidation. With τ the penalty factor and L the set of all valid ATE RMSEs computed by other SLAMs on the tested sequence, we define the Penalized ATE RMSE in Eq. 2:

Penalized ATE RMSE max(L).(1 + τ ), if ATE RMSE is invalid. ATE RMSE, otherwise.
(2) Conversely, we consider that a SLAM successfully processes a sequence if all these conditions are respected: 1) The ATE RMSE is known 2) The ATE RMSE is below a fixed threshold l max 3) The tracking rate is at least r gt -∆ r,max . The Success Rate of a SLAM on a dataset is the ratio of sequences that the SLAM successfully processes.

V. EXPERIMENTS

A. Experimental setup 1) Datasets and parameters: we evaluate our method on the TUM RGB-D dataset [START_REF] Sturm | A benchmark for the evaluation of RGB-D SLAM systems[END_REF] (dynamic sequences) and on the Consensus Inversion dataset. The purpose of the TUM RGB-D dataset is the evaluation of RGB-D SLAM systems; it was recorded using a Microsoft Kinect and the ground truth camera poses was obtained from a motion capture system. The sequences contain color and depth images at 640x480/30Hz. The dynamic sequences are a set of eight sequences that record people moving.

We use ORB-SLAM 2 [START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF] as the core SLAM algorithm for the main experiments and LDSO [START_REF] Gao | LDSO: Direct Sparse Odometry with Loop Closure[END_REF] specifically to test the extension to a Direct SLAM. We set the feature number to 3000 and use default/author settings otherwise. We use our method to train one network on the TUM RGB-D dataset (we use the sequences fr3 sitting static and fr3 walking static) Fig. 4. Consensus inversion in fr3 walking xyz (TUM RGB-D). Camera pose in blue and 3D map in red. Left: no masks, the camera trajectory is nonsensical as the SLAM uses features on moving people. Right: we apply masks using our method, the SLAM trajectory is coherent with the real motion. We empirically found suitable parameters: sliding windows of size 100x100 / 200x200 / 300x300 / 400x400 with a stride of 50px, a difference of 3 images to compute outliers scores, an interval k = 15 images (1s at 30Hz) for the unsupervised VOS and a max outlier score S max = 0.15 to determine if a window contains a dynamic object.

2) Metrics: To consider non-deterministic behaviors, we compute the median of the ATE RMSE of the keyframe trajectory with a Sim(3) alignment and the tracking rate over 10 executions for every sequence. We then compute the Penalized ATE RMSE and Success Rate (section IV-C) with penalty coefficient τ = 10%, max acceptable tracking rate decrease ∆ r,max = 15% and max acceptable ATE RMSE l max = 10cm. Fig. 6. We apply our method on LDSO and test it on a hard sequence. Without masks the SLAM fails, with masks it ignores the object and works correctly.

B. Results

1)

Comparison with the State-of-the-Art: dynamic objects, especially when causing consensus inversions, may cause early SLAM failure and decrease in the tracking rate of SLAMs, making the comparison of ATE RMSEs biased. To take both the trajectory error and the tracking rate into account we use our new metrics: the Penalized ATE RMSE and the Success Rate. The Penalized ATE RMSE integrates failures, so it is directly comparable between SLAMs, and a higher Success Rate expresses that a SLAM is less affected by dynamic objects. We evaluate the methods on the TUM RGB-D and Consensus Inversion datasets.

Table I (cols. 2-6) shows the Penalized ATE RMSE of the State-of-the-Art. Our method performed better than others on our dataset (all modes) and on TUM RGB-D in monocular mode. It is in third place on TUM RGB-D in RGB-D mode.

On TUM RGB-D in monocular mode the results of L-K2 [7], DynaSLAM3 [START_REF] Bescos | Dy-naSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes[END_REF] and SLAMANTIC4 [START_REF] Schorghuber | Slamantic -leveraging semantics to improve vslam in dynamic environments[END_REF] are explained by the harsh penalty we give to early failures (which happened to the three of them). The original ORB-SLAM 2 already performs well so removing dynamic objects is not really necessary. All Dynamic SLAMs5 performed well in RGB- 4 [10] Uni 5 [START_REF] Wang | A Unified Framework for Mutual Improvement of SLAM and Semantic Segmentation[END_REF] DS 5 [START_REF] Yu | DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments[END_REF] Segmentation baselines Our seg.

State-of-the-Art ORB-SLAM 2 [15] + • • • L-K 2 [7] Dyna 3 [14] ST

Test dataset

No seg. Mask R-CNN [START_REF] He | Mask R-CNN[END_REF] PWC-Net [START_REF] Sun | PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume[END_REF] RVOS [START_REF] Ventura | Rvos: End-to-end recurrent network for video object segmentation[END_REF] COSNet [START_REF] Lu | See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks[END_REF] Consensus Inversion -Mono 0,0547 0,0693 0,0692 N/A N/A 0,0860 0,0760 0,0237 0,0144 0,0297 0,0089

Consensus Inversion -Stereo N/A 0,0627 0,0699 N/A N/A 0,0756 0,0630 0,0803 0,0116 0,0148 0,0094 TUM RGB-D -Mono 0,0892 0,1108 0,1101 N/A N/A 0,0252 0,0235 0,0335 0,0331 0,0267 0,0222 TUM RGB-D -RGB-D N/A 0,0206 0,0173 0,0190 0,0802 0,1077 0,0172 0,0790 0,0218 0,0245 0,0185 D including our method. We reached the standard of other Dynamic SLAMs that rely on manually annotated networks. Fig. 4 illustrates how the SLAM can output nonsense in presence of consensus inversions (motions that do not exist) and that we prevent it. On our dataset other Dynamic SLAMs performed poorly: they failed in hard / very hard sequences when the object was of an unknown class, e.g., dragon or dromedary. Even SLAMANTIC, that tries not to segment mobile objects that are not moving (e.g. a parked car) also failed in the very hard sequences as the objects are static during most of the sequence.

Table II (cols. [START_REF] Klein | Parallel Tracking and Mapping for Small AR Workspaces[END_REF][START_REF] Rosinol | Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios[END_REF][START_REF] Besbes | An interactive augmented reality system: a prototype for industrial maintenance training applications[END_REF][START_REF] Gay-Bellile | A mobile markerless augmented reality system for the automotive field[END_REF][START_REF] Tamaazousti | The constrained slam framework for non-instrumented augmented reality[END_REF] shows the Success Rate of the State-ofthe-Art. Our approach has the best performance in all cases. The results are coherent with the Penalized ATE RMSE: a higher Success Rates correspond to a lower Penalized ATE RMSE. Except for TUM RGB-D in RGB-D mode (success rate ≥ 85%), all results are below 75%. The results in monocular mode show that it is more difficult to handle monocular dynamic sequences with geometrical approaches, possibly due to the arbitrary scale of the SLAM. The success rates on our dataset (about 60%) shows that both geometrical approaches and hybrid ones combining geometrical/semantic approaches at runtime fail if there are consensus inversions caused by unknown objects.

Our results prove that regarding the robustness to dynamic objects: 1) semantic networks should be fine-tuned to the dynamic objects of the working environment 2) geometrical approaches are unreliable under consensus inversions 3) Hybrid/combined approaches are unreliable under consensus inversion caused by unknown objects.

2) Comparison with baselines: Unsupervised Video Object Segmentation (UVOS) networks and semantic segmentation networks may appear as trivial solutions to make a SLAM Dynamic as their integration is straightforward.

To test this aspect, we integrated Mask R-CNN [START_REF] He | Mask R-CNN[END_REF] (we filter the same semantic classes as DynaSLAM), RVOS [START_REF] Ventura | Rvos: End-to-end recurrent network for video object segmentation[END_REF] (in zero-shot mode) and COSNet [START_REF] Lu | See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks[END_REF] (we only use past frames for inference) in ORB-SLAM 2. We also test a very simple optical flow solution with PWC-Net [START_REF] Sun | PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume[END_REF], a learned optical flow network, by masking the area of the image with the 50% most intense optical flow.

Table I (cols. [START_REF] Cheng | An Accurate Localization Scheme for Mobile Robots Using Optical Flow in Dynamic Environments[END_REF][START_REF] Cadena | Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age[END_REF][START_REF] Hartley | Multiple View Geometry in Computer Vision[END_REF][START_REF] Schorghuber | Slamantic -leveraging semantics to improve vslam in dynamic environments[END_REF][START_REF] Barnes | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments[END_REF][START_REF] Cheng | Improving monocular visual SLAM in dynamic environments: an optical-flow-based approach[END_REF] show that we perform better that all baselines. All results are comparable to the State-of-the-Art on TUM RGB-D except for PWC-Net, likely due to its naive integration (if there is no object moving the predicted mask will be wrong). However, on the Consensus Inversion dataset results are quite different: Mask R-CNN and PWC-Net both perform poorly while RVOS and COSNet have very good results. This shows that methods that are class-agnostic perform better than class-aware methods and -considering the other columns of the table -geometrical methods. The main issue with UVOS networks is in fact oversegmentation: RVOS and COSNet failed on the Static sequences of the Consensus Inversion / Static subset. They masked the only source of features in the image and made the SLAM fail. The same conclusions come from Table II. Fig. 7 illustrates failure cases.

The conclusion is that UVOS networks are better at making SLAMs robust to dynamic objects than the usual semantic and geometrical approaches. However, they also segment static objects and have an increased risk of failing early in static environments.

3) Evaluation of monocular false starts: monocular SLAMs as ORB-SLAM 2 require the camera to move to initialize, so we evaluate a specific kind of failure: false starts. Fig. 5 illustrates such a false start: since the camera is static, generated maps or trajectories are necessarily fake. We evaluate the State-of-the-Art and the baselines on the Consensus Inversion / Static subset. We performed best, never initializing incorrectly. All other methods failed, either because the object is unknown (semantic approaches), because the object caused a consensus inversion (geometrical approaches) or because it was not fully segmented (UVOS approaches). The results show again that it is essential to make a SLAM robust in a specific environment.

4) Extension to a Direct SLAM: : we test LDSO, a monocular direct SLAM. We improve it by integrating the network we trained using ORB-SLAM 2 after the feature detector. Fig. 6 shows the effect of masking: the 3D reconstruction is incorrect if the object is not masked. However, when masked, the SLAM operates normally. Table IV shows that both the Penalized ATE RMSE and Success Rate improve. While the Success Rate does not reach 100%, the result is very interesting: it is possible to use one SLAM to improve another one.

C. Limitations

There are some limitations to our method. We rely on video segmentation networks but they may not work in ambiguous situations, e.g., when a dynamic object passes in front of a very similar object. Dynamic objects must be reconstructed by the SLAM to generate outliers so it is difficult to mask objects that move all the time and we do not yet handle new dynamic objects at runtime. Evidently, the improved SLAM stops tracking if a known dynamic object covers the image.

VI. CONCLUSIONS

In this paper we proposed a novel method to learn to segment dynamic objects using only one monocular sequence per dynamic object, which is an advantage in comparison to previous methods. More importantly, we do not need any manual labelling which makes our method much easier to use.

We also presented the Consensus Inversion dataset and new metrics to evaluate the robustness of Dynamic SLAMs. We showed that consensus inversions can cause major SLAM failures, even to State-of-the-Art Dynamic SLAMs. We improved ORB-SLAM 2 monocular/stereo/RGB-D as well as LDSO and achieved top results in very challenging scenarios.

Another advantage, in addition to preventing SLAM failures and improving motion estimation, is the improvement in map reconstruction. Tasks like relocalization and loop closing need accurate maps and should benefit from our approach.

Fig. 3 .

 3 Fig. 3. Miniatures of our dataset Consensus Inversion. The camera moves in the Dynamic subset and stays static in the Static subset.

Fig. 5 .

 5 Fig. 5. Example of monocular false start. The SLAM cannot initialize as the camera is perfectly static. Yet if the object is not masked the SLAM generates absurd trajectories and 3D maps. Masking dynamic objects prevents such situations.

Fig. 7 .

 7 Fig. 7. Failure cases of baselines methods (ORB-SLAM 2 + existing network). Mask R-CNN ignores the dragon and considers the dromedary a cake. Other methods segment static objects. LDSO [16] + • • • No segmentation Our segmentation Avg. Penalized ATE RMSE (m) 0.0833 0.0581 Success Rate (%) 36.4% 63.6%

TABLE I AVERAGE

 I PENALIZED ATE RMSE (M) OF THE STATE-OF-THE-ART AND BASELINES ON CONSENSUS INVERSION/DYNAMIC AND TUM RGB-D/DYNAMIC DATASETS. N/A INDICATES THAT THE SLAM MODE IS NOT SUPPORTED.

			State-of-the-Art				ORB-SLAM 2 [15] + • • •
		L-K 2 [7] Dyna 3 [14] ST 4 [10] Uni 5 [22] DS 5 [23]		Segmentation baselines		Our seg.
	Test dataset						No seg. Mask R-CNN[30] PWC-Net[31] RVOS[32] COSNet[26]
	Consensus Inversion -Mono 54,5%	63,6%	63,6%	N/A	N/A	45,5%	54,5%	72,7%	72,7%	72,7%	100,0%
	Consensus Inversion -Stereo N/A	72,7%	63,6%	N/A	N/A	63,6%	63,6%	63,6%	81,8%	81,8%	100,0%
	TUM RGB-D -Mono	50,0%	62,5%	62,5%	N/A	N/A	87,5%	87,5%	62,5%	62,5%	100,0%	100,0%
	TUM RGB-D -RGB-D	N/A	100,0% 100,0% 100,0% 87,5% 62,5%	100,0%	62,5%	100,0%	100,0%	100,0%

TABLE II SUCCESS

 II RATE (%) OF THE STATE-OF-THE-ART AND BASELINES ON CONSENSUS INVERSION/DYNAMIC AND TUM RGB-D/DYNAMIC DATASETS. N/A INDICATES THAT THE SLAM MODE IS NOT SUPPORTED.

		State-of-the-Art				ORB-SLAM 2 [15] + • • •
	L-K 2 [7]	Dyna 3 [14] ST 4 [10]			Segmentation baselines			Our seg.
				No seg. Mask R-CNN[30] PWC-Net[31] RVOS[32] COSNet[26]
	53,3%	60,0%	60,0%	60,0%	60,0%	66,7%	86,7%	80,0%	100,0%

TABLE III EVALUATION

 III ON CONSENSUS INVERSION/STATIC DATASET. WE REPORT THE RATIO OF SEQUENCES THAT DO NOT CAUSE INITIALIZATION FAILS (FALSE STARTS).

TABLE IV AVG

 IV . PENALIZED ATE RMSE (M) AND SUCCESS RATE (%) OF LDSO AND OUR MASKED VERSION ON THE CONSENSUS INVERSION/DYNAMIC

DATASET.

Source: https://github.com/srihari-humbarwadi/person segmentation tf2.0

We implemented a simplified version of[START_REF] Cheng | An Accurate Localization Scheme for Mobile Robots Using Optical Flow in Dynamic Environments[END_REF] (which uses the Lucas-Kanade optical flow): we warp frames with an homography and we set the optical flow displacement threshold to 2px.

DynaSLAM randomly crashed in RGB-D mode on our system. We refer to the original results in this mode.

The publicly available code of SLAMANTIC does not support monocular mode so we adapted the available stereo code.

We report the results of DS-SLAM and Unified.

VII. ACKNOWLEDGEMENTS

This publication was made possible by the use of the FactoryIA supercomputer, financially supported by the Ile-de-France Regional Council.


Raphaèle Seror
email: <raphaele.se@gmail.com

Philippe Ravaud

Simon J Bowman

Gabriel Baron

Athanasios Tzioufas

Elke Theander

Jacques-Eric Eric Gottenberg

Hendrika Bootsma

Xavier Mariette

Claudio Vitali

Rapha Le Seror

Ö '

Karsten Asmussen

Soren Jacobsen

Johannes Wj Bijlsma

Aike A Kruize

Bombardieri Arthur

Cees Kallenberg

Johan G Brun

Roland Jonsson Steven

Nicoletta Del Papa

Alain Devauchelle

Thomas D Saraux

Roberto Gerli

Jean Sibilia

Sj Gren

Eric Hachulla

Gabor Illei

Adrian Jones

Menelaos Manoussakis

Roald Omdal

Ann Parke

Sonja Praprotnik

Matjia Tomsic

Manel Ramos Casals

Josef Smolen

Serge Steinfeld

Lennart Jacobsson

Frederick B Vivino

EULAR Sjogren's syndrome disease activity index: development of a consensus systemic disease activity index for primary

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

or 4 levels according to their severity. Data abstracted from 96 patients with systemic complications of primary SS were used to generate 702 realistic vignettes for which all possible systemic complications were represented. Using the 0 10 physician global assessment (PhGA) scale, each expert scored the disease activity of 5 patient profiles and 20 realistic vignettes. Multiple regression modelling, with PhGA used as the dependent variable, was used to estimate the weight of each domain.

Results

All 12 domains were significantly associated with disease activity in the multivariate model, domain weights ranged from 1 to 6. The ESSDAI scores varied from 2 to 47 and were significantly correlated with PhGA for both real patient profiles and realistic vignettes (r 0.61 and r 0.58, respectively, p<0.0001). Compared to 57 (59.4 ) of the real patient profiles, 468 (66.7 ) of the realistic vignettes

= = % %
were considered likely or very likely to be true.

Conclusion

The ESSDAI is a clinical index designed to measure disease activity in patients with primary SS. Once validated, such a standardized evaluation of primary SS should facilitate clinical research and should be helpful as an outcome measure in clinical trials.

Author Keywords primary Sjogren s syndrome ' ; outcome assessment ; systemic features ; activity index ; clinical vignettes ; disease activity Primary Sj gren s syndrome (SS) is a systemic disorder characterized by lymphocytic infiltration and progressive destruction of ö '

exocrine glands. The inflammatory process can, however, affect any organ. As a result, clinical features can be divided into two facets: (i) benign but disabling manifestations such as dryness, pain and fatigue, affecting almost all patients; and (ii) severe systemic manifestations that affect 20 to 40 of patients.

% %

Evidence-based therapy for SS is largely limited to treatments that improve sicca features. Clinical trials of disease-modifying

[1 ]
therapies have used a variety of ad hoc outcome measures mainly based on glandular features or patient symptoms, but not systemic features.

Valid activity indexes are needed to assess the effectiveness of new targeted therapies, such as B-cell targeted

[2 -6 ] [7 -9 ]
therapies that have shown promising results for both severe systemic , and glandular features. Two disease activity [10 11 ] [ 12 -15 ] indexes have recently been proposed: the SS disease activity index (SSDAI) and the Sj gren s Systemic Clinical Activity Index

[16 ] ö '

(SCAI). The development of these indexes was based on exploratory studies conducted in single countries, but they serve as the basis

[17 ]
of the present collaborative project. Thus, the European League Against Rheumatism (EULAR) has promoted an international collaboration to develop consensus disease activity indexes. Two indices are currently in development: (i) a patient-administered questionnaire to assess patient symptoms, the EULAR Sj gren s Syndrome Patient Reported Index (ESSPRI); and (ii) a systemic activity ö '

index to assess systemic complications, the EULAR Sj gren s Syndrome Disease Activity Index (ESSDAI).

ö '

We now describe the development and initial validation of the ESSDAI. This index was developed with the help of a worldwide panel of primary SS experts using physician global assessment (PhGA) of disease activity as an external criterion. The aim is for the ESSDAI to be used as outcome criteria to evaluate primary SS in a standardized way in both clinical trials and daily practice.

METHODS

This paper results from a collaboration of experts identified through their involvement in the primary SS field, headed by a steering committee of 7 physician experts in SS (HB, SB, JEG, XM, ET, AT, CV), a clinical epidemiologist (PR) and a rheumatologist, a fellow in clinical epidemiology (RS). The research protocol was endorsed by EULAR ( ). project code CLI 010

The steps of the development of the ESSDAI are summarized below; the entire methodology is available online (

). Appendix 1

Selection of relevant domains and definition of items

Domains of organ-specific involvement relevant to assess disease activity were selected in these steps. For each domain, the different clinical manifestations were ranked by level of activity (i.e., items). For selection of domains relevant to disease activity and definition of items for each domain, steering committee members prepared a preliminary proposal on the basis of their clinical experience, literature review and previous work.

, The preliminary selection of domains and items were successively submitted to the expert panel.

[16 17 ]
Experts had to rate the importance of each domain or suggest any additional domains or changes to proposed items. Intention-to-treat was used as a help for experts to define the different activity levels that ranged from no activity (requiring no treatment) to high activity (requiring high dose steroids or immunosuppressant). The experts proposals were analyzed, then discussed and voted on during a meeting.

'

Elaboration of clinical vignettes

In this step, realistic clinical vignettes were generated from real patient profiles.

Abstraction and standardization of real patient profiles

Five members of the steering committee supplied 96 profiles of their patients with systemic complications of primary SS. Each profile had to contain sections on History (demographic data and past medical history), Today (clinical symptoms and results of imaging " " " " examination) and Laboratory (biological features). Patient profiles included data from the baseline and 2 follow-up visits (3 and 6

" " months).

Abstraction of descriptions of items from real patient profiles

From patient profiles, 96 histories and 364 items, included in the Today and Laboratory sections, were extracted and standardized

" " " "
by the same investigator (RS). Description of all ESSDAI items were obtained and entered in a database with their corresponding scoring (domain and activity level). Each item had a median of 8.5 (interquartile range IQR: 4 15 descriptions.

[ -]

Generation of realistic clinical vignettes

Determination of construction rules

Data from primary SS patient cohorts of 5 members of the steering committee (SB, XM, ET, AT, CV) were used to construct [16 -20 ] a sample of vignettes with characteristics similar to European patient cohorts.

Generation of clinical vignettes

In total, 720 clinical vignettes were generated by a combination of History and items from the Today and Laboratory sections,

" " " " " "
with respect to the domain and item distribution defined previously. However, because items in the database referred to only systemic features, descriptors of symptoms such as dryness, pain and fatigue were generated and assigned to 30 of the patient vignettes.

%

Assessment

The 96 real patient profiles and the 720 clinical vignettes were randomly assigned to the 40 experts. Each expert had to rate 5 real patient profiles (rated by 2 raters each) and 20 clinical vignettes (18 were unique and 2 were common to 2 raters). For the survey, an

" " " "
internet-secure relational database was constructed. Patient data were presented chronologically, and the responses could not be changed.

For all visits of each profile or vignette, experts had to assess disease activity by use of the PhGA on a 0 10 numerical scale and a 5-point scale (inactive, low, moderate, high, very high activity). For the first visit of each profile or vignette, they also had to evaluate the plausibility of each patient case with use of a 5-point scale (very unlikely, unlikely, possible, likely, very likely) by answering the following question: Please indicate, according to your clinical experience and knowledge of the disease, the likelihood that this patient " scenario is a real case .

"

Statistical methods

Determination of domain weights and construction of the ESSDAI

Realistic clinical vignettes were used to determine domain weights. Disease activity assessed by the PhGA was used as an external criterion. Bivariate analysis involved Pearson s correlation between PhGA and each domain separately; for each domain, scores ranged ' from 0 no activity to 3 high activity . All domains were entered into multivariate models; the PhGA was used as a dependant = " " = " " variable and each domain was an explanatory variable. Two models were evaluated: a multiple linear regression model and a robust regression model with the least-median-of-squares method with an MM estimator. , The weights assigned to each domain were

[21 22 ]
derived from the regression coefficients of the multivariate model and rounded to form simplified indices. The weight of each item was obtained by multiplying the weight of the domain by the level of activity.

Preliminary validation

The ESSDAI was then calculated for all real patient profiles and realistic clinical vignettes. Construct validity was assessed by the strength of correlation between the ESSDAI score and the disease activity score assigned by the expert.

Sensitivity analyses

To evaluate the stability/robustness of the domain weight estimation, other models were tested: a logistic regression model with the 5-point scale used as an external criterion and different multiple linear regression models after pooling items that clustered.

Patient profile plausibility

Evaluation of patient profile plausibility of realistic clinical vignettes was compared to that of real patient profiles by a

Cochran-Armitage trend test.

Reliability of disease activity scoring

The evaluation of clinical vignettes common to 2 raters was used to assess inter-rater reliability:

For the 0 10 PhGA: intraclass correlation coefficient (ICC) and Bland and Altman graphical analysis ,

- [ 23 24 ]
For the 5-point scale: global agreement and Kappa statistics ,

[25 26 ]
The evaluation of real patient profiles was used to assess intra-rater reliability by the ICC, if at the first follow-up visit, the physician considered the disease activity unchanged. ICC confidence intervals were estimated with bootstrapping methods, with 1000 replications. [START_REF] Efron | An Introduction to the Bootstrap[END_REF] For all statistical analyses, a p-value less than 0.05 was considered statistically significant. All statistical analyses involved use of SAS release 9.1 and R release 2.2.1 statistical software packages.

RESULTS

Characteristics of expert panel

Of 40 invited primary SS experts, 39 took part in the study (35 Europeans from 13 countries and 4 North Americans). The median age of experts was 49 IQR: 46 58 years; 35 were rheumatologists, 3 were internists and 1 was an oral medicine practitioner. All but 2 (94.9

[ -] %

) had 10 years of experience in managing primary SS. All were involved in clinical research, and 23 (59.0 ) were also involved in basic

≥ %
science research into primary SS.

Selection of domains and definition of items

All 10 domains (constitutional and lymphadenopathy, glandular, articular, cutaneous, pulmonary, renal, muscular, peripheral nervous system, central nervous system, hematological) proposed by the steering committee were included. Experts decided to divide the " constitutional and lymphadenopathy domain into 2 domains and to add a biological domain but not add a hepatic domain (considered to " result from damage). The definition of the different activity levels (items) of each domain was obtained by consensus after discussion during meetings of the steering committee and experts.

Characteristics of real patient profiles and realistic vignettes

Thirty-nine of the 40 experts completed the rating of the 96 real patient profiles and 702 of the 720 clinical vignettes ( ). Real Table 1 patient profiles, selected for the extent of systemic involvement, had a significantly higher number of involved organs than did realistic clinical vignettes (2.83 1.46 vs. 2.14 1.08; p<0.0001).

± ±

Determination of domain weights and derivation of the ESSDAI

All domains, except haematological, glandular, articular and biological domains, showed a significant positive correlation with PhGA score (

). All domains were entered in 2 multivariate regression models. Multiple linear and least-median-of-squares regression Table 2 models provided similar results (R 0.29 and R 0.30, respectively). In both models, all domains were significantly associated with

2 = 2 =
disease activity (PhGA), and the weight estimation was similar. The weights derived from the regression coefficients were rounded to obtain a simple index ( and ). Tables 2 3

Preliminary validation of the ESSDAI in real patient profiles and realistic vignettes

The mean ESSDAI scores were 15.48 9.16 and 9.04 6.43 for real patient profiles and realistic vignettes, respectively. ESSDAI scores

± ±
were significantly correlated with the PhGA score (r 0.58 for realistic vignettes and r 0.61 for real patient profiles, p<0.0001;

).

= =

The maximum theoretical ESSDAI score is 123; however, only 25 of realistic vignettes and the real patient profiles had a score 13 and % ≥ 21, respectively. The highest score was 42 and 47 for the realistic vignettes and real profiles, respectively ( ).

≥ Figure 1

Sensitivity analyses

Other models for testing sensitivity analyses led to similar domain weights and similar correlation with the PhGA score.

Patient profile plausibility

Overall, experts considered 468 (66.7 ) of the 702 vignettes likely or very likely to be true, as compared with 57 (59.4 ) of the 96

% %
real patient profiles (p 0.09).

=

Reliability of disease activity scoring

Inter-rater reliability assessed by the ICC on 76 common vignettes was 0.41 0.18 0.60 for the PhGA. Bland and Altman graphical

[ - ]
analysis revealed no systematic errors (mean difference 0.16) but a variability of rating among experts (95 agreement interval 4.92 to

=- % [- 4.61
). The ratings of the same vignette by 2 different experts differed by 1 point for 37 vignettes (48.7 ), by 2 to 3 points for 30 (39.5

+ ] ≤ %
), and by >3 points for 9 (11.8 ). The weighted Kappa statistic for disease activity rating by the 5-point scale was 0.32 0.18 0.47 .

% % [ - ]
When grouping the highest activity scores (high and very high activity) and the lowest scores (inactive, low and moderate activity), the observed agreement was 72.4 and the Kappa coefficient 0.42 0.21 0.63 .

% [ - ]
Intra-rater reliability of the PhGA for 20 real patient profiles with unchanged activity at the first follow-up visit as assessed with the ICC was 0.86 0.68 0.94 .

[ -]

DISCUSSION

The ESSDAI is a consensus clinical index designed to measure disease activity in patients with systemic complications of primary SS.

This index is modelled on physician s judgment of disease activity. It results from a large collaboration of European and North American ' experts in primary SS. Compared to the PhGA, the ESSDAI performed satisfactorily for evaluation of disease activity in primary SS.

In the absence of an available gold standard or true understanding of disease process, the most accurate and meaningful method of

" "
disease activity assessment is to attempt to model the physician judgment. But any scale quantifying physician judgment of disease activity is a simplification of a complex mental process. For that purpose, 2 main gold standards have been used in the development of disease activity indexes: (i) the PhGA and (ii) the intention-to-treat approach. , The PhGA was used for the development of the systemic lupus disease activity index (SLEDAI) in systemic lupus erythematosus (SLE), whereas the intention-to-treat approach [28 -30 ] was used for the development of the British Isles lupus assessment group (BILAG) for SLE and the disease activity score (DAS) for [START_REF] Symmons | Development and assessment of a computerized index of clinical disease activity in systemic lupus erythematosus. Members of the British Isles Lupus Assessment Group (BILAG)[END_REF] rheumatoid arthritis (RA). However, unlike RA that quasi-exclusively affects articular system and where therapeutic decision is [START_REF] Van Der Heijde | Development of a disease activity score based on judgment in clinical practice by rheumatologists[END_REF] reproducible, the multisystemic nature of primary SS make therapeutic decision more variable. In addition, evidence-based therapeutic management of SLE is currently more advanced than in primary SS. Moreover, in the BILAG, this approach was used to define, in each domain, the different classes (A, B, C, D, E) and not as gold standard to determine domain weights. Therefore, in the absence of effective

" "
treatment or consensual therapeutic management and because of the variability of physician habits, the intention-to-treat approach might be more difficult to apply as a gold standard for primary SS at this time.

In addition, the extent to which each organ involvement or patient symptoms of fatigue and pain can influence the physician s ' evaluation of disease activity, in such a polymorphous disease, is extremely variable, as demonstrated by the limited reliability of the PhGA. These discrepancies among physicians views, even among disease experts, justify the necessity for a more objective and ' standardized scoring system to homogenise assessment of disease activity in different settings, by different physicians, experts but also less experienced physicians. Similar to correlation of SCAI scores with the PhGA, that of ESSDAI scores with the PhGA was about 0.60.

[17 ]

These correlations were lower than that from other studies evaluating disease activity scores for various systemic disorders , , .

[16 29 33 ] However, in most of these studies, the experts involved were trained to the rating of the PhGA and the different activity tools to improve reliability and homogeneity of this scoring. In the present study, to be closer to usual practice, we decided not to perform a training exercise.

The ESSDAI was developed by a large panel of primary SS experts and attempted to reflect their thought process. This may have ensured the content validity of the ESSDAI, including all relevant determinants of disease activity. The validity of the ESSDAI was further confirmed by the significant association of all domains with disease activity in our model. Previous primary SS activity indexes have been developed with use of cohorts in which about half of the patients had inactive or weakly active disease.

, Our strategy was to use [16 17 ] data from selected patients with systemic features to generate realistic clinical vignettes. This methodology enabled us to obtain a large number of vignettes (more than would have been possible with real patients) representing all possible systemic disease involvement (i.e., items). We then evaluated the extent to which each item influenced the evaluation of disease activity, which had not been possible in previous studies that did not include all organ-specific features.

, (same disease activity) may reflect different domains involved. As a further component of this project we will also be evaluating the most common patient-reported symptoms such as dryness, pain and fatigue in a patient-completed questionnaire the ESSPRI.

A major challenge in designing a systemic index is distinguishing between damage and disease activity. The most frequent approach, to avoid scoring damage, is to consider manifestations as active only if new or worsening. Under these scoring systems, when patients

" " " "
are evaluated at two time points, a persistent manifestation will not be rated at the second time point, which may cause an erroneous interpretation of improvement even though the patient s condition has not changed. To avoid this, all ESSDAI items were defined without ' reference to a previous assessment, but with an advice not to rate as active stable long-lasting features related to damage.

The ESSDAI is a systemic disease activity index developed to allow a standardized evaluation of disease activity in primary SS patients. Further studies are needed to assess the reliability and sensitivity to change of the ESSDAI. Once validated, if uniformly applied, the ESSDAI might enable comparison between studies and facilitate clinical research into primary SS. After the development of the patient-completed questionnaire (ESSPRI), the use of both the ESSDAI and ESSPRI for outcome assessment in randomized controlled trials should allow for assessing all facets of the disease. -For box plots of ESSDAI scores, the boxes represent the 25th and 75th percentiles; the lines within the box represent the median; the dot inside the box, linked by a line, represents the mean; and the whiskers extend to the most extreme data point, which is no more than 1.5 times the interquartile range (difference between the 75th and 25th percentiles) from the box. Values that are more extreme were considered outliers and are plotted individually (dots).

Table 2

Correlation between domains and disease activity, as assessed by the physician global assessment (PhGA) scale, and regression coefficients and domain weights obtained with the least-median-of-squares robust regression model with an MM estimator.

Domains

= = =
For bivariate analysis, Pearson s correlation coefficient (r) were obtained between PhGA and each domain; for each domain, scores ranged from 0 no activity to 3 high activity . All domains were

' = " " = " "
entered in multivariate regression with the least-median-of-squares model with an MM estimator. R 0.30 for the model. All domains were significantly associated with disease activity (as defined by 2 = the 0 10 PhGA numerical scale) in the multivariate model. The weights were derived from the regression coefficient of the multivariate model.

-

Table 3

The EULAR Sj gren s Syndrome Disease Activity Index (ESSDAI): Domain and item definitions and weights.

Figure 1

 1 Figure 1 Distribution of ESSDAI scores and correlation with disease activity in real patient profiles and realistic vignettes Figures 2A, 2B, 2C refer to the 702 realistic clinical vignettes, and figures 2D, 2E, 2F refer to the 96 real patient profiles. Distribution of ESSDAI scores in realistic vignettes (A) and real patient profiles (D), ESSDAI score for each level of global activity as defined by physicians on the 5-point scale (B and E), and correlation between ESSDAI scores and physicians ratings of disease activity by the physician global'

===≥ Moderate 2 == 2 ===

 22 Mild or intermittent fever (37.5 38.5 C)/night sweats and/or involuntary weight loss of 5 to 10 of body weight °-°% Moderate 2 Severe fever (>38.5 C)/night sweats and/or involuntary weight loss of >10 of body weight °% Lymphadenopathy [ 4 ] Lymphadenopathy 1 cm in any nodal region or 2 cm in inguinal region ≥ Lymphadenopathy 2 cm in any nodal region or 3 cm in inguinal region, and/or splenomegaly (clinically palpable or assessed by imaging) Small glandular swelling with enlarged parotid ( 3 cm), or limited submandibular or lachrymal swelling ≤ Moderate Major glandular swelling with enlarged parotid (> 3 cm), or important submandibular or lachrymal swelling Articular [ 2 ] Exclusion of osteoarthritis No 0 Absence of currently active articular involvement Low 1 Arthralgias in hands, wrists, ankles and feet accompanied by morning stiffness (>30 min) Moderate 2 = 1 to 5 (of 28 total count) synovitis

  rate; Hb hemoglobin; HRCT high-resolution computed tomography; IgG immunoglobulin G; NCS nerve conduction studies; NHYA New York heart association platelet; PNS peripheral nervous system; = =

Bivariate analysis Multivariate modelling Least median of square with MM estimator Correlation with

		PhGA	p-value	Regression coefficient	Standard error	Weight	p-value
	Constitutional	0.106	0.005	0.704	0.163	3	<0.0001
	Lymphadenopathy	0.134	0.0004	0.817	0.089	4	<0.0001
	Glandular	0.067	0.078	0.407	0.091	2	<0.0001
	Articular	0.063	0.095	0.489	0.068	2	<0.0001
	Cutaneous	0.156	<0.0001	0.559	0.097	3	<0.0001
	Pulmonary	0.170	<0.0001	1.066	0.115	5	<0.0001
	Renal	0.125	0.0009	1.090	0.170	5	<0.0001
	Muscular	0.156	<0.0001	1.193	0.191	6	<0.0001
	PNS	0.197	<0.0001	0.944	0.117	5	<0.0001
	CNS	0.159	<0.0001	0.936	0.157	5	<0.0001
	Hematological	0.041	0.277	0.361	0.126	2	0.004
	Biological	0.073	0.053	0.206	0.080	1	0.010
	PhGA physician global assessment; PNS peripheral nervous system; CNS central nervous system.

  Limited cutaneous vasculitis, including urticarial vasculitis, or purpura limited to feet and ankle, or subacute cutaneous lupus Evidence of mild active renal involvement, limited to tubular acidosis without renal failure or glomerular involvement with proteinuria (between 0.5 and 1 g/d) and without hematuria or renal failure (GFR 60 ml/min) Moderately active peripheral nervous system involvement shown by NCS, such as axonal sensory-motor neuropathy with maximal motor deficit of 4/5, pure sensory neuropathy with presence of cryoglobulinemic vasculitis, ganglionopathy with symptoms restricted to mild/moderate ataxia, inflammatory demyelinating polyneuropathy (CIDP) with mild functional impairment (maximal motor deficit of 4/5or mild ataxia), Or cranial nerve involvement of peripheral origin (except trigeminal (V) neralgia) High 3 = Highly active PNS involvement shown by NCS, such as axonal sensory-motor neuropathy with motor deficit 3/5, peripheral nerve involvement due ≤ to vasculitis (mononeuritis multiplex etc.), severe ataxia due to ganglionopathy, inflammatory demyelinating polyneuropathy (CIDP) with severe functional impairment: motor deficit 3/5 or severe ataxia Moderately active CNS features, such as cranial nerve involvement of central origin, optic neuritis or multiple sclerosis-like syndrome with symptoms restricted to pure sensory impairment or proven cognitive impairment High 3 = Highly active CNS features, such as cerebral vasculitis with cerebrovascular accident or transient ischemic attack, seizures, transverse myelitis, lymphocytic meningitis, multiple sclerosis-like syndrome with motor deficit. Cytopenia of auto-immune origin with neutropenia (neutrophils < 500/mm3), and/or or anemia (hemoglobin < 8 g/dl) and/or thrombocytopenia (platelets <50,000/mm3) Clonal component and/or hypocomplementemia (low C4 or C3 or CH50) and/or hypergammaglobulinemia or high IgG level between 16 and 20 g/L Moderate 2 Presence of cryoglobulinemia and/or hypergammaglobulinemia or high IgG level > 20 g/L, and/or recent onset hypogammaglobulinemia or recent decrease of IgG level (<5 g/L) CIDP chronic inflammatory demyelinating polyneuropathy; CK creatine kinase; CNS central nervous system; DLCO diffusing CO capacity; EMG electromyogram; FVC forced vital capacity;

	Exclusion of vitamin or iron deficiency,	High 3 = ≥ 6 (of 28 total count) synovitis Or lymphopenia (500 < lymphocytes < 1000/mm3)
	Cutaneous [ 3 ] Rate as No activity stable long-lasting features " " related to damage drug-induced cytopenia	No 0 = Low 1 Moderate 2 = = Moderate	Absence of currently active cutaneous involvement Erythema multiforma Cytopenia of auto-immune origin with neutropenia (500 thrombocytopenia (50,000 platelets 100,000/mm3) ≤ ≤ Or lymphopenia ( 500/mm3)	≤	neutrophils	≤	1000/mm3), and/or anemia (8	≤	hemoglobin	≤	10 g/dl), and/or
			2
	Pulmonary [ 5 ] Rate as No activity stable long-lasting features " related to damage, or respiratory involvement not " Biological [ 1 ]	High 3 = Diffuse cutaneous vasculitis, including urticarial vasculitis, or diffuse purpura, or ulcers related to vasculitis Or radiological or HRCT evidence of interstitial lung disease with: No breathlessness and normal lung function test. No 0 = No 0 = Absence of any of the following biological feature Absence of currently active pulmonary involvement Low 1 = Persistent cough or bronchial involvement with no radiographic abnormalities on radiography Low 1
	related to the disease (tobacco use etc.)	Moderate 2 =	Moderately active pulmonary involvement, such as interstitial lung disease shown by HRCT with shortness of breath on exercise (NHYA II) or abnormal lung function tests restricted to: 70 >DL 40 or 80 >FVC 60 % CO ≥ % % ≥ %
			High 3 = Highly active pulmonary involvement, such as interstitial lung disease shown by HRCT with shortness of breath at rest (NHYA III, IV) or with abnormal lung function tests: DL < 40 or FVC< 60 CO % %
	Renal [ 5 ] Rate as No activity stable long-lasting features " " related to damage, and renal involvement not	No 0 = Low 1	Absence of currently active renal involvement with proteinuria< 0.5 g/d, no hematuria, no leucocyturia, no acidosis, or long-lasting stable proteinuria to damage
	related to the disease.
	If biopsy has been performed, please rate activity
	based on histological features first
	Muscular [ 6 ]		No 0 =	Absence of currently active muscular involvement
	Exclusion of weakness due to corticosteroids	Low 1
			No 0
	"	"
	related to damage or PNS involvement not related
	to the disease
				≤
	CNS [ 5 ]		No 0 =	Absence of currently active CNS involvement
	Rate as No activity stable long-lasting features " "	Low 1
	related to damage or CNS involvement not related
	to the disease
	Hematological [ 2 ]		No 0
	For anemia, neutropenia, and thrombopenia, only
	auto-immune cytopenia must be considered
												Page / 10 11
				Ann Rheum Dis . Author manuscript

= = ≥ Moderate 2 = Moderately active renal involvement, such as tubular acidosis with renal failure (GFR <60 ml/min) or glomerular involvement with proteinuria between 1 and 1.5 g/d and without hematuria or renal failure (GFR 60 ml/min) or histological evidence of extra-membranous glomerulonephritis or ≥ important interstitial lymphoid infiltrate High 3 = Highly active renal involvement, such as glomerular involvement with proteinuria >1.5 g/d or hematuria or renal failure (GFR <60 ml/min), or histological evidence of proliferative glomerulonephritis or cryoglobulinemia related renal involvement = Mild active myositis shown by abnormal EMG or biopsy with no weakness and creatine kinase (N <CK 2N) ≤ Moderate 2 = Moderately active myositis proven by abnormal EMG or biopsy with weakness (maximal deficit of 4/5), or elevated creatine kinase (2N<CK 4N), ≤ High 3 = Highly active myositis shown by abnormal EMG or biopsy with weakness (deficit 3/5) or elevated creatine kinase (>4N) ≤ PNS [ 5 ] Rate as No activity stable long-lasting features = Absence of currently active PNS involvement Low 1 = Mild active peripheral nervous system involvement, such as pure sensory axonal polyneuropathy shown by NCS or trigeminal (V) neuralgia Moderate 2 = = = Absence of auto-immune cytopenia Low 1 = Cytopenia of auto-immune origin with neutropenia (1000 < neutrophils < 1500/mm3), and/or anemia (10 < hemoglobin < 12 g/dl), and/or thrombocytopenia (100,000 < platelets < 150,000/mm3) ≤ High 3 = = =

11

11

11

11

11

11

11

Ackowledgements:

This project was supported by a grant from the European League Against Rheumatism (EULAR). We thank Maxime Dougados and Alan Tyndall for their guidance and support.

References:

1 . Vivino FB , Al-Hashimi I , Khan Z , LeVeque FG , Salisbury PL 3rd , Tran-Johnson TK . Pilocarpine tablets for the treatment of dry mouth and dry eye symptoms in patients with Sjogren syndrome: a randomized, placebo-controlled, fixed-dose, multicenter trial. P92-01 Study Group . Arch Intern Med . 1999 ; 159 : 174 -81 2 . Drosos AA , Skopouli FN , Costopoulos JS , Papadimitriou CS , Moutsopoulos HM . Cyclosporin A (CyA) in primary Sjogren s syndrome: a double blind study ' . Ann Rheum Dis . 1986 ; 45 : 732 -5 3 . Kruize AA , Hene RJ , Kallenberg CG , van Bijsterveld OP , van der Heide A , Kater L . Hydroxychloroquine treatment for primary Sjogren s syndrome: a two year double 12 . Dass S , Bowman SJ , Vital EM , Ikeda K , Pease CT , Hamburger J . Reduction of fatigue in Sjogren syndrome with rituximab: results of a randomised, double-blind, placebo-controlled pilot study . Ann Rheum Dis . 2008 ; 67 : 1541 -4 13 . Devauchelle-Pensec V , Pennec Y , Morvan J , Pers JO , Daridon C , Jousse-Joulin S . Improvement of Sjogren s syndrome after two infusions of rituximab (anti-CD20)