Ruddy Théodose
email: [r.theodose@sherpa-eng.com

Dieumet Denis
email: d.denis]@sherpa-eng.com

Thierry Chateau

Vincent Frémont
email: vincent.fremont@ls2n.fr

Paul Checchin

A Deep Learning Approach for LiDAR Resolution-Agnostic Object Detection

Keywords: Deep Learning, 3D Object Detection, Sensor Fusion, Intelligent Vehicle, Camera Sensor, LiDAR Sensor

Existing neural network-based object detection approaches process LiDAR point clouds trained from one kind of LiDAR sensor. In the case of a different point cloud input, the trained network performs with less efficiency, especially when the given point cloud has low resolution. In this paper, we propose a new object detection approach, which is more resilient to variations in point cloud resolution. Firstly, layers from the point cloud are randomly discarded during the training phase in order to increase the variability of the data processed by the network. Secondly, the obstacles are described as Gaussian functions, grouping multiple parameters into a single representation. A Bhattacharyya distance is used as a loss function. This approach is tested on a LiDAR-based network and on an architecture using camera and LiDAR sensors. The networks are trained exclusively on the KITTI dataset and tested on Pandaset and the nuScenes Mini dataset. Experiments show that our method improves the performance of the tested networks on low-resolution point clouds without decreasing the ability to process high-resolution data.

I. INTRODUCTION

Autonomous driving in urban environments still remains a tremendous challenge. The autonomous vehicle must simultaneously manage multiple aspects of perception such as scene analysis, traffic sign recognition, or moving object localization. It is significant to control these aspects before deciding which actions can be engaged to preserve the safety of other users and the comfort of the passengers. 3D object detection for autonomous vehicles is an active research topic since identifying the parts of the scene that can interfere with the vehicle trajectory is a crucial task within the navigation pipeline. In this field, most works mainly rely, if not exclusively, on LiDAR sensor data.

LiDAR sensors collect 3D point clouds depicting the surroundings in great detail. However, the data structure of such 3D point clouds is usually not unique and normalized, and the spatial positioning of the points is also irregular. Moreover, the information provided on the appearance of surfaces is inadequate. By contrast, 2D cameras deliver structured data with high resolution as per the spectrum captured, e.g., color information, infrared. These characteristics make them particularly adapted for classification or identification tasks, even on small or noisy images. However, the loss of depth information and the narrow field of view makes the full 3D reconstruction of the scene geometry difficult. Over the past years, deep neural networks have proven their efficiency in perception tasks by using camera images. Most methods aim to extend these techniques to 3D point clouds. However, due to the differences between 2D images and 3D LiDAR information, the application of classical neural networks on 3D point cloud data is still an open problem. For example, some methods transform the sparse data into dense and structured grids through voxelization that can then be used by operators designed initially for image processing. Other works exploit architectures based on PointNet [START_REF] Qi | PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation[END_REF], [START_REF] Qi | PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[END_REF] with the aim to directly process the 3D point cloud by utilizing the geometrical relationships between points and their neighborhood [START_REF] Shi | PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud[END_REF], [START_REF] Chen | Fast Point R-CNN[END_REF]. No sensor is adapted for all the situations. Hence, autonomous vehicles are usually equipped with at least two types of sensors [START_REF]Waymo driver[END_REF], [START_REF]Motional[END_REF]. Currently, the majority of the 3D object detection methods are purely LiDAR-based [START_REF] Zhou | Voxelnet: End-to-end learning for point cloud based 3d object detection[END_REF], [START_REF] Yan | SECOND: Sparsely Embedded Convolutional Detection[END_REF], [START_REF] Shi | PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection[END_REF]. Fewer methods employ the data fusion of the two sensors [START_REF] Ku | Joint 3D Proposal Generation and Object Detection from View Aggregation[END_REF], [START_REF] Sindagi | MVX-Net: Multimodal voxelnet for 3D object detection[END_REF] because of their different data structures and constraints such as calibration, registration of 2D/3D images, etc. However, some methods exploit the outputs of 2D processing techniques to constraint the possible locations of the objects in the scene. For example, the helpful priors for the 3D detection task are the utilization of semantic image segmentation [START_REF] Vora | PointPainting: Sequential Fusion for 3D Object Detection[END_REF] or 3D frustums generated by 2D image detection [START_REF] Qi | Frustum PointNets for 3D Object Detection from RGB-D Data[END_REF].

While researches focused on the design of architectures that allows to maximize accuracy, most of the existing models are trained from a given set of LiDAR parameters and do not generalize well if tested on another set. Even though solutions have been developed for camera sensors (like domain adaptation [START_REF] Wang | Deep visual domain adaptation: A survey[END_REF]), to the best of our knowledge, a few works have analyzed the effects on pre-trained systems of 3D scans issued from different resolutions of LiDAR sensors. Therefore, we believe that this topic is important for three reasons. Firstly, to generalize a detection method for emerging LiDAR sensors. Secondly, we see that most of the existing systems are trained on restricted geographical areas and for a specific set of sensors rendering different networks useless if employed on data outside its training distribution. Finally, the annotation of new data is an expensive and time-consuming process. Moreover, a network performing decently on new data, even if the results are not perfect, can potentially speed up the annotation process.

In this paper, we study the consequences of input distribution changes on networks and propose an approach to alleviate these effects. First, we compare how methods exploiting both sensors and methods using only 3D point clouds react to variations in resolution. Our study revolves around a network inspired by PointPillars [START_REF] Lang | PointPillars: Fast Encoders for Object Detection From Point Clouds[END_REF] and MVX-Net [START_REF] Sindagi | MVX-Net: Multimodal voxelnet for 3D object detection[END_REF]. However, the main contribution concerns the way the network is trained. We propose a simple augmentation procedure in order to increase the variability of inputs seen in the training stage and then to make networks more robust to point cloud reduction. We have also developed a new representation for the obstacles and its corresponding loss function. Each target is represented by a Gaussian function. This loss function focuses on the occupation of the obstacle instead of regressing parameters which can only be accurately estimated if sufficient data are available. The performances of our approach are studied across data from three datasets: the KITTI dataset, the Pandaset dataset and the nuScenes Mini dataset. The results show that our approach can help a network to converge better and to produce accurate detections even if the input point cloud is extremely sparse. In this paper, we propose the following contributions:

• An extensive bibliographical study of the state-of-the-art methods for 3D object detection in Section II. According to the type of data used (images, 3D LiDAR points or both), a summary of published networks is presented and discussed; • A new training procedure to increase the variability of the inputs during the training stage; • A novel way to represent the obstacles as normal distribution is introduced in Section IV-A to use statistical distances and to study the similarities between a target and a prediction in terms of occupation instead of regressing directly the box parameters; • The evaluation of our approach is conducted on multiple datasets. The behaviors of our approach over inputs never seen during the training stage are detailed in Section VII.

II. RELATED WORK

In this section, some relevant networks aiming at 3D object detection on point clouds and RGB images are first introduced and then secondly, methods targeting the management of point clouds at different resolutions are presented.

A. 3D Object Detection Methods

We split the methods available in the literature into three groups. The first group includes networks working exclusively with 3D point clouds. Techniques from the second group only exploit images. The last group of methods process both types of data. Table I summarizes these approaches by mentioning the raw input data used and the main paradigms existing in the literature. These three groups of 3D detection methods are analyzed below.

1) 3D Object Detection using 2D Images: Camera-based 3D object detection methods are generally less accurate than LiDAR-based methods because of the lack of depth information in the image. Hence, most of them rely on geometrical constraints priors. Here, we focus only on monocular camera methods. In [START_REF] Roddick | Orthographic Feature Transform for Monocular 3D Object Detection[END_REF], a 3D grid is populated with features from the input image by projecting voxels on the image using the camera parameters. The resulting 3D grid is then collapsed on the vertical axis to reduce the computational cost. The authors of [START_REF] Shrivastava | CubifAE-3D: Monocular Camera Space Cubification on Autonomous Vehicles for Auto-Encoder based 3D Object Detection[END_REF] train an encoder-decoder architecture for depth estimation. The resulting intermediate features (after the encoder, before the decoder) are then used for 3D object detection. Monocular 3D Region Proposal Network [START_REF] Brazil | M3D-RPN: Monocular 3D Region Proposal Network for Object Detection[END_REF] proposes a twobranch architecture, where one branch uses regular convolutions across the whole image to estimate global features. The other splits the image into row blocks then applies kernels, distinct for each bin. These convolution kernels are called "depth aware convolutions". The primary assumption is that each row block can be associated with a discretized depth due to perspective projection (for example, lower rows often represent road and close objects) and then a different operation for each block is applied. In [START_REF] Simonelli | Disentangling Monocular 3D Object Detection[END_REF], a 2D object detection method is followed by a 3D object detection one. However, their loss function splits the box parameters into groups to simplify the optimization process.

2) Point Cloud-based Methods: This group only uses the 3D point clouds provided by LiDAR sensors as inputs. Methods belonging to this group can be split into two subgroups: grid-based methods and point-based methods.

Grid-based methods. The main idea of grid-based methods consists in turning the 3D point cloud into structured data to allow the use of concepts and operators which were successfully applied in 2D image processing such as convolutions. The main drawback is that the precision depends on the discretization of the grid. Some methods are based on existing 2D architectures to extract detections from BEV pseudo images. BirdNet [START_REF] Beltran | BirdNet: a 3D Object Detection Framework from LiDAR information[END_REF] and Complex YOLO [START_REF] Simon | Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds[END_REF] are based on Faster-RCNN [START_REF] Ren | Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[END_REF] and YOLO [START_REF] Redmon | You Only Look Once: Unified, Real-Time Object Detection[END_REF], [START_REF] Redmon | YOLOv3: An Incremental Improvement[END_REF], respectively. Their corresponding networks are applied on the 3-channel pseudoimages by concatenating a height map, a reflectance map, and a density map. VoxelNet [START_REF] Zhou | Voxelnet: End-to-end learning for point cloud based 3d object detection[END_REF] was a major milestone. In their paper, the authors describe an architecture including encoding voxels features through PointNet-like methods. The encoded grid is given to 3D dense convolutions and afterwards to 2D convolutions in order to return the selected prior boxes and their corrections. However, the use of 3D dense convolutions on a large 3D grid is slow and computationally expensive. SECOND [START_REF] Yan | SECOND: Sparsely Embedded Convolutional Detection[END_REF] rectifies this by applying sparse convolutions [START_REF] Graham | Submanifold Sparse Convolutional Networks[END_REF] on the grid. PointPillars [START_REF] Lang | PointPillars: Fast Encoders for Object Detection From Point Clouds[END_REF] extends the concept by using, directly, columns in a 2.5D pseudo image instead of voxels in a 3D grid in order to speed up the computation. Voxel-FPN [START_REF] Kuang | Voxel-FPN: Multi-Scale Voxel Feature Aggregation for 3D Object Detection from LiDAR Point Clouds[END_REF], inspired by the Feature Pyramid Networks [START_REF] Lin | Feature Pyramid Networks for Object Detection[END_REF], uses voxel grids with different resolutions to encode voxels at different scales. Anchor-free methods,

assimilating object detection to keypoint detection [START_REF] Zhou | Objects as points[END_REF], [START_REF] Law | Cornernet: Detecting objects as paired keypoints[END_REF] have also been transposed to 3D object detection [START_REF] Chen | Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots[END_REF], [START_REF] Yin | Center-based 3D Object Detection and Tracking[END_REF].

The main goal of these methods is not to change the input data representation but the output format, as detections are no more related to prior boxes.

Point-based methods. Point-based methods aim to deal with raw point clouds at some stage directly, in contrast to the grid-based approach. This paradigm can be applied for the whole method or jointly with grid representations. Moreover, most of these methods are two-stage detectors. PointRCNN [START_REF] Shi | PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud[END_REF] adopts a two-stage approach: A first stage generates proposals from foreground point segmentation followed by the refinement of the proposals to estimate targeted boxes. Fast PointRCNN [START_REF] Chen | Fast Point R-CNN[END_REF] and PV-RCNN [START_REF] Shi | PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection[END_REF] employ a VoxelNet-like to generate proposals processed by PointNet-like networks. Part-A 2 [START_REF] Shi | Part-A 2 Net: 3D Part-Aware and Aggregation Neural Network for Object Detection from Point Cloud[END_REF] initially estimates object parts (top left, bottom right, etc.) from point cloud to improve the 3D box refinement stage.

3) 3D Object Detection using Multi-Sensor Fusion: This section focuses on camera-LiDAR data fusion to localize surrounding objects, in contrary to the previously mentioned methods that exploit only one sensor. These methods can be divided into two subgroups. The first makes use of both sensor data within the same system. The second uses outputs from existing and pre-trained 2D methods as priors or inputs for methods that focus on point clouds. These are introduced in the following paragraphs.

Parallel flows. These methods generally use images as a second input to exploit the complementary nature of data. MV3D [START_REF] Chen | Multi-View 3D Object Detection Network for Autonomous Driving[END_REF] and AVOD [START_REF] Ku | Joint 3D Proposal Generation and Object Detection from View Aggregation[END_REF] are methods that simultaneously process images and point clouds to extract intermediate proposal. MV3D [START_REF] Chen | Multi-View 3D Object Detection Network for Autonomous Driving[END_REF] generates 3D proposals from a LiDAR scan and then projects them onto the LiDAR BEV. Features inside the projection of each modality are extracted through ROI pooling and merged to estimate final boxes. AVOD [START_REF] Ku | Joint 3D Proposal Generation and Object Detection from View Aggregation[END_REF] extends the idea by projecting all pre-built prior boxes on highresolution feature maps from LiDAR BEV and RGB images. MVX-Net [START_REF] Sindagi | MVX-Net: Multimodal voxelnet for 3D object detection[END_REF] adds pooled features from RGB feature maps to voxels used in VoxelNet. The method described in [START_REF] Liang | Deep Continuous Fusion for Multi-Sensor 3D Object Detection[END_REF] takes advantage of leveraging continuous convolutions to merge feature maps from each sensor. The authors of [START_REF] Liang | Multi-task multisensor fusion for 3d object detection[END_REF] improve this architecture by adding related tasks such as ground estimation and depth completion to boost the 3D refinement subnetwork.

Sequential streams. The methods introduced in this paragraph operate sequentially and are 2D-driven. These processes exploit results from RGB processing as filters on the point cloud, contrary to the previously mentioned techniques. Frustum PointNet [START_REF] Qi | Frustum PointNets for 3D Object Detection from RGB-D Data[END_REF], Frustum Convnets [START_REF] Wang | Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection[END_REF] and PointFusion [START_REF] Xu | PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation[END_REF] exploit results from a 2D detector to extract points that fall inside each detection and then reduce the search space for each object. The frustums are then processed to precisely localize objects. IPOD [START_REF] Yang | IPOD: Intensive Point-based Object Detector for Point Cloud[END_REF] removes the background 3D points by exploiting a segmentation map from RGB camera images. Each foreground point is used as a location for prior boxes and then processed afterwards. In [START_REF] Vora | PointPainting: Sequential Fusion for 3D Object Detection[END_REF], the authors show the improvement of performances on several LiDAR-based detectors only by integrating the semantic information from a segmentation map in the input.

B. Point Cloud-based Resolution-Agnostic Deep Learning

To the best of our knowledge, domain adaptation has become an active research field on images. However, less research has been conducted on domain adaptation or portability on 3D point clouds, especially in an outdoor sensor data. As shown in [START_REF] Lambert | Performance Analysis of 10 Models of 3D LiDARs for Automated Driving[END_REF], each 3D LiDAR sensor has its own set of characteristics corresponding to a range, point distribution, data coherency on disturbed conditions,etc. The authors of [START_REF] Piewak | Analyzing the Cross-Sensor Portability of Neural Network Architectures for LiDAR-based Semantic Labeling[END_REF] perform an in-depth analysis of the performance of their architecture for semantic segmentation of point clouds from a 32-channel and a 128-channel LiDAR. 3D Domain Adaptation is studied and analyzed on 3D CAD (Computer-Aided Design) object datasets such as ModelNet [START_REF] Wu | 3D ShapeNets: A Deep Representation for Volumetric Shapes[END_REF]. Studies on outdoor scenes are less usual and most of them target the semantic segmentation of point clouds. In [START_REF] Achituve | Self-Supervised Learning for Domain Adaptation on Point-Clouds[END_REF], a shared representation is learned in a self-supervised fashion through the reconstruction of deformed 3D point clouds. With PointDAN [START_REF] Qin | PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation[END_REF], an architecture is designed to learn cross domain local and global features to align objects from two distinct distributions. Some methods such as [START_REF] Wu | Squeezesegv2: Improved model structure and unsupervised domain adaptation for roadobject segmentation from a LiDAR point cloud[END_REF] have recourse to Generative Adversarial Networks to close the gap between synthetic data and real-world data. In [START_REF] Yi | Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds[END_REF], the authors propose a point cloud completion method by assimilating the task of reconstructing the underlying surface. A semantic segmentation network is then trained from the reconstructed surface that serves as the new canonical domain. Our method does not directly aim at the LiDAR domain adaptation. Instead, we aim at enforcing some aspects of the network in the training phase in order to make it more resilient to variations in point cloud resolution.

III. OBJECT DETECTION NETWORK INDEPENDENT OF POINT CLOUD RESOLUTION

In this paper, we propose an end-to-end trainable method for 3D object detection. The network takes point clouds and images as inputs.

A. LiDAR Data Preprocessing

This section lists the set of operations applied on the 3D LiDAR point cloud.

A 3D point cloud is defined as a set

P = {[x i , y i , z i ] T ∈ R 3 } i∈1...N with [x i , y i , z i ] T the
location of the i-th point in the 3D space. The common frame is defined with x-axis oriented forward, the y-axis leftward and the z-axis upward.

The input point cloud is discretized according to the method described in [START_REF] Lang | PointPillars: Fast Encoders for Object Detection From Point Clouds[END_REF]: The point cloud is turned into column voxels with no vertical discretization compared to a 3D voxel grid that slices the point cloud over the three dimensions. This way, a BEV pseudo-image can be generated where each non-zero pixel is assigned to a non-empty voxel. The terms "pixel" and "voxel" are used interchangeably when referred to the BEV image: One column voxel (volume) is linked to only one cell of the BEV map.

Each voxel has two data representations: A version containing the 3D points, name 3D voxel, and a version containing only the projections of these points on the image plane, named RGB voxel.

The search space is restricted to the interval [x min , x max ] (in meters) on the x axis and [y min , y max ] (in meters) on the y axis. This point cloud is discretized into a maximum of n in x × n in y voxels over the two dimensions.

B. Network Architecture

Figure 2 describes the global structure of the system. The RGB image is turned into a feature map which is sampled using the projections of the 3D points on the image plane. The 3D points and the extracted features are then concatenated. The resulting voxels are then processed by a PointPillars architecture [START_REF] Lang | PointPillars: Fast Encoders for Object Detection From Point Clouds[END_REF]. The approach looks like the one described in [START_REF] Vora | PointPainting: Sequential Fusion for 3D Object Detection[END_REF] except the extracted features are not gathered from a semantic segmentation output. Moreover, in opposition with MVXNet whose image network is a pre-trained Faster-RCNN, we train our network from the beginning. [START_REF] He | Deep Residual Learning for Image Recognition[END_REF]. As input images size may vary with the dataset, we chose a ResNet18 as a basis because of it small size and its small computation charge.

b) Sampling: The image use is heavily inspired by the PointFusion method described in [START_REF] Sindagi | MVX-Net: Multimodal voxelnet for 3D object detection[END_REF]. The feature map computed by is then sampled through the projections contained in each RGB voxel. c) BEV Network: As indicated, the network processing the voxels is a PointPillars network [START_REF] Lang | PointPillars: Fast Encoders for Object Detection From Point Clouds[END_REF]. Each voxel, containing 3D points and their corresponding RGB features, are first processed by a Voxel Feature Encoder (VFE) in order to represent its content as a vector. All these vectors are scattered across a BEV pseudo-image which is then fed to a Region Proposal Network (RPN). The network returns 3 outputs: a classification map, a direction map and a regression map.

C. Network Output

In common datasets, 3D obstacles are represented by a position [x, y, z] T , dimensions [h, w, l] T and an orientation θ . The orientation is often restrained to the vertical axis. The set of parameters is S = {x, y, z, h, w, l, θ }.

We decide to represent each obstacle as a normal distribution: N (µ µ µ, Σ Σ Σ):

µ µ µ = [x, y] T ∈ R 2 , Σ Σ Σ = a b b c ∈ R 2×2 (1) with a = cos 2 (θ ) 2σ 2 w + sin 2 (θ ) 2σ 2 l , b = - sin(2θ ) 4σ 2 w + sin(2θ ) 4σ 2 l , c = sin 2 (θ ) 2σ 2 w + cos 2 (θ ) 2σ 2 l , σ w = w 3 , σ l = l 3 ,
Σ Σ Σ being symmetric positive-definite. We note

F : R 7 → R 2 × R 2×2 (x, y, z, h, w, l, θ ) → µ µ µ, Σ Σ Σ
the function that turns the parameters S into a multivariate Gaussian function.

With this expression, an obstacle is defined as a Gaussian function. This representation focuses on the shape of the Gaussian function and the space occupied by its related obstacle. However, it does not provide information about the target's orientation. Therefore, ambiguities arise from the fact that multiple configurations can produce the same Gaussian function as illustrated in Figure 4.

The network returns three same size maps: A map for classification, a map for regression of boxes' parameters S, and one for orientation disambiguation. An anchor-free approach inspired from [START_REF] Zhou | Objects as points[END_REF] is used here to represent the object where each predicted object is described by its center. This method is contrary to many approaches defining prior boxes with various sizes and orientations located in fixed positions. Indeed, an anchor-based method requires a set of hyperparameters specifically tuned to work correctly (size of anchors, Intersection of Union (IoU) threshold to define a positive anchor, etc.). Anchor-free method removes this constraint, allowing to reduce the number of critical parameters that may significantly affect the training process.

IV. IMPLEMENTATION DETAILS

A. Training 1) Ground Truth Formatting: Each ground truth object i is defined by a class and its box parameters [x i , y i , z i , h i , w i , l i , θ i ] T with its position [x i , y i , z i ] T , its dimensions [h i , w i , l i ] T and its orientation θ i .

Ground truth maps are arranged using a process described in [START_REF] Zhou | Objects as points[END_REF]. Each label is represented by a Gaussian function, whose mean is the center of the object. A voxel is named positive if its ground truth value is 1 (the center of the object belongs to this voxel). One label is then assigned to one voxel in the classification map.

The regression map is composed of 7 channels, one for each box parameter. Let us denote ∆x the first channel, ∆y the second one and so on, see [START_REF] Qi | PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[END_REF]. Voxels are defined according to a static grid with a fixed position and each voxel on the classification map represents a portion on the metric space. For the ground truth object i, its associated voxel, whose center is located at ( xi , ỹi ) of the metric space and at (u, v) on the regression map, the corresponding target for the regression is defined as:

∆x = xi -x i , ∆y = ỹi -y i , ∆z = z i , ∆h = h i , ∆w = w i , ∆l = l i , ∆θ = θ i (2)
The orientation map is a four-channel image, each channel representing one of the four quadrants defined in Figure 5. This map is defined to select the correct case from the ones defined in Figure 4. The obstacles can be defined as probability laws, therefore the definition of a statistical distance between a target obstacle and a predicted obstacle is possible. In our case, we decided to use the Bhattacharyya distance. If P, Q are two multivariate Gaussian functions P = N (µ µ µ P , Σ Σ Σ P ), Q = N (µ µ µ Q , Σ Σ Σ Q ), the Bhattacharyya distance is expressed as:

D B (P, Q) = 1 8 (µ µ µ P -µ µ µ Q ) T Σ Σ Σ -1 (µ µ µ P -µ µ µ Q )+ 1 2 ln det Σ Σ Σ det Σ Σ Σ P det Σ Σ Σ Q .
(4) The regression loss is then defined as:

L reg = 1 N pos ∑ pos SL1(S c , Ŝc ) + D B (F(S c , Ŝamb ), F(S c , S amb )) (5)
with pos the locations of the positive voxels, N pos the number of positive voxels, SL1 the Smooth L1 loss, S * and Ŝ * respectively the targets and the predictions. c) Orientation disambiguation: L ori is a focal loss applied on the positive pixel locations of the estimated and the ground truth orientation maps.

3) Data Augmentation: In the field of autonomous driving, the most common LiDAR sensors used are rotating mechanical sensors, delivering 3D points from one emission location in a layered manner. Points from the same layer share the same latitude in a spherical frame centered on the sensor. For each sample, some layers are randomly discarded as illustrated in Figure 6. During the training, they are randomly discarded until between 25% and 60% in each point cloud. c) nuScenes Mini: nuScenes [START_REF] Caesar | nuScenes: A multimodal dataset for autonomous driving[END_REF] is a dataset that provides data recorded through Boston and Singapore on 360 • thanks to one 32-channel LiDAR and six cameras. One specificity of this dataset is that it encourages the accumulation of point clouds over multiple frames (called sweeps) to get a denser cloud. If the timestamps are taken into account, the estimation of the velocity of the targets is possible. For our experiments, we aim at detecting on synchronized point cloud and RGB image. Thus, the intermediate sweeps are not used. For the current experiments, we focus on the "Mini" subset containing 10 sequences.

B. Comparison Methodology

Each annotated dataset contains biases. Thus, for the same tasks, the required criteria may greatly vary among the datasets. In the following paragraphs, we detail our choices in order to compare the results on different datasets with the same setup. KITTI training split is used for training our experiments. Hence, we modify point clouds of testing point clouds sets to fit into the training conditions of the network. The x-axis is oriented forward, the y-axis leftward and the z-axis upward. All datasets have at least a 360 • mechanical LiDAR and at least one front camera. However, due to restrictions in the KITTI dataset, we only exploit the section of the point cloud inside the frustum of the front camera. Hence, this restriction is also applied on validation data. Moreover, the experiments are trained for a specific range. Consequently, all validation point cloud are cropped to fit into this range. Each dataset defines its class names and, for each similar category, the targets may greatly differ. For example, the "Car" class may include large personal vehicles such as SUVs and Vans in one dataset. In another dataset, the same "Car" class may only include small vehicles and two other classes for the SUVs and the Vans might be specifically created. For these experiments, we tried to establish correspondences of class names that represent the same entities in the different datasets. Each dataset defines its own obstacle representation for obstacles. For example on nuScenes, the orientation of annotations are defined as quaternions, allowing the rotation on the three axes. On KITTI however, the road is supposed horizontal, thus the orientation is defined as a scalar representing the rotation angle around the vertical axis. Some datasets also define attributes to describe the temporary states of the targets. For example, the dynamic status of a vehicle can be described as "Moving", "Stopped" or "Parked". For this study, each obstacle is defined according to the KITTI standards: Semantic class, position, dimensions and rotation around the vertical axis. Moreover, for nuScenes and Pandaset, we check if an annotation includes at least 5 points. If not, the annotation is discarded. All datasets use the Average Precision (AP) as a performance metric. However, depending on the method, criteria to separate true detections from errors may differ. For example, the nuScenes compute the AP using distances between the center of the targets and the proposals. In this study, we keep the process used in KITTI: An IoU-based AP computation. For the KITTI validation split, we consider the difficulty-based clustering introduced by the dataset. Moreover, the IoU threshold is fixed at 0.7. However, for Pandaset and nuScenes, all labels are considered as belonging to the same group. Easy and hard cases are grouped under the same value. In fact, KITTI difficulties are defined according to parameters on the image plane that are not explicitly defined on the other datasets. Moreover, for Pandaset and nuScenes, we also display the AP with an IoU threshold at 0.5. The reason is that each dataset has its own biases, especially on the box fitting. A high threshold would have supposed that the annotators on different bases have the same sensibility about data extrapolation and box fitting. We focus our study on BEV detection. In fact, this work also concerns the impact of low-resolution point clouds for the evaluation. Ground and car tops may not be defined enough to estimate features related to the vertical axis (height and altitude). However, the main goal remains the object detection on data recorded on unknown environments with different LiDAR sensors. Depending on the equipped sensors and the choices of the annotators, the maximum labeling range varies greatly among datasets. nuScenes limits its annotations to 50 m for cars while Pandaset can provide labels beyond 100 m. However, our networks are trained to work until 80 m. Thus, for a shorter range (nuScenes), the point cloud is used as if it reaches 80 m but the outputs beyond the labeling range are discarded. For a longer range, the input point cloud is cropped to fit into the 80 m along the x axis.

C. Conducted Experiments

In order to evaluate the contributions of the presented techniques, we conduct different experiments, each one with a different configuration.

The input range is set to [x min , x max ] × [y min , y max ] = [0.0, 70.4]× [35.2, 35.2]. The base surface of each voxel (on the xy plane) measures 0.22×0.22 m 2 , leading to a voxel grid with 320 × 320 cells. (γ cls , γ reg , γ ori ) are defined as (2.0, 1.0, 0.2). For the focal loss, (α, β ) is defined as [START_REF] Qi | PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[END_REF][START_REF] Chen | Fast Point R-CNN[END_REF]. These values come from an improved implementation of [START_REF] Yan | SECOND: Sparsely Embedded Convolutional Detection[END_REF]. All models, CL and L, are trained for 100 epochs using an Adam optimizer and a one-cycle learning rate scheduler with a maximum learning rate set to 0.001. About the common data augmentation techniques applied, the point cloud is first randomly mirrored along the y axis. We then perform random global translations, rotations and scalings on the point clouds and the annotations. On camera-LiDAR methods, all images in the dataset do not have the same sizes. Hence, to allow batch processing, each image is randomly cropped, so the result has 1024 × 256 pixels. Moreover, noise is added to images and parameters such as saturation and value are altered.

All the experiments are trained on the train split of the KITTI dataset. Each experiment is identified with a name whose structure is defined as Sensor-TrainNumLayers-Loss. The part Sensor can take the values L for LiDAR or CL for Camera + LiDAR. TrainNumLayers qualifies the number of layers in point cloud used for the training phase, 64/8 when the number of layers is fixed, Var when the randomized layer reduction is applied. Loss refers to the loss used, Std for the loss described in [START_REF] Lang | PointPillars: Fast Encoders for Object Detection From Point Clouds[END_REF], G for the loss function described in IV-A2. For example, L-Var-G refers to the experiment using 3D point clouds, layer reduction to generate randomized distribution and the statistical loss. Note that all experiments, including CL-experiments are trained from scratch. The RGB parts do not come from pre-trained networks. Consequently, both RGB and BEV parts are trained as a single network.

In order to get a reference, we also evaluate the performances of a pre-trained PointPillars network with its original parameters (anchor-based, 0.16 × 0.16 m 2 voxels, . . . ) and a pre-trained PV-RCNN [START_REF] Shi | PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection[END_REF].

VI. EXPERIMENTS ON KITTI DATASET

In this section, we introduce the experiments conducted on the KITTI dataset [START_REF] Geiger | Vision meets robotics: The KITTI Dataset[END_REF]. Results of all experiments on the validation split are displayed in Table III. We detail the different observations in the following paragraphs.

The CL-experiments run at 35 ms and the L-experiments run at 20 ms on a computer equipped with a GPU NVidia 1080Ti and a CPU Intel Core i7-7700K.

A. Differences between Camera-LiDAR Methods and LiDAR Networks

On 64 layer point clouds and for the same number of epochs, better performances are observed on networks using only point clouds in comparison with networks exploiting both sensors.

First, all networks are trained with the same hyperparameters. Hence, under the same conditions, CL-networks, which contain more parameters due to the image subnetwork, are more difficult to train. Moreover, in opposition with 3D point clouds which are produced by an active sensor and contain only geometric information, images tend to be denser and more sensitive to photometric phenomena and camera artifacts. These experiments show that random layer removal can be an easy way to improve the performances of a detector on lowresolution point cloud with minor loss (LiDAR methods) or even improvement (fusion methods) on high-resolution data.

C. Influence of Gaussian Representation

We focus now on the effects of the use of a statistical distance as a loss. On almost all cases, we observe a slight degradation of the results comparing to the experiments using Smooth-L1 losses.

Figure 7 shows the profiles of the Bhattacharyya loss D B (Eq. 4) for variations according w, l and θ . In Figure 7a, all the parameters are fixed except θ . The curve looks like the sine function with global minima at kπ, k ∈ Z, allowing multiple solutions. The main difference with the sine function concerns the impact of the target's dimensions. We tested D B for different target lengths. The closer to 1 the ratio l w is, the less the occupancy of the target changes if rotated, and less the error on θ affects the loss. Thus, this loss function acts as a weight on targets as it focuses on an angular error from more elongated targets. The loss function has one global minima and the function is convex. We note that the loss value is greater if the prediction is smaller than the target. However, the values returned by the loss tend to be much lower than the ones returned by a Smooth-L1 loss, leading to a more difficult convergence on the dimensions as shown in Figure 7b.

D. Conclusion

The effects of our training procedures are studied over data from the KITTI validation split. The layer reduction allows improving the performances of a network in various cases easily without any priors. However, while the point cloud distribution varied, the scene remains the same, which does not indicate if the network can be applied on other places and with other sensors. On the KITTI dataset, the Gaussian representation seems to degrade the network as it is less precise than the Smooth-L1. However, its benefits are more visible on out-of-distribution data.

VII. EXPERIMENTS WITH OTHER DATASETS

Until now, we have only generated situations where the point clouds vary but the environments and the annotation biases stay the same. In this section, we experiment with our trained networks on nuScenes Mini and Pandaset. By testing on other datasets, we can evaluate how our complete network reacts to stimuli of different environments and capture setups. Table III provides the values computed on the nuScenes and Pandaset datasets.

A. Common Analyzes

The main observation is that methods exploiting RGB images perform poorly in all cases. Networks using only point clouds tend to resist better to these changes. In fact, only bare geometrical information is available on 3D point clouds. Targets are often recognizable by their shape. Additional data such as RGB information that could be affected by external phenomena are integrated by the network during the training stage. When training on KITTI without constraints on images features, the networks implicitly learn the specificities of the camera (noise artifacts, dynamic range, deformations. . . ) but also what characterize the scenes. This could be lighting (e.g. twilight and night scenes missing in KITTI), the different urban infrastructures or even the car models available in restricted geographic regions. Hence, changing the image domain has an important impact on methods using both sensors.

B. Analyzes on nuScenes Mini

As expected, the performances drop significantly on the new dataset. The labels are originally designed for point cloud accumulation. Thus, detection on a single scan is more difficult. Furthermore, the results on experiments #8 L-64-G and #10 L-Var-G are now better on this dataset than they were before on the KITTI dataset and surpass their counterparts (experiments #7 L-64-Std and #9 L-Var-Std, respectively). For out-of-distribution data, the new loss function helped the network to acquire a more flexible (but less precise) representation of a car, allowing a better knowledge transfer on unknown data. Figure 8 illustrates qualitative results for experiment #10 L-Var-G on the nuScenes dataset. We can observe the contrast between the scene captured by the camera and the sparsity of one point cloud.

C. Analyzes on Pandaset

We observe the same general dynamics on the unaltered point clouds as during the comparison on the KITTI validation set. Despite the similar number of points available on both point clouds, AP values are much lower on Pandaset than on KITTI dataset. First, even if the KITTI train and validation splits are distinct, all samples come from the same sequences, inducing an important bias. Moreover, compared to KITTI, the density of labels is higher on Pandaset data. Among the labels, many of them are located outside the roadway, in a mall parking lot for example. Another reason related to possibly missed obstacles concerns the intra class variability. Depending on the city, the distribution of car models may vary considerably. On the recorded scenes, we observe vehicle models that do not exist in the KITTI dataset (recorded in Germany). Thus, the network tends to return the same parameters as it only observe a small part of the spectrum of vehicle models during the training phase.

Figure 9 presents qualitative results obtained for experiment #10 L-Var-G on Pandaset data. We observe that the network provides relevant BEV detections despite the context difference. Even if the number of targets is large in the scene, the network is still able to extract most of the obstacles.

VIII. CONCLUSION

In this paper, we study how 3D object detection networks react over datasets and point cloud distributions. We propose an approach to improve performances over unknown data without supplementary knowledge. The approach rests on two points. The first one consists in randomly discarding point cloud layers during the training. This technique allows to enforce the network to not overfit a 3D point distribution. The second one is a new obstacle representation. Each obstacle is represented by a Gaussian function, allowing the use of statistical distances as loss functions. Despite not being a domain adaptation method, our approach, exclusively trained on a subset of the KITTI dataset, provides accurate outputs on data from nuScenes and Pandaset datasets that were never revealed at the training stage. The work presented in this paper focuses on car detection. In future work, it would be needed to study how the method and the loss function act in multi-class detection. When working on low-resolution point clouds, it is hard to estimate the features of a target with complete exactitude. Finally, it would be interesting to investigate uncertainty estimation for object detection, especially for a critical field such as autonomous driving.

Fig. 1 :

 1 Fig. 1: The same network is able to process point clouds with different resolutions. The missing detections on the sparse case come from the lack of 3D points.

Fig. 2 :Fig. 3 :

 23 Fig. 2: Overview of the proposed network architecture.

2 Fig. 4 :

 24 Fig. 4: Parameter sets producing the same Gaussian function, w re f , l re f , θ re f are the reference values.

Fig. 5 :

 5 Fig. 5: The four quadrants of the orientation map.

2 )

 2 Loss Functions: A multitask-loss is used:L = γ cls L cls + γ reg L reg + γ ori L ori(3)with L cls , L reg , L ori the losses related to scene classification, parameter regression, orientation ambiguities removal. γ cls , γ reg , γ ori are their respective weights. a) Scene Classification: L cls is a mean of focal losses applied on each pixel of the classification map.b) Regression:The following computations occur only on the locations of positive voxels. Then, we suppose the box parameters extracted from the corresponding prediction and target maps. The box parameters are split into two sets, S c = {x, y, z, h} and S amb = {w, l, θ }, S amb is related to rotation ambiguities.

Fig. 6 :

 6 Fig. 6: Illustration of randomized layer removal (b). (a) Initial point cloud. On the top images, the point cloud projections are displayed. Colors represent distance from the LiDAR sensor.V. EXPERIMENTAL PROTOCOLWe describe here the conducted experiments and how they are evaluated across the datasets.A. Description of DatasetsIn this study, the experiments are conducted on three datasets: The KITTI dataset, the nuScenes dataset and the

  When trained on low resolution point clouds, experiments (#5 CL-8-Std, #6 CL-8-G) perform better than their Lcounterparts (#11 L-8-Std, #12 L-8-G) on high resolution data. The same observation is visible when the CL-networks are trained on 64-layer point clouds and applied on 8-layer data. This means that the network does not exclusively focus on the layout of the 3D points but also exploit the features extracted from the RGB image. Experiments (#11 L-8-Std, #12 L-8-G) only deal with extremely sparse point clouds that depict surfaces with low resolution. Hence, networks focus on optimizing this difficult train set and then are more prone to overfitting.B. Influence of Layer ReductionWe first analyze the effects of the layer reduction on CL-experiments with standard loss (experiments #1 CL-64-Std, #3 CL-Var-Std and #5 CL-8-Std). On 64-layer point clouds, the layer reduction seems to have few impact on the experiments. In fact, both image features and detailed point cloud are useful for target identification. However, on 8-layer point clouds, experiment #1 CL-64-Std returns lower scores as no examples of such type were seen during the training stage. Experiment #5 CL-8-Std is highly specialized on lowresolution point clouds. The randomized layer removal helped experiment #3 CL-Var-Std to get the best trade-off. Concerning the L-experiments, we focus on experiments #7 L-64-Std, #9 L-Var-Std and #11 L-8-Std. On 64layer point clouds, we notice an improvement between experiments #7 L-64-Std and #9 L-Var-Std. The layer removal helped the network to learn a better representation of the targets as more diverse examples were associated to the same goal. Once again, this operation also helped to improve the outputs on low-resolution point clouds without specializing the network on the latter point cloud distribution.

Fig. 7 :

 7 Fig. 7: Variations of D B as a function of a single parameter, the others being fixed at ground truth.

Fig. 8 :

 8 Fig. 8: Qualitative results for experiment #10 L-Var-G on nuScenes.

Fig. 9 :

 9 Fig. 9: Qualitative results for experiment #10 L-Var-G on Pandaset.

TABLE I :

 I Summary of published networks aiming at 3D object detection from point clouds and/or RGB images. An overview of the state of experimental evaluation of methods useful for autonomous driving.

	Reference	Raw Input	Raw Data Processing	Overview	Object Types	Dataset(s)
		Data				used
	OFT [16]	RGB Image	-	Resnet Backbone, projection of 3D voxels on image feature maps, each 3D	Cars	KITTI
				assigned with features inside the projection, then Voxelnet-like processing
	M3D-	RGB Image	-	DenseNet Backbone then 2 branches: (1) global feature extraction on the	Car, Pedestrian,	KITTI
	RPN [17]			whole image, (2) local "Depth-Aware" feature extractions. Horizontally	Cyclist
				sliced image. Each slice is fed to its own convolution
	MonoDIS [18] RGB Image	-	ResNet Backbone, 2D and 3D heads, box parameters divided into groups	Car, Pedestrian,	KITTI
				for better convergence	Cyclist
	CubifAE-	RGB Image	-	Multiple trainings: (1) depth estimation through encoder-decoder, (2) depth	Car, Pedestrian,	KITTI,
	3D [19]			latent space for 3D detection	Cyclist and oth-	nuScenes,
					ers	KITTI Virtual
						2
	BirdNet [20]	Point Cloud	Discretization into BEV	Faster RCNN-like network	Car, Pedestrian,	KITTI
			3-channel image (height,		Cyclist
			density, reflectance)
	Complex-	Point Cloud	Discretization into BEV	YOLO-like network, complex angles for regression	Car, Pedestrian,	KITTI
	YOLO [21]		3-channel image (height,		Cyclist
			density, reflectance)
	VoxelNet [7]	Point Cloud	3D voxelization	Learned voxel encoding for each voxel, 3D convolutions to flatten on the	Car, Pedestrian,	KITTI
				vertical axis then RPN	Cyclist
	SECOND [8]	Point Cloud	3D voxelization	Learned voxel Encoding, sparse 3D convolution to reduce the computa-	Car, Pedestrian,	KITTI
				tional burden, RPN	Cyclist	(nuScenes
						thereafter)
	PointPillars [15] Point Cloud	Column voxelization (no	Learned voxel encoding, 2D convolutions, RPN	Car, Pedestrian,	KITTI,
			slices on vertical axis)		Cyclist	nuScenes
	OHS [22]	Point Cloud	3D voxelization, voxels	Objects as sets of hotspots (non-empty voxels belonging to an object),	Car, Pedestrian,	KITTI,
			represented by the mean	architecture inspired by SECOND or PointPillars	Cyclist and oth-	nuScenes
			content		ers
	PointRCNN [3] Point Cloud	-	Two-stage method: (1) 3D proposal generation through point cloud seg-	Car, Pedestrian,	KITTI
				mentation, each foreground point generating its own 3D proposal, (2) 3D	Cyclist
				box refinement
	Fast Point R-	Point Cloud	3D voxelization	Two-stage method: (1) 3D region proposal through VoxelNet architecture,	Car, Pedestrian,	KITTI
	CNN [4]			(2) pooling on VoxelNets feature maps, fusion with the corresponding	Cyclist
				region of point cloud, the augmented point cloud is then processed by a
				refinement network
	CenterPoint [23] Point Cloud	3D or column voxelization	Objects represented by their center on the classification map, architecture	nuScenes	nuScenes
				inspired by VoxelNet & PointPillars	classes
	PV-	Point Cloud	3D voxelization + furthest		Car, Cyclist	KITTI
	RCNN [9]		point sampling to extract
			keypoints
	Part-A 2	Point Cloud	-	Two-stage detection: (1) point cloud semantic segmentation, part estima-	Car, Pedestrian,	KITTI
	Net [24]			tion for positive 3D points then 3D proposals, (2) point cloud pooling then	Cyclist
				refinement network to aggregate the estimated parts
	MV3D [25]	Point Cloud,	Discretization into BEV		Car	KITTI
		RGB image	3-channel image (height,
			density, reflectance) and
			Cylindrical projection
	Frustum	Point Cloud,			Car, Pedestrian,	KITTI
	Point-	RGB image			Cyclist
	Net [13]
	AVOD [10]	Point Cloud,	Discretization into BEV		Car, Pedestrian,	KITTI
		RGB	3-channel image (height,		Cyclist
			density, reflectance)
				the 3D boxes
	PointFusion [26] Point Cloud,	-	2D detector to extract 2D crop and 3D frustum, 2D crop and corresponding	Car, Pedestrian,	KITTI, SUN-
		RGB image		point cloud processed by a ResNet and a PointNet, respectively. Features	Cyclist	RGBD
				merged to estimate a refined 3D box
	ContFusion [27] Point Cloud,	-	The 2D feature extraction, feature sampling through continuous convolu-	Car	KITTI,
		RGB image		tion with the point cloud, reprojection of the sampled features on a BEV		TOR4D
				then 3D box prediction		(private)
	IPOD [28]	Point Cloud,	2D Semantic Segmentation	3 parts: (1) using the 2D segmentation, all the 3D background points are	Car, Pedestrian,	KITTI
		RGB image		discarded and proposals are generated from the remaining points, (2) a	Cyclist
				backbone network extracts local and global features from the whole point
				cloud, (3) the features are extracted from the proposals and used in a
				refinement network
	Frustum	Point Cloud,	-	The 2D detector on the image to create 3D frustums, for each result, point	Car, Pedestrian,	KITTI
	Con-	RGB image		cloud part inside the frustum is kept and voxelized along the frustum axis,	Cyclist
	vNet [29]			convolutions are used along this axis to estimate the 3D box
	PointPainting [12] Point Cloud,	2D Semantic Segmentation	Point features (3D coordinates, reflectance) concatenated with the score	Car, Pedestrian,	KITTI,
		RGB image		given by 2D segmentation. Experiments on multiple detectors (PointR-	Cyclist	nuScenes
				CNN, PointPillars, etc.)
	MVX-	Point Cloud,	3D voxelization
	Net [11]	RGB image

Two-stage detection: (1) architecture inspired by SECOND for the region proposal, the 3D feature maps sampled at multi-scales using the computed keypoints, (2) RoI-Grid Pooling on the keypoints then Refinement Network Two-stage detector: (1) BEV generates 3D proposals, (2) 3D proposals are projected on the BEV, the RGB image and on the cylindrical view, features from the 3 feature maps (each view) are pooled then merged to refine the boxes

-

The 2D detector on the image to create 3D frustums, for each result, the 3D points inside a frustum used to estimate the corresponding The 3D box with an architecture similar to PointNet++ Two-stage detector: (1) BEV and RGB images merged to generate 3D proposals (2) 3D proposals are projected on the BEV and the RGB image feature maps, the corresponding features are pooled then merged to refine VoxelNet used for the 3D detection and Backbone Faster RCNN for the 2D feature maps, 2 fusion methods tested: (1) 2D features sampled on point level and merged before voxel encoder, (2) 2D features sampled on voxel level and merged after voxel encoder Car KITTI MMF [30] Point Cloud, RGB image 3D voxelization Two-stage detector: (1) RGB image processed to generate intermediate feature maps and a depth map. Maps are merged with LiDAR features to generate 3D proposals. (2) LiDAR and RGB feature maps are pooled and merged to refine the estimated boxes Car (main) KITTI, TOR4D

TABLE II :

 II Datasets used for experimental evaluation.

	Name	Number of scenes	Weather conditions	Image size	LiDAR
	KITTI	7481 (Train)	Sunny	Variable (about 1242 × 375)	Velodyne HDL-64E
	nuScenes Mini	404 (10 sequences)	Sunny, Night, Cloudy	1600 × 900	Reference not provided
					(32 layers)
	Pandaset	8240 (103 sequences)	Sunny, Rainy, Night	1920 × 1080	Hesai Pandar64

ACKNOWLEDGMENTS

This work has been sponsored by Sherpa Engineering and ANRT. This work has been sponsored also by the French government research Program Investissements d'Avenir through the RobotEx Equipment of Excellence (ANR-10-EQPX-44) and the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01), by the European Union through the program Regional competitiveness and employment 2014-2020 (FEDER -AURA region) and by the AURA region.