Leo Milecki

Sylvain Bodard

Jean-Michel Correas

Marc-Olivier Timsit

Maria Vakalopoulou

3D UNSUPERVISED KIDNEY GRAFT SEGMENTATION BASED ON DEEP LEARNING AND MULTI-SEQUENCE MRI

Keywords: Unsupervised segmentation, kidney graft, multi-sequence MRI, Deep Learning, CNN

Image segmentation is one of the most popular problems in medical image analysis. Recently, with the success of deep neural networks, these powerful methods provide state of the art performance on various segmentation tasks. However, one of the main challenges relies on the high number of annotations that they need to be trained, which is crucial in medical applications. In this paper, we propose an unsupervised method based on deep learning for the segmentation of kidney grafts. Our method is composed of two different stages, the detection of the area of interest and the segmentation model that is able, through an iterative process, to provide accurate kidney draft segmentation without the need for annotations. The proposed framework works in the 3D space to explore all the available information and extract meaningful representations from Dynamic Contrast-Enhanced and T2 MRI sequences. Our method reports a dice of 89.8 ± 3.1%, Hausdorff distance at percentile 95% of 5.8±0.41mm and percentage of kidney volume difference of 5.9 ± 5.7% on a test dataset of 29 patients subject to a kidney transplant.

INTRODUCTION

Segmentation of organs or other structures such as vessels or cells in biomedical imaging is often an important first step or even a prerequisite to analyzing medical volumes. They are essential to describe and monitor shapes, volume characteristics [START_REF] Lee | Impact of the ratio of graft kidney volume to recipient body surface area on graft function after live donor kidney transplantation[END_REF] or further used in more complex pipelines, as in radiomics analysis [START_REF] Rizzo | Radiomics: the facts and the challenges of image analysis[END_REF] and precision medicine. In recent years, automatic segmentation in biomedical images has been subject to extensive search, with deep learning based techniques holding the state of the art performances in various applications and modalities, including Magnetic Resonance Imaging (MRI) [START_REF] Lundervold | An overview of deep learning in medical imaging focusing on mri[END_REF]. However, one of the main limitations of common deep Fig. 1. Overview of the proposed method for the automatic segmentation of kidney graft using multimodal MR Imaging. learning supervised approaches is the need for expensive and time-consuming annotations.

Nowadays, unsupervised approaches investigate methods that are able to generate models without the need for annotations. In particular, concerning unsupervised kidney segmentation, image-based methods have been proposed to obtain a segmentation through an image processing pipeline using tools such as thresholding [START_REF] Sandmair | Semiautomatic segmentation of the kidney in magnetic resonance images using unimodal thresholding[END_REF] or morphological operators [START_REF] Li | A renal vascular compartment segmentation method based on dynamic contrast-enhanced images[END_REF][START_REF] Yu | A watershed method for mr renography segmentation[END_REF]. Some other methods propose clustering frameworks based on K-means [START_REF] Yang | Renal compartment segmentation in DCE-MRI images[END_REF] or on wavelet-based clustering [START_REF] Li | Waveletbased segmentation of renal compartments in DCE-MRI of human kidney: initial results in patients and healthy volunteers[END_REF]. Deformable models were also designed in level-set methods [START_REF] Abdulahi | Fast chan-vese without edges and connected component analysis for kidney segmentation in mri images[END_REF] or exploiting active contours [START_REF] Yu | A novel active contour model for image segmentation using distance regularization term[END_REF]. Advanced deformable boundaries methods are also explored in [START_REF] Shehata | 3d kidney segmentation from abdominal diffusion mri using an appearance-guided deformable boundary[END_REF] with however the use of training samples for parts of their formulations.

This paper proposes an unsupervised method for the segmentation of kidney graft from Dynamic Contrast-Enhanced (DCE) and T2 sequences obtained 12 months after the transplantation. Our method consists of a twostep procedure that is summarised in Figure 1. First, we propose an unsupervised method to detect the kidney region using thresholding techniques and morphological operators. The second step corresponds to the pixelwise segmentation of the kidney graft. Our method is based on [START_REF] Kim | Unsupervised learning of image segmentation based on differentiable feature clustering[END_REF] extending the formulation to 3D to explore all the available information from the MRI modalities. Moreover, it proposes a multimodal formulation where multiple MRI sequences are integrated towards a robust kidney graft segmentation tool.

METHODS

Selection of the region of interest

The first step of the proposed method is the automatic detection of possible areas of interest. This step is essential to reduce the search space and boost the performance of the unsupervised segmentation technique eliminating the number of false positives. The detection of the region of interest is based on a relatively simple pipeline taking full advantage of the contrast-enhanced MRI. In particular, a thresholding operation is first applied into the entire DCE volume highlighting the highest intensities of the volume corresponding to regions that depict the graft kidney and blood vessels. In the pixels' volume histogram, the threshold is obtained as the abscissa of maximum intensities. After the volume thresholding, morphological operations are applied to remove artifacts and fill holes in the detected regions. The morphological step consists of a binary closing followed by a binary opening. For both operations, a sphere of radius 5 and 4 have been chosen as a structural element, respectively. Principal component analysis has been used in the thresholded volume. A boundary box indicating the highest volume component was selected as the region of interest and further processed. The same area is also indicated in the T2 sequence. Areas of interest ranged for

N x × N y × N z ∈ [125, 173] × [118, 199] × [49, 75].

Unsupervised Segmentation Model

The proposed unsupervised segmentation model then processes the boundary boxes of various sizes containing the region of interest. Our method is based on differentiable feature clustering [START_REF] Kim | Unsupervised learning of image segmentation based on differentiable feature clustering[END_REF]. A simple straight-forward 3D convolutional neural network (CNN) is used to extract features and optimize two loss functions without any need for ground truth annotations. The first loss is designed to refine and extract relevant features, and the second to favor smoothness on the obtained labels.

Let us denote X ∈ R 2×Nx×Ny×Nz the regions of interest extracted from the T2 and DCE sequences. Our unsupervised method is based on a 3D feature extractor CNN denoted as h(., θ), where θ ∈ Θ defines its trainable parameters. The p-dimentional feature vector obtained by the CNN is denoted as Y = h(X, θ) ∈ R p×Nx×Ny×Nz . From this feature vector we can propose a mapping strategy defined by a function g to obtain class labels for each of the voxels of the volume

C = g(Y ) ∈ Z Nx×Ny×Nz .
Two different loss functions are defined for the optimization of the method,

L sim (Y, C) = Nx×Ny×Nz ∑ i=1 p ∑ j=1 -δ(j -c i ) ln(y j,i ) (1)
where Y = {y j,i } j∈ [1,p],i∈ [START_REF] Lee | Impact of the ratio of graft kidney volume to recipient body surface area on graft function after live donor kidney transplantation[END_REF]Nx×Ny×Nz] and C = {c j,i } j∈ [1,p],i∈ [START_REF] Lee | Impact of the ratio of graft kidney volume to recipient body surface area on graft function after live donor kidney transplantation[END_REF]Nx×Ny×Nz] and δ is the kronecker function. This loss correspond to the cross entropy between the network output and class labels used as intermediate targets that are obtained by an argmax operation.

L cont (Y ) = Nx-1 ∑ k=1 Ny-1 ∑ l=1 Nz-1 ∑ m=1 ∥y k+1,l,m -y k,l,m ∥ 1 + ∥y k,l+1,m -y k,l,m ∥ 1 + ∥y k,l,m+1 -y k,l,m ∥ 1 (2)
where y k,l,m corresponds to the p-dimensional feature vector of the voxel at (k, l, m). This loss aims to enforce spatial smoothness on the obtained segmentation masks.

Finally, the overall optimization is achieved by,

L = L sim (Y, C) + λL cont (Y ) (3)
where λ is a weight that balances the contribution of each component to the final optimization.

The network parameters are optimized iteratively until the absolute difference of loss between two successive iterations is lower than a predefined value ϵ. Using this criterion, the number of classes obtained can be superior to 2 as the non-kidney elements can rarely be assigned in one class. Using the final predicted clustering, the kidney class can easily be obtained by selecting the first connected component class.

IMPLEMENTATION DETAILS

In this study, we first applied bias field correction to all the MRI images as a preprocessing step. The number of iterations was set to 2 and the number of fitting levels to 8, enabling to enhance peaks in volumes' histograms and, more particularly, contrast-enhanced regions. Moreover, even if the two sequences (DCE and T2) correspond to the same follow-up exam, they were acquired a few minutes apart, introducing misalignments. To address this issue and project properly one sequence to the other, we used an unsupervised deformable registration method [START_REF] Ferrante | Deformable registration through learning of contextspecific metric aggregation[END_REF]. The DCE modality has been used as target, and the T2 has been warped to it, using a variety of similarity metrics, including normalized cross-correlation, mean square error, and mutual information.

Our 3D CNN architecture consists of N convolutional blocks with p channels. Each block consists of a 3D convolution layer with a kernel size of 3 × 3 × 3 and stride 1, followed by a ReLU activation and a 3D batch normalization layer. The two different sequences were concatenated channel-wise and were given into the network. The choice to include as stacked channels the two sequences -early fusion -has been made and tested over middle (classification level fusion) or late fusion (decision level fusion) strategies [START_REF] Fang | Multi-modal medical image segmentation based on vector-valued active contour models[END_REF].

Concerning the other parameters of our network, we set λ = 5 and ϵ = 0.005. Our CNN model was trained using classic stochastic gradient descent with a momentum set to 0.9 and learning rate to 0.01. The overall optimization of a single volume needed approximately 1 minute on a GeForce GTX 1080 TI GPU (11.2 Go).

DATASET

The dataset used in this study corresponds to a total of 32 patients subject to a kidney transplant. All the MRIs were obtained 12 months after the transplantation. Our dataset consists of 2 different MRI sequences, namely T2 and DCE. DCE volumes being 512 × 512 × [64 -88] included spacing ranging from 0.78 × 0.78 × 1.9 to 0.94 × 0.94×2.5 and T2 volumes being 512×512×[25-36] from 0.70×0.70×5.0 to 0.98×0.98×6.0. For the optimization of the network's hyperparameters, 3 patients were used, while all the rest (29 patients) were used for evaluation of the proposed method. The 3 patients were selected as a sample of low, medium, and high segmentation difficulty volumes from preliminary studies. Pixel-wise kidney annotation was performed by a medical expert (a boardcertified radiologist), by tracing the kidney boundaries using the DCE MRI in sagittal view. The annotation was available in every 2 or 3 slices, resulting in 945 annotated slices. We obtained the 3D annotation by interpolating the obtained manual annotations per subject to assess the method's performance.

EXPERIMENTAL RESULTS

We evaluated the graft kidney's segmentation accuracy using five metrics commonly used to assess biomedical segmentation tasks. First, the mean dice coefficient (DSC) metric is used to indicate the similarity between the manual annotation and prediction. Secondly, we used the mean of Hausdorff distance at percentile 95% (HD95) to measure how far two surfaces are from each other. Finally, the mean percentage of kidney volume difference (PKVD), precision, and recall between our predictions and the annotations were used to evaluate different methods' performance.

Starting with the evaluation of the proposed method, an ablation study benchmarking the influence of the parameter λ and the 3D CNN architecture has been performed for both 1 sequence (DCE) and 2 sequences (DCE Fig. 2. Evaluation of the different components of the proposed method using different evaluation metrics on the validation set. The first column indicates the influence of λ and the second column the influence of the used architecture for different number of channels and layers. & T2) models (Figure 2). Starting with the influence of λ, we used grid search and benchmark values in {0.01, 0.1, 1, 5, 10, 25, 50, 100}. The superiority of the 2 sequence model seems to be indicated with the best performances to be found in [START_REF] Lee | Impact of the ratio of graft kidney volume to recipient body surface area on graft function after live donor kidney transplantation[END_REF]25]. Moreover, for the 1 sequence model, the variations are less monotone, and the selection of an optimum value more difficult.

Moreover, we analyzed the impact of the network's architecture in depth (number of convolutional blocks) and width (number of channels in each block) to extract relevant features. We evaluated these parameters (N convolutional blocks, p number of channel per block) in the set of {(2,30), (2,50), [START_REF] Lundervold | An overview of deep learning in medical imaging focusing on mri[END_REF]30), [START_REF] Lundervold | An overview of deep learning in medical imaging focusing on mri[END_REF]50), (4,30), (4,50), (5,20)}. The specific parameters seem to not really influence each model's performance, with again the 2 sequences one reporting more stable performance for all the experiments. Overall, the (2,50) configuration seems to report more stable performance, with less variance, on the validation set. This indicates that a relatively simple architecture is enough to address our problem.

To highlight our method's soundness, we compare it with other commonly used unsupervised methods in the literature and its 2D implementation [START_REF] Kim | Unsupervised learning of image segmentation based on differentiable feature clustering[END_REF]. In particular, our results were evaluated against the kidney graft segmentations obtained by Otsu thresholding [START_REF] Bindu | An efficient medical image segmentation using conventional otsu method[END_REF] and watershed 3D [START_REF] Yu | A watershed method for mr renography segmentation[END_REF] methods that are commonly used for unsupervised segmentation in a variety of studies. The obtained results are summarised in the mean and standard deviation (std) of each score on the whole test set. Even if additional postprocessing methods are presented in the literature [START_REF] Will | Automated segmentation and volumetric analysis of renal cortex, medulla, and pelvis based on non-contrastenhanced t1-and t2-weighted mr images[END_REF][START_REF] Yu | A watershed method for mr renography segmentation[END_REF] in our study, we did not perform any additional postprocessing in any of the exploit methods. The lowest performance in terms of all the metrics is reported by the Otsu thresholding, which reaches a mean precision of 57.07 ± 3.03%, highlighting the high number of false positives. Watershed seems to perform better, reporting a mean DSC of 84.06% with, however, a high standard deviation. The proposed method performs better in all the cases. Using only the DCE MRI as input, we obtain better (in terms of mean values) and more stable (in terms of std) performance than the previous two methods. The performance of the proposed method is further increased with the integration of the T2 sequence. The 2 sequence model reports 3% higher DSC than the model that uses only one sequence. The method's superiority is also indicated by the higher numbers of mean precision that increases from 83.93 ± 8.32% for the DCE to 91.00 ± 5.18% for the DCE & T2 models.Even if the two models report similar mean HD95, the DCE & T2 model's impact is further indicated by the significantly lower mean PKVD metric. Finally, our experiments indicated that the 2D [START_REF] Kim | Unsupervised learning of image segmentation based on differentiable feature clustering[END_REF] implementation is not performing well, highlighting the need for 3D formulations for this problem.

In Figure 3, we present some qualitative analysis of the obtained segmentation masks. One can observe that the accurate detection of the kidney between structuresessentially between the kidney and its irrigating vesselsis very challenging for unsupervised methods. However, the proposed formulation is able to separate the kidney graft from the different structures and provide an accurate segmentation.

CONCLUSION

In this paper, we proposed a 3D deep learning-based unsupervised kidney segmentation framework using multimodal MRI. Our framework consists of two losses, one that optimizes the cross-entropy between the predicted labels and the labels of clusters, while the second one is enforcing spatial continuity, producing smooth segmentation maps. Our 3D formulation works better than other commonly used unsupervised methods in the literature, including its 2D implementation [START_REF] Kim | Unsupervised learning of image segmentation based on differentiable feature clustering[END_REF]. Our experiments indicated that integrating more than one MRI sequence could significantly help remove detected structures close to the kidney graft, such as vessels irrigating the contrast agent. Our proposed method limitations include the number of hyperparameters that need to be defined, which are the same in all the deep learning based frameworks, and a lack of kidney prior information. Our quantitative results indicate that our tool could help clinical practice provide fast and robust monitoring of the kidney graft. In the future, we would like to investigate the impact of additional loss functions that could integrate more shape specific information. Moreover, we plan to investigate our segmentation framework's adaptation to the analysis of follow-up exams for patients subject to transplantation towards potential rejection.

COMPLIANCE WITH ETHICAL STANDARDS

Our study was approved by our Institutional Review Board, which waived the need for patients' consent. Data cohort corresponds to study reference ID-RCB: 2012-A01070-43 and ClinicalTrials.gov identifier: NCT02201537. All the data used in this study were anonymized.

Fig. 3 .

 3 Fig. 3. One testing subject depicting one slice in axial view and the segmentation results as overlay for each of the evaluated methods (in blue) and the annotation from the expert (in green): (A) DCE, (B) Warped T2, (C) Otsu thresholding, (D) watershed based method, (E) Proposed (DCE), (F) Proposed (DCE & T2).

Table ,

 , .71 57.34 57.07 13.03 97.53 0.91 Watershed 84.06 23.30 5.75 0.42 16.35 23.99 89.25 23.91 80.05 23.18 Proposed 2D (DCE only) 69.09 20.34 7.22 1.67 51.30 65.00 74.57 25.77 69.79 20.48

	Method	Metric	DSC (%)	HD95 (mm)	PKVD (%)	Precision (%)	Recall (%)
	Statistic		Mean Std	Mean Std	Mean Std	Mean Std	Mean Std
	Otsu Thresholding 0.65 84Proposed (DCE only) 71.04 10.97 6.55 86.48 4.01 5.92 0.42 11.08 10.13 83.93 8.32	89.97 3.67
	Proposed 2D (DCE & T2)	75.75 10.80 6.39	1.11 32.81 41.56 81.65 14.24 81.65 14.66
	Proposed (DCE & T2)		89.81 3.17	5.82	0.41 5.88	5.74	91.00 5.18	88.20 5.82
								1 presenting

Table 1 .

 1 Quantitative evaluation of the proposed method and other methods reporting: dice coefficient (DSC), Hausdorff distance at percentile 95% (HD95), percentage of kidney volume difference (PKVD), precision and recall.


Thanh Dat

Maurice Nguyen

Giuseppe Quach

Pierre Valenzise

Duhamel

LEARNING-BASED LOSSLESS COMPRESSION OF 3D POINT CLOUD GEOMETRY

Keywords: Point Cloud Compression, Deep Learning, G-PCC, context model

This paper presents a learning-based, lossless compression method for static point cloud geometry, based on contextadaptive arithmetic coding. Unlike most existing methods working in the octree domain, our encoder operates in a hybrid mode, mixing octree and voxel-based coding. We adaptively partition the point cloud into multi-resolution voxel blocks according to the point cloud structure, and use octree to signal the partitioning. On the one hand, octree representation can eliminate the sparsity in the point cloud. On the other hand, in the voxel domain, convolutions can be naturally expressed, and geometric information (i.e., planes, surfaces, etc.) is explicitly processed by a neural network. Our context model benefits from these properties and learns a probability distribution of the voxels using a deep convolutional neural network with masked filters, called VoxelDNN. Experiments show that our method outperforms the state-of-the-art MPEG G-PCC standard with average rate savings of 28% on a diverse set of point clouds from the Microsoft Voxelized Upper Bodies (MVUB) and MPEG. The implementation is available at https://github.com/Weafre/VoxelDNN.

INTRODUCTION

Recent visual capturing technology has enabled 3D scenes to be captured and stored in the form of Point Clouds (PCs). PCs are now becoming the preferred data structure in many applications (e.g., VR/AR, autonomous vehicle, cultural heritage), resulting in a massive demand for efficient Point Cloud Compression (PCC) methods.

The Moving Picture Expert Group has been developing two PCC standards [START_REF] Schwarz | Emerging MPEG Standards for Point Cloud Compression[END_REF][START_REF] Jang | Video-Based Point-Cloud-Compression Standard in MPEG: From Evidence Collection to Committee Draft [Standards in a Nutshell[END_REF][START_REF] Graziosi | An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)[END_REF]: Video-based PCC (V-PCC) and Geometry-based PCC (G-PCC). V-PCC focuses on dynamic point clouds and is based on 3D-to-2D projections. On the other hand, G-PCC targets static content and encodes the point clouds directly in 3D space. In G-PCC, the geometry and attribute information are independently encoded. However, the geometry must be available before filling point clouds with attributes. Therefore, having an efficient lossless geometry coding is fundamental for efficient PCC. A typical point cloud compression scheme consists in pre-quantizing the geometric coordinates using voxelization. In this paper, we also adopt this approach, which is particularly suited for dense point clouds. After voxelization, the point cloud geometry can be represented either directly in the voxel domain, or using an octree spatial decomposition.

In this work, we propose a deep-learning-based method for lossless compression of voxelized point cloud geometry. Using a masked 3D convolutional network, our approach (named VoxelDNN) first learns the distribution of a voxel given all previously decoded ones. This conditional distribution is then used to model the context of a context-based arithmetic coder. In addition, we reduce point cloud sparsity by adaptively partitioning the PC. We demonstrate experimentally that the proposed solution outperforms the MPEG G-PCC solution in terms of bits per occupied voxel with average rate savings of 28% on all test datasets.

The rest of the paper is structured as follows: Section 2 reviews the related work; the proposed method is described in Section 3; Section 4 presents the experimental results; and finally Section 5 concludes the paper.

RELATED WORK

Most existing point cloud geometry compression methods, including MPEG G-PCC test models, represent and encode occupied voxels using octrees [START_REF] Garcia | Context-based octree coding for point-cloud video[END_REF][START_REF] Garcia | Intra-Frame Context-Based Octree Coding for Point-Cloud Geometry[END_REF][START_REF] Garcia | Geometry Coding for Dynamic Voxelized Point Clouds Using Octrees and Multiple Contexts[END_REF] or local approximations called "triangle soups" [START_REF] Dricot | Adaptive multi-level triangle soup for geometry-based point cloud coding[END_REF]. Recently, the authors of [START_REF] Garcia | Geometry Coding for Dynamic Voxelized Point Clouds Using Octrees and Multiple Contexts[END_REF] proposed an intra-frame method called P(PNI), which builds a reference octree by propagating the parent octet to all children nodes, thus providing 255 contexts to encode the current octant. A frequency table of size 255 × 255 is built to encode the octree and needs to be transmitted to decoder. A drawback of this octree representation is that, at the first levels of the tree, it produces "blocky" scenes, and geometry information of point clouds (i.e., curve, plane) is lost. Instead, in this paper we work in the hybrid domain to exploit the geometry information. In addition, our method predicts voxel distributions in a sequential manner at the decoder side, thus avoiding the extra cost of transmitting large frequency tables.

Our work draws inspiration from the recent advances in deep generative models for 2D images. The goal of generative models is to learn the data distribution, which can be used for a variety of tasks, with image generation being probably the most popular [START_REF] Goodfellow | Deep learning[END_REF]. Among the various classes of generative models, we consider methods able to explicitly estimate data likelihood, such as the PixelCNN model [START_REF] Oord | Pixel Recurrent Neural Networks[END_REF][START_REF] Salimans | PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications[END_REF]. Specifically, these approaches factorize the likelihood of a picture by modeling the conditional distribution of a given pixel's color given all previously generated pixels. PixelCNN models the distribution using a neural network. The causality constraint is enforced using masked filters in each convolutional layer. Recently, this approach has been employed in image compression to yield accurate and learnable entropy models [START_REF] Mentzer | Conditional Probability Models for Deep Image Compression[END_REF]. This paper extends the generative model and masking filters to 3D point cloud geometry compression.

Inspired by the success in learning-based image compression, deep learning has been recently adopted in point cloud coding methods [START_REF] Huang | OctSqueeze: Octree-Structured Entropy Model for Li-DAR Compression[END_REF][START_REF] Guarda | Point cloud coding: Adopting a deep learning-based approach[END_REF][START_REF] Guarda | Point cloud geometry scalable coding with a single end-to-end deep learning model[END_REF][START_REF] Quach | Learning Convolutional Transforms for Lossy Point Cloud Geometry Compression[END_REF][START_REF] Quach | Improved Deep Point Cloud Geometry Compression[END_REF]. The proposed methods in [START_REF] Quach | Learning Convolutional Transforms for Lossy Point Cloud Geometry Compression[END_REF][START_REF] Quach | Improved Deep Point Cloud Geometry Compression[END_REF] encode each 64 × 64 × 64 sub-block of PC using a 3D convolutional auto-encoder. In contrast, in this paper we losslessly encode the voxels by directly learning the distribution of each voxel from its 3D context. We also offer block-based coding, which was successful in traditional image and video coding.

PROPOSED METHOD

Definitions

A point cloud voxelized over a 2 n × 2 n × 2 n grid is known as an n-bit depth PC, which can be represented by an n level octree. In this work, we represent point cloud geometry in a hybrid manner: both in octree and voxel domain. We coarsely partition an n-depth point cloud up to level n -6. As a result, we obtain a n -6 level octree and a number of non-empty binary blocks v of size 2 6 × 2 6 × 2 6 voxels, which we refer to as resolution d = 64 or simply block 64 in the following. Blocks 64 can be further partitioned at resolution d = {32, 16, 8, 4} as detailed in Section 3.3. At this point, we encode the blocks using our proposed encoder in the voxel domain (Section 3.2). The high-level octree, as well as the depth of each block, are converted to bytes and signaled to the decoder as side information. We index all voxels in block v at resolution d from 1 to d 3 in raster scan order with:

v i = 1, if i th voxel is occupied 0, otherwise. (1)

VoxelDNN

Our method encodes the voxelized point cloud losslessly using context-adaptive binary arithmetic coding. Specifically, we focus on estimating accurately a probability model p(v) for the occupancy of a block v composed by d × d × d voxels.

We factorize the joint distribution p(v) as a product of conditional distributions p(v i |v i-1 , . . . , v 1 ) over the voxel volume: Each term p(v i |v i-1 , . . . , v 1 ) above is the probability of the voxel v i being occupied given all previous voxels, referred to as a context. Figure 1(a) illustrates an example 3D context. We estimate p(v i |v i-1 , . . . , v 1 ) using a neural network which we dub VoxelDNN.

p(v) = d 3 Π i=1 p(v i |v i-1 , v i-2 , . . . , v 1 ). (2)
The conditional distributions in (2) depend on previously decoded voxels. This requires a causality constraint on the VoxelDNN network. To enforce causality, we extend to 3D the idea of masked convolutional filters, initially proposed in PixelCNN [START_REF] Oord | Pixel Recurrent Neural Networks[END_REF]. Specifically, two kinds of masks (A or B) can be employed. Type A mask is filled by zeros from the center position to the last position in raster scan order as shown in Figure 1(b). Type B mask differs from type A in that the value in the center location is 1. We apply type A mask to the first convolutional layer to restrict both the connections from all future voxels and the voxel currently being predicted. In contrast, from the second convolutional layer, type B masks are applied which relaxes the restrictions of mask A by allowing the connection from the current spatial location to itself.

In order to learn good estimates p(v i |v i-1 , . . . , v 1 ) of the underlying voxel occupancy distribution p(v i |v i-1 , . . . , v 1 ), and thus minimize the coding bitrate, we train VoxelDNN using cross-entropy loss. That is, for a block v of resolution d, [START_REF] Quach | Learning Convolutional Transforms for Lossy Point Cloud Geometry Compression[END_REF][START_REF] Quach | Improved Deep Point Cloud Geometry Compression[END_REF]. The motivation behind using focal loss is to cope with the high spatial unbalance between occupied and non-occupied voxels. The reconstructed PC is then obtained by hard thresholding p(v), and the target is thus the final classification accuracy. Conversely, here we aim at estimating accurate soft probabilities to be fed into an arithmetic coder.

H(p, p) = E v∼p(v)   d 3 i=1 -log p(v i )   . (3)
Figure 2 shows our VoxelDNN network architecture. Given the 64 × 64 × 64 input block, VoxelDNN outputs the predicted occupancy probability of all input voxels. Our first 3D convolutional layer uses 7 × 7 × 7 kernels with a type A mask. Type B masks are used in the subsequent layers.

To avoid vanishing gradients and speed up the convergence, we implement two residual connections with 5 × 5 × 5 kernels. Throughout VoxelDNN, the ReLu activation function is applied after each convolutional layer, except in the last layer where we use softmax activation. In total, our model contains 290,754 parameters and requires less then 4MB of disk storage space.

Multi-resolution encoder and adaptive partitioning

We use an arithmetic coder to encode the voxels sequentially from the first voxel to the last voxel of each block in a generative manner. Specifically, every time a voxel is encoded, it is fed back into VoxelDNN to predict the probability of the next voxel. Then, we pass the probability to the arithmetic coder to encode the next symbol.

However, applying this coding process at a fixed resolution d (in particular, on blocks 64), can be inefficient when blocks are sparse, i.e., they contain only few occupied voxels. This is due to the fact that in this case, there is little or no information available in the receptive fields of the convolutional filters. To overcome this problem, we propose a rateoptimized multi-resolution splitting algorithm as follows. We partition a block into 8 sub-blocks recursively and signal the occupancy of sub-blocks as well as the partitioning decision (0: empty, 1: encode as single block, 2: further partition). The partitioning decision depends on the output bits after arithmetic coding. If the total bitstream of partitioning flags and occupied sub-blocks is larger than encoding parent block as a single block, we do not perform partitioning. The details of this process are shown in Algorithm 1. The maximum partitioning level is controlled by maxLv and partitioning is performed up to maxLv = 5 corresponding to a smallest block size of 4. Depending on the output bits of each partitioning solution, a block of size 64 can contain a combination of blocks with different sizes. Figure 3 shows 4 partitioning examples for an encoder with maxLv = 4. Note that Vox-elDNN learns to predict the distribution of the current voxel from previous encoded voxels. As a result, we only need to train a single model to predict the probabilities for different input block sizes.

Table 1 reports the average rate in bits per occupied voxel (bpov) of the proposed method for 4 partitioning levels, compared with G-PCC v.12 [START_REF] Graziosi | An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)[END_REF]. We also report the results of the recent intra-frame geometry coding method P(PNI) from [START_REF] Garcia | Geometry Coding for Dynamic Voxelized Point Clouds Using Octrees and Multiple Contexts[END_REF] (the coding results are available only for some of the tested PCs). The results with 5 partitioning levels are identical to 4 partitioning levels and are not shown in the table. In all experiments, the total size of signaling bits for the high-level octree and partitioning accounts for less than 2% of the bitstream.

We observe that our proposed solution at all 4 levels and G-PCC outperform P(PNI) by a large margin. VoxelDNN solutions outperform G-PCC on the MVUB and MPEG 8i datasets with the highest average rate saving of 28.4% and 29.9%, respectively. As partitioning levels increases, the corresponding gain over G-PCC also increases; however, there is only a slight increase with 3 and 4 levels compared to the gain of the lower level. This can be explained with Figure 4, which shows the percentage allocation of the encoded points in each partitioning size, for different PCs, after optimal partitioning. It can be seen that most voxels are encoded using blocks 64 and 32, while very few ones are encoded with blocks of smaller size. While adding more partition levels enables to better adapt to point cloud geometry, smaller partitions entail a higher signalization cost. This is not often compensated by a bitrate reduction of the sub-blocks, since in the smaller partitions the encoder has access to limited contexts, resulting in less accurate probability estimations.

CONCLUSIONS

This paper proposed a hybrid octree/voxel-based lossless compression method for point cloud geometry. It employs for the first time a deep generative model in the voxel space to estimate the occupancy probabilities sequentially. Combined with a rate-optimized partitioning strategy, the proposed method outperforms MPEG G-PCC with average 28% rate savings over all tested datasets. We are now working on improving lossless coding of PC geometry by using more powerful generative models, and jointly optimizing octree and voxel coding.

  Fig. 1 (a): Example 3D context in a 5 × 5 × 5 block. Previously scanned elements are in blue. (b): 3 × 3 × 3 3D type A mask. Type B mask is obtained by changing center position (marked red) to 1.

Fig. 2

 2 Fig. 2 VoxelDNN architecture. A type A mask is applied in the first layer (dashed borders) and type B masks afterwards. 'f64,k7,s1' stands for 64 filters, kernel size 7 and stride 1.

Algorithm 1 : 4 if b is empty then 5 child 15 f

 14515 Block partitioning selectionInput: block: B, current level: curLv, max level: maxLv Output: partitioning flags: f l, output bitstream: bits 1 Function partitioner(B, curLv, maxLv): 2 f l2 ← 2 ; // encode as 8 child blocks 3 for block b in child blocks of B do , child bit ← partitioner(b, curLv + 1,maxLv); l2 ← [f l2, child f lag]; 16 bit2 ← [bit2, child bit]; 17 end 18 total bit2 = sizeOf (bit2) + len(f l2) × 2; 19 f l1 ← 1; // encode as a single block 20 bit1 ← singleBlockCoder(B); 21 total bit1 = sizeOf (bit1) + len(f l1) × 2;

Fig. 3

 3 Fig. 3 Partitioning a block of size 64 into: (a) a single block of size 64, (b): blocks of size 32, (c): 32 and 16, (d): 32, 16 and 8. Non-empty blocks are indicated by blue cubes.

Fig. 4

 4 Fig. 4 Percentage of encoded points in each block size. From top to bottom: block 8, 16, 32, 64.

Evaluation procedure: We evaluate our methods on both 9 and 10 bits depth PCs that have the lowest and highest rate performance when testing with G-PCC method from the MVUB and MPEG datasets. These PCs were not used during training. The effectiveness of the partitioning scheme is evaluated by increasing the maximum partitioning level from 1 to 5 corresponding to block sizes 64, 32, 16, 8 and 4.

Experimental results