Marco Calemme
email: calemme@telecom.paristech.fr

Senior Member, IEEE Marco Cagnazzo
email: cagnazzo@telecom.paristech.fr

Béatrice Pesquet-Popescu

Fellow, Beatrice Pesquet-Popescu
email: beatrice.pesquet@telecom.paristech.fr

Contour-Based Depth Coding: A Subjective Quality Assessment Study

Keywords: perceived quality, quality assessment, 3D video, multiple-views-plus-depth, contour coding, elastic curves

come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

The video-plus-depth representation for multi-view video sequences (MVD) consists of several views of the same scene with their associated depth information, which is the distance from the camera for every point in the view [START_REF]Emerging technologies for 3D video: content creation, coding, transmission and rendering[END_REF]. Depth information allows synthesizing virtual view points, for such applications as 3D television and free-viewpoint video, but it requires ad-hoc compression techniques, since those developed for texture images are not well suited for depths. As a consequence, MVD compression has attracted a huge amount of research effort in the last years, while ISO and ITU are jointly developing an MVD coding standard [START_REF] Sullivan | Standardized extensions of high efficiency video coding (HEVC)[END_REF].

A key point in depth images compression is that they are not meant to be visualized but only used for rendering of virtual views. Various techniques have been specifically proposed to code the depth information associated to the views and recent approaches include contour-based and object-based coding of depths. This approach seems reasonable because the properties of depth maps differ greatly from the ones of texture images. Objects within a depth map are usually arranged along planes in different perspectives; as a consequence there are areas of smoothly varying levels, separated by sharp edges which correspond to object boundaries. It is generally recognized that a high quality view rendering at the receiver side is possible only by preserving the contour information [START_REF] Gautier | Efficient depth map compression based on lossless edge coding and diffusion[END_REF], [START_REF] Daribo | Arithmetic edge coding for arbitrarily shaped sub-block motion prediction in depth video compression[END_REF], [START_REF] Shen | Edge-adaptive transforms for efficient depth map coding[END_REF], since distortions on edges during the encoding step would cause a sensible degradation on the synthesized view and on the 3D perception. The typical artifact consists in fact in a slight displacement of an object, with clear sharp edges. This leads to low scores for objective metrics like PSNR, which are very sensitive to this kind of errors, and at the same time the visual quality is claimed to be high. We want to investigate this claim by conducting for the first time a subjective quality assessment test to compare an object-based technique and a hybrid block-based techniques for the coding of depth maps.

In a previous work we proposed a technique using elastic deformation of curves to losslessly encode the object contours, and shape-adaptive wavelet to represent the inner parts of the objects [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF]. In a practical bit-rate range, this technique proved to be competitive with HEVC [START_REF] Sullivan | Overview of the high efficiency video coding (HEVC) standard[END_REF], stateof-the-art for hybrid block-based coding techniques, in terms of objective quality metrics (PSNR and SSIM). At that time, only very preliminary subjective evaluations were performed for this technique. Just like other contour-based methods, this techniques present totally different artifacts in synthesized images with respect to HEVC, and their impact is difficult to evaluate with classical metrics like the PSNR. We have thus set up a subjective test to make a quality assessment of synthesized views.

II. CODING OF DEPTH MAPS

Lossless contour coding using elastic curves. Srivastava et al. [START_REF] Srivastava | Shape analysis of elastic curves in euclidean spaces[END_REF] introduced a framework to model a continuous evolution of elastic deformations between two reference curves. According to the interpretation of the elastic metric, it is relatively easy to compute the geodesic between two curves: it consists in a continuous set of deformations that transforms one curve into another with a minimum amount of stretching and blending, and independently from their absolute position, scale, rotation and parametrization. The referred technique thus interpolates between shapes and makes the intermediary curves retain the global shape structure and the important local features such as corners and bends. An example of the geodesic connecting two curves is shown in Fig. 1. We show in black two contours extracted from a MVD sequence, corresponding to views 1 and 8. The curves in red are the contours extracted from the intermediate views, while in dashed blue we show a sampling of the elastic geodesic computed between the two extreme curves. The elastic deformations along the geodesic reproduce very well the deformations related to a change of viewpoint or a temporal evolution of an object in a sequence, given the initial and final shapes. Supposing that the encoder and the decoder share a representation of the initial and final shape, they can reproduce exactly the same geodesic path between them. Then, the decoder will use a suitable point of the geodesic, i.e. one of the dashed curves in Fig. 1, as context [START_REF] Cagnazzo | Mutual information-based context quantization[END_REF] to encode an intermediary contour (one of the solid curves in the same figure). The encoder will only have to send a value in [0, 1] to let the decoder identify this curve. The lossless coding of the contour is performed through an arithmetic coder, and the input symbol probability distribution is modified on the fly according to the elastic prediction [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF].

Object-based depth maps coding technique. A segmentation map of the scene with different objects can be coded with our lossless contour coding technique, and this map can be used in conjunction with an object-based coding technique to code the depth images. The method proposed in [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF] relies on the SA (Shape Adaptive) Wavelet Transform [START_REF] Li | Shape-adaptive discrete wavelet transforms for arbitrarily shaped visual object coding[END_REF], followed by SA SPIHT (Set Partitioning In Hierarchical Trees) [START_REF] Cagnazzo | Region-oriented compression of multispectral images by shape-adaptive wavelet transform and SPIHT[END_REF], followed by an arithmetic coder for the SPIHT symbols (memoryless, without context). This provides an entirely Intra technique for the inner part of the objects. From now on we will refer to this technique as "CC", contour coding.

Advantages of lossless coding. The validity of the objectbased approach is suggested by the relevance of the contour information in the synthesis step. A minimum bit budget is needed for a lossless representation of contours and this initial cost may be a relevant fraction of the total rate dedicated to the depth map. One question that arises is whether the benefits of a lossless contour will actually be perceived as relevant against its bit-rate cost. We investigated on this subject by simply skipping the lossless coding part in our coding technique and using the elastic prediction to directly generate the segmentation map to feed the objectbased coder. We rely on the SA SPIHT block to correct the possible imprecision of the coded depth map. This technique allows the rates to be lowered dramatically, however the quality of the synthesized images obtained from the depths compressed with this simple technique must be carefully evaluated. From now on we will refer to this technique as "NR", no refinement.

A concise scheme of the two methods is shown in Fig. 2. As inputs we have a depth image and its extracted contours, plus two reference contours to perform the elastic interpolation. First the elastic prediction is performed (EP block); then in case of method CC it is used in conjunction with the original contours of the depth image to perform the lossless contour coding (LLCC block); the depth image in input, with the coded (method CC) or predicted contours (method NR), arrange a segmentation map (Segm. block), and its objects are coded with a SA wavelet transform (SA WT block), followed by SA SPIHT.

III. PRODUCTION OF TEST MATERIAL

The multiview sequences beergarden (provided by Philips), lovebird (ETRI/MPEG Korea Forum) and mobile (Philips) have been used for test, an example for each content is shown in Fig. 3 The block EP performs the elastic prediction and takes as input two reference contours; the LLCC block performs the lossless contour coding using an original contour and its elastic prediction as context; the Segm. block arranges the segmentation map of the depth image with the coded or predicted contours, and it passes the objects of the segmentation map to the SA WT block, which performs the shape adaptive wavelet transform, followed in its turn by the shape adaptive SPIHT, represented by the SA SPIHT block.

compressing the selected images using the techniques CC, NR, and HEVC. For HEVC, used in mode Intra, we have chosen as QPs: 27, 38, 44, 51. QP 51 is the maximum level of compression allowed, corresponding to the smallest bit-rate for HEVC. For CC, the use of SA-SPIHT allows to chose the exact bit-rate, so we have chosen the lowest possible rate and the bit-rates corresponding to QPs 51, 44, and 38. And as for NR, we have chosen again the lowest possible rate and other two to match the low rates of the two other techniques. Details about the frames and views used for each sequence are reported in Tab. I. Moreover the whole dataset is available on the authors' website [12].

The use of the technique NR for this test is important to evaluate the effects of contour coding on the synthesized image. We will show in fact that the precision of the contour leads to remarkable gains in terms of perceived quality. Moreover we used this technique at a very low bit rate as the reference for the "very annoying" quality, for the evident artifacts that it produces on the synthesized image. If the predicted contour lies outside the object and the interior part is coded very coarsely, we will produce a change in the geometry of the scene, that will result in a "gap" in between the objects of the synthesized image.

Once the depths are compressed for two views, we used them to generate new synthetic views in between the two with the Depth-Image-Based Rendering (DIBR) software [START_REF] Fehn | Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV[END_REF], as shown in Fig. 4. The use of synthesized images for the test is justified by the plausible scenario of Free Viewpoint Video (FVV) [START_REF] Bosc | Towards a new quality metric for 3-d synthesized view assessment[END_REF].

To perform the elastic prediction we need two reference curves, be them from two different views or two different time instants of the same view. These curves represent a sort of "intra" curves and they are coded independently from the others (and losslessly, for the techniques CC and NR). The gap or distance between the two reference curves along the view or the time axis affects the final rate: the accuracy of the prediction decreases with the distance, but at the same time a wider gap between the two reference curves allows a lower bit-rate. The distance of the reference curves for the elastic prediction has to be chosen for the techniques CC and NR. To better illustrate this concept, in Fig. 4 we marked with a blue R the frames used to extract a reference curve, a blue arrow representing the geodesic path from one to another, and the distance from a reference curve to the other is defined as the number of frames that exist in between the two. For both the techniques the chosen value is three frames in between the two reference ones and we perform the elastic prediction along the time axis. The synthesis is then performed using the compressed depths of two views and their associated texture images.

Regarding the reference images, the unavailability of intermediate views and the presence of evident synthesis artifacts lead us to the use of synthesized reference images, obtained with the original uncompressed depths, as suggested in [START_REF] El-Yamany | Evaluation of depth compression and view synthesis distortions in multiview-video-plus-depth coding systems[END_REF].

IV. EXPERIMENTAL SETUP

Test environment. The images were displayed on a DELL P2210 screen, which has a resolution of 1680 × 1050 pixels. We set up a test space with mid gray non reflective background, isolated from external sources of light, as recommended in [START_REF]Methodology for the subjective assessment of the quality of television pictures[END_REF], [START_REF]General viewing conditions for subjective assessment of quality of SDTV and HDTV television pictures on flat panel displays[END_REF]. To avoid direct light sources in the field of view of the user, except for the screen, we placed a lamp at 6500K color temperature behind the screen to provide ambient illumination. The resulting ambient light measured in front of the screen, when this was off, was approximately of 10 cd/m 2 . Viewers participated to test sessions one at the time, sitting in front of the screen at a distance of its diagonal approximately, which corresponds to an angular resolution of about 35 pixels per degree.

Test methodology. The subjective evaluation has been performed following the Double Stimulus Impairment Scale (DSIS) methodology [START_REF]Methodology for the subjective assessment of the quality of television pictures[END_REF]. For each "round" a pair of images has been proposed to the user, stimuli A and B, in which the stimulus A was always the reference, and stimulus B was the image to be evaluated. The reference image is obtained using uncrompressed depths in the synthesis step, while the image to be evaluated is obtained with compressed depths. The viewer was informed of the presence of the reference image in the pair, and was asked to rate the quality of the second image with respect to the first one, using a continuous scale ranging from 0 to 100, in which five ranges were associated with five distinct adjectives ("Very annoying", "Annoying", "Slightly annoying", "Perceptible", "Imperceptible"). Each image was shown for 7 seconds and preceded by a gray screen to indicate the stimulus (A or B) for 1.5 seconds. Every round was composed of: stimulus A, stimulus B, stimulus A, stimulus B, voting screen. The voting screen is a pop-up window on mid gray background. The voting window contains a continuous slider with the five adjectives on its side, and a text box that reports the rating value and again the correspondent adjective with a larger font size to be more evident.The pairs of stimuli were presented in random order. The whole test took around 22 minutes to complete for each user.

V. RESULTS

A panel of twenty people took part in our test, 6 women and 14 men, aged from 23 to 32, with an average of 27.65 years. The subjects reported visual acuity or if needed wore corrective glasses.

The subjective scores were screened in order to detect and remove possible outliers, subjects whose scores differ greatly from the others'. We followed the procedure described in [START_REF]Methodology for the subjective assessment of the quality of television pictures[END_REF] for the DSIS test methodology. As we have not done any assumption on the distribution characteristics, values outside the interquartile range (from the 25% and the 75% percentiles) for more than 1.5 times are labeled outliers, and if there is an incidence of outliers in a subject's score of more than the 10%, he is considered an outlier and his scores are not taken in account. In our results three subjects have been marked as outliers and their scores have been discarded.

After the outlier removal, we verified that for each stimulus the score distribution is unimodal and we computed the Mean Opinion Scores (MOS), along with the 95% confidence interval (CI), with the assumption that the scores are following a t-Student distribution.

Objective measurements and perceived quality. In addition to the MOS, the images were also evaluated through different objective metrics. We considered pixel-based metrics: Peak Signal-to-Noise Ratio (PSNR) and Weighted Signalto-Noise Ratio (WSNR) [START_REF] Mitsa | Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms[END_REF]; as well as non-pixel-based metrics: Multi-scale Structural Similarity (MSSIM) [START_REF] Wang | Image quality assessment: from error visibility to structural similarity[END_REF] and Visual Information Fidelity (VIF) [START_REF] Sheikh | Image information and visual quality[END_REF]. In Fig. 5 the results of our subjective test and all the considered objective metrics are reported. Each column refers to a sequence, and each row to a metric. In particular, the computed MOS scores with their 95% CI are reported in the first row.

As we can see in Fig. 5, HEVC exhibits very regular trends in its MOS scores. The resulting visible artifacts are mainly edge fragmentation and blocking effect, which disappear as the bit-rate increases. MOS scores of HEVC generally vary from the "perceptible" to the "imperceptible" range. The technique CC exhibits in general very high scores compared to the other techniques (most of its scores are in fact in the "Imperceptible" range), thus the coding of the contours is generally worth its cost for the perceived quality of the resulting synthesized images. This is especially true if we compare the MOS scores of CC to the ones of NR at the same bit-rate: imperfections in the contour can lead to annoying artifacts and make the final quality drop, even if they are diminished by a finer coding of the interior part. Moreover we see that at low bit-rates a perfect contour and a coarsely coded interior part tend to lead to an excellent perception of the synthesized image. In contrast to this assertion we find slightly lower scores at low bitrates for the content beergarden, where the proximity of the objects to the camera demands a less coarse coding of the interior part to reach a very high level of perceived quality.

Dealing with objective metrics, again in Fig. 5, at low bit-rates HEVC generally shows better results. Pixel-based metrics like PSNR are in fact very sensitive to the most prominent artifact produced by the CC coding technique: the different geometry of the scene, given by the coarsely coded interior part, results in a slightly different disposition of the objects, with respect to the image synthesized from uncompressed depths. To a human observer it can be very difficult to tell apart the two images, but even the shifting of few pixels for an object or the background can cause a low PSNR. On the other hand, in terms of PSNR, the technique NR proves to be competitive with the technique CC. The explanation lies in the fact that very localized errors, especially if they affect the shape of the objects, are perceived as annoying by a human observer, and the tested objective metrics are not able to take this effect into account.

Apart from PSNR, also the more refined objective metrics WSNR, MSSIM and VIF produce scores that can differ greatly from the scores produced during the subjective test. The table II reports the Spearman correlation coefficients (in modulus) calculated for each content and technique. The metrics with the highest overall correlation coefficient are WSNR (0.68) and MSSIM (0.62), followed by the PSNR (0.56) and finally the VIF (0.55). While dealing with some of the techniques or some of the contents these objective metrics could provide a good subjective MOS prediction, but they are not consistent across several techniques and contents, in particular they showed a low correlation coefficient for the method CC.

VI. CONCLUSIONS

This paper addresses the issue of evaluating the effects of contour-preserving compression techniques on synthesized images by means of a subjective test. We compared, with subjective results and objective metrics, HEVC, the technique proposed in [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF], and the presented simple technique NR. The positive impact of contour preserving depth coding was confirmed by our study. Moreover by the comparison of subjective results and objective metrics it is also clear that the quality assessment of synthesized images in MVD can still present unresolved problems: algorithms can introduce non-perceptible or non-annoying artifacts and commonly used objective metrics can assign low scores for them even if for a human observer the degradation is relatively acceptable. Further experiments should be conducted to take into account also temporal artifacts and 3D video perception.

Figure 1 .

 1 Figure 1. Geodesic path of elastic deformations bs from the curve i 0 to i 1 (in dashed blue lines). b 3 is one of the contours bt extracted from the intermediate frames between the two reference ones, a good matching elastic curve b0.2 along the path is highlighted.

Figure 3 .

 3 Figure 3. Contents of the different sequences and their resolution.

Figure 2 .

 2 Figure 2. Coding scheme for the techniques CC (Contour Coding) and NR (No Refinement).The block EP performs the elastic prediction and takes as input two reference contours; the LLCC block performs the lossless contour coding using an original contour and its elastic prediction as context; the Segm. block arranges the segmentation map of the depth image with the coded or predicted contours, and it passes the objects of the segmentation map to the SA WT block, which performs the shape adaptive wavelet transform, followed in its turn by the shape adaptive SPIHT, represented by the SA SPIHT block.

Figure 4 .

 4 Figure 4. Elastic prediction and synthesis scheme.

Figure 5 .

 5 Figure 5. MOS scores with the 95% CI and different objective metrics for the three test sequences.

  . We produced the test material by

			NR
	Original	LL
	Contour	CC	CC
	Reference Contours	EP	CC NR	Segm.	SA WT	SA SPIHT
				objects
	Depth
	Image
				content	frames view L	goal	view R	crop
				beergarden	54-58	5	5.25	6	840×896
				lovebird	1-5	6	7	8	1024×768
				mobile	43-47	3	3.75	4	720×536
							Table I
				FRAMES USED FOR COMPRESSION AND SYNTHESIS FOR EACH
							SEQUENCE.

Table II SPEARMAN

 II CORRELATION COEFFICIENTS (IN MODULUS) CALCULATED FOR EACH CONTENT AND TECHNIQUE.

		overall beergarden lovebird	mobile	NC	CC	HEVC
	MSSIM	0.62	0.97	0.52	0.46	0.75 0.13	0.80
	VIF	0.55	0.97	0.50	0.37	0.52 0.02	0.79
	PSNR	0.56	0.96	0.50	0.35	0.52 0.15	0.75
	WSNR	0.68	0.99	0.63	0.45	0.80 0.14	0.79


Marco Calemme

Pietro Zanuttigh

Simone Milani

Marco Cagnazzo

Beatrice Pesquet-Popescu

Depth map coding with elastic contours and 3D surface prediction

Keywords: Depth Maps, Contour Coding, Elastic Deformation, Segmentation, HEVC

  L'archive ouverte pluridisciplinaire

INTRODUCTION

Depth maps can be represented by means of grayscale images and compressed with standard image compression tools. Better results can be obtained by exploiting ad-hoc approaches tailored to the specific properties of these representations and in particular the fact that depth maps are usually made by a set of smooth surfaces separated by sharp edges. The proposed compression scheme follows this rationale and is based on the idea of representing the depth field as a set of contours constraining the various regions together with a compact representation of the depth field inside each region. This allows to preserve the sharp edges between the various surfaces, a relevant property since the positive impact of contour-preserving compression techniques on synthesized images, obtained through DIBR approaches [START_REF] Fehn | Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV[END_REF], has been recently confirmed by means of subjective tests [START_REF] Calemme | Contour-based depth coding: a subjective quality assessment study[END_REF].

Segmentation information has been widely used for depth compression but the coding of the segments shape is critical and most available approaches are not able to propagate the information across multiple frames. An efficient contour coding technique has recently been proposed in [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF]: it uses elastic deformation of curves in order to exploit the temporal or inter-view redundancy of object contours and code more efficiently the contour information to be used for depth compression. After the main discontinuities have been captured by the contour description, the depth field inside each region is smooth and can be efficiently predicted from just a small number of samples. For this task we exploited an approach derived from [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF].

The general workflow of the proposed approach is shown in Fig. 1. The first step is the segmentation of the depth maps in order to extract the main objects and structures in the scene. The segment contours are then compressed using the approach described in [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF] and the average depth value in each segment is stored. The depth maps and the segmentation data are subsampled and the difference between the subsampled representation and the segment averages is compressed using a differential prediction scheme followed by the HEVC coder. Then a surface prediction algorithm derived from [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] is used to predict the input depth maps from the subsampled data and the contours. Finally, the residuals between the prediction and the input depth map are lossy compressed using the HEVC coder.

RELATED WORK

Many depth coding approaches exploits the key idea that depth maps are made of smooth regions divided by sharp edges. A possible solution is to exploit a segmentation of the depth map in order to decompose it into a set of homogeneous regions that can be represented by simple models or low resolution approximations. Examples of approaches belonging to this family are [START_REF] Zhu | View synthesis oriented depth map coding algorithm[END_REF][START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF][START_REF] Milani | A Depth Image Coder Based on Progressive Silhouettes[END_REF]. The approach of [START_REF] Zhu | View synthesis oriented depth map coding algorithm[END_REF] adopts a segmentation of the depth image followed by region-based coding scheme, while the solution in [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] exploits segmentation followed by a linear prediction scheme, and finally, the scheme in [START_REF] Milani | A Depth Image Coder Based on Progressive Silhouettes[END_REF] exploits a scalable decomposition into progressive silhouettes.

Another possibility is to exploit adaptive interpolation schemes able to correctly handle edge regions, for example the approach in [START_REF] Oh | Depth reconstruction filter and down/up sampling for depth coding in 3-d video[END_REF] subsamples the depth information and then reconstructs it with an adaptive interpolating filter. The coding scheme in [START_REF] Morvan | Depth-Image Compression based on an R-D Optimized Quadtree Decomposition for the Transmission of Multiview Images[END_REF] exploits platelets with support areas adapted to the object boundaries. Wavelet-based schemes have also been used, the solution in [START_REF] Sanchez | Edge-preserving depth-map coding using graph-based wavelets[END_REF] uses an edgepreserving lifting scheme while the approach in [START_REF] Mathew | Scalable coding of depth maps with r-d optimized embedding[END_REF] exploits a breakpoint field representing crack edges in order to avoid filtering across the edges in the wavelet reconstruction. The idea of exploiting crack edges is used also in [START_REF] Tabus | Context coding of depth map images under the piecewise-constant image model representation[END_REF].

Notice that these families of approaches need a representation of the segmentation or edge information that can have a relevant impact on the total bitrate, for this task efficient contour coding schemes are needed. Typical contour coding techniques rely on chain coding and differential chain coding [START_REF] Katsaggelos | MPEG-4 and rate-distortion-based shape-coding techniques[END_REF], or on polygon approximation [START_REF] Kim | Generalized predictive binary shape coding using polygon approximation[END_REF]. Even if they are quite effective, these techniques do not exploit the temporal redundancy of contours of the same objects in different time instants or views.

Color data can also been used to assist the compression of depth information as in [START_REF] Forster | Rgb-guided depth map compression via compressed sensing and sparse coding[END_REF], [START_REF] Milani | Efficient depth map compression exploiting segmented color data[END_REF], [START_REF] Milani | 3D Video Coding via Motion Compensation of Superpixels[END_REF] and [START_REF] Georgiev | Depth map compression using color-driven isotropic segmentation and regularised reconstruction[END_REF]. The coding of depth videos has also been considered, e.g., in [START_REF] Daribo | Arithmetic edge coding for arbitrarily shaped sub-block motion prediction in depth video compression[END_REF] the idea that pixels with similar depth have similar motion is exploited. In addition, the approach in [START_REF] Milani | A cognitive approach for effective coding and transmission of 3D video[END_REF] performs an object classification on the scene and adapts the depth coding strategy according to the motion characteristics.

CONTOUR CODING

Srivastava et al. [START_REF] Srivastava | Shape analysis of elastic curves in euclidean spaces[END_REF] introduced a framework to model a continuous evolution of elastic deformations between two reference curves. According to the interpretation of the elastic metric, it is relatively easy to compute the geodesic between two curves: it consists in a continuous set of deformations that transform one curve into another with a minimum amount of stretching and bending, and independently from their absolute position, scale, rotation and parametrization. The referred technique thus interpolates between shapes and makes the intermediary curves retain the global shape structure and the important local features such as corners and bends. An example of the geodesic connecting two curves is shown in Fig. 2. We show in black two contours extracted from a MVD sequence, corresponding to views 1 and 8. The curves in red are the contours extracted from the intermediate views, while in dashed blue we show a sampling of the elastic geodesic computed between the two extreme curves. The elastic deformations along the geodesic reproduce very well the deformations related to a change of viewpoint or a temporal evolution of an object in a sequence, given the initial and final shapes. The lossless coding of the contour is performed through an arithmetic coder, and the input symbol probability distribution is modified on the fly according to the context provided by the elastic prediction [START_REF] Calemme | Lossless contour coding using elastic curves in multiview video plus depth[END_REF].

Supposing that the encoder and the decoder share a representation of the initial and final shape, they can reproduce exactly the same geodesic path between them. Then, the decoder will use a suitable point of the geodesic, i.e. one of the dashed curves in Fig. 2, as context [START_REF] Cagnazzo | Mutual information-based context quantization[END_REF] to encode an intermediary contour (one of the solid curves in the same figure). The encoder will only have to send a value in [0, 1] to let the decoder identify this curve. Moreover, accurate probability values are obtained "following" the elastic prediction, so the decoder needs also a correspondence between the points of the curve b and its elastic prediction b to relate each point of b to a portion of b. An example of the result of the association of the two curves is given in Fig. 3 The correspondences between the two curves are indicated with thin dotted black lines. The vector p represents the most probable future direction.

As we compute the most probable future direction p, a distribution is defined for each symbol using the von Mises statistical model, and an efficient representation of the curve is obtained using these distributions in conjuction with an arithmetic coder.

DEPTH CODING

After the segmentation and the coding of the segment contours, the next step is the subsampling of depth data. The depth map is simply subsampled according to a regular grid (see Fig. 4c) and the average depth values in the segments are subtracted from the subsampled data. The sampling factor ∆ is selected according to the resolution and the amount of detail of the depth data. In the experimental set-up presented in Section 5, sampling grids of 16 × 16 or 32 × 32 were used. The subsampled data is basically a low resolution depth map of size (W/∆) by (H/∆), where W and H are the dimensions of the original depth map. This information is compressed in lossless way in order to prevent blurring on the edges and averaging between neighboring samples of different segments, which would make useless the benefits of the segmentation step. In order to obtain a high coding gain, the samples of the first depth image (Intra depth frame) are first scanned on a raster order and converted into a sequence of couples (r, l) according to a run-length coding strategy. Then, both runs and lengths are coded using a Lempel-Ziv 78 algorithm, which permits obtaining good coding gains with a limited complexity. For the following frames, a DPCM strategy is tailored in order to exploit temporal redundancy. Depth samples are first predicted from the previous frames (zero-motion prediction), and the prediction residual is coded like depth samples of the Intra depth frame.

Although technical literature provides many other examples of more effective coding strategies, the very low resolution of the input images permits obtaining very small file sizes. we used an approach derived from [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] that here we briefly describe. The key idea is that the depth map is made of a set of smooth surfaces represented by the segmented regions. For each region a set of samples is available, i.e., the subsampled depth map pixels belonging to that region. Each pixel p i of the high resolution depth map is thus surrounded by 4 samples of the grid (see Fig. 5) and the idea is to predict it by interpolating only the grid samples belonging to the same region. If all the 4 samples belong to the same region of p i , the estimation of p i is simply given by the bilinear interpolation of the 4 samples. When instead the sample is close to the contour of one of the segmented regions, some of the 4 samples could belong to different regions and the value of p i is estimated by interpolating only the values of the grid samples that belong to the same region. Up to the symmetry and excluding the trivial case of samples that correspond to grid points, there are 5 possible cases (see Fig. 5). a) If all the 4 samples are inside the same region p i is simply given by the bilinear interpolation of the 4 samples. b) If p i is surrounded by 3 samples belonging to the same region it is estimated by assuming that it lies on the plane defined by the 3 points. c) If p i is surrounded by 2 neighbors in the region and 2 outside (e.g., when it is close to an edge) a prediction of the two points outside of the region is performed by assuming that each of them lies on the line passing through the closest available point and the symmetrical point with respect to the available one (the orange points in Fig. 5c). These points are used to compute p i by bilinear interpolation. This approach allows to obtain a very accurate prediction of the input depth map, with only some small artifacts typically on the edges not captured by the segmentation.

Finally, the residual difference between the estimated and actual depth map is computed and lossy compressed using the HEVC coding engine. In this case, the main RExt profile is used disabling lossless mode and enabling rate-distortion optimization.

EXPERIMENTAL RESULTS

The performance of the proposed approach was evaluated on three different sequences with different resolutions and characteristics. The first is the undo dancer sequence, an high resolution (1920 × 1088) synthetic scene, the second is the lovebird sequence, a real world sequence with a resolution of 1024 × 768 and the third is the mobile sequence, another synthetic scene with resolution 720 × 540. We compared the proposed approach with the HEVC standard video coder and with the segmentation-based depth coding approach of [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF]. Fig. 6a shows the performance of the proposed and competing approaches on the undo dancer sequence. There is a large performance gain with respect to [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] at all bitrates. The two approaches share the idea of exploiting segmentation and low resolution approximation, but the contour coding strategy of this work is more efficient than the arithmetic coder of [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] and the proposed low resolution samples and residual compression strategies largely outperform the JPEG2000 based coding used in that work. The comparison with HEVC is more though: at low and medium bitrates (up to 0.01 bpp) the proposed approach is able to outperform HEVC thanks to the efficient representation of the contours and to the very low information content of the residuals. The performance gain reaches around 2 dB at around 0.006 bpp. Notice how the contours remain sharp and not blurred on the whole bitrate range while HEVC achieves this result only at high bitrates. Fig. 6b shows the results for the lovebird sequence: even if the resolution is different and the scene is a real world one (not synthetic), the results are very similar to the previous one, with the proposed approach able to outperform [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] at all bitrates and HEVC for bitrates up to 0.02 bpp corresponding to around 53 dB. The mobile sequence (Fig. 6c) is more challenging for our approach since the edges are already blurred in the input depth maps. This reduces the effectiveness of coding strategies based on the assumption of sharp edges between the various regions like the proposed one and [START_REF] Zanuttigh | Compression of depth information for 3d rendering[END_REF] (that on this sequence has very poor performances). However our approach is still able to outperform HEVC at low bitrates.

The edge preserving capabilities of the proposed approach are particularly evident when the depth is used for view warp-ing and interpolation. Fig. 7 shows a detail from a synthesized view from two frames of the lovebird sequence. Depth data compressed at around 0.008 bpp with both our approach and HEVC have been used. From the figure it is clear how depth compressed with the proposed approach leads to a better interpolation, in particular notice how the regions close to the people edges have much smaller artifacts. Other examples of interpolation are contained in the additional material.

CONCLUSION

In this paper we proposed a novel depth coding scheme. The proposed scheme is able to exploit the temporal redundancy through an efficient contour coding scheme based on elastic curves. The depth field inside each region is efficiently compressed by exploiting a surface interpolation scheme and efficient coding schemes for both the low resolution depth and the prediction residual. Experimental results demonstrated that the proposed approach is able to properly preserve the edges even at low bitrates, a very useful property for warping applications and that it is very effective at medium-low bitrates, where it outperforms the HEVC coder. Further research will be devoted to the replacement of the segmentation masks with edge-based schemes able to work also with partial and interrupted edges and to a better exploitation of the temporal redundancy.

Fig. 1 .

 1 Fig. 1. Overview of the proposed approach.

Fig. 2 .

 2 Fig.2. Geodesic path of elastic deformations bs from the curve i 0 to i 1 (in dashed blue lines). b 3 is one of the contours b t extracted from the intermediate frames between the two reference ones, a good matching elastic curve b0.2 along the path is highlighted.

Fig. 3 .

 3 Fig. 3. Extracts from the curves b (red) and b (dashed blue).The correspondences between the two curves are indicated with thin dotted black lines. The vector p represents the most probable future direction.

Fig. 4 .Fig. 5 .

 45 Fig. 4. a) Sample depth map from the Dancers sequence; b) segmented depth map; c) detail of the segmented depth map with the subsampling grid (red dots represent the position of the grid samples); d) depth map predicted from the contour and the low resolution samples.

  d) If p i has just one neighbor in the same region the value of this sample is taken as estimate. e) If p i has no neighbors the average depth value of the region is used.

Fig. 6 .

 6 Fig. 6. Comparison of the performances of the proposed approach with the HEVC coder and with the method of Zanuttigh et al. [4]: a) Undo Dancer sequence, b) Lovebird sequence, c) Mobile sequence.

Fig. 7 .

 7 Fig. 7. Detail of an interpolated view from the lovebird sequence: a) using depth compressed with our approach; b) using depth compressed with HEVC.