Heterogeneous Spatial Quality for Omnidirectional Video

Hristina Hristova * , Xavier Corbillon * , Gwendal Simon * , Viswanathan Swaminathan † , Alisa Devlic ‡ * IMT Atlantique † Adobe ‡ Huawei Abstract-A video with heterogeneous spatial quality is a video where some regions of the frame have a different quality than other regions (for instance, a better quality could mean more pixels and less encoding distortion). Such a quality-variable encoding is a key enabler of Virtual Reality application, with 360-degree videos. So far, the main technique that has been proposed to prepare spatially heterogeneous quality is based on the concept of tiling. More recently, Facebook has implemented another approach: the offset projection where more emphasis is put on a specific direction of the frame. In this paper, we study quality-variable 360-degree videos with two main contributions. First, we provide the theoretical analysis of the offset projection and show the impact of the parameter settings on the video quality. Second, we propose another approach which consists in preparing the 360-degree video from a Gaussian pyramid of downscaled and blurred versions of the video. We perform an evaluation of tiling, offset and Gaussian-based approaches in representative scenarios of heterogeneous spatial quality in 360-degree videos and highlight the main trade-off to consider when implementing these approaches.

I. INTRODUCTION

To deliver 360°videos, the content providers implement viewport-adaptive streaming solutions [START_REF] Corbillon | Viewport-adaptive navigable 360-degree video delivery[END_REF][START_REF] Petrangeli | An HTTP/2-Based Adaptive Streaming Framework for 360 Virtual Reality Videos[END_REF][START_REF] Sreedhar | Viewport-adaptive encoding and streaming of 360-degree video for virtual reality applications[END_REF], where the delivered video is characterized by heterogeneous spatial quality: some regions of the frame have a better quality than others [START_REF] Corbillon | Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming[END_REF]. The motivation is twofold: (i) display a high-quality video in the viewport of the client, and (ii) reduce the delivered bit-rate by encoding less information in the regions that are unlikely to be watched.

In a video with a spatially heterogeneous quality, each pixel is associated with a target quality, which ranges in a given ordered set. The rationale behind the mapping between quality and pixels, whether it comes from a statistical analysis of previous sessions [START_REF] Corbillon | 360-degree video head movement dataset[END_REF][START_REF] Lo | 360°video viewing dataset in head-mounted virtual reality[END_REF][START_REF] Wu | A dataset for exploring user behaviors in VR spherical video streaming[END_REF] or from a content analysis [START_REF] Lee | Efficient video coding based on audio-visual focus of attention[END_REF], is out of the scope of this paper. We assume that the content provider has defined a small set of qualities and a set of regions consisting of contiguous pixels with the same quality. An ideal implementation of spatially heterogeneous quality in a 360°video is characterized by: Precision The visual quality of a region in the frame reflects the specifications, both in terms of region boundaries and relative quality with respect to the quality in other regions. Smoothness The pixels at the boundary of two contiguous regions enable a smooth transition between both regions. Encoding Efficiency The bit-rate budget that is necessary to implement the spatially heterogeneous quality is low with respect to the obtained visual quality of the regions.

Universality

The process of preparing the video can be implemented and widely deployed. Requirement The resources (in particular, computing power and memory) that have to be provisioned to prepare the video are available in standard media servers.

To prepare heterogeneous spatial quality in videos, the concept of tiling has received the most attention. Tiling is offered by the motion-constrained tile sets (MCTS) feature in the High Efficiency Video Coding (HEVC) encoder [START_REF] Hosseini | Adaptive 360 VR video streaming based on MPEG-DASH SRD[END_REF][START_REF] Le Feuvre | Tiled-based Adaptive Streaming using MPEG-DASH[END_REF][START_REF] Sánchez | Compressed domain video processing for tile based panoramic streaming using HEVC[END_REF][START_REF] Zare | HEVCcompliant tile-based streaming of panoramic video for virtual reality applications[END_REF]. The key idea of tiling is to spatially cut the video frame into non-overlapping blocks of pixels, called tiles, and to encode them independently [START_REF] Misra | An overview of tiles in HEVC[END_REF]. Before streaming the video, a quality for each tile is selected depending on its location and the current quality-region input specifications [START_REF] Wang | Mixing Tile Resolutions in Tiled Video: A Perceptual Quality Assessment[END_REF]. The drawbacks of tiling become evident when analyzed with regards to the aforementioned fundamental characteristics. First, the precision of tiling depends on the number of tiles. A large number of tiles comes at the price of a lower encoding efficiency [START_REF] Concolato | Adaptive streaming of hevc tiled videos using mpeg-dash[END_REF] and an increased resource requirement (high signalling overhead and heavier content delivery network (CDN) management). Moreover, the visual quality changes abruptly at the boundary of two contiguous tiles. Finally, tiling has a low universality. So far, the only fast open-source HEVC encoder to implement tile encoding is Kvazaar [START_REF] Koivula | Performance evaluation of Kvazaar HEVC intra encoder on Xeon Phi many-core processor[END_REF]. Hence, despite being the most widely used method, tiling is not an ultimate solution, and the preparation of heterogeneous spatial quality in 360°videos is still an open research question.

To overcome the limitations of tiling, in this paper we propose two new approaches that implement heterogeneous spatial quality in 360°videos:

Gaussian Pyramid Composition An input video is processed pixel-wise using multiple decreasing qualities, arranged in a Gaussian pyramid. For each pixel in the output video, the content provider picks a pixel from a Gaussian level with respect to the expected quality at this pixel. Such a pixel-wise approach has never been explored for preparing heterogeneous spatial quality in 360°videos. Formal Study of Offset Projection The idea consists in modifying the sphere-to-plane projection before video encoding [START_REF] Zhou | A measurement study of occulus 360 degree video streaming[END_REF]. The content provider applies a patch to the projection to map more spherical pixels to a given area on the planar image. Despite the interest in this approach, the impact of the offset parameters on the visual quality after video encoding has never been formally studied.

We study these two approaches by introducing their mathematical principles, by analyzing their impact on the video visual quality, and by comparing them to the traditional tiling approach. In essence, this paper shows that three approaches exist to implement heterogeneous spatial quality in 360°videos: by leveraging encoder distortion (the tiling approach), by blurring images (the Gaussian approach), and by unequal pixel sampling (the offset approach). We reveal their respective advantages and weaknesses as well as open perspectives for this research challenge.

II. DEFINITIONS

The input of the content provider is a 360°spherical video, which is captured by an omnidirectional camera (either two fish-eye cameras, or a set of multiple stitched cameras). The content provider aims to generate a video, which is projected into a two-dimensional rectangular area (in this paper, we focus on the equirectangular projection but our results can be extended to other projections) and is then encoded with regards to some specific characteristics related to heterogeneous spatial quality.

The concept of heterogeneous spatial quality is defined by two input parameters. First, we define Q as an ordered set of qualities. This set is not formally associated with any precise measurable scale. Following the definition from MPEG [6], the quality is a rather vague indicator. We thus consider that there exist Q qualities in Q, denoted q i for i ∈ {0, 1, . . . , Q -1}, where q 0 is the lowest quality, and q Q-1 is the best quality.

Second, we define a set of regions, which are sets of contiguous pixels in a frame. The idea behind the concept of quality-region is the following. The content provider considers that the pixels in a given region have approximately the same probability to be displayed in the viewport, so they must be encoded at the same quality, i.e. the higher the probability to be displayed in the viewport, the higher the quality. Corbillon et al. [START_REF] Corbillon | Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming[END_REF] have considered a single region per video and have represented this region as a compact area on the sphere (i.e. spherical rectangle). Here, we do not restrict the regions to spherical rectangles. On the contrary, we assume that the regions may have any shape. We denote the set of regions by R. The regions do not overlap and cover the whole frame. The quality of a region R ∈ R is a spatial function denoted q(R) ∈ Q. The qualities of any two regions in R can be either different or the same.

III. GAUSSIAN PIXEL-WISE ENCODING

A. Background on Gaussian Pyramids

Image pyramids are multi-scale representations of an image. In particular, the Gaussian pyramid has been widely used to carry out various image processing tasks. The Gaussian pyramid is constructed using a function, denoted downSample(•). The function downSample (I, f (•)) performs two operations as follows: (i) blurs an input image I using the spatial weighing function f (•) and (ii) reduces both dimensions of image I by a scale of 2, i.e. (W 1 , H 1 ) = (W/2, H/2), where W × H is the original resolution of I . The blurring is carried out by convolving the input image I with the function f (•), also referred to as the blurring kernel. To generate the Gaussian pyramid of image I, I is convolved with a Gaussian kernel, i.e. each pixel of the blurred image G (1) is computed as a weighted average of the pixels of I in a 5×5 neighbourhood, as follows:

G (1) (i, j) = 2 x=-2 2 y=-2 f (x, y)I(2i + x, 2j + y) (1) f (x, y) = ω(x)ω(y) (2) ω(•) = (0.25 -p/2, 0.25, p, 0.25, 0.25 -p/2), (3)
where the parameter p ∈ [0.3, 0.6] controls the shape of the Gaussian kernel f (•). Once the blurred image G (1) is computed using Section III-A, the function downSample (•, f (•)) downsamples image G (1) by rejecting even rows and even columns, which reduces the resolution of the resulting image, g (1) , in half. Both the Gaussian blur and function downSample(•) can then be applied to the resulting image g (1) in order to compute a second scale of the input image I. The sequence of images G = {g (0) , g (1) , . . . , g (N ) }, where g (0) = I and

g (i) = downSample(g (i-1) , f (•)) for i ∈ (1, N ],
denotes the Gaussian pyramid of image I. The resolution of each image in the sequence G is twice as small as its predecessor.

We denote the inverse function of downSample(•) as upSample(•). The function upSample(g

(i+1) , f (•)) first upscales the image g (i+1) to dimensions (W i , H i ), such that (W i , H i ) = (2W i+1 , 2H i+1 ) (W i × H i
being the resolution of the Gaussian level i) by injecting zero even rows and columns. Then, the resulting upscaled image G i+1 is convoluted with a Gaussian kernel f (x, y), identical to the one used in function downScale(•), as follows:

G (i+1) (i, j) = 4 2 x=-2 2 y=-2 f (x, y)G i+1 i -x 2 , j -y 2 , (4)
where [a] denotes the nearest integer value to a.

B. Pixel-Wise Video Composition

We propose a multi-scale spatial approach for implementing heterogeneous spatial quality in 360°videos based on the Gaussian multi-scale representation. It consists in two processes: an offline process to construct the Gaussian pyramid, and an online process to compose the video version with heterogeneous spatial quality. Gaussian Pyramid Construction. For a given input video, we compute the sequence {g (i) } N i=0 , where each element g(i) contains the i th layer of the Gaussian pyramid, constructed for all frames of the video. We refer to {g (i) } N i=0 as the Gaussian pyramid of the original 360°video. The levels of the Gaussian pyramid represent the degradation of the video quality in N + 1 multi-scale levels. Then, we upscale each level g (i) to the resolution of the original video. Each level G (i) of the upscaled Gaussian pyramid is the

original raw G (0) encoded G (0)
g (1) raw G (1) encoded G (1) g (2) raw G (2) encoded G (2) encoded G (2) encoded G (1) encoded

G (0) raw G (0) raw G (1) raw G (2) raw O G encoded O G Pyramid Construction Video Composition
G (0) G (1) G (2) G (3) 0 5 10 15
Gaussian pyramid levels Bit-rate (Mbps) result of upscaling and convolving with the Gaussian kernel i times. This process, illustrated in Figure 1 (left part), leads to a certain amount of redundant information, which is exploited by the encoder to better compress the upsampled videos. To analyze the bit-rate gain, we chose three representative 1 min-long 360°equirectangular videos [START_REF] Corbillon | 360-degree video head movement dataset[END_REF] and we computed their upscaled Gaussian pyramids. Then, we encoded each level of the upscaled Gaussian pyramids using a constant Quantization Parameter (QP) value. In Figure 2, we present the distributions of the video bit-rates for each Gaussian level as well as the respective quality distortions. We show the efficiency of the Gaussian approach for generating videos at variable qualities and bit-rates by applying basic pixel operations. Create Quality-Variable Video. The upscaled Gaussian pyramid {G (i) } N i=0 is then exploited to create quality-variable videos as illustrated in Figure 1 (right part). We use a linear optimization to map the qualities, provided by the function q(R), to a given upscaled level G (i) . It takes into account the overall video bit-rate budget and chooses the most appropriate Gaussian levels with respect to this budget. Let φ(•) be a function that, for every pixel, maps q(R) to optimal Gaussian levels given a bit-rate budget B.

(a) Bit-rate G (0) G (1) G (2) G (3)
The video composition, prepared at the server side, is computed from the upscaled Gaussian pyramids for each pixel (u, v) as follows:

O G (u, v) = G (φ(q(x,y),B)) (u, v), (5)
where {O G } is the output video with spatially-varying quality.

C. Analysis

The Gaussian-based pixel-wise video composition approach requires the server to first compute the N + 1 levels of the Gaussian pyramid and encode them (for optimal storage). This process can be done only once, as the pyramid can then be re-used to create any number of different quality-variable videos. The preparation of a quality-variable video requires that the server decodes min (N + 1, |R|) Gaussian pyramid levels, performs the spatial operation to compose the spatially heterogeneous video, and encodes it. To sum up, the number of operations per video composition is: (i) decoding -min (N + 1, |R|) times; (ii) one spatial video composing; (iii) one video encoding.

IV. OFFSET PROJECTION The offset transformation [START_REF] Kuzyakov | End-to-end optimizations for dynamic streaming[END_REF] is a bijective sphere-to-sphere transformation, which increases the quality of a 360°video near an emphasized direction #» b. When two pixels on the sphere are close to (resp. far from) the emphasized direction #» b, the offset transformation decreases (resp. increases) the angular distance between the two pixels. The offset projection is a composition of a sphere-to-plane projection with an offset transformation. First, the spherical image is distorted using the offset transformation, then the distorted spherical image is projected to a plane using a sphere-to-plane projection. The offset projection maps on the plane more pixels close to #» b and fewer pixels far from #» b. The spatial sampling (thus, the quality) of the video decreases when the angular distance to #» b increases.

A. Theory

The offset projection has been experimentally studied by Zhou et al. [START_REF] Zhou | A measurement study of occulus 360 degree video streaming[END_REF] in the case of the cube-map projection. It is characterized by the following parameters: a projection function f : R 3 → R 2 , an emphasized direction #» b, and an offset amplitude α. The projection function f can be any sphere-to-plane projection. The emphasized direction #» b is a unit vector, pointing to the direction of space emphasized by the offset projection, and α is a real value in [0, 1).

A sphere-to-plane projection maps a spherical pixel, characterized by a unit vector #» a pointing from the origin of the sphere to the pixel, to a point (u, v) on the plane.

The offset transformation F : R 3 → R 3 distorts the sphere by mapping the vector #» a to the vector

#» a + α #» b / #» a + α #» b .
The offset projection g : R 3 → R 2 is then given as follows:

g( #» a ) = f • F ( #» a ) = f #» a + α #» b / #» a + α #» b (6)
In Equation (6), both projection functions f and g transform a given viewing direction #» a into planar coordinates (u, v). The inverse offset projection g -1 , transforming coordinates (u, v) into a viewing direction #» a , is defined as follows:

g -1 (u, v) = F -1 • f -1 (u, v) (7)
where f -1 is the plane-to-sphere projection, and F -1 is:

F -1 ( #» a ) = #» a • α #» b + ( #» a • α #» b) 2 -α + 1 #» a -α #» b (8)

Downsampling area

Oversampling area

0 • 30 • 60 • 90 • 120 • 150 • 180 • 0 0.5 1 1.5 45•
Angular distance from the emphasized direction Sampling density ratio α = 0.00, r = 100 % α = 0.16, r = 81 % α = 0.37, r = 64 % α = 0.98, r = 36 % with • being the inner vector product in R 3 .

To study how the offset projection continuously degrades the quality of the spherical image, we measure the sampling density of the projection on the sphere. The sampling density is a number that represents the number of pixels per surface unit on the sphere. It can be computed for any point on the sphere. If we consider only a finite number of pixels W × H on the plane, then the planar image can be split into W × H similar squares. A point p on the sphere is projected inside a unique square s p on the plane. The sampling density at point p on the sphere can be approximated by the inverse of the surface of the square s p , once s p is projected back on the sphere. In what follows, σ α,r denotes the sampling density function for the equirectangular offset projection with an amplitude α, an emphasized direction #» b = (1, 0, 0) and a resolution r(W ×H). Hereafter, r ∈ [0, 1] refers to the resolution ratio.

Increasing the amplitude α for a constant r increases the sampling near the emphasized direction and decreases it near the direction opposite #» b. Put differently, when α increases, more pixels from the plane are assigned to spherical angles near the emphasized direction and less in the opposite direction. Figure 3 depicts the offset sampling density compared to the sampling density at the same position in the original video, i.e. it depicts σ α,r /σ 0,1 . When α = 0, the sampling density ratio is constantly equal to r. As illustrated by the curve for r = 0.36 in Figure 3, when α tends towards 1, the sampling density ratio tends toward 0 for points farther than 90°from

#» b, and tends towards 4r in the emphasized direction #» b (i.e. tends towards 1.44 for r = 0.36). The latter means that the offset projection with an amplitude α > 0 cannot increase the sampling σ 0,r in the emphasized direction more than four times the sampling for α = 0. 1 Furthermore, when α > 0, the sampling decreases with the distance from the #» b and the transition between the emphasized and non-emphasized regions is smooth. Hereafter, we define the Quality Emphasized Region (QER) for the offset projection as being the zone of the video for which the sampling density ratio is at least equal to the original sampling ratio (in Figure 3, 1 Due to page limitations, the demonstration is not in the paper.

the original sampling ratio is constant and equal to 1). For a given resolution ratio r, there exists at most one value of α such that the angular size of the QER is equal to a given value. For instance, Figure 3 shows that for r = 0.64, a region of angular size 90°(i.e. 45°from #» b) is a QER when α = 0.37 (resp. α = 0.98 for r = 0.36 and α = 0.16 for r = 0.81).

B. Experiments and analysis

We now evaluate the impact of r and α on the visual quality of the videos. So far, only Zhou et al. [START_REF] Zhou | A measurement study of occulus 360 degree video streaming[END_REF] have studied the quality distortion caused by the offset approach. The authors compare the video quality degradation of the offset cube projection to the quality of the original equirectangular video. Their experiments show that, for a given α, the offset cube projection produces videos with similar quality as a video at a higher resolution within a certain angular distance from the offset center. However, Zhou et al. do not elaborate on the choice of α and, more importantly, how this choice influences the quality of the offset videos. Hereafter, we present a more thorough analysis of the quality distortion, caused by the offset (equirectangular) projection given various video resolutions and various values of the offset amplitude.

We applied the offset equirectangular projection on three 360°videos using three resolution ratios, i.e. r ∈ {0.81, 0.64, 0.36}, and five amplitude values, i.e. α ∈ {0, 0.25, 0.5, 0.75, 0.9}, for each resolution ratio r. To reduce the content-related bias in the quality measurement, we computed the offset projection eight times per video by applying different sphere rotations. To measure the distortion, introduced by the offset projection, we computed the Peak Signal to Noise Ratio (PSNR) between viewports from both the original equirectangular video and the offset videos. We extracted viewports at varying angular distances from the emphasized direction #» b to illustrate the relation between quality degradation and distance to the offset center.

Figure 4 shows the PSNR quality curves for each considered resolution ratio r and each amplitude α. The curves for α = 0, i.e. when no offset is applied, are referred to as baselines. Each baseline measures the distortion caused by the resolution decrease (by r). The PSNR remains constant regardless of the distance to the spherical center. In contrast, for α > 0, the video quality varies depending on the angular distance to the emphasized direction #» b. The PSNR curves show that, for α > 0, the quality of the viewports close to #» b is higher than the baseline quality. The farther we get from the center of emphasis, the higher the distortion.

Furthermore, for all resolution ratios r, the more we increase the amplitude α, the more we improve the video quality near the emphasized direction #» b and the more we degrade the quality near the opposite direction to #» b. The plots in Figure 4 show that the quality curves intersect the baseline at smaller angles when α increases. For instance, the curve for α = 0.5 intersects the baseline at 60°, whereas the curve for α = 0.9 intersects the baseline at 40°. The latter means that the higher the amplitude α, the smaller the emphasized region. As shown in Figure 3, for high amplitude values, e.g. α = 0.98, most of the samples are concentrated in a small region, centered at the emphasized direction #» b, and there are less samples for regions away from the center of emphasis. This explains the rapid decrease in the quality when the distance from #» b increases.

V. EVALUATION

In Table I, we summarize the main differences between the three approaches with regards to four out of the five main characteristics, introduced in Section I. Due to page limitations, the analysis of the computing and storage requirements for composing 360°videos as well as the user reaction to abrupt quality changes are left for future work. Here, we focus on the encoding efficiency by measuring the visual quality of the videos created using the three approaches, for a similar overall bit-rate. [START_REF] Yu | A Framework to Evaluate Omnidirectional Video Coding Schemes[END_REF]. The S-PSNR first maps each point on the sphere to a location in both the original and the encoded videos. Then, the pixel values, corresponding to these locations are used to determine the distortion between the two pixels. Finally, the errors for all spherical points are averaged to compute the S-PSNR. However, the concept of heterogeneous spatial quality in 360°videos takes its root from the fact that not all spherical pixels are equal (as some pixels have higher probability to be in the clients' viewports). To capture this heterogeneity, we propose to weigh the error of each pixel with respect to its expected quality. We thus use our quality function q(R) to weigh the S-PSNR and obtain a new weighted metric, denoted WS-PSNR. The highest weight is assigned to spherical points which, after projection, lie in the region with the highest quality. The weights of the spherical points lying within the other quality regions decrease exponentially.

To sum up, the S-PSNR measures the average quality of the entire encoded video, whereas the WS-PSNR measures the quality with respect to the given quality regions. We combine both metrics to obtain an interpolated quality estimation, denoted IS-PSNR.

Quality Regions. We consider two configurations of quality regions (Figure 5), which represent common viewport movements [START_REF] Corbillon | Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming[END_REF]. The -shape is common for a video where viewers stably focus on a single location, whereas the V-shape may appear when the object of interest is moving.

Video Preparation. We use three equirectangular 360°v ideos from a public dataset [START_REF] Corbillon | 360-degree video head movement dataset[END_REF] (rollercoaster, venice, and timelapse). Their resolution is 3840×2160 pixels (4K). We use the Kvazaar encoder [START_REF] Koivula | Performance evaluation of Kvazaar HEVC intra encoder on Xeon Phi many-core processor[END_REF] with three bit-rate targets: 6 Mbps, 9 Mbps and 14 Mbps. Specific settings include: Tiles We set 8×8 tiles. We encode four tiled videos with four bit-rate targets (3 Mbps, 7 Mbps, 13 Mbps and 21 Mbps). We extract each tile from each tiled video. To obtain a qualityvariable video matching the quality regions and the bit-rate target, we select each tile from one of the four qualities so that the overall bit-rate is close to the target. Offset We chose two resolutions: 3686×1944 (r = 81 % of the original video) and 2457×1296 (r = 36 %). The offset intensity α is set so that a 90°of the QER (resp. 50°) is oversampled for -shape (resp. V-shape). Gaussian The Gaussian pyramid is built by halving the video dimensions and applying a Gaussian kernel. The final video is composed from pre-selected upscaled Gaussian levels.

Result Analysis. In Figure 5a (resp. in Figure 5b), we show the IS-PSNR for the three bit-rate targets and for the -shape (resp. V-shape) quality regions. The lowest and the highest values of the bars correspond to the S-PSNR and the WS-PSNR respectively. In Figure 6, we show snapshots of the back viewports and thus highlight the different approaches (blurring, distorting, and reducing bit-rate by encoding). First, we observe that all approaches manage to prepare 360° videos with heterogeneous qualities. In each configuration, the visual quality in the emphasized regions, measured by the WS-PSNR), is higher than the visual quality, measured by the S-PSNR. The difference between both metrics is greater or equal to 10 dB, which is a significant gap. The offset approach is relevant for low encoding bit-rates, but it does not benefit from extra bit-rates. Also, the offset is sensitive to the viewport location, with large quality range between the S-PSNR and the WS-PSNR, especially at low resolutions. To obtain a high quality in a large viewport despite the low resolution, the price we pay is a severely degraded quality in the direction opposite #» b, as epitomized in Figure 6. The Gaussian approach is also characterized by large quality ranges. The latter is a result of the significant quality gap between G (0) and G (1) (Figure 2). The gain, obtained by exploiting the blurred Gaussian levels, enables the encoder to maintain very high quality in the viewport without decreasing the resolution. The Gaussian approach never reaches the low quality levels, observed for the offset. The Gaussian approach is sensitive to the video bit-rate: the quality increases steadily with the increase of the bit-rate budget. Finally, the tiling approach appears more stable. It offers a consistent good quality in the best cases and the quality never reaches very low levels (unlike the offset). The back viewport in Figure 6 has also a good visual quality. However, tiling requires specific encoders, and the abrupt changes between tiles can degrade user's experience.

VI. CONCLUSION

In this paper, we study the preparation of heterogeneous spatial quality in 360°videos. We propose a novel method, based on the Gaussian pyramid, and we analyze the theoretical principles of the offset projection. We identify three families of approaches: a quality degradation by the encoder (tiling), a Gaussian kernel (Gaussian), and an unequal projection (offset projection). We compare the three approaches in terms of encoding efficiency.

This paper is the first one to formally study heterogeneous spatial quality in 360°videos. Each of the proposed approaches deserves a deeper analysis to better understand the impact of the settings on the overall performance. Then, the integration of the propositions in the global delivery chain of viewport-adaptive streaming solutions also deserves a deeper analysis with respect to the constraints of the services, including definition of quality regions, live requirements, and implementation in CDN.

Figure 1 :

 1 Figure 1: Construction of the Gaussian pyramid (left part) and video composition (right part).

  ) (b) Distortion in PSNR

Figure 2 :

 2 Figure 2: The bit-rate and distortion distributions for the first four levels of the upscaled Gaussian pyramid.

Figure 3 :

 3 Figure 3: Sampling ratio σ α,r /σ 0,1 for the equirectangular offset projection for #» b = (1, 0, 0) at constant latitude π 2 .

0 40 50

 40 • 30 • 60 • 90 • 120 • 150 • 180 • Resolution: 81 % 0 • 30 • 60 • 90 • 120 • 150 • 180 • 20 30 Angular distance from the emphasized direction #» b (b) Resolution: 64 % 0 • 30 • 60 • 90 • 120 • 150 • 180 •

Figure 4 :

 4 Figure 4: PSNR between viewports, extracted from the original equirectangular video, and viewports, extracted from the equirectangular offset projection for various values of the amplitude α and three resolution ratios r.

Figure 5 :

 5 Figure 5: IS-PSNR bars for two quality regions

Figure 6 :

 6 Figure 6: Viewport in the non-QER for the 6 Mbps videos

Table I :

 I Summary of main characteristicsVisual Quality Metric. We lack a metric to capture the heterogeneity of quality in a 360°video. To fill this gap, we created a new metric based on the Spherical Peak Signal to Noise Ratio (S-PSNR) Yu et al.

	Tile	Offset Gaussian
	Precision depends on nb. tiles	low	high
	Smoothness no	yes	no
	Universality encoding issue	yes	yes
	Requirements depends on nb. tiles	low	depends on nb. videos


Mohammad-Mahdi Bazm

Marc Lacoste

Mario Südholt

Jean-Marc Menaud

Secure Distributed Computing on Untrusted Fog Infrastructures Using Trusted Linux Containers

published or not. The documents may come

in which each element extends its trust to the next element in the chain until launching applications. This mechanism allows identifying any modification in the host before launching a container [START_REF]Trusted docker containers and trusted vms in openstack[END_REF]. However, such mechanism is not necessarily implemented by remote hosts especially in heterogeneous infrastructures such as Fog, thus requiring new solutions. Intel SGX allows executing securely sensitive part/data of an application inside isolated zones (called enclaves) through the processor, by encryption mechanisms to protect data from operating system and hardware memory attacks. Consequently, it allows secure execution of the application on untrusted remote hosts. This technology can be leveraged in distributed computing where a distributed application should be run on remote hosts of the cloud infrastructure.

It is also integrated into Linux containers to protect packaged applications [START_REF] Arnautov | Scone: Secure linux containers with intel sgx[END_REF].

In this paper, we discuss the challenge of trusted computing on remote hosts. We review different scenarios to Running an application inside VMs/Linux containers may appear more secure than running it on hosts for isolation and confidentiality. For instance, all containers/applications in a host share the same kernel instance. Thus, any kernel compromise results in security vulnerabilities.

The same scenario applies for VMs although isolation is stronger between VMs than containers due to enforced isolation by the virtualization layer. An adversary may employ Virtual Machine Introspection (VMI) [START_REF] Garfinkel | A virtual machine introspection based architecture for intrusion detection[END_REF] to retrieve information of the VM. We may also run a container inside a VM, to enforce isolation [START_REF] Intel | Intel clear containers: Building a virtualization continuum[END_REF]. However, the real challenge is the execution of the application (either inside VMs or Linux containers) on remote hosts maintained by untrusted third-parties (the hypervisor/OS of hosts thus remain untrusted).

Hardware-assisted primitives provide an isolated execution environment between running applications/containers in the host OS, or between running VMs. These mechanisms provide integrity and confidentiality for the execution of applications/ containers/ VMs: the memory of applications/containers/VMs is hardware-encrypted, and therefore protected from any unauthorized access from untrusted OS/hypervisor. Memory encryption techniques create strong isolation between applications, application-OS, VMs, VM-hypervisor, containers, and container-OS. However, we cannot always trust the application running inside containers/VMs because the application may be modified or compromised by an adversary, thus resulting in information leakage or underlying system compromise. To meet such challenge, Remote Attestation allows remote parties to detect any change in the application. It enables a trusted device (i.e., a device with RoT) to present reliable evidence to remote parties about the state of application executing on remote hosts. This evidence may be for instance a cryptographic signature of a hash of application. by container c i through te on h j is defined as follows: tion frameworks enable to manage a set of containers e.g., Kubernetes, on hosts.

P rocess : D × C × H × T E → R, (d i , c i , h j , te) → r (

D R A F T

Trusted Cluster Creation: To build a trusted cluster composed of several trusted nodes, when a new container c i c i c i wishes to be integrated to the cluster C t C t C t , it initiates the join protocol (Fig. 3). It consists of a remote attestation mechanism to prove the container as a trusted node, by communicating with the master node. If the container is approved as a trusted node (i.e., the application deployed in the container has not been tampered with), the node is integrated to the cluster and the master starts to send data to the approved container for processing. Otherwise, the node is considered untrusted because the packaged application has been tampered with.

VI. IMPLEMENTATION AND EVALUATION

To implement our prototype, we need a container image (u) that provides Intel SGX mechanisms to the application running within container. We used OpenSGX [START_REF] Jain | Opensgx: An open platform for sgx research[END_REF],

a

  Bazm * , Marc Lacoste * , Mario Südholt † , Jean-Marc Menaud † * Orange Labs, 44 Avenue de la République, 92320 Châtillon, France † IMT Atlantique, 4 Rue Alfred Kastler, 44307 Nantes, France Abstract-Fog and Edge computing provide a large pool of resources at the edge of the network that may be used for distributed computing. Fog infrastructure heterogeneity also results in complex configuration of distributed applications on computing nodes. Linux containers are a mainstream technique allowing to run packaged applications and micro services. However, running applications on remote hosts owned by third parties is challenging because of untrusted operating systems and hardware maintained by third parties. To meet such challenges, we may leverage trusted execution mechanisms. In this work, we propose a model for distributed computing on Fog infrastructures using Linux containers secured by Intel's Software Guard Extensions (SGX) technology. We implement our model on a Docker and OpenSGX platform. The result is a secure and flexible approach for distributed computing on Fog infrastructures. Index Terms-Fog computing security, distributed computing, Intel Software Guard Extensions (SGX), trusted computing, secure computing, Linux containers. I. INTRODUCTION Fog computing extends computing resources and services to the edge of the network to provide QoS by supporting low latency and location awareness to meet enduser requirements. This paradigm is also well positioned for Big data and distributed computing applications due to the large amount of computing resources accessible through virtualization technology. Recently, Linux containers have received more attention in Fog and IoT clouds due to their flexibility and performance benefits compared to VMs. Distributed computing based on VMs and Linux containers offers more security (e.g., isolation) than traditional computing models e.g., userbased. However, integrity and confidentiality remain key concerns because of untrusted operating systems and hosts maintained by third-parties in the cloud infrastructure. The primary challenge is how to execute a distributed application securely inside VMs or in Linux containers running on remote hosts (i.e., guaranteeing isolated execution, data integrity and confidentiality). Hardware-assisted trusted execution mechanisms such as Intel's Software Guard Extensions (SGX) and ARM TrustZone, are highly promising to overcome such challenges. A secure component like the TPM (Trusted Platform Module) in a computing platform plays the role of Root of Trust (RoT) through specific embedded cryptographic mechanisms. To make the platform trusted, the RoT extends its trust to other components of the D R A F T 2 platform (e.g., OS) by building a Chain of Trust (CoT)

  secure execution of Linux containers on remote hosts in the cloud infrastructure through trusted execution mechanisms, and finally we propose a model for secure distributed computing through using Linux containers hardened by Intel SGX, to perform trusted execution on untrusted Fog computing infrastructures. The paper is organized as follows: Section II gives an overview of existing methods of distributed execution of applications in Fog computing infrastructures. Section III discusses security challenges of computing on remote hosts, and Section IV reviews two corresponding trusted execution scenarios. Section V gives an outlook on the model and implementation of our prototype and Section VI evaluates it. Section VII reviews related works. Section VIII concludes. II. DISTRIBUTED COMPUTING ON FOG INFRASTRUCTURES In traditional models, specific OS mechanisms allow multiplexing processor and memory between different processes and user accounts. A distributed application runs as a process on a remote machine through an operating system user. However, in highly heterogeneous infrastructures, legacy compatibility is an issue for the execution of distributed applications. Integrity and confidentiality of data and isolated execution cannot be guaranteed because security mechanisms are implemented in the operating system of remote hosts and managed by untrusted host administrators. In VMbased computing, a distributed application runs on a set of consolidated VMs. All VMs are managed by the administrator of the distributed computing platform, although VMs run on remote physical machines owned by third parties. This computing model offers more security for the execution of the distributed application because of the virtualization abstraction layer. It also provides stronger isolation and enables to implement and enforce security mechanisms inside VMs independently of the host, to guarantee data processing. However, the VMbased model suffers from flexibility limitations (e.g., VM migration) intrinsic to virtualization techniques [3]. In container-based computing, the distributed application is packaged in a set of containers. It then runs on hosts managed by third parties on the Fog infrastructure in the form of a set of containers to perform computations. This model provides more flexibility than the VMbased model regarding cluster building, because of fast provisioning and easy management of Linux containers. III. PROBLEM STATEMENT We address two main challenges related to secure and flexible distributed execution of applications on remote D R A F T 3 hosts. Remote Execution: In remote computing, if an application executes directly on hosts, inside VMs, or in Linux containers, there is always a security concern with such computing models: how to secure (i.e., guaranteeing isolated execution of application) remote computations?

1 )

 1 Performance and Flexibility: For distributed computing, a large set of computing resources is needed to run a distributed application on Fog infrastructures. How to provision such a resource pool in an efficient way with greater security and flexibility than the traditional approachs? Even when using trusted computing mechanisms[6] [7], challenges remain regarding fast provisioning and configuration on remote hosts. Linux containers provide a reliable way of encapsulating an application and its configuration with the flexibility of image-based deployment methods. Therefore, distributed computing using containers which are secured by trusted execution mechanisms has been getting more attention for Fog computing[START_REF] Korkhov | Distributed computing infrastructure based on dynamic container clusters[END_REF].IV. TRUSTED EXECUTION OVER FOGINFRASTRUCTURESWe distinguish two scenarios for trusted execution of Linux containers on remote hosts in the Fog infrastructure (Fig.1). a) Trusted host: Software CoT process: Containers are run on a cluster of trusted hosts in the infrastructure. Before launching a container image, a host proves itself as trusted host by following the CoT mechanism. The CoT starts from boot to launching the container engine that starts with an implicitly trusted component (i.e., RoT) and every other component trusted before being executed (Fig. 1a). If the host is trusted, it is added to the pool of trusted hosts which are maintained by third parties. Consequently, the host can launch container images. The container engine then D R A F T 4 verifies run-time trustworthy of the container image. If the proof is valid, the container is authorized to be executed on the host and the master node starts sending data to the container for processing. Trusted Docker containers [1], and CoreOS [8] provide such security solutions by extending the CoT to the Linux container image. b) Untrusted host: hardware trusted execution: Containers are directly launched on a cluster of untrusted hosts. Trusted execution mechanisms are leveraged in the container image encapsulating the application, through hardware mechanisms. The underlying layers (i.e., container engine, OS) are considered untrusted (Fig. 1b). The application executes in the trusted execution environement provided by hardware. If the application is proved trusted through remote attestation, it means it has not been tampered with. The master node then starts sending data to the application. This scenario is an optimized solution not based on a CoT which is a complex process to implement on hosts especially in heterogeneous infrastructures like Fog ones. This solution is preferred when third parties do not want to implement the CoT in their systems. SCONE [2] provides such capability by securing Docker containers leveraging Intel SGX technology. V. SECURE DISTRIBUTED COMPUTING MODEL We consider H = {h 1 , h 2 , ..., h n } a set of physical machines in the Fog infrastructure with a hardwarebacked trusted execution technology te ∈ T Eset of trusted computing technologies. We also consider C = {c 1 , c 2 , ..., c k } a set of containers to run on H. We introduce a software component sc ∈ SCset of software components, to be integrated in the container image c. c is composed of sc and u ∈ U -set of base images used to provide some libraries to the software. Any modification in sc and u results in new container image c ∈ C -set of tampered images. Thus the Tamper operation on c denoted as follows: T amper : SC × U → C , (sc, u) → c (We also consider dataset D = {d 1 , d 2 , ..., d k } to be processed by containers and R = {r 1 , r 2 , ..., r k } the result of data processing. The Process operation of d i

2 )

 2 Definition. c i is a trusted container if its whole image has not been tampered with and proved as such through a hardware-assisted remote attestation mechanism provided by the trusted execution technology te available on hosts in H. Trust Model: Our model leverages Intel SGX as trusted execution technology, and employs scenario b. Thus, critical parts of the application deployed in the container are protected inside enclaves. Other parts outside enclaves, are unprotected. Although all network links are encrypted between hosts by network security protocols such as IPSec, the network is not trusted.

Figure 1 :

 1 Figure 1: Different scenarios of trusted execution of containers.

Figure 2 :Figure 3 :

 23 Figure 2: End-to-end encryption between two enclaves.

  ment and Docker for application management. It also offers a trusted execution environment leveraging the CoT mechanism. CoreOS extends the CoT to container image by implementing specific mechanisms in rkt [11] container engine to provide the remote attestation. VIII. CONCLUSION We have seen that containers hardened by a trusted execution technology like Intel SGX bring more flexibility and security to distributed computing. We proposed a container-based model for secure distributed computing over Fog infrastructures. To illustrate the model, we implemented a container-based MapReduce prototype. Evaluation results show a reasonable performance overhead of using Intel SGX. As future work, we intend to explore AMD-SEV for Linux containers.

  Zhang et al.[START_REF] Zhang | Enabling trusted data-intensive execution in cloud computing[END_REF] proposed an approach for secure computing on cloud infrastructures based on trusted virtual machines. Their approach implements trusted computing mechanisms using TPM to guarantee data privacy and security. CoreOS[START_REF]Coreos container operating system[END_REF] is a minimal Linux OS to run Linux containers especially those that are Docker-based. It provides a secure and reliable cluster for running distributed applications. It leverages Kubernetes for cluster manage-

				•10 9
				2.8
	Execution Time(ms)	700 800	#CPU cycles	2.6 2 2.2 2.4	#Context-switch	40 30 35
				1.8
		600		1.6		25
		Mapper	Reducer	Mapper	Reducer	Mapper	Reducer
		Without SGX With SGX	Without SGX With SGX	SGX active
		(a) Node execution time	(b) Number of CPU cycles	(c) Number of context switches
		Figure 4: Performance of Mapper and Reducer nodes with and w/o Intel SGX.
	model, we consider that data to be processed by nodes	Context switching: Another phenomenon that has
	are sensitive. Thus, they must be protected inside Intel	impact on the performance of SGX applications is
	SGX enclaves. Lesser critical sections of the application	context switching between the enclave and the rest of
	such as network functions are thus placed outside of the	application. During the execution of the application,
	enclave and not protected by Intel SGX.		enclave data and code processing leads to CPU mode
	To evaluate the performance overhead of executing	changes. Therefore, each enclave enter/exit requires to
	SGX instructions on computing nodes (i.e., Linux con-	store/restore CPU states, operations which are costly in
	tainers) running Mapper and Reducer programs, we mea-	research platform for developing SGX-based applica-terms of performance. Fig. 4c shows the number of
	sure execution time, CPU cycles, and Context switching	tions. It implements and emulates Intel SGX ecosystem context-switches incurred in the execution of Mapper
	when SGX is enabled/not-enabled. We use a Dell i5-	instructions, and hardware components leveraging the and Reducer nodes. Overall, Intel SGX instructions
	3340M @2.70GHz with 8GB of RAM, running Ubuntu	QEMU emulator. OpenSGX provides certain mecha-impose 11.2% performance overhead to Mapper and
	14.04.				nisms available in SGX supported processors. Reducer nodes. This is essentially the overhead of using
	Node execution time: As shown in Fig. 4a, the	To evaluate the model, we implemented a MapRe-OpenSGX in our prototype (i.e., performance emulation
	execution time of a complete workflow of a client node	duce distributed computing framework based on Linux of Intel SGX).
	is higher than normal execution when using Intel SGX.	containers, to count number of words in a text file. Our prototype leverages Intel SGX enclaves, thus pro-
	We verified this difference through Linux perf. This	MapReduce being well-recognized for its scalability and vides the same security guarantees which are provided by
	additional execution time is caused by SGX special	flexibility, decomposing an application into microser-Intel SGX. Intel SGX (like other existing trusted comput-
	instructions, SGX library calls, enclave initialization, and	vices. In the prototype, we have three types of nodes: ing approaches), protects the software against memory
	memory encryption mechanisms.		Mapper, Reducer, and Master. The Master splits jobs/-attacks such as malicious OS, malicious hypervisor, and
	Number of CPU cycles: We measured the number of	data and distributes them to Mapper nodes. Then, Map-malicious firmware. However, it does not protect against
	CPU cycles when SGX was enabled (Fig. 4b). Using	pers perform their computing jobs and return the results cache-based side-channel attacks [10].
	SGX results execution of a higher number of instruc-	back to Reducers for merging. Finally, Reducers send VII. RELATED WORKS
	tions because of instructions used by SGX to pro-	results to the Master node. Our experimental prototype
	cess data and code in enclaves.		is composed of one Master and 10 Slaves nodes. In our

D R A F T