Hoai-Nam Nguyen

Ehsan Miandji

Fellow, IEEE Christine Guillemot

Multi-Mask Camera Model for Compressed Acquisition of Light Fields

Keywords: Light Field imaging, camera models, compressed sensing, regularization, inverse problems

We present an all-in-one camera model that encompasses the architectures of most existing compressive-sensing light-field cameras, equipped with a single lens and multiple amplitude coded masks that can be placed at different positions between the lens and the sensor. The proposed model, named the equivalent multi-mask camera (EMMC) model, enables the comparison between different camera designs, e.g using monochrome or CFA-based sensors, single or multiple acquisitions, or varying pixel sizes, via a simple adaptation of the sampling operator. In particular, in the case of a camera equipped with a CFA-based sensor and a coded mask, this model allows us to jointly perform color demosaicing and light field spatioangular reconstruction. In the case of variable pixel size, it allows to perform spatial super-resolution in addition to angular reconstruction. While the EMMC model is generic and can be used with any reconstruction algorithm, we validate the proposed model with a dictionary-based reconstruction algorithm and a regularization-based reconstruction algorithm using a 4D Total-Variation-based regularizer for light field data. Experimental results with different reconstruction algorithms show that the proposed model can flexibly adapt to various sensing schemes. They also show the advantage of using an in-built CFA sensor with respect to monochrome sensors classically used in the literature.

I. INTRODUCTION

Light field imaging is gaining in popularity for a number of applications, going from photo-realistic image-based rendering [START_REF] Levoy | Light Field Rendering[END_REF], [START_REF] Gortler | The Lumigraph[END_REF], [START_REF] Chen | Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields[END_REF] to computational photography [START_REF] Ng | Light Field Photography[END_REF], [START_REF] Veeraraghavan | Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing[END_REF], [START_REF] Xu | A high-resolution lightfield camera with dual-mask design[END_REF], [START_REF] Xu | High-resolution Lightfield Photography Using Two Masks[END_REF], glass-free 3D displays [START_REF] Lee | Additive Light Field Displays: Realization of Augmented Reality with Holographic Optical Elements[END_REF], [START_REF] Wetzstein | Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting[END_REF], and computer vision applications [START_REF] Wang | Occlusion-aware depth estimation using light-field cameras[END_REF]. A light field is a sampled version of the 7D plenoptic function that describes the intensity of the light rays interacting with the scene as received by an observer at every point in space, along any direction of gaze, for all times and every wavelength [START_REF] Adelson | The Plenoptic Function and the Elements of Early Vision[END_REF]. In computer vision, light fields are commonly represented as four-dimensional (4D) functions with two spatial and two angular (or directional) dimensions [START_REF] Levoy | Light Field Rendering[END_REF], [START_REF] Gortler | The Lumigraph[END_REF]. In this point-ofview, they can be seen as collections of two-dimensional (2D) images, which are called viewpoints or sub-aperture images or angular images, taken from different vantage points. The difference between viewpoints of the same scene enables not only reconstruction of 3D objects but also photo rendering with controlled depth-of-field.

Over the past decades, a number of acquisition devices have been developed by both academic and industrial groups, with the aim of efficiently capturing 4D light fields. Existing devices can be classified into two categories: multi-view imagers which directly acquire different viewpoints using optical and mechanical setups, and multi-view coders which encode angular information of 4D light fields onto 2D sensor This work was supported by the EU H2020 Research and Innovation Programme under grant agreement No 694122 (ERC advanced grant CLIM). images and then "compute" original viewpoints. The first category includes imaging systems involving single cameras mounted on mechanical gantries moving over a spherical or planar surface [START_REF] Levoy | Light Field Rendering[END_REF], and multiple cameras arranged in an array fashion (also called camera arrays) [START_REF] Yang | A Real-Time Distributed Light Field Camera[END_REF], [START_REF] Wilburn | High Performance Imaging using Large Camera Arrays[END_REF]. While moving camera gantries have major difficulties for capturing nonstatic scenes, specially those with fast moving objects and illumination changes, camera arrays suffer from low angular resolution due to physical constraints. In addition, both of these light field capturing systems are often quite bulky and not suitable for popular consumer uses.

In comparison with these cumbersome setups, compact devices have been designed by modifying the architecture of conventional cameras in order to capture the light field passing through the camera main lens. Early prototypes commonly known as "plenoptic cameras" [START_REF] Adelson | Single Lens Stereo with a Plenoptic Camera[END_REF], [START_REF] Ng | Light Field Photography with a Handheld Plenoptic Camera[END_REF] use micro-lens arrays (MLA) placed on the optical path before the camera sensor to separate incoming light rays by their incident angle. Thus, this separation allows for acquiring multiple viewpoints of the scene. Alternatively, several recent light-field camera designs consider coded masks instead of an MLA to modulate 4D light fields into 2D projections that are captured by the camera sensor, and use a reconstruction algorithm to restore unmodulated light fields from their projections [START_REF] Babacan | Compressive sensing of light fields[END_REF], [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]. Note that most mask-based cameras are actually multi-view coders; in contrast, MLA-based (plenoptic) cameras belong to the class of multi-view imagers.

While an MLA-based camera can be modelled as a virtual or equivalent camera array (ECA) [START_REF] Mignard-Debise | A Unifying First-Order Model for Light-Field Cameras: The Equivalent Camera Array[END_REF], there is no published work providing a similar equivalent system for mask-based prototypes. We introduce, in this work, a unifying model that represents any single-lens light-field camera with coded masks as an equivalent multi-mask camera (EMMC). The EMMC formalism enables the comparison of different camera designs and acquisition settings, e.g. using monochrome or CFA-based sensors, using single or multiple acquisitions, varying the pixel size. In particular, the architecture combining an attenuation mask and a CFA-equipped sensor can be seen as a dual-mask model: one is the coded mask which multiplexes angular (or directional) information and the other corresponds to the CFA pattern which samples color information. Preliminary results on joint demosaicing and light field reconstruction with CFA sensors have been published in [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF]. Also, please note that the proposed camera model concerns only (amplitude) mask-based cameras and does not include camera designs using phase plates or micro-lens arrays.

Regarding light field reconstruction methods, the EMMC model, in which the sampling operator modeling the coded projection process of light fields can be flexibly computed according to the camera design changes, can be used with any reconstruction algorithm. While we first validate the model with existing dictionary-based reconstruction algorithms, we the "Tulip" scene (5 × 5 views) from the Stanford Lytro Illum dataset [START_REF] Raj | Stanford Lytro Light Field Archive[END_REF]. The coded projection is computed with a RGBW mask as in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], and it is captured in three different ways: acquisition with CFA-sensor (b), full (RGB) color acquisition (c) and acquisition with monochrome sensor (d). Experimental results show that the reconstruction with our variational method from CFA-based acquisition is comparable to the result obtained with a state-of-the-art dictionary-based method [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] on full-color acquisition. In contrast, the combination of deep-learning-based algorithm [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] and monochrome acquisition produces a result with incorrect-color reconstruction.

also develop a generic variational algorithm by extending the total variation regularization for light field data. Experimental results on both synthetic and real light field data, show that our camera model can flexibly adapt to various acquisition schemes. They also show the advantage of using an in-built CFA sensor with respect to monochrome sensors classically used in the literature. In fact, when using the CFA sensor combined with a coded mask, the proposed model allows us to perform a joint demosaicing and light field reconstruction. Also, the model gives the flexibility of increasing the number of shots, thus giving better results compared to reconstructions from one-shot acquisitions. In summary, the contributions of this paper are the following: • The validation of the proposed model with various acquisition schemes using dictionary-based reconstruction methods as well as an approach combining a dictionary based reconstruction with a K-order differential regularization.

•
II. RELATED WORK This section gives an overview of main computational light field camera designs, with a focus on camera designs aiming at capturing a whole light field with a single sensor, e.g. micro-lens-based and mask-based cameras. A taxonomy of computational light field cameras, according to the approach used to project the 4D light field onto a 2D sensor, can be found in [START_REF] Zhou | Computational Cameras: Convergence of Optics and Processing[END_REF].

MLA-based cameras

The first single-lens plenoptic camera has been introduced in [START_REF] Adelson | Single Lens Stereo with a Plenoptic Camera[END_REF], using an array of pinholes in front of the sensor. Following principles of integral imaging pioneered by Lippmann [START_REF] Lippman | La photographie intégrale[END_REF], each pinhole acts as a camera that creates a micro-image of the main lens aperture on a small area of the sensor. An array of micro-lenses can be considered instead of the pinhole array for better light efficiency. The authors of [START_REF] Adelson | Single Lens Stereo with a Plenoptic Camera[END_REF] also proposed an elaborated design which places a field lens in front of the MLA to reduce aberrations and place a relay lens right after the MLA to separate it from the sensor. A decade later, by modifying the original setup in [START_REF] Adelson | Single Lens Stereo with a Plenoptic Camera[END_REF], Ren Ng has developed the first hand-held light field camera [START_REF] Ng | Light Field Photography with a Handheld Plenoptic Camera[END_REF], [START_REF] Ng | Light Field Photography[END_REF] that slightly modifies the original setup of [START_REF] Adelson | Single Lens Stereo with a Plenoptic Camera[END_REF]. A light transport framework for lenslet light field cameras is proposed in [START_REF] Liang | A light transport framework for lenslet light field cameras[END_REF] taking into account non uniform angular pixel sensitivity to understand the limits of lenslet-based light field cameras. The authors in [START_REF] Wei | Improving light field camera sample design with irregularity and aberration[END_REF] also take into account lens aberrations in the light field camera design.

Mask-based cameras

Programmable aperture approaches have also been considered to sequentially capture subsets of light rays, hence timemultiplexing 2D slices of the 4D light field on the sensor, using a programmable non-refractive mask at the aperture [START_REF] Liang | Programmable Aperture Photography: Multiplexed Light Field Acquisition[END_REF], and exploiting the fast multiple-exposure feature of digital sensors. Shield fields are introduced in [START_REF] Lanman | Shield fields: Modeling and capturing 3d occluders[END_REF] to quantify the 4D attenuation of light rays due to occluders in mask-based camera designs. A good overview of the above camera designs can be found in [START_REF] Wetzstein | State of the Art in Computational Plenoptic Imaging[END_REF].

Since the light field data is typically high dimensional and compressible, its acquisition can be placed in a compressive sensing framework, in which the sensing matrix is materialized by a coded physical mask. The application of the compressive sensing framework to the problem of light field acquisition thus led to novel camera architectures, referred to as compressive coded aperture light field cameras. Thanks to the use of a coded mask, instead of recording a spatial multiplex of 2D slices of the light field, the photosensor records a set of linear measurements from which a higher resolution light field can be reconstructed.

This compressive sensing principle is applied in [START_REF] Xu | A high-resolution lightfield camera with dual-mask design[END_REF], [START_REF] Xu | High-resolution Lightfield Photography Using Two Masks[END_REF] where the 2D sensor captures optically coded projections using two attenuation masks separately placed at the aperture plane and in front of the sensor. Given the measurements recorded on the sensor, the light field is then reconstructed using a least square minimization with a total variation regularization constraint. Similarly, the authors in [START_REF] Babacan | Compressive Light Field Sensing[END_REF] place a randomly coded mask on the aperture plane to obtain incoherent measurements of the light field. Multiple shots are captured as random linear combinations of angular images by separately opening one region of the aperture and blocking the light in the others. The authors in [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF] propose a camera architecture that records optically coded projections on a single image sensor, while the authors in [START_REF] Miandji | Compressive Reconstruction in Reduced Union of Subspaces[END_REF] and [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] use respectively a random binary mask or a moving colour coded mask affixed to the sensor to extract incoherent measurements. In both cases, the light field is then reconstructed using a compressive sensing framework, assuming that the light field is sparse in a domain defined by an overcomplete dictionary [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] or an ensemble of 2D separable dictionaries [START_REF] Miandji | Compressive Reconstruction in Reduced Union of Subspaces[END_REF]. Light field reconstruction from a sparse set of measurements is an inverse problem that can also be efficiently solved using deep learning techniques [START_REF] Vadathya | Learning light field reconstruction from a single coded image[END_REF], [START_REF] Gupta | Compressive light field reconstructions using deep learning[END_REF], [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], [START_REF] Inagaki | Learning to capture light fields through a coded aperture camera[END_REF]. The authors in [START_REF] Vadathya | Learning light field reconstruction from a single coded image[END_REF], [START_REF] Gupta | Compressive light field reconstructions using deep learning[END_REF], [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] assume a pre-defined mask pattern and propose convolutional neural network architectures to reconstruct the light field from the set of measurements given the coded mask. In contrast, the authors in [START_REF] Inagaki | Learning to capture light fields through a coded aperture camera[END_REF] pose the coded aperture acquisition and light field reconstruction as an auto-encoder and optimize the mask pattern together with the parameters of the reconstruction algorithm in an end-to-end auto-encoder learning. The higher resolutions achieved thanks to compressive acquisition is shown in [START_REF] Pegard | Compressive light-field microscopy for 3d neural activity recording[END_REF] to be useful in light-field microscopy for 3D neural activity recording. Although not related to acquisition, we would like to mention that similar principles using masks or light-attenuating layers have also been considered to realize compressive light field displays [START_REF] Wetzstein | Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting[END_REF].

Other single sensor camera designs

The authors in [START_REF] Venkataraman | Picam: An Ultra-thin High Performance Monolithic Camera Array[END_REF] also proposed a miniaturized and integrated camera array where multiple cameras share the same sensor, while in [START_REF] Iseringhausen | 4d imaging through spray-on optics[END_REF], the capture is done through a spray of water droplets on an acrylic surface placed in front of the camera [START_REF] Iseringhausen | 4d imaging through spray-on optics[END_REF]. Each drop of water produces a distorted view of the scene. A camera architecture is proposed in [START_REF] Hirsch | A switchable light field camera architecture with angle sensitive pixels and dictionary-based sparse coding[END_REF] allowing us to switch between a high resolution 2D image capture and a light field capture. This camera uses angular sensitive pixels (ASP) that allow for angular radiance information to be captured without the need for additional microlenses or light-blocking masks.

III. IMAGING MODEL: BACKGROUND

In this section, we recall basic concepts of light field parameterization and imaging based on geometrical optics for conventional cameras featuring a main lens placed on one side and a photo-sensitive surface (also called "sensor") on the opposite side. While the plane at which the camera lens is located is called the "lens plane", the plane containing the photo-sensitive surface is often referred to as the "sensor plane". We then establish the link between the light field induced by a world 3D scene and the image of this scene through the main lens as a projection onto the sensor plane.

A. Light field parameterization

We adopt the two-plane parameterization, in which any light field can be represented by a collection of light rays passing through two points on a pair of parallel planes. Given two constants Z and W , we denote by L Z (X, U , λ) the radiance in the wavelength λ along a ray passing through the point X = (X, Y ) on a plane of depth Z and the point U = (U, V ) on a plane of depth W , where Z = (Z, W ) is simply the depth vector of the two parameterization planes (see Fig. 2). The couple (X, Y ) (resp. (U, V )) represents the spatial coordinates (resp. angular coordinates) of the light field L Z . Here, the first reference plane (i.e. the X-plane) is called the "focal plane" and the second one (i.e. the U -plane) is called the "aperture plane" (which is often assumed to coincide with the "lens plane"). Common conventions often assume that the region between these two planes is a free space (i.e. there is no occlusion) to guarantee the preservation of radiance along light rays traveling in this region. Given a scene and a fixed camera position, the parameterization of the light fields can be extended for any two parallel planes of arbitrary depths (see Fig. 2 and [START_REF] Liang | A light transport framework for lenslet light field cameras[END_REF], [START_REF] Liang | Light field analysis for modeling image formation[END_REF] for details). Hence different light fields representing the same (static) scene can be obtained by changing the depth position of the focal plane or the aperture plane, or even both of them. Fig. 2: Lens imaging and light field parameterization. The light field induced by the scene can be parameterized by any two parallel planes, e.g. the pair of X-and U-plane (black and blue dashed lines respectively). The camera lens transforms the "world light field" L Z (also called the "outside light field") into the "in-camera light field" Lz which is a distorted version of L Z with L Z (X, U , λ) = Lz(x, u, λ). The x-plane (resp. u-plane) is the conjugate plane of the X-plane (resp. U -plane) through the camera lens. Note that the x-plane is not necessary coincide with the sensor plane of the camera.

B. Image formation in a conventional camera

In this paper, we consider single-lens architectures and without loss of generality, we assume that the main lens of the camera is centered at origin (i.e. at the point O = (0, 0, 0)). Let us consider a light ray starting from a point on an object of the scene and passing through two points X and U located on two planes of depth Z and W respectively. Due to refraction, the light ray bends after traveling through the camera lens and intersects the conjugate plane of the two reference planes at positions x and u as follows:

x = f Z + f X , z = f Z Z + f , u = f W + f U , w = f W W + f ,
where z (resp. w) denotes the depth of the conjugate plane of the X-plane (resp. U -plane), and where f denotes the camera focal length. Or in other words, the camera lens transforms the "world light field" L Z (also called the "outside light field") into the "in-camera light field" L z which is a distorted version of L Z with L Z (X, U , λ) = L z (x, u, λ) (see Fig. 2). When a translucent screen is placed exactly on the x-plane (i.e. the sensor plane coincides with the conjugate plane of the Xplane), classical radiometry states that the irradiance in the wavelength λ at a position x on the screen is equal to the following integral [START_REF] Ng | Light Field Photography[END_REF]:

I λ (x) = Θ L z (x, u, λ) du = f W + f W +f f Θ L Z Z + f f x, U , λ dU , (1)
where Θ ⊂ R 2 is the camera aperture domain. This equation represents the photograph formation in a conventional singlelens camera. It is nothing else than the projection of the 4D light field onto a 2D image by summing all rays intersecting the sensor plane at the same position.

IV. REVISITING ACQUISITION USING CODED MASKS

In this section, we study the image formation on a slightly modified conventional camera architecture, in which one or multiple (coded) masks can be inserted between the main lens and the sensor (see Fig. 3). In such an architecture, we denote by L zcam the in-camera light field parameterized by the sensor plane and the lens plane (here z cam = (z sensor , z lens ), where z sensor and z lens is the depth of the sensor plane and the main lens plane respectively). Also, let D = |z sensor -z lens | be the distance between two parameterization planes. We assume furthermore that L zcam is simply zero beyond the physical bounds of the camera sensor (denoted by Ω ∈ R 2 ) and the aperture Θ.

If the sensor is a translucent screen and a coded mask M (ξ, λ) (with feature size M , i.e. the size of each element or pixel of the mask) is placed at a distance d from the sensor plane (0 ≤ d ≤ D) in a way that d M is sufficiently small with respect to the visible spectrum (corresponding to wavelength in the range between 0.4 and 0.8 µm); under these conditions, the diffraction effect induced by the mask M can be ignored [START_REF] Asif | FlatCam: Thin, Lensless Cameras Using Coded Aperture and Computation[END_REF]. Then, the imaging equation can be rewritten as [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]:

I λ (x) = Θ M (ξ x,u,d , λ) L zcam (x, u, λ) du , (2)
where

ξ x,u,d = u + D-d D (x -u).
Each image I λ is called "coded projection" of the original light field L zcam in the color channel λ and can be seen as an angular-compressed version of the set of angular images {L zcam (•, u, λ)} u∈Θ . Note that Eqn. ( 1) is a particular case of Eqn. (2) when M = 1, i.e. when the mask is fully transparent.

Let us replace the translucent screen in the camera architecture by a simple monochrome sensor array that receives all incoming lights but does not distinguish the light colors. Accordingly, the intensity of an idealized infinitesimal pixel (i.e. the pixel size is considered as a point particle) located at the position x on the sensor plane can be computed as follows:

I(x) = Λ I λ (x) dλ = Θ×Λ M (ξ x,u,d , λ) L zcam (x, u, λ) du dλ , ( 3
)
where Λ is the range of light wavelengths that can be captured by the sensor. One can remark that if the mask M is not mounted directly on the sensor (i.e. d = 0) as suggested in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], all color channels of { I λ } λ∈Λ are merged to form the "grayscale" image I which contains only luminance information. In this case, both directional (angular) and spectral (color) information are compressed. Consequently, reconstructing the light field L zcam from its spectro-angular-compressed projection I is more challenging since it requires not only the estimation of the original viewpoints, but also the true color of each light field ray.

Fortunately, there are several acquisition schemes which permit avoiding the loss of all color information. a) Multi-spectral sensing: It is possible to place color filters on the camera optical path for capturing coded pro-jections in desired wavelengths. Each color channel can be obtained separately by considering the corresponding color filter. This enables multi-spectral imaging for arbitrary spectral (color) bands by the means of multiple (sequential) acquisitions; however, the total acquisition time increases proportionally to the number of colors that we want to capture. Another way to perform multi-spectral imaging without increasing the acquisition time is to use a prism to split incoming light beam into different color channels and use a large sensor array (or several small sensor arrays) to record each channel. Nevertheless, it requires major modifications of the camera architecture both in terms of optical setup and sensor implementation which are not convenient for popular consumers.

b) Monochromatic multi-shot acquisition: As in [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF], one can also consider multiple shots, each of which is taken using a different random colored pattern. The color-coded mask is indeed changed to create a new random pattern for increasing the incoherence. In order to retrieve color information, each pattern is generated in the same (color) tint. More interestingly, multi-spectral sensing can be seen as a particular case of monochromatic multi-shot acquisition, in which the random pattern used for each shot is actually a color component of the given color-coded mask.

c) Using built-in CFA: An alternative solution for color light field imaging is to use sensors with a built-in color filter array (CFA) in order to sample the color information. Accordingly, captured images are color mosaics, in which the color of each pixel is defined by the color filter located at the pixel position. Note that, in an equivalent setting with monochrome sensors, it amounts to placing the corresponding CFA pattern right in front of the (monochrome) sensor array.

V. EQUIVALENT MULTI-MASK CAMERA MODEL

In this section, we extend the conventional image formation model for single-lens single-mask cameras to more general situations, e.g. acquisition using multiple coded masks, multiple shots, and varying pixel sizes. We then introduce the notion of "Equivalent Multi-Mask Camera" that unifies various camera architectures and acquisition configurations. Using this EMMC formalism, we derive the imaging (sensing) matrix which maps a light field to a set of projected sensor images corresponding to a given situation. Also, we show the link between the proposed image formation model and the acquisition process on existing single-lens mask-based light field cameras.

A. Extended image formation model

The acquisition using a coded mask and a color sensor can be abstracted by an acquisition model using two coded masks and a monochrome sensor. In fact, the idea of using more than one mask has been introduced in [START_REF] Xu | A high-resolution lightfield camera with dual-mask design[END_REF], [START_REF] Xu | High-resolution Lightfield Photography Using Two Masks[END_REF] with a dual-mask design: one mask is placed before the sensor and the other one at the aperture.

1) Multiple masks at varying positions: By extending this concept, we propose a camera model equipped with a main lens, a monochrome sensor and a set of n m masks M = {M 1 , M 2 , . . . , M nm }, in which each mask M l is located between the camera sensor and aperture, at an arbitrary distance d l from the sensor plane. Accordingly, the coded projection of the in-camera light field L zcam , through the masks M and captured by the sensor, reads:

I(x) = Θ×Λ nm l=1 M l (ξ x,u,d l , λ) L zcam (x, u, λ) du dλ . (4)
The proposed multi-mask formalism does not only generalize the light field camera prototype introduced in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], but it also represents an equivalent model for most existing mask-based light field cameras. The readers are referred to section V-C for a detailed comparison of different mask-based designs and their equivalent multi-mask camera.

2) With varying pixel supports and multiple acquisitions: In reality, although (digital) sensor pixels are relatively small comparing to the main lens or the sensor itself, they have a certain size that can not be reduced to an idealized point particle. Each pixel therefore measures the quantity of incident light over a bounded region called "pixel support" to form the pixel value. Note that the pixel support may have various shapes depending on the sensor design. For the sake of simplicity, we only consider, in this work, regular rectangular sensors with square pixels. Under this assumption, let p be the pixel size and let k = (k 1 , k 2 ) ∈ Z 2 be the discrete coordinates of a sensor pixel. The support of this pixel is defined as the following rectangular domain:

C k = x k1 - p 2 , x k1 + p 2 × y k2 - p 2 , y k2 + p 2 ,
where (x k1 , y k2 ) = c k ∈ Ω denotes the pixel center. One can further envisage multiple acquisitions (shots) of the same scene and with the same camera position on the same sensor using different configurations of coded masks (e.g. changing mask position and pattern, inserting more masks or even removing some of them). Let n s be the number of desired acquisitions, then the s-th captured image can be expressed as:

I (s) k = C k ×Θ×Λ n (s) m l=1 M (s) l (ξ (s)
x,u,d l , λ) L zcam (x, u, λ) dx du dλ ,

where

1 ≤ s ≤ n s and M (s) = {M (s) 1 , M (s) 2 , . . . , M (s)
n (s) m } is the set of n (s)
m masks used for this acquisition. As each image I (s) can be seen as a finite collection of sensor pixel values, the imaging equation Eqn. [START_REF] Veeraraghavan | Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing[END_REF] establishes the link between the continuous light field L zcam and the discrete form of its coded projections measured by the sensor. In practice, light field reconstruction algorithms tend to produce (from the set of measurements {I (s) k } s,k ) an estimated discrete version of L zcam (up to a given spatio-angular resolution) instead of its continuous counterpart. For this purpose, we introduce in section V-B the imaging (sensing) matrix representing the transformation of a discrete 4D color light field into captured sensor images of its coded projection by taking into account the spatial, angular, and spectral resolutions, as well as the size of sensor pixels and the number of acquisitions to be performed.

B. Discretized sensing matrix

Given two strictly positive constant x and u , we assume that the in-camera light field L zcam is sampled in red, green and blue colors (which correspond to the wavelength λ R , λ G and λ B respectively) according to the grid of spatial and angular coordinates:

X = {x i = x 0 + x i ∈ Ω for i = (i 1 , i 2 ) ∈ N 2 } , U = {u j = u 0 + u j ∈ Θ for j = (j 1 , j 2 ) ∈ N 2 }
, where x 0 = (x 0 , y 0 ) and u 0 = (u 0 , v 0 ) are some reference coordinates. Let n = n x n y be the number of spatial samples and ν = ν u ν v the number of angular samples (also known

     I (1)
I (2) . . .

I (ns)      I =      H (1) 0 . . . 0 0 H (2) . . . 0 . . . . . . . . . . . . 0 0 . . . H (ns)      H      φ (1,R,1) . . . φ (ν,R,1) φ (1,G,1) . . . φ (ν,G,1) φ (1,B,1) . . . φ (ν,B,1) φ (1,R,2) . . . φ (ν,R,2) φ (1,G,2) . . . φ (ν,G,2) φ (1,B,2) . . . φ (ν,B,2) . . . . . . . . . . . . . . . . . . . . . . . . . . . φ (1,R,ns) . . . φ (ν,R,ns) φ (1,G,ns) . . . φ (ν,G,ns) φ (1,B,ns) . . . φ (ν,B,ns)      Φ                  L (1,R) . . . L (ν,R) L (1,G) . . . L (ν,G)
L (1,B) . . .

L (ν,B)                  L , (6)
as "number of views"). We then denote by L ∈ R 3νn the column vector which represents the sampled version of L zcam according to the sampling grid X × U defined as:

L c i,j = L zcam (x i , u j , λ c ) , for every c ∈ C = {R, G, B}.
By construction, the size of the discrete light field L is n x ×n y ×ν u ×ν v ×3. We denote the set of discrete spatial and angular coordinates by I and J respectively as follows:

i ∈ I = {1, 2, . . . , n x } × {1, 2, . . . , n y } ⊂ N 2 , j ∈ J = {1, 2, . . . , ν u } × {1, 2, . . . , ν v } ⊂ N 2 .
To simplify our notation, we adopt the following arrangement L = {L (j,c) } 1≤j≤ν,c∈C , where j = j 1 + ν u (j 2 -1) and each L (j,c) ∈ R n denotes the vectorized version of the j-viewpoint in the wavelength λ c . Using this arrangement, the discrete form of the multi-mask-based multi-shot acquisition scheme in Eqn. ( 5) can be expressed as a matrix-vector multiplication, as shown in Eqn. [START_REF] Xu | A high-resolution lightfield camera with dual-mask design[END_REF]. Here, I ∈ R nsrn denotes the collection of n s acquired images, each H (s) ∈ R rn×n is the matrix representing the integration (summation) of incident light rays on the same sensor pixels for the s-th acquisition, r = ( x p ) 2 is the squared ratio between the spatial sampling step and the pixel width, each φ (j,c,s) = n (s) m l=1 M (j,c,s) l ∈ R n×n is a sparse matrix and each M (j,c,s) l ∈ R n×n is a diagonal matrix containing the coefficients in the wavelength λ c of the mask M (s) l on its diagonal. Moreover, Φ ∈ R nsn×3νn is called the "coded projection matrix" containing the coefficients of implemented coded masks {M (s) } 1≤s≤ns , and H ∈ R nsrn×nsn is called the "sampling matrix" which models the sampling operations at the sensor level. By construction, the transform that relates the original (discrete) light field L to the sensor images I is given by: Ψ := HΦ ∈ R nsrn×3νn . (7) One can easily see that the imaging matrix Ψ is sparse, due to the structure of H and Φ. Note that when r = 1, H becomes an identity matrix, and thus implying Ψ = Φ, which represents the conventional compressive sensing (CS) framework of light fields as described in [START_REF] Babacan | Compressive sensing of light fields[END_REF], [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF], [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF]. Most early studies have proposed to reconstruct light fields with the same spatial resolution as of the captured sensor images. However, there is no published work that addresses the reconstruction of higherspatial-resolution light fields from low-spatial-resolution coded projections. Note that the spatial sub-sampling technique used in [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] assumes that the spatial resolution of the light field coincides with the sensor resolution.

In practice, for real camera systems using amplitude-coded masks [START_REF] Babacan | Compressive Light Field Sensing[END_REF], [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], the imaging matrix of the whole system is estimated using the so-called "whiteboard" scenes. Details of such estimation procedure can be found in the supplement material of [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF]. The estimation of the imaging matrix Ψ is also called as (PSF) "calibration".

C. Link with existing single-lens mask-based cameras

Most of existing single-lens camera designs represent special cases of Eqn. ( 5) by changing the number of masks n m , the mask pattern M l and the distance d l between the mask M l and the sensor plane. One trivial example is conventional (photography) cameras (which do not possess any coded mask) corresponding to n m = 1 and M 1 = 1. Regarding mask patterns, various classes of coded masks in the literature (such as broadband, sum-of-sines, random, monochrome, color etc.) can be easily modeled by using appropriate M l functions.

a) Coded aperture cameras (Fig. 4-a): Coded aperture cameras (also known as pupil plane coding cameras) use one mask located on the aperture plane of a traditional lens, i.e. corresponding to n m = 1 and d 1 = D. The mask pattern is either a sum of cosine signals in spatial domain (corresponding to a sum of Dirac delta functions in the Fourier domain) [START_REF] Veeraraghavan | Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing[END_REF], a random pattern [START_REF] Babacan | Compressive sensing of light fields[END_REF], or a learned pattern [START_REF] Inagaki | Learning to capture light fields through a coded aperture camera[END_REF].

b) Sensor-side mask-based cameras (Fig. 4-b): These camera architectures (also known as sensor side coding cameras) use a mask placed on the sensor plane or in front of the sensor [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF], [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], hence usually setting n m = 1 and assuming d 1 D. The mask patterns are in general random monochrome masks with Gaussian weights [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF] or random color coded masks [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF], [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF]. When d 1 = 0, the mask plane coincides with the sensor plane. This setting allows modeling specific implementations at the sensor level, e.g. color filter arrays (such as the Bayer filter array) can be considered as particular color coded masks to encode color information into photo-sensitive sensors.

c) Dual-mask cameras: Previously, the two-mask camera proposed in [START_REF] Xu | A high-resolution lightfield camera with dual-mask design[END_REF], [START_REF] Xu | High-resolution Lightfield Photography Using Two Masks[END_REF] is special case of the generic model of Eqn. ( 4), with n m = 2, d 1 = D and 0 < d 2 < D. In this camera model, one mask is placed in the aperture plane while the second mask is positioned in the optical path of the camera (see Fig. 4c). Recently, the authors of [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF] consider a camera architecture combining a coded (attenuation) mask and a CFAbased sensor which is often implemented in standard consumer cameras. This architecture can be abstracted by an equivalent two-mask model: one mask is placed before the sensor (d 1 > 0) and the other one is mounted directly on the monochrome sensor (see Fig. 4d).

VI. RECONSTRUCTION ALGORITHMS A. Dictionary-based reconstruction

The compressive sensing theory [START_REF] Candes | An introduction to compressive sampling[END_REF] relies on the assumption that the signal is sparse (or compressible) in some transform domain like wavelets, discrete cosine transform (DCT), or dictionaries learned from large datasets. In the context of compressed acquisition of light fields using coded masks, original light fields can be restored by solving a basis pursuit denoising (BPDN) problem [START_REF] Chen | Atomic Decomposition by Basis Pursuit[END_REF], given an overcomplete dictionary as described in [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]. We adapt this BPDNbased approach for the multi-mask-based multi-shot acquisition (Eqn. ( 6)).

Let us denote by D = D (1) D (2) . . . D (n d ) ∈ R 3νq×n d a light field dictionary, where n d ≥ 3νq is the number of dictionary atoms and each D (d) = {D (j,c,d) } 1≤j≤ν,c∈C ∈ R 3νq is a (color 4D) light field atom of size q x × q y × ν u × ν v × 3 (with q x q y = q). Here, the first two dimensions are spatial dimensions, the third and fourth dimensions are angular dimensions and the last one is for the number of color channels. In practice, due to the large size of light field data, one usually trains dictionaries using small patches obtained by dividing full-size light fields over the spatial domain while including all angular and color dimensions to reduce the learning time of dictionaries. Using spatially small patches instead of whole light fields also allows increasing the number of training examples (or training samples) for the learning stage. Given a pre-trained dictionary, the reconstruction is first performed on light field patches of the same size as the dictionary atoms, and afterward reconstructed patches are aggregated to compute the light field by averaging pixels on overlapped regions.

For the light field L ∈ R 3νn , we consider a set of patches L = {L (p) } 1≤p≤np extracted from L, where n p denotes the number of patches and each L (p) = {L (j,c,p) } 1≤j≤ν,c∈C ∈ R 3νq is a patch of size q x × q y × ν u × ν v × 3. We define the corresponding extracted sensing matrix Ψ (p) and extracted sensor image patches I (p) such that I (p) = Ψ (p) L (p) , where

Ψ (p) ∈ R nsrq×3νq and I (p) ∈ R nsrq . Assuming that L (p)
has a sparse representation in D, there exists a sparse vector

β = (β 1 , β 2 , . . . , β n d ) ∈ R n d such that Dβ = n d d=1 β d D (d)
approximates L (p) (i.e. Dβ L (p) ). Therefore, the reconstruction of L (p) from I (p) given Ψ (p) and D amounts to minimizing the following convex energy:

E (p) D (β) = 1 2 Ψ (p) Dβ -I (p) 2 2 + η β 1 , (8)
which can be efficiently solved by many existing algorithms (e.g. OMP [START_REF] Pati | Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition[END_REF], LARS [START_REF] Efron | Least angle regression[END_REF], ISTA [START_REF] Beck | A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems[END_REF], etc.) as used in [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] with different sensing matrices. Note that the proposed formalization is very generic and can be applied to various mask-based cameras and acquisition schemes. It also allows to flexibly use different BPDN solvers by considering the appropriate sensing operator Ψ (p) for a given acquisition scenario.

The computation complexity of dictionary-based reconstruction algorithms is dependent on the patch size, as well as the overlapping patch distance (i.e. the pixel-wise cornerto-corner distance between two neighboring patches that overlap with each other). A larger patch size will lead to a higher computational and storage complexity. Similarly, a smaller value for patch distances increases the computational complexity. On the other hand, both of these parameters affect the reconstruction quality. The larger the patch size, the sparser the representation, leading to a higher reconstruction quality. The smaller the overlapping distance, the higher the number of pixels to be reconstructed and averaged to get the final pixel value, hence a higher image quality. As shown in [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF], dictionary-based methods typically produce noise-like artifact when using non-overlapping patches, specially around the sharp edges between the foreground and background. In contrast, the use of overlapped patches is shown to be able to improve significantly the reconstruction quality (with a gain of several dB, see [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]); however it increases the computational complexity by orders of magnitude. As a result, finding suitable values for the patch size and overlapping distance that give an optimal trade-off between complexity and performance is often a tedious process. In the next section, we describe a differentialbased reconstruction algorithm that aims at reducing block and noise-like artefacts resulting from inadequate patch size and overlapping distance.

B. Differential-based reconstruction with patch-based constraint

Total variation (TV) [START_REF] Rudin | Nonlinear Total Variation Based Noise Removal Algorithms[END_REF] is one of the most well-known regularizers in image processing. In light field processing, TVbased regularization is commonly used to regularize angular images and epipolar plane images (EPIs) for light field denoising, impainting or depth estimation [START_REF] Goldluecke | The variational structure of disparity and regularization of 4d light fields[END_REF], [START_REF] Monteiro | Dense lightfield disparity estimation using total variation regularization[END_REF]. We introduce in this section a TV functional for 4D color light fields by taking into account the two-plane parameterization and the differential with arbitrary order.

Let us consider the following partial derivative operator applied to the light field L at the (discrete) spatio-angular coordinates (i, j) ∈ I × J given by:

(∂ α L) c i,j = ∂ α 1 L ∂x α1 ∂y α2 ∂u α3 ∂v α4 c i,j = ∂ α1 ∂x α1 ∂ α2 ∂y α2 ∂ α3 ∂u α3 ∂ α4 ∂v α4 L c i,j
,

where α = (α 1 , α 2 , α 3 , α 4 ) ∈ N 4 is a non-negative integer vector, α 1 = 4 i=1 α i is called the order of ∂ α and ∂ α ∂ dir α
denotes the α-order directional derivative with respect to the direction dir ∈ O = {x, y, u, v}. To simplify notations, let (∇ K L) i,j = {(∂ α L) c i,j } c∈C, α 1=K denote the vector that gathers all K-order partial derivative of L at (i, j), we then define the K-order-differential regularizer of L as follows:

DV K (L) = (∇ K L) i,j 1,2 := i∈I j∈J α 1=K c∈C (∂ α L) c i,j 2 ,
where • 1,2 denotes the "mixed l 1l 2 norm". In particular, when K = 1, (∇ 1 L) i,j is called the "spatio-angular gradient" computed at every position (i, j) of the light field L and DV 1 (L) can be seen as the total variation of L with respect to the spatio-angular gradient operator ∇ 1 .

Considering the proposed regularizer, we aim at solving the following minimization problem:

min L∈R 3νn 1 2 ΨL -I 2 2 + µ DV K (L) + ρ 2 PL -L 2 2 , (9)
where

L = { L (p)
} 1≤p≤ np is a collection of pre-estimated light field patches, P denotes the extraction operator applied to L with respect to the patch location L, and µ, ρ ≥ 0 are some nonnegative parameters. While the first quadratic term ΨL -I 2 2 measures the fidelity between the captured sensor images I and the projections of L according to the imaging matrix Ψ, the second quadratic term PL -L 2 2 measures the similarity between the extracted patches of L and the given estimates L. Note that the users are free to use any convenient methods (e.g., dictionary-based or deep-learning-based methods) in order to compute L. In practice, the latter can be obtained by gathering the reconstruction results of different methods or of the same method but with different parameters. For simplicity, we only consider here patches of the same size reconstructed using dictionary-based methods.

A possible choice to solve Eqn. ( 9) is the proximal gradient method (also known as forward-backward splitting method [START_REF] Combettes | Signal Recovery by Proximal Forward-Backward Splitting[END_REF], [START_REF] Beck | A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems[END_REF]) that requires computing prox DV K (the proximity operator of DV K ). It is however not easy to implement prox DV K in the case of high dimensional light fields. In this work, we consider instead the full-splitting approach [START_REF] Condat | A Generic Proximal Algorithm for Convex Optimization-Application to Total Variation Minimization[END_REF] which allows us to design an iterative algorithm using only "simple" operations as follows:

Choose the parameters γ, τ > 0 and the initial estimates

L (0) ∈ R 3νn , K (0) ∈ R 3νn4 K
. Then iterate, for ≥ 0:

L ( +1) = Ψ * (ΨL ( ) -I) + ρP * (PL ( ) -L) , (10)
L ( +1) = L ( ) -γ L ( +1) + (∇ K ) * K ( ) , (11)
K ( +1) = ∇ K 2L ( +1) -L ( ) , (12)
K ( +1) = prox τ (µ • 1,2 ) * K ( ) + τ K ( +1) , (13)
in which L ( ) converges to a solution of Eqn. ( 9) if the proximal parameter condition is satisfied:

γ 1 2 Ψ * Ψ + ρ 2 P * P + τ (∇ K ) * ∇ K < 1 .
Here, K ( ) are dual variables (in the sense of Fenchel-Rockafellar duality [START_REF] Bauschke | Fenchel-Rockafellar Duality[END_REF]) and K (0) is usually initialized with 0. The functional (µ • 1,2 ) * denotes the Fenchel-Rockafellar conjugate of µ • 1,2 , which satisfies the Moreau identity as:

prox τ (µ • 1,2) * (K) = K -τ prox (µ • 1,2)/τ (K/τ ).
Note that the proposed formulation can be applied not only to regularly overlapped patches, but also to patches that are non-overlapped, sparsely extracted or extracted in an irregular manner. It provides thus a flexible way to incorporate different patch-based estimators and improve the reconstruction quality. Moreover, in comparison with dictionary-based approaches, the proposed algorithm allows to obtain reconstructed results with homogeneous regions while avoiding block artifacts and noise-like artifacts that often happen when aggregating small patches as observed in [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] (see [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF] for more details).

VII. EXPERIMENTAL RESULTS

A. Datasets and acquisition setups

We evaluate the proposed algorithms on both synthetic and natural light fields. The synthetic light fields used in the experiments are collected from the MIT Media Lab archive [START_REF] Wetzstein | Compressive Light Field Photography Siggraph 2013 project[END_REF] and have 5×5 views of 840×593 pixels. The natural light fields come from the Stanford Lytro Light Field Archive [START_REF] Raj | Stanford Lytro Light Field Archive[END_REF] and Kalantari's dataset [START_REF] Kalantari | Learning-based View Synthesis for Light Field Cameras[END_REF]. These natural datasets have been captured using the Lytro Illum camera and have an angular resolution of 14 × 14 viewpoints for the first dataset (resp. 8×8 viewpoints for the second). However, they suffer from strong vignetting effects on peripheral views due to mechanical and optical imperfections. For that reason, we only take into account the 5 × 5 center angular images as reference (a.k.a ground truth) in our experiments.

A comparison between different masks patterns has been reported in [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]. In the scope of this paper, we will mainly focus on the comparison between different acquisition scenarios using different light field reconstruction methods. To have a homogeneous study on different light field reconstruction approaches, we consider the camera architecture featuring one main lens and one color (coded) attenuation mask inserted from the MIT Archive [START_REF] Wetzstein | Compressive Light Field Photography Siggraph 2013 project[END_REF]. The exposure time for each color channel of the RGB acquisition (scenario (i)) is t 3 For each shot of a three-shot acquisition using monochrome sensor, the exposure time equals to t 3 , while it is t for a one-shot acquisition using CFA sensor. The total exposure time for each acquisition scheme remains equal to t. The quality of captured images (in terms of brightness and signal-to-noise ratio) is indeed affected by the exposure time. Note that the zooms are contrast-enhanced for better visualization, while the original image intensities keep unchanged.

between the main lens and the camera sensor which can be either monochrome or CFA-based. Note that the combination of color coded mask and CFA sensor have been recently reported in [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF]. It is nothing else than a particular case of our EMMC model (see Fig. 4d) and can be straightforwardly modeled using Eqn. ( 4). We assume that the implemented CFA is the wellknown Bayer pattern [START_REF] Bayer | Color Imaging Array[END_REF] in all our simulations. Nevertheless, other CFA patterns such that those proposed in [START_REF] Hirakawa | Spatio-Spectral Color Filter Array Design for Optimal Image Recovery[END_REF], [START_REF] Condat | A New Color Filter Array With Optimal Properties for Noiseless and Noisy Color Image Acquisition[END_REF] can be also considered without modification of the reconstruction algorithm.

Regarding the choice of coded masks, we use a random RGBW (stands for red-green-blue-white) pattern as the attenuation mask which is placed before the sensor, representing the same setup as described in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], in order to provide a fair comparison with it. This kind of RGBW mask is actually a random color coded mask, similar to the one considered in [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]; the two masks differ from each other by the distributions used for generating the color patterns. Using the same RGBW attenuation mask, we consider three acquisition scenarios:

(i) color-by-color (or multi-spectral) acquisition as in [START_REF] Babacan | Compressive Light Field Sensing[END_REF], [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF], in which each coded projection is a (full color) RGB image; (ii) acquisition with a CFA-equipped sensor as in [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF], in which each coded projection is a color mosaic; (iii) acquisition with a monochrome sensor (i.e. bare photosensitive sensor without CFA) as in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF], in which each coded projection is just a gray-scale image. Examples of captured images on the sensor and reconstruction results for the three scenarios are illustrated in Figs. 1, 5 and6. The readers are refereed to Section VII-B for further technical details of the reconstruction algorithms.

As a side note, one may recall that the first scenario corresponds to performing three acquisitions (i.e. three shots) using an equivalent multi-mask camera equipped with red, green and blue color filters as specific color masks in addition to the used RGBW mask and monochrome sensors. In contrast, the second scenario is equivalent to performing only one acquisition using an EMMC composed of the RGBW attenuation mask and a CFA mask mounted on the monochrome sensor. For the two first acquisition scenarios, the number of masks in the EMMC representation is n m = 2. The third one, which is the simplest among the three acquisition scenarios, represents the "minimal" configuration of the EMMC model featuring a coded mask and a monochrome sensor array, in which each captured image corresponds to a single-shot acquisition. For the scenarios (ii) and (iii) which require only one shot per captured coded projection, the light field reconstruction from single and multiple shots are envisaged. Moreover, acquisitions with different pixel sizes are also considered for all three scenarios.

B. Light field reconstruction

In this section, we analyze the above acquisition scenarios, encompassed by the proposed camera model, and compare the proposed differential and dictionary-based light-field reconstruction approach with a dictionary-based approach. For sake of simplicity, we consider here the first-order differential regularization (i.e. the so-called "spatio-angular" total variation). Hereafter, we refer the reconstruction algorithm that incorporates total variation and dictionary-based estimation constraint (Section VI-B) as the "differential and dictionary based algorithm" (abbreviated by "TV-Dict"). For comparison purposes, we use the ADMM algorithm [START_REF] Boyd | Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers[END_REF] for the sparse decomposition task by adapting the sensing matrix in each acquisition scenario and name this dictionary-based reconstruction method the "Dict-ADMM" method. Note that the authors of [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF] use the SL0 algorithm [START_REF] Mohimani | A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed 0 Norm[END_REF] to compute the sparse decomposition, resulting to an other dictionary-based method which is different to Dict-ADMM. For the latter, we consider light field patches of spatial dimension 9×9 pixels with a stride of 5 pixels. In our experiments, the TV-Dict algorithm uses Dict-ADMM results for the patch-based constraint, however, results obtained with any dictionary-based reconstruction algorithms can be used instead. Also, for sake of reference, we provide the performance achieved by the deep-learning-based method The three scenarios produce visually similar results when the reconstruction is performed from noise-free coded projections. In this noiseless situation, the three-shot acquisition with monochrome sensor gets the best PSNR score compared to one-shot RGB and CFA acquisitions. When there is sensor noise, the reconstruction quality drops for all scenarios. However, the one-shot CFA acquisition scheme provides very close reconstruction results for the noise-free and noisy cases. [21] whose the developed network architecture can only deal with single-shot acquisitions using monochrome sensors. 1) Results on synthetic data: In this section, we present the reconstruction results obtained with the proposed TV-Dict algorithm for the three acquisition scenarios on four test light fields collected from [START_REF] Wetzstein | Compressive Light Field Photography Siggraph 2013 project[END_REF]. In this experience, we used a dictionary trained on randomly selected patches which were extracted from the scenes of the same data set (excluding the test set), using the K-SVD algorithm [61]1 .

a) Exposure time impact analysis: We note that all the acquisitions in this experiment are performed with the same amount of exposure time to conduct a fair comparison between them. Here, we consider three-shot acquisitions when the monochrome sensor is used, since only one monochrome captured image does not allow to recover color information as depicted in [START_REF] Miandji | Sparse representation of visual data for compression and compressed sensing[END_REF]. More precisely, given a total time t , the acquisition time of each color channel of full RGB images is t 3 , the latter is also the time for each shot of a threeshot acquisition using monochrome sensor, while the exposure time for a single-shot acquisition using CFA sensor remains equal to t . Acquired images in these above settings are illustrated in Fig. 5 showing the effect of the exposure time on the image quality in terms of brightness and signal-to-noise ratio (SNR). The RGB acquisition (c.f. the second column of Fig. 5) produces indeed very dark coded projections when comparing to the two other scenarios (due to short exposure time and color filtering). Consequently, its sensor image is much more noisier (with PSNR = 42.88 dB -the lowest among the three scenario). The PSNR is computed taking the noiseless acquisition as a reference, it hence measures the quality of the acquired measurements. Having the same exposure time, the acquisition using monochrome sensor provides brighter coded projections (c.f. the fourth column of 5), implying better signal-to-noise ratio (PSNR = 45.90 dB) since the sensor can integrate photons of all the light wavelengths. As expected, the acquisition using CFA sensor (c.f. the third column of Fig. 5), which benefits from longer exposure time, tends to achieve the best PSNR score (over 50 dB) while maintaining the same overall brightness level as obtained with monochrome sensor (see its color-interpolated version shown in Fig. 5e for better visual evaluation instead of using the original one). Note that this interpolation is obtained with the classical gradientcorrection-based method [START_REF] Malvar | High-quality linear interpolation for demosaicing of Bayer-patterned color images[END_REF]. Although it looks similar to the captured image obtained by a full (RGB) color acquisition scheme (see Fig. 5b), one can remark that the demosaicing tends to oversmooth color transition and thus fails to recover high frequency color information.

Corresponding reconstruction from the coded projections of the three acquisition scenarios is shown in Fig. 6. All the results are obtained with the proposed variational algorithm (TV-Dict) using the parameters µ = 2 -8 and ρ = 2 -7 . For the comparison purposes, we present the results gathered from two situations: with and without sensor noises. When no noise is added, the three scenarios produce visually similar results (c.f. the first row of Fig. 6) and they are very close to the original light field (see Fig. 6a). In this situation, the reconstruction from three coded projections acquired with monochrome sensor (see Fig. 6d) is slightly better both in terms of visual and quantitative evaluations when compared to those obtained from only one coded projection in the case of CFA or RGB acquisition. It is mainly due to the higher number of recorded measurements (three-fold over one-shot CFA acquisition) and higher amount of incident light (three-fold over one RGB captured image).

Among the three scenarios with the same total exposure time, the one-shot CFA acquisition, which achieves the best SNR of (noisy) sensor images while having the less recorded samples, produces decent reconstruction results: slightly inferior PSNR (27.82 compared to 28.40 obtained with one RGB acquisition) and visually pleasant restoration of fine details (c.f. Fig 6g). In contrast, the results obtained from the two other schemes look much more noisy (see Figs. 6f and6h) although the number of recorded samples is three times higher. One can see that the overall reconstruction quality depends strongly on how noisy the acquired images are and how many images to be acquired. In our opinion, considering the three acquisition scenarios, the one using CFA sensor provides the best trade-off between the SNR and the number of measurements. Moreover, it also allows to retain some color information thanks to the CFA pattern (compared to the lost of all color information in the case of monochrome sensor).

As a side note, we may inform the readers that the reconstruction from pre-demosaiced coded projection (i.e. the color-interpolated version of the CFA acquisition, c.f. Fig 5e) is depicted in Fig. 6e for illustration purposes. The obtained result, which suffers from heavy noise-like artifacts, can be seen as a reconstruction from very noisy RGB acquisitions. Similar results with a different coded mask pattern are reported in [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF]. Also, in order to demonstrate the performance of the TV-Dict algorithm in terms of parallax reconstruction, we show in Fig. 7b the epipolar plane image (EPI) visualization of the reconstruction result from CFA acquisition. Although the reconstructed EPIs is slightly noisy compared to the original one (see Fig. 7a), we can easily remark that most slopes have been well restored.

b) Noise impact analysis: In order analyse the behavior of our approach in presence of noise, the coded projections are corrupted by random noises at the sensor level (i.e. the digitization stage). For the sensor noise model, we consider a mixed Poisson-Gaussian noise combined with a quantization noise to cope with the quantum nature of the light as well as the on-sensor electronic fluctuations and the analog-to-digital conversion (ADC) procedure. Accordingly, the sensor image can be expressed as:

I sensor = g ADC P( t I) + n read + n quan , (14)
where n read ∼ N (0, σ 2 read ) is a Gaussian variable representing the electronic read-out noise, n quan ∼ U([0, 1]) follows the uniform distribution on the interval [0, 1] and models the quantization at the ADC level, g ADC > 0 denotes the ADC gain (also called "camera gain"), t is the exposure time and P( t I) is the Poisson variable modeling the photon shot noise with respect to the amount of incident light I and the exposure time t . In the experiments reported below, the camera gain is set as g ADC = 0.8 (i.e. each generated photoelectron corresponds to 0.8 digital unit of pixel intensity in the output sensor image), the standard deviation of the read-out noise σ read = 2.5 (electrons RMS) and the total exposure time t = 1 20 second. By considering this noise model, it is also possible to replace the data fidelity term 1 2 ΨL -I 2 2 (which is adapted to an additive Gaussian noise) in Eqn. ( 8) and ( 9) by a Bayesian-based Poisson-Gaussian data term as suggested in [START_REF] Snyder | Image Recovery from Data Acquired with a Charge-coupled-device Camera[END_REF], [START_REF] Lantéri | Restoration of astrophysical images -the case of poisson data with additive gaussian noise[END_REF], [START_REF] Benvenuto | The Study of an Iterative Method for The Reconstruction of Images Corrupted by Poisson and Gaussian noise[END_REF]. However, in the context of photography, when the illumination and the exposure conditions are sufficient, the photon shot noise involved in the imaging process can be approximated by a Gaussian noise. For that reason, we propose to simply substitute the noise-free coded projection I by its noisy version I = 1 tgADC I sensor in Eqn. ( 8) and ( 9) to scale the captured image I sensor into the same intensity space of the original light field L when performing the reconstruction from noise-corrupted coded projections.

We provide in Table II the PSNR values obtained on four test light fields, considering both noise-free and noisy acquisitions. When sensor noise is taken into account, one can remark that the reconstruction quality decreases drastically for all the three scenarios, regardless the choice of reconstruction algorithm (see Table II for more details). In this study, we compare the TV-Dict method with the ADMM-based implementation of the dictionary-based reconstruction (Dict-ADMM) for all the three acquisition scenarios. Based on reported results in Table II, the former method obtains higher PNSR values in most cases, compared to the latter. In our opinion, it is due to the fact that the differential regularization which favors homogeneous reconstructed regions, and thus reducing the noise and eventually block artifacts presented in patch-based reconstruction (see [START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF]). This also demonstrates the effectiveness of our regularization-based method on the improvement of light field reconstruction quality in different acquisition scenarios.

While there are several published works that already report the comparison between different types of coded masks (see [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF] and [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]), none of them actually considers a realistic sensor (e.g. CFA-equipped sensors) neither a realistic noise model (which takes into account the non-stationary nature of the noise in real life). Here, we illustrate in Fig. 8 the reconstruction results obtained with various coded masks, using the Dict-ADMM algorithm for the CFA-based acquisition scenario (ii) in the presence of sensor noise as in Eqn. ( 14) with the same parameters as above (i.e. σ read = 2.5, g ADC = 0.8 and t = 1 20 ). In this experiment, we consider the following mask patterns: the RGBW pattern, the random binary pattern (following a Bernoulli's distribution of probability 0.5), the "uniform color" pattern (i.e. each color component of the mask follows a uniform law in [0, 1]) and the "Gaussian color" pattern (where each color component of the mask follows a Gaussian distribution centered at 0.5 with the standard variation 0.2 and values greater than 1 or smaller than 0 are clipped). We can easily remark that all these masks have the same light transmission rate which is approximately equal to 50%. The results depicted in Fig. 8 show that best PSNR performances are obtained using the RGBW mask and the binary mask (see Fig. 8a and8b), compared to the uniform and Gaussian masks (see Fig. 8c and8d), although these latter masks give better randomness in terms of mask value variety. Based on this observation, we may think that the level of sensor noise has a significant impact on the choice of the implemented coded mask to achieve a target quality for light field reconstruction.

Indeed, this can be an interesting subject for future studies.

2) Results on real light fields: In this section, we compare the results obtained, with different reconstruction methods for the three acquisition scenarios, on real light field data captured using a Lytro Illum camera. Unlike the experiments reported in Table II, no simulation noise has been added in this experiment, since the tested light fields already contain noise and therefore the characteristics of sensor noise cannot be properly controlled when dealing with different acquisition scenarios and exposure times. Instead of studying the impact of noise on the reconstruction quality, we focus here on a scenario-by-scenario analysis as follows:

(i) Multi-spectral (or color per color) acquisition: Table III shows that scenario (i) gives a better reconstruction quality with both the dictionary-based and TV-based approaches. However, its compression ratio is three-time-lower (i.e. it captures three times more samples) than the ones of scenarios (ii) and (iii). In addition, this scenario (considered in [START_REF] Babacan | Compressive sensing of light fields[END_REF], [START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF], [START_REF] Gupta | Compressive light field reconstructions using deep learning[END_REF], [START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]) is not appropriate for standard consumer cameras due to major modifications of the camera architecture (e.g. using prisms to separate light colors and using a monochrome sensor to capture each color component).

(ii) With a CFA mask (sampled color acquisition): Table III shows that this dual-mask design (using a coded mask and a CFA mask) combined with the proposed joint demosaicingreconstruction approach, works well and gives decent results. The above solution (the dual-mask design combined with the TV-Dict algorithm) is compatible with existing sensors with in-built CFAs and allows reconstruction from multiple shot acquisitions, which is not the case of [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF]. In fact, the neural network developed in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] only allows the reconstruction from single coded projections and can not deal with multiple-shot acquisitions without changing the network architecture and retraining the modified network. In addition, it is sensitive to the distribution of mask colors since it needs to be trained for each specific mask pattern (or color distribution) in order to achieve its best performance (see Appendix A of [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] for more details). The TV-Dict approach, in contrast, does not possess this restriction and can handle both single and multiple shots as well as various coded mask patterns (including monochrome and color masks).

(iii) Using a monochrome sensor with one or multiple shots: This scenario, referred to as scenario (iii) in Table III, with one shot, records the same number of measurements as scenario (ii) but three times less than scenario (i). We can observe that in scenario (iii), all tested algorithms, including the deep learning approach (see Fig. 9f), often fail to correctly reconstruct colors of the original light fields from one singleshot acquisition (i.e. from only one coded projection) due to the lack of color information. In fact, the reconstructed colours (from one-shot acquisitions with the monochrome sensor) are less vivid than in the original images. The color can be better recovered by increasing the number of shots (see Fig. 9e and 9f respectively), implying longer total exposure times. Two acquisitions, i.e. two shots, are required when using a monochrome sensor for a good recovery of both colour and parallax. Table III (columns (iii) ‡) shows the PSNR values obtained with the different algorithms (Dict-ADMM and TV-Dict) using two shots. The results demonstrate that the combination of a global and local reconstruction integrating a TV-based and a dictionary-based regularization (i.e. the TV-Dict algorithm) improves the results for the three acquisition scenarios. Fig. 9: Comparative results between different reconstruction approaches in the three acquisition scenarios. We compare the results obtained on the "Rock" light field with the Dict-ADMM, the proposed TV-Dict algorithm and the deep-learning-based algorithm described in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF]. One can observe that single-shot acquisitions with monochrome sensor do not allow correct recovery of color information when comparing to single-shot acquisition with CFA sensor. In our opinion, regardless of reconstruction methods, multiple-shot acquisition is necessary to correctly recover the colors when using monochrome sensor.

C. Joint light field reconstruction and spatial super-resolution

In this section, we further show that the EMMC model also covers the case where the pixel size p varies. As explained in Section V-B, in the proposed model of Eqn.( 6), H (s) ∈ R rn×n is a matrix representing the integration (summation) of incident light rays on the same sensor pixels for the s-th acquisition, where r = ( x p ) 2 is the squared ratio between the spatial sampling step and the pixel width. In the experiments reported in Table IV, the pixel size is doubled in both the horizontal and vertical dimensions so that one pixel integrates the illumination of four light rays (corresponding to r = 1 4 ). The problem in this case becomes a joint problem of reconstruction and spatial super-resolution. Table IV shows the PSNR obtained when reconstructing the light field with a spatial magnification factor of r -1 2 = 2 in both the horizontal and vertical dimensions, with the three acquisition scenarios. Note that one RGB (full color) acquisition with r = 1 4 corresponds to a sampling rate equal to 1 4ν (i.e. 1 100 in the tests reported below), while the sampling rate of a one-shot acquisition using monochrome sensor as in [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] is 1 3ν (i.e. 1 75 in the tests below). One can see in Fig. ). ns denotes the number of coded projections, e.g. ns = 3 in the full RGB acquisition scenario means one coded projection for each color channel. Light fields indicated with a "*" come from the Stanford Lytro data set [START_REF] Raj | Stanford Lytro Light Field Archive[END_REF] while the others come from the data set of [START_REF] Kalantari | Learning-based View Synthesis for Light Field Cameras[END_REF]. 10 that, despite a higher compression factor, the model and proposed method with the full color acquisition scenario allow us to obtain a better color reconstruction quality compared with the deep learning approach. The deep learning approach of [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] assuming the use of monochrome sensors instead of CFA-based sensors, as mentioned above, cannot accurately reconstruct the image color, while the full colour acquisition scenario, despite the lower number of measurements, allows us to well recover color information. When using monochrome sensor with r = 1 4 , the sampling rate of one-shot acquisitions is even lower, i.e. 1 12ν (corresponds to 1 300 for color light fields with 5 × 5 views). One can observe the PSNR values significantly improve when increasing the number of shots. In fact, for 4 low-resolution shots with monochrome sensor, the reconstruction result reach a PSNR quality that is comparable to the one of [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] (slightly inferior) with the same number of measurements, but having better color reconstruction. In our opinion, not only the number of samples but also the way they are selected can have a huge impact on the reconstruction quality.

Our algorithm is implemented and tested on Matlab R2018b under Linux Ubuntu 18.04. All the experiments are done on a Dell Latitude 7490 featuring Intel i7 CPU with 4 cores. In terms of processing time, the reconstruction of a 9×9×5×5×3 light field patch for 10000 iterations takes 7.5 seconds using the dictionary-based reconstruction algorithm (Dict-ADMM). The use of the differential-based reconstruction step requires 4.5 extra seconds in addition to 7.5 seconds for the dictionary-based reconstruction step.

VIII. CONCLUSION

We have presented a unifying camera model for compressed acquisition of Light Fields using coded masks. The proposed equivalent multi-mask camera model allows a flexible configuration of a variety of acquisition schemes. Considering the CFA pattern present in sensors as a particular mask of the proposed model led us to introduce a joint demosaicing and reconstruction method using a TV or a dictionary-based regularization. Compared with a state of the art deep learning approach, the proposed model and reconstruction method offers the possibility of dealing with multiple-shot acquisitions without changing the network architecture and retraining the network. The EMMC model in addition supports the possibility of further increasing the pixel size, thus increasing the compression rate. The reconstruction algorithm in this case jointly performs spatial super-resolution and parallax reconstruction of original light fields. Future work will be dedicated to extending the proposed model in order to take into account the diffraction effect that can occur in real highresolution mask-based hardware imaging systems. . We compare with the deep-learning-based method [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] for a standard single-shot acquisition using monochrome sensor (i.e. r = 1).

Fig. 1 :

 1 Fig. 1: Reconstruction of a color light field from a single 2D coded projection captured in various acquisition scenarios. The original light field (a) is

  The construction of an Equivalent Multi-Mask Camera (EMMC) model which unifies most existing single-lens mask-based light field cameras, and which allows flexible configuration of a variety of sensing schemes related to the number and the positions of coded masks, monochrome and color filter array (CFA)-based sensors, and the pixel size. • The application of EMMC model in multi-shot light field reconstruction, joint light field reconstruction and demosaicing using a CFA-equipped sensor and coded masks, and joint light field reconstruction and superresolution in the case of variable pixel size. • The derivation of an image formation model using the proposed EMMC model to describe various acquisition schemes.

Fig. 3 :

 3 Fig. 3: Standard mask-based light field camera model. A coded mask is placed between the sensor and the main lens. Incoming light rays are filtered by the mask before they reach the sensor.

Fig. 4 :

 4 Fig.4: Mask-based light field camera architectures from the literature. Coded aperture[START_REF] Agrawal | Reinterpretable Imager: Towards Variable Post-Capture Space, Angle and Time Resolution in Photography[END_REF],[START_REF] Babacan | Compressive sensing of light fields[END_REF]: (a) with nm = 1 and d 1 = D. Sensor side coding[START_REF] Marwah | Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections[END_REF],[START_REF] Miandji | Multi-shot single sensor light field camera using a color coded mask[END_REF]: (b) with nm = 1 and d 1 D. Designs with two masks (nm = 2): (c)[START_REF] Xu | A high-resolution lightfield camera with dual-mask design[END_REF],[START_REF] Xu | High-resolution Lightfield Photography Using Two Masks[END_REF] model using sum-of-sinusoid masks, and (d) combination of one coded mask and CFA-based sensor[START_REF] Nguyen | Color and Angular Reconstruction of Light Fields from Incomplete-Color Coded Projections[END_REF].

Fig. 5 :

 5 Fig. 5: Compressed acquisition of light field with three different scenario. The original light field is the "Dragon and bunnies" scene (5 × 5 views)

Fig. 6 :

 6 Fig. 6: Visual comparison between reconstructions of the three acquisition scenarios using the proposed TV-Dict algorithm. The top-left viewpoint and corresponding zoom-in views are extracted from the 4D light fields for comparison All results are obtained with the parameters µ = 2 -8 and ρ = 2 -7 .The three scenarios produce visually similar results when the reconstruction is performed from noise-free coded projections. In this noiseless situation, the three-shot acquisition with monochrome sensor gets the best PSNR score compared to one-shot RGB and CFA acquisitions. When there is sensor noise, the reconstruction quality drops for all scenarios. However, the one-shot CFA acquisition scheme provides very close reconstruction results for the noise-free and noisy cases.

Fig. 7 :

 7 Fig. 7: EPI visualization of original and reconstructed light fields. The reconstruction result is obtained from the CFA acquisition scenario using the proposed TV-Dict algorithm. The reconstructed EPIs look similar to the original counterparts.

Fig. 8 :

 8 Fig. 8: Reconstruction results obtained with different coded masks and a CFA-based sensor using the Dict-ADMM algorithm. From left to right: (a)the RGBW pattern (as in[START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF]), (b) the random binary pattern (following a Bernoulli's distribution of probability 0.5), (c) the "uniform color" pattern (each color component of the mask follows a uniform law in [0, 1]) and (d) the "Gaussian color" pattern (each color component of the mask follows a Gaussian distribution centered at 0.5 with the standard variation 0.2 and values greater than 1 or smaller than 0 are clipped).

  (a) Original view (b) (i) Dict-ADMM (c) (ii) † Dict-ADMM (PNSR = 34.35) (PNSR = 27.30) (d) (ii) ‡ Dict-ADMM (e) (iii) † Dict-ADMM (f) (iii) ‡ Dict-ADMM (PNSR = 33.28) (PNSR = 25.91) (PNSR = 33.23) (g) (i) TV-Dict (h) (ii) † TV-Dict (i) (ii) ‡ TV-Dict (PNSR = 34.47) (PNSR = 27.15) (PNSR = 33.23) (j) (iii) † TV-Dict (k) (iii) ‡ TV-Dict (l) (iii) Deep learning (PNSR = 25.83) (PNSR = 33.30) (PNSR = 30.11)

Fig. 10 :

 10 Fig.10: Comparative results of super-resolution between different acquisition scenarios. Super-resolution results obtained on "Orchid" using the proposed TV-Dict algorithm (µ = 2 -8 and ρ = 2 -10 ) for the three situations: (i) full RGB acquisition, (ii) acquisition with CFA-sensor and (iii) acquisition with monochrome sensor with the super-resolution factor r = 1 4 . We compare with the deep-learning-based method[START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] for a standard single-shot acquisition using monochrome sensor (i.e. r = 1).

TABLE I :

 I Notations and Symbols.

TABLE II :

 II PSNR obtained with the three acquisition schemes, (i) full color (RGB), (ii) CFA-based and (iii) monochrome acquisitions, and three reconstruction methods, with both noise-free and noisy acquisitions. The light field used in the test is a synthetic light field from the MIT dataset[START_REF] Wetzstein | Compressive Light Field Photography Siggraph 2013 project[END_REF]. The regularization parameter µ has been set to 0.01 and 0.05 in the noise-less and noisy cases respectively.

TABLE IV :

 IV Super-resolution results with the Dict-ADMM method (top part of the table) and TV-Dict (bottom part of the table) based reconstruction methods (with r =1 4

The K-SVD toolbox is available at http://www.cs.technion.ac.il/~ronrubin/ software.html

ACKNOWLEDGMENTS

We thank Dr Ofir Nabati for providing the results obtained with [21], that are used for the comparison in this paper.

Dictionary-based

Differential and Learning-based Dict-ADMM dictionary-based (TV-Dict) [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF] (i) (i) color-by-color acquisition corresponding to three shots using color filters with monochrome sensors; (ii) acquisition with CFA-built-in sensor and joint light field reconstruction and demosaicing; and (iii) acquisition without CFA (one shot with monochrome sensors). †, ‡ and † ‡ denote reconstruction results from one-shot two-shot and three-shot acquisitions respectively. The two-shot acquisitions imply two times the number of samples compared with [START_REF] Nabati | Fast and accurate reconstruction of compressed color light field[END_REF]. Light fields indicated with a "*" come from the Stanford Lytro data set [START_REF] Raj | Stanford Lytro Light Field Archive[END_REF] while the others come from the data set of [START_REF] Kalantari | Learning-based View Synthesis for Light Field Cameras[END_REF] .