Joaquin Rodriguez
email: joaquin-jorge.rodriguez@u-bourgogne.fr

Lew-Fock-Chong Lew-Yan-Voon
email: lew.lew-yan-voon@u-bourgogne.fr

Renato Martins
email: renato.martins@u-bourgogne.fr

Olivier Morel
email: olivier.morel@u-bourgogne.fr

Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization

Keywords: polarization, computer vision, robotics, scene understanding, division-of-focal-plane

Polarization information of the light can provide rich cues for computer vision and scene understanding tasks, such as the type of material, pose, and shape of the objects. With the advent of new and cheap polarimetric sensors, this imaging modality is becoming accessible to a wider public for solving problems such as pose estimation, 3D reconstruction, underwater navigation, and depth estimation. However, we observe several limitations regarding the usage of this sensorial modality, as well as a lack of standards and publicly available tools to analyze polarization images. Furthermore, although polarization camera manufacturers usually provide acquisition tools to interface with their cameras, they rarely include processing algorithms that make use of the polarization information. In this paper, we review recent advances in applications that involve polarization imaging, including a comprehensive survey of recent advances on polarization for vision and robotics perception tasks. We also introduce a complete software toolkit that provides common standards to communicate with and process information from most of the existing micro-grid polarization cameras on the market. The toolkit also implements several image processing algorithms for this modality, and it is publicly available on GitHub: https://github. com/vibot-lab/Pola4all_JEI_2023.

Introduction

The polarization of the light is present in several real-world physical phenomena. Light coming from rainbows in the sky, reflections from water on highways, and monitors and cellphones based on LCD screens are typical examples. Light polarization is naturally generated when an unpolarized light source (e.g., light bulbs or the sun) hits a surface and is reflected. The polarized light generated in this way can be of two types: specular, when the reflection of the light is in a single direction; or diffuse, when the reflection is in all directions. The way the reflected wave oscillates depends on the characteristics and the shape of the material. This relationship between the observed light and the object properties is a key feature that vision algorithms can use to improve their accuracy with respect to another one that uses only texture information. These additional features can be leveraged, for instance, to improve object detection and scene segmentation results, detect mirrors and other surfaces that polarize the light, or uniquely identify places in a room that will serve as landmarks in navigation algorithms. It is worth noting that polarization cues are the main sources of information used by many biological agents such as insects and bees for their orientation in space [START_REF] Garcia | Bioinspired color-polarization imager for real-time in situ imaging[END_REF].

The introduction of micro-grid polarization sensors, such as Sony Polarsens, boosted the research in the polarization domain since they are capable of capturing the intensity, the color and the linear polarization information in a single snapshot, and they also allow measurements outside laboratory conditions. However, the number of approaches leveraging polarization for performing computer vision and robotics tasks is, unfortunately, still quite limited. For these reasons, to promote the usage of polarization cameras in robotics and computer vision tasks, we provide in this paper a comprehensive review of the latest advances in the field of polarization imaging. The revised papers have been chosen to show the potential of this modality to improve the results given by RGB-only methods, especially in challenging 1 arXiv:2312.14697v1 [cs.CV] 22 Dec 2023 situations, e.g., in the presence of transparent and textureless objects. Furthermore, to push even more the research in this field, we break the barrier of the practical problems by introducing a complete acquisition and processing software toolkit. This toolkit comes with a graphical user interface, and is capable of processing images coming from commonly available RGB-polarization sensors, such as the Sony Polarsens, regardless the camera manufacturer. The aim of the software is to provide standard acquisition and processing tools for researchers and practitioners in the field to facilitate the visualization and analysis of polarization images. In summary, the main contributions of this paper are:

• A comprehensive review of the most recent advances in the polarization imaging field for computer vision tasks. The selected works include the most representative application cases in which the polarization information contributes outstandingly to an improvement of the results compared to those that would have been obtained with an RGB-only camera.

• We provide a complete acquisition and processing toolkit, including a graphical user interface to fill the gap of a lack of a common software for RGBpolarization cameras. The objective of this tool is twofold. On one hand, it is a toolkit that any researcher can use as a base framework, to analyze and understand the images obtained by any RGB-polarization camera. On the other hand, due to its careful design, it enables easy development and inclusion of additional applications. We include in this paper some different use cases of the developed software with the available features to highlight the potential usages.

In what follows, we start with a brief introduction to the theory of the polarization state of light. We then present the main concepts used by all the reviewed papers, allowing the reader to better understand the contribution of polarization for each approach. Finally, we describe the developed software toolkit made publicly available to the community.

Polarization background 2.1 Mathematical polarization model

Light is an electromagnetic wave of high frequency, and when it propagates through space, it can be defined by its amplitude, its frequency, and the way it moves as it travels. The intensity of the wave is equivalent to the brightness of the light, the frequency is equivalent to its color, and the way it moves transversely to the propagation direction defines its polarization state. Let us consider the projection of the oscillation of the electric field vector of the light wave on a plane perpendicular to the propagation direction. The light is said to be linearly polarized if this projection gives a line. It is said to be circularly polarized if the projection describes a circle. If the vector moves in all directions in a random manner, the light is said to be unpolarized [START_REF] Born | Principles of optics: electromagnetic theory of propagation, interference and diffraction of light[END_REF]. Furthermore, a combination of linearly and circularly polarized light gives an elliptically polarized wave, and a light that has both an unpolarized and a polarized component is a partially polarized light. This last case is the most common type of polarized light that can be found in the nature.

There exist several models to depict mathematically the polarization state, but the most commonly adopted one is the Stokes model [START_REF] Goldstein | Polarized Light[END_REF]. This model defines the light wave as a 4D vector S = [S 0 , S 1 , S 2 , S 3 ] T . S 0 represents the total light intensity (polarized and unpolarized). S 1 and S 2 describe the amount of light that is linearly polarized horizontally / vertically, and in the direction of ±45 • , respectively. S 3 represents the amount of light that is circularly polarized. Using this model, it is possible to define two important physical polarization parameters:

ρ = S 2 1 + S 2 2 + S 2 3
S 0 and ϕ = 1 2 arctan S 2 S 1 ,

(1) where ρ is called the Degree of Polarization (DoP) which represents the portion of the light that is polarized, and ϕ is the Angle of Polarization (AoP). This angle represents the orientation of the line segment, or of the ellipse, when the light is respectively linearly or elliptically polarized. Circularly polarized light is rare in nature [START_REF] Horváth | Circulary Polarized Light in Nature[END_REF], thus considering only the first three components of the Stokes vector is a good approximation to model the polarization state. Therefore, in most applications, the S 3 component is set to zero, and the Stokes model is represented by a 3D vector of these physical variables as:

S =   S 0 S 0 ρ cos (2ϕ) S 0 ρ sin (2ϕ)   . (2)
If the linear components of the Stokes vector are used, we refer to the AoP and the DoP as the angle of linear polariza-tion (AoLP), and the degree of linear polarization (DoLP), respectively.

Polarization measurement

An advantage of the Stokes model is that the effect produced by an object (either by transmission, by reflection, or by scattering) on the incident wave can be modeled by a Mueller matrix M [START_REF] Goldstein | Polarized Light[END_REF]. More specifically, a Stokes vector S in that interacts with an object whose Mueller matrix is M is converted into a Stokes vector S out according to the following equation:

S out = MS in . (3)
If the full-Stokes vector is used, the matrix M has a shape of 4×4 elements. However, if only the linear part of the Stokes vector is considered, then this matrix has 3 × 3 components. Until now, the only way to measure the Stokes vector components is by using an indirect measurement method.

For the linear components of the light, this process consists in measuring the received intensity when the light passes through a Linear Polarization Filter (LPF) at different orientations. An LPF is an optical device that allows only the waves that have the same orientation as the filter axis to pass through. Any other wave is filtered with a gain that has a sine curve shape. The maximum gain occurs when the filter orientation matches the Angle of Linear Polarization (AoLP) of the incident light, and the minimum gain will occur when these angles are separated by π/2 radians. An optical device as the one described is modeled by the following Mueller matrix [START_REF] Ding | Calibration method for division-of-focal-plane polarimeters using nonuniform light[END_REF]:

M = 1 2   q + r (q -r) C 2θ (q -r) S 2θ (q -r) C 2θ m 11 m 12 (q -r) S 2θ m 21 m 22   , (4)
where q and r are the major and minor light transmittance of the linear polarizer, respectively, θ is the orientation of the filter, and:

S 2θ = sin (2θ) , C 2θ = cos (2θ) , m 11 = (q + r) C 2 2θ + 2 √ qrS 2 2θ , m 22 = (q + r) S 2 2θ + 2 √ qrC 2 2θ , m 21 = m 12 = q + r -2 √ qr S 2θ C 2θ .
If a camera is used to take the measurements of the filtered light, then, only the first component of S out can be retrieved, which corresponds to the total intensity of the ob-served light S out 0 . Thus, only the first line of the Mueller matrix M should be considered:

S out 0 θ = 1 2 q + r (q -r) C 2θ (q -r) S 2θ S in , (5)
where S out 0 θ is the S 0 component of the output Stokes vector, when the filter axis is oriented at an angle of θ radians. If the filter is considered ideal, then q = 1 and r = 0, and Eq. ( 5) becomes:

S out 0 θ = 1 2 1 cos (2θ) sin (2θ) S in . (6)
In general, S out 0 θ = I θ -d, where I θ is the readout intensity given by a pixel, and d is the pixel offset, often called dark current. Most works ignore d since in commercial cameras, this value is negligible compared to the camera measurement [START_REF] Lane | Calibration of a polarization image sensor andinvestigation of influencing factors[END_REF]. Thus, we have in general that:

I θ = 1 2 1 cos (2θ) sin (2θ) S in . (7)
To find the vector S in , several measurements at different angles θ are required. This can be done by using a division of focal plane (DoFP) polarization sensor composed of super-pixels. A super-pixel is a matrix of 2 × 2 pixels on top of which are four linear polarization micro-filters, one on each pixel and oriented at angles of 0 • , 45 • , 90 • , and 135 • . Moreover, to capture the color information, the superpixels are organized according to the Bayer pattern where each of its constitutive pixels shares the same color filter as shown in Figure 1 (a). This method has the advantage of capturing all the required information in a single shot: color and polarization. The choice of the four polarizer orientation values for the super-pixel is demonstrated by Tyo [START_REF] Tyo | Design of optimal polarimeters: maximization of signal-to-noise ratio and minimization of systematic error[END_REF] , in which it concludes that using equidistant angles in the range [0 • , 180 • ] optimizes the SNR of the computed Stokes vector.

For a DoFP sensor, four different orientations are used

(θ 1 , θ 2 , θ 3 , θ 4 ) = (0 • , 45 • , 90 • , 135 • ), thus four different intensity measurements are obtained (I θ 1 , I θ 2 , I θ 3 , I θ 4 ) = (I 0 • , I 45 • , I 90 • , I 135 • ).
Then, four expressions of Eq. ( 7) are obtained which when stacked together gives a linear system in the form I = AS in with: The law of reflection states that the angle of reflection is equal to the angle of incidence α 1 . Regarding the refracted light, its direction of travel will change, and it will be equal to α 2 .

    I θ 1 I θ 2 I θ 3 I θ 4     = 1 2     1 cos (2θ 1 ) sin (2θ 1 ) 1 cos (2θ 2 ) sin (2θ 2 ) 1 cos (2θ 3 ) sin (2θ 3 ) 1 cos (2θ 4 ) sin (2θ 4 )     S in , (8)
where I is the intensity measurements vector, A is called the pixel matrix, and S in is the Stokes vector we want to estimate. Then, it is possible to find the Stokes vector S in by computing the pseudo-inverse of A, which results in the following analytical form:

S in =    I 0 • + I 45 • + I 90 • + I 135 • 2 I 0 • -I 90 • I 45 • -I 135 •    . ( 9
)

Naturally generated polarization

An important property of the polarization state is that it conveys information about the shape and the composition of the objects when they are made of dielectric materials. After hitting a surface, an incident wave will create two new waves [START_REF] Goldstein | Polarized Light[END_REF]: a reflected and a refracted wave. In general, when a camera observes an object, the captured light is the result of the reflection. Furthermore, for insulator materials that produce specular light, there is a single angle in which the reflected light will be 100% linearly polarized [START_REF] Goldstein | Polarized Light[END_REF]. Any other direction will produce partially polarized light. The angle at which this type of reflection occurs depends on the index of refraction, which is a value related to the material type and the wavelength of the incident light. The interaction of the light, the material, and the observed intensity is depicted in Figure 1 (b). In this image, n 1 and n 2 are the indexes of refraction of the top and bottom medium respectively, α 1 the angle of the reflected light with respect to the normal vector n to the surface, and α 2 the angle of the refracted light ray with respect to the opposite to the normal vector. Another important property regarding the reflected light is that the polarization state is related to the surface orientation. It is for this reason that, through Fresnel theory [START_REF] Hecht | Optics, Pearson education[END_REF], if the ratio λ = n 2/n 1 is known, by measuring the AoP and the DoP, it is possible to retrieve the normal to the surface at each point of the object.

In summary, the polarization state of the light can provide rich geometric cues about the shape, pose and material of objects, notably in challenging conditions to classic imaging approaches, such as in the presence of highly reflective objects' surfaces (mirrors, windows, water, ... ) or transparent/translucent objects. We let the reader refer to these references: [START_REF] Hecht | Optics, Pearson education[END_REF][START_REF] Smith | Linear depth estimation from an uncalibrated, monocular polarisation image[END_REF][START_REF] Zhu | Depth from a polarisation + rgb stereo pair[END_REF][START_REF] Cui | Polarimetric relative pose estimation[END_REF].

Deep-learning background

In this section, we introduce the basic concepts of deep learning algorithms, with the aim of facilitating the understanding of the subsequent sections of the paper.

Overview of deep-learning networks

In the fields of computer vision and robotics, data-driven algorithms are the most performant ones, more particularly, deep-learning algorithms. These methods are based on a network model developed to solve a task through a training process where the network would learn to interpret the input data. Once the network is trained, it can be used on new and unseen data to perform the intended task.

The training process is an optimization routine in which a set of input data, known as the training data, is passed through a large equation with coefficients that can be tuned and the corresponding outputs determined. Next, an error function, called the loss function is computed based on the input data, the output data, and the constraints with which the output must comply. Finally, the equation coefficients, or network weights, are updated to try to reduce the loss function value. This process of updating the weights is called the back-propagation algorithm [START_REF] Rojas | The Backpropagation Algorithm[END_REF]. Once the weights have been updated, the process is repeated for a certain number of iterations, or epochs.

The theory of deep neural networks has existed for several years, but it is only after Krizhevsky et al. [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] that these algorithms have become popular. The authors have shown that Convolutional Neural Networks (CNN) are capable of outperforming hand-crafted theoretical algorithms. A CNN is composed of several layers, and each of them includes several convolutional, optimization, and normalization operations. At the output of each layer, we obtain the different images called feature maps. The deeper the layer in the network, the higher the level of information it represents. In other words, the output of the layers close to the network input represents low-level features (edges, lines, circles), and the output of the layers close to the output of the network represents high-level features such as object labels (chair, building, car, dog, etc), depth of the objects in the scene with respect to the camera coordinate frame, a text description of what the scene represents, or the pose of an object with respect to the camera frame. In a CNN, a large number of convolutions are done to the input image, and the training process is in charge of finding the convolution kernels coefficients to convert the input image into the expected output image.

Popular developed models

In the last decade, several deep neural network algorithms have been developed. They significantly improved the performance of their predecessors in various aspects such as achieving higher accuracies, maintaining the same precision with fewer parameters, and faster training times. Examples of convolutional neural networks are the VGGNet [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF], MobileNet [START_REF] Sinha | Thin mobilenet: An enhanced mobilenet architecture[END_REF], EfficientNet [START_REF] Koonce | [END_REF], DeepLabV3 [START_REF] Yurtkulu | Semantic segmentation with extended deeplabv3 architecture[END_REF], and ResNet [START_REF] He | Deep residual learning for image recognition[END_REF]. Even though there is a vast variety of models, in general, the models can be divided into two main blocks: an encoder block and a decoder block. The encoder is composed of layers designed to extract valuable information from the input data (also called features). The features are then passed to the decoder for interpretation and determination of the output data. Many authors use a known encoder block and tailor a specialized decoder module to meet their specific needs. Additionally, when dealing with color data, the encoder blocks of the most popular networks are already included in the deep learning frameworks, and they have already been pre-trained with the largest datasets available. This way, the optimization routine will converge faster than training the same model with a tiny dataset.

Very recently, a new type of neural network architecture has been developed, that challenges the existing convolutional neural networks. It is known as Transformers, and it has been presented by Vaswani et al.[20]. This new type of network does not use the convolution operation, and it has been shown to outperform all the known networks in the field of Natural Language Processing. Furthermore, Dosovitskiy et al. [START_REF] Dosovitskiy | An image is worth 16x16 words: Transformers for image recognition at scale[END_REF]. showed that Transformers can also provide more accurate results in the Computer Vision field. The network is based on what is called the attention mechanism, and more precisely, on the self-attention mechanism. This method computes the dot product between the different entries, and the output is an indication of the importance of the data at a given position for the task. Therefore, the attention mechanism can be used to ignore the areas where no important data is present and to keep the regions where the data is valuable to accomplish the final task. The disadvantage of this type of architecture is that it requires a large amount of data to outperform CNN. If the dataset used for training is not large enough, the performance of Transformers is poor.

Polarization and multi-modal networks

In data-driven approaches, it is not common to use polarization data exclusively, either because they are very noisy due to the acquisition process, or because there are not enough constraints based purely on the polarization theory for the network to converge. Therefore, polarization data are usually mixed with other modalities in a multi-modal network. This type of networks take several sources of data (at least color and polarization), and try to mix them efficiently. The process of mixing data in this way is called data fusion in the literature, and in general, one of the following three configurations is used: early, middle, or late fusion.

In the early fusion architecture, the data is mixed before entering the encoder network for feature extraction. This fusion mechanism is performed in general through operations that contain learnable parameters as the network, and these parameters are adjusted during the training process. In late fusion, the data from each modality passes through different encoders. The weights for each modality can be shared between the encoders, but it is not a common practice since each modality extracts different types of patterns from the inputs. Once the input modalities have passed through the encoders, the high-level feature maps are fused into a single one that is then passed to the decoder module. Finally, in the middle fusion architecture, as for the late fusion model, the data from each modality is inputted to an independent encoder, but in this case, the feature maps in the encoder at the different hierarchical levels are fused. In other words, at each network layer of the encoder, the corresponding feature maps are fused. Middle fusion is the heaviest fusion architecture in terms of parameters and forward pass time. Nonetheless, it ensures a complete mixture of the data at all levels in the network, avoiding information loss while the data passes from one layer to another.

In what follows, we will review the different approaches that make use of the polarization information in different ways, by using distinct approaches such as optimization algorithms, hand-crafted features, and data-driven approaches.

Related work

In this section, we discuss recent advances in the field of polarization imaging within both model-based and data-driven strategies. We have reviewed the latest applications in the computer vision and robotics field that utilize polarimetry between the years 2016 and 2022. Older applications have not been considered because the most significant advancements began after the release of the first micro-grid division-of-focal plane (DoFP) sensor around 2014. Since then, DoFP sensors have been adopted and widely used as it allows real-time imaging. In 2018, Sony introduced a DoFP sensor, named PolarSens, which became the core device of many commercial polarization cameras. It offers much better quality data and facilitates the development of more performant algorithms and real-time applications beyond laboratory conditions.. The works have been grouped into four representative categories and complementary tasks: image enhancement, segmentation, surface depth and normal esti- [START_REF] Wen | Polarization guided specular reflection separation[END_REF]. (c) Input image, ground truth, and resulting images of the polarization dehazing method [START_REF] Zhou | Learning to dehaze with polarization[END_REF]. Images courtesy of the respective works. mation, and pose estimation. Table 1 shows a summary of the papers included in each group. Most reviewed works focus on the application fields of computer vision and robotic vision. Particularly, we focus our study on the context of scene understanding in both ground and underwater environments. Polarization has also been extensively used in the field of remote sensing, notably in combination with synthetic aperture Radar (SAR) data. Applications in remote sensing are not the focus of this survey and a good review of the recent advances of applications of polarization in this field using data-driven algorithms can be found in [START_REF] Li | Polarimetric imaging via deep learning: A review[END_REF].

Image enhancement

In real-world applications, changes in viewing conditions can strongly impact the performance of computer vision algorithms. Thus, enhancing the quality of the visual information is often a required step to keep the accuracy of the developed computer vision system. The type of image

Survey section

Application Papers

Image enhancement

White balance, Demosaicking [START_REF] Ono | Degree-oflinear-polarization-based color constancy[END_REF][START_REF] Morimatsu | Monochrome and color polarization demosaicking using edge-aware residual interpolation[END_REF][START_REF] Wen | Joint chromatic and polarimetric demosaicing via sparse coding[END_REF][START_REF] Zhang | Sparse representation-based demosaicing method for microgrid polarimeter imagery[END_REF]] Dehazing, Reflection removal, Image restoration [START_REF] Hu | Iplnet: a neural network for intensity-polarization imaging in low light[END_REF][START_REF] Zhou | Learning to dehaze with polarization[END_REF][START_REF] Liu | Semantic-guided polarization image fusion method based on a dualdiscriminator gan[END_REF][START_REF] Lei | Polarized reflection removal with perfect alignment in the wild[END_REF][START_REF] Li | Underwater image restoration via stokes decomposition[END_REF][START_REF] Hu | Polarimetric underwater image recovery via deep learning[END_REF][START_REF] Tanaka | Polarized non-line-of-sight imaging[END_REF][START_REF] Amer | Enhancing underwater optical imaging by using a low-pass polarization filter[END_REF][START_REF] Shen | Underwater image enhancement based on polarization imaging[END_REF] Specular-diffuse separation [START_REF] Wen | Polarization guided specular reflection separation[END_REF] Image segmentation Material-based segmentation [START_REF] Liang | Multimodal material segmentation[END_REF] Glass segmentation [START_REF] Mei | Glass segmentation using intensity and spectral polarization cues[END_REF][START_REF] Kalra | Deep polarization cues for transparent object segmentation[END_REF] Urban scenes segmentation [START_REF] Li | Illumination-invariant road detection and tracking using lwir polarization characteristics[END_REF][START_REF] Li | Polarization-guided road detection network for lwir division-of-focalplane camera[END_REF][START_REF] Xiang | Polarization-driven semantic segmentation via efficient attention-bridged fusion[END_REF] Surface normal and depth estimation

Normal map estimation [START_REF] Ichikawa | Shape from sky: Polarimetric normal recovery under the sky[END_REF][START_REF] Lei | Shape from polarization for complex scenes in the wild[END_REF][START_REF] Fukao | Polarimetric normal stereo[END_REF][START_REF] Ba | Deep shape from polarization[END_REF][START_REF] Deschaintre | Deep polarization imaging for 3d shape and svbrdf acquisition[END_REF] Depth estimation [START_REF] Smith | Linear depth estimation from an uncalibrated, monocular polarisation image[END_REF][START_REF] Zhu | Depth from a polarisation + rgb stereo pair[END_REF][START_REF] Berger | Depth from stereo polarization in specular scenes for urban robotics[END_REF], 49] 3D reconstruction / scene rendering [START_REF] Shakeri | Polarimetric monocular dense mapping using relative deep depth prior[END_REF][START_REF] Zhao | Polarimetric multi-view inverse rendering[END_REF][START_REF] Kondo | Accurate polarimetric brdf for real polarization scene rendering[END_REF] Pose estimation

Object's pose [START_REF] Cui | Polarimetric relative pose estimation[END_REF][START_REF] Gao | Polarimetric pose prediction[END_REF][START_REF] Zou | Human pose and shape estimation from single polarization images[END_REF] Optical flow [START_REF] Tzabari | Polarized opticalflow gyroscope[END_REF] Underwater solar-tracking [START_REF] Hu | Solar-tracking methodology based on refraction-polarization in snell's window for underwater navigation[END_REF] Table 1: Summary of the reviewed works (2016 -2022) of this polarization survey.

quality improvement depends on the application itself. In some cases, this implies having high-quality measurements independently of the camera used, which can be achieved through camera calibration [START_REF] Lane | Calibration of a polarization image sensor andinvestigation of influencing factors[END_REF][START_REF] Rodriguez | A practical calibration method for rgb micro-grid polarimetric cameras[END_REF][START_REF] Ding | Calibration method for division-of-focal-plane polarimeters using nonuniform light[END_REF][START_REF] Powell | Calibration methods for division-of-focal-plane polarimeters[END_REF]. In others, the improvement can be related to removing highly bright, specular reflections, requiring a separation between this type of reflection from the diffuse ones [START_REF] Ono | Degree-oflinear-polarization-based color constancy[END_REF][START_REF] Kajiyama | Separating partially-polarized diffuse and specular reflection components under unpolarized light sources[END_REF][START_REF] Umeyama | Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images[END_REF]. In more complex cases, the background structure needs to be recovered when an atmospheric phenomenon is present such as mist or fog, as shown for some examples in Figure 2. In most cases, the physical constraints defined by the polarization state of the light can be used to improve the results obtained by conventional cameras.

In this context, Ono et al. [START_REF] Ono | Degree-oflinear-polarization-based color constancy[END_REF] present a white balance algorithm for RGB-polarization sensors based on the achromaticity of the Stokes vector in the visible spectrum. Rodriguez et al. [START_REF] Rodriguez | A practical calibration method for rgb micro-grid polarimetric cameras[END_REF] relax the experiment setup to calibrate micro-grid color-polarization sensors to achieve a flat-field response in all the polarization parameter images. Wen et al. [START_REF] Wen | Polarization guided specular reflection separation[END_REF] solve the separation of specular from diffuse reflections with a model-based optimization strategy [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF]. Their pipeline is independent of the illumination source by exploiting polarization and chromaticity images. Wen et al. [START_REF] Wen | Joint chromatic and polarimetric demosaicing via sparse coding[END_REF] propose to jointly demosaic RGB and polarization information to obtain high-quality, 12-channeled RGBpolarization images by using a sparse representation model. The model is obtained through an optimization algorithm that uses the ADMM scheme. Similarly, Morimatsu et al. [START_REF] Morimatsu | Monochrome and color polarization demosaicking using edge-aware residual interpolation[END_REF] obtain high-quality, polarization images by extrapolating the residual interpolation for RGB images [START_REF] Kiku | Beyond color difference: Residual interpolation for color image demosaicking[END_REF] to the monochrome and color polarization sensors. They achieve their results by changing the guidance image so that it is edge-aware, and by making use of the raw polarization intensity measurements. Tanaka et al. [START_REF] Tanaka | Polarized non-line-of-sight imaging[END_REF] achieve better quality images by improving the condition number of the transport matrix in comparison with conventional, passive Non-Line-Of-Sight (NLOS) system. This is done by using the polarization leakage effect model produced by the oblique reflection over a filter oriented at the Brewster angle [START_REF] Hecht | Optics, Pearson education[END_REF] of a wall.

Using hand-crafted theories provides good results when the scene and the effect to analyze are not complex, since a high-precision mathematical model of the problem can be established. When this is not possible, data-driven algorithms can be used, as they have the capability to learn complex theories during training. For example, they can handle scenes with several objects at the same time, or model effects for which no known mathematical model exists. Datadriven approaches as described in Lei et al. [START_REF] Lei | Polarized reflection removal with perfect alignment in the wild[END_REF] have been designed to remove the reflections produced by different types of glasses by using polarization theory. The input to their network architecture, composed of pre-trained U-net and VGG-19 networks, is an image which is a combination of the raw measurements, split by polarization channel and polarization parameters (I, ρ, ϕ), as presented in Eq. ( 1). Zhou et al. [START_REF] Zhou | Learning to dehaze with polarization[END_REF] use a single polarization image and a deep learning network composed of two U-net models and two autoencoders to dehaze urban scenes. Hu et al. [START_REF] Hu | Iplnet: a neural network for intensity-polarization imaging in low light[END_REF] developed a data-driven approach and a dataset to increase the brightness and quality of images under low-illumination conditions. They created a convolutional neural network that works in two steps based on the raw measurements of the camera: firstly, an enhancement in the intensity domain for all the color channels, and then each color channel is treated by a separate network. Liu et al. [START_REF] Liu | Semantic-guided polarization image fusion method based on a dualdiscriminator gan[END_REF] proposed a Generative Adversarial Network (GAN) architecture to fuse the DoLP and the intensity images into a single intensity image. By dividing the image into background and foreground, the network fuses these two polarization images into a single image that has an increased and better contrast than the original intensity image. The results produced by this network can be used to train other networks, i.e., to perform an improved data augmentation, in order to obtain models with increased generalization capabilities than the ones obtained when trained only with the original intensity images. Despite the outstanding results of data-driven algorithms with respect to the optimization-based approaches, the quality of these results depends on the data and the type of model used. Particularly for the data, if not all the cases have been considered in the images provided during training, the missing cases might produce in less accurate results during testing.

Several relevant image enhancing approaches have also been proposed to deal with the challenging scenario of underwater imaging. Li et al. [START_REF] Li | Underwater image restoration via stokes decomposition[END_REF] aim to improve the contrast of underwater images due to turbidity by using polarization and an optimization strategy. They propose to split the Stokes vector into three contributions (diffuse, specular, and scattered light), since they claim that the scattering reflection underwater cannot be neglected. In the same direction, Hu et al. [START_REF] Hu | Polarimetric underwater image recovery via deep learning[END_REF] present a novel CNN based on residual blocks that fuse the polarization features to restore the contrast of underwater images. By using the raw measurements of three polarization channels of a monochrome polarization camera, they are able to see through turbid water, and obtain a clear image of the hidden objects. Amer et al. [START_REF] Amer | Enhancing underwater optical imaging by using a low-pass polarization filter[END_REF] propose a static pipeline to increase the image quality for underwater applications. Based on the active cross-polarization technique, and an optimized version of the Dark Channel Prior, they achieve contrast improvement for underwater imaging, with a single snapshot in realtime. Shen and Zhao [START_REF] Shen | Underwater image enhancement based on polarization imaging[END_REF] developed an iterative pipeline to jointly improve the image contrast and denoise the image. With two polarization images taken with a rotative filter at 0 • and 90 • , they compute the transmittance and the irradiance maps in underwater conditions for each color channel. Then, they establish an iterative process to refine these results by using an adaptative bilateral filter, and an adaptative color correction routine.

Despite the remarkable improvement brought by polarization to image enhancement, as compared to similar applications for RGB-only cameras, several challenges still remain. For example, Ono et al. [START_REF] Ono | Degree-oflinear-polarization-based color constancy[END_REF] outperform different baselines in many scenes, but it is left as future work to improve the results obtained when the sky occupies a large portion of the image. Similarly, Zhou et al. [START_REF] Zhou | Learning to dehaze with polarization[END_REF] retrieve the hidden structure of objects behind the haze with good accuracy in real-world situations, after training the network with computer generated images. Despite this, the authors claim that the model does not produce adequate reconstructions for fog and mist due to the physical phenomena produced by these perturbations that are not the same as for the haze. It is important to note that one of the barriers in the polarization image enhancement field is the lack of standard benchmarks. Indeed, most works had to create a dataset to demonstrate their contributions. Some of the created datasets, which often required a huge amount of work, can be reused as in Lei et al. [START_REF] Lei | Polarized reflection removal with perfect alignment in the wild[END_REF] , where the authors realized acquisitions in a large variety of environments, and with different types of glasses. On the other hand, other applications have been demonstrated using small, non-available datasets, that may not necessarily cover all the required cases [START_REF] Wen | Joint chromatic and polarimetric demosaicing via sparse coding[END_REF], or they have created a polarization dataset based on RGB ones and a mathematical model of the polarization effect [START_REF] Zhou | Learning to dehaze with polarization[END_REF].

Image segmentation

The polarization state of the light is directly linked to the object's material and shape as introduced in Section 2.3. This property can provide insightful and complementary information to guide object segmentation approaches in scenarios where only the surface color is not discriminant. Indeed, the index of refraction depends on the internal structure of the objects, and on the wavelength of the incident light as defined by the Fresnel equations [START_REF] Born | Principles of optics: electromagnetic theory of propagation, interference and diffraction of light[END_REF]. It is for this reason that material classification is one of the most fruitful application of polarization theory. Previously, this task was accomplished using hand-crafted features, in controlled acquisition conditions, using rotative filters, and considering a single object to be analyzed at a time [START_REF] Wolff | Polarization-based material classification from specular reflection[END_REF][START_REF] Chen | Polarization phase-based method for material classification and object recognition in computer vision[END_REF][START_REF] Tominaga | Polarization imaging for material classification[END_REF][START_REF] Thilak | Polarization-based index of refraction and reflection angle estimation for remote sensing applications[END_REF]. Nowadays, with the advances in sensors and the advent of data-driven algorithms, object characterization with polarization cues has been ported to more complex, constraintrelaxed scenarios.

In the domain of infra-red imaging, Li et al. [START_REF] Li | Illumination-invariant road detection and tracking using lwir polarization characteristics[END_REF] succeed in efficiently detecting the road area in urban scenes by using the zero-distribution prior in the AoLP and the difference in the DoLP of the objects to increase the accuracy of the segmentation. This information is further used in a Input image with transparent objects, RGB-only method object segmentation results, and results obtained with the polarization data-driven method [START_REF] Kalra | Deep polarization cues for transparent object segmentation[END_REF]. (c) Input image, RGB-only method results, and the results of the Multimodal Material Segmentation algorithm [START_REF] Liang | Multimodal material segmentation[END_REF]. Images courtesy of the respective works.

visual tracking algorithm to continuously track the road online. In an extension of their previous work, Li et al. [START_REF] Li | Polarization-guided road detection network for lwir division-of-focalplane camera[END_REF] use the zero-distribution prior of the AoLP to create a coarse map of the road. Then, they developed a deep-learning network to refine the coarse road map. Their network consists of two branches that analyze different aspects of the scene. The main branch receives the information captured by an infra-red camera, already converted into a fake color image, i.e., a 3-channel image result of stacking the AoLP, the DoLP, and the total intensity together, and then converted from the HSV to the RGB color space. The objective of this branch is to extract multi-modal features of the scene. The other branch or polarization-guided branch also receives the AoLP and the DoLP of the scene, but it does not receive the intensity image. Instead, the coarse map obtained from the zero-distribution prior of the AoLP is provided. By doing so, the authors aim to guide the network based on the polarization properties of the road, and not of the entire scene. In the visible spectrum, Xiang et al. [START_REF] Xiang | Polarization-driven semantic segmentation via efficient attention-bridged fusion[END_REF] developed a fusion network to combine color and polarization data to better segment objects of urban scenes. They tested several combinations of polarization information with attention mechanisms and concluded that using only color and the AoLP is the best combination to improve the results. Kalra et al. [START_REF] Kalra | Deep polarization cues for transparent object segmentation[END_REF] improve the instance-semantic segmentation network Mask R-CNN [START_REF] Khaing | Transparent object detection using convolutional neural network[END_REF] to handle transparent objects (as shown in Figure 3) by adding monochrome polarization cues to the original mid-fusion pipeline. Each polarization parameter image (intensity, AoLP, and DoLP) is fed into a different backbone encoder, and the fusion of the feature maps is performed at each encoder level. Mei et al. [START_REF] Mei | Glass segmentation using intensity and spectral polarization cues[END_REF] extend the work presented in [START_REF] Kalra | Deep polarization cues for transparent object segmentation[END_REF] by using RGB polarization cues, instead of monochrome, with the aim of segmenting glasses in urban scenes. Since they use an RGB-Polarization camera, they can measure RGB intensity, RGB DoLP, and RGB AoLP. Each of the two RGB polarization images (DoLP and AoLP) is balanced by using an attention mechanism. Then, these two results, and the intensity RGB image are fed into three independent Conformer encoders [START_REF] Peng | Conformer: Local features coupling global representations for visual recognition[END_REF] and fused using local and global guidance. In a more complex scenario, Liang et al. [START_REF] Liang | Multimodal material segmentation[END_REF] build a network to fuse RGB, infra-red, and polarization cues to produce an outdoor scene segmentation based on the object material type. The proposed pipeline is composed of two core elements: a network that will classify the objects present in one class of a subset of the segmentation labels from the CityScapes dataset [START_REF] Cordts | The cityscapes dataset for semantic urban scene understanding[END_REF]; and a region-based filter selection module that chooses the modality that provides the most relevant information for determining the type of material of the constitutive elements of each detected object. The full network is composed of four encoders, one for the RGB intensity, one for the AoLP image, one for the DoLP image, and one for the infra-red image.

All these works outperform RGB systems when the polarization information is added to each developed pipeline. A higher gain in performance is also often obtained when the network is adapted to correctly process the AoLP and the DoLP, and not when an RGB network is trained with polarization images. This is why most of these works propose carefully designed fusion schemes. However, the lack of datasets including polarization information in the field of image segmentation poses limitations on the development of polarization-based approaches. It is important to highlight that all the previously discussed works have presented their own dataset to show that polarization is a path to consider in image segmentation. For instance, Kalra et al.

[39] used a private dataset acquired in a very specific environment, focused on the particular application with a pickand-place robotic arm. Xiang et al. [START_REF] Xiang | Polarization-driven semantic segmentation via efficient attention-bridged fusion[END_REF] provide a smallscale dataset of RGB-polarization images captured in various urban scenes. Although informative, the dataset contains only 394 annotated images segmented into 9 different classes. Mei et al. [START_REF] Mei | Glass segmentation using intensity and spectral polarization cues[END_REF] introduce a medium-scale dataset, with 4511 images annotated only for the labels glass and no-glass. Similarly, Liang et al. [START_REF] Liang | Multimodal material segmentation[END_REF] made publicly available a dataset of semantic segmentation of urban scenes, with multi-modal sensors, but it only includes 500 labeled images, and Li et al. [START_REF] Li | Illumination-invariant road detection and tracking using lwir polarization characteristics[END_REF] did it for road segmentation, and with their personalized infra-red polarization camera. Thus, there is a need for a common large-scale benchmark to evaluate the performance of these different segmentation algorithms to trace the direction toward a generalization of the polarization modality.

Surface normal and depth estimation

Polarization is well known to encode shape information of the different objects being observed. Most existing approaches consider an orthographic projection of the incoming light, and if the refractive index of the object is known, then the normal vectors to the surface can be estimated, and through integration, the depth map can be retrieved. These approaches are again based on the Fresnel's theory, which has been briefly introduced in Section 2.3, which establishes the relationship between the degree of linear polarization with the zenith angle of the normal, and the angle of linear polarization with the azimuth angle. Even though effective in several works and scenarios [START_REF] Miyazaki | Polarization-based inverse rendering from a single view[END_REF][START_REF] Morel | Polarization imaging applied to 3D reconstruction of specular metallic surfaces[END_REF][START_REF] Atkinson | Surface reconstruction using polarization and photometric stereo[END_REF][START_REF] Zhang | A comprehensive polarisation model for surface orientation recovery[END_REF], the fact that both the object's refraction index and the light direction must be known makes this approach limited to laboratory strict conditions. Additionally, the relationship between polarization and the normal vector has geometric ambiguities. Therefore, one important research direction is to reduce the constraints and priors required for the acquisition while maintaining low reconstruction errors.

Ba et al. [START_REF] Ba | Deep shape from polarization[END_REF] propose a learning-based approach to estimate the normal map of objects as shown in Figure 4. The ambiguous normal maps from Fresnel's theory are used as priors given directly to a deep neural network as inputs. Fukao et al. [START_REF] Fukao | Polarimetric normal stereo[END_REF] present a shape from polarization algorithm that uses a stereo pair of polarization cameras. The coarse map from the stereo vision is refined by filtering the normal maps with a belief propagation scheme. They exploit Fresnel's theory, and an improved modeling of the RGB image, corresponding rendered image, and the corresponding rendered degree of linear polarization. This result corresponds to the scene rendering technique implemented in [START_REF] Kondo | Accurate polarimetric brdf for real polarization scene rendering[END_REF]. (c) Input polarization images, ground truth, and estimated normal maps. These results are courtesy of [START_REF] Ba | Deep shape from polarization[END_REF]. micro-facet reflection effect by considering that it is a linear combination of the diffuse and the specular lobe reflection. Similarly, Ichikawa et al. [START_REF] Ichikawa | Shape from sky: Polarimetric normal recovery under the sky[END_REF] relaxed the constraints for shape from polarization by using the Rayleigh and the Perez models to estimate the sun polarization state and direction on a clear day. Then, through mathematical optimization, the normal and shading maps are obtained. The additional cues about the incident Stokes vector serve to determine how the object modulates the incident light (Mueller calculus), and jointly with the shading constraints, the normal map can be estimated. Deschaintre et al. [START_REF] Deschaintre | Deep polarization imaging for 3d shape and svbrdf acquisition[END_REF] propose a 3D object shape estimation, jointly with a spatially variable BRDF model estimation, by using a single-view polarization image fed to a U-Net based network architecture. The full input of the model is the intensity image, the normalized Stokes map, and the normalized diffuse color which encodes the object reflectance information. Lei et al. [START_REF] Lei | Shape from polarization for complex scenes in the wild[END_REF] propose a deep-learning network to estimate the normal map of complex scenes. Their aim is to improve the accuracy limits by incorporating viewing encoding as input to the network, which accounts for the non-orthographic projection. This input is an image where each pixel represents the direction of the incident light. When estimating the normal vectors from polarization under the orthographic assumption, all the incident light rays are supposed to be colinear to the Z axis of the camera coordinate frame. Thus, all the zenith angles are measured with respect to a common coordinate frame. When using a perspective lens, the zenith angle given by the polarization theory is measured with respect to the direction of propagation of the light, which in this case, will be different for each pixel. By providing the viewing encoding, the authors claim the network will understand the viewing direction of the polarization state and use this information to improve the results of a network that works under the orthographic assumption with a perspective lens. The other inputs to the model are the raw measurements of the camera separated by polarization channel, the AoLP, the DoLP, and the total intensity. Their network is also grounded by an architecture similar to a U-Net model, with a multi-head self-attention module in the bottleneck. Smith et al. [START_REF] Smith | Linear depth estimation from an uncalibrated, monocular polarisation image[END_REF] define the shape from polarization problem as a large linear system of equations. They combine the physics theory of polarization with the geometry of the problem to formulate the depth equations directly, without passing through the computation of the normals. Berger et al. [START_REF] Berger | Depth from stereo polarization in specular scenes for urban robotics[END_REF] present a depth estimation algorithm that uses the polarization cues in a stereo vision system. They improve the correspondence matching by adding the AoLP-normal constraint to the intensity similarity function. In the same direction, Zhu and Smith [START_REF] Zhu | Depth from a polarisation + rgb stereo pair[END_REF] propose a hybrid RGB-polarization acquisition system to obtain a dense depth reconstruction. By classifying the pixels into specular or diffuse, they make use of normal vectors obtained from Fresnel's theory to improve the estimation of the normal maps obtained from the stereo images. Blanchon et al. [START_REF] Blanchon | P2d: a self-supervised method for depth estimation from polarimetry[END_REF] extend the monocular depth estimation network Monodepthv2 [START_REF] Godard | Digging into self-supervised monocular depth estimation[END_REF] to consider polarization information by adding the azimuthal constraint to the deeplearning loss. Zhao et al. [START_REF] Zhao | Polarimetric multi-view inverse rendering[END_REF] extend the multi-view reconstruction system [START_REF] Kim | Multi-view inverse rendering under arbitrary illumination and albedo[END_REF] by adding polarization cues to the optimization. They introduce a continuous function that has four minimum values, each of them at one of the ambiguous normal azimuth possibilities of Fresnel's theory. Kondo et al. [START_REF] Kondo | Accurate polarimetric brdf for real polarization scene rendering[END_REF] developed a polarimetric BRDF model that does not constrain the illumination nor the camera position during acquisition. This model is used to synthesize polarization images out of the RGB images, easing the dataset cre-ation for data-driven algorithms. By acquiring images with different illumination of known Stokes vectors, they use Mueller calculus to model the object reflectance. Shakeri et al. [START_REF] Shakeri | Polarimetric monocular dense mapping using relative deep depth prior[END_REF] produce a dense 3D reconstruction by using polarization cues. This is done by optimizing an initial depth map obtained from MiDaS [START_REF] Ranftl | Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer[END_REF], and the coarse depth map from COLMAP [START_REF] Schönberger | Structure-frommotion revisited[END_REF]. The optimization routine constraints the normals with the ones from Fresnel's theory. The initial depth map is used to disambiguate the polarization normals, and the coarse map is used to regularize the optimization routine, since they are metrically correct but sparse.

The previously presented works have all been developed for shape/depth estimation while leveraging the polarization information. Combining the polarization state of the light with any geometry problem developed for the RGB space results in a significant improvement in accuracy and image quality. This is because the polarization measurements are dense since they are provided pixel-wise, which means that the normal constraints are also dense. Thus, passive, high-quality far-field 3D reconstructions can be retrieved using a multi-modal RGB-polarization camera, which cannot be done with active sensors such as LiDAR or Microsoft Kinect. However, these polarization constraints are still often dependent on knowing priors about the material (metallic vs insulator) and the reflection type (specular vs diffuse), thus sometimes they can produce low-quality results in the wild. To overcome this problem, some works decide to use only diffuse reflections [START_REF] Deschaintre | Deep polarization imaging for 3d shape and svbrdf acquisition[END_REF], or to classify the pixels into either diffuse dominant or specular dominant (such as in [START_REF] Smith | Linear depth estimation from an uncalibrated, monocular polarisation image[END_REF][START_REF] Shakeri | Polarimetric monocular dense mapping using relative deep depth prior[END_REF]). To deal with more complex cases, a better modeling of the reflection effect might be required [START_REF] Fukao | Polarimetric normal stereo[END_REF]. For data-driven algorithms, this field also suffers from the lack of large-scale datasets that can be used as benchmarks for research. Most papers propose their own dataset by doing acquisitions [START_REF] Lei | Shape from polarization for complex scenes in the wild[END_REF], or they model the polarization state assuming artificial conditions to simulate polarization over already existing RGB images [START_REF] Kondo | Accurate polarimetric brdf for real polarization scene rendering[END_REF].

In summary, the polarization clearly provides valuable cues in the field of shape estimation and 3D reconstruction but two main problems need to be addressed. Foremost, the lack of large-scale, standard evaluation benchmarks, that hinders the development of techniques using this modality in the current era of data-driven algorithms. On the other hand, there is no generic model that can effectively handle generic types of reflections over any type of material. Therefore the challenge of interpreting the measured data and determining which model to use remains open.

Pose estimation

The polarization of light can also play a significant role in object pose estimation since it provides valuable geometric constraints in the determination of the vectors normal to the observed surfaces (as discussed in Section 4.3). Hence, this additional information can be used to overcome the illposedness condition of many RGB problems, such as estimating the relative rotation and translation of textureless objects between two images. Particularly, these additional constraints are useful when the objects to analyze are highly reflective and translucent since the polarization measurements are independent of the intensity of the light. Cui et al. [START_REF] Cui | Polarimetric relative pose estimation[END_REF] use the normal vectors estimated from the Fresnel equations to add geometric constraints for pose estimation (some visualizations of the pose estimation can be seen in Figure 5). With this additional information, only two corresponding points in two views are required to estimate the rotation matrix and the translation vector. In the same direction, Gao et al. [START_REF] Gao | Polarimetric pose prediction[END_REF] propose a data-driven algorithm to find the pose transformation of an object in the image with respect to the camera coordinate frame. The algorithm uses three ambiguous normals as inputs to one encoder, and the polarization parameters into another. Then, the features are fused at different levels, and given to the decoder. Tzabari and Schechner [START_REF] Tzabari | Polarized opticalflow gyroscope[END_REF] present a static approach in which they use the AoLP and the DoLP to expand the optical flow theory. This new component accounts for the rotation speed estimation, which cannot be done with the classical optical-flow approach. Hu et al. [START_REF] Hu | Solar-tracking methodology based on refraction-polarization in snell's window for underwater navigation[END_REF] utilize a monochrome polarization camera to build a complete pipeline to estimate the sun's position based on the DoLP and the AoLP measured underwater. Jointly using the Snell and Fresnel theories, they revert the ray bending caused by the change in medium and handle the problem as if the measurements were done outside the water. Zou et al. [START_REF] Zou | Human pose and shape estimation from single polarization images[END_REF] push forward the accuracy in the human shape and pose estimation by building a two steps network with polarization cues. Assuming the human cloth to be diffuse dominant, they retrieve the human features by using the raw polarization intensity images, and the ambiguous normal maps obtained from Fresnel theory. The first network produces a high-quality normal map, and the second one uses this result, jointly with the output of the SMPL human shape model [START_REF] Loper | Smpl: A skinned multi-person linear model[END_REF] to estimate the final shape and pose of a clothed person.

Due to the geometric nature of the pose estimation problem, the polarization state of the light provides valuable clues that can be used in any computer vision algorithm in this field. The applications included in this section of the review demonstrate the potential of accuracy gain obtained with the polarization constraint, while at the same time relaxing the other algorithms hypothesis. In [START_REF] Cui | Polarimetric relative pose estimation[END_REF], it is shown that only two points are required to estimate the pose transformation between two views, resulting in an improvement in speed and accuracy when added to any structure from motion algorithm. Without any requirement in the type of clothes to be used, [START_REF] Zou | Human pose and shape estimation from single polarization images[END_REF] were able to estimate the human pose with a lower error and fewer constraints than competitors. In underwater applications, Global Positioning System (GPS) signals cannot be used because their intensities decrease rapidly with the depth in the water. To address this issue, Hu et al. [START_REF] Hu | Solar-tracking methodology based on refraction-polarization in snell's window for underwater navigation[END_REF] propose an autonomous underwater navigation system that uses polarization instead of a GPS signal. In their system, the camera's global position is estimated by applying geometrical constraints that link the sun's position to its known trajectory [START_REF] Du | An autonomous initial alignment and observability analysis for sins with bio-inspired polarized skylight sensors[END_REF][START_REF] Li | Bio-inspired attitude measurement method using a polarization skylight and a gravitational field[END_REF]. However, several limitations still remain when doing pose estimation with polarization. For example, in [START_REF] Gao | Polarimetric pose prediction[END_REF], only the position of one object can be done each time, while others adopt known object materials and physical properties. Furthermore, most algorithms only consider one type of reflection (either diffuse or specular), which limits their generalization to any type of scene.

Pola4all: Polarization Toolkit

From the presented study on the advances in polarization imaging, we have observed that in all the reviewed applications there are no common standards either for data structure, acquisition, or visualization tools. We can also notice a lack of a common library to boost the development, evaluation, and deployment of algorithms using polarization. For these reasons, this section presents a software library that allows to visualize and analyze polarization images, as well as to develop and integrate any custom application in a structured manner. In what follows, we briefly describe the software, including its basic components and the implemented image-processing algorithms. The objective is to contribute to the community by providing several tools integrated with a single, common graphical user interface (GUI) software that can interact with any DoFP RGB-polarization camera available on the market. Additionally, this software is meant to provide access to all the developed algorithms that showcase the power of polarization information, making it easier for anyone interested in working with polarization modality to get started. The library has been developed in C++ Object Oriented Programming language and is built on top of three main libraries/frameworks: OpenCV, ROS and QT5. An overview of the final graphical user interface can be found in the GitHub repository of this project.

Throughout this section, we will show the output images of our program. To be able to compare the polarization properties through the proposed software library toolkit, the same test image of an urban scene is used in all the experiments. This image, corresponding to the raw image obtained from the camera, is shown in Figure 6. The library is composed of several modules designed to be independent so as to enable easy maintenance and debugging. With such a structure the addition or removal of a functionality is straightforward. Regarding the architecture of the code, it is composed of two general components: the camera server, and the GUI client. They are detailed in the documentation of the library repository.

Core components

The first component is a ROS camera server package that interacts directly with the camera: getting images, and changing (or querying) its parameters such as the pixel gain, exposure time, and frame rate. It has been designed to support the integration of existing modern micro-grid polariza-Figure 6: Raw image of an urban scene used as a reference for all the algorithms implemented in the toolkit Pola4All. tion cameras. The second component of the software is the graphical user interface, which works as a client of the ROS server. The final user interface, with a detailed explanation, is included in the GitHub repository [START_REF] Rodriguez | Pola4all: A survey of polarimetric applications and an open-source software to analyze polarimetric images -repository[END_REF]. This interface allows the user to perform all the required tasks involving the camera such as changing the image parameters, and the super-pixels filter configuration. It also allows image processing, raw image display, and sensor calibration. To analyze the calibration performance, plotting functions are provided.

Basic processing

The basic polarimetric processing techniques included in the toolkit are detailed below.

Raw split images : It produces as output four images, one per filter orientation independently of the color. An example of the image for filter orientation of 0 • and red channel is shown in Figure 7 (a).

Polarized color images : It produces as output four images which are the demosaiced version of the output from the Raw split images mode obtained using the bilinear interpolation algorithm. Since the raw measurements of the camera are used, they are not white-balanced. Thus, the resulting images exhibit a greenish aspect, as shown in Figure 7 (b) for the 0 • polarization-red color channel. To this image the white-balance algorithm described in Section 5.2.1 can be applied to obtain the resulting whitebalanced image shown in Figure 7 (c). Polarized color images allow to identify regions in the scene where light is polarized. To illustrate this, the demosaiced, white-balanced image for the channel 135 • is shown in Figure 7 (d). Note that the windshields of the cars appear dark in this image compared to the one corresponding to filter orientation of 0 • . This indicates that the light reflected by the windshields is polarized because polarized light is filtered by polarimetric filters according to their orientations. As explained in Section 2, the intensity of linearly polarized light that passes through a rotating linear polarizer will exhibit a sine wave curve with respect to the rotation angle. This sine wave is maximum when the orientation of the filter is equal to the angle of the linearly polarized light and it decreases as the filter rotates. The intensity reaches a minimum when the orientation of the filter is shifted by π/2 radians with respect to the angle of linear polarization of the light. In the case of the RGB-polarization camera, four filter orientations are considered. If any of these orientations matches the AoLP of the incoming polarized light, then the corresponding image will have a bright spot, and the one that is shifted by π/2 radians will have a dark spot at the same position. Since the windshields of the cars in Figure 7 (d), for a filter orientation of 135 • , is dark where as in Figure 7 (c), for a filter orientation of 0 • , it is rather bright, this means that the 135 • filter has filtered out a great part, if not all, of the polarized reflections. Thus, we can deduce that the reflected light has an AoLP close to 45 • , and that it is highly linearly polarized.

Original color : Only one image is returned. This image is equivalent to the one obtained with a conventional RGB camera. It corresponds to the first component of the Stokes vector S 0 . It is computed by applying Eq. ( 9) to the output of the Raw split images mode. The obtained image is then demosaiced to obtain the three-channel color image.

Stokes images : After separating the input image by polarization channel, the three Stokes components images can be obtained by applying Eq. ( 9). Since the polarization state of the light depends on the frequency of the light, the Stokes vectors are split by color channel. As a consequence, this function will provide 3 × 4 = 12 images. Four color chan-nels are considered since the Bayer patterns consist of 2 × 2 patterns of red, green, green, and blue color filters. The three linear components, S 0 , S 1 and S 2 , of the Stokes vector for the red channel are shown in Figure 8 (a) to (c). The images for the other channels are included in the GitHub repository [START_REF] Rodriguez | Pola4all: A survey of polarimetric applications and an open-source software to analyze polarimetric images -repository[END_REF]. The S 0 image is the red channel image of the original color image, while the S 1 and S 2 images are functions of the orientation of the polarizer filters. Since S 1 and S 2 can have negative values, it is their absolute values that are considered. In the same way as polarized color images, Stokes component images can be used to detect regions with polarized light. Indeed, polarized light will be characterized by a bright or high Stokes value region in one of the S 1 and S 2 images and a corresponding dark or low Stokes value region in the other image. Conversely, regions that appear dark in both images correspond to areas where the reflected light is weakly polarized. In Figure 8 (a) and (c), we can deduce that the windshields exhibit highly polarized reflections. This is consistent with what we have already deduced from the polarization color images.

Raw I -Rho -Phi : As also explained in Section 2, the Stokes vector can be represented as a function of three physical parameters: the total intensity I, the degree of linear polarization ρ, and the angle of linear polarization ϕ. The equations to compute these parameters as a function of the Stokes vector are given in Eq. (1). Again, these three physical parameters can be computed for each color channel, and thus 12 images are also returned in this mode. Since each parameter has a different interval of values, all of them have been normalized to be in the range [0, 255]. The resulting images for the red channel only are shown in Figure 8 (a), (d), and (e), respectively. These images are a good representation of all the objects that reflect linearly polarized light. Note that a pixel value in the AoLP image has a meaning only if its corresponding pixel in the DoLP image has a non-zero value. In this set of images, it is possible to confirm that the road, the windshields, part of the body of the car, and some door glasses have a large DoLP. Thus, the AoLP and DoLP features are extremely valuable and can be used, for example, in Deep Learning models to improve the accuracy of the network results for these objects.

I -Rho -Phi : In the Raw I -Rho -Phi mode, all the images are single channeled, thus they are displayed in grayscale. Particularly for the angle of linear polarization, this is not an adequate representation, since it is a circular vari- able. A proper representation would be one that assigns the same color to the maximum and minimum values. In this mode, this is done by creating a color palette based on the HSV color space. Let X be a gray-scale value in the range [0, 255]. The HSV palette is defined as a function that assigns a 3D vector C hsv to each value X, such that:

C hsv = 179X 255 , 255, 255 T (10)
Then, the obtained three-channeled image is converted from the HSV to the RGB color space. On the other hand, the degree of linear polarization is colorized using the Jet palette, which maps blue colors to low values, green colors to middle values, and red colors to high values. As in the Raw I -Rho -Phi mode, 12 images are returned in this mode. The results of this mode for the colored AoLP and DoLP are shown in Figure 8 (f) and (g) respectively, for the red channel.

Fake colors : When dealing with polarization imaging, a correspondence is established between the HSV (hue, saturation, and value) color space and the intensity, degree of linear polarization, and angle of linear polarization [START_REF] Wolff | Polarization vision: a new sensory approach to image understanding[END_REF].

Since the AoLP is a circular variable, it is considered to be the hue of the color. The saturation is the purity of that color, meaning that a saturation of 100% is a pure color, and a saturation of 0% means that it is a gray-level value. Similarly, the degree of linear polarization indicates the "purity" of the light: if it is totally unpolarized, ρ is equal to zero, and if it is totally linearly polarized, ρ is equal to 1. Finally, if a conventional color image is considered, and the value channel is extracted from it in the HSV space, a gray-scale version of the original image is obtained. This information is similar to the total intensity measured by the I parameter. Thus, a color image can be obtained if the I, ϕ, and ρ images are stacked together, considered to be in the HSV space, and then converted back to the RGB space. The result obtained in this way is called a fake color image. The colors obtained with this processing algorithm have the following properties:

• Unpolarized light is represented by gray-scale colors.

• Highly polarized light will be colored.

• The colors in the image depend mainly on the AoLP, thus, to the surface orientation.

This processing is interesting since it helps to quickly identify the objects that reflect polarized light. As explained before, the polarization parameters depend on the frequency, thus the fake color images are separated by color channel. Therefore, this operation returns four images. The resulting image for the red channel is shown in Figure 9.

White-balance module

The observed light by a camera depends on two parameters: the observed object reflectance, and the illumination of the color [START_REF] Ono | Degree-oflinear-polarization-based color constancy[END_REF]. Thus, a white-balance algorithm needs to be used in order to restore the true colors in the scene. In some polarization cameras, and particularly in the Basler RGBpolarization camera, this type of algorithm is not correctly implemented. Thus, our software includes an implementation of a white-balance algorithm. The white balance is a gain applied to each color channel to get the true color of the objects despite the environment where the image is taken. In our implementation, the automatic white search is done globally, and not in a user-defined region of interest (ROI). The algorithm computes the average of all the color channels of a single orientation, and it searches for the pixels whose average is the highest. If there is a white piece in the scene, even with the color gains unbalanced, its average will be the highest. Thus, the pixel whose average is the highest is considered as white. Then, the highest channel value is left untouched (gain equal to 1), and the other channel gains are computed such that their values equal the highest channel value. This algorithm of automatic white balance is constrained to work with no high-level saturated images. It can be deactivated and the different gains set manually. An example of the input image to this algorithm, and its corresponding output are shown in Figure 7 (b) and (c), respectively.

Polarimetric camera calibration module

This module is an implementation of the polarimetric camera calibration algorithm described in [START_REF] Rodriguez | A practical calibration method for rgb micro-grid polarimetric cameras[END_REF]. The calibration algorithm is used to compute a series of matrices that are applied by a correction function to rectify the measurement errors due to manufacturing imperfections. In this way, two super-pixels of the same color channel that receive the same light source will provide the same output measurements.

The module also provides a function to correct the image, once the calibration algorithm has been run. This function takes a raw image from the camera, and if the calibration matrices have been computed, the function will return another image, with the same structure as the input, but with all the pixel measurements corrected by the calibration algorithm.

The calibration problem can be solved by taking several images of a uniform and linearly polarized light. If the light source is uniform but unpolarized, it can be polarized using a linear polarization filter. By turning the filter, the light received by the camera at each filter position will have a different AoLP. A sample of a polarized light source with an AoLP of 40 • is shown in Figure 10, for the 0 • , and 45 • polarization channels. These images correspond to the results before and after applying the calibration algorithm.

The calibration procedure will compute the pixel parameters given the model defined in [START_REF] Rodriguez | A practical calibration method for rgb micro-grid polarimetric cameras[END_REF], i.e., (T i , P i , θ i ). In this model: T i is the pixel gain; P i is a parameter that accounts for the non-ideality of the micro-polarization filter implemented on the pixel; θ i is the effective orientation of the micro-polarization filter of the pixel; i is the position of the pixel considered. From Mueller calculus, these parameters have an ideal value of (T i , P i ) = (0.5, 1.0) for all i, and θ i ∈ {0 • , 45 • , 90 • , 135 • }. However, in general, each pixel will have a set of parameters that will be different from these ideal values. Thus, the calibration algorithm allows to compensate for these differences.

To assess the acquisition quality, the different plot functions of the software can be used. With these plots, the user can determine if the acquisition is correct, and if the camera measurements are valid. It can also assess the quality of the sensor and confirm the effectiveness of the correction on the camera measurements.

Among the available plots, the histograms of the intensity, the DoLP and the AoLP of the incident light can be computed from the image currently displayed by the software. This information allows to evaluate the quality of the calibration results. After correction, these three parameters (intensity, DoLP and AoLP) will have a narrower distribution than the uncalibrated case. These histograms before and after calibration are represented in Figure 11 (a) to (f). They have been computed with the same sample image, mentioned before, of a polarized light with an AoLP of 40 • .

A real-time plot of these three parameters (intensity, DoLP, and AoLP) can be done for a given row of pixels of the sensor. These graphs are displayed in the last two columns of Figure 11, and they correspond to the measurements before and after applying the calibration. These plots illustrate the vignetting effect and show how calibration can reduce its impact over the three polarization parameters. It is important to note that this correction is achieved since the pixel model used considers the polarization parameters of the pixel, and not only the unbalanced sensing gain. A simple gain correction will only affect the intensity image, but not the AoLP nor the DoLP images.

Finally, the consequence of applying the calibration can be observed in the reference urban scene image. The effects of the calibration over the intensity, AoLP, and DoLP images are shown in Figure 12. A zoom has been added to the top and bottom right areas of the image where it is possible to see that the vignetting effect has been corrected. Particularly from these images, the contribution of the calibration can be observed mostly in the AoLP and the intensity images. In the scene, there are several walls that act as planes. Thus, they should reflect the same AoLP, i.e., they should have the same color. This happens only after calibration, mainly in the building situated in the far region of the image, and in the walls on both sides of the road. In the intensity image, since the pixel model includes a gain factor, the vignetting effect is also corrected with this system, making the darker areas in the borders to be as bright as the center of the image.

Polarization processing algorithms

Differently from Section 5.2 where basic polarimetric operations can be done, in this module two applications of the polarization concepts are implemented. The first one is the simulated polarization filter. As explained in Section 2, each super-pixel allows the computation of the Stokes vector S = [S 0 , S 1 , S 2 ] T of the incident light. Now, let us consider a light, described by the Stokes vector S in that passes through a linear polarizer. The filter is oriented at an angle θ and modeled by a Mueller matrix M, as explained in Section 2. Therefore, the effects of a linear polarizer in front of a normal camera can be simulated by computing Eq. [START_REF] Tyo | Design of optimal polarimeters: maximization of signal-to-noise ratio and minimization of systematic error[END_REF]. Thus in this functionality, the inputs are a raw image from the camera and the orientation of the filter θ that one would like to simulate. Then, this algorithm returns two images: the input image, and the filtered image after applying Eq. ( 7) to all the super-pixels. This functionality is commonly used in photography to remove annoying polarized reflections from the environment. In a real system with a conventional RGB camera, the filter is physically placed on top of the lens, and turned until the reflection is removed. With this software, the filter and its effects are simulated after the image has been captured, and the exact angle for reflection removal can be found. This orientation is equal to the AoLP of the incident light shifted by 90 degrees. As an example, the results of simulating a linear filter with the angles θ = 60 • and θ = 120 • are shown in Figure 13 (b) and (c), respectively. Note that the choice of an angle or another will either reinforce or erase the reflections on the windshields. The second functionality of this module is the polarized specularity removal. It is an extension of the simulated polarization filter, explained above. In the previous case, all the pixels are affected by a single polarization filter. But, in a scene, there might be several objects that produce this type of reflection, with different AoLP. To erase them all at once, let us consider the Stokes vector of the observed light. This vector can be split into two other Stokes vectors: one that represents totally unpolarized light and another that represents a totally linearly polarized light S = S unpol + S pol Removing the polarized reflection means erasing the component corresponding to S pol . This is equivalent to computing the S unpol vector. This functionality returns two images: the input and the filtered images, both demosaiced. In contrast to the previous case, this functionality does not require the user to enter an angle to each filter.

The filtering is done based on the measured DoLP at each super-pixel. The results of the filtered image is shown in Figure 13 (d). One can note that most reflections from the shiny surfaces, such as the windshield, the road and the door glasses, have been removed.

Discussion

Recent technological advances have made possible new sensors that allow for capturing RGB and polarization information in a single snapshot. This is an important step forward in the development of the polarization field since it reduces the movement constraints and the acquisition time for this modality. This is one of the main reasons why many research works leveraging polarization have been published in recent years with impressive results. From a global point of view, all of the works make use of the AoLP and the DoLP as principal added features to an RGB system. The Fresnel equations are used as geometrical constraints to better guide an optimization algorithm or a data-driven model training towards more consistent results. This last point can be used in two ways: either by computing the normal maps out of the Fresnel equations and providing these maps as input cues for a deep learning network or by using these relationships to compute an energy function that will be later optimized. Going further, some works [START_REF] Lei | Polarized reflection removal with perfect alignment in the wild[END_REF][START_REF] Zhou | Learning to dehaze with polarization[END_REF] use the raw measurements to avoid adding non-linearities to the relationships between the polarization state and the intensity measurements. However, there is still much to explore, since one of the reasons why polarization has not been widely adopted in the computer vision field is the required controlled acquisition conditions. In other words, many applications have been developed in laboratory conditions, or in environments where there are no perturbations that might affect the measurements. One example of this is in shape from polarization algorithms [START_REF] Ichikawa | Shape from sky: Polarimetric normal recovery under the sky[END_REF][START_REF] Fukao | Polarimetric normal stereo[END_REF][START_REF] Zhu | Depth from a polarisation + rgb stereo pair[END_REF], where often the properties of the object being reconstructed are known beforehand. Even though experiments can be done in outdoor conditions, the fact of having a single object to analyze prevents their generalization to a more generic scene in which several objects, with different properties, are present at the same time. Another example of the generalization limitation are algorithms developed under simulated conditions, such as the underwater image enhancement algorithm Li et al. [START_REF] Li | Underwater image restoration via stokes decomposition[END_REF] . In this particular case, the results show a clear improvement with respect to RGB systems, but the experiments have been done in a small-sized water tank, with known light, and no perturbations such as waves or the presence of other objects inside the tank. These constraints are not in line with the requirements of a real-world underwater situation, leaving its actual applicability unknown. It is worth noting that for data-driven algorithms, it is hard to make acquisitions of large number of images of highly reflective surfaces. These types of surfaces are known to be challenging for RGB-only algorithms and the polarization information can provide valuable cues. However, to obtain a ground truth regarding the shape or the depth, sensors such as the Microsoft Kinect or LiDAR must be used, but the reflective or transparent nature of the objects can result in inaccurate measurements by the sensors. Therefore, there is still challenges in handling these types of situations. As also mentioned in the different categories of Section 4, there is a lack of common standards to share data, acquisition and basic signal processing tools to process polarization information. In this context, we have proposed a software library toolkit that can serve to the objectives of facilitating the acquisition, application deployment, and comparison of techniques with polarization cues. We expect this toolkit to be a step further to help promoting the polarization imaging field in less constrained conditions.

Conclusion

In this paper, we have done a comprehensive review of recent works that have leveraged the polarization modality in robotics and computer vision fields, summarizing some of their main contributions and limitations. The rich information brought by RGB-polarization cameras has allowed a huge improvement in the analysis of scenes where RGB-only systems generally fail. The number of works in the recent years has increased due to the advent of new and cheap RGB-polarization sensors, and data-driven algorithms. These new devices facilitate data acquisition, in less constrained environments, and in real-time with a single snapshot. However, there is still a long way to the vision community to fully exploit the potential and the benefits of this modality. There is no general rule to follow to extend most existing conventional RGB algorithms to RGBpolarization cameras. As discussed before, just providing the polarization information as an additional input to an algorithm is unlike to provide better results. A good understanding of the physics and principles of light polarization is required to be able to get the most out of this modality. Furthermore, the lack of standards, common process-ing tools, large-scale benchmarks limits the development of methods that extend the results obtained for traditional vision systems since it is often not possible to train and compare different approaches over a common dataset. To allow a greater number of researchers to easily and rapidly acquire and analyze polarimetric images, we have developed an open-source software library toolkit that makes available commonly used processing algorithms. This software has been carefully developed to allow easy maintenance and the addition of new features. We hope this work will encourage other researchers to contribute to this field and to participate in the expansion of the functionalities included in the presented toolkit, with the aim of making it accessible to anyone who wants to explore polarization.

Figure 1 :

 1 Figure 1: (a) Micro-grid sensor scheme with four super-pixels with polarizer orientations of 0 • , 45• , 90 • , and 135 • arranged following a Bayer pattern of a typical RGB-polarization sensor[START_REF] Rodriguez | A practical calibration method for rgb micro-grid polarimetric cameras[END_REF]. (b) Sketch of the interaction between a light ray coming from the air with refractive index n 1 and an object with refractive index n 2 . When the incident light ray hits a point on the surface of the object, a portion of the light is reflected and a portion of it is refracted or transmitted in the second medium. The law of reflection states that the angle of reflection is equal to the angle of incidence α 1 . Regarding the refracted light, its direction of travel will change, and it will be equal to α 2 .

Figure 2 :

 2 Figure 2: Results of some of the reviewed algorithms for image enhancement. From left to right: (a) Input image, ground truth, and algorithm results of DoLP-based color constancy [23]. (b) Input image, obtained diffuse image, and obtained specular image of Polarization-guided specular reflection separation[START_REF] Wen | Polarization guided specular reflection separation[END_REF]. (c) Input image, ground truth, and resulting images of the polarization dehazing method[START_REF] Zhou | Learning to dehaze with polarization[END_REF]. Images courtesy of the respective works.

Figure 3 :

 3 Figure 3: Segmentation examples with polarization cues.From left to right: (a) Input image, results obtained with an RGB-only method, and the result of glass Segmentation using intensity and spectral polarization information[START_REF] Mei | Glass segmentation using intensity and spectral polarization cues[END_REF]. (b) Input image with transparent objects, RGB-only method object segmentation results, and results obtained with the polarization data-driven method[START_REF] Kalra | Deep polarization cues for transparent object segmentation[END_REF]. (c) Input image, RGB-only method results, and the results of the Multimodal Material Segmentation algorithm[START_REF] Liang | Multimodal material segmentation[END_REF]. Images courtesy of the respective works.

Figure 4 :

 4 Figure 4: Results of some of the reviewed algorithms for surface normal and depth estimation. From left to right: (a) Intensity input image, reconstructed mesh, and estimated normal map by the method introduced by [50]. (b) RealRGB image, corresponding rendered image, and the corresponding rendered degree of linear polarization. This result corresponds to the scene rendering technique implemented in[START_REF] Kondo | Accurate polarimetric brdf for real polarization scene rendering[END_REF]. (c) Input polarization images, ground truth, and estimated normal maps. These results are courtesy of[START_REF] Ba | Deep shape from polarization[END_REF].

Figure 5 :

 5 Figure 5: Results of some of the reviewed algorithms for pose estimation. From left to right: (a) Input RGB image, reconstructed surface with the algorithm provided in [12]. (b) Glass vase model, and the detected pose from two viewpoints. Results courtesy of [53].

Figure 7 :

 7 Figure 7: (a) Raw, mosaiced version of the reference image, for the polarization channel 0 • . (b) Demosaiced version of the reference image, without white-balance, for the 0 • polarization channel. (c) White-balanced, demosaiced images for the polarization channel 0 • . (d) White-balanced, demosaiced images for the polarization channel 135 • .

Figure 8 :Figure 9 :

 89 Figure 8: Polarization images for the red channel. (a) (b) (c) S 0,1,2 components of the Stokes vector. The parameter S 0 corresponds also to the total intensity image. (d) (e) Raw AoLP, and Raw DoLP. (f) Raw AoLP of the red channel, colored with the HSV palette. (g) Raw DoLP of the red channel, colored with the Jet palette.

Figure 10 :Figure 11 :Figure 12 :Figure 13 :

 10111213 Figure 10: Calibration effects over a uniform, linearly polarized light, with an AoLP of 40 • . These images are the demosaiced, RGB polarization channels. (a) and (c) Uncalibrated polarization channels 0 • and 45 • , respectively. (b) and (d) Calibrated polarization channels 0 • and 45 • , respectively.

Acknowledgements

We thank the Conseil Régional de Bourgogne-Franche-Comté for providing financial support for this research through the project ANER MOVIS, and the French government for the Plan France Relance initiative which also provided fundings via the European Union under contract ANR-21-PRRD-0047-01. We would also like to thank the IDRIS CNRS for granting us access to their High Performance Computing resources (Grant No. 2021-AD011013154).