Alice Fontbonne
email: alice.fontbonne@institutoptique.fr

Hervé Sauer

Caroline Kulcsár

Anne-Lise Coutrot

Franc ¸ois Goudail

Experimental validation of hybrid optical-digital imaging system for extended depth of field based on co-optimized binary phase masks

Keywords: image processing, binary phase masks, deconvolution, optical system codesign, wavefront coding, experimental validation

We experimentally investigate the performance of co-optimized hybrid optical-digital imaging systems based on binary phase masks and digital deconvolution for extended depth of field (DoF) under narrow-band illumination hypothesis. These systems are numerically optimized by assuming a simple generic imaging model. Using images of DoF targets and real scenes, we experimentally demonstrate that in practice, they actually reach the DoF range for which they have been optimized. Moreover, they are shown to be robust against small mask manufacturing errors and residual spherical aberration in the optical system. These results demonstrate that the optical/digital optimization protocol based on generic imaging model can be safely used to design DoF-enhanced imaging systems aimed at real-world applications.

Introduction

It is possible to enhance the depth of field (DoF) of an imaging system without reducing the light throughput by placing a phase mask in its aperture stop. Since these masks tend to blur images, digital deconvolution has to be performed on the acquired raw images to obtain a good visual quality. This joint use of a phase mask and of digital deconvolution constitutes "hybrid optical-digital" imaging system. This concept has been introduced by Cathey and Dowski [START_REF] Cathey | New paradigm for imaging systems[END_REF] for continuous phase DoF enhancing masks, generalized by Robinson and Stork 2 to other optical design tasks, and applied to the optimization of DoF enhancing phase masks for different imaging applications [START_REF] Ben-Eliezer | An optimal binary amplitude-phase mask for hybrid imaging systems that exhibit high resolution and extended depth of field[END_REF][START_REF] Diaz | Increase in depth of field taking into account deconvolution by optimization of pupil mask[END_REF][START_REF] Milgrom | Pupil coding masks for imaging polychromatic scenes with high resolution and extended depth of field[END_REF][START_REF] Elmalem | Polychromatic imaging with extended depth of field using phase masks exhibiting constant phase over broad wavelength band[END_REF][START_REF] Zhao | Axial intensity distribution analysis for a depth-offield-extended optical system using a lowfrequency binary phase mask[END_REF][START_REF] Zalevsky | All-optical axial super resolving imaging using a low-frequency binaryphase mask[END_REF][START_REF] Ryu | Design of binary phase filters for depth-of-focus extension via binarization of axisymmetric aberrations[END_REF][START_REF] Katkovnik | A novel binary and multilevel phase masks for enhanced depth-of-focus infrared imaging[END_REF][START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF][START_REF] Muyo | Infrared imaging with a wavefront-coded singlet lens[END_REF][START_REF] Katkovnik | Lensless broadband diffractive imaging with improved depth of focus: wavefront modulation by multilevel phase masks[END_REF][START_REF] Sahin | Phase-coded computational imaging for depth of field extension[END_REF] .

Among the many possible phase mask struc-tures, binary annular phase masks have the advantage of being easier to manufacture than nonrotationally symmetric masks, such as cubic phase masks. [START_REF] Dowski | Extended depth of field through wave-front coding[END_REF] They have been implemented in practice and have shown good performance on specific applications in infrared [START_REF] Diaz | Real-time increase in depth of field of an uncooled thermal camera using several phase-mask technologies[END_REF][START_REF] Elmalem | All-optical athermalization of infrared imaging systems using thermally dependent binary phase masks[END_REF] and visible imaging systems [START_REF] Burcklen | Experimental Demonstration of Extended Depth-of-Field F/1.2 Visible High Definition Camera with Jointly Optimized Phase Mask and Real-Time Digital Processing[END_REF][START_REF] Milgrom | Novel approach for extending the depth of field of barcode decoders by using rgb channels of information[END_REF][START_REF] Haim | Computational multi-focus imaging combining sparse model with color dependent phase mask[END_REF][START_REF] Elmalem | Learned phase coded aperture for the benefit of depth of field extension[END_REF] . A systematic study of their theoretical performance and of their robustness to aberrations has been done in a previous work [START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF] where we addressed in particular the fundamental questions of the maximum DoF reachable with these masks and of the minimum number of rings necessary to reach a given level of performance. It was shown that nearly optimal performance can be obtained with a limited number of rings and that imaging performance is robust to small amounts of aberrations.

This study was based on numerical optimizations assuming a generic scene model, a given noise level and theoretical models of spherical aberrations. However, there remain a number of questions about these results : how do the masks optimized in this way perform in practice, since observed real-world scenes may not be consistent with the generic image model used for their optimization ? In particular, do these masks really achieve the expected DoF extension? Are they robust to aberrations and manufacturing defects? The purpose of the present article is to answer these questions. For that purpose, we manufacture three generic phase masks optimized for three different DoF ranges and perform a systematic and quantitative experimental validation of an actual hybrid optical-digital imaging system based on these masks. The obtained results are shown to validate the numerical optimization protocol based on generic imaging model: this protocol can thus be safely used to improve DoF in real-world imaging applications.

This paper is organized as follows. In Section 2 we briefly review the basics of binary phase mask optimization using a generic imaging model. In Section 3, we describe the optical setup used for experiments and how the masks have been manufactured and tested. In Section 4, we establish the optical performance of the imaging systems including the manufactured masks by measuring their Modulation Transfer Functions (MTF) and their through-focus transfer functions. We also characterize the DoF extension ability of the global optical-digital system by analyzing its effective-MTFs at different defocus values. In Section 5, we evaluate the DoF extension provided by these masks on two types of real scenes: a printed circuit board and a floral scene. We also demonstrate the robustness of the imaging system to spherical aberration. [START_REF] Ojeda-Castañeda | Annular apodizers for low sensitivity to defocus and to spherical aberration[END_REF] 2 Phase mask optimization with a generic imaging model Binary annular phase masks are static spatial phase modulating optical elements consisting of a series of N concentric annular regions of phase modulation of alternatively 0 or π radians at the nominal wavelength λ, usually chosen in the middle of the working spectral range (see Fig. 1). Each annular constant-phase area corresponds to a so-called ring, so that an Nring mask of clear aperture radius R is parameterized by N -1 free normalized radius values φ = {ρ 1 , ..., ρ N -1 }, where the radius of the i'th phase transition is

r i = ρ i R and φ satisfies the conditions 0 < ρ 1 < ... < ρ N -1 < ρ N = 1.
Such masks can be used for extending the DoF of an imaging system. For that purpose, the mask is placed in the aperture stop of the imaging lens. Let us assume that one observes an object at a certain distance z o of the imaging system. For that purpose, we use an optical system of effective focal length f together with a sensor located at a fixed imaging distance z i . A wellfocused object is positioned at a distance z o0 of the imaging system and a defocused object is positioned at the object distance z o = z o0 + ∆z o of the imaging system. The defocus parameter is defined as:

ψ(∆z o ) = (z i × NA) 2 2 1 z i + 1 z o0 + ∆z o - 1 f (1)
where NA is the image Numerical Aperture of the system. It gives the peak-to-valley optical path difference of the Seidel defocus aberration that takes place for such an axial object shift from the nominal focus. Note that the defocus parameter ψ is equal to zero when ∆z o = 0. If one moves the object, the optical system is defocused. The induced object defocus distance ∆z o is negative when the object gets closer to the optical system, and positive when the object gets farther from the optical system. The behavior of a hybrid system containing an optimized binary phase mask with a targeted defocusing The image produced on the sensor can be modeled by h φ ψ (r) * O(r), where O(r) is the ideal sampled scene image (r represents the spatial coordinates), * denotes the convolution operator, and h φ ψ (r) is the Point Spread Function (PSF) of the optical system. In this work, we assume that the sampling satisfies the Shannon-Nyquist condition. The PSF depends on the defocus parameter ψ related to the object distance and on the phase function of the mask represented by the parameter set φ. We assume that the lens is otherwise ideal (with no-aberration, except defocus and phase mask behavior) with, thus, no other free parameter. This acquired image is then deconvolved with a digital deconvolution filter w(r) to restore its sharpness. The restored image at the output of this hybrid opticaldigital system can be modeled as:

Ô(r) = w(r) * h φ ψ (r) * O(r) + n(r) (2)
where n(r) is the detection noise. The reconstruction mean-squared error (MSE) is then defined as:

M SE(φ, ψ) = E Ô(r) -O(r) 2 dr (3)
where E[•] represents the mathematical expec-tation over the noise n(r) and the scene image O(r), which are both assumed to be zero-mean, stationary random processes of power spectral density (PSD) S nn (ν) and S oo (ν) respectively, with ν representing the spatial frequency coordinates. The co-design goal is to simultaneously find the deconvolution filter w(r) and the mask parameters φ that minimize M SE(φ, ψ) over a defocus parameter set such that ψ ∈ ψ 1 , ψ

w(ν) = 1 K K k=1 [ h φ ψ k (ν)] 1 K K k=1 h φ ψ k (ν) 2 + Snn(ν) Soo(ν) (4)
where stands for complex conjugate and for the Fourier transform. It is important to point out that, for a given phase mask, a unique filter is used for deconvolution over the whole field of view, regardless of the object defocusing. The masks designed in Ref. [START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF] and manufactured for the present study have been optimized using a generic ideal image model with a power-law PSD [START_REF] Ruderman | Origins of scaling in natural images[END_REF][START_REF] Van Der Schaaf | Modelling the power spectra of natural images: Statistics and information[END_REF] S oo (ν) = Kν -a , the parameter a being fixed to 2.5. The white noise PSD is such that the signal-to-noise ratio (SNR) on the raw image is 34 dB, with SNR = 10 log 10 S oo (ν)dν/ S nn (ν)dν . With this choice of deconvolution filter, the parameters φ opt of the optimal mask are optimized using a "minmax" criterion:

φ opt = arg min φ max k [MSE(φ, ψ k )] . (5)
The variation of image quality as a function of the number N of the mask rings was studied in Ref. [START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF] with a numerical global optimization method. It was shown that for a targeted defocus range of ψ max = 1λ, three rings are sufficient to reach the maximal performance, whereas six rings are preferable for ψ max = 2λ and seven rings for ψ max = 2.5λ. Optimal masks for these three DoF ranges are represented in Fig. 1 and the normalized radii of their rings are given in Tab. 1. In the following, they will be denoted, respectively, "1λ-phase mask", "2λphase mask", and "2.5λ-phase mask". These masks have been optimized using the simple generic and aberration-less imaging model described above. Our goal in this article is to experimentally validate their DoF extension performance on real-world scenes and in the presence of manufacturing defects and aberrations in the optical system. For that purpose, they have been manufactured and included in a specifically designed imaging system.

Experimental setup

In this section, we describe the masks manufacturing together with the verification of their optical quality. We then describe the optical setup we designed to evaluate their DoF enhancing performance.

Mask manufacturing

The binary phase masks have been manufactured on quartz substrates using UV pho-tolithography associated with Ion Beam Etching (IBE) followed by Inductive Coupled Plasma (RIE -ICP) etching. As they are used with a narrow spectral range illumination centered on the nominal wavelength λ, the etching depth h is calculated so that the phase shift between two rings is equal to π, according to the formula:

h = λ 2(n -1) (6)
where n is the refractive index of quartz at λ = 625 nm. Since quartz is a slightly birefringent material and as we employ unpolarized illumination, we use an empirical average index n = 3 4 n o + 1 4 n e where n o and n e are, respectively, the ordinary and extraordinary refractive indices. This leads to a required etching depth of h = 573 nm, on which we allow for a manufacturing tolerance of ±20 nm. This tolerance corresponds to 3% of the total depth and does not lead to any significant modification of the simulated MTF, while being reasonable regarding the manufacturing process.

The manufactured masks have been verified with a 1D mechanical profilometer to check the etching depth. An example of results obtained with the 2.5λ-mask is given in Fig. 2. It shows that this mask fits well with the manufacturing specifications. It is also the case of the 1λmask, whose step height has also been measured at about 573 nm. The manufactured 2λ-mask deviates a little more from specifications: the height of the three steps are 559 nm for the larger ring, 566 nm and 550 nm for the smaller ones. Simulations performed with the CodeV R software show that this deviation from the ideal step height induces only slight changes of the MTF with somewhat asymmetric behavior in the DoF range relatively to the nominal focus behavior.

Optical system

In Ref. [START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF] , the masks are optimized assuming that they are located exactly at the pupil of the image-forming optics. It is also assumed that the Phase mask ρ 1 ρ 2 ρ 3 ρ 4 ρ 5 ρ 6 1λ-phase mask 0.7684 0.9272 1 2λ-phase mask 0.1947 0.7558 0.9165 1 2.5λ-phase mask 0.5808 0.7047 0.7569 0.7879 0.8525 0.9424 Table 1 Normalized radii of the three optimal phase masks. imaging system without the phase mask is limited by diffraction, which means that geometric aberrations are negligible, and that diffraction PSF is sampled at least at Nyquist spatial frequency by the image sensor. It is also supposed that illumination is monochromatic. In order to be close to these conditions, we have designed a case-study optical system with two commercial doublets, with a small field of view and a low aperture in order to have nearly no aberration and fulfill the Shannon-Nyquist condition for our 3.45 µm pitch CMOS camera sensor. A scheme of the system is given in Fig. 3. We choose a magnification of -1 independent of the object axial position (of the object defocus distance ∆z o ) thanks to object telecentricity. To this end, the first doublet (Edmund Optics, with a focal of 250 mm and a diameter of ∅40 mm, see element (a) on Fig. 3) works in Focus/∞ configuration, with the system stop (of ∅20 mm) exactly placed in its Back Focus plane (element (b)). The removable and interchangeable phase-mask of clear aperture diameter ∅20 mm is placed as close as mechanically possible to the stop (element (c)). A second doublet (Edmund optics, with a focal of 250 mm and a diameter of ∅25 mm, see element (d)), that works in ∞/Focus configuration, forms an image on the camera sensor (element (e)). The second doublet to sensor distance is mechanically fixed. Focusing and defocusing are performed by axially moving the object relatively to the whole optical system. The optical components are set in place by an optomechanical closed housing (element (f)) build specifically for this system with a Cmount interface with the camera. This allows convenient experimental manipulations without stray-light concern. The optomechanical housing also provides the possibility of easily using a removable spherical aberration plate (element (g)). It enables to evaluate the robustness of binary phase masks to this aberration, as will be done in Section 5.2.

The system has an image F-number of 12.5 (i.e. an image Numerical Aperture of NA = 0.04), and an object and image field of 8.5mm × 7.09 mm. The scene is illuminated by a red backlight LED (element (h)) with a narrow spectrum centered at λ = 625 nm and a Full Width at Half Maximum of 20 nm. When it is used at its maximum power, its radiance is equal to 38 W.m -2 .sr -1 . The cut-off image spatial frequency of the system at λ = 625 nm is 128 mm -1 , whereas the sensor Nyquist frequency is 145 mm -1 : the system PSF is thus correctly sampled.

With these parameters, Eq. 1 (where z o should be taken from the H object principal plane, and z i from the H image principal plane, f being the effective focal length of the whole system) can be used to calculate the peak-to-valley defocus optical path difference ψ corresponding to any object defocus distance ∆z o . For example, the object defocus distances of -0.5 mm, -1 mm and -1.5 mm, that will be considered in the MTF measurements of Section 4, correspond respectively to optical path differences of ψ = 0.6λ, 1.3λ and 1.9λ.

(c) (g) (a) (c) (e) (f) (h) (i) (d) (b) (g)

Masks validation with Modulation Transfer Function measurements

When the manufactured phase masks are inserted in the optical system, they modify its behavior in order to enhance its DoF. As a primary validation procedure, we experimentally measure the raw MTFs of the optical system with the phase mask | hφ ψ (r)| at different values of the defocus parameter ψ to verify their match with the nominal ones given by numerical simulations. We also characterize the global effect of the masks on the hybrid optical-digital system by analyzing their effective-MTF after deconvolution for several defocus values. Finally, we measure the behavior of the hybrid system through focus at a given spatial frequency.

MTF measurements

The experimental method chosen to measure the MTF is to acquire images of Ronchi ruling test patterns. The detailed protocol is described on Appendix A. We use it to measure the MTF of the optical system in the presence of the different manufactured phase masks. These experimental measurements will be compared to the theoretical MTF computed with the optical design software CodeV R , taking into account the slight manufacturing defects of the masks and the real optical system.

We first measure the MTF when a non-etched quartz plate is inserted in the system, in order to characterize the system without phase mask while keeping all others configuration parameters unchanged. In the following, this component will be called "neutral plate". Fig. 4(a) displays the theoretical (solid line) and measured (dotted line with cross markers) MTFs obtained with this neutral plate for four different positions of the observed object: on-focus (ψ = 0λ), and increasingly defocused with ψ = 0.6λ, 1.3λ, and 1.9λ. The MTF values are represented only between 0 and 0.5 to facilitate comparison with the other masks. We first notice that the theoretical and experimental results are in good agreement. In particular, the experimental curves correctly follow the global shapes and absolute levels of the theoretical ones. This validates the method used for measurement. Secondly, we observe that on focus, the MTF has the familiar shape of a diffraction limited one. In the presence of defocus, the MTFs rapidly drop for ψ values are outside the natural DoF of a diffraction-limited system as given by the Rayleigh criterion (|ψ| < λ/4).

The MTFs obtained by inserting the 1λ-mask in the system are displayed in Fig. 4(b). We can see that in this case also, the experimental curves correctly follow the global shapes and absolute levels of the theoretical curves, which shows that the actual MTFs behave as predicted by the simulations. The 1λ-phase mask has been optimized to maintain a good imaging performance over a ψ range equal to [0, 1λ]. It should however be noted that binary phase masks throughfocus response is independent of the defocus parameter sign, so that the full DoF for this mask is in fact [-1λ, 1λ]. Nevertheless, for the sake of clarity, we do not include the MTF measurement for negative object defocus distance. We observe on Fig. 4(b) that the MTF at defocus parameter ψ = 0.6λ, which is within the defocus range of the mask optimization, is quite close to the on-focus MTF. This means that the imaging performance will be similar for ψ = 0 and ψ = 0.6λ, as the same deconvolution filter is used. By comparing with Fig. 4(a), it can also be noticed that the on-focus MTF is however much lower than the diffraction-limited one. This is due to the presence of the mask, and this is the reason why deconvolution is necessary to recover image sharpness. The MTFs for ψ = 1.3λ and ψ = 1.9λ are much lower, since these two values of ψ are outside the targeted defocus range for which the mask has been optimized.

Figure 4(c) displays the MTFs of the 2λmask. The entire range of ψ values taken into account in the measurements is included in the targeted defocus range of this phase mask. Hence all the MTFs are close to each other: the maximum differences between the MTFs remain below 0.1 over the entire range of interest, and is even closer to 0.05 for medium frequencies, between 20 lp.mm -1 and 60 lp.mm -1 . All the MTFs keep significant values until a spatial frequency of 60 lp.mm -1 . Moreover, in this case also, the measured MTFs follow the main features of the theoretical ones.

Finally, Fig. 4(d) represents the MTF curves for the 2.5λ-mask. It is observed that in this case also, the MTF curves are close to each other. However, since this mask has been optimized for a wider range of ψ values, there are some significant differences for medium frequencies due to the increasing difficulty of optimizing the compromise between sharpness and larger targeted defocus range.

In conclusion, the MTFs of the manufactured phase masks have a behavior close to what was anticipated by the simulations.

Effective-MTF

Although MTFs are a good way to check the compliance of the manufactured masks with the theoretical ones, they are not fully representative of the entire co-designed system which also includes image deconvolution. One way to represent this information is to visualize the effective-MTF, defined as: The effective-MTF describes the ability of the deconvolution filter to restore the attenuated scene spatial frequencies of the blurred raw image. From this point of view alone, the ideal effective-MTF should be equal to 1 at all frequencies. However, to avoid a strong amplification of the noise, the averaged Wiener filter w(ν) defined in Eq. 4 limits the restoration gain at the frequencies where the noise is higher than the signal transmitted by the optical system, leading to effective-MTFs lower than 1 in these areas. The frequencies at which this function is close to one are correctly restored by the hybrid system. By using an averaged filter, some spatial frequencies may be over amplified (when effective-MTF is higher than 1), leading to an image degradation by over-oscillation for some defocusing.

E φ ψ (ν) = | w(ν) hφ ψ (ν)| . (7)
We show in Fig. 5 the effective-MTFs of the hybrid system for the three manufactured masks. The theoretical MTFs and the average Wiener filter w(ν) defined in Eq. 4 have been used for this purpose. The Wiener filter has been calculated assuming an object PSD S OO (ν) ∝ ν -2.5 and an SNR equal to 34dB -equivalent to the camera noise under good lighting conditions.

The effective-MTFs are computed for five different defocus parameter ψ values ranging from 0λ (on-focus) to 2.5λ.

Figure 5(a) represents the effective-MTFs for the 1λ-mask. We observed that for defocus parameter values 0 and 0.6λ, which are within the optimized DoF range, the effective-MTF value is quite close to 1 until 60 lp.mm -1 , then drops and reaches 0 for a spatial frequency of 80 lp.mm -1 . This means that good image quality can be recovered in the spatial frequency range [0, 60] lp.mm -1 . On the other hand, for defocus parameter values larger than 1λ, the effective-MTFs are very low, since they are outside the DoF range for which the mask has been optimized.

With the 2λ-mask, the targeted defocus range is twice larger. It is observed in Fig. 5(b) that for all defocus parameter values below 2λ, the effective-MTF value keeps values above 0.5 up to the spatial frequency 55 lp.mm -1 , then drops and reaches 0 for a spatial frequency of 65 lp.mm -1 for ψ = 0.6, and for a spatial frequency above 65 lp.mm -1 for the other defocus parameter values. In consequence, image quality can be recovered until a spatial frequency of 55 lp.mm -1 at all defocus parameter values below 2λ. This means that good image quality can be recovered in this spatial frequency range. We can notice that this range is narrower than for the 1λ-mask. This is expected since the targeted defocus range is twice larger so the imaging task is more arduous. Conversely, within this spatial frequency range, the curves corresponding to ψ = 1.3λ and to 1.9λ are well above those in Fig. 5(a).

Figure 5(c) represents the effective-MTFs for the 2.5λ-mask. We can see that the curve corresponding to a defocus parameter of 2.5λ is now above 0.5 up to the spatial frequency 40 lp.mm -1 , whereas it was very low in Fig. 5(a) and Fig. 5(b), since the corresponding masks have not been optimized for this value of defocus parameter. Nevertheless, the global level of the curves is lower than in Fig. 5(b) because when the targeted defocus range gets larger, the optimization problem becomes more arduous.

In summary, effective-MTF graphs illustrate very well the necessary tradeoff between the targeted DoF extension range and the quality of the recovered image. [START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF] We have verified that the MTFs and effective-MTFs have the desired shape for a few values of the defocus. Phase masks can now be tested when the defocus varies continuously.

Through-focus behavior

As a further element of characterization, we now measure the through-focus behavior of the hybrid imaging system. For that purpose, we use a DoF target (Edmund Optics #54-440), which is made of a Ronchi ruling test pattern drawn on a surface set at 45 • of the optical axis. When observed with an object-telecentric optical system at the nominal 45 • surface tilt, this target is shown with a constant 15 lp.mm -1 spatial frequency and with an axial distance that varies linearly within the field of view. Fig. 6 represents images of the DoF target observed with the neutral plate (this image is not deconvolved) and with the three manufactured masks (these images are deconvolved with the corresponding averaged Wiener filter). The camera has been focused at the center of the field of view. In the left part, the target is closer to the optical system, and thus corresponds to negative object defocus distances. In the right part, it is farther from the camera and thus corresponds to positive object defocus distances. As the absolute value of the defocus increases, we can observe that contrast cancels out, indicating that the MTFs at spatial frequency 15 lp.mm -1 drop to zero for some defocus values. Comparing the different pictures of Fig. 6, we clearly see that the object defocus distance ∆z o between two nulling increases with the DoF range for which the mask has been optimized: it is ±1 mm for the neutral plate, ±2 mm for the 1λ-mask, ±2.5 mm for the 2λmask, and ±2.8 mm for the 2.5λ-phase mask. We can also notice that the images of the target are slightly asymmetric although in theory, the behavior of a binary phase mask is perfectly symmetric with respect to optical path difference. This may be due to slight defects of the optical system and of the phase mask. Asymmetry is larger for the 2λ-phase mask, for which we have seen that the measured phase delay between adjacent rings departs the most from their nominal values (see Section 3.1). For an optical designer using CodeV R , it may be interesting to have an insight of the DoF of the hybrid system during its design. For that purpose, it is possible to compute the Contrast Transfert Function (CTF) of the hybrid system as object defocus distance varies (by computing, with CodeV R , the MTF with the "square wave" (SQW) option). The CTF is the local contrast max -min max +min of the image of a perfect Ronchi ruling pattern, whereas MTF gives the local modulation of the image of a perfect gray-level sinusoidal pattern. It is a good indicator of the areas of a Ronchi pattern image that will have a good visual quality after deconvolution. The CTFs for the 1λ-mask and the 2.5λ-mask are plotted in Fig. 7 together with the DoF target deconvolved images for these two values of the targeted defocus range. It is observed that the contrast nulling in the images correspond exactly to the object defocus distance values where theoretical CTF reaches zero. This confirms that the hybrid imaging system using the manufactured phase masks fits very well with the expected behavior.

-3 -2 -1 0 1 2 3 Neutral plate -3 -2 -1 0 1 2 3 1 -mask -3 -2 -1 0 1 2 3 2 -mask -3 -2 -1 0 1 2 3 2.5 -mask

Experimental validation of the phase masks on real scenes

In this section, in order to experimentally validate the ability of binary phase masks to enhance DoF, we test the hybrid system on different types of real scenes. The first one is a printed circuit board featuring components at different depths corresponding to a maximal defocus parameter of ψ = 2.5λ (that corresponds to an object defocusing of about 2 mm). The second is a natural scene composed of two of flowers. It will be used for illustrating the robustness of the hybrid system to spherical aberration.

Performance of the masks on a real scene

Let us evaluate the imaging performance of the hybrid system on a printed circuit board. This object is interesting for our purpose since it contains many elements spread over a few millimeters of depth. For all the images in Fig. 8, the optical system has been focused, using the neutral plate, on the top of the board which appears on the far left of the field of view. The bottom of the board, located at the center of the field, is about 2 mm deeper than the top. The board is illuminated at oblique incidence with the same red LED source as previously used for the test patterns, which is centered on λ = 625 nm. Illumination intensity and integration time have been set so as to maximize image dynamic range while avoiding saturation. This is a difficult task on an object with many metallic parts, and a few light saturation points still appear on the component pins on the left and around the holes of the board. Figure 8(a) represents the board observed with the optical system and the neutral plate. Except for its far left part, on which the imaging system has been focused, the image is blurry. This shows that the DoF of the conventional optical system without mask is not sufficient to cover the whole depth of the board.

Keeping the same focus point, we inserted the different masks in the system and deconvolved the images with the corresponding average Wiener filter. The filters are built with a generic scene PSD ∝ ν -2.5 and the MTFs of the optical system with theoretical masks parameters. They are therefore not the ideal (but unknown) filters for this real scene. As the dynamic of the scene has been greatly reduced to avoid too much saturation on the metallic parts, the SNR of the scene is decreased. This is accounted for in the average Wiener filter by adjusting its regulation parameter, or equivalently by modifying the PSD S nn of the detector noise, allowing a deconvoluted image of better visual quality. Fig. 8(b) represent the deconvolved image obtained with the 1λ-mask. We first observe that this mask clearly increases the DoF. For example, the letters and numbers appear more sharply. However, the bottom of the board remains blurry, as it lies outside the DoF range for which the mask has been optimized. The whole image is much sharper with the 2λ-mask [Fig. 8(c)], and even sharper with the 2.5λ-mask [Fig. 8(d)], whose optimization range fits exactly the actual depth of the object. One can observe on all deconvolved images some ringing artifacts around the saturation points. They could be reduced by matifying the object or by processing these points in a different way.

To better analyze the increase in DoF on this object, two subparts of the image have been selected and appear in Fig. 9. On the second row of this figure, it is seen that the 1λ-mask enables the resistor to be almost sharp [Fig. 9(c)], but the bottom of the board is still blurry [Fig. 9(d)]. The 2λ-mask (third row) makes the resistor totally sharp [Fig. 9(e)], and the sharpness of the bottom of the electronic board is also improved [Fig. 9(f)]. However, there remains a sort of annular artefact around the written areas that looks like a circular element of the printed circuit line. This artefact almost disappears with the 2.5λmask (fourth row), as can be seen in Fig. 9(f). Some other benefits of the 2.5λ-mask can be noted: the long dust grain above the resistor is sharper, and the printed conductive track is no longer confused with the written area.

These two image subparts therefore confirm that DoF is increased by hybrid optical system including a phase mask and deconvolution even if the generic PSD model of the scene may not fit exactly with the real one. It also confirm that DoF extension is better when the DoF optimiza- tion range of the masks is equal to the actual depth of the object.

Robustness to spherical aberration

It is shown in Ref. [START_REF] Falcón | Performance limits of binary annular phase masks codesigned for depth-of-field extension[END_REF] through simulations that hybrid imaging systems based on co-designed binary phase masks are robust to residual aberrations present in the actual imaging system but not taken into account during optimization. Our goal in this section is to validate this result experimentally on real world system and scene.

For that purpose, we insert in the optical system a phase plate with a known level of spherical aberration. The corresponding wavefront error is expressed by the following Seidel term:

W S = αρ 4
, where ρ ∈ [0, 1] denotes the normalized radial position in the pupil. For this aberration, according to the Rayleigh's quarter wavelength rule, a peak-to-valley wavefront error due to an aberration larger than λ/4 in a conventional optical system is considered to be visually noticeable and designers may aim for lower values. To study the impact of this aberration, we thus choose two spherical aberration plates.

The first one has α = -0.31λ, which is close, but slightly above the Rayleigh criterion. The second one has a larger amplitude α = -0.62λ.

First, let us study the impact of spherical aberration on images of the DoF target when the neutral plate is inserted in the optical system. We have represented on Fig. 10 the throughfocus CTFs computed with CodeV R in the presence of the neutral plate only and of the neutral plus the aberration plates, together with the corresponding images of the DoF target. The curves and the images match exactly. We can observe that in the absence of aberration plate, the CTF is almost symmetric around the focusing depth. The slight asymmetry still present is due to the residual defects of the simulated optical system. The first contrast cancellations appear around ±1.2 mm, and it is possible to see small side lobes, with a much lower contrast than the central lobe. Adding a spherical aberration plate of -0.31λ shifts the best focus position of about 0.3 mm. This effect is also observable on the DoF target image, where the side lobes are larger for positive object defocus distance, and attenuated for negative object defocus distance. The phenomenon is even more obvious when we consider the spherical aberration plate of -0.62λ. It is also worth noting that in both cases, the contrast of the central lobe is not affected. This means that the effect of these levels of spherical aberrations is very weak at the spatial frequency of 15 lp.mm -1 . The DoF target is now observed with the 2λmask. In order to evaluate the robustness of the hybrid system, we do not take into account the presence of spherical aberrations in the expression of the deconvolution filter. We have represented in Fig. 11 the through-focus CTFs with the mask only and with simultaneously the mask and the aberration plates, together with the corresponding images of the DOF target. The curves and the images have a good match. In particular, the object defocus distance values for which the local contrast vanishes on the images correspond well to the zeros of the CTF curves. We can also observe, as in the case of the neutral plate, a shift of the zeros toward the positive object defocus distance values. Another important effect should be noted: the spherical aberration tends to make the CTF asymmetric. However, the main global features of the CTFs are conserved after inserting the spherical aberration plates. Let us now assess the influence of spherical aberration when imaging a natural scene. The chosen scene shows a buttercup on the right (whose petals are about 3mm long) above a common daisy on the left, as shown in the image description in Fig. 12. The optical system is focused using the neutral plate on the pistil of the buttercup. This scene has a DoF of several mil-limeters and is illuminated with a red LED light source. From the observer's point of view, the daisy is behind the buttercup, and the stamen 1 is further away than the pistil, itself further away than the stamen 2. Figure 13(a) represents the scene observed with the neutral plate and no aberration. The system has a very limited DoF: the pistil of the buttercup is sharp, but not the stamen 1 above the left petal nor the stamen 2 above the right petal. When the -0.32λ spherical aberration plate is added, it is seen in Fig. 13(c) that the stamen above the left petal of the buttercup becomes sharper whereas the petal on the right becomes blurry. This is because, as seen in Fig. 10, spherical aberration tends to shift the sharpness area towards positive object defocus distance values, i.e. further away from the optical system. The same effect, even more pronounced, is seen on Fig. 13(e), which represents the image observed with the neutral plate and the -0.62λ spherical aberration plate.

Figure 13(b) is obtained by inserting the 2λmask and no aberration plate in the optical system. The Wiener filter used to deconvolve the image is the one used during phase mask optimization; it does not take into account aberrations of the optical system. The increase in DoF is visible in the image. For example, the stamens above the petals on the left and right are now sharp. Details appear more clearly, espe-cially some of the daisy's disc florets on the left, the edges of the petals, or the leaf at the bottom right of the image. When the spherical aberration plate of -0.31λ [Fig. 13(d)] or -0.62λ [Fig. 13(f)] is added to the hybrid system, the difference with the images of the system without aberration is small. There are small differences on stamen 1, which appears blurrier when there is more spherical aberration. Nevertheless, the details remain well visible, which shows the robustness of the hybrid system to spherical aberration.

In summary, these experimental image acquisitions demonstrate the robustness of an hybrid system based on a jointly optimized binary phase mask: even if spherical aberration exceeds the Rayleigh criterion by more than twice, it does not disturb significantly the deconvolution. The achieved DoF remains close to the targeted one and is much larger than the one of a conventional system.

Conclusion

We have experimentally investigated the actual performance of hybrid optical-digital imaging systems for extended DoF based on binary phase masks co-optimized with a generic numerical imaging model. Phase masks optimized for three different DoF ranges were manufactured and implemented in a dedicated imaging test bench that allowed us to experimentally measure the MTFs of the system and to acquire images of real-word scenes. We have demonstrated that, in practice, good image quality is obtained within the whole targeted defocus range. The system is robust to mask manufacturing defects and to imaging characteristics different from those used in the optimization process, especially in terms of scene PSD and noise level. Moreover, although the masks were optimized for a nominal aberration-less optical system, they are robust to small amount of spherical aberration: as the aberration level grows, the DoF persists with a drop on image quality that is comparable with the one observed on a conventional systems at nominal focus. These results validate the mask optimization protocol based on generic imaging model and show that this protocol can be safely used to design DoF-enhanced imaging systems aimed at real-world applications. This work has many perspectives. A first one is to consider wide spectrum illumination to optimize and validate hybrid systems for panchromatic and/or color imaging. Moreover, we assumed in this study that the imaging system was well-sampled. It is of interest to thoroughly investigate the case of under-sampled imaging systems that are frequently encountered.

Appendix A: Experimental Methodology for MTF measurement

The experimental method chosen to measure the MTF is to acquire images of Ronchi ruling test patterns m(x, y) with different spatial frequencies from 5 lp.mm -1 to 120 lp.mm -1 by step of 5 lp.mm -1 (see element (i) on Fig. 3). This allows to cover the MTF spatial frequency range of interest up to the cut-off frequency of the optical system. If detector noise is neglected, the acquired raw image can be modeled as follows: s(x, y) = A h φ ψ (x, y) * m(x, y) + off ×Π M (x) × Π N (y)

where A is a factor accounting for parameters such as the medium illumination intensity or the gain of the camera, m(x, y) is a 1D square wave of frequency ν 0 with values 0 or 2, off is the offset of the camera, Π M (x) is a rectangle function of width M pixels representing the finite extent of the test pattern along the x-axis and Π N (y) is a rectangle function of width N pixels representing the finite extent of the test pattern along the y-axis. Note that, as we considere the PSF h φ ψ normalized with their energy h(x, y)dxdy, the medium gray level is directly given by A.

The goal is to estimate the MTF at the spatial frequency ν 0 of the test pattern with its acquired image. As the lens geometrical magnification is 1, object and image spatial frequencies are the same. Nonetheless, the test pattern is square wave shaped and not sinusoidal in gray levels, so we can not directly get the MTF value. Indeed, the periodic square wave shaped test pattern can be seen, through Fourier series analysis, as the superposition of fundamental and higher harmonic sine waves. A Fourier transform of the test pattern image allows us to extract the behavior at the ν 0 (fundamental) spatial frequency, but we must take care that the Fourier series coefficient at the fundamental frequency +ν 0 is

|s(ν 0 )| = 2 π | hφ ψ (ν 0 )| × C (9)
where C = A × M N depends only on the characteristics of the camera (gain, average gray level and number of pixels). We can also extract s(0) = (A hφ ψ (0) + off ) × M N from Eq. 8. It is therefore possible to estimate the MTF of the image of a test pattern of spatial frequency ν 0 oriented along the x-axis by:

| hφ ψ (ν 0 )| = π 2M N × |s(ν 0 )| |s(0)| M N -off . ( 10
)
To calculate the Fourier transform of the sampled test pattern without estimation error due to spatial frequency sampling, the calculation is not performed by Fast Fourier Transform but by computing the discrete-space Fourier transform at exactly ν 0 , the spatial frequency of the test pattern. Moreover, it is experimentally impossible for the test pattern to be perfectly parallel to the pixels of the matrix sensor, which introduces a bias due to the fact that the discretespace Fourier transform is not calculated for the correct rotation angle of the test pattern. In order to avoid this bias, we estimate the contrast

where px is the pixel pitch of the CMOS camera sensor. The maximization is performed by exhaustive search over a limited range of angles θ around 0. This algorithm enables us to experimentally estimate the MTF up to 120 lp.mm -1 with good accuracy.

Disclosures

Authors have no conflict of interest related to the work leading to the results presented in this manuscript.

Fig 1

 1 Fig 1The three chosen masks. (a) the "1λ" phase mask (b) the "2λ" phase mask and (c) the "2.5λ" phase mask. A ring is defined as an annular region with constant phase modulation. Dark gray areas induce a phase shift of 0 and light gray areas induce a phase shift of π radians at the nominal wavelength λ. (The "1λ", "2λ" and "2.5λ" mask names are their targeted defocus ranges as explained in the text below)

Fig 2

 2 Fig 2 One profile of the 2.5λ-phase mask. The arrows give the average step heights on several profiles.

Fig 3

 3 Fig 3 Schematic drawing of the optical system designed for the study of phase masks. (a) first Edmund Optics doublet, (b) STOP, (c) removable phase mask, (d) second Edmund Optics doublet, (e) camera sensor, (f) optomechanical closed housing, (g) optional removable spherical aberration plate, (h) red backlight and (i) scene, for instance, transmission Ronchi ruling test pattern.

Fig 4

 4 Fig 4 Experimental MTF and theoretical MTF for (a) the neutral plate (b) the 1λ-phase mask (c) the 2λ-phase mask and (d) the 2.5λ-phase mask. For (b) and (c) curves, the x-axis is cut above to respectively 100 lp.mm -1 and 80 lp.mm -1 for a better readabilily. The diffraction-limited cut-off frequency is 128 lp.mm -1 . The legend is the same for all graphs.

5 Fig 5

 55 Fig 5 Effective-MTFs of the simulated hybrid system with (a) the 1λ-phase mask, (b) the 2λ-phase mask and (c) the 2.5λ-phase mask.

Fig 6

 6 Fig 6 Images of a 15 lp.mm -1 DoF target (illumated with the red LED) obtained with different masks after deconvolution, except for the top one (neutral plate), with no deconvolution. Horizontal axis is ruled in millimeters.

Fig 7

 7 Fig 7 CodeV R computed CTF curves at 15 lp.mm -1 as a function of object defocusing of the optical system containing the 1λ-phase mask or the 2.5λ-phase mask. The deconvolved images of the experimentally acquired DOF target (illumated with the red LED) are given above and below for comparison.

Fig 8

 8 Fig 8 Images of an electronic card (illumated with the red LED) obtained with the optical system containing (a) the neutral plate, (b) the 1λ-phase mask, (c) the 2λ-phase mask and (d) the 2.5λ-phase mask. Images (b), (c) and (d) have been deconvolved with the Wiener filter. Histogram of all these images has been equalized for better visualization.

Fig 9

 9 Fig 9 Zoom on images of an electronic card (illumated with the red LED) obtained with the optical system containing different masks after deconvolution. Histogram of these images has been equalized for better visualization. (a)-(b) are zooms for the neutral plate, (c)-(d) for the 1λ-mask, (e)-(f) for the 2λ-mask, (g)-(h) for the 2.5λ-mask.

Fig 10

 10 Fig 10 CodeV R computed CTF curves at 15 lp.mm -1 as a function of object defocusing of the optical system containing the neutral plate, with a -0.31λ spherical aberration plate, a -0.62λ spherical aberration plate, or no aberration plate. Corresponding images of the DoF target (illumated with the red LED) are presented.

Fig 11

 11 Fig 11 CodeV R computed CTF curves at 15 lp.mm -1 as a function of object defocusing of the optical system containing the 2λ-phase mask (simulated with its defaults), with a -0.31λ spherical aberration plate, a -0.62λ spherical aberration plate, or no aberration plate. Corresponding deconvolved images of the DoF target (illumated with the red LED) are presented.

Fig 12

 12 Fig 12 Details of the objects of interest in the real scene.

Fig 13

 13 Fig 13 Images of a common daisy and a buttercup scene (illumated with the red LED) with the hybrid optical system, when (a) there is the neutral plate in the optical system and no deconvolution (b) there is the 2λ-phase mask in the optical system and a deconvolution. (c) and (d) are the same as (a) and (b) but with a spherical aberration plate of -0.31λ. (e) and (f) are the same as (a) and (b) but with a spherical aberration plate of -0.62λ.

2 π

 2 . Therefore, one has the following relation between |s(ν 0 )| and the MTF | hφ ψ (ν 0 )|:

  , l) exp -2iπ [(k -1)ν 0 .px . cos θ + (l -1)ν 0 .px . sin θ]

  2

Acknowledgments

The authors whish to thank the microfabrication platform at Thales Research and Technology Palaiseau and especially Julie Cholet and Etienne Eustache. We also thank Jean-René Rullier for the optomechanical housing design. The work reported in this paper is supported in part by the Agence de l'Innovation de Défense (AID) that provides half of a PhD fellowship to Alice Fontbonne.

Alice Fontbonne received her MS degree in optics from Institut dOptique Graduate School. She is now a PhD fellow at Institut dOptique Graduate School (Palaiseau, France) and works on joint design of optics and image processing as a new approach to high level optical design.

Hervé Sauer is a former student of the École Normale Supérieure de Fontenay/St-Cloud. He got the Agregation degree in physics in 1985, and a MSc and PhD in physics from Université Paris Sud-XI (Orsay) in 1985 and 1990 respectively. He is an associate professor at Institut d'Optique Graduate School (Palaiseau) since 1990. His main research topics deal with simulations of optical systems and optical system design.

Caroline Kulcsár is full professor at Institut d'Optique Graduate School (IOGS). After graduating from the École Supérieure d'Informatique-Électronique-Automatique (Paris) and from the control theory, image processing (ATS) master at University Paris 11-Orsay in 1991, she obtained her PhD in 20 1995 from University Paris 11. From 1999 to 2012, she was an assistant professor at L2TI-Université Paris 13. She is now with the Imaging and Information group at the Laboratoire Charles Fabry-CNRS-IOGS, University Paris Saclay. Her research activities include image sequences processing, codesign of imaging systems, and identification and control of stochastic dynamical systems, with applications in particular to adaptive optics systems for astronomy and retinal imaging.