Nina Colitti
email: nina.colitti@inserm.fr

Franck Desmoulin

Wafae Labriji

Alice Le Friec

Lorenne Robert

Amandine Michaux

Fabrice Conchou

Carla Cirillo

Isabelle Loubinoux

FIVE-WEEK INTRANASAL NERVE GROWTH FACTOR TREATMENT IS SAFE AND FAVORS BRAIN NEUROGENESIS

whether they are published or not. The documents may come

Introduction Materials & Methods

Malonate

Donnan GA et al., Stroke, 2008 ;Arvidsson A et al., Nat Med, 2002 ;Khelif Y et al., Theranostics, 2018 ;Zhu W et al., Drug Deliv, 2011 ;R. G. THORNE et al., Neuroscience, 2004 ;Davoust C et al., Stem Cells Res Ther, 2017

Results

 Stroke is the leading cause of acquired severe motor disability in adults.  The brain has limited ability to reconstruct itself by generating new neurons from stem cells.  New neurons survival is too low (0.2%) to induce recover.  Low survival is also due to the lack of growth factors.

This study focuses on Nerve Growth Factor (NGF) : neuroprotective/anti-inflammatory/stimulating neurogenesis effects.

The intranasal (IN) pathway was chosen as delivery route.

Malonate was used to induce M1 cortex and adjacent structure lesion in rats (n=19)

Sensorimotor function was evaluated by:

Conclusion

In this preclinical study, we show that five-week intranasal NGF treatment is safe and promotes tissue regeneration with a significantly higher proportion of neurons observed twelve weeks after brain injury.

  1/ grip strength test -2/ neurological severity score assessment -3/ limb-use asymmetry test Lesion volume and RT identification were evaluated by: -MRI T2 weighted images -Nissl staining coloration (RT calculated on Case viewer software ; % of RT is normalized to the size of the lesion) -Immunostaining: GFAP (glial scar, astrocytes) Iba1 (microglia) PDGFRb (vessels, pericytes) BrdU (proliferating cells) Doublecortin (neurons progenitors) beta 3 tubulin (immature neurons) NeuN (mature neurons)  MRI T2 weighted might be a reliable biomarker for reconstructed tissue evaluation  Clusters of BrdU+/DCX+ cells in RT demonstrates a de novo tissue  Long-term NGF treatment promotes tissue reconstruction and remodeling  NGF induces a significant increase in the percentage of mature neurons in the reconstructed tissue compared to PS (p=0.0043)  Five-week NGF treatment is safe; however ten-week treatment retarded motor recovery (p=0.01) • First study with long-term NGF treatment • Reconstructed tissue characterized for the first time at 3 month post injury • The nose-to-brain pathway is a valid strategy for repeated and non-invasive administrations : Hope for chronic stroke treatment Treatment: IN administration of NGF (n=9) Control: IN administration of Physiological Saline (PS ; n=9)


Antonin Gilles

Pierre Le Gargasson

Holographic near-eye display with real-time embedded rendering

Keywords: CCS Concepts:, Hardware → Emerging technologies holography, near-eye displays, augmented reality, computational displays

A C B

Fig. 1. (A) Proposed holographic near-eye display prototype. (B, C) Optical reconstructions of holograms captured through the prototype with a camera focusing on the foreground, illuminated using (B) laser diodes and (C) superluminescent diodes. Compared to laser diodes, superluminescent diodes significantly reduce speckle noise artifacts while preserving the sharpness of in-focus regions in the reconstructed images.

We present a wearable full-color holographic augmented reality headset with binocular vision support and real-time embedded hologram calculation. Contrarily to most previously proposed prototypes, our headset employs high-speed amplitude-only microdisplays and embeds a compact and lightweight electronic board to drive and synchronize the microdisplays and light source engines, allowing full-color holographic images to be displayed at 60 Hz. In addition, to enable a standalone usage of the headset, we developed a real-time hologram rendering engine capable of computing holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform. Finally, we provide a comparison of the e ciency of laser diodes and superluminescent diodes for the reduction of speckle noise, which greatly a ects the reconstructed image's quality. Experimental results show that our prototype enables full-color holographic images to be reconstructed with accurate focus cues and reduced speckle noise in real-time.

INTRODUCTION

Augmented reality (AR) has drawn considerable attention from the scienti c community during the last decade [START_REF] Billinghurst | A Survey of Augmented Reality[END_REF]]. Indeed, with the recent advances in Head-Mounted Display (HMD) devices, AR may nd application in many elds, including manufacturing [START_REF] Nee | Augmented reality applications in design and manufacturing[END_REF], architecture [START_REF] Chi | Research trends and opportunities of augmented reality applications in architecture, engineering, and construction[END_REF], medicine [START_REF] Barsom | Systematic review on the e ectiveness of augmented reality applications in medical training[END_REF] and education [START_REF] Antonioli | Augmented Reality Applications in Education[END_REF].

Unfortunately, conventional near-eye glasses -such as Microsoft HoloLens 2 ® -are based on Stereoscopy, which fails to create a natural and realistic depth illusion. Indeed, Stereoscopy does not reproduce all the Human Visual System (HVS) depth cues perceived in natural vision. In particular, it cannot provide the accommodation stimulus because the viewer has to focus on a xed focal plane whose depth does not match the actual location of perceived objects. As a consequence, it is often impossible to focus on real and virtual objects simultaneously, which signi cantly complicates natural interaction with virtual content. Moreover, it creates the Vergence-Accommodation Con ict (VAC) provoking eye-strain and headaches [START_REF] David | Vergence-accommodation con icts hinder visual performance and cause visual fatigue[END_REF]]. To solve these limitations, several alternative three-dimensional (3D) display technologies have been proposed for AR glasses, including varifocal [START_REF] Akşit | Neareye Varifocal Augmented Reality Display Using See-through Screens[END_REF]], multifocal [START_REF] Zabels | AR Displays: Next-Generation Technologies to Solve the Vergence-Accommodation Con ict[END_REF]], focal surfaces [START_REF] Matsuda | Focal Surface Displays[END_REF], light eld [START_REF] Ueno | Super multi-view near-eye display to solve vergence-accommodation con ict[END_REF] and holographic displays [START_REF] Chang | Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective[END_REF]]. Among these techniques, Holography is often considered as the most promising, since it provides all HVS depth cues without causing eye-strain [START_REF] Blanche | Holography, and the future of 3D display[END_REF]]. To create the depth illusion, a hologram di racts an illuminating light beam to give it the shape of the light wave that would be emitted, transmitted or re ected by a given scene. As a consequence, the viewers perceive the scene as if it were physically present in front of them.

Thanks to its attractive 3D visualization features, several holographic AR display prototypes were proposed in the last few years. An in-depth review of these works can be found in [START_REF] Chang | Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective[END_REF][START_REF] Park | Holographic techniques for augmented reality and virtual reality near-eye displays[END_REF]. However, a truly wearable full-color and binocular holographic near-eye display prototype with real-time embedded hologram calculation has never been proposed in the literature. Indeed, as detailed in Section 2, state-of-the-art prototypes still su er from several challenges that need to be tackled to build such a device. These limitations include the reproduction of full-color images with binocular vision support, the overall system miniaturisation, as well as the development of an embedded hardware calculation engine for real-time hologram rendering.

While solutions have been proposed in the literature to tackle each of these challenges independently, in this paper we demonstrate the non-trivial combination of these components into the rst fully integrated and wearable holographic AR headset with embedded hardware hologram calculation.

STATE-OF-THE-ART CHALLENGES

In the following, we describe the state-of-the-art challenges we tackled to design our holographic near-eye display prototype.

Reproduction of color images. Displaying full-color holographic images on compact headsets is still a challenging topic, and most published prototypes were therefore monochromatic [START_REF] Gao | Compact see-through 3D head-mounted display based on wavefront modulation with holographic grating lter[END_REF]Murakami et al. 2017;[START_REF] Park | Optical see-through holographic near-eyedisplay with eyebox steering and depth of eld control[END_REF]. A common approach is to synchronize a single phase-only Spatial Light Modulator (SLM) with three red, green and blue light sources to illuminate the color channels sequentially [Choi et al. 2022;[START_REF] Jang | Holographic Neareye Display with Expanded Eye-box[END_REF]Lee et al. 2022;Maimone et al. 2017;[START_REF] Zhang | A full-color compact 3D see-through near-eye display system based on complex amplitude modulation[END_REF]]. However, although phase holograms have a better di raction e ciency than their amplitude-only counterparts, they require time-consuming optimization algorithms to achieve an equivalent level of imaging quality, preventing their real-time embedded hardware calculation.

To overcome this limitation, in this paper we employ amplitudeonly LCoS microdisplays of resolution (2048 × 1024), capable of displaying holograms with icker-free color reproduction at 60 Hz. To assess their suitability for holographic near-eye displays, we quantitatively compare their imaging quality and energy e ciency with numerically reconstructed phase-only holograms in Section 5.

Wearable near-eye display design. Another issue of holographic near-eye displays is their overall system size and weight. Indeed, since the development of holographic near-eye displays is still an ongoing research topic, it is not surprising that most published works were limited to bulky and heavy benchtop prototypes [START_REF] Duan | Full-color see-through near-eye holographic display with 80° eld of view and an expanded eye-box[END_REF][START_REF] Park | Optical see-through holographic near-eyedisplay with eyebox steering and depth of eld control[END_REF]. Several compact headsets were developed using a miniaturized or folded optical path design to reduce the system size and weight [START_REF] Gao | Compact see-through 3D head-mounted display based on wavefront modulation with holographic grating lter[END_REF][START_REF] Gopakumar | Un ltered holography: optimizing high di raction orders without optical ltering for compact holographic displays[END_REF][START_REF] Jang | Waveguide Holography: Towards True 3D Holographic Glasses[END_REF][START_REF] Jang | Holographic Neareye Display with Expanded Eye-box[END_REF][START_REF] Kim | Holographic Glasses for Virtual Reality[END_REF][START_REF] Kim | Expanded Exit-Pupil Holographic Head-Mounted Display With High-Speed Digital Micromirror Device[END_REF]Maimone et al. 2017;Murakami et al. 2017;Yoneyama et al. 2018;[START_REF] Zhang | A full-color compact 3D see-through near-eye display system based on complex amplitude modulation[END_REF]]. However, none of them embedded the light source and microdisplay driving electronics, which is required for wearing them as HMDs.

In this work, we overcome this engineering challenge by designing a compact and lightweight electronic board to process the input holographic video stream, to drive and synchronize both microdisplays and light source engines as well as to measure and send the interpupillary distance to the hologram rendering engine. This electronic board is directly embedded onto the headset.

Embedded hardware hologram calculation. To enable a standalone usage of the headset, holograms must be computed in real-time using compact embedded calculation hardware, which is still very challenging due to the large amount of data to process. In [START_REF] Yamamoto | Special-purpose computer for electroholography in embedded systems[END_REF], the authors developed a compact holographic computer using a Xilinx Zynq UltraScale+ MPSoC with an ARM CPU and a FPGA on a single chip. They were able to compute full-HD monochrome holograms at 15 Hz for a scene containing 6500 points. In [START_REF] Kim | A Single-Chip FPGA Holographic Video Processor[END_REF]], a layer-based method was implemented on a FPGA to compute full-HD color holograms at 15 Hz. In [START_REF] Shi | End-to-end learning of 3D phaseonly holograms for holographic display[END_REF] the authors were able to compute full-HD color holograms at 5 Hz on an iPhone 13 Pro using a deep-learning based approach. Finally, in [START_REF] Gilles | Real-time embedded hologram calculation for augmented reality glasses[END_REF]], the author used a NVIDIA Jetson AGX Xavier embedded platform to compute a monocular full-color hologram of resolution (2048 × 1024) at more than 50 Hz with an object-based pre-computed angular spectrum approach [START_REF] Gilles | Real-time computer-generated hologram calculation using pre-computed angular spectra[END_REF]. Nevertheless, the calculation should be further accelerated for our binocular headset prototype, which requires two holograms for the left and right eyes of the viewer to be computed in real time.

To overcome this challenge, we propose a real-time hologram rendering engine capable of computing binocular color holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform, described in Section 4. Speckle noise reduction. Another limitation of holographic displays is the presence of signi cant speckle noise in the reconstructions, which strongly degrades their visual quality. Since speckle noise is due to the high degree of coherence of lasers used to illuminate the hologram, several authors proposed to use partially coherent light sources. In particular, a study published in [Deng and Chu 2017] demonstrated that while the speckle contrast depends on the temporal coherence of the light source, the image sharpness is directly proportional to its spatial coherence. As a consequence, the authors showed that superluminescent diodes (SLEDs), having high spatial and low temporal coherence values, were ideal for holographic display applications. This result was con rmed on holograms of still scenes using a benchtop opaque virtual reality (VR) setup in [START_REF] Peng | Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration[END_REF], demonstrating that SLEDs are able to e ciently reduce speckle noise artifacts compared to coherent lasers while improving the image sharpness over LEDs.

In this paper, we experimentally validate that the image quality enhancement brought by SLEDs is still clearly noticeable in the case of dynamic holographic videos, even when using a compact and wearable see-through AR headset. To the best of our knowledge, this has never been demonstrated before.

Field-of-view and eye-box size. Finally, one of the most important challenges of holographic near-eye displays is to provide a large eld-of-view and eye-box size to the viewer. Indeed, as detailed in the supplementary document, the eld-of-view is restricted by the SLM resolution and cannot be increased without shrinking the eye-box. Consequently, our headset provides a small eld-of-view and eye-box size that do not really support any pupil movement, thus requiring six adjustment wheels to t the position of the user's eyes. As further discussed in Section 6, SLMs with a much higher resolution are necessary to provide a eld-of-view and eye-box size as large as conventional stereoscopic AR displays. 1), a camera (2) to track the position of the user, as well as six adjustment wheels to modify the interpupillary distance (4) and set the position of the optical system in depth ( 5) and height (6). To measure the interpupillary distance, a potentiometer (3) is mounted on the beamspli er combiners. For each eye of the user, the optical system includes an amplitude-only LCoS microdisplay (7), a light source module (8), a beam expander (9), as well as a relay optics system (10). Finally, a compact electronic board ( 11) is embedded onto the headset to drive and synchronize the microdisplays with the light source modules for color reproduction.

HEAD-MOUNTED DISPLAY OVERVIEW

Figure 2 shows our proposed holographic head-mounted display prototype, which comprises a mechanical structure, an optical system and a compact electronic board.

Mechanical structure. The mechanical elements of the headset were designed using the Autodesk Fusion 360 software and manufactured by a Zortrax M300 Plus 3D printer with black Z-HIPS plastic lament. They enable the electronic board and optical components to be held on the head support, and include six adjustment wheels to modify the interpupillary distance and set the position of the optical system in depth and height. To measure the interpupillary distance, a potentiometer is mounted on the beamsplitter combiners. When the adjustment wheels are operated, the potentiometer moves accordingly and its value is sent to the hologram rendering engine through a USB-C port to update the calculation parameters. A camera is also embedded onto the headset to track the position and orientation of the user. Overall, the headset occupies a total volume of 222 mm × 282 mm × 323 mm and weights 2.1 kg.

Optical system. To support binocular vision, the optical systems corresponding to the left and right eyes of the viewer are constructed symmetrically using o -the-shelf components held with aluminum lens tubes of one inch diameter. Figure 3 shows the optical path of the right eye, which includes a RaonTech RDP551F LCoS microdisplay of resolution ( × ) = (2048 × 1024) with a pixel pitch of = 6.3 m and a refresh rate of 180 Hz, a red, green and blue (RGB) light source module, a beam expander to enlarge the illuminating light beam diameter, as well as a relay optics system to remove the zero di raction order and twin image artifacts and increase the eld-of-view. To evaluate and compare their impact on the imaging quality of full-color holograms of dynamic 3D scenes, we use a laser diode (LD) and a SLED RGB light source modules purchased from Exalos. While these two modules have the same red, green and blue wavelengths of 636 nm, 510 nm and 455 nm, respectively, LDs have a much higher temporal coherence than SLEDs, whose bandwidth full width at half maximum (FWHM) is more than ve times larger. Overall, the horizontal and vertical eld-of-view are given by for an eye-box size of ( × ) = (4 mm × 2 mm). It must be noted that the eld-of-view in the vertical direction is divided by two compared to the horizontal direction because the spatial lter removes the lower half of the hologram Fourier spectrum to suppress the zero di raction order and twin image artifacts [Bryngdahl and Lohmann 1968]. The implementation details of the optical system are provided in the supplementary document. Electronic board. Figure 4 shows the functional architecture of the compact electronic board (11), which includes a Mini DisplayPort (mDP) input, an USB-C input/output port, a front-end controller, two microdisplay controllers and a Microcontroller Unit (MCU). All the electronic components are powered by the embedded hologram rendering hardware through the USB-C port with a supply voltage of 5 V and a maximum current of 632 mA. First, the holograms computed by the hologram rendering engine are sent to the frontend controller through the mDP input as a color and binocular video stream of resolution (4096 × 1024). The front-end controller splits the holograms corresponding to the left and right eyes of the viewer into two video streams of resolution (2048 × 1024) which are sent to the microdisplay controllers. These modules sequentially plot the red, green and blue hologram channels onto the microdisplays at a frame rate of 180 Hz and drive the light source modules accordingly for time-multiplexing color reproduction.

EMBEDDED HOLOGRAM RENDERING ENGINE

In this section, we describe the proposed embedded hologram rendering engine. To compute the holograms corresponding to the left and right eyes of the user, we use a layer-based hologram synthesis method which comprises four steps, detailed in the following.

2D-plus-depth rendering. The rst step of our proposed method is to capture the 3D scene geometry from the user's viewpoint as a set of depth layers parallel to the hologram plane. To track the position and orientation of the user with respect to the scene, we used the ducial marker pipeline of the open-source SolAR framework [START_REF] Duong | AR Cloud: Towards Collaborative Augmented Reality at a Large-Scale[END_REF]. This pipeline detects the ducial marker image in the real-time video stream captured by the camera embedded onto the headset and estimates the pose of the camera in relation to the coordinate system of the marker.

Then, 2D-plus-depth perspective projections of the scene are synthesized from the positions of the observer's eyes using two virtual cameras separated by the measured interpupillary distance. Each virtual camera has a resolution of ( × ) and a horizontal and vertical eld-of-view of red , green , blue for the red, green and blue channels, respectively. As stated previously, the perceived eld-of-view in the vertical direction is divided by two compared to the horizontal direction because of the relay optics spatial lter. As a consequence, the lower halves of the color and depth images and captured by each camera are rendered with black pixels. Since the depth maps are encoded as 8-bits gray level images, the scene geometry is thus naturally sampled as a set of 256 evenly distributed parallel depth layers.

Depth map clustering. To accelerate the hologram calculation with a minimal decrease of visual quality, the second step of our method aims at reducing the number of depth layers. To this end, the depth map values of each virtual camera are clustered into a set of ≤ 256 depth intervals [ ; +1 [, with ∈ {0, ..., -1}. These intervals are computed using the k-means clustering algorithm, which is described in the following.

First, the depth map histogram H is computed and the depth intervals are initialized as

           0 = 0, = 256
+1 = min        = H [ ] >        = 0, ..., - 2
(1) where ⌈ ⌉ is the smallest integer greater than or equal to . The depth clusters are then progressively re ned through a sequence of assignment and update steps. During the assignment step, the depth intervals are modi ed such that

+1 = min | -| > | -+1 | for = 0, ..., -1 (2)
where is the average depth value of each cluster , which is then re-computed during the update step. The assignment and update steps are repeated until the clusters no longer change. Finally, for each depth interval, a layer is de ned to operate as a surface source of light emitting a complex wave sampled on a regular grid of resolution ( × ), such that

( , ; ) = ( , ; ) exp ( ( , )) if ( , ) ∈ [ ; +1 [ 0 otherwise, (3)
where is the color channel and ( , ) ∈ [0; 2 [ is the initial phase, set to a uniform random value to render a di use scene. This layer is located at depth

= 255 ( max -min ) + min , (4)
where min and max are the minimal and maximal depths of the scene, respectively.

Relay optics compensation. Once the depth layers have been initialized, the third step of our method aims at compensating the scene geometry distortion induced by the relay optics system. Indeed, the converging lenses L2 and L3 deform the geometry of the scene reconstructed by the hologram, and the position of each depth layer must be modi ed accordingly during the hologram calculation to compensate this e ect. We call 2 the distance between the microdisplay and L2, 3 the distance between L2 and L3, 2 and 3 the focal lengths of L2 and L3, respectively. According to the thin lens equation, to compensate the distortion induced by L2 and L3, the depth of layer should be modi ed to

ˆ = 2 ( 3 -ˆ ′ ) 3 -ˆ ′ -2 -2 , (5)
where

ˆ ′ = 3 ( -) --3 (6)
is the intermediate compensated layer depth and is the exit pupil position, given by

= - (( 2 + 3 ) 2 -2 3 ) 3 ( 2 -2 ) 3 -( 2 + 3 ) 2 + 2 3 . (7)
Layer-based di raction. In the last step of our method, the left and right holograms are computed from their corresponding depth layers. To this end, the light waves scattered by the compensated depth layers are numerically propagated towards the hologram plane and summed up to get the complex-valued object wave

( , ; ) = -1 =0 P ˆ { } ( , ; ), (8)
where P ˆ stands for the Fresnel di raction of light between two parallel planes separated by a distance ˆ , given by

P ˆ { } = 2 ˆ ˆ ˆ ( 2 + 2 ) F ( , ; ) ˆ ( 2 + 2 ) . (9)
In Eq. ( 9), is the wavelength of color channel and F is the forward Fourier transform. The amplitude-only hologram is nally obtained by shifting, scaling and quantizing the real part of on

= 256 discrete levels, such that ( , ; ) = (ℜ { ( , ; )} , , Δ) , (10)
where ℜ { } stands for the real part of , normalized between -1 and +1, is a uniform mid-rise quantizer, given by (11) and Δ refers to the input quantizer range clipping value, which controls the balance between the granular and overload distortions.

( , , Δ) =              0 if < -Δ, ( + Δ) 2Δ if -Δ ≤ < Δ, - 1 otherwise.
Embedded hardware implementation. To enable a standalone usage of the headset, we implemented the proposed hologram synthesis method on an NVIDIA Jetson AGX Orin embedded platform, which was released in 2022 as a successor to the Jetson AGX Xavier. It comprises a 12-core ARM Cortex-A78AE v8.2 64-bit CPU, a 2048core Ampere GPU and 32 GB of DDR5 memory on a single chip of size 100 mm×87 mm. While having ve times less CUDA cores than most recent desktop GPUs such as the NVIDIA 3080Ti, its compact size is perfectly suitable for head-mounted display applications. To facilitate the application deployment, we used the development kit operating Ubuntu 20.04 LTS 64-bit.

In our implementation, the 2D-plus-depth perspective projections of the scene are synthesized using the OpenGL 4.6 API, and all the hologram calculation steps are performed on the GPU using the CUDA application programming interface. To compensate for the reduced number of CUDA cores compared to a desktop GPU, the inner products in Eq. ( 9) are computed using 16-bit precision operations with one CUDA thread per sample. The Fourier transforms corresponding to the three color channels are then computed simultaneously using the CUDA cuFFT library with batched transforms to accelerate the calculation.

EXPERIMENTAL RESULTS

For the experiments, we used three input synthetic scenes: Animals, containing three animated models of a fox, a rabbit and a bear, Circuit, representing a race track with a snowy peak in the background, and Village, depicting a cartoon settlement of wooden mountain chalets. While Animals is representative of AR scenes showcasing distinct and isolated 3D models, Circuit and Village are typical VR scenes spanning a continuous and large depth range.

Calculation time. To evaluate the performance of our proposed hologram synthesis method in terms of calculation time, we implemented it on two di erent platforms: the NVIDIA Jetson AGX Orin embedded platform and a desktop computer employing an Intel Core i9-9900X CPU operating at 3.50 GHz, a main memory of 32 GB, as well as a GPU NVIDIA GeForce RTX 2080Ti.

Figure 5 shows the average calculation times and frame rates for the computation of binocular and colorful holograms of resolution (2048 × 1024) depending on the number of depth layers . We can see that for both implementations the calculation time is proportional to the number of depth layers. Using the desktop computer, the calculation time ranges between 4.7 ms and 16.6 ms for one to eight layers, corresponding to frame rates between 212.8 Hz and 60.2 Hz. On the other hand, since the NVIDIA Jetson AGX Orin contains less than half the number of CUDA cores than the NVIDIA GeForce RTX 2080Ti, the embedded hardware platform is slower than the desktop computer. For a single layer, the binocular hologram calculation frame rate is 73.5 Hz, which is higher than the microdisplays refresh rate. For two and three layers, the holographic video stream is computed at a frame rate of 48.1 Hz and 35.6 Hz, respectively, which is still su cient to provide a real-time AR experience without ickering. However, between four and eight layers, the calculation frame rate drops under 28.2 Hz, creating ickering in the reconstructions. For this reason, in the following we use only three depth layers. Power consumption. To measure the instantaneous power consumption of the headset prototype and embedded hologram rendering engine, we used the jtop command of the jetson-stats monitoring package [START_REF] Ra | Jetson stats[END_REF]]. This tool reported a total CPU, GPU and headset power consumption of 12 W, regardless of the number of layers. Consequently, with a Lithium-Ion battery having a capacity of 10000 mAh and a voltage of 3.7 V, our headset can be used autonomously for more than three hours. Overall, these experimental results demonstrate that the proposed embedded hologram rendering engine can be used in holographic AR applications.

Imaging quality and energy e ciency assessment. To assess the suitability of amplitude holograms for near-eye displays, we compared their imaging quality and energy e ciency with phase-only counterparts computed from the complex-valued object wave given in Eq. ( 8) using three di erent methods:

• the amplitude discard method, in which only the phase of the object wave is kept and quantized on 256 levels, • the double phase method described in [Maimone et al. 2017],

where each object wave sample is encoded using two adjacent phase values quantized on 256 levels, and • the binary phase method, in which the object wave amplitude is discarded and its phase is quantized on two levels. Iterative phase optimization algorithms were not considered in this study because their high computational burden is incompatible with real-time embedded hardware implementations.

To avoid any bias from the di erences in pixel pitches and resolutions or by the imperfections of the optical system, we numerically simulated their optical reconstructions on SLMs with a resolution of (2048 × 1024) and a pixel pitch of = 6.3 m. However, for double phase holograms we simulated a SLM with a resolution of (2048 × 2048) and non-squared pixels of size ( × /2). In addition, to simulate their time-multiplexing display on binary SLMs having a refresh rate of 4.5 kHz per channel (such as the M150 from Forth Dimension Displays), we computed 25 binary phase holograms for each scene using di erent initial random phase distributions in Eq. (3) and averaged their numerical reconstructions. It must be noted that we simulated the necessary spatial ltersi.e. a singlesideband lter for the amplitude-only and binary phase and a pupil aperture for double phase holograms -in the reconstructions.

To evaluate the imaging quality of amplitude and phase-only holograms, we computed the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) [Wang et al. 2004] of their simulated optical reconstructions with respect to those obtained from the complex-valued object wave given in Eq. ( 8). While the PSNR estimates the absolute error between two numerically reconstructed scene images, the SSIM is a measure of the structural similarity between two images that takes into account their luminance, contrast and structure and assigns a similarity index between 0 and 1, where 1 indicates identical images.

Figures 6a and6b show the PSNR and SSIM of the numerically reconstructed amplitude holograms computed from Circuit with three depth layers, depending on the input quantizer range clipping value Δ used in Eq. ( 11). For the sake of comparison, the corresponding metrics of phase-only holograms are plotted in dashed lines. As shown in these gures, the PSNR of amplitude-only holograms increases with Δ to reach a threshold value of 40.2 dB for Δ = 0.5. From Δ = 0.16, it exceeds the PSNR of phase holograms, which are of 25.2, 27.6, and 27.7 dB for the binary phase, double phase and amplitude discard methods, respectively. Similarly, the SSIM of amplitude holograms increases with Δ to reach a threshold value of 0.97 for Δ = 0.5. Regardless of Δ, it exceeds the SSIM of phase holograms, which are of 0.30, 0.40, and 0.62 for the binary phase, double phase and amplitude discard methods, respectively. Interestingly, while the amplitude discard and double phase methods outperform binary phase holograms with respect to PSNR, the latter exhibits superior SSIM performance. This is due to the fact that although the two-level quantization of binary phase holograms induces a signicant error on each pixel of the reconstructed image, the temporal averaging process reduces speckle noise compared to other methods and thus better preserves the structure of the reconstructed images.

Following these results, it is tempting to use the largest Δ value to obtain the best imaging quality. However, as shown in Figure 6c, the energy e ciency -already very low for amplitude holograms -decreases even further as Δ increases, dropping to 1.4% when Δ = 0.5. For comparison, the corresponding double phase hologram energy e ciency is 21.6% due to the spatial ltering by the used pupil aperture, and those of amplitude discard and binary phase holograms are always of 100% and 50% because of the single-sideband lter, respectively. In practice, we found that a value of Δ = 0.4 o ers a good compromise between energy e ciency and image quality. This value is used in the rest of the experiments.

These metrics are con rmed by the numerical reconstructions shown in Figure 7. Indeed, while the binary phase hologram reconstruction exhibits less speckle noise thanks to the temporal averaging process, it presents a non-zero background noise and a signi cantly reduced image contrast compared to the reconstructed complex-valued object wave. The amplitude discard and double phase holograms exhibit a similar behavior, with additional speckle noise. Overall, the numerically reconstructed amplitude hologram with Δ = 0.4 is the closest to the reference complex-valued object wave, con rming the superior imaging quality of amplitude-only holograms over phase holograms in terms of contrast and speckle noise. Further details on this imaging quality and energy e ciency assessment study are given in the supplementary document, together with experimental results for the Animals and Village scenes.

Optical reconstructions. To demonstrate the real-time hologram calculation pipeline from 3D pose estimation to display, we used a virtual scene containing a single animated fox model whose position was set a few centimeters above the ducial marker image. Figure 8 and the supplementary video show the photographs captured through our prototype using the LD module, with a varying ducial marker position and orientation. As shown in this video, the location of the fox is updated in real-time with a small latency by the embedded hologram rendering engine.

To evaluate the reproduction accuracy of the scene geometry and accommodation depth cue, we used the Animals scene with a xed user position. The animated fox, rabbit and bear models were located at a distance of 50 cm, 150 cm and 300 cm away from the user's eyes, respectively, and three real wooden toys -an acrobat, a ring of re and a tamer -were placed at the same distances in front of the headset. Figure 9 and the supplementary video show the optical reconstructions of holograms illuminated by the LD and SLED modules, recorded with a camera focusing on di erent depths. For both light sources, when the camera is focusing on a given 3D model, the corresponding wooden toy is also focused, demonstrating that our prototype accurately provides the accommodation depth cue. In addition, the scene geometry is properly reproduced without distortion thanks to the compensation step of our hologram synthesis method. Finally, compared to LDs, the SLED module signi cantly reduce speckle noise artifacts while preserving the sharpness of in-focus regions in the reconstructed images. For both light sources, the power passing through the exit pupil was measured to be under 0.2 mW, which is compliant with Class 1 laser safety conditions.

To assess the imaging quality of holograms computed from VR scenes Circuit and Village, we replaced the non-polarizing beamsplitter combiner by a mirror. As shown in Figure 10, when the scene spans a continuous depth range, the separation between consecutive depth layers is visible in the optical reconstructions, creating visual artifacts at the layers' boundaries. Therefore, while the layer-based calculation approach is suitable for typical AR content containing only isolated 3D models, it is not ideal for VR scenes with a continuous depth extent.

OPEN CHALLENGES

While our prototype enables full-color holographic images to be reconstructed with reduced speckle noise in real-time, it still su ers from several open challenges which are discussed below.

Headset size and weight. First of all, although the designed compact electronic board and embedded hologram rendering engine enable our prototype to be worn and used autonomously, its large form factor and weight are incompatible with a comfortable prolonged use. This is due to the fact that we only used o -the-shelf optical and optomechanical components to build the optical system. In particular, the prototype's weight and size come largely from the aluminum lens tubes and mounts used to hold and align the optical components. Indeed, these elements are made of aluminum, unlike the mechanical structure of the headset, which was made of plastic. In a future work, we plan to reduce the prototype's weight and size by designing and manufacturing a miniaturized optical system with custom optical and optomechanical components.

Holographic imaging quality. Secondly, although SLEDs enable to reduce speckle noise, our prototype does not reach the imaging quality of consumer stereoscopic near-eye displays. This is mostly due to optical components misalignment as well as geometric and chromatic aberrations. Indeed, since the optical system was built using o -the-shelf components, it was not optimized to reduce these aberrations. In a future work, we plan to improve the imaging quality by designing and manufacturing custom lenses to optimize the optical system in reducing these aberrations.

Field-of-view and eye-box size. Finally, since it cannot be increased without shrinking the eye-box, the eld-of-view of our headset is still too narrow for many practical consumer applications. To solve this trade-o , several consumer AR headsets use eye-box replication waveguides to obtain a wide eld-of-view without vignetting [Kress 2020]. However, these waveguides can only be used if the input eld is collimated, in other words when the image is formed in the far eld. Unfortunately, since holographic images are formed in the near eld, each replicated exit pupil will produce an image at a slightly di erent distance and the same image will be overlapped with di erent focal depths. For this reason, these waveguides cannot be used in the context of holographic displays.

Consequently, to provide a eld-of-view of (53 • × 26.5 • ) with an eye-box of (4 mm × 2 mm), SLMs of resolution (8192 × 4096) are required, resulting in slower hologram calculation frame rates. Alternatively, a promising solution is to use a beam steering element to deviate the illuminating light beam and shift the exit pupil to di erent locations sequentially depending on the user's gaze, as proposed in [START_REF] Jang | Holographic Neareye Display with Expanded Eye-box[END_REF]]. Nevertheless, this requires a precise and low latency eye-tracking system to follow the short and rapid eye movements called saccades, reaching up to 700 • / .

To conclude, while state-of-the-art approaches can be used to improve the headset size, weight and imaging quality, the limited eld-of-view of holographic near-eye displays is still an open challenge that poses a signi cant obstacle to their practical usability.

CONCLUSION

In this paper, we built the rst fully integrated and wearable holographic AR headset with embedded hardware hologram calculation capable of full-color reproduction with binocular vision support. Our proposed prototype embeds all the optical components and driving electronics on a single head support and employs amplitudeonly LCoS microdisplays instead of the commonly used phase SLM. In addition, we proposed and implemented a real-time layer-based hologram calculation method capable of computing binocular color holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform, enabling its standalone usage.

Experimental results show that our prototype enables full-color holographic images to be reconstructed with accurate focus cues in real-time. Thanks to the SLED illumination, speckle noise artifacts are reduced, enhancing the imaging quality of the headset. In a future work, we plan to reduce the prototype's weight and size and improve its imaging quality by designing a miniaturized optical system with custom optical and optomechanical components. In addition, the limited eld-of-view of holographic near-eye displays is still an open challenge that requires further research to be solved.

Field-of-view

Field-of-view Field-of-view

SLED

(c) Focused on the bear.

Fig. 9. Photographs captured through our prototype with a camera focusing on di erent depths, using the LD (top row) and SLED modules (bo om row). When the camera is focusing on a given 3D model, the corresponding real-world object is also focused while other parts of the scene appear blurred, demonstrating that our proposed holographic AR headset provides accurate accommodation depth cues. Compared to LDs, the SLED module significantly reduce speckle noise artifacts while preserving the sharpness of in-focus regions in the reconstructed images (see supplementary video). . Photographs captured through our prototype in virtual reality mode with a camera focusing on di erent depths, for two scenes covering a large depth extent: Circuit (first row) and Village (second row). When scene objects span a continuous depth range, the separation between consecutive depth layers is visible, creating visual artifacts at the layers boundaries. As a consequence, while the layer-based calculation approach is suitable for typical AR content containing only isolated 3D models, it is not ideal for VR scenes with a continuous depth extent.

Holographic near-eye display with real-time embedded rendering

S1 PROPOSED OPTICAL SYSTEM

In the following, we describe the optical system of the proposed holographic head-mounted display. Figure 1 shows the optical path corresponding to the right eye, which includes:

• a RGB light source module, described in Section S1.1,

• a beam expander made of a diverging lens L1 and a converging lens L2 to enlarge the illuminating light beam diameter, described in Section S1.2, • a polarizing beamsplitter and an amplitude-only LCoS microdisplay to display the computed holograms, • a relay optics system comprising two converging lenses L2 and L3 as well as a spatial lter to remove the zero di raction order and twin image artifacts, described in Section S1.3, • and a non-polarizing beamsplitter to superimpose the reconstructed virtual image on the real environment.

S1.1 Light source

To enable light di raction, the holograms must be illuminated by a spatially and temporally coherent light source. While laser diodes (LD) are commonly used in this context, a study published in [Deng and Chu 2017] showed that superluminescent diodes (SLED) are ideal for holographic display applications: thanks to their reduced temporal coherence, they bring less speckle noise in the reconstructed images. To evaluate and compare their impact on the imaging quality of full-color holograms of dynamic 3D scenes, we use two di erent RGB light sources purchased from Exalos: a LD and a SLED modules, whose characteristics are presented in Table 1.

As shown in this table, while these two modules share the same center wavelengths, LDs have a much higher temporal coherence than SLEDs, whose bandwidth full width at half maximum (FWHM) Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for pro t or commercial advantage and that copies bear this notice and the full citation on the rst page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). SA Conference Papers '23, December 12-15, 2023, Sydney, NSW, Australia © 2023 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-0315-7/23/12. https://doi.org/10.1145/3610548.3618179 is more than ve times larger. In addition, their maximum output power of 10 mW per color is su ciently high to compensate for the energy loss due to the low di raction e ciency of amplitude-only modulation. Finally, while the red, green and blue light beams are divergent, they are perfectly aligned at the output of the modules, allowing the use of a single optical path for the three colors.

S1.2 Beam expander

The objective of the beam expander is to enlarge the illuminating light beam at the output of the light source module to illuminate the entire surface of the microdisplay. Its schematic is shown in Figure 2. In this gure, L1 and L2 are two lenses of focal length 1 and 2 , respectively, separated by a distance 1 . The incident beam on L1, located at a distance 0 from the light source, has diameter and a full divergence angle . At the output of the beam expander, the beam diameter must be larger than the microdisplay size . To build the beam expander, we use a diverging lens L1 of focal length 1 = -6 mm located at a distance 0 = 15 mm from the light source module, and a converging lens L2 of focal length 2 = 150 mm. The light beam diameter incident on L1 is measured to be = 0.42 mm, with a full divergence angle of = 0.02 rad. In order to produce a collimated output beam, 1 must be set to

1 = ( 1 - ) 2 - 1 1 - = 145.33 mm. (1)
At the output of the beam expander, the beam diameter is therefore given by

= ( 1 + ) 1 - 1 1 = 13.5 mm. (2)
In our prototype, we use two RaonTech RDP551F LCoS microdisplays with a resolution of ( × ) = (2048 × 1024) and a pixel pitch of = 6.3 m to display the holograms corresponding to the left and right eyes of the viewer. The surface to illuminate is therefore given by ( × ) = (12.9 mm × 6.45 mm), which is smaller than the output beam diameter .

S1.3 Relay optics

Figure 3 shows the schematics of the relay optics system, whose objective is to lter out the zero di raction order and twin image artifacts and to increase the eld-of-view . In this gure, L2 and L3 are two converging lenses of focal length 2 and 3 , respectively, separated by a distance 3 . To reduce the optical path size, the L2 lens, located at a distance 2 from the microdisplay, is shared by the beam expander and relay optics systems.

In addition, a spatial lter is located in the focal plane of L2 to remove the unwanted di raction orders using the single-sideband technique [Bryngdahl and Lohmann 1968]. In this technique, only the object light rays that are propagating downwards -i.e. the higher half of the hologram Fourier spectrum -are computed, by rendering the lower halves of the color and depth images captured by each camera with black pixels. Then, during the reconstruction step, the spatial lter removes the zero-order and twin-image artifacts by cutting o the lower half of the hologram Fourier spectrum. For the sake of clarity, this lter is not shown in Figure 3.

The exit pupil position, also called eye relief, is given by At this position, the perceived eld-of-view and in the horizontal and vertical directions, respectively, are given by

= - (( 2 + 3 ) 2 -2 3 ) 3 ( 2 -2 ) 3 -( 2 + 3 ) 2 + 2 3 . (3)
= - (( 2 -2 ) 3 -( 2 + 3 ) 2 + 2 3 ) 2 3 and = 2 , (4)
where is twice the maximum di raction angle, given by

= 2 arcsin 2 , (5)
and is the microdisplay pixel pitch. The eld-of-view in the vertical direction is divided by two compared to the horizontal direction because of the single-sideband spatial lter.

As shown in Eq. ( 4), the eld-of-view can be widened by using a smaller focal length 3 or by increasing 3 . However, these parameters have an impact on the exit pupil size, also called eye-box, which de nes the region within which the whole image can be viewed without vignetting. Indeed, the eye-box in the horizontal and vertical directions is given by

= - 2 3 ( 2 -2 ) 3 -( 2 + 3 ) 2 + 2 3 . (6)
As a consequence, when 3 decreases or 3 increases, the eye-box is shrunk, creating vignetting when the eyes of the user move. This trade-o between the eye-box size and eld-of-view also exists in the case of conventional stereoscopic AR displays. In practice, an eyebox size of around 4 mm is often used in the literature [Murakami et al. 2017;Yoneyama et al. 2018].

In our holographic AR headset prototype, we use a converging lens L2 of focal length 2 = 150 mm located at a distance 2 = 10 mm from the microdisplay and a converging lens L3 of focal length 3 = 50 mm located at a distance 3 = 212 mm from L2. The horizontal and vertical eld-of-view are therefore given by red = 18.66

• green = 14.96 • blue = 13.34 • (7) red = 9.33 • green = 7.48 • blue = 6.67 • (8)
for an eye-box size of ( × ) = (4 mm × 2 mm).

S1.4 Detailed components list

In the following are listed the components required for assembling the proposed optical system, numbered in reference to Figure 1.

(1) Light source modules: Exalos RGB LD and SLED butter y modules with free-space output (2) Lens L1: Thorlabs biconcave lens LD2746-A (3) Lens L2: Thorlabs achromatic doublet AC254-150-A-ML (4) Polarizing beamsplitter: Thorlabs CCM1-PBS251/M (5) Microdisplays: RaonTech RDP551F LCoS panel (6) Spatial lter: custom 3D printed component (7) Mirror: Thorlabs dielectric turning mirror CCM1-E02/M (8) Lens L3: Thorlabs achromatic doublet AC254-050-A-ML (9) Non-polarizing beamsplitter: Thorlabs CCM1-BS013/M These components are held and aligned together using Thorlabs aluminium lens tubes of one inch diameter.

S2 IMAGING QUALITY AND ENERGY EFFICIENCY ASSESSMENT STUDY

In the following, we provide further details on the imaging quality and energy e ciency assessment study, together with additional experimental results.

S2.1 Compared hologram calculation methods

To assess the suitability of amplitude-only holograms for near-eye displays, we compared their imaging quality and energy e ciency with phase-only counterparts computed from the complex-valued object wave using three di erent methods: the amplitude discard, double phase and binary phase methods. In the following, we recall how the object wave and amplitude-only holograms are computed by the proposed embedded rendering engine, and provide additional details on the compared phase-only hologram calculation methods.

Complex-valued object wave. In our proposed embedded hologram rendering engine, the 3D scene geometry is reconstructed from the synthesized 2D-plus-depth images as a sparse set of parallel depth layers whose distance ˆ from the hologram plane is set as to compensate the distortion induced by the relay optics system described in Section S1.3. Then, the light waves scattered by the depth layers are numerically propagated towards the hologram plane and summed up to get the complex-valued object wave

( , ; ) = -1 =0 P ˆ { } ( , ; ), (9)
where is the color channel index, is the number of depth layers, is the light wave scattered by layer , and P ˆ is the Fresnel di raction of light between two parallel planes, given by

P ˆ { } = 2 ˆ ˆ ˆ ( 2 + 2 ) F ( , ; ) ˆ ( 2 + 2 ) . (10)
In Eq. ( 10), is the wavelength of color channel and F is the forward Fourier transform. Since the 2D-plus-depth images of the scene are synthesized from the position of the observer's eyes with perspective projection pinhole cameras, the occlusions are naturally taken into account in the complex-valued object wave for these speci c viewpoints only.

Amplitude-only holograms. Once the object wave has been computed, the amplitude-only hologram is obtained by shifting, scaling and quantizing the real part of on = 256 discrete levels, such that

( , ; ) = (ℜ { ( , ; )} , , Δ) , (11)
where ℜ { } stands for the real part of , normalized between -1 and +1, is a uniform mid-rise quantizer, given by (12) and Δ refers to the input quantizer range clipping value, which controls the balance between the granular and overload distortions. Since they do not modulate the phase of the incident light wave, amplitude-only holograms produce the unwanted zero di raction order and twin image artifacts. To remove them, we use a singlesideband lter in the focal plane of L2, as detailed in Section S1.3. Amplitude discard method. The amplitude discard method is the most trivial way of computing a phase-only hologram from the object wave. In this approach, only the phase of is kept and quantized on = 256 discrete levels, such that

( , , Δ) =              0 if < -Δ, ( + Δ) 2Δ if -Δ ≤ < Δ, - 1 otherwise.
( , ; ) = arg { ( , ; )} 2 , ( 13
)
where arg{ } is the phase angle of . Since they do not produce any unwanted di raction order artifact, phase-only holograms computed using the amplitude discard method do not need any spatial lter in the optical system.

Double phase method. The double phase method [START_REF] Hsueh | Computer-generated double-phase holograms[END_REF]] is an alternative approach for generating phaseonly holograms which involves encoding both the amplitude and phase information of the object wave into two separate phase values. Indeed, each sample ( , ; ) can be represented as the sum of two complex numbers with constant amplitude and variable phase, such that ( , ; ) = 0.5 1 ( , ; ) + 2 ( , ; ) , ( 14)

1 ( , ; ) = arg { ( , ; )} -cos -1 (| ( , ; )|) , ( 15
) 2 ( , ; ) = arg { ( , ; )} + cos -1 (| ( , ; )|) , (16)
where | | is the amplitude of , normalized between 0 and 1.

In our experiments, we simulated the double phase encoding arrangement described in [Maimone et al. 2017], where the pair of phase values 1 and 2 are quantized on = 256 discrete levels and placed adjacently in a checkerboard pattern on the Spatial Light Modulator (SLM). Since this arrangement implicitly creates a high frequency grating, the unwanted di raction orders are simply removed by using a pupil aperture in the focal plane of L2.

Binary phase method. Finally, another approach to compute phaseonly holograms is the binary phase method, in which the object wave amplitude is discarded and its phase is quantized on two levels, such that

( , ; ) = 1 if arg { ( , ; )} > , 0 otherwise. ( 17
)
This method enables to take advantage of the high display frame rates of binary phase modulation SLMs such as Ferroelectric LCoS (FLCoS) [Lee et al. 2022] or Digital Micromirror Devices (DMD) [Choi et al. 2022]. For instance, the M150 FLCoS SLM from Forth Dimension Displays is able to display binary monochromatic images at 4.5 kHz, i.e. 25 times faster than the RaonTech RDP551F LCoS panel used in our prototype. By displaying several holograms computed with di erent initial random phase di users sequentially, the speckle noise can be strongly reduced thanks to time multiplexing.

Nevertheless, the binary quantization process produces the unwanted zero di raction order and twin image artifacts, which need to be removed using a single-sideband lter in the focal plane of L2 as for amplitude-only holograms.

S2.2 Simulation of the optical reconstructions

To compare the imaging quality and energy e ciency of amplitude and phase-only holograms without any bias from the optical system's imperfections, we numerically simulated their reconstructions on SLMs with a resolution of (2048 × 1024) and a pixel pitch of = 6.3 m. However, for double phase holograms, which require twice the number of pixels, we simulated a SLM with a resolution of (2048 × 2048) and non-squared pixels of size ( × /2).

The numerical reconstruction process is shown in Figure 4. For a focus distance of ˆ , the simulated reconstructed image intensity from the object wave is calculated as

( , ; ) = P -1 ˆ { } ( , ; ) 2 , (18)
where P -1 ˆ is the inverse Fresnel di raction, given by

P -1 ˆ { } = ˆ 2 ˆ - ˆ ( 2 + 2 ) F -1 ( , ; ) - ˆ ( 2 + 2 ) .
(19) The corresponding numerically reconstructed image intensity of amplitude and phase-only holograms is computed as

ˆ ( , ; ) = P -1 ˆ G ˆ ( , ; ) 2 , ( 20
)
where ˆ is the complex-valued light wave in the hologram plane and G is the optional ltering operation to remove the unwanted di raction artifacts. For amplitude holograms, ˆ is set to

ˆ ( , ; ) = 4Δ ( , ; ). (21)
Note that the light wave amplitude is scaled by 4Δ instead of 2Δ to compensate for the light energy loss due to the single-sideband lter. For phase holograms, ˆ is given by

ˆ ( , ; ) = exp 2 ( , ; ) , (22)
where is the constant amplitude of channel , whose value is set for each hologram such as to maximize the Peak-Signal-To-Noise Ratio (PSNR) of the numerically reconstructed image intensity.

For amplitude-only, binary phase and double phase holograms, the unwanted di raction artifacts are removed by a spatial lter in the focal plane of L2. This ltering operation is simulated by

G ˆ = F -1 F ˆ ( , ) ( , ) , (23)
where is the spatial lter. For amplitude-only and binary phase holograms, is a single-sideband lter, given by

( , ) = 1 if > 0, 0 otherwise. ( 24
)
For double phase holograms, is a pupil aperture modeled as

( , ) = 1 if | | < 1 2 , 0 otherwise. ( 25
)
Since holograms computed using the amplitude discard method do not need any spatial lter, G is set to the identity function. Finally, to simulate their time-multiplexing display on DMDs or FLCoS having a refresh rate of 4.5 kHz per channel (such as the M150 SLM from Forth Dimension Displays), we computed 25 binary phase holograms for each scene using di erent initial random phase di users and averaged their numerical reconstructions.

S2.3 Imaging quality and energy e iciency assessment

To evaluate the imaging quality of amplitude and phase-only holograms, we computed the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) [Wang et al. 2004] of their simulated optical reconstructions with respect to those obtained from the complex-valued object wave. While the PSNR estimates the absolute error between two numerically reconstructed scene images, the SSIM is a measure of the structural similarity between two images that takes into account their luminance, contrast and structure and assigns a similarity index between 0 and 1, where 1 indicates identical images. To compute these metrics, ˆ and are rst scaled with respect to the maximum value of , and quantized on 256 levels. Then, the PSNR and SSIM metrics are computed for each reconstruction depth independently and their arithmetic mean is calculated.

To evaluate their energy e ciency , we computed the ratio between the illuminating and reconstructed light waves intensities over the hologram area, such that

= ∫ ∫ G ˆ ( , ) 2 ∫ ∫ | | 2 , ( 26
)
where is the illuminating light wave amplitude, given by = 4Δ and = for amplitude and phase-only holograms, respectively. While the energy e ciency of phase-only holograms computed using the amplitude discard method is always equal to 100%, those of binary phase holograms is equal to 50% because of the singlesideband lter used in the numerical reconstructions. Finally, the energy e ciencies of amplitude and double-phase holograms depend on the scene content. gures, the PSNR and SSIM of amplitude-only holograms increase with Δ to reach a threshold value around Δ = 0.5, and exceed those of phase holograms when Δ > 0.16. Interestingly, while the amplitude discard and double phase methods outperform binary phase holograms with respect to PSNR, the latter exhibits superior SSIM performance. This is due to the fact that although the two-level quantization of binary phase holograms induces a signi cant error on each pixel of the reconstructed image, the temporal averaging process reduces speckle noise compared to other methods and thus better preserves the structure of the reconstructed images.

S2.4 Experimental results

Following these results, it is tempting to use the largest Δ value to obtain the best imaging quality. However, the energy e ciencyalready very low for amplitude holograms -decreases even further as Δ increases, dropping around 1.4% when Δ = 0.5. For comparison, the corresponding double phase hologram energy e ciency are of 27.6%, 21.6% and 29.7% for the Animals, Circuit and Village, respectively. In practice, we found that a value of Δ = 0.4 o ers a good compromise between energy e ciency and image quality.

These experimental results are con rmed by the numerical reconstructions shown in Figures 8, 9 and 10. Indeed, we can see that the binary phase hologram reconstructions exhibit signi cantly reduced speckle noise artifacts compared to the amplitude discard and double phase holograms, at the cost of background noise. Overall, the numerically reconstructed amplitude holograms with Δ = 0.4 are always the closest to the reconstructions obtained with the reference complex-valued object waves, and present the best imaging quality in terms of contrast and speckle noise.

  Fig.2. Proposed holographic near-eye display prototype. The mechanical structure of the headset comprises a head support (1), a camera (2) to track the position of the user, as well as six adjustment wheels to modify the interpupillary distance (4) and set the position of the optical system in depth (5) and height (6). To measure the interpupillary distance, a potentiometer (3) is mounted on the beamspli er combiners. For each eye of the user, the optical system includes an amplitude-only LCoS microdisplay (7), a light source module (8), a beam expander (9), as well as a relay optics system (10). Finally, a compact electronic board (11) is embedded onto the headset to drive and synchronize the microdisplays with the light source modules for color reproduction.

Fig. 3 .

 3 Fig. 3. Proposed holographic near-eye display optical system (right eye).

Fig. 4 .

 4 Fig. 4. Functional architecture of the electronic board.

Fig. 5 .

 5 Fig.5. Calculation times and frame rates for the computation of binocular and colorful holograms of resolution (2048 × 1024) using the proposed method, depending on the number of depth layers .

Fig. 6 .

 6 Fig.6. Objective quality assessment and energy e iciency metrics of the numerically reconstructed amplitude holograms computed from Circuit with three depth layers, depending on the input quantizer range clipping value Δ defined in Eq. (11). The PSNR and SSIM metrics were computed with respect to the reference complex-valued object wave given in Eq. (8). For the sake of comparison, the corresponding phase-only holograms metrics are plo ed in dashed lines.

Fig. 7 .

 7 Fig. 7. Numerically reconstructed holograms computed from Circuit with three depth layers, focused on the foreground. Red squares indicate the position of zoom-in details shown in the upper-right corner of each reconstruction.

Fig. 8 .

 8 Fig. 8. Photographs captured through our prototype using the LD module, demonstrating the real-time hologram calculation pipeline from 3D pose estimation to display. The fox position is accurately updated by the hologram rendering engine in real-time and with a very small latency depending on the fiducial marker position and orientation (see supplementary video).

  (a) Focused on the foreground. (b) Focused on the middle ground. (c) Focused on the background.

Fig. 10

 10 Fig.10. Photographs captured through our prototype in virtual reality mode with a camera focusing on di erent depths, for two scenes covering a large depth extent: Circuit (first row) and Village (second row). When scene objects span a continuous depth range, the separation between consecutive depth layers is visible, creating visual artifacts at the layers boundaries. As a consequence, while the layer-based calculation approach is suitable for typical AR content containing only isolated 3D models, it is not ideal for VR scenes with a continuous depth extent.

Fig. 1 .

 1 Fig. 1. Optical system of the right eye.

Fig. 2 .

 2 Fig. 2. Beam expander system.

Fig. 3 .

 3 Fig. 3. Relay optics system.

Figures 5 ,Fig. 4 .

 54 Figures 5, 6 and 7 show the image quality and energy e ciency metrics of the numerically reconstructed amplitude holograms computed from Animals, Circuit and Village with three depth layers, depending on the input quantizer range clipping value Δ used in Eq. (12). For the sake of comparison, the corresponding metrics of phase-only holograms are plotted in dashed lines. As shown in these

Fig. 5 .Fig. 6 .Fig. 7 .

 567 Fig. 5. Objective quality assessment and energy e iciency metrics of the numerically reconstructed amplitude and phase holograms computed from Animals with three depth layers, depending on the input quantizer range clipping value Δ.

Fig. 8 .

 8 Fig. 8. Numerically reconstructed holograms computed from Animals with three depth layers.

Fig. 9 .

 9 Fig. 9. Numerically reconstructed holograms computed from Circuit with three depth layers.

Fig. 10 .

 10 Fig. 10. Numerically reconstructed holograms computed from Village with three depth layers.

  Supplementary document ANTONIN GILLES and PIERRE LE GARGASSON, Institute of Research & Technology b<>com, France GRÉGORY HOCQUET and PATRICK GIOIA, Orange Labs, FranceThis supplementary document describes the implementation of the proposed holographic augmented reality headset optical system, and provides additional details and experimental results on the imaging quality and energy e ciency assessment study.

	ACM Reference Format:
	Antonin Gilles, Pierre Le Gargasson, Grégory Hocquet, and Patrick Gioia.
	2023. Holographic near-eye display with real-time embedded rendering:
	Supplementary document. In SIGGRAPH Asia 2023 Conference Papers (SA
	Conference Papers '23), December 12-15, 2023, Sydney, NSW, Australia. ACM,
	New York, NY, USA, 9 pages. https://doi.org/10.1145/3610548.3618179

Table 1 .

 1 Light source modules characteristics.

	Color Wavelength	Bandwidth FWHM	Power
	Red	636 nm	LD: <1 nm SLED: 6 nm	10 mW
	Green	510 nm	LD: <1 nm SLED: 10 nm 10 mW
	Blue	455 nm	LD: <1 nm SLED: 5 nm	10 mW

ACKNOWLEDGMENTS

This work has been funded by the French government through the National Research Agency (ANR) Investment referenced ANR-A0-AIRT-07. All the 3D models used in this paper were downloaded from the Unity Asset Store (https://assetstore.unity.com).

ACKNOWLEDGMENTS

This work has been funded by the French government through the National Research Agency (ANR) Investment referenced ANR-A0-AIRT-07. All the 3D models used in this paper were downloaded from the Unity Asset Store (https://assetstore.unity.com).