Ulas K Ür Üm

Peter R Wiecha

Rebecca French

Otto L Muskens
email: o.muskens@soton.ac.uk

Deep learning enabled real time speckle recognition and hyperspectral imaging using a multimode fiber array

We demonstrate the use of deep learning for fast spectral deconstruction of speckle patterns. The artificial neural network can be effectively trained using numerically constructed multispectral datasets taken from a measured spectral transmission matrix. Optimized neural networks trained on these datasets achieve reliable reconstruction of both discrete and continuous spectra from a monochromatic camera image. Deep learning is compared to analytical inversion methods as well as to a compressive sensing algorithm and shows favourable characteristics both in the oversampling and in the sparse undersampling (compressive) regimes. The deep learning approach offers significant advantages in robustness to drift or noise and in reconstruction speed. In a proof-of-principle demonstrator we achieve real time recovery of hyperspectral information using a multi-core, multi-mode fiber array as a random scattering medium.

Introduction

Motivated by the need for imaging in complex environments and through opaque media, new techniques for characterizing and controlling multiple scattering are currently seeing a tremendous development [START_REF] Rotter | Light fields in complex media: Mesoscopic scattering meets wave control[END_REF]. This new toolbox opens up directions for controlling light in random media and exploiting it for applications such as imaging and optical information processing. At the same time, improved understanding allows us to retrieve more information from seemingly random scattering fields to see around corners and through opaque media [START_REF] Katz | Looking around corners and through thin turbid layers in real time with scattered incoherent light[END_REF][START_REF] Bertolotti | Non-invasive imaging through opaque scattering layers[END_REF]. Control over light scattering has led to exciting new applications such as programmable multiport optical elements for classical and quantum states [START_REF] Defienne | Nonclassical light manipulation in a multiple-scattering medium[END_REF][START_REF] Huisman | Programmable multiport optical circuits in opaque scattering materials[END_REF][START_REF] Strudley | An ultrafast reconfigurable nanophotonic switch using wavefront shaping of light in a nonlinear nanomaterial[END_REF][START_REF] Park | Scattering Optical Elements: Stand-Alone Optical Elements Exploiting Multiple Light Scattering[END_REF], quantum secure keys [START_REF] Sebastianus | Quantumsecure authentication of a physical unclonable key[END_REF] and compressive sampling imaging systems [START_REF] Liutkus | Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium[END_REF].

The multispectral characteristics of speckle fields have been used successfully in a range of studies to realize speckle spectrometers [START_REF] Cao | Perspective on speckle spectrometers[END_REF][START_REF] Redding | Using a multimode fiber as a high-resolution, low-loss spectrometer[END_REF][START_REF] Mazilu | Random super-prism wavelength meter[END_REF][START_REF] Chakrabarti | Speckle-based spectrometer[END_REF][START_REF] Liew | Broadband multimode fiber spectrometer[END_REF][START_REF] Valley | Multimode waveguide speckle patterns for compressive sensing[END_REF]. By exploiting wavelengthdependent speckle patterns from a multimode fiber, a spectral resolution of picometers in the near-infrared and nanometers in the visible region has been demonstrated [START_REF] Redding | High-resolution and broadband all-fiber spectrometers[END_REF][START_REF] Noel | High-resolution optical spectroscopy using multimode interference in a compact tapered fibre[END_REF]. Next to single-channel spectrometers, multiplexing of spectrally resolved speckle fields into hyperspectral imaging systems is of great interest. Compared to traditional approaches such as Integral Field Spectrometers [START_REF] Hagen | Review of snapshot spectral imaging technologies[END_REF][START_REF] Dwight | Lenslet array tunable snapshot imaging spectrometer (LATIS) for hyperspectral fluorescence microscopy[END_REF], complex media can offer new opportunities for combining broadband transmission with high spectral resolution [START_REF] Kumar Sahoo | Single-shot multispectral imaging with a monochromatic camera[END_REF][START_REF] Wang | Computational multispectral video imaging [Invited[END_REF][START_REF] French | Speckle-based hyperspectral imaging combining multiple scattering and compressive sensing in nanowire mats[END_REF]. In the spatial domain, multimode fibers as well as multi-core fiber bundles are a topic of study for a variety of imaging applications such as remote sensing and endoscopy [START_REF] Choi | Scanner-Free and Wide-Field Endoscopic Imaging by Using a Single Multimode Optical Fiber[END_REF][START_REF] Plöschner | Seeing through chaos in multimode fibres[END_REF][START_REF] Edgar | Two-photon imaging through a multimode fiber[END_REF][START_REF] Porat | Widefield lensless imaging through a fiber bundle via speckle correlations[END_REF].

Recently, a multicore multimode fiber bundle has been used as a frequency characterization element in a high-throughput imaging spectrometer for snapshot spatial and spectral measurements with sub nanometer spectral resolution [START_REF] French | Snapshot fiber spectral imaging using speckle correlations and compressive sensing[END_REF]. A compressive sensing (CS) algorithm was successfully employed to retrieve spectral information. Convex regularization techniques such as CS provide a suitable solution for a number of problems in computational imaging. However, in addition to their high computational cost, the applicability depends on a sparsity assumption and indeed performance is reduced for dense data. Faster data processing and requirement of robustness with respect to noise and drift could benefit from an entirely different approach based on artificial neural networks [START_REF] Villmann | Processing Hyperspectral Data in Machine Learning[END_REF][START_REF] Zhao | Spectral-Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach[END_REF][START_REF] Wang | Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking[END_REF][START_REF] Zhong | Spectral-Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework[END_REF][START_REF] Aptoula | Deep Learning With Attribute Profiles for Hyperspectral Image Classification[END_REF][START_REF] Li | Transferred deep learning for hyperspectral target detection[END_REF][START_REF] Yao | Intelligent Nanophotonics: Merging Photonics and Artificial Intelligence at the Nanoscale[END_REF]. Computational methods using neural networks that are trainable for specific problems have recently been shown to be highly efficient and fast [START_REF] Michael | Neural Networks and Deep Learning[END_REF][START_REF] Goodfellow | Deep Learning[END_REF][START_REF] Peter | Pushing the limits of optical information storage using deep learning[END_REF]. Recently, this approach has been used in various applications utilizing speckle patterns such as image reconstruction, object classification and recognition [START_REF] Yunzhe | Deep speckle correlation: A deep learning approach toward scalable imaging through scattering media[END_REF][START_REF] Satat | Object classification through scattering media with deep learning on time resolved measurement[END_REF][START_REF] Valent | Scatterer recognition via analysis of speckle patterns[END_REF][START_REF] Horisaki | Learning-based imaging through scattering media[END_REF][START_REF] Rahmani | Multimode optical fiber transmission with a deep learning network[END_REF][START_REF] Borhani | Learning to see through multimode fibers[END_REF][START_REF] Moran | Deep, Complex, Invertible Networks for Inversion of Transmission Effects in Multimode Optical Fibres[END_REF].

Here, we demonstrate the successful application of Deep Learning (DL) neural networks to the retrieval of spectral information from speckle images. Using a multi-mode, multicore fiber array as a multiplexed speckle spectrometer, we achieve real-time spectral imaging over several thousands of individual fiber cores. Besides being orders of magnitude faster than other, CS-based techniques, we investigate the robustness of DL to noise as well as to image shifts that could originate from thermal expansion or vibrations in the imaging system. We show the adaptability of DL in such conditions with good performance achieved by appropriate training. Results for DL are compared with CS and with analytical regularized inversion approaches. We find that DL performs well both in the compressive and oversampling regimes, combining a good balance in characteristics with fast reconstruction speeds and massive parallelized performance over many fiber cores.

Method

We use a multi-core, multimode fiber (MCMMF) as the complex scattering medium (Edmund Optics, Fiber optic image conduit). Mode mixing in each individual fiber results in a characteristic speckle pattern with a wavelength dependence determined by the fiber length and the angle of incidence of the incident light. For a direct comparison with previous results we used in our first calibration experiments the setup as described in [START_REF] French | Snapshot fiber spectral imaging using speckle correlations and compressive sensing[END_REF]. For the projection of animations we developed a new setup as shown in Fig. 1(a) based on an acousto-optic tunable filter (AOTF) and spatial light modulator (SLM). In short, a supercontinuum light source (Fianium SC400) was spectrally filtered using an AOTF with a resolution of 5 nm. The filtered light was projected onto a single-mode fiber to ensure stability of the illumination beam with wavelength and to eliminate any other forms of spectral drift in the setup. The fiber output was reflected off the liquid crystal spatial light modulator (SLM, Holoeye Pluto) and was projected onto the MCMMF at an incident angle of 4 • with an image demagnification of 5:1. The MCMMF consisted of 3012 fibers with individual core diameters of 50 µm. Fibers of different lengths can be used for different applications depending on the bandwidth required [START_REF] French | Snapshot fiber spectral imaging using speckle correlations and compressive sensing[END_REF]. After transmission through the fiber array, the output facet of the MCMMF was imaged onto the focal plane array of a 12-bit, 5 MPixel monochrome CMOS camera with a pixel size of 2.2 µm × 2.2 µm (AVT Guppy) using a 1:1 imaging system. Collected images were transferred to PC via IEEE 1394a and saved in uncompressed TIF format in 2592 (H) × 1944 (V) resolution in 12 bits.

Figure 1(b) illustrates the typical information obtained at the exit surface of the MCMMF in the form of wavelength dependent speckle patterns obtained from the individual fiber cores. Each pattern corresponds to a superposition of higher order fiber modes that is dependent on the wavelength, the length of the fiber and the angle of incidence. All fiber cores are slightly different and local variations in the material properties, strain, impurities and other random structural elements give rise to an individual set of speckle patterns for each fiber core in the array. The speckle patterns for every wavelength are stored into a multispectral transmission matrix for every core, which in principle allows retrieval of spectral information from arbitrary superposition states using a number of different techniques. Spectra consisting of more than one wavelength component, as well as continuous spectra, result in a superposition of many speckle patterns. Analytical inversion techniques like Moore-Penrose pseudo-inversion can be employed to reconstruct the spectra from these superpositions, but their performance is strongly dependent on noise and appropriate regularization is needed. In this work we compare our DL approach with analytical inversion using Tikhonov regularization (TR) [START_REF] Liutkus | Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium[END_REF]. Moreover, analytical inversion is limited to the oversampled regime, and its breakdown is observed at the Shannon-Nyquist sampling limit [START_REF] French | Speckle-based hyperspectral imaging combining multiple scattering and compressive sensing in nanowire mats[END_REF]. Compressive sensing (CS) extends reconstruction into the undersampling regime under conditions of sparsity, in our work CS was implemented for comparison to DL using the python package "cvxpy" [START_REF] Agrawal | A Rewriting System for Convex Optimization Problems[END_REF]. Spectral reconstruction via DL was implemented using a convolutional neural network (CNN), composed of a series of convolutional layers followed by two fully connected layers of 512 and 256 nodes with dropout regularization using 70% keep probability. The final, dense output layer of 43 neurons represents the spectrum, where each neuron corresponds to a discrete wavelength channel. The size of the CNN was manually optimized for each tested sampling condition. For a region of interest (ROI) size of 5x5 pixels, the best performing network consists of two convolutional layers with each a 2x2 kernel (CNN (i) in Fig. 2,yellow). On 20x20 pixels, a three-layer CNN with kernel size 3x3 throughout the network was found to perform best (CNN (ii) in Fig. 2, blue). Each convolution is followed by batch normalization and a leaky ReLU activation layer, all layers use valid padding. We found that any type of pooling consistently reduced the reconstruction quality, so no pooling was performed. The networks were implemented in python using keras as frontend for tensorflow [START_REF] Abadi | TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems[END_REF].

To test the performance of the different approaches, multiple patterns were digitally added up together to simulate a real signal made of a given number, N λ , of nonzero wavelength components with randomly varying intensities. The images of the speckle patterns were cropped to various ROI sizes to achieve different regimes of oversampling and undersampling, as given by the ratio of the total number of calibrated wavelengths, Y, to the total number of pixels of the ROI, X. For each multimode fiber, a total dataset of 31,000 images was generated, of which 29,000 were used to train the neural network, 1000 served for validation during training and the remaining 1000 were used for the final evaluation. A plot demonstrating training convergence is given in Fig. 7 in the Appendix.

Results and discussion

Deep learning reconstruction quality

A direct numerical illustration of the reconstruction capabilities of the DL approach is shown in Fig. 3. In Fig. 3(a) and 3(b) the performance is shown for different sampling regimes ranging from oversampling (Y /X = 9.30) down to deep undersampling (Y /X = 0.21). The cartoons in Fig. 3(a) illustrate the quality of the reconstruction, where a smiley emoticon was used as the ground truth in a single wavelength channel. For a single non-zero wavelength N λ =1, this is the only information contained in the spectrum. For the case N λ = 10, nine other wavelengths in the spectrum are filled with a crossed-out symbol in the form of a capital "X". We see that in both cases, the reconstruction of the target is very good in the oversampling regime, while the quality becomes poorer below the sampling limit, Y /X < 1. For N λ = 10 we can see the appearance of the cross in the image, indicating the presence of significant cross-talk between the spectral channels.

A quantitative analysis of this dependence of the reconstruction quality on the sampling rate is presented in Fig. 3(b), where the cross correlation of the reconstructed spectrum with the input spectrum (ground truth) is plotted versus the sampling ratio Y/X. We can clearly see the main trends identified in the illustration, namely a good quality of reconstruction in the oversampling regime and a degradation in performance for Y /X < 1. In the undersampling regime, we see that the reconstruction improves for lower number of nonzero wavelengths, with reasonably good performance (defined as correlation > 0.5) for sampling rates as low as Y/X=0.21 for just a single non-zero wavelength. Clearly, DL is able to infer meaningful results under conditions in which the information density is sparse and where analytical inversion techniques show a complete breakdown [START_REF] French | Speckle-based hyperspectral imaging combining multiple scattering and compressive sensing in nanowire mats[END_REF]. In other words, DL is able to cross over far into the compressive sensing regime and therefore exhibits similar characteristics to a CS-based approach. Having shown the strength of DL in the compressive sensing regime, it is of interest to investigate its capabilities in the regime of dense information, under conditions where the sparsity assumption underpinning CS ceases its validity. We start again with a numerical illustration in Fig. 3(c) to visualize the amount of information that can be encoded through speckle images. Using Y=43 available wavelength channels, we encoded separately the red, green and blue (RGB) channels of 14 independent RGB images using the experimentally obtained transmission matrix. The remaining unused wavelength channel allows to assess the residual cross-talk. Raw RGB reconstruction data are given in the Supporting Information. Figure 3(c) shows the reconstructed RGB images obtained using our DL approach in either the regime of undersampling (Y/X=0.84) or oversampling (Y/X=9.30). In case of oversampling (Y /X > 1), the neural network has much more input information to work with, which results in excellent image reconstruction quality and low residual cross-talk. For the undersampling regime, the images are still discernible but with significant reconstruction noise and crosstalk. These trends are again quantified in the accompanying analysis of Fig. 3(d), showing the cross-correlation with the ground truth versus number of non-zero wavelengths. We see that the network output is almost perfectly correlated in the oversampled case (perfect reconstruction), which holds even for dense spectra, where signal is present in all wavelength channels (N λ =43).

As seen in Fig. 3(c), for increasing number of wavelengths the effect of undersampling results in a rapid decrease of reconstruction fidelity. In the appendix we compare the same RGB data with reconstruction by TR and CS in the under-and oversampling regime and find that DL can compete with CS at a visual comparison and largely outperforms the analytical approach in the undersampled measurements.

Comparison of different reconstruction techniques

In Fig. 4 deep learning (DL) is compared directly with both the TR and CS reconstruction methods. In this benchmark, 1000 randomly generated spectra were generated numerically from the experimental transmission matrix. Figure 4(a) shows the oversampling case (Y/X=9.30) corresponding to an ROI of 20x20 pixels, while Fig. 4(b) shows the undersampling case (Y/X=0.58) corresponding to only 5x5 pixels. Several examples of typical spectra are shown (blue dash, ground truth), together with their corresponding reconstructions using DL (red), TR (orange) and CS (light blue). The lower two examples correspond to continuous spectra with a high density of spectral information. Figure 4(c) and 4(d) gives the full quantitative analysis of the average of the cross-correlation between each of the 1000 randomly generated spectra (ground truth) and its respective reconstruction. In the oversampling regime, all methods perform well with correlation values > 0.95. In the undersampling regime TR fails completely (average cross-correlation < 0.5) as it can be seen to produce a mostly flat spectrum for all cases irrespective of the spectral shape. DL yields a very good performance and even clearly outperforms CS for dense spectra in the undersampling case.

The slightly weaker performance of DL compared to CS in the oversampling case can be explained by the statistical training procedure in DL, while CS on the other hand is an analytical approach, yielding generally a close-to-optimum solution however at a significantly higher computational cost compared to the neural network reconstruction, as discussed further below. In the undersampling case we observe that DL tends to result in inferred spectra that are smoother than the original input spectrum, whereas CS results in more spikes in the spectra even when the input is smooth.

The previous tests considered speckle reconstruction using a multispectral transmission matrix in the complete absence of noise. In a real-life scenario, one can expect some level of noise to be present in the image, either shot noise, electronic camera noise or other non-specific backgrounds. An imaging system may also experience some drift caused by environmental effects, such as vibrations and thermal variations. To assess the robustness of the different approaches against typical perturbations, we compare in Fig. 5 the respective performance of DL, CS and TR. While CS and TR are based on analytic methods which are intrinsically inflexible to variations, DL has the advantage of allowing some level of adaptability through the choice of training data.

Robustness against noise and spatial shifts

To investigate the adaptability of the DL approach, we trained the neural network using noisy and spatially shifted training data with the aim of making it more robust against these effects. To account for the effect of noise, we added normally distributed random intensity noise to every pixel of the speckle pattern. Figure 5(a) shows a parameter map showing the relative performance of DL trained on noisy data (DL+N) against CS, which is quantified as the ratio of their respective cross-correlations with the input spectra. Noise-adapted DL consistently outperforms CS over a large parameter-space. Blue regions indicate a superior DL performance. DL always performs better for relatively dense spectra with N λ > 15. Even in most of the "white" parts of the colourplot, DL is outperforming TR and CS by at least some percent (see bar plots in Fig. 5(b)). CS is better performing only on sparse data at very low sampling rates. Furthermore TR contains a free parameter which has to be empirically adjusted to match a given noise level in order to achieve the specified performance [START_REF] Liutkus | Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium[END_REF]. The neural network trained on noisy data (DL+N) outperforms the normal DL when dealing with noisy spectra. This increased adaptability of the DL+N neural network comes at the cost of a relatively small hit on performance when dealing with noiseless data as seen in Fig. 5(b).

In principle, the performance of CS and TR on noisy data can be improved by adding an additional denoising step prior the spectral reconstruction, which of course adds additional computational cost. Since we are interested in developing a real-time-ready approach, computationally expensive image treatment on each speckle should be avoided. Thus, our analysis evaluates, how the different approaches can handle noisy data "out of the box".

A larger drift of the CCD or of the fiber would completely change the transmission matrix of the system and hence would require a full re-characterization of the setup. In the following we test whether deep learning can adapt to such situations, where analytical methods and compressive sensing break down definitely. To simulate spatial drift, the cropped ROI was randomly shifted by one pixel in arbitrary directions. We compare in Fig. 5(c) and 5(d) the performance of networks that were trained with and without spatial shift. If the neural network is trained without such perturbed data, the performance of DL decreases, just like for the conventional methods TR and CS. However, if the training datasets contain accordingly perturbed data, DL shows a distinct performance boost at the reconstruction from non-perfect speckle images. In the case of shift (Fig. 5(c) and 5(d)), accordingly trained DL dramatically outperforms the other techniques on perturbed data, while again the loss of performance on unshifted data is relatively small.

We note that the training set might not only contain data accounting for shifts of the CCD, but also for shifts of the fiber or variations in the incident angle of light. All these effects would affect the speckle patterns in a deterministic way and could be corrected by an accordingly trained artificial neural network, whilst this is impossible with an analytical method. Finally, apart from its universal applicability and superior stability, the most important achievement of using DL over CS is its reconstruction speed. We therefore demonstrate that our deep learning based hyperspectral reconstruction method is capable of real-time image processing. To this end we projected a grayscale video via an SLM onto the MCMMF. During video playback we randomly changed the wavelength of the illumination using the AOTF. We analyzed in real time the 2700 speckle patterns for each core, captured by the CMOS using artificial neural networks (one network for each fiber), which were trained in advance and preloaded into RAM. Figure 6(a) shows frames from the original video data in the top row, and the three first wavelengths of the spectral reconstruction in the bottom rows (for full videos see Visualizations). Upon switching of the wavelength, the reconstructed image changes channel with limited cross-talk between channels. At 1.74s, the AOTF was switched during the frame acquisition, briefly resulting in two spectral components. We note that the reconstruction quality is a bit worse compared to the synthetic data shown before, which is due to non-perfect intensity stability of the setup and some spurious residual wavelength correlations in the used, shorter (2.54cm) fiber. Further visualizations with the "La Linea" video and with an "Eclipse" video are also available. Finally, using microsecond-fast frequency sweeps with the AOTF, we were able to transmit spectrally broadened wavelength windows through the fiber bundle. Neural network based, multi-wavelength hyperspectral image reconstruction of experimental real-time data is demonstrated in some of the Visualizations in the supplementary material.

The training of the networks for 2700 fiber cores takes about 8 hours on an Nvidia Quadro P6000 GPU. This is a one-time procedure since the speckle patterns can be maintained stable over long times. The hyperspectral image reconstruction itself is the time-critical process in many applications. The trained neural networks are capable to reconstruct all 2700 fiber cores in only 132 ms on an Intel i7-3770 CPU. In comparison, CS requires around 2 minutes per frame for processing of the full fiber bundle. TR should potentially be even faster than deep learning, but interestingly, we found that in our python implementation, TR is around 1.5 times slower than the neural network inference (190 ms per video frame), which we attribute to the relatively slow python code, whilst the neural networks are evaluated through the highly optimized tensorflow backend. Hence it should be possible to further accelerate TR. Visually, all three methods perform similarly on the experimental video reconstruction.

Figure 6(b) shows the durations of the steps of DL hyperspectral image reconstruction on the Intel i7-3770 CPU, working with 32GB of RAM for network pre-loading. Including the preprocessing and rendering of the final hyperspectral images, the algorithm in its current state can reconstruct about 5 full images per second, which is faster than the total acquisition and transfer time of our 12bit CMOS camera (about 0.35s). In supplementary Visualizations, we also show >5fps reconstruction rate, using faster 8bit CMOS readout, which becomes essentially limited by the neural network inference speed.

There is significant scope for further increasing the reconstruction speed of DL by performance, in a first step already by optimization of the code. While not all 2700 networks fitted in the memory of our GPU, a benchmark on 1000 preloaded networks showed that the GPU offers around ×2.5 speed improvement, multi-GPU platforms will be accordingly faster. Training networks that each reconstruct speckles for multiple fibers in parallel can potentially result in a further significant performance boost and reduce the memory requirement of the approach, as is shown in figure 8 of the appendix. Even higher speed can potentially be obtained by developing hardware implementations of the networks, for example based on Field Programmable Gate Arrays (FPGAs) [START_REF] Ovtcharov | Accelerating Deep Convolutional Neural Networks Using Specialized Hardware[END_REF]. With respect to our current implementation using Python, a more direct hardware/software communication could also straightforwardly increase the frame collection rates, which however is outside the scope of this proof of principle study.

Conclusion

In conclusion, we have shown that with deep learning, a multicore multimode fiber bundle can be used as a real-time hyperspectral camera, robust to noise and spatial shifts. Using wavelength-and fiber-core dependent speckle patterns, we have used deep learning to cope with large amounts of data at a video rate of several frames per second on conventional computer hardware. The imaging spectrometer and deep learning technique are versatile by design, and the calibrated wavelength range can be tailored to specific applications. The approach can be easily scaled to any number of wavelength channels, to desired spectral resolution via the length of the multicore fiber bundle and with respect to imaging spatial resolution. Deep learning in combination with speckle spectrometry enables a new class of low-cost, compact hyperspectral imaging systems with real-time data processing capabilities.

Supplementary materials

Visualizations 1-4: La Linea short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm for several AOTF wavelength sweep sequences.

Visualization 5: Eclipse short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm.

Fig. 1 .

 1 Fig.1. a) Scheme of the experimental setup including broadband supercontinuum laser source, acousto-optical tunable filter (AOTF), Spatial Light Modulator (SLM) used for image generation and the multi-core, multimode fiber (MCMMF). b) Original projected image and detected camera image of the exit interface of the fiber bundle at a single selected wavelength with typical speckle patterns of a selected fiber core for different input wavelengths (1-n). La Linea, with permission, copyright CAVA/QUIPOS.

Fig. 2 .

 2 Fig. 2. Neural network structures used in this study for pixel areas of (i) 5x5 pixels (Y/X = 0.58) and (ii) 20x20 pixels (Y/X = 9.30).

Fig. 3

 3 Fig. 3. a) Numerical illustration and b) calculation of reconstruction quality using DL for different sampling rates Y/X, for N=1 and N=10 non-zero wavelengths. One wavelength carries the encoded image (smiley), all other non-zero channels encode the image of a capital "X", which becomes slightly visible at low sampling rates due to cross-talk (see Supporting Information). c) Numerical illustration of image reconstruction using DL for dense spectra (N=42) showing 14 RGB images that are encoded in 42 wavelength channels, the 43rd, blank channel serves for cross-talk control. Reconstructions are shown for undersampling Y/X=0.84 and oversampling Y/X=9.30 regimes. d) Cross-correlation with ground truth as a function of number of non-zero wavelengths in the spectrum for different sampling rates. Results in (b,d) are averaged over the whole fiber stack and for 100 spectra per fiber core. Light areas indicate the standard deviation of the data. Dashed line at Y/X=1 corresponds to Nyquist-Shannon sampling limit. Dashed line at a cross-correlation of 0.5 indicates the threshold below which the reconstruction is considered to have failed. Underlying full spectral data for (a) and (c) are presented in Fig. 9 and Fig. 10 of the Appendix. All shown cliparts are from www.openclipart.org and public domain.

Fig. 4 .

 4 Fig. 4. a) Examples of speckle images and reconstructed spectral information for sparse (three top rows) and dense spectra (two bottom rows, generated by a random-walk like algorithm). Left column: oversampling regime with Y/X=9.3, right column: undersampling regime with Y/X=0.58. The black box inside the speckle pattern shows the ROI used in the undersampling case. b) Histograms comparing average cross correlations from 1000 randomly generated sparse (< 50% sparsity) and dense (all wavelengths non-zero) spectra, obtained with deep learning (DL), Tikhonov regularization inversion (TR) and compressive sensing (CS).

Fig. 5 .

 5 Fig. 5. a) Map showing ratio of reconstruction quality (cross-correlations) for deep learning trained on noisy data (DL+N) over compressive sensing (CS) for 10% added noise. Blue indicates DL+N better than CS, red indicates CS better than DL+N. Contour lines indicate the cross-correlation of DL+N speckle reconstruction. b) Calculated cross-correlations without noise and in presence of 25% noise. DL+N outperforms CS over a large part of parameter space where the cross-correlation < 0.9. c) Robustness of the reconstruction against shifts of the speckle patterns by one pixel in a random direction. d) Calculated cross-correlations without shift and with shift of 1 pixel. DL can be trained on data including shift (DL+S), which renders the method very robust in such scenario, largely outperforming TR and CS.

3. 4 .Fig. 6 .

 46 Fig. 6. a) Real-time speckle-based hyperspectral video reconstruction via DL. A video is projected on the fiber bundle using an SLM in amplitude-modulation configuration. During playback the wavelength of the projecting light is changed. Top: Original frames of the input video (see also Visualizations). Bottom: Spectral reconstruction of three wavelength channels of the full multi-core fiber (approx. 2700 fiber cores). b) Bar graph showing the timings of the different execution steps. Full visualizations are included in the Supporting Materials of this study. La Linea, with permission, copyright CAVA/QUIPOS.

Fig. 8 .

 8 Fig.8. Training multi-channel 2D convolutional -1D convolutional upsampling networks on multiple fibers per network. a) architecture of a multi-fiber network: speckle images from N different fibers are fed into the network and translated into N spectra at the output. b) timing and reconstruction quality for a fixed network architecture for increasing number N of reconstructed fibers (corresponding to the number of input and output channels). Trained on 10,000×N speckles. On 32GB RAM, training data for up to 20 fibers could be kept in memory. The trained networks show no significant difference in reconstruction quality, while the timing per fiber reduces almost linearly. The Memory requirement of each pretrained network is approximately constant, hence required RAM linearly decreases with the number of reconstructed fibers per network.

Fig. 9 .

 9 Fig. 9. Reconstruction of all spectral channels with 10 channels containing images at a sampling rate of (a) Y/X=0.21, (b) Y/X=0.58, (c) and Y/X=1.14. Channels carrying information are 2; 3; 4; 9; 17; 18; 19; 24; 28; 33.

Fig. 10 .

 10 Fig. 10. a) Encoding and reconstruction of 14 RGB images in the speckle patterns of the multi-core fiber. Raw reconstructions of the spectral channels, containing the red, green and blue parts of the color images, at a sampling rates of (a) Y/X=0.84 and (b) Y/X=9.3.

Fig. 11 .

 11 Fig. 11. Visual comparison of reconstruction quality of the different methods (deep learning [DL], Tikhonov regularization [TR] and compressive sensing [CS]) on the RGB dataset (see also figure 3). a) sampling rate of Y/X=0.84 and (b) Y/X=9.3.

Acknowledgments

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research. La Linea CAVA/QUIPOS. La Linea usage rights granted to the University of Southampton for the purpose of this research has been approved by Quipos, Osvaldo Cavandoli's worldwide licensor. introduction to DBSCAN, and to Laurent Daudet for help with compressive sensing. All data supporting this study are openly available from the University of Southampton repository (DOI: 10.5258/SOTON/D0942).

Funding

German Research Foundation (DFG) Research Fellowship (WI 5261/1-1); EPSRC (EP/J016918/1).

Appendix


Clément Majorel
email: clement.majorel@cemes.fr

Vincent Paillard

Adelin Patoux

Peter R Wiecha

Aurelien Cuche

Arnaud Arbouet

Caroline Bonafos

Christian Girard
email: girard@cemes.fr

Aurélien Cuche

Theory of plasmonic properties of hyper-doped silicon nanostructures

Keywords: Surface plasmons, Hyperdoped nanostructures, Silicon

teaching and research institutions in France or abroad, or from public or private research centers.

During the three last decades, a huge amount of work has been devoted to the understanding of the plasmonic properties of complex metallic nanostructures, either individually or as the elementary brick of a metamaterial. This interest was also driven by potential applications in sensing devices [START_REF] Brolo | Plasmonics for future biosensors[END_REF], nanophotonic devices [START_REF] Barnes | Surface plasmon subwavelength optics[END_REF], photocatalysis [START_REF] Seh | Janus Au-TiO2 Photocatalysts with Strong Localization of Plasmonic Near-Fields for Efficient Visible-Light Hydrogen Generation[END_REF], and field-enhanced spectroscopies [START_REF] Huth | Resonant Antenna Probes for Tip-Enhanced Infrared Near-Field Microscopy[END_REF][START_REF] Law | All-Semiconductor Plasmonic Nanoantennas for Infrared Sensing[END_REF][START_REF] Hillenbrand | Phonon-enhanced lightmatter interaction at the nanometre scale[END_REF][START_REF] Aizpurua | Substrate-enhanced infrared near-field spectroscopy[END_REF]. However, the use of metals such as gold or silver is still an issue due to a poor compatibility with the semiconductor processing technology, and to strong dissipation losses at optical and infrared frequencies.

Recently, a new category of plasmonic materials emerged, with the study of Localized Surface Plasmon Resonance (LSPR) in heavily doped semiconductor nanostructures and metamaterials [START_REF] Luther | Localized surface plasmon resonances arising from free carriers in doped quantum dots[END_REF][START_REF] Kramer | Plasmonic Properties of Silicon Nanocrystals Doped with Boron and Phos-phorus[END_REF][START_REF] Faucheaux | Plasmon Resonances of Semiconductor Nanocrystals: Physical Principles and New Opportunities[END_REF][START_REF] Kriegel | Plasmonic doped semiconductor nanocrystals: Properties, fabrication, applications and perspectives[END_REF]. Compared to metals, the imaginary part of the dielectric constant in the visible and infrared range is low for most semiconductors, leading to weak losses. As for metals, the LSPR frequency is tunable by the constituent material, the size and shape of the nanostructure, and its dielectric environment. However, the active dopant concentration, thus the free carrier (electrons or holes) concentration, allows to finely tune the LSPR frequency [START_REF] Luther | Localized surface plasmon resonances arising from free carriers in doped quantum dots[END_REF][START_REF] Faucheaux | Plasmon Resonances of Semiconductor Nanocrystals: Physical Principles and New Opportunities[END_REF]. Notably, the lower order of magnitude of carrier density in degenerate semiconductors with respect to noble metals has an attractive consequence: even a minor modification of the carrier density has a strong effect on the spectral position of the LSPR. Quantum size effects have also an influence in the case of nanocrystals of a few nanometers in diameter [START_REF] Pi | Tight-Binding Calculations of the Optical Response of Optimally P-Doped Si Nanocrystals: A Model for Localized Surface Plasmon Resonance[END_REF]. For larger nanostructures, for which the Drude model can be used, the LSPR frequency is varying from the far-infrared (FIR) range for standard dopant concentration (a few 10 18 -10 19 cm -3 ) to the mid-infrared range (MIR) for heavily doped semiconductors (about 10 20 cm -3 ), and even near-infrared (NIR) if higher concentration can be obtained (above 10 21 cm -3 ) [START_REF] Luther | Localized surface plasmon resonances arising from free carriers in doped quantum dots[END_REF]. Nonstoechiometric alloy nanocrystals, such as Cu 2-x S or Cu 2-x Se, or metal-doped metal oxyde compounds (such as Al-or In-doped ZnO for instance) have shown intense plasmon resonances in the MIR to the NIR [START_REF] Luther | Localized surface plasmon resonances arising from free carriers in doped quantum dots[END_REF][START_REF] Faucheaux | Plasmon Resonances of Semiconductor Nanocrystals: Physical Principles and New Opportunities[END_REF][START_REF] Kriegel | Plasmonic doped semiconductor nanocrystals: Properties, fabrication, applications and perspectives[END_REF]. Few theoretical ab initio [START_REF] Pi | Tight-Binding Calculations of the Optical Response of Optimally P-Doped Si Nanocrystals: A Model for Localized Surface Plasmon Resonance[END_REF][START_REF] Della Sala | Ab Initio Plasmonics of Externally Doped Silicon Nanocrystals[END_REF][START_REF] Guerra | Preferential Positioning of Dopants and Co-Dopants in Embedded and Freestanding Si Nanocrystals[END_REF] and experimental [START_REF] Kramer | Plasmonic Properties of Silicon Nanocrystals Doped with Boron and Phos-phorus[END_REF][START_REF] Pi | Tight-Binding Calculations of the Optical Response of Optimally P-Doped Si Nanocrystals: A Model for Localized Surface Plasmon Resonance[END_REF][START_REF] Rowe | Phosphorus-Doped Silicon Nanocrystals Exhibiting Mid-Infrared Localized Surface Plasmon Resonance[END_REF][START_REF] Wheeler | Hypervalent surface interactions for colloidal stability and doping of silicon nanocrystals[END_REF] work concern heavily doped silicon nanocrystals (Si-NCs), which are potentially interesting nanomaterials for plasmonic applications in the infrared due to their low cost and low toxicity. However, nanocrystals are difficult to handle and ordered arrays of nanostructures are needed for many applications, for instance infrared sensing as described in ref [START_REF] Law | All-Semiconductor Plasmonic Nanoantennas for Infrared Sensing[END_REF]. Therefore, doping Si nanostructures (Si-NS) as the ones produced by electron beam lithography in the thin Si overlayer of a Silicon On Insulator (SOI) substrate could be attractive candidates for such applications [START_REF] Wiecha | En-hancement of electric and magnetic dipole transition of rareearth-doped thin films tailored by high-index dielectric nanostructures[END_REF]. High dopant concentrations are actually achievable in Si by using out of equilibrium methods as nanosecond Laser Thermal Annealing (LTA) treatments. Hence, active phosphorus concentrations as high as 5×10 20 cm -3 up to 2×10 21 cm -3 can be reached in respectively thin SOI [START_REF] Alba | Nanosecond Laser Annealing for Phosphorous Activation in Ultra-Thin Implanted Silicon-On-Insulator Substrates[END_REF] and bulk silicon [START_REF] Huet | Doping of semiconductor devices by Laser Thermal Annealing[END_REF]. Recently doping Si with deep chalcogen donor as Te by using the same non equilibrium processing allows also exceeding 10 21 cm -3 active dopants [START_REF] Wang | Breaking the Doping Limit in Silicon by Deep Impurities[END_REF]. This doping range corresponds to a plasmon resonance wavelength of about 1.5 µm of great interest in photonics. It is thus important to develop theoretical tools able to describe the far-field and near-field electromagnetic properties of doped Si-based plasmonic nanoantennas of arbitrary size and shape, in a similar way as for the metalbased plasmonic nanoantennas. In the following, we will focus on Si-NS with dopant concentration in the 10 20 to 5×10 21 cm -3 range and dimensions of a few tenths to a few hundreds of nanometers on top of a silica layer which mimics the SOI buried oxyde layer.

We introduce the Green Dyadic Method (GDM) implemented in the python toolkit pyGDM [START_REF] Wiecha | pyGDMA python toolkit for full-field electrodynamical simulations and evolutionary optimization of nanostructures[END_REF][START_REF] Girard | Near fields in nanostructures[END_REF]. GDM is an ab-initio electrodynamic method that relies on solving Maxwell's equations using Green's dyadic functions [START_REF] Girard | Near fields in nanostructures[END_REF]. This method is interesting because calculations can be performed for nanostructures of arbitrary size, shape, composition and, in the context of doped semiconductor NS, of dopant concentration. We show that, by solving self-consistent equations, it is possible to determine both electric and magnetic fields at optical or infrared frequencies inside and outside a set of Si-based plasmonic nanostructures of arbitrary size and shape. Thus, all physical quantities such as extinction, absorption and scattering spectra, as well as near-field maps, can be calculated with precision [START_REF] Girard | Near fields in nanostructures[END_REF].

The Green Dyadic Method (GDM) for submicrometer scale hyperdoped silicon structures

In the presence of an external illumination by an incident electromagnetic field characterized by the couple of electric and magnetic fields, {E 0 (r, t); H 0 (r, t)}, the local electric field E(r, t) induced inside and around a doped semiconductor system composed of N similar or different particles (see the illustrating example in figure (1)) can be calculated using the dyadic Lippmann-Schwinger equation:

E(r, ω) = E 0 (r, ω)+ 1 4π N j=1 Vj ( j dop (ω) -env (ω))S(r, r , ω) • E(r , ω)dr (1)
where E 0 (r, ω) and E(r, ω) are the time Fourier transforms of the incident and local electric fields (written in CGS Gaussian units), respectively. In this self-consistent equation, (j) dop (ω) and env (ω) represent the permittivity of the j th doped semiconductor nanostructure and the surrounding medium, respectively. Considering the submicrometer size of the nanostructures investigated here, we can apply the simple Drude model, which accurately describes the carrier dynamics in bulk doped semiconductors:

(j) dop (ω) = intr (ω) - (ω j p ) 2 ω(ω + iγ j ) (2)
where intr (ω) represents the dielectric constant of the intrinsic semiconductor and γ j is the damping term due to the collisions. ω j p defines the plasma frequency associated with the free carrier concentration of the j th particle:

ω j p = 4πN j dop e 2 m o (3)
where N j dop , e, and m o define the carrier density (in cm -3 ), the elementary charge, and the effective mass respectively. The effective mass was chosen equal to 0.3 times the mass of the electron [START_REF] Zhou | Comparative Study on the Localized Surface Plasmon Resonance of Boron-and Phosphorus-Doped Silicon Nanocrystals[END_REF]. For all the calculations in this article, we have arbitrarily chosen γ j = 0.1ω j p . This choice makes it possible to take into account the accumulation of defects in the structure when the doping concentration increases. To confirm the consistency of this choice, we calculated the free electron damping γ by using γ = e/m * 0 µ where µ represents the mobility of the electrons [START_REF] Cada | Theoretical and experimental study of plasmonic effects in heavily doped gallium arsenide and indium phosphide[END_REF][START_REF] Soltani | Free-carrier electrorefraction and electroabsorption in wurtzite GaN[END_REF]. In Ref. [START_REF] Zhou | Comparative Study on the Localized Surface Plasmon Resonance of Boron-and Phosphorus-Doped Silicon Nanocrystals[END_REF], the mobility lies between ∼360 and 425 cm 2 V -1 s -1 when the concentration of P increases from 4% to 18%. These percentages correspond respectively to carrier densities equal to 2 × 10 21 cm -3 and 9 × 10 21 cm -3 . By using the above values for mobility and effective mass, we found γ 1.63 × 10 13 Hz and γ 1.38 × 10 13 Hz, respectively. In our calculations, N dop lies between 1×10 20 cm -3 and 5×10 21 cm -3 , that means the number of impurities is less important than in the reference. If we replace N dop in Eq. ( 3) by these two values and multiply ω j p by 0.1 we obtain γ j = 1.64 × 10 13 Hz and γ j = 11.60 × 10 13 Hz, respectively. Therefore by choosing γ j = 0.1ω j p , γ j is higher than γ, hence we overestimate the number of collisions of the electrons inside the structure.

x y z In Equation ( 1), the response function S(r, r , ω) is the Green dyadic tensor associated with the bare silica surface (i.e. computed in the absence of any nanostructure).

This second rank tensor matching the boundary conditions on the planar surface is given as the sum of two distinct contributions:

S(r, r , ω) = S 0 (r, r , ω) + S surf (r, r , ω) , (4)
where S 0 (r, r , ω) represents the dipolar propagator of the homogeneous medium (characterized by env (ω)), and S surf (r, r , ω) takes into account the presence of the dielectric surface supporting the nanostructures:

S 0 (r, r , ω) = [k 2 0 I + 1 env (ω) ∇ r ∇ r ] e ik|r-r | |r -r | , (5)
with

k 0 = ω/c, k = env (ω)ω/c, and
S surf (r, r , ω) = ∆ surf (ω) (X 2 + Y 2 + Z 2 ) 5/2 × Z 2 + Y 2 -2X 2 -3XY 3XZ -3XY Z 2 + X 2 -2Y 2 3Y Z -3XZ -3Y Z 2Z 2 -Y 2 -X 2 (6)
with r = (x, y, z), r = (x , y , z ), and X = x -x, Y = y -y, and Z = z + z. The reflection coefficient ∆ surf (ω) is given by:

∆ surf (ω) = s (ω) -env (ω) s (ω) + env (ω) . (7)
As explained in reference [START_REF] Girard | Shaping and manipulation of light fields with bottom-up plasmonic structures[END_REF], numerical solutions of (1) need a volume discretization of the source region occupied by the nanostructures. Generally, each particle volume V j is discretized with n j identical elementary volumes v j . Such a procedure converts integrals into discrete summations:

E(r, ω) = E 0 (r, ω) + N j=1 η j (ω) nj i=1 × [S 0 (r, r j,i , ω) + S surf (r, r j,i , ω)] • E(r j,i , ω) . (8)
In this summation, the scalar parameters η j (ω) are proportional to the elementary volumes v j of the discretized cells in the j th particle:

η j (ω) = (j) dop (ω) -env (ω) 4π v j , (9)
and the vectors r j,i define the positions of the discretized cells inside the j th doped particle. The N nanostructures are positioned at the nodes of a cubic compact array. Notice that, in the case of most semiconductors including Si, the very high dielectric constant requires a fine discretization of the NS in order to have reliable calculations For instance a nanostructure with a dielectric constant Re dop . For a very high dielectric constant this effective wavelength may become comparable to the discretization step of the NS. In such case, the computation does not converge properly. The step d should be small compared to the wavelength inside the material and also with respect to the skin depth in an absorptive material [START_REF] Draine | The discrete-dipole approximation and its application to interstellar graphite grains[END_REF].

2. Spectral and near-field optical properties of square shaped-doped silicon pads deposited on a silica surface

In the different sections of the paper, we will consider a normal incidence monochromatic plane wave as illumination mode. In this case, the incident electric field can be defined by:

E 0 (r, t) = E 0 {cos(k • r -ω 0 t) + F env,surf cos(k • r + ω 0 t)} , (10)
where k = (0, 0, -k) and E 0 = E 0 (cos(φ), sin(φ), 0) (in which φ labels the incident polarization angle), and F env,surf defines the reflection Fresnel coefficient:

F env,surf = env (ω) - s (ω) env (ω) + s (ω) (11)
The Fourier transform of this field can be defined by:

E 0 (r, ω) = 1 2π E 0 (r, t)e iωt dt = E 0 2 ({e ik•r + F env,surf e -ik•r }δ(ω -ω 0 ) + {e -ik•r + F env,surf e ik•r }δ(ω + ω 0 )) (12)
For hyperdoped silicon structures the external excitation wavelength λ 0 = 2πc/ω 0 becomes an extremely sensitive parameter because of the known existence of plasmon resonances [START_REF] Luther | Localized surface plasmon resonances arising from free carriers in doped quantum dots[END_REF] and specific physical quantities can be used to characterize their new optical properties (near field intensity, absorption and scattering spectra, infrared local density of states, ...).

C ext (λ 0 ) = 8 √ env π 2 λ 0 |E 0 | 2 N j=1 nj i=1 (E 0 (r j,i , λ 0 ) • P(r j,i , λ 0 )) , (13)
where depicts the imaginary part and P(r j,i , ω 0 ) = η j (ω)E(r j,i , ω 0 ) (

A first example is shown in figure (2) for a square shaped pad of heavily doped (N dop =8×10 20 cm -3 ) silicon standing on a silica substrate. The spectrum of the extinction efficiency Q ext (the extinction cross-section divided by the geometrical cross-section) exhibits a well-defined plasmon peak located around 2.8 µm. The first resonance at 2.8 µm corresponds to the free carrier-induced LSPR. Interestingly, one can observe at longer wavelength a second spectral structure composed of different resonances, one of which dominating around 9.5 µm. This resonance, far from the plasmon band, is the signature of an optical phonon excitation inside the silica wafer supporting the doped Si-NS. Indeed, in the vicinity of 9.5 µm, the real part of the dielectric constant s (ω) of silica becomes rapidly negative while its imaginary part significantly increases [START_REF] Gunde | Vibrational modes in amorphous silicon dioxide[END_REF][START_REF] Kitamura | Optical constants of silica glass from extreme ultraviolet to far infrared at near room temperature[END_REF]. Consequently, the coefficient ∆ surf (ω) in the surface propagator S surf (r, r , ω) (see also Eq. ( 7)) resonates at the optical phonon frequency, yielding the additional peak at 9.5 µm in the extinction spectrum. In that case, silica phonons are excited by the virtual dipole image induced under the silica surface by the real dipole induced inside the Si-NS by the incident illumination field.

The behavior of the two absorption bands is evidenced in figure [START_REF] Seh | Janus Au-TiO2 Photocatalysts with Strong Localization of Plasmonic Near-Fields for Efficient Visible-Light Hydrogen Generation[END_REF] giving the extinction efficiency for the same Si pad as function of the dopant concentrations in the 1 × 10 20 -3 × 10 21 cm -3 range and the excitation wavelength. It can be seen that the LSPR band at short wavelength follows the square root of the carrier density, while the phonon band around 9.5 µm is rather insensitive to the Si-NS dopant concentration. At lower dopant concentration, a coupling between both resonances due to LSPR in Si-NS and phonon absorption in silica occurs. This kind of plasphonics phenomenon has been already studied in ref [START_REF] Marty | Plasphonics : local hybridization of plasmons and phonons[END_REF][START_REF] Hillenbrand | Phonon-enhanced lightmatter interaction at the nanometre scale[END_REF][START_REF] Anderson | Surface enhanced infrared absorption by coupling phonon and plasma resonance[END_REF].

For a high dopant concentration, the LSPR band is localized in the 1.5 -2 µm range. Such frequencies are accessible with strongly anisotropic gold structures. For instance, in ref [START_REF] Novotny | Effective Wavelength Scaling for Optical Antennas[END_REF], where nanorods of length 110 nm and radius 5 nm present a resonance at λ = 1170 nm. The use of hyperdoped Si-NS makes it possible to obtain the LSPR frequencies in the same NIR range even with isotropic structures.

(ii) Near-field intensity distribution around a single Si pad The function I nf (R, λ 0 ) gives the variation of the normalized optical near-field intensity recorded at a given location R above the sample and for the incident wavelength λ 0 . This signal can be computed for each location R from the total electric field and the incident electric field given by equations ( 8) and [START_REF] Pi | Tight-Binding Calculations of the Optical Response of Optimally P-Doped Si Nanocrystals: A Model for Localized Surface Plasmon Resonance[END_REF], respectively:

I nf (R, λ 0 ) = |E(R, λ 0 )| 2 /|E o (R, λ 0 )| 2 (15)
In order to get more insight on the impact of the dopant concentration, we calculated the infrared near-field intensity distributions of the electric field around the Si-NS for different wavelengths, and compared them to the case of intrinsic Si. In figure (4a), we show the scattering efficiency of a Si-NS with a dopant concentration N dop =2.66×10 21 cm -3 corresponding to a LSPR wavelength of about 1.54 µm (see solid dark curve in frame (a)). This wavelength corresponds to the state-of-the art active dopant concentration that can be obtained experimentally by pulsed laser annealing [START_REF] Huet | Doping of semiconductor devices by Laser Thermal Annealing[END_REF], and is of outmost interest in photonics. It indeed belongs to the optical telecommunication window and matches the emission of Erbium ions, which have been recently found new applications in Telecom-Band Quantum Optics [START_REF] Dibos | Atomic Source of Single Photons in the Telecom Band[END_REF].

Then we compute the normalized electric near field at two different locations shown in the inset of (4a), corresponding to a site at R = (0, 0, 70) on top of the Si-NS (magenta cross), and to a lateral site at R = (-60, 0, 70) (green cross). The near-field intensity distributions are shown in the two frames of figure (4b). The more striking feature is the presence of a strong contrast observed above the lateral site when changing the wavelength from 1200 µm to 1900 µm (see left frame of figure (4b)). The contrast change is less marked above the top site (right frame of figure (4b). For these wavelengths, the 2D maps of the electric field intensity distribution are given in (4c) and (4d). This reveals either the absence of contrast or a strong contrast around the doped Si-NS by controlling the excitation of the localized plasmons. By comparison, the maps computed around intrinsic Si-NS (4e) and (4f) are very similar independently of the wavelength. The absence of a strong local contrast in the case of the doped silicon can be explained by a vanishing contrast of the real parts of the permittivities between environment ( env = 1) and the doped silicon around λ 0 = 1.2 µm (close to the sign change of Re{ dop }).

Case of randomly oriented and/or doped Si-NS:

In figure (5), we complete our simulation sequence with a set of N = 5 randomly located and oriented Si-NS excited by an incident plane wave. We remark first that the disordered arrangement of the particles tends to slightly downgrade the image-object relation observed with single particles (cf. figure ( 4)). In addition, we analyse the changes of the near-field intensity maps when gradually increasing the pad doping rate. The analysis of these results raises the following comments. (i) Although the system is illuminated in the s-polarized mode (incident electric field parallel to the substrate), each Si-NS (either intrinsic or doped) displays a typical contrast in the electric near-field intensity maps with a large field gradient near the Si-NS edges. (ii) In the case of intrinsic Si, the contrast pattern does not depend on the incident wavelength. This is consistent with previous results reported in [START_REF] Barchiesi | Computing the optical near-field distributions around complex subwavelength surface structures: A comparative study of different methods[END_REF][START_REF] Weeber | Observation of Light Confinement Effects with a Near-Field Optical Microscope[END_REF], since in the absence of any plasmon resonance, those intrinsic Si-NS behave like dielectric pads. (iii) Contrary, within a wavelength interval ranging from 1200 nm to 1850 nm centered around the plasmon resonance at λ r = 1540 nm, we observe a significant change of the near-field intensity patterns when increasing the dopant concentration. In particular, close to λ 0 = 1200 nm, increasing the dopant concentration tends to flatten the contrast amplitude, illustrating how the incident wavelength can be used to control the visibility level expected in the near-field zone when approaching the plasmon resonance of the doped Si-NS.

Spectral and Polarization effect of a doped Si nanowire

Unlike doped cuboid Si-NS previously described, the ex- tinction cross-section of anisotropic doped Si-NS such as nanowires exhibit two polarization-dependent plasmon bands, that correspond to carrier oscillations either along their length (longitudinal mode) or across their width (transverse mode). This behavior, presented in figure [START_REF] Hillenbrand | Phonon-enhanced lightmatter interaction at the nanometre scale[END_REF], is well-known in the field of the "standard" (metal) plasmonics, where it was realized that one of the most effective approach in plasmon engineering was to break the confinement symmetry by modifying the nanoparticle aspect ratio from spherical to rod shape [START_REF] Link | Shape and size dependence of radiative, non-radiative and photothermal properties of gold nanocrystals[END_REF][START_REF] Link | Spectral Properties and Relaxation Dynamics of Surface Plasmon Electronic Oscillations in Gold and Silver Nanodots and Nanorods[END_REF]. The symmetry breaking lifts the plasmon mode degeneracy and gives rise to two distinct transverse and longitudinal bands. In our case, the most striking result is that two distinct regions of the infrared spectrum can be investigated using doped semiconductor nanowires.

Conclusion and Perspectives

In Summary, we have adapted GDM to describe the enhanced light-matter interaction and electromagnetic field confinement in the case of sub-wavelength doped Si-based plasmonic nanostructures. We show that such properties can be tuned over the MIR and NIR spectral ranges as function of NS size, dopant concentration. In addition, modifying the aspect ratio leads to two distinct transverse and longitudinal modes by lifting the plasmon mode degeneracy. Our results provide a useful tool for active dopant concentration measurement in doped semiconductor nanostructures using infrared absorption experiments, and for modeling near-field imaging in the infrared.

As shape and/or dopant concentration allow obtaining LSPR near the telecom wavelength of 1.54 µm, there are potential applications for silicon-based nanophotonics, and study enhanced emission from Erbium ions coupled to hyper-doped Si-NS. In addition, highly localized IR light depression or enhancement can be controlled in the vicinity of the highly doped Si-NS by adjusting the excitation wavelength. This property can be further exploited for either exalting (SERS effect or fluorescence exaltation) or on the contrary hiding (invisibility effect) the optical signal of molecules or nano-objects located in their near field.

These new objects open a new physics field for a mature nano-optics domain. Following up along this simple doping concept, extended networks of interconnected lattices of doped nanoparticles could generate unique and unusual sub-wavelength patterning of the infrared near-field. Several impacts are associated with light confinement around plasmonic structures. The most direct consequence is the improvement of the quality of the near-field infrared imaging and the increased local field enhancement, and finally the light energy storage in tiny volumes of matter [START_REF] Huth | Resonant Antenna Probes for Tip-Enhanced Infrared Near-Field Microscopy[END_REF][START_REF] Huth | Nano-FTIR Absorption Spectroscopy of Molecular Fingerprints at 20 nm Spatial Resolution[END_REF]. The optical physics related to the control of light confinement might have also important impact on the future solution for the miniaturization of both chemical and biological sensors in the infrared range.

Figure 1 :

 1 Figure 1: Example of three doped silicon nanostructures (j = 1, 2, or 3) of both arbitrary shape and intrinsic doping parameters N j dop . These objects are lying on a plane silica substrate of optical index ns(ω) = s(ω).

  (j) dop illuminated by a plane wave of wavelength λ 0 leads to an effective wavelength in the material λ ef f = λ 0 / Re (j)

2. 1 .

 1 Case of a single cuboid nanostructure of doped Si (100×100×50 nm 3 , N =1, n 1 =4000) (i) Extinction efficiency of a single doped Si pad The extinction cross-section can be deduced from the local electric polarization induced inside the doped nanostructure (see figure (2d)):

Figure 2 :

 2 Figure 2: (a) Extinction efficiency spectrum computed for a single hyper-doped silicon cuboid structure of size 100 × 100 × 50 nm 3 supported by a planar silica substrate. The dopant concentration N dop is 8 × 10 20 cm -3 . (b) Side view, and (c) top view of the discretized structure. Each point represents an elementary cell of volume v i . E 0 gives the incident plane wave polarization along the x-axis. (d) 3D representation of the snapshot of the charge density distribution inside the structure[28].

Figure 3 :

 3 Figure 3: 2D map representation of the extinction efficiency for heavily doped cuboid Si-NS of size 100 × 100 × 50 nm 3 lying on a silica substrate. The dopant concentration N dop varies between 1 × 10 20 and 9 × 10 20 cm -3 (a) and 1 × 10 21 and 3 × 10 21 cm -3 (b). The purple dashed line represents the evolution of the LSPR as a function of the dopant concentration.

Figure 4 :

 4 Figure 4: Simulation of the electric near-field behavior in the vicinity of doped (N dop = 2.66 × 10 21 cm -3 ) and intrinsic single Si-NS of same geometrical cross-section (100×100×50 nm 3 ). (a) Top view geometry and extinction efficiency; (b) Local near-field optical spectra computed above two different sites defined by R = (-60, 0, 70) (lateral site) and R = (0, 0, 70) (top site). The blue dots indicate both minimum and maximum values of the near-field intensity above the edge of the doped Si-NS, occurring at λ 0 = 1200 nm and λ 0 = 1862 nm, respectively. (c)-(d) Corresponding near-field intensity maps computed in a plane Z = 70 nm;(e)-(f) Same maps as in (c)-(d) for intrinsic Si-NS.

Figure 5 :

 5 Figure 5: Sequence of optical near-field images describing the evolution of near-field patterns when passing from pure to doped silicon pads. (color scale increasing from blue to red, window size 1000 × 1000 nm 2 ). All optical and geometrical parameters are identical to those of figure (3). (a) Geometry and dopant concentration considered in the numerical simulations; (b) Near-field map evolution computed at λ 0 = 1200 nm; (c) Same as (b) but for λ 0 = 1862 nm.

Figure 6 :

 6 Figure 6: (a) Side view, and (b) top view of the discretized structure of size 500 × 100 × 50 nm 3 . The two arrows in (b) represent the polarization of the incident excitation, parallel (red arrow) or perpendicular (blue arrow) to the nanowire axis. (c) Evolution of the extinction cross section due to longitudinal plasmon excitation as function of dopant concentration (from 1 × 10 21 to 5 × 10 21 cm -3 ). (d) Same spectrum for transverse plasmon excitation

Acknowledgments: This work was supported by the Agence Nationale de la Recherche (project ANR "DONNA" ANR-18-CE09-0034, project NEXT "LANCE" ANR-17-EURE-0009), and the computing facility center (CALMIP) in Toulouse (project P12167).