Philipp Del Hougne
email: philipp.del-hougne@univ-rennes1.fr

RIS-Based Radio Localization in Rich Scattering Environments: Harnessing Multi-Path with ANN Decoders

Keywords: Radio Localization, Reconfigurable Intelligent Surface, Rich Scattering Environment, Multi-Path, Artificial Neural Network

Radio localization is a key enabling technology for situational awareness but conventional techniques based on elaborate ray-tracing approaches naturally struggle in rich scattering environments (inside rooms, metro stations, planes, vessels, . . . ). Here, we discuss a completely different approach to radio localization: instead of attempting to understand rich scattering wave propagation in terms of rays, we harness the overwhelming complexity because it assigns unique wave fingerprints to each object position. We interpret wave propagation as a physical encoder of the sought-after localization information in multiplexed measurements and detail artificial neural network (ANN) architectures suitable to decode these measurements for a single or multiple, discrete or continuous, sought-after location variable(s). Capitalizing on recent physics-driven experiments, we clarify that the proposed technique is very robust to measurement noise and capable of achieving deeply sub-wavelength localization precision. The discussed technique can be implemented with multiplexing across spatial, spectral or configurational degrees of freedom, corresponding to sensor networks, broadband measurements and RIS-programmable environments, respectively. Specifically, multiplexing across a fixed random sequence of RIS configurations enables single-frequency localization with a single node. Finally, we propose an end-to-end vision of the technique in which programmable RIS elements take the role of physical weights in a hybrid analog-digital ANN. Thereby, relevant information for the localization task can be discriminated from irrelevant information already in the measurement process, enabling substantial latency improvements.

I. INTRODUCTION

Localization information is a fundamental ingredient of situational awareness and underlies several emerging technologies expected to impact society at large, including the areas of mobility (autonomous vehicles and robots), health-care (ambientassisted living [START_REF] Witrisal | Highaccuracy localization for assisted living: 5G systems will turn multipath channels from foe to friend[END_REF]), human-machine interaction (gesture-based commands, virtual reality), retail (asset tracking, customer analytics), and wireless communication (beamforming) [START_REF] Bourdoux | 6G white paper on localization and sensing[END_REF]. Radio technology is attractive for localization tasks because on the one hand it is independent of scene lighting, optical colors and optically opaque layers (clothes, fog, . . . ) while on the other hand it can often leverage existing wirless communication infrastructure, endowing it with a dual communication and sensing functionality. Conventional radio localization relies on ray-tracing, making use of information such as a radio wave's angle or time of arrival. These techniques were conceived for operation in quasi-free space but inherently struggle when confronted with the rich scattering that is typical for environments in which the above-listed applications are desired. In rich scattering environments, wave energy is not incident from one or a few well-defined directions at well-defined moments in time; instead, it is statistically spread across all possible directions and polarizations throughout the propagation environment [START_REF] Alexandropoulos | Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities[END_REF]. The amount of scattering depends on the specific environment's materials and geometry, as well as on the radio wave's frequency; for typical frequencies of a few GHz, as used in WiFi, many common scenarios taking place inside rooms, metro stations, planes or vessels are confronted with strong reverberation and multipath effects.

Dealing with rich scattering environments from a ray tracing perspective, in particular distinguishing objects of interest from other strong reflectors, is highly challenging. Current proposals tackle mildly scattering environments in which it may be feasible to account for known scatterers with identified position in ray tracing [START_REF] Meissner | Uwb for robust indoor Weighting of multipath components for efficient estimation[END_REF], [START_REF] Leitinger | A belief propagation algorithm for multipath-based SLAM[END_REF], to conduct a statistical analysis of the channels [START_REF] Patwari | Locating the nodes: cooperative localization in wireless sensor networks[END_REF], [START_REF] Mendrzik | Enabling situational awareness in millimeter wave massive MIMO systems[END_REF], to leverage large-scale antenna arrays [START_REF] Li | Massive MIMO-based localization and mapping exploiting phase information of multipath components[END_REF], [START_REF] Wen | 5G positioning and mapping with diffuse multipath[END_REF], or to use an array of programmable reflectors (a "Reconfigurable Intelligent Surface", RIS) as known and controllable reference or anchor points [START_REF] Wymeersch | Radio localization and mapping with reconfigurable intelligent surfaces[END_REF]. "Single-bounce" assumptions [START_REF] Wen | 5g synchronization, positioning, and mapping from diffuse multipath[END_REF] can be suitable simplifications of the multipath problem at higher frequencies. Some of the abovedescribed approaches have been tested experimentally in an indoor environment using 2-GHz-bandwidth signals [START_REF] Meissner | Uwb for robust indoor Weighting of multipath components for efficient estimation[END_REF], [START_REF] Leitinger | A belief propagation algorithm for multipath-based SLAM[END_REF], [START_REF] Leitinger | Multipath-based slam exploiting aoa and amplitude information[END_REF] or using a base-station equipped with a large amount of antennas [START_REF] Li | Massive MIMO-based localization and mapping exploiting phase information of multipath components[END_REF].

Instead of struggling to deal with multi-path in ray-tracing frameworks, a completely different approach consists in harnessing the overwhelming scattering richness via wave fingerprinting (WFPing). The wave field in a rich scattering environment is extremely sensitive to geometrical perturbations such that the wave fields act as WFPs for different object positions. To unambiguously identify the WFPs, they must be "inked" with sufficient distinct measurements. These measurements of the wave field can multiplex the sought-after location information across spatial, spectral or configurational degrees of freedom (DoF) of the system. Spatial DoF are accessed via large-scale antenna arrays [START_REF] Vari | mmWaves RSSI Indoor Network Localization[END_REF]- [START_REF] Vieira | Deep convolutional neural networks for massive mimo fingerprint-based positioning[END_REF], spectral DoF via (ultra-) broadband measurements [START_REF] Jin | Position location by time reversal in communication networks[END_REF]- [START_REF] Chen | Achieving centimeter-accuracy indoor localization on wifi platforms: A frequency hopping approach[END_REF], and configurational DoF via a pre-defined series of random configurations of a RIS-programmable propagation environment [START_REF] Del Hougne | Precise localization of multiple noncooperative objects in a disordered cavity by wave front shaping[END_REF]. The latter has the advantage of using only a single node and a single frequency, resulting in a very light hardware layer and minimal spectrum allotment concerns. Moreover, unlike RIS-based localization proposals in quasi-free space [START_REF] Abu-Shaban | Near-field localization with a reconfigurable intelligent surface acting as lens[END_REF], the use of random sequences for WFPing implies no need to model and optimize the RIS configuration. Not relying on a RIS model is of practical importance because to date no RIS models that faithfully capture all physically important details (inter-element coupling, dependence on angle of incidence, frequency and polarization, influence of nearby scatterers) is available.

Here, we discuss WFP-based radio localization in rich scattering environments from an information flow perspective in order to clarify important questions about its potential limitations (achievable localization precision, performance in dynamic environments). We interpret wave propagation in the rich scattering environment as physical encoder of the sought localization information, and discuss how this information can be decoded from the measurements. In particular, we argue that artificial neural networks (ANNs) are a good choice of digital decoder and discuss suitable architectures in various scenarios. Finally, the information flow perspective allows us to envision an end-to-end optimized localization scheme in which the programmable propagation environment plays the role of a first trainable ANN layer, holding the promise of significant latency improvements. With this unifying perspective on information flow, we bring together somewhat disjoint research threads on wave physics, metamaterials, and signal processing.

II. INFORMATION FLOW

To start, we analyze how information flows through the physical and digital layers of the localization sensing pipeline. For concreteness, we consider the use of RIS-based configurational diversity in Fig. 1. Wave propagation encodes the sought after information in the multiplexed measurements, and the digital layer serves as decoder of the desired information.

Many choices of digital decoder are conceivable. When the measurements are complex-valued, a phase retrieval problem is avoided and the problem can be framed as a matrix inversion, which can be further refined via various regularization strategies. A digital decoder that works irrespective of the availability or not of phase information is a correlation-based dictionary search, sometimes also marketed as "virtual time reversal" if spectral DoF are used. A more modern strategy is to train an ANN to approximate the inverse function of the physical encoder. Refs. [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF], [START_REF] Del Hougne | Deeply sub-wavelength localization with reverberation-coded-aperture[END_REF] both observed that ANN decoders are significantly more robust to noise than their competitors. A further advantage, in comparison to inversion or correlation, is that during inference the forward pass through the ANN is likely faster.

III. NEURAL NETWORK ARCHITECTURE

In this section, we discuss the choice of layer architecture, activation functions and cost functions in various scenarios.

A. Fully Connected or Convolutional Layers?

Convolutional architectures are currently very popular in deep learning, including in WFP-based localization in rich scattering environments [START_REF] Vieira | Deep convolutional neural networks for massive mimo fingerprint-based positioning[END_REF]. Their popularity originates from their success in computer vision, where convolutional layers excel at extracting local features from data [START_REF] Zheng | Processing global and local features in convolutional neural network (cnn) and primate visual systems[END_REF]. Computer vision deals with optical images in which there is a oneto-one mapping between scene and image. In sharp contrast, there is no one-to-one mapping between scene and multiplexed measurements in WFP-based localization data. Information from all parts of the scene is scrambled via wave propagation in the rich scattering environment, and each scene component contributes to each measurement. Therefore, local scene features are very likely encoded in long-range correlations in the multiplexed data and hence fully connected layers capable of identifying global features in data from multiplexed measurements appear preferable. Indeed, recent work on the recovery of scrambled optical images after propagation through a multimode fiber confirmed this hypothesis [START_REF] Zhu | Image reconstruction through a multimode fiber with a simple neural network architecture[END_REF], and successful WFP-based radio localization with a simple ANN consisting only of two fully connected layers was reported [START_REF] Alexandropoulos | Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities[END_REF], [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF].

B. Choice of Activation and Cost Functions

Depending on whether the sought-after location variable is discretized or continuous, the problem falls into the categories of classification or regression, respectively, with important consequences for the choice of activation and cost function.

Initial works [START_REF] Alexandropoulos | Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities[END_REF], [START_REF] Del Hougne | Precise localization of multiple noncooperative objects in a disordered cavity by wave front shaping[END_REF], [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF] discretized the location variable via a grid of predefined positions on which the object was allowed to be located. For a single object, this is a standard classification problem that can be tackled with ReLU activation functions, except for the last layer on which a SoftMax activation should be applied, in combination with a crossentropy cost function [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF]. If more than one object is to be localized, the classes are independent and not mutually exclusive, resulting in a multi-class multi-label classification problem. To that end, the activation function of the last layer should be a Sigmoid rather than a SoftMax function, again in combination with a cross-entropy cost function [START_REF] Alexandropoulos | Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities[END_REF]. 1More recently, in order to probe the achievable localization precision, Ref. [START_REF] Del Hougne | Deeply sub-wavelength localization with reverberation-coded-aperture[END_REF] treated the location variable as continuous. For such a regression problem, it was heuristically observed that Sigmoid activation functions should be used after the fully connected layers; no activation function should be applied to the output layer. The cost function should be the mean error (or root-mean-square error), i.e. not a cross-entropy based cost function.

IV. POTENTIAL LIMITATIONS

In this section, we address two potential limitations of WPFbased radio localization.

A. Achievable Precision

It is often assumed that the localization precision can be at best half the operating wavelength because of the diffraction limit. However, the diffraction limit applies to energy focusing whereas the WFP-based localization operates with multiplexed measurements without any energy focusing. Moreover, the crucial roles of a priori knowledge and signal-to-noise (SNR) ratio are ignored by the diffraction-limit argument. In fact, reverberation in a rich scattering environment encodes deeply sub-wavelength information in the multiplexed measurements, and the longer the waves dwell in the environment, the more efficiently they encode such information [START_REF] Del Hougne | Deeply sub-wavelength localization with reverberation-coded-aperture[END_REF]. Intuitively, this link between dwell time and localization precision is easily understood: the probability that a deeply sub-wavelength detail impacts the evolution of a wave is higher if the wave reverberates longer in the interaction domain. Specifically, this means that the localization precision in rich scattering media can vastly exceed that achievable in quasi-free space, underlining the potential of harnessing rich scattering. The challenge lies then only in extracting the sought-after information from the multiplexed measurements, and an ANN decoder is a suitable way of doing so [START_REF] Del Hougne | Deeply sub-wavelength localization with reverberation-coded-aperture[END_REF]. There is hence no fundamental wavelength-induced bound on the achievable localization precision. Instead, in practice, the latter depends on physical parameters (dwell time and SNR), the available a priori knowledge, and the choice of digital decoder.

B. Operation in Dynamic Environments

A common objection to WFP-based localization are doubts about its ability to perform well in dynamic environments. Indeed, WFPing is based on the sensitivity of chaotic wave fields to the object to be localized, but the sensitivity of course equally applies to parasitic perturbations. In principle, the perturbation can be interpreted and treated as an effective source of noise. The worse the effective SNR, the less useful information is obtained per measurement. However, this can be counterbalanced by taking more measurements with distinct measurement modes. Therefore, even if the parasitic perturbations are stronger than those of the object(s) of interest, in principle that does not automatically preclude the successful implementation of WFP-based localization [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF]. However, the stronger the parasitic perturber is, the more important it becomes to thoroughly characterize it such that the ANN can faithfully extract statistically relevant information from perturbed measurements. Specifically, this means that the training data set should capture a representative set of realizations of the perturbations that can be expected during on-line inference.

V. END-TO-END VISION

Reducing latency is a crucial goal for many localization applications. In terms of information flow, it is obvious that the above-discussed WFP-based schemes do not discriminate between relevant and irrelevant information during the physical encoding in multiplexed measurements. If only information relevant to the localization task was included in the measurements, fewer measurements would be needed to acquire the same amount of useful information, and less data would have to be processed, resulting in a double advantage in terms of latency.

If RIS-enabled configurational DoF are used for WFPing, it is possible to optimize the measurement modes. A random sequences of RIS configurations, as discussed thus far, yields pseudo-orthogonal measurement modes. This means that not only do the measurement modes not filter out irrelevant information but subsequent measurements in fact acquire to some extent redundant information due to the finite overlap of the measurement modes. This redundancy can be removed by optimizing the RIS sequence such that the measurement modes are truly orthogonal [START_REF] Del Hougne | Optimal multiplexing of spatially encoded information across custom-tailored configurations of a metasurface-tunable chaotic cavity[END_REF]. Such a sequence can be identified iteratively in the experiment, without needing to model the interaction of waves with the RIS. However, truly measurement modes still fail to separate relevant from irrelevant information. Fig. 2. "Learned Sensing" Architecture for RIS-Based Radio Localization. Learned Sensing is based on a hybrid analog-digital ANN, consisting of a physical layer on which the RIS elements' configurations are the trainable physical weights, followed by several traditional digital layers, with conventional digital weights. I denotes the identity function. The physical layer performs the function f P HY S ( r obj , {C i }) = M, where r obj is the object position, {C i } is a set of M RIS configurations, and M is the 1 × M measurement vector (one measurement per RIS configuration). M can be real or complex valued, depending on whether phase information is measured or not. A neural network with trainable weights θ is now trained by optimizing θ such that f F W M ( r obj , {C i }, θ) ≈ f P HY S ( r obj , {C i }) = M. Second, this learned forward model f F W M is integrated into the learned sensing pipeline. Now, θ remains fixed to the optimized values from the first step. The learned sensing network has a set of trainable physical weights, namely the RIS configurations {C i }, and a set of trainable digital weights, namely the weights of the digital decoding network φ. Overall, the learned sensing network is supposed to perform a function f LS ( r obj , {C i }, φ) ≈ r obj . To that end, its trainable physical and digital weights are jointly optimized.

Following the recently introduced "learned sensing" paradigm in electromagnetic imaging and gesture recognition [START_REF] Del Hougne | Learned integrated sensing pipeline: Reconfigurable metasurface transceivers as trainable physical layer in an artificial neural network[END_REF], to endow the physical layer with task-awareness, we propose to interpret the RIS-programmable rich scattering propagation environment as a trainable physical layer of a hybrid analog-digital neural network, followed by the digital layers discussed previously. The traditional WFP architecture from Fig. 1 also seeks to ultimately approximate an identity function by using f DIGIT ≈ f -1 P HY S , however, therein f P HY S is fixed and not trainable (a fixed sequence of random RIS configurations is used). In Fig. 1, only the digital layer can be optimized.

The architecture in Fig. 2 is reminiscent of an auto-encoder (see Ch. 14 in Ref. [START_REF] Goodfellow | Deep Learning[END_REF]) in that it seeks to map the information space to itself by approximating an identity function. It lacks, however, the common architectural symmetry of autoencoders. Most importantly, the encoding step of the proposed "learned sensing" pipeline happens in the physical rather than digital world -a fundamental difference from conventional alldigital auto-encoders. Conventional auto-encoders are a signal processing tool aimed at dimensionality reduction of measured data, whereas here we aim at optimal joint measuring and processing of data given a variable of interest. Auto-encoders can also be trained to yield representations of corrupted input data, known as denoising auto-encoders [START_REF] Vincent | Extracting and composing robust features with denoising autoencoders[END_REF]; this functionality is inherent in our proposal because the physical layer involves measurements that are inevitably affected by measurement noise.

In practice, an important challenge will be to obtain a for-ward model of the physical layer such that it can be integrated into the hybrid analog-digital end-to-end pipeline. Analytical forward models seem difficult to realize for localization in rich scattering environments for two reasons: (i) no models of RIS that faithfully capture the physics are available, and (ii) modelling a rich scattering environment via ray tracing is prohibitively expensive. Instead, it appears best to resort to a learned forward model [START_REF] Li | Intelligent electromagnetic sensing with learnable data acquisition and processing[END_REF]. To that end, as illustrated in Fig. 3, in a first step, an auxiliary neural network is trained such that the function f F W M that it performs approximates the physical layer; specifically, f F W M predicts the measurements M as a function of a set of RIS configurations {C i } and the object position r obj . To train this neural network, a representative sample of measurements obtained for different object positions and random RIS configurations is required. In a second step, this learned forward model is integrated into the "learned sensing" pipeline, where {C i } are the physical trainable weights and φ are the digital trainable weights. The learned sensing network is then trained by jointly optimizing {C i } and φ. Thereby, the RIS configurations learn to take into account the ultimate localization task, making it possible to discriminate between relevant and irrelevant information already in the measurement process. Since each measurement will then contain more information relevant to the localization task, fewer measurements will be needed to successfully perform the localization task. The shorter measurement process, combined with a lower data processing burden, will enable significant latency improvements. Moreover, the reduced number of necessary measurements will favorably impact other metrics such as power consumption and radiation exposure. We are confident that these latency trends which were evidenced in recent "learned sensing" studies of electromagnetic imaging in free space [START_REF] Del Hougne | Learned integrated sensing pipeline: Reconfigurable metasurface transceivers as trainable physical layer in an artificial neural network[END_REF], [START_REF] Li | Intelligent electromagnetic sensing with learnable data acquisition and processing[END_REF] will also apply to localization in richscattering environments: on a very abstract level, localizing an object can be interpreted as imaging of an extremely sparse scene; the environment's rich-scattering nature can be interpreted as part of the "scene illumination hardware".

VI. CONCLUSION

To summarize, we discussed WFP-based radio localization from an information flow perspective, with a focus on the use of configurational DoF offered by a RIS-equipped propagation environment. We considered technical details of ANN-based digital decoders (layer architecture, choice of activation and cost functions) in various scenarios. Moreover, we clarified the absence of wavelength-induced bounds on the achievable localization precision, as well as that operation in dynamic environments is feasible. Then, we introduced an end-to-end vision for a hybrid analog-digital localization pipeline which treats the RIS-programmable environment as a first trainable physical layer. On the one hand, thereby wave propagation acts as analog processor. On the other hand, the measurement process pre-selects information relevant to the localization task which avoids the acquisition of irrelevant information. We expect the end-to-end pipeline to enable significant latency improvements. Looking forward, this vision can be further refined within the realm of reinforcement learning to update the sequence of RIS configurations in real time [START_REF] Mnih | Recurrent models of visual attention[END_REF].

Fig. 1 .

 1 Fig. 1. Information Flow in WFP-Based Radio Localization using RIS-Based Configurational Diversity. Wave propagation in the RIS-parametrized rich scattering environment encodes the information about the object location, r obj , in a measurement vector M = f P HY S ( r obj , {C i }). The ith measurement is a single value (real or complex, depending on whether phase information is acquired) corresponding to the ith fixed random RIS configuration C i . M concatenates the M measurements obtained with the fixed series of M random RIS configurations, {C i }. The digital layer then attempts to decode M by approximating the inverse function of f P HY S to obtain an estimate of the object position: r obj = f DIGIT (M) ≈ r obj .

Fig. 3 .

 3 Fig. 3. Implementing a Realistic Learned Sensing Localization Pipeline. First, a learned forward model is established to approximate the physical layer.The physical layer performs the function f P HY S ( r obj , {C i }) = M, where r obj is the object position, {C i } is a set of M RIS configurations, and M is the 1 × M measurement vector (one measurement per RIS configuration). M can be real or complex valued, depending on whether phase information is measured or not. A neural network with trainable weights θ is now trained by optimizing θ such that f F W M ( r obj , {C i }, θ) ≈ f P HY S ( r obj , {C i }) = M. Second, this learned forward model f F W M is integrated into the learned sensing pipeline. Now, θ remains fixed to the optimized values from the first step. The learned sensing network has a set of trainable physical weights, namely the RIS configurations {C i }, and a set of trainable digital weights, namely the weights of the digital decoding network φ. Overall, the learned sensing network is supposed to perform a function f LS ( r obj , {C i }, φ) ≈ r obj . To that end, its trainable physical and digital weights are jointly optimized.

Authorized licensed use limited to: INRIA. Downloaded on November 12,2022 at 13:04:04 UTC from IEEE Xplore. Restrictions apply.

In both cases, it is important in practice to make sure the last activation function is not accidentally applied twice; for instance, in Tensorflow, crossentropy loss functions such as softmax cross entropy with logits compute internally the softmax of their inputs.

IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)


Chloé Saigre-Tardif
email: chloe.saigre-tardif@insa-rennes.fr

Philipp Del
email: philipp.del-hougne@univ-rennes1.fr

A self-adaptive RIS that estimates and shapes fading rich-scattering wireless channels

Keywords: reconfigurable intelligent surface, localization, sensing, channel estimation, fading channels, rich scattering, integrated sensing and communication

We present a framework for operating a selfadaptive RIS inside a fading rich-scattering wireless environment. We model the rich-scattering wireless channel as being doubleparametrized by (i) the RIS, and (ii) dynamic perturbers (moving objects, etc.). Within each coherence time, first, the self-adaptive RIS estimates the status of the dynamic perturbers (e.g., the perturbers' orientations and locations) based on measurements with an auxiliary wireless channel. Then, second, using a learned surrogate forward model of the mapping from RIS configuration and perturber status to wireless channel, an optimized RIS configuration to achieve a desired functionality is obtained. We demonstrate our technique using a physics-based end-toend model of RIS-parametrized communication with adjustable fading (PhysFad) for the example objective of maximizing the received signal strength indicator. Our results present a route toward convergence of RIS-empowered localization and sensing with RIS-empowered channel shaping beyond the simple case of operation in free space without fading.

I. INTRODUCTION

We introduce an artificial intelligence (AI) empowered approach to converging channel estimation and shaping enabled by reconfigurable intelligent surfaces (RISs) in fading rich-scattering wireless environments. The ability of RISs to shape wireless channels within the emerging "smart radio environment" paradigm pivotally relies on channel state knowledge. Most algorithmic explorations of RISs assume that the channels are known, but gaining this channel knowledge once per coherence time in practice will involve significant overhead [START_REF] Zappone | Overhead-Aware Design of Reconfigurable Intelligent Surfaces in Smart Radio Environments[END_REF], a realization that has even given rise to ideas for how to use RIS without deterministic channel information [START_REF] Abrardo | Intelligent Reflecting Surfaces: Sum-Rate Optimization Based on Statistical Position Information[END_REF]- [START_REF] Stylianopoulos | Deep-Learning-Assisted Configuration of Reconfigurable Iintelligent Surfaces in Dynamic Rich-Scattering Environments[END_REF].

If the receiver is cooperative, the channels can be measured via pilot signals, as recently studied for RISs in free space [START_REF] Wang | Channel estimation for intelligent reflecting surface assisted multiuser communications: Framework, algorithms, and analysis[END_REF]- [START_REF] Hu | Two-timescale channel estimation for reconfigurable intelligent surface aided wireless communications[END_REF]. Alternatively, and this is the case we focus on, the RIS can be equipped with a sensing functionality to determine itself the channel state information. First efforts to integrate sensing functionalities for channel estimation into RISs have been reported for operation in free space where the problem of channel estimation is comparatively simple, requiring, for instance, only a direction-of-arrival estimation [START_REF] Ma | Smart metasurface with self-adaptively reprogrammable functions[END_REF]- [START_REF] Dai | Simultaneous in-situ direction finding and field manipulation based on space-time-coding digital metasurface[END_REF].

But realistic propagation environments often involve complex fading effects significantly beyond the validity of a simple free-space approximation. Deployment scenarios of RISs inside metallic enclosures such as vessels, planes, trains or busses will give rise to channels with strong multipath characteristics; at low frequencies (< 6 GHz), even office rooms present substantial amounts of reverberation [START_REF] Kaina | Shaping complex microwave fields in reverberating media with binary tunable metasurfaces[END_REF], [START_REF] Del Hougne | Optimally diverse communication channels in disordered environments with tuned randomness[END_REF]. In such rich-scattering scenarios, the wireless channels are the superposition of many reflected waves arriving from seemingly arbitrary angles such that common free-space intuition and approaches cannot be applied [START_REF] Alexandropoulos | Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities[END_REF]. The complexity of the problem calls for data-driven AI approaches.

In our approach, we consider a double-parametrization of the fading rich-scattering wireless channels:

1) the desired RIS-parametrization.

2) the undesired channel dependence on some dynamic channel-perturbing parameters such as the location of the receiver, the location of moving objects, the orientation of rotating objects, etc. We will thus proceed in three steps:

1) Learn a surrogate forward model to predict the channel as a function of RIS parameters and perturbing parameters. 2) Estimate the perturbing parameters.

3) Optimize the RIS parameters, given the perturbing parameters, to achieve the desired shaping of the wireless channel. While step 1 is a one-off calibration step for a given scenario, steps 2 and 3 are repeated once within every channel coherence time.

The first step of learning a parametrized forward model of complex fading channels cannot be approached analytically in our rich-scattering setting (unlike its free-space counterpart).

The second step can build on existing literature for RISassisted sensing in rich-scattering environments. Specifically, a fixed series of random RIS configurations can offer sufficient configurational diversity to estimate the location of a non-cooperative object based on transmission measurements between a single pair of nodes [START_REF] Del Hougne | Precise Localization of Multiple Noncooperative Objects in a Disordered Cavity by Wave Front Shaping[END_REF], even deeply subwavelength precision with intensity-only measurements is attainable [START_REF] Del Hougne | Deeply Subwavelength Localization with Reverberation-Coded Aperture[END_REF]. This is essentially an instance of compressed sensing, where the rich-scattering environment acts as RIS-parametrized coded aperture. Importantly, the Fig. 1. Algorithmic summary. First, the pertuber status θ is determined using S-ANN based on measurements of the transmitted spectrum between the transmitter (TX) and an auxiliary receiver (AR) for a fixed series of random RIS configurations. Second, a simple iterative algorithm (Algorithm 1) is used to optimize a RIS configuration for RSSI boosting. At every iteration, the cost function evaluation is based on the prediction by CE-ANN of the transmitted spectrum between TX and receiver RX. CE-ANN takes the estimated perturber status from S-ANN and the RIS configuration to be evaluated from Algorithm 1 as input. C and C denote RIS configuration and cost function, respectively. nodes can be positioned at any fixed but arbitrary point within the environment. This wave-fingerprinting-based localization technique can be implemented robustly in dynamic environments [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF] and it may be possible to learn a shorter task-specific fixed series of RIS configurations to improve latency through "learned sensing" [START_REF]RIS-Based Radio Localization in Rich Scattering Environments: Harnessing Multi-Path with ANN Decoders[END_REF]. The related sensing task of object recognition (instead of localization) inside a rich scattering environment has also been implemented with compressed sensing using spectral diversity (broadband measurements) [START_REF] Frazier | Deep-learning estimation of complex reverberant wave fields with a programmable metasurface[END_REF].

The third step of optimizing the RIS parameters cannot be conveniently implemented by running the forward model from step 1 in reverse with fixed weights and trainable inputs. The reason is the 1-bit (or few-bit) programmability of the RIS elements and the fact that the learned forward model is not guaranteed to interpolate in a physically meaningful way between the discretized inputs from the training data.

Directly learning an inverse (instead of forward) model that outputs a suitable RIS configuration as a function of the desired wireless channel and perturber status does not present itself as viable alternative to following steps 1 and 3 because (i) the forward model is a many-to-one mapping (within reasonable precision limits), and (ii) determining desired wireless channel characteristics in accordance with the laws of physics is nontrivial.

In the remainder of this paper, we detail our preliminary implementation of the above approach to realizing a selfadaptive RIS for channel sounding and shaping beyond the simple free-space case. To work with the substantially more challenging case of fading rich-scattering channels, we use an end-to-end model for RIS-parametrized fading channels that fully complies with wave physics [START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF].

II. OPERATION PRINCIPLE

The fading rich-scattering wireless channel between transmitter TX and receiver RX, H RX-T X , is parametrized by (i) the RIS configuration, C, and (ii) the status of the dynamic perturber, θ. C is a 1 × N RIS vector, where N RIS is the number of RIS elements. θ is a vector concatenating all necessary parameters to fully describe the pertuber status. In the simple case of a single irregularly shaped object rotating around its own axis, θ is simply the object's angular orientation. In general, however, the perturbation can depend on multiple parameters that would be collected in θ. In addition, we assume the existence of an auxiliary wireless channel, H AR-T X , between the transmitter and an auxiliary receiver, AR, located at a fixed arbitrary position.

An algorithmic summary of the self-adaptive RIS' workflow performed during each coherence time is provided in Fig. 1. First, the self-adaptive RIS estimates the current state of the perturber, θ, by measuring the field transmitted from TX to AR, H AR-T X , for a fixed series of random RIS configurations. To this end, the artificial neural network (ANN) coined S-ANN is used. Second, based on θ, the self-adaptive RIS identifies a suitable configuration in order to implement a desired channel shaping functionality. To that end, the RIS uses the simple iterative Algorithm 1; every evaluation of the cost function in Algorithm 1 makes use of a second ANN coined CE-ANN that estimates the channel between TX and RX for a given θ and C.

We use the ANNs (S-ANN and CE-ANN) as faithful interpolators of functions for which no analytical expression exists.

Remark 1: For free-space operation, the channel H RX-T X is usually decomposed into the channel from the TX to the RIS, the RIS configuration, and the channel from the RIS to the RX. Such a simple linear cascaded model is not applicable in rich-scattering environments where each path may encounter different RIS elements along its complicated trajectory. The parametrization of the channel H RX-T X through RIS (and dynamic perturbers) is hence highly complicated and analytically intractable, which is why we learn the surrogate forward model CE-ANN.

Remark 2: The stronger the perturber alters the auxiliary wireless channel H AR-T X , the more accurately S-ANN can estimate θ with a limited sequence of measurements and/or in the presence of measurement noise [START_REF] Del Hougne | Deeply Subwavelength Localization with Reverberation-Coded Aperture[END_REF]. Various implementations of S-ANN (for different sensing tasks and measurement protocols) were reported in Refs. [START_REF] Alexandropoulos | Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities[END_REF]- [START_REF] Frazier | Deep-learning estimation of complex reverberant wave fields with a programmable metasurface[END_REF].

III. PROBLEM STATEMENT

For concreteness, we illustrate our proposed generic technique for the specific objective of maximizing the received signal strength indicator (RSSI, i.e., the magnitude of the transmitted spectrum) at a desired frequency between a fixed transmitter and a fixed receiver inside a rich-scattering enclosure including dynamic uncontrolled perturbers and a conformal distributed 1-bit-programmable RIS. As illustrated in Fig. 2a, the perturbers are four irregularly shaped objects rotating in an uncontrolled manner around their centers. We model the wireless channels with PhysFad [START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF], a physicsbased end-to-end communication model for RIS parametrized wireless environments with adjustable fading. PhysFad is based on a coupled-dipole formalism constructed from first principles and thereby inherently obeys all essential physical principles, including

• a notion of space and causality, • dispersion (frequency selectivity) and the intertwinement of a RIS element's phase and amplitude response, • any arising mutual coupling effects, including long-range mesoscopic correlations, and • the non-linear parametrization of wireless channels through RIS and perturbing objects. PhysFad is formulated, without loss of generality, in arbitrary units such that the central operating frequency as well as the medium's permittivity and permeability are all defined to be unity [START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF]. The utilized PhysFad parameters are summarized for the sake of repeatability in Table I -the interested reader is referred to Ref. [START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF] for details on PhysFad.

The impact of the RIS and/or perturbers on H RX-T X is illustrated in Figs. 2b-d. The standard deviation σ of the distribution of the complex-valued H RX-T X for random choices of C with fixed θ (Fig. 2c) is seen to be comparable to that for random choices of θ with fixed C (Fig. 2d). If both C and θ are chosen randomly, σ is of course higher (Fig. 2b). Overall, the non-trivial parametric dependence of H RX-T X on C and θ is obvious, excluding the use of analytical channel models as in free space.

IV. METHODS

In this section, we present preliminary results for the three-step procedure outlined in the Introduction to realize a self-adaptive RIS under fading rich-scattering conditions. We leave discussions of technical details like the influence of the amount of training data and the choice of ANN hyperparameters for future work. We work on purpose with "simple" fully connected layers because the rich-scattering enclosure scrambles information such that the features we seek to identify are likely to be encoded in long-range correlations [START_REF] Del Hougne | Deeply Subwavelength Localization with Reverberation-Coded Aperture[END_REF], [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF].

A. Step 1: CE-ANN

Our CE-ANN consists of two fully connected layers with 64 neurons and ReLu activation, followed by a linear output layer. Tackling a regression (rather than classification) problem, our cost function is the mean-squared-error (MSE) of the predicted H RX-T X . For training, we use a labeled dataset with 10 5 entries {C, θ, H RX-T X } of which 15% are used for validation to decide when to stop the training in order to avoid overfitting.

B. Step 2: S-ANN

Our S-ANN consists of three fully connected layers with 256, 128 and 26 neurons, respectively, and ReLu activation, followed by a linear output layer. Tackling another regression problem, our cost function is again a MSE, this time the MSE of the predicted θ (defined in terms of arc length on a unit circle to account for the periodicity). For training, we use a labeled dataset with 10 3 entries {θ, {H AR-T X,i }} where the set {H AR-T X,i } denotes the collection of transmission spectra of the auxiliary wireless channel for a fixed series of 10 random RIS configurations. 15% of the data are used for validation to decide when to stop the training in order to avoid overfitting.

C. Step 3: Algorithm 1

Algorithm 1 is implemented as detailed in Fig. 1, making use of CE-ANN everytime it evaluates the cost function. The θ-input of CE-ANN is obtained from the output of S-ANN.

V. RESULTS

Our preliminary results are summarized in Fig. 3. After training S-ANN and CE-ANN as detailed in Sec. IV, we test the performance of our self-adaptive RIS for the designated task of optimizing the RSSI on the wireless link H RX-T X for 100 random test instances of the dynamic rich-scattering environment. For each test instance, a random perturber status θ is chosen. The 10 measurements of the auxiliary channel H AR-T X for the 10 fixed random RIS configurations are made, and an estimate θ of the perturber status is obtained. The results in Fig. 3a reveal excellent precision in estimating the perturber status. As noted above, estimating the perturber status is relatively easy for large perturbers as in the considered case.

Having obtained θ in a given test instance, we next run Algorithm 1 to optimize the RIS configuration for the designated channel shaping functionality (RSSI maximization). The final optimized RIS configuration is evaluated in PhysFad to determine the achieved RSSI improvement independently from the quality of CE-ANN as a surrogate forward model. We also evaluate the RSSI for 500 random RIS configurations as benchmark. The results displayed in Fig. 3b indicate clear RSSI improvements in every test instance for our self-adaptive RIS. On average, the Fig. 3. Results obtained for 100 test instances with random perturber status θ. a) Estimation of perturber status θ (green) from S-ANN, and ground truth θ (violet). b) For each test instance, the average of RSSI over 500 random RIS configurations (blue) is contrasted with the RSSI of the RIS configuration optimized for the test instance using Algorithm 1 (red). While the identification of the optimized RIS configuration is solely based on CE-ANN, the final displayed RSSI results are evaluated in PhysFad [START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF] to avoid reliance on CE-ANN being a high-quality representation of the underlying model.

magnitude of H RX-T X is doubled, corresponding to a fourfold intensity improvement.

VI. CONCLUSION

We have introduced a paradigm through which a RIS can self-adapt its configuration inside a dynamic rich-scattering environment to reliably implement a desired channel shaping functionality despite dynamic perturbations. The conception of a self-adaptive RIS is particularly challenging to realize in our setting of a rich scattering environment where no simple analytical channel description exists. In practice, our technique will be useful when the channel coherence time is significantly longer than the time it takes our RIS to self-adapt through the procedure summarized in Fig. 1. Current RIS prototypes can refresh their configuration within 20 µs (e.g., in Ref. [START_REF] Zhao | Metasurface-assisted massive backscatter wireless communication with commodity wi-fi signals[END_REF]), with further room for improved refresh rates, suggesting that our method may serve in many realistic scenarios, e.g., where human motion is the perturbation.

Looking forward, our technique can be extended to scenes with more complicated dynamics (e.g., complex motion of multiple perturbing objects and moving receiver). In particular, the scalability of our approach to scenarios with more independent perturbers as well as the impact of the SNR on the performance deserve attention. The training procedure to adapt to a specific environment may be eased in the future through transfer learning. Furthermore, we envision smarter ANNs that anticipate motion and cognitively select intelligent sensing patterns [START_REF] Saigre-Tardif | Intelligent Meta-Imagers: From Compressed to Learned Sensing[END_REF]. Overall, it remains to be determined whether the system level advantages of smart radio environments justify the cost of the associated overhead (in our work, going through the procedure summarized in Fig. 1).

Fig. 2 .

 2 Fig.2. a) Considered setup in PhysFad[START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF]: An electrically large, irregularly shaped, rich-scattering enclosure inside which four irregular objects rotate (for simplicity in sync) in an uncontrolled manner. In addition, a distributed conformal RIS partially covers the enclosure's walls. The RIS consists of 100 1-bit programmable RIS elements which are clustered into macro-pixels of four RIS elements. An example configuration is indicated, where the RIS elements in the ON state are highlighted with a green circle. The wireless channel of interest, H RX-T X , is parametrized by the RIS configuration C and the perturber status θ. The self-adaptive RIS is equipped with an auxiliary receiver (AR) at a fixed arbitrary location such that it can estimate the perturber status based on measurements of the auxiliary wireless channel H AR-T X . b)-d) Distribution of H RX-T X in the complex plane for 50 random realizations of C and θ (b), of C with fixed θ (c), and of θ with fixed C. σ denotes the standard deviation of the complex-valued H RX-T X .

TABLE I SUMMARY

 I OF UTILIZED PHYSFAD[START_REF] Faqiri | PhysFad: Physics-Based End-to-End Channel Modeling of RIS-Parametrized Environments with Adjustable Fading[END_REF] PARAMETERS.

Authorized licensed use limited to: INRIA. Downloaded on November 12,2022 at 13:05:30 UTC from IEEE Xplore. Restrictions apply.


Hengxin Ruan

Siyuan Jiang

Hongrui Zhang

Hanting Zhao

Zhuo Wang

Shengguo Hu

Jun Ding

Jun Tie

Cui

Philipp Del Hougne
email: philipp.del-hougne@univ-rennes1.fr

Tie Jun Cui
email: tjcui@seu.edu.cn

Lianlin Li
email: lianlin.li@pku.edu.cn

Microwave Speech Recognizer Empowered by a Programmable Metasurface

We present an experimental prototype of a microwave speech recognizer empowered by a programmable metasurface that can recognize voice commands and speaker identities remotely even in noisy environments and if the speaker's mouth is hidden behind a wall or face mask. Thereby, we enable voice-commanded human machine interactions in many important but to-date inaccessible application scenarios, including smart health care and factory scenarios. The programmable metasurface is the pivotal hardware ingredient of our system because its large aperture and huge number of degrees of freedom allows our system to perform a complex sequence of tasks, orchestrated by artificial-intelligence tools. First, the speaker's mouth is localized by imaging the scene and identifying the region of interest. Second, microwaves are efficiently focused on the speaker's mouth to encode information about the vocalized speech in reflected microwave biosignals. The efficient focusing on the speaker's mouth is the origin of our system's robustness to various types of parasitic motion. Third, a dedicated neural network directly retrieves the sought-after speech information from the measured microwave biosignals. Relying solely on microwave data, our system avoids visual privacy infringements. We expect that the presented strategy will unlock new possibilities for future smart homes, ambient-assisted health monitoring and care, smart factories, as well as intelligent surveillance and security.

Introduction

Voice commands are arguably the most natural approach to human-machine interfaces because speech is the most direct communication method between humans. However, the most obvious approach for the machine to capture voice commands, namely the acquisition of the acoustic signals that are the primary information carrier, precludes many important deployment scenarios. On the one hand, voice commands can be drowned in ambient noise under operation in noisy environments such as streets, public transport, or restaurants. On the other hand, it is impossible to operate under silent-speech requirements [START_REF] Denby | Silent speech interfaces[END_REF] which may arise to preserve privacy, for use in quiet settings like libraries, or through verbally impaired users (e.g., post-laryngectomy). Therefore, a wide variety of indirect secondary carriers of information about voice commands has been explored.

Many of these techniques achieve high accuracy at the price of being highly invasive because they rely on placing sensors (e.g., magnetic 2 , surface electromyographic [START_REF] Meltzner | Silent Speech Recognition as an Alternative Communication Device for Persons With Laryngectomy[END_REF] , infrared [START_REF] Zhang | SpeeChin: A Smart Necklace for Silent Speech Recognition[END_REF] , electropalatographic [START_REF] Kimura | SilentSpeller: Towards mobile, hands-free, silent speech text entry using electropalatography[END_REF] , electromagnetic [START_REF] Birkholz | Non-Invasive Silent Phoneme Recognition Using Microwave Signals[END_REF][START_REF] Wagner | Silent speech command word recognition using stepped frequency continuous wave radar[END_REF] ) directly on the human's body to detect subtle vibrations that are correlated with the speech production. Obviously, such contactbased approaches are oftentimes inconvenient and, moreover, incompatible with largescale deployment in our daily lives. Similar limitations apply to contactless radar-like approaches based on the emission and reception of acoustic [START_REF] Gao | EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users[END_REF][START_REF] Zhang | CELIP: Ultrasonic-based Lip Reading with Channel Estimation Approach for Virtual Reality Systems[END_REF] or electromagnetic [START_REF] Eid | Ultrawideband Speech Sensing[END_REF][START_REF] Shin | Towards Contactless Silent Speech Recognition Based on Detection of Active and Visible Articulators Using IR-UWB Radar[END_REF] waves in the very close proximity (a few centimeters) of the speaker's face. A popular contactless technique for speech recognition that can operate remotely uses optical image sequences as secondary information carrier to recognize speech by analyzing lip or face motion [START_REF] Movellan | Visual Speech Recognition with Stochastic Networks[END_REF][START_REF] Zhao | Lipreading With Local Spatiotemporal Descriptors[END_REF] . However, such visual speech-recognition methods fail under unfavorable lighting conditions such as darkness as well as when the line of sight from the camera to the speaker's mouth is obstructed by a wall or, more recently, a face mask. In addition, the acquisition of camera images risks to infringe the user's privacy. Bi-modal speech recognition approaches, combining, for instance, acoustic and visual inputs [START_REF] Dupont | Audio-visual speech modeling for continuous speech recognition[END_REF] , benefit from richer input information but are also unable to tackle, for instance, the recognition of speech uttered behind a face mask in a noisy environment. An ideal voice-commanded human-machine interface would remotely capture relevant biosignals in a robust, noiseresilient, and privacy-respecting manner while being cheap, consuming little power, and being easy to deploy in our daily lives, even when the speaker's mouth is hidden behind an optically opaque tissue or wall.

The use of microwaves as remote contactless secondary information carrier of voice commands is predestined to meet this formidable challenge. The ability of microwaves as remote, non-ionizing sensing technology to penetrate through visually opaque layers is well known, for example, from airport security checks [START_REF] Gonzalez-Valdes | Improving Security Screening: A Comparison of Multistatic Radar Configurations for Human Body Imaging[END_REF] . However, capturing microwave biosignals that bear sufficient information about the sought-after voice commands is itself very challenging because most signal variations may be due to motion that is not related to speech. Therefore, it is of pivotal importance to focus the microwaves on the speaker's mouth, which in turn requires real-time tracking of the mouth as a pre-requisite. By focusing on the mouth, the weight of reflections from the region of interest (ROI) in the measured signals is drastically increased. Notable results toward that goal were reported in Ref. [START_REF] Wang | Can Hear You with Wi-Fi![END_REF] through a multiple-input-multiple-output (MIMO) beamforming approach at WiFi frequencies in the 2.4 GHz range. However, the underlying multi-channel coherent emission is costly and cumbersome because it requires synchronized sources and individual IQ modulation on each channel. Moreover, using only a few antennas, the setup's degrees of freedom were quite limited, resulting, for instance, in a focal spot that was so large that even winks perturbed the measured signals. A large antenna array emitting coherently controlled wavefronts would be necessary to efficiently localize and focus on the speaker's mouth. Yet this hardware is too costly and power hungry for widespread deployment in human-machine interaction.

In this article, we show that a deep-learning-controlled programmable metasurface fully reaps the benefits of microwave speech recognition with a drastically simpler hardware.

Our programmable metasurface [START_REF] Cui | Coding metamaterials, digital metamaterials and programmable metamaterials[END_REF] is an array of 1024 meta-atoms with individually controllable reflection properties, fed by a single source. Compared to a conventional antenna array comprising a few antennas, we have thus three orders of magnitude more degrees of freedom and, moreover, a much larger aperture. Building on recent results from intelligent computational meta-imaging [START_REF] Saigre-Tardif | Intelligent metaimagers: From compressed to learned sensing[END_REF] , these advantages allow us to localize and focus on the speaker's mouth with unprecedented efficiency. We use this ability to prototype a voice-commanded human-machine interaction scenario in which a speaker who is hidden behind a wall commands a mechanical hand. Our system is capable of tracking a moving speaker in real time, dynamically generating suitable spatial beams for focusing on the speaker's mouth and interpreting the measured biosignals with a deeplearning technique. We also demonstrate multi-speaker listening. Moreover, we shed new light on the mechanisms through which speech information is encoded in microwave biosignals: we demonstrate that, besides the obvious reflection off the mouth, the probing microwave signals partially penetrate through the skin and are affected by the tongue and other vocal entities. Finally, we evidence that our system can also be utilized as biometric identification technology because of the individual manner in which each subject utters speech. Our experimental results enable voice-commanded human-machine interaction at minimal cost in a plethora of challenging and to-date inaccessible scenarios such as health care for assisted living (see Fig. 1a for an example); our results may also be valuable in security applications requiring intelligent surveillance. Photographic image of one out of four panels of our one-bit programmable coding metasurface. The insets show the front and back sides of the designed individual meta-atom. c) Schematic drawing of the hardware configuration of our proposed microwave speech recognizer. The hardware of our system consists of a large-aperture one-bit reprogrammable metasurface, a pair of horn antennas, a vector network analyzer (VNA) and a host computer. d) Experimental characterization of the frequency-dependent phase and amplitude response of our designed meta-atoms in their two possible configurations ("0"/"ON" and "1"/"OFF"). e) and f) Maps of the spatial distribution of the microwave field magnitude (measured via near-field scans) corresponding to the indicated metasurface configurations that are chosen to focus on point A (e) or points B and C (f).

Results

System configuration. We start by elaborating on the system configuration of our implemented prototype for our proposed metasurface-empowered microwave speech recognizer. On the hardware level, our system comprises a large-aperture one-bit reprogrammable metasurface (1024 meta-atoms, 51.2 cm × 51.2 cm aperture), a pair of commercial horn antennas, a vector network analyzer (VNA, Agilent E5071C), and a host computersee Fig. 1c. Our programmable-metasurface-empowered system must accomplish a series of complex tasks on the fly. First, our system must localize the speaker's mouth. This involves imaging the scene and interpreting the resulting image to identify the ROI, that is, the mouth. Second, our system must focus microwaves on the mouth and capture the reflected biosignals. Third, our system must interpret the measured biosignals to extract the sought-after speech content.

Our system's autonomous ROI identification and analysis follows Ref. [START_REF] Li | Intelligent metasurface imager and recognizer[END_REF] : during the first step, the scene is imaged with a series of 18 random illuminations, generated through a known series of 18 random metasurface configurations. The measured data is processed by artificial intelligence (AI) tools to generate a 3D skeleton of the speaker. Details of the utilized AI architecture are provided in the Methods and Supplementary Note 4. Based on the estimated skeleton, the coordinates of the mouth, our ROI, can be deduced. Using a modified Gerchberg-Saxton algorithm, our system then identifies a metasurface configuration that focuses microwaves on the speaker's mouth. A series of probing focused microwave signals is emitted with a period of 70 ms, and the reflected signals are captured by the receiving horn antenna. Note that the use of directive horn antennas, in contrast to omnidirectional antennas, further helps with discriminating ROI signals from multipath scattered signals in the room. The measured biosignals are then interpreted by an artificial neural network (ANN) that directly maps the acquired microwave data to the desired speech content. Such a direct transcription of measured signals with text, without intermediate steps involving phonetic representations, is known as "end-to-end" speech recognition in the signal-processing literature [START_REF] Graves | Towards End-to-End Speech Recognition with Recurrent Neural Networks[END_REF] . Inspired by Ref. [START_REF] Vaswani | Attention is All you Need[END_REF] , we have developed a customized microwave-speech transformer for our system and trained it with supervised learning. To obtain labeled microwave-speech training data, we conveniently used the host computer's built-in microphone that was synchronized with our proposed microwave speech recognizer. Algorithmic details about our microwave-speech transformer are provided in the Methods and Supplementary Note 4.

The first step of our system's pipeline is a conventional instance of compressive imaging by leveraging the configurational diversity of a programmable meta-imager, first reported in Ref. [START_REF] Sleasman | Dynamic metamaterial aperture for microwave imaging[END_REF] (see also Sec. II.B in Ref. [START_REF] Saigre-Tardif | Intelligent metaimagers: From compressed to learned sensing[END_REF] for a balanced review of the field). Nonetheless, our system's pipeline in its entirety qualifies as an instance of "intelligent meta-imaging" according to the taxonomy from Ref. [START_REF] Saigre-Tardif | Intelligent metaimagers: From compressed to learned sensing[END_REF] (see Sec. III.A therein) because AI tools influence the choice of task-specific hardware configurations for our metasurface in the second step of its complex sequences of tasks. We note that related techniques for autonomous ROI identification were put forward in optical ghost imaging based on analytical rather than AI tools [START_REF] Averbuch | Adaptive Compressed Image Sensing Using Dictionaries[END_REF][START_REF] Aβmann | Compressive adaptive computational ghost imaging[END_REF][START_REF] Phillips | Adaptive foveated single-pixel imaging with dynamic supersampling[END_REF] . However, our AI-driven sensing pipeline is distinct from another class of intelligent meta-imagers which integrates the programmable meta-atoms as trainable physical weights directly into an end-to-end pipeline comprising both the physical and digital layers [START_REF] Del Hougne | Learned Integrated Sensing Pipeline: Reconfigurable Metasurface Transceivers as Trainable Physical Layer in an Artificial Neural Network[END_REF][START_REF] Li | Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing[END_REF] (see Sec. III.B-C in Ref. [START_REF] Saigre-Tardif | Intelligent metaimagers: From compressed to learned sensing[END_REF] ). The latter could be a future extension for the first step of our scheme.

The pivotal hardware ingredient of our microwave speech recognizer is an inexpensive one-bit reprogrammable coding metasurface composed of 32 × 32 electronically controllable meta-atoms. A photographic image of a 16 × 16 panel of meta-atoms is shown in Fig. 1b. Each meta-atom has dimensions of 16 mm × 16 mm × 1.88 mm and consists of a five-layer structure. A PIN diode (MADP-000907-14020x) is embedded on the top layer. By controlling the bias voltage of the PIN diode, we can electronically switch between two distinct meta-atom reflection properties in the microwave domain: The 0 ophase (denoted as digit '0') and 180 o -phase (denoted as '1') states are achieved when the PIN diode is biased with an externally applied DC voltage of 5V (ON) or 0V (OFF), respectively. The frequency-dependent magnitude and phase responses of our designed meta-atom are measured experimentally and plotted in Fig. 1d. Our designed meta-atom can efficiently manipulate the reflected electromagnetic field within the frequency range from 7.49 GHz to 8.3 GHz: the reflection phase of the meta-atom shows a roughly 180 o phase difference when the embedded PIN diode is switched from the ON state to the OFF state, while the reflection magnitudes are almost the same and close to unity. Thus, the radiation pattern can be flexibly manipulated by suitably biasing the PIN diodes of the metasurface through a field-programmable gate array (FPGA). More details about the reprogrammable coding metasurface are provided in Methods and Supplementary Note 1.

To examine the crucial role of our large-aperture programmable metasurface in discriminating between reflections from the mouth and undesired clutter as well as to improve the signal-to-noise ratio (SNR), we have conducted a series of preliminary experiments which seek to focus the microwave field at one or two desired location(s).

The spatial distributions of the microwave field are obtained through near-field scans. Two representative results are shown in Fig. 1e and Fig. 1f for focusing on a single point A (0.12m, 0.14m, 1m) or simultaneously on two points B (-0.05m, 0.14m, 0.65m) and C (0.1m, 0.14m, 0.65m), respectively. The corresponding metasurface configurations are identified via a modified Gerchberg-Saxton algorithm and also displayed in Fig. 1e and Fig. 1f. These results demonstrate the ability of our system to reallocate the microwave energy to one or multiple desired spot(s) in a dynamic manner using a suitable configuration of our programmable metasurface. The signal level at the desired spots is enhanced by a factor of around 20 dB. This focusing on the ROI is crucial in order to drastically increase the weight of reflections from the ROI and to suppress the influence of undesired clutter, noise and multipath reflections.

Encoding of speech information in microwave biosignals. Having described our system configuration, we now wish to investigate the mechanisms through which speech information is encoded in the microwave biosignals that our system acquires. The acoustic sounds that constitute a voice originate from air being exhaled by the lungs and causing vocal cord vibrations. The sound is additionally shaped through a series of further articulatory entities such as tongue position and mouth opening. Clearly, the more the acquired signals of a remote sensing technique interact with all of the involved articulatory entities, the more speech information is expected to be encoded in the biosignals.

Contactless visual speech recognition techniques solely rely on mouth motion because optical frequencies cannot penetrate through the skin and teeth. Similarly, Ref. [START_REF] Wang | Can Hear You with Wi-Fi![END_REF] based on a remote contactless microwave approach at 2.4 GHz suggests it relies solely on mouth motion. In contrast, Ref. [START_REF] Birkholz | Non-Invasive Silent Phoneme Recognition Using Microwave Signals[END_REF] reported that interior articulatory entities like the tongue decisively impact their contact-based measurements in the microwave regime. In fact, Ref. [START_REF] Birkholz | Non-Invasive Silent Phoneme Recognition Using Microwave Signals[END_REF] observed less microwave interaction with interior articulatory entities when the speaker had metallic tooth fillings, a clear indicator that the microwaves penetrated through the skin. These findings from Ref. [START_REF] Birkholz | Non-Invasive Silent Phoneme Recognition Using Microwave Signals[END_REF] are in line with our observations. As displayed in Fig. 2a, for our working frequency around 8 GHz, we find in full-wave simulations (CST Microwave Studio) that a significant portion of the microwave signal penetrates through the skin and teeth and interacts notably with the subject's articulatory entities such as the tongue. In addition, the spatial resolution is decent because the wavelength at 8 GHz is 3.75 cm in free space and even less inside the biological tissue. These results suggest that our technique is remarkably different from existing non-microwave-based strategies in that it can efficiently encode speech information not only from the mouth but also from interior articulatory entities such as the tongue.

To test this hypothesis experimentally, we next conduct a series of speech experiments with our microwave speech recognizer system. Therein, the subject pronounces alternately two syllables, an alveolar |s| and a cacuminal |∫|, and repeats them with a period of two seconds for five minutes (see Fig. 2b). The participant is asked to keep the mouth as close as possible during the whole pronunciation process so that only tongue motion inside the mouth is involved. Thereby, we minimize the encoding of speech information in the microwave biosignals through the mouth. If our system is nonetheless capable of extracting speech information, this proves that it efficiently probes articulatory entities other than the mouth, too. The amplitudes of the acquired microwave biosignals as a function of the Doppler frequency are plotted in Fig. 2c; each curve is averaged over 100 repeated acquisitions, and the experiment was performed for five different distances between the speaker and the metasurface (𝑑 = 0.9 m, 1.2 m, 1.5 m, 1.8 m, and 2 m). The corresponding sizes of the focal spot on the mouth are estimated to be on the order of

𝑂( 𝜆𝑑 𝐷

), where 𝐷 = 51.2 cm is the metasurface aperture, yielding values between 6.6 cm and 14.6 cm. As expected, the amplitude responses show the peaks at the Doppler frequency of around 0.5 Hz in all five cases. It is evident that the microwave signals can capture the hidden vocal vibrations and motions even though the motion of the subject's skin (lip and face) cannot be visually perceived. These microwave signals could be further processed to infer the speech content.

As a last set of preliminary experiments, the speaker is asked to pronounce a larger number of syllables and repeat them for five minutes with a frequency of 0.5 Hz per syllable. The corresponding microwave responses are collected by the developed microwave speech recognizer. Microwave speech recognition in line-of-sight setting. We begin to examine the performance of the developed microwave speech recognizer in a line-of-sight scenario, where the subject sits in front of the metasurface, as seen in Fig. 3a. To train our metasurface speech recognizer, we have recruited 22 participants (7 graduate students and 15 undergraduate students; 5 females and 17 males) and aim at recognizing 100 daily used English words. In the experiments, all participants were asked to read out loud the designated material five times at a normal speed (half a word per second on average) in a quiet environment. As mentioned above, the built-in microphone in our host computer is utilized for collecting the labeled voice data for supervised learning. In this way, we have acquired a total of 11000 pairs of labeled microwave-voice samples per location, in which 70% of the samples are randomly selected for training the microwave-voice transformer, and the rest are used for testing. Selected samples are provided in Supplementary Note

2.

First, we consider the simple case in which a single subject with or without wearing a face mask sits at P (0.04 m, 0 m, 1.15 m) in front of the reprogrammable metasurface in a static and quiet environment, as shown in Fig. 3a. Figure 3b demonstrates the learning and test performances of the developed microwave speech recognizer in terms of how the loss function evolves over the course of the training epochs. In addition, the dependence of the speech recognition accuracy over the training samples as the epoch index increases is plotted in Fig. 3b. We mix cases in which the subject wears a face mask or not because the results are almost identical in both cases. Figure 3b shows that the developed speech recognizer can be effectively trained to achieve near-perfect speech recognition, and that the trained system works very well on the 'unseen' test samples, even when the speaker wears a mask. In other words, these results indicate that the proposed microwave speech recognizer can 'hear' what people say without audio and visual clues and 'see' the speaker's mouth in a remote sensing manner.

Next, we evaluate the robustness of our speech recognition procedure in a dynamically changing environment. Body motion or a changing environment can lead to parasitic variations of the acquired microwave signals that deteriorate the speech recognition performance. We conducted a set of experiments in which a second person, referred to as the perturbation person, acts freely within the indicated region in Fig. 3c while the subject reads out loud the designated material. To interpret the microwave signals we use the ANN previously trained in a quiet environment without any disturbances. The results in this dynamically changing environment are shown in Fig. 3d and demonstrate the robustness of our approach. This robustness can be attributed to the efficiency with which our programmable metasurface focuses the microwave beam on the subject's mouth such that reflections off the perturbation person are very weak and hence their disturbing impact is efficiently suppressed.

We also examined the robustness with respect to body motion of the speaking subject.

Here, we consider a realistic scenario in which the subject walks around freely while speaking. This implies that the distance and orientation of the subject's mouth relative to the programmable metasurface are varying. Specifically, the subject spoke at the five different locations around the point P, indicated in Fig. 3e while performing three different kinds of body motion (making a phone call, typing, rhythmical leg movements). The achieved recognition accuracies, again based on the microwave-voice transformer network trained in the quiet environment, are reported in Fig. 3f. These results indicate that the speech recognition performance is almost unchanged when the subject makes phone calls or types while reading the designated text. Again, this robustness can be attributed to our efficient programmable-metasurface system that tracks the subject's mouth and ensures that the microwave focal spot follows the subject's motion. These characteristics are very encouraging with regard to real-life convenient and robust speech sensing, irrespective of the speaker's distance and orientation.

Microwave speech recognition in through-a-wall setting. Furthermore, we tackle microwave multi-speaker speech recognition in a more challenging through-a-wall scenario where two subjects sit behind a 5 cm-thick wooden wall and talk with each other.

The wall has a higher dielectric constant than air and possibly an additional microstructure, which previously motivated the conception of special wall-compensation algorithms to mitigate artefacts due to reflections and wavefront distortions in through-a-wall computational meta-imaging [START_REF] Sleasman | Computational through-wall imaging using a dynamic metasurface antenna[END_REF] . The multi-speaker problem now requires that the metasurface is configured such that it simultaneously focuses microwave energy on both speakers' mouthslike the example from Fig. 1f. To train the microwave metasurface speech recognizer, 22 participants read the assigned English reading material five times behind the 5 cm-thick wooden wall. As before, 70% of the samples are randomly selected for training the microwave-voice transformer, and the rest are used for testing. The corresponding experimental results are reported in Supplementary Note 3. The recognition accuracies of above 80% from Supplementary Fig. 3 confirm that our system performs well also for the very challenging multi-speaker through-a-wall speech recognition task, even if a third person acts freely in the room while the two subjects talk to each other. Voice-commanded human-machine interaction. Besides recognizing the content of voice commands which is the problem we have studied so far, ideally, a voicecommanded human-machine interface should additionally be able to recognize the speaker's identity. Voice is well-established as acoustic "fingerprint" of a user's identity because unique vocal features are encoded by unique properties of the individual's lung, vocal cords, vocal tract, and other articulatory entities. Therefore, even though multiple people pronounce the same words, the uttered sounds include distinctive features for each person. We now explore whether user identification is also possible with our acquired microwave biosignals. Interestingly, this variation across different users can be interpreted as a form of noise with respect to speech recognition whereas it is the salient feature for user identification. This double-sided-sword nature of signal variations as being either noise or the crucial feature is reminiscent of microwave-based complex localization problems [START_REF] Del Hougne | Robust position sensing with wave fingerprints in dynamic complex propagation environments[END_REF] .

We consider the through-a-wall microwave speech data of seven subjects, labelled with classes from 1 to 7 according to which subject they correspond to. The data processing problem is now a multivariate classification problem for which we have developed a deep convolutional neural network (see Supplementary Note 4 for more details). We explore two factors that we expect to have a major influence on the identification performance: the speech sample length and the number of subjects to be distinguished. Indeed, the results plotted in Fig. 4c-g demonstrate that as more subjects must be distinguished, acceptable classification accuracy can be achieved by analyzing longer speech samples. Using only six-second-long speech samples, we can distinguish between all seven individuals based on the microwave biosignals, without any vocal and visual clues.

Finally, we now discuss our demonstration of a voice-commanded human-machine interface in which a mechanical hand is controlled based on through-a-wall vocal speech that is recognized by our microwave speech recognizer. The corresponding experimental setup is depicted in Fig. 4a. The system now recognizes the speech content and subsequently sends out the recognized speech commands to a mechanical hand in order to control the motion of the latter in real time. The mechanical hand is integrated into a mobile vehicle and each finger is fitted with an anti-blocking joint servo (LFD-01) for the finger retraction control. An on-board Wi-Fi module (nRF24L01) is used to wirelessly receive control commands from the host computer based on the recognized vocal commands. The vehicle is equipped with an STM32 controller to process the received control commands into the control quantities for the corresponding finger servos (see Supplementary Note 5 for details). Five different speech commands are involved in this experiment: 'one', 'two', 'three', 'four', and 'five'. We have evenly collected 1000 pairs of microwave and acoustic samples for the five commands, and utilized 70% and 30% of samples to train and test the microwave speech recognizer, respectively. We report the classification confusion matrix in Fig. 4b, showing that near-perfect speech recognition of the five commands can be achieved by using our microwave speech recognizer. More details have been recorded in Supplementary Video 1. These primary experimental results demonstrate the important potential of our microwave speech recognizer for microwave-based contactless voice-commanded human-machine interfaces.

Conclusions

To summarize, we have proposed and experimentally prototyped the concept of a microwave speech recognizer empowered by a programmable metasurface, including a demonstration of a voice-commanded human-machine interface. Our system answers two fundamental questions in speech recognition -"what is being said?" and "who is speaking?"based on voice-modulated microwave biosignals. The unique advantages of a large-aperture programmable metasurface enable us to implement microwave-based speech recognition with unprecedented accuracy because we can dynamically track the speaker's mouth and focus microwaves on it with high efficiency. Our work is particularly timely in the current pandemic context: people always wear masks in public places such that their lip movements cannot be seen, and neither can their voice be heard given loud ambient noise sources. The demonstrated ability to implement contactless voicecommanded human-machine interfaces without reliance on optical or acoustic cues will enable numerous important but to-date inaccessible applications of human-machine interfaces such as in smart health care or industrial settings, as well as intelligent surveillance and security.

Figure 1 .

 1 Figure 1. System design of the metasurface-empowered microwave speech recognizer. a) Conceptual illustration of the proposed microwave speech recognizer in a challenging indoor scenario: an elderly person in the sleeping room voicecommands an appliance (e.g., lights) through a wall and despite loud music and motion in the neighboring guest room. b)

  Figure 2d shows amplitude and phase of the microwave responses corresponding to five different syllables. The corresponding sound signals obtained from the built-in microphone of the host computer are also plotted for comparison. It can be seen from Fig. 2d that the microwave responses are correlated with the corresponding sound signals in the time domain. The frequency spectrum of the microwave responses has distinct properties for each syllable.

Figure 2 .

 2 Figure 2. Physical mechanisms of speech encoding in the proposed microwave speech recognizer. a) Full-wave simulation of the interaction of the chosen microwave signal with the vocal organs, which is achieved by using a full-wave simulator, CST Microwave Studio 2012. In our simulation, the subject (Laura, a model of a 43-year-old female from Ref. 28 ) is illuminated with a linearly polarized plane wave. b) The experimental scheme according to which the subject is asked to alternately pronounce an alveolar |s| and a cacuminal |∫|, with a period of two second for a duration of five minutes. c) The experimental results corresponding to the scheme presented in b: the amplitudes of the microwave biosignals are plotted as a function of the Doppler frequency. The results are shown for five separations between the speaker and theprogrammable metasurface (0.9 m, 1.2 m, 1.5 m, 1.8 m, and 2 m). d) Selected microwave responses when the subject is asked to pronounce a short sentence, i.e., 'I am a student', three times. In addition, the corresponding sound signal acquired by the in-built microphone of our host computer is plotted for comparison.

Figure 3 .

 3 Figure 3. Experimental results of the microwave speech recognition in a line-of-sight setting. a) Experimental setting, where the subject is wearing or not wearing a face mask and sitting in front of the reprogrammable metasurface. b) The training and test behaviors of the microwave speech recognizer as a function of the training epoch. The logarithm of the loss function is plotted as blue solid line (left axis), and the test behavior is examined in terms of the recognition accuracy (right axis). Here, we consider four test cases with the microwave speech recognizer trained in the quiet environment: the simple test with the off-line collected test samples (called off-line test), the in-situ test with a subject in the same quiet environment as that for the training (called in-situ test in quiet environment), the in-situ test with a subject disturbed by an additional person freely acting in room (called in-situ test with perturbation), the in-situ test with a subject with different body motions (called in-situ test with body motion). c) Experimental setting for the investigation of the microwave metasurface speech recognizer's robustness to the disturbances of the ambient environment: an additional person acts freely within the region marked by the blue box while subject reads out loud the assigned material. d) Experimental speech recognition results for the setting from c. e) Experimental setting for examining the robustness when the speaking subject makes three

Figure 4 .

 4 Figure 4. Voice-commanded through-a-wall human-machine interaction based on our metasurface-empowered microwave speech recognizer. a) Experimental setting. Details about the mechanical hand are provided in the inserted figure. Further details can be found in Supplementary Video 1. b) Classification confusion matrix for the five different speech commands: 'one', 'two', 'three', 'four', and 'five'. c) Results of the dependence of the recognition accuracy on the speech length for the different number of speakers. d) Classification confusion matrices corresponding to the four red points marked in c.

Acknowledgments

This work was supported in part through the National Key Research and Development Program of China (2017YFA0700201, 2017YFA0700202, and 2017YFA0700203), and in part through the National Natural Science Foundation of China (61471006, 61631007 and 61571117).

Methods

Design of the one-bit reprogrammable coding metasurface. The reprogrammable coding metasurface is an ultrathin planar array of meta-atoms that are individually reconfigurable via electronic commands [START_REF] Cui | Coding metamaterials, digital metamaterials and programmable metamaterials[END_REF][START_REF] Yang | A programmable metasurface with dynamic polarization, scattering and focusing control[END_REF][START_REF] Huang | Dynamical beam manipulation based on 2-bit digitally-controlled coding metasurface[END_REF] . Thanks to its unique capabilities to manipulate electromagnetic wavefields in a reprogrammable manner, it has elicited many exciting physical phenomena (e.g., nonreciprocal reflection effects [START_REF] Zhang | Breaking Reciprocity with Space-Time-Coding Digital Metasurfaces[END_REF] ) and versatile functional devices, including computational imagers [START_REF] Saigre-Tardif | Intelligent metaimagers: From compressed to learned sensing[END_REF][START_REF] Sleasman | Dynamic metamaterial aperture for microwave imaging[END_REF][START_REF] Del Hougne | Learned Integrated Sensing Pipeline: Reconfigurable Metasurface Transceivers as Trainable Physical Layer in an Artificial Neural Network[END_REF][START_REF] Li | Intelligent metasurface imager and recognizer[END_REF][START_REF] Li | Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing[END_REF][START_REF] Li | Transmission-Type 2-Bit Programmable Metasurface for Single-Sensor and Single-Frequency Microwave Imaging[END_REF][START_REF] Sleasman | Microwave Imaging Using a Disordered Cavity with a Dynamically Tunable Impedance Surface[END_REF][START_REF] Sleasman | Single-frequency microwave imaging with dynamic metasurface apertures[END_REF][START_REF] Boyarsky | Single-frequency 3D synthetic aperture imaging with dynamic metasurface antennas[END_REF][START_REF] Li | Machine-learning reprogrammable metasurface imager[END_REF][START_REF] Ruan | Imaging Resolution Analysis of Single-Frequency and Single-Sensor Programmable Microwave Imager[END_REF][START_REF] Ruan | Programmable Metasurface Based Microwave Gesture Detection and Recognition Using Deep Learning[END_REF] , dynamic holography [START_REF] Li | Electromagnetic reprogrammable coding-metasurface holograms[END_REF] , wireless communications [START_REF] Yoo | Enhancing Capacity of Spatial Multiplexing Systems Using Reconfigurable Cavity-Backed Metasurface Antennas in Clustered MIMO Channels[END_REF][START_REF] Del Hougne | Optimally diverse communication channels in disordered environments with tuned randomness[END_REF][START_REF] Renzo | Smart radio environments empowered by reconfigurable AI meta-surfaces: an idea whose time has come[END_REF][START_REF] Dai | Reconfigurable Intelligent Surface-Based Wireless Communications: Antenna Design, Prototyping, and Experimental Results[END_REF][START_REF] Zhao | Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals[END_REF][START_REF] Shlezinger | Dynamic Metasurface Antennas for 6G Extreme Massive MIMO Communications[END_REF][START_REF] Zhang | A wireless communication scheme based on space-and frequency-division multiplexing using digital metasurfaces[END_REF][START_REF] Imani | Metasurface-Programmable Wireless Network-on-Chip[END_REF] , analog wave-based computing [START_REF] Del Hougne | Leveraging Chaos for Wave-Based Analog Computation: Demonstration with Indoor Wireless Communication Signals[END_REF][START_REF] Sol | Meta-programmable analog differentiator[END_REF][START_REF] Liu | A programmable diffractive deep neural network based on a digitalcoding metasurface array[END_REF] , and dynamic cloaks [START_REF] Qian | Deep-learning-enabled self-adaptive microwave cloak without human intervention[END_REF] .

We have designed a one-bit reprogrammable metasurface that is composed of 32×32 meta-atoms. The meta-atom is a five-layer structure as shown in Fig. 1b and Supplementary Note 1. The top layer is a square copper patch with dimensions of 11 mm × 11 mm, which contains a PIN diode to control the reflection phase of the metaatom. The second layer has a thickness of 1.58 mm and is made of Taconic TLX-8 which has a relative permittivity of 2.55. The fourth layer has a thickness of 0.3 mm and is made of FR-4 with a dielectric constant of 4.3. The third and fifth layers are ground planes made of copper, and a via hole is introduced on the third layer to isolate the bias voltage coming from the fifth layer. For the sake of easy fabrication, the entire reprogrammable coding metasurface is designed to be composed of 2×2 metasurface panels, and each panel consists of 16×16 electronically controllable digital meta-atoms. One such panel is depicted in Fig. 1b. Each metasurface panel is equipped with eight 8-bit shift registers (SN74LV595APW), and eight PIN diodes are sequentially controlled. The adopted clock rate is 50 MHz, and the ideal switching time of the PIN diodes is 10 μs.

Algorithmic overview. The first step of our microwave speech recognizer is to localize the ROI in the scene, i.e., the speaker's mouth. To this end, the scene is illuminated with 18 random patterns generated by a known series of random metasurface configurations.

The acquired data is directly mapped to a 3D skeleton of the speaker using a deep ANN (see details in Supplementary Note 4). [START_REF] Wang | High Resolution 3D Microwave Real Time Imaging System Based on Intelligent Metasurface[END_REF] Based on the 3D skeleton, it is then straightforward to localize the speaker's mouth. These ROI coordinates are needed to identify a metasurface configuration that efficiently focuses microwaves on the speaker's mouth. A suitable metasurface configuration for this focusing task is identified with a modified Gerchberg-Saxton algorithm based on the ROI coordinates. This is the basis for capturing clutter-resiliant microwave biosignals from which speech information can be extracted. The autonomous ROI identification here differs from that in Ref. [START_REF] Li | Intelligent metasurface imager and recognizer[END_REF] in that the acquired data from the first step is mapped to a 3D skeleton as opposed to a full image.

Microwave-speech transformer. The microwave-speech transformer is a deep artificial neural network which directly converts the sequence of microwave signals to the sequence of recognized speech information in an end-to-end fashion. The architecture is inspired by Ref. [START_REF] Vaswani | Attention is All you Need[END_REF] and detailed in Supplementary Note 4. The network adopts the typical Transformer structure and uses an encoder-decoder module structure, which is mainly composed of multi-head attention layer, feed-forward layer, residual connection and layer normalization. The network training is performed using the Adam optimization method [START_REF] Kingma | A method for stochastic optimization[END_REF] with a mini-batch size of 64, an epoch setting of 50, and a learning rate of 3×10 Configuration of proof-of-concept system. The experimental setup, as shown in Fig. 1b, consists of a transmitting (TX) horn antenna, a receiving (RX) horn antenna, a largeaperture reprogrammable metasurface, and a vector network analyzer. The two horn antennas are connected to two ports of the VNA via two 4m-long 50-Ω coaxial cables, and the VNA is used to acquire the response data by measuring transmission coefficients (S21).

In addition, an in-build sound microphone in the host computer has been integrated into our system for acquiring labeled training data. The computer controls the VNA and microphone to acquire the microwave data and voice signal, respectively, using the Python 3.1 software. These two procedures of data acquisition share the same starting time and ending time, but with different sampling intervals, 70 ms for the microwave data from the VNA and

s for the acoustic data from the microphone. Note that the sound signals are solely used to assist in labeling the microwave data with corresponding text to obtain labeled training data, since our primary goal is to infer the speech information from the microwave data. To that end, we cut the acoustic signal corresponding to a specific text by listening and writing down the start and end times. Since the microwave speech data and the acoustic data are acquired at the same start time, we can easily align each sampling interval. Finally, we can label the microwave speech data with the corresponding text. We input these microwave biosignals into the neural network and train the latter so that it outputs the corresponding speech.

Author Contributions

L.L. conceived the idea and conducted the theoretical analysis. P.

Additional Information

Supplementary Information accompanies this article.

Competing Interests:

The authors declare no competing interests.

Supplementary Files

This is a list of supplementary les associated with this preprint. Click to download.

SM.pdf

Video1.mp4