Claudio Battiloro

Mattia Merluzzi
email: mattia.merluzzi@cea.fr

Paolo Di Lorenzo

Sergio Barbarossa
email: sergio.barbarossa@uniroma1.it

DYNAMIC RESOURCE OPTIMIZATION FOR ADAPTIVE FEDERATED LEARNING EMPOWERED BY RECONFIGURABLE INTELLIGENT SURFACES

Keywords: Adaptive federated learning, Lyapunov optimization, resource allocation, Reconfigurable Intelligent Surfaces

The aim of this work is to propose a novel dynamic resource allocation strategy for adaptive Federated Learning (FL), in the context of beyond 5G networks endowed with Reconfigurable Intelligent Surfaces (RISs). Due to time-varying wireless channel conditions, communication resources (e.g., set of transmitting devices, transmit powers, bits), computation parameters (e.g., CPU cycles at devices and at server) and RISs reflectivity must be optimized in each communication round, in order to strike the best trade-off between power, latency, and performance of the FL task. Hinging on Lyapunov stochastic optimization, we devise an online strategy able to dynamically allocate these resources, while controlling learning performance in a fully data-driven fashion. Numerical simulations implement distributed training of deep convolutional neural networks, illustrating the effectiveness of the proposed FL strategy endowed with multiple reconfigurable intelligent surfaces.

INTRODUCTION

The future of wireless communication systems is to accomodate a plethora of new services for different verticals, on the same network infrastructure [START_REF] Strinati | 6G: The next frontier: From holographic messaging to artificial intelligence using subterahertz and visible light communication[END_REF]. Therefore, beyond 5G networks will have to be extremely flexible and dynamically reconfigurable from the application layer down to the wireless propagation environment. In particular, reconfigurability of the wireless environment is a completely new feature, foreseen to be enabled by the deployment of Reconfigurable Intelligent Surfaces (RIS) [START_REF] Strinati | Wireless environment as a service enabled by reconfigurable intelligent surfaces: The RISE-6G perspective[END_REF]. RISs are arrays of backscatters, where each element applies an individual phase-shift (and/or an amplitude and/or a polarization rotation) with which it backscatters an incident wave [START_REF] Renzo | Smart radio environments empowered by reconfigurable ai meta-surfaces: An idea whose time has come[END_REF][START_REF] Mursia | RISMA: Reconfigurable intelligent surfaces enabling beamforming for iot massive access[END_REF][START_REF] Zhang | Reconfigurable intelligent surfaces assisted communications with limited phase shifts: How many phase shifts are enough?[END_REF], with the aim of creating a controllable reflected beam. The deployment of such a flexible and complex architecture will enable Machine Learning (ML) and Artificial Intelligence (AI) at the edge, a paradigm known as edge machine learning. The challenge is that edge ML calls for jointly optimizing inference, training, communication, computation, and control under end-to-end latency, reliability, and learning performance requirements [START_REF] Park | Wireless network intelligence at the edge[END_REF][START_REF] Chen | Deep learning with edge computing: A review[END_REF][START_REF] Merluzzi | Wireless edge machine learning: Resource allocation and trade-offs[END_REF][START_REF] Battiloro | Lyapunovbased optimization of edge resources for energy-efficient adaptive federated learning[END_REF]. In this context, the exploitation of RISs in this joint design plays a key role, since it can strongly mitigate the presence of poor wireless channel conditions due to mobility, time-varying environment, and blocking events, especially using millimeter wave communications [START_REF] Strinati | Wireless environment as a service enabled by reconfigurable intelligent surfaces: The RISE-6G perspective[END_REF][START_REF] Renzo | Smart radio environments empowered by reconfigurable intelligent surfaces: How it works, state of research, and the road ahead[END_REF].

Related works. Training ML models at the edge mainly relies on Federated Learning (FL) [START_REF] Battiloro | Lyapunovbased optimization of edge resources for energy-efficient adaptive federated learning[END_REF][START_REF] Li | Federated learning: Challenges, methods, and future directions[END_REF][START_REF] Konečnỳ | Federated optimization: Distributed machine learning for on-device intelligence[END_REF][START_REF] Li | Federated optimization in heterogeneous networks[END_REF][START_REF] Konečnỳ | Federated optimization: Distributed optimization beyond the datacenter[END_REF][START_REF] Samarakoon | Distributed federated learning for ultra-reliable low-latency vehicular communications[END_REF]. In this approach, learning

The work of Di Lorenzo and Barbarossa was funded by the H2020 project RISE-6G no. 101017011. architectures perform (variants of) parallel stochastic gradient descent (SGD) across multiple edge devices, whose intermediate results are aggregated by an Edge Server (ES). FL has several benefits of data privacy, and is empowered by a large amount of device participants with modern powerful processors and low-delay mobileedge networks. Several works on FL explicitly focus on the optimization of radio resource allocation [START_REF] Chen | Learn-and-adapt network resource allocation[END_REF][START_REF] Bedi | Asynchronous incremental stochastic dual descent algorithm for network resource allocation[END_REF][START_REF] Wang | Adaptive federated learning in resource constrained edge computing systems[END_REF][START_REF] Chen | A joint learning and communications framework for federated learning over wireless networks[END_REF][START_REF] Tran | Federated learning over wireless networks: Optimization model design and analysis[END_REF][START_REF] Ma | Adaptive batch size for federated learning in resource-constrained edge computing[END_REF][START_REF] Khan | Federated learning for edge networks: Resource optimization and incentive mechanism[END_REF][START_REF] Yang | Energy efficient federated learning over wireless communication networks[END_REF][START_REF] Mills | Communication-efficient federated learning for wireless edge intelligence in IoT[END_REF][START_REF] Shi | Joint device scheduling and resource allocation for latency constrained wireless federated learning[END_REF][START_REF] Chen | Convergence time optimization for federated learning over wireless networks[END_REF]. The authors in [START_REF] Samarakoon | Distributed federated learning for ultra-reliable low-latency vehicular communications[END_REF] study FL and the problem of joint power and resource allocation for ultra-reliable low latency communication in vehicular networks. In [START_REF] Mills | Communication-efficient federated learning for wireless edge intelligence in IoT[END_REF], the authors propose adapting federated averaging to use a distributed form of Adam optimization along with a compression technique. The work in [START_REF] Jiang | Model pruning enables efficient federated learning on edge devices[END_REF] proposes a FL approach with adaptive and distributed parameter pruning. In [START_REF] Shi | Joint device scheduling and resource allocation for latency constrained wireless federated learning[END_REF], the authors propose a joint device scheduling and resource allocation policy to maximize the model accuracy within a given total training time budget. Reference [START_REF] Chen | Convergence time optimization for federated learning over wireless networks[END_REF] proposes a dynamic user selection scheme to minimize the FL convergence time. Very recently, some works have proposed to enhance the performance of FL tasks by exploiting RISs [START_REF] Ni | Federated learning in multi-ris aided systems[END_REF][START_REF] Liu | Joint communication-learning design for ris-assisted federated learning[END_REF]. In [START_REF] Ni | Federated learning in multi-ris aided systems[END_REF], the authors investigate the problem of model aggregation in FL systems aided by multiple RISs, designing the transmit power, selecting the participants in the model uploading process, and tuning the phase shift of the available RISs. Similarly, reference [START_REF] Liu | Joint communication-learning design for ris-assisted federated learning[END_REF] formulates a communication-learning design problem to jointly optimize device selection, over-the-air transceivers, and RIS phase shifts.

Contributions. In this paper, we introduce a novel dynamic optimization framework for adaptive federated learning empowered by RISs, jointly encompassing communication, computation, and learning aspects of the problem. Differently from previous works that mainly focused on static learning (where FL is carried out up to convergence and then the learning process stops), we consider adaptive FL strategies, with the aim of endowing wireless networks with continuous learning and adaptation capabilities. Hinging on Lyapunov stochastic optimization [START_REF] Neely | Stochastic Network Optimization with Application to Communication and Queueing Systems[END_REF], we develop a dynamic resource allocation strategy working at the same time-scale of the FL algorithm, while optimizing on the fly radio parameters (e.g., set of transmitting devices, bits and rates), computation resources (e.g., CPU cycles at devices and at ES) and RISs reflectivity parameters, in order to strike the best trade-off between energy, latency, and performance of the FL task. The method works in a fully data-driven fashion, estimating online the performance (i.e., accuracy and convergence rate) from streaming data. Finally, the proposed strategy is customized to deep convolutional network training, showing the advantages obtained by endowing FL architectures with multiple RISs.

SYSTEM MODEL

Let us consider a scenario with N edge devices, K RISs and an AP equipped with an ES. The devices are cooperatively performing a training task aimed at learning a weight vector w ∈ R m . To this aim, at each time t ≥ 0, the devices collect batches Bt of labelled i.i.d. data (i.e., input/output pairs) (xi,t, yi,t) ∈ R d × R, for all i = 1, . . . , N . Then, assuming that device i has a local loss function Ji(w; xi, yi), whose structure depends on the specific learning task, the goal of FL can be mathematically cast as:

min w N i=1 E Ji(w; xi, yi) , (1)
where the expectation is carried out over the data and/or batch distribution. Now, at each time t, letting wt be the instantaneous guess for w, we proceed by optimizing problem (1) using an adaptive stochastic optimizer (e.g. ADAM, Adagrad, etc.). In particular, let us denote g i,t (wt) = (x i,t ,y i,t )∈B i,t ∇wJi(wt; xi,t, yi,t), for all i, t, to shorten the notation. Then, a rather general gradient-based optimizer recursion reads as:

wt+1 = wt -µ • f i∈S t g i,t (wt) , (2)
where t ≥ 0, µ > 0 is a step-size, and St is the set of nodes that participate in the optimization at time t; the function f (•) depends on the specific optimizer and applies to the sum of received gradients a time t. To implement (2), the devices belonging to St (to be determined for all t) compute, in parallel, the gradients of the local cost functions and upload them to the AP. Then, the edge server aggregates the local information to compute the new estimate wt+1, which is finally fed back to the devices. To explore the trade-off between power, latency, and performance of FL, following [START_REF] Merluzzi | Wireless edge machine learning: Resource allocation and trade-offs[END_REF], we act on the source encoder of each transmitting device. In particular, we assume the devices transmit dithered quantized versions of g i,t (wt) [START_REF] Lipshitz | Quantization and dither: A theoretical survey[END_REF], encoding local gradients into m • bi,t bits, where bi,t is chosen at each t from a discrete set Ci, i.e., bi,t ∈ Ci.

RIS-enhanced communications

We consider a system endowed with K passive RISs, each one composed of M reflecting elements. Each element is characterized by a complex reflection coefficient v k,l,t = θ k,l,t e jφ k,l,t , where θ k,l,t ∈ {0, 1} (i.e., the l-th reflective element of RIS k is active or not at time t), and φ k,l,t ∈ 2nπ 2 br 2 br -1 n=0 (i.e., the phase of each element is quantized using br bits) [START_REF] Mursia | RISMA: Reconfigurable intelligent surfaces enabling beamforming for iot massive access[END_REF]. Equivalently, we have

v k,l,t ∈ R = 0, e j 2nπ 2 br 2 br -1 n=0 , ∀k, l, t. (3)
Let us denote

v k,t = [v k,1,t , . . . , v k,M,t ] T and vt = {v k,t } K k=1 .
Then, assuming a Single Input Single Output (SISO) communication system, and denoting by pi,t the transmission power of user i, the RIS-aided uplink transmission rate between user i and the AP is

Ri,t = Bi log 2 1 + hi,t(vt)pi,t N0Bi , ( 4
)
where Bi is the bandwidth assigned to user i, N0 is the noise power spectral density, and hi,t(vt) is the RIS-dependent channel coefficient given by [START_REF] Basar | SimRIS channel simulator for reconfigurable intelligent surface-empowered communication systems[END_REF]:

hi,t(vt) = h a i,t + K k=1 h T i,k,t diag(v k,t ) z a i,k,t 2 , (5)
where h a i,t represents the direct channel coefficient between user i and the AP; whereas, h i,k,t ∈ C M ×1 and z a i,k,t ∈ C M ×1 are vectors containing all the channel coefficients between user i and RIS k elements, and between RIS k elements and the AP, respectively.

Latency and power consumption

In this paragraph, we evaluate the latency and the power consumption needed to implement one iteration of the algorithm in (2). In particular, there are four main components to be taken into account.

(i) Local processing: At the i-th device, the latency and the power necessary to compute the local gradient g i,t (wt) are given by L loc i,t = B t J i f i,t and p c i,t = γi(fi,t) 3 , respectively, where Ji denotes the number of CPU cycles needed to compute the local gradient from one data unit, Bt = |Bt|, fi, t is the CPU frequency of device i, and γi is the effective switched capacitance of the i-th processor [START_REF] Burd | Processor design for portable systems[END_REF].

(ii) Uplink communication: Denoting by Ri,t the uplink data rate, the latency needed to upload the local gradients to the ES reads as

L u i,t = m•b i,t
R i,t . Then, inverting (4), the transmit power consumption is given by: pi,t

= B i N 0 h i,t (v t ) exp R i,t ln 2 B i -1 .
(iii) Edge server processing: At the ES, the latency necessary to produce the global estimate in ( 2) is given by

L s t = C|S t | f s t
, where C is the number of CPU cycles necessary to perform the single step of the gradient-based algorithm for each device, and f s t is the CPU frequency of the server. This operation entails a power p s t = γs(f s t ) 3 , with γs denoting the effective switched capacitance of the ES.

(iv) Downlink communication: It is necessary to send the global estimate back to the agents. For simplicity, it is assumed to be given and not optimized in this work.

To have synchronous training updates, we need to consider the maximum among communication delays of all transmitting devices in the overall latency that, at a given time t, reads as:

Lt = max i∈S t L loc i,t + L u i,t + L r t . (6)
Similarly, the overall power consumption for the FL task at time t is:

p tot t = i∈S t pi,t + p c i,t + p s t . (7)

RIS-AIDED ADAPTIVE FEDERATED LEARNING

The aim of this paper is to jointly allocate radio (i.e., set of transmitting devices, powers, quantization bits and RISs reflectivity parameters) and computation (i.e., CPU cycles at devices and at server) resources to minimize the long-term average system power consumption in [START_REF] Chen | Deep learning with edge computing: A review[END_REF], with constraints on the average learning performance and the average latency in [START_REF] Park | Wireless network intelligence at the edge[END_REF]. Let Gt and αt be task-dependent learning performance and convergence rate metrics, respectively, which will be explained in the sequel (see, e.g., ( 9)). Then, the problem can be cast as:

min Ψ t lim t→∞ 1 t t-1 τ =0 E p tot τ subject to (a) lim t→∞ 1 t t-1 τ =0 E {Lτ } ≤ L; ( 8
) (b) lim t→∞ 1 t t-1 τ =0 E{Gτ } ≥ G; (c) lim t→∞ 1 t t-1 τ =0 E{ατ } = α, bi,t ∈ Ci, ∀i ∈ St, t; R min i ≤ Ri,t ≤ R max i,t , ∀i ∈ St, t; f min i ≤ fi,t ≤ f max i , ∀i ∈ St, t; v k,l,t ∈ R, ∀k, l, t; Bt ∈ B, ∀t; f s,min ≤ f s t ≤ f s,max , ∀t;       

Xt

where Ψt = [vt, {bi,t}i∈S t , {Ri,t}i∈S t , {fi,t}i∈S t , f s t , Bt], and the expectations are taken with respect to the random channel states, whose statistics are supposed to be unknown. The constraints of (8) have the following meaning: (a) the average latency of training iterations does not exceed a predefined value L; (b) the average performance metric Gt has to be greater or equal to predefined value G; (c) the average convergence rate is constrained to be equal to α. Also, some constraints in Xt impose that the batch size Bt can take values only from the discrete set B, the number of quantization bits {bi,t}i∈S t can take values only from a finite set Ci, and the phase shifts v k,l,t of the RISs obey to (3). Finally, the last constraints in Xt impose instantaneous bounds on the resource variables {Ri,t}i∈S t , {fi,t}i∈S t , f s t . In several practical cases (e.g., neural network training), we do not have a closed-form expression for Gt and αt. Therefore, we propose to exploit an online mechanism that estimates the learning performance and the convergence rate in a totally data-driven fashion. We assume that either the ES is provided with a validation set T or, in the absence of a validation set, the agents can sense an additional batch T of data at each time-slot, compute their local learning perfomance and send it (one scalar) to the server for the computation of the overall learning perfomance. Then, two task-dependent functions Gt and αt are introduced to measure online the unknown Gt and αt, respectively. As an example, if we consider a classification task, the validation (or batch) accuracy and its moving average with length 2κ can be used to estimate learning performance and convergence rate as:

Gt = 1 |T | y∈T I( yt = y), αt = 1 κ κ-1 τ =t-κ ( Gτ -Gτ-1) (9)
where yt is the prediction for data unit y at time-slot t.

Algorithmic design via stochastic optimization

We now introduce a method to transform Problem (8) into a stability problem, building on the tools of stochastic Lyapunov optimization [START_REF] Neely | Stochastic Network Optimization with Application to Communication and Queueing Systems[END_REF].In particular, to deal with the long-term constraints (a)-(c), we introduce three virtual queues, respectively associated with them:

Zt+1 = max 0, Zt + z Lt -L , (10)
Qt+1 = max 0, Qt + q G -Gt (11)
Yt+1 = [Yt + y,t ( αt -α)] • I Gt ≥ G , (12)
where z > 0, q > 0 and y,t > 0 are step-sizes used to speedup the convergence of the algorithm. The queue evolution defined in [START_REF] Konečnỳ | Federated optimization: Distributed machine learning for on-device intelligence[END_REF], including the adaptive step-size y,t, is motivated by the fact that, if the distribution of the data is stationary, there is no need to overshoot the convergence rate after the target level of learning performance is reached. At the same time, non-stationary behaviors can be detected observing sharp changes in Gt, which reactivate the virtual queue Yt. Interestingly, ensuring the mean-rate stability of the virtual queues in ( 10)-( 12) is equivalent to satisfy the three corresponding constraints [START_REF] Neely | Stochastic Network Optimization with Application to Communication and Queueing Systems[END_REF]. To this aim, letting Ut =

1 2 (Z 2 t + Q 2 t + Y 2 t )
, we introduce the following drif-plus-penalty function [START_REF] Neely | Stochastic Network Optimization with Application to Communication and Queueing Systems[END_REF], which reads as:

∆ p t = E{Ut+1 -Ut + V • p tot t Φt}, (13)
which also incorporates a penalty factor that weights the objective function of ( 8), with a weighting parameter V . Now, it is possible to prove that minimizing ∆ p t , if [START_REF] Li | Federated optimization in heterogeneous networks[END_REF] is lower than a finite constant for all t, the virtual queues are mean rate stable, and the optimal solution of ( 8) is asymptotically reached as V increases [START_REF] Neely | Stochastic Network Optimization with Application to Communication and Queueing Systems[END_REF]Th. 4.8].

Using stochastic optimization arguments [START_REF] Neely | Stochastic Network Optimization with Application to Communication and Queueing Systems[END_REF], we proceed by optimizing a suitable upper-bound of (13), while removing the expectation per each time-slot t. However, the estimates Gt and αt appearing in ( 10)-( 12) (and, thus, in [START_REF] Li | Federated optimization in heterogeneous networks[END_REF]) are not explicitly related to the number of quantization bits {bi,t}i∈S t and to the batch size Bt, which must be optimized and adapted to drive the learning performance and the convergence rate. Then, we propose to exploit two surrogate functions, say Gt({bi,t}i∈S t ) and αt({bi,t}i∈S t , Bt). The rationale for the selection of the surrogates comes from the assumption that the true performance metrics Gt and αt typically show a non-increasing behavior with respect to the quantization bits {bi,t}i∈S t and the batch size Bt (examples will be given in the sequel). In other words, a finer representation of the data typically leads to better learning performance [START_REF] Merluzzi | Wireless edge machine learning: Resource allocation and trade-offs[END_REF][START_REF] Battiloro | Lyapunovbased optimization of edge resources for energy-efficient adaptive federated learning[END_REF].Thus, Gt and αt have only to be non-increasing functions of the quantization bits and the batch size. After some algebra manipulations (omitted due to the lack of space), the method requires to solve the following deterministic problem at each time-slot t:

min Ψ t ∈X t Zt Lt -Qt Gt -Yt αt + V • p tot t ( 14
)
where Ψt is the dynamic set of variables in [START_REF] Merluzzi | Wireless edge machine learning: Resource allocation and trade-offs[END_REF], and Lt is an upperbound of ( 6) where the maxi∈S t operator is substituted with the summation over all i ∈ St. Because of the structure of the variable set Ψt, ( 14) is a mixed-integer nonlinear optimization problem. However, for any given {bi,t} N i=1 , Bt, and vt at time t, it is easy to see that ( 14) is separable into three sub-problems that admit closed form solution for the optimal uplink data rates, the optimal CPU clock frequency of devices, and the optimal CPU clock frequency of the edge server. The expressions, for the optimal parameters are given by (derivations are omitted due to the lack of space):

Ri,t = 2Bi ln(2) W ln(2) Bi Zt m • bi,t hi,t(vt) 2V N0 R max i,t R min i (15) fi,t = ZtBtJi 3γiV 1 4 f max i f min i f r t = ZtC |St| 3γsV 1 4 f r,max f r,min (16)
for all i ∈ St, where W (•) is the principal branch of the Lambert function. In principle, to find the optimal solution of ( 14) at each time-slot t, one should compute the optimal allocation of edge resources for all possible combinations of vt, {bi,t} N i=1 and Bt, evaluate the corresponding objective function using ( 15) and ( 16) in ( 14), and then select the one that yields the lowest value. This approach has a complexity that grows exponentially with N , maxi |Ci|, |B|, K, M and |R|. Therefore, to find a manageable implementation, in the next paragraph we propose a two-stages greedy method that first optimizes the reflectivity parameters of the RISs {v k,t } K k=1 and then, given the RISs configuration, proceeds selecting the batch size Bt and the number of quantization bits {bi,t} N i=1 .

Two-stage greedy algorithm for resource optimization

In the first stage, the method greedily selects the RISs coefficients {v k,t } K k=1 in order to maximize the following objective: where

∆ R ({v k,t } K k=1 ) = N i=1 δi,t h a i,t + K k=1 h T i,k,t diag(v k,t ) z a i,k,t 0
δi,t = 1/|h a i,t | 2 N i=1 1/|h a i,t | 2
is a weighting factor aiming at assigning more importance to devices that, without the aid of the RISs, experience worse instantaneous channel conditions. The rationale of this lies on the fact that, having more devices with good channel conditions, increases the degree of freedom in selecting the set of transmitting devices, thus improving the overall performance of the FL task. This greedy procedure experiences a polynomial complexity with respect to K, M and |R|.

In the second stage, for each possible batch size Bt ∈ B (the number of selectable batch sizes is assumed to be small, e.g., 3 or 4), the method starts from the empty set of transmitting nodes and iteratively adds the most convenient devices, selecting jointly the best number of quantization bits {bi,t} N i=1 and the associated edge resources in [START_REF] Samarakoon | Distributed federated learning for ultra-reliable low-latency vehicular communications[END_REF] and [START_REF] Chen | Learn-and-adapt network resource allocation[END_REF]. The method keeps adding devices until the resulting value of the objective in [START_REF] Konečnỳ | Federated optimization: Distributed optimization beyond the datacenter[END_REF] decreases, and stops when there is no more incentive in letting other nodes to transmit any bit of information. Finally, the batch size and the associated transmitting set and edge resources that raise the lowest value of the objective function are chosen. Such greedy method hugely reduces the complexity, which becomes polynomial in N , max i {|Ci|}, |B|.

NUMERICAL RESULTS

In this section, we assess the performance of the proposed method, considering a federated learning task aiming at a training a deep convolutional neural network (CNN) classifier. We exploit a CNN architecture made of four convolutional layers with 32, 32, 10 and 10 filters, respectively, with final flatten and dense layers. The loss is the well-known cross-entropy, and the model is trained using a federated ADAM optimizer over the MNIST dataset. We consider a scenario with one AP equipped with an edge server, N = 9 devices, 6 of whom having their direct path to the AP attenuated by an obstacle (with an additive pathloss of 5 dB), and one RIS equipped with 1-bit discrete phase shifters. Wireless channels are generated with the tool presented in [START_REF] Basar | SimRIS channel simulator for reconfigurable intelligent surface-empowered communication systems[END_REF]. We set the radio and computation parameters as follows: N0 = -174 dBm/Hz, A = 25, t. Furthermore, the set of quantization bits is Ci = [START_REF] Strinati | Wireless environment as a service enabled by reconfigurable intelligent surfaces: The RISE-6G perspective[END_REF][START_REF] Mursia | RISMA: Reconfigurable intelligent surfaces enabling beamforming for iot massive access[END_REF][START_REF] Merluzzi | Wireless edge machine learning: Resource allocation and trade-offs[END_REF], for all i = 1, . . . , N . The ADAM step-size is set to 0.001, with forgetting factors β1 = 0.9, and β2 = 0.99. For this experiment, we use the performance estimate Gt and αt as in ( 9) with K = 10. Also, as a surrogate function for the accuracy metric, we exploit: Gt = i∈S t σ(bi,t -Median{Ci}), where σ(•) is the logistic sigmoid function, and Median{•} represents the median value. Regarding the convergence rate, we use instead the surrogate αt = Bt i∈S t bi,t. Finally, the set of batch sizes is B = [START_REF] Strinati | 6G: The next frontier: From holographic messaging to artificial intelligence using subterahertz and visible light communication[END_REF][START_REF] Renzo | Smart radio environments empowered by reconfigurable ai meta-surfaces: An idea whose time has come[END_REF][START_REF] Chen | Deep learning with edge computing: A review[END_REF]. As a first result, in Fig. 1 (a), we illustrate the temporal behavior of the estimated accuracy of the FL algorithm, obtained for different values of the learning rate α, fixing the accuracy to G = 0.8; also, at time slot 210, we change the accuracy requirement from G = 0.8 to G = 0.9 for the curve with α = 0.1, introducing a level of non-stationarity. Then, in Fig. 1 (b), we show the instantaneous latency required by the proposed FL strategy to perform one iteration, together with the latency constraint L = 50 ms. As we can notice from Fig. 1, the proposed method is able to obtain the desired learning performance, while controlling the convergence rate, and satisfying the required latency constraint. Furthermore, the method is able to react promptly to changes in the accuracy requirement, exhibiting powerful learning and adaptation capabilities in a fully data-driven fashion. Finally, in Fig. 2, we show the total average uplink transmission power expenditure versus the average latency L, for different values of average accuracy G, comparing the cases where RISs are exploited or not and a baseline given by an equal-rate (chosen to respect the latency and the accuracy constraints) policy with all the agents always transmitting. As expected, from Fig. 2, the trade-off gets worse imposing a stricter requirement on the accuracy, due to the larger power (and number of bits) necessary to obtain the target performance. Also, we can see the gain obtained thanks to the presence of the RIS in the FL task, and the superior performance of the proposed method w.r.t. the baseline.

f max s = 3.3 GHz, f max i = 2.5 GHz, R max i,t = Bi log 2 (1 + hi,tP max i / (

CONCLUSIONS

In this paper, we proposed an online strategy for adaptive federated learning empowered by reconfigurable intelligent surfaces. The method dynamically minimizes the power expenditure of the system, while guaranteeing target learning performance and latency constraints. The approach builds on stochastic Lyapunov optimization, dynamically and jointly optimizing radio and computation resources, learning parameters, and RISs phase shits. Numerical results illustrate the advantages obtained by the proposed strategy.

Fig. 1 :

 1 Fig. 1: (a) Accuracy vs t. (b) Latency vs t.

Fig. 2 :

 2 Fig. 2: Total avg. comm. power vs L vs for different G.


Ichraq Lemghari

Sylvie Le Hégarat-Mascle

Emanuel Aldea

Jennifer Vandoni

Handling noisy annotations in

Keywords: Supervised learning, classification, noisy labels, set-valued decisions, partial ignorance modelling

INTRODUCTION AND PROBLEM STATEMENT

Deep Learning (DL) relies heavily on large annotated training datasets. [START_REF] Daniely | Generalization bounds for neural networks via approximate description length[END_REF] However, accurate manual expertlabelling of each instance on a large scale is not feasible and often prohibitively expensive for some applications, 2 such as anomaly or defect detection. In these circumstances, three factors negatively impact the dataset creation and subsequently the learning process: the scarcity of some types of samples, the scarcity of qualified annotators, and the higher level of noise being present in the provided labels due to the difficulty of the task. Therefore, imperfect datasets presenting label noise are often employed, [START_REF] Xiao | Learning from massive noisy labeled data for image classification[END_REF] especially in the context of NDT. The impact of this label noise is very clear on the results that are strongly degraded, because deep neural networks are known for memorizing and eventually fitting label noise. [START_REF] Zhang | Understanding deep learning (still) requires rethinking generalization[END_REF] In this context, our main objective is to find a way to effectively train deep neural networks with modern architectures under label noise in a supervised learning classification setting.

In this work, we formalize the problem in the context of a c-class classification. Let D be the clean (i.e., not noisy) dataset, which consists of a d-dimensional feature space X ⊂ R d and a true label space Y = {0, 1} c , and let D be the noisy dataset, which consists of the same d-dimensional feature space X ⊂ R d and a noisy label space Ȳ = {0, 1} c . The goal of the learning step in the classification task is to learn the mapping function f (.; Θ) : X → [0, 1] c such that the parameter Θ minimizes the empirical risk R D (f ) = E D [l(f (x; Θ), y)] = 1 |D| Σ (x,y)∈D l(f (x; Θ), y) where l is the considered loss function. However, since data labels are usually noisy in real-world scenarios, this risk is calculated with noisy data, and the learned classifier is obtained as: f = arg min f ∈F E D [l(f (x; Θ), ȳ)]. Thus, the model can be biased by the corrupted labels, and in order to find a way to mitigate the impact of those noisy labels, one has to follow a noise-tolerant training process.

To do so, we will handle set-valued labels corresponding to imprecise or partially ignorant labels. More specifically, if Ω = {ω 1 , . . . , ω c } denotes the label set (associated to Y which is defined in the one-hot encoding space), we denote by 2 Ω the powerset of Ω, i.e. the set of the Ω subsets:

2 Ω = {∅, ω 1 , ω 2 , {ω 1 , ω 2 }, ω 3 , {ω 1 , ω 3 }, {ω 2 , ω 3 }, . . . , Ω}.
We consider the belief function theory 5 in order to model the belief on 2 Ω , and a mass function:

m : 2 Ω → [0, 1],
A⊆Ω m(A) = 1. Finally, let us call imprecise any decision function g with g : V → 2 Ω , where V is the score space which is usually equal to [0, 1] c (e.g., an output layer providing a normalized score vector of dimensionality equal to the number of classes) but can be equal to [0, 1] 2 c if an evidential layer is used and the output is a mass function m.

Further author information: Ichraq Lemghari. E-mail: ichraq.lemghari@safrangroup.com

RELATED WORK

Noisy labels

Learning from noisy labels is a high-stakes but challenging task in modern Deep Learning (DL) applications. Several DL approaches that have been studied in this context address the problem from different perspectives.

The first type of methods, based on robust architectures, focuses on the architectural modifications that can model the transition between true and noisy labels 3 . [START_REF] Sukhbaatar | Training convolutional networks with noisy labels[END_REF] It mainly includes two strategies, which work by either adding a noise adaptation layer at the top of the Deep Neural Network (DNN) model, or developing a dedicated architecture to handle the label noise.

A second category of methods, called robust regularization, consists of advanced regularization techniques that specifically enforce a DNN to overfit less the false-labeled samples. [START_REF] Tanno | Learning from noisy labels by regularized estimation of annotator confusion[END_REF] It can have an implicit form such as data augmentation or adversarial training [START_REF] Goodfellow | Explaining and harnessing adversarial examples[END_REF] , or an explicit form that modifies the expected training loss such as in Bilevel learning, [START_REF] Jenni | Deep bilevel learning[END_REF] which uses a clean validation dataset to regularize the overfitting of a model by introducing a bilevel optimization approach.

Sample selection is a third strategy whose objective is to separate the true-labeled samples from the noisy ones based on a specific metric. [START_REF] Lyu | Curriculum loss: Robust learning and generalization against label corruption[END_REF][START_REF] Malach | Decoupling 'when to update' from 'how to update[END_REF][START_REF] Song | Selfie: Refurbishing unclean samples for robust deep learning[END_REF][START_REF] Song | Robust learning by self-transition for handling noisy labels[END_REF] Hence, only the samples selected as clean (i.e., not noisy) are used during the update of the DNN model in the training step.

Finally, a fourth type of methods, known as loss adjustment, proposes to adjust the loss computation on the noisy training data before updating the DNN model, so that the impact of noisy labels can be reduced. [START_REF] Patrini | Making deep neural networks robust to label noise: A loss correction approach[END_REF][START_REF] Hendrycks | Using trusted data to train deep networks on labels corrupted by severe noise[END_REF] For example, Forward Loss Correction 14 is a popular method, which adjusts the loss function by integrating an estimated label transition matrix T in the training in order to mimic the behavior of the noisy labels. The procedure is divided into two main steps, namely the transition matrix estimation, which summarizes the probability of one class being flipped into another one, and the forward loss correction, that adjusts the loss.

Evidential Convolutional Neural Network

In order to handle uncertainty in DL classification tasks, Evidential Convolutional Neural Networks (EC-NNs) [START_REF] Denoeux | A neural network classifier based on dempster-shafer theory[END_REF][START_REF] Denoeux | A new evidential k-nearest neighbor rule based on contextual discounting with partially supervised learning[END_REF] have been proposed. ECNNs are built upon the framework of Convolutional Neural Networks (CNNs) in which the output is a probability distribution over all possible subsets of classes, along with different measures of uncertainty. [START_REF] Tong | An evidential classifier based on dempster-shafer theory and deep learning[END_REF] Following the evidential layer that replaces the usual softmax layer, a so-called utility layer 18 is used for decision making. This layer allows for the adoption of set-valued solutions corresponding to imprecise decisions. [START_REF] Chen | Evidential knn-based condition monitoring and early warning method with applications in power plant[END_REF][START_REF] Guettari | Blind image steganalysis based on evidential k-nearest neighbors[END_REF] The basic idea behind a partial classification decision is "better imprecise than wrong", which can be formulated using the utility matrix U ∈ [0, 1]

(2 c -1)×c , which specifies the utility u A,k of any decision A ⊆ Ω, A ̸ = ∅, when the true label is k ∈ Ω. This matrix allows us to calculate the expected-utility for each decision in favor of an hypothesis A ∈ 2 Ω using the Hurwicz criterion: 18

E m,ν (A) = νE m,ν (A) + (1 -ν)E m,ν (A), ( 1
)
where m is the mass function [START_REF] Tong | An evidential classifier based on dempster-shafer theory and deep learning[END_REF] corresponding to the general case and ν ∈ [0, 1] is a pessimism index that weights the lower and upper expected utilities E m,ν (A) and E m,ν (A):

E m,ν (A) = B⊆Ω m(B) min ωj ∈B u A,j , E m,ν (A) = B⊆Ω m(B) max ωj ∈B u A,j . (2)

PROPOSED METHOD

As mentioned in the problem statement, in the presence of complex data, the task of finding a hard partitioning among the classes is particularly difficult even for expert annotators. It thus results in noisy labeled datasets, which leads to poor ML model performance. Now, labelling errors are mainly due to ambiguities between some specific classes. They are thus not completely random but rather correspond to some confusion between two or three classes. Hence, we propose to handle these confusions by considering set-valued labels, also called "imprecise" labels, e.g. {ω i , ω j } when we doubt between ω i and ω j classes for a given sample.

In this context, starting with our dataset D containing noisy labels, we derive a two-step strategy that relies on: (1) A set-valued classifier able to identify the noisy samples; (2) A method to correct our model trained with noisy samples. The interactions between these two steps, that are detailed in the two following subsections, are summarized on Fig. 1. More specifically, the overall training procedure can be described in Algorithm 1.

Input image

Set-valued convolutional neural network classifier

In this section, we describe the proposed classifier that is derived from the ECNN classifier presented in Section 2.2. We first present the overall structure of the model and then explain the role of each part.

The architecture

As illustrated in Figure 1, our model involves mainly three components:

• a CNN backbone, which is used to extract some relevant latent features for the classification by propagating the input samples through its different hidden layers, and which consists mainly in convolution and pooling operations.

• a Softmax layer, that specifically converts the feature vectors into dogmatic Basic Belief Assignments (BBAs, cf. Section 3.1.2) which quantify the probability distribution of the sample class.

• a Utility layer, which calculates for each sample the expected-utility vector defined on the 2 Ω \ {∅}, based on the considered utility matrix U .

From Non Dogmatic to Bayesian BBA

In the ECNN, the hypotheses A ⊆ Ω with a non-null mass value (i.e., m(A) > 0) , called the focal sets, are the singleton hypotheses, namely ω 1 , ω 2 , . . . , ω c , and the hypothesis representing the complete ignorance, namely Ω. Now, using Eq. ( 1), we can show that introducing some mass on Ω promotes decisions in favor of singleton classes or Ω. For illustration, Figure 2 shows, for a 3-class problem (Ω = {ω 1 , ω 2 , ω 3 }), how the different decision areas evolve when we increase the mass on Ω (from left to right), and we clearly notice that the areas corresponding to decisions in favor of the hypotheses with cardinality equal to 2 ({ω i , ω j }, (i, j) ∈ {1, 2, 3}, i ̸ = j) decrease with m(Ω) until they completely disappear when m(Ω) = 0.3.

To illustrate further the evolution of these decision areas, we draw some samples in the different decision areas defined when m(Ω) = 0 (Fig. 2-a), and then we analyse the distribution of their predicted label in 2 Ω (i.e. possibly imprecise based on the Hurwitz criterion) when m(Ω) increases. Figure 3 shows some distribution examples versus m(Ω) for the samples initially in {ω 1 , ω 2 } decision area. We notice that, when m(Ω) increases, the samples are redistributed between the two singleton classes ω 1 and ω 2 , and the Ω class, with a different final behaviour against ν, i.e. either completely redistributed in the two singleton classes when ν = 0.2 or in Ω when ν = 0.5. Furthermore, according to some distribution results not shown in this paper, the samples which are initially in a singleton class can be redistributed only in Ω when this latter increases (for ν ≥ 0.5). Now, recall that we assume that confusions mostly occur when two classes are mislabeled as each other and our objective with set-valued classification is to assign classes with a cardinality of two for the uncertain samples, i.e. optimally the noisy samples. Then, relaxing the constraint of decision among singleton classes, we nevertheless aim at keeping the predicted imprecise classes with as low cardinality as possible. Thus, based on the previous analysis, in this work, we decide to set m(Ω) = 0 and only keep singletons as focal sets, i.e. to handle only Bayesian BBAs.

Utility layer

The utility layer output is the expected-utility vector of size equal to 2 c -1. To compute it, we connect any unit j of the softmax layer to each output unit representing a hypothesis A ⊆ Ω with a weight equal to u A,j , which specifies the utility of assigning the input sample to A ⊆ Ω when the true label is ω j ∈ Ω.

The coefficients u A,j of the utility matrix U ∈ [0, 1]

(2 c -1)×c are defined as an Ordered Weighted Average (OWA), [START_REF] Yager | On ordered weighted averaging aggregation operators in multicriteria decisionmaking[END_REF] and thus only depend on the cardinality of the considered imprecise class A. In, [START_REF] Ma | Partial classification in the belief function framework[END_REF] they are obtained by maximizing the entropy of the OWA operator, given an imprecision tolerance degree γ. In this work, we rather propose an estimation of u A,j ∈ [0, 1] in order to make the noisy samples be classified in an imprecise class of cardinality 2 (the noisy label and the true one) and the not noisy samples be classified in a singleton class (cardinality 1). Since we mostly focus on imprecise classes of cardinality 2, ∀A such that |A| = 1 we set u A,j = 1, ∀A such that |A| > 2 we set u A,j to a low value, and we set to a same value, denoted by u 2 ∈ (0, 1), all the utilities u A,j ∀A such that |A| = 2. Then, the estimation of u 2 is as follows. Based on Eq. ( 1), and (2), for any BBA, having as only focal sets the singletons ω i and Ω

∀i ∈ 1, c , E m ({ω i }) = m ({ω i }) + (1 -ν) m (Ω) , (3)
∀ (i, j) ∈ 1, c 2 , i ̸ = j, E m ({ω i , ω j }) = u 2 [m ({ω i }) + m ({ω j }) + (1 -ν) m (Ω)] . (4)
Then, we propose to plot the distributions of the quantity

α 1,2 = m({ωi})+(1-ν)m(Ω) m({ωi})+m({ωj })+(1-ν)m(Ω)
, which allows us to distinguish the clean samples correctly labelled ω i from the noisy samples wrongly labelled ω j with true label ω i . Indeed, we aim to set the u 2 value such that it allows us to separate the clean samples from the noisy ones by classifying the first ones in singleton classes.

Training with soft labels

As mentioned before, the main objective of the set-valued classifier is to identify the samples and assign them new class sets with cardinality greater than one. Indeed, we aim to train again our classifier model after having replaced the noisy labels of these samples, that correspond to a false class, by soft labels associated to a subset of classes which optimally includes also the true class. A possible way to fulfill this objective is to create some soft labels as follows.

First, let us recall that in the simple case of Bayesian BBAs, the mass function is equal to the belief function and to the plausibility function, and that the mass function restriction to singletons is also equal to the pignistic probability and to the contour function. Then, for any sample i classified as belonging to A ⊆ Ω, its soft label is defined as:

∀i ∈ 1, n , y sof t i := y sof t i (ω j ) = m(ωj ) ω k ∈A m(ω k )
if ω j ∈ A, y sof t i (ω j ) = 0 otherwise.

(5)

EXPERIMENTAL RESULTS

We tested our method on the MNIST [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF] and CIFAR-10 24 datasets.

Simulating realistic label corruption

Before carrying out the experiments, we artificially integrate noise in the datasets by noisifying samples in a manner that mimics the mistakes that can occur in a realistic dataset for similar classes, namely the classes 1 ←→ 7 for MNIST and CAT (3) ←→ DOG(5) for CIFAR-10, with varying ratios of noise. Figure 4 shows some examples of confusing samples in MNIST and CIFAR-10 respectively. With this aim, we first train a CNN backbone with a softmax layer on the original dataset without noise in order to get the class score vectors for each sample. Then, for a specific ratio of noise n ∈ [0, 1], we noisify the proportion n × N of the samples that have the smallest scores for their ground-truth classes, i.e. samples that are hard for the model to predict. The noise is introduced symmetrically between the classes in question, i.e. labels of samples of classes 1/CAT are replaced by 7/DOG and vice versa.

Set-valued classification 4.2.1 Learning parameters

Common to all experiments, the loss function used is cross entropy. The training dataset is divided into 20% for validation and 80% for training and the test dataset is not noisified. Adam optimizer is used during the training. For MNIST, a convolutional network with two convolutional hidden layers, initialized with Kaiming method, is used with a batch of size 128. Training is performed during 50 epochs with a learning rate of 0.001. For CIFAR-10, we employ a ResNet-34 [START_REF] He | Deep residual learning for image recognition[END_REF] pretrained on ImageNet with a batch size of 256. Training is performed during 10 epochs with a learning rate of 10 -5 .

Utility matrix estimation

Starting from the noisy dataset, we first estimate the values of the utility matrix needed for the set-valued classification, based on the estimation process presented in Section 3.1.3. By plotting the distribution of α 1,2 for the different classes, we notice that for the noisified classes, there is a difference between the distribution of the noisy not noisy samples, which gives an approximation interval of the value u 2 (cf. Section 3.1.3). Figures 5 and6 represent this distribution in the case of the noisified MNIST and CIFAR-10 datasets with different ratios of noise. Specifically, we empirically find that u 2 belongs to the range [0.55, 0.6] with a default value about 0.57. Hence, in the following, we employ a utility matrix where u 2 = 0.57, while ∀A ⊆ Ω such that |A| > 2, u A,j = 0.1 (low value to avoid decisions in favor to classes of cardinality greater than two).

Set-valued classification

The set-valued classifier enables us to obtain multi-labels classification. The distribution of the obtained multi-labels among noisy and not noisy samples is represented in Figures 7a and7b for MNIST and CIFAR-10 respectively, for a noise ratio varying as {0.1, 0.2, 0.3, 0.4}. The model is mostly able to identify the noisy samples, especially when the noise ratio is small, by assigning them to some sets of classes of cardinality two, while the majority of the not noisy samples were assigned to singleton classes.

Final experimental results and comparison with the related works

After obtaining the set-valued classification, the dataset is relabelled with soft labels as explained in Section 3.2 and the model is finetuned to give the final classification results, that we present in this section. In order to assess the given results, Table 1 presents the evolution of the accuracy against the noise ratio for three methods, namely training the considered CNN model on the original noisy dataset, training using the T-forward correction procedure [START_REF] Patrini | Making deep neural networks robust to label noise: A loss correction approach[END_REF] As we can see, for both CIFAR-10 and MNIST datasets, our method was able to reach higher values of accuracy than the T-forward procedure. The improvement is clearly robust to the noise ratio introduced in the dataset, and is more relevant when using MNIST dataset since it is more easy to handle. These results show that identifying the noisy samples helps more the model to adjust its classification results than using an estimated transition matrix to correct the loss function because the transition matrix models the class ambiguities without taking into account the dependence to the samples.

CONCLUSIONS AND FUTURE WORKS

In this paper, we have proposed a framework for dealing with noisy labels in datasets, using a set-valued (imprecise) classification. We derived a new classifier that combines a CNN with a utility layer in order to relabel the noisy samples in the dataset based on their predicted multi-labels. The performance of the method has been assessed in case of using the MNIST and CIFAR-10 datasets with different ratios of noise, and the accuracy was consistently better than the training using a CNN model and the popular T-forward correction method.

Several perspectives remain open. Firstly, recall that ECNNs integrate the uncertainty by placing a given amount of mass on Ω, without considering subsets of classes. Thus, in this first work, we end up using Bayesian BBAs because using the (c + 1) mass values does not reflect pair-wise class ambiguities since Ω is the only compound focal element. Hence, in future work, to benefit from the powerful modeling of belief functions, we aim to define an evidential layer whose output is a BBA with focal elements of intermediate cardinality values between one (singleton classes) and |Ω|, which feeds the utility layer. Secondly, we aim to integrate this utility layer in the training by adjusting the loss function to depend also on the utilities.

Figure 1 :

 1 Figure 1: Scheme of the proposed method.

Figure 2 :Figure 3 :

 23 Figure 2: Evolution of the decision areas for m(Ω) ∈ {0., 0.15, 0.3} (from left to right) and ν = 0.5.

Figure 4 :

 4 Figure 4: Examples of confusing samples for MNIST(1 -7) and CIFAR-10(Cat-Dog)

Figure 5 :

 5 Figure 5: Distribution of α 1,2 for MNIST with different values of noise: 0.1, 0.2, 0.3 and 0.4

Figure 6 :

 6 Figure 6: Distribution of α 1,2 for CIFAR-10 with different values of noise: 0.1, 0.2, 0.3 and 0.4

Figure 7 :

 7 Figure 7: Distribution of multi-labels (red) with noise (noisy samples on the left/not noisy on the right) for (a) MNIST and (b) CIFAR-10

Table 1 :

 1 and finally the proposed method. Accuracy values for different ratios of symmetric noise for MNIST and CIFAR-10.

	Method/Dataset		MNIST			CIFAR-10
		n = 0.1 n = 0.2 n = 0.3 n = 0.4 n = 0.1 n = 0.2 n = 0.3 n = 0.4
	CNN model	0.9717	0.9585	0.9395	0.9232	0.9241,	0.9141	0.8925	0.8701
	T-forward 14	0.9743	0.9605	0.9523	0.9477	0.9298	0.9204	0.9048	0.8781
	Our method	0.9897 0.9830 0.9731 0.9644 0.9411 0.9346 0.9131 0.8852