Manon Dampfhoffer
email: manon.dampfhoffer@cea.fr

Thomas Mesquida
email: thomas.mesquida@cea.fr

Alexandre Valentian
email: alexandre.valentian@cea.fr

Lorena Anghel
email: lorena.anghel@grenoble-inp.fr

Backpropagation-based Learning Techniques for Deep Spiking Neural Networks: A survey

Keywords: Deep learning, spiking neural networks, braininspired computing, neuromorphic computing

With the adoption of smart systems, Artificial Neural Networks (ANNs) have become ubiquitous. Conventional ANN implementations have a high energy consumption, limiting their use in embedded and mobile applications. Spiking Neural Networks (SNNs) mimic the dynamics of biological neural networks by distributing information over time through binary spikes. Neuromorphic hardware has emerged to leverage the characteristics of SNNs, such as asynchronous processing and high activation sparsity. Therefore, SNNs have recently gained interest in the machine learning community as a brain-inspired alternative to ANNs for low-power applications. However, the discrete representation of the information makes the training of SNNs by backpropagation-based techniques challenging. In this survey, we review training strategies for deep SNNs targeting deep learning applications such as image processing. We start with methods based on the conversion from an ANN to a SNN and compare these with backpropagation-based techniques. We propose a new taxonomy of spiking backpropagation algorithms into three categories, namely: spatial, spatio-temporal and singlespike approaches. In addition, we analyze different strategies to improve accuracy, latency and sparsity, such as regularization methods, training hybridization and tuning of the parameters specific to the SNN neuron model. We highlight the impact of input encoding, network architecture and training strategy on the accuracy-latency trade-off. Finally, in the light of the remaining challenges for accurate and efficient SNNs solutions, we emphasize the importance of joint hardware-software codevelopment.

I. INTRODUCTION

A RTIFICIAL Neural Networks (ANNs) are machine- learning algorithms inspired by the computations done by the brain while solving complex tasks. Deep ANNs (with more than a few hidden layers) have been successfully used in many applications such as image recognition [START_REF] Krizhevsky | Learning multiple layers of features from tiny images[END_REF], object detection [START_REF] Girshick | Rich feature hierarchies for accurate object detection and semantic segmentation[END_REF], speech recognition [START_REF] Hinton | Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups[END_REF], medical diagnosis [START_REF] Esteva | Dermatologist-level classification of skin cancer with deep neural networks[END_REF], game playing [START_REF] Silver | Mastering the game of go with deep neural networks and tree search[END_REF], etc. ANNs can solve difficult tasks using a large amount of data and large network architectures. This comes at the cost of a high energy consumption [START_REF] Li | Evaluating the energy efficiency of deep convolutional neural networks on cpus and gpus[END_REF], which limits their use in embedded and even more in mobile applications. It is known that biological neural networks in the human brain consumes very few energy while performing much complex tasks, which motivates brain-inspired computing.

Spiking Neural Networks (SNNs) are among the new generations of ANNs pushing the brain-inspired approach. In the This work has been partially supported by MIAI @ Grenoble Alpes, (ANR-19-P3IA-0003).

brain, neurons use electrical pulses to transmit information through the synapses in a sparse and asynchronous manner. Similarly, the neurons of SNNs communicate through temporal events (called spikes) and follow Integrate and Fire dynamics [START_REF] Lapicque | Recherches quantitatives sur l'excitation electrique des nerfs traitee comme une polarization[END_REF], [START_REF] Gerstner | Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition[END_REF]. Information can be encoded sparsely in the temporal dimension, in the spike rates or in the timing of spike emissions, and processed asynchronously. SNNs are sometimes seen as Recurrent Neural Networks (RNNs) in a wide sense [START_REF] Neftci | Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks[END_REF], as the state of a spiking neuron depends on its previous state, therefore naturally introducing recurrence. SNNs are also often compared to Binarized Neural Networks (BNNs), which use binarized weights and activations. However, in RNNs and BNNs the operations are executed in a synchronized manner, while in SNNs the computation can be event-driven (computation only when an input spike arrives).

SNNs aim not only at realistically emulating biological computation of the brain, but also target better energyefficiency than ANNs when implemented in hardware. ANN computations involve matrix multiplications which can be done in parallel on GPUs. On the other hand, SNN computations can be realized efficiently on dedicated neuromorphic hardware [START_REF] Rueckauer | Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification[END_REF], [START_REF] Pfeiffer | Deep Learning With Spiking Neurons: Opportunities and Challenges[END_REF]. Neuromorphic computing aims at mimicking the asynchronous information processing and colocalization of memory and computing units in the brain, as opposed to the traditional von Neumann paradigm. Large-scale neuromorphic accelerators [START_REF] Davies | Loihi: A neuromorphic manycore processor with onchip learning[END_REF]- [START_REF] Benjamin | Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations[END_REF] and application-specific accelerators [START_REF] Yin | Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations[END_REF], [START_REF] Mostafa | Fast classification using sparsely active spiking networks[END_REF] have been demonstrated, either for onchip training, or for inference only. While ANNs process the analog information in a one-shot fashion using matrix multiplications, information in SNN is coded in a binary signal distributed over time. The use of binary spikes is one of the motivations for SNNs as it allows replacing costly multiply-accumulate (MAC) operations in ANNs by simpler accumulate (AC) operations, which consume less energy and occupy less area [START_REF] Rueckauer | Conversion of analog to spiking neural networks using sparse temporal coding[END_REF]. Moreover, not all neurons spike during an inference phase. Thus, this sparsity of activations decreases the number of ACs operations to be performed, which is another motivation for the use of SNNs. In addition, the asynchronous behavior of SNNs can allow faster inference in some neuromorphic hardware such as the neuromorphic asynchronous processors proposed in [START_REF] Moradi | A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)[END_REF]. Note that most of the energy consumption of neural networks in specialized architectures comes from memory accesses associated with arithmetic operations rather than from the arithmetic operations themselves [START_REF] Horowitz | Computing's energy problem (and what we can do about it)[END_REF]. However, spike sparsity and event-based processing can help reduce the memory accesses by reducing the number of associated operations.

Various methods have been proposed to train SNNs in supervised or unsupervised manners. The Spike-Timing-Dependent Plasticity (STDP) rule is a local learning rule demonstrated by neurobiologists and is inspired by chemical mechanisms with computation capability existing in nervous system [START_REF] Dan | Spike timing-dependent plasticity: From synapse to perception[END_REF]. This rule makes it possible to train SNNs in an unsupervised and biologically-plausible manner and is suitable for on-chip learning due to its locality. Although it has been shown that STDP can approximate backpropagation update rules [START_REF] Tavanaei | BP-STDP: Approximating backpropagation using spike timing dependent plasticity[END_REF], [START_REF] Eshraghian | Training Spiking Neural Networks Using Lessons From Deep Learning[END_REF], it is not the most appropriate rule to train deep networks with high accuracy in the context of deep learning applications. On the other hand, backpropagation-based training methods are able to accurately train deep networks. In that case, deep SNNs can be trained either directly with backpropagation, or indirectly by converting a pre-trained ANN to a SNN formalism. In this survey, we focus on training deep SNNs, and therefore we will not discuss the applications of the STDP rule, for which we refer the reader to [START_REF] Pfeiffer | Deep Learning With Spiking Neurons: Opportunities and Challenges[END_REF], [START_REF] Eshraghian | Training Spiking Neural Networks Using Lessons From Deep Learning[END_REF]- [START_REF] Han | A Survey on Spiking Neural Networks[END_REF].

Surveys on SNNs [START_REF] Pfeiffer | Deep Learning With Spiking Neurons: Opportunities and Challenges[END_REF], [START_REF] Eshraghian | Training Spiking Neural Networks Using Lessons From Deep Learning[END_REF]- [START_REF] Roy | Towards spike-based machine intelligence with neuromorphic computing[END_REF] have been published the past few years, addressing spiking neuron models, learning rules (unsupervised and supervised), neuromorphic hardware and applications. Motivated by the success of many recently proposed backpropagation-based training methods for deep SNNs, this survey takes a different approach by focusing on the latter. A novel taxonomy of backpropagation-based learning algorithms for the direct training of SNNs is proposed and strategies to improve these algorithms in terms of accuracy, latency and sparsity are discussed. In addition, the impact of the input encoding and network architecture on the accuracylatency trade-off is highlighted.

The remainder of the paper is organised as follows. In Section II, the SNN model is presented, addressing the choices of neurons, synapses and coding strategy. In Section III, the different strategies to train deep SNNs from conversion methods to backpropagation-based methods are reviewed. In Section IV, strategies to improve the training of deep SNNs are discussed. Targeting accurate and energy-efficient hardware inference, the accuracy-latency trade-off is studied in Section V. Finally, promising solutions and remaining challenges for SNNs are summarized in Section VI.

II. THE SPIKING NEURAL NETWORKS MODEL

A. Neurons and synapses

The basic ANNs and SNNs units in a neural networks are shown in Fig. 1. In ANNs, the output of a neuron is a function defined as:

y i = φ( j x j w ij + b i ) (1)
where y i is the output activation of neuron i, b i is the bias of neuron i, x j is the input activation from presynaptic neuron j, and w ij is the synaptic weight between neurons i and j. φ is an activation function, such as the Rectified linear unit (ReLU). While in ANNs the information propagates synchronously on a layer-by-layer basis, the information processing in SNNs is asynchronous and in a depth-first manner. Indeed, neurons in a layer fire spikes without waiting for other neurons in the same layer to fire. Due to the temporal dynamics of the neurons, SNNs operate in the spatio-temporal domain, while standard ANNs operate only in the spatial domain.

The most popular neuron model for SNNs is the Leaky Integrate and Fire model (LIF) [START_REF] Lapicque | Recherches quantitatives sur l'excitation electrique des nerfs traitee comme une polarization[END_REF], [START_REF] Gerstner | Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition[END_REF]. More complex neuron models exist [START_REF] Hodgkin | A quantitative description of membrane current and its application to conduction and excitation in nerve[END_REF], [START_REF] Izhikevich | Simple model of spiking neurons[END_REF], but have not yet demonstrated superior performance than the simple LIF model for deep learning applications. In the LIF model, the membrane potential V i (t) of the neuron i is described as :

λ dVi dt = -V i + j w ij k ϵ(t -t jk ) V i (t) = V reset , if V i (t) ≥ θ i ( 2
)
where λ is the membrane time constant, w ij is the synaptic weight from neuron j to i, ϵ(.) is the synaptic kernel, t jk is the k th spike of the input neuron j, V reset is the reset membrane potential and θ i is the membrane potential threshold. It is also possible to add a bias in the SNN model, for example by setting V i (0) = b i . Similar to biological neurons, the neuron integrates the weighted input spikes into its membrane potential. When the latter reaches its threshold, the neuron fires an output spike and the membrane potential is reset. Depending on the chosen membrane time constant, the Integrate and Fire neuron can also be non-leaky (IF). The difference is that the membrane potential of the IF neuron is constant in time between two spikes, while in the LIF neuron the membrane potential decays over time. The synapse model is defined by the kernel function ϵ(.), corresponding to the response of the neuron membrane potential to the presynaptic spike. The synapse can be instantaneous (Dirac kernel function) or continuous (e.g. linear, exponential, or alpha kernel functions). Neurons and synapses are shown in Fig. 1b. The inference phase of SNNs is usually discretized in timesteps, each timestep corresponding to one forward pass in the network. In this case, an iterative version of the LIF model is used, as shown in [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF]. The number of timesteps is generally used to estimate the future latency of the SNN inference in hardware.

B. Coding strategy

Contrary to ANNs, SNNs use the temporal dimension to code the information. Indeed, while ANNs use static realvalued activations per neuron per inference, SNNs use one or several binary spikes for which the temporal information is used to code the information. Therefore, the coding strategy refers to how the temporal information in the spikes is taken into account. There are several coding strategies based on the spike rate, timing, rank, phase, etc. (for a recent review see [START_REF] Auge | A survey of encoding techniques for signal processing in spiking neural networks[END_REF]), but the majority of works in deep SNNs use either the spike rate or the spike timing. The rate coding strategy uses several spikes to represent one unit of information while in temporal coding, the information is carried by individual spike times. Note that, in order to be efficient in neuromorphic hardware, the coding strategy should use a minimum number of spikes, as the energy consumption is strongly correlated to the spiking activity. Moreover, the coding strategy affects the behavior of the entire system, the characteristics of the neurons and synapses, and the efficiency of the learning process.

The coding strategy must consider both the encoding of the input to the network and the decoding of the output. Indeed, to process real-valued data, such as pixels for images, these values can be converted into spikes in order to be processed further by the SNN. Note that neuromorphic sensors can provide data already in the form of spikes which can be fed directly to the SNN without pre-processing. Fig. 1c represents a typical pixel-to-spike conversion in rate and time. The ratebased strategy matches each pixel intensity with a firing rate, using a probabilistic sampling (generally Poisson) to generate the spike trains: the higher the pixel value, the higher the firing rate of the corresponding input. A simple time-based strategy, also called latency coding, consists in associating the pixel intensity with the latency of a single spike. In that case, the latency is inversely proportional to the pixel intensity: earlier spikes encode higher values and later spikes encode lower values. Decoding the output in a classification task consists in determining the most activated neuron in the output layer, each neuron being associated with a class. With rate coding, this is done by using the highest spike rate or membrane potential value. With temporal coding, a popular solution called Time-To-First-Spike (TTFS) uses the first spike fired by a neuron.

Temporal codes are sparse and can have a lower latency (e.g. when the TTFS decoding is used). However, temporal coding requires high temporal resolution because each spike carries important information, which may be difficult to implement efficiently in neuromorphic hardware [START_REF] Stöckl | Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes[END_REF]. Moreover, the analogy with ANNs is more straightforward with rate coding, and therefore the latter is easier to use for supervised training, either with conversion or direct training approaches.

III. TRAINING DEEP SNNS

Learning is the process by which the weights (i.e. the values of the synapses) of the neural network are determined such that the network performs a specific task. Most deep learning applications use supervised learning, meaning that the expected outputs are known (each data is associated to a label). A cost metric is defined as a function of the error between the desired and actual output. The learning process consists of tuning the values of the synaptic weights such that the cost function is minimized. In ANNs, the backpropagation algorithm is used to calculate the gradients of the cost function with respect to each synaptic weight in order to perform synaptic weights updates. The calculation starts from the last layer of the network and proceeds backwards layer by layer. This survey focuses on supervised strategies inspired by the backpropagation algorithm for training deep SNNs. Applying backpropagation to SNNs is challenging due to the nature of the neuron activation function (step function) which is not differentiable, thus the error can not be backpropagated correctly. Different strategies have been proposed to mitigate this problem, such as approximating the derivative with a surrogate gradient [START_REF] Neftci | Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks[END_REF] (see Fig. 3), or directly differentiating the spike times [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]. An alternative solution is to convert a trained ANN to a SNN formalism, also called indirect training or ANN-to-SNN conversion, which bypasses the training difficulty of SNNs.

A. ANN-to-SNN conversion

The ANN-to-SNN conversion is an indirect training strategy consisting in training an ANN and then mapping the trained weights to a SNN, assuming equivalence of the SNN computing units to the ANN ones. The ANN must be trained under constraints to fit the SNN model, such as removing biases of neurons (which are usually not represented in the SNN model) and batch normalization layers (which rely on biases). Then, either the thresholds of the spiking neurons or the weights are normalized equivalently, so that the transfer function (inputoutput mapping) of the SNN unit matches the transfer function of the ANN unit [START_REF] Diehl | Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing[END_REF], [START_REF] Sengupta | Going deeper in spiking neural networks: VGG and residual architectures[END_REF].

1) Conversion with rate coding: Rate coding is a straightforward approach to conversion, in which the firing rate of neurons is considered equivalent to the analog output (activation value) of the ANN neuron [START_REF] Rueckauer | Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification[END_REF], [START_REF] Cao | Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition[END_REF]. However, the conversion process results in errors in some cases, for instance when the ANN activation is too high and can not be accurately represented by the spike rate given a fixed simulation duration. An effective data-based weight normalization, consisting in rescaling the weights in each layer according to the maximum ANN activation in the corresponding layer within the training set, is presented in [START_REF] Diehl | Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing[END_REF] to mitigate this problem. Another solution is proposed in [START_REF] Sengupta | Going deeper in spiking neural networks: VGG and residual architectures[END_REF], which balances the thresholds in each layer according to the maximum SNN activation. The difference with the method in [START_REF] Diehl | Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing[END_REF] is that SNN statistics are used instead of ANN statistics to determine the normalization, which leads to more accurate results. They report high accuracy for a converted SNN with VGG-16 network architecture, such as 91.55% on CIFAR-10 and 69.96% on ImageNet using 2500 inference timesteps. The accuracy can be further improved using a soft reset mechanism instead of a hard reset [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF]. The soft reset consists in subtracting the threshold value from the membrane potential after the neuron fires a spike, instead of setting it to the reset potential value. The residual membrane potential above the threshold is thus kept for the next spike, which reduces the loss in the spiking quantization process. Their method yields a near lossless conversion, showing 93.63% accuracy on CIFAR-10 and 73.09% on ImageNet with a VGG-16 architecture using 2048 and 4096 timesteps, respectively. Similar results are shown with ResNet-20 and ResNet-34 on CIFAR-10 and ImageNet achieving 91.36% and 69.89% accuracy, respectively.

2) Conversion with temporal coding: Another approach to conversion is based on temporal coding. This approach is attractive because the number of spikes emitted can be decreased drastically, thus further reducing the energy consumption. This is first proposed in [START_REF] Rueckauer | Conversion of analog to spiking neural networks using sparse temporal coding[END_REF] using the equivalence between the activation value of the ANN unit and the inverse of the spike time of the SNN unit. In [START_REF] Han | Deep spiking neural network: Energy efficiency through time based coding[END_REF] an accuracy as high as the one obtained with rate coding [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF] is demonstrated on CIFAR-10 and Imagenet using VGG-16 and ResNet architectures, with at most two spikes per neuron. They propose a novel temporal coding with one positive and one negative spike per neuron. In addition, they introduce a threshold balancing method to improve the accuracy while using fewer timesteps compared to the rate-based conversion of [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF]. Furthermore, the paper [START_REF] Stöckl | Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes[END_REF] introduces a temporal code associated with a new spiking neuron model using logN different values of spike times to transmit integers between 1 and N in order to reduce the required temporal resolution. They achieve 83.57% accuracy on ImageNet with the EfficientNet-B7 architecture (75.10% with ResNet-50), with on average less than 2 spikes per neuron per inference. However, the proposed neuron model is complex, requiring additional parameters and additional state functions computed at each timestep, and might be costly to implement in neuromorphic hardware.

3) Limitations: The ANN-to-SNN conversion results in SNNs with accuracy close to the accuracy of the original ANN. Although originally designed for rate coding, the conversion process is compatible with many coding strategies. In fact, conversion with temporal coding is a promising way to increase the spike sparsity, but it might be less robust to noise in hardware implementation. However, both coding strategies require hundreds to thousands of inference timesteps, leading to a much higher latency and a degraded energy efficiency [START_REF] Roy | Towards spike-based machine intelligence with neuromorphic computing[END_REF]. The long inference time required to achieve high accuracy is inherent to the equivalence chosen for the conversion. Indeed, the ReLU activation function approximates the firing rate of the IF model only if the SNN inference is discretized with a sufficient number of timesteps. However, recent improvements in conversion methods can mitigate this problem. For instance, the authors in [START_REF] Bu | Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks[END_REF] propose to replace the standard ReLU in the ANN model by the quantization clip-floor-shift activation function, which better approximates the firing rate of the IF model. Indeed, by clipping and flooring, they suppress the conversion errors due to large activations in the ANN and the discretization of the SNN activation (resp.), if the quantization step is properly chosen according to the time discretization of the SNN inference. Therefore, they reduce the conversion error between the ANN and SNN model and make it possible to obtain SNNs with a much smaller number of timesteps. For instance, on ImageNet with ResNet-34, they achieve 69.37% (resp. 72.35%) accuracy with 32 (resp. 64) timesteps. However, there is still a gap between the accuracy of the ANN and the converted SNN due to unevenness errors, which result from the fact that a different order of input spikes produces a different output [START_REF] Bu | Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks[END_REF]. Besides, the conversion process does not allow the optimization of the temporal dynamics of the SNN, contrary to direct training approaches [START_REF] Rathi | Exploring spike-based learning for neuromorphic computing: Prospects and perspectives[END_REF].

B. Backpropagation-based learning algorithms for deep SNNs

The direct training of SNNs with backpropagation was partly motivated by the objective of reducing the SNN latency. In order to apply backpropagation to SNNs, different strategies are used to circumvent the problem of the non-differentiability of the spiking activation function, such as approximating the derivative with a surrogate gradient [START_REF] Neftci | Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks[END_REF], or directly differentiating the spike times [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]. The surrogate gradient technique consists in approximating the neuron's activation function (which is usually a step function) by a differentiable function during the backward pass, to enable informative gradients to backpropagate through the layers (see Fig. 3). The approach was first introduced by [START_REF] Hinton | Neural networks for machine learning, coursera[END_REF], [START_REF] Bengio | Estimating or propagating gradients through stochastic neurons for conditional computation[END_REF] with the straight-through estimator used to train quantized neural networks. Furthermore, the SNN inference being discretized in timesteps, there are several strategies to apply backpropagation to SNNs (see Fig. 2). We classify these methods into three categories. [START_REF] Krizhevsky | Learning multiple layers of features from tiny images[END_REF] The spatial approach uses accumulated quantities which are retrieved at the end of the inference phase (such as the spike count or membrane potential value) to serve as an activation value for each neuron. Then, backpropagation can be applied on these quantities. This strategy does not take into account the SNN temporal dynamics (i.e. the precise timing and order of the spikes and temporal components of the SNN model).

(2) The spatio-temporal approach considers each timestep of the inference phase by using backpropagation through time (BPTT), which was originally used to train RNNs. The BPTT consists in applying the backpropagation method on an unrolled SNN network, where all the timesteps are simultaneously considered (one neuron unrolled for T timesteps will be instantiated T times in this virtual network). This method thus considers both temporal and spatial dynamics of SNN, and therefore takes into account the temporal dependencies associated with LIF neurons and continuous synapses. (3) The single-spike approach uses only one spike per neuron (using a latency temporal coding), and computes directly the spike time of each neuron during the inference phase. Then, the backpropagation algorithm can be applied using the spike time of each neuron as its activation value.

1) Spatial approaches: Spatial approaches, such as [START_REF] Lee | Training Deep Spiking Neural Networks Using Backpropagation[END_REF]- [START_REF] Wu | Deep spiking neural network with spike count based learning rule[END_REF], use a rate-coding strategy with IF neurons and instantaneous synapses and do not consider temporal dependencies in the gradient computation. They consist in approximating the SNN forward pass during the training in order to obtain a lighter backpropagation, only in the spatial domain, as in ANN training. The spatial approach resembles the ANN-to-SNN conversion with rate coding, however the training targets are different. In conversion methods, the ANN is trained under constraints and then, there is a conversion procedure to transfer the ANN trained weights to the SNN. On the other hand, in the spatial approach, the SNN is directly trained, but viewed as an ANN, and thus can be trained in a similar way using accumulated quantities during the SNN forward pass. Hence, contrary to the ANN-to-SNN conversion, there is only one network and no conversion procedure.

For instance, the authors in [START_REF] Lee | Training Deep Spiking Neural Networks Using Backpropagation[END_REF] consider the membrane potential without the spiking discontinuities, using low-pass filtered spike signals. Therefore the signal is continuous (considering the spiking activity as noise) and backpropagation can be applied on it. They use an architecture with lateral inhibitions and neurons with a refractory periods. To keep the performance stable, error normalization, threshold regularization and a novel exponential weight regularization are used to reach 99.31% accuracy on MNIST with a 4-layer CNN architecture. In [START_REF] Wu | Deep spiking neural network with spike count based learning rule[END_REF] the authors take the spike count of neurons as a surrogate for gradient backpropagation. They define the equivalent of the neuron activation as the sum of its spikes temporal approaches use a rate coding while the single-spike approach use a temporal (latency) coding. The spatio-temporal approach considers the activation of each neuron at each timestep a t i , corresponding to the emission or not of a spike s t i . The backpropagation through time is used to backpropagate the error in both space and time dimensions. On the other hand, the spatial and single-spike approaches consider for each neuron a single activation a i for the forward pass, which can correspond to the spike count c i for the former or the timing of the unique spike emitted by the neuron t i for the latter. Therefore, the backpropagation is used to backpropagate the error only in the space dimension. (b) Backpropagation and backprogation through time training. Notations: n number of layers, T number of timesteps used in the SNN inference, A (l) activation of neurons in layer l, W (l) weight vector from layer l to l + 1, τ membrane potential and postsynaptic potential update (for LIF neurons and continuous synapses). Fig. 3. The activation function of neurons has derivative equals to zero everywhere except in θ where it is infinite. Therefore, the derivative of a surrogate function is used to compute the gradients during the backward pass.

𝜏 𝑨 (𝟐) 𝑨 (𝟏) 𝑨 (𝟑) 𝑨 (𝟏) … 𝑨 (𝒏) … 𝐒𝐩𝐚𝐜𝐞 𝑡 = 0 𝐓𝐢𝐦𝐞 𝐴 (1) 𝑨 (𝟐) … 𝑨 (𝟐) 𝑨 (𝒏) … 𝑨 (𝒏) 𝑨 (𝟑) … 𝑨 (𝟑) … … 𝑡 = 1 𝑾 (𝟏) 𝑾 (𝟐) 𝑾 (𝒏-𝟏) 𝑾 (𝟏) 𝑾 (𝟐) 𝑾 (𝒏-𝟏) 𝑾 (𝒏-𝟏) 𝑾 (𝟐) 𝑾 (𝟏) 𝑡 = 𝑇 𝐸𝑟𝑟𝑜𝑟 … 𝐒𝐩𝐚𝐜𝐞 𝐸𝑟𝑟𝑜𝑟 𝐁𝐚𝐜𝐤𝐩𝐫𝐨𝐩𝐚𝐠𝐚𝐭𝐢𝐨𝐧 𝑨 (𝟏) 𝑨 (𝟐) 𝑨 (𝟑) 𝑨 (𝒏) … 𝑾 (𝟏) 𝑾 (𝟐) 𝑾 (𝒏-𝟏)

Rate

produced during the simulation time [START_REF] Wu | Deep spiking neural network with spike count based learning rule[END_REF]:

a n i = T t=1 s t,n i (3)
with a n i the equivalent activation of neuron i from layer n, and s t,n i a spike produced by this neuron at timestep t. Therefore, the aggregated input current of a neuron can be expressed as [START_REF] Wu | Deep spiking neural network with spike count based learning rule[END_REF]:

z n i = θ j w n-1 ij a n-1 j + b n i (4)
They use a cross-entropy loss, as it is commonly used for classification tasks:

E(a N i , y i ) = -log( exp(a N i ) k exp(a N k ) ) (5)
with a N j the output spike count of neuron j in the last layer and y j the target one-hot label for this neuron. Then, the error derivative with respect to weights and biases is computed using standard backpropagation. For instance, in the hidden layer, the error derivative with respect to the weights is computed using the chain rule [START_REF] Wu | Deep spiking neural network with spike count based learning rule[END_REF]:

∂E ∂w n-1 ij = ∂E ∂a n i ∂a n i ∂z n i ∂z n i ∂w n-1 ij (6)
with

∂a n i ∂z n i = 1 θ .(z n i > 0)
. They achieve 99.26% accuracy on MNIST with a 3-layer CNN architecture. However, they argue that considering only the spike count generates a quantization error, as the surplus membrane potential of spiking neurons is not taken into account, which could become a problem for deeper neural networks. Furthermore, the paper [START_REF] Thiele | SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes[END_REF] shows that the backpropagation phase can also be realized with spikes by considering the error in a discrete form. Therefore, the same hardware infrastructure can be used for both inference and learning, which makes it attractive for on-chip learning. Moreover, they show that the spike discretization error can be reduced to zero by adding some constraints on the ANN. Therefore, they can perform the offline training directly with the equivalent ANN. They demonstrate 89.99% accuracy on CIFAR-10 with a 8-layers VGG architecture. However, because they perform training and inference with the equivalent ANN, they do not indicate the number of inference timesteps which would be used for the inference with the SNN.

Overall, the spatial approach is purely rate-based and therefore is the closest to ANNs. This approach was not used in many works as it appears to not benefit from the spatiotemporal dynamics of SNNs. However, in [START_REF] Wu | Deep spiking neural network with spike count based learning rule[END_REF] the authors argue that spatial rules are less complex than spatio-temporal rules and can be more efficient in terms of computation and memory due to the use of backpropagation instead of BPTT. Moreover, the most recent ANN-to-SNN conversion techniques resemble the spatial SNN training. Indeed, they convert a SNN from an ANN that matches more closely the SNN such that the differences between the two models decrease and the conversion becomes more straightforward [START_REF] Bu | Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks[END_REF].

2) Spatio-temporal approaches: Spatio-temporal approaches, such as those described in [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF], [START_REF] Shrestha | SLAYER: Spike layer error reassignment in time[END_REF]- [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF], also use a rate-coding strategy, but propagate the gradient both in spatial and temporal dimensions using the BPTT. They usually have temporal components, such as in the neurons (LIF), synapses (continuous), or in the loss function. The majority of works use rate-coded loss functions (based on the output firing rate or output membrane potential) and LIF neurons with instantaneous synapses.

Paper [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF] proposes a spatio-temporal backpropagation for SNN based on an iterative LIF model and approximated the non-differentiable spiking activity with a surrogate gradient.

Their iterative LIF obtained by solving the differential equation of the LIF is defined as [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF]:

               u t+1,n i = u t,n i f (o t,n i ) + x t+1,n i + b n i x t+1,n i = l(n-1) j=1 w n ij o t+1,n-1 j o t+1,n i = g(u t+1,n i ) f (x) = τ exp(-x/τ ) g(x) = 1 if x ≥ θ , 0 otherwise. (7) with u t,n i , o t,n i , x t,n
i , respectively the membrane potential, output, and input current of neuron i in layer n at time t. They use a loss based on the mean square error for all samples of the batch (of size S) between the target vector (y) and the sum of the output vector of neurons in the last layer during the simulation time T [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF]:

L = 1 2S S s=1 ||y s - 1 T T t=1 o t,N s || 2 2 (8)
The derivative of the loss with respects to the membrane potential and output spikes is needed to compute the error derivative with respect to weights and biases. These values are obtained by unrolling the network in time and space. For instance, in the hidden layers at a given timestep, the loss derivative with respect to output spikes depends on the output spikes of the next layer (space) and the output spikes at the next timestep (time). Similarly, the loss derivative with respect to the membrane potential depends on the output spikes produced by this neuron (space) and the output spikes at the next timestep (time) [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF]:

∂L ∂o t,n i = l(n+1) j=1 ∂L ∂o t,n+1 j ∂o t,n+1 j ∂o t,n i + ∂L ∂o t+1,n i ∂o t+1,n i ∂o t,n i = l(n+1) j=1 ∂L ∂o t,n+1 j ∂g ∂u t,n+1 i w ij + ∂L ∂o t+1,n i ∂g ∂u t+1,n i u t,n i ∂f ∂o t,n i (9) ∂L ∂u t,n i = ∂L ∂o t,n i ∂o t,n i ∂u t,n i + ∂L ∂o t+1,n i ∂o t+1,n i ∂u t,n i = ∂L ∂o t,n i ∂g ∂u t,n i + ∂L ∂o t+1,n i ∂g ∂u t+1,n i f (o t,n i ) (10)
All the terms can be derived using the equations ( 7)-( 8), except the derivative of the spiking activation function ( ∂g ∂u ), which must be approximated by a surrogate gradient. They demonstrate 99.42% accuracy on MNIST with a 4-layer CNN architecture. The authors in [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF] follow the approach of [START_REF] Lee | Training Deep Spiking Neural Networks Using Backpropagation[END_REF] by considering the membrane potential without the spiking discontinuities, but with a leak in the neuron model. They achieve 90.95% accuracy on CIFAR-10 with a ResNet-11 using 100 timesteps for inference with on average 1.53x10 6 spikes per inference, which leads to x15 efficiency gain with respect to MAC and AC operations (using the energy values given in [START_REF] Horowitz | Computing's energy problem (and what we can do about it)[END_REF]) compared to the ANN version. In [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF], they do not use temporal components in the neuron and synapse models, but use a temporal loss inspired by the van Rossum distance. Therefore, they minimize the distance between the desired and actual output spike train.

The use of surrogate gradient to approximate the nondifferentiable behavior has the effect of smoothing the spiking activity. This leads to an inconsistency between the computed gradient and loss, which degrades the accuracy [START_REF] Jin | Hybrid macro/micro level backpropagation for training deep spiking neural networks[END_REF], [START_REF] Zhang | Temporal spike sequence learning via backpropagation for deep spiking neural networks[END_REF]. Therefore, the authors in [START_REF] Jin | Hybrid macro/micro level backpropagation for training deep spiking neural networks[END_REF] introduce a backpropagation at the spike train level with a decoupled macro factor accounting for the rate-coded loss function and a micro factor to incorporate temporal dependencies at the spike train level. They show that this decomposition improves the precision of the temporal learning. In the same spirit, in [START_REF] Zhang | Temporal spike sequence learning via backpropagation for deep spiking neural networks[END_REF], the derivative of the error is decomposed in two factors, one accounting for the inter-neuron dependencies and one accounting for the intraneuron dependencies. They use LIF neurons with continuous exponential synapses and consider incremental changes in the postsynaptic potential with regards to the membrane potential of a neuron to compute the gradients. Moreover, they use a temporal loss function analogous to the van Rossum distance. They proposed a warm-up mechanism with a sigmoid instead of a step function as activation function to enable backpropagation when there is no spike. They improved the temporal learning compared to [START_REF] Jin | Hybrid macro/micro level backpropagation for training deep spiking neural networks[END_REF] and scaled to CIFAR-10, while reducing the inference latency and showing high spike sparsity with on average 0.49 spike per neuron per inference.

In summary, the spatio-temporal approaches consider spatial and temporal dependencies using the BPTT. This is relevant to take into account the temporal components of the model (such as in the neurons, synapses and loss function) in the backpropagation. Therefore, they can benefit from the full spatio-temporal dimension of the SNN dynamics.

3) Single-spike approaches: Single-spike approaches, such as [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF], [START_REF] Coms ¸a | Temporal coding in spiking neural networks with alpha synaptic function: Learning with backpropagation[END_REF]- [START_REF] Zhou | Temporal-coded deep spiking neural network with easy training and robust performance[END_REF], circumvent the non-differentiability problem when applying backpropagation in SNNs by directly differentiating the spike times. These methods have been applied to image processing, the spatial information contained in the images being directly converted to the spike timing using a temporal (latency) coding.

The single-spike approach was introduced very early with the SpikeProp learning rule [START_REF] Bohte | Error-backpropagation in temporally encoded networks of spiking neurons[END_REF]. SpikeProp defines the firing time of neurons as a function of their membrane potential and thus approximates their derivative using the changes of the membrane potential around the firing time. Later, the authors in [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF] demonstrated that by using single-spike IF neurons with exponential synapses, the differential equation of the neuron membrane potential has a simple solution. Indeed, they define the IF neurons and exponential synapses as [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]:

dV j mem (t) dt = i w ji r ϵ(t -t r i ) (11)
with t r i the time of the rth spike from neuron i and ϵ the synaptic current kernel given by [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]:

ϵ(x) = exp(- x τ syn ) if x ≥ 0 , 0 otherwise. ( 12
)
with τ syn the synaptic time constant. Then, assuming a neuron will spike at time t out , they integrate the membrane potential for t < t out , which allows to obtain the output spike time of a neuron (t out ) as a function of the spike times of its presynaptic neurons [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]:

exp(t out ) = i∈C w i exp(t i ) i∈C w i -θ (13)
C = {i : t i < t out } is the causal set of the neuron, containing the input spikes that had arrived before its output spike and thus have influenced the output spike. Due to this analytic input-output relation, the spike times can be computed directly without simulating the whole network dynamics (i.e. computing membrane potentials at each timestep). Therefore, there is no need to do a BPTT, but a direct backpropagation only on the spike times is possible. A particularity of this algorithm is that, due to the form of the analytical relation, for a given neuron, the backpropagation rule only applies to the presynaptic neurons in its causal set. Furthermore, backpropagation is not performed on the neurons that do not spike. Therefore, the authors in [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF] apply weight regularization to push neurons to spike, by forcing its presynaptic sum of weights to be superior to its threshold. Using a change of variable exp(t x ) -→ z x , the derivative of the output spike time with respect to the weight and with respect to an input spike time is computed [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]:

dz out dw p = z p -z out i∈C w i -θ if p ∈ C, 0 otherwise. ( 14
)
dz out dz p = w p i∈C w i -θ if p ∈ C, 0 otherwise. ( 15
)
The loss function for the output spike times vector of the output neurons z L and a target class index g is defined as [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]:

L(g, z L ) = -ln exp(-z L [g]) i exp(-z L [i]) (16)
The loss function takes the negative of the spike times (in the z-domain) to encourage the neurons corresponding to the correct class to fire earlier. They achieve 97.2% accuracy on MNIST with a 784-800-10 fully-connected network.

Papers [START_REF] Coms ¸a | Temporal coding in spiking neural networks with alpha synaptic function: Learning with backpropagation[END_REF] and [START_REF] Göltz | Fast and deep neuromorphic learning with first-spike coding[END_REF] take their inspiration from [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF] but use IF neurons and synapses with alpha synaptic kernel, and LIF neurons and synapses with dual exponential kernel, respectively. Their models are more biologically plausible, but the differential equations have complex solutions. Performance in hardware is demonstrated in [START_REF] Göltz | Fast and deep neuromorphic learning with first-spike coding[END_REF] by implementing the algorithm in BrainScaleS-2. They yield 95.9% accuracy with 25µJ per classification on MNIST (16x16 images) with a 256-128-10 fully-connected network.

Another work [START_REF] Zhang | Spike-timing-dependent back propagation in deep spiking neural networks[END_REF] uses IF neurons with linear synapses, taking inspiration from the ReLU units used in ANNs, demonstrating 99.2% accuracy on MNIST with a 5-layer CNN architecture. They argue that the linear synapse alleviates the problem of exploding gradient and dead neurons compared to the alpha synapse (and more generally to a decaying kernel function). Indeed, the derivative of the linear synapse is never close to zero, and there is no leak in the model therefore neurons are more likely to spike. The authors in [START_REF] Sakemi | A supervised learning algorithm for multilayer spiking neural networks based on temporal coding toward energy-efficient vlsi processor design[END_REF] also use linear synapses and added a temporal penalty term to stabilize the timing of the output spike instead of the penalty on the weights used by [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF]. This allows to control the output spike times of the network and therefore to tune the inference latency. Furthermore, considering analog VLSI implementations, they show that their SNN dynamics can be mapped to circuits with analog resistive memory. Using numerical simulations, the robustness of their model to device variations (when taken into account during the training) was demonstrated for a small fully connected network on the MNIST task. Besides, the authors in [START_REF] Kheradpisheh | S4nn: temporal backpropagation for spiking neural networks with one spike per neuron[END_REF] simplify the model of [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF] by using instantaneous synapses with IF neurons. However, because the synapse is not continuous, they must approximate the derivative of the spike time with regards to the membrane potential, and so the gradients are not exact. They obtain 97.4% accuracy on MNIST with a 784-400-10 fully-connected network.

Finally, the paper [START_REF] Zhou | Temporal-coded deep spiking neural network with easy training and robust performance[END_REF] extends the work of [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF] by proposing an efficient way of computing the spike times taking advantage of parallel tensor computations in deep learning frameworks running on GPUs, thus speeding-up the offline training. They achieved 92.68% and 68.8% accuracy on CIFAR-10 and ImageNet using VGG-16 and GoogleNet architectures, respectively. Targeting efficient inference, they show that their method is robust to weight quantization. Indeed, they yield 90.93% accuracy for CIFAR-10 with 2-bits quantization, and 65.2% (resp. 60.0%) for ImageNet with 4-bit (resp. 2bit) quantization. In addition, they report high sparsity of activations, with on average 0.62 (resp 0.56) spike per neuron per inference on CIFAR-10 (resp. ImageNet). However, as they directly compute the spike times without simulating the SNN dynamics, they do not discretize the inference phase with timesteps. Therefore, the number of timesteps which would be used to simulate the SNN inference with the desired temporal resolution in hardware is unknown.

In summary, single-spike approaches have the advantage of using at most one spike per neuron and thus appear promising for energy-efficient hardware implementations. Moreover, using the analytical input-output relation allows us to compute exact gradients instead of using approximate gradients, which is better for the optimization process. However, the backpropagation is only performed on presynaptic neurons which have spiked before the given neuron. Therefore, fewer synaptic weights are updated per iteration, which could lead to a slower training convergence. Moreover, encouraging neurons to fire during training improves the learning [START_REF] Mostafa | Supervised learning based on temporal coding in spiking neural networks[END_REF], [START_REF] Zhou | Temporal-coded deep spiking neural network with easy training and robust performance[END_REF] but can limit the sparsity reached. In addition, while the single-spike approach is adapted to static input data, it is hardly compatible with dynamically changing input data, as neurons can fire only once [START_REF] Eshraghian | Training Spiking Neural Networks Using Lessons From Deep Learning[END_REF].

4) Limitations: Considering the spatial and single-spike approaches, only the works [START_REF] Thiele | SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes[END_REF], [START_REF] Zhou | Temporal-coded deep spiking neural network with easy training and robust performance[END_REF] have shown scalability to CIFAR-10, and ImageNet for the latter. The spatio-temporal approach is the only approach considering spatial and temporal dimension in the backpropagation, which can lead to a better optimization of the SNN in terms of accuracy and latency. Moreover, spatio-temporal approaches are compatible with dynamically changing input data, such as neuromorphic data.

However, the computational and memory cost of training with BPTT is huge, as this method requires storing the activations and computing the gradient at all timesteps. This is a major obstacle to the direct training of SNNs. For instance, training VGG-16 on CIFAR-10 for one epoch of BPTT (using 100 timesteps) with the method of [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF] takes 78 min and 9.36 GB of GPU memory (using Nvidia GeForce RTX 2080 Ti TU102 GPU with 11GB memory). For comparison, the VGG-16 ANN training of one epoch requires only 0.57 min and 1.47 GB, which is x137 less time and x6 less memory.

In addition, SNN training is still challenging. Most of the works use an approximate derivative for the spiking activity, therefore there is an inconsistency between the model which is optimized with the backpropagation and the actual SNN model. These approximation errors could accumulate through the layers. Moreover, [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF] argued that, for SNNs trained with BPTT, reducing the number of timesteps is crucial, not only to reduce the latency and energy consumption, but also to improve the training convergence. Indeed, with the same network architecture, the unrolled SNN in the BPTT is much bigger compared to the ANN trained with backpropagation. Therefore, similar to RNNs, the vanishing and exploding gradients problem is increased. These problems prevent SNNs from scaling to very deep architectures. Hence, in addition to an increased offline training cost, the accuracy of SNNs trained with backpropagation is still limited and lags behind conversion approaches.

IV. OPTIMIZING FURTHER DEEP SNNS

Recent works have proposed additional strategies to improve the supervised training of SNNs with backpropagation, such as transferring ANNs techniques to SNNs, improving the encoding and decoding, using wider network architectures, hybridization between training strategies, and tuning the parameters specific to the SNN model. This methods have demonstrated significant improvements regarding the accuracy, latency and spike sparsity. The main results of backpropagation-based methods (spatial, spatio-temporal and single-spike methods) on static and neuromorphic vision datasets are summarized in Tables I and II, while showing the effect of using the improvements described in the following subsections.

A. Adapting successful ANN techniques to SNNs

ANNs training is more mature and already benefits from successful techniques developed in the past few years. Therefore, taking inspiration from these techniques could be a solution to improve SNNs accuracy. For instance, regularization is a powerful ANN technique to decrease the overfitting occurring during the training process and thus improving the generalization of the network. One regularization method especially designed for SNN is proposed in [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF]. This neuron normalization method (NeuNorm) consists in using auxiliary neurons at each convolutional layer. These auxialiary neurons compute a weighted summation of spike counts of the other neurons to be fed to the next layer in addition to the raw spike signals. This additional signal is used to balance the input current received by neurons and leads to an increased accuracy. However, this normalization step requires additional multiplications as both the spike counts and the associated weights are real-valued quantities.

In the ANN domain, dropout [START_REF] Srivastava | Dropout: A simple way to prevent neural networks from overfitting[END_REF] is a simple yet effective regularization technique transferable to SNNs [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF], [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF], [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF], [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF], [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF], [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF]. It consists in disconnecting some units of a layer with a given probability during the training to avoid the network relying too much on certain connections. For example, [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF] introduced dropout for SNN trained with BPTT by keeping the same random subset of dropped units at each timestep. Batch Normalization (BN) [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF] is a another powerful regularization technique widely used to train deep ANNs. It consists in rescaling the activations of a layer, and learning this scaling per batch, in order to maintain the variance of the activations throughout the network, which leads to better convergence. Several works propose to transfer the BN technique to the SNN training [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF], [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF], [START_REF] Kim | Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch[END_REF]. The authors in [START_REF] Kim | Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch[END_REF] show that standard BN should not be applied directly to SNNs, because it considers the timesteps all at once. Instead, they propose a BN "through time" to decouple the parameters of the BN across the timesteps. They show that their BPTT method could converge on CIFAR-100 and TinyImageNet datasets, which was not the case without the BN. Furthermore, they report a decreased number of spikes per inference by one order of magnitude compared to the BPTT without BN and an increased robustness to noisy inputs. They showed x9 efficiency gain compared to the ANN version in terms of AC/MAC operations using the energy values of [START_REF] Horowitz | Computing's energy problem (and what we can do about it)[END_REF]. In addition, [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF] propose a threshold-dependent spatio-temporal BN that normalizes the variance of the inputs to the threshold. By adding this BN to the BPTT algorithm described in [START_REF] Wu | Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks[END_REF] (further improved by the neuron normalization in [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF]), they demonstrate scalability to deep residual networks with high accuracy while using fewer timesteps. In addition, they report sparse spiking activity (less than 2 spikes per neuron per inference on average). Therefore, it seems that BN enables the convergence of deeper networks while potentially reducing the spiking activity.

Scalability of SNNs can also be improved using optimized network architectures. For instance, the ResNet architecture can alleviate the gradient vanishing problem by adding residual shortcut connections, which enables the effective training of deeper networks [START_REF] He | Deep residual learning for image recognition[END_REF]. Papers [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF], [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF], [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF], [START_REF] Fang | Deep Residual Learning in Spiking Neural Networks[END_REF] achieve high accuracy using ResNet architectures and regularization techniques. For instance, [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF] and [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF] show scalibility up to a 50-layer ResNet on CIFAR-100 and ImageNet, respectively. However, the accuracy gap with respect to ANN for the same architecture is still high. On ImageNet, the SNN ResNet-50 in [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF] yields 64.88% accuracy while the ANN ResNet-50 achieves 76.13% [START_REF]Pytorch torchvision models[END_REF], both being trained with BN. However, [START_REF] Fang | Deep Residual Learning in Spiking Neural Networks[END_REF] proposes a novel implementation of the identity mapping in spiking ResNet, which can mitigate the problem of SNNs scaling with depth. Indeed, they are able to scale to very deep residual networks while increasing the accuracy with depth. For instance, they reach 69.26% accuracy on ImageNet with ResNet-152 using only 4 timesteps.

B. Improving encoding and decoding

Input encoding and output decoding of the network seriously impact the SNN accuracy and latency [START_REF] Rueckauer | Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification[END_REF], [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF], [START_REF] Garg | DCT-SNN: Using DCT to distribute spatial information over time for learning lowlatency spiking neural networks[END_REF], [START_REF] Deng | Rethinking the performance comparison between SNNS and ANNS[END_REF]. Therefore, recent works proposed alternative coding methods.

1) Encoding: It is known that the higher the number of timesteps used in the inference phase, the higher the precision of the encoding and in turn, the higher the network accuracy. However, this induces an increased inference latency. Therefore, reducing the number of timesteps while preserving the accuracy is challenging. The authors in [START_REF] Garg | DCT-SNN: Using DCT to distribute spatial information over time for learning lowlatency spiking neural networks[END_REF] point out that Poisson spike generation (the most common encoding scheme used in both spatial and spatio-temporal approaches) is inefficient, because it relies only on spike rate to encode information and does not make use of the temporal dimension. They propose a Discrete Cosine Transform which encodes distinct information at each timesteps, by decomposing the input signal into a basis of spectral components. Therefore, the latency is defined as the number of vectors in the basis, starting from coarse to fine grain resolution to maximize the accuracy while reducing the latency. However, this encoding requires two matrix multiplications in the pre-processing step.

Another solution to improve the encoding, which can be applied to real-valued signals from non-spiking datasets (such as image pixels), consists in using an encoding layer, also called direct input encoding, to do the conversion from real values to spikes [START_REF] Rueckauer | Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification[END_REF], [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF], [START_REF] Zhang | Temporal spike sequence learning via backpropagation for deep spiking neural networks[END_REF], [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF], [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF], [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF], [START_REF] Deng | Rethinking the performance comparison between SNNS and ANNS[END_REF], [START_REF] Wu | A tandem learning rule for effective training and rapid inference of deep spiking neural networks[END_REF] (see Fig. 4b). This method consists in directly feeding the real values to the first layer at each timestep, without discretization of the input, the discretization process with the spikes being done in the first layer. Such encoding layer is thus a hybrid ANN-SNN layer, as the synapses perform real-valued inputweight multiplications but the neurons are spiking units. The authors in [START_REF] Deng | Rethinking the performance comparison between SNNS and ANNS[END_REF] report that by using the encoding layer, a small VGG-5 trained with BPTT on CIFAR-10 could achieve 74.23% accuracy instead of 63.19% with a standard probabilistic sampling. Moreover, it seems that, as it improves the accuracy, the number of timesteps can be reduced without losing in accuracy. Indeed, it appears that since the inputs are encoded with full precision in the first layer, a lower precision can be used in the subsequent layers while maintaining a high accuracy. They show that the VGG-5 with the encoding layer can achieve already the same accuracy than the one obtained without the encoding layer by using only 3 timesteps instead of 15 [START_REF] Deng | Rethinking the performance comparison between SNNS and ANNS[END_REF]. Besides, other spatio-temporal approaches [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF], [START_REF] Zhang | Temporal spike sequence learning via backpropagation for deep spiking neural networks[END_REF], [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF], [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF] demonstrate accuracy above 90% on CIFAR-10 with 5-12 timesteps, while usually the same approach with Poisson spike generation requires about a hundred timesteps for accurate inference [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF]. Similar conclusions are derived in the recent study [START_REF] Kim | Rate coding or direct coding: Which one is better for accurate, robust, and energy-efficient spiking neural networks?[END_REF] comparing the rate coding and the encoding layer strategies. Note that this hybrid layer must be supported in the neuromorphic hardware used for inference, or computed at the interface between the data and neuromorphic hardware.

2) Decoding: A simple way to increase the precision of the output layer is to apply the loss function on the high precision membrane potential of the output neurons instead of their spike rate [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF], [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF], [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF]. Alternatively, population decoding can be used to improve the robustness of the classification when using the output spike rates [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF], [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF], [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF]. In this decoding scheme, each group of output neurons represents a class, and the choice is made based on a voting strategy. The drawback is to use more neurons in the output layer and therefore more parameters. Besides, improving the softmax layer can also improve the accuracy by a few percents. For instance, [START_REF] Panda | Toward scalable, efficient, and accurate deep spiking neural networks with backward residual connections, stochastic softmax, and hybridization[END_REF] show about 2 to 3% accuracy improvement and a sparser spiking activity, by using a stochastic softmax instead of the standard softmax classifier. The stochastic aspect of this softmax can be considered as a regularization technique.

C. Wide network architectures

Using wider network architectures, by increasing the number of neurons per layer, improves the accuracy when the number of timesteps is constrained. For instance, the works [START_REF] Lee | Enabling spike-based backpropagation for training deep neural network architectures[END_REF], [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF], [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF], [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF], [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF] achieving low latency with high accuracy on CIFAR-10 use very large ResNet architectures (11 to 18M parameters), following the architecture designed in the ResNet paper [START_REF] He | Deep residual learning for image recognition[END_REF] for the ImageNet task instead of CIFAR-10. Indeed, ResNet architectures for CIFAR-10 are originally very small architectures, such as the ResNet-20 with only 0.27M parameters, and yet are able to achieve good accuracy on CIFAR-10 (91.25%). On ImageNet, [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF] report that by doubling the number of filters per convolutional layers in the ResNet-34 architecture, they increase the accuracy from 63.72% to 67.05%. Interestingly, when they compare the SNN with the ResNet-50 and ResNet-34 original architectures, the ResNet-50 yields better accuracy (64.88%), but not compared to the large ResNet-34. This shows that, in SNNs, from a certain depth, increasing the width of the network is more beneficial compared to further increasing the depth. Indeed, the number of neurons is increased, which can be considered equivalent to increasing the precision of a smaller number of neurons, without degrading the backpropagation phase. On the other hand, increasing the depth of the network makes the backpropagation more difficult due to accumulation of errors (for instance due to the surrogate gradient training) through the layers. This is on par with the work [START_REF] Mishra | WRPN: Wide reduced-precision networks[END_REF] proposing to use wider architectures to improve the accuracy of reducedprecision ANNs (with quantized weights and activations). Indeed, SNNs with few timesteps behave similarly to ANNs with strongly-quantized activations.

D. Training hybridization

Hybrid training approaches have also been proposed to further reduce the offline training and hardware inference costs while preserving the accuracy.

1) ANN-SNN network hybridization: Mixing ANN and SNN layers is one strategy to improve the accuracy (see Fig. 4a). For instance, the authors in [START_REF] Panda | Toward scalable, efficient, and accurate deep spiking neural networks with backward residual connections, stochastic softmax, and hybridization[END_REF] use a network with ANN layers at the inputs to improve the encoding accuracy, and SNN layers at the output, the whole network being trained with backpropagation (spatio-temporal for SNN layers). This approach demonstrates benefits for the CIFAR-10 classification task, for which the hybrid version yields 84.98% accuracy (+1.65% compared to the full ANN) using 25 timesteps with a VGG-9, showing x4 efficiency in terms of MAC/AC operations (using the energy values in [START_REF] Han | Learning both weights and connections for efficient neural networks[END_REF]) compared to the ANN. However, the benefits are smaller on the Imagenet dataset (x1.3 efficiency), as they could not use more than two spiking layers in the VGG-13 architecture to achieve satisfactory accuracy (-2% compared to the full the ANN).

2) Tandem learning: Another strategy proposed in [START_REF] Wu | A tandem learning rule for effective training and rapid inference of deep spiking neural networks[END_REF] is to couple each SNN layer with an ANN layer with weight sharing. In the training phase, the inference is performed by the SNN and the obtained spike counts are used as activation values by the ANN to perform the backpropagation. Therefore, the offline training phase is accelerated and needs less memory because the backpropagation is done on the ANN (thus removing the need for BPTT). The obtained SNN yields 90.98% accuracy on CIFAR-10 with 8 timesteps (7layer VGG architecture), and 50.22% on ImageNet with 10 timesteps (AlexNet architecture). In comparison, the ANN versions achieve 91.77% and 57.55%, respectively. In addition, they report a sparse spiking activity (less than 0.4 spike per neuron per inference on the CIFAR-10 task), and thus up to x20 energy-efficiency compared to the ANN in terms of MAC/AC operations (using the energy values of [START_REF] Han | Learning both weights and connections for efficient neural networks[END_REF]). Note that the temporal information can not be used in the backpropagation as the training is performed in the ANN domain.

3) Conversion and direct training hybridization: A hybrid approach between conversion and supervised direct training, thus mixing direct and indirect training, is proposed in [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF]. Indeed, the ANN-to-SNN conversion yields very good accuracy but at the cost of a high number of inference timesteps, while SNN supervised training yields a lower number of timesteps but the spatio-temporal training with BPTT is expensive. Taking the best of both worlds, they used ANN-to-SNN conversion as a pretraining and further applied SNN fine-tuning with a spatio-temporal learning rule. For instance with a VGG-16 architecture on CIFAR-10, after the ANN pretraining (250 training epochs), the SNN converged with 20 training epochs, showing the effectiveness of the pretraining. Therefore, the total training duration is reduced to 28 hours (using Nvidia GeForce RTX 2080 Ti TU102 GPU with 11GB memory) compared to 325 hours with SNN training from scratch. However, the memory requirements for training are not reduced, as the SNN still requires training with BPTT. In addition, the latency is decreased while yielding high accuracy. For instance, on ImageNet with 250 timesteps, a VGG-16 (resp. ResNet-34) with hybrid training achieves 65.19% (resp. 61.48%) accuracy, while with the same number of timesteps the converted SNN without the fine tuning only achieves 62.73% (resp. 56.87%).

E. Leveraging the specificity of the SNN model

The methods to improve the supervised training proposed in the previous sections are mostly inspired by the ANN training. However, the SNN model have some specificity that do not exist in the ANN version, such as the threshold parameter, or time constants for LIF neurons and continuous synapses. This section shows that, by making efficient use of the rich dynamics of the SNN model, further benefits can be expected in terms of accuracy and efficiency (latency and sparsity).

1) Neuron's leak and threshold: Typical parameters of spiking neurons such as leak (for LIF neurons) and threshold are usually defined as hyperparameters and not considered in the training. However, neuron's leak and threshold are important parameters determining the SNN behavior. For a given set of weights, the threshold determines how much the input neurons must spike in order for the neuron to spike. The leak parameter controls how close to each other input spikes must be for a temporal coincidence to be detected. Thus, tuning leak and threshold parameters with backpropagation can allow a better optimization of the SNN model. For instance, the authors in [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF] propose to learn the leak parameters and reported higher accuracy than other spiking approaches on neuromorphic datasets CIFAR-10-DVS and DVSGesture (see Table II). In [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF], both leak and threshold are learned with the hybrid conversion pretraining and supervised finetuning method of [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF]. They show an accuracy improvement compared to learning only the synaptic weights of about 1% on CIFAR-10 and up to 5% on CIFAR-100 and ImageNet, as well as a lower spike rate (average number of spikes per neuron per inference). The effect of tuning the thresholds when the weights are learned is surprising, as adjusting either the weights or the thresholds (from pre-trained weights) is usually considered equivalent in conversion methods. However, they demonstrate that, with iso-accuracy (using VGG-16 on CIFAR-10), tuning the thresholds leads to a reduction in timesteps from 25 to 15 and in spike rate from 1.94 to 1.47. Tuning the leaks reduces even more the number of timesteps from 15 to 5 and the spike sparsity from 1.47 to 0.39. The threshold and leak parameters are shared between neurons in a layer, thus the number of added parameters is negligible compared to the total number of parameters in the model.

2) Synapse dynamics: The choice of the synaptic kernel function, when continuous synapses (rather than instantaneous) are used, is another parameter to explore in order to optimize the SNN model. For instance, in [START_REF] Fang | Exploiting neuron and synapse filter dynamics in spatial temporal learning of deep spiking neural network[END_REF], the authors propose a SNN with LIF neurons where the synapses are described as second order infinite impulse response filters, allowing to model various types of kernel (e.g. instantaneous, exponential, alpha, dual-exponential). The coefficients of the synapse filter are jointly learned with the synaptic weights using BPTT. They achieved 96.09% accuracy on DVSGesture.

3) Surrogate gradient: Finally, the spatial and spatiotemporal approaches use a surrogate gradient to approximate the derivative of the spiking activity. The derivative of sigmoid functions is often used as a surrogate, as the sigmoid function can be seen as a smooth approximation of the step function (see Fig. 3), but other surrogate derivatives, such as exponential or piece-wise linear functions, can also be used [START_REF] Neftci | Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks[END_REF]. While the training performance is robust to the shape of the surrogate function, it is strongly affected by its scale [START_REF] Zenke | The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks[END_REF]. For instance, paper [START_REF] Ledinauskas | Training deep spiking neural networks[END_REF] shows that, by tuning the scale of the surrogate gradient function, the variance of the gradients can be preserved through the layers, avoiding exploding and vanishing gradients. Therefore, by optimizing the width of the surrogate gradient function (and additionally using BN), they can scale to a ResNet-50 architecture, achieving 58.5% accuracy on CIFAR-100 and 81.2% accuracy on Imagenette (a subset of ImageNet), with 40 and 20 timesteps for evaluation, respectively (10 timesteps are used for training).

V. IMPACT OF INPUT ENCODING, TRAINING AND NETWORK ARCHITECTURE WIDTH ON THE ACCURACY-LATENCY TRADE-OFF

Finally, as SNNs distribute the information through binary events over time, there is an inherent trade-off between accuracy and latency (and thus the energy cost). Indeed, the latency directly impacts the accuracy, as the precision of the coding depends on the number of timesteps used for inference [START_REF] Diehl | Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing[END_REF], [START_REF] Sengupta | Going deeper in spiking neural networks: VGG and residual architectures[END_REF], [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF]. In order for SNNs to replace ANNs for efficient inference on neuromorphic hardware, the accuracy-latency trade-off should be carefully considered. In particular, parameters such as the coding, training strategies, and network architecture width, impact this trade-off. State-ofthe-art approaches in conversion, direct training with BPTT, and hybrid training, are compared in terms of accuracy-latency trade-off with regards to these parameters in Table III.

First, the training strategy appears to have an effect on the accuracy-latency trade-off. For instance, the conversion with temporal switch coding proposed in [START_REF] Han | Deep spiking neural network: Energy efficiency through time based coding[END_REF] yields better results than the conversion with rate coding of [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF] when a reduced number of timesteps is used (256). This is explained by the use of a better threshold balancing. Moreover, fine-tuning with BPTT after the rate-coded conversion in [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF] improves the accuracy by 1% on CIFAR-10 and by 3% (resp. 5%) on ImageNet when using a VGG (resp. ResNet) architectures with the same number of timesteps (250). Adding further the finetuning of the threshold and leak improves also the accuracy by 1% (resp. 5%) on CIFAR-10 with ResNet architecture (resp. ImageNet with VGG architecture) when using the same number of timesteps (5) [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF].

Considering the network architecture, as mentioned in section IV-C, increasing the width can improve the accuracylatency trade-off. For instance, we observe that the gap between the SNN and ANN accuracy decreases as much as the architecture width increases. This explains the bigger differences between SNN and ANN accuracy on ImageNet with ResNet-34 architecture (21M parameters) than with VGG-16 (138M parameters). For instance, 69.00% accuracy is achieved with 5 timesteps using VGG-16 in [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF], which is close to the ANN accuracy (71.59% without BN). The best accuracy among SNNs for a 34-layer ResNet is 67.05% with the spatiotemporal approach (trained with BN) of [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF] by using a wide ResNet (≈85M parameters). The problem is mitigated by [START_REF] Fang | Deep Residual Learning in Spiking Neural Networks[END_REF] with a modified implementation of the spiking ResNet achieving 67.04% accuracy with 4 timesteps using the original ResNet-34 architecture. However, both SNNs are still far from the ANN ResNet-34 accuracy (73.31% with BN). Similarly in conversion approaches [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF], [START_REF] Han | Deep spiking neural network: Energy efficiency through time based coding[END_REF], using the ResNet-34 architecture on ImageNet with a reduced number of timesteps (256 vs 4096), the degradation in accuracy is larger than the one observed for VGG-16. This highlights the accuracy-size trade-off in SNNs: the loss in accuracy due to the quantization of information (which is further increased by reducing the number of inference timesteps) is compensated by increasing the number of neurons in each layer of the network.

In addition, the encoding layer impacts a lot the accuracylatency trade-off for both hybrid conversion and direct training approaches. Indeed, when compared to the same architecture (ResNet-20 large or VGG-16) with the same training methodology (i.e. conversion with fine-tuning), the encoding layer can allow to reduce the number of timesteps from 250 to 5 with only 1 to 2% accuracy loss [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF]. In addition, direct training approaches yielding high accuracy with very few timesteps (5 to 12), such as [START_REF] Wu | Direct training for spiking neural networks: Faster, larger, better[END_REF], [START_REF] Zhang | Temporal spike sequence learning via backpropagation for deep spiking neural networks[END_REF], [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF], [START_REF] Zheng | Going deeper with directly-trained larger spiking neural networks[END_REF], use the encoding layer. The paper [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF] studies the effect of the input encoding and leak and threshold optimization on both the spike rate and latency for the hybrid conversion with fine-tuning approach (with VGG-16 on CIFAR-10). Using an encoding layer instead of probabilistic sampling induces a significant improvement in the latency (from 150 to 25 timesteps)and in the spike rate (from 26 to 1.94).

Hence, it seems that the training strategy, the network architecture width and the use of an encoding layer impact the accuracy-latency trade-off. In particular, the use of wide architectures and encoding layer seems to be the key to compensate for the quantization process inherent to the spike coding with a limited number of timesteps and thus to reach the best accuracy-latency trade-off. However, they bring additional costs, such as using MACs operations for the encoding layer, and increasing the area and memory requirements for large architectures compared to smaller one.

VI. CONCLUSION

Compared to ANNs, the SNN model is more complex and requires to choose the neuron, synapse and coding strategies. These choices are intertwined and impact the learning rule and the system efficiency. Related to SNN training, the conversion and direct training backpropagation-based approaches have the potential to replace ANNs for accurate and energy-efficient hardware inference on typical deep learning tasks such as image classification. From the conversion approaches, the temporal switch coding [START_REF] Han | Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[END_REF] is a promising strategy to increase the spike sparsity. Moreover, it is important to use ANN model which are closest to SNN models to minimize the conversion errors [START_REF] Bu | Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks[END_REF]. From the direct training methods, the spatio-temporal approaches have demonstrated the best results. Only two algorithms [START_REF] Thiele | SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes[END_REF], [START_REF] Zhou | Temporal-coded deep spiking neural network with easy training and robust performance[END_REF] from the spatial and singleapproaches were shown to scale, but they lack information related to the number of timesteps required for inference. Thus, further work is needed to evaluate the performance of those approaches compared to the spatio-temporal ones.

Conversion methods suffer from high latency while direct training spatio-temporal approaches show an increased offline training cost and a degraded accuracy. Thus, to reach ANN capabilities, further methods to improve SNN training and inference are necessary, such as applying regularization techniques during training and tuning parameters specific to SNNs.

Moreover, feeding directly real values at the inputs with an encoding layer instead of performing spike conversion improves both SNN accuracy and latency. In the same spirit, the hybrid ANN-SNN approach mixing ANN and SNN layers also has a potential to improve the accuracy-latency trade-off and should be further explored. The hybrid training approaches such as conversion pre-training and supervised fine-tuning in [START_REF] Rathi | Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation[END_REF], [START_REF] Rathi | Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[END_REF] or tandem ANN-SNN learning [START_REF] Wu | A tandem learning rule for effective training and rapid inference of deep spiking neural networks[END_REF], can improve the accuracy, latency and spike sparsity. In addition, they reduce the cost of the offline training due to the use of ANN training, which was a hurdle to direct training approaches. Furthermore, wide network architectures allow reducing the accuracy loss when the number of timesteps is limited. This shows the importance of finding network architectures suitable for SNNs, which can be done using Neural Architecture Search [START_REF] Kim | Neural Architecture Search for Spiking Neural Networks[END_REF]. Finally, the accuracy-latency trade-off in SNN should be carefully considered. For instance, the use of an encoding layer and wide architectures can significantly improve this trade-off.

In the interests of comparing the different training algorithms, we have focused on image classifications tasks. However, SNNs may offer greater benefits in other applications, but these remain to be fully explored [START_REF] Roy | Towards spike-based machine intelligence with neuromorphic computing[END_REF]. For instance, some works have studied SNNs for speech processing [START_REF] Bellec | Long short-term memory and learning-to-learn in networks of spiking neurons[END_REF], [START_REF] Wu | Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition[END_REF], biomedical applications [START_REF] Kasabov | NeuCube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data[END_REF]- [START_REF] Kumarasinghe | Brain-inspired spiking neural networks for decoding and understanding muscle activity and kinematics from electroencephalography signals during hand movements[END_REF] or in the context of privacy and security scenarios [START_REF] Sharmin | Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-linear Activations[END_REF]- [START_REF] Kim | PrivateSNN: Privacy-Preserving Spiking Neural Networks[END_REF] including federated learning [START_REF] Skatchkovsky | Federated neuromorphic learning of spiking neural networks for low-power edge intelligence[END_REF], [START_REF] Venkatesha | Federated learning with spiking neural networks[END_REF]. These works show a broad range of applications where SNNs could bring both high accuracy and energy-efficient solutions.

The sparsity of activations and event-based processing result in efficient SNNs in neuromorphic hardware. However, note that, although the sparsity and the number of timesteps provide initial elements of comparison at the algorithmic level, they are not sufficient to accurately evaluate the efficiency of an algorithm implemented in hardware. For instance, the energy consumption and the latency of the SNN hardware inference also depends on the complexity of the neuron model and the network architecture. Moreover, the characteristics of SNNs may be particularly suited to the constraints of memristorbased neural networks accelerators. Indeed, activation sparsity and information coding through time could mitigate the current limitations of this emerging hardware, such as heat dissipation, device variability or limited variable precision [START_REF] Eshraghian | Memristor-based binarized spiking neural networks: Challenges and applications[END_REF]. Hence, we emphasize the importance of a hardware-software co-design strategy in order to enable highly energy-efficient solutions.

Fig. 1 .

 1 Fig. 1. (a) Feedforward fully-connected neural network. (b) ANN and SNN neuron and synapse models. (c) Input encoding: example of pixel-to-spike conversion with a rate coding or temporal (latency) coding.

Fig. 2 .

 2 Fig.2. (a) Backpropagation-based learning algorithms. Spatial and spatiotemporal approaches use a rate coding while the single-spike approach use a temporal (latency) coding. The spatio-temporal approach considers the activation of each neuron at each timestep a t i , corresponding to the emission or not of a spike s t i . The backpropagation through time is used to backpropagate the error in both space and time dimensions. On the other hand, the spatial and single-spike approaches consider for each neuron a single activation a i for the forward pass, which can correspond to the spike count c i for the former or the timing of the unique spike emitted by the neuron t i for the latter. Therefore, the backpropagation is used to backpropagate the error only in the space dimension. (b) Backpropagation and backprogation through time training. Notations: n number of layers, T number of timesteps used in the SNN inference, A (l) activation of neurons in layer l, W (l) weight vector from layer l to l + 1, τ membrane potential and postsynaptic potential update (for LIF neurons and continuous synapses).

Fig. 4 .

 4 Fig. 4. (a) Example of ANN-SNN hybridization. Here, the first convolutions are done in ANN mode (MACs with real-valued inputs) and the last convolutions are performed in SNN mode (ACs with binary spikes). A conversion from analog values to spikes is performed between ANN and SNN layers. (b) Encoding layer. The first layer is hybrid ANN-SNN, as synapses perform MAC operations between weights and real-valued inputs, but neurons are spiking. Therefore, the conversion from analog values to spikes is performed directly by this layer.

Manon

  Dampfhoffer received the M.Sc. degree in Informatics and Applied Mathematics from the Grenoble Institute of Technology, France, in 2019. She is currently working toward the Ph.D. degree at Univ. Grenoble Alpes, France, in the Systemson-Chip and Advanced Technologies (LSTA) laboratory at CEA LIST and Spintec laboratory. Her current research focuses on models and algorithms for implementing energy-efficient Spiking Neural Networks on neuromorphic hardware at the edge. Thomas Mesquida joined CEA in 2019, after a PhD in microelectronics. His past research included information encoding within Spiking Neural Network, with a focus on inter-spike interval, its hardware implementations and applications. He is currently pursuing the development lightweight spiking and hybrid neural network implementations for embedded applications, combining memory technology, information encoding and learning method for quantized networks. Alexandre Valentian joined CEA LETI in 2005, after an MSc and a PhD in microelectronics. His past research activities included design technology cooptimization, promoting the FDSOI technology (notably through his participation in the SOI Academy), 2.5D/3D integration technologies and non-volatile memory technology. He is currently pursuing the development of bio-inspired circuits for AI, combining memory technology, information encoding and dedicated learning methods. Since 2020, he heads the Systems-on-Chip and Advanced Technologies (LSTA) laboratory at CEA LIST. Dr Valentian has authored or co-authored 80 conference and journal papers. Lorena Anghel received the Ph.D. degree (cum laude) from Grenoble INP in 2000. From 2016 to 2020, she was the Vice President of Grenoble INP, in charge of industrial relationships, where she is currently the Scientific Director. She is also a Full Professor with Grenoble INP and a member of the Research Staff of the Spintec Laboratory. She has published more than 130 publications in international conferences and symposia. She has supervised 24 Ph.D. students. Her research interests include hardware design and test of neural networks, on-line testing, fault tolerance, and reliable design and verification. She was a recipient of several best paper and outstanding paper awards. She had fulfilled positions, such as the General Chair and the Program Chair for many prestigious IEEE conferences including the IEEE VTS, the IEEE ETS, and the IEEE On-Line Test Symposium.

TABLE III IMPACT

 III OF INPUT ENCODING, TRAINING AND NETWORK ARCHITECTURE WIDTH ON THE ACCURACY-LATENCY TRADE-OFF

	Paper	Architecture	Encoding layer	Training	Timesteps Acc. (%)
	CIFAR-10
	[38]	ResNet-20	✕	conversion (rate)	2048	91.36
	[38]	ResNet-20	✕	conversion (rate)	256	89.37
	[39]	ResNet-20	✕	conversion (time)	2048	91.42
	[39]	ResNet-20	✕	conversion (time)	256	90.10
	[40]	ResNet-20	✓	conversion (rate)	16	91.62
	[66]	ResNet-20 (L)	✕	conversion (rate)	250	91.12
	[66]	ResNet-20 (L)	✕	conversion (rate) + backpropagation	250	92.22
	[70]	ResNet-20 (L)	✓	conversion (rate) + backpropagation	5	90.29
	[70]	ResNet-20 (L)	✓	conversion (rate) + backpropagation (+ leak & threshold tuning)	5	91.78
	[57]	ResNet-19 (L)	✓	backpropagation	6	93.16
	[53]	ResNet-11 (L)	✕	backpropagation	100	90.95
	[58]	ResNet-11 (L)	✕	backpropagation (+ batch normalization + surrogate gradient tuning)	20	90.20
	[72]	ResNet-20	/	ANN (+ batch normalization)	/	91.25
	[70]	ResNet-20 (L)	/	ANN	/	92.79
	ImageNet
	[38]	ResNet-34	✕	conversion (rate)	4096	69.89
	[38]	ResNet-34	✕	conversion (rate)	256	≈20
	[39]	ResNet-34	✕	conversion (time)	4096	69.93
	[39]	ResNet-34	✕	conversion (time)	256	55.65
	[40]	ResNet-34	✓	conversion (rate)	64	72.35
	[66]	ResNet-34 (M)	✕	conversion (rate)	250	56.87
	[66]	ResNet-34 (M)	✕	conversion (rate) + backpropagation	250	61.48
	[57]	ResNet-34	✓	backpropagation (+ batch normalization)	6	63.72
	[57]	ResNet-34 (L)	✓	backpropagation (+ batch normalization)	6	67.05
	[68]	ResNet-34	✓	backpropagation (+ batch normalization)	4	67.04
	[73]	ResNet-34	/	ANN (+ batch normalization)	/	73.31
	[38]	VGG-16	✕	conversion (rate)	4096	73.09
	[38]	VGG-16	✕	conversion (rate)	256	48.32
	[39]	VGG-16	✕	conversion (time)	2560	73.46
	[39]	VGG-16	✕	conversion (time)	256	69.71
	[40]	VGG-16	✓	conversion (rate)	64	72.85
	[66]	VGG-16	✕	conversion (rate)	250	62.73
	[66]	VGG-16	✕	conversion (rate) + backpropagation	250	65.19
	[70]	VGG-16	✓	conversion (rate) + backpropagation	5	64.32
	[70]	VGG-16	✓	conversion (rate) + backpropagation (+ leak & threshold tuning)	5	69.00
	[73]	VGG-16	/	ANN (+ batch normalization)	/	73.36
	Number of parameters of the architectures (estimated according to the details given in the associated papers): ResNet-20: 0.27M. ResNet-20
	(L): 11M. ResNet-19 (L): 13M. ResNet-11 (L): 18M. ResNet-34: 21M. ResNet-34 (M): 22M. ResNet-34 (L): 85M. VGG-16: 138M.