Fabio Pavanello

Ioana Vatajelu

Alberto Bosio

Thomas Van Vaerenbergh

Peter Bienstman

Benoit Charbonnier

Alessio Carpegna

Stefano Di Carlo

Alessandro Savino

Elena Ioana Vatajelu

Special Session: Neuromorphic hardware design and reliability from traditional CMOS to emerging technologies

Keywords: augmented silicon photonics, neuromorphic hardware, artificial neural networks, spiking neural networks, reliability, phase change materials

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

I. INTRODUCTION

Artificial Neural Networks (ANNs) have enabled complex computations but require significant computational resources for both training and inference [START_REF] Marchisio | Deep learning for edge computing: Current trends, cross-layer optimizations, and open research challenges[END_REF]. The main bottleneck in these networks is the transfer of large amounts of data to support different tasks. However, the trend in computing is moving towards edge devices, such as the Internet of Things (IoT), to improve security and reduce power consumption and latency [START_REF] Shafique | An overview of next-generation architectures for machine learning: Roadmap, opportunities and challenges in the iot era[END_REF]. Furthermore, there is a growing demand for powerful yet energy-efficient accelerators in various fields, including fault detection in microprocessors [START_REF] Dutto | Exploring deep learning for in-field fault detection in microprocessors[END_REF] and intrusion detection systems [START_REF] Sedjelmaci | A lightweight anomaly detection technique for low-resource iot devices: A game-theoretic methodology[END_REF].

The design space for ANNs is vast, and it involves choices in four fundamental aspects: the neuron model, the architecture This project has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No. 101070238. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. It was also partially supported by the ANR within the EMINENT Project ANR-19-CE24-0001 structure, the information encoding, and the training method [START_REF] Zhou | A brief survey on deep neural networks[END_REF]. These choices can significantly impact the hardware design and optimization process. Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), and Spiking Neural Networks (SNNs) are three popular types of ANNs, each with unique characteristics and applications.

CNNs are commonly used in image recognition and processing tasks, and they rely on convolutional layers to extract spatial features from input images [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. DNNs, on the other hand, are used in a wide range of applications, from speech recognition to natural language processing and game playing [START_REF] Goodfellow | Deep learning[END_REF]. They are characterized by multiple layers of neurons that learn increasingly abstract features of the input data.

The limitations of traditional computing architectures, particularly the communication bottleneck between memory and processor and the latency of information propagation and manipulation, have highlighted the need for alternative approaches to ANNs. While software approaches for ANNs offer advantages when implemented on specialized hardware such as Graphical Processing Units (GPUs), these limitations persist [START_REF] Schuman | Opportunities for neuromorphic computing algorithms and applications[END_REF].

SNNs have emerged as the next generation of ANNs that exchange information in spikes, inspired by the behavior of biological brains [START_REF] Maass | Networks of spiking neurons: The third generation of neural network models[END_REF]. This allows for more efficient computation and reduced power consumption and is particularly interesting when working with time sequences such as audio, video, and electrical signals [START_REF] Cramer | The heidelberg spiking data sets for the systematic evaluation of spiking neural networks[END_REF], [START_REF] Amirshahi | Ecg classification algorithm based on stdp and r-stdp neural networks for real-time monitoring on ultra lowpower personal wearable devices[END_REF]. The model complexity and internal parameters determine the model's suitability to the input data, with shorter time constants detecting shorter temporal correlations and higher values catching more prolonged time effects.

Eventually, the resilience of the ANN is crucial when designing the entire system and cannot be ignored [START_REF] Vallero | Syra: Early system reliability analysis for cross-layer soft errors resilience in memory arrays of microprocessor systems[END_REF], [START_REF] Ruospo | A survey on deep learning resilience assessment methodologies[END_REF]. Retaking inspiration from biology, the human brain is intrinsically resilient to malfunctioning and faults. It loses approximately 50000 neurons daily but can still perform complex tasks and learn new ones, creating connections between the remaining neurons. ANNs have inherited this characteristic at 979-8-3503-4630-5/23/$31.00 ©2023 IEEE a certain level, but they still need to improve significantly, particularly in mission-critical and safety-critical applications. Therefore, a deeper study of ANNs from this perspective is required.

Neuromorphic computing, which merges memory and processing units within neurons and synapses and maps computing architectures more closely to Neural Networks (NNs) models, offers a promising solution to address the limitations of traditional computing architectures [START_REF] James | A historical survey of algorithms and hardware architectures for neural-inspired and neuromorphic computing applications[END_REF]. Various technologies, including CMOS, memristors, and optoelectronics/alloptical approaches, have been explored to develop neuromorphic hardware. This paper provides an overview of these approaches, highlighting their advantages and limitations. Background on ANNs is given in section II, while section III discusses the design and performance of the digital version of ANNs, with a focus on the SNNs. section IV discusses the potential of silicon photonics for ANNs, and section V covers reliability challenges in the usage of ANNs.

II. BACKGROUND

The choice of the neuron model, architecture structure, information encoding, and training method can significantly impact the hardware design and optimization process for each of these ANNs. Therefore, it is essential to consider these aspects carefully when designing and implementing hardware for ANNs.

CNNs and DNNs use different neuron models. In CNNs, the neuron model is typically based on the Rectified Linear Unit (ReLU) function, a non-linear activation function commonly used in deep learning. The ReLU function is simple and computationally efficient, making it a popular choice for CNNs. The ReLU function is f (x) = max(0, x), where x is the input to the neuron. The output of the ReLU function is zero if the input is negative and equal to the input if the input is positive. The ReLU function effectively reduces overfitting in DNNs and has been used in various computer vision tasks, such as object detection and recognition [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF].

In DNNs, the neuron model is typically based on a nonlinear activation function. The sigmoid function, defined as f (x) = 1 1+e -x , where x is the input to the neuron, is one of the simplest. The output of the sigmoid function is between 0 and 1, which makes it useful for tasks such as classification. Another popular activation function in DNNs is the hyperbolic tangent (tanh) function, similar to the sigmoid function but outputs values between -1 and 1. The choice of activation function depends on the task at hand and the NNs's architecture. For example, the sigmoid and tanh functions were commonly used in early DNN architectures, such as the Multilayer Perceptron (MLP) [START_REF] Rumelhart | Learning representations by back-propagating errors[END_REF] but have since been largely replaced by the ReLU function in more recent architectures. However, the sigmoid and tanh functions are still used in certain NNs, such as Recurrent Neural Networks (RNNs) and autoencoders [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF].

State-of-the-art implementations of CNNs and DNNs often use 32-bit floating-point numbers in software or model-based approaches. However, implementing such algorithms in hardware is challenging due to their extensive data requirements, high energy consumption, and large memory bandwidth. To address these challenges, quantization, and regularization techniques have been explored, using fixed-point computations with 16 bits, 8 bits, or lower precision. Although these methods reduce the precision of synaptic weights and inter-layer signals, IBM's TrueNorth [START_REF] Akopyan | Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip[END_REF] chip has achieved acceptable precision using only five synaptic states, albeit at high design and energy costs [START_REF] Draghici | On the capabilities of neural networks using limited precision weights[END_REF], [START_REF] Merolla | A million spiking-neuron integrated circuit with a scalable communication network and interface[END_REF].

In the case of SNNs, the neuron models and architectures are the most complex to target due to the nature of the information they carry and how they treat it. Many different mathematical models describe and mimic the behavior of biological neurons, such as the Hodgkin-Huxley model [START_REF] Hodgkin | A quantitative description of membrane current and its application to conduction and excitation in nerve[END_REF], the Izhikevich model [START_REF] Izhikevich | Simple model of spiking neurons[END_REF], the Leaky Integrate and Fire (LIF) model [START_REF] Teeter | Generalized leaky integrate-and-fire models classify multiple neuron types[END_REF], and the Integrate and Fire (IF) model [START_REF] Burkitt | A review of the integrate-and-fire neuron model: Ii. inhomogeneous synaptic input and network properties[END_REF]. These models range from very complex and detailed to much simpler and more suitable for machine learning and hardware applications, with varying degrees of biological plausibility and computational efficiency.

Neurons in SNNs are generally treated as leaky integrators where input spikes are integrated over time after being weighted by corresponding synapses, affecting the neuron's state, usually the electrical potential across its membrane. The neuron membrane depolarizes due to internal charge leakage without spikes, except in the IF model, where it is kept at a constant value. A new spike is generated when the membrane potential exceeds a specific threshold value, causing the neuron to fire, and the potential drops suddenly into a reset state.

The architecture of the SNN can also be very flexible. The literature reports Fully-Connected (FC) Feed-Forward (FF) SNN [START_REF] Carpegna | Spiker: an fpga-optimized hardware accelerator for spiking neural networks[END_REF], regularly recurrent structures [START_REF] Cramer | The heidelberg spiking data sets for the systematic evaluation of spiking neural networks[END_REF] or randomly recurrent architectures [START_REF] Corradi | A neuromorphic event-based neural recording system for smart brain-machine-interfaces[END_REF], used for example, in Reservoir Computing (RC). There can be only excitatory connections, with positive weights, or adding inhibitory connections, with negative weights [START_REF] Carpegna | Spiker: an fpga-optimized hardware accelerator for spiking neural networks[END_REF], or in more detailed models, separated excitatory and inhibitory neurons, as observed in some regions of the human brain.

RC is an efficient technique where a randomly initialized RNN trains only a linear combination of the signals at each node. It has shown promising results in various applications, including photonics [START_REF] Der Sande | Advances in photonic reservoir computing[END_REF]. Different RC architectures have been investigated using photonics, such as spatial-multiplexing and time-multiplexing approaches. Spatial-multiplexing involves physically separating the nodes, while time multiplexing requires a faster sampling speed and more complex processing of the read-out layer.

The choice of input encoding can significantly affect the performance of the CNN or DNN. It often requires careful consideration and experimentation to determine the optimal encoding for the given task. Factors to consider when selecting an input encoding include the nature and complexity of the input data, the available computational resources, and the application's performance requirements.

The most common input encoding for CNNs is raw pixel values, where each pixel in the image is represented as a numerical value. In contrast, for DNNs that process nonimage data, input encoding may involve feature engineering techniques such as transforming the raw input data into a set of meaningful features more amenable to learning by the network [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. For example, in natural language processing, input encoding may involve converting text into a numerical representation, such as bag-of-words or word embeddings. Due to their nature, SNNs require more advanced methods for encoding and interpreting information. The main approaches are rate coding [START_REF] Enoka | Rate coding and the control of muscle force[END_REF], temporal coding [START_REF] Petro | Selection and optimization of temporal spike encoding methods for spiking neural networks[END_REF], and population rank coding [START_REF] Pan | Neural population coding for effective temporal classification[END_REF]. Rate coding uses the average spike frequency to encode information and is suitable for static input data. It is less efficient regarding spike activity but more robust to noise. Temporal coding encodes information in the precise arrival time of spikes or their relative distance, requiring fewer spikes to process information, and is suitable for encoding time-varying signals. However, it is more sensitive to noise. Population rank coding uses the joint activity of a group of neurons to process information.

Training approaches involve optimizing the model parameters to minimize a given loss function for CNNs and DNNs. In supervised learning, this involves iteratively adjusting the weights and biases of the network to reduce the difference between the predicted output and the ground truth labels. The optimization is typically performed using gradient descent methods, which involve calculating the gradient of the loss function concerning the model parameters and updating them in the direction of the negative gradient. Commonly used gradient descent methods include Stochastic Gradient Descent (SGD) [START_REF] Bottou | Large-scale machine learning with stochastic gradient descent[END_REF], AdaGrad [START_REF] Duchi | Adaptive subgradient methods for online learning and stochastic optimization[END_REF], etc.

Regularization techniques such as Dropout [START_REF] Srivastava | Dropout: A simple way to prevent neural networks from overfitting[END_REF], etc., are often used to prevent overfitting and improve generalization. Additionally, data augmentation methods such as flipping, rotating, and cropping the input images are employed to increase the size of the training dataset and improve model robustness [START_REF] Takahashi | Ricap: Random image cropping and patching data augmentation for deep cnns[END_REF].

Unsupervised learning approaches, such as Autoencoders [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF] and Restricted Boltzmann Machines [START_REF] Fischer | An introduction to restricted boltzmann machines[END_REF], can also be used for pretraining the model parameters. Transfer learning approaches can also be employed, where a pre-trained network is fine-tuned on a new dataset with similar or related features [START_REF] Pan | A survey on transfer learning[END_REF]. Finally, reinforcement learning can also be used to train CNNs and DNNs, where the model learns to take actions based on a reward signal, such as in game-playing agents [START_REF] Mnih | Playing atari with deep reinforcement learning[END_REF].

Training SNNs is challenging due to the nondifferentiability of the thresholding function of neurons. Classical back-propagation methods cannot be directly applied, but several approaches have been developed, including supervised and unsupervised training. In supervised training, the most common approach is to convert an ANN into an SNN, where the ANN's differentiable non-linear function is trained using back-propagation, and the weights are used directly in the SNN. Alternatively, a Back-Propagation Through Time (BPTT) can be applied directly to the SNN, where a surrogate gradient replaces the neuron's thresholding function with a differentiable function during the backward pass. In contrast, inspired by biology, most unsupervised approaches update weights locally based on the relative spike timing between the inputs and the output without depending on a global error signal propagating across the network, resulting in a lighter memory footprint and computational overhead, with Spike Timing Dependent Plasticity (STDP) being the most common method [START_REF] Bi | Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type[END_REF].

III. DIGITAL ACCELERATORS

As seen before, the architecture of an ANN, and in the same way of an SNN, is generally composed of many independent neurons and, as such, is intrinsically strongly parallelizable. This poorly fits the common CPU-based computing approach, in which the parallelism is limited to a few tens of very powerful cores. For this reason, one active research branch in the field of ANN and SNN is directed towards accelerating such algorithms, using computing platforms to execute them more efficiently. The goal is to broaden their application to many contexts, such as performance-constrained, powerconstrained, or real-time tasks. When discussing acceleration in the digital domain, several solutions have different degrees of optimization, efficiency, and cost. One first approach, already mainstream for many ANN models, is to exploit the computational parallelism offered by general-purpose hardware accelerators like GPUs. Regarding SNNs, several software frameworks natively support the deployment of the code on GPUs, for example, the ones based on pyTorch, like snnTorch [START_REF] Eshraghian | Training spiking neural networks using lessons from deep learning[END_REF] and spikeTorch, or CUDA accelerated C++ frameworks, like SLAYER [START_REF] Shrestha | SLAYER: Spike layer error reassignment in time[END_REF] and CARLsim 4 [START_REF] Chou | Carlsim 4: An open source library for large scale, biologically detailed spiking neural network simulation using heterogeneous clusters[END_REF].

However, SNNs have many features unsuitable with a GPU execution. For example, spikes can be represented in the digital domain as single-bit events (high in the presence of a spike and low otherwise). The numerical representation used in GPU, based on words with 8, 16, 32, 64, or similar bit widths, is inefficient. Moreover, the computation can be based on events: a neuron can react only in the presence of an active input spike, remaining in a quiescent state otherwise. Such an update policy would allow exploiting the sparsity of spikes typical of SNNs, with considerable savings in switching power, but again is not supported by GPUs.

This incompatibility has pushed for developing specialized hardware accelerators explicitly designed to support SNN features. Interestingly SNNs are intrinsically more suitable for developing these accelerators than other ANN models. The main reason is again how SNN encodes the information, which in the digital domain corresponds to single-bit signals. This drastically reduces the interconnection and memory requirements. When designing hardware accelerators, the roads that can be followed are two: fixed hardware or reconfigurable hardware. In the first case, the accelerator becomes an Application Specific Integrated Circuit (ASIC), able to perform in a very efficient way the specific tasks for which it has been designed, but nothing more. On the other hand, specific hardware platforms can be reconfigured multiple times, allowing for flexible architectures. This is the case of Field Programmable Gate Arrays (FPGA). The choice between the two depends a lot on the kind of application and constraints. Generally, designers try to fit as many functionalities as possible when developing an ASIC since the hardware remains fixed.

Companies and universities are investing in developing programmable chips to enable fast simulation of large-scale SNN. Among them, a first attempt is SpiNNaker [START_REF] Furber | The spinnaker project[END_REF]. The idea behind SpiNNaker is to use classical CMOS architectures, particularly ARM968 RISC processors, to simulate the neurons' dynamics and optimize the routing between them to fit a spikebased communication perfectly. A specific communication protocol, i.e., Address Event Representation (AER), is used to do this. This protocol is designed explicitly for neuromorphic circuits, in which a spike is represented through the ID of the neuron that generated it and the corresponding generation time stamp. The IBM TrueNorth [START_REF] Akopyan | Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip[END_REF] is a fully custom ASIC realized in 28nm CMOS technology. Again the neurons use standard CMOS digital gates, but the communication is asynchronous. The Intel Loihi [START_REF] Davies | Loihi: A neuromorphic manycore processor with onchip learning[END_REF] has an approach that is a hybrid between the previous two. It has 128 custom chips implementing 1024 LIF primitives. The custom connection mesh optimizes the typical sparse communication between spiking neurons. Additionally, Loihi includes specific components to perform learning directly on-chip. It provides a set of configurable parameters to allow different local learning rules, from a basic STDP to more complex alternatives. Finally, Tianjic [START_REF] Pei | Towards artificial general intelligence with hybrid tianjic chip architecture[END_REF] is a hybrid ANN/SNN accelerator with a custom interconnection between the various cores.

In the field of ASICs, it is worth citing ODIN [START_REF] Frenkel | A 0.086-mm 2 12.7-pj/sop 64k-synapse 256-neuron online-learning digital spiking neuromorphic processor in 28-nm cmos[END_REF], a small network implemented in 28nm Fully-Depleted Silicon-On-Insulator (FDSOI) CMOS technology and targeting lowpower applications. Neurons can be configured to implement a LIF model or one of the 20 possible Izhikevich behaviors. The routing of the spikes between neurons is again performed through an AER protocol.

Table I compares the accelerators in terms of area and energy efficiency, expressed in Giga-Synapse Operations (GSOPS) per Watt. For a more detailed comparison, see [START_REF] Basu | Spiking neural network integrated circuits: A review of trends and future directions[END_REF]. The last approach in designing the SNN accelerators is to target a reconfigurable hardware platform, such as an FPGA. The FPGA can host different accelerators. This is the reason why many embedded platforms are starting to include them. Second, online reconfigurability allows hardware modification while the system is on. This can be used to add an accelerator after the system has been deployed, remove it once it is no longer required, and modify its functionality. Taking SNN as an example, the architecture of the network can be modified to target a different set of data if necessary. Finally, partial reconfigurability can add and remove functionalities to a specific accelerator. For example, online learning can be activated and deactivated on request, enabling and disabling the corresponding circuitry and physically adding or removing the required piece of hardware, guaranteeing the optimal architecture for the required application. This is the idea behind [START_REF] Carpegna | Spiker: an fpga-optimized hardware accelerator for spiking neural networks[END_REF], where the authors started to design a tiny hardware accelerator to fit an FPGA together with other components employing a LIF neuron model.

The idea is then to have a first degree of reconfigurability, making the accelerator programmable in many aspects, such as weights and thresholds. Then, to add a layer of flexibility by allowing easy modification of the hardwired network hyperparameters, such as the membrane time constant, the network architecture, the internal bit-widths, etc. Several other works are targeting a more standard but still configurable implementation, such as [START_REF] Neil | Minitaur, an event-driven fpga-based spiking network accelerator[END_REF], [START_REF] Wang | Energy efficient parallel neuromorphic architectures with approximate arithmetic on fpga[END_REF], [START_REF] Ma | Darwin: A neuromorphic hardware co-processor based on spiking neural networks[END_REF], [START_REF] Li | A fast and energy-efficient snn processor with adaptive clock/event-driven computation scheme and online learning[END_REF]. Table II compares different accelerators. For more details, see [START_REF] Li | A fast and energy-efficient snn processor with adaptive clock/event-driven computation scheme and online learning[END_REF].

In general, digital accelerators can help a lot in increasing the execution efficiency of SNNs. CMOS technology is decades old and nowadays widespread, low-cost, and highly optimized. However, the intrinsic behavior of digital devices is very far from that observed in biological components, and the response time and power consumption are still a burden. Augmented silicon photonics platforms can cover most design aspects with more efficient solutions.

IV. AUGMENTED SILICON PHOTONICS PLATFORMS

Integrated photonics is one of the key technologies that has been investigated to build neuromorphic hardware [START_REF] Paquot | Optoelectronic Reservoir Computing[END_REF]- [START_REF] Feldmann | All-optical spiking neurosynaptic networks with self-learning capabilities[END_REF]. In particular, Photonic Neural Networks (PNNs) based on silicon photonics have been extensively investigated for developing lightweight, low-latency, high-speed computing hardware with ultra-low power consumption [START_REF] Feldmann | All-optical spiking neurosynaptic networks with self-learning capabilities[END_REF]- [START_REF] Feldmann | Parallel convolutional processing using an integrated photonic tensor core[END_REF].

Such properties arise from the intrinsic nature of light manipulation and propagation and the capabilities currently integrated photonics platforms can offer. For example, different frequencies of light can be used to encode different data streams separately onto each frequency and then be processed in parallel, thus increasing computing density [START_REF] Peng | Neuromorphic Photonic Integrated Circuits[END_REF]. Such wavelength multiplexing approaches are beneficial for increasing the parallelization degree of architectures. This key feature is used in broadcast and weight protocol where each frequency channel has a specific weight assigned (for each layer) before being summed up together, e.g., by a photodetector [START_REF] Nahmias | Neuromodulation of hebbian plasticity: relevance and mechanisms[END_REF]. In this approach, ring resonators are key devices enabling the multiplexing (filtering) and weighting of the signals at different frequencies [START_REF] Feldmann | All-optical spiking neurosynaptic networks with self-learning capabilities[END_REF].

Indeed, photonic approaches allow light manipulation (e.g., weights application) while preserving signals propagation at the speed of light throughout the photonic network, thus resulting in ultra-low latencies, limited only by the physical size of the network leading to orders of magnitude lower values compared to electronics implementations [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF].

Another essential feature of photonic neuromorphic systems is the possibility of operating with analog complex-valued signals, which is beneficial to leverage non-linearities in neuromorphic hardware such as the electro-optic conversion between complex-valued optical fields into intensities (i.e., photocurrents at the photodetection), but also thanks to connection matrices presenting a more considerable richness in degrees of freedom [START_REF] Vandoorne | Experimental demonstration of reservoir computing on a silicon photonics chip[END_REF], [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF], [START_REF] Abdalla | Minimum complexity integrated photonic architecture for delay-based reservoir computing[END_REF].

Furthermore, PNNs can operate at much higher speeds than digital accelerators, with their main limitation coming from electro-optic conversion stages, e.g., at the read-out of a PNN where photodetectors allow to operate at speeds of hundreds of GHz depending on the technology and responsivity required [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF].

Among the various integrated photonic platforms that have been considered, Silicon-on-Insulator (SOI) platforms are those that have attracted the most vital interest thanks to the availability of both active and passive components and their reduced footprint compared to platforms with lower refractive indices contrasts, such as Silicon Nitride-on-Insulator (SiNOI).

Table III shows a comparison under different metrics for digital accelerators, flash technology, and three different types of PNNs based on hybrid lasers, co-integrated silicon photonics, and sub-λ nanophotonics. For the latter, the device footprint shrinks by at least an order of magnitude due to the robust localization of the optical fields, e.g., in photonic crystal cavities. It is worth noting that such constrained photonic approaches can expect a significant gain in energy consumption and latency. More information on the specific implementations can be found here [START_REF] Nahmias | Photonic Multiply-Accumulate Operations for Neural Networks[END_REF].

In particular, SOI platforms can provide high-speed modulators, e.g., in MZI or Ring Resonator (RR) configurations, as well as high-speed broadband photodetectors (> 50 GHz) and low propagation losses (< 3 dB/cm) [START_REF] Margalit | Perspective on the future of silicon photonics and electronics[END_REF]. More specifically, MZI devices allow to modulate light and change its amplitude (and phase), therefore providing a practical way to implement ANN weights [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF]. They can be arranged in meshes in precise ways, e.g., matrix multiplications as shown in Fig. 3(a) [START_REF] Clements | Optimal design for universal multiport interferometers[END_REF]. The traditionally used physical mechanisms for modulation are either (i) thermo-optic, (ii) electro-optic, (iii) carrier plasma effect [START_REF] Bogaerts | Silicon microring resonators[END_REF] as schematically shown in Fig. 3(b). For the thermooptic approach, one of the interferometer arms is heated up by TABLE III: Photonic versus electronic approaches comparison table. Latency is the time for a single matrix multiplication operation to compute at the given vector size. Speed is the time between subsequent matrix multiplies [START_REF] Nahmias | Photonic Multiply-Accumulate Operations for Neural Networks[END_REF].

Technology

Google TPU Flash Hybrid laser NN Co-Integrated Si NN Sub-λ nanophotonics [START_REF]Proceedings of the 44th Annual International Symposium on Computer Architecture[END_REF] (Analog) [ a micro-heater, resulting in a change in the refractive index of the arm. In the second case, electrodes can establish an electric field (for electro-optic materials -not present in native Si platforms) that modifies the refractive index. In the third case, a p-n/p-i-n junction is used where carriers concentration in the depletion region is modified by an applied voltage, thus changing the refractive index by the plasma carrier effect [START_REF] Bogaerts | Silicon microring resonators[END_REF]. All these approaches are of interest (depending on the platform available) and have enabled vector-matrix multiplication and PNNs for classification tasks [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF]. However, in silicon platforms, one of the main limitations during the inference process is the need to dissipate energy to keep the values of the weights. This energy can account for up to 10 mW with a total π phase shift per MZI. Such a shift allows routing full signals from one output to the other of an MZI with two output ports (see inset of Fig. 3)(a) starting from a signal only coming from the former port.

In [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF], the authors used a specific arrangement of mesh devices to perform matrix multiplication through singular value decomposition. This mapping is precisely defined between the elements of the matrix and the phase shifters MZIs in the mesh. Unlike conventional electronics like CPUs and GPUs, the energy consumed per calculation (Floating Point Operations per Second (FLOP)) reduces to zero as the size of the mesh grows larger (assuming non-volatile weights). In contrast, conventional electronics require a fixed amount of energy per FLOP, and the overall energy consumption scales quadratically with the size of the problem, i.e., N 2 instead of just N for PNNs with N representing the mesh size. The authors in [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF] also used this MZIs as weights to build a Feed-Forward Neural Network (FFNN) that could analyze vowels, achieving a simulated correctness of 90%, comparable to that of a digital computer which would reach 91.7%.

One solution to avoid constant power dissipation is setting the weights using waveguides integrating non-volatile materials such as PCMs, e.g., above the waveguide as in (iv) approach in Fig. 3)(b). Their response (change in the degree of crystalline to amorphous ratio) can be set using electrical or optical pulses [START_REF] Rios | Controlled switching of phase-change materials by evanescent-field coupling in integrated photonics[END_REF]. Such materials are exciting also to implement STDP thanks to their very rapid response time (sub-ns) to stimuli, thus allowing to build of optical plastic synapses and spiking architectures based on ring resonators [START_REF] Feldmann | All-optical spiking neurosynaptic networks with self-learning capabilities[END_REF].

The recently started Horizon Europe NEUROPULS project investigates a series of approaches that leverage silicon photonics platforms with the addition of PCMs and III-V materials for building more efficient neuromorphic hardware.

An approach to improving CNN performance is to use an accelerator designed for matrix-vector products, such as a mesh of modulators programmed to perform a specific matrix multiplication. Singular value decomposition can factor the matrix, resulting in more efficient computation on the accelerator. This approach can also handle other matrix operations, including convolutions, and optimize hardware architecture for specific tasks, leading to significant training and inference times speed-ups [START_REF] Shen | Deep learning with coherent nanophotonic circuits[END_REF].

Although optical components can benefit matrix operations, their size can limit the size of matrices that can be implemented. To overcome this limitation, alternative approaches can be explored. One possible solution is to employ pruning techniques that remove unnecessary connections and weights from the network, thereby reducing the overall size of the matrix. This makes it possible to implement larger systems using the available optical components.

Another option is to use block matrix decompositions, which involve dividing the matrix into smaller blocks that can be processed separately. This approach can enable the implementation of larger matrices using a smaller number of optical components. The smaller blocks can be computed independently and combined for the final result. This technique can also be combined with other optimization strategies, such as quantization and compression, further to reduce the size and complexity of the system.

The tensor-train approach proposed in [START_REF] Xiao | Large-scale and energy-efficient tensorized optical neural networks on iii-v-on-silicon moscap platform[END_REF] will be explored. This approach represents the matrix as a product of low-rank tensors, which can be processed more efficiently using optical components. The goal is to develop a scalable optical architecture capable of handling larger matrices and more complex neural networks using the abovementioned approaches. This will facilitate faster and more efficient training of neural networks using photonics.

The research will also explore RNN applications, including fully trainable RNNs as well as RCs. We will investigate the potential applications of RC in photonics for various applications, including nonlinear dispersion compensation of telecom signals, as demonstrated in [START_REF] Sackesyn | Experimental realization of integrated photonic reservoir computing for nonlinear fiber distortion compensation[END_REF]. The research aims to develop new techniques and approaches for utilizing photonics in Machine Learning (ML) and other fields, which could lead to significant advancements in the performance and efficiency of these systems.

To implement non-volatile optical weights, the proposed architecture will incorporate PCMs. Previous studies, such as [START_REF] Feldmann | All-optical spiking neurosynaptic networks with self-learning capabilities[END_REF] and [START_REF] Miscuglio | Artificial synapse with mnemonic functionality using gsst-based photonic integrated memory[END_REF], have explored these materials and shown a strong potential for neuromorphic systems. Incorporating nonvolatile weights can significantly reduce power consumption compared to volatile weights, which require continuous driving or periodic refreshing. Including these materials in the proposed architecture will be crucial in developing low-power and high-performance systems for machine learning and other applications.

However, in addition to their non-volatility, PCMs have another advantage: their nonlinear dynamics. E.g., by exciting the material with pulses rather than continuous-wave excitation, the nonlinear behavior of the material enables other computing paradigms, such as SNNs. In such networks, the neurons communicate using brief pulses or spikes rather than continuously varying signals. This opens up the implementation of energyefficient and highly parallel neural networks. To generate the spikes injected into the system, Advanced high-extinction ratio (ER > 8 dB) Q-switched spiking lasers can be used, which will be monolithically integrated into III-V materials on the same platform [START_REF] Mekemeza-Ona | All optical q-switched laser based spiking neuron[END_REF]. These hybrid III-V-on-Si spiking lasers are a scalable and cost-effective alternative to previous Q-switched lasers made purely from III-V materials.

III-V-on-Si spiking lasers will generate highly precise and controlled optical spikes, essential for many photonics and ML applications. These lasers offer several advantages, including high extinction ratios, low power consumption, and compatibility with standard silicon processing techniques.

V. RELIABILITY STUDIES AND CONCERNS

ANNs have an intrinsic error tolerance from an algorithmic point of view thanks to their redundant nature. However, hardware designs to deploy such algorithms must be analyzed to assess the impact of hardware restrictions or faulty manifestation on the network functional's behavior.

Due to manufacturing issues, hardware faults can occur randomly, provoked by neuron and synapse defects and imprecisions. Still, they can also be malicious, introduced by different kinds of attacks (i.e., laser beams fault injection or row hammer attack) [START_REF] Liu | Fault injection attack on deep neural network[END_REF], [START_REF] Breier | Deeplaser: Practical fault attack on deep neural networks[END_REF]. The authors of [START_REF] Breier | Deeplaser: Practical fault attack on deep neural networks[END_REF] have analyzed the misclassification rate of MLP based deep neural networks face to models derived from physical phenomena. Faulty-neuron behaviors have been injected randomly or deterministically, with injection scenarios considering single and multiple faulty neurons per layer. This is usually done during the function activation timeframe, which can be hundreds to thousands of cycles. Results indicate that in some cases, even a relatively small number of faulty neurons (≈ 10%) can lead to a high risk of misclassification (≈ 62%). As seen in section II, different activation functions have been studied for DNNs. They show that for a higher miss-classification rate (>50%), at least half of the neurons in a given hidden layer should be faulty, which is the case for sigmoid and tanh activation functions. In the case of ReLU, at least 3/4 of the neurons should be faulty to achieve the same miss-classification rate.

Extensive work has been dedicated in the last years to studying and evaluating AI hardware accelerators' errors and fault tolerance. An overview of fault tolerance techniques for feedforward neural networks is presented in [START_REF] Torres-Huitzil | Fault and error tolerance in neural networks: A review[END_REF]. In this paper, the authors review fault types, models, and measures used to evaluate performance and provide a taxonomy of the main techniques to enhance the intrinsic properties of some neural models based on the principles and mechanisms they exploit to achieve fault tolerance passively.

In [START_REF] Salami | On the resilience of rtl nn accelerators: Fault characterization and mitigation[END_REF], the authors present a study of the fault characterization and mitigation of Register-Transfer Level (RTL) model of NN accelerators by characterizing the vulnerability of NNs to application-level specifications, network topology, and activation functions, as well as architectural level specifications. In [START_REF] Li | Understanding error propagation in deep learning neural network (dnn) accelerators and applications[END_REF], authors present an experimental evaluation of the resilience of DNN systems (i.e., DNN software running on specialized accelerators) under Soft errors caused by highenergy particles.

An empirical study of DNNs resilience can be found in [START_REF] Reagen | Ares: A framework for quantifying the resilience of deep neural networks[END_REF], where a fault injection framework named Ares, which can deal with fully connected and CNN-based DNN accelerators, is presented. It uses hardware fault models related to technology and environment variability, single event faults transients in memory elements, and algorithmic level faults models such as faults occurring in weights, activation, and hidden states. Fault injection is performed static, offline, before the inference process, and dynamically during inference execution.

The analysis of SNNs fault tolerance and reliability is a relatively newer field of research since their hardware implementations are much more recent than classical DNNs.

Consensus has yet to be reached on the main applications of these networks. Moreover, the variety of signal-to-spike coding and training algorithms brings a specific heterogeneity in their characteristic which should be accounted for when their fault tolerance and reliability is discussed. One of the first works in this field, [START_REF] Vatajelu | Fully-connected single-layer stt-mtj-based spiking neural network under process variability[END_REF], estimates the accuracy of a FC SNN, capable of STDP learning designed with spintronic devices under the effect of process variability. Both the neuron and synapse behavior are strongly affected by process variability, and the accuracy drops by approximately 10% when assuming moderate variability compared to the ideal case.

Another interesting study is presented in [START_REF] Vatajelu | Special session: Reliability of hardwareimplemented spiking neural networks (snn)[END_REF], where a taxonomy of faults was defined for spiking neural networks. The accuracy of a hardware-implemented spiking neural network designed to perform STDP online training was analyzed under the assumption that (i) both learning and inference were performed on faulty hardware, (ii) only inference was performed on faulty hardware. This paper shows that performing learning directly on faulty hardware reduces the impact of faults on the network accuracy by an average of 15% and, in extreme cases, can reach up to 30%. Moreover, in [START_REF] Bosio | Rebooting computing: The challenges for test and reliability[END_REF], faults affecting the signal-to-spike conversion layer have the strongest impact on the network accuracy, with the synaptic stuck-at faults coming to a very close second. These faults strongly affect the learning process, which only exacerbates during inference.

On the other hand, faults, like delayed synapse activation or stuck lateral inhibition, have a marginal effect. The study presented in [START_REF] Spyrou | Reliability analysis of a spiking neural network hardware accelerator[END_REF] takes a different approach. The fault tolerance study is performed on an SNN inference accelerator, a multi-layer SNN with supervised off-line learning, designed in VHDL and implemented on an FPGA. The fault injection experiments identify the parts of the design that need to be protected against faults and the inherently fault-tolerant parts. They have shown that the behavior under faults of the chosen type of SNN is similar to the behavior under faults of an ANN in that faults injected in the most significant bits of the synaptic values affect have a more substantial effect on network accuracy that when the faults are injected in the less critical bits, and also faults affecting the last layer (where the classification is performed) are more relevant than faults affecting the first layer (where there is higher computing redundancy). In addition, the authors have shown that their proposed SNN implementation is much more sensitive to faults injected in the routing of signals than in the synaptic weight or neuron computation.

Several studies have compared SNNs and their ANNs counterparts, mainly focusing on performance or power consumption rather than their relative fault tolerance. Therefore, a study was conducted to assess the fault resilience of both network types, assuming different quantizations of weights and neural computation and various training scenarios for the SNN. The study compared the fault tolerance of an MLP and an SNN with the same topology and bit precision. A 784 X 100 X 10 network was implemented to solve the MNIST classification problem [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. The MLP uses a sigmoid activation function for all layers and has a base accuracy of 98% after 20 training epochs. The SNN used rate coding for signal-to-spike conversion and several training algorithms: (i) Shadow Training (ST) (with a base accuracy of 96%), (ii) BPTT (with a base accuracy of 97%), (iii) STDP (with a base accuracy of 87%). The precision of synaptic weights was 16, 8, and 4 bits. Figure 4 illustrates selected results of the analysis, showing that the SNNs are more fault-tolerant than the MLP, and the SNN trained using BPTT is the most resilient of the three. This study also demonstrates how the SNN is trained in its fault resilience, with online training being more resilient than offline training. The last two columns in Figure 5 correspond to the SNN being trained by STDP, and the network accuracy degradation is computed when the presence of faults is assumed only at inference (AT) or during both training and inference (BT). These results show that the fault tolerance of the SNN depends on the training mechanism and that an SNN trained online is more resilient to faults than an SNN trained offline. The results suggest that SNNs have the potential to be more resilient to faults than ANNs, and further comparison between the two is necessary to consolidate these findings. were introduced, and their digital counterparts were described using standard CMOS technologies. The discussion then focused on augmented silicon photonics improvements and the reliability aspects of Artificial Neural Network. The benefits and drawbacks of each technology were thoroughly analyzed, including its reliability. Our analysis suggests that the choice of technology for Artificial Neural Network design should depend on the specific application requirements and design constraints. While CMOS technology offers established fabrication processes and high reliability, it may have limitations in power consumption and scalability. Silicon photonics, on the other hand, provides high power efficiency and scalability, but their reliability is still under ongoing research. We hope this paper will serve as a valuable reference for researchers and practitioners in the field of Artificial Neural Network design as they explore the potential of these emerging technologies.

  (a) Classical Von Neuman computing approach (b) Neural network's computation and memory co-location

  (a) Multi-bit digital coding (b) Spikes information encoding

Fig. 3 :

 3 Fig. 3: (a) Photonic implementation of a network for 9x9 matrix multiplication based on Mach-Zenhder Interferometers (MZIs) [60]. Inset describes what each crossing consists of, i.e., a phase shifter (ϕ), then a 50/50 splitter, another phase shifter (2θ), and then a 50/50 combiner. (b) Top-view of phase shifter implementations in waveguides based on (i) thermooptic effect, (ii) electro-optic effect, (iii) carrier plasma effect, and (iv) Phase Change Material (PCM)-induced shift. Blue lines where no crossing is present are optical waveguides.

Fig. 4 :

 4 Fig. 4: Network accuracy under synaptic fault injection: a comparison between MLP and SNN. The synaptic weight precision is 8 bits, and 10 -4 of the synaptic bits are considered faulty.

Fig. 5 :

 5 Fig. 5: Maximum network accuracy degradation for different training scenarios under synaptic fault injection: a comparison between MLP and SNN. The synaptic weight precision is 8 bits, and 10 -4 of the synaptic bits are considered faulty. AT = after training, BT = before training VI. CONCLUSION In this paper, the fundamentals of Convolutional Neural Network, Deep Neural Network, and Spiking Neural Network were introduced, and their digital counterparts were described using standard CMOS technologies. The discussion then focused on augmented silicon photonics improvements and the reliability aspects of Artificial Neural Network. The benefits

TABLE I :

 I ASIC comparison table[START_REF] Basu | Spiking neural network integrated circuits: A review of trends and future directions[END_REF]

	Design	[42]	[18]	[43]	[44]	[45]
	# of neurons	18k	1M	128k	39k	256
	# of synapses	18M	256M	128M 9.75M	64k
	Area (mm 2 )	88.4	413	60	14.4	0.086
	Process (nm)	130	28	14	28	65
	Energy (GSOPS/W) 0.033	400	-	649	78.7

TABLE II :

 II FPGA comparison table[START_REF] Li | A fast and energy-efficient snn processor with adaptive clock/event-driven computation scheme and online learning[END_REF]

	Design	[48]	[49]	[50]	[47]	[25]
	Clock frequency(MHz)	75	120	25	100	100
	Data format	16bit Fixed	8bit Fixed	32bit Fixed	16bit Floating	16bit Fixed
	Computing scheme	Event-Driven	Clock-Driven	Event-Driven	Adaptive Clock/Event-Driven	Clock-Driven
	Neuron model	LIF	LIF	LIF	LIF	LIF
	FPGA platform	Spartan 6	Virtex 6	Spartan 6	Virtex 7	Artix 7
	Neurons	1794	1591	1794	1094	1384
	Synapses	647000	638208	647000	177800	313600
	Task	MNIST	MNIST	MNIST	MNIST	MNIST
	Computation time	0.53s/image	8.40s/image	0.16s/image	3.15ms/image	215µs/image
	Computation time @100MHz	0.40s/image	10.08s/image	40.00ms/image	3.15ms/image	215µs/image
	Energy	0.80J/image	1.12J/image	Not reported	5.04mJ/image	13mJ/image
	Energy/Synapse	1.2µJ/synapse 1.76µJ/synapse	Not reported	0.028µJ/synapse	0.041µJ/synapse