Annachiara Ruospo

Ernesto Sanchez

Marcello Traiola

Ian O'connor

Alberto Bosio

Investigating data representation for efficient and reliable Convolutional Neural Networks

Keywords: Approximate Computing, Convolutional Neural Networks, Reliability, Fault Injections

HAL is

Introduction

In the last decades, Convolutional Neural Networks (CNNs) have gained popularity due to their excellent performance in solving complex learning problems [START_REF] Albawi | Understanding of a convolutional neural network[END_REF]. Indeed, they provide very good results for many tasks such as object recognition in image/videos, drug discovery, natural language processing up to playing games [START_REF]Recent Advances in Deep Learning for Speech Research at Microsoft[END_REF][START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF][START_REF] Silver | Mastering the game of go with deep neural networks and tree search[END_REF]. They are categorized as a class of Deep Neural Networks (DNNs) [START_REF] Lecun | Deep learning[END_REF] and their name origins from the mathematical linear operation between matrixes called convolution.

Being brain-inspired models, convolutional neural networks have an inherent resilience attributed to their distributed and parallel structure and the space redundancy introduced because of over-provisioning [START_REF] Sung | Resiliency of deep neural networks under quantization[END_REF]. Several studies have investigated the fault tolerance of DNNs, for example [START_REF] Sequin | Fault tolerance in artificial neural networks[END_REF], and have demonstrated that NNs are highly robust against computation errors [START_REF] Tchernev | Investigating the fault tolerance of neural networks[END_REF][START_REF] Vialatte | A study of deep learning robustness against computation failures[END_REF]. As a consequence, hardware-specific NNs, ranging from circuits for embedded machine learning applications to custom Very Large Scale Integration (VLSI) of neural networks in silicon, are traditionally considered tightly robust to hardware faults (HW faults). However, several recent studies have demonstrated that HW faults induced by an external perturbation (i.e., in a harsh environment) or due to silicon wearout and aging effects can significantly impact the inference leading to CNN prediction failures [START_REF] Torres-Huitzil | Fault and error tolerance in neural networks: A review[END_REF][START_REF] Ruospo | Evaluating convolutional neural networks reliability depending on their data representation[END_REF][START_REF] Lotfi | Resiliency of automotive object detection networks on gpu architectures[END_REF]. Therefore, ensuring the reliability of CNNs is crucial, especially when they are deployed in safety-critical and mission-critical applications, such as robotics, aeronautics, smart healthcare, and autonomous driving [START_REF] Chen | Deepdriving: Learning affordance for direct perception in autonomous driving[END_REF]. Reliability can be defined as the probability that a HW fault causes a failure [START_REF] Di Natale | Cross-Layer Reliability of Computing Systems[END_REF]. The reliability analysis and its evaluation are regulated by standards depending on the application domain (e.g., IEC 61508 for industrial systems, DO-254 for avionics, ISO 26262 for automotive), and it is usually assessed through fault injection (FI) campaigns.

Nowadays, to address data confidentiality issues and bandwidth limitations, the trend is to push deep learning based systems from the cloud to edge devices [START_REF] Peluso | Monocular depth perception on microcontrollers for edge applications[END_REF][START_REF] Ottavi | A mixed-precision risc-v processor for extreme-edge dnn inference[END_REF][START_REF] Ravaglia | Memory-latency-accuracy trade-offs for continual learning on a risc-v extreme-edge node[END_REF], such as Internet-of-Things (IoTs) devices, given the ever-increasing internet-connected IoTs. One of the principal advantage is that it alleviates the communication latency which is unacceptable for real-time safety-critical decisions, e.g., in autonomous driving. For this reason, the design of custom Application Specific Integrated Circuit (ASIC) hardware accelerators to support the energy-hungry data movement, speed of computation, and memory resources that CNNs require to realize their full potential at the edge is crucial [START_REF] Moons | 5 envision: A 0.26-to-10tops/w subword-parallel dynamic-voltage-accuracy-frequencyscalable convolutional neural network processor in 28nm fdsoi[END_REF].

In parallel with reliability assessments [START_REF] Schorn | Accurate neuron resilience prediction for a flexible reliability management in neural network accelerators[END_REF], a significant effort has been done to reduce the memory and energy footprint of CNNs leveraging on reduced bitwidth data type in either training or inference phase. Indeed, one important limitation about the adoption of the newer version of CNNs is the memory required for storing the network parameters (e.g., synaptic weights). For example, VGG-Net [START_REF] Simonyan | Very deep convolutional networks for largescale image recognition[END_REF] requires 500 MB of memory, a complexity that simply cannot fit the constrained hardware of many embedded systems. Recently, Approximate Computing (AxC) has become a major field of research to improve both speed and energy consumption in embedded and high-performance systems [START_REF]A survey of techniques for approximate computing[END_REF]. By relaxing the need for fully precise or completely deterministic operations, AxC substantially improves energy efficiency and reduces the memory requirement.

Various techniques for AxC augment the design space by providing another set of design knobs for performance-accuracy trade-offs. As an example, the gain in energy between a low-precision 8-bit operation suitable for vision and a 64-bit double-precision floating-point operation necessary for high-precision scientific computation can reach up to 50× by considering storage, transport, and computation. The gain in energy efficiency (the number of computations per Joule) is even larger since the delay of basic operations is greatly reduced. Having simpler operators also reduces the implementation cost, which allows the network to use more resources in parallel.

Therefore, the major challenge is to find an adequate data representation for CNNs that fits well with the application and hardware constraints. CNNs lend themselves well to AxC techniques, especially with fixed-point arithmetic or low-precision floating-point implementations, which expose large fine-grain parallelism. For instance, in [START_REF] Courbariaux | Binaryconnect: Training deep neural networks with binary weights during propagations[END_REF] the authors describe a binary network which exploits only two values {-1, 1} for the weights representation. Another proposed solution is the ternary network [START_REF] Zhu | Trained ternary quantization[END_REF]; it quantizes weights into 3 different values {-1, 0, 1}. Finally, XNOR-Net [START_REF] Rastegari | Xnor-net: Imagenet classification using binary convolutional neural networks[END_REF] uses a slightly different methodology: all computations are performed through XNOR and bit counting operations, at the same time reducing the precision of the operands involved during the computation. Consequently, it is necessary to understand if those optimized models are reliable enough to tolerate failures that propagate throughout the system. It starts to be crucial to evaluate CNNs behaviour in a faulty scenario to determine if they can still be safely deployed in a safety-critical system. These doubts are justified if considering the growing technology scaling in chip manufacturing.

Due to the transistors shrinking, newer hardware platforms are more complex and, at the same time, more susceptible to faults, albeit faster.

The end-goal of this paper is to characterize the impact of permanent faults affecting a CNN by means of fault injection campaigns, when custom data types are used for representing the network parameters. We analysed different implementations of the same CNN architecture, when different data types are exploited, and we identified the best representation leading to achieve a 4x reduction of memory footprint with the highest resilience to faults. Two case studies are presented: the former targets LeNet-5, a popular CNN, the latter focuses on a deeper CNN (YOLO) used for the task of detecting objects in real time.

The rest of the paper is structured as follows: after summarizing the related works in the field, our main contribution is highlighted (Section 1.1). Section 2 presents the methodology, focusing on the custom data type description and the data conversion technique. In the same section, both the fault injection scenario and environment are described. Next, in Section 3, we present the two case studies (i.e., LeNet-5 and YOLO), along with the experimental results coming from the FI campaigns. Finally, Section 4 concludes the article and outlines some of the possible future research directions.

Related Works

In the literature, increasing attention is paid to the Neural Network Reliability. Depending on multiple factors, such as FI typology, level of abstraction, and fault models, it is possible to identify different sets of interesting research activities.

A significant set focuses on analysing a specific fault model: the soft error (i.e., bit flip). In [START_REF] Neggaz | Are CNNs reliable enough for critical applications? an exploratory study[END_REF][START_REF] Neggaz | A reliability study on CNNs for critical embedded systems[END_REF] authors performed a deep analysis of CNN reliability when 32-bit floating-point values are used as data type for weights representation, it turned out that bit 30 th is the most critical among the overall 32.

Interestingly, we found out the same results with our methodology as shown in next sections. In [START_REF] Santos | Evaluation and mitigation of soft-errors in neural network-based object detection in three gpu architectures[END_REF][START_REF] Ibrahim | Soft errors in DNN accelerators: A comprehensive review[END_REF], the authors evaluate the reliability of one CNN executed on three different GPU architectures (Kepler, Maxwell, and Pascal).

The soft errors injection has been done by exposing the GPUs running the CNN under controlled neutron beams. A similar but wider approach is detailed in [START_REF] Lotfi | Resiliency of automotive object detection networks on gpu architectures[END_REF], where the authors assess the reliability of a 54-layers DNN (NVIDIA and 16-bit fixed-point. Overall, they conclude that, the larger the dynamic value range of the network's data type, the higher the likelihood of having large deviations in values in the event of faults leading to wrong predictions. Furthermore, a different framework is shown in [START_REF] Reagen | Ares: A framework for quantifying the resilience of deep neural networks[END_REF]: Ares, a light-weight DNN fault injection framework. The authors present an empirical study on the resilience of three prominent types of DNNs (fully connected, CNNs and Gated Recurrent Unit). In particular, they focus on two fixed-point data types for each network: Q 3,13 i.e, 3 integer and 13 fractional bits, and Q 2,6 . Their experiments demonstrate that the optimized Q 2,6 data type is 10x more fault tolerant. The reason lies in the fact that the unnecessary larger range of integer values increases the chance of failure happening. It is worth noting that this result is in line with our gathered results, presented in Section 4.2. It is a common trend to explore fixed-point computations for ultra-low power embedded systems with a limited power budget [START_REF] Palossi | A 64mw dnn-based visual navigation engine for autonomous nano-drones[END_REF]. Finally, in [START_REF] Salami | On the resilience of rtl nn accelerators: Fault characterization and mitigation[END_REF], the authors analyse the reliability of a DNN accelerator by following a High Level Synthesis (HLS) approach. They characterize the effects of both permanent and transient faults by exploiting a fault injector framework embedded into the RTL design of the accelerator. Faults are injected during the inference cycle only on a subset of registers: those that are in charge to store weights, input values, and intermediate ones used throughout the inference job, without considering the effects of faults in the other data-path units. As for the used data representation, they perform the experiments by only adopting a 16-bits fixed-point low precision model, claiming a negligible accuracy loss with respect to a full-precision data model.

Additionally, it is worth also mentioning the research contribution of [START_REF] He | Fidelity: Efficient resilience analysis framework for deep learning accelerators[END_REF] and [START_REF] Libano | How reduced data precision and degree of parallelism impact the reliability of convolutional neural networks on fpgas[END_REF], where the authors investigate the reliability of CNNs exploiting both the 16-bit and 8-bit integer data representation. Then, moving towards an even smaller data dimension, related works in [START_REF] Sabbagh | Evaluating fault resiliency of compressed deep neural networks[END_REF] and [START_REF] Du | On the reliability of convolutional neural network implementation on SRAM-based FPGA[END_REF] exploit reduced bitwidths. The former uses 5-bit and 3-bit fixed-point data types and a binary representation. The latter performs reliability assessment analysis on a binary neural network, where only 1 bit is used to represent the parameters (weights and biases).

In this light, the principal contribution of this paper is a comprehensive analysis on the behavior of CNNs depending on their data representation. In a previous work [START_REF] Ruospo | Evaluating convolutional neural networks reliability depending on their data representation[END_REF][START_REF] Bosio | A reliability analysis of a deep neural network[END_REF], we evaluated the impact of permanent faults affecting CNNs through software FI campaigns. Compared to the state-of-the-art analyses [START_REF] Reagen | Ares: A framework for quantifying the resilience of deep neural networks[END_REF][START_REF] Salami | On the resilience of rtl nn accelerators: Fault characterization and mitigation[END_REF][START_REF] Ruospo | Evaluating convolutional neural networks reliability depending on their data representation[END_REF][START_REF] Bosio | A reliability analysis of a deep neural network[END_REF], a wider spectrum of floating-and fixed-point representations is given (five typologies ranging from 32 bits up to 8 bits). As a result, we identified the best data representation leading to the highest memory footprint reduction and the highest resilience to faults.

Methodology

This section presents the adopted methodology. We exploit the darknet open source DNN framework [START_REF] Redmon | Darknet: Open source neural networks in c[END_REF]. Implemented in C and CUDA language, it is suitable to perform end-to-end deployment of neural network architectures in a very simple way. It further supplies a very simple environment where several configurations of DNNs, including CNNs, can be executed either to perform training or inference jobs. In our work, we modified darknet framework to (i) approximate the DNN and (ii) inject faults at the inference time.

DNN Data Type Approximation

In the neural networks field, a common approximation approach is to reduce the precision and data type of weights and activation's values. More in detail, we intend to use custom floating-point and fixed-point representations with different precision (i.e., bit-width) at the inference time.

The darknet framework leverages on 32-bit floating-point data types only.

Therefore, we modified the darknet source code to allow data type conversions.

All the conversions between the standard 32-bit floating-point and custom data type have been carried out by integrating two open source libraries into the darknet framework: the libfixmath library [START_REF]Libfixmath library[END_REF] for managing fixed-point and the FloatX library. • N : it determines the data bit-width;

… … 0 N-1 i
• i: it determines the dynamics and the precision of the data type depending on the data representation:

-Floating Point: i is the mantissa width, N -1 -i is the exponent width;

-Fixed Point: i is the fractional width, N -1 -i is the integer width.

Since the end-goal is to characterize fault effect propagation through the network (speeding up computations and compacting the model size are out of the scope of this work), we performed on-line conversions while maintaining all internal operations in floating-point (Figure 2). The benefits coming from this approach are two-fold: first, it is not necessary to change the framework structure every time new experiments with a different data type have to be performed; second, it allows to change the representation without retraining the DNN model for each data type, exploiting the same set of trained parameters.

In this way, the assessment of the CNN reliability is quicker; it is possible to switch between experiments with different numerical formats in a reasonable amount of time. To confirm the choice suitability, we performed a preliminary experiment to confirm that the use of the online conversion does not introduce important accuracy differences while providing execution time benefits. To this end, in the experiments, we converted the data representation of the darknet framework from 32-bit floating point to 32-bit fixed point. In this way, all the operations were performed in the fixed-point arithmetic domain by using the libfixmath library [START_REF]Libfixmath library[END_REF]. We executed the inference of 70,000 images from the MNIST database with the LeNet-5 CNN (more details in Section 3.1) with both versions, i.e., the original floating-point one and the modified fixed-point one.

Specifically, we did not retrain the CNN. The results of the experiments showed that the average accuracy error of the fixed-point version with respect to the floating-point one was of -0.01%, i.e., a slight accuracy increase. Moreover, while For the sake of completeness, we describe how the on-line conversion of a 32-bit floating-point weight is applied before reaching a single neuron (Figure 2). The applied scheme works in the following way:

1. The weight is converted from standard 32-bit floating-point to a given custom data type representation.

2. The custom data type weight is corrupted according to a chosen fault list and a fault location, i.e., the fault is injected.

3. The custom data type weight is converted back to the standard 32-bit floating-point representation in order to preserve the native implementation of the framework. In such a way, the gained value reflects the same fixed-point corrupted value, while still remaining a floating-point data.

4. The weight is multiplied by the input value.

The neuron performs the arithmetic computations.

Although all network operations are executed between 32-bit floating-point variables, it should be outlined that the loss of precision caused by the first conversion is preserved. Indeed, when moving from the standard 32-bit floatingpoint representation to a low-precision one (e.g., [START_REF] Ottavi | A mixed-precision risc-v processor for extreme-edge dnn inference[END_REF]-bit fixed-point), we are witnessing a truncation error effect. Then, converting back from a narrow range of value to a wider one, the truncation error still remains.

Fault Injector

The intent of the section is to describe the Fault Injection (FI) scenario and the environment built on the darknet framework. outputs are stored (i.e., the "Golden Std" and the "Golden Custom") and compared to determine the Accuracy Loss due to the approximation. The fault injection campaign is carried out on the Custom CNN and the faulty inference outputs are stored in the "Faulty Custom" log. The latter is then compared with the "Golden Custom" and the "Golden Std" to assess the resilience.

The Custom CNN (i.e., approximate) is intended to replace the Standard CNN in edge/resource-limited devices. Thus, we have to assess the reliability of the Custom CNN with respect to the standard CNN. On the other hand, it is also possible to train directly the custom-data-type CNN. In this case, the reference with respect to the reliability has to be assessed is the Custom CNN itself.

FI Environment: The hardware system can be affected by faults caused by physical manufacturing defects. As Figure 4 highlights, faults could propagate through the different hardware structures composing the full system. However, it could happen that they are masked during the propagation either at the technological or at the architectural level [START_REF] Kooli | Cache-and register-aware system reliability evaluation based on data lifetime analysis[END_REF]. When a fault reaches the soft-

Hardware masking

Hardware independent ware layer of the system, it can corrupt data, instructions or the control flow.

Hardware dependent

These errors may impact the correct software execution by producing erroneous results or prevent the execution of the application leading to abnormal termination or application hangs. The software stack can play an important role in masking errors; at the same time, this phenomenon is implicitly important for the system reliability but a hard challenge for engineers that have to ensure the safeness of their systems.

As

where Layer corresponds to the CNN layer, Connection is the edge connecting one node of the Layer, and Bit is one of the bits of the weight associated with the Connection. Finally, the P olarity can be '0' or '1' depending on the SaF. The Fault Injector actually works at the software layer, and its pseudocode is provided in the P seudo -Code (Listing 1). It corresponds to a simple serial fault injector that modifies the CNN topology as described by [START_REF] Albawi | Understanding of a convolutional neural network[END_REF].

The FI process follows the scenario depicted in Figure 3 and consists of the following: once the CNN is fully trained, a golden run is performed collecting the golden results (i.e., standard and custom golden prediction), i.e., lines 1 and 2 in Listing 1. Then, the actual fault injection process is performed. The initial step requires generating the list of faults to be injected. This fault list should be seen as a list of places to inject the faults, as previously described. Then, for any fault in the fault list (line 3), a prediction run is performed and the results collected and named as faulty prediction. It is necessary to underline that faults are injected regardless of their polarity (stuck-at-0 or stuck-at-1).

Once the fault location is fixed, the target bit is inverted (if '0' it becomes a '1' and vice-versa). In this way, we do not distinguish between the singular effect of the two fault models while obtaining great flexibility for the considerable amount of performed simulations. At this point (lines 6 and 7), the obtained results are compared with the expected ones (again for both the standard and custom CNN), and the results are logged for further analysis.

In detail, the function compare of the Pseudo-code (Listing 1) classifies the prediction/classification of the faulty CNN with respect to the golden one. The classification depends on the CNN type. In our paper, we consider two types of CNNs: (i) a classifier and (ii) an object detector. Concerning the classifier, outputs of the faulty CNN are labelled as follows:

• Masked: No difference is observed between the faulty CNN and the golden one.

• Observed: A difference is observed between the faulty CNN and the golden one. Depending on how much the results diverge, we further classify these as:

-Good: The confidence score of the top-ranked is higher with respect to the golden CNN. In other words, the faulty CNN provides a better inference than the golden one;

-Accept: The confidence score of the top-ranked element is reduced by less than 5% with respect to the golden CNN;

-Warning: The confidence score of the top-ranked element is reduced by more than 5% with respect to the golden CNN;

-Critical: The top-1 prediction is different. In other words, the faulty CNN makes a wrong inference.

From a safety assessment perspective, we consider three classes of faults Critical, Warning, and Accept as events reducing the CNN safety. Indeed, whenever one of those fault classes occurs, either the top-1 prediction is different (Critical ) or the top-1 prediction confidence level decreases (Warning, Accept).

On the other hand, the two fault classes Masked and Good either leave the safety of the CNN unaltered (Masked) or even improve it (Good).

On the other hand, the effects of the injected faults on the object detector CNN are classified differently. The CNN output is an image having bounding boxes indicating the detection of the objects. To assess whether any two bounding boxes overlap, we use the intersection over union (IoU) metric as defined in [START_REF] Lotfi | Resiliency of automotive object detection networks on gpu architectures[END_REF]. IoU is the ratio of the intersection area over the union area of two bounding boxes. The closer IoU is to 1, the higher overlap the two bounding boxes have.

Thanks to the IoU metric, we can redefine the fault outcome as follows:

• Masked: No difference is observed between the faulty CNN and the golden one.

• Observed: A difference is observed between the faulty CNN and the golden one. By using the IoU calculated between the boxes of the golden CNN and those of the faulty one, we further classify the observed outcomes as follows:

-Accept: The IoU is lower than 1 and higher than 0.95;

-Warning: The IoU is lower than 0.95 and higher than 0.9;

-Critical: The number of bounding boxes is different, or the label associated with the boxes does not match the good ones. In other words, the faulty CNN identified wrong objects. Moreover, if the IoU is lower than 0.9, the fault is classified as critical, meaning that the faulty CNN correctly identifies the objects, but it is not able to locate them precisely enough.

We did not consider the "Good" outcome since it does not make sense for object detection. It is worth noting that the fault classification used for the object detector is more stringent than the one used for the classifier. Indeed, we consider the object detection task more critical than the classification task.

From a safety assessment perspective, we consider faults falling in the Critical, Warning, and Accept classes as events reducing the CNN safety. Conversely, Masked faults leave the safety of the CNN unaltered.

Experimental Results

This section first details the two case studies and the related fault injection campaigns, then it discusses the experimental results.

First Case Study

Among the existing CNNs, our interest falls on LeNet-5 [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF], a well-known classifier for handwritten digit recognition tasks introduced by Y. Lecun Moreover, since we are focusing on the inference phase and on the response of the network in a faulty scenario, a set of pretrained weights has been adopted.

It is available from the darknet website and includes all weights in a 32-bit floating-point representation.

Custom Data Type

The selection of the custom data type is not trivial. Let us resort to an example to illustrate that point. In our preliminary work [START_REF] Ruospo | Evaluating convolutional neural networks reliability depending on their data representation[END_REF], we analyzed different fixed-point data types in terms of resilience to faults. One of those was configured with N = 16 and i = 8, meaning that 8 bits were devoted to represent the fractional part and 8 bits the integer one. Figures 5a and5b show the "criticality" of each bit of the data type. It can be seen that in the 32-bit floating-point data type, only one bit (i.e., the bit 30 th ) is the main responsible for the Critical Observed faulty behaviors: up to 95% of critical observed faulty behaviors is due to a fault at bit 30 th . On the other hand, in the 16-bit fixedpoint representation (with 8 bits for integers and 8 bits for the fractional part), the number of bits responsible for the Critical Observed faulty behaviors is six.

This means that the custom CNN has a lower memory footprint of 50%, but it also shows a lower resilience with respect to the standard CNN, since a higher number of faulty bits may seriously affect the inference results. analysis, we simply deduced that the data type does not need higher dynamics while a high precision is preferred. Hence, we selected the custom data types reported in Table 1. Two data types are used, the fixed and floating point with different bit width. Moreover, we computed the accuracy loss of the CNN showed that only when reducing the bit width to 8 bits, the network exhibited some accuracy loss. In detail, for the network with weights encoded by using 8-bit floating-point variables (scenario FP8), the accuracy loss was 0.02%, while it was 0.04% when the weights were encoded by using 8-bit fixed-point variables (scenario FxP8).

Fault List

In this section, we discuss the complexity of the considered CNN in terms of fault injection. Furthermore, we illustrate the approach that we used to manage this complexity by performing a statistically meaningful subset of fault injections. For this complex evaluation, we consider only four LeNet-5 layers that perform arithmetic computations involving trainable weights, i.e., two convolutionals and two Fully Connected. Indeed, we consider the resilience of the CNN against faults striking the memory, where the weights are stored. Table 2 provides details about the configuration as well as the fault list of each layer. The first two rows (labeled "Layer" and "Detail") of the table present the target layers; the third one ("Connections") specifies the amount of their connection weights. The number of possible faults is computed as the multiplication between the connections number ("Connections") and the weight size ("Bit-width").

As the rows "#Faults" point out, the overall number of possible faults is very high and this reflects in a non-manageable FI campaign execution time.

Thus, to reduce the fault injection execution time, we can randomly select a subset of faults. To obtain statistically significant results with an error margin of 1% and a confidence level of 99%, an average of 15.6k fault injections have to be considered for 32-bit scenarios (FP32 and FxP32), 15k for 16-bit scenarios (FP16 and FxP16), and 13.8k for 8-bit scenarios (FP8 and FxP8). The precise numbers are given in the rows of Table 2 labeled #Injections" and they have been computed by using the approach presented in [START_REF] Leveugle | Statistical fault injection: Quantified error and confidence[END_REF]. In details, we resorted to the following formula:

f ault injections = N 1 + e 2 • N -1 t 2 •0.25 (2)
where N is the total number of fault locations (i.e., row #Faults of Table 2), e is the desired error margin (1%), and t depends on the desired confidence level (t=2.58 corresponds to 99% confidence level [START_REF] Leveugle | Statistical fault injection: Quantified error and confidence[END_REF]). Equation 2 has an horizontal asymptotic value (N → ∞) equal to 16,641, thus limiting the number of fault injections necessary to achieve an evaluation with an error margin of 1% and a confidence level of 99%. Moreover, it is worth underlining that the injections are performed by randomly selecting the faulty bit among all bits of the connection weights.

To perform the FI experiments, the weight conversions are performed as described in Section 2.1, i.e., the weights are mapped to the custom type and then reconverted back to the original format. In particular, every time a fault is placed, the randomly selected weight value must be converted to the custom representation, injected with the fault, and reconverted back to the original format to perform the inference. On the one hand, this approach requires an overhead during the fault injection phase; however, on the other hand, this technique allows performing the experiments without introducing overhead due to external computationally intensive custom floating-and fixed-point libraries for the arithmetic operations.

Fault Injection Outcomes

We conducted two sets of experiments (see Figure 3). In the first one, we evaluated the reliability by using as reference the Standard 32-bit floatingpoint CNN. This is useful in the case where a designer wants to approximate the CNN (i.e., change its data type and/or bit-width) after that it has been trained. In the second one, we assess the CNN reliability by using as reference the Custom fault-free CNN. This is useful in the case where a designer wants to train directly the custom data-type/bit-width CNN. To discuss the results, we refer to the classification presented in Section 2.2. In particular, we want to evaluate the safety of the different CNN versions, when subject to faults. Therefore, we consider faults in the classes Critical, Warning, and Accept as events reducing the CNN safety. The sum of these contributions is represented by symbol '<' in tables. Conversely, we consider the faults in the classes Masked and Good as events either leaving the safety of the CNN unaltered or even improving it. The sum of these contributions is represented by symbol '≥' in tables.

The results of the first set of experiments (i.e., having the Standard 32-bit floating-point CNN as reference) are shown in Table 3 where each row corresponds to one of the CNN variants (FP32-FP16-FP8-FxP32-FxP16-FxP8 defined in Table 1). Each column corresponds to a faulty behavior class as described in Section 2.2.

First, we can note a different resilience to faults depending on the data type: floating versus fixed. More in detail, the safety decreasing effect is lower for the fixed-point than for the floating-points, for a given bit-width.

As an example, we may resort to scenarios FP32 and FxP32 (32-bit CNNs): the safety increasing (decreasing) effect varies from 69% (31%) of the floatingpoint version (scenario FP32) to 74% (26%) of the fixed-point version (scenario FxP8). This corresponds to a difference of 5%. The average difference between floating-and fixed-point versions with respect to safety increasing/decreasing effect over the three variants (32, 16, and 8 bits) is 8.96% over all the layers.

This can be seen by comparing the scenarios FP32 with FxP32, FP16 with FxP16, and FP8 with FxP8, in terms of the average safety increase/decrease effect variation (columns 8 and 9). Is it worth highlighting that, in general terms, the safety decreasing effect is critical only in a few cases. The percentage of critical faults is always lower than 3.42% for all the variants. In particular, fixed-point variants have a very small percentage of critical faults, always lower than 0.46%. Moreover, the contribution of Good faults to the safety increasing effect turns out to be significant, especially for 16-and 8-bit versions. As an example, in the scenario FxP8 for the layer L0, we observed a safety increasing effect in 52.21% of the cases, with a 52.18% of Good faults.

Furthermore, the bit-width plays an important role for the reliability: the lower the bit-width the lower the resilience. Therefore, a designer who wanted to use a more efficient version of the CNN (reduced memory footprint)

has to be aware that it would be also less resilient with respect to the original CNN (FP32). However, it is worth also remarking that using fixed-point data representation, instead of the floating-point counterpart, provides the better results in terms of trade-off between resilience and efficiency. This is reported in the last two columns of Table 3. For instance, we may compare scenarios FP8 and FxP8 (8-bit CNNs) for layer L0: we observed a safety loss with respect to FP32 of 37% in the floating-point version (scenario FP8) and only of 16% in the fixed-point version (scenario FxP8). Therefore, choosing the CNN in the scenario FxP8, namely 8-bit fixed-point (1 bit for integer and 7 bits for fractional), allows the designer to compact the memory footprint by a 4x factor while reducing the safety only by 16%. Moreover, by looking more closely, the occurrence of critical faults in scenario FxP8 even decreases from 1.32% of FP32 to 0.45%, while in scenario B it increases to 3.41%. Additionally, for scenario FxP32 (32-bit fixed-point CNN), it has been observed that the CNN achieves improved safety with respect to FP32 scenario for the same memory footprint for layers L0 and L2 (+5.34% and +1.43%, respectively). Thus, simply changing the CNN data type to a fixed-point representation may improve its resilience for some layers.

Table 4 reports the results of the second set of experiments (i.e., having the Custom CNN as a reference). While in the first set we compared the FI results of each scenario to the ones obtained with the fault-free 32-bit floatingpoint CNN (FP32), in this set of experiments we compare the results of each scenario to the results obtained with the corresponding fault-free custom CNN.

This scenario corresponds to directly training the custom-data-type CNN, so the reliability has to be assessed with respect to the Custom CNN itself. For details, see Figure 3. Note that the row related to the FP32 version is the same one in both experiment sets. In general, the trends highlighted in the first set of experiments are observed also in this scenario: (i) the safety is impacted by the bit-width reduction;

(ii) for fixed-point CNN versions, we observed a more graceful safety decrease compared to floating-point versions. Besides that, an interesting effect is that for floating-point custom variants (FP16 and FP8) the difference between the two sets of experiments (Tables 3 and4) in terms of Safety gain with respect to FP32 is higher than for fixed-point ones (FxP32, FxP16, and FxP8). For example, in layer L0, for variant FP8 (8-bit floating-point) the Safety gain with respect to FP32 is -37.1% for the first set of experiments (Table 3) and -19.9% for the second set of experiments (Table 4), that is a 17.2% difference. Conversely, for variant FxP8 (8-bit fixed-point) the Safety gain with respect to FP32 is -16.4% for the first set of experiments (Table 3) and -17.9% for the second set of experiments (Table 4), which is a difference of 1.5%. On average, the difference between the two sets of experiments for floating-point variants (FP16 and FP8) over all the layers is 33.72%, while for fixed-point custom variants (FxP32, FxP16, and FxP8) is 14.81%. Practically, this means that a designer who chose to approximate the original Standard FP32 CNN version by using a custom floating-point variant without retraining it, would be exposed to higher safety degradation than by using the fixed-point alternative with the same bitwidth. When a training is performed directly on the custom-data-type CNN, the safety degradation difference between the floating-point variants and the fixedpoint ones is smaller. However, fixed-point ones still guarantee less critical fault occurrence, i.e., less than 0.12%.

Finally, as a result of the analysis, we think that in general, the CNN in FxP8 scenario provides the best results in terms of memory footprint reduction, i.e., 4X, with a significant resilience, i.e. less than 0.45% critical faults.

Second Case Study

The second case study targets YOLO [START_REF] Redmon | You only look once: Unified, real-time object detection[END_REF], a CNN capable of detecting objects in real time, analyzing up to 45 frames per second. YOLO is a predictor CNN used to detect a certain set of objects (i.e., the list of recognizable objects came from [START_REF] Redmon | Darknet: Open source neural networks in c[END_REF]). In Figure 7, the prediction results are shown highlighting the identified objects including the object name and area occupation. This example presents an image containing three relevant objects to be detected: one dog, one bike, and one car. Moreover, the car is further recognized as a truck for a total of four objects detected.

Custom Data Type

To carefully select the custom data representation, we first analyzed the the weight distribution of YOLO. It is shown in Figure 8. As evidenced, all The results show that for the FxP32 scenario there is no degradation, for the 16-bit data types (both FP16 and FxP16) in 42% of the cases (3 images out of 7) there is no degradation and for 58% of the cases (4 images out of 7) there is an acceptable degradation (i.e., all objects are correctly recognized and the IoU metric is between 0.95 and 1). Finally, when using the 8-bit floating-point data type (FP8), the CNN is able to deliver usable results (i.e., classified as warning, 0.9≤IoU≤0.95 ) for 28% of the inputs (2 images out of 7) and cannot produce correct outputs (critical ) for the rest (5 images out of 7). On the contrary, with the 8-bit fixed-point data type (FxP8), the CNN is unable to provide the correct results at all, i.e., for all input images the output was classified as critical.

Fault List

In this section, we present the complexity of YOLO in terms of fault injection. As for LeNet-5, also for YOLO we consider only the layers performing arithmetic computations involving trainable weights, i.e., the thirteen convolutional layers. Table 6 provides details about the configuration as well as the fault list of each layer. The first column (labeled "Layer") reports the target layers; the second one ("Connections") specifies the number of connection weights.

The number of possible faults is computed as the multiplication between the connections number ("Connections") and the weight size ("Bit-width"). As the columns "#Faults" point out, the overall number of possible faults is very high and this reflects in a non-manageable fault injection campaign execution time. As done for LeNet-5, we selected a subset of faults in order to reduce the fault injection execution time. For each layer, we injected the number of faults reported in the columns "#Injections". The number of faults to inject was obtained by using the approach presented in [START_REF] Leveugle | Statistical fault injection: Quantified error and confidence[END_REF] (see Section 3.3) and it While for LeNet-5 we figured out that the fixed-point versions of the CNN had a higher average safety level (8.96%) with respect to the floating-point counterparts, for YOLO the difference is not as significant. Indeed, for YOLO, the fixed-point versions of the CNN have a slightly lower average safety level, i.e., -0.44%, with respect to the floating-point versions. This is calculated for the 32and 16-bit versions, as the 8-bit fixed-point one always yields critical errors (see Table 5). If we include also the 8-bit versions, the fixed-point versions have a lower average safety level of -3.42% compared to the floating-point ones. This is probably due to the different distribution of the pretrained weight values for the YOLO network compared to LeNet-5 (compare Figure 8 with Figure 6). Indeed, for YOLO, the need of more bits for the integer part of the weights reduced the representation precision. Furthermore, it is worth highlighting that, in general terms, YOLO turns out to be much less resilient than LeNet-5. Indeed, while for LeNet-5 the percentage of critical faults is always lower than 3.42%, the YOLO network reaches up to 48.46% of critical faults (layer L0, FP8 scenario) and, as already mentioned, the FxP8 version produces critical faults even in a fault-free scenario. This is probably due to the much more stringent definition that we used for critical faults. Indeed, for LeNet-5 we classified a fault as critical only when the top-1 prediction was wrong; conversely, for YOLO a fault is classified as critical also when an object is correctly classified but it is not located perfectly (IoU<0.9). Finally, as in the LeNet-5 case, also for YOLO reducing the bit-width implies reducing the CNN resilience.

In Table 8, we report the results of the second set of FI campaigns, where we compare the results of each scenario with the results obtained with the corresponding fault-free custom CNN. First, it is immediately clear that, also for this FI campaign, the DNN safety is impacted by the bit-width reduction. Second, also in this case, YOLO exhibits high critical fault occurrence, i.e., up to 42.08% in the layer L0 in FP8 scenario.

Finally, for YOLO we notice the same phenomenon that we observed for LeNet-5: for floating-point custom variants (FP16 and FP8), the difference between the two sets of experiments (Tables 7 and8) in terms of Safety gain with respect to FP32 is higher than for fixed-point ones (FxP32, FxP16, and FxP8). Indeed, on average, the difference for floating-point variants over all the layers is 59.38%, while for fixed-point custom variants is 13.92%. As already mentioned, this means that approximating the FP32 CNN version by using a custom floatingpoint variant without retraining exposes to higher safety degradation than by using the same bit-width fixed-point alternative.

Conclusions

This paper presents a characterization framework for analyzing the impact

  DriveWorks) through fault injection experiments and accelerated neutron beam testing for permanent and transient faults, respectively. Faults are injected on the DNN's weights and on the input images. All inferences are executed on Volta GPU only targeting 32-bit floating-point values. Moving forward, Li et al. present in [30] a different analysis. They characterize the propagation of soft errors from the hardware to the application software of different CNNs. The injections are performed by using a DNN simulator based on open-source simulator framework, Tiny-CNN [31]. Thanks to the flexibility of the simulator, it is possible to characterize each layer for a more precise analysis. In this article, CNNs with six different data types are considered: 64bit double precision floating-point, 32-bit single precision floating-point, 16-bit half precision floating-point, 32-bit fixed-point (with two different radix points),

Figure 1 :

 1 Figure 1: Custom Data Type.

Figure 1

 1 Figure 1 illustrates our custom data type. It is defined as following:

Figure 2 :

 2 Figure 2: On-line Weights Conversions.

FI Scenario: Figure 3 Figure 3 :

 33 Figure 3: Fault Injection Scenario.

Figure 4 :

 4 Figure 4: System Layers and Fault Propagation.

  (a) Standard 32-bit floating-point resilience (b) Custom 16-bit fixed-point resilience

Figure 5 :

 5 Figure 5: Critical Bits

Figure 6 :

 6 Figure 6: Distribution of pretrained Weights Values for LeNet-5 network.

  reports the bit-width of the weights. The fourth column shows the amount of bits allocated to encode the different parts of the number, i.e., sign, exponent, and fractional parts in the case of floating-point representations, and the integer and fractional part in the case of fixed-point data. To compute the accuracy of the CNN in the different scenarios, the inference of the images belonging to the validation set of the MNIST database (10,000 images) has been run on LeNet-5, clearly without injecting any faults, i.e., in a golden scenario. The results

Figure 7 :

 7 Figure 7: YOLO CNN prediction result

Figure 8 :

 8 Figure 8: Distribution of pretrained Weights Values for YOLO network

  of permanent faults affecting a Convolutional Neural Network intended to be deployed in safety-critical and resource-constrained systems. The characterization is done by means of fault injection campaigns on the darknet open source framework. The experiments are performed at the software level with the aim of being independent from the hardware architecture and, on the whole, to derive a common characterization of the behavior of CNNs affected by permanent faults. This could be considered an interesting outcome since the designer, starting from these results, could be able to select the most convenient data type for his or her CNN application. Compared to previous works, we show that depending on the data type definition, we can deeply impact the reliability of the CNN. In particular, the results of the fault injection campaign showed that YOLO is less resilient than LeNet-5, according to the fault classifications used. Furthermore, for LeNet-5 CNN, fixed-point data provide a better tradeoff between memory footprint reduction and network resilience compared to floating-point data. In conclusion, depending on the width of the distribution of the CNN weight values, a designer can decide to use either the fixed-point or the floating-point data type, with different bit-widths, to obtain different trade-offs between resilience and resource gain. For example, the use of 8-bit fixed-point data type for LeNet-5 provides 4X memory footprint reduction at the cost of less than 0.45% critical faults. In the future, we intend to investigate the reliability of DNNs by exploiting other data representations, such as the integer data type.

  et al. in 1998. The network architecture is composed of 1 input layer, 5 hidden layers, and 1 output layer, whose typology ranges from Convolutional, Fully-Connected, and Max-Pool layers. For running the experiments, the MNIST database [44], a well-known dataset used to evaluate the accuracy of new emerging models, has been selected. It is composed of 60,000 images for training and 10,000 for

test/validation of the model, encoded in 28 × 28 pixels in grayscale. However, to lower the computational cost and time, a workload of 2,023 images was randomly selected from the MNIST test/validation dataset for the experiments.

Table 1 :

 1 LeNet-5 Data Type Accuracy Loss [%]

	Scenario	Data type	Bit-width	Bit encoding	[%] Accuracy Loss
	FP32	floating-point	32	1 sign, 8 exponent, 23 fractional	Ref.
	FP16	floating-point	16	1 sign, 5 exponent, 10 fractional	0%
	FP8	floating-point	8	1 sign, 4 exponent, 3 fractional	0.02%
	FxP32	fixed-point	32	1 integer, 31 fractional	0%
	FxP16	fixed-point	16	1 integer, 15 fractional	0%
	FxP8	fixed-point	8	1 integer, 7 fractional	0.04%
	resulting from the adoption of custom data type weights. As highlighted in

Table 1 ,

 1 five different scenarios have been analyzed. The second column of the table reports the data type used in each FI campaign, while the third column

Table 2 :

 2 LeNet-5 Fault List for Fault Injection Campaigns

	Layer	L0	L2	L4	L6
	Detail	Convolutional Convolutional Fully Connected Fully Connected
	Connections	2,400	51,200	3,211,264	10,240
	Bit-width	32	32	32	32
	Scenarios
	#Faults	76,800	1,638,400	102,760,448	327,680
	FP32, FxP32
	#Injections	13,678	16,474	16,638	15,837
	Bit-width	16	16	16	16
	Scenarios
	#Faults	38,400	819,200	51,380,224	163,840
	FP16, FxP16
	#Injections	11,610	16,310	16,636	15,107
	Bit-width	8	8	8	8
	Scenarios
	#Faults	19,200	409,600	25,690,112	8,1920
	FP8, FxP8
	#Injections	8,915	15,991	16,630	13,831

Table 3 :

 3 LeNet-5 Fault Injection outcomes w.r.t. Golden Std.

		Observed		Safety		Gain w.r.t. FP32
	Layer Data	Critical Warning Accept Good	Masked	<	≥	Safety * Memory
	FP32 1.32% 0.06% 29.96% 28.98% 39.68% 31.34% 68.66%	-	-
	FP16 2.61% 0.12% 53.28% 41.27% 2.71% 56.01% 43.98% -24.67%	2X
	FP8 3.41% 0.91% 64.13% 31.53% 0.02% 68.45% 31.55% -37.11%	4X
	L0
	FxP32 0.03% 0.04% 25.93% 25.54% 48.47% 26.00% 74.01% +5.34%	0
	FxP16 0.05% 0.08% 49.98% 46.94% 2.96% 50.11% 49.90% -18.77%	2X
	FxP8 0.45% 0.60% 46.74% 52.18% 0.03% 47.79% 52.21% -16.45%	4X
	FP32 1.37% 0.02% 23.88% 23.39% 51.34% 25.27% 74.73%	-	-
	FP16 2.59% 0.08% 55.00% 38.57% 3.76% 57.67% 42.33% -32.40%	2X
	FP8 1.10% 0.91% 65.86% 32.10% 0.03% 67.87% 32.13% -42.60%	4X
	L2
	FxP32 0.01% 0.01% 23.82% 23.62% 52.54% 23.84% 76.16% +1.43%	0
	FxP16 0.03% 0.02% 49.87% 46.61% 3.47% 49.92% 50.08% -24.65%	2X
	FxP8 0.42% 0.45% 46.57% 52.53% 0.03% 47.44% 52.56% -22.17%	4X
	FP32 0.71% 0.00% 3.85% 3.86% 91.57% 4.56% 95.44%	-	-
	FP16 0.84% 0.13% 57.79% 36.12% 5.13% 58.75% 41.25% -54.19%	2X
	FP8 0.49% 0.06% 66.69% 32.72% 0.04% 67.24% 32.76% -62.67%	4X
	L4
	FxP32 0.00% 0.00% 10.99% 11.49% 77.52% 10.99% 89.01% -6.43%	0
	FxP16 0.00% 0.00% 49.40% 45.59% 5.01% 49.40% 50.60% -44.84%	2X
	FxP8 0.40% 0.36% 46.38% 52.82% 0.04% 47.14% 52.86% -42.58%	4X
	FP32 0.61% 0.01% 7.69% 8.14% 83.54% 8.32% 91.68%	-	-
	FP16 1.19% 0.03% 56.34% 37.65% 4.78% 57.57% 42.43% -49.25%	2X
	FP8 1.76% 0.40% 65.07% 32.74% 0.04% 67.22% 32.78% -58.91%	4X
	L6
	FxP32 0.01% 0.04% 13.50% 14.11% 72.33% 13.55% 86.45% -5.24%	0
	FxP16 0.03% 0.08% 49.15% 46.11% 4.63% 49.26% 50.74% -40.94%	2X
	FxP8 0.44% 0.53% 46.28% 52.72% 0.04% 47.25% 52.75% -38.93%	4X

* Safety increasing effect difference between a given scenario and FP32

Table 4 :

 4 LeNet-5 Fault Injection outcomes w.r.t. Golden Custom

	Layer	Observed		Safety		Gain w.r.t. FP32
	Data	Critical Warning Accept Good	Masked	<	≥	Safety * Memory
	FP32 1.32% 0.06% 29.96% 28.98% 39.68% 31.34% 68.66%		-
	FP16 2.61% 0.12% 42.10% 40.86% 14.30% 44.84% 55.16% -13.50%	2X
	FP8 3.30% 0.88% 47.08% 44.67% 4.07% 51.26% 48.74% -19.91%	4X
	L0
	FxP32 0.03% 0.04% 24.45% 23.81% 51.67% 24.51% 75.49% +6.83%	0
	FxP16 0.05% 0.08% 41.96% 40.81% 17.10% 42.09% 57.91% -10.75%	2X
	FxP8 0.11% 0.15% 49.03% 46.94% 3.76% 49.29% 50.71% -17.95%	4X
	FP32 1.37% 0.02% 23.88% 23.39% 51.34% 25.27% 74.73%		-
	FP16 2.59% 0.08% 35.66% 34.59% 27.07% 38.34% 61.66% -13.07%	2X
	FP8 0.96% 0.89% 46.21% 43.17% 8.77% 48.06% 51.94% -22.79%	4X
	L2
	FxP32 0.01% 0.01% 21.41% 20.93% 57.64% 21.43% 78.57% +3.84%	0
	FxP16 0.03% 0.02% 38.70% 37.74% 23.52% 38.75% 61.25% -13.47%	2X
	FxP8 0.06% 0.05% 47.94% 45.79% 6.17% 48.05% 51.95% -22.77%	4X
	FP32 0.71% 0.00% 3.85% 3.86% 91.57% 4.56% 95.44%		-
	FP16 0.84% 0.13% 6.21% 6.23% 86.59% 7.17% 92.83% -2.61%	2X
	FP8 0.32% 0.06% 10.45% 10.11% 79.06% 10.83% 89.17% -6.26%	4X
	L4
	FxP32 0.00% 0.00% 4.06% 4.02% 91.92% 4.06% 95.94% +0.50%	0
	FxP16 0.00% 0.00% 7.83% 7.75% 84.42% 7.83% 92.17% -3.27%	2X
	FxP8 0.01% 0.00% 10.67% 10.33% 78.99% 10.68% 89.32% -6.12%	4X
	FP32 0.61% 0.01% 7.69% 8.14% 83.54% 8.32% 91.68%		-
	FP16 1.19% 0.03% 11.47% 12.63% 74.67% 12.70% 87.30% -4.39%	2X
	FP8 1.59% 0.39% 15.87% 17.10% 65.05% 17.85% 82.15% -9.53%	4X
	L6
	FxP32 0.01% 0.04% 7.15% 7.27% 85.52% 7.21% 92.79% +1.11%	0
	FxP16 0.03% 0.08% 13.48% 13.72% 72.69% 13.59% 86.41% -5.27%	2X
	FxP8 0.05% 0.16% 17.25% 18.27% 64.27% 17.47% 82.53% -9.15%	4X
	* Safety increasing effect difference between a given scenario and FP32

Table 5 :

 5 YOLO Data Type Accuracy Loss [%]

	Scenario Data type Bit-width	Bit encoding	[%] Accuracy Loss
	FP32	floating-point	32	1 sign, 8 exponent, 23 fractional	Ref.
	FP16	floating-point	16	1 sign, 5 exponent, 10 fractional	42% masked, 58% acceptable
	FP8	floating-point	8	1 sign, 4 exponent, 3 fractional	28% warning, 72% critical
	FxP32	fixed-point	32	3 integer, 29 fractional	100% masked
	FxP16	fixed-point	16	3 integer, 13 fractional	42% masked, 58% acceptable
	FxP8	fixed-point	8	3 integer, 5 fractional	100% critical
	Masked: IoU=1; acceptable: 0.95<IoU<1; warning: 0.9≤IoU≤0.95; critical: IoU<0.9 or different objects recognized
	also for this CNN, the data type does not need higher dynamics while a high
	precision is preferred. Therefore, we selected the custom data type reported in

Table 5 .

 5 Moreover, we computed the accuracy loss of the CNN resulting from the adoption of custom data types. As highlighted in Table5, five different scenarios have been analyzed. The second column of the table reports the data type used in each scenario, while the third column reports the bit-width of the weights. The fourth column shows the amount of bits allocated to encode the different parts of the number, i.e., sign, exponent, and fractional parts in the case of floating-point representations, and integer and fractional part in the case of fixed-point ones. To compute the accuracy loss of the network in the different scenarios, the inference for 7 images has been run on YOLO, clearly without injecting any faults, i.e., in a golden scenario. The results are reported according to the classification reported in Section 2.2 for the object detector.

Table 6 :

 6 YOLO Fault List for Fault Injection Campaigns

			FP32 and FxP32	FP16 and FxP16	FP8 and FxP8
			Bit-width=32	Bit-width=16	Bit-width=8
	Layer Connections	#Faults #Injections	#Faults #Injections	#Faults #Injections
	L0	432	13,824	7,551	6,912	4,884	3,456	2,862
	L2	4,608	147,456	14,954	73,728	13,577	36,864	11,466
	L4	18,432	589,824	16,184	294,912	15,752	147,456	14,954
	L6	73,728	2,359,296	16,524	1,179,648	16,410	589,824	16,184
	L8	294,912	9,437,184	16,612	4,718,592	16,583	2,359,296	16,524
	L10	1,179,648	37,748,736	16,634 18,874,368	16,626	9,437,184	16,612
	L12	4,718,592 150,994,944	16,639 75,497,472	16,637 37,748,736	16,634
	L13	262,144	8,388,608	16,608	4,194,304	16,575	2,097,152	16,510
	L14	1,179,648	37,748,736	16,634 18,874,368	16,626	9,437,184	16,612
	L15	130,560	4,177,920	16,575	2,088,960	16,509	1,044,480	16,380
	L18	32,768	1,048,576	16,381	524,288	16,129	262,144	15,648
	L21	884,736	28,311,552	16,631 14,155,776	16,621	7,077,888	16,602
	L22	65,280	2,088,960	16,509	1,044,480	16,380	522,240	16,127

Table 8 :

 8 Yolo Fault Injection outcomes w.r.t. Golden Custom Safety increasing effect difference between a given scenario and FP32

			Observed		Safety		Gain w.r.t. FP32
	Level Data	Critical Warning Accept	Masked	<	=	Safety * Memory
		FP32 20.17% 0.66% 16.25% 62.92% 37.08% 62.92%	-	-
		FP16 30.70% 1.03% 28.10% 40.16% 59.84% 40.16% -22.76%	2x
	L0	FP8 42.08% 5.63% 37.02% 15.27% 84.73% 15.27% -47.64%	4x
		FxP32 13.35% 0.54% 12.43% 73.68% 26.32% 73.68% 10.76%	0
		FxP16 27.23% 1.13% 23.50% 48.14% 51.86% 48.14% -14.78%	2x
		FP32 6.81% 0.10% 14.46% 78.63% 21.37% 78.63%	-	-
		FP16 12.07% 0.18% 23.04% 64.71% 35.29% 64.71% -13.92%	2x
	L2	FP8 14.19% 2.64% 38.61% 44.55% 55.45% 44.55% -34.07%	4x
		FxP32 7.40% 0.43% 12.22% 79.95% 20.05% 79.95% 1.33%	0
		FxP16 14.39% 0.84% 23.60% 61.16% 38.84% 61.16% -17.46%	2x
		FP32 5.05% 0.05% 11.24% 83.66% 16.34% 83.66%	-	-
		FP16 9.88% 0.10% 18.29% 71.73% 28.27% 71.73% -11.94%	2x
	L4	FP8 11.84% 1.70% 30.70% 55.76% 44.24% 55.76% -27.90%	4x
		FxP32 6.32% 0.35% 11.86% 81.47% 18.53% 81.47% -2.19%	0
		FxP16 12.45% 0.70% 22.51% 64.34% 35.66% 64.34% -19.33%	2x
		FP32 4.02% 0.01% 7.59% 88.37% 11.63% 88.37%	-	-
		FP16 8.29% 0.04% 13.34% 78.33% 21.67% 78.33% -10.04%	2x
	L6	FP8 8.37% 1.25% 22.66% 67.72% 32.28% 67.72% -20.65%	4x
		FxP32 4.49% 0.26% 11.04% 84.21% 15.79% 84.21% -4.17%	0
		FxP16 8.82% 0.53% 20.99% 69.66% 30.34% 69.66% -18.72%	2x
		FP32 3.41% 0.01% 3.94% 92.65% 7.35% 92.65%	-	-
	L8

is statistically representative with an error margin of 1% and a confidence level of 99%. On average, 15.7k faults are injected in each layer for 32-bit scenarios (FP32 and FxP32), 15.3k for 16-bit scenarios (FP16 and FxP16), and 14.8k for 8-bit scenarios (FP8 and FxP8). The injections are performed by randomly selecting the faulty bit among all bits of the connection weights.

Fault Injection Outcomes

In this section, we report the results of the FI campaign on the YOLO CNN. In line with the LeNet-5 FI campaign, two sets of experiments are carried out. Firstly, we evaluated the reliability by using as reference the Standard 32-bit floating-point CNN and then by using the Custom fault-free one. The results are reported in terms of the classification presented in Section 2.2. In the following, we define faults belonging to the classes Critical, Warning, and Accept as events reducing the CNN safety. The sum of these contributions is represented by the symbol '<' in the tables. On the other hand, we consider the masked faults as events leaving the safety of the CNN unaltered. Thus, their contribution is represented by the symbol '=' in Tables.

Table 7 reports the results of the first set of FI campaigns.